Mark-VI Communication with ABB DCS Over OPC

On our site we have integrated our Mark-VI systems to the Power Plant DCS which is ABB Freelance 800F. The DCS reads the data from the gas tubrines' local station and the OPC link is established via fiber optic. The OPC server is configured in 2 of our ABB Stations, one for each turbine.

We have been facing an issue that parameters of the turbine (being read through OPC) often get halted on the ABB stations. The issue is rectified by restarting the respective gas turbine's local station and in certain cases the ABB station running the respective OPC Server. However, on a few occasions the same issue aggravated to the extent that it resulted in hanging of our ABB Stations leaving the boardman totally blind of the plant. Even though the problem was again rectified by rebooting the turbines' local stations, the situation is concerning as it holds the potential of plant shutdown or serious incident.

I would greatly appreciate if anyone advise as to what can be the possible reason for this scenario and what should we do to ensure it does not recur.
 
Ali,

Mark VI systems are no longer provided by GE; Mark VIe systems are now the standard. This means that the turbine control systems at your site have been in service for some time, and it would seem this problem has just recently started (or that it has not persisted since commissioning--we hope!). So, if the system ran for some time with few or no problems, then something triggered this problem.

So, the question is: <b>What changed just before this problem started?</b>

Was there some upgrade to the Mark VI, or the GE HMI? Or to Toolbox? Or to CIMPLICITY?

<b>OR</b>, was some upgrade done to the ABB system?

Usually, this communication is done over the PDH so is there some overloading of the PDH that was precipitated by some new hardware or software that running on or connected to the PDH?

These are the usual causes of these types of problems. Someone made what seemed, at the time, to be an innocuous change to one end or the other of the OPC link, or some piece of software was upgraded, or some new piece of equipment was added to the PDH. Usually, these things don't just start happening by themselves.

Cabling could also be a cause--but not likely.

Network switches could also be a cause--but not likely. Not impossible, but not likely.

What OPC client/server arrangement is being used on the "local station"? GE CIMPLICITY or some other third-party OPC software?

Did someone recently try to add signals to the CIMPLICITY Project(s)? Or to the OPC configuration at one end or the other?

Most problems like this are the result of changes that weren't performed properly, or add just one too many signal to the list--there are limits and those limits are usually less than advertised and painfully discovered.

And, sometimes, new hardware added to the PDH can cause these kinds of problems. Even changing CPUs or adding new network printers can cause these kinds of problems if not properly configured.

So, when something like this happens it's best to step back, and try to think of what changed just before the problem started. Because, again, if the system was running with few or no problems for some time and then "all of a sudden" something goes awry it's usually the result of some change that someone made that seemed very innocent at the time, but wasn't done properly or resulted in excess network traffic.

As hard as it is to believe, Ethernet cables do fail. If hardware was moved (say for housekeeping/cleaning) and cables were disturbed, this has been found to be a cause of some problems. It usually takes a LONG time to find, but it's like a loose termination in a circuit--everyone assumes the worst, and it can easily be the simplest thing that causes the problem. I marvel at how much of our world is entrusted to Ethernet cabling, and how poorly the cabling is treated or regarded. It's a critical link.

If you suspect network problems, get someone to site who can "sniff" the network (PDH) with something like Wireshark (sp?) to see if there are lots of collisions or other problems. There are many network utilities which can be useful in troubleshooting these kinds of problems.

And finally, it's best to remember that troubleshooting is quite often a logical process of elimination. One makes a best guess as to what the problem might be caused by, and then formulates a plan to prove--or disprove--their theory. If they prove that it was NOT the cause of the problem (conclusively!), then they form another test plan for the next logical possible cause, and set out to prove--or disprove--their theory (conclusively!). I can tell you I've been to a LOT of site where someone did a meaningless or minimal test of some piece of equipment or process only to find weeks or months or even years later--it was the cause of the problem and had they taken the time to properly test and eliminate that piece of equipment they would have discovered the cause of the problem much sooner. So, a proper plan to conclusively prove--or disprove--some theory is necessary when troubleshooting a problem like this.

Please--write back to let us know how you fare in troubleshooting and resolving the problem!

One more thing, it's not clear from your post whether or not this happens more frequently on one turbine/local station or if it occurs just as frequently on both units. This could be helpful by narrowing the problem primarily to one unit or the other.

Again, please write back to let us know how you fare!
 
Top