PDIO replacement, Markvie controller failed to boot

We have two mark vie controllers for Gas turbines. One of the PDIO module failed while GT online and alarmed, Control system fault inhibit. PDIO was showing without any indications.

We removed the load on the machine and while at FSNL, replaced the PDIO with a spare one. The module didn't load even though it is auto configured. We build and downloaded from the HMI. While downloading machine tripped. All the controllers show, input enabled state, not going online.

Tried downloading again, but in vain. Ping shows the controllers are healthy. After a few attempts, Controller came on line.

My question is what is enabled state in Controller and what could be the reason to go in to this state. Any expert advice on this matter??
>We have two mark vie controllers for Gas turbines.

What does that mean? That you have a DUAL Redundant Mark VIe control system with two UCSx controllers, or that you have two Mark VIe turbine control systems for two gas turbines? If you have two Mark VIe control systems for two gas turbines are they TMR or DUAL Redundant?

The post and the conditions are unclear. As for your question regarding 'input state enabled', a Mark VIe turbine control system with multiple UCSx controllers (either DUAL Redundant or TMR) must be fully synchronized with all controllers before the outputs are enabled. Part of the process of synchronization (and synchronized operation) involves reading inputs, then sharing the vzlues of inputs with the other redundant controllers, voting the inputs, and when fully synchronized, writing the outputs. So, enabling inputs is <b>ONE</b> of the steps of synchronization.

>We build and downloaded from the HMI.

What did you build and download--the application code? Why?

What did you download to--the "controller" is the UCSx. The PDIO is an I/O Pack.

>After a few attempts, Controller came on line.

Are you referring to the UCSx controller, or the I/O Pack? (They are not both controllers. One is a controller (the UCSB), and the other is an I/O Pack.

It has been my experience that replacing a Mark VIe I/O Pack on a running turbine sometimes requires several minutes for everything to get synched up and fully working--especially on a DUAL Redundant Mark VIe turbine control system. Patience is a virtue when replacing an I/O Pack on a running turbine.

And in no case should it be necessary to build & download application code when only replacing an I/O Pack (not in my experience).

Quite often people get frustrated with the pace of re-synchronizing controllers and start building and downloading which can lead to all sorts of unimaginable consequences and trips if the unit is running.

Another possibility, especially if the Mark VIe is a DUAL Redundant turbine control system is that the I/O Pack had some critical I/O connected to it that after the inputs were "unfrozen" during the synchronization process (synchronization of controllers and I/O Packs) the controllers believed the turbine should be tripped--and you DID NOT tell what alarm(s) were annunciated when the turbine tripped....

I also have some reservations about how DUAL Redundant controllers with single I/O Packs for critical I/O actually re-synchronize and "choose" which controller contrils the I/O. That has never been very well documented by GE.

Finally, when performing downloads while a unit is running sometimes people mistakenly download to 'permanent storage' which will most often re-boot the controller (UCSx)--which isn't a good thing for DUAL Redundant systems. And, it's also not clear if you were downloading to one or both controllers (UCSx's) and how you chose one controller to download to--or why.

The most important lesson here is: Don't be in a hurry to build & download. And be aware of what you are downloading to, and what method you have allowed for downloading.

Also, please be clear when posting. It makes replying much easier. And, I bet you will find if you take the time to make your post clearer you will actually discover what you may have done in error.

Hope this helps!
Sorry for the confusion. Thanks a lot for the quick response.

I mean two control systems for two GT's. Each one is TMR.

we waited for five to ten minutes. When the auto config didn't work. build and downloaded. Download command selected only failed PDIO files such as firmware and parameters.

Downloaded to PDIO. Not to UCSA controller.

What is meant by
"Finally, when performing downloads while a unit is running
sometimes people mistakenly download to 'permanent storage'"

There is no option permanent or temporary?

Please explain me what is the status called "Input Enabled" in controller status.


I believe I answered the main question, but I'll try once more.

As part of the boot-up process of an I/O Pack (or of the main controllers) one of the steps is to enable the inputs to the I/O pack so that the values can be read, then shared with the other controllers, and then used to vote the input values, then execute the application code, then write to the outputs based on the results of the executing the application code with the voted values of inputs.

Again, all I/O Packs and controllers must be synchronized to read their inputs at the beginning of a frame, then communicate their input values to other controllers who then vote on the values of inputs, then use the voted values in executing the application code, then write values to the outputs.

It all has to be synchronized so that every controller reads its inputs at the same instant in time and then shares the values with the other controllers who then vote the input values (this is called SIFT--Software-Implemented Fault Tolerance) and its very important to a TMR (or DUAL Redundant panel) because it should help to prevent nuisance trips due to erroneous input values.

So, during boot-up, enabling inputs is one of the "states" the I/O pack has to pass (successfully complete) before it then shares the values from the I/O Pack with the other controllers, who each then vote the values from the three I/O Packs (PDIOs) and then execute the application code and then write to the outputs of the various I/O Packs.

In my experience, when replacing a single I/O Pack it is not necessary to build the application code--it's only necessary to download the firmware (since the I/O Pack is new), and then download the configuration to the I/O Pack. It should only be necessary to download firmware and configuration to the I/O Pack.

I was using "old" terminology from the Mark VI days--downloading to permanent storage was the equivalent of downloading to EEPROM, and to make the changes active (to get the application code from EEPROM to RAM) it was necessary to re-boot the controller. Also known as an off-line download, as opposed to an on-line download--that doesn't require a re-boot. I believe in this case, when downloading to one I/O Pack of three on a terminal board (you didn't say which I/O Pack, nor if there were three I/O packs, one for each of <R>, <S> & <T>--nor did you provide the alarms from when the unit tripped) you would have to download firmware then configuration. (I believe the firmware and the configuration are both written to the I/O Pack's EEPROM and a reboot is require at least for the firmware to take effect.)

I think it might be necessary to re-boot the associated controller of the I/O Pack that was replaced, because I don't believe the firmware will take effect without re-booting the associated controller--or re-booting the I/O Pack after the firmware is downloaded to it by temporarily removing the 28 VDC supply in order for the firmware to take effect, then it would be necessary to download configuration to the I/O Pack--but I don't think it would require re-booting the I/O Pack for the configuration to take effect. (It's been a few years since I've had to replace an I/O Pack--just lucky, I guess!)

Hope this helps!

In short, what CSA is saying is that one of the I/O packs is not talking to the controllers. This could be the new PDIO you installed or another pack that could have been bad for sometime. You will only see this 'inputs enabled' when you start up the controller.

Go online with the controller and look for any I/O packs that have the red X...that is your culprit.