Hi, everyone,
We are having a very odd problem with our Mark VI PLC, where all tests we did are pointing to the core R UCVE failure. May you please see our tests descriptions and give your opinion about the diagnostics?
First impressions: the gas turbine (F6B, Mark VI PLC) was stopped for a 10 day maintenance with 100% good diagnostics of the PLC. As usual, 3 days before start-up we went to do a full check-up, where we found out the core R VSVO card indicating some failures, with its redundant signals (tripled) showing frozen values: VSVO S and T cores was showing live values changing (very slightly), but the R core was completely frozen and showing negative values.
At first we thought it was a VSVO problem, so we got a spare card and went to substitute. Before doing that, GE suggested a reboot of rack R (power off and back on), to see if the failure would persist. When we did that, the R core did not came back. We tried a full reboot (rack R, S and T), and the result is:
SCENARIO "A" [INIT]
- When connected to S or T: Core S in 0xc3 / Core T in 0xc3 / Core R in UNKNOWN state.
- When connected to R: Core R in FAIL (red) / Core S in UNKNOWN state / Core T in UNKNOWN state.
Then, we tried a full offline download (for the 3 cores), getting the error: "-Error D165 - SYSERR sys loader (165): cannot online download an unconfigured UC2000". When download to S and T only, the download completes fine.
SCENARIO "A" [END]
The next step was doing a complete restart of core R UCVE:
- Switch off rack R; removed UCVE; removed CF card;
- Using a CF USB reader, we downloaded the compact flash using Control Toolbox (configuring IP);
- Put UCVE CF carf back; put UCVE back to rack R; disconnected VCMI card (as requested by the GE manual);
- Ping to core R UCVE: successful
- Downloaded the product code (runtime) to core R only, using Control Toolbox (select.dnl file), selecting only "download to permanent storage" and NOT selecting "download to memory". The options "download symbol table" and "download compressed controller file" did not even appeared.
- Download was successful.
- Power off rack R; re-inserted VCMI; power on rack R;
- SCENARIO "A" was maintained.
After that, we tried a full power cycle of the Mark VI (turn off all racks R, S, T, X, Y, Z; then turn on in the order X, Y, Z, R, S, T). SCENARIO "A" was maintained.
To try to verify/confirm if the problem was in fact the UCVE, we did the following:
- Switched off core R and S;
- Removed UCVE R and S and swapped the CF card between them;
- Inserted the R_UCVE in the S rack and the S_UCVE in the R rack; did a full power cycle;
- we noticed that the failure followed the R_UCVE card, that is:
* When connected to R or T: Core R in 0xc3 / Core T in 0xc3 / now Core S in UNKNOWN state.
* When connected to S: now Core S in FAIL (red) / Core R in UNKNOWN state / Core T in UNKNOWN state.
Some questions (please think out of the box):
1) Can we say with 100% certainty that the issue is related to the R_UCVE card?
2) What is the "D165 - SYSERR sys loader (165)" error and what is the UC2000?
3) We have a PC MIP board (RJ 45 connector) installed in all 3 UCVE controllers, and the one in the R_UCVE is not working (LED off). What is the function of this board? Is this only for redundancy with the LAN RJ45 UDH port? Can we disconnect this board from the R_UCVE and try a download without it? We disconnected the network cables from PC MIP of S_UCVE and T_UCVE and did a download to them, and it went just fine.
Unfortunately we don't have a UCVE spare (we tried to buy, but GE informed they don't supply this card anymore). We are now working with two possible solutions:
- Migrate UCVE to UCVG or UCVH (if GE have them for sale);
- We are looking for GE clients to buy UCVE cards for replacement;
What do you think, any other ideas?
We are very thankful for any help you may provide. Very best regards.
We are having a very odd problem with our Mark VI PLC, where all tests we did are pointing to the core R UCVE failure. May you please see our tests descriptions and give your opinion about the diagnostics?
First impressions: the gas turbine (F6B, Mark VI PLC) was stopped for a 10 day maintenance with 100% good diagnostics of the PLC. As usual, 3 days before start-up we went to do a full check-up, where we found out the core R VSVO card indicating some failures, with its redundant signals (tripled) showing frozen values: VSVO S and T cores was showing live values changing (very slightly), but the R core was completely frozen and showing negative values.
At first we thought it was a VSVO problem, so we got a spare card and went to substitute. Before doing that, GE suggested a reboot of rack R (power off and back on), to see if the failure would persist. When we did that, the R core did not came back. We tried a full reboot (rack R, S and T), and the result is:
SCENARIO "A" [INIT]
- When connected to S or T: Core S in 0xc3 / Core T in 0xc3 / Core R in UNKNOWN state.
- When connected to R: Core R in FAIL (red) / Core S in UNKNOWN state / Core T in UNKNOWN state.
Then, we tried a full offline download (for the 3 cores), getting the error: "-Error D165 - SYSERR sys loader (165): cannot online download an unconfigured UC2000". When download to S and T only, the download completes fine.
SCENARIO "A" [END]
The next step was doing a complete restart of core R UCVE:
- Switch off rack R; removed UCVE; removed CF card;
- Using a CF USB reader, we downloaded the compact flash using Control Toolbox (configuring IP);
- Put UCVE CF carf back; put UCVE back to rack R; disconnected VCMI card (as requested by the GE manual);
- Ping to core R UCVE: successful
- Downloaded the product code (runtime) to core R only, using Control Toolbox (select.dnl file), selecting only "download to permanent storage" and NOT selecting "download to memory". The options "download symbol table" and "download compressed controller file" did not even appeared.
- Download was successful.
- Power off rack R; re-inserted VCMI; power on rack R;
- SCENARIO "A" was maintained.
After that, we tried a full power cycle of the Mark VI (turn off all racks R, S, T, X, Y, Z; then turn on in the order X, Y, Z, R, S, T). SCENARIO "A" was maintained.
To try to verify/confirm if the problem was in fact the UCVE, we did the following:
- Switched off core R and S;
- Removed UCVE R and S and swapped the CF card between them;
- Inserted the R_UCVE in the S rack and the S_UCVE in the R rack; did a full power cycle;
- we noticed that the failure followed the R_UCVE card, that is:
* When connected to R or T: Core R in 0xc3 / Core T in 0xc3 / now Core S in UNKNOWN state.
* When connected to S: now Core S in FAIL (red) / Core R in UNKNOWN state / Core T in UNKNOWN state.
Some questions (please think out of the box):
1) Can we say with 100% certainty that the issue is related to the R_UCVE card?
2) What is the "D165 - SYSERR sys loader (165)" error and what is the UC2000?
3) We have a PC MIP board (RJ 45 connector) installed in all 3 UCVE controllers, and the one in the R_UCVE is not working (LED off). What is the function of this board? Is this only for redundancy with the LAN RJ45 UDH port? Can we disconnect this board from the R_UCVE and try a download without it? We disconnected the network cables from PC MIP of S_UCVE and T_UCVE and did a download to them, and it went just fine.
Unfortunately we don't have a UCVE spare (we tried to buy, but GE informed they don't supply this card anymore). We are now working with two possible solutions:
- Migrate UCVE to UCVG or UCVH (if GE have them for sale);
- We are looking for GE clients to buy UCVE cards for replacement;
What do you think, any other ideas?
We are very thankful for any help you may provide. Very best regards.