Hi

I'm from Bangladesh. Till I've used this site for troubleshooting and other issues by just searching and browsing various posts which helped me a lot. But this the first time I have to post a new thread as I couldn't find anything that relate my current problem. So I badly need help from you guys/bosses/gurus.

I'm working in GT Power Station having two units. The 100MW PG9171E unit with GE MarkV recently showing an weird problem. A set of Thermocouple (No: 2,5,8,11,14,17,20,23) is showing value -18 0c. All other thermocouples are showing ambient temperature as the GT unit is in shutdown for other issue (not even TG is running). All these negative valued thermocouples are related with <S> core because when we Hard Reset <S> core, the problem goes temporarily. But after sometime (like 2 or 3 hours or some times ever after more) -18 0c back again. This odd values comes along with following alarms also:

1. LOAD TUNNEL THERMOCOUPLE TROUBLE
2. COMP INLET TEMP TC TROUBLE
3. COMP DISCHARGE TEMP HIGH SPREAD
4. COMPRESSOR INLET THERMOCOUPLES DISAGREE

I don't know why the no. 2,3,4 alarms are coming. Also, No Diagnostic Alarm is coming when thermocouple value shows -18 0c.

But if we open any thermocouple connection in MarkV, then value shows -83 0c. Please give your valuable suggestion.

 

The MarkV monitors the exhaust TC on each core RST. Note 2+3=5, 5+3=8 .... etc.

Your post indicates No: 2, 5, 8, 11, 14, 17, 20, 23 each 3 away for the other.

Reboot <S> Core

for example 1, 4, 7.... I would respond..... Reboot <R> core.

 

All triple redundant RST.

> 1. LOAD TUNNEL THERMOCOUPLE TROUBLE
> 2. COMP INLET TEMP TC TROUBLE
> 3. COMP DISCHARGE TEMP HIGH SPREAD
> 4. COMPRESSOR INLET THERMOCOUPLES DISAGREE

Without a complete tutorial, I hope you will investigate and find out why it works this way.

 

What is the ambient temperature in the MarkV cabinet near <S> core?

 

Hi,

Is <S> core at A7? if not check the IO states which stuck to reach A7. Working towards will sove your issues.

best of luck

take care
g.rajesh

 

Thanks a lot to everyone for your valuable replies. First let me give some feedback.

>Is <S> core at A7? if not check the IO states which stuck to
>reach A7. Working towards will sove your issues.

1. Yes, all four cores i.e. <C>, <R>, <S> and <T> are in A7 states and healthy.

2. I've told about ambient temperature at Exhaust Thermocouple where all other TC showing between 28 to 31. Anyway, as the MarkV cabinet is at Local Control Room where air cooler is cooling the room, the ambient temperature in the MarkV cabinet near <S> core will be around 24-25 0c.

3. I've mentioned the TC no: 2, 5, 8, 11, 14, 17, 20, 23 are showing negative value which are related with <S>. TC no. 1, 4, 7, 10, 13, 16, 19, 22 related with <R> and 3, 6, 9, 12, 15, 18, 21, 24 related with <T> core. We have Hard RESETed all cores, but same thing as I described.

4. Actually, I've asked about the alarm no. 2, 3, 4 i.e. COMP INLET TEMP TC TROUBLE, COMP DISCHARGE TEMP HIGH SPREAD, COMPRESSOR INLET THERMOCOUPLES DISAGREE

Because, if I have any problem with Exhaust Thermocouple, why these alarms are coming?

In my opinion, Exhaust Thermocouples, may be, have not any physical problem. Coz, to find out that, we have opened a TC connection in MarkV (which was showing positive ambient temp) then it shows -83 0c which is a Healthy situation.

Finally, I would like to add another question. Is this a software problem?

 

As GRajesh has suggested use the LCC/SLCC display and keypad to determine the I/O States of all of <S>'s cards. More than likely, there's a problem with the TCQA card.

When did this problem start? After a card in <S> was replaced? After <S> was re-booted? After a turbine trip? If a card was recently replaced, which one was it?

Also, WHAT DIAGNOSTIC ALARMS ARE PRESENT? It's pretty certain there are more than one Diag. Alarm related to <S> and or the power supplies of <S>.

What alarms are being annunciated on the LCC/SLCC display?

Have you tried pushing firmly on all the cable connectors of the cable that connects the LCC/SLCC to the DCC/SDCC to the TCQA?

If the unit isn't running, what's the harm in re-booting <S>?

Write back to let us know how you fare.

 

> 4. Actually, I've asked about the alarm no. 2, 3, 4, i.e. COMP INLET TEMP TC TROUBLE,
> COMP DISCHARGE TEMP HIGH SPREAD, COMPRESSOR INLET THERMOCOUPLES DISAGREE

> Because, if I have any problem with Exhaust Thermocouple,
> why these alarms are coming?

Triple Redundant
3 COMP INLET TEMP
3 COMP DISCHARGE
3 COMPRESSOR INLET THERMOCOUPLES

<S> core is having problems getting good data or no data (basically default date).

It is not software related.

<S> core is monitoring 1 inlet temp and voting
<S> core is monitoring 1 comp discharge and voting
<S> core is monitoring 1 comp inlet and voting
<S> core is monitoring 2, 5, 8........ exhaust tc

All listed TC inputs are physically landed on the same input boards and then shared via ribbon cables to each core.

Please check ribbon cables from TC input board that supplies <S> core. They are generally clearly labeled.

 

-18C on a MKV panel Thermocouple usually means there is no T/C connected (-18C is usually the zero value for the TC thermocouple scale). I would be suspicious that you have a controller problem, by the thermocouples you have mentioned, it should be <S>. Best bet is the TCQA in <S>, if not the <S> terminal board TBQA but not so common.

Do you have a MKV Application Manual? if yes there is some good info. there which should help you.

 

I don't remember where the A/D conversion for thermocouples takes place, but I would check there. A -18 deg C is 0 deg F, and internally GE does all the calculations in English units. I do believe that all the thermocouples in a given core use the same A/D converter board.
The conversion to metric units takes place in the HMI.

 

CuriousOne,

What you're saying--without saying it outright--is that T/Cs are connected to the TBQx (I can't remember the last character; it's either B or C) and then groups of fifteen of them are connected to the TCQA cards in <R>, <S>, and <T>. So there are three ribbon cables on the TBQx to carry the T/C signals to the TCQA cards in the three control processors. There might be a problem with the cable between the TBQx and the TCQA in <S>. If no conductive grease has been applied to the ribbon cable connectors and receptacles or it's been a long time since it has been, that may be a part of the problem.

My money's on an issue with either the TCQA or the TCPS in <S>, or the 3PL cable connecting the LCC/SLCC, the DCC/SDCV and the TCQA (and the optional TCQB, if present). A blown fuse on the TCPS may also be the problem--but there's probably a Diagnostic Alarm to indicate that.

Rarely does a problem like this occur without some "stimulus" like a ground or short or a card replace phone bad or something like that.

Hopefully the original poster will write back with information and details.

 

First try to restart the <S> network switch.

If it doesn't work try to reset the <S> I/O card.

 

CSA,
I have awaited you input.

I only have a mere couple of decades of experience with the Mark V.

From my limited experience: I have found that a core that is overheating will reject its inputs that are shared from terminal boards that are actually installed on <R> core.

I had hoped that the original individual that posted would investigate how a TMR ACTUALLY shares a single wire with multiple cores via ribbon cables.

Only servos and such are actually landed on the TMR cores

On multiple visits to different sites: I have found air conditioning in the PEECC not functioning which resulted in a core reporting problems with odd numbered exhaust TC, Comp inlet, load tunnel temp, etc.

All triple redundant landed on <R>.

 

Hi,

I would suggest you to check the compressor inlet TC pre-voted value first (CTIF1 & CTIF2) and identify the faulty one. Next check the one which is showing bad value is connected to "S" core TCQA board through TBQA IO board or not. Probably the faulty compressor inlet TC (as per alarm COMP INLET TEMP TC TROUBLE) and Exhaust TC No: 2, 5, 8, 11, 14, 17, 20, 23 are connected to the same board. If so you need to investigate whether anybody replaced the some boards in either in S or R, T cores. Does the problem started after that or not. Try to interchange TCQA board in R with S temporarily and investigate the problem has been shifted to R or not. Even you can shift TC No: 2, 5, 8, 11, 14, 17, 20, 23 to R core temporarily to identify the fault. Probably I feels that TCQA board or some of its ribbon cable connections are bad.

 

CuriousOne,

I have never heard of an overheated core in a Mark V. Each processor does its own cold junction compensation for T/C temperature measurement, and that can be seen by using the Prevote Data Display or Logic Forcing Display and viewing the signal TCQA_CJ_Q. If the discharge of an air conditioner is blowing directly towards <S>--and that I have seen!!--then <S>'s cold junction compensation will be lower than <R>'s or <T>'s and that will play havoc with the readings of the thermocouples that are routed to <S>. I've seen this cause DLN-I units to go out of emissions compliance intermittently which drove the Operations Manager crazy. (The DLN combustion hardware was not OEM and the DLN tuning was marginal to begin with, but of course the problem was the Mark V. And the Operations Manager had relocated the Mark V to a location directly beneath an A/C discharge duct which is how the cold air happened to be blowing mostly through the louvers of the left door onto <S>.)

As for people investigating how I/O is terminated in a Mark V, that's not very intuitive or easy. And our function is to understand that not everyone has twenty years of experience with the Mark V and to detail our explanations accordingly. I'm often accused of being too verbose in my responses, but I'm trying to recognize that there are a LOT of people reading these posts who can benefit from putting as much detail as possible in the responses--which, while it may make them long and unnecessarily detailed for some, it helps those who don't have the background or detailed knowledge to learn even more than they might otherwise have. That's the real beauty of this forum--helping many people learn from the questions and tribulations of one. We must use our knowledge to help as many people as we can in these posts by including as much detail as possible--even if it seems obvious to us and doesn't seem to need explanation.

AND--remember when this wasn't obvious to us, and how learning these little gems helped us to be even better at investigating and troubleshooting? Not just the Mark V, but other things, as well.!

 

Adel_S,

This post is about <b>Mark V</b>, not Mark VIe. The Mark V does not have network switches.

 

Hi, everyone

Thanks a lot for your valuable replies. My problem is still persisting. When I Hard RESET the <S> core the problem gone for sometime and comes again. According to your valuable suggestions, let me describe our MarkV Case wiring.

There are 24nos. Exhaust Thermocouple installed in our system. All of them are terminated to TBQA board of <R> core and then equally divided for each <Q> core. From <R> core TBQA board JAS connector goes to <S> core TCQA card JA connector. From <S> core TCQA card 2PL and 3PL runs daisy chain from SDCC to SLCC.

To CSA: You have mentioned about Diagnostic alarm. But no Diagnostic Alarm comes when TC value goes negative and aforementioned alarms come. We have tighten all connectors, replaced TBQA board, TCQA card, but result was same. Rebooting <S> core keep the system healthy for some time. TCQA_CJ_Q signal value in Prevote Data Display was
Voted value= 33, <R> = 34, <S> = 33, <T> = 32 deg C

Most probably Overheating problem did not happened this time, coz an A/C is directly opposite to the MarkV panel.

Rama2014: There are 3 CTIF i.e. CTIF1A, CTIF2A, CTIF3A. All 3 are connected to <R> core TBQA board. I've found that CTIF2A = -18 degc. During diagnosing, one thing I've seen that TCQA_STAT_S was 0000 in hex value but other two i.e. TCQA_STAT_R = 00A7 and TCQA_STAT_T = 00A7 in Logic Forcing Display, whereas in LCC display all IO cards are showing A7 status even in <S> core also. I've removed the 2PL connector in <S> core TCQA card and put it firmly again. After some time, all TC value become positive. I don't know how much time it will be Ok.

So, according to all of your suggestion, I think, anyhow there is a communication problem between TCQA and SDCC card. Tomorrow I'll interchange the TCQA card with other core to ensure about TCQA card.

Thanks a lot again for now.

 

Thanks for your feedback and waiting to know your furhter progress.

Good luck
g.rajesh

 

bpsemd,

What's the status of the situation?

Are you aware the ribbon cable connectors (on the ribbon cables themselves, as well as the stationary receptacles on the printed circuit cards) are prone to corrosion in some environments? The OEM has recommended that a light film of conductive grease be applied to the connectors periodically. The best way to do this is to remove the ribbon cable connector from the receptacle on the printed circuit card (when the processor is powered down) and wipe a thin film of grease on the ribbon cable connector, then reinsert the cable connector into the printed circuit card receptacle, removing and re-inserting the connector two or three times to try to spread the grease on the receptacle pins as evenly as possible. The OEM usually provides tubes of conductive grease in the boxes with the spare printed circuit cards. Too much grease is as bad as no grease, so care should be used when applying it. A thin layer/film of grease is all that's required--but it should be thick enough to cover all the pins very lightly and penetrate into the ribbon cable connector female sockets.

The grease should be re-applied every couple of years or so.

I'm extremely surprised there are no Diagnostic Alarms--REALLY surprised. Usually, most people have the exact opposite experience on a normal basis, that is, there are too many unintelligible Diagnostic Alarms. And, Diagnostic Alarms are indicative of hardware problems.

The DIAG_STAT value you cited is indicative of a problem with the TCQA. If you were using the LCC/SLCC display/keypad to view the individual I/O Card states, it would also report a similar inability to read the TCQA card. the 3PL cable used to connect the TCQA to the DCC/SDCC and LCC/SLCC cards is known to be very prone to failure--because the OEM foolishly did not use pull tabs on the cable to make it easy to remove them from the printed circuit card receptacles. More than one intermittent problem has been magically resolved by replacing the 3PL cable, or just making sure that all the connectors are firmly attached to the cable and have not come loose. You have reported replacing the cards the 3PL cable connects, and it's very likely that unless great care was used that the cable connectors may have been disturbed or even damaged.

Please write back to let us know how you have fared in resolving this issue.

 

>What's the status of the situation?

Thanks a lot CSA and everyone. My situation is still same. By this time we have replaced the TCCA card of <C> core with a new one. Same.... Interchanged the TCQA of faulty <S> core with <R> core. Still same.. Lastly, we have replaced the SDCC card of <S> core with a new one last evening and still didn't get any news of same thing from control room. So, hope, till now the system is OK. But, if it is failed again, I don't know what should we do.

Thankful to CSA. You have mentioned a very important matter about greasing. The control system is about 14 years old and we never greased the connection points. Very frankly, we didn't knew about this. That's why, tomorrow I'll search for the conductive grease if there is any piece in our store. And still now there is no Diagnostic Alarms and also SLCC shows all cards are in A7 status.

Actually we are in very much odd situation. This disaster started on 1st December 2014. With a heavy vibration, the machine tripped and we found that the Permanent Magnet Generator (PMG) of the unit has been damaged. One of its pole has broken. A total disaster. After that we have tried to procure a new PMG from the OEM BHEL. Being a Government organization we have some protocols to be followed and there several things happened by this time which causes this delay. Anyway, I've got the news that the PMG has been shipped from India and may be within a week it'll be available in our hand. So, after installing the new PMG we must have to Start the machine. Even, by this time, we have completed our scheduled HGPI. We don't know what will happen. Except the Control System, everything is ready, will fall me in a very much awkward position.

Wish we have overcome the situation by this time....