Dear all,

I hope you are in the best of health and spirits.

I need help in understanding root-cause of a recent tripping on one of the generator units at my workplace. The generator is coupled with Gas Turbine Frame-5001PA.

Generator details are:


Type DG185ZL

Rated Power - 26 MVA

Power Factor - 0.85

Rated Voltage (L-L)- 6600 V

Rated Current- 2295 A

Rated Frequency-50 Hz

Year of Manufacture- 2009

Number of Poles - 4

Rated Speed - 1500 rpm

What happened?


Recently the generator tripped on Seismic probe security i.e. going beyond the trip limit.

There are two Seismic probes installed on the generator(See the Train Diagram attached), 1ea on the in-board and 1ea on the outboard along with 2ea proximity probes. Bentley Nevada system is installed which feeds their value to Mark 6 system for Turbine control. The Sequence-of-Events was as follows:

1. Both the Seismic probes started to go above the alarm limit of 4.5mm together. This kept happening for 5 minutes.(Both went above alarm limit and came back, then again both went above alarm limit and came back and this repeated)
2. Then finally one of them went above the trip limit of 7mm(followed by the other) and this tripped the machine as the logic is 1oo1 for these probes in Mark-6.
3. It is important to note that the proximity probes did not read any abnormality during this time. It was only the seismic probes.

Note: We do not have any trending of probe value for both Seismic and Proximity Probes in Mark-6. Only events are logged i.e. if something goes above alarm limit or trip limit.

Our Instruments team has checked both the seismic probes by knocking on them and seeing the response signal on Mark-6, checked the cabling and the Bentley cards. All was found Okay. There was no seismic activity in area nor any sudden load change on generator i.e. All the parameters were running business as usual. (There was a road being made about 50m away form the generator and a roller was being used on it.)

The machine was started again after forcing the seismic probes and it is running normally now.

Questions?

I would request the forum to give their insights on following questions:

1. What can cause Seismic probe to read the values in this manner i.e. as mentioned above, one by one going above alarm level?
2. Can something inside the generator i.e. excitor/PMG assembly, cause the seismic probe to detect intermittently?
3. Is the Seismic probe even needed here when we have proximity probes as well? The machine is well protected with the proximity probes as well.
4. Should we change the trip logic on Seismic probes from 1oo1 to 1.5oo2 or 2oo2 and increase the limits?(Current values are for: Alarm: 4.5mm, Trip: 7.5mm)
5. What is recommended maintenance or replacement regime for Seismic probes on this application? Do they have a life after which they need replacement?
6. What more should we be looking at to identify Root- Cause of Event?

The generator was running normally, no sudden load changes and no lightening strikes or seismic activity observed/recorded in the areas.

We have already floated the queries to OEM, so please don't suggest that. Posting here to benefit from collective wisdom.

Looking forward to your responses.

Regards,
MRasool

 

It appears from your post that BOTH the seismic (velocity) sensors AND the proximity probes are connected to the same Bently-Nevada monitor.

Who is the manufacturer of the seismic (velocity) sensors on the generator (DE (Drive End) and NDE (Non-Drive End))?

Who is the manufacturer of the generator?

Who is the packager of the turbine-generator?

Typically, the seismic (velocity) sensors are mounted to the top of the bearing housing (sometimes called the "bearing caps"). They should be mounted using screws to which a medium-strength thread locking fluid/gel is applied AND some kind of safety wire (usually stainless steel) is used to secure the bolts from loosening/turning, not just lock washers. Sometimes the seismic (velocity) sensors are bolted to a small metal plate that is bolted to the bearing cap, and the bolt(s) holding that plate in place should also have medium-strength thread-locking fluid/gel applied AND some kind of restraint to keep them from loosening (safety wire or locking plates--not just lock washers).

Because of the mass of the generator rotor (and the turbine compressor, as well) the OEM's philosophy is generally that seismic (velocity) sensors were adequate protection against the damaging effects of high vibration (the generator rotor is relatively heavy and if there is some kind of vibration problem then the "bouncing" mass of the generator rotor will be enough for the seismic (vibration) sensors to detect the vibration and alarm/trip the unit.

With the advent of proximity probes and monitors, of which Bently-Nevada are (are at least were) the market leader in terms of share and adoption, GE reluctantly started adding them to their heavy duty gas turbines and generators. It was mostly because Customers were insisting on them, but GE soon realized that having the proximity probes was beneficial in helping to determine the magnitude of any vibration as well as being able (with additional equipment) to know where the problem might exist (bearing; imbalance; etc.) and quickly resolve it (instead of having to send special equipment and personnel to site in some cases. BUT, for many years, they still alarmed and tripped on the seismic (velocity) sensors--because if it ain't broke, why fix it, right?

But, then GE bought Bently-Nevada and suddenly it became the default vibration measuring/sensing system on their heavy duty gas turbines....! And, slowly, the seismic (velocity) sensors were removed from machines and alarming and tripping was accomplished using the proximity probes either directly connected to the Mark* or through the B-N monitor.

Many generators used by GE on their packages had one or two sensors on one end of the generator (usually the NDE end with the collector rings or sometimes a PMG and diode wheel (brushless exciter) which extended past the NDE bearing housing. On some machines there was only one sensor on the DE because the DE is usually coupled to either the turbine or to a reduction gear, as in your case. Most of the GE packages I worked on used a system that would require two seismic (velocity) sensors to be in alarm (above the alarm setpoint) AND one of the sensors to be above the trip setpoint to actually initiate a trip from the generator vibration seismic (velocity) sensors. We don't know if that kind of logic is used in the application code running in the Mark* VI of this machine, or who the packager of the turbine and generator is/was. Or who the generator manufacturer is.

If I recall correctly, the B-N monitor uses cards which can accept multiple seismic (velocity) sensors. If both of the generator sensors are connected to the same card in the B-N monitor, that makes that card common to both seismic (velocity) sensors and a definite suspect for the cause of the problem you are describing.

Also, if this happened after a recent maintenance outage when the generator bearing caps and/or seismic (velocity) sensors were removed and re-installed, that could also be a contributing factor to the problem. Perhaps the sensors weren't handled properly during removal, storage and re-installation, OR, the wiring terminations (if they were disturbed when the sensors were removed and re-installed) may not have been done properly, or the terminations weren't tight (INCLUDING the cabling where it connects to the sensor (if that cabling connects with a BNC-style connector, it may not have been properly tightened before safety wiring was installed).

Before making any changes to the logic/application code, you should be consulting the generator manufacturer for assistance with the logic they recommend.

This should get you pointed in the right direction for troubleshooting.

As for periodic maintenance of seismic (velocity) probes, I don't think very many sites do periodic maintenance on them (sounds like your site may be in that very large group also). You could get a "shaker table" and remove the sensors from the bearing caps and test them with that.

Based on the information provided, and the fact that the machine uses a Mark* VI turbine control system the machine is not new and the sensors could theoretically be replaced simply based on age (but many sites are reluctant to spend the money for new probes unless they are proven bad). (The control system might even be a Mark* VIe, which many people refer to as a Mark* VI (mistakenly). In this case, the machine is newer but not knowing who the sensor manufacturer is, it's difficult to say if it's "time" to replace the sensors or not.)

My money is on something wrong with either the B-N rack or slot where the seismic (velocity) probe interface card is located, or the interface card itself (especially if both generator seismic (velocity) sensors are connected to the same card). But that's the way I think, based on the information provided.

In my personal opinion, if a machine uses both seismic (velocity) sensors AND proximity probes the seismic sensors are "redundant" and should be secondary indication and protection to the proximity sensors and monitor. But, that's just my opinion.

Finally, if you have a Mark* VI (or Mark* VIe) then a trending function IS available in Toolbox (or ToolboxST). It's very powerful, it WON'T trip the turbine when practicing on how to configure it or if it runs in the background gathering data. Some versions of Trend Recorder (or Trender) can even have a function that monitors a logic point (say seismic (velocity) alarm setpoint exceeded) and capture some values BEFORE that event happened as well as AFTER the event (called a "triggered" trend).

You should ALSO have a Trip History function or something similar that captures preconfigured points and generates a "report" on each and every trip. It's possible the seismic (velocity) probe data isn't included in the preconfigured Trip History function--but they can typically be added using Toolbox (or ToolboxST), with some effort--but it's NOT impossible. That report should be available somewhere on the GE Mark* HMI, and when opened it can be exported to a .csv (comma-separated value) file for importation into some spreadsheet app for viewing when away from the GE Mark* HMI and Toolbox (or ToolboxST). Trend Recorder (or Trender) is a VERY POWERFUL tool for troubleshooting high-speed data especially--but just in general. It's not difficult to learn, and can even be used to capture data during normal STARTS and shutdowns so that when (perceived) problems do occur they can be compared to data captured during the suspect period/process. It's INFINITELY easier--and faster--than many data archival and retrieval systems (such as PI which is provided on many GE Historian computers; PI DOES NOT capture high-speed data by default, and configuring it to do so almost requires a Master's degree in computer science (and quite often plug-ins and extensions). Trend Recorder (or Trender) is FREE and included in Toolbox (or ToolboxST). (Anyone who invests a little time and effort can become the site "hero" by learning how to use Trend Recorder (or Trender), especially during trips and unstable events. And even just by being able to analyze the Trip History reports and data. HINT. HINT. Might be worth a little more money in a pay envelope.... Just sayin'.... The 'Help' feature of Trend Recorder (or Trender) is not too bad; I used it to become more proficient, but it didn't get me a raise... :-( )

Go forth and conquer!

Many people would be very grateful to know what you find and how you resolve the problem(s).

 

It appears from your post that BOTH the seismic (velocity) sensors AND the proximity probes are connected to the same Bently-Nevada monitor.

Who is the manufacturer of the seismic (velocity) sensors on the generator (DE (Drive End) and NDE (Non-Drive End))?

Who is the manufacturer of the generator?

Who is the packager of the turbine-generator?

Typically, the seismic (velocity) sensors are mounted to the top of the bearing housing (sometimes called the "bearing caps"). They should be mounted using screws to which a medium-strength thread locking fluid/gel is applied AND some kind of safety wire (usually stainless steel) is used to secure the bolts from loosening/turning, not just lock washers. Sometimes the seismic (velocity) sensors are bolted to a small metal plate that is bolted to the bearing cap, and the bolt(s) holding that plate in place should also have medium-strength thread-locking fluid/gel applied AND some kind of restraint to keep them from loosening (safety wire or locking plates--not just lock washers).

Because of the mass of the generator rotor (and the turbine compressor, as well) the OEM's philosophy is generally that seismic (velocity) sensors were adequate protection against the damaging effects of high vibration (the generator rotor is relatively heavy and if there is some kind of vibration problem then the "bouncing" mass of the generator rotor will be enough for the seismic (vibration) sensors to detect the vibration and alarm/trip the unit.

With the advent of proximity probes and monitors, of which Bently-Nevada are (are at least were) the market leader in terms of share and adoption, GE reluctantly started adding them to their heavy duty gas turbines and generators. It was mostly because Customers were insisting on them, but GE soon realized that having the proximity probes was beneficial in helping to determine the magnitude of any vibration as well as being able (with additional equipment) to know where the problem might exist (bearing; imbalance; etc.) and quickly resolve it (instead of having to send special equipment and personnel to site in some cases. BUT, for many years, they still alarmed and tripped on the seismic (velocity) sensors--because if it ain't broke, why fix it, right?

But, then GE bought Bently-Nevada and suddenly it became the default vibration measuring/sensing system on their heavy duty gas turbines....! And, slowly, the seismic (velocity) sensors were removed from machines and alarming and tripping was accomplished using the proximity probes either directly connected to the Mark* or through the B-N monitor.

Many generators used by GE on their packages had one or two sensors on one end of the generator (usually the NDE end with the collector rings or sometimes a PMG and diode wheel (brushless exciter) which extended past the NDE bearing housing. On some machines there was only one sensor on the DE because the DE is usually coupled to either the turbine or to a reduction gear, as in your case. Most of the GE packages I worked on used a system that would require two seismic (velocity) sensors to be in alarm (above the alarm setpoint) AND one of the sensors to be above the trip setpoint to actually initiate a trip from the generator vibration seismic (velocity) sensors. We don't know if that kind of logic is used in the application code running in the Mark* VI of this machine, or who the packager of the turbine and generator is/was. Or who the generator manufacturer is.

If I recall correctly, the B-N monitor uses cards which can accept multiple seismic (velocity) sensors. If both of the generator sensors are connected to the same card in the B-N monitor, that makes that card common to both seismic (velocity) sensors and a definite suspect for the cause of the problem you are describing.

Also, if this happened after a recent maintenance outage when the generator bearing caps and/or seismic (velocity) sensors were removed and re-installed, that could also be a contributing factor to the problem. Perhaps the sensors weren't handled properly during removal, storage and re-installation, OR, the wiring terminations (if they were disturbed when the sensors were removed and re-installed) may not have been done properly, or the terminations weren't tight (INCLUDING the cabling where it connects to the sensor (if that cabling connects with a BNC-style connector, it may not have been properly tightened before safety wiring was installed).

Before making any changes to the logic/application code, you should be consulting the generator manufacturer for assistance with the logic they recommend.

This should get you pointed in the right direction for troubleshooting.

As for periodic maintenance of seismic (velocity) probes, I don't think very many sites do periodic maintenance on them (sounds like your site may be in that very large group also). You could get a "shaker table" and remove the sensors from the bearing caps and test them with that.

Based on the information provided, and the fact that the machine uses a Mark* VI turbine control system the machine is not new and the sensors could theoretically be replaced simply based on age (but many sites are reluctant to spend the money for new probes unless they are proven bad). (The control system might even be a Mark* VIe, which many people refer to as a Mark* VI (mistakenly). In this case, the machine is newer but not knowing who the sensor manufacturer is, it's difficult to say if it's "time" to replace the sensors or not.)

My money is on something wrong with either the B-N rack or slot where the seismic (velocity) probe interface card is located, or the interface card itself (especially if both generator seismic (velocity) sensors are connected to the same card). But that's the way I think, based on the information provided.

In my personal opinion, if a machine uses both seismic (velocity) sensors AND proximity probes the seismic sensors are "redundant" and should be secondary indication and protection to the proximity sensors and monitor. But, that's just my opinion.

Finally, if you have a Mark* VI (or Mark* VIe) then a trending function IS available in Toolbox (or ToolboxST). It's very powerful, it WON'T trip the turbine when practicing on how to configure it or if it runs in the background gathering data. Some versions of Trend Recorder (or Trender) can even have a function that monitors a logic point (say seismic (velocity) alarm setpoint exceeded) and capture some values BEFORE that event happened as well as AFTER the event (called a "triggered" trend).

You should ALSO have a Trip History function or something similar that captures preconfigured points and generates a "report" on each and every trip. It's possible the seismic (velocity) probe data isn't included in the preconfigured Trip History function--but they can typically be added using Toolbox (or ToolboxST), with some effort--but it's NOT impossible. That report should be available somewhere on the GE Mark* HMI, and when opened it can be exported to a .csv (comma-separated value) file for importation into some spreadsheet app for viewing when away from the GE Mark* HMI and Toolbox (or ToolboxST). Trend Recorder (or Trender) is a VERY POWERFUL tool for troubleshooting high-speed data especially--but just in general. It's not difficult to learn, and can even be used to capture data during normal STARTS and shutdowns so that when (perceived) problems do occur they can be compared to data captured during the suspect period/process. It's INFINITELY easier--and faster--than many data archival and retrieval systems (such as PI which is provided on many GE Historian computers; PI DOES NOT capture high-speed data by default, and configuring it to do so almost requires a Master's degree in computer science (and quite often plug-ins and extensions). Trend Recorder (or Trender) is FREE and included in Toolbox (or ToolboxST). (Anyone who invests a little time and effort can become the site "hero" by learning how to use Trend Recorder (or Trender), especially during trips and unstable events. And even just by being able to analyze the Trip History reports and data. HINT. HINT. Might be worth a little more money in a pay envelope.... Just sayin'.... The 'Help' feature of Trend Recorder (or Trender) is not too bad; I used it to become more proficient, but it didn't get me a raise... :-( )

Go forth and conquer!

Many people would be very grateful to know what you find and how you resolve the problem(s).

Hello,

First off, thank you very much for the detailed reply. It was insightful.

Now to answer your questions:

1- Make of Generator is BRUSH Electric.(Commissioned in 2008)
2- Packager is Nuovo Pignone.
3- The Seismic Sensors are Bentley Velomitor® Sensors.
4- We have a Mark-6. (Not a Mark-6e)

With regards to the mounting tips you have given, although we do use a lock nut, we do-not use a thread locking fluid like locktite nor do we use a lock-wire/safety-wire. We will incorporate your recommendations in our practice in future. In our current scenario however, this can not be a source since both the Seismic sensors started going above and below the alarm level together. They are considerably apart i.e. one at DE other at NDE, so I can't think looseness can occur at both the same time.

I was checking the logs again and the S.O.E(Sequence-of-Events) is that both the seismic probes start to go above the alarm threshold and then return below and then go up again and then return below... this keeps on happening for 5 to 6 minutes before one of them reaches the trip limit(followed by the other) and trips the machine.

So our first bet was to check the 'common-point' of both the sensors as well. Which in the B-N card. You are correct in guessing, as in our case both the Seismic probes terminate at the same card. The communication then from B-N to Mark-6, is on Modbus. The card was checked by running the Module Self-test, which was found to be OKAY. So we did not look further on this side. Is there something else we can check at the B-N panel?

The interesting thing is that after forcing this security, when the machine was started, it ran smoothly and has been running smoothly for quite some days now. No alarm on the seismic sensors as well. It's like it was a ghost that came and went. But we all know too well, nothing happens by itself.

As for the 'Trip History Function', you are right, the Analog values of Seismic probes are not configured in it. The Mark-6 HMI came pre-configured and none of us tried to modify what was initially configured in the trending/history or toolBox. Something we should do now.

We don-not have a shaker-table for checking the probes. However the field techs use the old knock-and-check technique. They knock the sensor in the field and check the response on mark-6 HMI. This is always done when the machine is shutdown for whatever reason.

Can I have your email? Will share some more data.

Thank you once again.

-MRasool

 

Mutahir...
1. Do the trips occur with some regularity with regard to activity, such as shift change?
2. Do the trips occur with some regularity with regard to time of day?
3. Are there other physical operations, such as heavy construction, occurring nearby?
Regards, Phil Corso

 

@MRasool,

"Can I have your email? Will share some more data."

I happen to share a previous contributor's position that taking a thread "off-line" by using private emails to share data and troubleshooting deprives the many other people that read and follow these posts to gain knowledge and learn what works--and what doesn't. This thread has already been viewed 57 times (as of this writing), and another post from just a couple of months ago has already been viewed 577 times--so a lot of people read and follow threads on this forum. Both now, and in the future since all threads and posts are archived and can be searched using the very fast and powerful 'Search' feature available at the top of every Control.com page (the magnifying glass icon).

Okay; so the anecdotal data has changed; because if I recall correctly you were saying that one generator velocity sensor would indicate a high value then go back down while the other sensor then went high and went back down and at some point at least one went over the trip setpoint (and I assume the other sensor was also in alarm at that time.?.?.?). This is why hard, "actionable" data is so helpful in troubleshooting these kinds of problems. Files can be attached to posts in Control.com and can be very informative to those of us who are not there with you to see what's going on and can't see the data you're looking at.

Shaker tables can be rented in most industrialized areas of the world; they're small and relatively inexpensive. I've even some relatively unused looking shaker tables available on ebay, and there are other industrial equipment websites that specialize in previously used instruments and test equipment. (By the way, tapping on velocity sensors can damage them if appropriate care is not used when doing so. Personally, I used the handle of a screwdriver (wooden if I could find one) and just a very gentle tap on the top for the most basic "loop check" test of a velocity sensor circuit. I've watched people use a ball peen hammer and beat velocity sensors not just on the tops but also the sides (???). True stories (because I've witnessed it several times more than just once...).

Have you checked the generator shaft grounding brush(es)? Is (Are) the brushes in good condition and not hardened? You say the machine has a PMG brushless exciter. Are you certain all of the components of the system are in good working order? Is the site required to use intrinsically safe (IS) barriers for devices and instruments? If so, are they used for the velocity sensors on the generator, and do you know for a fact they are working correctly and not experiencing intermittent problems? Is the shield drain wires of the low-level twisted shielded pair wiring properly grounded?

It would seem that some kind of "stray" voltage or currents might be affecting the generator velocity sensors.

It's been said many times before on Control.com, by many contributors: Often, troubleshooting is process of elimination. One makes a list of all the potential causes, then tries as best as possible to prioritize the list based on highest likelihood of the cause actually being the problem (including loose screw terminals and poor wire crimps (sometimes just tugging on wires in crimp terminals can result in a wire or wires coming loose because of a poor crimp)), and then working through that list. Each time something is proven to NOT be the cause that cause is eliminated. (Personally, on machines that have been running for a decade or more, I like to check things like wiring, screw terminations and crimp terminals (especially on small gauge wire!) first and am often rewarded with quick success this way. Not always, but often enough it's one of my staple troubleshooting methods for intermittent problems.

Have you removed and replaced the common velocity sensor card in the B-N monitor rack, several times? If the site has corrosive atmospheres that can cause issues with backplane connections and sometimes just unseating and reseating cards can loosen/remove corrosion that has built up over time. A combination of humidity and dust in the room where the B-N racks are located can also cause connection problems.

Again, the list for troubleshooting an intermittent problem must be as detailed as possible, and not only that, the test procedures need to detail what is considered a positive result (if I had a nickel (a USD0.05 coin) for every time I've heard, "We tested that but it didn't solve the problem!" and yet "that" turns out to be the problem when properly tested I would be a much richer man than I am (at least monetarily, anyway; I'm rich beyond my wildest dreams in many ways in my life, especially now).

Lastly, I AM NOT a fan of using MODBUS for high-speed data gathering (and, yes, vibration qualifies as high-speed on large rotating equipment, though I note the generator is a four-pole, 50 Hz machine (operating at 1500 RPM)). A LOT can happen in the time between MODBUS data transmissions--so a LOT can be unseen using such a scheme, and TRIPPING on signals transmitted using MODBUS, too...that's kind of scary (to me, anyway)!!! (If I understand what you're saying correctly.) I'm NOT saying the MODBUS link is a problem, only that a lot of data can be unmonitored between data transmissions on large rotating equipment (especially high-speed rotating equipment). Does the B-N rack have the capability of outputting 4-20 mA signals based on the velocity sensor inputs? If so, they could be connected to the Mark VI for a more dedicated indication.

 

Mutahir...
1. Do the trips occur with some regularity with regard to activity, such as shift change?
2. Do the trips occur with some regularity with regard to time of day?
3. Are there other physical operations, such as heavy construction, occurring nearby?
Regards, Phil Corso

Hello Phil!

I hope you are doing well.

1. Do the trips occur with some regularity with regard to activity, such as shift change?
No, this actually happened for the first time since commissioning of this machine. Time was almost a shift change i.e. morning outgoing and evening incoming.

2. Do the trips occur with some regularity with regard to time of day?
This kind of trip on Seismic probes happened for the first time since 2008.

3. Are there other physical operations, such as heavy construction, occurring nearby?
There is a road being constructed about 50M away from the generator and they were using a roller on the road at that time. My though went towards that activity as well but a roller simply presses the surface, doesn't knock. So how can it generate vibrations.

Regards,
Mutahir

 

@MRasool,

"Can I have your email? Will share some more data."

I happen to share a previous contributor's position that taking a thread "off-line" by using private emails to share data and troubleshooting deprives the many other people that read and follow these posts to gain knowledge and learn what works--and what doesn't. This thread has already been viewed 57 times (as of this writing), and another post from just a couple of months ago has already been viewed 577 times--so a lot of people read and follow threads on this forum. Both now, and in the future since all threads and posts are archived and can be searched using the very fast and powerful 'Search' feature available at the top of every Control.com page (the magnifying glass icon).

Okay; so the anecdotal data has changed; because if I recall correctly you were saying that one generator velocity sensor would indicate a high value then go back down while the other sensor then went high and went back down and at some point at least one went over the trip setpoint (and I assume the other sensor was also in alarm at that time.?.?.?). This is why hard, "actionable" data is so helpful in troubleshooting these kinds of problems. Files can be attached to posts in Control.com and can be very informative to those of us who are not there with you to see what's going on and can't see the data you're looking at.

Shaker tables can be rented in most industrialized areas of the world; they're small and relatively inexpensive. I've even some relatively unused looking shaker tables available on ebay, and there are other industrial equipment websites that specialize in previously used instruments and test equipment. (By the way, tapping on velocity sensors can damage them if appropriate care is not used when doing so. Personally, I used the handle of a screwdriver (wooden if I could find one) and just a very gentle tap on the top for the most basic "loop check" test of a velocity sensor circuit. I've watched people use a ball peen hammer and beat velocity sensors not just on the tops but also the sides (???). True stories (because I've witnessed it several times more than just once...).

Have you checked the generator shaft grounding brush(es)? Is (Are) the brushes in good condition and not hardened? You say the machine has a PMG brushless exciter. Are you certain all of the components of the system are in good working order? Is the site required to use intrinsically safe (IS) barriers for devices and instruments? If so, are they used for the velocity sensors on the generator, and do you know for a fact they are working correctly and not experiencing intermittent problems? Is the shield drain wires of the low-level twisted shielded pair wiring properly grounded?

It would seem that some kind of "stray" voltage or currents might be affecting the generator velocity sensors.

It's been said many times before on Control.com, by many contributors: Often, troubleshooting is process of elimination. One makes a list of all the potential causes, then tries as best as possible to prioritize the list based on highest likelihood of the cause actually being the problem (including loose screw terminals and poor wire crimps (sometimes just tugging on wires in crimp terminals can result in a wire or wires coming loose because of a poor crimp)), and then working through that list. Each time something is proven to NOT be the cause that cause is eliminated. (Personally, on machines that have been running for a decade or more, I like to check things like wiring, screw terminations and crimp terminals (especially on small gauge wire!) first and am often rewarded with quick success this way. Not always, but often enough it's one of my staple troubleshooting methods for intermittent problems.

Have you removed and replaced the common velocity sensor card in the B-N monitor rack, several times? If the site has corrosive atmospheres that can cause issues with backplane connections and sometimes just unseating and reseating cards can loosen/remove corrosion that has built up over time. A combination of humidity and dust in the room where the B-N racks are located can also cause connection problems.

Again, the list for troubleshooting an intermittent problem must be as detailed as possible, and not only that, the test procedures need to detail what is considered a positive result (if I had a nickel (a USD0.05 coin) for every time I've heard, "We tested that but it didn't solve the problem!" and yet "that" turns out to be the problem when properly tested I would be a much richer man than I am (at least monetarily, anyway; I'm rich beyond my wildest dreams in many ways in my life, especially now).

Lastly, I AM NOT a fan of using MODBUS for high-speed data gathering (and, yes, vibration qualifies as high-speed on large rotating equipment, though I note the generator is a four-pole, 50 Hz machine (operating at 1500 RPM)). A LOT can happen in the time between MODBUS data transmissions--so a LOT can be unseen using such a scheme, and TRIPPING on signals transmitted using MODBUS, too...that's kind of scary (to me, anyway)!!! (If I understand what you're saying correctly.) I'm NOT saying the MODBUS link is a problem, only that a lot of data can be unmonitored between data transmissions on large rotating equipment (especially high-speed rotating equipment). Does the B-N rack have the capability of outputting 4-20 mA signals based on the velocity sensor inputs? If so, they could be connected to the Mark VI for a more dedicated indication.

Hello,

I am all for sharing knowledge as well, however few documents you can't share on public forums.

Yes, I was rechecking the log and I noticed that both the vibration probes are going to alarm levels together and coming back together as well. However, there is a delay of 1 to 2 secs between them, which is not significant enough to state them as sensing one-by-one(my previous understanding). So I corrected my SOE as well.

I agree with your concerns on tap checks, but we do it carefully and it is never done on the probe but on area around it. We are planning to procure a shaker-table now.

The machine has a PMG Brushless excitor indeed and we have not checked the shaft grounding brushes presently. The machine was taken down in Nov 2022 and checks were made at that time. As for the PMG and excitor, had there been any issue on those components, the machine would be responding accordingly. All excitation parameters were normal before the event and even after start-up again, these parameters are okay. Why do you think this can be linked to the issue? Have you experienced anything of the sort or heard about it? I actually did want to know if the excitation system can somehow impact the Seismic values?

No we don't use IS barriers on the GT. I think they create one more points of failure... just saying.

As for the shield drain wires of the twisted pair cable, I check in the panel the it was connected to ground. However, didn't confirm it with meter. I will have the grounding checked.

As for the plugging in and out the cars... no. We don't touch the panel when everything is running smoothly. These panels are opened only in shutdowns. Last was in Nov 2022. The B-N panel is placed in a switchgear room with 26 degree C maintained temp, and no humidity. However out side there is 40 to 45 degree C. There is dust in the panel though.

As for the Modbus, the trip signal is hard wired to Mark-6. Only the Analog monitoring is done through Modbus and It has been working well for us so far but I get your point. There is an inherent delay. What do you suggest?
The B-N cards don't have a direct 4-20mA outputs. It has a relay output card that sends trip signals to Mark-6.

You are right about troubleshooting, elimination is the way to go. We are also checking things one-by-one. But the fact that everything went to normal since we restarted the machine, without changing anything, is a bit confounding.

Will update more as we go along. Thanks for chipping in and suggesting things!

Regards,
MRasool

 

@WTF?
Can the power supply to B-N cards or the Vibration cards cause this issue? I mean if there is a power dip in DC circuit or some sort of intermittent supply failure? Have you witnessed anything of this sort due to power supply issues?

Regards,
MRasool

 

MANY “rollers” used for road construction have a hydraulic vibrating feature that helps to compact soil and tarmac/asphalt/pavement. So PhilCorso may be on to something with this question…. However, it’s somewhat difficult to understand how those vibrations could be sensed by the velocity sensors and not by the proximity probes. A four-pole generator rotor is probably very large and heavy, though, and it might not actually move but the bearing supports might—though that would still be relative motion between the bearing supports (in which the proximity probes are mounted) and the generator shaft so one would still think the proximity probes would have detected movement.

Yes; I have seen generator shaft currents cause problems with instruments mounted on a generator (including vibration sensors). But it wasn’t intermittent; it was steadily increasing over time (weeks to be exact).

One thing about B-N equipment is that it is pretty robust. It usually either works, or it doesn’t. There’s very little “middle ground” (no reference to earthing/grounding).

Very odd scenario. I am not at all familiar with B-N velocity sensors. AND the way the vibration system is being used is just very unusual—to my experience. MODBUS for monitoring and hard-wired discrete signals for tripping (and alarming I presume) just seems odd, also.

IS barriers are usually passive devices, but can be damaged by earthing issues as well as nearby lightning strikes and voltage spikes. But for the most part they are pretty robust. Properly applied, I have seen very few failures of IS barriers (but have long considered them to be suspects in troubleshooting intermittent problems but can honestly say they have not been the cause of many problems in my experience.

Sometimes one can over-analyze a situation and in the process overlook something which wouldn’t really require much time or effort to check (unseating and reseating cards from a backplane mount, for example). But keeping good written notes about what was done and the results (as well as how it was done) can be very helpful in the long term. It’s one of the reasons why I am not a fan of the shotgun approach to troubleshooting—trying multiple things at once, because if the problem is resolved there’s no clear indication of the which thing was the actual problem. Of course, most people are just happy the problem was resolved; but if the root cause is not identified and understood I still have fears about whether or not the problem was actually resolved or what the fix was. My technician mind wants to know the root cause and the solution or it just doesn’t seem right.

 

Mutahir...
Last question (I promise)!
5. How about details on generator enclosure: Drip proof; TEFC; air cooled with air from non-hazardous area. If the latter what is shaft-bushing material? If your answer is positive, then the problem could be related to something called the "MORTON" effect. It's a phenomenon related to machines with synchronously-related vibration!
I hope all on this forum have happy, and safe Holidays!
Regards, Phil Corso

 

Mutahir...
Last question (I promise)!
5. How about details on generator enclosure: Drip proof; TEFC; air cooled with air from non-hazardous area. If the latter what is shaft-bushing material? If your answer is positive, then the problem could be related to something called the "MORTON" effect. It's a phenomenon related to machines with synchronously-related vibration!
I hope all on this forum have happy, and safe Holidays!
Regards, Phil Corso

Hey Phil,

Sorry for the late response as I was away on leaves. Happy new year to you!

Getting straight to your questions:

5. How about details on generator enclosure: Drip proof; TEFC; air cooled with air from non-hazardous area. If the latter what is shaft-bushing material? If your answer is positive, then the problem could be related to something called the "MORTON" effect. It's a phenomenon related to machines with synchronously-related vibration!

The generator is TEFC, air-cooled from a Hazardous area actually i.e. Class 1 Div2 Group IIA. Shaft Material is Solid Steel. Can you explain what you mean by Shaft Bushing? Do you mean the sleeve bearing inner face material(bearing is RENK EM type with external cooling oil)?

Thanks for sharing about Morton Effect. I read about it but can't be sure about it, I need more case studies(If you have any I will be delighted to read.) From the text I read, the shaft needs to have a significant overhang/coupling for it to bend due to thermal impact in case of Morton Effect. At NDE side, there is no overhang, as PMG and the excitor are inside of the bearing. However at the DE side, we have a gearbox which can be considered as an overhang but not too sure about it.

Regards,
Mutahir

PS: Happy New Year to all of you!

 

Mutahir...
Thanks for your Blessing! Even celebrated our 65th Wedding Anniversary, and 7 more Great-Grand-Kids!

The reason I asked about enclosure detail is "The Rest of the Story"!

A 5-year old 21,000 Hp induction-motor, driving a refinery compressor, was returned to service after some routine maintenance! And yes, its enclosure was "TEFC". When started It began to trip randomly. There was no recognizable pattern, i.e., day or night, weekday or weekend, rain or shine, shift-change, all weather conditions, space-shuttle launches, even during different phases of the moon, or ocean tides (the refinery was located near an ocean), etc., etc.!

A 15-man team from every known engineering discipline (Before AI) was assembled. I was the "Electrical-side"! It also included Professor J. P. Den Hartog, considered then as the father of Vibration Analysis! The team also included a Psychologist (looking for an irate employee), the manufacture's Engineers, as well as a competitive company's Engineers.

After 1 week, Professor Den Hartog exclaimed, "Yep, it's Classic Morton's" and left the site! After 2 more weeks of head-scratching scrutiny, it seemed impossible for the team to solve the problem. Actually one test was to lift the motor's shaft (while rotating under power) and "bounce" it. However, we did learn that by immediately restarting after a trip, production was maintained.

The problem was eventually traced to shaft-rub! It occurred where the shaft passed through a wood bushing (yes, I said wood) in the motor's fiber-glass enclosure. A slight unbalance of the motor’s rotor produced the rub, which, in turn generated enough heat to bow (bend) the motor’s 6-8" Diameter shaft, thus tripping the seismic vibration sensors.

The cause was that the motor's rotor-structure (part holding the rotor's magnetic-iron to the shaft) was "Y" shaped and not the old-fashioned "+" shape. The manufacturer eventually admitted it was a new, never used before, design!
The fix was to replace the motor. I will have a picture in a few days.
Regards, Phil Corso

 

Mutahir...
Thanks for your Blessing! Even celebrated our 65th Wedding Anniversary, and 7 more Great-Grand-Kids!

The reason I asked about enclosure detail is "The Rest of the Story"!

A 5-year old 21,000 Hp induction-motor, driving a refinery compressor, was returned to service after some routine maintenance! And yes, its enclosure was "TEFC". When started It began to trip randomly. There was no recognizable pattern, i.e., day or night, weekday or weekend, rain or shine, shift-change, all weather conditions, space-shuttle launches, even during different phases of the moon, or ocean tides (the refinery was located near an ocean), etc., etc.!

A 15-man team from every known engineering discipline (Before AI) was assembled. I was the "Electrical-side"! It also included Professor J. P. Den Hartog, considered then as the father of Vibration Analysis! The team also included a Psychologist (looking for an irate employee), the manufacture's Engineers, as well as a competitive company's Engineers.

After 1 week, Professor Den Hartog exclaimed, "Yep, it's Classic Morton's" and left the site! After 2 more weeks of head-scratching scrutiny, it seemed impossible for the team to solve the problem. Actually one test was to lift the motor's shaft (while rotating under power) and "bounce" it. However, we did learn that by immediately restarting after a trip, production was maintained.

The problem was eventually traced to shaft-rub! It occurred where the shaft passed through a wood bushing (yes, I said wood) in the motor's fiber-glass enclosure. A slight unbalance of the motor’s rotor produced the rub, which, in turn generated enough heat to bow (bend) the motor’s 6-8" Diameter shaft, thus tripping the seismic vibration sensors.

The cause was that the motor's rotor-structure (part holding the rotor's magnetic-iron to the shaft) was "Y" shaped and not the old-fashioned "+" shape. The manufacturer eventually admitted it was a new, never used before, design!
The fix was to replace the motor.
Regards, Phil Corso

Happy Anniversary Phil! May your family always find love in their hearts and blessings on their table.

Interesting case study you shared. Thanks. In our case, we had proximity and seismic probes both installed on bearings, so had it been Morton, the proximity probes would have picked up something as well.

We will keep on investigating and share what we find.

Sincerely,
Mutahir

 

Mutahir...
I believe you have misunderstood my interpretation of the "Morton Effect". It is bending of a shaft because of heating occurring at just one point, but the same point each revolution of the shaft!
Since it only occurred once in your case, then you are lucky. If it happens again then use a Forehead thermometer!
Regards, Phil Corso

 

Mutahir...
To satisfy my own curiosity:
Did you notice a smooth increase in vibration magnitude of any non-Seismic sensors for a short interval prior to 'trip'?
Regards, Phil Corso

 

Hey Phil,

Well, we did not observe any increase in Vibration magnitude of any proximity sensors prior to trip. Actually it would have been so much easier had we configured trending on these values. Unfortunately in the packaged unit only events triggering is captured i.e. if vibration increases beyond a certain set point. If the vibration is increasing or decreasing within the event setpoint, it just displays it in real-time and does not capture any trend. So it can be the case that there was a slight increase in proximity sensor values prior to the event, which was not picked up by GT operator and only when it tripped on seismic probe values, he gave attention. But when I asked the operator, he said he did not pick up and increase in proximity probes values.

I have written to OEM 'BRUSH' about it as well.

Regards,
Mutahir