My current level of understanding regarding automation/ process control is very limited at this stage in my research. My task is to develop survey that shows the current uses of SCADA technology in the water industry and barriers they face/reasons why systems are not being used to their fullest extent. It has been relatively easy to find out what can be done in regards to remote monitoring and control WRT water treatment and distribution systems with all the available info on the net (including this site which I have found to be particularly useful).

Not being a system user/ developer it is hard for me however to understand the reasons why many water utilities do not have the most sophisticated SCADA system possible implemented to protect us on something as vital to national infrastructure and security as water supply. I have visited a few water/wastewater treatment facilities and spent extensive hours on the internet trying to understand why systems across the country do not continuously upgrade their systems and what are the main barriers that exist when trying to add new functions to a SCADA system such as various water quality sensors, energy management equipment/sub-metering , security monitoring equipment, maintenance prediction, etc.

So far the number one item I have heard/read would be lack of money/time. Other reasons (correct me if I am wrong) include; no onsite (in house) SCADA expert to perform upgrades, HMI/SCADA software limitations, incompatibility of 3rd party components with proprietary systems, and many others I am sure. If any of you have personal experience with upgrading SCADA systems I would greatly appreciate hearing about problems you may have had or why upgrading a system is not as easy as just plugging in a new sensor. Reasons I am interested in would be from both the management and technical perspective.

Thanks,
Jesse

 

The biggest reason may be that there is no good reason to do many of the things you suggest. people only want to spend money for good reasons. Unless there is some reasonable payback, or the upgrade is needed to keep something in service that is considered important enough, why spend the money?

--
Bob

 

Hi Jesse,

Within the water treatment and distribution arena, I see many that do continuously upgrade their systems, and it generally turns out well, since they keep their systems from turning obsolete. It pays off in the long run.

I also see many that don't, and the reason is usually keeping (short term) expenses down.

BTW... Are you a student at Missouri-Columbia, MO, or the School of Mines/Metallurgy-Rolla, MO? (I'm a graduate from the latter.)

 

I contest your built-in-assumption that the latest and greatest in SCADA, the implementation of which will require constant upgrades, is by definition the "safest" most conservative alternative, the one which is going to best protect such unquestionably vital infrastructure.

In industrial type applications, change is a source of risk. Is there a bug in the new SCADA? What if something gets broken because of the non-routine procedures performed during the upgrade process itself? Will it require a new version of the computer OS that will break something else?

Not saying that a plant should never ever perform an upgrade, but please realize that for critical applications, any risky procedure which is even arguably unnecessary or optional, will have to offer VERY substantial guaranteed benefits to even consider undertaking the risk.

 

Steve,
Thank you, this is exactly the type of information I am looking for. I agree with you, change is risky and making unsubstantiated changes, especially in regards to water treatment, could pose serious risks. But I am curious, especially in regards to water distribution security, does "if its not broke don't fix it" still hold true? In my prior military service, I found that government entities especially, usually do not fix something until a problem arises (i.e Started Iraq war with unarmored Humvee's....finished with up-armored vehicles named JERRVS after many deaths had already occurred).

If a utility has never had a problem with, lets say an intentional attack/poisoning of a water supply, does that mean they should not have some kind of event detection capabilities? Is cost still the driving force? In my mind implementing additional sensors should not pose risks unless the current system cannot accommodate them. The reasons you all stated will be very useful to me.

My goal is to develop a list of reasons why it is difficult/why one would not upgrade a SCADA systems when the technology is out there, and implement the reasons in to my survey. People do not like long surveys, so I would like to have some pre-determined answers to speed things up in addition to educating myself. @scadametrics I am at the Columbia campus, and I am a Civil Engineering major.

BTW Thank you all so much for your input.

 

When did you graduate? I used to work with a couple UMR boys. Both excellent engineers!

 

I think "if it ain't broke, don't fix it" is probably a little too strong. There are times when the guaranteed or nearly guaranteed advantages of an upgrade make the risk worthwhile. It's just that the managers have to weigh the risks against the benefits rather than plunging ahead wanting the benefits while assuming there are no risks.

Yes, it definitely makes sense to think about hazards that have never struck your plant and think about how to prevent or deter them, as well as how to mitigate the effects should they occur.

I think you're talking about two types of control upgrade scenarios in your questions, or at least two different points on a continuum. When you say "upgrade SCADA" that puts me in mind of a system-wide replacement of the control system software. Then OTOH you're talking about adding monitoring for parameters that may help detect a hazard (intentional attack or otherwise). This is a whole different kettle of fish than a "system upgrade". The risk of some disaster occurring due to implementation is lower, so the bar to committing to that kind of an upgrade is lower.

 

Jesse,
I thought and still think like you do. I work in power, but at a public utility. There are many cases where spending a few bucks and upgrading stuff will give great returns IF DONE CORRECTLY. But there is a risk associated with everything (including the status quo).

With that said, many reasons wise things aren't done is because unwise people are in charge. It's that simple.

In the real world, very few people are innovators and forward thinking. In the public utility world (of which most water folks are), there is even less forward thinking.

In the power generation sector (and I'm generalizing here), the plants that strive to use the most cost effective and efficient approaches like upgrading automation to reduce workload, improve reliability and efficiency, etc...are usually the independent guys that have to make a profit--the IPP's. Public utilities have the mentality to provide the service and all is well as long as reliability is there AND the regulating board isn't bugging them about asking for too high of a rate.

As engineers we tend to overlook the human aspects. When people feel no pain with the status quo, they have no reason to change. The public is happy because the water flows or the lights are on. They don't know any better about rates, etc. Then you have the politicians that get involved.

And from your experience in the military you can see how bureaucracy is ALWAYS reactive, not proactive. And this is why entrepreneurs are the innovators of the world.

 

>When did you graduate?<

Steve... 1988.

 

I think you were probably behind my guys then. Jeff Bacon and Roger Herrscher, probably graduated in the 83-84 time frame IIRC.

 

Well, if it isn't broken why upgrade it? There has to be some value to the new SCADA that offsets the costs of upgrading. Even if the upgrade is "free" there are costs--downtime for the upgrade, retraining operators and maintenance people, establishing new procedures and documentation, etc.

In my experience in the energy industry, SCADA and HMI systems are only upgraded A) when the hardware fails and the site has no choice because the hardware is obsolete, or B) the site is upgrading the entire control system anyway and the SCADA is bundled. Occasionally I've seen a company with lots of sites upgrade SCADA at all sites to establish a common platform.

Sites are almost never upgraded just for prettier graphics; the one exception I can remember was a "model" site that the customer used for VIP tours.

All that said, regulatory requirements for access control, logging, auditing, and cybersecurity are going to force a lot of SCADA upgrades in the near future.