AI-Generated Code: To Trust or To Verify?

Artificial intelligence is rapidly disrupting the way engineers develop software nowadays. AI-enabled tools with the capability of generating code from natural language prompts can now create scripts, database queries, application logic, and even portions of industrial control programs in a matter of seconds.

Using AI-enabled tools in industrial controls has brought significant potential for increased productivity. AI can substantially accelerate control engineering tasks that previously required extensively searching through documentation, manually reviewing old code, or writing repetitive subroutines from scratch.

Figure 1. AI assists, but the engineer remains in control. All images used courtesy of the author and generated with ChatGPT, since, after all, this is an article about AI

However, although using AI assistance has many benefits, it does not eliminate the fundamental responsibility of engineering, which is to ensure that the solution actually works. This is particularly important in industrial automation, where writing code is not even the hardest part of the job. Engineers in this domain need to understand the process, define correct (and incorrect) system behavior, account for exceptions, and validate the system, all before typing even the first line of code.

This is the reason why AI is best viewed as an aid, often referred to as a copilot. AI can help us develop solutions faster, but it does not replace human judgment.

The Value of AI-Generated Code

One of the strongest applications for AI in controls engineering is reducing the time developers spend on well-defined, repetitive coding tasks. Some examples of helpful AI tasks might include creating a Python script to process production data, generating an SQL query, or providing an initial structure for a new subroutine. In a way, these applications are agentic-like.

AI can also help when working with unfamiliar code or when learning a new programming language. It can interpret the program and provide insights into its purpose, architecture, and working mechanisms, so that the developer can spend less time manually interpreting line-by-line and get started on their own task sooner.

Figure 2. AI accelerates development, allowing the engineer to focus on what matters.

Artificial intelligence does not necessarily produce better code than an experienced developer (although it might, in some cases). The advantage is speed. With AI, the developer can begin a project with a proposed solution already established instead of just an empty screen. As we all know, in many engineering applications, getting from zero to the first working concept consumes a significant amount of time. By shortening this process through AI, the engineer can spend more time reviewing the proposed solution and improving it, reaching the testing and go-live milestones sooner.

In many ways, using AI-enabled tools for code generation is no different than using other well-established methods. We rely on simulations to understand, visualize and scope projects, but that does not eliminate the need for engineers to know the process, challenge assumptions, and validate the results. Computer-aided design software does not eliminate the need to understand mechanical design.

The concept of “human in the loop” is familiar to both the controls engineering and AI fields. This principle is particularly relevant when it comes to AI code generation. While AI can accelerate the work, engineers must remain in control.

Better AI Prompts Lead to Better Results

The quality of AI-generated code depends heavily on the information provided to the model by the engineer. Vague or generic requests will likely produce irrelevant or incorrect code.

Let’s emphasize here that most AI models will always output “something”, no matter how open-ended the request is. You will see your screen populated with line after line of code and you’ll get excited that you created a program to control a pump in a matter of seconds. But if the model was simply told to “write a program to control a pump,” then the generated code is most likely useless in a real situation.

Figure 3. AI accelerates coding, but engineers must validate the results.

A better prompt should look very different from that simplistic input. It should define all important requirements, including the target programming language, all relevant details about the controller platform, the input and output addresses, and probably much more. Your prompt should also include a narrative of the operating sequence and expected behavior. Many AI models can interpret a diagram if one is available. Finally, include your considerations around permissives, alarms, and failure conditions. In fact, include the entire functional specifications of your project if they are available.

The more clearly the system or project is defined, the more useful the AI-generated result becomes. There is actually an entire field within the AI domain that is called Prompt Engineering, which is concerned with structuring instructions so that an AI model produces more accurate, useful, and predictable results.

From Generation to Validation

AI-generated code can look convincing even when it may contain significant problems, which is why it must be reviewed and validated, just like any other code. Even when the code compiles, we must be very careful because it may not do what is intended. There is also the hidden risk that an AI-generated program that works well under normal conditions may fail when an exception occurs.

Therefore, it is crucial that we test and validate our code using the same tools and processes already available for human-generated code. We can compile the code, run simulations, test inputs, and compare the results against the expected behavior. In PLC applications, we can simulate the logic and test any interlocks, communication failures, abnormal conditions, and varying operating sequences before deploying in production.

Common best practices that we already use in industrial automation, such as design reviews and factory/site acceptance tests, remain relevant. The same practices apply regardless of whether the program was generated by a human, by AI, or by a combination of both. At the end of the day, the ultimate responsibility belongs to the engineer.

In the next article, we’ll explore another common AI trust concern: how to prevent ‘hallucinations,’ or the hidden addition of unanticipated assumptions and inputs which mar the results and compound the sense of mistrust.