Combining Machine Vision and Neural Networks in IIoT

The cerebral cortex is the part of the brain that processes images. Human beings have the largest cerebral cortex compared to other mammals. This superior vision is one of the evolutionary traits that gave humans an edge over other animals. Evolutionary biologists try to unfold the mystery behind this trait, and technology researchers try to replicate it.

 

Figure 1. A machine vision system in a factory.

 

Humans learn through experience and practice. Machine learning is the field of artificial intelligence (AI) that tries to make computers mimic how human brains learn. Image recognition and processing is an important subsection of the field. Using image recognition, engineers can classify objects without direct human interaction.

 

Neural Networks and Machine Vision

In simple terms, machine vision in conjunction with algorithms or neural networks is the ability for computers to see and act on data. Images are fed to the computer’s processing unit in digital format to analyze, interpret, and act based on it. Prior to this setup, we must first train the system to learn the data.

 

Training the Neural Networks in Machine Vision

Neural networks are an advanced area of machine learning. They are widely used for image recognition and other tasks that require complex cognition. The first step to build an artificial neural network is to train the computer algorithms with images. 

 

Figure 2. An example of training algorithms with images. Image used courtesy of Teledyne DALSA

 

Humans are tasked with labeling the input images based on which the algorithms learn. The algorithm will learn by itself to identify the objects after being trained on a large labeled dataset. Data scientists can tilt the models to improve the learning speed and other parameters. A complete model with consistent accuracy can be deployed for commercial applications.

But where do we get the data to train the model?

 

Data Sources for Machine Vision

Machine vision coupled with neural networks is more complex than simply identifying images. It is also spatial cognition to recognize 3D (three-dimensional) objects. The source for machine vision programs can be photographs, video, or even live camera feed. In such instances, a camera collects the images or video fed to the algorithm.

There are machine vision algorithms that take other forms of input. Radar and LiDAR can track the objects around the sensor. The output of such equipment is not images, but coordinates of the various objects around it. Machine vision algorithms can also process this information to gain spatial understanding. A combination of LiDAR data and images from a camera can also be used as input to a machine vision algorithm.

Machine vision has a wide range of industrial and non-industrial applications. The applications range from identification and navigation to giving vision to the blind.

 

Machine Vision in Industry

Machine vision or computer vision has plenty of applications in the industry. Since it is still in a nascent stage, more industrial applications are identified each passing day. 

 

Figure 3. A machine vision system inspecting objects in a factory.

 

Some of the known applications include:

• Sorting products
• Picking products
• Automated inspection
• Optical character recognition
• Identifying defects
• Identifying colors
• Motion detection
• Identifying patterns
• Quality assurance
• Guided movement (automated guided vehicles, or AGVs)
• Self-driving automobiles
• Reading bar codes and QR codes
• Dark factories

 

As machine vision usage increases, the cost of implementing such solutions comes down, and new applications are identified. For instance, machine vision can be used in conjunction with IIoT.

 

Security

Human security is used to patrol the grounds of various institutions. However, a common problem is the fatigue experienced by such personnel while performing duties at odd hours. Machine vision with IIoT can help.

Closed-circuit television (CCTV) feed of the secured premises can be hooked with machine vision algorithms. The algorithm will constantly monitor the feed for anomalous activities. Once intruders or other anomalous activities are detected, the system can trigger an alarm for human guards to check. 

It can also automatically execute security protocols like electrifying the fence, deploying spikes at exit points, alerting police, etc. All the communication will happen over the network between internet-enabled devices to execute any protocol.

 

Chemical Plant

Chemical plants are dangerous for human employees to work in. It would be beneficial if the processes can be automated. Take, for example, a process in a chemical plant that requires the material to come to a boil before the heat is turned off. Without machine vision, a human technician has to wait and observe the process. 

 

Figure 4. A chemical plant vent stack and boiler.

 

With machine vision, a camera can focus on the solution, and a neural network trained to detect boils can detect if the mixture has started boiling. Once the algorithm detects boil in the solution, it can trigger a flag. This is sent to the central computing system over the network. The system is designed so that the receipt of this signal will trigger the mechanism to turn off the heat.

In such a scenario, a human does not have to be in the plant inhaling the chemical fumes to monitor the process. Machine vision and IIoT can automate the complete process. In this particular example, infrared or other temperature detecting sensors can also be used to detect a boiling mixture.

 

Dark Factories

Dark factories, as a concept, are the extension of utilizing machine vision, automation, and IIoT capabilities to the fullest extent. A dark factory is a manufacturing facility where no human operator is required to run or monitor factory operations. In essence, the factory need not have lights, and hence, can operate in the dark.

From raw material delivery to packing, the final product will be completely automated. Raw materials will be picked and sorted with forklifts and robotic arms aided with machine vision. Robotic hands assembling various parts will use machine vision to identify where each part goes. AGVs, which use machine vision to navigate, handle the transportation of materials within the factory.

Finished products are inspected and tested with robots coupled with machine vision capability. Sorting defective products and packing finished products also uses machine vision.

Applications such as dark factories and chemical processes, to name a few, can benefit from machine vision. Its algorithms can keep personnel safe, processes safe, and enhance efficiency. What types of applications do you use machine vision for on your shop floor?