Technical Article

Robot-aided Visual Inspection in Manufacturing Industries

June 23, 2020 by Akshay Kumar

Learn all about how robotic visual inspection is helping factories with manufacturing and assembly processes by keeping the system accurate and producing quality products.

Industries suffer huge losses due to poor standards of manufacturing and/or assembly processes. Traditionally, manual inspection of manufactured goods by human experts has been the way to go for quality assurance. While this works for coarse objects, measuring the accuracy and precision of critical goods with consistent, quantifiable results is challenging for the human eye.

Manual visual inspection is gradually being replaced by cameras programmed to look for features of the product and robots that move the camera around to inspect multiple points on the product. This article explains how robotic visual inspection is helping industries and factories ensure their manufacturing and assembly processes are accurate and their product quality is consistent.


What is Visual Inspection?

Visual inspection is the inspection of equipment and products for flaws, and feature consistency and accuracy against the ideal specification. Humans trained to compare objects can suffice for simple tasks like confirming the shape of every can made is perfectly round. However, it is difficult for humans to check and quantify if the quality of a  food ingredient is as desired, if all components of a PCB have been mounted and soldered correctly, or if a subassembly is correct.

Visual inspection is non-invasive and allows image optimization for better flaw evaluation, automated documentation, and process optimization based on the documentation. Visual inspection is faster than humans in most applications and can detect defects almost immediately while addressing a huge variety of features. 


Figure 1. Multiple cameras for visual feature inspection


Camera-based visual inspection has been very readily adopted in factories for all stages of production, from raw product analysis to finished goods monitoring. While visual inspection using fixed cameras is efficient, it needs multiple sophisticated cameras mounted on appropriate frames, configured individually to detect feature points on a product, and synchronized with the assembly line. Industrial cameras are expensive and using one camera for one particular task/feature inspection is a severe under-utilization of the versatility they provide, cost-prohibitive, rigid, and cumbersome to manage.

Most industrial cameras are easy to use via a GUI that lets even non-expert users inspect certain areas of the product and configure the features that the camera should look for in the image. Once features are detected, the user can configure a threshold for classification success and the communication and documentation strategy. 


Common Visual Features

Visual features inspected in a test piece depend on the shape, texture, and nature of the product. Common features observed in products are edges, lines, regular shapes, color blobs, and patterns (like checkerboards). These features can be observed in different structures of the test piece. Regular concentric circles are observed when a bolt is torqued to a hole with a designated groove. 

For certain subassemblies, objects in the image are detected, classified, and compared against the ideal subassembly. Conventional computer vision applications along with deep learning and AI are used in the backend for detection.


Robotic Visual Inspection

Robotic visual inspection refers to a camera and lighting gear mounted on the end-effector of a robot and the robot moves to inspect multiple points on the same object/test piece for features. The robot can be programmed to automatically detect a sequence of locations on the object. 


Figure 2. Robotic visual inspection 


The addition of a new location for inspection and its features characteristics into the programmed sequence is an easy step. With the option to control the incidence angle of the camera using the robot’s end-effector orientation, the best images can be obtained for inspection.


Pipeline for Robotic Visual Inspection

The pipeline involves camera configuration, robot motion configuration, communication with PLC for data logging, and final result generation. This helps today's factories ensure their products are not sub-standard and the fault is detected at the origin and not multiple steps down the complete production line.


Figure 3. Robotic visual inspection pipeline


High-resolution images from the camera are processed in real-time for feature matching and success is determined by whether the matching percentage is greater than the user-configured threshold value. Usually, the robot is connected to a PLC that synchronizes the motion of the robot against the test piece and its various test locations. It also communicates with the conveyor and how the test piece is presented to the robot.


Common Industrial Tools Used

Common industrial camera companies like Keyence, Cognex, Flir, and Balluff support the configurable feature detection ideal for visual inspection applications. These products usually have an onboard computing platform with a web-based UI accessible over the network to configure the feature to be detected, its threshold, the type of data to be conveyed to the PLC, and further inputs.

Robotic visual inspection ideally needs small, agile robots with low payload capacity — they only enough to carry a camera while satisfying the kinematic requirements. Also, most production lines with such inspection mechanisms have humans working nearby, thus, collaborative robot arms are the most suitable for these applications. Universal Robots, Fanuc, Doosan Robotics, and Franka Emika are well-known companies that manufacture affordable collaborative robots ideal for visual inspection applications.

Allen-Bradley, Schneider Electric, Siemens, and Mitsubishi Electric are well-known  PLC companies that are used in industries, readily integrable with robots and conveyor systems.


Figure 4. Vision-based solution for handling variance. Image courtesy of UR Webpage.


Challenges in Robotic Visual Inspection

The number of points to inspect, the cycle time, resolution of the images, and budget constraints are all factors that affect the implementation of a robotic visual inspection pipeline. While robotic visual inspection has impressive advantages, it does have some limitations and disadvantages because of the robot motion as well as the camera including:

  • Low cycle times are difficult on high-speed production lines due to computational delay in image processing
  • Obtaining steady-state pose for the robot for stable and non-blurred images causes a delay in cycle time
  • Real-time computations need expensive computing hardware and cameras
  • They are not immune to ambient lighting troubles and repeatability and accuracy efficiencies in robot pose 
  • The process needs efficient robot hardware which is expensive
  • They do not qualify as acceptable solutions for space-restricted installations


Review of Visual Inspection

While robotic visual inspection is technologically appealing, commercial considerations limit the adoption rate. The ROI is usually longer and robotic visual inspection systems need more maintenance and configuration due to more moving parts in the system. Also, these mechanisms need more engineering personnel for robot control, computer vision, and communication pipeline management.

Visual inspection is a proven tool for industries, but robot-aided visual inspection competes against fixed frame visual inspection due to budget, logistics, and personnel requirements. However, as collaborative robots get cheaper and more reliable, they will soon become the go-to platform for visual inspection.