MIRAI: Using AI to Detect Variances and Improve Robot Operations

Imagine that you have a production task to be accomplished, and it has many of the marks of an automatable process. There is a nearby infeed and outfeed section, the products to be picked are large enough to be handled by a standard gripper, and there are only 2 sizes of products with similar shapes, so how hard can it be?

Then, you notice a problem. The products to be picked aren’t neatly arranged as they come down the infeed belt. Moreover, there are some blue products and some red products. Each color must be placed in a specific location. To add to this, the outfeed section contains the main blocks, where each colored product is to be inserted, and they are not always aligned the right way.

Suddenly, this seemingly simple pick-and-place task is met with some common robotic obstacles: variations in position and color.

 

MIRAI vision and AI system linked to a Universal Robot with an OnRobot adapter and Robotiq gripper. Images used courtesy of Micropsi

 

Micropsi Industry’s MIRAI System

Recently, Micropsi has announced the expansion of their AI-powered training tool for robotics, called MIRAI, designed to replicate the ‘teaching’ process used to train human workers, allowing the robot to effectively learn the proper way to perform the tasks, and what details and characteristics are important to the process, even when it’s not always presented the same way every time.

I got a chance to chat with the CTO of Micropsi, Dominik Boesl, to get a better understanding of the purpose and process of how a camera with AI software can be used to overcome a few of the biggest hurdles in automation.

 

How Do You ‘Train’ an AI System?

Like many engineers, I’ve heard a LOT about artificial intelligence recently, but it’s helpful to hear an explanation of how a device is actually programmed. They are touted to be very simple, since a computerized algorithm actually does the computations, but what information does it need, and how does it get this data?

Typical vision systems take an image, or a series of images, locating key points, edges, and features to identify the presence of a part. With more and more images, some of the variances in light, orientation, placement, etc can be compensated. But comparing against still images can only go so far. You can teach what to look for and what to avoid, but what if an untrained situation occurs? Should it guess?

The MIRAI solution is to record short videos, from one dozen to perhaps hundreds, depending on the complexity of the task. These videos range less than 2 minutes in length typically, and they show the operator simply guiding the robot through the required task, including all the required steps. New camera angles are introduced, along with all the variations in light, colors, orientations, and everything. The videos are submitted to a secure server to be processed, and are not combined with larger mass databases, to ensure data security.

 

Example application of using MIRAI to pick and assemble RJ45 connections. Images used courtesy of Micropsi

 

When the processing is finished, the controller will be able to understand the subtle details of the process, the goals, and the constraints, becoming adaptable to nearly any variance.

According to CTO Boesl, “we're not building any CAD or 3D models, we're not making it understand exactly how to perform the process, no, we just show it essentially, this is how it looks, and this is how we want it to look. The more confident it is in knowing how it should look, the better it will perform.”

 

What Hardware and Software Does MIRAI Use?

The hardware includes lighting, a camera, and a controller that communicates with the robot. The program supports UR, FANUC, and in this most recent announcement, KUKA, with expectations to expand to other brands in a controller-agnostic fashion under a partnership with software developer voraus robotik.

 

The most recent introduction to the robot software lineup for MIRAI is the inclusion of the KUKA collaborative robotics platform compatibility. Image used courtesy of voraus robotik.

 

One of the benefits of this learning process is that there are no special add-ons for new tasks. Boels explains, “This is where the flexibility comes from. Today, it could be cable plugging. Tomorrow, it could do sealing, it could do polishing, grinding, or something like that, you could simply retrain it.” This provides consistency and confidence in the potential to launch new projects with the same robot and realize a faster ROI.

 

How is the MIRAI System Implemented?

The installation process works in many ways. If the company has a mature engineering department with many robots already in good standing, a test pilot project kit may prove the application, then scale that deployment to all machines.

On the opposite end of the spectrum, a company with very little robotics experience may require the aid of an integrator or the robot manufacturer, so Micropsi comes alongside to offer the AI solution at any part of the process as needed.

 

Actual case study image of a small end effector being used to refurbish gas turbine vanes. Images used courtesy of Micropsi

 

The relationship with customers can come from any level of the integration process from OEM to end user. “We have end users contacting us to say, ‘I have tried desperately to solve this [cable plugging] application, and I can't. I came across your solution, please ship me your box.’ In most cases, it works with the robot they have.” Boesl said in answer to my question about the ideal target audiences for the MIRAI system. “Another user category is integrators. We have a partner network of integrators that now are more and more taking us as a stock component to assist end users who are just getting started in automation.”

 

AI and Vision, Working Together for Ease of Programming

Truly, there are many tasks that computers will never be able to accomplish. But for many of the others, the ones that are more simple and repetitive, vision partnered with AI is solving some of the challenges associated with variances, challenges that might have been unable to be overcome only a few years ago.