NVIDIA and Intrinsic Collaborate To Advance Robotic Gripping

NVIDIA has announced the integration of its artificial intelligence (AI) and Isaac Sim technologies with Intrinsic's developer environment. The collaborating companies have developed a prototype focused on advancing robotic grasping capabilities. Intrinsic is an independent Alphabet company specializing in innovative software and AI-driven robotics technologies. As highlighted in a previous article, NVIDIA is dedicated to the integration and implementation of generative AI to help make the industrial metaverse a reality.

NVIDIA and Intrinsic merge platform expertise to advance robotic grasping for industrial environments. Image used courtesy of NVIDIA 

 

NVIDIA Isaac Manipulator

NVIDIA’s Isaac Manipulator is a repository of modular general processing unit (GPU)-accelerated libraries (for computing-intensive applications) and foundation models that allow robotics software developers to construct and deploy reproducible and scalable workflows for orchestrating robotic tasks requiring dexterous manipulation.

Isaac Manipulator uses foundation models (FoundationPose, Synthetica DETR, NVIDIA cuMotion, and FoundationGrasp)—a kind of AI neural network nested within the field of machine learning and combined with a transformer deep learning architectures (comprised of decoder/encoder blocks for data processing)—which can make sense of extensive unlabeled data by tracking data features and recognizing relationships within sequential data.

Using unlabeled data is less costly and time-consuming than using labeled data. Foundation models are described as forms of generative AI, creating generated output based on input (prompts of human language instructions).

Foundation models centralize data from a variety of sources and can be used for many applications. Image used courtesy of the Center for Research on Foundation Models (CRFM)

 

According to NVIDIA, when developing robotic industrial applications through Isaac Manipulator, developers can save valuable programming time and speed up path planning by as much as 80 times. Zero-shot learning (without the need for prior examples) also increases the speed with which robots can be deployed, increasing efficiency and productivity and enabling developers to automate a greater number of new robotic tasks. Early ecosystem partners include Universal Robots, a Teradyne company, READY Robotics, PickNik Robotics, Yaskawa, and others.

 

Robotic Gripping Capabilities

The downsides of executing robotic gripping tasks include the expense and time-sapping labor of programming autonomous manipulation to complete a variety of dynamic tasks. This leaves a critical gap in adaptable software innovations that might be used to simplify and speed up robotic manipulation-based application development.

As part of its collaboration with NVIDIA, Intrinsic leveraged the capabilities of NVIDIA Isaac Sim to generate cost-saving synthetic data to train an AI neural network for a robot to execute vacuum grasping using computer-aided design models of sheet metal and suction grippers. The resulting prototype used Intrinsic’s virtual developer environment, Intrinsic Flowstate, that enables path planning, pose estimation, and testing of robotic cell workflows through simulation before calibrating real hardware.

 

A virtual and real-world view of the execution of a grasping skill developed from an NVIDIA foundation model and implemented in Intrinsic Flowstate. Video used courtesy of NVIDIA 

 

Isaac Manipulator and Isaac Sim can be combined to generate application-appropriate poses for robotic grippers, Compute Unified Device Architecture (CUDA)-accelerated robot motions, and evaluate virtual prototypes before real-world implementation through the Intrinsic platform. 

 

Advancing Robotic Dexterity 

Together, NVIDIA and Intrinsic have set out to accelerate the development and deployment of robotic automation tasks across global industrial environments by employing tunable generative AI-based models through their latest prototype, which employs the companies' respective technologies.