AI Meets Coffee: How Robots Are Transforming Hospitality

Researchers have set out to test the hypothesis that combining sensorimotor abilities and AI-based large language modeling would enable a robot arm to deconstruct complex tasks and execute them, using objects within an unknown environment to make a blissful cup of coffee. The researchers tested their embodied large-language-model-enabled robot (ELLMER) framework in a scenario where a robot was tasked with coffee-making and plate decorating.

The ELLMER framework allows the coffee-making robot to respond to customer queries in real time, find previously unknown objects, mix, and prepare plates as part of a dynamic and efficient customer-focused service. Image used courtesy of Nature Machine Intelligence

Conventional, Rigid Systems

The challenge facing the development of new, flexible, and responsive robotics technologies is the reliance on pre-programmed systems that operate using specific inputs to action tasks in pre-conditioned environments. There is a need to develop more adaptive and responsive robotics customer service technologies with advanced AI and integrated machine vision-led physical and auditory responsiveness. Real-time responsiveness is an important facet of customer service, which means that any given bot will need to process voice and/or image data quickly and respond most efficiently and appropriately.

Pre-trained models can fall short if a customer asks for a difficult request or one that is a bit outside of the box and not included in the pre-configured library of code a service bot has at its disposal. This is where comprehensive AI model training comes into play, with continuous training on customer service scenarios.

How Does it Work?

ChatGPT (Generative Pre-Trained Transformers) is a type of large language model (LLM), an AI model that analyzes an extensive load of text data, using it to deconstruct and build human-like text. Here, the researchers incorporate GPT-4 within its framework to ingest user queries (input data) and combine this with image data captured by an Azure Kinect depth camera.

The transformer retrieves query-relevant code from a knowledge base and constructs Python code to action the tasks required. The code is sent to the controller, which sends signals to the robot to command action (through force and vision feedback).

In this way, the team’s bot was able to successfully take in and process verbal instruction, assess its surroundings, access cupboard doors, and select mugs that it had not seen before. The robot also demonstrated its ability to mix coffee with hot water at a fixed ratio.

The researchers say that the robot can also adapt to disturbances to its work, such as if someone moves the mug it is using.

The opening of the automated robotic SELF restaurant at Josep Tarradellas Barcelona-El Prat Airport marks a new era in the food and customer service industry. Image used courtesy of KUKA

Kick-Starting the Catering Revolution

KUKA has been working alongside travel restaurant company Areas and other partners to develop the SELF project, a unique catering offering bringing together a KUKA robotic arm and machine vision to manage and deliver restaurant orders.

Up to six orders can be processed concurrently by its AI-driven system, which optimizes inventories and helps improve speed and precision while maintaining quality control. High standards for food preparation are upheld by machine vision, and sustainability is enhanced by energy-saving devices and recyclable materials.

SELF, which is situated in Josep Tarradellas Barcelona-El Prat Airport, demonstrates how robotics and AI are revolutionizing the food sector by improving productivity, precision, and customization. While AI maximizes food production, the method frees up human employees to concentrate on customer service. This is a step toward completely automated eating experiences, where waste is decreased, and overall customer service is enhanced through the intelligent use of resources.

ADAM engages in conversation with its customers while serving cocktails, coffee, and tasty bubble tea. Image used courtesy of Richtech Robotics

ADAM

In a previous article from Control Automation, we saw another nifty coffee-brewing barista getting to the daily grind for coffee lovers and cocktail enthusiasts. Yes, you heard right, ADAM, the sensational robot mixologist, makes cocktails, too. Like other technologies exhibited above, ADAM can work around the clock, unlike human workers, which saves companies money and increases revenue.

ADAM (from Richtech Robotics), ELLMER, and SELF are just some of the game-changing robotics catering technologies placing AI, machine vision, and optimal resource management at the forefront of evolving automated customer service delivery and customer satisfaction.