NVIDIA’s breakthrough AI teaches robots human

The Voyager AI agent was built using the large language model (LLM) GPT-4, and the team at NVIDIA has now used the same model to create a new agent that can perform complex skills at par with humans.

How does Eureka work?

Reinforcement learning (RL) has been used extensively in AI for a few years. “Reinforcement learning has enabled impressive wins over the last decade, yet many challenges still exist, such as reward design, which remains a trial-and-error process,” said Anima Anandkumar, senior director of AI research at NVIDIA, in the press release.

Eureka goes a step further and works with generative AI to write software code that rewards robots through reinforcement learning. This is done using GPT-4 LLM and does not need task-specific prompting or a predefined template. Moreover, the agent can incorporate human feedback to modify rewards to improve the results.

Eureka's library of algorithms works can be used on Isaac Gym, NVIDIA's physics simulations reference application built on Omniverse and used for reinforcement learning research. Powered by NVIDIA's GPUs, the Isaac Gym can rapidly evaluate even large batches of reward candidates, further improving training efficiency.

The AI agent then prepares a summary fed into the LLM to improve reward functions using the critical stats from training results. The approach was used for a wide range of robot types, such as quadruped, bipedal, and quadrotor, with dexterous hands or cobot arms with equal ease.

The results generated from Isaac Gym environments can be visualized in NVIDIA Omniverse, showcased below.

How well does Eureka work?

Apart from rapid pen spinning, NVIDIA researchers successfully trained the robots to complete other complex tasks, such as opening cabinets and drawers, tossing and catching balls, and manipulating scissors.

According to a research paper by the NVIDIA team, Eureka-generated rewards outperformed human-written ones on more than 80 percent of the tasks performed. They led to more than 50 percent performance improvements in the robots.

The paper also lists performance improvements in in-depth evaluations of 20 tasks on which robots were trained using Eureka that involved complex manipulation skills.

NVIDIA is confident its recent progress will encourage developers to take up more ambitious and challenging projects shortly. "We believe that Eureka will enable dexterous robot control and provide a new way to produce physically realistic animations for artists," added Jim Fan, a senior research scientist at NVIDIA, who was involved in the research.