NVIDIA’s making some big moves in the world of robotics! Let me break it down for you in simple terms:
At SIGGRAPH 2024, NVIDIA announced they’re giving robot makers and AI developers many excellent new tools. These tools are designed to help create the next generation of humanoid robots – the ones that look and move like humans.
Jensen Huang, the big boss at NVIDIA, is pretty excited about this. He thinks the next big thing in AI will be all about robotics, especially humanoid robots. NVIDIA’s not just sitting back – they’re jumping in with both feet, offering their AI tech to anyone who wants to build these advanced robots.
So, what exactly are they offering? Here’s the scoop:
- NVIDIA NIM: This is a set of tools that help robots learn and simulate stuff. Think of it as a training gym for robots.
- NVIDIA OSMO: This service lets developers run multiple robot tasks at once. It’s like multitasking for robots.
- Particular Workflows: These step-by-step guides show developers how to train robots using AI and just a little human data. It’s like teaching a robot to learn from watching humans, but without needing tons of examples.
This is exciting because it means more people and companies can start working on advanced humanoid robots. NVIDIA’s saying, “Hey, if you want to build cool robots, we’ve got the tools to help you do it!”
This could lead to some really interesting developments in robotics. We might see more human-like robots in various industries sooner than we thought. It’s like NVIDIA’s giving everyone a head start in the robot-building race!
What are NVIDIA NIM and OSMO?
NVIDIA NIM is a set of microservices with pre-built containers for the company’s inference software. These containers and their instructions help developers cut deployment time from weeks to minutes.
NIM allows for the simulation of workflows for generative AI and the creation of synthetic motion data from devices like the Apple Vision Pro. This makes it easier to train humanoid robots and optimize their movements while also significantly reducing the time needed to install and update their embedded software.
On the other hand, OSMO is a cloud-based service that helps professionals create and manage complex workflows using various computing resources. This service helps companies and developers speed up training and simulation processes by providing a dashboard to visualize and manage tasks. These tasks include generating synthetic data, training models, reinforcement learning, and implementing software for testing and scaling.
How does NVIDIA humanoid robot training work?
Creating training models for humanoid robots requires vast amounts of data, and NVIDIA aims to simplify this process with new tools announced at SIGGRAPH 2024. They focus on capturing human data through teleoperation using devices like the Apple Vision Pro. This device records data that is then sampled by Isaac Sim and MimiGen NIM, two tools for synthetic motion simulation and data generation now available from NVIDIA.
This collected data is fed into Project GROOT, a generalist learning and training model designed to help robots recognize human facial expressions, tone of voice, and gestures. The Robocasa NIM framework takes this a step further by generating experiences from the data provided to Project GROOT and retraining the humanoid robot models.
NVIDIA’s approach aims to simplify the complex process of developing humanoid robots through a continuous cycle of streamlined data acquisition, synthetic motion data generation, and retraining of models.
“Developing humanoid robots is an incredibly complex task that requires a huge amount of real-world data that is tediously captured,” said Fourier CEO Alex Gu. “NVIDIA’s new generative AI development and simulation tools will help jumpstart and accelerate the development workflow.”
Availability
NVIDIA is offering three computing platforms to support the development of humanoid robots:
- AI Supercomputers for Model Training: These supercomputers are designed to train complex AI models needed for humanoid robots.
- NVIDIA Isaac Sim: This platform allows developers to teach and refine robot skills in a virtual environment, simulating real-world scenarios.
- NVIDIA Jetson Thor: This humanoid robot tests and runs AI models.
Businesses and developers can utilize all these platforms or opt for specific features through the NVIDIA Humanoid Robot Development Program. While all these features are currently in early access, NVIDIA NIM will be available soon.
