SANTA CLARA — Hugging Face and NVIDIA have announced a strategic collaboration at the Conference for Robot Learning (CoRL) in Munich, aimed at advancing robotics research and development across industries like healthcare, manufacturing, and logistics. By combining Hugging Face’s LeRobot open-source platform with NVIDIA’s AI technology, Omniverse, and Isaac robotics tools, the two companies are providing a powerful toolkit for researchers and developers to drive the future of robotics.

The collaboration marks a pivotal moment in the evolution of physical AI, a rapidly growing field where robots can understand and interact with their environments. To sustain this transformation, there is a clear need for open-source, extensible frameworks that span the entire development process—from robot training to simulation and inference. Hugging Face’s platform, which serves over 5 million machine learning researchers and developers, offers access to millions of models, datasets, and applications through its Hugging Face Hub, enabling AI developers to accelerate their projects without reinventing the wheel.

Hugging Face’s LeRobot framework builds on the success of its popular Transformers and Diffusers libraries by bringing them to the world of robotics. This open AI platform offers a comprehensive set of tools for data collection, model training, and simulation, along with designs for affordable manipulator kits. When integrated with NVIDIA’s cutting-edge AI and simulation tools, such as Isaac Lab, LeRobot’s data collection, training, and verification workflows are greatly accelerated, helping to foster a collaborative ecosystem where researchers and developers can share their models and datasets.

Creating physical AI systems is no easy task. Unlike language models that rely on large amounts of text data from the internet, robotics demands real-world physical interaction data, which is more challenging to collect. Isaac Lab, built on NVIDIA Isaac Sim, helps overcome this challenge by enabling robot training in high-fidelity simulated environments. This platform allows researchers to generate vast amounts of training data—equivalent to thousands of real-world experiences—using GPU-accelerated physics simulations and parallel environment execution.

The synthetic data generated in Isaac Lab can then be used to train robot policies through imitation learning. These policies, once validated in simulation, are deployed on real robots where they can be fine-tuned to deliver optimal performance. By sharing these models, policies, and datasets on the Hugging Face platform, a “data flywheel” is created that accelerates innovation in the robotics community.

Animesh Garg, an assistant professor at Georgia Tech, highlighted the importance of collaboration: “The robotics community thrives when we build together. By embracing open-source frameworks like Hugging Face’s LeRobot and NVIDIA Isaac Lab, we accelerate the pace of research and innovation in AI-powered robotics.”

This partnership also promotes community engagement through a shared workflow. Data collected via teleoperation and simulation in Isaac Lab is stored in the LeRobotDataset format, enabling seamless integration. This data is then used to train robot policies, which are tested and evaluated in simulation before being deployed to real-world robots with NVIDIA Jetson for real-time inference.

The collaboration has already yielded results, with a physical picking setup demonstrated using LeRobot software running on the powerful, compact NVIDIA Jetson Orin Nano platform. “Combining Hugging Face’s open-source community with NVIDIA’s hardware and Isaac Lab simulation has the potential to accelerate innovation in AI for robotics,” said Remi Cadene, principal research scientist at LeRobot.

This work builds on NVIDIA’s ongoing contributions to generative AI at the edge, furthering support for open models and libraries like Hugging Face’s Transformers. By optimizing inference for a range of AI models—including large and small language models, multimodal vision-language models, and vision-language action models—NVIDIA continues to strengthen its role in the AI-driven robotics revolution.

Through this collaboration, Hugging Face and NVIDIA aim to fuel the progress of a global community of robotics researchers and developers, accelerating innovation across key sectors such as transportation, logistics, and manufacturing.

Learn about NVIDIA’s robotics research papers at CoRL, including VLM integration for better environmental understanding, temporal navigation and long-horizon planning. Check out workshops at CoRL with NVIDIA researchers.