—
Exploring the Future of Robotics: NVIDIA’s Groundbreaking Advances
In the rapidly evolving world of robotics, creating machines that can seamlessly work alongside humans in environments such as factories, hospitals, and public areas poses significant technical challenges. These robots need to exhibit human-like abilities in terms of dexterity, perception, cognition, and full-body coordination to efficiently navigate unpredictable real-world environments.
A novel "sim-first" approach is proving to be a game-changer in this arena. This method allows for the parallel training of numerous robot instances using a combination of data captured from real robots and synthetic data within simulation environments. Central to this development is the Universal Scene Description (OpenUSD), a robust framework that offers a scalable and interoperable data standard. This enables developers to construct highly accurate virtual worlds where robots can hone their skills before applying them in real-world scenarios.
Accelerating the Development of Physical AI
This week, at the Conference on Robot Learning, NVIDIA unveiled significant advancements in open-source physics simulation, foundational models, and development frameworks. These include:
- Newton Physics Engine: Robots, particularly humanoid ones with intricate joints and movement dynamics, learn more efficiently and securely in simulated environments. Developed in collaboration with Google DeepMind, Disney Research, and NVIDIA, and overseen by the Linux Foundation, Newton is an open-source, GPU-accelerated physics engine designed to advance robotic learning. By leveraging NVIDIA Warp and OpenUSD, Newton allows robots to master complex tasks with greater precision. It integrates smoothly with robot learning frameworks such as MuJoCo Playground and NVIDIA Isaac Lab.
- Isaac GR00T N1.6: To perform tasks akin to humans, robots must comprehend ambiguous instructions and navigate unforeseen scenarios. The latest Isaac GR00T N1.6 model, soon available on Hugging Face, incorporates NVIDIA Cosmos Reason. This reasoning vision language model, built for physical AI, acts as the robot’s cognitive core, transforming vague instructions into detailed action plans grounded in prior knowledge, common sense, and an understanding of physics.
- NVIDIA Isaac Lab: The newest version of Isaac Lab, an open-source, modular robot learning framework built on NVIDIA Isaac Sim and OpenUSD, is now available as an early developer release. Version 2.3 introduces numerous features for robotics researchers and developers, including advanced whole-body control and enhanced teleoperation for data gathering.
OpenUSD’s interoperability ensures that these advanced physics simulations, foundational models, and learning frameworks work harmoniously. This enables developers to build unified robot learning pipelines that can scale across different platforms and deployment scenarios.
How Developers are Accelerating Robotic Learning
Leading companies in humanoid and robotics development, such as Agility Robotics, Lightwheel, Mentee, and Universal Robots, are adopting simulation technologies and libraries to expedite the development and deployment of physical AI.
- Agility Robotics is utilizing NVIDIA Isaac Lab to train a comprehensive control model for its Digit robot. With Isaac Sim and OpenUSD, they can create precise digital twins of customer facilities, providing a scalable method to optimize the robot’s operations before actual deployment.
- Lightwheel has developed the Lightwheel Simulation Platform, based on NVIDIA Omniverse. They are also creating simulation-ready assets using the NVIDIA USD Search application programming interface, which streamlines asset discovery and aids in assembling accurate digital twins to accelerate training and simulation processes for robotics developers.
- Mentee Robotics leverages NVIDIA’s three-computer architecture to enhance the learning capabilities of MenteeBot. By using OpenUSD as the foundation, they are developing pipelines for generating synthetic data in Isaac Sim.
- Universal Robots employs the NVIDIA Isaac platform for comprehensive robot simulation and learning. They use OpenUSD to create interoperable digital twins of manufacturing environments, ensuring cobot safety protocols are validated and optimizing human-robot interaction in various industrial settings. In collaboration, Inbolt provides dynamic vision guidance systems that allow robots to adapt to their environment dynamically and handle production variations effortlessly.
- Wandelbots, a German robotics software firm, is aiding Volkswagen in reducing the duration of automation projects at its Transparent Factory in Dresden. By using Wandelbots NOVA, an Isaac Sim-integrated, no-code teaching platform, assembly workers can train robots in virtual settings before deploying them.
NVIDIA’s open frameworks and libraries are also being embraced by the broader robotics community. For instance, community member and NVIDIA Omniverse ambassador Dylan Tobin has developed an AI chatbot trained on Isaac Sim workflows to assist developers in navigating Omniverse more efficiently.
To see how other developers in the community are using Isaac Sim and Isaac Lab for innovation in robotics navigation, control, and reinforcement learning, you can watch the replay of a recent livestream.
Additionally, an NVIDIA Robotics office hours session demonstrates how Brev simplifies running Isaac Sim and Isaac Lab on Omniverse.
Getting Connected with OpenUSD
For those interested in learning more about robot learning with OpenUSD and NVIDIA’s latest robotics technologies, a wealth of resources is available:
Stay informed by subscribing to NVIDIA Omniverse news, joining the Omniverse community, and following them on various social platforms like Discord, Instagram, LinkedIn, Threads, X, and YouTube.
For further exploration, visit the Alliance for OpenUSD forum and the AOUSD website. These platforms provide valuable insights and updates on the latest advancements and applications of OpenUSD in robotics and beyond.
In conclusion, as NVIDIA continues to push the boundaries of what’s possible in the field of robotics through their pioneering work in simulation and AI, developers and enterprises alike are presented with exciting opportunities to revolutionize their workflows and applications in the real world.
For more details on this topic, you can visit the original article on NVIDIA’s website.
For more Information, Refer to this article.
