NVIDIA, known for its powerful GPUs, is now turning its attention to robotics. The company recently launched Cosmos, a platform that accelerates the development of physical AI systems such as autonomous vehicles (AVs) and robotics.
“NVIDIA will ultimately be a robotics company, not just a semiconductor. Few people understand how it works at the lowest level, from manufacturing to software,” said Dylan Patel, founder of Semianalysis. he said.
NVIDIA CEO Jensen Huang said in a recent interview that the world needs AI to understand the physical world. “You need to understand the mechanics of the physical world, such as gravity, inertia, and friction, and you need to understand spatial and geometric relationships,” Huang said.
According to him, the world needs more robots today because there is a shortage of workers. “The population is aging, the types of jobs people want are changing, and birth rates are falling. The world needs more workers, so the timing for robotic systems is really critical.” is important.”
NVIDIA is primarily focused on integrating robotics into manufacturing processes, self-driving cars, and healthcare. For example, humanoid robots in manufacturing can perform repetitive tasks, handle materials, and collaborate with human workers. Huang predicts that the $50 trillion manufacturing industry will become software-defined.
The future Huang envisions is already taking shape. For example, BMW uses the humanoid robot shown in Figure 02 on its production line. The company claims that Figure 02 is now able to operate as an “autonomous fleet,” delivering 400% faster speeds and a 7x higher success rate.
NVIDIA Robotics Fundamentals
Huang called Cosmos “the ChatGPT or Llama of the world’s foundational models.” The platform focuses on dynamic interactions such as human gait and hand movements and is trained on 20 million hours of video.
He further stated that the real magic happens when Cosmos is integrated with the Omniverse. The combination of the two provides “ground truth” to AI and helps it understand the physical world. Huang compared the connection between Cosmos and Omniverse to the concept of an LLM connected to a search augmentation generation (RAG) system.
In addition, Huang introduced three basic computer concepts that are essential to building robotic systems. The first is Deep GPU Xceleration (DGX), which is used to train AI. Once the AI is trained, the next computer, AGX, will be used to deploy AI into real-world applications such as cars and robots.
The third component is the digital twin. This is a simulated environment where AI practices, hone its capabilities, and undergo further training before deployment.
Backed by powerful research
This isn’t the first time NVIDIA has discussed humanoid and self-driving cars. Over the past year, the company has been actively researching this area.
“It’s such a relief to know that we are the last generation where advanced robots don’t exist anywhere. Our children will grow up as ‘robot natives.’ “You’ll cook a Michelin dinner for a humanoid, tell a bedtime story to a robot teddy bear, and the FSD will drive you to school,” said NVIDIA’s Senior Research Manager and Embodied AI (GEAR Lab). said Jim Huang, Director of .
In a separate interview, Huang said the company chose humanoids because the world is built around human form factors. “Our restaurants, factories, hospitals, and all our equipment and tools are designed to fit the human body.”
Notably, the company recently announced Project Eureka and demonstrated a demonstration of training a five-fingered robotic hand to spin a pen.
Additionally, NVIDIA recently developed HOVER (Humanoid Versatile Controller), a 1.5 million-parameter neural network designed to coordinate the motors of humanoid robots for locomotion and manipulation.
“Not every underlying model needs to be huge. We trained a neural network with 1.5 million parameters to control the body of a humanoid robot,” Huang revealed.
NVIDIA launched Project GR00T and the Isaac platform last year. GR00T is a framework that allows developers to generate large synthetic datasets from a limited number of human actions.
The company is also developing a new generation of small computers for humanoid robots, Jetson Thor, which is expected to be released in the first half of 2025.
NVIDIA is collaborating with companies such as 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics to develop humanoid robots.
World model x humanoid
It appears NVIDIA is not alone in the robot development race. According to OpenAI’s career page, the startup is hiring for positions in mechanical engineering, robotic systems integration, and program management. The goal is to “integrate cutting-edge hardware and software to explore a wide range of robotic form factors.”
Last year, the company hired Caitlin Kalinowski to lead its robotics and consumer hardware divisions. Previously, he oversaw the development of Orion augmented reality (AR) glasses at Meta. OpenAI has also invested in Figure AI and robotic AI startup Physical Intelligence.
Similarly, Apptronik, one of the leaders in AI-powered humanoid robotics, recently announced an exciting new partnership with Google DeepMind’s robotics team to create truly intelligent and autonomous robots. did.
Tesla is also ramping up its strategy. Last year, CEO Elon Musk unveiled the company’s humanoid robot, Optimus, at the “We, Robot” event held at Warner Bros. Studios in California. They can walk the dog, mow the lawn, do housework, babysit, and even speak like Gen Z with smooth body language.
Meanwhile, fan mentor and “AI Godmother” Feifei Li recently launched her own company to build large-scale world models (LWM) that can perceive, generate, and interact with 3D worlds. Founded World Labs.
