New tech helps humanoids run, jump and handle tasks with one control

Republished by Newlife - MIM. All rights belong to the original publisher; see Source below.

The motion generator could use inputs from human teleoperation, text, video, and music and directly convert them into whole-body humanoid control signals. Researchers have designed a large-scale framework to control humanoid robots using inputs ranging from teleoperation and video to text and music. The system, named SONIC, enables robots to perform natural whole-body movements, including running, jumping, grasping, and transitioning smoothly between different actions. Unlike many existing controllers built for specific tasks, SONIC is designed to support a broad range of movements through a single policy. The researchers at NVIDIA demonstrated the framework in both simulations and a physical humanoid robot, showing its ability to perform varied motions and everyday tasks.

Scalable humanoid control SONIC is a large-scale humanoid control framework that uses motion tracking as the foundation for generating natural whole-body robot movements. Rather than training separate controllers for individual skills, SONIC uses a single policy trained on more than 100 million motion frames, allowing a humanoid robot to reproduce diverse movements while improving its ability to generalize to motions not seen during training. The system addresses a major limitation in humanoid robotics: existing control policies are typically relatively small neural networks designed around specific tasks, with new behaviors often requiring separately engineered objectives and training. SONIC instead treats motion tracking as a scalable learning problem, using dense frame-by-frame supervision from human motion data to provide a consistent training signal across a large and diverse collection of behaviors.

The researchers scaled SONIC across three dimensions: training data, model capacity, and computing power. The training dataset contains more than 100 million frames derived from about 700 hours of motion-capture data, while the largest model contains 42 million parameters. Training used as many as 128 GPUs and approximately 21,000 GPU hours, with experiments showing progressively better tracking and generalization as data, model size, and compute increased. A central component of SONIC is its universal token space, which allows different types of motion inputs to be translated into a shared representation. Specialized encoders process robot motion, human motion, and hybrid commands before converting them into a quantized token that feeds a common robot control decoder. This architecture allows the same control policy to connect motion tracking with VR teleoperation, video-based control, text commands, music-driven motion generation, and vision-language-action models. “We expect SONIC to serve as a practical foundation upon which higher-level perception and reasoning can be built to advance general-purpose humanoid autonomy,” said the team in a statement.

Whole-body robot control The framework also includes a real-time kinematic motion planner that generates short motion segments between keyframes. The planner can continuously replan based on incoming commands, enabling interactive changes in direction, speed, and movement style without retraining the underlying policy. This gives SONIC a control layer capable of turning high-level movement intent into continuous whole-body motion. For multimodal control, SONIC can receive human motion generated from video, natural-language instructions, or music and feed those motions into its tracking policy. Video input can be processed from prerecorded clips or live webcam streams, while text prompts can specify actions such as walking or kicking. Music-based control generates dance movements from rhythmic and melodic information, with the system able to transition between input modalities. The researchers deployed SONIC on a Unitree G1 humanoid and demonstrated its ability to track diverse motions in the real world. The system achieved a 99.2 percent success rate across 123 real-world motion sequences, compared with 100 percent in simulation, showing a relatively small simulation-to-reality performance gap. SONIC was also connected to a vision-language-action model through its universal token interface. In five whole-body tasks, the system achieved an average success rate of 75 percent, including object pickup, opening a trash can with a foot pedal, and coordinating hand and foot movements to move a soda can into the bin. The researchers argue that this shared action representation could make it easier for higher-level AI systems to control the entire humanoid body rather than treating locomotion and manipulation as separate problems.

Source

- interestingengineering.com (2026-08-12) - Original article: New tech helps humanoids run, jump and handle tasks with one control