GMT: General Motion Tracking for Humanoid Whole-Body Control
Authors: name: Zixuan Chen, name: Mazeyu Ji, name: Xuxin Cheng, name: Xuanbin Peng, name: Xue Bin Peng, name: Xiaolong Wang

The ability to track general whole-body motions in the real world is a useful
way to build general-purpose humanoid robots. However, achieving this can be
challenging due to the temporal and kinematic diversity of the motions, the
policy's capability, and the difficulty of coordination of the upper and lower
bodies. To address these issues, we propose GMT, a general and scalable
motion-tracking framework that trains a single unified policy to enable
humanoid robots to track diverse motions in the real world. GMT is built upon
two core components: an Adaptive Sampling strategy and a Motion
Mixture-of-Experts (MoE) architecture. The Adaptive Sampling automatically
balances easy and difficult motions during training. The MoE ensures better
specialization of different regions of the motion manifold. We show through
extensive experiments in both simulation and the real world the effectiveness
of GMT, achieving state-of-the-art performance across a broad spectrum of
motions using a unified general policy. Videos and additional information can
be found at https://gmt-humanoid.github.io.