The Quest for Generalizable Motion Generation: Data, Model, and
  Evaluation
Authors: name: Jing Lin; name: Ruisi Wang; name: Junzhe Lu; name: Ziqi Huang; name: Guorui Song; name: Ailing Zeng; name: Xian Liu; name: Chen Wei; name: Wanqi Yin; name: Qingping Sun; name: Zhongang Cai; name: Lei Yang; name: Ziwei Liu
Published: 2025-10-30 17:59:27

Abstract: Despite recent advances in 3D human motion generation (MoGen) on standard
benchmarks, existing models still face a fundamental bottleneck in their
generalization capability. In contrast, adjacent generative fields, most
notably video generation (ViGen), have demonstrated remarkable generalization
in modeling human behaviors, highlighting transferable insights that MoGen can
leverage. Motivated by this observation, we present a comprehensive framework
that systematically transfers knowledge from ViGen to MoGen across three key
pillars: data, modeling, and evaluation. First, we introduce ViMoGen-228K, a
large-scale dataset comprising 228,000 high-quality motion samples that
integrates high-fidelity optical MoCap data with semantically annotated motions
from web videos and synthesized samples generated by state-of-the-art ViGen
models. The dataset includes both text-motion pairs and text-video-motion
triplets, substantially expanding semantic diversity. Second, we propose
ViMoGen, a flow-matching-based diffusion transformer that unifies priors from
MoCap data and ViGen models through gated multimodal conditioning. To enhance
efficiency, we further develop ViMoGen-light, a distilled variant that
eliminates video generation dependencies while preserving strong
generalization. Finally, we present MBench, a hierarchical benchmark designed
for fine-grained evaluation across motion quality, prompt fidelity, and
generalization ability. Extensive experiments show that our framework
significantly outperforms existing approaches in both automatic and human
evaluations. The code, data, and benchmark will be made publicly available.