The Quest for Generalizable Motion Generation: Data, Model, and Evaluation
Authors: Jing Lin
Ruisi Wang
Junzhe Lu
Ziqi Huang
Guorui Song
Ailing Zeng
Xian Liu
Chen Wei
Wanqi Yin
Qingping SUN
Zhongang Cai
Lei Yang
Ziwei Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Motion Generation
Generalizable
Primary area: generative models
TL;DR: This work introduces a unified framework of spanning data (ViMoGen-228k), modeling (ViMoGen and ViMoGen-light), and evaluation (MBench), to advance generalizable 3D human motion generation by transferring knowledge from other generative fields.

Abstract: Despite recent advances in 3D human motion generation (MoGen) on standard benchmarks, existing models still face a fundamental bottleneck in their generalization capability. In contrast, adjacent generative fields, most notably video generation (ViGen), have demonstrated remarkable generalization in modeling human behaviors, highlighting transferable insights that MoGen can leverage. Motivated by this observation, we present a comprehensive framework that systematically transfers knowledge from ViGen to MoGen across three key pillars: data, modeling, and evaluation. First, we introduce ViMoGen-228k, a large-scale dataset comprising 228,000 high-quality motion samples that integrates high-fidelity optical MoCap data with semantically annotated motions from web videos and synthesized samples generated by state-of-the-art ViGen models. The dataset includes both text–motion pairs and text–video–motion triplets, substantially expanding semantic diversity. Second, we propose ViMoGen, a flow-matching-based diffusion transformer that unifies priors from MoCap data and ViGen models through gated multimodal conditioning. To enhance efficiency, we further develop ViMoGen-light, a distilled variant that eliminates video generation dependencies while preserving strong generalization. Finally, we present MBench, a hierarchical benchmark designed for fine-grained evaluation across motion quality, prompt fidelity, and generalization ability. Extensive experiments show that our framework significantly outperforms existing approaches in both automatic and human evaluations. The code, data, and benchmark will be made publicly available.

Published as a conference paper at ICLR 2026
THE QUEST FOR GENERALIZABLE MOTION GENERATION: DATA, MODEL, AND EVALUATION
Jing Lin Ruisi Wang"* Junzhe Lu Ziqi Huang', Guorui Song", Ailing Zeng', Xian Liu Chen Wei', Wanqi Yin?, Qingping Sun", Zhongang Cai Lei Yang _ Ziwei Liu 18
Junzhe
1 Nanyang Technological University 2 SenseTime Research
3 Tsinghua University
4 The Chinese University of Hong Kong
6 NVIDIA Research
jing@66@c.nt.cdu... e.ntu.edu.sg wangruisil @ @ensetime.com {caizhongang, yanglei] semsttmme.com ziwei.iu@ntu.eduu..
lu-jz24 1ete244mmill.ccm mailss.singhua.cdu.cm
ABSTRACT
Despite recent advances in 3D human motion generation (MoGen) on standard benchmarks, existing text-to-motion models still face a fundamental bottleneck in their generalization capability. In contrast, adjacent generative fields, most notably video generation (ViGen), have demonstrated remarkable generalization in modeling human behaviors, highlighting transferable insights that MoGen can leverage. Motivated by this observation, we present a comprehensive framework that systematically transfers knowledge from ViGen to MoGen across three key pillars: data, modeling, and evaluation. First, we introduce ViMoGen-228K, large-scale dataset comprising 228,000 high-quality motion samples that inte- grates high-fidelity optical MoCap data with semantically annotated motions from web videos and synthesized samples generated by state-of-the-as ViGen mod- els. The dataset includes both motion pairs and vext-video-ae motion triplets, substantially expanding semantic diversity. Second, we propose ViMoGen, flow-matchingg-aase diffusion transformer that unifies priors from MoCap data and ViGen models through gated multimodal conditioning. To enhance efficiency, we further develop ViMoGen-light, a distilled variant that eliminates vidco gen- eration dependencies while preserving strong generalization. Finally, we present MBench, hierarchical benchmark designed for fine-grad tin-inaieed evaluation across motion quality, prompt fidelity, and generalization ability. Extensive experiments show that our framework significantly outperforms existing approaches in both automatic and human evaluations. The code, data, and benchmark will be made publicly available.
INTRODUCTION
Breakthroughs in generative models have profoundly transformed human visual creativity, opening new avenues for artistic expression and content production across modalities, including text (Touvron et al., 2023; Guo et al., 2025), image (Rombach et al., 2021; Labs, 2024), and video (Hong et al., 2022: Wang et al., 2025). critical factor underlying these successes their strong generalization capability: such models not only excel within their training domains but also extend effectively to novel instructions, thereby empowering human creativity in a broad spectrum of applications. In contrast, 3D human motion generation SpeGffi)-seeciffcllll in the texi-to-motion domain remains comparatively underdeveloped, particularly
capacity to generalize to diverse and long-tail instructions. In this work, we propose holistic approach toward generalizable 3D human motion generation, advancing the field through coordinated innovations data, modeling, and evaluation. Our effort is guided by broader vision: to unify and transfer insights from adjacent generative fields, particularly video generation (ViGen), while building motion-speciio innovations that lay the groundwork for future exploration toward general-purposs motion foundation model. While
Equal contribution; Prooecct Lead; Corresponding Author.

---

Published as a conference paper at ICLR 2026
motion generation broadly encompasses audio and music modalities, study focuses exclusively on text-to-motion synthesis to address the critical semantic generalization gap.
MoCop
Gallig
XN
D:T
plaaao sito
ViGmn
shoot arroe, then return the house plop anto sofa"
ViMoGen
(a)
Vi (Duss) Eumm Enaate CHAmLCC CIICAA
Wai canile, plallal kieas, and energetic alde leeps.
Generiizaniity
MelsasCondiian Canasicuuy
Jikts Deguce
Syntheti Video
Pose Quiiiy
Fror
Flotiia
(b)
(c) Optina Mocap Data
In-Dh Wild Video
Pandtmniss
Stiling
Figure 1: Overview of our approach toward generalizable 3D human motion generation. (a)
ViMoGen: Our model demonstrates superior generalization on challenging prompts including martial arts, dynamic sports, and multi-step behaviors. (b) MBench: Comprehensive benchmark evaluating models across dimensions, showing ViMoGen's 's significant improvements over existing methods. (c)
ViMoGen-2228H 228K:: Large-scale dataset with 228,000 motion sequences from diverse sources covering simple indoor to complex outdoor activities.
First, we curate ViMoGen-228K, a large-scale, diversely sourced, and carefully balanced dataset designed to enable generalizable motion generation. A major bottleneck for motion generation lies in data scarcity: unlike other modalities such as text, image, and video (where massive datasets read- ily available online), motion data is orders of magnitude smaller, leading to severely limited semantic coverage. To overcome this, source heterogeneous data and design meticulous curation pipeline. ViMoGen- 228K ultimately comprises 171.5K text-motion pairs and 56.6K text-video- video-motion triplets from three complementary sources. (1) Optical motion capture (MoCap) datasets, which, despite physical constraints, provide high-quality motion signals that serve as critical priors for modeling human dynamics. We standardize 30 MoCap datasets, aligning conventions and augmenting them with text annotations. (2) in-the-wild video-derived motions, which significantly broaden semantic coverage. Following prior efforts (Wang et al., 2024c; Liang et al., 2024a; Lin et al., 2023), we con- struct a rigorous cascaded filtering pipeline that enforces a strict selection scheme: only about ~ % of motions extracted from a large-scale internal video database are retained, ensuring motion fidelity. (3) Synthetic video-derived motions, generated by a state-of-the-art video generation model (Wan et al., 2025) trained on billions images and videos, where humans constitute a substantial subset. Unlike scraping Internet videos, leading ViGen models can produce surprisingly realistic, strategically controlled human videos that are easier to capture with MoCap pipelines, when prompted with crafted long-tail instructions extend behavioral coverage.
Second, we introduce ViMoGen, a versatile diffusion transformer (DiT)-based model designed for generalizable motion generation. The central idea of ViMoGen is harness knowledge from proxy generation tasks that have already demonstrated strong human-centrie generalization (most notably ViGen models, which excel producing high-fidelity human motions). Yet, effectively exploiting diverse and heterogeneous knowledge sources remains underexplored: existing frameworks are often inadequate for systematic knowledge transfer, limiting their ability to generalize across contexts. For instance, prior works (Millán et al., 2025; Albaba et al., 2025) rely on costly optimization- based pipelines to extract motion knowledge from ViGen models. contrast, ViMoGen employs gated fusion and dual-branch design that balances complementary contributions from optical MoCap, in-the-wild videos, and synthetic video data: (i) a Text-to-Motion (T2M) branch that conditions generation on text paired with MoCap priors, and (ii) Motion-to-Motion (M2M) branch that leverages ViGen-derived motion tokens to broaden semantic coverage. This unified architecture enables more efficient and integrated transfer of ViGen priors into the motion diffusion process, combining the semantic richness video models with the high fidelity motion speccifc synthesis. Furthermore, develop ViMoGen-light, lightweight distilled variant that inherits knowledge
2

---

Published as a conference paper at ICLR 2026
directly from ViGen models while bypassing expensive sokt-to-vided inference. As illustrated in Fig. 1, both ViMoGen and ViMoGen- substantially improve generalization over prior approaches.
Third, we present MBBench, comprehensive benchmark for accurate and tine-graing evaluation of motion generation algorithms, with a particular emphasis on generalization capability. The field currently lacks unified evaluation suite that systematically assess different models, measure various aspects of the generated motions, and provide actionable feedback for researchers. Existing evaluation protocols (Guo et al., 2022a) primarily rely on distribution-based metrics (e.g.. FID), which reduce model performance into a single score, sacrificing granularity and often diverging from human judgment (Wang al., 2024a). Moreover, commonly used test prompts (Guo al., 2022a; Punnakkal et al., 2021) are dominated by simple, indoor actions, offering limited insight into generalization. To address these gaps, MBench provides comprehensive and structured assessment framework that it evaluates generated motions along three principal axes: motion quality, motion-condition consistency, and generalization ability. In particular, we emphasize the use of carefully curated open-worts wenndorldd vocabulary to rigorously assess generalization beyond conventional action categories. Overall, MBench spans nine distinct dimensions, each guided by mindfully designed prompts tailored to its specific characteristics. Importantly, the benchmark undergoes human validation to confirm that the automated components closely align with human judgments and preferences.
We also reveal some valuable insights that we find during our extensive experiments. Our contributions can be summarized as:
We curate ViMoGen-228K, large-scale human motion dataset comprising both text-video- motion triplets and text-motion pairs. The dataset features both broad semantic coverage and motion quality.
We propose ViMoGen, DiT-based model with innovative gating and masking mechanisms that seamlessly integrate knowledge from multiple modalities while preserving learnet motion priors, along with its efficient distilled variant, ViMoGen-light.
We build MBench, comprehensive, fine-gratd grainrdi and numan-alignen benchmark that effectively assesses motion generalization, motion-condition consistency, and motion quality,
2 METHOD
We present comprehensive framework for generalizable text-driven human motion generation, built on a flow-based diffusion transformer architecture. Our approach, which we call ViMoGen, tackles the generalization bottleneck by systematically unifying the strengths of high-quality motion capture (MoCap) data and the broad semantic knowledge from large-scale video generation (ViGen) models.
2.1 PRELIMINARIES
Problem Definition. Our objective is synthesize high-quality human motion from textual descrip- tions, formulated as text-to-motion translation task. Following DART (Zhao et al., 2025), we employ an overparameterized representation based on the SMPL-X (Pavlakos al., 2019) model, where each frame is represented as D = 276 dimensional vector encompassing body root translation, root rotation, joint rotations, joint locations, and joint velocities.
Flow Matching for Motion Generation. We employ continuous normalizing flows (Lipman et al., 2022) to model the data generation process. Given a clean motion sequence Zy E NXD random noise (0, I), and timestep E [0, 1]. we define the forward process using rectified flow interpolation: F, = (1 - t)c + taxg. The velocity field v; = .ig represents the optimal transport path. Our model fe is trained to predict this velocity field given the noisy motion sequence Ir. timestep t, and conditioning information c which represents the textual description of the desired motion. The training objective is formulated as: L Ezo.e.t.ellll fo (x,, t, vollā), where encodes semantic guidance for generating motions that align with the given text prompt.
2.2 UNIFYING VIDEO AND MOTION GENERATION MODEL PRIOR
The fundamental challenge in motion generation lies in reconciling two competing objectives: achieving high motion quality and maintaining broad generalization capability. Traditional optical MoCap datasets provide precise motion dynamies but suffer from limited semantic diversity due
3

---

Published as a conference paper at ICLR 2026
Video Generation
Timestep
(b) Gating Diffusion Block
Text
martial sequence
KoCop
Gating Diffusion Block
e Video Motion
TToions
! T2M Crass
@ M2M
V-Tolums
Cross
Linear
36 Noisy Motion
Gating
Sei Sef-Attention
M-Tokons
Denoised Motion
Figure 2: Overview of ViMoGen. (a) Our model takes a text prompt as input and leverages both text encoder and an offline video generation model to produce textual and video motion tokens. These are fused with noisy motion inputs through a stack of gating Diffusion Blocks. (b) Each block includes self-attention, an adaptive gating module, and two cross-altention branches: Text-to-Motion (T2M) and Motion-to Mosoontt--Motinn (M2M). Only one branch is activated at time, enabling the model to adaptively balance robustness and generalization.
to studio constraints. Conversely, video generation models trained on web-scale data demonstrate remarkable generalization across diverse actions but produce motions with reduced fidelity and amplitude accuracy. Our approach bridges this gap through novel cross-modal fusion architecture that adaptively leverages the complementary strengths of both modalities.
Pipeline Overview. As illustrated in Fig. 2(a). ViMoGen processes textual descriptions through three parallel pathways to extract complementary representations. The text encoder produces semantic embeddings Ztent that capture linguistic structure and action semantics. During inference, offline video generation model produces videos from the text, from which we extract reference motions using visual MoCap models (Patel & Black, 2025). These encoded into video motion tokens Rvideo- During training, to bridge the domain gap and improve efficiency, we simulate Zvideo by perturbing ground-truth motions with controlled noise patterns that mimic visual MoCap errors. Interestingly, we find that using intermediate ViGen features as Svideo leads to slow convergence and minimal performance gains. This may be attributed to the substantial modality gap between videos and motions that is difficult to bridge with current data scales. The noisy motion tokens 2motion. text embeddings Stext and video motion tokens Zvideo are then processed through a stack of specialized cross-modal fusion blocks.
Dual-Branch Fusion Mechanism. Each fusion block, depicted in Fig. 2(b), employs dual-branch architecture with mutually exclusive activation. Inspired by modern designs from Wan Video (Wang al., 2025), our architecture begins with standard self-attention over motion tokens, followed by two specialized cross-attention branches. Notably, the Text-to-Motion (T2M) and Motion-to-Motion (M2M) branches share approximately 66% of the DiT parameters (including self-attention and FFN layers), separating only at the attss-attention mechanism. The T2M branch enables motion tokens to attend directly to text embeddings, leveraging the precise motion dynamics learned from high-quality optical MoCap data. This branch performs well in generating physically plausible motions for actions well-represented attend video motion tokens, transferring semantic knowledge from the video generation domain. This branch captures the rich contextual understanding and generalization capabilities derived from web-scale video training data, enabling the model handle novel rare actions absent from MoCap datasets. The key insight is that these branches operate in complementary regimes: the T2M branch provides robustness and quality assurance, while the M2M branch extends semantic coverage and generalization.
traditional mocap datasets. The M2M branch allows motion tokens to
Adaptive Branch Selection. The selection between branches governed by an adaptive mechanism that assesses the semantic alignment between the text prompt and the generated video motion. During inference, we first generate video motion offline using the text prompt, then employ a Vision- Language Model to perform a binary alignment check between the generated video and the text prompt. If aligned, the M2M branch is activated to refine the video motion; otherwise, the model falls back to the T2M branch to generate motion from text. This instance-level adaptive gating strategy enables ViMoGen to dynamically balance between leveraging video model generalization for novel actions and maintaining motion quality through established mocap priors. During training,
4

---

Published as a conference paper at ICLR 2026
employ a curriculum approach based dataset characteristics to regulate branch activation. For manually annotated datasets, we prioritize the T2M branch to ensure precise semantic alignment; for large-scale auto-labeled datasets, we encourage broader generalization by increasing M2M branch usage. When the M2M branch is activated, we generate Evideo by applying a compound noise strategy to ground-trutt motions-inclln mimic the artifacts of visual MoCap. Furthermore, since visual MoCap often yields inaccurate global trajectories, we mask the global translation in Svideos forcing the M2M branch rely on local pose dynamics. This simulation avoids the need for online video generation during training.
random corruption, jitter simulation, and temporal dropout-
2.3 DISTILLING MOTION PRIOR FROM VIDEO GENERATION MODEL
While ViMoGen unifies video generation generalization with motion generation quality, the require- ment for video generation model inference introduces significant computational overhead. To address this limitation, we propose efficient variant, ViMoGen-light, which exclusively uses the T2M branch and eliminates video generation dependencies.
Semantically-Diveres Data Synthesis. Our preliminary experiments reveal that model generalization closely correlated with the diversity of action verbs and phrases encountered during training (detailed in Section 5). Based this key insight, systematically construct a comprehensive set of action descriptions to capture the full spectrum of human motion semanties. We begin by extracting feegurryy verbs and action phrases from linguistic resources, followed by semantic clustering and expansion using large language models (LLMs). This process yields a comprehensive synthetic dataset of 14,000 novel prompts covering diverse action categories, such a person is breakdancing and a knight is jousting, which are crucial for enhancing generalization beyond standard benchmarks.
Knowledge Distillation. The ViMoGen-light model is created through knowledge distillation, where the generalization capabilities of the T2V prior are distilled into a lightweight architecture. The complete ViMoGen pipeline serves as powerful teacher model. For each of the synthesized prompts, the teacher generates corresponding high-quality motion. The student model, ViMoGen-lin -iggt,, is then trained on this synthetic dataset using standard flow matching objectives. Through this process, the student model inherits broad semantic knowledge and generalization priors from the teacher's 's dual-branch architecture, thereby enabling inference without ViGen model. Ablation studies confirm that this distilled data is highly effective for improving motion generation model generalization while maintaining computational efficiency.
3 VIMOGEN-228K DATASET
In this section, we introduce ViMoGen- 228K, a dataset carefully designed from broad spectrum of sources, featuring a competitive scale, high motion quality and broad semantic diversity. Table I presents a comparison between ViMoGen-228KK and existing datasets. Optical MoCap datasets (e.g., HLman (Guo et al., 2022a)) are indispensable for establishing strong tion priors but they are limited in scale. In contrast, web video-based datasets scale far more easily but typically compromise motion quality and often exhibit semantic biases (c.g.. Motion-X (Lin et al., 2023) focuses predominantly on sports and gam- ing scenarios). ViMoGien-228K is designed to strike a deliberate balance between motion quality (172K high-fidelity text-motion pairs) and semantic diversity (56K diverse text-video-motion triplets) through three complementary strategies: (1) aggregating large collection of 30 optical MoCap datasets, surpassing the scale of any previous optical MoCap resource; (2) filtering a massive pool of web videos through a rigorous selection pipeline to maximize motion fidelity while enriching in- the-wild representations; and (3) strategically generating synthetic videos to further expand semantic coverage, particularly in domains underrepresented difficult to capture in real-world datasets.
Dataset
#Clip
#Hour
Scene
Motion
Video
KFT-ML
3911
11:2
Indocr
GT
GT
BABEL
13,220
43.5
Indoor
GT
HumanMI.3 3D
14,616
28.6
Indoce
GT
Motion-X
81,084
Pscudo GT
Motion-Xre
120,500
350
In- Wild
Pseudo GT
MotionMillin
2M
2000
In-the- Wild
Pscndo GT
ViMoGen: 228K
228,236
369.4
Optical MoCup'
171,542
292.7
Indoce
GT
In-the-Wild Wild Video!
41,971
Wild
Pseudo GT
Synthetic Video"
14,723
16.6
In-the Wild Psendo GT
Table 1: Comparison of ViMKGen-228K with ex- isting human motion datasets. Unified 29 datasets. 'ggresssivlyy filtered from 10M clips. # "Strategically generated for semantic coverage.
5

---

Published as a conference paper at ICLR 2026
We explain the data collection, filtering, and annotation below. Due to space limitations, more details are available in Appendix D.2.
Optical MoCap Dataset. We unify 30 publicly available datasets (for the complete list of datasets, please refer to Appendix D.2.1), standardizing to SMPL-X (Pavlakos et al., 2019) format and 20 frames-per-second (fps). Long sequences segmented into 5-second clips (100 frames) without overlap. Multi-stage filtering includes T-pose removal, minimum 3-second length requirements, and quality assessment via MBench eliminate jitter and low-dynamics motions. Empirical analysis led us to exclude several datasets due to poor text-motion consistency and challenging high-speed motion characteristics in actions like dancing. Eventually we select 17 of the 30 candidate datasets (172k of 267k clips) for our final training corpus. For existing MoCap datasets like HumanML3D, we utilize their original text annotations when available. For optical MoCap datasets that lack text annotations or only have class lahels, we design a pipeline to generate structured textual descriptions. Specifically, we render depth videos of the human mesh for each motion sequence, then provid frames from these videos at 5 fps to Gemini 2.0 for motion description. We engineer system prompt to elicit structured annotations, including a brief summary, key actions, frame-level details, and motion style. We apply heuristic filtering multimodal LLM-generated annotations, removing invalid responses.
In-the-Will Video Data. From an internal mega-scale video database (consisting of large public datasets (Wang et al., 2024b; Chen et al., 2024; Ju et al., 2024; Grauman et al., 2024)), and videos obtained via keyword searches across sports, daily activities, cultural performances, and occupational tasks), obtain 10M human-centric video clips after quality assessment (removing clips with low visual quality, shaky camera, excessive motion, poor lighting, or heavy occlusion), making the source competitive scale against existing works (Lu et al., 2025; Fan et al., 2025). We annotate the clips with CameraHMR (Patel & Black, 2025) and SMPLest-X (Yin et al., 2025) to extract 3D motion in the SMPL-X representation. We then canonicalize the global orientation of each motion so that the initial frame faces the y+ axis. Visual MoCap artifacts, such as jitter and inaccurate global translation, are mitigated by applying post-processing smoothness algorithms. Furthermore, only local body poses are supervised during training. For video clips without text annotation, we provide the original RGB videos as input to Gemini 2.0 Flash for motion captioning.
Synthetic Video Data. With the rapid development in ViGen, leading models (Wan et al., 2025) trained on billions of images and videos demonstrate unprecedented controllability and generalization capability. Leveraging these characteristics, videos that are easy to perform vision- based MoCap (e.g.. stable camera movement, full body, single person) that are difficult to find the equivalent in real-world videos. To systematically expand semantic coverage into underrepresented domains, we construct a long-tail vocabulary derived from large-scale video captions in ViGen datasets such as OpenHuman Vid (Li al., 2024), L.LVA- Video- 178K (Zhang et al., 2024c), and an internal collection. The compiled action verbs and descriptive nouns undergo semantic clustering and deduplication, yielding over 20,000 diverse prompts, These prompts are then processed with the 1an2.1 (Wan et al., 2025) text-to-video model to produce 81-frame clips at 16 frames per second (fps). The results further undergo standard filtering and annotation pipeline with additional refinement via our pre-trained ViMoGen M2M branch to obtain 14,000 high-quality samples.
explicitly instruct the ViGen model to generate
MBENCH
In this section, we introduce the main components of MBench, a novel hicrarchical benchmark designed to provide granular and multifaceted assessment of generated human motions. Section 4. outlines the motivation behind the design of the evaluation dimensions, along with their definitions and evaluation methods. To validate MBench 's alignment with human perception, conduct human preference annotations for each evaluation dimension, as discussed in Appendix C.1.
4.1 EVALUATION DIMENSIONS
As shown in Fig. 3(b), MBench offers structured approach by decomposing the evaluation into nine dimensions in three primary pillars: Motion Generalization, Motion Alotion-Condition Consistency, and Motion Quality. Finally, we construct and release curated prompt lists designed go beyond

---

Published as a conference paper at ICLR 2026
Commoiizabiley
127
8.31
2LI5
ho di
Sand
Motion Quality
Motia Condition
Salk K
Tenpural
Conssteacy
1S DR POU
Oereeiad Malee
put 1E bend Lay
tur do
Croadd Reartater
1000
move
Jampac C. Ljaoko
Segret
(a) Top 100 Verbs (Left: MBench. Right: HumanML3D)
(b) Evaluation Dimensions of MBench
Figure 3: Overview of MBench. (a) MBench features more balanced distribution and vastly different prompt designs compared to HumanML3D. (b) MBench designed is to systematically evaluate motion generation algorithms across nine dimensions, focusing on motion quality, prompt-following and generalization capability.
Bodysurfing
Putting on shoes
Martial arts
Climbing a ladder
MotionL.CM
T2M-GPT
MoMask
ViMoGen (ours)
Figure 4: Qualitative comparison on MBench prompts. We show keywords in prompts for simplicity.
simple memorization and dataset overlap, serving as challenging cases for model evaluation. Details of the prompt suits are included in Appendix C.2.
Motion Generalizability. This dimension specifically targets motions that are rare entirely absent from commonly used MoGen and ViGen datasets. By evaluating such out-of-distribution actions, directly measures model's ability transcend memorization of seen patterns and instead generalize to novel semantic inputs. In doing so, it highlights the capacity of the model to produce diverse, contextually coherent movements. single action (e.g., walking) can be expressed through numerous linguistic variations that convey subtleties of emotion, speed, and intent. To construct an open- world vocabulary for this dimension, we systematically extract near-synonyms and related expressions from the Oxford English Dictionary. These are then composed into descriptive sentences that encode contextual variations physical style and temporal dynamics. For example, the simple action trample is expanded into a semantically richer phrase such trample over flowerbeds with heavy, lumbering steps. This methodology ensures that the benchmark probes deeper language-motion associations rather than merely testing surface-level mappings. For evaluation, generated motions are rendered as videos with realistic ground planes and natural camera movements, thereby avoiding oversimplified stick-figure visualizations. A Aision-language model (VLM) is then tasked with assessing whether the generated motions faithfully depict the specified rare or unseen actions.
Motion-C Motion-Condition Consistency. This metric quantifies semantic fidelity between generated motion and text prompts. While pre-trained action recognition models like TMR (Petrovich et al., 2023) could assess action accuracy, they suffer from training data biases and limited generalization. UMT (Li
7

---

Published as a conference paper at ICLR 2026
Models
Motion Condition Consistency
Motion Gemeralizabliiy
Jitter Degrev
Dynamic Degree
Foot Floating
Foot Sliding
Body Pemetration
Pose Quality
MDM (Tever et al, 2023)
0.42
0.51
0.0136
0.03766
0.156
0.01%
1.68
2.67
T2M-GPT (Zhung es 2023x)
0.39
0.38
0.0156
0.0349
0.209
0.0156
1.33
2/3
FineMoCien (Zhang
20254)
0.37
0.42
0.0118
0,03866
0.781
0,0091
1.18
2:28
MotionLCM
al. 2024)
0.48
0.193
2,40
MoMask (Gun et al,, 2024)
0.38
0.44
0.0147
0,0396
0.178
0,0147
1.48
2,67
MetionDiffise (Zhang et 201400
0.44
0,0289
0.126
2,21
MotionCraft (Bian al., 2025)
0.42
0.45
0.0132
0,0420
0.402
0,0090
1,15
2.12
ViMoGen (Ouns)
0.0251
0.204
2.38
ViMoGen light (Ouns)
0.47
0.55
0.0129
0.0294
0.155
0,0051
1443
2.10
Table 2: Quantitative comparison on MBench. The best performance is bolded.
et al., 2023b), used in VBench (Huang et al., 2024a:b), evaluates action execution via video-text alignment but underperforms in motion-only scenarios lacking rich visual context. For this assessment, adopted an evaluation pipeline similar to that used for Motion Generalizability. Using chain- of-thought prompting, the VLM describes observed motion, then selects the most suitable label from ten candidates: one ground-trut and nine distractors. Distractors are formed using cosine similarity thresholds from three percentile ranges (below 5th, 47-52nd, above 95th), ensuring semantic variability and consistent evaluation.
Motion Quality. Motion Quality evaluation is decomposed into temporal and frame-wise aspects. Temporal Quality assesses cross-frame consistency: jitter degree (quantifying articulatory instability via joint accelerations and orientation changes). foot contact metrics including ground penetration, foot floating, and foot sliding (evaluating physical plausibility of foot-ground interactions), and dynamic degree (measuring overall motion intensity through joint velocities). Frame-Wiin Quality evaluates individual pose characteristics, including body penetration rates computed via BVH-based collision detection and pose naturalness assessed using Neural Riemannian Distance Field (NRDF) model (He et al., 2024) that captures anthropomorphic plausibility priors.
EXPERIMENTS
In this section, we compare ViMoGen with state-of-the-art (SOTA) methods on MBench (Section 5. 1), and conduct ablation studies (Section 5.2). We include implementation details (Section E) and results (Section F) on the standard HumanML3D benchmark (Guo et al., 2022a) in the Appendix.
5.1 COMPARISON WITH SOTA METHODS
We benchmark ViMoGen and its distilled variant, ViMoGen-light, against leading motion generation models: MDM (Tevet et al., 2023), MotionLCM (Dai et al., 2024), T2M-GPT (Zhang et al., 2023a), MotionDiffuse (Zhang et al., 2024b), FineMoGen (Zhang al., 2023d), MotionCraft (Bian et al., 2025), and MoMask (Guo et al., 2024) using our comprehensive MBench evaluation framework.
Quantitative Results. Table 2 presents the quantitative comparison MBench. Our full model, ViMoGen, significantly outperforms all baselines on the key semantic metrics of Motion Condition Consistency and Generalizability, showcasing the powerful advantage of leveraging a T2V model's rich semantic prior. The distilled ViMoGen-light variant achieves a Generalization Score on par with the strongest baseline, demonstrating that this knowledge can be effectively transferred an efficient model that does not require video generation at inference. Our improved semantic performance stems from using more powerful T5 text encoder and a multi-token aross-attention mechanism, which provides richer textual guidance than the single-token CL.IP features used in prior works. This pursuit of generalization introduces trade-off in motion quality. By incorporating diverse, video-sourced motions (c.g.. "tying shoelaces"), our training data features more complex actions with less global movement compared to comotionn-havy datasets. This data characteristic leads our models to produce more stable patterns, explaining both their superior Jitter Degree and lower Dynamic Degree.
Qualitative Analysis. Fig. 4 presents qualitative results using representative prompts of each MBench dimension. ViMoGen demonstrates superior rext-to-motion alignment and physical plausibility across diverse scenarios. For out-of-domain prompts like 'body surfing'. ViMoGen leverages semantic knowledge from video generation priors to produce plausible motions despite this action being absent
8

---

Published as a conference paper at ICLR 2026
Neanch Selection
Motion Cundition Conditency
Mution
Jiteer Degree
Foot Sllding
Training
Testing
Mation Condition
Modion Generalizabliit
Foot
Generaltrabliiy
Consistency
Süiding
Video Generation Baselmee
0.51
0.58
0.0193
0,0151
Motion
0.40
TEM Only
Motion
Video
0.52
0.39
0,0031
MEM Ouly
0.51
0.59
0.0145
0,0113
Video
Adapiive Caring
0L53
0.68
0,0108
0.0064
Video
Vidao
0.41
0.44
0.0032
Table 3: Ablation study on different branch selec- tion strategies for ViMoGen. Our adaptive method significantly outperforms single-branch basclines in generalization and accuracy.
Table Ablation on text prompt style. Train-
ing with descriptive video-style text and test- ing on concise motion-style text yields the best overall performance.
Training Butswts
Malken Clip Numbler
Matian Coudition Coasistancy
Mution
Fost Staling
Text Encoder
Motion Condition Consistency
Modion
Foot Stiding
Body
Goserdicahhiit
Generaliabiliiy
Penetration
HanacklLSg
Mioer Dars
Viani Macip Dara
59k
428
11:3
0:41
L.41
0.50
0.88
0.44
01002
010055
0.663
15
CLIP
0.32
0.35
0.44
0.0023
105
1.39
Systeeii Valeo Duta
14k
0.47
0.55
0.0051
MLLM
0.38
0.46
0.0022
1.51
Table 5: Ablation on the composition of the train- ing data. Adding diverse data sources progres- Compared to CLIP (Radford et al., 2021) and sively improves generalization, with synthetic MLLM (Kong et al., 2024), T5-XXL (Raffel et al., 2020) provides the best balance of generalization and motion quality.
Table 6: Ablation on the choice of text encoder.
data providing the largest gains.
from standard training datasets. Competing methods such as T2M-GPT (Zhang et al., 2023a) generate implausible or generic motions for such novel prompts. ViMoGen also excels on common daily actions, confirming robust performance across the full semantic spectrum. These results validate core contribution of bridging the semantic gap between limited mocap data and diverse real-world motion requirements.
5.2 ABLATION STUDY
Effectiveness of Adaptive Selection. We validate our adaptive branch selection architecture by comparing against several baselines in Table 3. The video generation baseline, using motion estimated directly from text-to-video models, produces semantically relevant but low-quality motions with significant jitter and foot sliding. The M2M model uses this noisy video-based motion as a reference, improving quality metrics but without significant gains in accuracy or generalization. Conversely, the T2M model, trained solely on high-quality MoCap data, achieves optimal motion quality but exhibits limited generalization capabilities. Our adaptive gated model successfully integrates the strengths of both approaches. The source of this improved generalization lies in its intelligent fallback mechanism. For novel prompts where the ViGen prior is semantically accurate, our model leverages its rich guidance. However, for highly dynamic actions like turning around or falling, where ViGen models can be unstable and produce distorted motions (e.g.. "body twisting"), ") our adaptive mechanism switches to the more robust T2M branch. This branch, guided by the powerful semantic priors of the text encoder, ensures plausible motion generation when the iideo-based guidance is unreliable. We further provide qualitative examples illustrating this adaptive process in Appendix G.
Impact of Multi-Source Training Data. We analyze the contribution of each data component within our ViMoGen-222K dataset by incrementally adding data sources to a baseline ViMoGen-light model trained solely HumanMI.30 (Guo et al., 2022a). The results in Table 5 show that progressively adding data from diverse sources steadily improves the model's 's action accuracy and generalization. Notably, the inclusion of synthetic video data, despite its small size (14k clips), provides substantial boost to the generalization score (from 0.50 to O.55). This progression confirms that semantic diversity, even from smaller datasets, significantly impacts generalization.
Impact of Text Encoder and Representation. We investigate how different text encoders and representations affect model performance. First, we compare three pre-trained text encoders: CLIP (Radford et al., 2021), T5-XXL (Raffel et al., 2020), and a Multimodal Large Language Model (MLLM) (Kong et al., 2024). As shown in Table 6, both T5-XXL and the MLLM significantly outperform CLIP in generalization capacity. While prior works like MoMask (Guo et al., 2024) successfully employed CLIP on the HumanML3D (Guo et al., 2022a) benchmark, our findings indicate that more powerful text encoders are essential for handling the greater linguistic complexity

---

Published as a conference paper at ICLR 2026
T2M Beunch
MEM Branch
Modian Condition Consisieccy 1
Motion Generalizability
Jitter Degree
Dynomic Degree
Foot Fleoting
Foot Sldting-
Body Penetration
Quality
Pose
ViMaGien
VIMoGes
0.68
0J1108
0.0064
2.382
ViMotinn (Chan)
DNO
0.65
0.1188
0.005
2.45%
MotooLCM
Vi5ooGen (Oues)
0.49
0.66
0U0179
0.0074
0.217
0/0136
1,570
2.485
MotioLCMM
DNO
0.63
0.0205
0/0217
2.567
Table 7: Analysis of mutual benefits between T2M and M2M branches. We compare our joint training approach against baselines using MotionI.CM (Dai et al., 2024) and DNO (Karunratanakul et al., 2024).
required for true generalization. Next, we analyze the impact of prompt style, comparing concise motion-style text with rich, descriptive video-style text. The detailed prompts and visualization examples are available in Appendix G. Table 4 reveals that training with descriptive video-style text while testing concise motion-style style descriptions yiclds the best performance. This suggests rich descriptions function as effective data augmentation, improving model robustness and alignment with pretrained text encoder expectations.
Mutual Benefits of Dual-Branch Training. To clarify whether the T2M and M2M branches benefit each other, analyze our joint training strategy where branches share approximately 66% of DiT parameters (self-attention and FFN layers). In Table 7, we compare our unified approach against selection-based baseline that uses external VLM scores choose between a SOTA T2M model (MotionLCM (Dai et al., 2024)) and a motion refinement pipeline (DNO (Karunratanakul et al., 2024)). We also perform ablations replacing one of our branches with these baselines. The results demonstrate that our joint training significantly outperforms simple model selection and single-branch hybrids. The M2M branch introduces diverse motion priors from optical MoCap data (often lacking text labels) into the shared parameters, enhancing the plausibility of T2M generation. Conversely, the T2M branch enforces semantic alignment, benefiting the overall representation. This synergy results in superior condition consistency and generalizability compared to using separate SOTA models.
CONCLUSION AND DISCUSSION
In this work, we tackle the fundamental challenge of generalizable 3D human motion generation through coordinated innovations in data, modeling, and evaluation. (1) ViMoGen-228K dataset curated with more than 228,000 motion clips featuring both high-quality and broad semantic coverage. (2) ViMoGen model unifies priors from video generation with motion-specific knowledge through innovative gated diffusion blocks, achieving state-of-the-art performance in both action accuracy and generalization. Additionally, the lightweight variant, ViMoGen-light effectively distills generalization capabilities while significantly reducing computational overhead. (3) MBench provides the first comprehensive benchmark for fine-grained evaluation across generalization capabilities, condonncondition consistency motion quality. Moreover, a detailed discussion on related work is included in Appendix B.
Limitations and Future Work. Despite these advancements, our method currently faces two primary limitations. First, the architecture is designed for single-person motion generation and does not yet support multi-person interactions. Second, for complex, high-dynamic motions (e.g., gymnastic twists), the model relies heavily the video generation model for initialization; if the video prior is distorted, the M2M branch may struggle to fully correct the dynamics. We also observe a trade-off where superior generalization does not always yield the highest scores specific quality metrics (e.g.. Dynamic Degree). This is primarily attributed to artifacts in current visual MoCap data (such as foot sliding) and the limited dynamic range inherent in generated videos. Future work will address these challenges by incorporating advanced contact-aware MoCap algorithms to enhance data precision and exploring training strategies that force the model to extrapolate high-quality dynamics from video initializations.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. Our study relies on motion datasets that may contain data derived from human subjects. This may raise potential concerns regarding privacy and consent. To mitigate these issues, we only use publicly available optical motion capture datasets released under sesearch-friendie licenses. For video data collected from the internet, we focus solely on motion information, and no personally identifiable information was used in our study. Regarding the motion generation model, while and rchabilitation, it could also be misused for deceptive content creation surveillance. We acknowledge this risk and emphasize that our model intended solely for scientific research and beneficial applications.
potential for positive applications in animation, roboties,
REPRODUCIBILITY STATEMENT
We have taken extensive steps ensure the reproducibility our work. All code, datasets, and benchmark configurations will be made publicly available in anonymous repository. Details of data processing, experimental setup, and training protocols are thoroughly described in the main paper and supplementary materials. Hyperparameters, implementation choices, and evaluation metries also documented to enable independent verification and fair comparison.
ACKNOWLEDGEMENTS
This research is supported by cash and in-kind funding from NTU S-Lab and industry partner(s). This study is also supported by the Ministry of Education, Singapore, under MOE AcRF Tier (MOE-T2EPP2022--0012 MOE-T2EP20023-0000)
REFERENCES
Mert Albaba, Chenhao Li, Markos Diomataris, Omid Taheri, Andreas Krause, and Michael Black. Nil: No-data imitation learning by leveraging pre-trained video diffusion models. arXiv preprint arXiv:2503. 2025.
Simon Alexanderson, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter. Listen, denoise, action! audio-driven motion synthesis with diffusion models. ACM Trans. Graph., 42(4):44:1444:00 2023. doi: 10.1145/3592458.
Joao Pedro Araújo, Jiaman Li, Karthik Vetrivel, Rishi Agarwal, Jiajun Wu, Deepak Gopinath. Alexander William Clegg, and Karen Liu. Circle: Capture in rich contextual environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
21211-21221, 2023.
Bharat Lal Bhatnagar, Xianghui Xie, Ilya A Petrov, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. Behave: Dataset and method for tracking human object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
15935-15946, 2022.
Yuxuan Bian, Ailing Zeng, Xuan Ju, Xian Liu, Zhaoyang Zhang, Wei Liu, and Qiang Xu. Motioncraft: Crafting whole-body motion with plug-and-play multimodal controls. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 1880-1888, 2025.
Kang Chen, Zhipeng Tan, Jin Lei, Song-Hai Zhang, Yuan-Chen Guo, Weidong Zhang, and Shi-Min Hu. Choreomaster: choreography oriented music-driven dance synthesis. ACM Transactions on Graphics (TOG), 40(4):1-13, 2021.
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wer wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin- Ying Lee, Jian Ren. Ming-Hsuan Yang, al. Panda-70m: Captioning 70m videos with multiple cross-mouee coos-modalityy teachers. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13320- 13331, 2024.

---

Published as a conference paper at ICLR 2026
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18000-18010, 2023.
Wenxun Dai, Ling- Hao Chen, Jingbo Wang, Jinpeng Liu, Bo Dai, and Yansong Tang. Motionlem: Real-time controllable motion generation via latent consistency model. In European Conference on Computer Vision, pp. 390-408. Springer, 2024.
Ke Fan, Shumlin Lu, Minyue Dai, Runyi Yu, Lixing Xiao. Zhiyang Dou, Junting Dong. Lizhuang Ma, and Jingbo Wang. Go to zero: Towards zero-shot motion generation with million-scale data. arXiv preprint arXiv:2507.07000 2025.
Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J. Black, and Otmar Hilliges. ARCTIC: A dataset for dexterous bimanual hand-object manipulation. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Mihai Fieraru, Mihai Zanfir, Elisabeta Oneata, Alin-Ionut Popa, Vlad Olaru, and Cristian Smin- chisescu. Learning complex 3d human self-contact. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 1343-1351 2021a.
Mihai Fieraru, Mihai Zanfir, Silviu Ciiit-Cristian Pirlea, Vlad Olaru, and Cristian Sminchisescu, Aifit: Au- tomatic 3d human-intess intmann-itterretablle feedback models for fitness training. In The IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), June 2021b.
Mihai Fieraru, Mihai Zanfir, Elisabeta Oncata, Alin-lonut Popa, Vlad Olaru, and Cristian Sminchis- escu. Reconstructing three-dimensional models of interacting humans. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattrrn Recognition, pp. 19383-19400, 2024.
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 5152-5161, 2022a.
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang. Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp. 5152-5161, 2022b.
Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In European Conference on Computer Vision, pp. 580-597. Springer, 2022c.
Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked modeling of 3d human motions. In Proceedings of the IEEF/CVE Conference on Computer Vision and Patlern Recognition, pp. 1900-1910, 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 2025.
Félix G Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal. Robust motion in- betweening. ACM Transactions on Graphics (TOG), 39 2020.
Yannan He, Garvita Tiwari, Tolga Birdal, Jan Eric Lenssen, and Gerard Pons-Moll. Nrdf: Neural riemannian distance fields for learning articulated pose priors. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 1661-1671, 2024.
Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205. 15868, 2022.
12

---

Published as a conference paper at ICLR 2026
Chun-Hao P. Huang, Hongwei Yi, Markus Höschle, Matvey Safroshkin, Tsvetelina Alexiadis, Senya Polikovsky, Daniel Scharstein, and Michael Black. Capturing and inferring dense full- body human-scene contact. In Proceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 13274-13285, June 2022.
Ziqi Huang. Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11807-21818 2024a.
Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang, Yaohui Wang, Xinyuan Chen, Ying-Cong Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench++: Comprehensive and versatile benchmark suite for video generative models. arXiv preprint arXiv:2411.13503, 2024b.
Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Zhiyuan Zhang, Yixin Chen, He Wang, Yixin Zhu. and Siyuan Huang. Full-body articulated human-object interaction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2023.
Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin
Zhu, and Siyuan Huang. Scaling up dynamic ncenn-scene interaction modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1737-1747, 2024.
Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralytics yolov8, 2023. URL https: //gtt/ub. http/uutraatticsa//ooooo wtw.yytiiiitttrrattttt...t....
Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. Miradata: A large-scale video dataset with long durations and structured captions. Advances in Neural Information Processing Systems, 37:48955-4488 2024.
Korrawe Karunratanakul, Konpat Preechakul, Emre Aksan, Thabo Beeler, Supasorn Suwajanakorn and Siyu Tang. Optimizing diffusion noise can serve as universal motion priors. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1334-1345, 2024.
Manuel Kaufmann, Jie Song, Chen Guo, Kaiyue Shen, Tianjian Jiang, Chengeheng Tang, Juan José Zárate, and Otmar Hilliges. Emdb: The electromagnetic database of global 3d human pose and shape in the wild, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14632-14643, 2023.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprini arXiv:2412.03600 2024.
wtt.c//occooooreeeeeeecee Black Forest Labs. Flux. httpps///gthhb...cob/bbaccttttt
flux, 2024.
Nhat Le, Tuong Do, Khoa Do, Hien Nguyen, Erman Tjiputra, Quang D Tran, and Anh Nguyen. Controllable group choreography using contrastive diffusion. ACM Transactions Graphics (TOG), 4-16) 2023.
Buyu Li, Yongchi Zhao, Shi Zhelun, and Lu Sheng. Danceformer: Music conditioned 3d dance generation with parametric motion transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 1272-1279, 2022.
Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, et al. Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation. arXiv preprint arXiv:24/2.001/5 2024.
Jiaman Li. Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Transactions Graphics (TOG), -21,
Kunchang Li, Yali Wang, Yizhuo Li, Yi Wang. Yinan He, Limin Wang, and Yu Qiao. Unmasked teacher: Towards training-efficienn efficient video foundation models. In Proceedings of the IEEE/CVF international conference computer vision, pp. 19948-19964 2023b.
13

---

Published as a conference paper at ICLR 2026
Ronghui Li, Junfan Zhao, Yachao Zhang, Mingyang Su, Zeping Ren, Han Zhang, Yansong Tang, and Xiu Li. Finedance: fine-gres teee-gaaied choreography dataset for full body dance generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10234-10243, 2023c.
Ruilong Li, Shan Yang, David A. Ross, and Angjoo Kanazawa. Learn to dance with aist++; Music conditioned 3d dance generation, 2021.
Han Liang. Jiacheng Bao, Ruichi Zhang, Sihan Ren, Yuecheng Xu, Sibei Yang, Xin Chen, Jingyi Yu, and Lan Xu. Omg: Towards vocabuacyb motion generation via mixture of controllers. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp.
482-493, 2024a.
Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: iiffusion-basee multi- human motion generation under complex interactions. International Journal of Computer Vision, pp. 1-21, 2024b.
Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x: A large-scale 3d expressive whole-body human motion dataset. Advances in Neural Information Processing Systems, 36:25268 -25280, 2023.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:22/0.02747, 2022.
Haiyang Liu, Zihao Zhu, Giorgio Becherini, Yichen Peng, Mingyang Su, You Zhou, Xuefei Zhe, Naoya Iwamoto, Bo Zheng, and Michael Black. Emage: Towards unified holistic co-speech gesture generation via expressive masked audio gesture modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 114--1154, 2024.
Shunlin Lu, Jingho Wang, Zeyu Lu, Ling-Hao Chen, Wenxun Dai, Junting Dong, Zhiyang Dou, Bo Dai, and Ruimao Zhang. Scamo: Exploring the scaling law in autoregressive motion generation model. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 27872- 27882, 2025.
Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5442-5451, 2019.
Ian Mason, Sebastian Starke, and Taku Komura. Real-time style modelling of human locomotion via feature-wise transformations and local motion phases. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 5(1):1-18, 2022.
Mare Benedí San Millán, Angela Dai, and Matthias Nießner. Animating the uncaptured: Humanoid mesh animation with video diffusion models. arXiv preprint arXiv:2503. 15996, 2025.
Multi-Linguality
Multi- Multi-Granullrity.. M3-embedding: Multi- lingualingg,,ity, multi- functionality, multi-granuuarity text embeddings through self-knowledge distillation, 2024.
Priyanka Patel and Michael J. Black. CameraHMR: Aligning people with perspective. In International Conference on 3D Vision (3DV), 2025.
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani. Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 10975-10985, 2019.
Mathis Petrovich, Michael J Black, and Güil Varol. Tmr: Text-to-motion retrieval using contrastive 3d human motion synthesis. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 9488-9497, 2023.
Matthias Plappert, Christian Mandery, and Tamim Asfour, The KIT motion-language dataset. Big Data, 2016.
14

---

Published as a conference paper at ICLR 2026
Abhinanda R Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez,, and Michael J Black. Babel: Bodies, action and behavior with english labels. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 722-731, 2021.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-00 67, 2020.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models, 2021.
Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. In The Eleventh International Conference on Learning Represen- tations, 2023. URL https: wpwpeeveew.eeepe////pp/e. net/ / forum 2iddssiksss222 jwu.
Hugo Touvron, Thibaut Lavril, Gautier Izacard. Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv: /3971, 2023.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng. Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng. Ningyi Zhang, Pandeng Li. Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang. Wei Wang. Wenmeng Zhou, Wente Wang. Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang. Yingya Zhang, Yitong Huang. Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong. Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large- scale video generative models. arXiv preprint ar arxiv:2503 203314, 2025.
Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.200
2025.
Haoru Wang, Wentao Zhu, Luyi Miao, Yishu Xu, Feng Gao, Qi Tian, and Yizhou Wang. Aligning human motion generation with human perceptions. arXiv preprint arXiv:2407.02272 2024a.
Qiuheng Wang, Yukai Shi, Jiarong Ou, Rui Chen, Ke Lin, Jiahao Wang, Boyuan Jiang, Haotian Yang, Mingwu Zheng, Xin Tao, Fei Yang, Pengfei Wan, and Di Zhang. Koala-36m: A large-scale video dataset improving consistency between fine-grained grained conditions and video content, 2024b. URL https://arrxie org/ wtgg/abs//// 2410 0 08260.
Ye Wang. Sipeng Zheng, Bin Cao, Qianshan Wei, Qin Jin, and Zongqing Lu. Quo vadis, motion generation? from large language models to large motion models. arXiv preprint arXiv:2410.03311, 2024c.
Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, et al. Inter-x: Towards versatile In CVPR, pp. 12260-22271 2024.
interaction analysis.
Wanqi Yin, Zhongang Cai, Ruisi Wang, Ailing Zeng, Chen Wei, Qingping Sun, Haiyi Mei, Yanjun Wang, Hui En Pang, Mingyuan Zhang, Lei Zhang, Chen Change Loy, Atsushi Yamashita, Lei Yang, and Ziwei Liu. Smplest-x: Ultimate scaling for expressive human pose and shape estimation. arXiv preprint arXiv:2501.09782 2025.
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2m-gpt: Generating human motion from textual descriptions with discrete representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023a.
15

---

Published as a conference paper at ICLR 2026
Juze Zhang, Haimin Luo, Hongdi Yang, Xinru Xu, Qianyang Wu, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Neuraldome: A neural modeling pipeline on multi-view human-object interactions. In CVPR, 2023b.
Juze Zhang, Jingyan Zhang, Zining Song, Zhanhe Shi, Chengfeng Zhao, Ye Shi, Jingyi Yu, Lan Xu, and Jingya Wang. Hoi-m3: Capture multiple humans and objects interaction within contextual environment. arXiv preprint SKKi:22404002299 2024a.
Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong. Huirong Li, Lei Yang. and Ziwei Liu. Remodiffuse: Retrieval-augmente motion diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 364-373, 2023c.
Mingyuan Zhang, Huirong Li, Zhongang Cai, Jiawei Ren, Lei Yang, and Ziwei Liu. Finemogen: Fine-grained spatio-temporal motion generation and editing. In Advances in Neural Information Processing Systems, volume 36, pp. 13981-13992, 2023d.
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. IEEE transactions on pattern analysis and machine intelligence, 46(6):4115 4128, 2024b.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma. Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713, 2024c.
Kaifeng Zhao, Gen Li, and Siyu Tang. DanConnrool: A diffusion-based autoregressive motion model for real-time text-driven motion control. In The Thirteenth International Conference Learning Representations (ICLR), 2025.
16

---

Published as a conference paper at ICLR 2026
A AI USAGE DECLARATION
Artificial intelligence tools (e.g., ChatGPT) were utilized solely for the purposes of grammar checking and language refinement. No content was generated that contributed the intellectual or analytical aspects of this work.
B RELATED WORK
B.1 MOTION DATASET
Optical MoCap Dataset. Pioneering works in texi-to-motion generation relied almost exclusively on optical MoCap datasets. These resources, exemplified by KIT-ML (Plappert et al.. 2016). AMASS (Mahmood et al., 2019). BABEL (Punnakkal et al., 202 HumanMLL3 (Guo et al., 2022a), aioz (Le et al., 2023), etc, were instrumental in advancing the field. Optical motion capture provides precise. low-noise joint position and rotation data, resulting in motions that are physically plausible and temporally coherent. This precision is critical for modeling the complex dynamies of the human body, such as maintaining balance and ground contact. The controlled, studio-based collection environment also ensures that the data is meticulously clean and free from common artifacts like jitter or body-part penetration. Despite these strengths, these datasets orders of magnitude smaller than those in other modalities. HumanML3D, for example, contains approximately 14,000 clips. This scarcity is a direct consequence of the labor-intensive and expensive collection process. The controlled environment and small scale also lead to a severely limited semantic coverage, failing to capture the broad distribution of human movements. This limitation means that models trained solely these datasets tend to overfit to small set of indoor, locomotion-heast -havy actions, thereby losing the ability to generalize to novel instructions. This bottleneck has been primary driver for the field's search for alternative data sources.
Visual MoCap Datasets. To break free from the MoCap bottleneck, the community has turned to large-scale, video-based data. These "in-the-wild" datasets leverage advancements in visual motion capture (visual MoCap) to extract motion from public web videos. Motion-X (Lin et al., 2023) was an early step in this direction, compiling approximately 100,000 sequences, but it were still considered limited in scale and primarily focused on specific scenarios such as sports and gaming. The MotionMillion (Fan et al., 2025) dataset represents the pinnacle of this approach, with over 2 million high-quality motion sequences and 2,000 hours of content, making 20 times larger than existing resources. While immensely valuable, this approach introduces new set of challenges. The motion data is fundamentally derived from the visual mocap method, which has lower fidelity manifests kinematic artifacts and jitter. This transition reveals critical shift: the problem of data scarcity has been replaced by the problem of data quality heterogeneity. New methods must be developed not just to collect data, but effectively handle and learn from data of varying fidelity.
B.2 TEXT--TOMMOTOO GENERATION MODEL.
Conventional Motion Generation Models. The current state of the art has been overwhelmingly driven by two paradigms: diffusion and autoregressive models. Diffusion models like MDM (Tevet al., 2023), MotionDiffuse (Zhang et al., 2024b), and MotionLCM (Dai et al., 2024) have become the dominant paradigm for generating high-fidelity motion. They are praised for their ability to synthesize natural motions by denoising random signal over time. More recently, ReMoDiffuse (Zhang et al., 2023c) proposed models on diverse motions. However, unlike models that leverage knowledge from other modalities like video generation, ReMoDiffuse's ' approach is based on retrieving and refining from a database of existing motion samples. Autoregressive approaches like T2M-GPT (Zhang et al., 2023a) and ScaMo (Lu et al., 2025) model motion as sequence of discrete tokens. This approach, akin to Large Language Models (LLMs), is inherently scalable and well-suited for modeling long, compositional sequences. However, critical limitation shared by these models is their reliance constrained datasets. While they achieve impressive results on benchmarks like HumanML33D, their generalization to -fo-oi- instructions remains challenge.
diffusion model to address the unsatisfactory performance of
Bridging Motion and Video Generation Priors. A new class of models is emerging to address this limitation by leveraging the vast semantic knowledge encoded in large-scale video generation
17

---

Published as a conference paper at ICLR 2026
models. Early works (Millán al., 2025; Albaba et al., 2025) attempted to leverage knowledge from large-scale video generation models by animating images and then extracting motion through optimization- -aase methods. The key critique of these methods is their heavy dependence generation model performance, which often lacks robust motion quality; their neglect of existing motion datasets that provide valuable human motion priors and patterns; and high computational costs and slow inference speeds.
vidco
C MBENCH DETAILS
C.I HUMAN PREFERENCE ANALYSIS
UR. 93: ret
ondioninttee description attooaa
Aperion doing
niht match descristonaaiiee
WWhchhotdee action matchee the toot description abiow better?
The munchee
maanightmantee batter
Figure Human Preference Annotation Interface. Two rendered videos side-by-side with annotator choices.
To validate MBench 's alignment with human perception across the nine evaluation dimensions, conducted large-scale human preference labeling on rendered motion videos. The annotators range in age from 20 to 35 and are equipped with fundamental domain knowledge relevant to the task.
C.1.1 DATA PREPARATION
For each text prompt Pi. generate videos using five text-to-motion models M {M1, .2,, Ma, M4, Ma). producing outputs G; (Vi,1. Vi,,, V1,,, V4,4, V.s).. We construct all 10 unique pairs per prompt.
pairwise combinations within each group, yielding
Different evaluation protocols target specific metrics. For Motion-Condition Consistency and Motion Generalizability. annotators perform pairwise comparisons to determine which video better satisfies the text prompt. For physical and perceptual quality metrics (Jitter Degree, Ground Penetration, etc.), annotators rate each video individually using 3-point Likert scale (0-2) with natural language descriptions. Each video appears 5 times during evaluation with randomized presentation order to minimize bias.
Across N prompts. this yields N 10 pairwise comparisons. Aggregated results quantify the correlation between MBench automatic metrics and human judgments, enabling rigorous assessment of metric reliability and model performance.
C.1.2 VALIDATING HUMAN ALIGNMENT OF MBENCH
Given the human annotation results, compute the win ratio for each model. In each pairwise comparison, a model's video is judged as better, the model receives a score of 1 while the other receives 0. In the case of tie, both models assigned score of 0.5.
Per-Dimension Evaluation For each evaluation dimension, we compute the win ratio using two sources: (1) scores derived from MBench 's automatic evaluation metrics, and (2) human annotation
18

---

Published as a conference paper at ICLR 2026
Foot Flootiig
Foot Sliding
Foot Penetration
0.7
0.3000
0.7
0.5000
0.7
0.9000
0.6
0.6
0.5
0.5
0.5
0.5
0.4
0.4
0.4
0.3
3.3
0.4
0.5
0.6
0.7
0.3
0.4
0.5
0.6
0.7
0.3
0.4
0.5
0.6
MBenchh Svaluation Win Rato
0.7
0.9000
Dynamic Degree
0.7
0,0000
Jitter Degree
0.7
Body Penetration
1,0000
0.6
0.6
0.5
0.5
0.5
0.5
0.4
0.4
0.4
0.3
0.4
044
0.4
Pose Quality
Motion Condition Consistency
Motion Generalizability
0.7
0.8000
0.8
1,0000
0.7
1,000
0.7
0.8
0.5
0.5
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.3
0.4
0.7
0.4
Human Preference Win Ratio
HDM
Mosien.CM
T2N--PT
Mccaask
Vi4:Gen(Durs)
Figure 6: MBench's Human Alignment. In each plot, a dot represents the human preference win ratio (horizontal axis) and MBench automatic evaluation win ratio (vertical axis) for motion generation model. We linearly fit straight line to visualize the correlation and calculate the Spearman's correlation coefficient (p) for each dimension.
results. We then calculate the correlation between these two sets of win ratios. As illustrated in Fig. 6, the scatter plots show strong positive correlations between human-pree prefennneeeeeeeeeee win ratios and MBench per-dimension evaluation win ratios, indicating the reliability of the proposed automatic metrics. Notably, the Foot Floating dimension has a lower correlation. This because human generally have limited sensitivity to subtle foot-floating artifacts, causing their preference scores to cluster around 0.5. contrast, the physics-base -baee metric in our benchmark can precisely distinguish these differences. We acknowledge the slight difference in calculating the MBench evaluation win ratio for Motion-Conditee Condition Consistency. As mentioned Sec. 4.1, these two dimensions assessed through multiple-choice tasks, where the VLM first describes the motion and then selects the most matching text description from 10 carefully selected text options. The selection is correct, the sample scores; otherwise it doesn't. This results in binary correctness rather than continuous [0, I] scores. Consequently, when computing MBench evaluation win ratios, two motion generation methods that both receive either for sample result in tie, which reduces the variation in their MBench win ratio distribution.
Motion Generalizabillity and Motion
Usage of Single-Vides Video Evaluations for Disambiguation. In addition to pairwise comparisons, annotators also provide single-video ratings for each video from each method. These individual scores are used to mitigate potential biases or unfairness in pairwise annotations. Specifically, in cases where the single-video scores clearly indicate that one video is superior but the corresponding pairwise annotation contradicts this assessment, the decision is revised in favour of the single-video judgment.
C.2 PROMPT SUITE PER MBENCH EVALUATION DIMENSION
The prompts used in our benchmark are carefully curated to balance evaluation efficiency with representativeness. We aim to avoid excessive number of prompts that could lead to long evaluation time. While we still need to ensure the suite is comprehensive enough to cover diverse motion types and possesses the discriminative power to reveal differences among evaluation dimensions. In total, we designed a suite of 450 distinct prompts, each tailored to specific MBench dimension:
150 prompts for Temporal Quality(Motion Quality)
100 prompts for Frame-Wise Quality(Motion Quality)
19

---

Published as a conference paper at ICLR 2026
100 prompts for Motion-Condition Consistency
100 prompts for Motion Generalizability
C.2.1 MOTION QUALITY PROMPTS
For the Temporal and Frame- Wise Quality dimensions, the primary goal is to assess the intrinsic quality of the generated motion, rather than the model's ability to simply produce an output. Thus, these 250 prompts were sourced from established benchmarks, such as HumanML33 (Guo et al., 2022a) and AMASS (Mahmood et al., 2019), as well as various open-source motion libraries. The final test set was manually curated to specifically include challenging scenarios where current models are known to fail, such as those prone to artifacts like foot-skating, ensuring a rigorous evaluation of motion fidelity.
C.2.2 MOTION-CONDITION CONSISTENCY PROMPTS
The prompts for Motion-Condition Consistency is designed to probe robustness of motion generation models. The prompts intentionally also include common, everyday actions that are typically under- represented in traditional motion generation datasets, testing the model's ability generalize beyond its training data.
The creation process for this prompt suite was as follows:
Data Sourcing: We drew from diverse range of actions found in large-scale video generation datasets, including LLaVA-178K (Zhang al., 2024c) and an extensive internal dataset containing millions of video-text pairs.
Automated Extraction:
The DeepSeek-RI (Guo al., 2025) model was utilized to parse and extract motion-specific descriptions from the video annotations and text prompts.
Semantic Embedding and Clustering: After removing duplicates, we mapped the textual descriptions into shared embedding space using several encoders, including T5 (Raffel et al., 2020), CLIP (Radford et al., 2021), and BGE-M3 (Multi-Granularity, 2024). BGE-M3 was ultimately selected for its superior semantic clustering performance. We then applied K-means clustering to the resulting embeddings, generating 1000 distinct prompt clusters.
Final Curation:
The final set of prompts was selected through a hybrid approach, combining automated filtering with DeepSeek-R and expert curation to ensure relevance, diversity, and difficulty.
C.2.3 MOTION GENERALIZABILITT
The prompts are designed to evaluate a model's generalization capacity by requiring it to handle motions described by an open-world vocabulary, rather than just the limited terms found in standard datasets. A single action, such as walking, can be expressed with numerous linguistic nuances that convey subtleties in emotion, speed, and intent. A good generalized model can translate subtle semantic differences into distinct kinematic profiles, proving it possesses genuine understanding of the relationship between language and motion.
For prompt creation, our methodology involved the extraction of near- mear-synonyms from the Oxford English Dictionary. We then manually composed descriptive phrases that fully articulate the meaning and nuances of particular motion. This approach creates a challenging set of prompts specifically designed to push the boundaries of the motion generation model's generalizability.
C.3 NORMALIZATION FOR RADAR CHART VISUALIZATION
For the radar charts in Main Paper Fig. 1, we apply normalization to enable fair and intuitive comparison of relative performance. Specifically, for each metric, we first scale all values by dividing them by the maximum value observed across models. For those metrics where smaller values indicate better performance, we then apply a positive transformation by taking the reciprocal of the scaled values. Finally, we normalize all metrics so that the best score among the models is mapped to 1.0 and the worst score is mapped 0.2. The radar chart axes have range of 0.0 to 1.0.
20

---

Published as a conference paper at ICLR 2026
D MMMOGEN-228K DATAILS
D.1 VIMOGEN-2288 VISUALIZAAIOON
We provide some examples our ViMoGen-228k in Fig. including optical Mocap data, in-the-wild video data, and synthetic video data.
BETW
BETY
BETW
(a) Optical Mocap Data
(b) In the Wild Data
Synthetic Video Data
Figure 7: Visualization of our ViMoGen-2228K dataset. (a) High-fidelity Optical MoCap data. (b) Diverse In-the-Wild Video Data. (c) Precisely controlled Synthetic Video Data.
D.2 VIMOGEN-228B COLLECTION, FILTERING, AND ANNOTATION
This section provides detailed breakdown of the curation process for the ViMoGen-228K dataset, covering the collection, filtering, and annotation pipelines for each of our three primary data sources.
D.2.1 OPTICAL MOCAP DATA
Collection and Aggregation. Our high-fidelity motion data is built upon a large-scale aggregation of 30 publicly available optical motion capture datasets. The initial pool of datasets includes: Hu- manML33 (Guo et al., 2022a), 100style (Mason et al., 2022), AIOZ-GDANCE (Le et al., 2023), AIST+ (Li et al., 2021), ARCTIC (Fan al., 2023), BEAT (Liu et al., 2024), BEHAVE (Bhatnagar et al., 2022), Chairs (Jiang et al., 2023), CHI3D (Fieraru et al., 2025), ChoreoMaster (Chen et al., 2021), CIRCLE (Araújo al., 2023), CMU (Tevet et al., 2023), EGO-BODY (Grauman al., 2024), EMDB (Kaufmann et al., 2023), FineDance (Li et al., 2023c), FIT3D (Fieraru et al., 2021b), HOI-M3 (Zhang et al., 2024a), HumanSC3D (Fieraru al., 2021a), IDEA400 (Lin al., 2023), Inter- human (Liang et al., 2024b), InterX (Xu et al., 2024), KIT-MI (Plappert et al., 2016), LAFANI (Har- vey et al., 2020), Mixamo, MoTORICA (Alexanderson et al., 2023), NeuralDome (Zhang al., 2023b), OMOMO (Li et al., 2023a), PhantomDance (Li et al., 2022), RICH (Huang et al., 2022), and TRU-MANS (Jiang al., 2024). All motion sequences were standardized the SMPL-X (Pavlakos et al., 2019) format and resampled to consistent 20 fps.
Filtering and Selection. To ensure high quality, we applied rigorous multi-stage filtering pipeline. First, all long sequences were segmented into overlaaalg 5-second clips (100 frames). We then filtered out any clips shorter than seconds and removed static T-pose frames. Subsequently, we used our MBench evaluation suite programmatically assess motion quality, removing clips that exhibited significant jitter or lacked sufficient dynamics. Our dataset selection was an empirical, iterative process designed to maximize performance on downstream tasks. We began by establishing baseline using the HLmanML3D dataset, chosen for its high-quality, human numan-annotated text and its widespread use in the community. We then systematically evaluated the remaining 29 candidate datasets. For each candidate, we trained a text-to motion model on a corpus combining the HumanML3D dataset with the candidate dataset. The performance of this model then evaluated MBench
21

---

Published as a conference paper at ICLR 2026
benchmark. This data-driven approach allowed us to quantify the contribution of each dataset. We observed that several datasets, particularly those with high-speed or stylistic dance motions and less semantic descriptive text, degraded overall model performance and text-motion consistency on MBench. This led us to exclude them from the final training set. Through this comprehensive curation process, selected the 16 additional datasets that demonstrated a positive impact, resulting in a final corpus of 17 high-quality datasets (including the HumanML3D base) comprising 172k clips. The selected datasets are: 100style, ARCTIC, BEHAVE, Chairs, CIRCLE, EMDB, FTT3D, HumanML3D, HumanSC3D, IDEA400, InterX, KIT-ML, LAFAN1, Mixamo, OMOMO, RICH, and TRU-MANS.
D.2.2 IN-THE-WII WILD VIDEO DATA
Collection. To capture a diverse range of real-world motions, we sourced data from an internal mega-scale video database, which includes large public datasets such as Koala36M (Wang et al., 2024b), PANDA-36M (Chen et al., 2024), MIRA (Ju et al., 2024), and Ego-Exo (Grauman et al., 2024). This resulted in an initial pool of approximately 60 million video clips.
Filtering. The raw videos underwent a multi-stage filtering process. We first applied quality assessment filters to remove clips with low resolution, poor lighting, excessive motion blur, or heavy occlusion. This step yielded more refined set of around 10 million clips suitable for further analysis. We then perform second, more granular filtering pass. This process focused on selecting clips where at least 80% of the human skeleton is visible. This was supplemented by keyword searches for videos across categories like sports, daily activities, and cultural performances. While the current pipeline applies strict filtering to ensure the highest fidelity for model training and to balance between different data sources(Optical Mocap Data, In-the-Will Video Data, Synthetic Video Data), we acknowledge the massive volume of discarded data. With the development of advanced MoCap algorithms (such as SAM 3D), we believe that the "imperfect" data can be used more efficiently in the future.
Scare
Course Arnet
Coarse Arnot
Coans Arnot
fame Annol
Twining Fee Generation
Video
Config
Corfi
Corfo
Corlig
Filor
Data Proprosen Pipelre
Detection
Traiiig
fiow
Quality
Carreaa Masion
Kayplits
Scene Cut
Sconer Num Furson
Scaver Scaee Move
Scorer Nuri Test
Langet Filst
Detabane
Scores Copeiio
Optical Flow Lineart
Camera Movemant
Figure 8: Comprehensive Data Preprocess Pipeline. The pipeline is organized into sequential modules, utilizing a central database to aggregate metadata like scores, keypoints, captions, and text masks.
22

---

Published as a conference paper at ICLR 2026
70020000
61947286
90030000
40000000
34184107
20020000
0492167
120350
118542
41971
poot (NarASRM, PANDA- 36M, NIAA, Ego Bio.)
Rarrove too-quallls wassts [Bu lightigging!
Bemowe excassiN shary heny
Eody
NoCup 12arigg
Motton quality quaaityttarrgggg
Figure 9: Quantitative Results of the Filtering Process.
D.2.3 SYNTHETIC VIDEO DATA
Prompt Construction. To systematically expand the semantic coverage of our dataset, leveraged the generative power the state-of-the-an text-to-video model, Wan2. (Wan et al., 2025). We constructed long-tail vocabulary by analyzing large-scale video caption datasets, including Open- Human Vid (Li et al., 2024) and -LaVA-Video- 178K (Zhang et al., 2024c). From this vocabulary, compiled list of action verbs and descriptive nouns, which underwent semantic clustering and deduplication to produce over 20,000 unique prompts.
Generation and Refinement. These prompts were used with the Wan2.1 model generate 81-frame video clips at 16 fps. We explicitly instructed the model to generate videos conducive to visual MoCap, such as those with stable camera, a single full-body person, and clean background. The generated videos and their extracted motions were passed through our standard filtering pipeline. We also interpolated the motion sequences from 16 fps motion datasets. Crucially, the motions were further refined using our pre-trained ViMoGen Motion- to-Motion (M2M) branch to enhance fidelity, especially the world-space global translation that usually estimated poorly by visual MoCap models. This process yielded final set of 14,000 high-quality synthetic samples that fill critical semantic gaps in existing real-workd datasets.
fps linearly for aligmment with common
D.2.4 DATA ANNOTATION
Motion Annotation. For all human video data (in-the-wild and synthetic data), we employed a two-stage pipeline. We first used the YOLOVS tracking model (Jocher al., 2023) to obtain human bounding boxes and then applied a state-of-the-arr human mesh recovery model, CameraHMR (Patel Black, 2025), extract 3D motion in the SMPLX representation. Following DART (Zhao al., 2025), we canonicalize the global orientation of each motion so that the initial frame faces the y+ axis. To mitigate common visual MoCap artifacts like jitter and foot-sliding, we applied post- post-processing smoothness algorithms based on temporal Gaussian smoothing.
Text Annotation. For MoCap datasets with existing high-quality annotations, such as H3manML.3D, we used the provided text. For MoCap clips that lacked annotations or only had class labels, we generated descriptive text using multi-modal LLM. We first rendered depth videos the human mesh and provided these video frames to the Gemini 2.0 Flash model. For the in-the-wild videos, the original RGB clips were used as input. Our annotation process was guided by the detailed system prompt shown in Box which engineered to elicit structured and comprehensive motion descriptions. All LLM-generater annotations were heuristically filtered to remove invalid responses such "I can't 't help with that" "'The speech content is:
23

---

Published as a conference paper at ICLR 2026
D.2.5 SYSTEM PROMPT FOR MOTION DESCRIPTION ANNOTATION
We utilize Gemini 2.0 Flash model to generate structured text annotations for our motion data. The model was provided with sequence of rendered frames and the system prompt shown in Box 1. This prompt was carefully engineered to elicit detailed, multi-faceted descriptions covering holistic summaries, motion styles, and fine-grained temporal dynamics.
Box 1: System Prompt for Gemini 2.0 Motion Description Annotation
You are an expert in human motion analysis and biomechanics. Your task is to provide a detailed, structured annotation for human motion sequence. You will provided with a series of frames from rendered video showing 3D human motion. Analyze the sequence carefully and generate a description following the exact 7-part structure below. Be precise, objective, and use clear, descriptive language.
1. Detailed description of whole human motion sequence:
Provide a comprchensive, narrative summary of the entire motion from start to finish. Describe the overall action, the flow of movement, and the purpose of the action if is apparent.
2. Styles, emotions of human motion sequence:
List 2-3 descriptive adjectives that capture the qualitative aspects of the motion. Examples include: energetic, fluid, tense, relaxed, deliberate, clumsy, focused.
3. Description of the global trajectory and interaction with the environment: Describe the overall path of the person's s center of mass. Specify if the person is stationary (in-place) or moving through the environment. Note any interactions with the ground (e.g., foot contact, sliding) or implied objects.
4. Key actions temporally:
Provide concise, chronologically ordered list of the main actions or phases in the sequence. Use short verb phrases. For example: "Start standing, raise right leg, kick forward, return to standing."
5. Action description every 2 frames:
Provide fine-grained, haaee-b--frane breakdown of the motion. For each 2-frame interval, describe the specific changes in body posture and limb positions.
Frame to [Description]
Frame 3 to 4: [Description]
...and so on for the entire sequence.
6. Action description every 4 frames: Provide slightly more -gaaine breakdown of the motion. For each 4-frame interval, summarize the primary action or transition occurring within that window.
Frame to [Description]
Frame 5 to 8: [Description]
...and so on for the entire sequence.
7. Key motion status with corresponding essential body part movements: Identify the most critical poses or phases of the motion (e.g.. the peak of jump, the moment of impact). For each key status, describe the configuration and movement of the essential body parts involved (e.g.. torso, limbs, head).
Following the system prompt in Box 1, the model generates structured annotation for each motion sequence. Box provides representative example for the motion of a person performing a drag curl. In our model training, only use the information from Part 1.
Box 2: Example of Generated Motion Description
1. Detailed description of whole human motion sequence: The person starts semi-crouched position, with knees slightly bent and arms
24

---

Published as a conference paper at ICLR 2026
close to the body. As the sequence progresses, the person gradually lowers their torso, moving from an upright position to a deeper squat, flexing at the hips and knees. The elbows remain bent, with the hands moving downward in controlled motion, as holding an object. The rhythm is steady and deliberate, emphasizing the smooth transition from high to low position, reflecting the controlled nature of the drag curl movement.
2. Styles, emotions of human motion sequence: Focused, deliberate.
3. Description of the global trajectory and interaction with the environment: The person stands in place, gradually lowering their torso and arms while maintaining balance on the spot.
4. Key actions temporally: Stand semi-crouched, lower torso and arms, deepen squat, maintain position.
5. Action description every 2 frames:
Frame 1 to 2: Bending slightly more the knees, hands move downward.
Frame 3 to 4: Torso lowers further, arms start to extend slightly downward.
Frame 5 to Knees continue to bend, torso moves forward and downward.
Frame 7 to 8: Torso reaches lower position, hands nearing midpoint of descent.
Frame to 10: Full squat position with arms fully extended downward.
Frame 11 to 12: Torso remains low, slight adjustment in arm position.
Frame 13 to 14: Maintaining position, minor adjustments to balance.
6. Action description every 4 frames:
Frame 1 to 4: A gradual increase in knee flexion, causing the torso to lower and extend arms downward.
Frame 5 to 8: Transition into deeper squat with arms moving in sync, maintain- ing alignment and halance.
Frame 9 to 12: Stabilization in the squatted position, ensuring balanced posture and extended arms.
7. Key motion status with corresponding essential body part movements: The person transitions smoothly from a standing position into a deep squat, charac- terized by a gradual lowering of the torso and extending arms downward. The upper body leans slightly forward to maintain balance, while the knees and hips are flexed deliberately, emphasizing stability and control the motion sequence.
E VIMOGEN IMPLEMENTATION DETAILS
Our ViMoGen model builds upon the --B-parameter Wan2.1 (Wang et al., 2025) text-to-video foundation model. Motion sequences are represented using the 276-dimensional SMPL-based vector from DartControl (Zhao et al., 2025).
Training Configuration. The model initializes with Wan2.1 weights and trains on 8 H800 GPUs using AdamW optimization (Ir=0.0002, batch size=128) with FSDP for memory efficiency. Training ViMoGen on our MotionAtlas dataset requires 40,000 iterations (1.5 days), while ViMoGen-light completes in approximately one day.
Adaptive Training Strategy, To maximize the utility of heterogeneous data sources, we implement a data-aware training protocol. Synthetic data receives double weighting to compensate for potential quality variations, while visual mocap data supervision focuses only on local body poses to mitigate global motion artifacts. For branch selection during training, adjust probabilities based data quality: HumanML3D (Guo et al., 2022a) samples use 80% text-to-motion and 20% motion- ton--ttoooooon generation, while other data uses balanced 40%/60% probabilities.
25

---

Published as a conference paper at ICLR 2026
R Precision!
Methods
FID,
MultiModal Dist.
MultiModality
Top
Top
Top 3
TM2T (Guo et al., 2022c)
0.424 4x0
0.618 o00
0.7291..om
1.501 1.501 1.7
3.467--011 011
2.424 Low
al., 2022b)
0.455 2000---000
0.636 008
0.736*-c02 000
1.087
3.347
2.219
MDM (Tevet al., 2023)
0,320
0.498
0.611 t:007
0.544
5.566 -027
2.799
MilionDiifuuee (Zhang et al., 2024b)
0,4911 =000
0.681 001
0.782
0.630 co1
3.113
1.55.3
T2M-GPT (Zhang al., 2023a)
0,492 0.492
0.679 002
7775*.002
0.141
3.121
1.831
MoMask (Guo et al..
0.521 =
0.713
0,807
0.045
2,958 008
1,211
Motion-LCM (Dai et al,, 2024)
0.502
0.698
0.798+.002
0.304
3.012
2.259
MLD (Chen et al., 2023)
0,481 000
0.673 000
0.772 con
0,473
3,196
2.413+077 078
MLDD ViMoGen-light (Ours)
0.542
0.733
0,825 0225*.002
0.1 0.114
2.826 0.007
1.973
Table 8: Quantitative evaluation on the HumanMI.31 test set. + indicates a 95% confidence interval during 20 times repeating evaluations, Bold indicates the best result.
Noise Simulation Parameters. To simulate realistic visual MoCap errors for the M2M branch during training, we apply the following controlled noise parameters to the ground truth motion:
Gaussian Corruption: Applied with probability range of [0.0, 0.4] and a scale range of
(0.05, 0.15).
Jitter Simulation: We simulate temporal jitter by blending neighboring frames with probability of [0.03, 0.08] and a jitter strength of 0.3.
Temporal Dropout: Small temporal spans are masked with a rate 0.02 to mimic tracking loss.
Global translation is strictly masked for the M2M branch to eliminate domain gaps caused by trajectory estimation errors in monocular MoCap.
F TEXT-TO-MOTION EXPERIMENT ON HUMANML3D BENCHMARK
To further validate the effectiveness and generalizability of our proposed model architecture, we con- duct supplementary experiment the widely-used HlmanMIL3D benchmark (Guo al., 2022a). This experiment isolates our core architectural contribution from our large-scale ViMoGen-22285 dataset, demonstrating its strong performance in a traditional training setting. For this purpose, we integrate our network architecture into the established Motion Latent Diffusion (MLD) frame- work (Chen al., 2023).
F.1 EXPERIMENTAL SETTINGS
Dataset. We conduct this experiment on the HumanMLLD (Guo et al., 2022a) dataset, a standard benchmark for text-to-motion generation. contains 14,616 human motion sequences, paired with 44,970 descriptive text annotations. Following standard practice (Guo et al., 2022b; Dai et al., 2024), we utilize the same redundant motion representation, which includes root velocity and height, local joint positions, velocities, rotations relative to the root, and binary foot contact labels.
Evaluation Metrics. To ensure fair and comprehensive comparison with prior work, we adopt standard suite of evaluation metrics established by previous methods (Guo et al., 2022b; Chen et al., 2023). These metrics assess the generated motions from three key perspectives:
Motion Quality and Diversity: the distributional similarity between generated and real motions. We also report Diversity and MultiModality (MModality) to evaluate the variety of motions generated from different texts and the same text, respectively.
We use the Frechet Inception Distance (FID) to measure
Text-Motion Consistency: We evaluate the semantic alignment between the generated motion and the input text using R-Precision (Top-1. Top-2, Top-3), which measures motion retrieval accuracy, and Multimodal Distance (MM Dist), which calculates the average feature distance between text and motion pairs.
26

---

Published as a conference paper at ICLR 2026
Implementation Details. Our implementation is built upon the reproduced version of MLD (Chen et al., 2023) from the official PyTorch codebase of MotionLCMM (Dai et al., 2024). The core modification is the replacement of the original diffusion denoiser with our proposed text-to-motion network architecture. We leverage the pre-trained motion VAE from MotionI.CS to operate within the same latent space, ensuring fair comparison of the denoising network's capabilities.
To maintain consistency with the baseline, we retain most of the original hyperparameters from the code repository, including the choice of text encoder, the number of diffusion timesteps, and the Classifier-Free Guidance (CFG) scale used during inference. To optimize for our hardware and accelerate convergence, we tuned the training configuration by using larger batch size and correspondingly adjusted learning rate. The model was trained using the Adam W optimizer with cosine learning rate scheduler for 36000 iterations.
F.2 RESULTS AND ANALYSIS
We compare method, ViMoGen-light against range of state-of- the-art models the Hu- manML3D test set. The quantitative results are presented Table The experiment was repeated 20 times report the mean and 95% confidence interval, following standard evaluation protocol (Guo et al., 2022b; 2024).
As shown in the table, integrating our architecture into the MLD framework leads the art in text-motion consistency. Our model, MLD + ViMoGen-light, surpasses all prior methods on every text-alignment metric, achieving the best R-Precision (Top-1, Top-2, Top-3) and the lowest Multimodal Distance. We attribute this significant leap in performance to our full-transformer denoiser, which enables a more nuanced and effective fusion of text and motion features compared to the original MLD architecture, thereby improving the interpretation of complex prompts. This superior text-motion alignment is further demonstrated in the qualitative examples presented in Fig, 10.
state of
Furthermore, this substantial gain in semantic accuracy is achieved without sacrificing motion quality, Our model attains a highly competitive FID of O. 0.114, marking a dramatic improvement over the MLD baseline (0.473) and placing on par with other top-performing methods like T2M-GPT. While maintaining strong MultiModality score, these results validate that architectural contributions are robust and independently effective. The experiment confirms that our approach is not only powerful when combined with our large-scale OmniMotion dataset but also serves potent, generalizable component that can advance existing frameworks on established benchmarks.
G ADDITIONAL QUALITATIVE RESULTS
In this section, we provide additional qualitative results to complement the analysis presented the main paper. Fig. 11 visualizes our adaptive branch selection mechanism action, showcasing how ViMoGen intelligently switches between its M2M and T2M branches. Fig. 12 illustrates the impact of using descriptive video-style versus concise motion-style text for training and inference. Finally, Fig. 13 presents further side-by-side comparisons with state-of-the-as methods on MBench prompts, highlighting the superior semantic fidelity of our approach.
27

---

Published as a conference paper at ICLR 2026
MDM
MoMask
T2M-GPT
MotionLCM
ViMoGen-light
Prompt: "a person raises both amss towards their face and pushes one leg up out."
Promot: person squats the and squals again thighs parallel the floor and quickly
Promot: "a person stands in a wide stance and looks as if they are swinging golf club."
Figure 10: Qualitative comparison with sfate-of-the-an art methods on the Human.ML3D benchmark. For complex, multi-step prompts, our ViMoGen-light model generates motions that are more plausible and demonstrate superior text-motion alignment compared to prior works.
28

---

Published as a conference paper at ICLR 2026
ViGen Results
Mocap Baseline
ViMoGen-M2M
ViMoGen-T2M
they their arching anchingggaat twee iits amoo suived for Iedenno Mitheet, spiining sowllly hesbes handhg
aroa. As may contacto the retation Sher ostono steet ground, aboording the imporr with soft, conmalee landing mantaiiing perfect poice and stoedcy."
strole confidentty
sky. The able anpll explores dycumii kiwwer slifilly bann /amcc gripaing loosm fimmly the wind Ally mindiurree Aoarse back, chait ledy
svoty aquuring the sttting guats. The board giles smoothy across the water showcasing the windturfor's belance and fuld control"
Prompt Tau--ody asgled gnyy greasy probien, her fuvt bellared the buckidinn of bue
away,
Iedweow The angle thair the instinstieey, cwweling atlempp alwatty thersseluee Daegshe the wilutls, powseble pusling them doverwend They twit sigityy alr linaiin on thai side with ponto mudt anto splayed outfeardion the grass, abcoibing the impoctith resigned acccntance of fot."
Figure 11: Qualitative examples of our adaptive branch selection mechanism. This figure showcases how ViMoGen intelligently chooses between its Motion-to-Motien (M2M) and Text-to-Motion (T2M) branches based on the quality of the initial motion extracted from generated videos (Mocap Baseline). (Rows 1-2) For prompts where the ViGen model produces plausible motion sequence (e.g., "backffipp" "windsurfer"), the adaptive gate selects the M2M branch. This branch successfully refines the semantically correct but noisy Mocap Baseline, reducing jitter and improving physical realism. (Rows 3-4) For prompts involving sudden movements where the ViGen model fails and produces distorted or incomplete motions (e.g., "twist and throw". "stumble and fall'),, the Mocap Baseline unreliable. Here, the adaptive gate correctly falls back to the more robust T2M branch, which generates stable motion directly from the text prompt, ignoring the flawed video prior.
29

---

Published as a conference paper at ICLR 2026
Model Training Text
Testing Example1: swaggering into the room
Testing Example2: chopping wood
Video Style Prompt
Motion Style Prompt
Video Style Prompt
Motion Style Prompt
Video Style
Motion Style
Examplet:
Molion dyyle prompt "The persor ohiche contotently unto the room eacn stop oolberate and measored shoulders slppily took head held high Thik arme swing loosely of aidea, couding velased you acceetie enagy. sceale in weigh, they passae manomnialyy aurvpgnng the room aul-aoauee guas, continuing fowward
Video promet Fut body stote
whne woman
blrck chouss and strppl thts oiiches mestibeety
misimetn
packichop. Her atops are dellerrte and micoorrd abouldors aigndy bock nead how hight Hor ams awing looacty hor skies, prosicing relorod essortive energy. She easoes momontadyy, by wwitting hor woight subty 20 she survays tie spece with a sot assured pazo, ocforo continunng foneard with an unmutalioo bil of chonsmo and prosence."
Motion
porson
feet
shoulder with
with
powerful the mution, diving doommndd with expicote faree ganeestedd from
They
whoww that right attukkser thai tanao heiding allptily the post the beokawingg
their hege and come. the sems garking the iolede in precise, and fre striew the toeget bahrs.
video style prompt Full body snot stitch comera cidh view person in gray tracksue stanos with feot planted shoutfewwoth sport on clear oottoor space with a greasy lavms orepiep use with botk hands They swing the upward smooth somerfal d0ing above right abosidwe ther toro imaing slately All pask the oxsibotive
the belos.
Figure 12: Qualitative examples illustrating the impact of different text prompt styles used during training and inference. The rows represent the style of text used to train the model (video-style vs. motion-style), while the columns show the prompt style used for testing. We display results for two prompts: "swaggering into the room" and "chopping wood." The visualizations demonstrate that the model trained on descriptive video-style text (top row) is more robust and generalizes better, producing high-ge high-quality motions for both concise motion-style and rich video-style prompts at test time. Full prompt examples are provided below the images for clarity.
30

---

Published as a conference paper at ICLR 2026
MDM
MoMask
MotionLCM
ViMoGen
ViMoGen-light
Prompt perner alapss ficweed with purpross wach food lwoding limely syri with steady biel arms masinging uniion with the strick The postiow cerraits spright and disopincs, syes tocsed shood, the rhytiic cadencee the march ochoos with procsiion and unlty ambicdying strenghh and dedermmaatio..
Prompt "hhe person lowers thair body vito a deep crouch Roeping thei center of grevty close to the ground They propal Phomoot forward with sertss raped. stoet, chopoy ataps, thait fent baeety iiting off the foor. Thai emms are netr slghtty our from thei alsse maneen balenoo chiiig the humeed eiveement and their moso remoiss buschhed as moy move scrcos the space with an uggoit porprotle gal."
Angers sptayed
sightly: leeving
bnoresaion.
Additional qualitative comparisons with tfate-oftthe-ae methods on MBench prompts. Both ViMoGen and ViMoGen-light consistently generate motions that more faithfully adhere to the detailed text descriptions, showcasing their superior semantic understanding and generation quality compared to prior works.
Figure 13:
31


Reviews:
[{"review_id": "LtScw6w0GE", "reviewer": "Reviewer_gdGg", "summary": "The authors aim at bridging video generation and 3D human motion generation by proposing a new large-scale dataset ViMoGen-228k, a novel diffusion transformer ViMoGen for unifying the video priors and mocap priors, and a new benchmark MBench for evaluating motion generalizability, motion-condition consistency, and motion quality.", "questions": "Please refer to the weaknesses section for details. If the identified issues and questions are properly addressed, I would consider raising my score.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Contribution a large-scale motion datasets with 228,000 motion sequences, including high fidelity MoCap data and automatically annotated motion datas from web videos and synthesized videos, which is great for pushing forward the field of motion generation.\n\n1. Great efforts on trying to provide a robust and quantifiable evaluation benchmark for motion generalizability and motion-condition consistency. The proposed based on gated multimodal conditioning is proven to be effective under the proposed benchmark.", "weaknesses": "1. Poor presentation for qualitative results.  Take Fig.4 for example, for several samples, the SMPL frame shots are highly overlapped and the resolution are too low for the readers to compare the visual qualities, e.g. doing jumping jacks, martial arts, climbing a ladder. Similar issues can be observed in the qualitative results figures throughout the paper. Also, the axe or reference for temporal direction should be mentioned alongside.\n\n1. No demo videos are provided. Given the limitations of static frame shot figures, it’s crucial to provide convincing video demos of the generated samples and dataset samples to justify the qualitative comparison. However, the authors haven’t provided any video demos in supplementary materials. \n\n1.  In the proposed gating diffusion block, the T2Mbranch seems to be totally parallel with the M2Mbranch, performing totally different tasks with only one shared self-attention layer.  And the adaptive branch selection at inference relies heavily on alignment scores from external VLM to determine which branch to go with. It’s then questionable whether the two branch can really benefit from each other in learning the underlying distribution, or the performance is mainly gained from the external VLM assessment and selection for the task to proceed with.\n\n1. The training pipeline is not clear. The curriculum approach mentioned in line 204 - 207 is worth elaborated:\n    - “we simulate video motion references by perturbing ground-truth motions with controlled noise.” — What is the controlled noise mentioned here? And it’s very confusing to me how perturbing the ground-truth motions can simulate the video motion references. \n    - For training with text-motion pairs of high quality MoCap data,  would there still be a z_video for M2Mbranch? If so, how to ensure the alignment of the z_video and ground-truth high quality motion? The video generated from the same text may contain totally different motion from the ground-truth motion.\n\n1. The details of inference phase is not clear either:\n    - How to assess if the alignment scores from VLM is high or low? Through a scalar threshold?\n    - What are the differences and advantages of using the M2Mbranch, other than extracting the MoCap motion directly from the generated videos and performing motion refinement with a SOTA refinement method? Any empirical results on this?\n\n1. The benchmark metrics haven’t been fully evaluated whether it aligns with human perception or not. Although Appendix C.1.2 presents the alignment results for the temporal motion quality, the human preference study for the abstractive metrics (motion generalizability and motion-condition consistency) based on VLM are not provided.\n\n1. The frame-wise quality seems to replace the old distribution-based metrics (e.g. FID) with another “new” distribution-based metrics, except that the definition for the underlying distribution is different. Why would it be better to set up this way? Moreover, any empirical results of why this whole set of motion quality metrics is more reliable and robust than before? \n\n1. The baselines and ablations in Table 3 are not very convincing. Practically, the M2Mbranch is playing the role of a motion refinement module in this context, so the compared baseline should additionally includes text-to-video +  a SOTA motion refinement, to clearly evaluate the power of M2Mbranch. \n\n1. To fairly ablate the impact of external VLM assessment during inference, there should also be a specific ablation for not using the gated diffusion block, and instead only use the external VLM alignment scores to select from a SOTA T2M model and a SOTA text-to-video + motion refinement model, and other two ablations with replacing the one of two models with T2M or M2M respectively. \n\n1. Please pay attention to the typos, e.g. line 110, line 363.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bHSccFFsyy", "reviewer": "Reviewer_4ZNa", "summary": "This paper addresses the limited generalization of current 3D human motion generation models and proposes a unified framework that transfers knowledge from video generation models. They introduce ViMoGen-228K, a large-scale dataset combining high-fidelity MoCap data, in-the-wild video motions, and synthetic ViGen-generated samples to enhance semantic diversity. Besides, the authors present a flow-matching diffusion-based model with adaptive gating to fuse MoCap and ViGen priors. Finally, an evaluation benchmark MBench, is proposed for comprehensive evaluation.", "questions": "Please refer to weaknesses section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper addresses the generalization limitation in 3D human motion generation by offering a comprehensive solution across data, models, and evaluation benchmarks.\n* The paper is well-written and easy to follow, with well-organized structure.\n* The authors introduce ViMoGen-228K, a large-scale and diverse motion dataset, and MBench, a fine-grained evaluation benchmark. This provides training and evaluation solutions for the field of motion generation, which is beneficial for advancing technological development in this domain.", "weaknesses": "* Despite the authors claims strong generalization, the paper does not thoroughly examine where the model fails (e.g., on highly dynamic, multi-person scenarios). Analysis on this part is expected.\n* While the model is excels at the generalization, there appears to be a trade-off, as it does not outperform all baselines on certain motion quality metrics like dynamic degree. Is there any solution to alleviate this problem?\n* MBench relies heavily on VLM-based automatic scoring and curated prompts. I'm curious about the metric‘s robustness and sensitivities. Has the author analyzed these factors?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tFnO9zQIjF", "reviewer": "Reviewer_ixmY", "summary": "This paper proposes a comprehensive framework to enhance generalization in 3D human motion generation by transferring knowledge from video generation. \nThe contributions are threefold. \n1) The ViMoGen-228K dataset is introduced, integrating 228,000 high-quality motion samples from optical MoCap, web videos, and synthetic ViGen data to expand semantic diversity. \n2) The ViMoGen model is presented, a diffusion transformer through flow matching that unifies MoCap and ViGen priors via gated multimodal conditioning, alongside a distilled variant, ViMoGen-light, for efficient inference. \n3) MBench, a hierarchical benchmark, is developed for fine-grained evaluation across motion quality, prompt fidelity, and generalization. Experiments demonstrate superior performance over existing methods.", "questions": "Please see the weakness section.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The strengths of this paper lie in the following aspects:\n1) The construction of the ViMoGen-228K dataset demonstrates is encouraging, combining high-fidelity optical MoCap data with semantically diverse motions from in-the-wild and synthetically generated videos. \nThe multi-stage filtering pipeline encourages a balance between motion quality and semantic coverage.\n\n2) The design of the ViMoGen model's gated, dual-branch diffusion transformer is interesting as well. The adaptive selection mechanism between the text-to-motion and motion-to-motion branches dynamically balances high-quality MoCap priors with the broad semantic knowledge from ViGen models.\n\n3) The paper is very easy to understand and easy to follow. A straightforward presentation of both the data-set and methodology.\n\n4) As for the experiments, I notice the empirical gains on MBench: higher motion-condition consistency (0.53) and generalizability (0.68) than MDM, T2M-GPT, MotionLCM, and MoMask; reduced jitter (0.0108) and foot sliding (0.0064) with analyzed trade-offs in dynamic degree.", "weaknesses": "My major concern is that \n1) this paper lack of key comparison with methods: \n(i) MotionCraft (Bian et al., 2025) which shows state-of-the-art performance on text-to-motion on the HumanML3D subset of Motion-X dataset (Lin et al., 2023)\n(ii) FineMoGen(Zhang et al., 2023), \n(iii) MotionDiffuse both of which are also representative recent approaches and show promising results.\nAs well, the comparison is only conducted on the proposed MBench benchmark. How about the proposed approach on the widely adopted benchmarks? e.g. text-to-motion on the HumanML3D subset of Motion-X dataset; speech-based motion generation on the BEAT2 dataset; music-based motion generation on the FineDance.\n\n2) The topic and focus of this paper is too broad, while motion generation, has many aspects: speech-based, text-to-motion, music-based. I recommend the authors to narrow down the focus and scope and propose the advantages and difference the previous proposed approaches.\n\nReferences:\n\n[1] Yuxuan Bian, Ailing Zeng, Xuan Ju, Xian Liu, Zhaoyang Zhang, Wei Liu, and Qiang Xu. Motioncraft: Crafting whole-body motion with plug-and-play multimodal controls. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 1880–1888, 2025.\n\n[2] Mingyuan Zhang, Huirong Li, Zhongang Cai, Jiawei Ren, Lei Yang, and Ziwei Liu. Finemogen: Fine-grained spatio-temporal motion generation and editing. Advances in Neural Information Processing Systems, 36: 13981–13992, 2023c.\n\n[3] Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. IEEE transactions on pattern analysis and machine intelligence, 46(6):4115–4128, 2024b.\n\nSome other technical problems:\n1) There may exist potential data contamination and selection bias: (i) synthetic video motions are “refined” with a pre-trained ViMoGen M2M branch used later for training, creating circularity; (ii) optical MoCap subset selection is optimized using MBench, risking overfitting the training data to the proposed evaluation.\n2) How about the gating mechanism robustness? branch selection depends on VLM-based alignment to video-derived motion; training substitutes real video references with noise-perturbed GT motions, introducing a distribution gap between training and inference.\n3) How about the text annotation validity? large portions of text labels are produced by an MLLM from rendered depth/RGB frames with heuristic filtering only; no inter-annotator agreement or systematic quality audit is provided, risking noisy supervision for text–motion alignment.\n4) The dataset provenance and ethical are recommended to clarify: the internal video pool and synthetic data prompts are derived from large external caption corpora; consent, licensing of source material, and redistribution rights of derivative 3D motions are insufficiently detailed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B3MJkAGnpQ", "reviewer": "Reviewer_b56g", "summary": "This paper proposes a comprehensive framework of data, models, and evaluation to address the generalization capability of 3D human motion generation (MoGen). It leverages prior knowledge from ViGen.\n\nFor data, the authors built ViMoGen-228K, a large-scale dataset with 228,000 motion clips. It innovatively fuses high-fidelity optical motion capture (MoCap) data, diverse data from web videos, and long-tail data synthesized by Video Generation (ViGen) models. This wide variety of actions helps improve the generalization of MoGen.\n\nFor the model, the authors proposed ViMoGen, a flow-matching-based diffusion Transformer. It uses a novel gated dual-branch (T2M and M2M) architecture to adaptively unify the quality priors from MoCap data and the generalization priors from ViGen models. A distilled, efficient version, ViMoGen-light, is also provided.\n\nFor evaluation, the authors designed MBench, a new hierarchical benchmark for the comprehensive and fine-grained assessment of motion quality, text-motion consistency, and especially generalization capability.", "questions": "1. Regarding the data filtering pipeline, neither the main paper nor Appendix D.2 provides a detailed procedure. Key details are missing, such as the specific quality assessment filters employed and the retention ratios (or absolute clip counts) at each filtering stage. This lack of transparency makes the drastic data reduction (from 10M to 40k clips) , raises concerns.\n\n2. In Appendix D.2.3 (Synthetic Data), the authors state they \"compiled a list of action verbs and descriptive nouns.\"\nIt is recommended that the authors provide this complete list in the appendix and, crucially, conduct a comparative analysis against the vocabulary used in MBench (to ensure the generation vocabulary does not overly align with the benchmark).\nAdditionally, clarification is required on how potentially ambiguous verbs (e.g., \"have\") were programmatically handled or disambiguated during text prompt generation.\n\n3. ViMoGen-light: Table 2 reveals a substancial performance gap between ViMoGen-light and the full ViMoGen. The primary source of this discrepancy is not clearly analyzed.\nFurthermore, the distillation (referred to as \"distill\") used to create ViMoGen-light is critically underspecified. What specific distillation methodology was employed (e.g., DMD2, or )?\nWhat performance benefits  provide by ViMoGen-light (e.g., computation cost)?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality:\n\n1. To expand semantic coverage, the dataset leverages a Video Generation (ViGen) model to synthesize long-tail motion data, which is then integrated with traditional MoCap data.\n\n2. The paper introduces MBench, a hierarchical benchmark meticulously designed for the fine-grained evaluation of motion generalization capabilities, featuring a curated open-world vocabulary.\n\nQuality:\n\n1. The paper presents a complete and systematic solution—spanning data collection, model architecture, and a novel evaluation benchmark—demonstrating a thorough and solid investigation.\n\n2. The method's effectiveness is rigorously validated through extensive experiments, including comprehensive comparisons against state-of-the-art (SOTA) methods and detailed ablation studies. Furthermore, the MBench metrics are corroborated by a large-scale human preference study, ensuring their alignment with human judgment.\n\nClarity:\n1. The paper is well-structured and clearly delineates its three core contributions: the dataset (ViMoGen-228K), the model (ViMoGen), and the benchmark (MBench). \n\n2. The manuscript includes high-quality figures that effectively aid comprehension. For instance, Figure 1 clearly illustrates the overall framework and comparative radar charts; Figure 2 intuitively presents the model's dual-branch gated architecture; and Figure 3 vividly details the evaluation dimensions of MBench.\n\nSignificance:\n\n1. This work identifies and addresses a critical bottleneck in the field of 3D human motion generation: poor generalization capability. It proposes an effective and integrated solution to tackle this fundamental challenge.\n\n2. The authors commit to the public release of their code, the ViMoGen-228K dataset, and the MBench benchmark. These artifacts will serve as a valuable public resource, poised to stimulate further research and development in this area.", "weaknesses": "1. ViMoGen-228K leverages a mixture of three data sources: high-fidelity MoCap, alongside in-the-wild and synthetic videos. The filtering process for the in-the-wild data yields an extremely low retention rate (a reduction from 60M to 40k clips). This raises the question of whether in-the-wild video data, in itself, is inherently unsuitable for human motion extraction. Compounding this, based on Table 5, the \"Visual Mocap Data\" component does not appear to yield significant improvements. \n\n2. Regarding the generalization capabilities discussed (relative to ViGen), the 228k (370h) dataset used for MoGen is still considerably smaller than ViGen's pre-training corpus. How do the authors think MoGen's generalization capabilities compared with ViGen's generalization?\n\n3. The primary quantitative results (Table 2) are reported on MBench, a benchmark concurrently introduced by the authors. While MBench appears to be well-designed, this presents a potential risk: namely, that the new data and the proposed model may have \"overfit\" to the specific evaluation criteria of this new benchmark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)