SnapMoGen: Human Motion Generation from Expressive Texts
Authors: chuan guo
Inwoo Hwang
Jian Wang
Bing Zhou
Venue: NeurIPS 2025 poster ()
Keywords: Human Motion Generation
Primary area: applications

Abstract: Text-to-motion generation has experienced remarkable progress in recent years. However, current approaches remain limited to synthesizing motion from short or general text prompts, primarily due to dataset constraints. This limitation undermines fine-grained controllability and generalization to unseen prompts. In this paper, we introduce SnapMoGen, a new text-motion dataset featuring high-quality motion capture data paired with accurate, \textit{expressive} textual annotations. The dataset comprises 20K motion clips totaling 44 hours, accompanied by 122 detailed textual descriptions averaging 48 words per description (vs. 12 words of HumanML3D). Importantly, these motion clips preserve original temporal continuity as they were in long sequences, facilitating research in long-term motion generation and blending. We also improve upon previous generative masked modeling approaches. Our model, MoMask++, transforms motion into \textbf{multi-scale} token sequences that better exploit the token capacity, and learns to generate all tokens using a single generative masked transformer. MoMask++ achieves state-of-the-art performance on both HumanML3D and OmniMotion benchmarks. Additionally, we demonstrate the ability to process casual user prompts by employing an LLM to reformat inputs to align with the expressivity and narration style of SnapMoGen.

SnapMoGen: Human Motion Generation from Expressive Texts
Chuan Guo
Inwoo Hwang", Jian Wang', Bing Zhou
Snap Inc.
2 Scoul National University
ht https://snap-reeearre htt.::/eaaarra..i........iioooee /o/SnaMoGnn/
Abstract
Text-to-motion generation has experienced remarkable progress recent years. However, current approaches remain limited to synthesizing motion from short or general text prompts, primarily due to dataset constraints. This limitation un- dermines fine-grained controllability and generalization unseen prompts. In this paper, we introduce SnapMoGen, a new text-motion dataset featuring high- quality motion capture data paired with accurate, expressive textual annotations. The dataset comprises 20K motion clips totaling 44 hours, accompanied by 122K detailed textual descriptions averaging 48 words per description (vs. 12 words of HumanML3D) Importantly, these motion clips preserve original temporal con- tinuity as they were in long sequences, facilitating research long-tern motion generation and blending. We also improve upon previous generative masked model- ing approaches. Our model, MoMask++, transforms motion into multi-scale token sequences that better exploit the token capacity, and learns to generate all tokens using single generative masked transformer. MoMask++ achieves state-of-the-art performance on both HumanMI.31 and SnapMoGen benchmarks. Additionally, we demonstrate the ability to process casual user prompts by employing an LLM reformat inputs to align with the expressivity and narration style of SnapMoGen.
1 Introduction
Generating human motions from text has garnered increasing attention in recent years and has experienced notable progress. These advances have been made possible by existing large-scale text-motion datasets 10
19
30
28), and variety of deep generative models such as VAEs [00 25), GPTs
[1] 37 16), and generative masking 271. Nevertheless, current models encounter critical limitations when processing complex prompts, falling short in achieving fine-grained control and capturing nuanced variations in human movements. key contributing factor is the restricted expressivity of text descriptions in existing motion-text datasets. Textual annotations in these datasets are typically brief and general (e.g., "eprsn jumps up and lands"), ), lacking specific execution details. For instance, in Human [IO], motion sequences of approximately seconds are described by texts averaging only 12 words, which is insufficient to capture motion complexity.
diffusion models (34
5
The importance of expressive visual content synthesis fields 22 22 13 6 7]. Descriptive prompts notably enhance the accuracy and and improve temporal coherence video generation [[].
annotations has been well-established in other text-conditioned
aesthetic quality of generated images These models understand complex visual content compositions by learning from rich text semantics,
Project lead: guochuan5513 13@ wrgggal.cmm. com; bzhou@ cooouu@tsaphha..cmm
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

ro
m
Text Amntaaton (6s-33 Aixi "the person stands in heron nose, bending the right leg and bittop the body to loft white hoiding the head with bech hands. They then refrare the head lower The right leg, and spread the anns out before lowerisg them.
Text Annotation (13.66; 225): "A persos or right RicO, with right
ground they lowers thair loft amm, cutirrecceed luf lep. they puts they left rousea thei right hand abowe their hend,
lef, raiers right sightf,, stretches
The person dltss left, penalig the left leg and ratsing both arms above the head to join
hand benind back, leoru
Them Noxt, they crouch with the right lng bent bock, resting the right hand the
and busnerr left and right repratealyy After thes, they lowers right arm,
ground. The Lgf best leg postiioned forward the loft hard pioced on
S
leans to the right dightly and beacs thee left leg'
Figure I: Samples from SnapMoGen dataset. Our datasct features temporally continuous motion segments
paired with highly expressive text annotations. Each motion clip is accompanied by six distinct textual descrip-
tions. We show LLM a.Ma-uugmetted annotation for the first segment, and human annotation for the second.
enabling ine-grained visual content editing 17 [7.3], 3], adaptation [29 32 33 , and understanding 4 36 [8]. More importantly, when dealing with casual user prompts, rich LLM knowledge can be leveraged to enhance prompts with specific details and nuances that models have learned from fine-grained textual training data, effectively improving generalization capability. To foster this research direction in motion synthesis, we introduce SnapMoGen, which features high-quality motions captioned with accurate and highly expressive descriptions. SnapMoGen is created by segmenting long motion sequences into meaningful 4-12 second clips, each accompanied by six text deseriptions- two manually annotated by human experts and four augmented by LLM that introduces diversity while preserving semantics and temporal consistency. In total, SnapMoGen comprises 20K motion clips, amounting 40 hours of mocap data, accompanied by 122K detailed text descriptions. As shown in Fig. [ our text annotations contain extremely rich semantic cues of human movements, with an average length of 48 words-thrr times longer than HL3a Furthermore, our continuous motion segments facilitate research in long-term motion synthesis and motion localization. Tab. presents statistical comparison between SnapMoGen and related motion-text datasets.
To generate motions from expressive texts, we build an improved model upon the previous state-of- the-art approach- MaoMask [9. MoMask applies residual quantization to motion latent features, transforming them into multiple ordered sets of same-length discrete token sequences. Despite achieving pleasing VQ reconstruction through extensive tokens, many of them are not utilized to their full capacity. For instance, tokens following the first quantization layer carry only marginal information. This inefficiency, combined with its layer-specific token vocabulary design. creates inflexibility in subsequent text-to-token generation-necessiiiee necessitating separate models for different token sequences: a primary model for the first sequence and a secondary model for remaining tokens. To overcome these limitations, we adopt multi-scale approach for motion tokenization and generate all motion tokens using single generative masked transformer. our residual VQ, tokens each quantization layer focus on a particular temporal scale, following coarse-to to-fie gradual progression. Additionally, we share one codebook across all layers ensure a universal token vocabulary. As shown in Fig. our multi-scale RVQ continually learns meaningful semantics with more layers, outperforming conventional RVQ [ with 45% less tokens. Then, we simply concatenate tokens along the temporal dimension and train generative transformer to produce tokens from text by predicting randomly masked tokens. Our new framework, dubbed MoMask++, outperforms MoMask on text-to-motion generation with only a quarter of their token count, in Tab. 3
In summary, our key contributions are threefold. First, we introduce SnapMoGen, large-scale dataset comprising 20K temporally continuous motion capture clips described by 122K highly expressive text prompts. We also establish comprehensive benchmarks and evaluation protocols for this new dataset. Second, we advance beyond the existing state-of-the-ar -hr- approach by proposing Mo.Mask++, which optimizes motion token capacity through multi-scale quantization and models text-conditioned token generation using single generative masked transformer. Third, we demonstrate effective handling of casual user prompts through M.L-eb prompt rewriting, enabled by the descriptive captions in SnapMoGen.

---

Datasets
Year
Clips
Duration
Texts
Avg. words
Avg. length per clip
Mocap?
Continuous?
per text
KIT-ML
28
2016
3,911
10.3
6,278
9.5s
x
BABEL
30
2021
52,937
33.2h
52,937
2
2.38
M33
10
2022
14,616
28.6h
44,970
12
7.1s
X
Motion-X
19
2023
81,084
144.2h
81,084
9
6.4s
X
X
SnapNoGen
2025
20,450
43.7h
122,565
48
7.8s
Table 1: Comparisons with public datasets. SnapMoGen highlights its accurate and expressive text descrip- tions, high-quality motion capture data, continuous motion segmentation. f indicates values calculated only from the publicly available BABEL subset. * denotes a combination of 40,859 manual text annotations and 81,706 LLM-augmented annotations, both with an average text length of 48 words.
2 Related Work
Human Motion- Text Dataset. KIT Motion-Language Dataset [28] pioneered this domain with 3.9K motions and 6.3K numan-annotann descriptions but was limited in scale and text diversity. BABEL [30] introduced temporally precise frame-level labels across 33 hours of motion capture data; however, its annotations primarily consist of short phrases (c.g., 'lift something') )) for approxi- mately 2-second atomic actions rather than descriptions of extended sequences. HumanMI.3D (TO) expanded the field with 14.6K motions and 44.9K texts by aggregating data from AMASS (2 and HumanAct12 2 [12. Despite its size, the text descriptions remain brief and general (e.g.. "a person was pushed but did not fall"), failing to capture nuanced movement details. Motion-X [19] increased diversity by extracting motions from monocular videos and generating descriptions using video captioning models [36]. However, these motions often contain estimation artifacts such as jitter and foot-sliding, while their descriptions still lack expressivity. Recently, HuMMan- -Moenn introduced fine-grained descriptions for specific body parts in motions. In contrast, our SnapMoGen introduces highly expressive text descriptions for holistic 4-12 second motion segments.
Human Motion Generation. Recent advances in human motion generation, particularly in text- conditioned synthesis, have significantly improved the realism and text controllability of generated motions. Early methods explored continuous motion representations using generative models such as VAES 1T0 25 The introduction of diffusion models (34 38 39 23 has significantly advanced the field. By iteratively refining motion through denoising steps, these models generate realistic sequences that align closely with textual prompts. A parallel line of research models motion as sequences of discrete tokens using quantization techniques such as VAES [35]. These approaches represent motion as compact, structured token sequences. typically generated autoregressively 20 L3
37
16 or through generative masking schemes 9 27
To reduce quantization error, MoMask g applies multiple quantization layers to iteratively approximate the residuals. Nevertheless, as all quantization is applied at the same (and full) temporal scale, the information captured at each successive layer decreases drastically, leading to an overproduction of tokens with notably uneven information content. This inefficiency also makes text-to-token generation rather inflexible. These limitations directly inspire the multi-scale residual quantization process in our framework.
SnapMoGen Dataset
SnapMoGen encompasses 43.7 hours of high-quality motion data captured at 30 frames per sec- ond. The dataset comprises total of 4.7M motion frames, featuring a diverse range of actions including daily activities, fitness routines, social interactions, dances, and more. We deliberately incorporate various stylized performances (e.g., princess, elderly person, zombie) enhance diversity. SnapMoGen captures performances from 10 participants, resulting in 20,450 motion clips ranging from to 12 seconds in length. Each motion clip is accompanied by 6 detailed textual descriptions (2 manually annotated, 4 ALLM-auu M-augmeetted) totaling 122,565 textual descriptions with an average length of 48 words. A comparison between SnapMoGen and existing motion-text datasets presented in We further augment the dataset by mirroring motion data [10] throughout our experiments.
Table] I
3

---

Motion Coverage. We aim to cover a wide range of actions while ensuring high-quality 3D motion capture. All motions are recorded using Xsens and Bokokdmmotion capture suits. To determine motion content, we combine two resources: (i) LLMM-generated action scenarios covering diverse topics, and (ii) a curated collection of videos and images from the internet featuring content of interest, such as stylized movements. These text instructions and video demonstrations are presented to performers prior to recording sessions as reference material. Performers are then encouraged to execute these or related actions in their own interpretive style. Following data collection, motions with notable artifacts (e.g., jittering, foot sliding) are filtered out to maintain data quality.
Segmentation. We deliberately capture long motion sequences containing multiple actions for broader applications. Subsequently, we develop an automated pipeline to segment these sequences into shorter clips of appropriate lengths. The key principle is prioritize segmentation motionless moments. Specifically, we first calculate the average positional velocities of hip and end-effector joint each frame, smoothed by Gaussian filter. We then detect velocity troughs and normalize their values within each sequence, yielding Pl.m. E [0, 1J. where indicating the number of troughs. Each trough is selected segmentation point with probability 0.5p;, which typically results in clips averaging 8 seconds. Hard constraints of minimum (4s) and maximum duration (12s) are enforced during segmentation. Segmentation examples are provided in supplementary files.
Text Annotation. Each motion clip is rendered as video using a 3D character for annotation. We collect descriptions from two distinct annotators for each motion clip. The entire annotation process involves 55 professional native English-speaking annotators who are instructed to address following aspects in their textual descriptions: action, context, style, moving direction posture (if applica-
speed, trajectory shape, body parts, spatial relation/locatiin ble), and timing (if applicable). All annotations undergo a second-round review to ensure descriptive accuracy. Typographical errors in the collected textual descriptions corrected using LLM. To enhance textual diversity, we further employ the LLM to re-describe each manual description twice, maintaining precise action semantics while varying expression. This results in total of six distinct descriptions per motion clip.
4
Method
Our goal generate a 3D human pose sequence mı-V of length N guided by textual description where m; E R and D denotes the dimension of pose features.
4.1 Preliminary: Motion Tokenization via Residual Quantization
In traditional motion VQ-VAEs
37,, motion encoder E(-) encodes the motion sequence m E N to a latent feature sequence f Raxd which is further mapped discrete token sequence q [K]" through vector quantization:
f = {m).
q Q(),),
where ((-) denotes a quantizer. The quantizer typically consists of a learnable codebook C e R of K codes. During quantization, each feature vector fü mapped to the code index qi its nearest code entry in the codebook:
Ø argminke [K] (Cookup(C,A)) -fill) [K]
where lookup(C, k)) means taking the k-th vector in codebook C. The quantized feature vector sequence is finally fed into decoder D to reconstruct the input motion:
lookup 9),
D(})
To effectively reduce quantization errors, MoMask 9 introduces additional V quantization layers
Q L (-). Specifically, starting from the initial residual 0 = f. each @*(-) calculates token indices
http:://www ella www.prodacttstmoniieee capturelssens animate
hntps://ww pww.uctst hew.uuucttemtttttttt....ooo.

---

Motion token
Motion VQ codl
Text embxdding
Motion embedeing
Madked tokan
Codebook tookup
Up interpciation
Predicted tokens
In-Context
Transformer
Concatonate
embordiing g
embeddinge
Toxt taken embessiing
Myston soken entecliing
Croas Attantion
T5
Mask
Milh twas
2V
Text takens
Motion tokens
token
onbediing
(a) Multi-scate VQVAE
(b) Masked Transformer
(c) Conditioning Schemes
Figure 2: Approach overview. (a) A multi-scale VQVAR encodes a motion sequence into V + discrete token sequences (q";-...")) where each sequence qº operates specific temporal resolution h". Their corresponding quantized features upsampled full resolution ) y and summed before being fed into the decoder. (b) A single masked transformer operates on tokens from all scales. Token sequences from a motion are concatenated along the temporal dimension and randomly masked with a variable rate. The transformer trained to predict the masked tokens conditioned on text and the partially masked sequence. (c) We implement two methods for text conditioning: in-context learning and cross-attention.
qº and their corresponding codes fr as an approximation of residual r", and then computes the next residual y u++ as:
9" (r"),,
j" lookup(C"'
y
fr
Each quantization layer (.) contains a separate codebook C E Rxd, This approach yields V + 1 discrete token sequences [q"]]" E [K]V++))> of length n for a motion sequence. The final approximation of the latent sequence f is the sum of all quantized features E"=of"
Overall, this quantized auto-encoder model trained using compound loss function that combines motion reconstruction and per-layer latent embedding losses:
Lrog SmoothLi(m - m)) + 3
Σ || "rr"-sgl""" "gl/""]l2, V=U
where agl-] denotes the stop-grade gradient operation, and a weighting factor for embedding alignment. The codebook entries are updated using exponential moving average [37].
Discussion. Although high-fidelity VQ reconstruction can be achieved through an extensive set ((V 1) x of motion tokens, this approach introduces inflexibility in learning the text-to-token mapping, primarily due to two factors: i) information disproportionately distributed across quanti- zation layers-first-layer tokens typically contain the predominant features, while subsequent layers capture only incremental refinements; and ii) tokens at different layers are indexed by independent codebooks. To address this heterogeneity, MoMask [9] applics expressive generative masked transformer for the principal first-layer tokens, while modeling other-layer tokens with a secondary transformer conditioned on first-layer results. This hierarchical approach further diminishes the representational capacity of tokens in non-first layers.
4.2
Our Approach: MoMask+
In our approach, tokens at different quantization layers are designed to capture information at specific temporal resolutions, with common codebook shared across all layers. This design allows us to model the generation of all tokens using single generative masked transformer.
5

---

Full scale RVQ Nutti scale RVQ
Full scals RVQ
Nulti scale RVQ
Joint Position Error
(log scales
120
184
101-
40
Tokens
Tokens
Figure 3: Illustration of token capacity in a pretrained traditional 6-layer, 480-token full-scale RVQ 9 compared a 10-layer, 266-token multi-scale RVQ" Starting from zero-sequence, we incrementally add one quantized feature sequence for motion decoding and measure the reconstruction performance. The multi-scale VQ learns tokens more efficiently with meaningful semantics at each quantization layer.
4.2.1 Multi-scale Motion Quantization
As depicted in Fig. (a), for motion latent feature sequence f Raad our quantizer employs series of residual quantization operations at progressively increasing temporal resolutions {h") where ht" common codebook Quantized features at coarse level f E TR xd are bilinearly interpolated to full resolution f = I(f", h)), where residuals are calculated and then downsampled to next scale (h+41) to be quantized by the succeeding layer. Mathematically, Eq. (3) is reformulated as:
Je=0-
hv n. We denote all quantization operations uniformly as 2(-) as they share
oº Q( ,h")),
y r -I( I(f" h h),
r p0=f,
(5)
where f"= lookup(C,, 4"). Then, the final approximation of latent sequence f is the sum of all up-interpolated quantized sequences, which then fed into decoder D for motion reconstruction:
= =()
We further add self-attention layers after each res-block in existing motion VQVAE architectures 9 37 for higher-fideltyy motion reconstruction. During training, we introduce additional emphasis on reconstructing essential rotational features Less on the top of Lrug in Eq. (4), weighed by \ess- The final learning objective becomes:
Lme Grog + NeeeCese
(6)
After all, a motion sequence represented as V + ordered discrete token sequences with a hierarchy of temporal scales q = (q" .g' ..... q² q²), where each q" has length of h". Since a shared codebook is utilized across all scales, tokens from each q" belong to the same vocabulary [K].
As shown in Fig. compared to previous all full-scale residual VQ [9 our multi-scale VQ effectively exploits token capacity and continually learns meaningful semantic features at each quantization layer. It achieves superior reconstruction quality significantly with fewer tokens (266 vs. 480).
4.2.2 Learning Text-to-token Mapping
We employ single bidirectional transformer for token generation from text descriptions. Our framework is illustrated in Figure 2 (b-c). We utilize base [31] to extract word-level features from complex textual descriptions c. Motion tokens from all scales are concatenated along the temporal axis, yielding an extended token sequence q. which is then embedded through an MLP. We investigate two primary architectures for text conditioning: motion tokens and text tokens are concatenated and processed uniformly as transformer input, and (ii) Cross-attention, which incorporates additional multi-heaa cross- aooss-attnttonn layers that enable motion features to query relevant text features.
In-context learning. where embeddings of
Token counts encoding 320 pase seguence. VQQ, use [2.
60, 80)

---

R Precisiont
Methods
FIDJ
MM Dist.
MModality
Top
Top
Top
Real motions
0.511 #5003
0.703 00s
0.797
0.002 000
2.974
TM2T 11
0.424#0003
6618:t.00s
0.72
1.501 4.01
3. 3.467#.011
2.424+005
T2M [0]
0.4555#0003
0.636
0. 0.736tt.000
1.087
3.3477
2.21g:t.074
MDM 34
0.611 0.611# +00
0.544+.004
5.566
2. 2.799
MLD
0.481
0.673--008
0.7722..00 002
0.473 1-13
3.196 2.010
2.413
MotionDiffuse 38
0.491 2..01
0.681 =001
0.7822.000
0.630 001
3.113#..001
1.553+.002
T2M-GPT 37
0.49222.003
0.679-..002
0.775
0.141 0.141 1.005
3.121 1.000
1.831 1.8314.008
MMM 27
0.515#t.002
0.708 0.708+t002
+0022
0.089
2.926 20226tt0000
1.226++0000
MoMask
0.521 00.0
0.713
t.807
0.045 0002
22958tt.000
008
1.241
Ma
0.528 1-288-000
0.718
0.811 + 002
0.072
2.912 008
1.227
4oMask4455
0.517#t0022
0. 700
0.803tt.002
0.069 -003
2.948
1.192
Table 2: Quantitative evaluation on HumanML3D test set. + indicates 95% confidence interval. Bold indicates the best result, while underscore refers to the second best. "in": in-context. "cra": cross-attention.
In training, a varying fraction 7(T) = cos( E [0, 1], where T (0 1). of sequence elements is uniformly selected, masked out, and replaced with special [MASK] token. The transformer is trained to predict these masked tokens given text input e and the partially masked token sequence i. by maximizing the likelihood:
Smask
Σ
llog pe (qx (qxlĀ.c).
(7)
4N [MASK]
We adopt the replacing and remasking strategy 9 g
to enhance contextual reasoning ability. Ad-
ditionally, the model is trained without text condition c with probability of 10% enable classifier free guidance (CFG).
During inference, complete sequence of q be generated in constant number (L) of iterations. This process begins with an empty sequence [[MASK]]* where all tokens are masked, with N Ev h" denoting the total number of tokens in At each iteration (I), the model predicts categorical token distributions at masked locations, samples tokens, and re-masks the [Y[ .N) lowest- confidence tokens. This process repeats until reaches L. We also adopt classifier-free guidance in [9] with guidance scale s. Finally, all generated tokens are decoded and projected back to motion sequence through the VQ-VAE decoder.
5 Experiments
Besides SnapMoGen, we also conduct experiments on HumanML3D [10), popular motion-text dataset comprising 14,616 motions with 44,970 textual descriptions.
Dataset Setup. We process motions in SnapMoGen following procedures established in Hu- manML3D, including motion mirroring and standardization. To prevent data leakage, we deliberately hold out a test (%10) set and a validation (%5) set where the motion scenarios (e.g., fashion) differ from the training motions. We primarily adopt the feature representation from HumanML3D, consist- ing of root angular velocity along Y-axis jd E IR, root linear velocity on XZ-plane juz E R2, root height fu' R. 6D local joint rotations j E R local joint positions j" R and local joint veloc- ities j" 3 where denotes the number of joints. We empirically find that this comprehensive set of pose features leads to slightly better performance (Tab. 4 Our SnapMoGen follows a skeletal topology comprising 24 joints, resulting in 296-dimensional pose features. Unlike HumanML.3) our pose features are directly convertible to standard motion capture file formats (e.g., BVH).
Evaluation Setup. We adopt established metrics including FID, R-Precision, MultiModal Distance,
and Multimodality following previous works 14 34]. The evaluator from prior research [10] was exclusively trained to align motion and text embeddings. However, the resulting motion embeddings may be biased toward text alignment while overlooking motion fidelity. Additionally, its redundant motion feature design lacks flexibility for broader evaluation scenarios (23]. Therefore, adopt
7

---

R Precisiont
Methods
FID
CLIP Scoret
MModality
Top
Top
Top
Real motions
0.9410#.000
0.976
0.985 4001
0.001 0.000
0.837 000
MDM
34
0.503-1.02
0.653--002
0.727
57.7831 092
0,481-000 001
13.412
01.412--229
T2M-GPT 37
0.618#-002
0.773=-002
0.812 002
32. 22.6299 087
0.573=000
9.172#
StableMoPusion 14
0.679 002
0.823=-000
0.888 002
27. 97.801* + 0G3
0.605=000 001
9.0644
MARDM
2R
0.659
0.812#-002
0.860 002
26.878 1:31
0. 0.602
9.812#
MoMask
9
0.7774.000
0.888 0.8884.002
0.9274..00 902
17.404 051
0.664
8.1834..14
0.805 =005 0022
0.90-1
=002
0.938
15.56
0.684 0.684=0001
6.556
+oMask++
0.802#.001
0.905 #002
0.938
15.06
0.685
7.259
Table 3: Quantitative evaluation SnapMoGen test set.
the TMR [26] approach for our evaluation model, utilizing only essential 148 148-dimensional motion features. This method extracts separate latent vectors from motion and text, requiring the motion vector to both align well with corresponding text features and accurately reconstruct the source motion (ensuring fidelity). We use the T5-base model to extract word-level text features. For R-Precision calculations, we employ a candidate pool size of 100. We also use the CLIP score [23] evaluate lext-motion alignment, which measures the cosine similarity between text and motion features.
Comparison Models. On SnapMoGen, We reproduce haseline methods across three mainstream generative paradigms: diffusion models (MDM 34. StableMoFusion [14), and MARDM (23) autoregressive models (T2M-GPT (37), and generative masking approaches (MoMask M.. We utilize their official codebases. Each experiment is repeated 20 times, with final results reported as means with 95% confidence intervals. For all baselines, we replace the original text encoder with
T5-Base. For MoMask, we implement 6-layer RVQ. Please refer to supplementary materials for baseline implementations.
Implementation Details. Our VQVAE encoder and decoder consists of three dilated rlocksocks,
with down(up)-scale factor of 33 9
The temporal quantization scales follows the progression [n/2V, ,..., n1/2"] with n denoting the full-scale length. We employ (i.e., V = 3) quantization layers for HumanML3D and 2 for SnapMoGen, with codebook sizes of 512 512 and 2048 x 512, respectively. The paper-parameters 3 and deso are set to 0.02 and 2.0. Our transformer architecture comprise 8 layers with feedforward size of 1024, latent dimension of 384, attention heads, and a dropout ratio of 0.2, totaling 13.5M parameters for in-context model, and 18.3M parameters for cross-attention model. During inference, we use classifier-fre guidance scales of 5 and 4, and iteration counts (L) of 10 and 18 for SnapMoGen and HumanML3D. respectively. All models are trained on a single Tesla V100 GPU, with batch size of 256 for VQVAEs and 64 for transformers.
5.1
Comparison to State-of-the-art Approaches
The quantitative results on HumanML33 and SnapMoGen are reported in Tables and respectively. Overall, MoMask++ attains state-of- of---h--thttr performance on both datasets, demonstrating consistent improvements motion-text alignment and motion quality. These advantages are particularly pronounced in our SnapMoGen dataset, partially due to the more expressive evaluation model. We observe that previous works struggle with the complex, lengthy text inputs SnapMoGen, and fall short in maintaining multimodal semantic coherence, as evidenced by the relatively low CLIP scores and R- precision values. Notably, our method outperforms MoMask with only two VQ layers (a quarter of MoMask's token count) with similar model size. Between the two variants of Mo.Mask+. we find that the in-context model generally performs better on HumanML3D. It however tends to overfit on long text prompts in SnapMoGen (Fig. (6) and underperforms compared to the cross- attention model. Nevertheless, a significant gap to real motions still exists, suggesting substantial room for future improvements.
Figure 4 displays pose sequences generated by MoMask++, demonstrating ability to produce precise motions following fine-grained text prompts. We further showcase the capability to handle out-of-domain user prompts by employing an LLM to rephrase the inputs. For additional generation results and comprehensive visual comparisons, please refer to the supplementary materials.

---

"The perios intottr zumbe. They thet fenward thev nody sightiy biy dowerendd malk mobally fenward, then mcbely step beckward and level off while standing stll."
dunsou stands nestrally, iron steeoding thas legs wide
semulnce
with gostrezz moving thasr hops seft havd havd di disssin actsrs. They Thoypmone turn put the right big forword, iquet slghtly sefore sonuliting bostettiol shot small
"Boxing lnvilbble oppenere"
ooswaing lightly on They didgge contterpuneees Thro incosity, weeking
person
fow
fort
Figure 4: MoMask+ generated samples for SnapMoGen test prompts (#1,2) and a casual user prompt (#3).
Pose Dim.
VQ Config
VQ Reconstruction
T2M Generation
MCoues
*Juantt
F/M
w/ Al.
FID
Joinr Pos. lir.
FID
CLI Scor"
0.673
(A) Only essential pase feat.
148
1024
4.71
7.12
15.95
0.667
(B) Smaller codehook
1024
3.30
15.61
0.668
512
3.77
8.95
16.96
0.665
3.13
8.40
16.21
0.652
(C) Varying #quant
4.57
15.56
0.684
8.81
10.48
16.25
0.677
Full scale multi-scale scale
F
2.64
6.53
18.02
0.667
(IT) Wio attention
3.39
8.57
16.18
0.662
Table Ablation analysis of VQ configuration on SnapMoGen test set. "F/M" denotes full-scale versus multi-scale residual quantization, For text2motion, this experiment use 284 latent dimension, 1024 forward size, and 8 attention layers. with T5-bast text encoder and in-context conditioning.
5.2 Component Analysis
We perform comprehensive ablation experiments to evaluate the effects of various hyper-paranee syper-parameters and technical designs, as shown Tab, 4 and Tab. 5 In Tab, (A), we observe that compact pose representation leads to smaller VQ reconstruction error, while it slightly underperforms for text-to-motion synthesis.
In terms of VQ configuration, we observe from Tab. (B) that while increasing codebook size consistently enhances VQ reconstruction and text-motion alignment (CLIP score), motion quality does not necessarily follow this trend (best FID at |C| 1024). Tab. (C) show that additional VQ layers effectively improve reconstruction, but more token hierarchies also introduce complexity for text-to-motion synthesis, with optimal results at 2 layers. In Tab. (D), we apply full-scale for all layers, yet despite achieving better VQ performance, inefficient token utilization leads to suboptimal generation quality. Finally, incorporating self-attention layers in the encoder and decoder (Tab. (E) improves both VQ learning and motion synthesis performance.
We then examine the effects of text augmentation and text-to-motion transformer design in Tab. 5 In Tab. (A), caption augmentation clearly improves model performance across all evaluation metrics. In Tab. (B), we observe that the CLIP text encoder is inadequate for handling the long and complex textual descriptions
SnapMoGen. From Tab. and Fig. 6 we further find that cross-attention conditioning is less prone to overfitting and leads to higher motion quality and better text-motion alignment. Meanwhile, Tab. (D) show that transformers with higher latent dimensions or more attention layers counterintuitive,, degrade motion generation quality. Figureló provides additional insight, indicating that larger transformer models (Base: 18.3M, Medium: 36.6M, Large: 53.4M) tend to overfit the dataset more severely.
How are tokens at different scales modeled? To investigate this question, we analyze which tokens are "favored" by MoMask++. During iterative inference, MoMask+ generates complete

---

T2M Config.
T2M Generation
Text Aug.
Text Enc.
Conditionigg
Architecure
FID.
CLIP Score
base
TS-base
In-conteat
(B) 384, 1024,
15.56
0.684
(A) W/o text
x
17.98
0.656
(B) CLIP text enc.
CLIP
19.96
0,478
Cross-att cond.
Cross-att.
15.06
0.685
Cross-att. (M) 2048,
15.58
0.679
(D) Larger model
Cross-att.
51) 2048,
16.02
0.670
Table Ablation analysis T2M model configuration SnapMoGen to transformer hyperparameters including latent dimension, feedforward size, and number of layers. This experiment use quantization layers for VQ, with 2048 cndehook size and 296 pose features.
"Archi tecture" refers
Progress of Token Decoding across Scaless
In-Context (B)
1.0
7.0
Cross-attention (B)
Cross-attention (M)
Cumulative Decoding
Val. Loss
6.0
Cross-attention
Scale 3 |40 takens)
Scale 4 [80 takens)
0.0
10
100
200
300
400
500
terations
Num. of Epochs
Figure 5: Decoding progress over iterations for different token scales.
Figure 6: Validation loss curves of different model sizes on SnapMoGen.
motion sequence by selectively retaining and re-reeeee re-masking tokens at each step, allowing to track which tokens the model prioritizes over time. We conduct an experiment using four token scales (from coarse to fine, with 10, 20, 40, and 80 tokens, totaling 150 tokens per sequence) and 10-iteration inference process over 32 text prompts, recording the token completion ratio at each scale. The results, shown in Fig. 5 reveal that the model naturally prioritizes coarse-scale tokens (Scale 1) in the early stages of generation and progressively shifts its focus toward finer scales. This behavior demonstrates "global-to-local" generation strategy, indicating that the attention mechanism effectively captures and prioritizes information based on semantic importance (coarse-to-fine)
Conclusion
In this paper, we introduced SnapMoGen, a high-quality text-motion dataset featuring temporally continuous motion segments with expressive textual annotations. Comprising 20K motion clips and 122K detailed deseriptions averaging 48 words each, SnapMoGen provides significantly richer semantic information than existing datasets. We also proposed MoMask++, novel text-to-motion generation framework that employs multi-scale residual vector quantization and single genera- tive masked transformer for token prediction. Extensive experiments both HumanML3D and SnapMoGen demonstrate the state-of-the-ar performance of MoMask+t.
10

---

References
[1] Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang. and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models, arXiv preprint Stti:24005042233. 2024.
[2] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang. Juntang Zhuang, Joyce Lee. Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://edn. openai. com/papers/dall-e-. pdf. 2(3):8, 2023.
[3] Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transformers. arXiv preprint arXiv:2301. 30.0000444 2023.
[4] Guo Chen, Yin-Dong Zheng, Jiahao Wang, Jilan Xu, Yifei Huang, Junting Pan, Yi Wang, Yali Wang, Yu Qiao, Tong Lu, et al. Videollm: Modeling video sequence with large language models. arXiv preprint arXiv:2305. 12992, 2023.
[5] Xin Chen, Biao Jiang. Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Exccuting your commands via motion diffusion in latent space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18000-18010, 2023.
[6] Zijie Chen, Lichao Zhang, Fangsheng Weng, Lili Pan, and Zhenzhong Lan. Tailored visions: Enhancing text-to-image generation with personalized prompt rewriting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7727-7736, 2024.
[7] Siddhartha Datta, Alexander Ku, Deepak Ramachandran, and Peter Anderson. Prompt expansion for adaptive rext-to-image generation. arXiv preprint arXiv:2312.16721 2023.
[8] Jacob Devlin, Ming-War Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume I (long and short papers), pages 4171-4186, 2019.
[9] Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked modeling of 3d human motions. In Proceedings the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 1900-1910, 2024.
[10] Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages $152-5161, 2022.
[11] Chuan Guo, Xinxin Zuo Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts. In European Conference Computer Vision, pages 580-597. Springer, 2022.
[12] Chuan Guo, Xinxin Zuo. Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Action2motion: Conditioned generation of 3d human motions. In Proceedings of the 28th ACM International Conference on Multimedia, pages
2020.
[13] Yaru Hao, Zewen Chi, Li Dong, and Furu Wei. Optimizing prompts for text-to-image generation. Advances in Neural Information Processing Systems, 36:66923-66939, 20223
[14] Yiheng Huang, Hui Yang, Chuanchen Luo, Yuxi Wang, Shibiao Xu, Zhaoxiang Zhang, Man Zhang, and Junran Peng. Stablemofusion: Towards robust and efficient diffusion-basedd mo- tion generation framework. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 224-232, 2024.
[15] Yiming Huang, Weilin Wan, Yue Yang, Chris Callison-Burdes Mark Yatskar, and Lingjie Liu. Como: Controllable motion generation through language guided pose code editing. In European Conference Computer Vision, pages 180-196. Springer, 2024.
[16) Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as foreign language. Advances Neural Information Processing Systems, 36:20067-20079 2023.
11

---

[17] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pages 6007-6017, 2023.
[18) Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint 1..1122 2023.
[19] Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x: A large-scale 3d expressive whole-hody human motion dataset. Advances in Neural Information Processing Systems, 36:25268- -222800 2023.
[20] Shunlin Lu, Jingho Wang, Zeyu Lu, Ling-Hao Chen, Wenxun Dai, Junting Dong, Zhiyang Dou, Bo Dai, and Ruimao Zhang. Scamo: Exploring the scaling law in autoregressive motion generation model. arXiv preprint arXiv:2412.1455: 2024.
[21] Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. Amass: Archive of motion capture surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442-5451, 2019.
[22] Oscar Mañas, Pietro Astolfi, Melissa Hall, Candace Ross, Jack Urbanek, Adina Williams, Aishwarya Agrawal, Adriana Romero- -Soiann,, and Michal Drozdzal. Improving text-to-image consistency via automatic prompt optimization. arXiv preprint arXiv:2403.1780 2024.
[23] Zichong Meng, Yiming Xie, Xiaogang Peng. Zeyu Han, and Huaizu Jiang. Rethinking diffusion for text-driven human motion generation. arXiv preprint arXiv:2411.16575. 2024.
[24] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint 1a0n::0084440 2018
[25] Mathis Petrovich, Michael J Black, and Gül Varol. Temos: Generating diverse human motions from textual descriptions. In European Conference on Computer Vision, pages 480-497. Springer, 2022.
[26] Mathis Petrovich, Michael J Black, and Gül Varol. Tmr: Text-to-motion retrieval using con- trastive 3d human motion synthesis. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9488-9497, 2023.
[27] Ekkasit Pinyoanuntapong, Pu Wang. Minwoo Lee, and Chen Chen. Mmm: Generative masked motion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1546-1555, 2024.
[28] Matthias Plappert, Christian Mandery, and Tamim Asfour. The kit motion- dataset. Big data, 444):236-22,, 2016.
[29] Ryan Po, Guandao Yang, Kfir Aberman, and Gordon Wetzstein. Orthogonal adaptation for modular customization of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7964-7973, 2024.
[30] Abhinanda R Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramiree and Michael Black. Babel: Bodies, action and behavior with english labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 722-731, 2021.
[31] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-6) 2020.
[32] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 22500-223 12:00022510, 10, 2023.
12

---

[33] Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, al. Styledrop: Text-to-image generation in any style. arXiv preprini arXiv:2306.00983 2023.
[34] Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H Bermano. Human motion diffusion model. arXiv preprint arXiv:2209.1499 2022.
[35] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[36] Hang Zhang, Xin Li, and Lidong Bing. Video-Ilama: An instruction-finetuned visual language model for video understanding. 2023.
[37] Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Shen. T2m-gpt: Generating human motion from textual descriptions with discrete representations. arXiv preprint arXiv:2301.00052, 2023.
[38] Mingyuan Zhang. Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. arXiv preprint 1500011208.150 2022.
[39] Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Remodiffuse: Retrieval-augmentee motion diffusion model. arXiv preprint arXiv:2304.01 2023.
13

---

panoe sanii rell peotiiel
more and love
utlo
Bec tooe on lond gerut
Foo righteal ham bas or cioeer
The by
Figure Text annotation for SnapMoGen.
A Annotation Details
Our text annotation interface is presented in Fig. 7 We first visualize all motions using 3D character to help annotators better understand the motion content. However, since the character may walk out of the camera view and inter-penetration artifacts sometimes occur, also display the motions using stick-figure representations that remain centered in the camera view. These two visualizations are synchronized and presented simultaneously to annotators. Annotators can also flag low-quality motions during the annotation process.
B Evaluation Details
B.1 Baseline Implementation.
All baseline models SnapMoGen dataset leverage the T5- model for extracting word-level features from text descriptions and are trained using a single NVIDIA RTX A6000 GPU.
For MDM 3d we use an 8-layer transformer decoder where the text encoding is injected via cross-attention layers. The model trained for 600K steps with a batch size of 1024 using diffusion process with T 1000 steps. For T2M-GPT 337, we first learn a codebook size of 1024 512 with a downsampling rate of Then, we model sequence of codebook indices via an 18-layer transformer. During training, text embeddings and motions are concatenated and processed as input, and a random portion the ground-truth code indices is replaced with random ones to improve robustness. The model is trained for 600K steps with a batch size of 128. For StableMoFusior [14), we use a Conv ID-based U-Net incorporating residual cross-ates cross-attention to align motion features with word-level semantics, along with group normalization. The model is trained for 500K iterations with = 1000 denoising steps and a batch size 1024. For MARDM [23], we first encode motion into latent representation using 3-layer ResNet-based auto-encoder. These motion latents then modeled using masked autoregressive transformer with a dimension 1024 and 16 attention heads, where text encodings are injected via cross coss-attention layers. The model is trained for 600K steps with a batch size of 128.
B.2 Evaluation Model
Our evaluation model accounts for both motion fidelity and text-motion alignment. We adopt the TMR framework, as shown in Figure 8 This framework comprises three network components: motion encoder that encodes motion sequences into global vectors, sequences into global vectors, and motion decoder that reconstructs motions from either motion or text vectors. All three networks are 6-layer transformers with a latent dimension of 256, attention heads, and feedforward hidden size of 1024. The T5-base model first extracts word-level features from texts. For motions, we use only the essential 148-dimensional root motion and local rotational features. All encoders output Gaussian distribution parameters (mean and variance,, from which
encoder that encodes text
14

---

Somcone is kicking
man walks quarter circle the left.
human making a counterclockwise turn.
Text
Motion Decoder
with the left leg
cf
of
41
S/2
S/3
Motion Encoder
S21
S22
S23
$31
$32
S33
Figure 8: Architecture of the evaluation model [26]. Three network components are trained with two main goals: multimodal alignment and reconstruction. The cosine similarity between motion embeddings and text embeddings from positíve pairs (green) is maximized, while similarity for negative pairs minimized. Meanwhile, both embeddings are required to reconstruct the corresponding motion sequence through the motion decoder. Image adapted from TMR [26
vectors are sampled. We append two temporal timesteps at the end of the sequence input for outputing these vectors.
This evaluation model is trained with compound loss:
Lime
Liee + CKL + dEhE + ANCECNCE.
where free measures the motion reconstruction given text motion input (via smooth L.1 loss). A KL-divergence loss fKL regularizes each embedding distribution to be close to a unitary Gaussian distribution (O,, I), and also encourage these two distributions to be close to each other. LE enforces both mean vectors to be similar each other. Finally, a InfoNCE [24] loss used for constrastive learning of motion-text batches with batch size of 64. We set λE. AKL. and ANCE to le - 5, le - 5. and 0.I. For more model details, we recommend to read the original TMR work (26).
In inference, we employ the evaluation metrics designed in [10]. We increase the pool size for R-precision 100 and directly use the mean vectors of the latent distributions as embedding vectors
C LLM-based Prompt Augmentation
Dataset Augmentation. During training, we enhance data diversity by employing an LLM to rewrite human-providee annotations, generating paraphrased versions with varied linguistic structures while preserving core semantics. This approach ensures each motion sequence is associated with multiple textual descriptions, improving model robustness. The prompt instructions for ChatGPT provided in Tab. 6
Inference-time Prompt Augmentation. During inference, the LLM rewrites each input prompt into a richly detailed description, incorporating explicit motion cues such as body posture, timing, and stylistic elements. This expanded form more effectively guides motion generation models. The instructions for ChatGPT are provided in Tab. 7
D Limitation
We present several representative failure motions in the static webpage. Here we discuss limitations from both data and model perspectives.
Dataset. Despite extensive calibration and post-processing of the collected motions, quality issues rooted in the inertial-basee mocap suit persist. For example, global positions may lack precision, and jitters can oceur during fast or complex motions. Additionally, we are unable to capture highly skilled motions such as cartwheels, backflips, outdoor activities (c.g., climbing).
15

---

Task Deseription The goal rewrite textual descriptions from text-to-motion dataset correct typos and grammar while rephrasing them for better readability and fluency. The key requirement semantics of the described human motion must be preserved exactly, with no loss or modification of motion detail. You may vary the sentence structure, use synonyms, merge split phrases, or improve flow, but the described body movements and temporal order must remain unchanged.
the
Instructions
Correct all spelling, grammar, and stylistic issues in the input text.
2. Rephrase the input to make clearer, more fluent, and more readable.
You must remove, simplify, or alter the described body movements.
4. Keep all motions, ordering, and temporal logic intact.
Add mild clarifications only if they help with motion clarity.
Examples
Original: The person takes two steps forward, starting with his left foot, bends down and reaches down with his right hand to the floor, rises with his arms out to the sides and steps back, and abruptly takes step forward with his right foot with his arms bent the elbows, and shaking both arms slowly steps forward standing in a fighting stance and provoking a fight.
Augmented:
The person steps forward twice, beginning with the left foot. They bend down, reaching the floor with their right hand. Rising, they extend arms sideways step back. Suddenly, they step forward with the right foot, elbows bent, arms shaking. They stand in a fighting stance, slowly advancing. as if provoking a fight.
Original:
The person stands with their legs wide apart. Then they take steps back and slightly to left, lowering their head down and raising their left hand to their head. Then they lower their left hand and take two steps to the right, stopping. Then walks forward, turning to the left and waving their arms.
Augmented:
The person stands with legs spread wide apart. They move steps backward and slightly to the left while lowering their head and raising their left hand to touch it. Afterward, they lower their left hand, take two steps to the right, and pause. Then they walk forward, turning towards the left while waving their arms in fluid motion.
Table Prompt instruction for grammar-correcting and semaatically-preservvng text augmentation.
Model. Opportunities for improving text-to-motion models also remain. As MoMask++ relies on VQ, quantization errors inevitably degrade motion quality. We observe that MoMask++ struggles with rare motion patterns or uncommon text prompts. Furthermore, it does not yet maintain physical plausibility, such as proper foot contacts.
16

---

Task Description: Your task is rewrite prompts user inputs for text-to-motion generation model inference. This model generates 3D human motion data from text, you need to understand the intent user input and describe how the human body should move in detail, and give me the proper duration of the motion clip, usually from 4 to 12 seconds.
Instructions:
Make sure the rewritten prompts describe the human motion without major information loss.
Be related to human body movements- the tool not able to generate anything else.
The rewritten prompt should be around 60 words, more than 100.
Use a clear, descriptive, and precise tone.
Be creative and make the motion interesting and expressive.
Feel free to add physical movement details.
Examples:
Input: Shooting basketball.
Rewrite: The person stands neutrally, then leans forward, spreading their legs wide. They simulate basketball dribbling with hand gestures, moving their hips side side. The left hand performs dribbling actions. They pause, turn left, put the right leg forward, and squat slightly before simulating a basketball shot with a small jump.
Length: 8 seconds
Input:
Zombie walk.
The person shuffies forward with a stiff, dragging motion, one foot scraping the ground as moves. His arms hang loosely by its sides, occasionally jerking forward as staggers with uneven steps. Length: 6 seconds
Rewrite:
Table Instructions for re-writing casual user prompts.
NeurIPS Paper Checklist
17


Reviews:
[{"review_id": "TPOTcI3PGB", "reviewer": "Reviewer_xbdR", "summary": "The authors propose a new text2mtoion dataset and revisit the Momask model to get better results. The dataset is useful and meaningful. It comprises 20K motion clips totaling 44 hours, accompanied by 122 detailed textual descriptions averaging 48 words per description (vs. 12 words of HumanML3D).", "questions": "1. How the author handle the halluciation problem when only using the text as the prompt?", "limitations": "N/A", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Dataset is good. The t2m field nees more high quality data.\n2. The authors provides extensive experiments results and visualization results, which are not observed before.\n3. The evaluation process is standardized and meaningful. \n\nWeaknesses:\n1. The paper do not propose a new generation model or architecture, but an incremental improvment on the existing method.\n2. The rewriting process is intuitive, since 2/3 of the texts are generated by LLM. And the generated texts have it own bias. I do not think this should be highlighted in the abstract and it is just a trick everybody knows. \n3. Although the dataset is useful, the paper do not provide any new insight for me. It is more like an engineering project. Thus, I will give a boarderline this time.", "was_revised": true, "final_justification": "The authors solved most of my questions and further clarify their framework contirbution to me. This work is still more like an engineering work to me. I decided to keep my score as a Borderline accept.", "rebuttal": "**We thank the reviewer for the comments, and hope the following response properly addresses your concerns.**     \n\n\n***Q1***: *The paper does not propose a new generation model or architecture, but an incremental improvement on the existing method. Although the dataset is useful, the paper does not provide any new insight for me. It is more like an engineering project.*\n\n**A1**: Thanks for your comment. While our model builds on existing methods, our paper's core novelty and insight lie in two areas:\n\nFirst, we highlight the **critical, under-explored role of expressive text input** in text-to-motion generation. Our OmniMotion dataset, with its uniquely rich and detailed annotations, proves that fine-grained text-based control and generalization are possible when models are provided with sufficient textual cues. This enables generating motions from complex prompts like \"a bird taking flight,\" which were previously impossible. We also benchmark a wide range of baseline models on OmniMotion under fair settings.\n\nSecond, our MoMask++ model introduces a more efficient multi-scale motion quantization and uses a single transformer, making it a neater and more effective solution. It also demonstrably outperforms previous methods on long text prompts within our challenging OmniMotion dataset, pushing the state of the art in practical application.\n\nThese two contributions collectively offer new insights and practical advancements for the field.\n\n---\n\n***Q2**: The rewriting process is intuitive, since 2/3 of the texts are generated by LLM. And the generated texts have their own bias. I do not think this should be highlighted in the abstract and it is just a trick everybody knows.*\n\n**A2**: Thanks for the feedback. We want to clarify that our LLM integration is a **strategic enabler**, not just a trick. We use LLMs in two ways:\n1. **Dataset Curation**: For minor corrections and semantic expansions to ensure OmniMotion's high-quality, expressive annotations. I also use LLM to increase the text variations in the dataset\n2. **Inference-Time Prompt Enhancement**: Crucially, LLMs enrich casual user prompts with motion details, bridging the gap between typical user input and the detailed motion cues our model requires.\n\nWe are highlighting the second point in the abstract to underscore how our expressive dataset enables generalizable motion generation from diverse user queries. We'll revise the abstract to clarify this key application.\n\n---\n\n***Q3**: How does the author handle the hallucination problem when only using the text as the prompt?*\n\n**A3**: We assume the reviewer is referring to **LLM hallucination during prompt rewriting**. Please correct us if this is not the case.\nWe address this by:\n1. **Strict Prompting**: We follow best practices from image/video generation [1,2], using highly descriptive prompts to guide the LLM.\n2. **Task Simplicity**: For dataset augmentation, the LLM's role is limited to correcting typos and rephrasing for diversity while strictly preserving original motion semantics. For inference, it adds motion details without altering user intent.\n3. **Quality Control**: We provide several in-context examples to reduce ambiguity and have meticulously checked rewriting samples.\n\nGiven the **relatively low complexity** of these rewriting tasks, we observed negligible instances of factual errors or \"hallucinations\" in the LLM-generated prompts (refer to Tables 5 & 6 in the supplementary material). We are happy to discuss this further in the limitations section if the reviewer deems it necessary.\n\n---\n\n***Q4**: The ablation table needs to be refined. It is hard to recognize.*\n\n**A4**: Thank you for the suggestion. We will reorganize the ablation table to improve readability in the version. \n\n---\nWe sincerely hope our response addresses all your concerns. Please feel free to let us know if you have further questions.\n\n[1] Betker et al., Improving image generation with better captions. 2023.     \n[2] Bao et al., Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models. 2024."}, {"review_id": "vIVsD1F0zO", "reviewer": "Reviewer_PaWz", "summary": "This paper introduces MoMask++, a multi-scale masked transformer for text-to-motion generation, alongside the OmniMotion dataset with detailed text descriptions. While the work shows solid technical execution and experimental effort, several concerns regarding novelty and methodology limit its contribution.", "questions": "Questions to Address\n\nNovelty: How does your multi-scale approach fundamentally differ from prior works like MoFusion or T2M-GPT, beyond standard cross-scale attention? Can you better justify the novelty given the modest performance improvement over MoMask?\n\nMulti-Scale Fusion: Why was summation chosen for fusing multi-scale features? Have you explored alternative, weighted fusion methods to avoid over-emphasizing low-frequency information? What analysis can you provide on the post-fusion features?\n\nAttention & Positional Encoding: How does the attention mechanism handle concatenated sequences of very different lengths (e.g., 2 vs. 80 tokens)? How do positional encodings function after this multi-scale concatenation to preserve temporal order?\n\nDataset Timestamps: Could you enhance the dataset by aligning text descriptions with timestamps to allow for motion speed and pace control?\n\nMotion Quality: How do you plan to address the foot-ground contact issues mentioned in the limitations?\n\nVisualization: Can you revise Figure 2 to make the visualizations for \"Motion VQ code\" and \"Motion embedding\" more distinct to avoid confusion?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The OmniMotion dataset with extended text descriptions fills a practical gap and could benefit the community.\n2. The paper is well-structured and easy to follow, with comprehensive experiments and useful ablation studies.\n3. The LLM-based prompt rewriting feature shows thoughtful consideration for real-world usability.\n4. The multi-scale residual quantization framework demonstrates some improvements.  \n\nMain Concerns:\n1. Limited Novelty. This is the main concern with the paper. The core contribution appears to be primarily an engineering combination of existing approaches. While the multi-scale extension of MoMask is reasonable, similar multi-scale strategies have been widely explored in computer vision and motion generation (e.g. MoFusion, T2M-GPT) and cross-scale attention is fairly standard. The improvements shown in Tables 2 and 3 are relatively modest compared to MoMask, which doesn't strongly validate the method's effectiveness.\n\n2. Methods.\na. The paper employs summation for multi-scale information fusion, but it remains unclear whether this is the optimal approach. The use of equal-weight summation may lead to overemphasis of low-frequency information (global patterns) while underweighting high-frequency information (local details). A more detailed analysis of the post-fusion features is needed.\nb. When concatenating sequences of very different lengths (e.g., 2 vs 80 tokens), the attention mechanism might naturally favor longer sequences.  Some discussion of this issue would strengthen the paper. Also, It's unclear how positional encodings work after multi-scale concatenation, which seems important for preserving temporal relationships.\n\n3. Dataset: Figure 1 shows the text annotations for motion, which include a series of continuous action descriptions. However, these are merely descriptions of consecutive sub-action contents, lacking alignment with time steps. Therefore, we cannot determine the speed or pace of each sub-action from these descriptions. I believe supplementing this information would be beneficial for text-to-motion generation.\n\n4. Minor Issues\na. The foot-ground contact issues mentioned in limitations suggest room for improvement in basic motion quality.\nb. In Figure 2, the Motion VQ code in (a) and the Motion embedding in (c) look too similar, which could cause confusion. A more distinguishable visualization approach should be used.", "was_revised": true, "final_justification": "Almost all reviewers have expressed similar views, acknowledged the contributions of this paper, and agreed that it falls within the borderline acceptance range.", "rebuttal": "**We thank the reviewer for the comments, and hope the following response properly addresses your concerns.**\n\n***Q1**: Novelty: While the multi-scale extension of MoMask is reasonable, similar multi-scale strategies have been widely explored in computer vision and motion generation (e.g. MoFusion, T2M-GPT). Can you better justify the novelty given the modest performance improvement over MoMask?*\n\n**A1**: Thanks for your feedback. We wish to clarify that our work offers distinct innovations **beyond a mere engineering combination**, specifically in:\n\n1. **Novel Multi-Scale Token Design for Motion**: We respectfully clarify that neither MoFusion nor T2M-GPT employs a multi-scale token design as ours does. MoFusion is a continuous diffusion model, and T2M-GPT uses single-layer tokens. To the best of our knowledge, we are the **first** to introduce this multi-scale tokenization for motion generation, enabling each layer to capture distinct, meaningful semantics (Fig. 3).\n\n2. **Compact & Flexible Token Modeling**: Our design models motion tokens significantly more efficient than MoMask's: achieving the same VQ reconstruction with **2/3 fewer** tokens and requiring **3/4 fewer** tokens for generation. Crucially, a single generative transformer models all tokens, allowing the model to adaptively prioritize important tokens across **all scales** during inference, unlike MoMask's fixed layer order.\n\n3. **Demonstrable Gains on Complex Prompts**: While HumanML3D metrics are overall modest, our core strength is handling long, complex text prompts. On OmniMotion, MoMask++ shows notable improvements over MoMask (FID 17.4 to 15.06; CLIP 0.66 to 0.685). Qualitatively, our model demonstrates superior understanding of nuanced semantics (e.g., \"right leg in front\") and complex motions (e.g., \"figure skater spinning\") that MoMask struggles with (see supp. files).\n\nFinally, we encourage a **holistic** evaluation of our contribution. Our LLM-based prompt rewriting method, made possible by our unique **OmniMotion dataset**, further enhances the model's practical utility to handle novel and complex user prompts (e.g., \"bird flight,\" \"frog jump\"), beyond what existing methods can achieve.\n\n---\n\n***Q2**: Why was summation chosen for fusing multi-scale features? Have you explored alternative, weighted fusion methods to avoid over-emphasizing low-frequency information? What analysis can you provide on the post-fusion features?* \n\n**A2**: **Equal-weight summation is mathematically necessary** for our VQ learning framework. Specifically, we decompose the input feature $f$ into hierarchical residuals across temporal resolutions, and at each resolution the residual features are quantized. For the final summed feature $\\hat{f}$ to precisely approximate the input $f$, the quantized features from all levels must be summed **uniformly**, being consistent with the *residual decomposition* process.\n\nOur design inherently prevents over-emphasis on low-frequency information:\n* **Early quantization layers** (i.e., global pattern) are explicitly assigned with shorter token sequences.\n* **Later layers** capture high-frequency local details with more tokens.\n---\n\n***Q3**: When concatenating sequences of very different lengths (e.g., 2 vs 80 tokens), the attention mechanism might naturally favor longer sequences.  How does the attention mechanism handle concatenated sequences of very different lengths (e.g., 2 vs. 80 tokens)?*\n\n**A3**: Thanks for this very insightful comment. To investigate this question, we look into the tokens *\"favored\"* by MoMask++. During iterative inference, MoMask++ generates a complete sequence by retaining and re-masking certain tokens. This allows us to track which tokens the model prioritizes. We conducted an experiment with four token scales and 10-iteration inference using 32 text prompts, tracking the **token completion ratio** at each scale. The results are shown below:\n| Iterations | Scale 1 (10 tokens) |  Scale 2 (20 tokens)  |  Scale 3  (40 tokens)  |   Scale 4 (80 tokens)  |   Total (150 tokens) |\n|:----:|:----:|:----:|:----:|:----:|:----:|\n|Iter 0|0.0|0.0|0.0|0.0|0.0|\n|Iter 1|0.1500|0.0111|0.0280|0.0021|0.0133|\n|Iter 2 |0.5389|0.1194|0.0139|0.0083|0.0600|\n|Iter 3|0.6056|0.4000|0.0889|0.0299|0.1333|\n|Iter 4 |0.7667|0.5306|0.2750|0.0715|0.2333|\n|Iter 5 |0.8333|0.6389|0.4597|0.1813|0.3600|\n|Iter 6 |0.9278|0.7667|0.6278|0.3160|0.5000|\n|Iter 7|0.9500|0.8639|0.7708|0.5174|0.6600|\n|Iter 8|0.9667|0.9389|0.8750|0.7569|0.8267|\n|Iter 9 |1.0|0.9972|0.9958|0.9903|0.9933|\n|Iter 10 |1.0|1.0|1.0|1.0|1.0|\n\nAs the table shows, despite their shorter length, the model naturally prioritizes coarse-scale tokens (Scale 1) early in the generation process. It then gradually shifts its focus to finer-scale tokens, demonstrating a \"global-to-local\" generation strategy. This indicates that the attention mechanism effectively captures and prioritizes information based on its semantic importance (coarse-to-fine) rather than simply token length.\n\nWe will include this in our revision as suggested.\n\n---\n\n***Q4** : It's unclear how positional encodings work after multi-scale concatenation, which seems important for preserving temporal relationships.*\n\n**A4**: First, each token sequence at every scale is padded to a **fixed maximum length** before concatenation. Then, all token sequences are concatenated by scale (e.g., Scale 1 tokens, then Scale 2, etc.), forming a single long sequence. This resulting sequence is then **incrementally positional encoded** as a standard sequential input. The model recognizes a token's scale from its fixed positional region within this concatenated sequence, and temporal order is also maintained within each scale's token sequence.\n\nWe also explored a \"separate encoding\" strategy, where tokens within each scale were separately PE'd and added with a scale-specific embedding before concatenation. However, this approach led to worse performance (FID increased from 15.06 to 16.45, CLIP score dropped from 0.685 to 0.670).\n\n---\n\n***Q5**: Dataset Timestamps: Could you enhance the dataset by aligning text descriptions with timestamps to allow for motion speed and pace control?*\n\n**A5**: Thank you for the suggestion. In OmniMotion, each motion clip is already associated with **precise start and end frame timestamps**, which are encoded directly in the clip filenames. For example, clips named *ep1_00000#0#281* and *ep1_00000#281#503* indicate consecutive segments spanning frames 0–281 and 281–503, respectively.\n\nThese timestamps enable downstream applications to control motion speed and temporal alignment. We encourage the reviewer to refer to the provided *Sample_annotations.json* file for more examples and details.\n\n--- \n\n***Q6**: Motion Quality: How do you plan to address the foot-ground contact issues mentioned in the limitations?*\n\n**A6**: We agree that foot-ground contact is a critical aspect of motion realism. The issue typically stems from two factors:\n\n1. **Mismatch between global trajectory and local motion** (e.g., the character appears to slide or move unnaturally fast/slow), and\n2. Local foot jittering or penetration artifacts.\n\nTo address these, we see several promising directions:\n1. For global-local mismatch, recent methods like GenMoStyle [1] and NeMF [2] learn a regressor to infer global motion from local features. A similar approach could be integrated into our framework.\n2. For local foot artifacts, classical foot inverse kinematics (IK) can help enforce contact constraints.\n\nMore generally, integrating **physics-based priors or simulators** could further enhance motion plausibility and enforce realistic contact dynamics.\n\n---\n***Q7**: Visualization: Can you revise Figure 2 to make the visualizations for \"Motion VQ code\" and \"Motion embedding\" more distinct to avoid confusion?*\n\n**A7**: Thanks for pointing this out. We will revise accordingly.\n\n---\nHope our reply satisfactorily addresses your concerns. Otherwise, we will be happy to further discuss.\n\n[1] Generative Human Motion Stylization in Latent Space, ICLR 2024.    \n[2] NeMF: Neural Motion Fields for Kinematic Animation, NeurIPS 2022"}, {"review_id": "ak5po97Jjk", "reviewer": "Reviewer_bxuM", "summary": "This paper tackles the problem of human motion synthesis from text. The authors propose a new motion dataset called OmniMotion which consists of around 20k motion sequence each with 6 detailed textual description. Different from preivous motion datasets whose text description is brief and general, OmniMotion provides highly expressive text descriptions for each motion enables the potential of fine-grain control in motion synthesis. Building upon the SOTA method MoMask, the authors further introduce MoMask++. The main contribution of MoMask++ is the multi-scale motion quantization which enables better motion reconstruction with less tokens.", "questions": "L202, how does the CFG used in an auto-regressive model?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The overall quality of the writting is good. The arguments are clear and valid. \n- The motivations of the new dataset as well as the multi-scale motion quantization are convincing.\n- The highly detailed text description provided by the new dataset can be very helpful to the future reseach of text-to-motion.\n\n**Weakness**\n- The technical contribution is limited. The proposed MoMask++ is similar to the MoMask pipeline with differences on residual quantization and text encoding. And the only contribution is the multi-scale residual quantization. There is no novelty on the text encoding since it simply uses an existing model. \n- The performance of the method on HumanML3D dataset is either marginally better or even worse comparing to MoMask. While on OmniMotion dataset, the proposed method is consistently better. Are there any explainations on this? Is it because the T5-based text encoder better leverages the highly detailed descrition of OmniMotion dataset? If so, what is the benefits of the multi-scale residual quantization in text-to-motion?", "was_revised": true, "final_justification": "I would suggest accept this paper. The reubuttal addressed most of my major concerns. The authors provide a clarification of the contribution between their method and MoMask++ and detailed discussion about the performance on HumanML3D dataset.", "rebuttal": "**We thank you for your thoughtful comments. The following responses aim to address your concerns point by point.**\n\n***Q1**: The proposed MoMask++ is similar to the MoMask pipeline with differences on residual quantization and text encoding. And the only contribution is the multi-scale residual quantization.*\n\n**A1**: Besides multi-scale residual quantization, we also employ an **unified generative transformer** which allows more **flexible token prioritization**. Unlike MoMask, which uses separate models for different scales and follows a fixed, layer-by-layer generation schedule, MoMask++ employs a single, universal Transformer to generate tokens across all scales. This design is not only more efficient but crucially enables the model to **dynamically and flexibly prioritize token generation** at any scale during iterative inference. As detailed in our response to Reviewer xbdR (Q3), MoMask++ naturally learns a \"global-to-local\" strategy, prioritizing coarse-scale tokens early while also selectively valuing important finer-scale tokens.\n\n**LLM-Enhanced Generalizability**: Our novel integration of LLMs to rephrase casual, out-of-domain user prompts into detailed, expressive descriptions is a critical technical contribution. This approach, uniquely enabled by our OmniMotion dataset's expressive annotations, allows MoMask++ to achieve unprecedented generation on novel, complex prompts like \"a bird taking flight\" or \"chasing after a fluttering butterfly.\" This is a practical advancement for real-world text-to-motion applications.\n\n---\n\n***Q2**: The performance of the method on HumanML3D dataset is either marginally better or even worse comparing to MoMask. While on OmniMotion dataset, the proposed method is consistently better. Are there any explanations on this? Is it because the T5-based text encoder better leverages the highly detailed description of OmniMotion dataset? If so, what is the benefits of the multi-scale residual quantization in text-to-motion?*\n\n**A2**: First, to clarify, all baselines, including MoMask, were adapted to use the **T5 text encoder for fair comparison** on OmniMotion (an ablation is provided in Table 4, F Row). The performance difference is not due to a text encoder advantage.\n\nThe disparity arises from the nature of the datasets: HumanML3D typically features simple text semantics, where MoMask performs well. In contrast, OmniMotion's **much longer and more complex texts** critically stress a model's capacity for fine-grained text-motion mapping. MoMask++ is uniquely advantaged on OmniMotion due to:\n\n1. **Compact, Detail-Preserving Multi-Scale VQ**: Our multi-scale VQ design effectively preserves motion details while yielding significantly more compact token sets (e.g., 45% fewer tokens than MoMask Residual VQ). This eases the burden of generating extensive tokens from long, complex text prompts.\n\n2. **Token Exploitation**: By confining temporal scales at each quantization layer, our VQ encourages tokens to continuously capture meaningful semantics hierarchically (as shown in Figure 3). This contrasts with MoMask's VQ, where first layer is overly dominant and latter layers are often \"lazy\" to capture incremental information. Our design's efficient information encoding facilitates superior text details to motion mapping.\n\n3. **Flexible, Single-Stage Generative Transformer**: Unlike MoMask's rigid, fixed-order two-stage generation, our one-stage masked Transformer dynamically and flexibly prioritizes token generation across all scales during iterative decoding (as elaborated in our A1 to Reviewer xbdR (Q3)).\n\nCollectively, these advantages make MoMask++ a better candidate for coping with the long and complex texts found in OmniMotion.\n\n---\n\n***Q3**: L202, how is the CFG used in an auto-regressive model?*    \n\n**A3**: We'd like to clarify that **MoMask++ is not an auto-regressive model**. Instead, it is a non-autoregressive masked generative model where **all tokens are predicted in parallel** at each iteration and then re-masked for subsequent iterations.\n\nConsistent with the MoMask approach, Classifier-Free Guidance (CFG) is applied during inference at the final linear projection layer before softmax. The guided logits $ω_g$ are computed by: $ω_g =(1+s)⋅ω_c −s⋅ω_u$ where $ω_c$ are the conditional logits, $ω_u$ are the unconditional logits, and $s$ is the guidance scale.\n\n---\nWe sincerely hope that we have properly addressed your concerns. If not, we are happy to open further discussions."}, {"review_id": "6J0sFv7sBJ", "reviewer": "Reviewer_zsJy", "summary": "This paper introduces OmniMotion, a new human motion dataset and generation framework. It presents a large scale collection of continuous motion capture sequences paired with rich, paragraph length textual descriptions that cover action details, style, and context. The paper also proposes MoMask++, a multi-scale residual vector quantization and unified masked Transformer approach for tokenization and parallel generation of motion sequences. Experiments demonstrate state-of-the-art performance on benchmarks and ablation studies identify optimal quantization and model configurations.", "questions": "(1) The paper relies on LLM‐based caption augmentation, yet the examples suggest mostly simple synonym swaps. Could the authors include an ablation study that compares training with and without these augmented captions to quantify its actual impact on model performance? If such an experiment shows a clear gain in generation quality or retrieval metrics, my confidence in the dataset’s contribution would increase.\n\n(2) During inference, prompts are likewise expanded via an LLM to match the training style. How do you verify that these expanded prompts faithfully capture the user’s intent without introducing bias? Have you measured whether the added detail reduces motion diversity? Providing some visualization on diversity vs. prompt length would help clarify the model’s robustness and could strengthen the paper.", "limitations": "Please add a discussion of the limitations, for example, whether LLM-annotated text is truly expressive or merely a longer, semantically equivalent restatement.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n(1) The paper delivers a large-scale, richly annotated human motion dataset.\n(2) Proposed MoMask++ model that fuses multi scale RVQ and a unified masked Transformer to reduce token usage while improving reconstruction quality.\n(3) Leverages LLM-based prompt augmentation both to enrich dataset annotations and to harmonize inference inputs with training style.\n(4) Presents extensive benchmark comparisons and ablation studies that clearly demonstrate the dataset’s value and the model’s performance gains.\n\nWeaknesses:\n(1) During dataset generation, augmented captions rely mostly on simple synonym swaps and lack deeper structural or semantic variation\n(2) The shift from MoMask to RVQ, while effective, represents a modest, incremental improvement over existing quantization approaches.\n(3) Inference appears to depend on LLM-expanded, detailed prompts—raising questions about the model’s ability to generate diversity results.", "was_revised": true, "final_justification": "I thank the authors for their responses. In the rebuttal, the authors used experimental results to demonstrate that their caption augmentation can improve performance, which shows the value of their dataset. Overall, I think this paper proposes a valuable dataset, with relatively comprehensive experiments and good writing. However, using LLMs to enhance text captions is a standardized practice, and the method itself is an incremental engineering effort. Therefore, I tend to maintain my original rating.", "rebuttal": "**We appreciate your time and effort spent on reviewing our paper. Below, we answer the raised questions one by one.**\n\n***Q1**:  Could the authors include an ablation study that compares training with and without these augmented captions to quantify its actual impact on model performance? If such an experiment shows a clear gain in generation quality or retrieval metrics, my confidence in the dataset’s contribution would increase.*\n\n**A1**: Thank you for your valuable suggestion. Our LLM-based caption augmentation specifically aims to increase textual variation without altering or missing core motion semantics. We conducted an ablation study comparing our model trained with and without these augmented captions, with results below:\n\n|    | FID$\\downarrow$ |  Top1$\\uparrow$  |  Top2$\\uparrow$  |  Top3$\\uparrow$  |  CLIP Score$\\uparrow$  |\n|:--:|:--:|:--:|:--:|:--:|:--:|\n|  Ours  |  15.56|  0.802  |  0.905  |  0.938   |  0.684  |\n|  Ours w/o text aug  |  17.98  |  0.758  |  0.873  |  0.917  |  0.656  |\n\nThe table clearly shows that caption augmentation significantly enhances model performance across all metrics. Our training curves also show that text augmentation makes the model less prone to overfitting. \n\n---\n\n***Q2**: During inference, prompts are likewise expanded via an LLM to match the training style. How do you verify that these expanded prompts faithfully capture the user’s intent without introducing bias?*\n\n**A2**: During inference, we instruct the LLM with descriptive prompts and in-context examples (Table 6 in supplementary) to \"understand the user's intent and rewrite a clear, descriptive, precise-tone prompt without major information loss.\"\n\nOur primary verification involved **meticulous manual checks on hundreds of samples**, comparing the original user prompt, the rewritten prompt, and the resulting generated motion. Given the capabilities of current LLMs for this specific task, we found that the rewritten prompts and their generated motions reliably and precisely reflect the original user intent.\n\nNevertheless, we acknowledge that potential risks may still exist, and we are happy to discuss this in the limitations and outlook sections.\n\n---\n\n***Q3**: Have you measured whether the added detail reduces motion diversity? Providing some visualization on diversity vs. prompt length would help clarify the model’s robustness and could strengthen the paper.*\n\n**A3**: Thank you for the suggestion. We have not measured the effect of added detail on motion diversity, as it’s difficult to be quantitatively evaluated. Intuitively, more detailed prompts reduce ambiguity and naturally constrain diversity. We agree that qualitative examples would help clarify this trade-off and will include them in the revision. It's worth noting that many raw user prompts (e.g., “walk like a robot”, “bird taking flight”) fail without added detail, as shown in our ablation videos.\n\n---\n***Q4**: Please add a discussion of the limitations, for example, whether LLM-annotated text is truly expressive or merely a longer, semantically equivalent restatement.*\n\n**A4**: Thanks! We will add this to our limitation discussion.\n\n---\n***Q5**: The shift from MoMask to RVQ, while effective, represents a modest, incremental improvement over existing quantization approaches.*\n\n**A4**: Thank you for the comment. Our multi-scale motion quantization improves over MoMask's RVQ by achieving comparable reconstruction with only one third of token counts. In motion generation, MoMask++ notably outperforms MoMask on complex prompts in OmniMotion, reducing FID from 17.4 to 15.06. As shown in our supplementary videos, MoMask++ better handles fine-grained control (e.g., “right leg in front”) and rare actions (e.g., “figure skater spinning”). Nevertheless, we acknowledge that there are still much room to improve in future.\n\n---\nThanks again for your comment. We sincerely hope our responses have properly addressed your concerns."}]

Author rebuttal:

Meta-review: This paper was reviewed by four experts in the field, with 4 Borderline Accept recommendations. All the reviewers appreciate that this paper proposes an important dataset for human motion generation, with reasonable technical improvements and relatively comprehensive experiments. Thus, this paper achieves the NeurIPS acceptance requirement. AC highly urges the authors to go through the detailed comments carefully to polish the writing and provide extra experimental details, so as to ensure the final acceptance.
Decision: Accept (poster)