Autoregressive Motion Generation with Gaussian Mixture-Guided Latent Sampling
Authors: Linnan Tu
Lingwei Meng
Zongyi Li
Hefei Ling
Shijuan Huang
Venue: NeurIPS 2025 poster ()
Keywords: Motion generation
Gaussian Mixture
VAE
Autoregressive Generation
Primary area: applications
TL;DR: A continuously autoregressive approach for text to motion generation with gaussian mixture-guided latent sampling.

Abstract: Existing efforts in motion synthesis typically utilize either generative transformers with discrete representations or diffusion models with continuous representations.
However, the discretization process in generative transformers can introduce motion errors, while the sampling process in diffusion models tends to be slow.
In this paper, we propose a novel text-to-motion synthesis method GMMotion that combines a continuous motion representation with an autoregressive model, using the Gaussian mixture model (GMM) to represent the conditional probability distribution.
Unlike autoregressive approaches relying on residual vector quantization, our model employs continuous motion representations derived from the VAE's latent space. This choice streamlines both the training and the inference processes.
Specifically, we utilize a causal transformer to learn the distributions of continuous motion representations, which are modeled with a learnable Gaussian mixture model. 
Extensive experiments demonstrate that our model surpasses existing state-of-the-art models in the motion synthesis task.

Autoregressive Motion Generation with Gaussian Mixture-Guided Latent Sampling
Linnan Tu', Lingwei Meng", Zongyi Li', Hefei Ling', Shijuan Huang'
'Department of Computer Science and Technology, Huazhong University of Science and Technology
"The Chinese University of Hong Kong
(nntu, zongyili, lhefei, shi ahiiuan_huanggae huang)/hust.eddi Imengèse. cuhk. edu. hk
Abstract
Existing efforts motion synthesis typically utilize either generative transformers with discrete representations or diffusion models with continuous representations. However, the discretization process in generative transformers can introduce mo- tion errors, while the sampling process in diffusion models tends to be slow. In this paper, we propose a novel text-to-motion synthesis method GMMotion that combines continuous motion representation with autoregressive model, us- ing the Gaussian mixture model (GMM) to represent the conditional probability distribution. Unlike prior autoregressive approaches relying on residual vector quantization, our model employs continuous motion representations derived from the VAE's latent space. This choice streamlines both the training and the inference processes while mitigating discretization errors. Specifically, we utilize a causal transformer to learn the distributions of continuous motion representations, which are modeled with a learnable Gaussian mixture model. Extensive experiments demonstrate that our model surpasses existing state-of-the-ar models in the motion synthesis task.
Introduction
3D human motion synthesis. i.e., generating vivid action sequence by control conditions, holds promising applications in game development, embodied intelligence, and the animation. Two main paradigms are used today: (1) One paradigm uses generative transformers with discrete motion representation, such as GPT-like 2 or BERT-like 4 5) models, to synthesize motions based on specific conditions. Typically, these methods require vector quantization model (6) convert continuous motion sequences into discrete codebook tokens. Subsequently, a generative transformer is trained either using feacher-forcing approach generate discrete motion tokens autoregressively or using a masked filling strategy to generate them non-autoregrresivelly Finally, decoder synthesizes the final motion sequence. (2) The other paradigm employs diffusion models with continuous motion representation 7 8 10). They first train continuous autoencoder, primarily using VAE- based (1 models, to create a compressed and semantically rich representation of motion in latent space. The diffusion model then utilizes various sampling strategies 12 9 [3 and conditional control methods (44 [5) to generate latent vectors of motion that align with given conditions. However, both paradigms have vantdees
The VQ process inevitably disrupts the continuity of the time series, which can lead errors during the token connection process. Some works employ residual VQ (RVQ) 4 16). which involves iteratively summing residuals using multiple codebooks to mitigate compression loss. However, models using RVQ often have structure similar to VALL-E (17), which requires separate processing
"Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

of tokens for the primary layer and the residual layers, increasing the complexity of the model. Some studies use binary codebooks (T8) or model body joints separately (I9), but this can increase the codehook size and lead to higher storage costs. Moreover, the highly compressed nature of the motion can lead to less diverse outputs, favoring common patterns found in the training data.
The diffusion models offer better diversity with text conditions and generate high-quality motion (20 21 8): however, their inference speed is limited since the sampling process requires multiple iterations. Efforts to address these issues, such as MotionLCMM (12) and B2A-HDM (22), have successfully decreased sampling steps through distillation techniques. Nevertheless, they require pre-specifying maximum sequence length, which limits the scalability of motion generation. Therefore, we are motivated develop an approach that reduces compression loss in motion representation while enabling continuous generation in molti--odal spaces.
In this paper, propose novel framework called Autoregressive Modeling with Gaussian Mixtures (GMMotion), which aims to synthesize motions with continuous GMM latent sapce, demonstrating that vector quantization not a necessary prerequisite for autoregressive motion modeling. We lead the motion sequence into the Gaussian mixtures' latent space with learnable parameters in the first stage. By constraining the latent representation to be continuous multi-modal distribution during VAE training and recovering it in the second stage with continuous autoregressive (AR) model, we can build AR model that retains all the advantages of Large Language Modeling (prompting, integrated duration modeling, and sampling) without many of the challenges associated with VQ. Additionally, our key advantage lies eliminating the need predefine the duration of generated content, enabling the synthesis of longer motion sequences based on the complexity of control conditions.
Our approach includes three major aspects:
We utilize a learnable Gaussian mixture model to represent motion sequences multi- distributions.
We introduce an autoregressive causal transformer that learns the distribution of continuous representations and employs Gaussian mixture sampling to generate motion representations.
We design a straightforward architecture that benefits from single-step Gaussian mixture sampling and AR generation, leading to extrapolatable inference and high-quality motion synthesis.
2 Related Work
2.1 Autoregressive Generation with Continuous Tokens
Autoregressive models (33 24 typically generate content utilizing quantized representations
extracted from raw data 6 25).. However, recent studies (76) find that as long as the per-token
distributions are modeled, autoregressive models can be approached without vector quantization. MAR (27) introduced an image-masked autoregressive modeling method that uses an ML.P head to perform diffusion sampling on several consecutive tokens at each iteration. LatentL. (2X) introduces multi-modal latent language model with next-token diffusion approach, achieving good results in both speech synthesis and image generation tasks. However, the sequential iterative diffusion sampling process can lead to extremely slow inference speeds. MELLE (29) proposes autoregressive text- to-speech synthesis model using uni-modal Gaussian sampling to accelerate the inference process. However, as shown in Figure I time series data often follow multi-moda distributions, making difficult to accurately fit with an unimodal Gaussian distribution. VAE (12) exhibits an unimodal distribution but does not cluster the representations of movements. RVQ 4 captures the slightly chaotic multimodal distribution. Due to sampling errors and the varying movements of different joints, the raw data do not conform to typical normal distribution, resulting in multi-modal distributions even more complex distributions.
2.2 Motion Generation Methods
have been employed in earlier studies. Some motion synthesis models leverage raw motion data for training generative models (7). However, these models can be affected by measurement errors in the
Research on human motion analysis has long history (30) statistical models 33
32 1

---

GM4 (Ours)
(Ours)
Come.1
Crmp-2
Comp. 1t
Comp
C
Come d
C
Comp
VAE
[a] Raw Motion's Distribution
(b) Comparison of Latent Distributions
Figure 1: (a) Violin map of the motion's 's moving components randomly selected from the normalized HL3D datasets:(b) Visualization of different representation model's '' latent space with t-SNE. Yellow indicates dense sample distribution, while blue represents sparse sample distribution.
data, which often result from inaccuracies in the motion capture process (33 34). Furthermore, the motion vectors corresponding to different joints exhibit specific distributions, making it challenging for these models to learn representations of complex movements. To maintain the continuity of time series, some approaches (3) (3) employ continuous representation learning models such as variational autoencoders (VAEs) (1)) In these models, an encoder predicts the mean and variance of motion latent vectors, followed by sampling from Gaussian distribution obtain the latent vector representation, which is then decoded to reconstruct the motion. Although VAEs ( achieved satisfactory reconstruction results, they struggled to differentiate between various motions, thereby increasing the difficulty of learning for the subsequent generative model.
Recent work has leveraged Vector Quantized- -ariiattoaa Autoencoders (VQ-VAEs) 5 36 18 to achieve discrete representation learning for motion. These models use codebook indices as motion tokens, enabling the application of language modeling (37) techniques and resulting in impressive generation outcomes. However, due to the inevitable quantization error introduced by the discrete process, autoregressive masked generation methods based VQ models have been constrained
(29 27).
Methodology
Given motion description like "A person rolls forward once, then raises their hand and quickly shoots a ball.", our goal is to create a 3D motion sequence that reflects this description. In this paper, propose a learnable Gaussian mixture model to represent motion sequences multi-moda distributions, regularized by the Kolmogorov-Snie -orirov (KS) distance [Sec.B.Il. We then introduce masked autoregressive transformer that learns the distribution of continuous representations and employs Gaussian mixture sampling to generate motion representations (Sec. 3.2).
Our proposed model comprises two primary components: Gaussian Mixture Variational Autoen- coder (GM-VAE) and an AR model. The GM-VAE incorporates causal convolutions to preserve temporal consistency in sequential data processing, along with learnable prior distribution to reg- ularize the latent space. The AR model consists of a text encoder, a causal transformer, a latent sampling module, and PostNet. The latent sampling module applies Gaussian resampling the autoregressively generated latent vectors from the transformer to produce coarse reconstructions, which the PostNet then refines into detailed motion outputs.
3.1 Stage 1: Learning Continuous Motion Representation
Motion data, which describes the movement of different parts of the body, can be highly variable due to the unique motion patterns of individual joints. This complexity makes it difficult to represent data with a single normal distribution. For example, the movement of your feet when walking or running (including speed and rhythm) can differ significantly from the movements of other parts of your body. This motivated to model multi-modal distributions in a continuous latent space to preserve the temporal integrity of motion data (as shown in Figure

---

Learned misture (Ta, Na. Pa. (-1
Cortruous blons
Sampled coanse tikers
Rgguaarbtton
Post Net
GMM Latent ampling
Encoder
Decoder
Text Encader
Causal Transformer
Rcconsmuution
person running"
Loss
(a) Continuous mation representation
(b) Teacher-forcing training with GM samping
The training stage of the proposed GMMotion model. First, VAE is trained to reconstruct motion data. The encoder obtains the posterior distribution of motions, which shaped by learnable Gaussian mixture, and then the decoder generates reconstructed motions from latent features. Next, causal transformer generates latent representations, which are regularized by VAE's shared GMM parameters. Text embeddings and latent representations control the transformer to predict Gaussian variables for masked latent. Finally, continuous latent is restored through Gaussian mixture latent sampling.
Figure 2:
suggest that structured VAEs can effectively train deep models using GMM as a prior distribution, replacing the normal distribution typically used in vanilla VAEs. Inspired by (39), an unsupervised clustering model using Gaussian mixture variational autocncoders, propose the learnable mixture of Gaussians as prior distribution to replace the single normal distribution. The evidence lower bound is:
Learnable GMM as prior. Some works 38
(1) where λ controls the strength of the regularization, (E)))]=)) [log [ogg(=(=)) the log-likelihood of the reconstructed data. Similar to VQ-based models, we obtain the latent vector from the encoder through a deterministic mapping. However, since the posterior is deterministic function and the prior is composed of mixture of Gaussians, directly calculating the difference between these two distributions (known as KL divergence) not straightforward (40). The reconstruction term, therefore, can be estimated by drawing Monte Carlo samples:
ELBO [x(=/x) [lop=x)) AKL gmm.
Σ
KL gmm =KL(g(z)x)
TN Ex)
(2)
G (L(g (x|m E,, 1)),
where denotes the mixing exponent index, is the total number of mixtures, and is a one-hot vector sampled from the mixing probability n, which chooses component from the Gaussian mixture. M is number of samples, (1) = p(t; = 1|2(i)) is the conditional probability of t; being equal when given the sampled latent z ()) The gradient can be backpropagated with the standard reparameterization trick (35). The prior term can be calculated analytically.
KS distance for latent regularization. Our loss definition is based on the Kolmogorov- Smiiroy (KS) test (4T) for equality of one -ie-dmmnsioonal probability distributions. The KS test serves statistical tool to determine whether set of N one-dimensional data points is drawn from specified reference distribution. This determination made by comparing the cumulative distribution function (CDF) of the reference distribution with the empirical CDF F N. which is derived from the observed samples.
For each mode < in the GMM, let u represent the mean, Σ, represent the covariance matrix, and πi represent the weight of that specific mode. The CDFs for univariate Gaussian distributions can be defined
[Mi] [Σ]d
)
FGMM. (2)
Σ
t
1=1

---

and the covariance matrix of the GMM can be computed as:
ΣGMM Σ E
Σ
p1ly x)44 m)
where zz = L it, applying our proposed regularization method to multi-modal GMMs is simple extension. Given d-dimensional latent samples EN; the empirical marginal CDF in dimension j is given by:
N
(2)
(5)
n2
where ll is an indicator function, the primary term in our loss function is specified as:
C 2N) MSE (F) (z)). FGMM,(
Based on a motion- VAE 42), we use an L.2 loss between the ground truth poses x and predictions x. We an L2 loss between the root-centered vertices of the SMPL mesh (43) and predictions v:
Cree (R, | x
vlī
The final loss function includes motion reconstruction and latent space distribution constraints:
Lioss AKSCKS,L
Lese.
(8)
3.2 Stage 2: Learning Autoregressive Latent Sampling with GMM
Considering that the motion representation in the first stage constrained to be continuous and follows a GM distribution, in the second stage, directly generate this distribution from variational inference perspective, thereby avoid- ing the multi-step iterations of diffusion sam- pling (44).
Residual MLP
The model uses causal transformer architecture like T2M-GPT (36). Additionally, we designed Residual-MLP composed of three-layer Mul- tilayer Perceptrons to reorganize the sampled latent representations. Then, the coarse latent representations are refined through a PostNet with residual connections to reconstruct finer latent representations. In the case of Gaussian mixture generative modeling, we no longer need an embedding layer or a softmax layer. Since there no mask/padding token IDs in the tinuous case, we design learnable mask latent and padding latent replace them.
N(O.))
Gumbell Softmae
1
Linear Layer
Figure 3: GMM latent sampling process.
Gaussian mixture latent sampling. We aim for the transformer to predict the parameters of mixture of Gaussian distributions, from which we can sample to obtain latent vectors( Figure B3. We define an autoregressive model for the continuous random variable 2, E RR,, where the conditional probabilities are represented as a mixture of Gaussian distributions:
p(zz 1. z Σ wfN (ZGr).
where wf is the n-th GM's weights, Da and T represent the n-th GM's mean and diagonal variances for generating step 1. The mixture parameters are generated by neural network f(), which takes the previous inputs and conditional information as its inputs:
Wi Vi Ti.r] f(zt- Z1, C"),
(10)
5

---

where C' is the condition embeddings, 2 is the AR transformer latent representations. We use one Linear layer to obtain the weights, means, and diagonal variances. We use the negative log- logllikeihood loss function as:
M CNLL= = Σ
logp (z Hwi (. + KL wt witii N (ul ( ))
MW E().
1=1
(11)
To ensure the mixture weights and variances are valid, we apply softmax to normalize the wt values and softplus to the r¹ values in the network output.
Head pre-padding and learnable rotary position encoding. Motion sequences in the same batch vary in length. Previous AR motion generation models pad shorter sequences with padding tokens at the tail to maintain uniform length and apply absolute position encoding to indicate positional relationships. Inspired by large language models (45), fill padding tokens at the head of shorter sequences and employ learnable relative position encoding (46) to preserve positional relationships. Our goal is to enhance the scalability of the autoregressive model, enabling synthesize longer motion videos even when trained on shorter sequences from the HumanML3D dataset.
On the other hand, adopt the same text-conditioning injection method SALAD (47), where cross-attention module with residual connections is added after each transformer block to inject text embeddings. We use the same text encoder as LAMP (16). Compared embedding text conditions as the first token in the AR iteration process, this approach allows for more flexible control and better compatibility with our head pre-filling method.
4 Experimental Results
4.1 Datasets and evaluation metries
Datasets. To fairly and accurately compare method with the baseline, we used main motion- language benchmarks: KITML. (34) and HumanML33 (33) The KITMI. dataset comprises 3,91 actions from KIT motion data, with each action accompanied by one four text notes (a total of 6,278 notes). The KITML motions are set at 12.5 frames per second (FPS). HumanML3D includes 14,616 actions from the AMASS (48) and HumanAct12 (49) datasets. Each action is described by three text scripts (a total of 44,970 notes). The HumanML3D motions are set at 20 FPS and last up to 10 seconds. We augmented the data by flipping motions and split both datasets into training, testing, and validation sets.
Evaluation metries. We evaluate the generated motions in three aspects: (1) Quality of the generated motions. We use the Frechet Inception Distance (FID) to measure how close the generated motion patterns are to the real ones. (2) Text-motion alignment. We use the Matching Score to measure how well the texts match the generated motions. Additionally, we apply R-Precision to assess how accurately motions can be retrieved based their corresponding texts within set of N motion-texi textt pairs. (3) Motion disversity. MultiModality (MModality) measures the generation diversity conditioned on the same text and Diversity calculates variance through features (33).
4.2 Experimental setup
We use the same CNN-based encoder and decoder as Momask (4). We introduce linear layer after the encoder (the same as (50)) and replace the vector quantization step with a learnable Gaussian mixture distribution. To maintain training stability, we make the mean learnable, initialize the weights with a uniform distribution, and fix the variance to be the identity matrix. The dimension of the 8-layer Causal Transformer is set to 512, with 8 heads and a dropout rate of 0.1, using the GELU activation function. Learnable RoPE embeddings are applied. The diagonal covariance matrices are set to be diagonal.

---

R- P-Prrcision 7
Methods
FID 4
MM Dist
Diversity + MultiModality
Top-1
Top-2
Top-3
Real motion
0.511
0.703%.000
0.797 0013
0.002
2.974
9.503+-003
T2M-GPT 36
0.492
0.679:--00
0.775
0.141
3.1211.008
3.121
9.7221-002
1.831
AET2M (51)
0.499 003
0.6904.000
0.786 001
0.112 005
3.038
9.7004.000
2.452
ParCo [2
0.51 003
0. 0.706
0.801 0012
0.109
2.927
ons
9.576
1.382 060
MoMask
0.521 002
0.713
0.807 002
0.045*.002
2.958 oos
2241+.04
MoGenTS [9
0.529 003
0.719
0.812 2
0. 0.033 0.033+.001
2.867
9.570
LaMP 16
0.557 o03
0.751 #0000
0.843
0.002 002
2.75g
9.571
DiverseMotion 535
0.515 003
0. 0.706
0. 0.802 0012
0.072
2.941
9.68 9.683#t110
1.869 .89
MDM y
0.320 o0s
0. 2.04
0.611 00007
0.54
5.568
9.559
2. 7..799 7999
MLD
0.481 A: 003
0. 0.673 17733-000
0.772
0.473
3.196
0.72g2.082
2. 2.413--009
MotionDifluse 55
0.491
0.681 21:001
0.782
0.630
3.113:1-01
9.410:1-019
1.5534-01
ReMoDiffuse 55
0.510 o0s
0. 1.6988* 1.08t.00s
0,795 0004
0.103
2.97/1
9.018
1. 1.795
Fg-T2M++ (56)
0,513 002
0. 70220000
0.801 000
t.00
2.925
9. 0.223
2 2.625
GMMotion (Ours)
0.572 003
0.761 20000
0.852 001
0.086*.00x
0003
2.743 00s
9.792
2.033 108
Real motion
0.4244.000
0.649 00G
0.779 000
0.031 000
2.788 2.7884-012 012
11.08= 107
T2M-GPT 36
0.416:t.006
0.627=2000
0.745
0.514
3.007
10.92 108
1.570 039
ArT2M [51
0.413
0.632#.006
0.751 #.005
0.870+0039
3.039+-021
10.96 123
2,281
ParCo (22
#..44
0.6-99#.007
0.772#
0,453
2.820
10.93=.004
1.245
MoMask (4
+..03
0.6364.00 0005
0.781 4.005
0.2044.011
2.779
1.131 043
DiverseMotion S3
0.006:::005
0.637=.000
0, 0t
0.468 fogt-000 006
2. 2.892
10.87= 101
2.062 079
MoGenTS [9]
0.4455+.006
0.671# #0000
0.671
0.797
0.143
2. 2.711 2.711
10.92 090
LaMP [66
0. 0.179
0.691 =000
0.826
0,141 #:03
2. 2.704
10.93=.101
MDM 7
0.164 0.16444.004
0.291 0.291#.004 2.004
0.396
0.4974.002
9.191 2.022
10.83=.10h 100
1.907
MLD 8
t.390
0.60g=-000 0008
0.734
0.404
3.204
10.80 117
2.192 071
MotionDiffusc 554
0.417
0.621 #.004
0.739
0.709
1.954
2.958
11.10=:148 148
0.730 013
ReMoDiffuse
555
0.127
0.6-11 #.04
0.765
+.055
0.155
2.814#-012
10.80= 105
1.230
028
Fg. T2+++
(56
0.44224006
0.637
0.7814.004 004
4..354.0004
2.6964.0111
10.99 105
1.255 4 078
GMMotion (Ours)
0 2.005
0. 0.703=.000
0.819
0.198
2.604
11.12 015
1.457
Table 1: Quantitative evaluation results on the test sets of HumanML3D (top) and KIT-ML (bottom).
f and J. denote that higher and lower values are better, respectively, while - denotes that the values closer to the real motion are better. Red and blue colors indicate the best and the second best results.
4.3 Motion representation performance
In Table 2 we compared our motion GM-VAE with other motion tokenizers, such as RVQ-VAE 4. Transformer- VAE (8), and VQ-VAE (36), and found that our model demonsirates superior results in motion reconstruction.
R-Precision f
Methods
FID 4
MPIPE
Top
Top
Top
VQ-VAE
4.001
72.6* 000
0.483=.000 -00
0. 0.680-t.00
0.700
RVQ-VR -VQ
0.029#.001
31.5# 001
0.497 0022
0.693 003
0.791#.002
Trans-VAE
0.0234.00 001
13.7 001
0.499 002
0.695 003
0.791 000
GM-VAE-s (Ours)
0. 0.004++00 +000
9.4
0.514 0022
0. 7.703
0.819#.000 9000
GM- VAE. (Ours)
0.0084.001 001
10.2# 001
0.518
002
0.710 002
0.81
44.002
Table 2: Reconstruction results of latent encoders in our method vs baseline methods(VQQ-AA (36). RVQ 4 and VAE (12) on HumanML3D (33) data. and / mean 128 dims and 512 dims in GMM latent spaces.
We also analyzed how our latent space representation compares with others, as shown in the t-SNE plot (see Figure [D. Constrained by the standard normal distribution, VAE (12) exhibits an unimodal distribution but does not cluster the representations of movements. Meanwhile, due to uneven utilization of the codebook, RVQ (4) captures representations of high-frequency movements but less sensitive to low-frequency movements. Our model not only represents multi-modal motion distributions more effectively but also achieves better clustering of motion data. This is facilitated by the use of Gaussian mixture distributions, which allows the model to capture detailed motion characteristics through unsupervised clustering.
7

---

Ours
lift knees
stride Yorwward ruise right arm
tawardss the opgiad bxhitow walk forward
le down
press on
crawls forward
Jower down
Momask
lift
towards the oppasitss walk forward
raise rigot
crawis forward
mise arm and Jower down
Motion
lift knets
stride torward miise right arm
towards the oppositselbsoo walk forward
Li down crawls forward
press on raise and lower doown
"A person strides forward, raising their right arm."
"A person lifts each knee towards the oppositaelbow while walking forward." forward."
"A person lies down and then slowly crawls
person presses on while raising their left
arm and lower down."
Figure 5: Visualization of qualitative results vs. diffusion based model (12) and VQ based model (4)
The color from light yellow to dark yellow indicates the motion sequence order.
4.4 Text to motion Generation
7) which can be broadly divided into two types: VQ-based models and diffusion- models. The results indicate that GMMotion achieves favorable outcomes in both text alignment and motion quality. In the qualitative analysis, we evaluated the quality of motion synthesis, the alignment between text and motion, and the diversity of motions. For the quantitative analysis, we compared our model's 's motion synthesis results with those of other baselines using the same text instructions.
Quantitative results. We compared model with other state-of-the-ait methods 5 36 12
Additionally, to demonstrate the feasibility of applying our method, we compare average infer- ence time results in Appendix.
Qualitative results. We also visualize our qual- itative results in Figure 5 GMMotion demon- strates more accurate and natural- tions compared to the other models. For in- stance, in the action of striding forward with raised arm, our model captures the movement fluidly, whereas Momask and Motionl LCM ex- hibit some blurriness and less precise limb sitioning. Similarly, the knee-lifting motion, model shows clearer and more realistic leg movements, while the other models struggle with the finer details. Overall, our model outper- forms Momask and MotionLCM in generating coherent and lifelike human movements across different actions.
mo-
nalve
leaeablle
Linear layer
Caussions
Gansssions
Figure 4: Visualization of different GMM settings with t-SNE.
4.5 Ablation Study
focused on two aspects: (1) The effectiveness of the GMM. We explored its reconstruction performance under various conditions, including the number of components and regularization weights. (2) The continuous motion Transformer structure. We investigated how the transformer module, designed for continuous sampling, influences generation outcomes. We also discuss the

---

effectiveness of the generation methods in the Appendix. We examined the combined effects of
different sampling and training approaches.
Ablation of Gaussian mixtures. In Table 3 we examine how the number of components and the loss weight À influence the generative per- formance. It is evident that the reconstruction quality improves significantly when the number of components exceeds one, suggesting that Gaussian mixture distribution has better fitting capabilities than single Gaussian. We also adjusted the weight of the regularization con- straint in the GM-VAF, and the results indicate that the model performs best when the weight 1. In Figure we find that adding linear layer in the vanilla VAE and setting the GMM parameters to be learnable can improve the latent space representation.
No Gous
KS weight
Matching
R-Pre.1
score]
0.1
0.121 t0.01
2,939 2.9039
0.815
0.145
2.942
0.805 1M15
10.0
0,165
3.080
0,799
0:1
0.088 T0000
2.781 7.000
0.841 7.004
1.0
0.0866
2
0
10.0
0.142
2.802 1007
0,827 (08
0.1
0,101 91:002
3.011 f-002
0.801 9.004
1.0
0
3.
0.782 SM14
10.0
0.650
3.556
0.679
Table 3: Text-to-motion results with different Gaus- sian components and KS weights.
Ablation of AR architecture. In Table 4 we present the results of ablation studies on the structure of the AR model. The findings indi- cate that the head pre-padding and RoPE mod- ules improve the quality of motion, while the motion reorganization module has a significant impact the synthesis effect. We believe the motion reorganization module plays crucial role in refining the rough latent representations after sampling. When we removed Gaussian sampling, reverting the model to deterministic sampling, there notable decline in performance. This suggests that the stochastic nature of Gaussian sampling is essential for maintaining high-quality motion synthesis.
Components
FID
Matching score].
R-Pre.1
Full
0.086
2.743 ocs
0.852
who RoPE
0.133
2.851 2.8511.010 d.00
0.823
wdo Res
0.945-.c2
3.423 L.18
0.7524.00 000
w/o GM Samp. who
0.485
3.241
4.014
0.791
0.791 f.004
0.141 10004
2.939 1.000
2.939
0.813 008
Table Results of AR. Where Res the residual net, GM Samp. is the Gaussian mixture sampling stage, and Post the PostNet.
User study. To evaluate the perceptual quality of text-driven motion generation, we conducted a user study with 19 participants comparing our method against two baselines: LaMP (16), which generates motions by discrete AR mod- els; and MotionLCM (50), which generates mo- tions by continuous diffusion models. For each method, participants were presented with 15 video examples and asked to evaluate them based on two criteria: visual quality and text-motion alignment. All ratings were collected using a 5-point Likert scale ranging from 1 (poorest) to 5 (best). The results demonstrate that our model exhibits advantages in terms of visual quality and text-motion alignment.
Methods
Visual Quality
Text-motion Alignment
LaMP 16
3.9624.111
3.775 186
MotionLCM 50
3.118
3.219
GMMotion (Ours)
4.392
4.121 4 mss
Table 5: User study results.
Conclusion
We introduced GMMotion, a novel text-to-motion synthesis framework that employs continuous motion representation and GMM to capture multimodal human motions. GMMotion streamlines training and inference by avoiding vector quantization, instead sampling from learnable GMMs the latent space. Our two-stage model-finn modeling motion sequences into multimodal distributions with GMM, then using a causal transformer for efficient generation outperrornssss existing models in quality and alignment.
Limitations. While GMMotion achieves efficient motion synthesis through autoregressive genera- tion, the sequential nature of the process may occasionally lead to minor error accumulation over long sequences. Early prediction inaccuracies (e.g., subtle joint angle deviations) could propagate temporally, potentially affecting the smoothness extended motions.

---

Acknowledgments
This work was supported in part by the Natural Science Foundation of China under Grant 62372203 and 62302186, in part by the Major Scientific and Technological Project of Shenzhen (2023 10211 in part by the National key research and development program of China(20222FF22611 by the Major Scientific and Technological Project of Hubei Province (2022BAA046, 2022BAA042).
in part
References
[1] L. Siyao, W. Yu, T. Gu, C. Lin. Q. Wang. Qian, C. C. Loy, and Z. Liu, "Bailando: dance generation by actor-critic gpt with choreographic memory." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 11 050-11 059.
[2] Z. Zhang, A. Liu, I. Reid, Hartley, B. Zhuang, and H. Tang, "Motion mamba: Efficient and long sequence motion generation with hierarchical and bidirectional selective ssm," arXiv preprint arXiv:2403.07433 2024.
[3] H. Kong. K. Gong, Lian, M. Mi, and Wang. "Priority-centric human motion generation discrete latent space," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp.
14 886-148811 816.
[4] C. Guo, Mu, M. G. Javed, S. Wang, and L. Cheng, "Momask: Generative masked modeling human motions," " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1910-
[5] E. Pinycanuntapong, P. Wang, M. Lee, and Chen, "Mmm: Generative masked motion model," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[6] A. Van Den Oord, O. Vinyals et al., "Neural discrete representation learning" " Advances in neural information processing systems, vol. 30, 2017.
[7] G. Tevet, S. Raab, B. Gordon, Y. Shafir, D. Cohen-or, and A. H. Bermano, "Human motion diffusion model," in The Eleventh International Conference on Learning Representations, 2023. [Online]. Available: htnps//ognneeeewwn////ouuuttssss 1SyO22ju
[8] X. Chen, B. Jiang, W. Liu, Z. Huang, B. Fu, T. Chen, and G. Yu, "Executing your commands via motion diffusion in latent space." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 18 0800180000.
[9] H. Liang, J. Bao, R. Zhang, S. Ren, Y. Xu, S. Yang. X. Chen, J. Yu, and L. Xu, "Omg: Towards open- vocabulary motion generation via mixture of controllers," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 482-493.
[10] K. Karunratanakul, K. Precchakul, E. Aksan, T. Beeler, S. Suwajanakom, and S. Tang, "Optimizing diffusion noise can serve universal motion priors," in arxiv:2312. 19944 2023.
[11] C. Felce, S. Liorsdóttir, and L. Pachter. "The virial theorem and the price equation." 2024. [Online]. Available:
[12] Dai, L.-H. Chen, Wang. Liu, Dai, and Y. Tang, "Motionlem: Real-time time controllable motion generation via latent consistency model," arXiv preprint arXiv:2404. 19759, 2024.
[13] G. Barquero, S. Escalera, and C. Palmero, "Seamless human motion composition with blended positional encodings," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[14] Q. Zhang, J. Song, X. Huang, Y. Chen, and M.-Y. Liu, "Diffcollage: Parallel generation of large content with diffusion models," in 2023 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR). IEEE, 2023, pp. 188-10198,
[15] Y. Shafir. G. Tevet, R. Kapon, and A. H. Bermano, "Human motion diffusion a generative prior," arXiv preprint 2303.02003. 2023.
[16] Z. Li, W. Yuan, Y. He, L. Qiu, S. Zhu, X. Gu, W. Shen, Y. Dong. Z. Dong, and L. T. Yang. "Lamp: Language motion pretraining for motion generation, retrieval, and captioning." 2025. [Online]. Available: https///rriv.org/abs/// http://wwrr/g/b///1000000// 2410.07093
[17] S. Chen, S. Liu, L. Zhou, Y. Liu, X. Tan, J. Li. S. Zhao, Y. Qian, and F. Wei, "Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers." 2024. [Online]. Available: https//aniiv.oggadb//44.....5.7
10

---

[18] S. Lu, J. Wang, Z. Lu, L.-H. Chen, W. Dai, J. Dong. Z. Dou, B. Dai, and R. Zhang, "Seamo: Exploring the scaling law in autoregressive motion generation model," 2024. [Online]. Available: https//arii..or/ab///22...2
[19] W. Yuan, W. Shen, Y. He, Y. Dong, X. Gu, Z. Dong. L. Bo, and Huang. "Mogents: Motion generation based on peaporll joint modeling." 2024. [Online]. Available: https://arii..or//a///440...
[20] K. Karunratanakul, K. Preechakul, S. Suwajanakom, and S. Tang, "Guided motion diffusion for controllable human motion synthesis," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 2151-216-
[21] O. Bar-Tal, Yariv, Y. Lipman, and T. Dekel, "Multidiffusion: Fusing diffusion paths for controlled image gencration." " Proceedings of Machine Learning Research, vol. 202, pp. 1737-1752, 2023.
[22] Z. Xie, Wu, X. Gao, Z. Sun, Yang. and X. Liang. "Towards detailed rext-to-motinn synthesis via basic- to-advanced hicrarchical diffusion model," 2023. [Online]. Available: httpss//arxxw.orr/ab///22
[23] A. Radford, Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, "Language models are unsupervised multitask learners," 2019.
[24] L. Yu, J. Lezama, N. B. Gundavarapu, L. Versari, K. Sohn, D. Minnen, Y. Cheng, V. Birodkar, A. Gupta, X. Gu, A. G. Hauptmann, B. Gong, M.-H. Yang, I. Essa, D. A. Ross, and L. Jiang, "Language model beats diffusion tokenizer is key to visual generation," 2024. [Online]. Available:
hutps//araiv.orr/aa//231111.3330
[25] P. Esser, R. Rombach, and B. Ommer, "Taming transformers for high-resolution image synthesis," 2021. [Online]. Available:
[26] L. Fan, T. Li, S. Qin, Y. Li, C. Sun, M. Rubinstein, D. Sun, K. He, and Y. Tian, "Fluid: Scaling autoregressive texi-to-image generative models with continuous tokens," 2024. [Online]. Available: https:/aaniv.orr//s/21111..033
[27] T. Li, Y. Tian, H. Li, M. Deng. and K. He. "Autoregressive image generation without vector quantization," 2024. [Online]. Available: https://arxiv.or//ab////0000 11838
[28] Y. Sun, H. Bao, W. Wang. Z. Peng, L. Dong. S. Huang. J. Wang, and F. Wei, "Multimodal latent language modeling with next-token diffusion." " 2024. [Online]. Available:
[29] L. Meng, L. Zhou, S. Liu, S. Chen, B. Han, S. Hu, Y. Liu, J. Li, S, Zhao, X. Wu, H. Meng. and F. Wei, "Autoregressive speech synthesis without vector quantization." " 2024. [Online]. Available: http://anii..orr//bb//447....55
[30] V. M. Spitzer and D. G. Whitlock, "The visible human dataset: the anatomical platform for human simulation." " The Anatomical Record: An Official Publication of the American Association of Anatomists, vol. 253, no. 2, pp. 49-57, 1998.
[31] R. Bowden, "Learning statistical models of human motion," in IEEE Workshop on Human Modeling, Analysis and Synthesis, CVPR, vol. 2000, 2000.
[32] A. Galata, N. Johnson, and D. Hogg. "Learning variable- length markov models of behavior," Computer Vision and Image Understanding, vol. 81, no. 3, pp. 398-413, 2001.
[33] C. Guo, S. Zou, X. Zuo, S. Wang, W. Ji, X. Li, and L. Cheng, "Generating diverse and natural 3d human motions from text." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5152-5161.
[34] Plappert, C. Mandery, and T. Asfour, "The kit motion-language dataset," Big data, vol. 4, no. 4, pp. 236-252. 2016.
[35] D. Kingma and Welling, "Auto-encoding variational bayes." 2014. [Online]. Available: https://axxv.orr/ab//III 1312611
[36] Zhang, Y. Zhang, X. Cun, S. Huang. Y. Zhang, H. Zhao, Lu, and X. Shen, -T2m-gpt: Generating human motion from textual descriptions with discrete representations"" in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[37] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, "Learning transferable visual models from natural language supervision," in ICML, 2021.

---

[38] I. Higgins, L. Matthey. X. Glorot, A. Pal. B. Uria, C. Blundell, S. Mohamed, and A. Lerchner. "Early visual concept learning with unsupervised deep learning," " 2016. [Online]. Available:
https//aaxi..or///////6....... 1o0.abb5/666600055799
[39] N. Dilokthanakul,, P. A. M. Mediano, M. Garnelo, M. C. H. Lee, H. Salimbeni, K. Arulkumaran, and Shanahan, "Deep unsupervised clustering with gaussian mixture variational autoencoders." 2017.
[Online]. Available:
[40] A. Saseendran, K. Skubch, Falkner, and M. Keuper, "Shape your space: A gaussian mixture regularization approach to deterministic autoencoders," in Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin. P. Liang. and J. W. Vaughan. Eds.. vol. 34. Curran Associates, Inc., 2021, pp. 7319 7332. [Online]. Available: https
[41] R. H. Lopes, I. Reid, and P. R. Hobson, "The two-dimensional kolmogorov- smirnov test," 2007.
[42] M. Petrovich, M. J. Black, and G. Varol, cAniioooccccooniioonneeee 3d human motion synthesis with transformer vac," 2021. [Online]. Available: hutps://arrtiworrr////2//0.....0
[43] M. Loper, N. Mahmood, J. Romero, G. Pons- Moll, and M. J. Black, "Smpl." ACM Transactions on Graphics, p. 1-16, Nov 2015. [Online]. Available: htt../o.....or/11111111119999999 20119113
[44] Z. Meng. Y. Xie, X. Peng. Z. Han, and H. Jiang, "Rethinking diffusion for text-driven human motion generation," 2024. [Online], Available: https://aniiv.or//a////11111111 16575
[45] J. Xu. Z. Guo. J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang. Y. Fan, K. Dang. B. Zhang. X. Wang, Y. Chu, and J. Lin, "Qwen2.5- ommi technical report." 2025. [Online]. Available:
[46] J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu, "Roformer: Enhanced transformer with rotary position embedding," 2023. [Online]. Available: https//aaxiioorr//b///1.......
[47] S. Bae, H. Kim, Y. Choi, and J. Lee, "Salad: Improving robustness and generalization through contrastive learning with structure-awes -wrrr and llm-driven augmented data," 2025. [Oaline]. Available: hutps://ariv.orr//as//55444 12185
[48] N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. Black, "Amass: Archive of motion capture as surface shapes." in 2019 JEEE/CVF International Conference on Computer Vision (ICCV), Oct 2019. [Online]. Available: httt:://bx.doi.orr/lu w1wtee/w/c/..w./111.....o 000544
[49] C. Guo, Zuo. Wang, S. Zou, Q. Sun, A. Deng, M. Gong, and L. Cheng, "Action2motion: Conditioned generation of 3d human motions," in Proceedings of the 28th ACM International Conference on Multimedia, Oct 2020. [Online]. Available: http:/dx..oii
[50] W. Dai, L.-H. Chen, Y. Huo, J. Wang. J. Liu, B. Dai. and Y. Tang. "Motionlem-v22 Improved compression rate for multi-latent-token diffusion," December 2024. [Online]. Available: http://ggggggtaoooooo/oooo///o/o
[51] C. Zhong, L. Hu, Z. Zhang, and S. Xia, "Att22m: Text-driven human motion generation with multi- perspective attention mechanism." 2023. [Online]. Available: httpss//aarxivoorr////////////00
[52] Q. Zou, S. Yuan, S. Du, Y. Wang, C. Liu, Y. Xu, J. Chen, and X. Ji, "Parco: Part-coordinating text-to-motion synthesis," 2024. [Online], Available: https://axii.org//b///33...
[53] Y. Lou, L. Zhu, Y. Wang, X. Wang, and Y. Yang. "Diversemotion: Towards diverse human motion generation via discrete diffusion." 2023. [Online]. Available: https://axxiv.rr//bb/////
[54] M. Zhang, Z. Cai, L. Pan, F. Hong, X. Guo, L. Yang, and Z. Liu, "Motiondiffuse: Text-driven human motion generation with diffusion model," arXiv preprint arXiv:2208.1 15001, 2022.
[55] M. Zhang. X. Guo, L. Pan, Z. Cai, F. Hong. H. Li, L. Yang, and Z. Liu, "Remodiffuse: Retrieval- motion diffusion model," arXiv preprint 11X1v:200..01116, 2023.
[56] Y. Wang. M. Li. J. Liu, Z. Leng. F. W. B. Li, Z. Zhang, and X. Liang. "Fg-t2m++: augmente fine- fine-grained text driven human motion generation." 2025. [Online]. Available: https://ari..oggalbz25522.555..
12

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We demonstrate our novel GMM-based design.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the impact of the error accumulation process.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

Justification:
We have provided a thorough theoretical explanation for GMM sampling.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The code will be made publicly available after the review process.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification:
The code will made publicly available after the review process.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pipps.cc/pullc//uuui wwwtc//ggui////eeeeceoocc CoodeSubeiiieeee ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We repeated the description of the experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We presented the results in detail.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
15

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided the relevant content in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http:///eeiis..eeeeeeeee http:/is....puuulc/c/tttcccualaa
Answer:
[Yes]
Justification:
We have included the relevant content.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed the content related this issue.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
16

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We discussed the content related to this issue.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We discussed the content related to this issue.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee www.ree.wthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
17

---

Answer: [Yes]
Justification: we communicate the details of the model as part of submissions via structured templates.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification:
We discussed the content related to this issue.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification:
We discussed the content related this issue.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMSs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality the research, declaration is required.
Answer: [Yes]
18

---

Justification:
We used LLMs in accordance with the conference guidelines.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy http:ees/heerrppcoccccooootrrere for what should or should not be described.
19


Reviews:
[{"review_id": "F4rCu7OGC6", "reviewer": "Reviewer_QPHE", "summary": "The paper addresses the challenge of text-to-motion generation and introduces GMMotion, a novel framework that combines a Gaussian Mixture Model (GMM)-based motion compression VAE with an autoregressive transformer. This design circumvents limitations in prior approaches: unlike VQ-VAEs, which introduce discretization errors, the proposed continuous latent representation avoids quantization altogether; and compared to standard VAEs with unimodal Gaussian priors, the use of a GMM better captures the inherently multimodal nature of human motion.\n\nGMMotion operates in two stages: (1) learning a continuous, structured latent space through a GMM-based VAE (GM-VAE) regularized by the Kolmogorov-Smirnov distance, and (2) training an autoregressive causal transformer that models these latent representations and samples from the learned GMM to generate motion sequences. This approach enables prompt-conditioned generation without predefining output length.\n\nEmpirical results show the superiority of the method: the GM-VAE outperforms alternative motion tokenizers (including VQ-VAEs and RVQ-VAEs) in reconstruction metrics, and the full generative pipeline produces higher-quality, better-aligned, and more diverse motions across multiple automatic metrics (e.g., FID, R-Precision) and human evaluations. The model also demonstrates faster inference than diffusion-based methods, making it both effective and efficient.", "questions": "Q1: The method advocates for using a GMM prior in the VAE. Could you elaborate on why the task of human motion generation specifically benefits from such a prior, as opposed to other generative modeling domains like image, video, or audio generation, where the use of a GMM prior in VAEs is relatively uncommon? These tasks similarly have a multi-modal nature (e.g. different kinds of audio - speech, music, ambient noises).\n\nQ2: In many generative models that use a VAE as a pre-processing stage, the predicted variances tend to collapse to very small values (often due to low KL loss weights), effectively making the VAE behave like an autoencoder. In your case, the diagonal variances are fixed to 1. Doesn’t this introduce significant noise into the latent representations? Alternatively, do the mean values compensate by increasing in magnitude? It would be helpful to clarify how this design choice affects the latent space behavior and model stability. Please also provide some analysis about the magnitude of the predicted means.\n\nQ3: Table 3 shows that the model performs best with only 4 Gaussian components, while using 8 components leads to worse performance than a single Gaussian. How do you explain this? Intuitively, I would expect a more expressive multimodal prior to benefit from a larger number of components. Additionally, could you report performance with 2 components? It would also be helpful to include statistics about the learned GMM (e.g., entropy of the component weights or effective number of components used) to better understand how the model utilizes the mixture.", "limitations": "There is some discussion about the limitations, in section 5.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nClarity: The paper is very well structured, clear, and easy to follow. In my opinion, there is a good balance between motivating the problem, providing the required background, formulating the solution, describing the experiments, and presenting a thorough ablation study.\n\nQuality: The method appears to be technically sound, and the empirical evidence convincingly demonstrates its success in improving the quality of text-to-motion generation. The ablation study in Section 4.5 further contributes to understanding the role of different components in the proposed method.\n\nOriginality: To the best of my knowledge, the idea of combining a continuous VAE with a GMM prior in the context of an autoregressive model for motion prediction is novel.\n\n**Weakness**\n\nI have some questions and concerns about the proposed method, which may affect its overall significance (and also caused me to reduce my Quality rating for the paper). These concerns focus on how effectively the Gaussian Mixture Model (GMM) prior is utilized in the method. Specifically:\n- Table 3 shows that the best performance is achieved with only 4 components, and using 8 components actually performs worse than a single Gaussian. This raises concerns about whether the GMM prior is being fully leveraged. It is unclear if this reflects a limitation in training stability, an issue of overfitting, or whether the added expressivity of larger mixtures does not translate to meaningful gains in this setting.\n- The covariance matrices are fixed as identity matrices (i.e. all diagonal variances are 1).\n\nPlease see expanded questions in the section below.", "was_revised": true, "final_justification": "After reading the authors' rebuttal and the other reviews, I opt for keeping my initial \"accept\" rating.", "rebuttal": "We thank reviewer QPHE for the constructive suggestions and thorough review.\n\nThe reviewer has provided detailed thoughts on our GMM design and raised many interesting questions. We acknowledge and appreciate the reviewer's inquiries and feedback, and will include corresponding content and discussions in the revised version.\n\n---\n\n**[Q1]**: *Why is a GMM prior beneficial for human motion generation in VAEs?*\n\n- *Human motion sequences are continuous time series with specific multimodal distributions.* The representation pattern of human motion is similar to the Mel-spectrogram in speech representation, but each dimension has a specific meaning (e.g., the relative coordinates of the left elbow joint, the angular velocity of the right knee joint). We observed that the motion of each dimension has specific patterns and constraints (e.g., the human forehead cannot rotate 360 degrees). Therefore, unlike sound synthesis and image synthesis, human motion representation has a unique motion distribution. After visualization, we found that it resembles a mixture of Gaussian distributions.\n\n - *We aim to avoid learning motion patterns through the diffusion denoising process.* Current state-of-the-art methods use diffusion models for motion synthesis and have achieved good results. Many of these methods draw inspiration from image generation and speech synthesis, sometimes overlooking the temporal characteristics and spatial regularities of human motion. We believe treating human motion as an image or a noise sequence introduces redundant steps (for example, motion diffusion models require multiple sampling steps, which may introduce additional computational overhead).\n\n  - *We believe that GMM also holds potential in speech and image synthesis, it is suitable for modeling continuous tokens.* In the first stage of training, a VAE is typically trained for content compression. In practice, the KL divergence loss weight is often constrained to a very small value, making the VAE function more like an autoencoder, where the KL loss primarily serves to limit the numerical range of the latent vectors. In the second stage, diffusion models are frequently employed to learn the noise distribution, mainly because the original time series and images do not exhibit clear distributional patterns. Inspired by MAR [1], an increasing number of approaches （MELLE [2], GMM-LM [4], DiCoDe [3]） are attempting to impose strong constraints on the latent space and then perform sequence synthesis through a Gaussian sampling head. Therefore, we believe that Gaussian Mixture Models remain a promising direction worth exploring.\n\n---\n\n**[W2/Q2]**: *The VAE's diagonal variances are fixed to 1,  doesn't this introduce significant noise into the latents? Or do the mean values compensate by growing larger?*\n\n - *The diagonal variances are fixed to 1 to provide uncertainty.* Similar to your observation, when we make the variances learnable, the model exhibits a \"laziness\" behavior, causing all variances to converge towards 0. We share the same perspective as DiCoDe: even though the text-to-motion task is deterministic, we can explicitly introduce variance into the target. Introducing variance allows for better capture of the uncertainty inherent in the deep latent distribution, which is crucial for generation diversity.\n\n - *The mean magnitudes of different components vary to adapt to specific motion distributions.* In our reparameterization design, each dimension in the latent space corresponds to an independent GMM. Taking our 128-dimensional model as an example, we analyze the GMMs at dim 1, 32, and 64, as shown in the table below. It can be observed that while the magnitudes of some Gaussian components' means increase, the majority fluctuate around 0.\n\n| position | Number of Gaussians | Mean Value |\n| --- | --- | --- |\n| dim=1 | 1 | [0.1628] |\n| dim=32 | 1 | [-0.1259] |\n| dim=64 | 1 | [0.1924] |\n| dim=1 | 4 | [-0.3429, 0.7895, -0.0567, 0.9218] |\n| dim=32 | 4 | [0.4462, -0.1294, 1.6289, -0.8481] |\n| dim=64 | 4 | [-0.2350, 0.5608, 0.1413, -0.4146] |\n| dim=1 | 8 | [0.3429, -0.7695, 1.5345, 0.0467, -0.9218, 0.6989, -0.1334, 0.4762] |\n| dim=32 | 8 | [0.0042, -1.0907, 0.0902, -0.2431, -0.2554, 0.2300, 0.4215, -0.9123] |\n| dim=64 | 8 | [-2.6504, 0.1296, -0.0042, -0.8976, 0.7694, -3.4231, 0.0008, 0.0321] |\n\n---\n\n**[W1/Q3]**: *Why does performance degrade with more components, contrary to the expectation that greater expressiveness helps? Could you also report results with 2 components and include GMM usage statistics?*\n\n- *More number of Gaussian mixture components can lead to more uncertainty.* Since we use GMM as a learnable prior (while VAEs generally use a Gaussian distribution as a non-learnable prior), a larger number of Gaussian components allows latent vectors more freedom to select different components, which weakens the constraining power of the GMM prior. More detailed results (we add GMM with 2, 6 components) are shown in the table below. We will include visualizations of the utilization of different GMM components in the next version to further illustrate the above points.\n\n- *GMM can be somewhat analogous to the MoE structure in LLMs, where an excessive number of components leads to increased model complexity.* As the components number increases, the proportion of parameters allocated to each component decreases, and the learning difficulty increases. Our method assigns a Gaussian component for reparameterization based on the probability proportional to the weight during the sampling process, a design similar to the routing mechanism and expert models in MoE. An excessive number of components would result in an increase in model parameters and computational load, making training more unstable. Considering this, we plan to supplement the training loss curves in the next versions.\n\n\n| Number of Gaussians | FID ↓ | Matching score ↓ | R-precision ↑ |\n| ------------------- | ----- | ---------------- | ------------- |\n| 1                   | 0.145 | 2.942            | 0.805         |\n| 2                   | 0.097 | 2.801            | 0.829         |\n| 4                   | 0.086 | 2.743            | 0.852         |\n| 6                   | 0.139 | 2.861            | 0.815         |\n| 8                   | 0.196 | 3.110            | 0.782         |\n\n- We select the same textual instruction to perform inference when the number of components is 4 or 8, and observe the weights corresponding to the dim at 0. The results of GMM weight are\n\ncomponents=2 : [0.5423, 0.4577], \n\ncomponents=4 : [0.2849, 0.2163, 0.2733, 0.2255], \n\ncomponents=8 : [0.1146 0.1348, 0.1053, 0.1069 0.1379, 0.1199, 0.1494, 0.1312].\n\nHere, the results are obtained by printing the GMM sampling head parameters. Although somewhat incomplete, they indicate that the GMM weight distribution is roughly uniform. In future versions, we plan to provide results with higher visualization quality.\n\n---\nReviewer prompted us to further explore GMM's potential and limitations in human motion generation. We found that too many components can degrade performance, and fixing variance to 1 requires balancing uncertainty and training stability. We will add discussion and further experiments in the next version.\n\n---\n### Reference\n\n[1] Li T, Tian Y, et al. Autoregressive Image Generation without Vector Quantization. Neurips 2024.\n\n[2] Meng L, et al. Autoregressive Speech Synthesis without Vector Quantizatio. ACL 2025.\n\n[3] Li et al. DiCoDe: Diffusion-Compressed Deep Tokens for Autoregressive Video Generation with Language Models.\n\n[4] Lin et al. Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesis. ICLR 2025."}, {"review_id": "z85FnLIVmZ", "reviewer": "Reviewer_kPaJ", "summary": "This paper proposes a new text-to-motion generation framework, which incorporates GMM latent space and Auto-regressive modeling to achieve AR generation scheme without discrete tokenization.", "questions": "Please kindly refer to the weaknesses part.", "limitations": "Limitations have been well discussed.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n1. This paper includes several valuable ablation studies, which is beneficial for the community.\n2. The implementation details are described in a clear and systematic manner, making the overall behavior of the proposed method easy to follow and reproduce.\n\nWeaknesses\n\nMy primary concerns lie in two areas: the novelty of the proposed generative approach and the overall generation quality.\n\n1. From the perspective of novelty, the use of a GMM latent space combined with an autoregressive (AR) decoder is not new, and has already been explored in other domains such as video generation (e.g., DiCoDe[1]) and speech synthesis (e.g., GMM-LM[2]). The current method appears to largely transfer these ideas to the text-to-motion domain without incorporating domain-specific adaptations or insights that account for the unique structure or semantics of motion data.\n2. Regarding performance, while the authors claim that discrete token representations introduce significant quantization errors, the proposed method does not show a clear advantage in generation quality over existing approaches. In particular, it underperforms both discrete-token-based methods such as MoMask and its successors, and continuous representation methods like MotionLCM-v2, in terms of overall motion realism and diversity. Although the proposed model achieves the best R-Precision, I would caution against overemphasizing this metric as a primary indicator of generation quality. R-Precision values surpassing the ground truth may suggest that the generated results closely resemble samples from the training set, which may reflect mode memorization rather than true semantic alignment or diversity. Moreover, the metric can disproportionately benefit generations that are statistically similar to frequent training patterns, making it less reliable for evaluating generalization or creativity.\n3. Additionally, the claim in Appendix D.1 that AR exhibits strong generative capacity appears to conflict with results in Table 6, where the AR variant shows no clear advantage over MAR. In fact, when the latent dimension is set to 128, MAR significantly outperforms AR, which undermines the claim of AR’s superiority.\n\n[1] Li et al. DiCoDe: Diffusion-Compressed Deep Tokens for Autoregressive Video Generation with Language Models\n[2] Lin et al. Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesis", "was_revised": true, "final_justification": "The authors' rebuttal has addressed most of my concern. Therefore I vote for borderline accept.", "rebuttal": "We would like to thank the reviewer for their thoughtful comments.\n\n---\n\n**[W1]** :  *The novelty of the proposed generative approach. there have been similar works in the fields of video generation and speech synthesis.*\n\nOur motivation for choosing autoregressive motion synthesis within the GMM latent space is as follows: In the domain of human motion, the movement characteristics of a specific joint often exhibit a multimodal distribution (as shown in Figure 1.a, in the violin plot, it is clearly visible that human motion follows a multi-modal distribution, unlike images and audio.). GMMs are inherently well-suited for modeling such multimodal distributions. This is different from the motivation of DiCoDe and GMM-LM. We recognize and appreciate these two excellent works—GMM-LM has already been cited in this paper, and we will include related work in the updated version.\n- In DiCoDe, the Gaussian Mixture Model is used to model the uncertainty of deep tokens in the continuous value space and learns the variance during the autoregressive process. **It still requires a diffusion model to complete the video generation.**\n- In GMM-LM, it estimates the KL divergence through Monte Carlo sampling, whereas our approach adopts the KS distance for regularization.\n- In our method, our autoencoding process keeps the time sequence of human motion and uses Gaussian mixture models for spatial relationships, specifically designed for text to human motion generation.\n\nOn the other hand, we agree that it’s valuable to adapt and further design outstanding works from similar domains to fit the characteristics of motion:\n\n- T2M-GPT [1] (text to motion) draws inspiration from VQGAN [2] (image generation), utilizing VQ-VAE + Generative Transformer to achieve autoregressive motion synthesis.  \n- Momask [3] (text to motion) references VALL-E [4] (audio synthesis), employing RVQ with two transformers to model motion features at both the base layer and residual layer, achieving excellent generation results.  \n\n- MotionLCM [5] (text to motion) takes cues from the application of consistency models [6] in image generation, designing a one-step sampling method for motion diffusion models, which significantly accelerates the sampling speed.\n\nThus, there have been many successful cases where excellent algorithms from external domains (image generation, speech synthesis, video generation) were adapted to achieve good results. To the best of our knowledge, we are the first to introduce GMM sampling into the motion generation field. We have also been exploring whether new methods can replace diffusion models or VQ-VAE. In this work, we initiated such an exploration.\n\n---\n\n**[W2]** :  *The overall generation quality.* \n\n- *It's not just R-Precision， our model surpasses existing sota models across many metrics.*\n\nCompared to discrete-token-based methods such as MoMask [3], our model shows clear gains in R-Precision↑ (0.852 vs 0.807), MM-Dist↓ (2.743 vs 2.958), and MultiModality↑ (2.033 vs 1.241). When compared to continuous representation methods like MotionLCMv2 [5], our model also demonstrates significant improvements in R-Precision↑ (0.852 vs 0.837), MM-Dist↓ (2.743 vs 2.773), and MultiModality↑ (2.033 vs 1.758). Furthermore, our model leads in most metrics when compared to T2M-GPT [1], another autoregressive generative model. In the first-stage motion representation, RVQ used by Momask, etc. (FID=0.029) performs worse in motion reconstruction than our method (FID=0.008), which further supports that continuous motion representation achieves better reconstruction.\n\n- *R-Precision still remains a meaningful reference for evaluating text-motion alignment.* \n\nThe calculation of R-Precision involves mixing the text description corresponding to the target motion with 31 other negative text samples from the test set, then using a text feature extractor trained from HumanML3D [7] to obtain text vectors for matching with the target motion. As noted in MoMask and MARDM [8], this feature extractor has been in use for almost four years, and it is indeed possible for the R-Precision value to exceed that of the ground truth. \n\nThe controversy surrounding R-Precision arises more from the relatively outdated feature extractor used for evaluation. We acknowledge your concern regarding potential mode memorization due to certain historical reasons. To address this, MARDM [8] introduced the HumanML3D dataset with non-redundant data and trained a new feature extractor. Results on this new dataset further validate the overall generation quality of our model.\n\n### Table 1 Text to motion results on MARDM's new HumanML3D benchmark\n\n| Methods | R-precision | FID       | Matching score | MModality |\n| ------- | ----------- | --------- | -------------- | --------- |\n| Momask  | 0.786       | 0.116     | 3.353          | 1.263     |\n| MARDM   | 0.795       | 0.114     | 3.270          | 2.231     |\n| Ours    | **0.799**   | **0.103** | **3.259**      | **2.427** |\n\nOn the new dataset proposed by MARDM, the evaluator has been retrained to better suit current state-of-the-art models. Under this dataset, our method outperforms both Momask and MARDM.\n\n---\n\n**[W3]**: *Advantages of the AR approach over the MAR approach*\n\n- *With all other hyperparameters being equal, our model (AR) has only half the number of parameters compared to the MAR approach.*\n\nThe specific MAR structure we refer to in Appendix D.1 follows MARDM, which consists of a Transformer autoencoder and a Transformer autodecoder, implementing masking and pseudo-reordering between them.\n\n- *The benchmark used in the appendix is the MARDM-HumanML3D with redundant parameters removed, rather than the original HumanML3D.*\n\nMARDM argues that a large portion of the human motion parameters in the original HumanML3D are not effectively utilized (about over 60%), and thus removes these redundant features, further reducing data abundance. This reduction may make it more difficult for small-parameter models to learn complex motion synthesis.\n\nTherefore, we believe the reason the AR approach underperforms when the dimension (dim) is set to 128 is due to the model having too few parameters, combined with lower data abundance in this specific dataset, making it difficult to learn motion patterns. Additionally, the AR approach performs better than MAR when the dim is set to 512. We see strong potential in the AR approach for long-motion synthesis and continuous motion generation, and believe it holds future promise for integration with motion LLM-style models.\n\n-----\n\nOverall, we hope this paper can inspire new thinking about text-to-motion synthesis, potentially paving the way for future approaches that could replace either continuous diffusion or discrete AR generation. One key reason we explored the AR approach is to align motion synthesis models with the autoregressive frameworks common in large language models. Additionally, through designs like rotary position encoding, we aim to preserve the ability to generate long sequences, hoping to offer a novel method for motion synthesis.\n\nWe hope our response improves your perception of this work, and welcome any further questions or discussions.\n\n---\n### **Reference**\n[1] Zhang J, Zhang Y, Cun X, et al. Generating human motion from textual descriptions with discrete representations. CVPR 2023.\n\n[2] Esser P, Rombach R, Ommer B. Taming transformers for high-resolution image synthesis. CVPR. 2021.\n\n[3] Guo C, Mu Y, Javed M G, et al. Momask: Generative masked modeling of 3d human motions. CVPR 2024.\n\n[4] Wang C, Chen S, Wu Y, et al. Neural codec language models are zero-shot text to speech synthesizers.arXiv:2301.02111.\n\n[5] Dai W, Chen L H, Wang J, et al. Motionlcm: Real-time controllable motion generation via latent consistency model. ECCV 2024.\n\n[6] Luo S, Tan Y, Huang L, et al. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv:2310.04378.\n\n[7] Guo C, et al. Generating Diverse and Natural 3D Human Motions From Text. CVPR 2022.\n\n[8] Meng Z, Xie Y, Peng X, et al. Rethinking diffusion for text-driven human motion generation. CVPR 2025."}, {"review_id": "RzPyGZO9Ud", "reviewer": "Reviewer_Nig7", "summary": "To address the motion errors introduced by the discretization process in generative transformers as well as the slow sampling of diffusion models, this paper presents a text-to-motion synthesis method GMMotion which aims to achieve autoregressive motion generation and leverage Gaussian mixture model for the representation of the conditional probability distribution. The paper shows comprehensive evaluations and the proposed method achieves SOTA performance.", "questions": "1. Please provide the results of training and inference times to fully verify the proposed method.\n2. Please explain why the proposed method has relatively low motion disversity (i.e., MultiModality).", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-motivated and significant. The motion errors in generative transformer and the slow sampling of diffusion model are two significant and valuable issues in text-to-motion synthesis. \n2. The proposed method is quite novel. It leverages VAE’s latent space and Gaussian mixture sampling for continuous motion representations to reduce the discretization error.\n3. The authors conduct comprehensive experimental evaluations and ablation studies, and effectively demonstrate the superiority of the proposed method.\n\nWeaknesses:\n1. Lack of time comparisons in the experiments. As one of the motivation is the slow sampling in diffusion model, the authors should provide the results of training and inference times to fully verify the proposed method.", "was_revised": true, "final_justification": "After reading the author rebuttal, my cncerns have been addressed. So I keep my score (Accept).", "rebuttal": "We thank the reviewer Nig7 for their careful and constructive review.\n\nWe agree with the reviewer's suggestion. Since one of our motivations is to improve the sampling speed of diffusion models, it is important to provide inference speed metrics in the experimental results section of the main text. Additionally, we will further supplement the training details, including the type of GPU used for training and the training duration, to enhance the completeness of our experiments.\n\n-----\n\n**[W1/Q1]**: *Provide the results of training and inference times to fully verify the proposed method.*\n\n*   **Training Time:**\n    We use 4 NVIDIA RTX 4090s to complete the experiments. The training of the GM-VAE takes approximately 14.5 hours. Specifically, the first stage (training the GM-VAE) takes about 14.5 hours, and the second stage (training the AR transformer) takes about 13.6 hours.\n    According to our experiments, in the first-stage training (for VAE, VQ-VAE [1], etc.), our speed is comparable to discrete motion representations (MoMask [2], ~ 16 hours) and continuous motion representations (MLD [3], ~ 15 hours). In the second-stage training, our speed is faster than MoMask ( ~ 24 hours) and MLD (2-3 days). MoMask requires training two separate transformers (base and residual, total parameters ~ 44.9M), while MLD's skip transformer has more parameters (~ 30M) and its diffusion denoising validation process is time-consuming. Our model only requires training a single transformer (~ 11.6M), and its validation process allows for single-step sampling (vs. MLD's DDIM=50), resulting in faster training.\n\n*   **Inference Time:**\n    We use the average inference time per sentence (AITS) over 100 samples on a single Nvidia 4090 device to evaluate inference time. Our model's AITS is 0.033, which is better than MotionLCMv2 [4] (0.045) and MoMask (0.046). More detailed results can be found in Appendix D.3. We will indicate the AITS for different models in the next version.\n\n----\n\n**[Q2]**: *Explain why the proposed method has relatively low motion diversity (i.e., MultiModality).*\n\nWe think this reduction in diversity may stem from the random fluctuations introduced by the Gaussian Mixture parameters. During training, the diagonal values of the covariance matrix are fixed to 1, meaning each Gaussian component is orthogonal. This design helps the model learn the motion distribution more effectively, but at the cost of some diversity.\n\nMultiModality measures the diversity of generated motions within each textual description [5]. On the HumanML3D [5] dataset, our model's MultiModality score (2.033) is still better than existing SOTA models (MoMask with 1.241, MotionLCMv2 with 1.758), but lags behind some classic models (MDM [6] with 2.799, MLD with 2.413). Our model also shows the degree of performance degradation in diversity. \n\nBalancing the quality (e.g., FID) and diversity (e.g., MultiModality) of motion synthesis is a problem worth our consideration. GMM, through uncertainty modeling, struggles to achieve high diversity while maintaining good motion quality, and we will continue to work further in this direction.\n\n----\n### **Reference**\n\n[1] Yuan W, Shen W, et al. Motion Generation based on Spatial-Temporal Joint Modeling. Neurips 2024.\n\n[2] Guo C, Mu Y, Javed M G, et al. Momask: Generative masked modeling of 3d human motions. CVPR 2024.\n\n[3] Chen X, Jiang B, et al. Executing your Commands via Motion Diffusion in Latent Space. CVPR 2023.\n\n[4] Dai W, Chen L H, Wang J, et al. Motionlcm: Real-time controllable motion generation via latent consistency model. ECCV 2024.\n\n[5] Guo C, et al. Generating Diverse and Natural 3D Human Motions From Text. CVPR 2022.\n\n[6] Tevet G, et al. Human Motion Diffusion Model. ICLR 2023."}, {"review_id": "0QPmtob8ld", "reviewer": "Reviewer_eYpb", "summary": "This paper proposes a novel text-to-motion method GMMotion combines the continuous motion representation (GM-VAE) and autoregressive generation model.\nThe method is trained in two stages:\n1. GM-VAE encodes motion sequences into a multimodal latent space regularized by GMM and KS distance to match real data distribution.\n2. Autoregressive Transformer predicts GMM parameters and samples latent vectors directly without diffusion or discrete token decoding.\nSuch approach avoids vector quantization, which causes a discretization errors and breaks temporal continuity.", "questions": "1. Inference time\nThe autoregressive approach could take long time generating frame-by-frame, presenting the inference time could provide better understanding on the proposed method.\n2. Long sequence\nDoes the proposed method generalizes to the long sequences?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1. Novel approach\nAuthors proposed novel approach to learn the latent space of the motion with GMM.\n2. Good generation quality\nQualitative and quantitative results are good.\n\nWeakness\n1. Unclear comparison over previous ideas\nSince the previous observation on VAE indicates that the generation results using VAE are blurry, authors could provide more detailed explanation why their proposed methods achieved the better performance compared to VAE and VQ-VAE.", "was_revised": true, "final_justification": "Since the concerns are well addressed by rebuttal, I retain my original recommendation score.", "rebuttal": "We thank the reviewer eYpb for their insightful comments and suggestions.\n\nWe acknowledge the issues and weaknesses raised by the reviewer. Given that our focus is on text to motion synthesis, explaining inference time is indeed crucial. Furthermore, we will further clarify the advantages of GM-VAE and update this in the paper.\n\n---\n\n**[W1]**: *Unclear comparison over previous ideas about GM-VAE.*\n*   The blurry results generated by VAEs are commonly observed in image synthesis. We believe this occurs because the KL divergence encourages the latent space distribution to be close to a standard normal distribution, which may cause latent samples to cluster too tightly, resulting in generated images lacking diversity and appearing blurry. In the context of human motion reconstruction studied in this paper, this might manifest as **missing fine-grained motion details between frames** (e.g., subtle hand joint movements or slight head shakes failing to be reconstructed accurately).\n*   Compared to a standard VAE, we define a learnable Gaussian Mixture Model (GMM) prior to prevent the latent space distribution from collapsing into a single, restrictive mode. Under the same model architecture, when training a VAE with a fixed standard normal prior, we obtain an FID of 0.040. In contrast, our GM-VAE achieves a significantly lower FID of 0.008. We will include these comparative results in the appendix.\n*   Compared to VQ-VAE, our method synthesizes motion continuously over time, avoiding motion errors introduced by the discretization process. Since motion sequences are continuous time-series data, VQ-VAE typically discretizes every frame into a single ID. This approach can lead to discontinuities in the reconstructed motion (e.g., MoMask [1] with RVQ achieves an FID of 0.029).\n*   GMM better reconstructs the spatial distribution of human motion. Unlike images and audio, each joint of the human skeleton follows specific motion patterns (which we visualize in Figure 1.a as a **multimodal distribution**), whereas the KL divergence constraint in VAE assumes a **unimodal distribution**, making it difficult to fully reconstruct motion details. The visualization results of the latent space are shown in Figure 1.b.\n\n---\n\n**[Q1]**: *The inference time should be presented.*\n\n- We evaluate inference time using the average inference time per sentence (AITS) over 100 samples on a single Nvidia 4090 device. Our model's AITS is 0.033, which outperforms MotionLCMv2 [2] (0.045) and MoMask (0.046). More detailed results are available in Appendix D.3.\n- Although the autoregressive (AR) process requires frame-by-frame synthesis, in the first stage we condense every four motion frames into one latent frame. This design gives our model an advantage in parameter efficiency compared to the diffusion-based transformer MotionLCMv2 and MoMask, which requires two separate transformers. MoMask first uses a base transformer for masked generation to obtain base tokens and then employs a residual transformer for non-autoregressive, multi-step iterative synthesis to obtain residual tokens, introducing additional computational overhead. MotionLCM [2] achieves its best results after distillation using 4 iterative steps; our model achieves competitive results with single-step sampling while having significantly fewer parameters (MotionLCM can also use 1-step sampling, but with a noticeable drop in synthesis quality).\n\n- We will clearly state the inference time in the main text of the revised manuscript.\n\n---\n\n**[Q2]**: *Does the proposed method generalize to long sequences?*\n\n- Yes. We appreciate the reviewer notes this special design and future direction. Some video demonstrations of long-sequence synthesis were attached at the end of the abstract when the main paper was submitted. Currently, the visual quality of long-sequence synthesis still requires refinement due to rendering problem.\n\n- In this paper, we introduce two additional designs to facilitate long-sequence synthesis: prefix padding and rotary position encoding. Since motion sub-sequences within a batch often have varying lengths, autoregressive generation methods like T2M-GPT [3] in the motion generation domain typically pad sequences at the end (suffix padding) for training convenience. However, this approach limits long-sequence synthesis because the model stops inference as soon as it generates a specific stop token.\nInspired by LLM inference designs, we opt for prefix padding to align the end of motion sequences, combined with rotary positional encoding instead of absolute positional encoding. This design reduces the model's strong dependence on absolute positional relationships, thereby enhancing its scalability. \n\n- We will include a discussion section in the new version elaborating on the capability for long-sequence synthesis. Additionally, we will provide qualitative results of long-sequence synthesis in the appendix.\n\n----\nWe sincerely appreciate the reviewer's constructive suggestions and believe that the additional analysis and explanations improve the quality of our submission. We hope that this provides sufficient reasons to raise the score.\n\n----\n### **Reference**\n\n[1] Guo C, Mu Y, Javed M G, et al. Momask: Generative masked modeling of 3d human motions. CVPR 2024.\n\n[2] Dai W, Chen L H, Wang J, et al. Motionlcm: Real-time controllable motion generation via latent consistency model. ECCV 2024.\n\n[3] Zhang J, Zhang Y, Cun X, et al. Generating human motion from textual descriptions with discrete representations. CVPR 2023."}]

Author rebuttal:

Meta-review: This paper proposes GMMotion, a text-to-motion synthesis framework that integrates a Gaussian Mixture VAE (GM-VAE) with an autoregressive transformer, enabling continuous latent motion representation without discrete tokenization or diffusion-based sampling. The method addresses key issues of quantization error in VQ-based models and slow inference in diffusion models. Across evaluations, it demonstrates competitive or superior performance in reconstruction, generation quality, efficiency, and alignment with text prompts.

Three reviewers (QPHE, Nig7, eYpb) are clearly supportive, highlighting novelty, technical soundness, and thorough empirical evaluation. One reviewer (kPaJ) initially questioned the novelty and generation quality but, after the rebuttal and clarifications, raised their score to borderline accept. Overall consensus is positive.

Strengths identified:
1) Novel combination of GM-VAE latent representation and AR transformer for text-to-motion.
2) Clear improvements in reconstruction quality and faster inference compared to diffusion baselines.


Remaining weaknesses and revision requests for the camera-ready version:
1) Clarify novelty relative to prior GMM-based generative models in other domains (e.g., video, speech) and better articulate domain-specific contributions for motion synthesis.
2) Inference and Efficiency: Include explicit runtime comparisons (training and inference) against baselines, as requested by multiple reviewers.
3) Generation Diversity: Discuss the slightly lower diversity compared to some baselines and outline potential improvements.
4) Clarity: Improve explanations on why GM-VAE avoids blurriness typically observed in VAEs, and expand on generalization to longer sequences.
Decision: Accept (poster)