SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure B: Additional qualitative motion results are generated with text prompts of the HumanML3D test set. The darker colors indicate the later in time. These results demonstrate that our method can generate diverse and accurate motion sequences.
Where  $\alpha^{\prime}$  is the guidance scale and  $\alpha^{\prime} > 1$  can strengthen the effect of guidance [8]. We set  $\alpha^{\prime}$  to 7.5 in practice following MLD. Please refer to our code for more details.
B.3 Implementation Details for Different Datasets
Following MLD [8], we utilize a frozen text encoder of the CLIP-ViT-L-14 [45] model for text representation. The dimension of node representation  $D$  is set to 768. The dimension of latent embedding  $D^{\prime}$  is set to 256. For the motion variational autoencoder, motion encoder  $\mathcal{E}$  and decoder  $\mathcal{D}$  all consist of 9 layers and 4 heads with skip connection [46]. We set the token sizes  $C^m$  to 2,  $C^a$
Figure C: Qualitative analysis on the imbalance problem. The darker colors indicate the later in time. When the verbs and action names are masked, existing models tend to generate motion randomly. In contrast, our method can generate motion based solely on the action specifics. These results show that our method is not overly focused on the verbs and action names.
Figure D: Qualitative comparison of different hierarchies. The output at the higher level (e.g., specific level) has more action details. Specifically, the motion level generates only coarse-grained overall motion. The action level generates local actions better than the motion level but lacks action specifics. The specific level generates more action specifics than the action level.
to 4, and  $C^s$  to 8. We set  $\lambda$  to 1e-4. All our models are trained with the AdamW [26, 38] optimizer using a fixed learning rate of 1e-4. We use 4 Tesla V100 GPUs for the training, and there are 128 samples on each GPU, so the total batch size is 512. The number of diffusion steps of each level is 1,000 during training, and the step sizes  $\beta_{t}$  are scaled linearly from  $8.5 \times 1\mathrm{e - }4$  to 0.012. We keep running a similar number of iterations on different data sets. For the HumanML3D dataset, the model is trained for 6,000 epochs during the motion variational autoencoder stage and 3,000 epochs during the diffusion stage. For the KIT dataset, the model is trained for 30,000 epochs during the motion variational autoencoder stage and 15,000 epochs during the diffusion stage. Code is available at https://github.com/jpthu17/GraphMotion. In this code, we provide the process of the training and evaluation of the proposed method, and the pre-trained model.
Table B: Ablation study about the total number of diffusion steps on the HumanML3D test set. “↑” denotes that higher is better. “↓” denotes that lower is better. We repeat all the evaluations 20 times and report the average with a  $95\%$  confidence interval. “×” denotes that this method does not apply this parameter. To speed up the sampling process, we use DDIM in practice following MLD.
Methods	Diffusion Steps	R-Precision ↑	FID ↓
Motion Tm	Action Ta	Specific Ts	Top-1	Top-2	Top-3
The total number of diffusion steps is 1000 with DDPM [18]
MDM [54]	1000	×	×	0.320±.005	0.498±.004	0.611±.007	0.544±.044
MotionDiffuse [62]	1000	×	×	0.491±.001	0.681±.001	0.782±.001	0.630±.001
The total number of diffusion steps is 50 with DDIM [50]
MLD [8]	50	×	×	0.481±.003	0.673±.003	0.772±.002	0.473±.013
GraphMotion (Ours)	20	15	15	0.489±.003	0.676±.002	0.771±.002	0.131±.007
GraphMotion (Ours)	15	15	20	0.496±.003	0.686±.003	0.778±.002	0.118±.008
The total number of diffusion steps is 150 with DDIM [50]
MLD [8]	150	×	×	0.461±.002	0.649±.003	0.797±.002	0.457±.011
GraphMotion (Ours)	50	50	50	0.504±.003	0.699±.002	0.785±.002	0.116±.007
The total number of diffusion steps is 300 with DDIM [50]
MLD [8]	300	×	×	0.473±.002	0.664±.003	0.765±.002	0.403±.011
GraphMotion (Ours)	100	100	100	0.486±.003	0.671±.004	0.767±.003	0.096±.008
The total number of diffusion steps is 1000 with DDIM [50]
MLD [8]	1000	×	×	0.452±.002	0.639±.003	0.751±.002	0.460±.013
GraphMotion (Ours)	400	300	300	0.475±.003	0.659±.003	0.756±.003	0.136±.007
GraphMotion (Ours)	300	300	400	0.484±.003	0.694±.003	0.787±.003	0.132±.008
C Additional Qualitative Analysis
C.1 Qualitative Comparison of the Existing Methods
We provide additional qualitative motion results in Fig. A. Compared to other methods, our method generates motions that match the text descriptions better, indicating that our method is more sensitive to subtle differences in texts. The generated results demonstrate that our method achieves superior controllability compared to well-designed baseline models.
C.2 Additional Visualization Results
In Fig. B, we provide additional qualitative motion results which are generated with text prompts of the HumanML3D test set. These results demonstrate that our method can generate diverse and accurate motion sequences from a variety of motion descriptions.
C.3 Qualitative Analysis on the Imbalance Problem
To demonstrate the imbalance problem of other methods and prove that our method does not have this problem, we mask the verbs and action names in the motion description to force the model to generate motion only from action specifics. As shown in Fig. C, when the verbs and action names are masked, existing models tend to generate motion randomly. In contrast, our method can generate motion that matches the description based solely on the action specifics. These results show that our method is not overly focused on the verbs and action names.
C.4 Qualitative Comparison of Different Hierarchies
We provide different levels of qualitative comparison in Fig. D. The results show that the output at the higher level (e.g., specific level) has more action details. Specifically, the motion level generates only coarse-grained overall motion. The action level generates local actions better than the motion level but lacks action specifics. The specific level generates more action specifics than the action level.
Table C: Evaluation of Inference time costs on the HumanML3D test set. We evaluate the average time per sample with different diffusion schedules and FID. “↓” denotes that lower is better. Please note the bad FID of MDM with DDIM is mentioned in their GitHub issues #76. “×” denotes that this method does not apply this parameter. We use DDIM in practice following MLD.
Methods	Reference	Diffusion Steps	Average time per sample (s) ↓	FID ↓
Motion Tm	Action Ta	Specific Ts
The total number of diffusion steps is 1000 with DDPM [18]
MDM [54]	ICLR 2023	1000	×	×	178.7699	0.544
MLD [8]	CVPR 2023	1000	×	×	5.5045	0.568
The total number of diffusion steps is 50 with DDIM [50]
MDM [54]	ICLR 2023	50	×	×	20.5678	7.334
MLD [8]	CVPR 2023	50	×	×	0.9349	0.473
GraphMotion	Ours	20	15	15	0.9094	0.131
GraphMotion	Ours	15	15	20	0.7758	0.118
The total number of diffusion steps is 150 with DDIM [50]
MLD [8]	CVPR 2023	150	×	×	2.4998	0.457
GraphMotion	Ours	50	50	50	2.5518	0.116
The total number of diffusion steps is 1000 with DDIM [50]
MLD [8]	CVPR 2023	1000	×	×	16.6654	0.460
GraphMotion	Ours	400	300	300	22.1238	0.136
GraphMotion	Ours	300	300	400	17.0912	0.132
D Additional Experiments
D.1 Analysis of the Diffusion Steps
In Tab. B, we show the ablation study of the total number of diffusion steps on the HumanML3D test set. Following MLD [8], we adopt the denoising diffusion implicit models [50] (DDIM) during interference. As shown in Tab. B, our method consistently outperforms the existing state-of-the-art methods with the same total number of diffusion steps, which demonstrates the efficiency of our method. We find that the number of diffusion steps at the higher level (e.g., specific level) has a greater impact on the result. Therefore, in scenarios requiring high efficiency, we recommend allocating more diffusion steps to the higher level. Moreover, with the increase of the total diffusion steps, the performance of our method is further improved, while the performance of MLD saturates. These results further prove the superiority of our design.
D.2 Analysis of the Inference Time
In Tab. C, we provide the evaluation of inference time costs. Our method is as efficient as the one-stage diffusion methods during the inference stage, even though we decompose the diffusion process into three parts. This is because we can control the total number  $T^{m} + T^{a} + T^{s}$  of iterations by restricting it to be the same as those of the one-stage diffusion methods. As shown in Tab. C, the inference speed of our method is comparable to that of the existing state-of-the-art methods with the same total number of diffusion steps, which demonstrates the efficiency of our method.
D.3 Analysis of the motion VAE models
We provide the evaluation of the motion VAE models. In Tab. D, we show the results on the HumanML3D test set. Tab. E shows the results on the KIT test set. Among the three levels, the performance of the specific level is the best, which indicates that increasing the token size can improve the reconstruction ability of the motion VAE models.
E Motion Representations and Metric Definitions
E.1 Motion Representations
Motion representation can be summarized into the following four categories, and we follow the previous work of representing motion in latent space.
Table D: Evaluation of the VAE models on the motion part of the HumanML3D test set. “↑” denotes that higher is better. “↓” denotes that lower is better. “→” denotes that results are better if the metric is closer to the real motion. The performance of the specific level is the best.
Methods	Token Size	R-Precision ↑	FID ↓	Diversity →
Top-1	Top-2	Top-3
Real motion	-	0.511	0.703	0.797	0.002	9.503
Motion Level	2	0.498	0.692	0.791	1.906	9.675
Action Level	4	0.514	0.703	0.793	0.068	9.610
Specific Level	8	0.525	0.708	0.800	0.019	9.863
Table E: Evaluation of the VAE models on the motion part of the KIT test set. “↑” denotes that higher is better. “↓” denotes that lower is better. “→” denotes that results are better if the metric is closer to the real motion. The performance of the specific level is the best.
Methods	Token Size	R-Precision ↑	FID ↓	Diversity →
Top-1	Top-2	Top-3
Real motion	-	0.424	0.649	0.779	0.031	11.08
Motion Level	2	0.431	0.623	0.745	1.196	10.66
Action Level	4	0.413	0.644	0.770	0.396	10.85
Specific Level	8	0.414	0.640	0.760	0.361	10.86
Latent Format. Following previous works [42, 8, 61], we encode the motion into the latent space with a motion variational autoencoder [27]. The latent representation is formulated as:
$$
\hat {x} ^ {1: L} = \mathcal {D} (z), \quad z = \mathcal {E} \left(x ^ {1: L}\right). \tag {B}
$$
HumanML3D Format. HumanML3D [14] proposes a motion representation  $x^{1:L}$  inspired by motion features in character control. This motion representation is well-suited for neural networks. To be specific, the  $i_{th}$  pose  $x^i$  is defined by a tuple consisting of the root angular velocity  $r^a \in \mathbb{R}$  along the Y-axis, root linear velocities  $(r^x, r^z \in \mathbb{R})$  on the XZ-plane, root height  $r^y \in \mathbb{R}$ , local joints positions  $j^p \in \mathbb{R}^{3N_j}$ , velocities  $j^v \in \mathbb{R}^{3N_j}$ , and rotations  $j^r \in \mathbb{R}^{6N_j}$  in root space, and binary foot-ground contact features  $c^f \in \mathbb{R}^4$  obtained by thresholding the heel and toe joint velocities. Here,  $N_j$  denotes the joint number. Finally, the HumanML3D format can be defined as:
$$
x ^ {i} = \left\{r ^ {a}, r ^ {x}, r ^ {z}, r ^ {y}, j ^ {\mathbf {p}}, j ^ {\mathbf {v}}, j ^ {\mathbf {r}}, c ^ {\mathbf {f}} \right\}. \tag {C}
$$
SMPL-based Format. SMPL [37] is one of the most widely used parametric human models. SMPL and its variants propose motion parameters  $\theta$  and shape parameters  $\beta$ .  $\theta \in \mathbb{R}^{3\times 23 + 3}$  is rotation vectors for 23 joints and a root, while  $\beta$  represents the weights for linear blended shapes. The global translation  $r$  is also incorporated to formulate the representation as follows:
$$
x ^ {i} = \{r, \theta , \beta \}. \tag {D}
$$
MMM Format. Master Motor Map [52] (MMM) representations propose joint angle parameters based on a uniform skeleton structure with 50 degrees of freedom (DoFs). In text-to-motion tasks, recent methods [2, 12, 42] converts joint rotation angles into  $J = 21$  joint XYZ coordinates. Given the global trajectory  $t_{root}$  and  $p_m \in \mathbb{R}^{3J}$ , the preprocessed representation is formulated as:
$$
x ^ {i} = \left\{p _ {m}, t _ {\text {r o o t}} \right\}. \tag {E}
$$
E.2 Metric Definitions
Following previous works, we use the following five metrics to measure the performance of the model. Note that global representations of motion and text descriptions are first extracted with the pre-trained network in [14].
R-Precision. Under the feature space of the pre-trained network in [14], given one motion sequence and 32 text descriptions (1 ground-truth and 31 randomly selected mismatched descriptions), motion-retrieval precision calculates the text and motion Top  $1/2/3$  matching accuracy.
Frechet Inception Distance (FID). We measure the distribution distance between the generated and real motion using FID [17] on the extracted motion features [14]. The FID is calculated as:
$$
\mathrm {F I D} = \left\| \mu_ {g t} - \mu_ {p r e d} \right\| ^ {2} - \operatorname {T r} \left(\Sigma_ {g t} + \Sigma_ {p r e d} - 2 \left(\Sigma_ {g t} \Sigma_ {p r e d}\right) ^ {\frac {1}{2}}\right), \tag {F}
$$
where  $\Sigma$  is the covariance matrix. Tr denotes the trace of a matrix.  $\mu_{gt}$  and  $\mu_{pred}$  are the mean of ground-truth motion features and generated motion features.
Multimodal Distance (MM-Dist). Given  $N$  randomly generated samples, we calculate the average Euclidean distances between each text feature  $f_{t}$  and the generated motion feature  $f_{m}$  from that text. The multimodal distance is calculated as:
$$
\mathrm {M M - D i s t} = \frac {1}{N} \sum_ {i = 1} ^ {N} \| f _ {t, i} - f _ {m, i} \|, \tag {G}
$$
where  $f_{t,i}$  and  $f_{m,i}$  are the features of the  $i_{th}$  text-motion pair.
Diversity. All generated motions are randomly sampled to two subsets  $\left(\{x_{1}, x_{2}, \ldots, x_{X_{d}}\}\right.$  and  $\left.\{x_{1}^{\prime}, x_{2}^{\prime}, \ldots, x_{X_{d}}^{\prime}\}\right)$  of the same size  $X_{d}$ . Then, we extract motion features [14] and compute the average Euclidean distances between the two subsets:
$$
\text {D i v e r s i t y} = \frac {1}{X _ {d}} \sum_ {i = 1} ^ {X _ {d}} \| x _ {i} - x _ {i} ^ {\prime} \|. \tag {H}
$$
Multimodality (MModality). We randomly sample a set of text descriptions with size  $J_{m}$  from all descriptions. For each text description, we generate  $2 \times X_{m}$  motion sequences, forming  $X_{m}$  pairs of motions. We extract motion features and calculate the average Euclidean distance between each pair. We report the average of all text descriptions. We define features of the  $j_{th}$  pair of the  $i_{th}$  text description as  $(x_{j,i}, x_{j,i}')$ . The multimodality is calculated as:
$$
\text {M M o d a l i t y} = \frac {1}{J _ {m} \times X _ {m}} \sum_ {j = 1} ^ {J _ {m}} \sum_ {i = 1} ^ {X _ {m}} \| x _ {j, i} - x _ {j, i} ^ {\prime} \|. \tag {I}
$$