# MVAE: Motion-conditioned Variational Auto-Encoder for tailoring character animations

Jean-Baptiste Bordier, Marc Christie

► To cite this version:

Jean-Baptiste Bordier, Marc Christie. MVAE: Motion-conditioned Variational Auto-Encoder for tailoring character animations. Expressive-WICED, May 2025, Londres, United Kingdom. 10.2312/exw.20251059. hal-05239274

HAL Id: hal-05239274

https://inria.hal.science/hal-05239274v1

Submitted on 4 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# MVAE: Motion-conditioned Variational Auto-Encoder for tailoring character animations

JB. Bordier¹ and M. Christie¹

¹University of Rennes 1, IRISA, Rennes France

<figure><img src="image_2.png" /><figcaption>**Figure 1:** We propose MVAE, a Motion-conditioned Variational Auto-Encoder designed to enable animators to generate and adapt animations based on motion characteristics such as rhythm, pace, and amplitude, facilitating the tailored creation of a diverse range of animations. In the top row, we display an example where the user input motion generates a walking motion which amplitude is controlled from the user's amplitude in the gesture, and in the bottom row the input motion creates a jumping animation and the jump height is also affected by the amplitude in the gesture.</figcaption></figure>

## Abstract

The design of character animations with enough diversity is a time-consuming task in many productions such as video games or animated films, and drives the need for more simple and effective authoring systems. This paper introduces a novel approach, a motion-conditioned variational autoencoder (VAE) with Virtual reality as a motion capture device. Our model generates diverse humanoid character animations only based on a gesture captured from two Virtual reality controllers, allowing for precise control of motion characteristics such as rhythm, speed and amplitude, and providing variability through noise sampling. From a dataset comprising paired controller-character motions, we design and train our VAE to (i) identify global motion characteristics from the movement, in order to discern the type of animation desired by the user, and (ii) identify local motion characteristics including rhythm, velocity, and amplitude to adapt the animation to these characteristics. Unlike many text-to-motion approaches, our method faces the challenge of interpreting high-dimensional, non-discrete user inputs. Our model maps these inputs into the higher-dimensional space of character animation while leveraging motion characteristics (such as height, speed, walking step frequency, and amplitude) to fine-tune the generated motion. We demonstrate the relevance of the approach on a number of examples and illustrate how changes in rhythm and amplitude of the input motions are transferred to coherent changes in the animated character, while offering a diversity of results using different noise samples.

## CCS Concepts

* Computing methodologies → Neural networks; Animation; Machine learning; Deep learning; Motion processing;
# 1. Introduction

Character animation remains one of the most time-consuming tasks when it comes to the production of video games or animated films. For many years now, researchers have been working to propose solutions and systems that ease and optimise the complex task of animating characters. The recent advent of generative deep learning techniques has facilitated the development of animation tools for 3D and 2D characters. Editing tools with different input modalities have made it possible to create animations from sketches [GRGC15, TBvdP06, BRL*16, GXMZ14, GCR13], from text [GB08, SKK09, TGBCO22], from mobile phone movements [YKSF20] or from action labels [PBV21, SW22, LAZ*22], sometimes combined with inputs from joysticks for example [LZCdP20]. Virtual reality has also made it possible to explore new modes of interaction in order to create character animations, by benefiting from precise tracking of positions and orientations and has been proven useful for the task of posing and animating characters [BMCC22, LCM20, GRC19, JPY*23, CDML19]. In fact, the 6 DoF (Degree of Freedom) offered by VR controllers allow very natural, direct editing at the character's joints or on different multidimensional animation controllers (a controller offers a way to manipulate multiple joints simultaneously). Capturing information on rotations and positions is very easy, making it possible to collect sequences with lots of information on acceleration, speed, amplitude and frequency. Yet, recent works have not successfully integrated artistic control with the efficacy of deep learning models. The need for artistic control remains unfulfilled when relying on labels, textual descriptions, or even sketches as inputs. This challenge is particularly evident when generating intricate animations that require a specific level of stylization. Such inputs often fail to convey the nuanced details necessary to precisely articulate the desired stylized movements. For example, if a user wants a walking animation with a particular step frequency, achieving this quickly and accurately can be very difficult, if not impossible. This is where the integration of controllers becomes particularly valuable. By using controllers, an animator can, much like a choreographer, express a wide range of characteristics through precise movements. These characteristics include jump or walk amplitude, step frequency, animation length, speed, and acceleration. Controllers enable animators to directly shape the motion, providing a more intuitive and expressive approach to animation creation.

In our work, we address the challenge of providing expressive artistic control while simultaneously maintaining the productivity and diversity afforded by generative deep-learning models. In contrast with text-conditioned approaches which have a limited degree of artistic control, we propose a motion-conditioned variational autoencoder, inspired by [PBV21], to generate transform-based humanoid character animations from controllers motions. Our algorithm is therefore able to take as input a sequence of positions and rotations of the two VR controllers, providing a fine artistic control by extracting the movement's underlying characteristics (such as the desired walking frequency of an animated walk or run, the amplitude of the steps, the height of a jump), and generate diverse humanoid animations respecting these motion characteristics (as illustrated in Fig. 1). This empowers users to quickly and precisely describe the type and style of desired animation. For instance, a swift, abrupt gesture or one with high acceleration and amplitude

can be used to precisely convey the desired animation style. In the case of a jump animation, this method permits users to define the jump's height, force, speed, and other characteristics.

Our contributions include the following:

1. Creation of a database pairing humanoid motion animations with motion using VR controllers as motion capture device.

2. Provision of a motion-conditioned Variational Autoencoder (VAE) model capable of processing complex sequences captured by VR controller motions and generating stylized animations altered by the inherent characteristics of the input motion.

The rest of the paper is organized as follows: a literature review is performed in Section 2, our overall method is presented in Section 3, including the design of the paired motions dataset, the proposed motion-driven VAE and the ablation study. Results are finally presented in Section 4 before concluding (Section 5).

# 2. State of the art

In this state of the art section, we will mostly focus on artistic-driven approaches to crafting and editing animations, leaving aside the large literature on incarnation (i.e. real-time tracking of a person with VR controllers and headset position to create realistic motion in CG). The primary distinction between animation generation/editing and works addressing the incarnation problem lies in their goals: while the former focuses on artistic creation and manipulation, the latter emphasizes accurate and realistic pose estimation based on limited input data from VR headsets and controllers. Incarnation methods (e.g. [JSQ*22, DDC*21, AOGF*21, PYAP22, DKP*23, WWY22]) are inherently constrained by the physical limitations of the user and the scope of their motion databases, posing significant challenges for generating complex or stylized poses that the user cannot physically perform. For instance, while recent works like [AOGF*21] blend user input with pre-recorded motion capture data to accentuate upper-body gestures, and [PYAP22] leverage Motion Matching for plausible lower-body reconstruction, these systems remain bound to the physical feasibility of motions. A backflip would be infeasible to generate using incarnation techniques unless explicitly included in the database and triggered by near-physical user input, that would requires the user to almost perform the movement with the headset and hands controllers. Consequently, these limitations underscore the importance of differentiating between artistic animation approaches and realistic pose estimation methods in VR applications. Our work diverges from incarnation approaches by decoupling animation synthesis from physical user motion. Rather than replicating or extrapolating movements performed by the user (e.g., walking, running), we treat the VR controllers as a gestural interface for authoring animations. This is akin to puppeteering, where users craft motions by defining stylistic intent, such as a backflip, through abstract hand gestures, independent of their own physical capabilities.

Many heuristic methods for generating artistic animations have been developed over years using different input modalities. For example, [BRL*16, CÃS19] proposed a sketch- and trajectory/tangent-based interface for editing an animation. The results showed a good ability to modify animations to make them expres-
sive, including character posing, global and local trajectory modi-
fication, or visualization of spatial and temporal relationships.

Deep learning techniques have shown to be successful in gen-
erating animations by leveraging large datasets and incorporating
stylistic features through discriminative capabilities. However, to
our knowledge, no research has attempted to merge the power of
deep learning with VR's capture capabilities to generate anima-
tions for content creators. Notably, [GRC19] utilized VR to con-
dition character animation generation using a token matching algo-
rithm, albeit with limitations in generating diverse animations due
to heuristic approaches.

To tackle the challenge of animation diversity, deep learning
approaches have emerged as effective solutions. Holden et al.
[HSKJ15] pioneered the use of deep learning to project erroneous
motion data onto a manifold, addressing noise and errors in anima-
tion clips. Subsequent works like Taylor et al. [THR11] employed
Conditional Restricted Boltzmann machines for motion prediction.
Building on this, Holden et al. [HSK16] improved their method-
ology by employing deep convolutional autoencoders and incor-
porating a disambiguation network for enhanced realism and pre-
cision in generated animations. Since then, various research en-
deavors have aimed to enhance artistic control in character an-
imation through deep learning techniques. This discussion will
explore prominent architectures in character animation, including
Generative Adversarial Networks (GANs) [GPAM*14] and Varia-
tional Autoencoders (VAEs). GANs have been extensively applied
in animation, as evidenced by works such as [LAZ*22, MMFN20,
MPD22, SW22]. For instance, [SW22] introduced a GAN-based
framework incorporating a Transformer architecture for encoding
non-local correlations and leveraging a Gaussian Process prior for
temporal correlations. Meanwhile, [LAZ*22] proposed GANima-
tor, a framework for synthesizing motion sequences with various
capabilities such as conditional sequence generation, key-frame
editing, and motion style transfer.

VAEs have also been widely utilized in generating diverse an-
imations, often driven by inputs such as labels or joystick ac-
tions. Notably, [PBV21] introduced ACTOR, a model combining
Transformers and VAEs to generate realistic 3D human motions,
addressing issues such as motion regression to mean pose and
creating action-conditioned sequence-level embeddings. Similarly,
[LZCdP20] integrated VAEs with reinforcement learning for stable
and high-quality human motion generation on a frame-by-frame
basis, particularly beneficial for dynamically altering animation ac-
tions in video game systems. Moreover, studies such as [WYF*21]
and [PBV21] have highlighted the efficacy of Transformers over
alternatives like Long Short-Term Memory (LSTM) or Gated Re-
current Units (GRU) in animation tasks.

In our work, in order to generate animations based on gesture,
we (i) regarded the utilization of deep learning as a means to mit-
gigate ambiguities in the input data, a challenge stemming from
the multidimensional nature of the data that must be mapped to
a complex rig with multiple joints, (ii) depart from existing text-
to-motion approaches by improving the level of control through
motion-conditioning, and (iii) we rely on a conditional VAE to en-
sure diversity in the generated motions through random noise sam-
pling.

3. Method

Current approaches to character animation generation often rely on
abstract labels (e.g., "walk," "jump") or textual prompts to condi-
tion motion synthesis. While effective for producing predefined ac-
tions, these methods lack the granularity and expressiveness needed
for dynamic, user-guided animation design. Labels constrain cre-
ativity by forcing animations into categorical boxes, limiting the
ability to author nuanced, stylized motions that blend multiple ac-
tions or evolve over time. Our objective is to introduce a model
that enables sequence-aware animation generation, where motion
itself serves as the conditioning signal. Instead of using static la-
bels or text, we propose motion-conditioning, leveraging temporal
sequences of VR controller inputs (e.g., hand trajectories, rota-
tions) to guide the synthesis of animations. This paradigm shift al-
lows users to sculpt motions interactively, such as choreographing
a dance by gesturing with controllers or refining a jumps trajectory
by adjusting hand movements. During the sampling stage, the user
simply records a gesture in Unity using VR controllers. The model
then extracts high-level characteristics and conditions the anima-
tion output to follow these features.

3.1. Inputs and Outputs

During the training phase, our system receives two distinct in-
puts. It utilizes an encoder that processes two primary inputs: the
first input, denoted as $C^{T \times 14}$, represents a sequence containing
the temporal evolution of positions (a 3D vector) and rotations (a
quaternion) for both VR controllers $C^{T \times (3+4)*2}$ (the conditioning
data). The second input is the target output animation, denoted as
$A^{T \times 68 \times 4}$, consisting of T poses $P^{68 \times 4}$, where each of the 67 joints
is defined by their local rotations represented in quaternions along
with the global position of the root.

During the test phase, the input consists of a random latent code,
denoted as $z^{1 \times T \times 512}$, where T represents the desired number of
frames, along with the conditioning sequence of gesture VR con-
trollers C.

In both the training and runtime phases, the output is an anima-
tion sequence with T frames denoted as $A^{T \times 68 \times 4}$.

3.2. Overall Framework

The architecture of our deep learning model is presented in Fig-
ure 2.

1. During training: Each input is initially processed through linear layers in the encoder. The subsequent task of the transformer encoder is to extract distribution parameters from these inputs. To enable the transformer encoder to pay attention to the positioning of data within the sequence, we subject the inputs to a positional encoding procedure. Subsequently, a latent sequence is computed based on these learned parameters. It is important to note that our model's latent space is sensitive to the temporal dimension, resulting in a latent sequence characterized by two dimensions (sequence length, 512). This latent sequence is then fed into the transformer decoder along with the conditioned data C, which has also been projected. The decoder's output is further subjected to linear layers to achieve the desired output
shape. We trained the model for 20 000 epochs, with a batch size of 40 and a learning rate of $1.0e^{-4}$ and used the AdamW optimizer on an NVIDIA RTX A5000 GPU with 24GB RAM. The model consists of 4 Transformer layers for both encoder and decoder with a hidden size of 512, plus the Linear layers for projection, totaling 21M parameters.

2. **During testing:** The encoder is no longer used. The inputs now consist of the condition, which corresponds to the motion performed in VR, projected through a linear layer and concatenated with a random latent code. This concatenation is then passed through a positional encoding before being processed by the transformer decoders. The output of the transformer layers is subsequently projected through a linear layer to obtain a shape identical of a character animation.

The model is a sequence-to-sequence motion-conditioned Variational Autoencoder (VAE). This design allows the model to capture and represent time-dependent patterns and variations in the data.

### 3.2.1. Losses description

In our model, we employed a combination of two losses, drawing inspiration from existing Variational Autoencoder (VAE) models: a reconstruction loss and a Kullback-Leibler (KL) loss, which contributes to generalization. Unlike the approach in [PBV21], we excluded the weighting of the KL loss. It was observed that the most favorable outcomes were achieved by the model trained without any emphasis on the KL loss.

For the reconstruction loss, we opted for a composite of two losses. The initial two losses involve Mean Squared Error (MSE), calculated on the root translation data, ensuring the model grasps the accurate movement of the root joint. Subsequently, the second component of the reconstruction loss is determined by a geodesic distance [Huy09], computed between the ground truth quaternions representing the local rotations of joints and the generated joints' quaternions.

The reconstruction loss can then be expressed as:

$$ \mathcal{L}_{reconstruction} = \mathcal{L}_{MSE_{root}} + \mathcal{L}_{Geodesic_{rotations}} \quad (1) $$

where $\mathcal{L}_{Geodesic_{rotations}}$ is the geodesic error between output generated animation's joint local rotations represented as quaternions ($q_o$) and the ground truth animations ($q_g$). It can be expressed as $\mathcal{L}_{Geodesic_{rotations}} = 1 - \cos(\text{geodesic}(q_o, q_g))$, with

$$ \text{geodesic}(q_o, q_g) = \cos^{-1}(2\langle q_o, q_g \rangle^2 - 1) \quad (2) $$

where $\langle q_o, q_g \rangle$ stands for the inner product of generated and groundtruth quaternions: $\langle a_1 + b_1i + c_1j + d_1k, a_2 + b_2i + c_2j + d_2k \rangle = a_1a_2 + b_1b_2 + c_1c_2 + d_1d_2$.

### 3.2.2. Dataset

In this context, acquiring an appropriate dataset posed a challenge, as there were no existing dataset in the literature that matched the requirements of pairing human motion data with the positional and rotational information of two controllers over time. While prior

work such as [PYA*23] explores mapping VR controller data to animations, these efforts focus on realistic pose reconstruction, aligning avatar motions with the users physical movements. In contrast, our goal is not to replicate or reconstruct motion, but to leverage VR controllers as a creative interface for authoring animations. Existing datasets bind ground-truth animations to positional and rotational data from controllers to mirror real-world user actions. Our approach instead decouples animation synthesis from physical constraints, enabling stylized or non-physical motions (e.g., exaggerated jumps) that transcend the limitations of incarnation techniques.

To address this gap, we undertook the task of creating a new dataset. We leveraged the resources provided by Mixamo [Inc], a comprehensive database featuring a wide array of locomotion animations. We selected the Mixamo dataset over alternatives such as [MGT*19] due to its unique capability to parametrically adjust animations via intuitive sliders. These sliders enable fine-grained control over motion characteristics such as jump height, stride length, or gait speed, allowing us to generate multiple variations of the same base animation with targeted modifications. This feature was critical for training our model to isolate and learn specific stylistic or kinematic traits, as it provided a controlled dataset where variations in desired attributes (e.g., exaggerated jumps or altered walking cadence) could be systematically introduced without confounding factors. In contrast, datasets like AMASS, while rich in motion diversity, lack explicit parametric controls, making it challenging to curate aligned samples for studying granular animation properties. We composed a dataset encompassing 71 distinct animations, each consisting of variable length frames (from 60 to 350 frames). This collection encompassed various motion types, such as walking, running, jumping, rolling, climbing, sliding, and even boxing.

During the dataset creation process, we directed users to perform movements that corresponded to the animations displayed in the headset. Notably, approximately three-quarters of the users tended to replicate the evolving foot animations when dealing with locomotion animations, although we acknowledge that the response may vary depending on the specific type of animation. We preprocessed the conditions to have the left hand's first frame position as a reference point, meaning that the first position of the left hand is set as the origin (null vector). Right hand positions are also adjusted to be in the reference frame of the left hand's first frame.

Regarding the representation of the character animation data, our aim was to provide a dataset with a variety of representations such as joints local or global rotation, positions.

We then have a dataset composed of 71 different and complex humanoid locomotion animations, each paired with 14 different inputs for a total of 994 paired animations with inputs resulting in 189720 frames with an average of 203 frames per animations.

### 3.3. Ablation study

In the following, we assess the model's learning performances with various hyperparameters and explore the relevance of our architecture design through testing different architectural configurations. To accomplish this, we utilized a metric that measures the difference in joint angles, measured in degrees, between the output gen-
<figure><img src="image_3.png" /><figcaption>**Figure 2:** Our model architecture. During training, the entire model is utilized and optimized. The left blue section represents the VAE encoder, while the right blue section corresponds to the VAE decoder. So during training, the encoder receives two inputs, a sequence that encapsulates the movement of VR controllers (C), encompassing both rotation and position data along with the groundtruth animation composed of T poses (P<sub>0...T</sub>). During testing, the encoder is no longer used; instead, the latent code is randomly generated before being fed into the VAE decoder. The model then receives this random Z along with the condition (C). The output of the model is an animation composed of multiple poses (P'<sub>0...T</sub>) that incorporates considerations of frequency, amplitude, speed, length, and the specific type of animation.</figcaption></figure>

erated animation ($q_o$) and the ground truth animation ($q_g$) for all the train set paired inputs. To calculate this metric, we first obtained the quaternion representing the rotational difference between $q_o$ and $q_g$ named $q_{dif}$ following this:

$$q_{dif} = q_o \cdot q_g^{-1} \qquad (3)$$

Most of the following evaluations will use this metric and then will be converted and expressed in degrees. We average the angles for all joints of the skeleton and then average it for all frames of the animations. So for an animation of the evaluation set, we will have a value expressed in degrees, representing the average angle D between the generated animation and the ground truth over all frames (T) and all joints (J):

$$D = \frac{1}{T \cdot J} \sum_{t=0}^{T} \sum_{j}^{J} \theta(t, j) \qquad (4)$$

### 3.3.1. Hyperparameters

In the following we explore the impact of hyperparameter values: different batch size and different layers number. The batch size had a small impact on the results. As shown in Table 1, the best model based on the evaluation set is the one trained with a batch size of

<table><caption>**Table 1:** Comparison table for batch size hyperparameters. Each value is in degrees.</caption><thead><tr><th>Batch size</th><th>20</th><th>40</th><th>60</th></tr></thead><tbody><tr><td>Average value</td><td>5.683°</td><td>3.476°</td><td>5.605°</td></tr><tr><td>Minimum value</td><td>1.306°</td><td>1.3914°</td><td>1.421°</td></tr><tr><td>Maximum value</td><td>31.524°</td><td>29.815°</td><td>32.086°</td></tr><tr><td>Standard deviation</td><td>5.659°</td><td>3.186°</td><td>5.399°</td></tr><tr><td>Variance</td><td>32.030</td><td>10.151</td><td>29.159</td></tr></tbody></table>

40. This is the one with the best average value, while minimizing the variance.

We train our system with different number of layers for both encoder and decoder (Table 2). The most effective learning solution appears to be the one trained with 4 layers.

### 3.3.2. Model architecture

During our ablation study, we explored different architectures, including an LSTM-based model – a recurrent network commonly used for character animation in the literature. We compared this LSTM-based VAE model with the transformer-based architecture.
<table><caption>**Table 2:** Comparison table for the different number of layers hyperparameter. Each value is in degrees.</caption><thead><tr><th>Layers number</th><th>2</th><th>4</th><th>8</th></tr></thead><tbody><tr><th>Average value</th><td>5.363°</td><td>3.476°</td><td>5.319°</td></tr><tr><th>Minimum value</th><td>1.307°</td><td>1.391°</td><td>1.460°</td></tr><tr><th>Maximum value</th><td>32.383°</td><td>29.815°</td><td>31.200°</td></tr><tr><th>Standard deviation</th><td>5.641°</td><td>3.186°</td><td>5.153°</td></tr><tr><th>Variance</th><td>31.830</td><td>10.151</td><td>26.561</td></tr></tbody></table>

<table><caption>**Table 3:** Comparison table between errors for Transformers-based and LSTM-based models. Each value is in degrees.</caption><thead><tr><th>Architecture</th><th>Transformers</th><th>LSTM</th></tr></thead><tbody><tr><th>Average value</th><td>3.476°</td><td>11.046°</td></tr><tr><th>Minimum value</th><td>1.391°</td><td>1.788°</td></tr><tr><th>Maximum value</th><td>29.815°</td><td>38.748°</td></tr><tr><th>Standard deviation</th><td>3.186°</td><td>6.093°</td></tr><tr><th>Variance</th><td>10.151</td><td>37.131</td></tr></tbody></table>

In a similar fashion to the study conducted in [WYF*21], which demonstrated that transformer based models are better than LSTM using the Silhouette Coefficient (SCoeff) and the Calinski-Harabaz Index (CHI) metrics (12.5% on SCoeff and 15.7% on CHI). Our results favored the transformer-based architecture, producing an average error nearly 3.2 times lower than the LSTM-based model when using the training dataset inputs, as depicted in Table 3.

### 3.3.3. Exploring loss representations

We experimented a version where our angle error in the reconstruction loss was calculated using mean squared error (MSE) applied to each dimension (x, y, z, w) of the quaternions instead of the geodesic. This approach aimed to minimize individual quaternion components, rather than treating the quaternion as a rotation angle. As shown in Table 4 the model trained using the geodesic loss on angles exhibited an average improvement of 1,405 degree on the train set when contrasted with the model trained with Mean Squared Error (MSE). It's noteworthy that the model trained with MSE exhibits a variance four times higher, indicating significant variability in its learning performance across different animations. This suggests that while the model may have learned certain animations effectively, its performance on others was notably poorer.

In our further exploration, we sought to conduct an experiment

<table><caption>**Table 4:** Comparison table between models trained with a Geodesic loss and a MSE loss for rotations error. Each value is in degrees.</caption><thead><tr><th>Architecture</th><th>Geodesic loss</th><th>MSE loss</th></tr></thead><tbody><tr><th>Average value</th><td>3.476°</td><td>4.881°</td></tr><tr><th>Minimum value</th><td>1.391°</td><td>0.605°</td></tr><tr><th>Maximum value</th><td>29.815°</td><td>33.071°</td></tr><tr><th>Standard deviation</th><td>3.186°</td><td>6.559°</td></tr><tr><th>Variance</th><td>10.151</td><td>43.026</td></tr></tbody></table>

<table><caption>**Table 5:** Comparison table between models trained with a different rotation representation i.e. quaternion versus 6D. Each value is in degrees.</caption><thead><tr><th>Architecture</th><th>4D</th><th>6D</th></tr></thead><tbody><tr><th>Average value</th><td>3.476°</td><td>13.916°</td></tr><tr><th>Minimum value</th><td>1.391°</td><td>2.244°</td></tr><tr><th>Maximum value</th><td>29.815°</td><td>32.480°</td></tr><tr><th>Standard deviation</th><td>3.186°</td><td>4.832°</td></tr><tr><th>Variance</th><td>10.151</td><td>23.353</td></tr></tbody></table>

using the 6D representation ( [ZBL*18]) as the output format for the loss computation and input rotations. As shown in Table.5, the learning performance is better using the 4D representation than the 6D.

## 4. Results

In this section, we will showcase the outcomes of our model. Our primary objectives encompassed: (i) proposing a model with a large range of animation, avoiding limitations in animation possibilities; (ii) developing a model capable of producing variable-length animations with frequency adaptation, considering factors such as step frequency for walking or running animations, and jump frequency in more intricate multi-jump animations; and (iii) enabling the model to consider the amplitude of the input movement, allowing for the generation of animations with varying amplitudes.

### 4.1. Animation diversity

We showcase the capacity of our model to generate various and diverse animations. We decided to use a VAE to be able to generate diverse animations using the same condition. Since the model learned a probability distribution over the latent space that closely matches the true distribution of the data, we are able to generate diverse animations using the same condition. In Fig. 3a, we provide a condition specifying a jump with a predetermined height. The model effectively maintains this condition while introducing variability through the random aspect of the latent code z, resulting in a range of diverse animations. Similarly, in Fig. 3b, we repeated this process one hundred times and visualized the outputs in a graph using t-SNE dimension reduction. This visualization underscores the model's capacity to generate multiple distinct animations based on the same condition, yet similar in the type of motion performed, further emphasizing its capacity to generate diverse animations.

Similarly, we sought to demonstrate the model's capacity to produce diverse animations based on varying conditions. This is illustrated in Fig. 4. From the training set, we selected 13 inputs targeting the same action output (i.e., jumping animation). Additionally, we chose an input paired with a completely different action output (i.e., running animation). Subsequently, we introduced incremental levels of noise (20%, 40%, 60%, ..., 140%) to one of the 14 inputs. To introduce noise to the condition tensor C of shape (seq_len, 14), the following steps were taken:

1. Compute the total range for each axis of rotation and position
<figure><img src="image_4.png" /><figcaption>(a) Example of three generated outputs using the same condition with three different latent code z.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(b) t-SNE representation of a hundred generated output using the same single condition and 100 random sampled z. Each point of same color is a frame of the same generated animation. The overall pose is well maintained, and variations appear coherent.</figcaption></figure>

<caption>**Figure 3:** Evaluation results for random sample z, combined with the same condition. The model is capable of generating diverse animations, while keeping the desired characteristics and global type of motion (e.g. a jump, run, walk).</caption>

for each controller:

$$ \text{total\_range}_i = \begin{cases} \max_i - |\min_i| & \text{if } \min_i < 0 \\ \max_i - \min_i & \text{otherwise} \end{cases} \qquad (5) $$

where i ranges over the axes.

2. Generate a random tensor R of the same shape as C.

3. Add noise to C to obtain the noisy tensor C<sub>noisy</sub>:

$$ C_{\text{noisy}} = C + \text{total\_range} \cdot \mathbf{R} \cdot (\text{noise\_percentage}/100) \quad (6) $$

Here, max<sub>i</sub> and min<sub>i</sub> represent the maximum and minimum values, respectively, for the i-th axis across all controllers in C.

The resulting animations from these 21 inputs are depicted in Figure 4b. In this figure, we employed TSNE classification to categorize the generated animations. As anticipated, the introduction of additional noise led to greater divergence in the output animations. Notably, animations generated from inputs intended for a distinct action (i.e., running animation) were effectively segregated from the others. This observation underscores the model's ability to produce diverse and coherent outputs based on the input, as well as its adeptness in comprehending and categorizing the conditions, even in the absence of explicit labels.

## 4.2. Animation length and frequency adaptation

Our model underwent testing with varying input lengths and frequencies to ensure its adaptability and consistency in generating locomotion animations (in Fig. 5, a wall running animation). The outcomes were compelling, demonstrating that the generated animation length precisely corresponds to the input length, and the

speed and frequency of the wall running and simple walking are intricately linked to the frequency and speed of the VR controller movement input.

As illustrated in Fig. 5, where two distinct inputs were examined, each accompanied by its corresponding output. The first input represented a gradual movement with a low frequency of steps, resulting in an extended input. The key parameter to observe is the y-position axis for the left and right controllers (depicted by the solid curves in the figure). The green animated character corresponds to this initial input. Notably, each rise and fall of the VR controllers in the first input spans multiple frames, culminating in a gradual movement with a low frequency. Consequently, the generated animation exhibits a corresponding slowness and a low frequency of steps.

Conversely, the second input is characterized by abrupt movements, featuring a high frequency of up-and-down motions on the y-position axis in a condensed timeframe. The resulting animation (depicted by the red character in Fig. 5) is succinct, marked by a high frequency of steps.

For slow, extended inputs with a lower frequency of steps, the output manifests as an animation with a correspondingly slower step frequency and a prolonged duration. This test further underscores the model's adaptability, showcasing its proficiency not only in harmonizing the overall animation to maintain a stable frequency but also in adeptly adjusting each individual step to align with the input speed. For instance, if the input signals a slow lifting phase followed by a swift posing movement, the model adeptly captures
<figure><img src="image_6.png" /><figcaption>(a) Input visualisation using t-SNE dimension reduction. 13 inputs are paired with the same target action animation (jumping animation), 1 input is paired with a complete different target animation (running animation), and 7 (yellow stars) are based on one of the 14, but with a percent of noise added.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(b) t-SNE representation of generated outputs. Pink points are the frame of the complete different animation target. Centered points are frames from noisy inputs.</figcaption></figure>

**Figure 4:** This figure demonstrates the capacity of the model to handle different but close conditions, achieving to generate close but different outputs, that respect the desired characteristics and action encoded in the condition.
and integrates these distinct characteristics into the generated animation.

### 4.3. Animation amplitude adaptation

In this subsection, we will illustrate the model's capacity in analyzing inputs with varying amplitudes and generating adapted animations. Fig. 6 and Fig. 7 serve to illustrate the model's capabilities. In Fig. 6, in the first input provided to the model, both the x and y axes exhibit low amplitude signals, resulting in a wall-jumping animation with a modest amplitude. Conversely, the second input entails a substantial amplitude signal for both the x and y axes, yielding an animation with a more forceful jump. In this scenario, the character extends their body during the jump (corresponding to the larger amplitude of the y axis) and executes a longer jump (corresponding to the larger amplitude of the x axis). It adeptly handles the amplitude of walking and running animations (Fig. 7), adjusting the step amplitude to align with the input. Furthermore, the model demonstrates its ability to consider amplitude not only for jump height but also for the stretching of the character's body. It successfully addressed the challenges we aimed to overcome, particularly the limitations observed in our heuristic approach. In the heuristic approach, a constraint arose where, for certain inputs, the framework's ability to suggest diverse edits was confined to the lower body part. In contrast, our current approach enables the model to edit multiple joints for a single input, contingent on the generated animation conditioned by that input. As depicted in Fig. 7, we can, for instance, use the rotation of the VR controller during a walk to simultaneously edit the height of the knees and the amplitude of the steps.

### 4.4. Formal evaluation

For the formal evaluation, an animator was tasked with utilizing the model to generate a target animation. This assessment was conducted using an Oculus Quest 2. The animator was presented with the target animation and informed that the model was primarily trained to align its conditioning with the feet movements of the target animation. Subsequently, the animator was tasked with devising new conditions aimed at reproducing the target animation. This involved careful consideration of factors such as step frequency, amplitude, animation duration, and the specific actions within the animation. To assess the model's performance, we compared the generated animations in real-time with the target animations, as outlined in Equation 3. Additionally, for the ablation study, we evaluated the animator's real-time recorded conditions against different model architectures and hyperparameters.

The animator recorded 60 conditions paired with the target animations. Overall, the model demonstrated strong performance in comprehending the desired actions outlined by the animator and generating animations that correlated well with the input characteristics. Across the 60 animations, the model exhibited an average degree error of 12.762° for each joint per frame. Both the animator's conditions and the resulting generated output effectively captured the characteristics of the target animation. This suggests that in a real-world scenario, an animator envisioning a specific animation stylized by these characteristics could feasibly generate animations

<table><caption>**Table 6:** Comparison table between errors for Transformers based and LSTM based models. Each value is in degrees.</caption><thead><tr><th>Architecture</th><th>Transformers</th><th>LSTM</th></tr></thead><tbody><tr><td>Average value</td><td>12.945°</td><td>16.380°</td></tr><tr><td>Minimum value</td><td>1.890°</td><td>4.101°</td></tr><tr><td>Maximum value</td><td>32.070°</td><td>35.265°</td></tr><tr><td>Standard deviation</td><td>7.172°</td><td>6.337°</td></tr><tr><td>Variance</td><td>51.441</td><td>40.166</td></tr></tbody></table>

<table><caption>**Table 7:** Comparison table between models trained with a different rotation representation ie quaternion versus 6D. Each value is in degrees.</caption><thead><tr><th>Architecture</th><th>4D</th><th>6D</th></tr></thead><tbody><tr><td>Average value</td><td>12.945°</td><td>14.993°</td></tr><tr><td>Minimum value</td><td>1.890°</td><td>3.071°</td></tr><tr><td>Maximum value</td><td>32.070°</td><td>31.416°</td></tr><tr><td>Standard deviation</td><td>7.172°</td><td>5.646°</td></tr><tr><td>Variance</td><td>51.441</td><td>31.885</td></tr></tbody></table>

very close to the target, in a straightforward manner using a VR headset.

As mentioned earlier, we repurposed the animator's conditions to evaluate other model architectures and hyperparameter values. Tables 6, 7, 8, and 9 illustrate the discrepancies between LSTM-based models, 6D rotation representations, MSE loss for angles, and different batch models. Key observations include:

* **LSTM vs. Transformer:** Unlike the training dataset, the LSTM model exhibited lower variance compared to the transformer-based model, despite having an average error of nearly 4 degrees higher than the transformer.

* **6D vs. 4D Representation:** In contrast to the LSTM comparison, the highest variance shifted between the training and testing datasets. Although the average error across the test set converged closer between both models than in the training set, the model utilizing 6D representation for joint rotations encountered challenges in precisely learning the target animations but displayed good generalizability. Overall, the model with the 4D rotation representation still exhibited the least error.

* **Model trained with MSE for Joint Rotations:** Similar to the training phase, the average error did not differ significantly. However, the variance difference suggests that the model trained with MSE loss faced slightly more difficulty in identifying the desired actions.

* **Model trained with different batch size:** The evaluation on the test set revealed that the model trained with a batch size of 20 exhibited slightly better results than the one trained with a batch size of 40. Despite both models having nearly similar average errors, the variance of the model trained with a batch size of 20 was marginally lower than that of the model trained with a batch size of 40.
<figure><img src="image_8.png" /><figcaption>(a) Generated animations. Green character corresponds to first slow and long (250 frames) input while red character corresponds to second sudden (140 frames) with high frequency of step inputs.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>(b) The first input presented to the model features a lower step frequency (a step each 50 frames) and an extended duration, resulting in the generation of the green animation where the walking steps are slow.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>(c) The second input exhibits a higher step frequency (a step each 20 frames) and a shorter duration, leading to the generation of the red animation where the walking steps are fast.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>(a) Generated animations. Green character corresponds to first input with small amplitude while red character corresponds to second inputs with large amplitude.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>(b) The first input presented to the model features a small amplitude for both x and y axis, resulting in the generation of a green animation.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>(c) The second input exhibits the large amplitude for both x and y axis, leading to the generation of a red animation.</figcaption></figure>

<caption>**Figure 6:** Results showcase the model's capability to consider varying input amplitude, effectively generating animations corresponding to these diverse input parameters.</caption>
<figure><img src="image_14.png" /><figcaption>(a) Generated animations. Green character correspond to first input with high amplitude length while red character correspond to second inputs with small amplitude length.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>(b) The first input presented to the model features a high amplitude for both y and z axis, resulting in the generation of a green animation.</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>(c) The second input exhibits the small amplitude for y and z axis, leading to the generation of a red animation.</figcaption></figure>

**Figure 7:** Results showcase the model's capability to consider varying input amplitude for 2 axis, here z and y, resulting in adapted animation, where one animation is more of a running animation, with knees following the y axis of the largest amplitude input and with larger amplitude steps (green animation). In the other hand, the red generated animation is the results of both axis lower amplitude.

<table><caption>**Table 8:** Comparison table between models trained with a Geodesic loss and a MSE loss for rotations error. Each value is in degrees.</caption><thead><tr><td>Architecture</td><td>Geodesic loss</td><td>MSE loss</td></tr></thead><tbody><tr><td>Average value</td><td>12.945°</td><td>12.536°</td></tr><tr><td>Minimum value</td><td>1.890°</td><td>0.926°</td></tr><tr><td>Maximum value</td><td>32.070°</td><td>34.268°</td></tr><tr><td>Standard deviation</td><td>7.172°</td><td>8.35°</td></tr><tr><td>Variance</td><td>51.441</td><td>69.888</td></tr></tbody></table>

<table><caption>**Table 9:** Comparison table for batch size hyperparameters. Each value is in degrees.</caption><thead><tr><td>Batch size</td><td>20</td><td>40</td><td>60</td></tr></thead><tbody><tr><td>Average value</td><td>12.540°</td><td>12.945°</td><td>14.533°</td></tr><tr><td>Minimum value</td><td>2.317°</td><td>1.890°</td><td>2.388°</td></tr><tr><td>Maximum value</td><td>32.696°</td><td>32.070°</td><td>32.157°</td></tr><tr><td>Standard deviation</td><td>6.825°</td><td>7.172°</td><td>7.282°</td></tr><tr><td>Variance</td><td>46.582</td><td>51.441</td><td>53.035</td></tr></tbody></table>

## 5. Conclusion

We introduce MVAE, a Motion-conditioned Variational Encoder model designed to generate diverse humanoid locomotion animations based on movements captured by Virtual reality controllers. In contrast to existing works such as [PBV21, JPY*23, WYF*21, SW22], where conditions are provided by labels, our model enables finer artistic control over the generated animations. This approach

empowers animators, akin to choreographers, to articulate desired animation characteristics such as step frequency, amplitude, jump height, length, timing, and more. By encoding and understanding these desired traits, the model produces diverse animations that align with the specified characteristics, facilitating the creation of stylized animations in an intuitive and efficient manner. We have explored various architectures and hyperparameters to identify optimal learning and real-time application models.

The proposed model has certain limitations. Firstly, the training dataset comprises only 71 animations, potentially restricting the range of animations the model can generate. While the model is capable of interpolation, certain desired animations may be beyond its capabilities. For instance, static backflips diverge significantly from the learned animations and may not be reproducible by the model. Expanding the dataset and soliciting inputs from more participants to cover a broader spectrum of actions could mitigate this limitation. Additionally, most conditions in the training dataset are oriented along the z-axis. Animations involving backward movement along this axis may pose challenges for the model due to a lack of similar training instances. Similarly, the model may struggle to generate animations involving multiple actions within a single movement sequence. For example, seamlessly transitioning from running to stopping to performing a static jump in one continuous motion is currently infeasible. Future work aims to address these limitations by developing a model capable of understanding and generating complex, multi-action animations within a single movement sequence, thereby enabling smoother scene expression for animators.
## References

[AOGF*21] AHUJA K., OFEK E., GONZALEZ-FRANCO M., HOLZ C., WILSON A. D.: Coolmoves: User motion accentuation in virtual reality. *Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies* (2021). 2

[BMCC22] BORDIER J.-B., MIRABILE A., COURANT R., CHRISTIE M.: Smart motion trails for animating in vr. *IEEE International Conference on Artificial Intelligence and Virtual Reality* (2022). 2

[BRL*16] BYUNGKUK C., ROGER B. I. R., LEWIS J. P., SEOL Y., HONG S., EOM H., JUNG S., NOH J.: Sketchimo: Sketch-based motion editing for articulated characters. *ACM Transactions on Graphics* (2016). 2

[CDML19] CANNAVO A., DEMARTINI C., MORRA L., LAMBERTI F.: Immersive virtual reality-based interfaces for character animation. *IEEE Access* (2019). 2

[CĀS19] CICCONE L., ĀŪZTIRELI C., SUMNER R. W.: Tangent-space optimization for interactive animation control. *ACM Transactions on Graphics* (2019). 2

[DDC*21] DITTADI A., DZIADZIO S., COSKER D., LUNDELL B., CASHMAN T., SHOTTON J.: Full-body motion from a single head-mounted device: Generating smpl poses from partial observations. *International Conference on Computer Vision* (2021). 2

[DKP*23] DU Y., KIPS R., PUMAROLA A., STARKE S., THABET A., SANAKOYEU A.: Avatars grow legs: Generating smooth human motion from sparse tracking inputs with diffusion model, 2023. URL: https://arxiv.org/abs/2304.08577, arXiv:2304.08577. 2

[GB08] GLASS K., BANGAY S.: Automating the creation of 3d animation from annotated fiction text. *Proceedings of Computer Graphics and Visualization* (2008). 2

[GCR13] GUAY M., CANI M.-P., RONFARD R.: The line of action: an intuitive interface for expressive character posing. *ACM Transactions on Graphics* (2013). 2

[GPAM*14] GOODFELLOW I. J., POUGET-ABADIE J., MIRZA M., XU B., WARDE-FARLEY D., OZAIR S., COURVILLE A., BENGIO Y.: Generative adversarial networks. *Advances in Neural Information Processing Systems* (2014). 3

[GRC19] GARCIA M., RONFARD R., CANI M.-P.: Spatial motion doodyles: Sketching animation in vr using hand gestures and laban motion analysis. *ACM Conference on Motion, Interaction, and Games* (2019). 2, 3

[GRGC15] GUAY M., RONFARD R., GLEICHER M., CANI M.-P.: To appear in acm tog 34(4).space-time sketching of character animation. *ACM Transactions on Graphics* (2015). 2

[GXMZ14] GOUVATSOS A., XIAO Z., MARSDEN N., ZHANG J. J.: Automatic 3d posing from 2d hand-drawn sketches. *Proceedings - Pacific Conference on Computer Graphics and Applications* (2014). 2

[HSK16] HOLDEN D., SAITO J., KOMURA T.: A deep learning framework for character motion synthesis and editing. *Transactions On Graphics* (2016). 3

[HSKJ15] HOLDEN D., SAITO J., KOMURA T., JOYCE T.: Learning motion manifolds with convolutional autoencoders. *SIGGRAPH Asia* (2015). 3

[Huy09] HUYNH D.: Metrics for 3d rotations: Comparison and analysis. *Journal of Mathematical Imaging and Vision* (2009). 4

[Inc] INCORPORATED A. S.: Mixamo. URL: https://www.mixamo.com/. 4

[JPY*23] JI B., PAN Y., YAN Y., CHEN R., YANG X.: Stylevr: Stylizing character animations with normalizing flows. *IEEE Transactions on Visualization and Computer Graphics* (2023). 2, 11

[JSQ*22] JIANG J., STRELI P., QIU H., FENDER A., LAICH L., SNAPE P., HOLZ C.: Avatarposer: Articulated full-body pose tracking from sparse motion sensing. *European conference on computer vision* (2022). 2

[LAZ*22] LI P., ABERMAN K., ZHANG Z., HANOCKA R., SORKINEN-HORNUNG O.: Ganimator: neural motion synthesis from a single sequence. *Transactions on graphics* (2022). 2, 3

[LCM20] LAMBERTI F., CANNAVO A., MONTUSCHI P.: Is immersive virtual reality the ultimate interface for 3d animators? *IEEE Computer* (2020). 2

[LZCdP20] LING H. Y., ZINNO F., CHENG G., DE PANNE M. V.: Character controllers using motion vaes. *Transactions on Graphics* (2020). 2, 3

[MGT*19] MAHMOOD N., GHORBANI N., TROJE N. F., PONS-MOLL G., BLACK M. J.: Amass: Archive of motion capture as surface shapes. *ICCV* (2019). 4

[MMFN20] MIRZAEI M. S., MESHGI K., FRIGO E., NISHIDA T.: Animgan: A spatiotemporally-conditioned generative adversarial network for character animation. *International Conference on Image Processing* (2020). 3

[MPD22] MALEK-PODJASKI M., DELIGIANNI F.: Adversarial attention for human motion synthesis. 3

[PBV21] PETROVICH M., BLACK M. J., VAROL G.: Action-conditioned 3d human motion synthesis with transformer vae. *International Conference on Computer Vision* (2021). 2, 3, 4, 11

[PYA*23] PONTON J. L., YUN H., ARISTIDOU A., ANDUJAR C., PELECHANO N.: Sparseposer: Real-time full-body motion reconstruction from sparse data. *ACM Transactions on Graphics* 43, 1 (Oct. 2023), 1âĂŞ14. URL: http://dx.doi.org/10.1145/3625264, doi: 10.1145/3625264. 4

[PYAP22] PONTON J. L., YUN H., ANDUJAR C., PELECHANO N.: Combining motion matching and orientation prediction to animate avatars for consumer-grade vr device. *Computer Graphics Forum* (2022). 2

[SKK09] SHIM H., KANG B., KWAG K.: Web2animation - automatic generation of 3d animation from the web text. *ACM International Conference on Web Intelligence* (2009). 2

[SW22] SONG L. X. Z., WANG D.: Actformer: A gan-based transformer towards general action-conditioned 3d human motion generation. 2, 3, 11

[TBvdP06] THORNE M., BURKE D., VAN DE PANNE M.: Motiondooles: An interface for sketching character motion. *ACM SIGGRAPH* (2006). 2

[TGBCO22] TEVET G., GORDON B., BERMANO A. H. A. H., COHEN-OR D.: Motionclip: Exposing human motion generation to clip space. *European Conference on Computer Vision* (2022). 2

[THR11] TAYLOR G. W., HINTON G. E., ROWEIS S. T.: Two distributed-state models for generating high-dimensional time series. *Journal of Machine Learning Research* (2011). 3

[WWY22] WINKLER A., WON J., YE Y.: Questsim: Human motion tracking from sparse sensors with simulated avatars. In *SIGGRAPH Asia 2022 Conference Papers* (Nov. 2022), SA âĂŽ22, ACM, p. 1âĂŞ8. URL: http://dx.doi.org/10.1145/3550469.3555411, doi:10.1145/3550469.3555411. 2

[WYF*21] WEN Y.-H., YANG Z., FU H., GAO L., SUN Y., LIU Y.-J.: Autoregressive stylized motion synthesis with generative flow. *Conference on Computer Vision and Pattern Recognition* (2021). 3, 6, 11

[YKSF20] YE H., KWAN K. C., SU W., FU H.: Aranimator: In-situ character animation in mobile ar with user-defined motion gestures. *SIGGRAPH* (2020). 2

[ZBL*18] ZHOU Y., BARNES C., LU J., YANG J., LI H.: On the continuity of rotation representations in neural networks. *Conference on Computer Vision and Pattern Recognition* (2018). 6

## 6. Appendix
<figure><img src="image_17.png" /><figcaption>(a) Generated animations: the red character corresponds to the first input, characterized by a slow step frequency (almost 100 frames between steps) and extended duration (400+ frames), while the blue character corresponds to the second input, featuring sudden, high-frequency steps (50 frames between each steps, 250 frames long).</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>(b) The initial input introduced to the model demonstrates a reduced step frequency and prolonged duration, yielding the red animation.</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>(c) The subsequent input showcases an increased step frequency and a shorter duration, producing the green animation</figcaption></figure>

<caption>**Figure 8:** Results showcase the model's capability to consider varying input frequencies and lengths, effectively generating walking animations corresponding to these diverse input parameters.</caption>
<figure><img src="image_20.png" /><figcaption>(a) Generated animations. Green character correspond to first input with high amplitude height while red character correspond to second inputs with small amplitude height.</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>(b) The first input presented to the model features a high amplitude for y axis, resulting in the generation of a green animation.</figcaption></figure>

<figure><img src="image_22.png" /><figcaption>(c) The second input exhibits the small amplitude for y axis, leading to the generation of a red animation.</figcaption></figure>

**Figure 9:** Results showcase the model's capability to consider varying input amplitude for jump height animation generation.