MoSA: Motion-Coherent Human Video Generation via Structure-Appearance Decoupling
Authors: Haoyu Wang
Hao Tang
Donglin Di
Zhilu Zhang
Wangmeng Zuo
Feng Gao
Siwei Ma
Shiliang Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Human Video Generation
Coherent Video Generation
Human Video Dataset
Primary area: generative models
TL;DR: We propose a structure-appearance decoupling framework with human-aware dynamic control, dense tracking constraints, 3D contact constraints,and a large-scale human video dataset to generate motion-coherent human videos from given prompts.

Abstract: Existing video generation models predominantly emphasize appearance fidelity while exhibiting limited ability to synthesize complex human motions, such as whole-body movements, long-range dynamics, and fine-grained human–environment interactions. This often leads to unrealistic or physically implausible movements with inadequate structural coherence. To conquer these challenges, we propose MoSA, which decouples the process of human video generation into two components, i.e., structure generation and appearance generation. MoSA first employs a 3D structure transformer to generate a human motion sequence from the text prompt. The remaining video appearance is then synthesized under the guidance of this structural sequence. We achieve fine-grained control over the sparse human structures by introducing Human-Aware Dynamic Control modules with a dense tracking constraint during training. The modeling of human–environment interactions is improved through the proposed contact constraint. Those two components work comprehensively to ensure the structural and appearance fidelity across the generated videos. This paper also contributes a large-scale human video dataset, which features more complex and diverse motions than existing human video datasets. We conduct comprehensive comparisons between MoSA and a variety of approaches, including general video generation models, human video generation models, and human animation models. Experiments demonstrate that MoSA substantially outperforms existing approaches across the majority of evaluation metrics.

Published as a conference paper at ICLR 2026
MoSA: MOTION-COHERENT HUMAN VIDEO GENER- ATION VIA STRUCTURE-APPEARANCE DECOUPLING
Haoyu Wang Hao Tang', Donglin Di Zhilu Zhang',
Wangmeng Zuo",
Feng Gao", Siwei Ma', Shiliang Zhang
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China 2 School of Arts, Beijing, China
Li Auto, Beijing, China,
4 Harbin Institute of Technology, Harbin, China
haoyu. wang@ stu. pku. edu. en, aaotanglpku edu. donglin dd1 @gmail 1 com, cs sazllzaan@outlook. com, wmzuo@hit www.sseghttttt....c..mmm en, gaofêpku. edu. cn, swma@pku. edu...dduuccoo en, slzhang. jdl@pku.edu.co
ABSTRACT
Existing video generation models predominantly emphasize appearance fi- delity while exhibiting limited ability to synthesize complex human motions, such as whole-body movements, long-range dynamics, and fine-grained hu- man can-eenvronmmet interactions. This often leads to unrealistic or physically im- plausible movements with inadequate structural coherence. To conquer these chal- lenges, we propose MoSA, which decouples the process of human video gener- ation into two components, i.e., structure generation and appearance generation. McSA first employs 3D structure transformer to generate human motion se- quence from the text prompt. The remaining video appearance is then synthesized under the guidance of this structural sequence. We achieve fine-grained control over the sparse human structures by introducing Human-Aware Dynamic Con- trol modules with a dense tracking constraint during training. The modeling of human-environment interactions is improved through the proposed contact con- straint. Those two components work comprehensively and appearance fidelity across the generated videos. This paper also contributes large-scale human video dataset, which features more complex and diverse mo- tions than cxisting human video datasets. We conduct comprehensive compar- isons between MoSA and variety of approaches, including general video gener- ation models, human video generation models, and human animation models. Ex- periments demonstrate that MoSA substantially outperforms existing approaches across the majority of evaluation metrics.
ensure the structural
INTRODUCTION
General human video generation from text or image prompts (Jiang et al. 2023 Song et al. 2024
Wang et al.) 2025a Zhang et al., 2024b Huang et al. 2024a) has recently garnered substantial research interest due to its broad application potentials. A core challenge lies in maintaining the structural plausibility of the human body, particularly for complex motions such as whole-body dy- namies, long-range movement, and comman-nnvirommen interactions, while preserving appearance
fidelity in the generated videos. (Chefer et al.) 2025).
Existing video generation models (Kong et al. 2024 Yang et al. 2024b Wan et al. 2025 often lack explicit guidance from human structural priors and are typically trained with noise reconstruction objectives in pixel space. Previous studies (Chefer et al. 2025 Jeong al. 2024) have demon- strated that this paradigm leads to an overemphasis on appearance fidelity while neglecting human structural coherence, resulting unrealistic human motion in the generated videos, as shown in
"Corresoonding author.

---

Published as a conference paper at ICLR 2026
Mochi
CelehV
Cog VidenxX
HumanVid
Wan 2.1
TikTnk
Ours
Comparison with existing video generation models
(b) Comparison with open-source human video datasets
Figure 1:
Illustration of the motivation. (a) shows sampled frames from videos generated with the
prompt "running", where existing works (Genmo 2024; Yang et al. 2024b) struggle to generate human videos with reasonable structures. (b) compares existing human video datasets (Wang et al.) 2024d, Dafarian & Park (20TT) and our Movid, where existing datasets mostly focus on facial or upper-body regions, or consist of vertically oriented dance videos. More samples Movid are pro- vided Fig. and supplementary materials.
Fig. Ta). Since human appearance and motion convey different cues, they should adhere to differ- generation paradigms. This intuition leads to our MoSA, which generates human video through decoupling the structure and appearance generation. Specifically, as it is difficult to generate human videos with complex motions directly from the given text prompt, we first generate human motion structures conditioned on the text prompt. Given the sparse motion structures, MoSA subsequently synthesizes the visual appearance.
To generate the human motion structures, we first generate 3D human keypoints via 3D structure transformer, which is pretrained on large-scale human motion datasets (Guo al. 2022] Zhang al.) 2025b Plappert et al. 2016). 3D human keypoint sequences are hence projected into 2D skeleton sequence. Compared with directly producing Song et al.) 2024 Wang et al., 2025a), such as skeleton sequences, leveraging 3D human keypoints presents better robustness and accuracy because: i) 3D structure transformer leverages human priors to efficiently generate human keypoints, thereby ensuring the plausibility of the predicted human structure, and ii) by operating in 3D space, it can exploit implicit depth information to maintain structural plausibility in the presence of limb occlusions.
structural representations (Huang et al. 2024a
The subsequent video appearance is then synthesized under the guidance this structural sequence. As the skeleton representation inherently provides only sparse structural guidance, its capability for fine-grained supervision in subsequent appearance generation is limited. To address issue, we propose the Human-Aware Dynamic Control module. It employs learnable dynamic weight pre- dictors to generate weight maps corresponding to the skeleton features, hence further refines these maps using tailored mask loss. This mask loss encourages the propagation of sparse skeleton guidance across the entire motion region and assigns dynamic weights to different spatial locations, the sparse skeleton. In addition, previous stud- ies (Chefer et al.) 2025 Jeong et al.) 2024) have shown that relying solely on the noise prediction objective during training may cause models to favor appearance fidelity over motion coherence. To mitigate this issue, we introduce dense tracking loss aimed enhancing the model's ability to pre- serve coherent motion structures. We further incorporate a contact constraint to accurately model human-environmen interactions.
thereby enhancing the fine- mne-grai controllability
Besides the above methodology, this paper also contributes a novel human video dataset presenting complex human motions. Most existing human video datasets (Yu et al. 2023 Wang et al. 2024d Li et al. 2024) primarily capture facial and upper-body movements with relatively simple move- ments, illustrated in Fig. [b). Similarly, existing dance datasets (Jafarian & Park 2021) exhibit
2

---

Published as a conference paper at ICLR 2026
restricted background diversity and motion complexity, which confines corresponding generation
approaches (Hu 2024; Hu et al. to dance videos and often requires auxihary pose inputs. The fact that most existing open-
2025; Zhu et al.] 2024 Gan et al. 2025 Zhang et al. 2024b)
human video datasets (Wang et al. 2024d Li et al. 2024 Jafarian & Park 2021 Yu et al. 2023)
primarily focus on simple movements, makes models trained on such datasets struggle generate realistic and physically plausible motions. We thus introduce MoVid, a novel dataset comprising 30K human motion videos exhibiting diverse action categories and complex motions.
We employ Mo as the training set for MoSA, and conduct comprehensive comparisons between MoSA and a broad spectrum of baselines, including general video generation models, human video generation models, and human animation models. The results demonstrate that MoSA substantially outperforms existing methods across most evaluation metrics, achieving superior performance in measures such as FVD, CLIP similarity, and VBench scores. As shown in Fig. T a),, our method presents more reasonable body structure and more fluent motion.
In summary, our key contributions lies in three aspects: i) This is original effort on struc- ture-appearance decoupling framework for human video generation. As shown extensive experi- ments, disentangling structural consistency from appearance synthesis benefits physically plausible human video generation. ii) Our proposed modules like fluman-Award Dynamic Control, dense tracking loss, and contact constraint lead to effective implement of the proposed decoupling framework. They work well to enhance the fine-grained structural guidance, the modeling of mo- tion coherence, as well as numan-ennironnen structed to offer more diverse and complex motions than existing datasets. Extensive experiments demonstrate the superior performance of our method. The code and dataset will be released.
interactions. iii) A large-scale dataset MoVid is
2 RELATED WORK
Human Video Generation. Most existing human video generation approaches rely on additional
inputs beyond the text prompt, such as reference images of the target person (He et al. [0244] Yuan
et al. 2024| Zhang et al. 2222a c Cao al. (025). , driving pose sequences (Hu 2024 Liu al.
2025
Hu et al. [2025] Wang et al. 2024d Gan et al. 2025 Zhang et al.) 2024b; Zhu et al. 2024).
or speech conditions Sun al.] 2025 Lin al. 2025 Tian et al. 2025 Meng et 2024a Cui
let al. 2024). Among these works, ID AD-Ammttor (He et al. (2024) introduced framework capa- ble of generating identity-specific videos from reference facial images, along with corresponding identity-orientedd dataset. Building upon this foundation, subsequent studies have further advanced the task. For instance, ConsisII) (Yuan et al. 2024) proposed a frequency decomposition strategy to improve identity fidelity. Moreover, AnimateAnyone (Hu) 2024) introduces an additional skele- ton sequence as input to animate static human images. Building on this, AnimateAnyone2 (Hu eet al. 2025) extends the framework to support environment-aware generation, enabling more co- herent background migration. AnimateAnywhert (Liu et al. 2025) incorporates a camera motion learner model background movement, thereby enhancing the realism of generated videos. How- ever, the aforementioned methods are typically constrained to generating minor facial or upper-body movements, or specialized for vertically oriented dance videos.
Some recent work (Song et al.) 2024 Wang et al. 2025a Huang et al. 2024a Liang et al., 2025) has focused on more general text-drivee human motion video generation, but due the limitations of human video datasets (Wang et al. 2024d: Li et al. 2024), it is also difficult to generate realistic and physically compliant motion. To address these challenges, we propose structure-appearance decoupling framework for generating motion-coherent human videos, and construct a large-scale dataset Vidd to support the learning of complex human motion.
Human Motion Generation. Text-driven human motion generation aims to produce sequence of human keypointss conditioned on a given text prompt, which can then be transformed into structural representations such as skeletons. Several existing approaches (Zhang et al.) 2023a; Chen et al.)
2023 Tevet et al. 2022 Zhang et al. 2024a Yuan et al. 2023 Meng et al. 2024b Guo et al. 2024 Fan et al. (2025) leverage models trained on 3D pontion-annotatee datasets (Guo et al. 2022 Plappert Zhang et al. 2025b) to generate 3D keypoint sequences. For exam- extends latent diffusion models to support text-to-motion generation, while T2M-GPT (Zhang et al. 2023a) employs a generative pre-trained transformer (Radford et al,
et al. 2016 Lin et al.
2023
ple, MLD (Chen et al. 2023)
2018) as the backbone and utilizes vector-quantized variational autoencoder (Van Den Oord al.
3

---

Published as a conference paper at ICLR 2026
p': "A girl is running
3D Structure Transformer
g5
AdaLN
Structural Full Att.
AdaLN
Structrue Gen Block
Human-Awar Dynamic Control (HADC)
DA
Feed Forward
a
p
up"
AdaL.
AdaLN
Uk m
0
Structure Generation Branch Gs
54
sk+1
"A girl is running up stairs urban"
Appearance- Gen Block
Appearance- Gen Block
Multiplication Addition
AdaLN
Attention
AdaL.
T5
Feed Forward
7p
HADC
Concatenation
Projection
AdaLN
AdaLN
Motion
Dataset
ZT
Output V
Appearance Generation Branch Ga
Figure 2: Overview of the proposed MoSA. Given text prompt p. first employ 3D structure transformer to generate structure sequence, which is subsequently encoded as structural features guide the appearance generation. To further enhance motion consistency, introduce human- aware dynamic control modules. For brevity, the Gate modules in blocks have been omitted.
2017 to encode and reconstruct keypoint features. In addition, several studies (Wang et al.) 2025a 2024c| Song et al., 2024) have explored directly generating 2D keypoints or skeleton sequences as representations of human motion.
These generated motion representations serve structural priors for video generation and contribute improve the plausibility of human motion. However, existing methods typically produce relatively sparse motion representations, limiting their capacity for fine-grained control. To address this, introduce Human-Ave Aware Dynamic Control modules that adaptively emphasize human-relevan re- gions and incorporate dense tracking loss to further enhance the model's 's ability to learn structurally coherent and temporally coherent motion patterns.
3 METHODOLOGY
While recent video generation models (Yang al. 2024b Kong al. 2024 Wan al., 2025)
have achieved impressive visual quality, they frequently fail to generate physically plausible and structurally coherent human motion, especially in scenarios involving complex movement. Our MoSA aims to enhance structural consistency while preserving high-quality visual appearance, We present our method using the text-to-video setting the primary example, while providing details of the image-to-video variant in Sec. H of the appendix.
Specifically, we begin by introducing the preliminaries of video generation models in Sec. Afterwards, we describe the structure-appeaarance decoupling in detail in Sec. 3.2 To enhance the fine-grained controllability of sparse structural guidance, we propose the Human-Aware Dynamic Control (HADC) modules in Sec. 3.3 Finally, the training objectives are summarized in Sec. 3.4 which encompass the proposed dense tracking loss and contact constraint.
3.1 PRELIMINARY
Video Generation Model. Diffusion transformer (DiT) Peebles & Xie 2023) based generative models have attracted increasing attention due to their strong performance and scalability. Building on this (Yang et al.) 20246] Wan et al., 2025), we adopt DiT as the backbone and extend it to support motion-coherent human video generation.
4

---

Published as a conference paper at ICLR 2026
Given the text prompt p and the corresponding video V, first employ pretrained T5 coder (Raffel et al. 2020) to obtain the text embeddings Ep E R BBx Bxxx,,, where B, L, denote the batch size, token length and token dimensions, respectively. The video V is encoded using a VAE encoder & and Gaussian noise is added to the resulting latent to obtain the noisy latent Zo RBBxPXXHXX where F, C, H, W denote the temporal, channel and spatial dimensions, respectively, The embeddings 2p and latent 2, are then fed into the backbone Gg. The training ob- jective is to learn noise predictor Go that estimates the added noise e. The loss function formally defined as
= Ee,e.a.. ||e Go(vorz + aae; Ep, 012 where is a random time step and a represents the predefined variance schedule.
During inference, random Gaussian noise 27 ~ ~( 1) is sampled as =o. and then ør is iteratively denoised through the backward process conditioned on the text embeddings Ep- The resulting latent 2o is subsequently decoded by the VAE decoder D to generate the output video V'.
3.2 STRUCTURE-APPEARRR
DECOUPLING
Our method decouples the generation process into two branches, i.e., structure generation and ap- pearance generation as illustrated in Fig. 2 Following parts proceed to provide detailed description of this decoupled generation paradigm.
3.2. STRUCTURE GENERATION BRANCH
Given a text prompt p. the structure generation branch G, aims to produce human motion struc- ture that aligns with the motion semantics conveyed by p. Since the prompt p may also include appearance-related descriptions such as details of the surrounding environment, which are irrelevant motion structure, we preprocess the input by extracting motion-specine subset of p. denoted as p'. As illustrated in Fig. []] retains only motion-relevant information. This filtering process can be performed automatically using Large Language Model (Yang et al. 2024a) or specified by the user. The resulting motion-spectio prompt p' is then used as the input the structure generation branch.
This branch is dedicated to generating human structures from the motion-specific prompt p' without incorporating appearance information. However, directly training text-driven model to produce 2D structural sequences, such skeletons, often fails to guarantee the anatomical plausibility of the generated structures. We thus reformulate the text-driven structure generation task as a 3D keypoint sequence generation task. This formulation could i) leverage human priors to efficiently generate K human keypoints, thereby ensuring the plausibility and coherence of the predicted human structure, and ii) benefit from the generation in 3D space. In other words, it can utilize implicit depth infor- mation to preserve structural consistency in scenarios involving limb occlusions. Once the keypoint sequence is obtained, it is rendered into 2D space. Following common practice conditional hu- man video generation (Hu et al., 2025 Gan et al. (025). we convert the keypoint sequence into a skeleton representation, which serves as the final structural guidance Ja- The above process can be formally described as follows:
9s Projection( (2f"
(2f:
where TTD denotes the 3D structure transformer, and = represents Gaussian noise sampled from a standard normal distribution (). Following previous work (Fan al.) 2025 Meng et al., 2024b), 9 adopts the same autoregressive architecture and is first pretrained on million-scal mo- tion datasets. Details are described in Sec. of the appendix.
After obtaining J.. we employ it as an additional control signal encoding human structural informa- tion to guide subsequent appearance generation. To effectively encode and incorporate this condition into the appearance generation process, we introduce specialized structure generation blocks within the DiT architecture. The overall process can be formalized as follows:
I:N
(E
(3)
where - (I: = 1,.., N) represents the output of the k-th structure generation block in G., which is used guide the subsequent appearance generation, and N is the total number of such blocks.
5

---

Published as a conference paper at ICLR 2026
3.2.2 APPEARANCE GENERATION BRANCH
The appearance generation branch Go designed to synthesize realistic video content conditioned on and 51:N capturing both the environmental appearance and human subjects, while preserving the realism of human motion. As described in Sec. 3.1, a pretrained T5 encoder is employed to extract the text embedding &p from the prompt and the initial latent 2y is sampled from ((0, I), both of which are served as the input of this appearance generation branch.
To enhance the controllability of structure guidance, particularly in the context of sparse skeleton representations, we introduce the Human-Aware Dynamic Control (HADC) modules within this branch. After T steps of iterative denoising, the resulting latent 2u is decoded by VAE decoder D (Kingmal 2013), yielding human videos V with coherent motion and high-fidelity appearance that align with the semantics of the text prompt
3.3 HUMAN-AWARE DYNAMIC CONTROL
The human structure features ..:N can be utilized as auxiliary conditions Ga to enhance the plausibility of human motion. However, the structural guidance ys. typically represented as a sparse skeleton, lacks the expressiveness required for fine-grained motion control.
To address this limitation, Human-Aware Dynamic Control (HADC) modules, which are inserted between adjacent DiT blocks within the appearance generation branch. Each k-th HADC module takes the structural signal s² the intermediate video latent af, and the text embedding z produced by the preceding DiT block as input. By leveraging the structural cues embedded s² the HADC module refines at to enable fine-grad mne-grained control over human motion, producing motion-enhanced latents a that are subsequently passed to the next DiT block. Specifically, we design human-aware dynamic weights predictor pk which aims to i) facilitate the propagation of x throughout the whole motion region in of and ii) assign spatially-varyiin -varyin control weights to these human motion regions within af i...
w pk a
(4)
where w denotes the human-aware dynamic control weights. Leveraging w², the sparse skeleton feature s can exert fine-grained control over the video latents, i.e.,
t = a 4 (w ④ 8*)
(5)
where a denotes the motion- enhanced video latents, while b and @ indicate element- wise addition and multiplication, respectively. In addition, to ensure the effectiveness of w we design a learnable network th to convert w into mask latents and constrain it through mask loss L. during training, ie.,
Lm =
N ||2 (w*) E
where U4 (w*) denotes the predicted mask latent m.. M denotes the ground truth video mask, and E(M) the corresponding latent m. By incorporating the proposed HADC modules, the consis- tency of human motion within the video latent at is significantly improved, as illustrated in Fig.
3.4 TRAINING OBJECTIVES
During the training of these two branches, the pretrained 3D structure transformer G.TH is excluded, and the skeleton sequence extracted from the ground truth video V is directly used as the structural condition 9s. To further improve the model's capacity for learning temporally coherent motion, we introduce a dense tracking loss Liracks i.e.,
||u U Un-
Ltrack
where 4,, are video timestamps, U and represent the 2D tracks corresponding to the gen- erated video V and the ground truth video V, respectively. These tracks are extracted by Co-
[t denotes temporal weighting function that assigns higher
Tracker3 (Karaev al. 20241.

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative comparison with existing methods. Lower FVD values indicate better perfor- mance, whereas higher values on the other metrics correspond to better results. Bold indicates the best performance, and underline denotes the second-best.
Subject Consistency
Background Consistency
Motion Smoothness
Dynamic Degree
Imaging Quality
Method
FVD
CLIPSIM
ModelScope
1945
0.2739
90.87%
93.41%
96.22%
48.57%
60.12%
VideoCrafter2
1959
0.2801
93.43%
97.01%
97.31%
35.71%
60.32%
LaVie
1778
0.2895
93.80%
95.51%
97.21%
53.73% %
62.57%
Mochi
1207
0.2903
94.67%
95.32%
97.75%
51.14%
54.65%
CogVide VideoX
1 1360
0.2899
93.75%
94.02%
97.78%
51.42%
62.98%
Hunyuan Video
1235
0.2948
94.41 %
95.17%
98.95%
50.42%
58.13%
Wan 2.1
1251
0.2951
94.43%
95.55%
98.36%
51.71%
65.21
Ours
1093
0.3035
96.83% %
97.43%
99.25%
52.86%
65.43%
"A man skating an indoor ice rink. Windows allow natural light to flood in, creating a dim atmosphere.
male gymmast in shorts walks to blue tram- poline and jumps, all under the bright natural light."
ModelSccoe
VC2
La
Mochi
CogVideoX
Hunyuan
Wan 2.1
Ours
Figure 3: Visual comparison with existing video generation models. For clarity, VideoCrafter2 (Chen et al. 2024) is denoted as VC2, and Hunyuan Video (Kong et al. 2024) is de- noted as Hunyuan. demonstrates that our method is capable of generating realistic human motion with with reasonable structures, while simultaneously preserving high fidelity in video appearance.
loss weights to longer time intervals, thereby encouraging the model to better capture long-range motion dependencies. S represents the set of time intervals, and (tu, (.,) E S. i.e.
((1 (.) S < To, To. to (.),
(8)

---

Published as a conference paper at ICLR 2026
Table 2: Effect on the Wan2.1 base model.
Table 3: Effect of the contact constraint.
Models
FVD
CLIPSIM
Constraint Lcont
FVD
CLIPSIM
Wan 2.1
1251
0.2951
x Ours
1108
0.3021
Our Decoupling Framework
1108
0.3044
1093
0.3035
Table 4: Effect of the decoupling framework.
Table 5: Effect of the HADC modules.
Structure Generation Branch
FVD
CLIPSIM
HADC Modules
FVD
CLIPSIM
x
1262
0.2971
x 1188
0.2973
2D Structure Generation
1230
0.2998
w/o Lm Ours
1112
0.3009
Ours
1093
0.3035
1093
0.3035
Table 6: Effect of the dense tracking loss Ltrock-
Table Necessity of our MoVid.
Dense Tracking Loss Livack
FVD
CLIPSIM
Training Dataset
FVD
CLIPSIM
x
1172
0.3009
x
1360
0,2899
Static Weights Ours
1114
0.3016
Human Vid
1217
0,2949
1093
0.3035
MoVid (Ours)
1093
0,3035
where T. is the video length. Additionally, propose 3D contact constraint Lcont to further enhance the modeling of human environment interactions. Due to page limitations, full details are presented in Sec. B of the appendix. The overall training objective is formulated
= L + Cm + Atrack Ctrack + Ccont
(9)
where Ams Atrack and Acont the weights used to balance different loss terms.
4 EXPERIMENTS
4.1 EXPERIMENTAL DETAILS
Datasets. Existing human video datasets are largely restricted to simple movements, limiting mod- els' ability to synthesize realistic and physically plausible motion in complex scenarios. To over- come these limitations, we curated MoVid, a dataset of 30K real-world human motion vidcos with annotations. Details are provided in Sec. Cf of the appendix. For evaluation, collected more than 300 text prompts spanning motion types and environmental contexts following previous work.
Evaluation Metrics. Following previous work, adopt Fréchet Video Distance (FVD) (Un- terthiner et al., 2019) and CLIP similarity (CLIPSIM) (Radford et al. 202I) to measure the per- formance. Furthermore,
utilize VBench (Huang al. (2024b) to conduct more comprehensive assessment of model performance across multiple dimensions, including subject consistency, back- ground consistency, motion smoothness, motion dynamics, and visual quality.
4.2 COMPARISON WITH EXISTING METHODS
We adopt Cog VideooXX-BBB Yang et al. 2024b) as the backbone of the appearance gener- ation branch, with additional implementation details provided in Sec. D of the appendix. We first compare our method with video generation models, including ModelScope (Wang et al. 2023 VideoCrafter2 (Chen et al. 2024). LaVie (Wang et al. 2024a), Mochi (Genmo 2024). CogVideoX-5B-11 (CogVideox) (Yang et al. 2024b), Hunyuan Video (Kong et al. 2024). and Wan2.I-T2V-149 (Wan 2.1) (Wan et al. 2025), accompanied by a user study (Sec.) E.3 For text- driven human video generation methods (Huang et al. 2024a| Wang et al. 2025a Song et al. 2024) without public implementations, we conduct qualitative comparisons based on their released videos, if available (Sec. E.2). We further compare MoSA with pose-driven human animation methods (Hu) 2024 Wang et al. 2024d Men et al.] (025) Tu et al. 2025), with additional results and visualizations
presented in Sec. E.5 of the appendix. More video types shown in Sec. G
8

---

Published as a conference paper at ICLR 2026
"A dressed in athletic attire stretching his legs park, silhouetted against a free and bridge.
"An elderly with short, graying hair performing exercise black mat a well-equipped gym.
+ Our MoSA Wan 2.1
Figure 4: Effect of our decoupling framework MoSA when applied to Wan 2.1 Wan et al. 2025
"A man with long black hair, is seated on blue rug in modern living room, playing classical guitar."
"A woman in red sports bra and pink shorts performs leg exercise gym, with leg lifted high.
x
2D Structure
Ours
Figure 5: Effect of structure appearance decoupling. For experiments that employ the structure generation branch, we also visualize the corresponding generated human structure.
structure-appearance
Quantitative Comparison. Quantitative comparisons with existing methods are shown in Tab. Compared with previous models, our MoSA achieves excellent performance in various metrics.
Qualitative Comparison. Visual comparisons are presented in Fig. 3 As illustrated, our approach capable of generating realistic human motion, both for basic actions such walking and jumping, as well as for more complex activities like skating. In contrast, existing methods often struggle to generate physically plausible motion with coherent structural integrity for complex movements. More visual results are provided in Sec.
ABLATION STUDY
5.1 EFFECT OF OUR DECOUPLING FRAMEWORK WHEN APPLIED TO WAN 2.1
The proposed decoupled generation framework exhibits high compatibility with state-of the-art video generation models and can be seamlessly integrated to enhance their performance. Specifi- cally, the structure branch focuses producing plausible and coherent motion, while the appearance branch leverages the intrinsic strengths of these models to synthesize realistic textures and environ- mental details. To further validate its effectiveness, we apply our MoSA to Wan 2.1 by incorporating the proposed components and finetuning the base model. As shown in Tab. and Fig. the results demonstrate both the transferability and the effectiveness of our MoSA framework.
5.2 EFFECT OF STRUCTURE- -PREUURRAEAPPEECEEEEEE DECOUPLING
Tab. presents the analysis for the effect of structure-appearance decoupling. In the first row, the structure generation branch removed, and the base model is directly finetuned using MoVid. The second row utilizes outputs from independently trained 2D skeleton sequence generation model as the structure guidance 9.. Details this model are described in Sec. of the appendix. Visual comparisons provided in Fig. 5 Compared with 2D structure generation, our 9 effectively

---

Published as a conference paper at ICLR 2026
preserves structure correctness, demonstrated by the missing leg in the second-row the left side of Fig. Additionally, our G:th leverages the depth information in 3D space to maintain spatial coherence in scenarios involving limb occlusion. For example, in the second row the right of the right leg is incorrectly placed behind the left, leading to an implausible body structure.
Fig. 5
5.3 EFFECTS OF OTHER PROPOSED COMPONENTS
We further investigate the effects of other proposed components, including the HADC modules (Tab. 5 and Sec. F.2), the dense tracking loss (Tab. 6 and Sec. F.3) the contact constraint (Tab, and Sec. [F.4), and the necessity of the MoVid dataset (Tab. and Sec. H.5). Due to page limitations,
detailed analyses and extended visual comparisons are presented in the Sec. Sec.Fof the appendix.
CONCLUSION
We present MoSA, aructure-appearacce decoupling framework for realistic human video gen- eration. 3D structure transformer synthesizes motion structures guide appearance generation, while human-aware dynamic control and a dense tracking loss enhance fine-grained motion coher- ence. To better capture human environment interactions, we introduce a contact constraint. Further- more, we curate large-scale human video dataset to overcome the limitations of existing datasets. Extensive experiments show that MoSA consistently outperforms previous approaches.
DISCUSSIONS AND FUTURE WORK
Generating complex inter-person contact and fine-grained hand interactions has long been chal- lenging problem for video generation models. Built upon a structure apperaraeee disentanglement paradigm, our method substantially improves the quality of multi-person interactions and fine- grained motion synthesis. As shown in Fig. 17] the girl's bowstring-pulling action in the second row and the two-person jumping and ball-contesting scene in the third row demonstrate clear en- hancements.
Nevertheless, we found that generating highly intricate hand motions remains challenge, where current methods may still produce artifacts such as distorted or blurred finger structures. How- ever, our structure-appearance disentanglement paradigm naturally compatible with incorporating denser structural cues, such as hand keypoints. This inherent compatibility offers promising path- way to enhance the realism finger-level actions by integrating such detailed guidance.
Our further investigation reveals that the primary obstacle lies in the training data. Existing mo- tion datasets used for training 3D structure transformers typically include only SMPL body joints. Consequently, incorporating hand keypoints would necessitate augmenting these datasets with ad- ditional 3D annotations for hand joints to enable effective model training. We believe this a very worthwhile direction explore and will pursue it in future work, with extending the proposed MoVid dataset.
ETHICS STATEMENT
This work focuses on human video generation and the collection of a human motion dataset. All data were obtained from public sources. The dataset will be released for research under license that prohibits misuse such as surveillance, deepfake creation, or other harmful applications. This research adheres to institutional ethical standards and legal regulations.
REPRODUCIBILITY STATEMENT
We provide the core codes and data samples in the supplementary materials, The appendix provides additional implementation details of our work. Furthermore, the pre-preee pre-processing steps for the datasets described in the supplementary materials. The full codebase and MoVid dataset will be released to the public upon final preparation, ensuring that the results in this paper can independently verified.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported in part by under Grant 2023-JCIQ-LA-001-088. in part by the Nat- ural Science Foundation of China under Grant U20B2052, Grant 61936011, in part by under 2025ZD160) 161DD600000 in part by the Okawa Foundation Research Award, in part by the Ant Group Research Fund, and in part by the Kunpeng&Ascend Center of Excellence, Peking University.
REFERENCES
Chenjie Cao, Jingkai Zhou, Shikai Li, Jingyun Liang, Chaohui Yu, Fan Wang. Xiangyang Xue, and Yanwei Fu, Uni3e: Unifying precisely 3d-enhanced camera and human motion controls for video generation. arXiv preprint arXiv:2504.44897 2025.
Hila Chefer, Uriel Singer, Amit Zohar, Yuval Kirstain, Adam Polyak, Yaniv Taigman, Lior Wolf, and Shelly Sheynin. Videojam: Joint appearance-motion representations for enhanced motion generation in video models. arXiv preprint arXiv:2502.02499) 2025.
Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 7310-7320, 2024.
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands motion diffusion in latent space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18000-18010, 2023.
Jiahao Cui, Hui Li, Yun Zhan, Hanlin Shang, Kaihui Cheng, Yuqi Ma, Shan Mu, Hang Zhou, Jingdong Wang, and Siyu Zhu. Hallo3: Highly dynamic and realistic portrait image animation with video diffusion transformer. arXiv preprint arXiv:2412.00755 2024.
Ke Fan, Shunlin Lu, Minyue Dai, Runyi Yu, Lixing Xiao, Zhiyang Dou, Junting Dong, Lizhuang Ma, and Jingbo Wang. Go to zero: Towards zero-shot motion generation with million-scale data. arXiv preprint 07X1v:2507.07095, 2025.
Qijun Gan, Yi Ren, Chen Zhang, Zhenhui Ye, Pan Xie, Xiang Yin, Zehuan Yuan, Bingyue Peng, and Jianke Zhu. Humandit: Pose-guided diffusion transformer for long-form human motion video generation. arXiv preprint (rXiv:2502.0484/ 2025.
Genmo. Mochi 1: A new sota in open-source video generation models. 2024.
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5152-5161, 2022.
Chuan Guo. Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Gener- ative masked modeling of 3d human motions. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1900-1910, 2024.
Xuanhua He, Quande Liu, Shengju Qian, Xin Wang. Tao Hu, Ke Cao, Keyu Yan, and Jie Zhang. Id-animator: Zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404. 15275, 2024.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08413 08405, 2016.
Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji. Zhao Xue, et al. Cogvlm2: Visual language models for image and video understanding. arXiv preprint arXiv:2408. 16500, 2024.
Li Hu. Animate anyone: Consistent and controllable image-to-vidde synthesis for character anima- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8153-8163, 2024.
11

---

Published as a conference paper at ICLR 2026
Li Hu, Guangyuan Wang. Zhen Shen, Xin Gao, Dechao Meng, Lian Zhuo, Peng Zhang. Bang Zhang, and Liefeng Bo. Animate anyone 2: figh-fidelity character image animation with environment affordance. arXiv preprint 25Xt:22502.06145 2025.
Hsin Hsin-Ping Ping Huang, Yang Zhou, Jui-Hsien Wang, Difan Liu, Feng Liu, Ming Ming-Hsuan Yang, and Zhan Xu. Move-in-2d: 2d-conditioned human motion generation. arXiv preprint arXiv:2412.133:3 2024a.
Ziqi Huang, Yinan Hc, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Champaisit, et al. Vbench: Comprchensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11807-21818,, 2024b.
Yasamin Jafarian and Hyun Soo Park. Learning high fidelity depths of dressed humans by watching social media dance videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2021.
Hyeonho Jeong. Chun-Hao Paul Huang, Jong Chul Ye, Niloy Mitra, and Duygu Ceylan. Track4gen: Teaching video diffusion models to track points improves video generation. arXiv preprint arXiv:2412.06016 2024.
Yuming Jiang, Shuai Yang, Tong Liang Koh, Wayne Wu, Chen Change Loy, and Ziwei Liu. Text2performer: Text-driven human video generation. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pp. 22747-22757, 2023.
Nikita Karaev, Iurüi Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pscudo-labelling real vidcos. arXiv preprint arXiv:2410.1183/ 2024.
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv: 1e...111226.41 2013.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision, pp. 4015-4026, 2023.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprini arXiv:2412.03600 2024.
Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, et al. Openhumanvid: large-scale high-quality dataset enhancing human-centre video generation. arXiv preprint oxXiv22412.00115, 2024.
Jingyun Liang, Jingkai Zhou, Shikai Li, Chenjie Cao, Lei Sun, Yichen Qian, Weihua Chen, and Fan Wang. Realismotion: Decomposed human motion control and video generation in the world space. arXiv preprint arXiv:2508.0855 2025.
Gaojie Lin, Jianwen Jiang, Jiaqi Yang, Zerong Zheng, and Chao Liang. Omnihuman-1: Re- thinking the scaling-up of one-stage conditioned human animation models. arXiv preprint arXiv:2502.01061 2025.
Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x: A large-scale 3d expressive whole-body human motion dataset. Advances in Neural Information Processing Systems, 36:25268-25280 2023.
Xiaoyu Liu, Mingshuai Yao, Yabo Zhang, Xianhui Lin, Peiran Ren, Xiaoming Li, Ming Liu, and Wangmeng Zuo. unimateanywhere: Rouse the background human image animation. arXiv preprint arXiv:2504. 19834, 2025.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17110:1711.05101 2017.
12

---

Published as a conference paper at ICLR 2026
Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Mimo: Controllable character video syn- thesis with spatial decomposed modeling. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
Rang Meng, Xingyu Zhang, Yuming Li, and Chenguang Ma. Echomimiev2: Towards striking, simplified, and semi-body human animation. arXiv preprini urXiv:2411.10000 2024a.
Zichong Meng. Yiming Xie, Xiaogang Peng, Zeyu Han, and Huaizu Jiang. Rethinking diffusion for text-driven human motion generation. arXiv preprint arXiv:2411. 2024b.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Pexels. https//wwww.pexel..cmm. URL
Matthias Plappert, Christian Mandery, and Tamim Asfour. The kit motion-languan dataset. Big data, 4(4):236-252, 2016.
Alec Radford, Karthik Narasimhan, Tim Salimans, lyya Sutskever, al. Improving language under- standing by generative pre-training, 2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140))1-67, 2020.
2020.
Kunpeng Song, Tingbo Hou, Zecheng He, Haoyu Ma, Jialiang Wang. Animesh Sinha, Sam Tsai, Yaqiao Luo, Xiaoliang Dai, Li Chen, et al. Directorlim for human-centric video generation. arXiv preprint ArXiv:2412. 14484, 2024.
Yasheng Sun, Zhiliang Xu, Hang Zhou, Jiazhi Guan, Quanwei Yang, Kaisiyuan Wang, Borong Liang Yingying Li, Haocheng Feng. Jingdong Wang, et al. Cosh-dit: Co-speech gesture video synthesis via hybrid audio-visual diffusion transformers. arXiv preprint arXiv:2503.09942, 2025.
Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen and Amit Bermano.
Human motion diffusion model. arXiv preprint arXiv:2209. /4916, 2022.
Linrui Tian, Siqi Hu, Qi Wang, Bang Zhang, and Liefeng Bo. Emo2: End-effector guided audio- driven avatar video generation. arXiv preprint arXiv:2501.10687 2025.
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spa- tiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision, pp. 4489-4497, 2015.
Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Sta- bleanimator: High-quality preesevng human image animation. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for vidco generation. 2019.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wec Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20111 2025.
13

---

Published as a conference paper at ICLR 2026
Boyuan Wang, Xiaofeng Wang, Chaojun Ni, Guosheng Zhao, Zhiqin Yang, Zheng Zhu, Muyang Zhang, Yukun Zhou, Xinze Chen, Guan Huang, et al. Humandreamer. Generating controllable human-motion videos via decoupled generation. arXiv preprint arXiv:2503.2400: 24026,, 2025a.
Haoyu Wang, Zhilu Zhang, Donglin Di, Shiliang Zhang, and Wangmeng Zuo.
Mv-vton: Multi- view virtual try-on with diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 7682-7690, 2025b.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5294-5299599 5306, 2025c.
Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Mod- elscope text-to-video technical report. arXiv preprint arXiv:2308 20:::057780777,, 2023.
Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang. Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. International Journal of Computer Vision, pp. 1-20, 2024a.
Yihan Wang, Lahav Lipson, and Jia Deng. Sea-raft: Simple, efficient, accurate raft for optical flow. In European Conference on Computer Vision, pp. 36-54. Springer, 2024b.
Yuan Wang, Zhao Wang, Junhao Gong, Di Huang, Tong He, Wanli Ouyang, Jile Jiao, Xuetao Feng, Qi Dou, Shixiang Tang, et al. Holistic-motion2d:: Scalable whole-bodi human motion generation in 2d space. arXiv preprint arXiv:2406. /2553, 2024c.
Zhenzhi Wang, Yixuan Li, Yanhong Zeng. Youqing Fang, Yuwei Guo, Wenran Liu, Jing Tan, Kai Chen, Tianfan Xue, Bo Dai, al. Humanvid: Demystifying training data for camera-controllable human image animation. In The Tiirty-eight Conference on Neural Information Processing Sys- tems Datasets and Benchmarks Track, 2024d.
Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 15211:2412.15115. 2024a.
Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effective whole-body pose estimation with two-stages distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4210-4220, 2023.
Zhuoyi Yang. Jiayan Teng. Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, al. Cogvidcox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072 2024b.
YouTube. https://www.yuutube.cm
URL httts:::: http:///wwwwwbbbbee com
Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Weidong Cai, and Wayne Wu. Celebv-text: A large-scale facial text-video dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14805-14814, 44,4, 2023.
Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan. prentity text-to-video generation by frequency decomposition. arXiv preprint arXiv:2411.17440) 2024.
Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guider human motion diffusion model. Proceedings of the IEEF/CVE international conference on computer vision, pp. 16010-16021, 2023.
Polina Zablotskaia, Aliaksandr Siarohin, Bo Zhao, and Leonid Sigal. Dwnet: Dense warp-based network for pose-guided human video generation. arXiv preprint arXiv:1910.09139 2019.
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. Generating human motion from textual descriptions with discrete representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14730-14740, 2023a.
14

---

Published as a conference paper at ICLR 2026
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. IEEE transac- tions pattern analysis and machine intelligence, 46(6):4115-4128, 2024a.
Yuechen Zhang, Yaoyang Liu, Bin Xia, Bohao Peng. Zexin Yan, Eric Lo, and Jiaya Jia. Magic mirror: Id-preserved video generation in video diffusion transformers. arXiv preprint arXiv:2501.03923 033933,, 2025a.
Yuhang Zhang, Yuan Zhou, Zeyu Liu, Yuxuan Cai, Qiuyue Wang, Aidong Men, and Huan Yang. Fleximo: Towards flexible text-to-human motion video generation. arXiv preprint arXiv:2411.19454 2024b.
Yuhong Zhang, Jing Lin, Ailing Zeng, Guanlin Wu, Shunlin Lu, Yurong Fu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x++: A large-scale multimodal 3d whole-body human motion dataset. arXiv preprint arXiv:2501.0500 2025b.
Yunpeng Zhang, Qiang Wang, Fan Jiang, Yaqi Fan, Mu Xu, and Yonggang Qi. Fantasyid: Face knowledge enhanced id id-rreseg video generation. arXiv preprint 13X19.250..13995, 2025c.
Zhilu Zhang, Haoyu Wang, Shuai Liu, Xiaotao Wang, Lei Lei, and Wangmeng Zuo. Self-supervised high dynamic range imaging with multi-exposure images in dynamic scenes. arXiv preprint arXiv:2310.01840, 2023b.
Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. Celebv-hq: A large-scale video facial attributes dataset. In European ference on computer vision, pp. 650-667. Springer, 2022.
Shenhao Zhu, Jumming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference Computer Vision, pp. 145-162. Springer, 2024.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
The content of this appendix involves:
Details of the 3D Structure Transformer in Sec. A
Details of the proposed 3D contact constraint in Sec.
B including its differential process. Description of the proposed Mo dataset, including data cleaning, annotation, and statistic com-
parison in Sec. C
Implementation details of MoSA in Sec. D
More comparative experiments in Sec. including more comparisons with general video diffu- sion models, text-driven human video generation models, commercial video generation models, human animation models, and generation results with occlusion situations.
More ablation studies in Sec. including details of the 2D structure generation model, final visualization of the HADC module outputs, effect of human-aware dynamic control modules, effect of dense tracking loss, effect of the contact constraint, necessity of the MoVid dataset, effect of the selected camera poses, and the effect of the data from different views our dataset..
More video types generated by MoSA Sec.
Image-to-video generation variant of MoSA in Sec.
Evaluation more motion-centrie metrics in Sec.
LLM usage declaration in Sec.
A DETAILS OF 3D STRUCTURE TRANSFORMER
We employ a 3D structure trans- former g to generate 3D hu- man keypoints. Following previ-
p': "A girl is running up
X3
ous work (Fan et al. 2025 Meng jet al. 2024b), 9THH adopts uni- lied autoregressive architecture and pretrained on million-scale mo- tion datasets (Guo et al. 2022 Fan al., (225)), as illustrated These large-scale datasets encompass a wide variety of motion categories, including diverse and complex hu- man movements, which enable ro- bust pretraining. During inference, the model takes as input a motion-specifi text prompt p' and initial motion latents T1.N- The prompt p' is first encoded by text tokenizer (Raffel et al. (2020) to obtain conditional embeddings, which, together with II,N, are fed into an autoregressive transformer (Vaswani] 2017) to predict the final motion latents. These latents are then decoded by the motion decoder into human keypoint sequence aligned with the semantics of p'. Finally, standardized skeleton sequence gs is derived from the predicted keypoints.
X2
Autoregressive Transformer
Motion Decoder
Text Tokenizer
Fig.
Figure 6: Architecture of the 3D structure transformer Gmm. The final skeleton frames are obtained after projection.
B DETAILS OF THE PROPOSED 3D CONTACT CONSTRAINT
To ensure physically plausible human-environment interactions, we introduce 3D contact loss that penalizes unrealistic interpenetrations. The computation proceeds in four steps. First, each video frame is lifted into 3D point cloud representation using pretrained VGGT (Wang et al. 2025c) model. Formally, for the f-th frame, we obtain point set P3 p; E R3 Second, human and scene points are separated based on 2D segmentation masks. Each 3D point p; is projected onto the image plane using the intrinsic matrix K₁ and extrinsic matrix E, predicted by the VGGT model. the projected pixel (u, va) lies within the segmented human region My. the point is assigned to the human set H y; otherwise, is assigned to the scene set Sy. Third, scene points from all frames
16

---

Published as a conference paper at ICLR 2026
aggregated to reconstruct a mesh M, via convex hull estimation. This mesh is then converted into a signed distance function (SDF), denoted SDF(.), where negative values indicate locations inside the scene surface and positive values denote outside regions. Finally, the 3D contact loss is defined. If human points penetrate the scene (SDF(h) < r), the loss penalizes the penetration depth; otherwise, it encourages human points remain close the scene surface by minimizing their distance to the mesh. The resulting loss is formally expressed as:
Σ SSDP( T/, if if 0, hell min SDF(h) T, otherwise, hEMl
Conn
(10)
where H thE Hy SDF(h) <) denotes the of penetrating points, and is the pene- tration threshold (set to zero by default). The final contact loss Leont calculated by summing the values across all frames and then normalizing by the number of frames. By incorporating the constraint Leont during training, we effectively mitigate issues such as penetration and other physi- cally implausible behaviors that frequently arise in the generation of complex human-cnvironment interactions.
Differential Process. For the f-th frame, the 3D contact loss is defined as Eq. 10
The propagation path is follows, where h denotes human point, V denotes the generated video, 0 denotes the model parameters.
c SDF(h) - V +
(11)
(1) The process begins with the loss function C.ant itself. Since is set to 0, for human point h, the loss form can be simplified to:
f.h = max(0, SDF(h))
(12)
(2) The partial derivative of this loss with respect to the SDF value is non-zero only in the case of interpenctration:
ach (-1, OSDF(h)
SDF(h) O, 0, otherwise.
(13)
(3) The next step is propagate the gradient from the SDF value the 3D coordinates (æ, y, 2)
of the point h. The gradient of a scalar field like the SDF with respect to point's coordinates is vector:
SDS @SDF(h) Ōh
(14)
This gradient vector provides the most efficient direction to displace the point order move towards the exterior of the scene mesh, thereby resolving the penetration.
Using the chain rule, we combine the gradients from the previous two steps to compute the gradient of the loss with respect to the 3D coordinates of the human point h:
achA Oh
aChA
@SDF(h)
(15)
OSDF(h)
Oh
Sc 9h.
(4) Then, the gradient can be backpropagated through the layers of VGGT [2] obtain the gradient with respect to its input, i.e. the generated video frame V":
ach.
a
ah
(16)
OV
ØV
(5) Finally, the generated video frame V is the output of model, V MoSAA 0). With acts the gradient 01 computed, continue the hackpropagation through the MoSA network to
17

---

Published as a conference paper at ICLR 2026
obtain the gradients with respect to all of its learnable parameters 0:
ac
ac oV
OV" 00
(17)
00
These final gradients are then used by the optimizer to update the model's parameters.
C IN-DEPTH DESCRIPTION OF THE MOVID DATASET
C.I DATA CLEANING AND ANNOTATION
Following HumanVia (Wang et al. 2024d), we construct the MoVid dataset by collecting a large number of source videos from public sources (Pexels.) YouTube.) using motion-related keywords. To ensure data quality, we employ the "Motion Smoothness" "Dynnane Degree" and "Image Qual- ity" metrics from VBench (Huang et al. 2024b) to filter out low-quality samples. Coarse-grained textual annotations are then generated using CogVLM2 (Hong et al.) (2024), followed by manual verification to ensure accuracy. For spatial supervision, we apply SAM (Kirillov et al.) 2023) to obtain human masks and utilize DWPose (Yang et al. 2023) to extract human keypoints and skele- tons. To ensure motion richness, we compute the average offset ö of each keypoint across the entire video and discard samples with S 0.1. Additionally, videos containing only simple motions, such as isolated facial or upper-body movements, are excluded, retaining examples that exhibit complex dynamics. Through this rigorous pipeline, curate dataset comprising approximately 30K high- quality real-world video clips encompassing more diverse and complex human motions and scenes.
C.2 DATASET STATISTICS
Tab. g provides a detailed comparison with several representative real-world human video datasets,
including CelebV-HQ (Zhu al. 2022), CelebV-Text (Yu et al. (023). TikTok (Jafarian & Park
2021) UBC-Fashion (Zablotskaia et al. 2019), IDEA-400 (Lin et al. 2023) and Human Vid (Wang
al. 2024d). Most existing datasets, such as CelebV (Yu et al.) 2023 Zhu al. 2022). Human- Vid (Wang et al. 2024d). and the recent OpenHuman Vid (Li et al. 2024), predominantly focus on facial or upper-body motions. addition, other datasets are often constrained by limited mo- tion diversity or specific video formats. For example, TikTok (Jafarian & Park] (021) and UBC- Fashion (Zablotskaia al., 2019) primarily consist of vertically oriented videos with restricted mo- tion ranges. Others, such as IDEA-400 (Lin et al.) 2023). lack fine-grained and accurate textual annotations, which limits their applicability in text-conditioned generation tasks. To address these limitations, we introduce the Mo Vid, high-quality whole-body human video dataset with millions of frames. Examples are provided in dataset sample.zip of the provided supplementary materials, and are also shown in Fig. 7
Tab. also shows the comparison in terms of action complexity and action types. For Action Com- plexity, inspired by VMBench, we first segment the regions related to human and calculate the average optical flow of pixels in these regions. We utilize SEA-RAFT (Wang et al. 2024b) to esti- mate optical flow, which serves as a measure of the magnitude of human motion within the dataset. Specifically, smaller optical flow values correspond to subtler movements, whereas larger values are indicative of more significant and intricate motion. Results demonstrate that our proposed MoVid dataset has more complex human actions. For Action Types, CelebV-HQ and CelebV-Text are re- stricted to facial movements, while the Tik dataset focus on a singular category dance. The UBC-Fashion dataset, in turn, is composed exclusively of standing persons exhibiting only subtle motion. For the IDEA-400, Human Vid, and our proposed MoVid datasets, we first generate text annotations for videos using CogVLM2, and then analyze these annotations to identify all verb and verb-object phrases related to human, calculating the number of unique action types in each dataset. The results show that MoVid dataset has richer variety of human motion. The above results provide strong evidence that the Mo Vid dataset encompasses a broader, more diverse, and more complex spectrum of human motion videos.
18

---

Published as a conference paper at ICLR 2026
Table Comparison of MoVid with existing representative real-world human video datasets.
Dataset
Clips
Resolution
Action Types
Action Complexity
Fine-grate Line-grained Caption
Ce
35K
512x512
Facial type
0.6891
Text
70K
512x512
Facial type
0,7070
Text
TiKTok
340
x04 1080
Dancing
0,6816
UBC-Fashion
500
720x964
Standing
0,3321
IDEA-400
12K
720P
0,5969
Human Vid
20K
1080P
7K
0.6669
MoVid (Ours)
30K
1080P
17K
1.1124
Text
Video samples of our Vid Datasct
Figure 7: Examples randomly selected from the proposed MoVid dataset.
D IMPLEMENTATION DETAILS OF MOSA
We use CogVideoX-SB-T2Y Yang et al. 2024b) the base model of the appearance generation branch. During training. we freeze the original weights and train only the structure generation blocks and HADC modules. We also present the results for the version using Wan 2.1 (Wan et al., 2025) as the base model Tab. 2 and Fig. 4 following the same training settings as described above. For the proposed HADC modules, p comprises three linear layers interleaved with two activation
19

---

Published as a conference paper at ICLR 2026
"A person repeatedly lifting dumbbells a well-lit gym filled with mirrors and fitness equipment..
ModelS IScpe
VC2
La
1
Mochi
CogVidoXX
Hunyuan
Wan 2.1
Ours
Figure More visual comparison with existing video generation models.
functions (Hendrycks & Gimpel 2016 Wang et al. 2025b Zhang et al. 2023b). 24 shares a similar
architecture with P. but includes additional up-sampling layer followed by 3D convolution layer (Tran et al. 2015) at the end. During training, the skeleton sequence is extracted from the input video V. We train and test resolution of 720x480, and train for 20,000 iterations on NVIDIA A800 GPUs with batch size of 16. The AdamW (Loshchilov & Hutter 2017) optimizer used with a learning rate of lc-5. We the loss weights Am. Atrack and Acone 0.001, 0.01 and 10.0, respectively.
E MORE COMPARATIVE EXPERIMENTS
E.1 VISUAL COMPARISON WITH EXISTING VIDEO GENERATION MODELS
We provide more visual comparison with ModelScope WWang al. 2023
VideoCrafter2 (Chen
ct al. 2024), La (Wang et al. 2024a), Mochi (Genmo) 2024), CogvideoX (Yang et al. 2024b),
Hunyuan Video (Kong al. 2024) and Wan 2.1 (Wan al. 2025) in Fig. 8 and Fig. 9
We also show
20

---

Published as a conference paper at ICLR 2026
"A young man stands on an indoor baskethall court, holding basketball with both hands at his chest
VC2 ModelScope
La
Mochi 1
CogVideoX
Hunyuan
Wan 2.1
Ours
Figure 9: More visual comparison with existing video generation models.
the video results in the provided video.mp4. The video.mp4 in the supplementary materials presents additional and more compelling visual comparisons. As illustrated, MoSA demonstrates superior capability in generating human vidcos with coherent and physically plausible motion compared to existing approaches. Additionally, we alson showcase a broader range generation results to highlight the diversity and generalization ability of MoSA in video.mp4.
E.2 VISUAL COMPARISON WITH TEXT-DRIVEN HUMAN VIDEO GENERATION METHODS
Since existing text-driven human video generation methods (Wang et al. 2025a Huang et al. 2024a Song et al. 2024) are not yet open source, we perform visual comparison based their re released videos, accessible. Fig. [] presents a visual comparison with Move-in-2 2n-2D (Huang et al. 2024a) and HumanDreamer (Wang al.) 2025a). As shown, method generates more realistic and visually coherent results, demonstrating improved fidelity and motion consistency. The video com- is shown in video.mp4.
parison with Move-in-2D and
21

---

Published as a conference paper at ICLR 2026
"An elderly man with short, graying hair is performing exercise on black mat in a well- equipped gym
VC2 ModelScope
La
Mochi
CogVideoX
Hunyuan
Wan2.11 1
Ours
Figure 10: More visual comparison with existing video generation models.
E.3
USER STUDY
To enable a more comprehensive evaluation, we conduct manual assessment of the generated re- sults across different methods. Specifically, participants are presented with video samples generated by various models for each text prompt and are asked to select the most preferred video based on two criteria: Motion Quality, which assesses the realism and coherence of human motion, and Video Quality, which evaluates the overall visual fidelity and realism of the generated appearance. We then calculated the proportion of times each method selected as the best, As reported in Tab, and Tab. I0 our MoSA achieved the highest preference rates in both Motion Quality and Vidco Quality, demonstrating its superior performance in generating visually realistic and cochionn human videos.
22

---

Published as a conference paper at ICLR 2026
"A man hitting tennis return.
Move-in-21
Ours
"A man is performing a yoga pose on a mal, and he is seen moving his legs
HDreamer
Ours
Figure 11: Visual comparison with text-driven human video generation models Move-in-2D and HumanDreamer (HDreamer) on their released samples.
Table 9: User study with existing video generation models. Motion Quality assesses the realism and plausibility of human motion, while Video Quality evaluates the overall perceptual quality of the generated videos.
Method
Motion Quality
Video Quality 1
ModelScope (Wang al.
2023
3.91%
1.92%
VC2 (Chen et al. 2024)
4.14%
2.33%
LaVie Wang al.
2024a
10.62%
5.75%
Mochi
Wang ct al
2024a
8.38%
10.34%
Cog Video
Yang al. 2024b)
12.07%
16.01%
Hunyuan
Kong al. 2024
15.21%
15.55%
Wan Wan ct al.
2025
15.41%
18.98%
Ours
30.26% %
29.12%
E.4 COMPARISON WITH THE COMMERCIAL MODELS KLING AND SEEDANCE
To better illustrate the performance of our MoSA model, we provide comparison with Kling and Seedance in Fig. 12 Given that Kling and Seedance are commercial models requiring a queue for access, we only able to evaluate their performance within limited timeframe using text prompts from our paper. We observe that despite the impressive capabilities of commercial models like Kling and Seedance, they are still susceptible the issue of human anatomical distortion.
E.5 COMPARISON WITH HUMAN ANIMATION METHODS
We test the performance of human animation methods, incluing Animate Anyone (Hu 2024), Hu- man Vid (Wang et al. 2024d), MIMO (Men et al., 2025) and StableAnimator (Tu et al. 2025). as shown in Tab. These methods use the same 2D skeletons in MoSA as input, with reference im- ages generated from the text prompts. The results further demonstrate the superiority of our method. Furthermore, train additional I2V version of our MoSA the human animation task in which the 3D structure transformer removed during both training and testing. Following Human- Vid (Wang al. 2024d), then compare this version with pose-driven human video generation
23

---

Published as a conference paper at ICLR 2026
Table 10: User study with Move-in-2D and HrmanDreamer on their released videos.
Method
Motion Quality t Video Quality
Move-in-2D (Huang et al. 2024a)
24.3%
24.7%
HumanDreamer Wang et al. 2025a
26.2%
23.4%
Ours
49.5%
51.9%
"A man stretches on a wooden pier, placing one hand on his foot and the other reaching towards his leg."
Seedance
Kling
Ours
"A young woman, clad in sleeveless beige top, stretches her leg on a gravel path in a tranquil park." "
Seedance
Kling
Ours
Figure 12: Visual comparisons with Kling and Seedance. The watermark in the bottom right corner automatically added by their models.
methods using the same training and test sets Humanvid (Wang et al. 2024d), well identical prompts and pose conditions. Results are presented the Tab. 12 which further demonstrate the superiority of our method. Since MIMO and StimleAnimator trained self-collected in-house datasets, they are not included in these tables.
E.6 GENERATION RESULTS WITH OCCLUSTON SITUATIONS
The core role of the HADC modules is to enhance the model's fine-grained control over human motion. During training, we introduce mask loss Lm calculated from the ground-truth human masks. This process guides the model to learn how to effectively propagate the sparse structural guidance the corresponding visiible regions** of the human body in the image. Through this supervision, the model learns the association between the skeletal structure and the actual visible body parts.
24

---

Published as a conference paper at ICLR 2026
Table 11: Comparison with existing human animation methods on our test sets for human video generation in general scenarios.
Subject Consistency
Background Consistency
Motion Smoothness
Dynamic Degree
Imaging Quality
Method
FVD
CLIPSIM
Animate Anyone
1362
0.2850
94.09%
95.33%
97.23%
41.28%
57.06%
Vid
1374
0.2876
95. 15%
94.77%
97.42%
42.34%
54.05%
MIMO
1285
0.2904
94.82%
95,49%
97.38%
45.57%
53.83%
StableAnimator
1326
0.2895
95.37%
94.89%
97.88%
42.85%
56.96%
Ours
1108
0.3021
97.74%
97.37%
99.31 %
52.86%
64.68% %
Table 12: Comparison with pose-driven methods on the Human Vid dataset.
Method
SSIM
PSNR f
LPIPS
FVD
FID
Animate Anyone (Hu 2024
0.602
16.108
0.368
1248.4
97.74
Champ (Zhu et al. 2024
0,653
15.028
0.426
1985.2
100.59
Human (Wang ct
2024d
0.672
19.534
0.275
732.7
46.06
Uni3C (Cao et al. 2025
0.687
20.779
0.221
562.7
35.11
Ours
0.691
20.802
0.209
568.3
34.76
Consequently, this mechanism naturally handles scenarios with partial occlusions. During training, a part of the human body occluded by an object (e-g., leg hidden behind a table), the corre- sponding ground-truth human mask for that area will be absent. The model learns that even when the structural guidance indicates that a leg should present, it should avoid generating the leg's appearance in regions where it is actually occluded. In other words, the HADC module learns to render the visual appearance only the **non-occluded areas corresponding to the structural guidance. This capability is demonstrated properly handles person being occluded by objects.
the left side of Fig. 5 in paper, where model
To further substantiate the effectiveness of approach, add additional visual results in Fig. 18 in the appendix. These results showease more diverse occlusion scenarios and clearly demonstrate that MoSA can generate high-quality and physically plausible videos even when partial occlusions are present.
F MORE ABLATION STUDIES
F.1 DETAILS OF THE 2D STRUCTURE GENERATION
To assess the effectiveness of the 3D Human Structure Generator, we conducted an ablation study reported in Tab. of the main paper. The "2D Structure Generation" row indicates that the output of an additionally trained 2D skeleton generation model is used as 9.- Specifically, we fine- tune the base model Cog VideoX-BB using text-skeleton pairs from the MoVid dataset. However, this approach introduces two major issues: (1) directly generating 2D skeletons often compromises structural plausibility, as exemplified by the result on the left side of Fig. 5 in main paper, where a leg is entirely missing; (2) 2D representations struggle to resolve ambiguities arising from limb occlusions, frequently leading to physically implausible poses, such as the example on the right side of Fig. 5 where the right leg is incorrectly generated behind the left. In contrast, the 3D Human Structure Generator effectively mitigates these issues by leveraging depth information and human priors.
F.2 EFFECT OF HUMAN-AWAR DYNAMIC CONTROL
Tab. presents the quantitative analysis of the HADC modules. The first row indicates discarding the entire HADC modules, and the second indicates discarding only Lm- We can that they could improve the model's performance. To further illustrate their effect, the corresponding visual results shown in Fig. 13 a). When HADC modules removed, the generated human structure remains
25

---

Published as a conference paper at ICLR 2026
woman performs yoga calm lake, with leg lified behind her and arms reaching skyward." "
"A stretches wooden pier, placing hand his foot and the other reaching towards his leg.
x
wo Lm
Static
Ours
Ours
(a) Rffect of the HADC modules
(b) Fiffect of the dense tracking loss
Figure 13: Effect of the HADC modules and dense tracking loss. "Static" means applying fixed weight.
"A man equipped with backpack is walking fallen free trunk in dense forext." "
wil Eegont
Ours
Figure 14: Visual effect of the proposed contact constraint Ccont-
roughly aligned with the expected body layout but lacks fine- fine-grained motion control. Without L.s. details of corresponding human motion will also be unrealistic.
F.3 EFFECT OF THE DENSE TRACKING Loss
Tab. and Fig. (3) present the effect of the dense tracking loss. "Static weights" means applying a fixed weight different time intervals. By introducing temporal tracking optimization, the model's ability to learn temporally coherent motion is enhanced. Furthermore, assigning greater loss weights motion pairs with longer temporal intervals further improves the overall consistency of human motion across time.
F.4 EFFECT OF THE CONTACT CONSTRAINT
Tab. and Fig. 14 present the effect the proposed contact constraint. As illustrated in Fig. [4 the absence of the proposed contact constraint leads noticeable interpenetration when the human walks on a fallen tree trunk. In contrast, our method, with the constraint incorporated, generates more realistic motions without penetration artifacts. This highlights the effectiveness of the contact constraint in modeling physically plausible human environment interactions.
F.5 NECESSITY OF THE MOVID DATASET
To explore the necessity of the proposed Mo dataset, we conduct corresponding experiments, as shown in Tab. 7 The first row represents the performance of the base model without any additional human video dataset for training. The second row represents the performance after fine-tuning with the existing open-source human video dataset. We selected the widely used Human Vid dataset with various human motions and expanded to the same scale as MoVid. Fig. [5] illustraas the visual effects. As most existing datasets primarily capture simple motion like facial expressions upper-
26

---

Published as a conference paper at ICLR 2026
with long hair running on sandy beach, feeling the cool breeze on her face."
Human
Ours
Figure 15: Necessity of the MoVid dataset.
Table 13: Results of different camera poses when rendering the 3D human structure.
Method
FVD CLIPSIM
Fixed Camera
1093
0.3035
Fixed Camera
1089
0.3032
Time-varying Camera
1106
0.3029
Time-varying Camera
1096
0.3031
body movements, they fall short in supporting the generation of more complex motions. In contrast, models trained the MoVid dataset are able synthesize physically plausible and structurally coherent human motions. This is due to the fact that the proposed MoVid dataset covers wide variety of motion types and more complex dynamics, so that the model trained on can capture variety of motions in the real world, enhancing the model's ability to generate realistic and physically plausible human motion videos.
F.6 EFFECT OF THE SELECTED CAMERA POSES WHEN RENDERING THE 3D STRUCTURE
During inference, users can freely specify camera poses, including elevation angle, azimuth angle, and distance from the center of the coordinate system. These camera poses can be either fixed or time-varying. The quantitative results reported in the paper obtained by rendering the 3D human pose from fixed camera view. To further evaluate the impact of camera poses, we provide additional quantitative results under different fixed camera poses and two time-varying camera trajectories. It is important to note that this does not require retraining the model. The corresponding results are presented in Tab. [3] demonstrating that our method is robust to variations the rendering views of the generated 3D human structures.
In addition, for the camera movements of the background, we build on the capabilities of the Cogvideo-X X to generate coherent camera movements based on text descriptions and video content. Visual results are shown in Fig. 16
G MORE VIDEO TYPES GENERATED BY MOSA
In Fig. 17 we show examples of half-body and multi-person results generated by our MoSA. For half-body results, our method allows users specify which keypoints to retain when rendering the 3D pose, thereby generating partial body skeletons. For example, by discarding lower body key- points (knees, feet, etc.) during rendering, skeleton containing only the upper body can be gener- ated. For multi- results, during inference, we can optionally use the 3D structure transformer generate multi-person structures, based which generate subsequent videos.
27

---

Published as a conference paper at ICLR 2026
Dynamic camera trajectory for human
Dynamic camera trajectory for background
Figure 16: Visual results of dynamic camera trajectory for human and background.
H LMAGE-TO-VIIE VIDEEO GENERATION VARIANT OF MOSA
We additionally train an image-to-video generation variant based on the CogVideoX-5B-12 (Yang et al. (2024b). During training, frame is randomly sampled from the ground-truth video to serve as the image prompt, while the text prompt is retained as input. Moreover, data augmentation applied to the skeleton conditions by randomly translating and scaling the input skeleton sequences, enhancing the model's generative capability. During inference, the 3D structure transformer 9T¹ first produces the corresponding skeleton sequence based the text prompt. The text prompt, image prompt, and skeleton sequence are then fed into corresponding branches to generate the target video. Visualization results presented in Fig. 17
EVALUATION ON MORE MOTION-CENTRIC METRICS
We design more motion-centric metrics and add these evaluation metries to the ablation study of the structure-apy strueture-appeerrnce appearance decoupling paradigm, as shown in Table [4] Furthermore, we use these metrics to compare existing T2V models with our approach, and the corresponding results pre- sented in Table 15 The Pose Confidence metric is computed by applying the DWPose [1] model to extract human keypoints from the generated videos and averaging their confidence scores. Higher confidence indicates more accurate pose estimation, which further reflects the plausibility of the generated human poses, avoiding structural failures such missing limbs. Temporal Smoothness is derived by calculating the mean jerk of tracking points within human regions, where lower val- ues correspond to smoother motion. The Contact- Violation metric is evaluated using Qwen3- VL, powerful multimodal model equipped with extensive world knowledge. It identifies physically im- plausible artifacts such as interpenetrations the generated videos and computes the corresponding violation rate. Action Recognizability is also assessed using Qwen3-VL, which determines whether the generated actions faithfully match those specified in the text prompt, such as the text prompt "walks the blue trampoline and jumps" in Fig. Motion Plausibility and Action Correctness are the evaluation criteria used in the user study.
Moreover, have conducted user study, reported in Table 9 of the paper, to assess the motion quality of the generated videos. To more comprehensively demonstrate the advantages our ap- proach, further evaluate the motion plausibility and action correctness of the generated videos,
28

---

Published as a conference paper at ICLR 2026
Half-body videos and multi-person videos generated by our MoSA
Image-to-video generation results of our MoSA
Figure 17: Half-body and multi video generation results in this figure.
multi-person person results generated by our MoSA. We also provide image-to-
with the corresponding results presented in Table 16 tional evidence that our method exhibits clear superiority in generating high-quality human motion.
These motion-centre metrics provide addi-
Table 14: Effect of the decoupling paradigm, evaluating motion-centrie metrics.
Pose Confidence
Temporal Smoothness
Contact- Violation
Action Recognizability
Motion Plausibility
Action
Structure Branch
Correctness
X
0.885
0.122
2.04%
96.85%
5.09%
8.24%
2D Structure Generation
0.896
0.114
1.60%
98.42%
15.13%
15.60%
Ours
0.911
0.103
1.08 %
99.45%
79.78 %
76.16%
J LLM USAGE DECLARATION
Large Language Models (ChatGPT) used exclusively to improve the clarity and fluency of writing. They were not involved in research idcation, experimental design, data analysis, or inter- pretation. The authors take full responsibility for all content.
29

---

Published as a conference paper at ICLR 2026
Table 15: Quantitative comparison with more motion-centric metrics.
Method
Pose Confidence t Temporal Smoothness + Contact- Violation Action Recognizability
VideoCrafter2
0.831
0.155
4.06%
89.70%
LaVie
0.865
0.138
3.25%
92.14%
Mochi
0.886
0.121
2.04%
96.75%
VogVideoX
0,882
0.124
2.17%
96.47%
Humyuan Video
0.894
0.119
1.89%
97.28%
Wan2.1
0.893
0.117
1.62%
97.56%
Ours
0.911
0.103
1.08% %
99.45% %
Table 16: User study with more motion-aware metries.
Method
Motion Plausibility
Action Correctness f
VideoCrafter2
0.85%
2.04%
LaVie
3.37%
2.14%
Mochi 1
12.18%
11.59%
Cog VidexX
12.36%
14.82%
Hunyuan Video
16.45%
16.99%
Wan2.1
17.47%
17.14%
Ours
37.32% %
35.28 %
Table 17: Effect of the data from different views in our dataset.
Model
FVD |
CLIPSIM
w/o side-view data
1172
0,2959
back-view data
1165
0.2963
Ours
1096
0.3031
Table 18: Ablation study on the human animation task.
Model
SSIM 1
PSNR f
LPIPS
FVD
FID 1
w/o Strue. branch
0.677
20.603
0.266
682.1
42.74
w/o HADC
0.682
20.716
0.247
627.9
39.55
w/o Tracking loss
0.688
20.785
0.218
594.1
36.64
Ours
0.691
20.802
0.209
568.3
34.76
30

---

Published as a conference paper at ICLR 2026
Figure 18: Visual results in which the human body is partially occluded.
31


Reviews:
[{"review_id": "bxm0d5BIyv", "reviewer": "Reviewer_sL3c", "summary": "This paper introduces a video generation method with finegrained human motion controlled from text prompt. The key idea is to guide the appearance generation on a set of 2D keypoints, projected from 3d keypoints, with the assistant of a spatial variation control weight regions. It also introduces a large scale dataset with 30k diverse and complex human motion. The proposed method shows superior results over prior baselines.", "questions": "Please respond to the weekness section", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Decoupling of motion and appearance to guide the video generation to more complex human motion makes sense\n2. The Movid dataset is an important contribution to the field", "weaknesses": "1. The method uses motion corerspondences to constrain video consistency. These correspondences can be noisy, especially for textureless regions. Since these correspondences are only estimated from RGB frames, ie., the final output within a diffusion process, unrolling the gradient from multi-step diffusion is expensive. \n2. It is unclear how the 3D contract constraint is differentiable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZxAWnaI21P", "reviewer": "Reviewer_74M1", "summary": "This paper presents a novel framework which decouples the process of human video generation into structure generation and appearance generation.", "questions": "please refer to weakness", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The motivation is interesting, and decoupling structure from appearance addresses a long-standing challenge in text-to-human video generation.\n\nExperiments on multiple benchmarks show substantial improvements compared with existing methods.", "weaknesses": "Does the proposed method ensure physical realism—for example, the deformation of the trampoline shown in Figure 3?\n\nThe overall framework just combines known ideas (DiT backbone + structural priors + control modules), thus the innovation is mainly architectural integration.\n\nThis paper adopts a mask-based generation approach. How does this method handle problem where the human body is partially occluded?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vHIBTxfCia", "reviewer": "Reviewer_gVyP", "summary": "MoSA proposes a structure–appearance decoupling pipeline for human video generation: starting from a motion-specific prompt, a pretrained 3D structure transformer predicts 3D keypoint sequences that are projected to 2D skeletons and injected via specialized structure blocks to guide a DiT-based appearance model. To turn sparse skeleton cues into precise motion control, the authors add Human-Aware Dynamic Control (HADC) with a learned mask loss that assigns spatially varying weights over human regions. Motion realism is further enforced with a dense tracking loss for temporal coherence and a 3D human–scene contact constraint that discourages interpenetration. They also curate MoVid, a large-scale human video dataset emphasizing diverse and complex motions. Empirically, MoSA plugs into strong base models (e.g., CogVideoX, Wan 2.1) and shows consistent gains over prior work, and the paper additionally provides an image-to-video (I2V) variant in the appendix.", "questions": "Please see the weaknesses. Please correct me if my understanding about some points are wrong.\n\nOverall, it is a good attempt to incorporate human skeleton and motion generation methods into T2V human video generation. However, I still have some concerns about the evaluation part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper proposes to use 3D to 2D structure guidance, and HADC integrates cleanly into DiT backbones. Experiments shows that it works with CogVideoX and Wan 2.1. Removing the structure branch or replacing 3D with direct 2D pose generation degrades plausibility (missing/occluded limbs). \n\n2. To enhance motion coherence and physics, dense-tracking and contact terms are proposed and proved to be effect.\n\n3. Dataset contribution addresses a real gap (full-body, complex motion) and is used in comparisons.", "weaknesses": "1. Is projected 2D structure necessary for T2V? The authors argue that generating 3D keypoints and projecting to 2D improves plausibility and occlusion handling over directly predicting 2D skeletons. Ablations show failures like missing limbs when using a 2D structure generator, which supports that claim. However, the current metrics (FVD, CLIP similarity, and some VBench dimensions) are not targeted at structural correctness, so they don’t decisively isolate the benefit of 3D to 2D conditioning for T2V. Please complement FVD/CLIP/VBench with motion-centric metrics, e.g., pose detector, self-occlusion correctness, temporal smoothness/jerk on tracked keypoints, foot-sliding/contact-violation rate, and action recognizability via a pretrained classifier, to more directly validate structure quality. FVD, CLIP similarity and VBench-style scores are useful but still loosely connected to human motion fidelity, so they may not fully capture improvements in structure plausibility or contact realism. I recommend adding a small targeted user study (preference on “motion plausibility” and “action correctness”) and reporting standardized, motion-aware metrics as noted above to strengthen claims beyond general perceptual quality.\n\n2. Training–inference structure gap. During training, the 3D structure transformer is excluded and the model conditions on 2D skeletons extracted from the ground-truth video, whereas at inference it conditions on 2D projections of generated 3D structures. This creates a distribution shift that the paper doesn’t directly address. While Table 13 suggests robustness to user-specified fixed or time-varying cameras at inference, it would help to report results when training includes similar view randomization.\n\n3. Representation limits (sparse skeletons & multi-person). The approach is bottlenecked by the sparsity of skeletons: this paper explicitly notes that skeleton guidance lacks expressiveness for fine-grained control, requiring HADC to propagate structure cues into denser motion regions. This leaves open limitations for hands, garments, and challenging multi-person occlusions. The paper shows half-body and multi-person qualitative results, but a more systematic analysis of failure modes (e.g., hand interactions, inter-person contact) would clarify scope. A brief discussion of these limits and whether denser cues (SMPL-X joints, hand keypoints) would slot into the same framework would help readers understand when MoSA may still struggle.\n\n4. The authors do provide an image-to-video (I2V) variant in Appendix H to report an I2V “human animation” setting where the 3D structure transformer is removed. Is the good performance from better pretrained model (CogVideoX and Wan 2.1) or the model design? Because most of the compared methods use a U-net based pretraining model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sgBqBZ3ITD", "reviewer": "Reviewer_4dkv", "summary": "This paper proposed a motion-coherent human video generation model, called MoSA. Formally, MoSA decouples the structure and appearance generation through a 3D structure generation and an appearance generation branch. The structure generation branch comprises a text-to-motion Transformer, structure-gen blocks, and Human-Aware Dynamic Control (HADC) blocks, while the appearance generation is based on a pretrained video generation model. Mask loss and tracking loss are additionally incorporated to improve the performance. Detailed experiments show that the proposed method enjoys visually coherent results. Moreover, this paper proposed a new human video dataset, called MoVid, including 30K human motion videos exhibiting diverse action categories and complex motions.", "questions": "I recommend that the authors further compare their method to the SOTA human animation methods, such as Uni3C, to confirm their performance. Morevoer, an interesting discussion should be included for the unified training of both text-to-motion and controllable video generation.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper proposed an interesting idea for text-to-human generation, unifying both 3D text-to-motion and controllable video generation.   \n2. The writing quality of this paper is good, while qualitative and quantitative experiments are sufficient to cover SOTA video generation and user studies.\n3. The proposed dataset, MoVid, seems to be useful to the community.", "weaknesses": "1. As detailed in Appendix.D, both the video generation model and text-to-motion transformer are frozen during the training. It would be helpful and interesting to investigate whether the text-to-motion model can also be enhanced with this end-to-end training strategy.\n\n2. Although this paper included detailed experiments, some results are still lacking. For example, a qualitative ablation study for HADC and its visual outcomes can help the readers to understand the effect of HADC better.\n\n3. For the comparison of human animation methods, the authors only compare to Animate Anyone, Champ, and HumanVid with inferior video backbones. More in-depth discussion and comparison are required, such as the Wan-based Uni3C (text-to-smpl can be used to simulate the Uni3C's input conditions as discussed in their paper).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)