MoAlign: Motion-Centric Representation Alignment for Video Diffusion Models
Authors: Aritra Bhowmik
Denis Korzhenkov
Cees G. M. Snoek
Amir Habibian
Mohsen Ghafoorian
Venue: ICLR 2026 Poster (2026)
Keywords: video diffusion models
physical plausability
Primary area: generative models

Abstract: Text-to-video diffusion models have enabled high-quality video synthesis, yet often fail to generate temporally coherent and physically plausible motion. A key reason is the models' insufficient understanding of complex motions that natural videos often entail. Recent works tackle this problem by aligning diffusion model features with those from pretrained video encoders. However, these encoders mix video appearance and dynamics into entangled features, limiting the benefit  of such alignment. In this paper, we propose a motion-centric alignment framework that learns a disentangled motion subspace from a pretrained video encoder. This subspace is optimized to predict ground-truth optical flow, ensuring it captures true motion dynamics. We then align the latent features of a text-to-video diffusion model to this new subspace, enabling the generative model to internalize motion knowledge and generate more plausible videos. Our method improves the physical commonsense in a state-of-the-art video diffusion model, while preserving adherence to textual prompts, as evidenced by empirical evaluations on VideoPhy, VideoPhy2, VBench, and VBench-2.0, along with a user study.

Published as a conference paper at ICLR 2026
MOALIGN: MOTION-CENTRIE REPRESENTATION ALIGNMENT FOR VIDEO DIFFUSION MODELS
Aritra Bhowmik
Denis Korzhenkov
Cees G. M. Snoek
University of Amsterdam knhowiikuuva nl
Qualcomm Al Research
University of Amsterdam
dkorzhen@qt.i qua comm. com
c-g. m. snoe k @uva m1
Amirhossein Habibian
Mohsen Ghafoorian'
Qualcomm Al Research
Qualcomm AI Research
ahabibia@qti. qualcomm. com
mghafoor@gti. www.lii...coooo.m..mmm com
ABSTRACT
Text-to-video diffusion models have enabled high-quality video synthesis, yet of- ten fail to generate temporally coherent and physically plausible motion. A key reason is the models' insufficient understanding of complex motions that natural videos often entail. Recent works tackle this problem by aligning diffusion model features with those from pretrained video encoders. However, these encoders mix video appearance and dynamics into entangled features, limiting the benefit such alignment. In this paper, we propose motion-centric alignment framework that learns a disentangled motion subspace from a pretrained video encoder. This subspace is optimized true motion dynamics. We then align the latent features of text-to-video dif- fusion model this new subspace, enabling the generative model to internalize motion knowledge and generate more plausible videos. Our method improves the physical commonsense in state-of-the-ar art video diffusion model, while pre- serving adherence to textual prompts, as evidenced by empirical evaluations on VideoPhy, VideoPhy2, VBench, and VBench-2.0, along with a user study.
predict ground-truth optical flow, ensuring it captures
INTRODUCTION
Text-to-video diffusion models have enabled high-fidelity video synthesis across domains from tertainment to simulation. Recent systems like Wan2./ (Wan et al.) 2025), CogVideoX (Yang et al. 2025b), Humyyan Video (Lab) 2025), PyramidalFlow (Jin et al.) (025), and Open-Sora Plan (Lin et al. 2024a) leverage Diffusion Transformers (DiTs) and large-scale training to achieve impressive visual quality and scalability. Despite high visual quality, these models often generate videos with unnatural motion and physics violations, such as unsupported floating objects, implausible colli- sions, or inconsistent trajectories. These artifacts reveal a key limitation: while current models excel at generating photorealistic frames, they lack a deep understanding of motion dynamics, which is crucial for producing videos that are both visually and physically plausible.
Efforts to improve the physical plausibility of video generation generally fall into three broad cate- gories: (i) Simulation-ber based methods incorporate physics engines or differentiable simulators the generation process to model rigid-body dynamics, fluid interactions, thermodynamic effects (Lin al. 2024b Liu et al.) 2024| Xie et al. 2025) 2024, Zhang et al. 2024 Lin et al. 2025). While ef- fective, these approaches are computationally intensive, Comain--seciii and hard to scale to diverse open-world content. (ii) Non-simulation based methods aim to enhance realism without explicit simulation, often by scaling model capacity, leveraging L.LM-guided self-refinement,
introduc-
ing auxiliary objectives such as 3D point regularization and representation alignment to encourage physically coherent motion (Chen et al. 2025b; Wang al. 20255] Xue al. 2025 Zhang et al.) 20256, Hwang et al. 2025). These strategies improve appearance and sometimes temporal con- sistency, but often prioritize visual semantics over true motion dynamics. (iii) Conditioning- based
"Work was completed while an intem at Qualcomm AI Research 1 "Qullcomm AI Research initiative of Qualcomm Technologies, Inc.

---

Published as a conference paper at ICLR 2026
Text Prompt: A coin fails into a cup
Hoton-Spacilic
CogVidnok 2B
Festures
Physics violated: The coin keeps floating
Foature
Mollign
'A Cotwoik slowty the groinn"
Drikasion Model
Physics law maintained: The coin falls into the cup under gravity
todint align
Figure I: Problem (Left): Physics laws often violated in outputs of video diffusion models. Base CogVideo' model (top) cannot generate a coin falling into a cup: the coin floats in the air instead. Our MoAlign method (bottom) improves this. Proposed solution (Right): Our finetuning pipeline aligns internal representations of the diffusion model with motion-specife features extracted from VideoMAEv2.
approaches motion cues like trajectories, optical flow, pose sequences guide generation via control mechanisms (Shi et al. 2025; Geng et al. 2025 Zhang et al. 2025c). While effective for temporal coherence, they rely on these extra inputs and preprocessing inference time, making them impractical for text-only generation.
More broadly, existing methods either rely on external guidance (e.g., physies engines or inference- time controls) or influence motion only indirectly through capacity scaling appearance-centric aligmment, leaving the core issue unresolved: embedding motion understanding directly into the model's 's latent space. Recent benchmarks reveal that current video diffusion models under-encode motion dynamies in their latent space, leading to identity inconsistency, unstable trajectories, and physics violations, even when individual frames appear realistic (Huang et al. 2024 Zheng et al.) 2025 Bansal et al., 2025a). We aim to bridge this gap by aligning the diffusion model's 's latent fea- tures with representations from video encoders trained real videos, which inherently encode mo- tion observed in the physical world (Zhang et al. 20256 H Hwang et al. 2025). This, however, creates two challenges: (i) alignment may default to matching static appearance features instead of captur- ing true motion dynamics (Zhang et al. 2025b), and (ii) hard feature matching risks destabilizing pretrained representations during fine-tuning (Hwang al. 2025). As result, current alignment- based methods enhance visual fidelity but fall short in enforcing coherent motion. This raises key question: how can we design fine-tuning strategy that explicitly targets motion dynamics, without introducing extra inference -tme requirements or compromising model stability?
To tackle these challenges, we propose a motion-centric fine-tuning framework that disentangles dynamic structure from static appearance. We leverage features from pretrained video encoder, e.g. VideoMAEv2 (Wang et al., 2023). and learn projection into www-dimensiona subspace, supervised to predict optical flow, encouraging the subspace isolate motion-relevant information from entangled semantics. then align the diffusion model's ' latent features to these motion repre- sentations via soft relational alignment mechanism. In contrast to prior
setssenttttionaliinnnen approaches, e.g. REPA (Leng et al., 2025) or VidcoREPA (Zhang et al. (2025b) that relied on joint mpoeorance-motion representations,
make of motion-only feature space. And unlike Video- JAM (Chefer et al. 2025) which also used optical flows to improve motion coherence, our method does not expand the output space of the diffusion model and does not increase the cost of the infer- ence procedure. We summarize our contributions as follows:
We suggest method to learn a motion- -opion- subspace from pretrained video encoder by optimizing its projected features predict ground-true optical flow, enabling a disen- tangled motion representation.
We propose to align diffusion model features to this learned motion subspace using soft relational alignment, internalizing motion dynamics without external conditioning or sim- ulation.
We demonstrate improved temporal coherence and physical plausibility on Cog VideoX (Yang et al. 2025b), state-of-the-art diffusion model, through
2

---

Published as a conference paper at ICLR 2026
study and evaluations physics benchmarks VideoPhy (Bansal et al. 2025a). VideoPhy2 (Bansal et al., 2025bb) while maintaining high visual fidelity in VBench (Huang et al. 2024), and VBench-2.0 (Zheng et al. 2025).
2 RELATED WORKS
We group prior efforts to improve physical and temporal realism in text-to-vided generation into four areas: architectural advancements, simulation- bbased methods, conditioning-base motion control, and representation alignment. Each addresses part of the problem, yet none fully internalizes motion dynamics within the generative model as our framework does.
Text-to-video diffusson models. Early video diffusion models adapted image pipelines with frame- wise synthesis (Esser et al. 2023) Geyer et al. 2023 Karjauv et al.) 2024) and U-Nets (Ho et al., 2022 Hong et al. 2023 Blattmann al.) 2023b a Yahia et al.] 2024) but struggled with temporal
consistency and realistic motion. Transformer- based designs soon improved spatiotemporal mod-
eling via token compression and self-attention (Villegas et al. 2023 Yan et al. 2021 linear attention (Chen et al., 2025a Ghafoorian et al.) 2025). Recent systems like Cog VideoX, Wan2. PyramidalFlow, or Sora push fidelity and scale with 3D-aware representations, pyramidal flow, and spacetime patches. yet videos still show identity drift and physically implausible dynamics (Yang al. 2025b; Wan al. [025 Jin et al.) 2025/OpenAS (2024). We address this gap by internalizing motion through fine-tuning strategy that disentangles appearance from motion without external conditioning or simulation.
Simulation- based approaches. Methods that integrate physics engines or differentiable simulators capture rigid-body, fluid/elastic, or material-aware interactions (Liu et al. 2024 Zhang et al. 2024] Xie et al. 2024 Liu et al., 2025). Some combine simulation with LLM-guided reasoning or hand- crafted priors (Xue et al. 2025] Zhang et al.) 2025a), and others employ physics-guided generation (Xie al.) 2025] Montanaro et al. 2024h. While realism improves, these approaches are domain- specific, compute-heavs and hard to scale to open-world content. Our method avoids simulation and instead embeds motion understanding directly into the model.
Condition-based motion control. Another line conditions generation on motion cues such as op- tical flow, trajectories, or poses, injected via encoders/adapters (Koroglu al. 2025 Geng al.) 2025 Terauchi & Yanai (02T). Plug-and-play customization and temporal in-context fine-tuning further enhance control (Bian et al. 2025 Kim et al. 2025). These methods achieve strong coher- ence when accurate conditions exist, but require extra inputs or preprocessing at inference, limiting practicality for text-only generation. We instead internalize motion priors within the latent space.
Alignment methods match internal features of generators to pretrained encoders to improve semantics and training efficiency, but are largely spatial and image-centric (Yu et al. 2025 Leng et al. 2025). Video extensions, e.g. VideoREPA, distill spatiotemporal relations loken-levee relational matching (Zhang al.) 20256). However, hard alignment can destabilize pretrained representations and entangled features can mix appearance with motion (Zhang et al.) 20256. |Hwang al., 2025). We build on this direction with soft relational alignment specific subspace, disentangling dynamics from appearance internalize motion without sacrific- ing stability. Besides VideoREPA, closest works to ours are Track4Gen (Jeong et al.) 2025) and VideoJAM (Chefer al. 2025). which introduce motion supervision. Track4Gen operates in image-to-video setting, using optical-flows based point trajectories to enforce local correspondence at single UNet block, but it does not address global motion dynamics physical plausibility. VideoJAM jointly predicts RGB optical flow, and appearance, injecting motion via inference-time inner-guidance through learned auxiliary output. In contrast, our method neither predicts flow nor requires inference-time changes: we learn a motion-only subspace from frozen VideoMAE and align the diffusion transformer its spatio-temporal geometry, enabling motion priors without altering the generation interface.
Representation alignment.
motion-
METHOD
Our method builds upon recent advances in video diffusion modeling and representation alignment. We first review the fundamentals of video diffusion models and the REPA framework, which form
3

---

Published as a conference paper at ICLR 2026
the basis of our motion-centie fine-tuning strategy. Then, introduce proposed approach for internaizzing motion dynamics via soft relational alignment.
3.1 PRELIMINARIES
Video diffusion models. Modern text-to-video diffusion models, such as CogVideoX (Yang al. 20256), generate videos by learning to reverse a forward noising process applied to latent repre- sentations of video frames. These models operate in the latent space of a pretrained 3D Varia- tional Autoencoder (VAE), which compresses the input video both spatially and temporally. Let 3o FxxxxxxC denote a clean video with F frames. The VAE encoder maps 2 to latent representation so E = xH''X">'"'' xw''xw>''' xc", where F' F due to temporal downsampling. The forward diffusion process perturbs zo by adding Gaussian noise over I timesteps. The goal of the model is to learn a denoising function eg (2,, c.) that predicts the added noise c, conditioned on the text prompt and timestep The training objective minimizes the mean squared error Lai between the true and the predicted noise. During inference, the model samples 2r ~ .N (0, I) and iteratively denoises obtain 2p. which is then decoded by the VAE to produce the final video. Cog VideoX employs a wransfommerbased architecture (MM-DiT) as a denoiser cg. uses bidirectional spatio-temporal attention to model dependencies across frames.
Representation alignment. Diffusion Transformers (DiTs), including those used in Cog VideoX, learn internal representations during the denoising process. However, these representations often lag behind those learned by self-supervised visual encoders in terms of semantic richness and dis- criminative power. REPresentation Aligmment (REPA) addresses this gap by introducing a simple yet effective regularization that aligns the hidden states of the diffusion model with pretrained visual features (Yu et al., 2025). Originally, REPA was proposed for image models: Let x² be a clean input frame and & pretrained visual encoder (e.g., DINOv2). The encoder produces a patch-wise E(x") E NgNxD, where N is the number of patches and De the embedding dimension. During training, the image diffusion model D& processes a noisy latent input 2, along with condition c and timestep s and produces hidden states H = De(2g. 8, c). These are projected via a small trainable network Ps to match the dimensionality of Y REPA encourages alignment by maximizing the similarity between corresponding patches:
representation Y
N
CREPA (E, 6) = -E ,6,8
Σ
sim (Y;, Po (o
(I)
N
where sim(... denotes cosine similarity. This loss is added to the standard diffusion objective:
Lusal Lan + ACREPA
with À controlling the strength of alignment. Empirically, REPA improves convergence speed and generation quality, especially when applied to early transformer layers. However, in case of video models aligning each latent frame independently may lead to temporal inconsistencies, motivating extensions such as cross-frame alignment (Hwang et al. 2025).
3.2 MOTION-CENTER FINE-TUNING
Our goal is to internalize motion understanding within the diffusion model by aligning its latent features motion-specific subspace. We achieve this through two-stage fine-tuning framework: (i) learning motion-centric features from a pretrained video encoder, and (ii) aligning the diffusion model's hidden states to this motion subspace via soft relational alignment. This approach avoids reliance on external simulators or conditioning inputs, and instead distills dynamic structure directly into the generative model.
Stage 1: Learning motion-centric features. The objective of this stage is to extract features that encode motion dynamics, disentangled from static appearance and context. This is a challeng- ing task: motion is inherently relational, emerging from temporal changes across frames, whereas appearance is directly observable in individual frames. Consequently, features extracted from pre- trained video encoders often entangle motion with appearance, object identity, and scene seman- tics (Assran et al.) 2023) Wang et al. 202T Zhu et al. (020). Without explicit supervision, there is guarantee that learned representations isolate motion-specifie information.
4

---

Published as a conference paper at ICLR 2026
Learn Motion-centin
Representations
Teacher Encoderr
Motion Projector
Predictor
Stage 1:
M
VAE Enooder
MM--DIT
Black
Feature Projector
Align Diffusion Features with Motion
Block
PL
Stage 2:
MM DIT Block
cat walks slowly on the ground"
Alignment
Loss
Encoder
Teecher
Motion Projector
Ma
Figure 2: Overview of our motion-centrie fine-tuning framework. Stage / trains motion-aware teacher by extracting features from a pretrained video encoder and supervising them with ground- truth optical flow. Stage aligns the latent features of the video diffusion model (MM-DiT) to the motion-speciiic subspace via a soft relational alignment loss. This two-stage process internalizes motion understanding without requiring external conditioning simulation at inference time.
To address this, construct motion-specific subspace by supervising projection of pretrained video features to predict optical flow. Given a video clip E Fx xC we extract spatiotem- poral features S = Vxxo) E IR F" " H² xw" D. using frozen video encoder V, e.g. VideoMAEv2. These features are projected into a lower-dimensional space via learnable head Me:
M = (S) R F" xW" X Dm < D.,.
This dimensionality bottleneck is critical. By compressing the feature space, we constrain the model retain only the most salient information relevant the downstream task. Prior work has shown that such compression promotes abstraction and suppresses irrelevant appearance cues (Yang et al., 2025a 2024 Lew et al., 2025). In our case, biases the representation toward motion by limiting capacity for encoding static content.
To enforce motion specificity, we supervise M using ground-truue truth optical flow O computed between consecutive frames of Øg- A lightweight decoder F., maps M to predicted flow Ô (M), and the training objective is:
Lnow (v), w) = 0 0
Optical flow provides dense, low-level supervision that directly encodes pixel-wise motion. By forc- ing the compressed features predict flow, we constrain the subspace to encode dynamic structure rather than static semantics. This approach is supported by recent work in motion-aware video mod- cling, which demonstrates that blowd supervision improves temporal coherence and physical
plausibility (Koroglu al. 2025 Yang et al. 2024 Lew al. 2025
In summary, this stage constructs motion-specine subspace by (i) compressing figh-dimensional video features suppress appearance, and (ii) enforcing motion supervision via optical flow predic- tion. The resulting features serve as a distilled representation of dynamics, which we use as a target for aligning the diffusion model in Stage 2.
5

---

Published as a conference paper at ICLR 2026
Stage 2: Aligning diffusion features to motion. To internalize motion dynamics within the gen- erative model, we align the latent features of the video diffusion model to the motion-specifie sub- space learned in Stage 1. We adopt a soft relational alignment strategy based the Token Relation Distillation loss introduced in VideoREPA paper (Zhang et al. 2025b), which matches the pairwise similarity structure of token-level features across space and time.
Consider latent features of the diffusion model Y, E @RFxHxxx extracted from a noisy input at- We apply small projection network Pc to obtain the tensor Z E Rp " xD" the same size M, output of MMg. We denote the corresponding spatial features of f-th latent frame as Z and My, respectively, I S F". We reshape them to token matrices Z Mr, R w" xD=. The spatial similarity matrix for frame f is defined as:
(5) where Zz,. M E R denote the i-th token of 22 and M 1 i,J J.J H". For temporal similarity, we flatten all frames into sequence of F H" W tokens and compute cross-frame similarities. Let (() and M denote the i-th token in the full sequence. The temporal similarity matrices are:
5% Sepatial (f]i..) = f Z5,s);
Sepania (f) = M.
Sepanial
temporal
[. (Z)) Z())
Stomporal
[1,]] sim(M (MM))
2
aM
As in Sec. 3.1 we employ cosine similarity as the sim(.,-) function. To emphasize inter-frame dynamics, we exclude intra-frame pairs and apply temporal weighting scheme. Namely, let A.j denote the distance between frames that tokens with indices i and belong to, and define the tem- poral weight matrix W:
Wi exp -ي
if As #
0,
otherwise
where 7 is a temperature hyperparameter. The final alignment loss combines spatial and weighted temporal components:
pd
Calign (0,5)
52 Sepatial (f) Sypatial (f)], W
temporal
W
demponi
M
||
(8)
FM
where O denotes element-wise multiplication and || is the mean absolute error. This formulation extends the original Token Relation Distillation loss by introducing temporal weighting W which prioritizes temporal consistency in the local vicinity of frame. The final training objective equals
Liaal = Lain + ACalign
(9)
where À controls the strength of motion supervision. This strategy enables the diffusion model to internalize motion dynamics without requiring external conditioning compromising the stability.
4 EXPERIMENTAL SETUP
We detail the implementation of our motion-centric fine-tuning framework, including model archi- tecture, training configurations, and optimization strategies.
4.1 MODEL AND TRAINING CONFIGURATION.
We build upon VogeiooX YYagg et al. 2025b). a transformer- latent video diffusion model composed of MM-DiT blocks with joint spatio-temporal attention. CogVideoX operates in the latent space of 3D VAE compressing input videos by factor of 4 along the temporal axis.
Learning motion-centric features. For motion supervision, we use Video- MAEv2 (Wang et al. 2023) as a frozen video encoder to extract spatiotemporal features. In Stage 1, these features compressed using 3D convolutional network that reduces the channel dimension from 768 to 64 while preserving temporal structure, encouraging the retention of motion-red relevann information. The compressed features then decoded into dense optical flow using lightweight
Stage 1:

---

Published as a conference paper at ICLR 2026
Table VideoPhy2 results. We report semantic adherence (SA) and physical correctness (PC). Our model achieves highest joint score and demonstrates better ora than alternatives.
Table 2: VideoPhy results. While fine-tuning on our data
lowers SA across models, our method maintains compet- itive SA and achieves the highest PC scores across all four interaction types, demonstrating robust physical modeling through motion-centric alignment.
Method
Solid
Fluid
Fluid
Method
CogVideoX-22
27.1
64.55
22.3
SA
PC
SA
PC
SA
PC
SA
PC
CogVideoX-28 (FT)
Static haseline
15.6
91.0
15.1
CideoXXX
24.7
16.9
67.5
69.0
40.0
49.8
23.9
VideoREPA-2B (ncimpl.)
VideoREPA-2D (paper)
21.0
26.1
72.5
73.3
23.0
Cog Vod (FT)
VideoREPN- 2D (reimpl.)
23.2
22.5
29.6
31.0
62.1
66.9
34.5
39.3
54.6
58.2
45.5
52.7
41.9
46.7
17.9
34.1
MoAlign-2B (ours)
28.8
75.0
24.9
MoAlign-2H (ours)
24.7
31.7
67.3
56.4
39.4
transposed convolutional network that progressively upsamples spatial resolution in UNet-like fashion. This setup ensures that the compressed features capture dynamic structure, as they are explicitly trained to regress RAFTT-omputed ground-truth flow using L1 loss. All VideoMAEv2 weights remain frozen during this stage. We train this stage using the AdamW optimizer with a learning rate of 1 x 10-1, 3, 0.9, 32=0. and weight decay of 1 x 10-3, Training is conducted for 50,000 iterations using four NVIDIA H100 GPUs (80GB VRAM each) with batch size of 128.
Stage Aligning diffusion features to motion. In Stage 2, we align the latent features of Cog VideoX to the motion-specific subspace learned in Stage 1. We use a lightweight MLP pro- jector that maps high- ligh-dimensionll MM-DiT features (1920 channels) to a compact 64-dimensional space via 4-layer MLP with SiLU activations. The projected features temporally upsampled and spatially downsampled using a convolutional head. This transformation ensures compatibil- ity with the motion subspace dimensions while preserving relational structure. The alignment is applied to the 18th MM-DiT layer, and optimized using our soft relational alignment loss with tem- poral weighting. We set A=0.5 and 1=10.0, and train using Adam V with a learning rate of 2 x 10-6 and batch size 32. Training is conducted for 4000 iterations using four NVIDIA H100 GPUs (80GB VRAM each). We use the Adam W optimizer with a learning rate of 2 x 10 6, batch size of 32, and enable mixed precision training via Py PyTorch AMP.
Dataset. For fine-tuning our models we used 350K subset of the video dataset used by Open- Sora Plan (Lin et al.] (2024a), along with set of 16K synthetic video samples generated by the Wan2.1 14B model, with prompts sourced from the same set as for the Open-Sora Plan dataset.
RESULTS
5.1 COMPARISON WITH BASELINES
We evaluate method across three complementary axes of video generation: (i) physical plausi- bility, using the VideoPhy (Bansal et al. 2025a) and VideoPhy2 (Bansal et al. (025b benchmarks: (ii) general generation quality, using VBench (Huang al. (024) and VBench-2.0 Zheng et al.) 2025): and (iii) perceptual realism, via a blind user study. Each evaluation targets distinct aspect of generative fidelity, from adherence to physical laws semantic aligmment and human preference.
VideoPhy2. This recent benchmark evaluates physical plausibility while focusing on action- centric scenarios involving -ojant interactions. Videos are generated from 591 extended prompts, and scored using the VkdeoPhy2-AutoEl model. This model predicts two metrics on 5-point scale: Semantic Adherence (SA) which measures how well the video matches the prompt and Physical Commonsense (PC) which assesses whether the motion and interactions are physically plausible. The primary metric is the Joint score, defined as the fraction of vidcos rated > 4 on both SA and PC dimensions. Tab. [ [highlights the importance of holistic evaluation: a degenerate static baseline, which simply repeats the first frame, achieves deceptively high PC score by avoiding motion violations, but fails on SA, resulting in a low Joint score.
We compare our MoAlign method against the base model CogVideo and recent VidcoREPA approach that aimed to improve physical plausibility. To decouple the effect of training data and different alignment methods, also finetuned the base model the same dataset and with the

---

Published as a conference paper at ICLR 2026
Table 3: General video quality, All methods maintain the original technical quality, as indicated by VBench. On VBench-2.0 MoAlign demonstrates improvement in Total score, mainly driven by improved instance preservation, dynamic spatial relationship and human anatomy.
Model
ViBecch
VBench-2.0
Total
Quality
Semantic
Total
Creativity
Commonsense
Controliability
Iuman Fidelity
Physics
Cog VideoX
81.6
52.8
60.2
81.1
Cog VidenX-28 (FT)
80.3
81.1
77.1
54.7
58.7
60.8
25.6
83.4
44.9
VideoREPA-20
80.5
81.3
27.2
55.0
$6.9
61.4
25.9
85.4
45.1
MoAlign-2B (ours)
82.0
52.8
68.5
25.7
86.7
same training budget, this checkpoint referred to as FT. As shown in Tab. T our training data has marginal effect on Joint score.
Compared to the base model, MoAlign-2B achieves improvement both for individual dimensions and Joint score. Since Since/Zhang al. (2025b) have not reported the Joint score for their VideoREPA method, we reimplemented it and evaluated independently in the same setup. While VidcoREPA-2B improves PC, it suffers from noticeable drop SA, resulting in a lower gain in Joint score than our method. This suggests that alignment to entangled features may improve physical realism at the cost of prompt fidelity. In contrast, our method aligns to disentangled motion features, improving both motion realism and semantic alignment. This indicates that internalizing dynamic structure via motion-specific supervision leads to more coherent and faithful video generation. We have further investigated the effectiveness of MoAlign on Wan2.1 (1.3B); the results are available on Appendix A.5
VideoPhy. The second benchmark focuses material-centric interactions across three categories: solid-solid, solid-fluid, and fluid-fluid. Videos are generated from 343 prompts. Scoring is per- formed using the VideoConPhysics auto-rater, which evaluates SA and PC dimensions. In contrast with VideoPhy2, extended prompts were not standardized in this benchmark. Therefore we opted for sampling videos from all models with the short prompts provided by [Bansal et al. (2025a).
As shown in Tab. 2 we observe a consistent trend across all method: finetuning on our data tends to reduce SA scores compared the base model (note that FT model without any alignment performs the worst). We attribute this to the shortage of relevant examples in the dataset. At the same time, PC reflects the plausibility of generated physics irrespective of the fact if it follows the textual prompt. Notably, our method most effectively mitigates the drop in SA among all finetuned variants while achieving the highest PC scores across all interaction types. This proves that the proposed MoAlign training strategy overcomes the limitations of training data better than other considered methods.
VBecc and VBench-2.0. To ensure that improvements in physical plausibility do not come at the cost of overall video quality, we evaluate all methods with two other commonly used toolkits. VBench focuses more on perceptual characteristics such as aesthetics, temporal smoothness, object- scene consistency, etc., while its second version targets intrinsic faithfulness of generated videos.
We report the aggregated metrics for both benchmarks in Tab. 3 Please refer to the Supplementary for more fine-grained results. First of all, note that all methods keep the VBench Total score approximately constant which suggests that none of them worsens the technical quality genera- tions. For VBenchh-2.0, VideoREPA is on par with the base model in terms of Total score, while our method brings noticeable improvement. This gain is achieved mainly by means of Commonsense and Human Fidelity metries which cover, among others, such dimensions as instance preservation, dynamic spatial relationship. and human anatomy highly important aspects for physical plausibil- ity. The significant drop of Physics score for all funetuned models has the same explanation in VideoPhy case: our training data lacks samples related to thermotics and materials which are pivotal for this category.
User study. To complement automated metrics, we conduct a blind user study to assess temporal coherence and physical plausibility from a human perspective. We compare three models: the base CogVideoX-2D VideoREPA-2B. and our MoAlign-2B. For each model, generate 50 vidcos ing extended prompts sampled from mix of VBench-22 2.0 and VideoPhy2, and collect 672 pairwise preferences in side-by-side comparisons.
8

---

Published as a conference paper at ICLR 2026
Table 4: Ablation study. Both proposed components are important for the performance of our method, as measured by VideoPhy2.
Table 5: Layer for alignment. Impact of the feature alignment layer on model quality based on VidePhy2 scores.
Method
SA
PC
Joint
Layer
PC
REPA
25.7
22.3
27.3
71.9
23.2
Cog VideoX (FT)
26.4
73.1
22.8
12
27.6
71.7
22.9
VideoREPA
26.1
73.3
23.0
28.2
73.9
24.2
MoAlign who motion features
27.8
73,8
23.5
18
28.8
75.0
24.9
MoAlign wlo soft-TRD loss
28.2
74.4
24.1
27.1
73.2
23.8
MoAlign (ours)
28.8
75.0
24,9
27.6
72.8
23.4
Table 6: User study. Our results are preferred over those from the baselines.
Comparison
MoAlign
Baseline
vs. CogVideoX-2B
68 % 78 %
32%
vs. VidcoREPA-2B
22%
As shown in Tab. our method is preferred significantly more often in both comparisons, indicating that motion-centric alignment not only improves physical plausibility and automated scores, but also enhances perceived realism and prompt fidelity. These results confirm that our framework leads to more coherent and visually compelling video generation, as validated by human judgment.
We provide examples of videos used for the user study in the Supplementary.
5.2 ABLATION STUDY
We also conducted experiments to isolate the contributions of our method. First, we considered modification of MoAlign that employs untouched feature extractor from VideoMAEv2 for super- vision. Second, we try vanilla TRD loss without weighting based on distance between frames, i.e. + 00 in Eq. 7 Note that these two ablations done simultancously result in the VideoREPA method. Additionally, we report the performance of classical REPA alignment, which pushes ternal diffusion features towards those from VideoMAEv2 instead of distilling their self-relational structure. Following Zhang et al. (0055b), we use VideoPhy2 for the evaluation of ablated mod- els. Results are reported in Tab. 4 As shown, both proposed components are important for the performance, and both of them individually improve the VideoREPA baseline. Notably, REPA loss provides the worst results and cannot surpass simple finetuning of the diffusion model data.
Prior works noted the sensitivity of alignment results the selection of the layer from denoising network 6g for extraction of latent features (Leng et al. 2025) Zhang et al. 20256). Therefore, we also examined several layers and chose the 18-th transformer block of Cog VideoX for our MoAlign method, see Tab. 5 This choice is consistent with REPA-style approaches, which typically align at single mid-level layer rather than across the entire network. Empirically, we observe that aligning at layer 18 yields the best VideoPhy2 performance, while both earlier and deeper layers lead to lower physical commonsense scores. We also found that distributing the loss across multiple layers slightly degrades results, suggesting that overly broad regularization disrupts the denoising trajectory. These findings indicate that motion-relate relational structure is naturally concentrated around mid-depth blocks, motivating our focus single alignment layer. More details provided in Appendix A.4
We also conducted additional analysis on the Stage-I training dynamics, including convergence behavior and training cost. The full details of this study are provided in Appendix A.3
5.3 QUALITATIVE RESULTS
We show a qualitative comparison in Figure 3 Our MoAlign accurately depicts both the source and target glasses throughout the video, with clear and coherent pouring motion. In contrast, CogVideoX fails to represent the scene structure, omitting key elements like the second glass and the liquid flow. VideoREPA partially captures the target glass but lacks temporal consistency in the pouring dynamics. This example highlights our model's ability to preserve scene semantics and physical realism. Additional qualitative comparisons are provided in appendix A.

---

exciting direction Published as a conference paper at ICLR 2026
Cog VideoX
VideoREPA
MoAlign
Prompt: Glycerin from clear glass is poured into glass of water
Figure 3: Qualitative results. MoAlign shows both glasses and coherent pouring motion, while baselines miss key elements or exhibit inconsistent fluid behavior.
LIMITATIONS
While MoALign consistently improves motion plausibility and physics-centric behavior, it inherits two important limitations. First, its performance is bounded by the coverage available training data: noted in Tab. 3 the VBench-2.00 Physies category includes domains such thermotics, material deformation, liquids, and granular media, none of which appears in the datasets used for our Stage-1 motion-teacher training, or Stage-2 diffusion finetuning. Second, MoAlign provides implicit physics through motion statistics learned from real videos, but does not explicitly model forces, material properties, long-horizon causal reasoning, and may therefore struggle with sce- narios requiring reasoning beyond motion alone, limitation also discussed in prior works like VideoJAM (Chefer al., 2025). While addressing these limitations primarily requires video data containing such phenomena, incorporating them would only require retraining the lightweight Stage- module followed by standard Stage-2 alignment, without re-training VideoMAF or Cog VideoX from scratch. We view extending MoAlign toward richer physics domains for future work.
CONCLUSION AND FUTURE WORK
In this work we presented a method for improving temporal coherence and physical plausibility in pretrained video diffusion models. Our pipeline called MoAlign is based the trainable alignment of internal diffusion features to motion-specifie representations extracted from original videos. We demonstrated that such representations result in better quality than general features extracted from a pretrained self-supervised video encoder. Also, we showed that such alignment works better if long-range temporal dependencies. We evaluated
prioritizes local vicinity of each frame approach with four recent and commonly used benchmarks, as well as user preference study.
In our experiments found that several quality metrics were hurt for all finetuning- methods that we tried, We attribute that to the limitations of our training dataset, and suggest that MoAlign may further benefit from better data curation. Nevertheless, our method demonstrated stronger resilience to shortcomings of the collected dataset than the baselines.
As downside of our method, we noted that sometimes improves the physical commonsense the expense of reduced motion in the generated videos. We consider this limitation as viable direction for future work.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15619-15629, 2023.
Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, and Aditya Grover. Videophy: Evaluating physical com- monsense for video generation. In The Thirleenth International Conference Learning Repre- sentations, 2025a. URL Dectps::/cpperevii/ net htte://oommmmdmd00000
Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, and Kai-Wei Chang. Videophy-2: A challenging action-centric physical commonsense evaluation in video generation.
arXiv preprint Stti::250330680000 2025b. 3
Yuxuan Bian, Ailing Zeng, Xuan Ju, Xian Liu, Zhaoyang Zhang, Wei Liu, and Qiang Xu. Motion- craft: Crafting whole-body motion with plug-and aangg--ndplay multimodal controls. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 1880-1888, 2025.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominík Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rom- bach. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint ar wtXXv:2331.151222 2023a. URL
abs/ 2311.1121 15127
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Scung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2052222633222 22575, 2023b.
Hila Chefer, Uriel Singer. Amit Zohar, Yuval Kirstain, Adam Polyak, Yaniv Taigman, Lior Wolf, and Shelly Sheynin. VideoJAM: Joint appearance-motion representations for enhanced motion generation in video models. In Forry-second International Conference on Machine Learning, 2025.
10 19
Junsong Chen, Yuyang Zhao, Jincheng Yu, Ruihang Chu, Junyu Chen, Shuai Yang. Xianbang Wang, Yicheng Pan, Daquan Zhou, Huan Ling, al. Sana-video: Efficient video generation with block linear diffusion transformer. arXiv preprint trXiv:2509.24693 2025a. 3
Yunuo Chen, Junli Cao, Anil Kag, Vidit Goel, Sergei Korolev, Chenfanfu Jiang, Sergey Tulyakov, and Jian Ren. Towards physical understanding in video generation: A 3d point regularization approach. arXiv preprini (tlt:25502.03639, 2025b.
Patrick Esser, Johnathan Chiu, Parmida Atighehchian. Jonathan Granskog, and Anastasis Germani- dis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 7346-7356, 2023. 3
Daniel Geng, Charles Herrmann, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevara, Yusuf Aytar, Michael Rubinstein, Chen Sun, et al. Motion prompting: Control- ling video generation with motion trajectories. In Proceedings of the Computer Vision and Pattern
Recognition Conference, pp. 1-12, 2025. 2
Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307. 10373, 2023. 3
Mohsen Ghafoorian, Denis Korzhenkov, and Amirhossein Habibian. Attention surgery: An efficient recipe to linearize your video diffusion transformer. arXiv preprint arXiv:2509.24899 2025. 3
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi. and David J Fleet. Video diffusion models. Advances in neural information processing systems, 35:8633- 8646, 2022. 3
11

---

Published as a conference paper at ICLR 2026
Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pre- training for text-to-video generation via transformers. In The Eleventh International Confer- ence Learning Representations, 2023. URL http::/ hwpperreiiwwnne//eeee fww.:/mmmeeooouuiieec /ttppas////ppeeeeeeeee....ttet
rB6TpjAuSRR
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si. Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vhench: Comprehensive henchmark suite for video generative models. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21807-21818, 2024 7
Sungwon Hwang, Hyojin Jang, Kinam Kim, Minho Park, and Jaegul Choo. Cross-frame represen- tation alignment for fine-tuning video diffusion models. arXiv preprint arXiv:2506.09229, 2025.
4
Hyeonho Jeong, Chun-Hao Huang, Jong Chul Ye, Niloy Mitra, and Duygu Ceylan. Track4gen: Teaching video diffusion models track points improves video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7276-7287, 2025.
Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang. Nan Zhuang, Quzhe Huang, Yang Song, Yadong MU, and Zhouchen Lin. Pyramidal flow matching for efficient video gener- ative modeling. In The Thirteenth International Conference on Learning Representations, 2025. URL
net Forummddd
Adil Karjauv, Noor Fathima, Ioannis Lelekas, Fatih Porikli, Amir Ghodrati, and Amirhossein Habib- ian. Movie: Mobile diffusion for video editing, arXiv preprint arXiv:2412.06578. 2024. 3
Kinam Kim, Junha Hyung, and Jaegul Choo. Temporal in-context fine-tuning for versatile control of video diffusion models. arXiv preprint arXiv:2506. 200009,06.00999, 2025. 3
Mathis Koroglu, Hugo Caselles- Dupré, Guillaume Jeanneret, and Matthieu Cord. Onlyflow: Optical flow based motion conditioning for video diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 6226-6236, 2025.
3 5
Tencent AI Lab. Hunyuanvideo: A systematic framework for large video generation model, arXiv preprint ar arXiv:2412.03600 2025. https://:///ax http:////axxxoccc/a/////////ss// org. 2412.2240 03603 [
Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking for end-to-end tuning with latent diffusion transformers. In ICCV, 2025.
9
Jaihyun Lew, Jooyoung Choi, Chachun Shin, Dahuin Jung, and Sungroh Yoon. Disentangled motion modeling for video frame interpolation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 4607-4615, 2025.
Bin Lin, Yunyang Ge, Xinhua Cheng, et al. Open-sora plan: Open-source large video generation model. arXiv preprint a:211:2412.00111, 2024a. URL ftttps://arxdi..org/ab////// 00131
Jaajing Lin, Zhenzhong Wang, Yongjie Hou, Yuzhou Tang, and Min Jiang. Phy124: Fast physics- driven 4d content generation from a single image. arXiv preprint arXiv:2409.07179, 2024b. [
Jajjnng Lin, Zhenzhong Wang, Shu Jiang, Yongjie Hou, and Min Jiang. Phys4dgen: A physics- driven framework for controllable and efficient 4d content generation from a single image. arXiv e-prints, pp. arXiv-2411, 2024c. [
Wang Lin, Liyu Jia, Wentao Hu, Kaihang Pan, Zhongqi Yue, Wei Zhao, Jingyuan Chen, Fei Wu, and Hanwang Zhang. Reasoning physical video generation with diffusion timestep tokens via reinforcement learning. arXiv preprint arXiv:2504. 15932, 2025.
Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang. Physgen: Rigid-body physics- grounded image-to-vides generation. In European Conference on Computer Vision, pp. 360-378. Springer, 2024. 3
12

---

Published as a conference paper at ICLR 2026
Zhuoman Liu, Weicai Ye, Yan Luximon, Pengfei Wan, and Di Zhang. Unleashing the potential of multi-modal foundation models and video diffusion for 4d dynamic physical scene simulation. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), 2025.
Antonio Montanaro, Luca Savant Aira, Emanuele Aiello, Diego Valsesia, and Enrico Magli. Mo- tioncraft: Physics-based zero-shot video generation. Advances in Neural Information Processing Systems, 37:123155-12311 2024. 3
OpenAI. Video generation models as world simulators, 2024.
Qingyu Shi, Jianzong Wu, Jinbin Bai, Jiangning Zhang, Lu Qi. Yunhai Tong, and Xiangtai Li. De- couple and track: Benchmarking and improving video diffusion transformers for motion transfer. In ICCV, 2025. 2
Kento Terauchi and Keiji Yanai. Pose sequence generation with a gen and an initial pose generator. In Asian Conference Pultern Recognition, pp. 417-430. Springer, 2021.
Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Vari- able length video generation from open domain textual descriptions. In International Confer- ence on Learning Representations, 2023. URL battps://openzeeeee
net forum?idk
VCEXS39nOF
Team Wan et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 2025.
Jing Wang, Ao Ma, Ke Cao, Jun Zheng, Zhanjie Zhang, Jiasong Feng, Shanyuan Liu, Yuhang Ma, Bo Cheng, Dawei Leng, et al. Wisa: World simulator assistant for physics-aware text-to-video generation. arXiv preprint NXiv:2503.08153 2025. T
Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yinan He. Yi Wang, Yali Wang, and Yu Qiao. Videomae v2: Scaling video masked autoencoders with dual masking. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14549-14560, 2023.
Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, and Hanwang Zhang. Self-supervised learning disentangled group representation as feature. Advances in Neural Information Processing Systems, 34:18448 14:18225-18240, 2021. 4
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Physgaussian: Physics-integrated 3d gaussians for generative dynamics. In Proceedings of the JEEE/VVF Conference on Computer Vision and Pattern Recognition, pp. 4389- 2024. [
Tianyi Xie, Yiwei Zhao, Ying Jiang, and Chenfanfu Jiang. Physanimator: Physics-guided generative cartoon animation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10793-10804, 2025.
Qiyao Xue, Xiangyu Yin, Boyuan Yang, and Wei Gao. Phyt2v: LIm-guided iterative self-refinement for -rrssics-groudeed rext-to-vided generation. In Proceedings of Computer Vision and Pattern Recognition Conference, pp. 11822-18836, 2025.
Haitam Ben Yahia, Denis Korzhenkov, Ioannis Lelekas, Amir Ghodrati, and Amirhossein Habibian. Mobile video diffusion. arXiv preprint arXiv:2412.07583, 2024. 3
Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.10157 2021.
3
Haosen Yang, Deng Huang, Bin Wen, Jiannan Wu, Hongxun Yao, Yi Yijin,, Xiatian Zhu, and Zehuan Yuan. Motionmae: Self-supervised video representation learning with motion-aware masked autocncoders. In BMVC, 2024. 5
Jiange Yang, Yansong Shi, Haoyi Zhu, Mingyu Liu, Kaijing Ma, Yating Wang, Gangshan Wu, Tong He, and Limin Wang. Como: Learning continuous latent motion from internet videos for scalable robot learning, arXiv preprint arXiv:2505.1 17006, 2025a. 5
13

---

Published as a conference paper at ICLR 2026
Zhuoyi Yang. Jiayan Teng, Wendi Zheng. Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Yuxuan. Zhang, Weihan Wang. Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-vides diffusion models with an expert transformer. In The Thirteenth International Conference on Learning Representations, 2025b. URL
net. wtt.:eteet/oommmm2oo.i./.
123
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong. Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is casier than you think. In The Thirteenth International Conference on Learning Representations, 2025.
URL https: /ttp:///opeeeeeeeeeee net/ tenee/ffrree forum? JDDJSZGGGYV1
Lingyum Zhang, Yu Xie, Yanwei Fu, and Ping Chen. Concept replacer: Replacing sensitive concepts in diffusion models via precision localization. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 8172-8181 2025a. S
Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Feng. Changxi Zheng, Noah Snavely, Jiajun Wu, and William T Freeman. Physdreamer: Physics-basee basee interaction with 3d objects via video generation. European Conference on Computer Vision, pp. 388-406 Springer, 2024. I
Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang, Fanqing Meng, Xiangpeng Wan, Junchi Yan, and Yu Cheng. Videorepa: Learning physics for video generation through relational alignment with foundation models. In NeurlPS, 2025b.
2
S 8
Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffusion transformer for video generation. In Pro- ceedings of the Computer Vision and Pattern Recognition Conference, pp. 2063-2073, 2025c.
Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, et al. Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness. arXiv preprint arXiv:2503.21755, 2025. 2
Yizhe Zhu, Martin Renqiang Min, Asim Kadav, and Hans Peter Graf. S3vae: Self-supervised sequential vae for representation disentanglement and data generation. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5538-6547, 2020.
14

---

Published as a conference paper at ICLR 2026
Cog VideoX
VideoREPA
MoAlign
Prompt: A man is jumping
Cog VideoX
VidcoREPA
MoAlign
Prompt: A person is eating pudding with spoon
CogVideoX
VideoREPA
Moalien
Prompt: person is squeezing ketchup onto a plate
methods for three prompts. In the first video, method (MoAlign) preserves realistic human motion without deformation. In the second, it captures accurate hand-mouth interaction while baselines fail represent the subject. In the third, it models physically plausible ketchup flow, unlike the erratic behavior seen in baselines. See supplementary videos for full comparisons.
Figure A.1: Qualitative comparison
A APPENDIX
The Appendix consists of the following sections: Qualitative results (Sec. A.I), VBench and VBench-2.0 full results (Sec. A.2)). Stage 1 training details (Sec. A.3). and Stage training details (Sec. A.4. Sec. A.7 provides details about usage of LLMSs in this project.
15

---

Published as a conference paper at ICLR 2026
A.1 QUALITATIVE RESULTS
We present qualitative comparisons in Figure A.1 across three scenarios involving human motion, object manipulation, and fluid dynamics. In each case, our method demonstrates superior temporal coherence and physical plausibility compared to existing baselines.
In the first video, depicting a man jumping, our method produces smooth and anatomically consis- tent motion. The subject maintains realistic posture and limb articulation throughout the sequence. In contrast, both CogVideok and VideoREPA exhibit noticeable distortions, including unnatural body twisting and implausible joint movements. In the second video, where a person is eating pud- ding with a spoon, our model accurately captures the interaction: the spoon visibly scoops pudding, and the subject's mouth moves in coordination. The baselines fail to preserve this interaction the person is either missing or the spoon disappears mid-sequence. breaking temporal and seman- tic consistency. In the third video, showing man squeezing ketchup onto a plate, our method correctly models the accumulation of ketchup over time. The quantity on the plate increases as ex- pected. Conversely, the baselines display erratic behavior: ketchup appears, disappears, and even flows back into the bottle, violating intuitive physical dynamics.
To facilitate further comparison, we include eight video files in the supplementary material. Each file contains grid of six videos arranged in two rows and three columns. Each row corresponds to a different prompt, and each column shows the output from one of the three methods: Cog VideoX, VideoREPA, and our MoAlign. This layout allows viewers easily compare the outputs across methods for the same prompt and observe differences in motion consistency, semantic fidelity, and physical realism.
A.2 VBNNCH AND VBENCH-2.0 RESULTS
We present fine-grained results on the henchmarks in Tabs. A. A.2
A.4 and A.5
Notable decrease in dynamic degree and side-effect. We observe that by applying MoAlign, dy- namic degree drops most notably among all the other VBench metrics. The Dynamic Degree metric in VBench primarily measures the magnitude of pixel-space motion (e.g., optical-flow amplitude) rather than the physical plausibility of that motion. Upon closer inspection, we find that the higher dynamic degree scores achieved by the baseline CogVideoX largely stem from exaggerated or un- stable motions - such as abrupt limb jumps, temporal jitter, or transient body parts which inflate flow magnitude without corresponding to realistic dynamics.
On the other hand, MoAlign tends reduce these unstable high-amplitude artifacts, leading to smoother and more physically grounded trajectories, which as expected lowers amplitude-based motion metrics. Importantly, note that this does not indicate a degenerate "Iow-motion" solution: physics-centric metrics such as VideoPhy and VideoPhy2 improve (e.g., +3.1 Joint), and the Joint metric is explicitly designed to penalize low- mowmmotion outputs. As shown in the Table [ the Static Baseline, containing no motion, achieves very high PC score but suffers dramatic collapse in Joint, whereas MoAlign improves Joint over both the base model and the static case. This hints that MoAlign preserves meaningful motion while reducing mostly the nonphysical components. Qualitatively, the model continues to produce clear global motion (e.g., in dancing/running prompts), but with more realistic velocities and fewer nonphysical transitions.
A.3 STAGE TRAINING DETAILS
To learn a motion-centric representation disentangled from static appearance, we have a two-stage network to predict ground-truth optical flow from features extracted by frozen VideoMAEv2 en- coder. The encoder outputs token-level features of dimension 768, which are passed through a 3D convolutional compressor that reduces them to 644-dimensional subspace. This compression network consists of two convolutional layers: the first is a temporal convolution with kernel size (3, 1, and padding (1, (1,0,00 0). which captures motion patterns across frames and maps the input to 256 channels. The second is pointwise (1 x x 1) convolution that compresses the channel di- mension to 64. Both layers are followed by SiLU activations. The input this network is of shape [B. 768, 24, H, W], and the output is [B, 64, 24, H, W), representing our learned motion subspace optimized to retain dynamic structure while suppressing static semantics.
16

---

Published as a conference paper at ICLR 2026
0.34
0.32
0.3
0.28
0.26
0.24
Flow Loss
0.22
0.2
0.18
0.16
0.14
0.12
0.1
-5k
5k 10k
15k 20k 25k 30k 35k 40k 45k 50k 55k
Training Iteration
Figure A.2: Stage-1 I motion projector head training convergence. The training converges at ~30k iterations. The full 50k training takes ~22 hours.
The compressed features are then fed into flow prediction network designed to regress dense optical flow maps. This network begins with 3D convolution that refines the temporal resolution from 24 to 23 using a kernel of (2,3, 3, 3) and padding (0, 1, 1). followed by a ReLU activation. It then applies two stages of spatial upsampling using transposed convolutions: the first upsamples 2x with kernel of (1, 4, 4) and stride (1, 2, 2), mapping to 32 channels; the second upsamples again by 2x 16 channels. Each transposed convolution is followed by ReLU. A final 3D convolution with kernel size 3 and padding produces the flow vectors with output channels. The resulting tensor is interpolated to a fixed shape [B, 2, 23, 128, 192] using trilinear interpolation, and then permuted to [B, 23, 2, 128, 192] for compatibility with the ground-truth flow format.
The model is trained using LI loss against RAFT-computed ground-truth flow, with all Video- MAEv2 weights kept frozen. We use the AdamW optimizer with a learning rate of x 10-4 31 0.9, B2 0.95, and weight decay of 10-3, Training conducted for 50,000 iterations on four NVIDIA H100 GPUs (80GB VRAM each), with batch size of 128 and mixed precision enabled via PyTorcl AMP. Input videos are resized 160 240 and truncated to 49 frames, with center cropping applied to ensure consistent input dimensions. Validation is performed every 1000 steps using a held-out set.
Stage-1 training saturation. Figure A.2 shows the training curve for learning the motion subspace from VideoMAE representations. We observe that the training saturates around 30k iterations. The full training of 50k iterations for the first stage takes ~22 hours. Note that the learned motion subspace projector is independent of the stage-2 training. the intrinsics of the diffusion model and its fine-tuning: thus the same motion sub-space projector once trained can be reused for various diffusion representation alignment episodes.
Table A. VBench results (part 1/2): consistency, motion, object-level metrics.
Method
Subject Consistency
Background Consistency
Temporal Flickering
Motion Smoothness
Dynamic Degree
Aesthetic Quality
Imaging Quality
Objoct Class
Multiple Ohjects
Human Action
CogVideoX-22 2B
92.9
94.7
97.1
97.6
70.3
62.9
63.2
86.8
66.8
97.2
Videid (PT)
VideoRIPAA
95.7
96.1
98.9
98.2
44.4
63.2
61.1
88.2
71.1
96.2
MoAlign (ours)
95.8
96.4
99.0
98.4
42.2
64.5
64.5
89.6
75.2
98.4
Table A.2: VBench results (part 2/2): appearance, style, and overall scores.
Spatial Relationship
Appearance Style
Temporal Style
Overall Consistency
Quality Score
Semantic Score
Total
Method
Color
Scene
Score
Cog VideoX-28 2B
78.6
71.8
50.8
24.5
24.4
26.7
81.6
76.6
80.6
Cog VidenX (FT)
84.4
69.6
52.4
24.4
24.4
26.3
81.1
77.1
80.3
VideoREPA
83.8
50.2
24.6
24.4
26.4
81.3
77.2
80.5
MoAlign (ours)
80.4
75.4
49.9
24.2
24.3
26.4
82,0
78.2
81.3
17

---

Published as a conference paper at ICLR 2026
Table A.3: VBench-2.0 results (part 1/3).
Method
Human Identity
Dynamic Spatial Relationship
Complex Landscape
Instance Preservation
Multi- View Consistency
Human Clothes
Dynamic Attribute
Complex Plot
CogVidcoX-28
75.1
19.8
14.0
84.8
20.3
85.6
23.8
8.1
Cug VideoX (FT)
79.7
18.8
91.8
8.7
88.1
17.6
9.2
VidcoREPA
81,9
22.2
15.1
92.4
6.3
90.4
16.5
8.5
MoAlign
80.6
26.1
16.2
95.3
10.0
94.2
16.5
8.6
Table A.4: VBench-2.0 results (part 2/3).
Method
Mechanics
Human Anatomy
Composition
Human Interaction
Motion Rationality
Material
Diversity
Understandngg
Motion Order
CogVideoX-28
64.2
82.6
$5.6
35.6
68.1
10.1
VogVidooX (FT)
$7.5
82.3
54.9
64.7
29.9
58.5
62.6
9.9
VideoREPA
55.6
84.0
55.6
30.5
61.5
58.2
10.6
MoAlign
57.9
85.3
$3.3
64.7
35.6
64.6
52.3
8.9
Table A.5: VBench-2.0 results (part 3/3).
Method
Camera Motion
Thermotics
Creativity Score
Commonsensee
Controllability Score
Human Fidelity Score
Physies
Total
Score
Coy VideoX 2B
49.7
63.0
52.8
60.2
26.6
81.1
53,9
$4.9
CogVideoX (FT)
45.7
$4.7
58.7
60.8
25.6
83.4
44.9
54.7
VidcoREPA
42.3
57.1
56.9
61.4
25.9
85.4
55.0
MoAlign
38.9
62.8
52.8
65.5
25.7
86.7
48.8
55.9
A.4 STAGE TRAINING DETAILS
In the second stage, align the latent features of the Cog VideoX diffusion model to the motion- centric subspace learned Stage 1. This is achieved by introducing a lightweight projection network that maps internal representations from Cog VideeX to the same 64-dimensional space used for mo- tion supervision. Specifically, we extract hidden states from the 18th MM-DiT block of Cog VideoX and pass them through a temporal- spatial projection head. This head first applies temporal con- volution stack consisting of (3, 1, 1) 3D convolution followed by SiLU activation, and then a pointwise (1 x 1) convolution with another SiLU activation. These layers reduce the input chan- nel dimension from 1920 to 256 and then to 64, preserving temporal structure while compressing appearance information.
After temporal processing, the features are interpolated along the time axis by factor of 2 using trilinear interpolation. The resulting tensor is then passed through spatial downsampling layer to match with the spatial dimension of the videomae compressed features. This layer is single 3 x 3 convolution with stride 3. This operation reduces the spatial resolution by approximately 3x, yielding final output of shape that is compatible with the motion subspace.
During training, we freeze the VideoMAEv2 encoder and the motion compressor from Stage 1, and optimize only the VideoX transformer and the projection head. The training objective com- bines the standard denoising loss from the diffusion model with a soft relational alignment loss that matches the pairwise similarity structure of the projected features space. We use a temporal weighting scheme with - = 10 to emphasize long-range inter-frame consistency, The alignment loss is weighted by a factor of 0.5 and added to the diffusion loss. Train- ing is performed using AdamW with a learning rate of 2 x 10-6 batch size 32, and mixed precision enabled. The model is trained for 4000 steps on four NVIDIA H100 GPUs (80GB VRAM each), with validation conducted every 500 steps using a held-out set of prompts.
those from the motion sub-
Stage-2 alignment depth. As noted in Section 5.2 the choice which Cog VideoX block to align is a key factor for effective motion transfer. Prior alignment-based works (e.g., REPA, VideoREPA) similarly report that applying the alignment loss at a single, well-chosen mid-level layer yields the best performance, whereas spreading supervision across multiple blocks can overly constrain the denoising trajectory. In our case, we evaluated several transformer blocks of Cog VideoX and found that aligning at the th transformer block provides the strongest improvement in VideoPhy2 scores (see Tab. | in the main paper). We attribute the better performance of layers from the middle of the network to the internal dynamic of the denoising transformer. Namely, earlier layers primarily
18

---

Published as a conference paper at ICLR 2026
encode low-level appearance features, while later blocks mainly refine high-frequency details and translate the representation to the output domain (since Cog VideoX not zg pe-parametrized) While finding the theoretically justified algorithm for layer selection is appealing research direction, in this work we opted for empirical comparison several inner blocks.
A.5 EXPERIMENTS ON WAN2.1 1.3B
To evaluate the effectiveness of our motion representation alignment method, we applied similar alignment strategy to fine-tune 1 (1.3B), widely used state-of-the-art efficient video diffu- sion model. Table A.6 presents the results. As shown, incorporating MoAlign fine-tuning improves all relevant metries (SA, PC, and Joint score), though the gains are more moderate compared to those observed with Cog VideoX.
Table A.6: Quantitative assessment of effectiveness of incorporating MoAlign on Wan2.1 (1.3B), as measured on VideoPhy2.
Method
SA
PC
Joint
Wan-1.3B
28.7
64.1
23.7
Wan-1.3B + MoALign
29.8
64.6
25.1
A.6 COMPARISON WITH VIDEOJAM
To compare our MoAlign method with other approaches, we attempted train the recently proposed VideoJAM Chefer et al., 2025 modification of the video model. The training setup, base diffusion model, and dataset were identical to those used for Stage of MoAlign (see Sec. 4.1). our experiments, found that under this training budget, VideoJAM remained far from convergence and produced unsatisfactory video outputs.
VideoJAM extends the original video generation model by simultaneously predicting the RGB vi- sualization of the video's optical flow. This design introduces capabilities that were absent in the base model. The dynamics of training losses support this claim: the start of training, the 'flow' component of the diffusion MSE loss is very high but gradually decreases, while the 'video' compo- nent initially low reflecting the model's prior video deder-generttion pretraining then rises, peaks, and slowly declines. This indicates that during the early phase of VideoJAM training, the video and flow objectives conflict, and model requires substantial adaptation align with the new joint objective. However, this adaptation is resource-intensive.
In the original work by Chefer al. (2025). VideoJAM was trained on million video clips (50K iterations on 32 GPUs) at spatial resolution of 256 256. This far exceeds the requirements for MoAlign's Stage 2, which uses only 4K steps on 4 GPUs. We consider these findings strong evidence that our method offers better training efficiency for practitioners.
A.7 USE OF LARGE LANGUAGE MODELS
We used Microsoft Copilot (a large language model) to aid in polishing the writing of this sub- mission. The model was employed solely for improving clarity and readability; all ideas, technical content, and conclusions are our own.
19


Reviews:
[{"review_id": "FGGGsk7AM4", "reviewer": "Reviewer_DP4f", "summary": "This paper proposes MoAlign, a motion-centric representation alignment framework for video diffusion models. The method learns a motion-specific subspace from a pretrained video encoder by supervising its projection to predict optical flow, and then aligns diffusion features to this subspace through a soft relational alignment loss.", "questions": "See the weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**1) Relevance and Timeliness**\n\nMotion-centric alignment addresses a timely limitation in large video diffusion models, namely the lack of explicit motion understanding, and the approach fits well within the ongoing trend of representation-based fine-tuning.\n\n**2) Clear Methodology**\n\nThe two-stage design (motion feature learning → diffusion feature alignment) is logically presented and easy to follow, providing a clean conceptual link between motion representation learning and diffusion adaptation.\n\n**3) Comprehensive Evaluation** \n\nThe paper conducts extensive experiments on diverse benchmarks and includes both quantitative metrics and human preference studies, demonstrating consistent improvements in motion and physical realism.", "weaknesses": "**1) Literature Coverage**\n\nWhile the introduction claims that prior alignment-based methods mostly focus on visual semantics rather than true motion dynamics, the paper overlooks several recent studies that explicitly target motion-aligned or dynamics-centric representations (e.g.[1], [2]). These works similarly aim to internalize motion dynamics rather than relying on appearance cues. Clarifying how the proposed framework surpasses such motion-aware alignment approaches would help position this work more precisely within the current landscape and better substantiate its claimed novelty and advantage.\n\n**2) Efficiency and Practical Overhead**\n\nAlthough the paper emphasizes that the proposed framework internalizes motion understanding without external simulators or inference-time conditioning, the Stage 1 training involves over 50 K iterations using multiple H100 GPUs. This introduces substantial computational overhead and weakens the claim of simplicity or efficiency. It would be informative to discuss whether this stage truly needs such heavy training to be effective. For instance, how does performance change if the motion projector is trained with fewer iterations or a lighter backbone? \n\n**3) Limited Discussion on Motion Representation Across Layers**\n\nPrior alignment-based works (e.g., REPA, VideoREPA) have shown that the choice of layer for alignment is a critical design factor, often focusing on mid or multiple layers and providing useful insights into how different layers contribute to the denoising process and what types of information they encode. In this work, however, the alignment is applied to a single layer without further discussion, and despite the paper’s focus on learning motion-related representations, it lacks sufficient analysis or discussion of how the aligned layer captures motion information. It would strengthen the paper to elaborate on what insights this approach provides regarding how motion-related representations are distributed across layers, as this understanding could offer a deeper contribution to motion-centric representation learning.\n\n[1] *Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generation*  \n[2] ​​*VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models*", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XOGE339kJG", "reviewer": "Reviewer_mRuk", "summary": "This paper proposes MoAlign, to improve the physical plausibility and temporal coherence of T2V diffusion models. This framework first creates a motion-centric feature with a frozen video encoder to predict optical flow, then the diffusion model's features are aligned to this motion subspace with alignment loss. The results show improvements in physical commonsense benchmarks over baselines, without quality drops.", "questions": "Refer to weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper identifies a key limitation of existing representation between appearance and motion. The proposed two-stage approach is a novel and intuitive solution.\n\n2. Using optical flow as an explicit signal to capture motion-centric representation space is a logical and strong choice into the diffusion model.\n\n3. The paper validates its approach using a wide range of benchmarks.", "weaknesses": "1. Although the effectiveness of proposed method is validated by physical-centric benchmarks, it is demonstrated exclusively on CogVideoX-2B. It is unclear if this motion alignment approach would yield similar benefits for other SoTA video diffusion architectures, which may have different internal representations.\n\n2. The claim of improved plausibility is accompanied by a drop in metrics related to motion dynamism. Specifically, MoAlign shows a large reduction in VBench's 'Dynamic Degree' (70.3 $\\rightarrow$ 42.2) and VBench-2.0's 'Dynamic Attribute' (23.8 $\\rightarrow$ 16.5). Is the model learning to be safer by simply reducing overall motion to avoid making physical mistakes?\n\n3. The model also shows a notable drop in VBench-2.0's 'Physics Score' (53.9 $\\rightarrow$ 48.8). The authors attribute this (L410) to a lack of training samples for phenomena like thermotics and materials. I wonder if the model can easily incorporate these missing physics domains via fine-tuning (e.g., SFT or MoAlign method on a few examples) or if it would require a complete retraining.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zcAaMDfMh7", "reviewer": "Reviewer_evkb", "summary": "MoAlign addresses a critical and timely problem of poor temporal coherence and physical implausability in videos generated by text-to-video diffusion models. The authors hypothesize that this failure stems from the entanglement of appearance and motion dynamics withon the model's internal representations. To address this, the authors provide a two stage fine-tuning framework. In the first stage, a motion centric feature subspace is learned by addition of an optical flow prediction head on top of a frozen video encoder. In the second stage, the internal features of a base text-to-video model are aligned with the distilled motion subspace using a soft relational alignment loss. This procedure is designed to internalize the motion knowledge disentangled from the appearance without adding overhead at inference time.", "questions": "- How does MoAlign compare with VideoJAM (Chefer et al. 2025) in approach and results? Both use optical flow to improve motion realism, VideoJAM adds a flow-prediction objective during training and uses an inner-guidance at inference.\n- Your results consistently show a drop in Semantic Adherence (Table 2) and VBench-2.0 Physics score (Table 3) for all fine-tuned models. While you attribute this to the dataset, could you discuss the possibility that the alignment process itself, by strongly regularizing the model's feature space, inherently creates a trade-off that limits general capabilities? Was a \"Pareto frontier\" of plausibility vs. fidelity observed when tuning the alignment strength $\\lambda$?\n- You report that semantic fidelity is preserved, and user studies even noted improved realism. Can you comment on whether introducing the motion alignment had any side-effects on visual quality or diversity of generations?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The work addresses an important and timely problem. Although video models achieve generating visually plausible videos, most models lack the capability of generating physically plausible motion. \n- The two stage framework is conceptually simple and well-explained. The idea of first learning a motion-specific latent subspace and then aligning the diffusion model's representations to that subspace is innovative. This decoupling of motion from appearance is a neat solution to force the model to internalize dynamics without confusion from static content.\n- The paper's primary goal is to improve physical plausibility, and the results clearly demonstrate success. MoAlign consistently outperforms the base model, a fine-tuned baseline, and the re-implemented VideoREPA on the key \"Physical Commonsense\" (PC) and \"Joint\" metrics across both VideoPhy and VideoPhy2.\n- The paper provides a wide range of experiments: four benchmarks + a user study + ablations. The ablation studies confirm the importance of each proposed component (motion-centric features and relational alignment). This thoroughness increases confidence in the approach. The paper also discusses examples of improved physical common sense (e.g., motion of objects following expected physics) which help illustrate the benefits.", "weaknesses": "- The paper's most significant flaw is its failure to cite, discuss, or compare against \"VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models\" by Chefer et al. (2025). This omission is critical because VideoJAM addresses the exact same problem with a similar philosophy of incorporating an explicit motion signal, but through a fundamentally different mechanism. Both MoAlign and VideoJAM identify that the standard pixel-reconstruction objective is insufficient and biases models towards appearance at the expense of motion. Both propose to solve this by explicitly incorporating a motion-based objective during training.\n- The improvements are mainly in motion consistency, as ensured by optical flow alignment. This addresses many physics issues (e.g., objects move more naturally). The paper doesn’t deeply analyze failure cases, but one can suspect that while MoAlign reduces obvious physics violations (like unnatural motion trajectories), it might still struggle with scenarios requiring understanding beyond motion (e.g., a generation might conserve motion but still violate gravity in subtle ways, etc.). A discussion of limitations would be useful.\n- Although MoAlign is effective, parts of the approach are incremental. Prior work established that aligning diffusion features with a video model can inject knowledge (VideoREPA’s token relation loss is a direct precursor). MoAlign’s novelty mainly lies in isolating motion features via flow supervision. This is a sensible extension rather than a fundamentally new paradigm. The paper could do more to highlight what is truly new. For example, the idea of a learned motion projection head and the specific loss formulation could be emphasized as key innovations. Without clear differentiation, some readers may interpret MoAlign as a minor variant of VideoREPA (when in fact it has important differences). Ensuring the contributions are framed in contrast to related methods (especially REPA methods and VideoJAM) would strengthen the impact.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)