Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance
Authors: Ruihang Chu
Yefei He
Zhekai Chen
Shiwei Zhang
Xiaogang Xu
Bin Xia
Dingdong WANG
Hongwei Yi
Xihui Liu
Hengshuang Zhao
Yu Liu
Yingya Zhang
Yujiu Yang
Venue: NeurIPS 2025 poster ()
Keywords: Video generation
Motion-controllable video generation
Diffusion models
Primary area: applications

Abstract: We present Wan-Move, a simple and scalable framework that brings motion control to video generative models. Existing motion-controllable methods typically suffer from coarse control granularity and limited scalability, leaving their outputs insufficient for practical use. We narrow this gap by achieving precise and high-quality motion control. Our core idea is to directly make the original condition features motion-aware for guiding video synthesis. To this end, we first represent object motions with dense point trajectories, allowing fine-grained control over the scene. We then project these trajectories into latent space and propagate the first frame's features along each trajectory, producing an aligned spatiotemporal feature map that tells how each scene element should move. This feature map serves as the updated latent condition, which is naturally integrated into the off-the-shelf image-to-video model, e.g., Wan-I2V-14B, as motion guidance without any architecture change. It removes the need for auxiliary motion encoders and makes fine-tuning base models easily scalable. Through scaled training, Wan-Move generates 5-second, 480p videos whose motion controllability rivals Kling 1.5 Pro's commercial Motion Brush, as indicated by user studies. To support comprehensive evaluation, we further design MoveBench, a rigorously curated benchmark featuring diverse content categories and hybrid-verified annotations. It is distinguished by larger data volume, longer video durations, and high-quality motion annotations. Extensive experiments on MoveBench and the public dataset consistently show Wan-Move's superior motion quality. Code, models, and benchmark data are made available.

Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance
Ruihang Chu 22+f
Yefei He
Zhekai Chen
Shiwei Zhang Xihui Liu
Xiaogang Xu
Bin Xia
Dingdong Wang'
Hongwei Yi
Hengshuang Zhao
Yu Liu
Yingya Zhang'
Yujiu Yang?
Tongyi Lab, Alibaba Group
"Tsinghua University
"HKU
*CUHK
Github: https://githu..com/aaee wilaa//aiilaaaaaaaaaaaa Move
Abstract
We present Wan-Move, simple and scalable framework that brings motion control to video generative models. Existing motion-controllable methods typically suffer from coarse control granularity and limited scalability, leaving their outputs insuffi- cient for practical use. We narrow this gap by achieving precise and high-quality motion control. Our core idea is directly make the original condition features motion-aware for guiding video synthesis. To this end, we first represent object motions with dense point trajectories, allowing fineggrined control over the scene. We then project these trajectories into latent space and propagate the first frame's 's features along each trajectory, producing an aligned spatiotemporal feature map that tells how each scene element should move. This feature map serves as the updated latent condition, which is naturally integrated into the off-the-shelf image-to- to-video model, e.g., Wan-I2V-14BB
motion guidance without any architecture change. It removes the need for auxiliary motion encoders and makes fine-tuning base models easily scalable. Through scaled training. Wan-Move generates 5-second, 480p videos whose motion controllability rivals Kling 1.5 Pro's 's commercial Mo- tion Brush, indicated by user studies. To support comprehensive evaluation, we further design MoveBench, rigorously curated benchmark featuring diverse content categories and verrffee annotations. It is distinguished by larger data volume, longer video durations, and high-quality motion annotations. Extensive experiments MoveBench and the public dataset consistently show Wan-Move's superior motion quality. Code, models, and benchmark data are made available.
Introduction
Motion lies at the heart of video generation as it fundamentally transforms static images into dynamic visual narratives. Recognizing its importance, both the research community [1, 2, 3] and commercial players [4, 5, 6] have devoted considerable effort controlling motion in video generative models.
The essence motion control lies injecting a motion guidance signal into the video generation process. Thus, the two key choices are (i) how to represent the guidance signal and (ii) how integrate into the generator. First, existing motion guidance representations can be broadly classified into sparse and dense types. Sparse representations include bounding boxes [7, 8] and segmentation masks [1, 9, 10]. Although these signals can steer an object's global movement, they fail to control
"Equal contribution
Corresponding authors
Project leader
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Mukti-object motion
Complex motion
Object Camera motion
Primittie-level control
Motion transfer
3D rotation
Input first frome
Generated video (5s)
Figure 1: Wan-Move is image-to-vine generation framework that supports diverse motion control applications. The generated samples (832x480p. 5s) exhibits high visual fidelity and accurate motion.
local motions. In contrast, dense representations, such pixel-wise optical flow [2, 11, 12, 3] and point trajectories [13, 14], offer more fine- fine-grained control ability. Yet, optical flow requires an additional model for flow estimation during inference, which adds cumulative errors across frames and hampers scalability. While point trajectories are easy to specify during inference, each track only single-pixe thread and lacks surrounding spatial context. This makes it hard align textures and motion patterns across neighboring regions. Second, to inject the guidance signal into generative models, a range of motion encoders have been designed [14, 15, 1, 13, 16, 17], with ControlNet [18] being popular way to fuse motion cues. However, all of these pipelines introduce extra motion-aware modules, which may degrade the motion signal during processing and make harder to fine-tune the video-generation backbone at scale.
To tackle these challenges, we present Wan-Move, a novel motion-control framework that builds on the existing image-to-video (12V) generation model without adding auxiliary motion-processing modules. Our core idea is inject motion information by directly editing the image condition features. We turn it into an updated latent guide that conveys both appearance and motion throughout video generation. Thanks this simple and effective design, Wan-Move delivers high-quality motion control and scales easily by fine-tuning the powerful 12V backbone.
Specifically, represent motion trajectories using point tracks [13] as they capture fine fine-grained local and global movement. Unlike prior work [13] that embeds point trajectories into latent features, we transfer each trajectory from pixel space into latent coordinates. As I2V generation aims to animate the first frame, we guide this process by copying the first-frame feature at each tracked position to its corresponding location in later frames along the latent trajectory. Each copied feature preserves rich context, thus the propagated signal drives more natural local motion, verified in Sec. 5.3. Moreover, since motion guidance injected by editing the image condition features, we add no extra modules. As a result, Wan-Move can plug straight into the 12V backbone, such as Wan-12V-144 [19], and support scalable fine-tuning with fast convergence. Fig. shows that Wan-Move generates high- fidelity video clips (832 x832, 5s) with precise motion control, enabling a diverse set of applications, as illustrated Sec. 5.4. To our best knowledge, the first research model (to open- open-sourced) to match the visual quality of commercial products such Kling 1.5 Pro's Motion Brush [4].

---

To set rigorous, comprehensive evaluation for motion-control methods, we introduce a free-license benchmark termed MoveBench. Compared with existing benchmarks [20, 21, 22] that offer fewer clips, shorter durations, and incomplete motion annotations, Moveßench provides more data, greater diversity, and reliable motion annotations (Fig. 5). Concretely, we design a curation pipeline to categorize the video library into 54 content categories, 10-25 videos each, giving rise to over 1000 cases to ensure a broad scenario coverage, All video clips maintain a 5-second duration to facilitate evaluation of long-range dynamics. Every clip is paired with detailed motion annotations for single or multiple objects. They include both precise point trajectories and sparse segmentation masks to fit wide range of motion-contron models. We ensure annotation quality by developing interactive labeling pipeline. It combines human labeling with SAM [23] predictions, marrying annotation precision with automated scalability. In summary, contributions are follows:
We propose Wan-Move for motion control in image-to-video generation. Unlike prior approaches that require motion encoding, it injects the motion guidance by editing condition features, adding no new modules and allowing the easy fine-tuning of base models at scale.
We introduce MoveBench, a comprehensive and well-curated benchmark to assess motion control. A hybrid human+SAM labeling pipeline ensures annotation quality.
Extensive experiments on MoveBench and public datasets show that Wan-Move supports diverse motion-control tasks and delivers commercial-gnee -grade results with scaled training.
2 Related Work
Video generation models. Video diffusion models [24] pioneer the extension of denoising diffusion probabilistic models (DDPMs) to video generation through a 3D U-Net architecture. Subsequent advancements, such as Imagen Video [25] and Phenaki [26], enhance this framework produce longer and higher-resolution sequences. Nevertheless, these CNN-based approaches [27, 28, 29] face limitations in capturing long-range spatiotemporal dependencies. Transformer-
architectures [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46] overcome this bottleneck and greatly improve training scalability. Recent innovations, including CogVideoX: [47] and Hunyuan Video [6]. further validate the efficacy of spatio-temporal attention mechanisms for coherent video synthesis. Notably, Wan [19] introduces an efficient framework for both text-to-video and image-to-video generation, setting new standard for open-source video models. Our Wan-Move, introduces how to leverage latent trajectory guidance to enable motion control upon the image-to-video diffusion model, enabling precise motion control while preserving visual fidelity.
Motion-controllabbe video generation. To adapt pretrained video generation models for motion- controllable synthesis, training-free methods [48, 49, 50, 51] optimize input noisy latents or manip- ulate attention mechanisms, enabling zero-shot control. However, these approaches often exhibit performance degradation when controlling fine-grained or multi-object motion. In contrast, fine- tuning based methods [52. 2, 15, 1. 14, 20, 53, 16, 54, 55, 56, 10, 12, 57, 58, 59] leverage diverse motion signals and introduce various techniques to integrate them into the base model. While these methods significantly enhance output quality, they typically require auxiliary encoders or fusion modules, complicating the model architecture and limiting training scalability. Among these studies, the most relevant to our work is Motion Prompting [13], as both employ point trajectories to represent motion guidance. However, we differ in two key aspects. First, Motion Prompting [13] encodes point tracks via random embeddings in pixel space, where the guidance is pixel-level threads that lack surrounding context to offer local control. We express point trajectories in latent space using the image feature, providing rich local information and finer control. Second, Motion Prompting [13] integrates motion guidance through separate ControlNet [18], whereas directly use the pretrained base model without architectural modifications, which facilitates its scalable fine-tuning. Sec. 5.3 provides quantitative and qualitative evidence of our advantages.
For more specific robotic scenarios, works [60, 61] rely on pretrained DINOv2 features [62] to transfer object representations across frames for motion control in generated videos, yet both delivers DINOY2's limitation in representing motion signals. DINOv2 excels at high-level semantic encoding but lacks fine-grained object details. Thus, GEM [1] employs additional identity embeddings to distinguish objects and train an ObjectNet to bridge the domain gap between DINOv2 and the UNet's feature space. Moreover, the 14 patch size in DINOv2 may restrict the granularity of the proposed
3

---

Original video
Nakey feature
Timesteps
VAE
-Nosee
W
VAE
Trajectory transfer
O-padding fromes
Concot
DIT blacks
Fenture replication
replication
WAE
Enc
//f
++T T/f
Generated video
W/l.
First frame feature
Candition feature
(a) Latent feature replication
(b) Training pipeline of Wan-Move
Figure 2: (a) To inject motion guidance, we transfer point trajectories from videos to latent space, then replicate the first frame feature into subsequent sero-padded frames along each latent trajectory. (b) Wan-Move trained upon an existing image-to- saage-to-videe video generation model (e.g., work [19, 29]), with efficient latent feature replication step (as in (a)) to update the condition feature. The CLIP [65] image encoder and umT5 [66] text encoder from the base model are omitted for simplicity.
control. In contrast, our approach mitigates these limitations by employing native VAE in I2V foundation models without relying on any auxiliary modules (e.g., identity embeddings).
Benchmark for motion-controllabbe video generation. Most motion-controllabble work evaluates on small, task-specific datasets, which are typically ad hoc, limited in scope, insufficient for evaluating long-range dynamics and multi-object interactions. For example, datasets such as DAVIS [21] and VIPSeg [22] have been repurposed for trajectory control methods, yet their short clip durations and sparse annotations make them inadequate for assessing long-term consistency or complex interactions. While MagicBench [20] expands to 600 video clips, it categorizes samples solely by object count and relies on automatically generated labels from noisy pipelines, limiting the annotation precision. To address these limitations, we introduce MoveBench, comprehensive benchmark for motion- controllable video generation. It includes carefully selected 1018 videos with extensive annotations, long-range dynamics, and 54 well-classified content patterns.
3 Method
3.1 Preliminary
Video diffusion models [24, 19, 6, 63] apply Gaussian noise to clean data during the forward process and learn reverse process to denoise and generate videos. To reduce computational costs, the denoising network typically operates on latent video representations obtained from a pretrained VAE [64]. Given an input video V [[]+T)xH>*'> x3 the encoder compresses both the temporal and spatial dimensions with compression ratios fo (temporal) and fa (spatial), while expanding the channel dimension to C, yielding x E(V) E (1+ f I W. xC The decoder D then reconstructs the video from the latent representation as V = (xx).
Our work focuses on cootion-controllable image-to-vided (12V) generation, where models are required to generate motion-coheren videos based on the input first-frame image and motion trajectories. While the first frame will be encoded into the condition feature Zimuge by the VAE, motion trajectories, which can be represented diverse formats, remain in pixel space. Thus, the key challenge lies in effectively encoding motion trajectories into the condition feature Zmtatonn and injecting into the generative model. To avoid the signal degradation and training difficulties associated with additional motion encoder and fusion modules, we aim to develop motion-control framework that leverages existing I2V models without architectural modifications, as detailed below.
3.2 Latent Trajectory Guidance
To enable video generation conditioned on the first frame, an effective approach of popular 12V models [19, 29] concatenates the latent noise x, and the fist-fframe condition feature Zimage along

---

the channel dimension. Zimge is obtained by encoding the first frame I along with zero cero-padded subsequent frames OT 1 CC using pretrained VAE encoder E:
Zimage & (concat [I. OTxHxWx3)) E R x +x
(I)
For motion guidance representation, we adopt point trajectories, following prior studies [13, 14], as they provide fine-grained control and capture both local and global motion. Formally, a point trajectory of length + T can be represented as p E I(++))x2 where p(n] = (Z Vvs) specifies the trajectory location in the rethh frame in the pixel space. Existing methods often employ auxiliary modules to encode and integrate trajectories into the backbone. Yet, this approach may degrade motion signals during motion encoding. In addition, training extra modules increases the complexity of fine-tuning the base model at scale. This raises a key question: Can we inject pixel-space motion guidance without auxiliary modules?
Intuitively, 12V generation aims to animate the first frame, while motion trajectories specify object positions in each subsequent frame. Given the translation equivariance of VAB models, latent features corresponding trajectory positions should closely resemble those in the first frame. Motivated by this, we propose encoding trajectories directly into latent space via spatial mapping, eliminating the need for an extra motion encoder:
pinl fo Di
if n = 0,
p|n]
(2)
1 5 n S T F
fe
The latent trajectory position at the first frame is derived by spatial mapping, while for subsequent frames, it is averaged over each consecutive ft frames. This deterministically transforms pixel-space trajectories into latent space. To inject the obtained latent trajectories, we extract the latent features of the first frame at the initial trajectory point p/0 and replicate them across subsequent frames according to p. leveraging the translation equivariance of latent features, as shown in Fig. 2 (a):
T
Zimage (n2, p[n, p(n, 1],:] = Zimage p0. p(0, 1]. for n ==
(3)
fe
Here, Zimagele [t, h, w denotes the feature vector at temporal index t. height h, and width W. This operation efficiently injects motion guidance into the condition feature by updating Zimago eliminating the need for explicit motion condition features and injection modules. An overview of the Wan- Move generation framework is presented in Fig. 2(b). When multiple visible point trajectories coincide at a given spatiotemporal position, we randomly select one trajectory's 's corresponding first-frame feature.
3.3 Training and Inference
Training data. We curate a high-quality training dataset, which undergoes rigorous two-stage filtering to ensure both visual quality and motion consistency. First, we manually annotate the visual quality of 1,000 samples and use them to train an expert scoring model for initial quality assessment. To further enhance temporal coherence, we introduce motion quality filtering stage. Specifically, for each video, we extract SigLIP [67] features from the first frame and compute the mean SigLIP features for the remaining frames. The cosine similarity between these features serves as our stability metric. Based on empirical analysis of 10,000 samples, we establish threshold to retain only videos where the content remains consistent with the initial frame. This two-stage pipeline produces a final dataset of 2 million high-quality 720p videos with strong visual quality and motion coherence. Additional details on the training data sources are provided in the supplementary material.
Modeling training. Based on our training dataset, we use CoTrracker [68] to track the trajectories of dense 32x32 grid of points. For each training iteration, we sample trajectories from mixed distribution: with 5% probability, no trajectory is used (k- = 0); with 95% probability, k uniformly sampled from to 200. Notably, we retain a 5% probability of dropping motion conditions, which effectively preserves the model's original image-to- mngde-to-video generation capability. For the selected trajectories, we extract the first-frame features and replicate them to subsequent zero-pade zero-padded frames, formalized by Eq. (3). Since CoTracker distinguishes between visible and
5

---

Poxels row videos
Expert Scoring
Video Processing
Video disploy
Click interfoce
Point tracks
Mask tracks
Video Clustering
Detailed caption sooude
Selected points
Negative points
They ond whi fistness bbot neads and brots, snout us oef sullug rosed cocens dutt win slen with whthhlllg owsere skaii
Manual
Mask of object
Mask of object 2
Video curation
(b) Interactive onnotation
(c) Sample contents
Figure 3: Construction pipeline of MoveBench obtain high-quality samples with rich annotations.
Statistics
Trajectory annotations
Benchmark
Videos
Frames
Video categorizaionn
Mask track
Point track
DAVVIS
35-100
ViPSag (val)
343
24
MagicBench
600
MoveBench
1018
Figure 4: Balanced sample number per class.
Figure 5: Comparison with related benchmarks.
occluded point trajectories, perform feature replication only along the visible trajectories. During training, the model parameters is initialized from the I2V model [19] and fine-tuned to predict the vector field (x,) that transports samples from the noise distribution to the data distribution [69]:
(0) xx. (x,, (, ve (x.))
where denotes the union of the generation condition.
Inference with Wan-Move. The inference process closely resembles the original 12V model, with an additional latent feature replication operation. Specifically, Wan-Move conditions generation three inputs: (1) a text prompt, (2) an input image as the first frame, and (3) sparse or dense point trajectories for motion control. Pretrained umT5 [66] and CLIP [65] models are employed to encode global context from the text prompt and first frame, respectively. The resulting image embedding Zylobal and text embeddings Ziea are then injected into the DIT backbone via decoupled cross- attention [18]. Additionally, a VAE is used to extract the first-frame condition feature Zumage, which will be injected through latent feature replication (as detailed in Sec. 3.2). Classifier-free guidance is applied to enhance alignment with conditional information. Formally, let unconditional vector field V = (x( I, Zimage; Zglobal), ) and conditional vector field Voond=V Xx, Zimage: Zelohal, Ziexr). The guided vector field Vo(x,, L, Zimage; Zelohal, Ziet) is weighted combination of the conditional and unconditional outputs, with the guidance scale wt
Ve (X1, t, Zemage; Zelloall Zrexa) V Vuncond w(( (Veond Vuncond)
(5)
4 MoveBench
Current benchmarks for and lack precise, comprehensive motion annotations, thus introducing bias and limiting granularity. To address these gaps, we introduce MoveBench, high-quality benchmark with 1018 videos (480x832 resolution, 5-second duration), designed for comprehensive evaluation motion-controllable gener- ation, as illustrated in Fig. 3-5. The evaluation videos are selected from Pexels [70], a large-scale, high-quality dataset containing about 400K videos, all released under a free license.
video generation suffer from small scale, short duration,
MoveBench combines algorithmic curation with human expertise to ensure diverse, representative, and precisely annotated motion data. Compared to prior works, offers three key features: (i) High quality. We curate videos through rigorous four-stage pipeline, as illustrated in Fig. 3(a). We first utilize the expert scoring model obtained from Sec. 3.3 score videos based on visual quality, filtering out low-quality content. Then, the selected videos are cropped to 480p and uniformly sampled to 81 frames to ensure temporal consistency. Finally, videos are clustered into 54 content categories and we manually select the 15-25 most representative examples for each category, balancing diversity and quality (Fig. 4). (ii) Precise annotations. As shown Fig. 5, we provide both point and mask annotations, so that methods using mask guidance signals can also be evaluated using our benchmark.

---

Table 1: Performance comparisons MoveBench and DAVIS. Wan-Move consistently yields
substantial improvements both visual fidelity and motion quality across all metrics.
MoveBench
DAVIS
Method
FID4
FVD
PSNR1
SSIM
EPE4
FIDL
FVD.
PSNR1
SSIMI
EPEL
ImageConductor [53]
34.5
424.0
13.4
0.49
15.6
54.2
513.6
11.6
0.47
14.8
Levilor [16]
18.1
98.8
15.6
0.54
3.4
22.0
115.4
13.3
0.51
3.7
Tora [14]
22.5
100.4
15.7
0.55
3.3
25.9
129.2
13.7
0.49
3.5
MagicMotion [20]
17.5
96.7
14.9
0.56
3.2
24.2
113.4
12.8
0.53
3.5
Wan-Move (Ours)
12.2
83.5
17.8
0.64
2.6
14.7
94.3
16.5
0.61
2.5
prompt
dooreup shat shows person usigg on diettre pinerr to amocen wooden board. bath hu handl are grippin the planer, plane movel, wood showspp off bourd.
In dedicon,, womon to sitting the bed using loptos, Nevr hap stands mon wearep o vit headset holddng e shoped controller each playing gome.
Image
Image Conductor
Kliing 1.5
Kling Pro
Won Move
Wan- Move
Figure 6: Qualitative comparisons between Wan-Move and recent approaches, including both academic methods [53, 16] and commercial solutions [4]. Motions that deviate from the specified trajectories and major visual artifacts are marked with red dots and boxes, respectively.
To precisely annotate motion regions, we design an interactive annotation interface as shown in Fig. 3. Annotators click on a target region in the first frame, prompting SAM [23] to generate an initial mask. When the mask exceeds the desired area, annotators add negative points to exclude irrelevant regions. This is critical for isolating articulated motions small objects in cluttered scenes. After annotation, each video contains at least one point indicating a representative motion, with 192 videos additionally including multiple-object motion trajectories. (iii) Detailed captions. We use Gemini [71] to generate dense descriptions covering objects, actions, and camera dynamics. Unlike segmentation datasets like DAVIS [21], our captions are tailored for video generation tasks.
5
Experiment
5.1 Experimental Setup
Wan-Move is implemented top of Wan-12V-14B [19], a state-of-the-art image-to-video (12V) generation model. As described in Sec. 3.3, we fine-tune Wan-Move on a high-quality dataset consisting of 2M high-quality videos. Only the DiT backbone is trainable, while the image and text encoders remain frozen. During inference, we use a classifier-free guidance scale w of 5.0 unless otherwise specified. Detailed training configurations are provided in the supplementary material.
To quantitatively evaluate the fidelity of generated videos, we compute standard video quality metrics including FID [72], FVD [73]. PSNR, and SSIM [74]. To assess motion accuracy, measure the L2 distance between ground truth tracks and those estimated from generated videos, following [13] in denoting this metric end-point error (EPE). All evaluations are performed at a resolution of 480p.
5.2 Main Results
Single-object motion control. We present an extensive comparison between Wan-Move and recent
motion-controllabbee video generation methods [53, 16, 14, 20, 4]. Quantitative results on MoveBench and the public DAVIS [21] are shown Table 1. Qualitative visualizations are presented Fig. Unlike other methods that rely point tracks for motion guidance, MagicMotion [20] takes input
7

---

Wan- Move MogisMotion
Input first frame
Generated videos
Input first frome
Generated videos
Figure 7: Qualitative comparisons with MagicMotion [20], controlling motions using sparse signals (i.e., bounding boxes) as input. Motions that break the guidance are marked with red dots.
Table 2: MoveBench multi-object motion results.
Table 3: Our win rates in 2AFC human study.
Method
FID
FVD]
PSNR
SSIMt
EPEL
Method
Motion accuracy
Motion quality
quality
ImageConductor
77.5
764.5
13.9
0.51
LeviTor
98.2
98.8
Tora
53.2
350.0
14.5
0.54
3.5
Tora
96.2
93.8
98.4
Wan-Move (Ours)
28.8
226.3
16.7
0.62
2.2
MagicMotion
Kling 1.5 Pro
89.4
47.8
53.4
50.2
98.2
Table 4: Ablation motion guidance strategies.
Table 5: Ablation condition fusion strategies.
Motion guidance
FID FVD. PSNR SSIM, EPE]
Cond. fusion
FID
FVD.
EPE/
Latency (s)
Pixel replication
Random track embedding
17.3
15.4
89.2
91.0
16.1
15.3
0.56
0.59
3.7
2.7
ControlNet
12.4
84.6
2.5
987 (+:220)
Latent feature replication
12.2
83.5
17.8
0.64
2.6
Concat. (Ours)
12.2
83.5
2.6
765 (4:3)
sparse masks and bounding boxes. Since boxes can be directly derived from segmentation masks, they are inherently compatible with MoveBench that covering mask annotations. Hence, also conduct comparisons using box-based inputs in Fig. 7. Among these methods, ImageConductor [53] exhibits poor performance in both image and motion quality, which can be attributed to its reliance on direct pixel-level trajectory injection, where single-pixel features lack sufficient semantic and texture information. The remaining methods report similar EPE (3.2-3.44 3.44, despite differing motion guidance approaches: Levitor [16] and MagicMotion [20] utilize the complex ControlNet [18], while Tora [14] adopts the lightweight adaLN [30]. Notably, method achieves the best motion control performance (lowest EPE) and video quality (highest PSNR and SSIM) through latent trajectory replication without introducing additional parameters. This underscores the effectiveness our latent trajectory guidance in adhering to motion constraints. Consistent results on the DAVIS dataset further validate the robustness of our approach.
Multi-object motion control. As MoveBench includes 192 cases with annotated multi-objed motion, further evaluate Wan-Move against baselines [53, 14] this challenging setting, presented in Table 2. Our method achieves significantly lower FVD and reduced EPE compared to other methods, highlighting its precise adherence to motion constraints in more complex scenarios.
Human study, We conduct a two-alternative forced-choice (2AFC) human evaluation comparing Wan-Move with SOTA approaches [14, 4, 16, 20J. Each method generated 50 conditioned samples, which are evaluated by 20 participants. The results, presented in Table 3, report Wan-Move's 'ss win rates across three metries: motion accuracy, motion quality, and visual quality. Compared to Tora [14], Wan-Mov Move achieves win rates exceeding 96% in all categories. When evaluated against the commercial model Kling 1.5 Pro, method demonstrates competitive performance, with superior win rates in motion quality. This narrows the gap between research- oriented and commercial models.
5.3 Ablation Study
Trajectory guidance strategy. We investigate the impact of motion guidance strategies on video quality and motion consistency. Quantitative and qualitative results as presented in Table and Fig. 8, respectively. Pixel replication applies pixel-level ppste along the orig- inal trajectory, followed by VAE encoding. Yet, since single-pixel features contain lim- ited semantic and texture information, the resulting motion control weak, as reflected by high EPE value of 3.7 and generation
Irpur firtt from
Plal ryploction
Bondom Irani smbed. Talere fuct, repiination
Figure 8: Visualization of various guidance strategies.
8

---

Table 6: Ablation maximum number of point
Table 7: Ablation on actual number of point
trajectories (see Sec. 3.3) during training.
trajectories during inference.
Number
FID
FVD!
PSNR1
SSIM
EPE.
Number
FIDL
FVD1
PSNRT
SSIM
EPE!
N=
12.8
86.6
17.6
0.62
3.3
N =
12.8
87.9
17.9
0.64
12.4
N 100
12.9
84.7
17.7
0.65
2.7
N
12.2
83.5
0.64
2.6
N 200
12.2
83.5
17.8
0.64
2.6
N = 16
10.6
78.3
18.2
0.67
2.2
500
13.3
83.9
17.6
0.63
3.0
N 512
7.7
51.0
20.3
0.75
1.5
1024
13.4
0.61
3,9
N= 1024
45.2
0.79
Table 8: Ablation different 12V backbones and training data
Table 9: Large-motion and out- of-distribution-moot subset.
scale. Wan-Move attains better results under the same setting.
Method
Backbone
Data scale FVD.FDD PSNR1 SSIM EPE.
FIDI
FVDj
EPE]
MagicMotion
Cog VideoXXBB
23K
17.5
96.7
14.9
0.56
3.2
Torn
29.1
126.3
4,3
Tora
Wan-Move-Cog-2555
CogVidcoX-SB
CogVideoX-25 VideoX-SB
630K
23K
22.5
16.0
100.4
92.3
16.8
15.7
0.59
0.55
2.8
3.3
Large
Wan-Mowe
MagicMotion
24.6
14,5
119.3
86.6
4,1
3,0
Wan-Move-Cog-6300
Cog VideoX-SB
630K
14.1
87.3
17.2
2.8
Toma
MagicMotion
115.7
Wan-Move
Nan2.1-12V-1444
2000K
12.2
83.5
17.8
0.64
2.6
Wan Move
13,5
86.0
2.8
failures (see Fig. 8). The random track embedding approach, originally proposed for pixel space representations [13]. is adapted to assign randomly initialized embeddings in latent space for injecting motion guidance. While effective for rigid single-region control, this approach fails to incorporate contextual information from surrounding regions, resulting in suboptimal video quality (lower PSNR and SSIM) and stiff motion near tracked points. For example, the hand moves, but surrounding bread remains static in Fig. 8. In contrast, proposed latent feature replication method achieves superior video quality (highest PSNR 17.8) and precise motion control (lowest EPE 2.6).
Condition fusion strategy. We compare different motion condition approaches, namely Control- Net [18] and direct concatenation (our approach). The results are presented in Table 5. Notably, simple concatenation of motion conditions with input noise achieves performance comparable to ControlNet motion-controllable generation. Yet, ControlNet introduces significant additional modules, substantially increasing inference latency by 225 seconds over the original 12V model. In contrast, Wan-Move preserves the base model architecture and only adds a one-time trajectory extraction process, increasing just 3-second inference time.
Number of point trajectories during training. Table 6 evaluates the impact of the maximum number of point tracks (N) during training. As N increases from 10 to 200, the model's 's motion-followee capability improves progressively, evidenced by the decreasing EPE. The optimal performance, in terms of both structural similarity (SSIM) and EPE, is achieved at N=200. However, further increasing the number point tracks leads to a rise EPE. This can be attributed to the mismatch between the dense point tracks in the training and the sparse point tracks during evaluation.
Number of point tracks during inference. Table 12 ablates the performance of Wan- Move across varying numbers of point trac- jectories over MoveBench. As the number of tracks increases, EPE drops significantly, indicating better motion guidance and en- hanced temporal coherence. When reaching the maximum number of point trajectories extracted by CoTracker, Wan-Move achieves the lowest EPE of 1.1. Thougg it is trained with at most 200 tracks, the model shows strong generalization capability. Notably, naive 12V inference (with no point tracks) yields PSNR and SSIM scores comparable to motion- controlled generation, confirming that our model strongly retains its inherent I2V quality, Naive 12V samples generated by Wan-Move are presented in Fig. 9.
group onts dusay nowing and of th nest. farr bookss assicyy altoroting poinemm of brown and balon.
On a woodon piece of brown burlop ungoNicd, revesting DONqUeT papes Bouquet concuts layers of purale flowers fowerrs blossons
Figure 9; I2V results of Wan-Move (no point tracks).
Backbones and data scale. In pursuit of the best generation quality, we initially train Wan-Move with a large-scale dataset and a strong backbone. ensure a fair comparison with the leading approaches MagicMotion and Tora, we align Wan-Move's 's backbone and training data scale with them. This yields two variants, iee, Wan-Move-Cog-22 and Wan-Move-Cog-63N which are trained on 23K and 630K data samples respectively, using CogVideoX1..-SB-DD [47] backbone. The

---

detailed comparison on MoveBench is shown in Table 8. Under the same backbone and data scale, Wan-Move still outperforms these two powerful methods.
Evaluation on large-motion and out-of-distribution motion scenarios. To further verify the model generalizability, we curate subsets from MoveBench containing high-amplitude and out-of- distribution motion control cases. For each video, its motion amplitude score is computed as the average of the top 2% largest optical flow values extracted by RAFT [75]. The top 20% highest-score videos are selected as large-motion videos. Besides, we manually curate 50 uncommon motion cases as -itt----iitrbuttion subset, including complex foreground-backgroundd interactions, objects moving out of frame, and rare camera motions. Evaluation results on these challenging examples are shown in Table 9. Notably, Wan-Move consistently outperforms two leading baselines, with performance gaps further widening under these difficult condition. In addition. Wan-Mover performance only marginally drops compared to results on the full benchmark, demonstrating robustness.
5.4 Motion Control Applications
As point trajectories can flexibly represent various types of motion, Wan-Move supports a wide range of motion control applications, as showcased in Fig. 1. First, rows 1-2 show object control using single multiple point trajectories. For camera control (row 3),
either drag background elements directly or follow the approach of work [13]. The latter estimates a point cloud from a monocular depth predictor [76], projects it along camera pose trajectory, and applies z-buffering to obtain aamera-aligned 2D trajectories. Following work [13], we perform primitive-level control by rotating a virtual sphere to generate projected 2D trajectories for globe motion (row 4). row we enable motion transfer by applying trajectories extracted from one video to update the condition features of different image. Row 6 shows 3D rotation control by estimating depth-based positions, applying rotation, and projecting the results to 2D. We refer readers to the supplementary file for more visualizations and full videos.
Conclusion and Discussion
We propose Wan-Move, a simple and scalable framework for precise motion control in video generation. represents motion with point trajectories and transfers them into latent coordinates through spatial mapping, requiring extra motion encoder. We then inject trajectory guidance into first-frame condition features via latent feature replication, achieving effective motion control without architectural changes. For rigorous evaluation, further present MoveBench, a comprehensive and well-curated benchmark featuring diverse content categories with hybrid-verified annotations. Extensive experiments
MoveBench and public datasets show that Wan-Move generates high- quality, long-duration (5s, 480p) videos with motion controllability on par with commercial tools like Kling 1.5 Pro's 's Motion Brush. We believe our open-sourced solution offers an efficient path to scale motion-controllable video generation and will empower a wide range of creators.
Limitations and broader impacts. Wan-Move unreliable when tracks are missing due to occlusion. While we observe that short-term occlusions can be recovered once the point reappears, showing degree of generalization, prolonged absence may lead to loss of control (see Appendix). As with other generative models, Wan-Move carries dual-use potential. Its ability produce realistic, controllable videos can benefit creative industries, education, and simulation, but also risks misuse for generating misleading or harmful content.
point trajectories to guide motion, which be
10

---

7 Acknowledgment
This work was supported by the National Natural Science Foundation of China (Grant No. 62576191).
References
[1] Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for anything using entity representation. In ECCV, 2024.
[2] Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: tne-grained control in video generation by integrating text, image, and trajectory. arXiv:2308.0800 088089 2023.
[3] Ryan Burgert, Yuancheng Xu, Wengi Xian, Oliver Pilarski, Pascal Clausen, Mingming He, Li Ma, Yitong Deng, Lingxiao Li, Mohsen Mousavi, Michael Ryoo, Paul Debevec, and Ning Yu. Go-with-the-flow: Motion-controllable video diffusion models using real-time warped noise. In
CVPR, 2025.
[4] Kuaishou. Kling ai. http:///kinndd.kkuissoo..cm 2024.06.
[5] Runway. Gen-3. https//rumwwayml.com, 2024.06.
[6] Weijie Kong, Qi Tian, Zijian Zhang, Rox Min. Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv:2412.03603; 2024.
[7] Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen, Motionbooth: Motion-aware customized text-to-video generation. NeurIPS, 2024.
[8] Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guoqiang Wei, Liping Yuan, and Hang Li. Boximator: Generating rich and controllable motions for video synthesis. arXiv:2402.01 01 566. 2024.
[9] Zuozhuo Dai, Zhenghao Zhang, Yao Yao, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Animateanything: Fine-grained open domain image animation with motion guidance. arXiv:2311.12886 2023.
[10] Haitao Zhou, Chuang Wang, Rui Nie, Jinlin Liu, Dongdong Yu, Qian Yu, and Changhu Wang. Trackgo: A flexible and efficient method for controllable video generation. In AAAI, 2025.
[11] Mathis Koroglu, Hugo Caselles- Duppeé, Guillaume Jeanneret Sanmiguel, and Matthicu Cord. On- lyflow: Optical flow based motion conditioning for video diffusion models. 2.X55:2411.10001, 2024.
[12] Xiaoyu Shi, Zhaoyang Huang, Yu-Yun Wang. Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. In SIGGRAPHI, 2024.
[13] Daniel Geng, Charles Herrmann, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Ta- tiana Lopez-Guevara, Carl Doersch, Yusuf Aytar. Michael Rubinstein, al. Motion prompting: Controlling video generation with motion trajectories. arXiv:2412.02700, 2024.
[14] Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trnjectory-oriente diffusion transformer for video generation. arXiv:2407.22 175,, 2024.
[15] Zhouxia Wang. Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: unified and flexible motion controller for video generation. In SIGGRAPH, 2024.

---

[16] Hanlin Wang, Hao Ouyang, Qiuyu Wang, Wen Wang, Ka Leong Cheng, Qifeng Chen, Yujun Shen, and Limin Wang. Levitor: 3d trajectory oriented image-to-vice synthesis. arXiv:24/2.1522) 15214, 2024.
[17] Yuqing Chen, Junjie Wang, Lin Liu, Ruihang Chu, Xiaopeng Zhang, Qi Tian, and Yujiu Yang. O-disco-edit: Object distortion control for unified realistic video editing. arXiv preprint arXiv:2509.0159 2025.
[18] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023.
[19] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-We Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. et al. Wan: Open and advanced large-scale video generative models, arXiv preprint arXiiv2503.20314 2025.
[20] Quanhao Li, Zhen Xing, Rui Wang. Hui Zhang, Qi Dai, and Zuxuan Wu. Magicmotion: Controllable video generation with dense-to-sparse trajectory guidance. arXiv:2503. 16421, 2025.
[21] Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Sorkine- and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv: 170.000704 2017.
[22] Jiaxu Miao, Xiaohan Wang, Yu Wu, Wei Li, Xu Zhang, Yunchao Wei, and Yi Yang. Large-scale video panoptic segmentation in the wild: A benchmark. CVPR, 2022.
[23] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan- Lo, Piotr Dollár, and Ross Girshick. Segment Anything. In ICCV, 2023.
[24] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. arXiv:2204.0344 2022.
[25] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv:2210.02303, 2022.
[26] Ruben Villegas, Mohammad Babacizadch, Pieter-Jan Kindermans, Hernan Moraldo. Han Zhang. Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual description. arXiv:2210.02399, 2022.
[27] Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In CVPR, 2024.
[28] Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv:2308, 06571, 2023.
(29) Omer Bar- Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, al. Lumiere: A space-time diffusion model for video generation. In SIGGRAPH Asia, 2024.
[30] William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
[31] Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. 2rX5v:2205.15860 2022.
[32] Junsong Chen, Jincheng Yu. Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-a: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In ICLR, 2024.
[33] Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. In ICLR, 2025.
12

---

[34] Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richard- son. Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon. Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Ltx-video: Realtime video latent diffusion. arXiv:2501.00/0 2024.
[35] Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. TMLR, 2025.
[36] Bin Lin. Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv:2412.0013 2024.
[37] OpenAl. Video generation models as world simulators, 2024.
[38] Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv:2410.1377 1372200....... 2024.
[39] Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models. arXiv:2405.0422 2024.
[40] MiniMax. Hailuo ai. https://aaitocii.cm//iic 2024.09.
[41] Zangwei Zheng, Xiangyu Peng. Tianji Yang. Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, March 2024.
[42]
GemmoTeam. Mochi 1.
2024.
[43] Guoqing Ma, Haoyang Huang. Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming. Xiaoniu Song, Xing Chen, et al. Step-video-12v technical report: The practice, challenges, and future of video foundation model. arXiv:2502.10244, 10248 2025.
[44] Junsong Chen, Yuyang Zhao, Jincheng Yu, Ruihang Chu, Junyu Chen, Shuai Yang. Xianbang Wang, Yicheng Pan, Daquan Zhou, Huan Ling, et al. Sana-video: Efficient video generation with block linear diffusion transformer. arXiv preprint arXiv:2509.24685 2025.
[45] Shuai Yang. Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, et al. Longlive: Real-time interactive long video generation. arXiv preprint MiXXi:25099226222 2025.
[46] Shentong Mo, Enze Xie, Ruihang Chu, Lanqing Hong. Matthias Niessner, and Zhenguo Li. Dit-3d: Exploring plain diffusion transformers for 3d shape generation. Advances in neural information processing systems, (1867960-679711 2023.
[47] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Xiaotao Gu, Yuxuan Zhang, Weihan Wang. Yean Cheng, Ting Liu, Bin Xu, Yuxiao Dong, and Jie Tang. VogVideoX: Text-to- Vidco Diffusion Models with An Expert Transformer. In ICLR, 2025.
[48] Koichi Namekata, Sherwin Bahmani, Ziyi Wu, Yash Kant, Igor Gilitschenski, and David B Lindell. Sg-i2v: Self-guided trajectory control in image-to-video generation. arXiv:24/1.04989 2024.
[49] Haonan Qiu, Zhaoxi Chen, Zhouxia Wang, Yingqing He, Menghan Xia, and Ziwei Liu. Freetraj: Tuning-free trajectory control in video diffusion models. 2xXi:22466.66663 2024.
[50] Wan-Duo Kurt Ma, John Lewis, and W Bastiaan Kleijn. Trailblazer: Trajectory control for diffusion-based video generation. In SIGGRAPH Asia, 2024.
[51] Yash Jain, Anshul Nasery, Vibhav Vineet, and Harkirat Behl. Peekaboo: Interactive video generation via masked-diffusion -ifffusion. In CVPR. 2024.
13

---

[52] Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. VideoComposer: Compositional video synthesis with motion controllability. In NeurIPS, 2023.
[53] Yaowei Li, Xintao Wang, Zhaoyang Zhang, Zhouxia Wang, Ziyang Yuan, Liangbin Xie. Ying Shan, and Yuexian Zou. Image conductor: Precision control for interactive video synthesis. In AAAI, 2025.
[54] Xiao Fu, Xian Liu, Xintao Wang, Sida Peng, Menghan Xia, Xiaoyu Shi. Ziyang Yuan, Pengfei Wan, Di Zhang, and Dahua Lin. 3dtrajmaster: Mastering 3d trajectory for multi-entity motion in video generation. In ICLR, 2024.
[55] Qinghe Wang, Yawen Luo, Xiaoyu Shi, Xu Jia, Huchuan Lu, Tianfan Xue, Xintao Wang, Pengfei Wan, Di Zhang, and Kun Gai. Cinemaster: A 3d-aware and controllable framework for cinematic text-to-video generation. arXiv:2502.08633, 2025.
[56) Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, al. Diffusion as shader: 3d-aware video diffusion for versatile video 2025.
generation control. wwx.v:2501.03844
[57] Yingjie Chen, Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Perception-as- control: Fine-grained controllable image animation with 3d-aware motion representation. arXiv:2501.05020 2025.
(58) Xiang Wang, Shiwei Zhang, Haonan Qiu, Ruihang Chu, Zekun Li, Yingya Zhang, Changxin Gao, Yuehuan Wang, Chunhua Shen, and Nong Sang. Replace anyone in videos. arXiv preprint arXiv:2409. I9911, 2024.
[59] Bin Xia, Jiyang Liu, Yuechen Zhang. Bohao Peng, Ruihang Chu, Yitong Wang. Xinglong Wu, Bei Yu, and Jiaya Jia. Dreamve: Unified instruction-basee image and video editing, arXiv preprint arXiv:2508. 060800000 2025.
[60] Mariam Hassan, Sebastian Stapf, Ahmad Rahimi, Pedro Rezende, Yasaman Haghighi, David Brüggemann, Isinsu Katircioglu, Lin Zhang, Xiaoran Chen, Suman Saha, et al. Gem: A generalizable ego-vision multimodal world model for fine-grained ego-motion, object dynamics. and scene composition control. In Proceedings of the Computer Vision and Puttern Recognition Conference, pages 24414-2415,, 2025.
[61] Aram Davtyan, Sepehr Sameni, Bjöm Ommer, and Paolo Favaro. Cage: Unsupervised visual composition and animation for controllable video generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 16163-16171, 2025.
[62] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
[63] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv:2311.15127, 2023.
[64] Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
[65] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In JCML, 2021.
[66] Hyung Won Chung, Noah Constant, Xavier Garcia, Adam Roberts, Yi Tay, Sharan Narang, and Orhan Firat. Unimax: Fairer and more effective language sampling for large-scale multilingual pretraining, arXiv:2304.09151. 2023.
[67] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lueas Beyer. Sigmoid loss for language image pre-training, In ICCV, 2023.
14

---

[68] Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: is better to track together. In ECCV, 2024.
[69] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv:2210.02747 2022.
[70]
JovianZM. Pexels-400k dataset. http::/hungginngeec..oooaaasssss
2024.
[71] Gemini Team, Rohan Anil, Sehastian Borgeaud, Jean- Sean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. 0rXiv:2312.11800 2023.
[72] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. In NeurIPS, 2017.
[73] Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphal Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. In ICLR, 2019.
[74] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. 77P, 2004.
[75] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In European conference on computer vision, pages 402-419. Springer, 2020.
[76] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. In NeurlPS, 2024.
[77] Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang. Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, and Sergey Tulyakov. Panda-70m: Captioning 70m videos with multiple cross modality teachers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[78] Pixabay. Discover and download free videos pixabay, 2025.
[79] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song. Kai Dang. Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar 2xx::2502.113233 2025.
[80] PyTorch Team. Pytorch fully sharded data parallel (fsdp). http::/byyoocch..ggdooossaaaaae html, 2021. Accessed: [Insert Date].
[81] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
[82] Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309. 230,0,9909,4509, 2023.
[83] Aaron Hurst. Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-40 system card. arXiv preprint arXiv:2410.21276 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, detail our task objectives, weaknesses in the current methods, and the improvements we propose for these weaknesses.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
In the end part of main paper, we discuss the limitations of this paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We fully disclose all the experimental information Section 5.1 and supple- mentary material.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification:
The code will be open-sourced upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (htips:/niips.ccppbbiic Sondes//oodeSubmisiioooollii
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (htips: SoaieddseeCCeSStmmiiiooooliiii
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
We specify the training and test details in Section 5.1 and supplementary material.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: For motion-contrenn controllable video generation tasks, the eтror bar is uncommon, and there is currently no mature error evaluation system in this community.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide training details in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurips.. http://wwwppp.ooppiillleecesccia
Answer: [Yes]
Justification: We have carefully reviewed the NeurIPS Code of Ethics to ensure that our submission complies with all regulations.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: have discussed broader impacts in Section
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The pre-trained model used in paper is image-to-video model, and the content of the generated video is specified by the input image. The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The models and data used in this paper are open-sourced and authorized.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
assetts are released, the license, copyright information, and terms of use package should be provided. For popular datasets, paperswithcode. https//ihhood..o//tttteeeee has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
20

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy Ottp::/leeiiisccccccocceeeeeesss for what should or should not be described.
22

---

Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance
Supplementary Material
Contents
Implementation Details
23
8.1
Training Data Details
23
8.2
MoveBench Construction Details
23
8.3
Training and Inference Configuration
24
9
Additional Experiments
25
9.1
Choice of Feature Replication Strategies under Trajectory Overlap
25
9.2 Choice of Different Training Strategies
25
9.3 Model Performance Under Trajectory Disappearance
25
9.4 Failure Cases
25
10 Qualitative Visualizations
26
10.1 More Qualitative Comparisons
26
10.2 More Camera Control Results
26
10.3 More Motion Transfer Results
26
10.4 More 3D Rotation Results
27
8 Implementation Details
8.1 Training Data Details
Table 10 presents the composition of the filtered training datasets, which are sourced from Panda70M [77], Pixabay [78], Pexels [70], and YouTube. YouTube videos are independently collected for this study. To prevent data leakage, the videos from Pexels are strictly separated from those in the proposed MoveBench.
All videos for training are captioned using Qwen2.5-VL. [79], with the prompt structure illustrated in Fig. 10. This prompt emphasizes motion and camera attributes while preserving the fundamental scene descriptions, ensuring that the model semantically understands the context and generates physically plausible motions. The same captioning prompt is applied to the videos in MoveBench.
8.2 MoveBench Construction Details
Video content clustering. Following the initial filtering stage, we conduct a rigorous content clustering process to ensure broad scenario coverage in our benchmark. Specifically, we sample 16 frames per filtered video and compute the average of their SigLip [67] features. Using k-means clustering, group these features into 54 distinct content categories. Each category label, e.g., Tennis, is then automatically captioned using Qwen2.5- VL [79]. Finally, we manually select the 15-25 most representative videos per category to maintain balance of diversity.
Interactive labeling. Existing models often fail to accurately identify representative motion regions in videos, the most prominent motion may not be optimal, and many motions terminate prematurely.
23

---

Table 10: The statistics of the training datasets.
Figure 10: Prompt for video caption.
Video Caption Prompt
Dataset source
Number
Captioner
pangraph.
orainaass Desciits their chrorologiaa ordecurr
Panda70M [77]
0.56M
QW
dreely deseribe the aposarance and number of these subjects, indled detais the coloe sim and orlentation yV"
"o Mestion caatss retationsips between su subjects mitean In hron of to the left ot seclis"
Pixabay [78]
0.42M
Qwen2.5-VI.
You shou desor be the camoro persocctive and movement at the end of the
decriition, induding the eooo amgle (e top-dwwn, thomal vind especia contilbutte
(e& ssom our,
Pexels [70]
0.25M
Qwen2.5-VV
percestion motion baciground
YouTube
0.75M
Qwen2.5-VL
include tee
romed dharamers analysk realikk
snimated) urlect ecovria for undentarding the motion (n") "Koop your dosu poin condle and flucnt, idoally within scntenoos Your description
Annotations!
Regeciov pain!
Posttive point
Segrented
spectly different onjerts
change point Cype
Clear Pulida
Figure 11: The interactive annotation interface displays the video (left) and its first frame (right). Users click green positive points to specify the start point of a motion trajectory, and red negative points exclude irrelevant regions if needed. SAM segments the mask of moving objects & regions for user review. To annotate multiple motion trajectories, users must assign different object IDs.
To facilitate precise annotation of motion regions, introduce interactive labeling interface (Fig. 11) for selecting the initial motion point and its corresponding mask in the first frame. Annotators begin by selecting target point in the initial frame, prompting SAM [23] generate a preliminary segmentation mask. If the mask extends beyond the desired area, negative points can be added to exclude irrelevant regions. This method effectively isolates articulated motions small objects. For subsequent frames, point trajectories are automatically extracted using CoTracker [68].
8.3 Training and Inference Configuration
As illustrated in Sec. 5.1 of the main paper, we employ Wan-12V-14B [19] as the base 12V model. During training, both the DiT and umT5 components of Wan are wrapped with Fully Sharded Data Parallel (FSDP) [80], with parameters cast torch. bfloatle 6 for memory efficiency. The training employs the Adam W optimizer [81] with weight decay of 1e-3 and a base learning rate of 5e-6. The first 2,000 steps are used for linear warm-up enable smooth transition from the initial 12V generation (corresponding to 0 point trajectories) to motion-controllable video generation. We adopt flow matching objective for optimization, where number time sampling steps set to 1,000 during training. To enable large-scale training with long sequences (e.g., 5s video clip), adopt the Ulysses sequence parallelism strategy [82] following Wan, setting the sequence parallel size 4. We train model using 64 NVIDIA A100 GPUs, with each GPU processing quarter of
24

---

Table 11: Impact of feature replication strategies when multiple motion trajectories overlap.
Table 12: Impact of using dense-to-sampling training strategy.
strategy
FIDI
FVD.
PSNR1
SSIM
EPEL
Strategy
FIDI FVDJ
PSNRT
SSIM
EPEL
Average
13.1
83.4
17.5
0.63
2.7
Sase-to-Sampling
12.9
84.2
17.5
0.62
2.6
Random
12.2
83.5
17.8
0.64
2.6
Sumpling
12.2
83,5
17.8
0.64
2.6
Input first frome
Generated video
Figure 12: Wan-Move generalizes to continue controlling motion trajectories when they temporarily disappears. Green circles indicate visible segments, while red circles mark invisible segments, e.g., occluded or out-of-frame parts.
the sequence length, for a total of 30,000 steps. During inference, we follow Wan's 's sampling scheme with 50 sampling steps.
Additional Experiments
9.1 Choice of Feature Replication Strategies under Trajectory Overlap
We analyze the impact of feature replication strategies in cases of trajectory overlap. The results, presented in Table II, demonstrate that randomly selecting single trajectory's first-frame feature for replication when multiple trajectories coincide yields superior video quality and motion control. This is evidenced by lower FVD and EPE compared to feature averaging. We hypothesize that averaging features from overlapping trajectories leads information loss, thereby degrading performance.
9.2 Choice of Different Training Strategies
This subsection evaluates the performance differences between tonse-to-sampling and direct sampling training strategies, presented Table 12. Prior work [14, 15, 2, 53] commonly employs two-stage dense-to-sampling pipeline, where the first stage uses dense motion trajectories to enhance motion control followed by sparse trajectories in the second stage. However,
find that model, trained with randomly sampling of 1-200 points (as refer to Sec. 3.3 in the main paper), achieves comparable EPE and lower FVD compared the two-stage approach. These results demonstrate that our method provides generalization capability in point trajectory numbers while simplifying the training process. This generalization ability is also verified in Table 12 of the main paper.
9.3 Model Performance Under Trajectory Disappearance
Fig. 12 illustrates our model's ''s capability to generate motion-coheren videos when handling temporar- ily invisible trajectories. The Wan-Move maintains stable generation quality in these challenging scenarios, which we attribute to both the presence of similar cases in the training data and the model's inherent generalization capacity.
9,4 Failure Cases
This subsection analyzes and visualizes three primary failure modes Wan-Move, as illustrated in Fig. 13. First, control degradation occurs when motion trajectories remain invisible for extended durations, causing the model to lose conditional guidance. Second, performance deteriorates visu- ally complex scenes with multiple interacting objects in crowded environments. Third, implausible
25

---

Won- Move
Ins50le due body occluion
Renl video
Invsible
(o) Losing motion control due trojectory disoppearonce
Won- Move
Real video
(b) Visual artifacts under complex and crowded environment
Won Move
Real video
(c) Vialating rigorous physical laws
Figure 13: Three primary failure modes of Wan-Move. (a) Loss of motion control due to persistent trajectory disappearance; (b) Visual artifacts in overly complex, crowded environments; and (c) motion outputs that violate rigorous physical laws.
motion trajectories that violate fundamental physical laws result in Furthermore, erroneous tracking points identified by CoTracker [68] may compound these failure
predictions.
modes.
10
Qualitative Visualizations
10.1
More Qualitative Comparisons
We present additional qualitative comparisons with fate--of-the-an academic [14] and commercial [4]
approaches, as shown Fig.
10.2 More Camera Control Results
As demonstrated in Fig. 15, Wan-Move enables camera control. This can be accomplished, following the work [13], by estimating point cloud using monocular depth predictor [76], projecting it along a predefined camera trajectory, and applying z-buffering to derive occlusion flags and camera-aligned 2D trajectories.
10.3 More Motion Transfer Results
This subsection presents dense motion transfer visualizations generated by Wan-Move using dense point trajectories (1,024 in our implementation). illustrated Fig. 16, Wan-Move achieves nearly identical appearance quality and motion alignment compared to the original videos given dense
26

---

Toro
repeated press
Kiln 1.5 Pro
repeoted
Won-Move
repeoted press
Tora
Kiing 1.5 Pro
Won-Move
Toro
bbock-andforthh motion
Kiing 1.5 Pro
back-and-forth mation
Won-Move
back- bback-andfoor and-forth motion
Input first frome
video
Figure 14: Additional qualitative comparisons with Tora [14] and the commercial model Kling 1.5 Pro [4]. Wan-Move demonstrates superior motion accuracy and visual quality. Major motion control failures visual artifacts are denoted with red boxes.
trajectory conditions and the same first frame. Moreover, Wan-Move also enables video editing by copying the motion while using an additional image editing model to modify the content the first frame, maintaining the original video's 's motion trajectories, as shown in Fig. 17.
10.4 More 3D Rotation Results
As illustrated in Fig. 18, Wan-Move additionally supports 3D object rotation. This capability is realized by first estimating depth-based 3D positions, applying rotational transformation, and then reprojecting the results into 2D trajectories. These trajectories subsequently serve conditioning inputs for our model to rotate the objects in videos.
27

---

Linear displocement
Linear displacement
Dolly in
TIME BBQU
BBO
Dolly out
Input frame
Generated video
Figure 15: Wan-Move enables effective and flexible camera control through different point trajectories, such as linear displacement, dolly in, and dolly out.
Original video
Dense motion copy
Won-Mern Move
Original video
copy
Won- Won-Mon Move
Input first frame
Figure 16: Wan-Move enables accurate video motion copy using dense point trajectories (e,g., 1024 points). The synthesized video preserves high fidelity in both appearance and object-level motion alignment with the original video, even under complex environmental conditions.
28

---

Original video
Edit the first frome: "chonge the color of the cloth to yellow"
Won Move
Origina video
Edit the first frome: "Transform Imoge into a Studio Ghibli style"
Wan Move
Input first frame frame
Generated video
Figure 17:
Wan-Move enables video editing through motion copy and additional image editing
models. It first applies the image editing model (e.g., ControlNet [18), GPT-4o [83] modify the style or content of the first frame, then uses the original video's motion trajectories to animate the edited image frame.
3D rototion
3D rotation Input first frome
Generated video
Figure 18: Wan-Move enables object 3D rotation by estimating depth-based positions, applying rotation, and projecting the results to 2D.
29


Reviews:
[{"review_id": "iYo71Fyllz", "reviewer": "Reviewer_b1sz", "summary": "The paper proposes a representation for trajectory conditions and achieving such guidance in image-to-video generation. The representation basically transforms a sequence of points into the latent space and then uses the new representation to create feature maps for subsequent frames after the 1st frame. These condition features are concatenated with the noisy patches as input to the DiT backbone without having the need to modify the overall architecture or introduce new tunable modules. The authors curated a large-scale dataset for training and also a benchmark called MoverBench for evaluation. Experimental results show that Mover outperforms many previous baselines.", "questions": "See weaknesses", "limitations": "See weaknesses", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "(+) The paper comes up with a very smart way to represent trajectories in the latent space and then uses these representations to create conditional features to achieve motion control. This eliminates the need for an additional control network or a new tunable module, making the entire model more compact and unified. \n\n(+) A large-scale benchmark is introduced, which seems of high quality and could benefit the community by providing a standard benchmark beyond DAVIS. \n\n(+) Experimental results suggest superior control achieved by Mover compared to previous baselines. It also works well with multiple trajectories. \n\n(-) While Mover saves from using an additional module, it also has the drawback of changing the whole parameter of the original DiT backbone. But with ControlNet, one gets to freeze the backbone and tune half of the parameters and use multiple trained ControlNets in a plug-and-play manner. \n\n(-) The evaluation comparison is not fair. Mover has used a lot more high-quality videos compared to all the baselines and potentially a stronger base model (Wan-14B v.s. e.g., CogVideoX in baselines). Of course, it's never easy for this type of work to make a fair comparison due to all kinds of reasons and limitations. For example, the most comparable model is Motion Prompting, which trains a naive ControlNet with a similar number of videos as Mover -- but it is not open-sourced. However, the work needs to clarify the differences between Mover and the baselines in terms of base models and the number of training videos.", "was_revised": true, "final_justification": "The authors have addressed all my concerns.", "rebuttal": "Dear Reviewer b1sz,\n\nThank you for the valuable comments and for recognizing the value of our method, the benchmark design, and the experimental results. We are happy to respond to each question below.\n\n**Q1: Compared to ControlNet that freezes the backbone and tunes half of the parameters, Mover changes the whole parameters of the original DiT backbone.**\n\nA1: We fully update the DiT backbone due to a better trade-off between video quality and inference cost. As shown in Table 5 of the main paper, compared to the ControlNet-based Mover variant, our full fine-tuning approach performs favorably ($\\downarrow$ 0.2 FID and $\\downarrow$ 1.1 FVD), while reducing test time by 22.4% and overall parameter size by over 30%. Moreover, despite updating the entire backbone, Mover effectively preserves the original image-to-video (I2V) capability of the Wan2.1-I2V backbone, as verified both quantitatively (Table 7) and qualitatively (Figure 9). This is because motion condition is occasionally dropped during training.\n\nWe also agree that plug-and-play modules have their advantages. In addition to the ControlNet variant, we now train a Mover-LoRA version, which retains Mover’s design while replacing full tuning with LoRA. This version is also a plug-and-play solution and adds only 1.8% parameters. As compared in **Table R1** below, Mover (full fine-tuning) achieves the best overall performance over its two variants, i.e., Mover-ControlNet and Mover-LoRA. Notably, Mover-LoRA also consistently outperforms all academic baselines in the main paper. We are happy to release all variants to support diverse usage scenarios.\n\n**Table R1:** Comparisons of training strategies on MoverBench.\n| Approach    | FID  $\\downarrow$ | FVD  $\\downarrow$ | EPE  $\\downarrow$ | Latency (s) $\\downarrow$|\n|------------------|:-------:|:-------:|:-------:|:--------------:|\n| Mover-LoRA   | 13.9   | 85.4   | 2.7    | 765   |\n| Mover-ControlNet       | 12.4   | 84.6   | **2.5**    | 987|\n| Mover (full fine-tuning)   | **12.2**   | **83.5**   | 2.6    | **765**  |\n\n\n**Q2: Among existing methods, Motion Prompting is the most comparable in terms of training data scale, but it is not open-sourced. While a fully fair comparison is difficult, the work needs to clarify the differences in backbones and training data scale with baselines.**\n\nA2: We train Mover with large-scale data and a stronger backbone so as to pursue the best possible generation quality. As a result, Mover performs competitively with the commercial model Kling 1.5 Pro Motion Brush in human evaluations, which has not been achieved by prior academic works. \n\nWe fully acknowledge your suggestion and will clarify more setting difference. Although Motion Prompting is closed-source, we have compared one of its core design with Mover fairly. Specifically, Motion Prompting formulates motion guidance using random track embeddings, while Mover adopts latent feature replication. Based on the same dataset and backbone, we directly compare the two designs in Table 4 (quantitative results) and Figure 8 (qualitative comparisons). Our approach consistently outperforms the Motion Prompting's design in video quality. \n\nAdditionally, during the rebuttal phase, we adjust Mover's settings to align with two leading methods MagicMotion [1] and Tora [2], denoted as Mover-Cog-23K and Mover-Cog-630K, respectively. The detailed comparison on MoverBench is shown in **Table R2** below. Under the same backbone and training data scale, Mover still outperforms these two powerful methods. We will include this table in our revised paper for clarity.\n\n**Table R2:** Comparisons of model performance under different training setups.\n\n| Method               | Backbone         | Data Scale | FID  $\\downarrow$ | FVD  $\\downarrow$ | PSNR  $\\uparrow$ | SSIM $\\uparrow$ | EPE $\\downarrow$ |\n|----------------------|:----------------:|:----------:|:-----:|:-----:|:------:|:------:|:-----:|\n| MagicMotion [1]    | CogVideoX-5B      | 23K        | 17.5  | 96.7 | 14.9  |0.56  | 3.2 |\n| Mover-Cog-23K           | CogVideoX-5B      | 23K        | **16.0** | **92.3**  | **16.8**   | **0.59**  | **2.8**   |\n|||||||||\n| Tora [2]     | CogVideoX-5B      | 630K       | 22.5  | 100.4 | 15.7  | 0.55  | 3.3  |\n| Mover-Cog-630K                    | CogVideoX-5B      | 630K       | **14.1**  | **87.3**  | **17.2**   | **0.61**  |**2.8**  |\n|||||||||\n| Mover               | Wan2.1-I2V-14B   | 2000K      | **12.2** | **83.5** | **17.8** | **0.64** | **2.6**   |\n\n****\n**References**\n\n[1] Li, et al. MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidance. In ICCV 2025.\n\n[2] Zhang, et al. Tora: Trajectory-oriented Diffusion Transformer for Video Generation. In CVPR 2025."}, {"review_id": "h9MKbwirxC", "reviewer": "Reviewer_rUr2", "summary": "The paper introduces Mover, a framework to add precise motion control to image-to-video (I2V) diffusion models without modifying their architectures. The key ideas are:\n\nLatent Trajectory Guidance: Convert dense point trajectories from pixel space into the latent feature map, then replicate first-frame features along these latent paths to inject motion without auxiliary encoders (Sec. 3.2).\n\nMoverBench: A new benchmark of 1,018 videos (5 s, 480 p) with hybrid human+SAM annotations covering 54 content categories for rigorous evaluation of motion-controllable generation (Sec. 4).\n\nExtensive quantitative (FID, FVD, PSNR/SSIM, EPE) and human studies demonstrate that Mover outperforms recent academic methods and rivals commercial tools such as Kling 1.5 Pro’s Motion Brush.", "questions": "Please according to the weakness section.", "limitations": "The author adequately addressed the limitations in the main paper.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nComprehensive benchmark (MoverBench) with precise point and mask tracks along with detailed captions, enabling standardized evaluation of diverse motion tasks.\n\nStrong empirical results: Significant improvements in motion accuracy (EPE) and video fidelity across single- and multi-object scenarios, supported by user studies with win rates exceeding 95% against top baselines.\n\n**Weakness**\n\nUsing trajectories to control video generation is not in itself a particularly novel paradigm. There are several recent works employing similar mechanisms, and in essence it may not differ substantially from using optical flow, segmentation masks, or bounding boxes for control. The primary contribution of this paper appears to be the assembly of a large, trajectory‐annotated dataset and the demonstration that scaling up training yields strong results. \n\nThat said: (1) I am not a specialist in video synthesis, and different subfields sometimes value contributions in distinct ways; and (2) some related works are currently only on arXiv and could be seen as concurrent efforts. Accordingly, I would prefer to frame this review positively and defer to other reviewers’ judgments as well.", "was_revised": true, "final_justification": "After the rebuttal, most of my concerns are resolved, thus I will keep my positive score.", "rebuttal": "Dear Reviewer rUr2,\n\nThanks for your valuable feedback. We are happy to address your comments below.\n\n**Q1: Novelty of using trajectory control for video generation.**\n\nA1: We would like to clarify that using trajectories for motion control is not our novelty. Point trajectories, optical flow, and segmentation masks are all widely-used motion control forms in prior works, as explicitly discussed in the Introduction (Lines 28–36). Our method simply adopts point trajectories as the motion signal.\n\nOur novelty lies in the feature replication mechanism, which edits the image condition features directly to enable motion control. This design removes the need for training extra motion encoders or injection modules like ControlNet, leaving the base I2V model architecture unchanged. Based on this insight, we validate the method at scale, construct a well-curated benchmark for evaluation, and will release an open-source model that even rivals Kling 1.5 Pro’s commercial Motion Brush.\nTogether, our method, scaled training, evaluation benchmark, and model performance form an integrated contribution to the community.\n\nWe are also glad to see your recognition of the overall value of our work. We hope our insights and the released model will be helpful to both the research community and video creators."}, {"review_id": "FQtpEwCafx", "reviewer": "Reviewer_wpp4", "summary": "The paper introduces Mover, a framework for precise motion control in video generative models. It leverages dense point trajectories to enable fine-grained control over scene motion. These trajectories guide the propagation of features from the first frame, generating motion-aware latent conditions. Mover integrates seamlessly into existing models without requiring architectural changes or additional encoders. Additionally, the paper presents MoverBench, a new benchmark for evaluating motion-controllable video generation. Experiments and user studies demonstrate that Mover outperforms existing methods.", "questions": "Please see weakness section.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The technical description is concise and easy to understand.\n2. The use of point trajectories as control signals makes the framework user-friendly.\n\nWeaknesses:\n1. Limited Technical Novelty: The core technique of the paper—latent feature replication along point trajectories—is conceptually similar to the widely tested pixel replication methods. Extending this idea to the latent domain offers only marginal novelty. Additionally, concatenating noisy video latents with condition inputs has also been widely adopted, such as in Wan [a]. It would be helpful if the authors could briefly clarify the distinctions between their method and related approaches like ATI [b], which also leverage VAE latent propagation.\n2. Scalability Claim Lacks Sufficient Support: The paper's claim that Mover is scalable is not convincingly substantiated. Many existing methods can also be fine-tuned with minimal architectural changes and computational overhead to achieve similar capabilities. While the paper compares inference speed with ControlNet-style branching methods, a more comprehensive evaluation is needed—including training cost across different backend models, convergence speed, and resulting model performance.\n3. Inconsistent Baseline Models: The evaluation lacks rigor due to differing backend models used across comparisons. For instance, Tora is based on CogVideoX, while Mover is tested on Wan-I2V-14B. Given the performance gap between these two models, it is difficult to draw fair conclusions about Mover's advantages solely from these comparisons.\n4. It remains unclear how Mover handles ambiguous control signals. Specifically, when a control point and its associated trajectory are provided, it is not evident how Mover distinguishes whether the signal is intended to control local object deformation, global object motion (translation or rotation), camera movement, or background object motion. Further clarification on this aspect would strengthen the paper.\n\n[a] Wan, Team, et al. \"Wan: Open and advanced large-scale video generative models.\" arXiv preprint arXiv:2503.20314 (2025).\n\n[b] Wang, Angtian, et al. \"ATI: Any Trajectory Instruction for Controllable Video Generation.\" arXiv preprint arXiv:2505.22944 (2025).", "was_revised": true, "final_justification": "I have carefully reviewed the authors' response as well as the comments from the other reviewers. Most of my concerns have been addressed. Although I still feel that the technical contribution of the paper could be further improved, I will increase my score to borderline accept.", "rebuttal": "Dear Reviewer wpp4,\n\nThanks for the constructive comments. Please find our response to the specific points below.\n\n\n**Q1: Latent feature replication is conceptually similar to the pixel replication methods.**\n\nA1: We would like to clarify that latent feature replication is non-trivial. Compared to pixel-level replication, it offers two advantages crucial for overall performance. **(i)** Pixel-replicated videos produces out-of-distribution visual inputs for pretrained VAEs, typically requiring auxiliary encoder networks for enhanced performance (e.g., Tora [1], discussed in Lines 93-97). This complicates the fine-tuning of video-generation backbones, which contrasts with our core insight. **(ii)** Individual pixels lack semantic and contextual information (referred in Lines 34-35). Directly copying them hinders motion pattern alignment across surrounding regions. In contrast, latent features carry richer contextual cues, enabling more coherent motion conditions.\n\nThese advantages were rigorously validated in Table 4 and Figure 8 of the main paper, both quantitative and qualitatively. Specifically, our 'latent feature replication' approach demonstrates significant performance gains over the 'pixel replication', lowering FVD by 7.5.\n\n\n\n**Q2: Concatenating noisy video latents with condition inputs has been widely adopted such as in Wan.**\n\nA2: Yes, as clearly stated in Lines 135–138, this operation is used in prior I2V backbones such as Wan [2] and Lumiere [3] for fusing image conditions. Because this strategy is already integrated into the backbone's design, we follow the same approach to maintain compatibility and avoid modifying the model architecture. We also provide an ablation in Table 5, which confirms that the concatenation strategy works well for our task.\n\n**Q3: Clarify the distinctions between Mover and ATI.**\n\nA3: We note that ATI [4] (arXiv: May 28) was submitted after our NeurIPS 2025 submission deadline (May 15). Our work was developed independently.\n\n**Q4: Scalability claim needs more sufficient support.**\n\nA4: Thanks for your suggestion. Mover’s scalability mainly stems from preserving the base I2V model architecture. To this end, we **(i)** simplify motion control extraction by eliminating trajectory encoders like Tora’s [1], and **(ii)** avoid ControlNet-style fusion module that adds a new branch like MagicMotion’s [5]. To address your concern, we perform ablation studies to evaluate how these two designs facilitate scalability. The first compares Mover with a variant that adds a trainable motion encoder (denoted as Mover-Motion_Encoder), the second compares Mover with its ControlNet-style variant (denoted as Mover-ControlNet). Experiments are extensively conducted on two backbones, using the same data for fairness.\n\nAs shown in **Table R3**, our method achieves both better performance and faster convergence compared to alternatives. Mover-Motion_Encoder increases optimization complexity and degrades performance, Mover-ControlNet converges more slowly, possibly due to using zero convolutions, as also observed in ControlNext [6]. We are happy to incorporate the tables to better clarify our scalability.\n\n**Table R3:** Scalability comparison of different approaches on two backbones.\n\n| Approach                     | Backbone       | Convergence Step $\\downarrow$  | FVD $\\downarrow$  | FID $\\downarrow$ | EPE $\\downarrow$  |\n|-----------------------------|:--------------:|:-------------------:|:------:|:------:|:------:|\n| Mover-Motion_Encoder   | Wan2.1-I2V-14B | 33000 | 13.5 | 87.8 | 2.7    |\n| Mover-ControlNet   | Wan2.1-I2V-14B | 36000  | 12.4   | 84.6   | **2.5**    | \n| Mover      |Wan2.1-I2V-14B  | **29500** | **12.2** | **83.5** | 2.6   |\n|||||||\n| Mover-Motion_Encoder   | CogVideoX-5B| 26500 | 14.5 | 88.2 | 3.0 |\n| Mover-ControlNet   | CogVideoX-5B| 29500  | **14.1** | 87.0   | 2.8 |\n| Mover      | CogVideoX-5B  |  **23000** | 14.2  | **86.6**   | **2.8** |\n\n\n*Setting illustration:* In **Table R3**, Mover-Motion_Encoder refers to that we add convolutional layers (following 'Trajectory Extractor' in Tora) after the feature replication step to further encode the motion condition. We define convergence as the point where evaluation metrics stabilize within a small range.\n\n**Q5: Mover and baseline methods use different backbones.**\n\nA5: We choose Wan as the backbone to pursue performance upper bound among open-source models. Notably, we can compare our performance with commercial model (Kling 1.5 Pro). To address your concern, we adjust Mover's backbone and training data scale to match those of two leading methods, MagicMotion and Tora. The comparisons under aligned settings are shown in **Table R4**. Under the same backbone and training data scale, Mover still outperforms these powerful methods. We are happy to include this table in our revised paper for clarity.\n\n**Table R4:** Comparison of model performance under different training setups. Mover-Cog-23K and  Mover-Cog-630K denote Mover variants under different backbones and training data scales.\n\n| Method               | Backbone         | Data Scale | FID  $\\downarrow$ | FVD  $\\downarrow$ | PSNR  $\\uparrow$ | SSIM $\\uparrow$ | EPE $\\downarrow$ |\n|----------------------|:----------------:|:----------:|:-----:|:-----:|:------:|:------:|:-----:|\n| MagicMotion [5]    | CogVideoX-5B      | 23K        | 17.5  | 96.7 | 14.9  |0.56  | 3.2 |\n| Mover-Cog-23K           | CogVideoX-5B      | 23K        | **16.0** | **92.3**  | **16.8**   | **0.59**  | **2.8**   |\n|||||||||\n| Tora [1]     | CogVideoX-5B      | 630K       | 22.5  | 100.4 | 15.7  | 0.55  | 3.3  |\n| Mover-Cog-630K                    | CogVideoX-5B      | 630K       | **14.1**  | **87.3** | **17.2**   | **0.61**  | **2.8**  |\n|||||||||\n| Mover               | Wan2.1-I2V-14B   | 2000K      | **12.2** | **83.5** | **17.8** | **0.64** | **2.6**   |\n\n\n**Q6: How does Mover handle ambiguous control signals?**\n\nA6: Mover naturally addresses the ambiguity issue through two key mechanisms. **(i)** During inference, users can easily clarify intended motion by specifying more detailed point trajectories. For example, placing multiple points on an object with similar motion suggests global movement, while adding stationary points nearby helps indicate local movement. These adjustments can be made directly through the user interface. \n\n**(ii)** Text prompt provides complementary motion guidance, further resolving potential ambiguities in control signals. During training, text prompts clearly describe the moving foreground subject and background context, paired with randomly sampled point trajectories. With large-scale training on diverse aligned samples, even given a single point trajectory, the model can rely on text prompts to infer the desired motion.\n\n\n****\n**References**\n\n[1] Zhang, et al. Tora: Trajectory-oriented Diffusion Transformer for Video Generation. In CVPR 2025.\n\n[2] Wan Team, et al. Wan: Open and Advanced Large-Scale Video Generative Models. arXiv preprint arXiv:2503.20314 (2025).\n\n[3] Bar-Tal, et al. Lumiere: A Space-Time Diffusion Model for Video Generation. In SIGGRAPH Asia 2024 Conference.\n\n[4] Wang, et al. ATI: Any Trajectory Instruction for Controllable Video Generation. arXiv preprint arXiv:2505.22944 (2025).\n\n[5] Li, et al. MagicMotion: Controllable Video Generation with Dense-to-Sparse\nTrajectory Guidance. In ICCV 2025.\n\n[6] Peng, et al. ControlNeXt: Powerful and Efficient Control for Image and Video Generation. arXiv preprint arXiv:2408.06070 (2024)."}, {"review_id": "Rmj4lOXolX", "reviewer": "Reviewer_TUep", "summary": "The paper proposes Mover, a method for repurposing a pre-trained video generation model to enable motion-controllable generation. Mover achieves this by conditioning the pre-trained backbone on a sequence of latent frames constructed as follows: the first frame contains the complete latent representation of the first image in the video, while the remaining latent frames are initially zeroed out and then partially filled with features from the first frame along sparsely sampled ground truth motion trajectories. At test time, this allows the model to condition the generation on a sparse set of motion trajectories. Extensive experiments show that Mover outperforms previous methods in terms of video quality, motion accuracy, and human preference. In addition, the authors introduce MoverBench, a curated benchmark designed specifically for evaluating motion-controllable video generation.", "questions": "1) Could the authors include a discussion on how the control in Mover differs from [1]? The baselines in the ablations seem too weak. Have the authors tried using external visual encoders instead of the native encoder or random track embeddings?\n\n2) Could the authors discuss how the magnitude of the control affects the accuracy of the model following the specified trajectories?\n\n3) The control signal is applied at a low-resolution feature level. Could the authors provide insight into how this impacts fine-grained control, especially for small or subtle motions?", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths \n1) The paper is well-written, easy to follow, and presents the method clearly.\n2) It includes sufficient implementation details to support reproducibility.\n3) The experiments are comprehensive, and the quantitative results are convincing.\n\n## Weaknesses\n1) The core idea of conditioning video generation on sparse moving visual features is not novel. Prior works [1, 2] have employed similar approaches to condition the generation on motion-trajectories. Notably, these methods leverage external feature encoders that are robust to small appearance changes, eliminating the need for ground truth trajectories and allowing flexibility during inference. Although Mover demonstrates improved controllability and applies the technique to a general-purpose video model with stronger quantitative results, a thorough discussion of these related works would help contextualize the contribution.\n2) The qualitative results are less convincing. The magnitude of motion control appears limited, and the number of qualitative examples is small, raising questions about the method's performance under larger and more out of distribution control inputs.\n3) One limitation of the proposed control format is its low resolution. Although the tracks themselves can be of high-resolution, they are applied to low-resolution features. This may restrict the granularity of the proposed control. \n\n[1] Hassan et. al, Gem: A generalizable ego-vision multimodal world model for fine-grained ego-motion, object dynamics, and scene composition control. CVPR 25\n\n[2] Davtyan et. al, CAGE: Unsupervised Visual Composition and Animation for Controllable Video Generation. AAAI 25", "was_revised": true, "final_justification": "The authors did a good job during the rebuttal and addressed most of my concerns. The proposed method to cast a pretrained video model to a trajectory-controlled one is simple, clearly presented, and demonstrates good empirical results.", "rebuttal": "Dear Reviewer TUep,\n\nThank you for the constructive comments. We are glad that you found our implementation details sufficient for reproducibility. We will also open-source our code and model.\n\n**Q1: A thorough discussion of related works [1,2] would help contextualize the contribution.**\n\nA1: Thanks for your constructive feedback. We are happy to clarify their differences and incorporate this discussion into the revised Related Work section.\n\nWorks [1,2] and Mover are simlar in that they condition the generation on latent motion trajectories. However, they differ in the methodology and task. **(i) Methodology difference:** Works [1,2] rely on pretrained DINOv2 features to transfer object representations across frames, yet both papers highlight DINOv2’s limitations in video generation. First, DINOv2 excels at high-level semantic encoding but lacks fine-grained object details. Thus, GEM [1] employs additional identity embeddings to distinguish objects and train an ObjectNet to bridge the domain gap between DINOv2 and the\nUNet’s feature space. Second, CAGE [2] observes that DINOv2 features inherently retain spatial positioning. This makes it hard to disentangle object identity from the original location, finally reducing controllability. Moreover, the 14 $\\times$ 14 patch size in DINOv2 may restrict the granularity of the proposed control. In contrast, our approach mitigates these limitations without relying on any auxiliary modules (e.g., identity embeddings). **(ii) Task difference:** As you mentioned, we focus on general-purpose video generation, supporting a significantly broader range of motion control tasks compared to [1-2].\n\n**Q2: The qualitative results contain a limited number of large-motion examples, raising questions about the model performance under larger and more out-of-distribution motion conditions.**\n\nA2: Our visualization results include a number of large-motion cases, such as boxing (supplementary video: 10–15s) and aerobics (47–52s). The supplementary video also shows that Mover performs noticeably better than compared methods in high-amplitude motion scenes (47s-1:11). This capability is supported by our data and benchmark construction process, where we specifically retained a wide range of large-motion cases using expert models and motion-based filtering (Lines 165–170 and 204–207).\n\nTo further address your concern, we curate subsets from MoverBench containing high-amplitude and out-of-distribution motion control cases. Evaluation results on these challenging examples are shown in **Tables R5** and **R6**. Notably, Mover consistently outperforms two leading baselines [3,4], with performance gaps further widening under these difficult condition. In addition, Mover’s performance only marginally drops compared to its results on the full benchmark, demonstrating its robustness. Due to rebuttal guidelines, we cannot include additional visual examples here, but we will provide more such cases in our upcoming project page and include these two tables for clarity.\n\n**Table R5:** Comparisons on MoverBench large-motion subset. For each video, its motion amplitude score is computed as the average of the top 2% largest optical flow values extracted by RAFT [5]. The top 20% highest-score videos are selected.\n| Method     | FID  $\\downarrow$ | FVD  $\\downarrow$ | EPE  $\\downarrow$ |\n|------------------|:-------:|:-------:|:-------:|\n| Tora [3]  | 29.1  | 126.3   | 4.3     |\n| MagicMotion [4]   | 24.6   | 119.3  | 4.1    |\n| Mover  | **14.5**   | **86.6**   | **3.0**    |\n\n**Table R6:** Comparisons on MoverBench out-of-distribution motion subset. We manually curated 50 uncommon motion cases, including complex foreground–background interactions, objects moving out of frame, and rare camera motions.\n| Method    | FID  $\\downarrow$ | FVD  $\\downarrow$ | EPE  $\\downarrow$ |\n|------------------|:-------:|:-------:|:-------:|\n| Tora [3]  | 28.9   | 120.2   | 4.0    |\n| MagicMotion [4]   | 23.5   | 115.7   | 3.9    |\n| Mover   | **13.5**   | **86.0**   | **2.8**  |\n\n\n**Q3: One limitation of the proposed control format is its low resolution. Could the authors provide insight into how this impacts fine-grained control?**\n\nA3: We use a VAE encoder to compress motion trajectories with a spatial downsampling ratio of 8, which is a relatively low compression. It means that when the trajectory displacement exceeds approximately 8 pixels, the model can theoretically distinguish them. Even when the motion amplitude is very small (a particularly demanding case), the model may still infer the intended motion by leveraging the text prompt (e.g., \"slightly move to the left\") and the contextual motion patterns from nearby regions. \n\nTo further address your concern, we conduct a targeted evaluation. We sample 100 videos from MoverBench and, for each video, select one point trajectory with total displacement falling into one of four ranges: <8 pixels, 8–16 pixels, 16–32 pixels, and >32 pixels. We perform motion control using the selected trajectories and evaluate the control precision by end-point error (EPE). As shown in **Table R7**, Mover maintains reasonable control accuracy even for very small motions, demonstrating fine-grained motion control capability.\n\n**Table R7:** Comparisons of motion control performance at varying motion amplitudes.\n| Trajectory displacement |  <8 pixels | 8–16 pixels | 16–32 pixels | >32 pixels |\n|------------------|:-------:|:-------:|:-------:|:-------:|\n| **EPE**   | 3.0   |  2.6  | 2.5    |2.5    |\n\nMoreover, there are two potential solutions to further alleviate the resolution limitation. **(i)** We have already trained Mover based on the I2V-14B-720P backbone, increasing the video resolution from 480P to 720P and enabling finer motion granularity. We are happy to release this model. **(ii)** Another direction is to incorporate explicit pixel-level position embedding into our feature replication mechanism. We leave this as a promising future work.\n\n**Q4: Have the authors tried using external visual encoders instead of the native encoder or random track embedding?**\n\nA4: Thanks for the suggestion. During the rebuttal phase, we evaluate two external visual encoder designs. **(i)** Mover-Motion_Encoder: we add learnable convolutional layers (like Tora’s 'Trajectory Extractor') after the feature replication step to further encode the motion condition. **(ii)** Mover-DINOv2: We extract features from a pretrained DINOv2 model and resize them to match the latent feature map for conditioning. As shown in **Table R8**, both variants underperform compared to our approach. Mover-Motion_Encoder increases optimization complexity and degrades performance. Mover-DINOv2 yields the weakest results. It aligns with prior findings (see Q1 & A1) that DINOv2 features, while semantically rich, introduce limitations for motion-controllable video generation. We are happy to incorporate this experiment for a more complete ablation in the revised paper.\n\n**Table R8:** Comparisons of visual encoders on MoverBench.\n| Approach    | FID  $\\downarrow$ | FVD  $\\downarrow$ | EPE  $\\downarrow$ |\n|------------------|:-------:|:-------:|:-------:|\n| Mover-DINOv2   | 16.6   | 97.2   | 3.5    |\n| Mover-Motion_Encoder       | 14.5   | 88.2   | 3.0   |\n| Mover (full fine-tuning)   | **12.2**   | **83.5**   | **2.6**    |\n\n****\n**References**\n\n[1] Hassan, et al. Gem: A Generalizable Ego-vision Multimodal World Model for Fine-grained Ego-motion, Object Dynamics, and Scene Composition Control. In CVPR 2025.\n\n[2] Davtyan, et al. CAGE: Unsupervised Visual Composition and Animation for Controllable Video Generation. In AAAI 2025.\n\n[3] Zhang, et al. Tora: Trajectory-oriented Diffusion Transformer for Video Generation. In CVPR 2025.\n\n[4] Li, et al. MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidance. In ICCV 2025.\n\n[5] Teed, et al. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. In ECCV 2020."}]

Author rebuttal:

Meta-review: The paper introduces Mover, a framework for precise motion control in video generative models. It leverages dense point trajectories to enable fine-grained control over scene motion. The work is further supported by the release of MoverBench, a large-scale benchmark with precise annotations that is likely to benefit the community.

Reviewers consistently acknowledge the strengths of the paper: the method is simple yet effective, the benchmark is valuable, and empirical results show strong performance. Two reviewers gave clear accept ratings.

The main concerns raised relate to limited novelty, fairness of comparisons due to stronger base models and larger training data, and insufficient discussion of scalability and relation to recent concurrent works. However, the rebuttal addressed many of these issues, and reviewers increased or maintained their positive stance.

All reviewers reached a consensus to accept the paper.
Decision: Accept (poster)