Title: Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance
Venue: NeurIPS 2025
Accepted: yes
Keywords: Video generation

Motion-controllable video generation

Diffusion models
Abstract: We present Wan-Move, a simple and scalable framework that brings motion control to video generative models. Existing motion-controllable methods typically suffer from coarse control granularity and limited scalability, leaving their outputs insufficient for practical use. We narrow this gap by achieving precise and high-quality motion control. Our core idea is to directly make the original condition features motion-aware for guiding video synthesis. To this end, we first represent object motions with dense point trajectories, allowing fine-grained control over the scene. We then project these trajectories into latent space and propagate the first frame's features along each trajectory, producing an aligned spatiotemporal feature map that tells how each scene element should move. This feature map serves as the updated latent condition, which is naturally integrated into the off-the-shelf image-to-video model, e.g., Wan-I2V-14B, as motion guidance without any architecture change. It removes the need for auxiliary motion encoders and makes fine-tuning base models easily scalable. Through scaled training, Wan-Move generates 5-second, 480p videos whose motion controllability rivals Kling 1.5 Pro's commercial Motion Brush, as indicated by user studies. To support comprehensive evaluation, we further design MoveBench, a rigorously curated benchmark featuring diverse content categories and hybrid-verified annotations. It is distinguished by larger data volume, longer video durations, and high-quality motion annotations. Extensive experiments on MoveBench and the public dataset consistently show Wan-Move's superior motion quality. Code, models, and benchmark data are made available.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Mover, a method for repurposing a pre-trained video generation model to enable motion-controllable generation. Mover achieves this by conditioning the pre-trained backbone on a sequence of latent frames constructed as follows: the first frame contains the complete latent representation of the first image in the video, while the remaining latent frames are initially zeroed out and then partially filled with features from the first frame along sparsely sampled ground truth motion trajectories. At test time, this allows the model to condition the generation on a sparse set of motion trajectories. Extensive experiments show that Mover outperforms previous methods in terms of video quality, motion accuracy, and human preference. In addition, the authors introduce MoverBench, a curated benchmark designed specifically for evaluating motion-controllable video generation.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths \n1) The paper is well-written, easy to follow, and presents the method clearly.\n2) It includes sufficient implementation details to support reproducibility.\n3) The experiments are comprehensive, and the quantitative results are convincing.\n\n## Weaknesses\n1) The core idea of conditioning video generation on sparse moving visual features is not novel. Prior works [1, 2] have employed similar approaches to condition the generation on motion-trajectories. Notably, these methods leverage external feature encoders that are robust to small appearance changes, eliminating the need for ground truth trajectories and allowing flexibility during inference. Although Mover demonstrates improved controllability and applies the technique to a general-purpose video model with stronger quantitative results, a thorough discussion of these related works would help contextualize the contribution.\n2) The qualitative results are less convincing. The magnitude of motion control appears limited, and the number of qualitative examples is small, raising questions about the method's performance under larger and more out of distribution control inputs.\n3) One limitation of the proposed control format is its low resolution. Although the tracks themselves can be of high-resolution, they are applied to low-resolution features. This may restrict the granularity of the proposed control. \n\n[1] Hassan et. al, Gem: A generalizable ego-vision multimodal world model for fine-grained ego-motion, object dynamics, and scene composition control. CVPR 25\n\n[2] Davtyan et. al, CAGE: Unsupervised Visual Composition and Animation for Controllable Video Generation. AAAI 25\n\nQUESTIONS:\n1) Could the authors include a discussion on how the control in Mover differs from [1]? The baselines in the ablations seem too weak. Have the authors tried using external visual encoders instead of the native encoder or random track embeddings?\n\n2) Could the authors discuss how the magnitude of the control affects the accuracy of the model following the specified trajectories?\n\n3) The control signal is applied at a low-resolution feature level. Could the authors provide insight into how this impacts fine-grained control, especially for small or subtle motions?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Mover, a framework for precise motion control in video generative models. It leverages dense point trajectories to enable fine-grained control over scene motion. These trajectories guide the propagation of features from the first frame, generating motion-aware latent conditions. Mover integrates seamlessly into existing models without requiring architectural changes or additional encoders. Additionally, the paper presents MoverBench, a new benchmark for evaluating motion-controllable video generation. Experiments and user studies demonstrate that Mover outperforms existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The technical description is concise and easy to understand.\n2. The use of point trajectories as control signals makes the framework user-friendly.\n\nWeaknesses:\n1. Limited Technical Novelty: The core technique of the paper—latent feature replication along point trajectories—is conceptually similar to the widely tested pixel replication methods. Extending this idea to the latent domain offers only marginal novelty. Additionally, concatenating noisy video latents with condition inputs has also been widely adopted, such as in Wan [a]. It would be helpful if the authors could briefly clarify the distinctions between their method and related approaches like ATI [b], which also leverage VAE latent propagation.\n2. Scalability Claim Lacks Sufficient Support: The paper's claim that Mover is scalable is not convincingly substantiated. Many existing methods can also be fine-tuned with minimal architectural changes and computational overhead to achieve similar capabilities. While the paper compares inference speed with ControlNet-style branching methods, a more comprehensive evaluation is needed—including training cost across different backend models, convergence speed, and resulting model performance.\n3. Inconsistent Baseline Models: The evaluation lacks rigor due to differing backend models used across comparisons. For instance, Tora is based on CogVideoX, while Mover is tested on Wan-I2V-14B. Given the performance gap between these two models, it is difficult to draw fair conclusions about Mover's advantages solely from these comparisons.\n4. It remains unclear how Mover handles ambiguous control signals. Specifically, when a control point and its associated trajectory are provided, it is not evident how Mover distinguishes whether the signal is intended to control local object deformation, global object motion (translation or rotation), camera movement, or background object motion. Further clarification on this aspect would strengthen the paper.\n\n[a] Wan, Team, et al. \"Wan: Open and advanced large-scale video generative models.\" arXiv preprint arXiv:2503.20314 (2025).\n\n[b] Wang, Angtian, et al. \"ATI: Any Trajectory Instruction for Controllable Video Generation.\" arXiv preprint arXiv:2505.22944 (2025).\n\nQUESTIONS:\nPlease see weakness section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Mover, a framework to add precise motion control to image-to-video (I2V) diffusion models without modifying their architectures. The key ideas are:\n\nLatent Trajectory Guidance: Convert dense point trajectories from pixel space into the latent feature map, then replicate first-frame features along these latent paths to inject motion without auxiliary encoders (Sec. 3.2).\n\nMoverBench: A new benchmark of 1,018 videos (5 s, 480 p) with hybrid human+SAM annotations covering 54 content categories for rigorous evaluation of motion-controllable generation (Sec. 4).\n\nExtensive quantitative (FID, FVD, PSNR/SSIM, EPE) and human studies demonstrate that Mover outperforms recent academic methods and rivals commercial tools such as Kling 1.5 Pro’s Motion Brush.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nComprehensive benchmark (MoverBench) with precise point and mask tracks along with detailed captions, enabling standardized evaluation of diverse motion tasks.\n\nStrong empirical results: Significant improvements in motion accuracy (EPE) and video fidelity across single- and multi-object scenarios, supported by user studies with win rates exceeding 95% against top baselines.\n\n**Weakness**\n\nUsing trajectories to control video generation is not in itself a particularly novel paradigm. There are several recent works employing similar mechanisms, and in essence it may not differ substantially from using optical flow, segmentation masks, or bounding boxes for control. The primary contribution of this paper appears to be the assembly of a large, trajectory‐annotated dataset and the demonstration that scaling up training yields strong results. \n\nThat said: (1) I am not a specialist in video synthesis, and different subfields sometimes value contributions in distinct ways; and (2) some related works are currently only on arXiv and could be seen as concurrent efforts. Accordingly, I would prefer to frame this review positively and defer to other reviewers’ judgments as well.\n\nQUESTIONS:\nPlease according to the weakness section.\n\nLIMITATIONS:\nThe author adequately addressed the limitations in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a representation for trajectory conditions and achieving such guidance in image-to-video generation. The representation basically transforms a sequence of points into the latent space and then uses the new representation to create feature maps for subsequent frames after the 1st frame. These condition features are concatenated with the noisy patches as input to the DiT backbone without having the need to modify the overall architecture or introduce new tunable modules. The authors curated a large-scale dataset for training and also a benchmark called MoverBench for evaluation. Experimental results show that Mover outperforms many previous baselines.\n\nSTRENGTHS AND WEAKNESSES:\n(+) The paper comes up with a very smart way to represent trajectories in the latent space and then uses these representations to create conditional features to achieve motion control. This eliminates the need for an additional control network or a new tunable module, making the entire model more compact and unified. \n\n(+) A large-scale benchmark is introduced, which seems of high quality and could benefit the community by providing a standard benchmark beyond DAVIS. \n\n(+) Experimental results suggest superior control achieved by Mover compared to previous baselines. It also works well with multiple trajectories. \n\n(-) While Mover saves from using an additional module, it also has the drawback of changing the whole parameter of the original DiT backbone. But with ControlNet, one gets to freeze the backbone and tune half of the parameters and use multiple trained ControlNets in a plug-and-play manner. \n\n(-) The evaluation comparison is not fair. Mover has used a lot more high-quality videos compared to all the baselines and potentially a stronger base model (Wan-14B v.s. e.g., CogVideoX in baselines). Of course, it's never easy for this type of work to make a fair comparison due to all kinds of reasons and limitations. For example, the most comparable model is Motion Prompting, which trains a naive ControlNet with a similar number of videos as Mover -- but it is not open-sourced. However, the work needs to clarify the differences between Mover and the baselines in terms of base models and the number of training videos.\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nSee weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}