Time-to-Move: Training-Free Motion-Controlled Video Generation via Dual-Clock Denoising
Authors: Assaf Singer
Noam Rotstein
Amir Mann
Ron Kimmel
Or Litany
Venue: ICLR 2026 Poster (2026)
Keywords: Computer vision
Generative models
Primary area: generative models
TL;DR: We introduce Time-to-Move (TTM), a training-free, plug-and-play framework for motion- and appearance-controlled video generation with image-to-video (I2V) diffusion models.

Abstract: Diffusion-based video generation can create realistic videos, yet existing image- and text-based conditioning fails to offer precise motion control. Prior methods for motion-conditioned synthesis typically require model-specific fine-tuning, which is computationally expensive and restrictive. We introduce Time-to-Move (TTM), a training-free, plug-and-play framework for motion- and appearance-controlled video generation with image-to-video (I2V) diffusion models. Our key insight is to use crude reference animations obtained through user-friendly manipulations such as cut-and-drag or depth-based reprojection. Motivated by SDEdit’s use of coarse layout cues for image editing, we treat the crude animations as coarse motion cues and adapt the mechanism to the video domain. We preserve appearance with image conditioning and introduce dual-clock denoising, a region-dependent strategy that enforces strong alignment in motion-specified regions while allowing flexibility elsewhere, balancing fidelity to user intent with natural dynamics. This lightweight modification of the sampling process incurs no additional training or runtime cost and is compatible with any backbone. Extensive experiments on object and camera motion benchmarks show that TTM matches or exceeds existing training-based baselines in realism and motion control. Beyond this, TTM introduces a unique capability: precise appearance control through pixel-level conditioning, exceeding the limits of text-only prompting. Visit our project page (https://time-to-move.github.io) for video examples and code.

Published as a conference paper at ICLR 2026
TO-E-E- TRAINING-FREE MOTION CON- TROLLED VIDEO GENERATION VIA DUAL-CLOCK DE-
NOISING
Assaf Singer!*
Noam Rotstein
Amir Mann
Ron Kimmel
Or Litany1.2
Technion Israel Institute of Technology
NVIDIA
Equal contribution
Local Object Motion Control
Camera Motion Control
Joint Motion Appearance Control
Figure 1: Qualitative results of Time-to-Move on various tasks.
ABSTRACT
Diffusion-based video generation can create realistic videos, yet existing image- and text-based conditioning fails offer precise motion control. Prior methods for motion-conditioned synthesis typically require model-specific fine-tuning, which is computationally expensive and restrictive. We introduce Time-to-Move (TTM), training-fire free, plug-and-play framework for motion- and appearance-controlled video generation with image-to-video (12V) diffusion models. Our key insight is use crude reference animations obtained through sser-friendly manipulations such as cut-and-drag or depth-based reprojection. Motivated by SDEdit's 's use of coarse layout cues for image editing, we treat the crude animations coarse mo- tion cues and adapt the mechanism to the video domain. We preserve appearance with image conditioning and introduce dual-clock denoising. region-dependen strategy that enforces strong alignment in motion- motion-specified regions while allowing flexibility elsewhere, balancing fidelity to user intent with natural dynamics. This lightweight modification of the sampling process incurs no additional training runtime cost and compatible with any backbone. Extensive experiments on ob- ject and camera motion benchmarks show that TTM matches exceeds existing training- based baselines in realism and motion control. Beyond this, TTM intro- duces unique capability: precise appearance control through pixel-level condi- tioning, exceeding the limits of text-only prompting. Visit our project pagel for video examples and code.

---

Published as a conference paper at ICLR 2026
1 INTRODUCTION
Diffusion- video generators have recently achieved remarkable visual quality, yet their con- trollability remains limited. Image-to-video (12V) models partially alleviate this limitation by con- ditioning a single input frame, which gives users direct control over the appearance of the gen- erated video. However, motion control remains largely prompt-driven which is often unreliable, coarse, and insufficiently fine-gratd fine-grained for interactive use. To address this gap, a practical generative video system should provide an interface that defines both what moves and where it moves, ensur- ing realistic, temporally coherent motion while preserving the appearance of the input image. Such fine-grained control would enable interactive content authoring, post-production, and animation pro- totyping, where creators require precise, local adjustments with fast feedback. Existing approaches for controllable motion in generation typically encode user intent through auxiliary control signals such as optical flow or point-trajectorie and then heavily fine-tune generator to ingest this motion conditioning (Burgert et al. 2025) Geng et al., 2025). Such methods are computationally expensive to train, often compromise the quality of the original model, and remain specif,,,,,,,e, requir- ing architectural modifications to incorporate the controls. This motivates a framework that can be applied to off-the-shelf video diffusion backhones without expensive tuning or additional data.
We introduce Time-to-Move (TTM), training-free, architecture-agnoste plug-and-play inference procedure for video diffusion models that matches the speed of standard generation. We observe that crude animation inputs, such as those created by simple cut-and-drag manipulation or by straight- forward reprojection of the image into novel views using estimated monocular depth, can serve as a useful proxy for the intended target. Such references capture coarse structure and convey the desired motion, while remaining easy to produce and flexible enough to be made as specific or detailed as the user intends. To transform these crude signals into realistic videos, draw on SDEdit (Meng) let al., (021), which shows that coarse structure can be imposed by adding noise to the timestep where the layout is determined. By analogy, we hypothesize that noising the synthetic reference video to the point where motion is established by the video diffusion model can embed the intended dynamics. Indeed, this strategy successfully injects motion, but at this noise level fidelity to the reference appearance is lost. To mitigate this, we turn to image-conditions video diffusion models, which preserve the identity and scene details of the initial frame and thereby maintain appearance consistency throughout the generated sequence.
Even with appearance preserved, motion cues remain uneven across regions. the synthetic guiding reference video, some regions, such as dragged objects, may contain strong user-specified dynam- ics, while others remain unspecified. These unconstrained regions are not meant to stay static, but rather to adapt naturally in support of the intended movement. To this end, we introduce novel region-depended dual-clock denoising process, which assigns one of two distinct SDE timesteps to different regions across frames: strong alignment for specified motion and weaker alignment for unconstrained areas, allowing spatially varying conditioning strength. To realize this effect without retraining the model, we employ simple yet effective diffusion blending strategy akin to (Avrahamil let al. 2022) Lugmayr et al. 2022). This allows the model to adhere closely to a specified motion where it exists while allowing greater freedom to invent plausible dynamics elsewhere.
Unlike prior approaches that rely solely on (either sparse or dense) displacement fields as a guiding signal, our method is conditioned directly on the reference video itself. This provides richer super- visory signal: In regions where alignment is strongly enforced, we not only constrain motion, but can also dictate appearance attributes such as color, shape, or style. As a result, TTM enables joint control of motion and appearance, extending the conditioning space beyond motion-only interfaces. We exploit this capability to support appearance- senaaiiieeeeeevveeeeeeeeeeeeeeee prompting in tandem with motion control, for example, animating an object along a user-specified trajectory while simultaneously changing its color (See Fig. 7. In summary, our work makes the following contributions:
Training-free motion control using crude animations. We show that simple user- user-provice per-provided ani- mations (e.g., cut-and-drag manipulations or depth- based reprojections) serve as effective motion proxies. Adapting SDEdit-style noise injection to video diffusion and anchoring appearance with image conditioning converts these coarse inputs into realistic motion without training.
Region-dependent dual-clock denoising. We propose denoising process that operates with two distinct noise schedules-strong alignment in motion-speciffed regions and weaker alignment elsewhere enabling spatially varying conditioning without retraining.
2

---

Published as a conference paper at ICLR 2026
Joint motion-appearance control. Conditioning full reference frames, rather than mo- tion trajectories alone, enables simultaneous control of both motion and appearance, a capability previously limited to ambiguous text prompts.
Extensive experiments show that TTM consistently ranks among the best performing methods on both object and camera motion benchmarks, outerforming even training- baselines. The ap- proach is training-fite ffree and plug-and-play validated across three 12V backbones, and matches the efficiency of standard video sampling.
2 RELATED WORK
Video generation has recently shown great progress, achieving high visual quality and temporal coherence (Blattmann et al., 2023 Hong et al. 2023 Yang et al. (2025) and demonstrating utility across diverse tasks (Rotstein et al. 2025 Wiedemer et al. 2025] Voleti et al., 2024). However, re-
lying solely on a prompt or initial frame provides limited control, particularly for specifying motion.
Learning motion control in video generation. A common strategy in motion control learn a cenettoo-cooniiioneed representation and fuse it throughout the network. Methods inject user trajec- tories via multi-scale fusion of trajectory maps U-Net blocks (Yin et al. (223), parameter-efficient LoRA modules that decouple camera and object motion (L.i et al.) 2025b), or motion-patch tokens integrated across transformer blocks (Zhang et al. 2025a): ATI encodes point tracks as Gaussian- weighted latent features (Wang et al. 2025), TrackGo inserts auxiliary branches into SVD's tem- poral self-attention (Zhou et al. 2025 Blattmann et al. 2023), and MotionPro uses region-wise trajectories plus motion mask to distinguish object vs. camera motion (Zhang al., 2025b). Other methods, like ours, incorporate explicit motion-basee cues rather than relying solely on learned in- jection. Drag Anything extracts entity representations from first-frame diffusion features and injects trajectory conditioning via a ControlNet-style branch (Zhang et al.) 2023). Motion Prompting con- ditions a video diffusion model on point-track "motion prompts " via a track-conditioned ControlNet Geng et al. 2025). For camera control, GEN3C (Ren et al. 2025) constructs depth-lifted 3D cache and, similar to our approach, conditions generation on projected renders along the target cam- era path. Go-with-the-Flow (Burgert et al. (2025) warps diffusion noise to align with the intended motion. While this shares our sechitecture-agnosii goal, unlike our method and the approaches above, it extensively fine-tunes the base model to leverage the input conditioning.
Training-free Motion-controllabbe video generation. Several approaches avoid additional train- ing by reusing pretrained models. Recent text-to-video (T2V) approaches manipulate attention to control motion: TrailBlazer modifies spatial and temporal attention early in denoising (Ma et al. 2024), PEEKABOO gates regions using masked spatio-temporal attention (Jain et al. (024). and
FreeTraj shapes low-frequency noise with attention biases to follow bounding boxes (Qiu et al.
2024). However, these models tie bounding boxes to text, limiting part-level motion and preventing precise appearance control or in in-place animation of a given image. Video-MSG (Li et al. 2025a) is training-free backbone-agnostie guidance method: it uses an MLLM video sketch, inverts it to initialize a T2V model's noise, then denoises to the final video. Unlike TTM, it targets T2V, requires inversion, and relies on a single global noise field shared across all spatial regions. In parallel, motion transfer methods synthesize videos by applying motion from a driving sequence to a still image, yet depend on suitable reference video that is difficult to obtain (Jeong et al., 2024 Yatim al. 2024 Pondaven et al.] 2025). Targeting 12V without reference vidcos, SG-12V (Namekata et al. 2025 enforces cross-frame consistency by replacing each frame's spatial self-attention keys/values with those of the first, then optimizes the latent with a box-restricted similarity loss, and re-injects high- frequency detail via an FFT. Although zero-shot and conceptually aligned with our concept of align- ing moved objects to their first-frame representation, This method is demonstrated on SVD-specific layers, so generality for other backbones is unclear; moreover, as shown in Sec. 4.I. it often induces unintended camera motion. Finally, (Yu et al. 2024) proposes a training-free trajectory guided I2V using gated self-attention for layout-conditionel control, temporal attention for propagation, and a Motion Afterimage Suppression step. Its modular inpainting design inherits fimitations-fided tied to T2I inpainting and grounding tokens, heuristic handling of large displacements, while also being designed for a specific video generation model.
Heterogeneous Denoising. Asynchronous denoising has been explored in several contexts. RAD reformulates inpainting with element-wise noise schedules and spatial timestep embedding, enabling
3

---

Published as a conference paper at ICLR 2026
Motion Signal
Dual Clock Denoising
Output
Imput Image
Strong
wwekk
Ssrrogg
M
X
12V
12V
Motion Control
Apply Mask
X
Apply
Figure 2: Overview of Time-to-Move. Given an input image and a motion instruction, mask marks the region under strong control. A motion signal is then generated automatically and, together with the image, conditions an image-to-video (12V) diffusion model. During sampling, denoising starts at different noise levels-lower inside the mask to enforce the specified motion, and higher outside to allow natural deviations the background. Joint sampling then yields realistic video that preserves input details while accurately following the motion control.
ggiion-asynchoonour denoising while adapting a pretrained model via LoRA (im et al., 2025 SVNR addresses spatially variant sensor noise by training with per-pixel timesteps and starting the reverse process directly from the noisy input (Pearl et al. 2023). Diffusion Forcing (DF) introduces temporal heterogeneity by assigning each token (e.g., video frame) its noise level during train- ing, and at sampling uses 2D scheduling matrix over time and noise levels so tokens are denoised different rates (Chen al. 2024). In contrast RAD, SVNR, and DF, our approach is training- free: We impose region-specific schedules directly at inference. RePaint (Lugmayr et al. 2022) is also training-free free, but repeatedly re-noises unmasked regions and denoises solely the mask, so only part of the image is actively denoised. Differential Diffusion (Levin & Fried (2023) generalizes this to continuous per-pixel schedules, enabling soft spatial editing without binary masks. Our method heterogeneously denoises the entire image, extending these scheduling principles to video motion, and eliminating RePaint's resampling loops since no region is excluded.
METHOD
Our goal is enable precise motion control in generative video models. Inspired by SDEdit, which injects coarse edits into images via noising and denoising, we treat a crude warped anima- tion (Sec. 3.I) as the video analogue of such edits and adapt SDEdit to inject intended motion into video diffusion (Sec. 3.2). To avoid the loss of identity that occurs when noising alone drives the process an inherent limitation of SDEdit we opt for image conditioning, anchoring the genera- tion to the clean first frame so that the appearance is preserved throughout the sequence. Building on these foundations, we introduce a novel dual-clock denoising process (Sec. (333 that assigns dif- ferent noise levels to distinct regions, allowing spatially varying motion guidance. An overview of these components is shown in Fig. Finally, our procedure naturally extends to appearance control, allowing simultaneous specification of both dynamic and visual attributes.
Problem Formulation. Our method takes input (i) single image I E 33xHxW (ii) coarse, user-specified warped reference video with frames, V/8 E F x3xXXXX and (iii) a binary mask video M {0, 1 indicating, for each frame, the regions where motion guidance is pro- vided by the reference video. The objective is to generate a realistic video 2p E R Fx3x xW that maintains fidelity to the input image while accurately following the prescribed motion.
3.1 MOTION SIGNAL
We begin by describing how the motion signal T is generated. To facilitate user-frient user-friendly interac- tion. the user selects a region in the first frame to produce an initial binary mask Mp. then specifies coarse motion by dragging this region along trajectory, yielding the sequence M. This defines

---

Published as a conference paper at ICLR 2026
Last Frame
Last Frame Output
Denoising Schedule
Init. Noise
SDEdit (/,o)
8
Input Image
SDEdit
Dual-clock-
Motion Control
denoising (ours)
Timestap
Figure 3: Region-dependent denoising strategies. SDEdit (single clock): low noise levels over- constrain the video, suppressing non-masked region dynamics; high noise levels improve realism but drift from the prescribed motion. RePaint (foreground override): motion is enforced in the object, but uncontrolled regions exhibit artifacts such as duplication. Dual-clock (ours): masked regions follow the intended motion with strong fidelity, while the background denoises more freely, yielding realistic dynamics without artifacts.
piecewise-smoote displacement field within the masked region, which induces per-frame warps of the input image. The warped video V is obtained by forward warping I, with identity mapping outside the mask, with further details in Sec. 5.1] Forward warping naturally introduces disocclu- sions where previously occluded background becomes visible; these holes are filled using a simple nearest-neighbor inpainting strategy. Although presented here dragging. both V and M can be constructed in multiple ways. For example, in Sect. 4.2 we show that V can also be produced by pixel-wise warping of the input image according to monocular depth estimation. As demonstrated in our demo page, we support additional interactions, such as rotation and scaling of the selected region, which integrate seamlessly into the same formulation. While such warped animations are vi- sually unrealistic, they faithfully capture the user-intended object placement and temporal structure. We exploit these properties by using them as a guiding signal for the video diffusion model dur- ing generation, and note that V can also encode appearance modifications, such color changes, within the same framework.
3.2 SDEDIT ADAPTATION FOR MOTION INJECTION
Inspired by the use of coarsely edited images as structural priors in SDEdit (Meng et al. 2021), adapt the method to videos by using the crudely warped video V itseif as a coarse yet explicit motion instruction. We initialize sampling from noisy version of the warped reference, X++ ((I2- VW), Previous publications (Shaulov et al. 2025) show that coarse motion is determined early in the denoising trajectory; By noising to t" the intended dynamics are injected at precisely this stage. If we were to apply this procedure in text-conditioned video diffusion model, the fidelity the input image would be quickly lost; The model's only knowledge of appearance comes from the noised v", so fine details cannot be preserved. To overcome this limitation, we instead opt for an inage-conditioned video diffusion model, which anchors generation to the clean first frame I. The resulting sampling process, xo po xx+,/)) /), faithfully integrates the motion guidance from V while preserving identity and appearance throughout the generated sequence.
3.3 REGION-
DUAL-CLOCK DENOISING
SDEdit employs a single noising timestep t" to corrupt the reference signal before denoising. In our setting, this creates a trade-off. The warped video V contains regions where motion is explicitly
5

---

Published as a conference paper at ICLR 2026
Warped
Ours (SVD)
Drag Anything
SGI2V
MotionPro
First
Frame
Last
Frame
Figure 4: Qualitative comparison MC-Bench Competing methods exhibit artifacts (red), whereas TTM achieves clean placement and appearance consistency.
Method
Training Free?
CTD
BG OGj
Dynamic
Subject
Background Consistency, Smoothness,
Motion
Aesthetic Imaging Quality+ Quality+
CTD₁
Degree, Consistency,
SVD-Based Models
DragAnything
x
10.645
50.885
0.981
0.956
0.942
0.983
0.531
0.554
SG-12V*
/
5.796
12.042
0.803
0,976
0,953
0.991
0.553
0.621
MotionPro
X
8.685
24.485
0.422
0.979
0.975
0.993
0,559
0.617
Ours
7.967
35.340
0.427
0.979
0.967
0.993
0.548
0.617
CogVideoX-Baseee Based Models with Longer Generated Videos
GWTF 0007
x 32.548
86.614
0.736
0.963
0.965
0,989
0.517
0.539
GWTF 0.5
x
27.844
87.708
0.764
0.958
0.963
0.988
0.513
0.539
Ours
/
13.665 70.608
0.357
0.980
0.977
0.995
0.531
0.579
Table 1: Quantitative results on MC-Bench object motion control.
specified (masked areas), alongside regions without explicit instruction. For the masked regions, we want the generated video to closely follow the prescribed motion. In unmasked areas, we not want them to stay static; instead, they should adapt naturally to support the motion. For example, in Fig. 3 when the boat is cut and dragged to follow a trajectory, the wake of the boat should modify accordingly, even though it was not directly manipulated. With a single timestep, SDEdit cannot accommodate this asymmetry. t" small, the denoised video adheres closely to the warped signal but inherits artifacts such frozen backgrounds (top row). If t" is large, the results look realistic but drift away from the intended motion (second row). We therefore conjecture that different regions require different effective noising levels: masked regions demand strong adherence to the motion signal, achieved with less noising (4stnong), while unmasked regions benefit from weaker enforcement, achieved with increased noising (fweak))
The challenge is that standard pretrained diffusion models, which assume inputs are corrupted by single uniform noise level, cannot directly accommodate region-dependent noise. To overcome this, we propose dual-clock denoising. Given a mask M, we noise the warped video reference 1/8 to timestep tweak and initialize the denoising process. At each denoising step with Estrong I tweaks we override the masked region with the corresponding region of the warped video noised to t 1. This constrains the masked regions follow the intended trajectory, while the background free to denoise more aggressively and achieve realism. Once f = (stongg we stop overriding and continue the standard sampling process, allowing the model to refine both regions for a coherent result. Let I, denote the noisy sample at timestep and á.- ( t) the denoiser prediction. The update rule is
Tt-1 (1 - M) @ - (x,, + M ④ xx-11
where Te" is the warped reference video noised to timestep t 1.
Efficiency and Applicability. Our method is lightweight modification to standard sampling, adding no extra computation or latency over regular video diffusion. In practice, it can also be faster than vanilla inference, since TTM runs the denoising process only up l.weak < T, while the remaining motion and masking operations incur negligible overhead. is entirely training-free and plug-ant and-play for image-conditonned I2V models; In Section 4 backbones, demonstrating broad applicability.
we show that integrates with three

---

Published as a conference paper at ICLR 2026
Warped Video
Ours (Wan 2.2)
Ours (CogVideoX)
GWTF
Input Image
Figure 5: Comparison on challenging cut-and-drag example. GWTF exhibits strong artifacts
under large motion (right); TTM follows the prescribed motion realistically across various models.
CLIP Cons.t
Optical flow
Method
MSE
FID,
LPIPS
SSIM,
GWTF 0.5.5
0.033
25.990
0.371
0.526
0.981
76.714
GWTF y.0.77
0,042
28.483
0.370
0.410
0.985
81.738
Warped
0.025 33.443
0.339
0.560
0.981
65.494
Ours
0.022 21.966
0.332
0.586
0.983
60.558
Table Quantitative results DL3DV camera motion control.
4 EXPERIMENTS
We evaluate TTM in three complementary settings: Object motion control (Sec. 4. camera motion control (Sec. 4.2 and joint motion applarracee editing (Sec. [.3)) These cover the primary modes of user intent: animating selected object, inducing global motion via viewpoint changes, and mod- ifying the appearance of scene elements. For the first two, we report quantitative benchmarks and qualitative comparisons against state-of-the-art training-basee -aaed and training-free baselines. For ap- pearance editing, where no standard benchmark exists, we present qualitative results highlighting capabilities unique our approach. We also demonstrate plug-am wogg-and-pay generality across multi- ple 12V backbones (Sec. (4.4) and analyze the dual-clock schedule via ablations (App. A). Video demonstrations are included in the project page
4.1 OBJECT MOTION CONTROI.
We evaluate TTM for object-level motion control. The inputs are a single source image, a binary mask of the target object, and 2D trajectory defining the desired motion. We benchmark MC- Bench (Zhang et al. 2025b) under its official protocol. Notably, the benchmark masks are coarse human-annotated brush regions rather than pixel-accurate segmentations, effectively simulating alistic and noisy user-provided inputs. Further details of the evaluation protocol and implementation provided in App. D.I.
Baselines. We compare against both training-based methods Baathoss-DragAnllhiii (Wu et al. 2024), MotionPro (Zhang et 2025b), and Go-With-thee-Flov (GWTF) (Burgert et al. 2025 and the training-fiee SG-12V (Namekata et al., 2025). Results are grouped by backbone: SVD (hybrid conv/attention, 1..B parameters) and Cog VideoX (Diffusion Transformer, 5B parameters). We apply our bookkone-agnostte method to both architectures, denoting them as TTMsvD and TTMcog- For fairness, we report GWTF with both recommended noise-degradation values (7 E (0.5. 0.7)).
Metrics. We evaluate motion adherence and perceptual quality. For adherence, we use MC-Bench's CoTracker Distance (CTD) for object trajectories and BG-Obj CTD to detect unintended back- ground co-motion. For perceptual quality, adopt VBench (Huang et al. 2024). reference-free suite of automated video metrics. See App. D.I for more details.
Results. Tab. Tab.[summan summarizes the results. Across both backbones. TTM attains the lowest CoTracker distance (best adherence to the prescribed motion), excluding SG-I2V. On SVD, our VBench qual- ity matches MotionPro, with minor metric trade-offs, and surpasses DragAnything and SG-12V on most measures. TTM's dynamic degree is lower than Dnag and SG-I2V; we attribute this to DragAnything often inducing unintended scene motion and local deformations (Fig. 4). whereas

---

Published as a conference paper at ICLR 2026
Warped Video
Ours (Wan 2.2)
Ours (CogVideoX)
GWTF
Input Image
Figure 6: Qualitative comparison of camera-motion control. GWTF drifts from the target camera path, while TTM leverages the warped reference enforce motion, yielding smooth, artifact-free results beyond simple depth warping.
SG-I2V frequently triggers camera co-motion, moving the whole scene rather than just the object (e.g., a rightward pan in the same figure, where the camera shifts right and the moon exits the frame). This effect-ate also noted by [Burgert et al. (2025 is reflected in SG-12V's substantially lower BG-Obj CTD, indicating strong bbject-background co-motion. On the CogVideoX back- bone, TTM achieves substantially stronger adherence to motion conditioning and higher scores on nearly all video-quality metries compared to GWTF. The only exception the "dynamic" score, where GWTF reports higher values; however, these gains often come at the cost of scene defor- mations and inconsistencies,
evident from the background- and subject-consistency metrics in and in Fig. 5 Overall, TTM exceeds the performance of both training-based and training- ffree
Tab. [ baselines on most metrics, while remaining entirely training-free
Qualitative Examples. In Fig. mark, using SVD as the common 12V backbone. Competing methods introduce noticeable artifacts (highlighted in red), while our TTM produces clean foreground placement at the intended location and preserves fidelity to the first-frame appearance. Additional videos and benchmark results are provided in Fig. App. and on our supplementary results page
present representative example from the MC-Bench bench-
4.2 CAMERA MOTION CONTROL
We evaluate TTM synthesizing realistic videos from single image under prescribed camera motion. Following GWTF, we use a subset DL3DV-10K (Ling et al.] 2024]. which contains statie-scene videos with per-frame camera annotations. From the first frame, depth with DepthPro (Bochkovskil et al.] 2025), back-project to a 3D point cloud, and reproject along the prescribed motion to construct reference video. Pixels that are not assigned value by the warp (i.e., holes) are filled by copying the color of the nearest valid warped pixel. The collection of these inpainted pixels constitutes the mask M. We evaluate 150 sequences with 49 target views each, comparing generated results against the original frames at the same viewpoints. Further details appear in App. D
estimate metric
Baselines. We benchmark TTM against GWTF, the leading prior method for free-form motion control. The protocol for constructing depth-based warped videos is identical to that used in our approach; however, GWTF further extracts optical flow from them to synthesize noise warping.
evaluate frame-level alignment using MSE, LPIPS (Zhang et al. 2018), and SSIM (Wang et al.] (004). Motion consistency is assessed by the MSE between #AFT-estimated optical flows (Teed Deng, 2020) of generated and ground-truth videos. Distributional similarity is assessed with FID (Heusel et al. (017)) between all generated and original frames. Temporal consistency is measured as the average CLIP (Radford et al. 2021) cosine similarity between consecutive generated frames.
Metrics. With ground-truth videos available.
Results. Tab. 2 reports quantitative results, and App. E.2 together with our video appendix present qualitative results. Our method delivers the best camera-mont control, outperforming baselines in motion fidelity and pixel quality: vs. the best GWTF variant, pixel MSE drops by 33% and FID by 15.5%; optical-flow flow MSE also decreases, indicating better temporal alignment across the scene.
8

---

Published as a conference paper at ICLR 2026
Input Image
Motion Appearance Control
Input Image
Object Addition
Input Image
Mation Shape Control
I movE
Wurped Video
Ours
EWTF
Warped Video
Ours
Warped Video
Ous
time
move
hove
Time To Move
Time Move
Figure 7: Joint motion and appearance control. TTM leverages a user-provided warped reference to control both motion and per-pixel appearance in diverse tasks, with per-task details given in 4.3
Qualitative Examples. Fig. compares TTM with GWTF input images and user-specified camera trajectories. GWTF struggles with long motions as it relies on noise warping for scene consistency and drifts from the prescribed path. In contrast, TTM precisely follows the target camera motion and preserves identity across frames, yielding smooth, realistic sequences. Depth warping is shown as coarse reference; TTM removes its tearing and holes while retaining the intended motion.
4.3 APPEARANCE CONTROL
Beyond motion, TTM enables pixel-level appearance specification across the scene. By conditioning on full reference frames, the crude animation constrains both where objects move and how they look. Users can guide motion and evolving appearance jointly, without retraining or additional cost. In contrast, prior methods rely on trajectories and text alone, limiting them to ambiguous appearance changes. In Fig. 7 we illustrate three setups: (i) Motion and appearance control: chameleon follows a user-drawn trajectory while changing its color from green to purple. For comparison, GWTF is with optical flow and text prompt describing the desired color (see App- D.3); method preserves both motion and appearance, whereas GWTF fails to satisfy both constraints. full frames allows adding cowboy looking in a mirror; the hat blends naturally into the scene and appears consistently in the reflection. (iii) Joint motion and shape control: TTM preserves the intended graphic deformations while harmonizing appearance with the scene as clouds are revealed.
(ii) Object insertion: conditioning
objects. We place hat
4.4 PLUG-AND-PLAY MODEL ADAPTATION
With video generators evolving rapidly and parameter counts rising, adding motion control with- out relraining becomes especially valuable. Beyond SVD and CogVideoX, TTM applies as is to any image-to-video diffusion model. We demonstrate this on the recently released 14B-parameters WAN _ WWn al., 2025): with only brief adaptation. TTM enables both local object trol and explicit comera-moticn conditioning. In Figs. 5 and 6 we present set of challenging examples. By contrast, GWTF achieves motion control only after fine-tuning Cog VideoX-5B with worped-noise training, requiring ~ A100-80GB GPU-hours.
CUT-AND-DRAG
To make Time-to-Move accessible beyond scripted experiments, we provide lightweight "cut-and- drag" user interface that, given an input image. creates the warped reference video and the mask video annotated by the user. Given an input frame, the user selects one or more objects by drawing polygons around them, or by using single click with Segment Anything (SAM) (Kirillov al. 2023 for automatic full-object masks. Each selected region can then be dragged over time in
https://github.
deo/ wateeo//an....0 www.aan2.2ttateeeeeae

---

Published as a conference paper at ICLR 2026
sequence of segments, with per-segment controls for rotation, uniform scaling, and simple hue- based recoloring. The GUI interpolates these user-defined key poses into a crude warped animation and automatically builds the corresponding binary mask sequence. External images also be imported and animated with the same cut-and-drag operations, enabling object insertion. Fig. provides an illustration of the GUT's workflow.
5.1 WARPING
We parameterize the motion in forward manner, while rendering the warped video with standard backward warping. Let Mo E {0, 1 1 be the initial object mask, either defined by the user using the interactive GUI or taken directly from MC-Bench. The dragging interaction induces, for each frame E fo..... F 1), a 2D transform Φ acting pixel coordinates E R2² In the MC-Bench setting this transform reduces pure translation with displacement A, E IR?
de = x + A..
The warped guidance video then obtained by sampling from the input image inverse- transformed coordinates,
(x) = (00,
for pixels belonging to the moving foreground, while background pixels retain their original value. To handle occlusions and disocclusions, we remove the masked region from I and fill it once using simple nearest-neighbor inpainting procedure that propagates nearby background colors. The mov- ing foreground "sprite" extracted from is then rendered at each frame using de and composited onto this background. For general interactive use, the same mechanism extends beyond pure trans- lation: rotation and uniform scaling correspond to using & as a similarity transform rather than just shift, but the backward warp- wwwrr-anddcompae and-composite procedure remains unchanged. In practice, the user specifies only a few key poses in the GUI, and the intermediate transforms di are interpolated so that the sprite motion and the resulting warped video evolve gradually.
Beyond the cut-and-drag tool, provide lightweight interface for camera control, and Fig. illustrates an example.
6 CONCLUSIONS, LIMITATIONS AND FUTURE DIRECTIONS
We introduced a training- free framework for motion and appearance control in I2V diffusion mod- els. By extending the SDEdit principle to videos, we treat warped reference animations as direct motion guidance, while image conditioning preserves fidelity to the input. To balance striet ad- herence in user-specified regions while enabling natural adaptation in the remaining regions, we proposed region-dependen. dual-clock denoising, a plug-and-pi aad-ppay strategy that produces realistic and faithful generations. Our method has several limitations. Although our framework adapts seam- lessly to different 12V backbones, the dual-clock scheme still requires tuning of (fweak: (atrog)) for each model. Identity preservation is restricted to content visible in the first frame; objects entering later cannot be anchored beyond what implicitly recovered during denoising. Finally, our frame- work requires full object masks when specifying motion, unlike some motion potion-promptig methods that are explicitly trained to operate from partial markings. Nevertheless, our method remains ro- control, in other
bust to imperfect masks, demonstrated in MC-Bench. For camera-mont recent free- frr-form motion-control methods, our setup relies on an off-the-shelf monocular depth esti- mator successfully construct the crude 3D animation. Our framework accommodates extensions beyond our current implementation. In particular, the dual-clock scheme could be generalized to support multiple regions, soft masks, smoother noise schedules, offering more fine-grained con- trol. We leave richer appearance edits (e.g., stylistic transformations), exploration of alternative warping schemes, articulated motion, and long-horizon video generation for future work.
ACKNOWLEDGEMENTS
Or Litany acknowledges support from the Israel Science Foundation (grant 624/25) and the Azrieli Foundation Early Career Faculty Fellowship.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 18208-18218, 2022.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023.
Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. In International Conference on Learning Representations, 2025, URL
abs/ 000/2440.00011
Ryan Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski, Pascal Clausen, Mingming He, Li Ma, Yitong Deng. Lingxiao Li, Mohsen Mousavi, et al. Go-wwihhthhe-floe Motion-controllable video diffusion models using real-time warped noise. In Proceedings of the Computer Vision and Patlern Recognition Conference, pp. 13-23 2025.
Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitz- mann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081-24125, 2024.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261 2025.
Daniel Geng, Charles Hermaann Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevara, Yusuf Aytar, Michael Rubinstein, Chen Sun, et al. Motion prompting: Control- ling video generation with motion trajectories. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1-12, 2025.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to local nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp- 6629 6640, Red Hook, NY, USA, 2017. Curran Associates Inc ISBN 978151086096
Jonathan Ho and Tim Salimans. arXiv:2207.12558, 2022.
Classifier-free diffusion guidance.
arXiv preprint
Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: training for text-to-video generation via transformers. In International Conference on Learning Representations, 2023.
Large-scale pre-
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 21807-21818, 2024.
Yash Jain, Anshul Nasery, Vibhav Vineet, and Harkirat Behl. Peekaboo: Interactive video generation In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 8079-8088, 2024.
via
Hyeonho Jeong, Geon Yeong Park, and Jong Chul Ye. Vme: Video motion customization using temporal attention adaption for text-to-video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9212-9221, 2024.
Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track together. In European conference on computer vision, pp. 18-35. Springer, 2024.
11

---

Published as a conference paper at ICLR 2026
Sora Kim, Sungho Suh, and Minsik Lee. Rad: Region-aware diffusion models for image inpainting. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1439-2445 2025.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF International Conference Computer Vision, pp. 4015-4026, 2023.
Eran Levin and Ohad Fried. Differential diffusion: Giving each pixel its strength. In Computer Graphics Forum, pp. e70400. Wiley Online Library, 2023.
Jialu Li. Shoubin Yu, Han Lin, Jaemin Cho, Jachong Yoon, and Mohit Bansal. Training- frree guid- ance in text-to-video generation via multimodal planning and structured noise initialization. arXiv preprint arXiv:2504.0m 1,XX4:25004006411, 2025a.
Yaowei Li, Xintao Wang, Zhaoyang Zhang, Zhouxia Wang, Ziyang Yuan, Liangbin Xie, Ying Shan, and Yuexian Zou. Image conductor: Precision control for interactive video synthesis. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 5031-5038, 2025b.
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. D13dv-10k: A large-scale scene dataset for deep learning- baasin 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22160-22169, 2024.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11461-11471, 2022.
Wan-Duo Kurt Ma, John P Lewis, and W Bastiaan Kleijn. Trailblazer: Trajectory control for diffusion-based video generation. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-11, 2024.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.0107. 2021.
Koichi Namekata, Sherwin Bahmani, Ziyi Wu, Yash Kant, Igor Gilitschenski, and David B Lindell. Sg-i2v: Self-guided trajectory control in image-to-video generation. In International Conference on Learning Representations, 2025.
OpenAl. Gpt-4o system card, 2024. URL htt
Naama Pearl, Yaron Brodsky, Dana Berman, Assaf Zomet, Alex Rav Acha, Daniel Cohen-Or, and Dani Lischinski. Svnr: Spatially-variant noise removal with denoising diffusion. arXiv preprint arXiv:2306. 16052, 2023.
Alexander Pondaven, Aliaksandr Siarohin, Sergey Tulyakov, Philip Torr, and Fabio Pizzati. Video motion transfer with diffusion transformers. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22911-22991 2025.
Haonan Qiu, Zhaoxi Chen, Zhouxia Wang, Yingqing He, Menghan Xia, and Ziwei Liu. Freetraj: Tuning-free trajectory control in video diffusion models. arXiv preprint arXiv:2406. 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pp- PmLR, 2021.
Xuanchi Ren, Tianchang Shen, Jiahui Huang. Huan Ling, Yifan Lu, Merlin Nimier- David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3e: 3d-informed world-consistent video generation with precise camera control. In Proceedings of the IEEE/CVF Conference on Com- pufer Vision and Pattern Recognition (CVPR), pp. 6121-6132, June 2025.
12

---

Published as a conference paper at ICLR 2026
Noam Rotstein, Gal Yona, Daniel Silver, Roy Velich, David Bensaid, and Ron Kimmel. Pathways on the image manifold: Image editing via video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7857-7866, 2025.
Peter Sand and Seth Teller. Particle video: Long-range motion estimation using point trajectories. International journal of computer vision, 80(1):72-91, 2008.
Ariel Shaulov, Itay Hazan, Lior Wolf, and Hila Chefer. Flowmo: Variance-based flow guidance for coherent motion in video generation. arXiv preprint arXiv:2506.01144, 2025.
Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In European conference on computer vision, pp. 402-419. Springer, 2020.
Narek Tumanyan, Assaf Singer, Shai Bagon, and Tali Dekel. Dino-tracker: Taming dino for self- supervised point tracking in single video. In European Conference on Computer Vision, pp- 367-385. Springer. 2024.
Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Chris- tian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from single image using latent video diffusion. In European Conference Com- puter Vision, pp. 439-457. Springer, 2024.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint 1rXiv:2503.203144 2025.
Angtian Wang, Haibin Huang, Jacob Zhiyuan Fang, Yiding Yang, and Chongyang Ma. Ati: Any trajectory instruction for controllable video generation. arXiv preprint arXiv:2505.22944 2025.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Ecro Simoncelli. Image quality assessment: from error visibility to structural similarity. JEEE transactions on image processing, 13(4))600- 612, 2004.
Thaddaus Wiedemer, Yuxuan Li, Paul Vicol, Shixiang Shane Gu, Nick Matarese, Kevin Swersky, Been Kim, Priyank Jaini, and Robert Geirhos. Video models are zero-shot learners and reasoners. arXiv preprint arXiv:2509.20328 2025.
Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for anything using entity representation. In European Conference on Computer Vision, pp. 331-348. Springer, 2024.
Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, al. Cogvideox: Text-to-video diffusion models with an expert transformer. In International Conference on Learning Representations, 2025.
Danah Yatim, Rafail Fridman, Omer Bar-Tal, Yoni Kasten, and Tali Dekel. Space-time diffusion features for zero-shot text-driven motion transfer. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 8466 8466-8476, 2024.
Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Drag- nuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint ar arXiv:208..0800 2023.
Shoubin Yu, Jacob Zhiyuan Fang, Jian Zheng, Gunnar Sigurdsson, Vicente Ordonez, Robinson Pira- muthu, and Mohit Bansal. Zero-shot controllable image-to- -ovve animation via motion decompo- sition. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 3332-3341, 2024.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3836-3847, 2023.
13

---

Published as a conference paper at ICLR 2026
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The Unrea- sonable Effectiveness of Deep Features as a Perceptual Metric In 2018 IEEE/CVF Confer- Computer Viston and Pattern Recognition (CVPR), pp. 586-595, Los Alamitos, CA, URL https:
USA, June 2018. IEEE Computer Society. doi: 10.1 P0110///V...11..0000.
7/doi. hew.ececcttttrrrooi.ytyyyoo////. 11t.0111111/PPPP 2010 2018 00068
Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: rrrientory-oriented diffusion transformer for video generation. In Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp. 2063-2073, 2025a.
Zhongwei Zhang, Fuchen Long, Zhaofan Qiu, Yingwei Pan, Wu Liu, Ting Yao, and Tao Mei. Mo- tionpro: A precise motion controller for image-to-vides generation. In Proceedings of the Com- puter Vision and Pattern Recognition Conference, pp. 27957-27967, 2025b.
Haitao Zhou, Chuang Wang, Rui Nie, Jinlin Liu, Dongdong Yu, Qian Yu, and Changhu Wang. Trackgo: A flexible and efficient method for controllable video generation. In Proceedings the AAAI Conference on Artificial Intelligence, volume 39, pp. 10743- 2025.
14

---

Published as a conference paper at ICLR 2026
A ABLATION STUDY: DUAL-CLOCK DENOISING
We ablate the dual-clock denoising scheme, presented in 3.3 using the same evaluation protocol described in 4.1 In TTM, the first tick tweak sets the initialization noise level for sampling, while the second tick Estrong sets when we stop overriding the masked part with the noisy warped ref- erence; after this point, all pixels denoise together. For this section, we evaluate this procedure under different set- tings. In these experiments, we use different timing ticks, denoting the first as 1 and the second as t2
First Second tick (l1)
CoTracker distance
Dynamic Imaging quality
tick (12)
degree
tweak
tweak
27.316
0.265
0.623
Estrnn
Etrong
5.528
0.353
0.620
0
2.954
0,411
0.578
tweak
0
2,923
0.404
0.576
Estrong
0
2.942
0.353
0.579
The resulting behaviors under different settings, together with their quantitative outcomes, are summarized below and in Table
fweak Estrong
29.399
0.254
0.622
9.228
0.430
0.615
tweak
Lurog
7.967
0,427
0.617
Single-clock baseline (f1 = tz). This implies apply- ing SDEdit the warped video (tweak fxtong). When
Table 3: Dual-Clock Ablation.
4 11=4 // = !weak, too little conditioning is induced: the Co- Tracker distance is high, reflecting poor motion adherence.
When 4 = 12 strong non-masked regions become over-constrained to unintended motion, sup- pressing dynamies (e.g., the background freezes); see Fig. 3. where the boat's foam remains static although the boat moves.
RePaint-style (t2 0). Here denoising occurs only outside the masked reference (equivalent to RePaint). As expected, for any t the CoTracker distance drops sharply, since the warped masked region is injected throughout denoising. However, this comes at the cost of Imaging quality: the videos appear nearly perfect in motion adherence but unnatural overall, due to the lack of flexibility inside the mask region.
Unconstrained background (t: T). No constraint is applied to non-masked regions. For 22 tweak, motion is not enforced and the model tends to generate overly static videos. For Istrnng performance improves, but tracking error remains unsatisfactory; in practice, this setup often produces duplicate copies of the source object, which harms adherence.
Dual clock (ours). t tweak. t2 Estrong- This setting achieves the best overall trade-off, com- bining strong motion coniiinnnniiiininnggg adherence (low CoTracker distance) with higher dynamic degree and robust visual quality.
B ABLATION STUDY: MASK PERTURBATIONS
Our object-control evaluation on MC-Bench in Sec. [ implicitly measures robustness to inaccu- rate masks: the dataset provides human-annotated brush masks that are coarse, include background regions, and often miss fine object details, rather than pixel-accurate segmentations. To further vali- date this, we add an experiment that explicitly per- turbs the input masks and reports the resulting per- formance, complementing Sec. 4.1 We follow the same experimental setup, but apply morphological crosion and dilation with varying kernel sizes to the MC-Bench masks, simulating under- and over- segmented masks with different boundary charac- teristies. As shown in Tab. 4 these perturbations lead to only minor changes in all metrics, indicat- ing that our method is robust mask inaccuracies.
Morphological Kemell Operation
CoTracker distance
Dynamic Imaging degree
Size
quality
7.967
0.427
0.617
Dilate
3
8.059
0.435
0.618
Dilate
5
8.527
0.425
0.618
Dilate
7
7.898
0.438
0.617
Erode
3
8.499
0,416
0.618
Erode
8.276
0.430
0.617
Erode
7
9.125
0.433
0.617
Table 4: Mask Perturbations Ablation.
15

---

Published as a conference paper at ICLR 2026
C ABLATION STUDY: TIMESTEP SENSITIVITY
Tuning small set of inference-time parameters is a common requirement diffusion-based meth- ods, analogous to adjusting the guidance scale in classifier-free guidance (CFG) (Ho & Sali- mans 2022) or selecting timestep schedule in SDEdit. To characterize the sensitivity of TTM to its timestep parameters, we perform an ablation in which we vary tweak and fstonng around their default values on MC-Bench and measure the resulting per- formance. As summarized in Tab. [s smaller Earong values increase motion adherence and reduce the CoTrrcckrr distance, but slightly degrade imaging quality, as the object becomes more rigid and less free to adapt. Conversely, larger twaa values (i.e.. more initial noise) generally lead to a higher dy- namic degree. Overall, the trends are smooth, and our default settings lie a stable operating regime that provides favorable trade-off between motion control and visual fidelity.
CoTracker
Dynamic Imaging
tweak Estrong
distance
degree
quality
38
27
11.571
0.419
0.620
36
27
10.596
0.454
0.619
34
27
9.546
0.416
0.619
38
25
8,576
0.433
0.617
36
25
7.967
0.427
0.617
34
25
8,031
0,419
0.618
38
6.500
0.438
0.612
36
23
6.757
0.419
0.612
34
23
6.130
0,414
0.615
Table 5:
Timestep Sensitivity Ablation.
The original experiment used wweak = 36 and (sttrnng 25.
D IMPLEMENTATION DETAILS
D.1 OBJECT MOTION CONTROL
This subsection complements Sec. with concise protocol and implementation details:
Trajel-Trajector.. To avoid ambiguity stemming from masks linked to multiple ob- jects/trajectories in the original MC-Bench dataset, we restrict evaluation single- trajectory cases (over 91% of the dataset).
Input handling. Inputs resized to each model's native size and padded to match pect ratio; after generation, padding is removed and outputs are resized back. Exceptions: MotionPro its original benchmark pipeline; DragAnything is run with its default input handling (we observed best results without external resizing).
Trajectory scaling: the 2D trajectory points are affinely transformed with the same resize- and-pad mapping applied to the frames. After generation, we remove padding and invert the scaling when mapping tracks back for evaluation, ensuring geometric consistency.
Clip length. Standardized (2025b)) and 49 frames for Videooeaa-aaa frames thus TTM5VD generates 25 and keeps the first 16; DragAnything emits 20 and we keep the first 16; SG-12V produces 14 and we evaluate the native 14-frame output, which may be slightly favorable to its metrics. If the output has more frames than the provided trajectory, we trim the trajectory to the target length.
16 frames for SVD-based methods (as Zhang al.
methods. Concretely, SVD emits 14 or 25
Preppost-processing and prompts. SG-I2V is conditioned on bounding boxes rather than masks, unlike the other methods. Therefore, following Burgert et al. (2025). we supply the tight bounding box of the provided mask. For prompts, SVD-based methods are text-free, while VodeoxXoo
methods use the MC-Bench prompts.
Mask resizing: Since both SVD and CogVideoX operate in a downsampled latent space, we project the binary masks to the latent resolution with nearest-neighbor interpolation. For SVD this is spatial-only; for CogVideoX also subsample in time match temporal compression (nearest-neighbor in time).
Hyperparameters. All methods (tweak- (strong) = (36, 25) and fix MotionPro's 's motion bucket to 17 (as in their release). For TTMcag use (46, 41). Other run-time settings follow each method's defaults.
T=50 denoising steps. For TTMsvD set
16

---

Published as a conference paper at ICLR 2026
VBench. For CogVideoX- -baseedd-d-dddd--- 19-frame models, we use the long benchmark varian Dynamic Degree. VBench flags a clip as dynamic when the mean of the top 5% RAFT flow magnitudes in frame exceeds cecpoolutionsscalle threshold a. min(H.W) in at least 25% of sampled frames. The default a= 6.0, tuned for VBench's source videos, is too strict for our MC-Bench aedtingppredomianttee static camera with small, localized motions- nearly all clips are marked static. We therefore set a = 3.5 (keeping the 25% rule unchanged), which yields a more meaningful separation of dynamic
256
static.
CoTracker Distance (CTD), Following MC-Benchi MD-Vid Vid protocol. we leverage re- cent advances in point tracking (Sand & Teller 2008 Tumanyan et al. 2024) and employ CoTracker (Karaev et al. 2024) to assess the alignment between the imput 2D trajectory and the motion in the generated video, measured by the mean per-frame Euclidean distance in pixel space (lower is better).
Background-Objed CoTracker Distance (BG-Obj CTD). This metric measures whether the background unintentionally moves together with the controlled object. We run CoTracker on the generated video for both (i) the tracked object trajectory and (ii) a uniform 16 16 grid of points sampled in the first frame. Let o, E R² denote the object's tracked position at frame I, and Pj.t E IR² the tracked position of grid point j at frame We convert all tracks to displacements from frame 1: Ao; = 01 Apja Pj.t P1.1- For each frame t 2 2 and grid point j. we compute d,,t = | - Aor (pixels). The
BG-Obj CTD is then the average over frames and grid points:
BG-Obj CTD
111 ΣΣde
1
dz.
(T- ==
Higher values indicate stronger object-backgroume disentanglement (less co-motion).
D.2 CAMERA CONTROL ON DL3DV
For camera control experiments, we use a subset of the DL3DV-10K dataset. The reference warped videos are created at resolution of 960p using PoToc333D
This subsection provides additional details for Sec. 4.2
We utilize the official DL3DV camera transition data and align the coordinate systems with a sign flip of the z-axis and flip of the camera pitch due convention differences between Py PyTorch3D and NerfStudio, which was used to create the DL3DV dataset. The point cloud is generated in the origi- nal camera frame, with the camera extrinsics derived from parameter estimations and the estimated transition of the first frame. To resolve the inherent depth ambiguity, we perform binary search to find the transition scale that maximizes the MSE alignment between the warped and original videos. This aligns the transitions to metric scale consistent with the output of DepthPro.
To select a robust subset for evaluation, we first filter out videos from the 10K subset with an esti- mated scale of less than 0.3, as these were found to exhibit minimal real camera movement. We then select the 150 scenes with the lowest MSE loss between the warped and ground truth vidcos. The masks used for the TTM process generated by first marking pixels with point cloud contri- bution as regions denoised freely from weeak. while all other pixels are regions denoised freely from tstong- The strong guidance masks generated by first marking pixels with no point cloud contri- bution as O (no guidance) and all other pixels as (guidance). To ensure only regions with dense point cloud data are used for guidance, we apply a morphological "open" operation to the mask us- ing a kernel size of 5. This operation serves to remove isolated noise and expand the non-guidance areas, resulting in cleaner, more reliable mask. For text guidance, we automatically generate a text prompt for each scene using GPT-4o (OpenAl 2024). following CogV VideoX's protoco
D.3 APPEARANCE CONTROL
For the chameleon example demonstrating joint motion and appearance control, we use the prompt: "A realistic video of a four-legged chameleon walking slowly and naturally from left to right along
1ong http:///wnnn/nieeeeeeee///eeeee/
py
17

---

Published as a conference paper at ICLR 2026
thick, textured vine in a lush jungle. Its limbs move in coordinated, controlled reptilian gait as adjusts its body to the curve the vine. The chameleon gradually changes color from green to purple."
E
EXTRA QUALITATIVE COMPARISONS
For the video versions of the comparisons in this paper, well additional results, please visit our project page
E.1
QUALITATIVE COMPARISONS FROM MC-BENCH
Following the experiment described in Sec. we present additional results beyond those shown in Fig. 4 further illustrating our method's performance against leading approaches on the MC-Bench dataset using the SVD backbone.
Firtt Frome
Quarter Frome
Three Quarter
Last
Prome
18

---

Published as a conference paper at ICLR 2026
Warped
(5/0)
SGDV
VolosPro
Firt frome
Midde
Theee Querter
Proma
Werped
(5:0)
Dreg
SGDV
Vollesho
Frome
S
Midde
Querter
M
Prome
19

---

Published as a conference paper at ICLR 2026
E.2 QUALITATIVE COMPARISONS FROM DL3DV
Following the experiment described in Sec. 4.2 we present qualitative results for camera-motion control the DL3DV benchmark, comparing method with GWTF given input image, its monocular depth estimate, and depth-warped video. These examples demonstrate superior perfor- mance in maintaining the intended camera motion and overall visual fidelity.
Ground Truth
Worped
Ours
CWTF
First
Frome
Quarter
Frome
Middle
Frome
Three
Quarter
Frome
Lost
Frame
20

---

Published as a conference paper at ICLR 2026
Ground Truth
Worped
GWTF
First
Quarter
Frame
Middle
Frome
Three
Quarter
Last Frame
21

---

Published as a conference paper at ICLR 2026
Ground Truth
Worped
GWTF
First
UO
BT
Quarter
Frame
Middle Frome
Three Quarter
Frome
Last Frame
22

---

Published as a conference paper at ICLR 2026
F CHALLENGING USER- CREATCD EXAMPLES
To produce the examples shown in Fig. 5 Fig. and on the demo page, we collected 53 test cases, hand-crafted by users, spanning both object-motion and camera-motion control. For each case, the initial reference frame was generated with Gemini (Comanici et al., 2025), and object- control inputs were specified via GUI adapted from the interface introduced (Burgert et al. 2025).
Generation.
Additional Results As explained in Sec. 4.4 we leverage the plug-and-play nature our method to run the recently released WAN2.2. Below we present additional "cut-and-drag" examples that complement Fig. | These real-world cases illustrate scenarios which the current state-of-the-an baseline, GWTF, often struggles to produce coherent results.
(Cagvideok)
Frome
Midd Frome
Worped Video
Cura (Wan2.2)
Ours (Cogliseex)
CNTF
Hile Frome
Froms
23

---

Published as a conference paper at ICLR 2026
Worped Video
Ours (Mon2.2)
Ous (Coglaseox)
GWTF
Firt Frons
lat Troms
Worped Video
Ours (Man2.2) 2.2)
Owr (Cozhideok)
CWTF
Firt Froms
lat Troms
24

---

Published as a conference paper at ICLR 2026
(1) Choose Image
(2) Polygong
(3) Drag
Rotate
(5) Scale
(6) Final Crude Video
Figure 8: Example interaction with our cut-and-drag GUL (1) The user selects an input image. (2) A region (e.g., the monkey) is defined via polygon. (3) The object is dragged to define motion segment. The final key pose is then refined by applying controls for (4) rotation and (5) uniform scaling. (6) The GUI interpolates these key poses, automatically generating the warped video and corresponding mask sequence, where the object gradually moves, rotates, and scales to its final the final motion signal for the TTM framework.
position; these
25

---

Published as a conference paper at ICLR 2026
Loading DepthPro, (CPU)..,
(1) Image
Depth
(3) Select Trajectory Key-Frames
(4) Final Crude Video
Figure 9: Example interaction with camera-control GUL. (1) The user selects input image. (2) The GUI automatically estimates depth and constructs a 3D reprojection for view synthesis. (3) The user navigates the 3D scene and selects camera-pose keyframes define trajectory, which is visualized and interpolated by the GUI. (4) The resulting depth- reprojection yields a warped "crude" video, which is used the motion signal for TTM.
26


Reviews:
[{"review_id": "lG3HcNcTaN", "reviewer": "Reviewer_bHQm", "summary": "The paper proposes Time-to-Move (TTM), a training-free, plug-and-play motion control method for image-to-video diffusion models. Core ideas: (i) generate a crude reference animation (cut-and-drag or depth reprojection) and adapt SDEdit to inject its motion, and (ii) a region-dependent “dual-clock” denoising that uses a lower noise level inside a motion mask and a higher one elsewhere to balance adherence vs. realism. The method claims strong object/camera motion control across SVD, CogVideoX, and Wan backbones with no extra training, with quantitative gains on MC-Bench and DL3DV.", "questions": "please see weaknesses section", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Training-free & backbone-agnostic; integrates with multiple I2V backbones.\n\n2. Clear articulation of the dual-clock idea with ablations.\n\n3. Competitive metrics and clean qualitative examples for object and camera motion control.", "weaknesses": "1. Missing related-work discussion: structural-noise initalization like in video-MSG [1]\n\n2. Sensitivity to the applied template (reference animation + mask + schedule). TTM critically depends on: (a) how the warped video is produced (cut-drag vs. depth reprojection, inpainting strategy), (b) the mask quality, and (c) the chosen (t_weak, t_strong) schedule. The paper acknowledges per-model tuning and that identity is anchored only by the first frame, but doesn’t deeply analyze robustness to template variance \n\n3. Even though figures highlight cleaner results vs. baselines, the pipeline can inherit issues from the template (e.g., depth reprojection tearing/holes) and from hard masking. The paper partly recognizes this but artifact analysis is light.\n\n[1] Training-free Guidance in Text-to-Video Generation via Multimodal Planning and Structured Noise Initialization, 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sI7CikjK4b", "reviewer": "Reviewer_Bb3Y", "summary": "This paper proposes Time-to-Move (TTM), a training-free framework for controllable video generation using image-to-video diffusion models. TTM introduces the concept of using crude reference animations (e.g., cut-and-drag edits or depth-based warping) as direct motion guidance signals. TTM uses dual-clock denoising, which assigns different noise schedules to masked (motion-controlled) and unmasked regions, allowing strict enforcement of motion guidance where specified and natural evolution elsewhere. Experiments across multiple diffusion backbones show that TTM offers strong performance compared to state-of-the-art training-based and training-free baselines.", "questions": "- What is the trade-off in inference time and computational overhead? What is the average inference time increase (or decrease) when using TTM on a base model compared to its vanilla inference process?\n- How robust is the method to segmentation errors? For example, when there is motion blur or non-rigid motion, it is difficult to provide an accurate mask.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The training-free and plug-and-play natures of TTM are valuable. No fine-tuning or architectural changes are required, making the method broadly applicable across various existing models and lightweight.\n- Region-dependent noising is effective, balancing fidelity to user-specified motion with natural background dynamics.\n- Generalization results across backbones look good.", "weaknesses": "- Objects introduced later rather than in the first frame cannot be anchored, limiting use cases involving scene entry, new dynamic object, or sudden occlusion.\n- The dual-clock denoising scheme requires tuning specific settings per backbone, which could hinder plug-and-play deployment in practice.\n- Strict full-object masks must be provided. Control signals relying on sparse cues are more user-friendly in some workflows.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ocWNczkmAL", "reviewer": "Reviewer_4gbG", "summary": "This paper Introduces a training-free method for controllable video generation. It firstly generates a crude edited video according to user's input. Then the edited video serves as the guiding signal for a pretrained video diffusion model and inferences in a SDEdit-style. To achieve better performance, authers modify SDEdit and propose a Dual-clock inference pipeline which apply different denoising timestep for mask and unmasked region.", "questions": "1. The warping method. Can you describe more about the forward warping method? Also, in the provided Object Control demos (second column, fifth row), the warped video not only warps the white ball but also warps the two other balls it collides with. I’m curious how this was achieved. Additionally, I wonder how the method performs on actions that are difficult to warp, such as complex interactions or occlusions between objects. If the warping quality is poor in such cases, would it significantly affect the subsequent generation?\n2. The dual-clock denoising strategies. Could the authors provide some empirical guidelines or example values for selecting  $t_{weak}$ and $t_{strong}$? Is there a generally applicable choice for these parameters, or do they need to be tuned separately for each video?\n3. Experiments. This paper lacks ablation study and the authors should provide more ablation studys on different warping method and the dual-clock denoising strategy.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The method adopts a user-friendly setup that allows users to drag the object they want to move. Studying this problem plays an important role in advancing controllable generation toward practical applications. \n2. The proposed training-free method is straight-forward and efficient. The method also shows board application for various pretrained video diffusion models, extending the function of video foundation model.\n3. The author’s writing and figures in the paper are clear and easy to understand.", "weaknesses": "1. The warping method. The quality of this method highly depends on the quality of the directly warped video. However, the author only briefly mentioned that the warped video is generated through forward warping, without providing detailed steps or references for the process.\n2. The dual-clock denoising strategies. The dual-clock is a slight adaptation of SDEdit and lacks insight or improvement of SDEdit. And I think it's highly depend on the choice of $t_{weak}$ and $t_{strong}$.\n3. Experiments. The authors do not provide ablation study. I think the author should ablate different warping method and the dual-clock denoising strategy to see which part contributes more to the final result.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OFZcWxEjJe", "reviewer": "Reviewer_X1Wj", "summary": "This paper proposes TTM, a training-free framework for motion control in image-to-video (I2V) models. The core of the method is a \"dual-clock denoising\" strategy: it uses less noise ($t_{strong}$) in user-specified motion regions to strongly constrain motion, while using more noise ($t_{weak}$) in other regions (like backgrounds) to allow for natural dynamics. Experiments show this training-free method achieves results comparable to, or even better than, state-of-the-art (SOTA) methods that require training.", "questions": "See weaknesses above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  **Practicality and Novelty:** The paper addresses a key pain point in video generation (training-free motion control). \"Dual-clock denoising\" is a concise and effective innovation that elegantly solves the SDEdit trade-off between motion fidelity and background artifacts. The framework can be used as a plug-and-play module for different I2V models.\n2.  **Experimental Validation:** The experiments are thorough, covering object motion, camera motion, and joint appearance control. Comparisons against SOTA methods like DragAnything and GWTF are convincing, demonstrating the method's effectiveness both quantitatively (Tables 1, 2) and qualitatively (Figs 4, 6). The ablation study in Appendix A also strongly supports the necessity of the dual-clock design.", "weaknesses": "1.  **Relation to RePaint:** The paper should more clearly articulate its distinction from RePaint. The blending operation during the $t_{strong} \\le t < t_{weak}$ phase appears very similar to RePaint's. The core innovation seems to be the **temporal control** of this blending (i.e., stopping at $t_{strong}$ for joint refinement) rather than the blending mechanism itself, which needs clarification.\n2.  **Hyperparameter Sensitivity:** The key hyperparameters $(t_{weak}, t_{strong})$ require manual tuning for different models (e.g., (36, 25) for SVD vs. (46, 41) for CogVideoX), which somewhat reduces its \"plug-and-play\" convenience. An analysis of sensitivity to these parameters is recommended.\n3.  **Dependence on Input Quality:** The method's effectiveness depends on the quality of the \"crude animation\" $V^w$ and the mask $M$. Particularly for the camera motion task, it relies on external depth estimation models and complex 3D reprojection (as detailed in Appendix B.2), which adds a \"hidden cost\" to its application.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)