FastVMT: Eliminating Redundancy in Video Motion Transfer
Authors: Yue Ma
Zhikai Wang
Tianhao Ren
Mingzhe Zheng
Hongyu Liu
Jiayi Guo
Kunyu Feng
Yuxuan Xue
Zixiang Zhao
Konrad Schindler
Qifeng Chen
Linfeng Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Video Motion Transfer; Efficiency; Diffusion model;
Primary area: generative models
TL;DR: We emphasize the motion redundancy and gradient redundancy existing in the training-free motion transfer task and propose FastVMT, an efficient framework using DiT-based video generative model to transfer motion efficiently.

Abstract: Video motion transfer aims to synthesize videos by generating visual content according to a text prompt while transferring the motion pattern observed in a reference video. Recent methods predominantly use the Diffusion Transformer (DiT) architecture. To achieve satisfactory runtime, several methods attempt to accelerate the computations in the DiT, but fail to address structural sources of inefficiency. In this work, we identify and remove two types of computational redundancy in earlier work: **motion redundancy** arises because the generic DiT architecture does not reflect the fact that frame-to-frame motion is small and smooth; **gradient redundancy** occurs if one ignores that gradients change slowly along the diffusion trajectory. To mitigate motion redundancy, we mask the corresponding attention layers to a local neighborhood such that interaction weights are not computed unnecessarily distant image regions. To exploit gradient redundancy, we design an optimization scheme that reuses gradients from previous diffusion steps and skips unwarranted gradient computations. On average, FastVMT achieves a 3.43× speedup without degrading the visual fidelity or the temporal consistency of the generated videos.

Published as a conference paper at ICLR 2026
FASTVMT 4: ELIMINATING REDUNDANCY IN VIDEO MOTION TRANSFER
Yue Ma 7, Zhikai Wang Tianhao Ren
Mingzhe Zheng Hongyu Liu P J.iayii Guo
Kunyu Feng, Yuxuan Xue
Zixiang Zhao Konrad Schindler 5, Qifeng Chen 2, Linfeng Zhang
EPIC Lab, Shanghai Jiao Tong University, HKUSS, THU, Meta, ETH Zürich
Ref Output Ref
wolf is running on the snow
firefighter is running on the street.
A rocket standing at the launch sièe
A Spider MAn is running in the garden
Output
Oure
Oure
DrTFlww
Motionclone
317
Mohiertinecter
SMM
BITFlww
Def
Mationtav
Lateney6)
Figure I: Efficient motion transfer with FastVMT: By eliminating redundant attention computa- tions and reusing previously computed gradients, we achieve faster motion transfer for single-as well as multi-object motion, camera ego-motion, and complex articulations.
ABSTRACT
Video motion transfer aims to synthesize videos by generating visual content according to text prompt while transferring the motion pattern observed in ref- erence video. Recent methods predominantly use the Diffusion Transformer (DiT) architecture. achieve satisfactory runtime, several methods attempt to accelerate the computations in the DiT. but fail to address structural sources of inefficiency. this work, we identify and remove two types of computational redundancy in earlier work: motion redundancy arises because the generic DiT architecture does not reflect the fact that Trame-to-frame motion small and smooth; gradient redundancy occurs if one ignores that gradients change slowly along the diffusion trajectory. To mitigate motion redundancy, we mask the corresponding attention layers local neighborhood such that interaction weights not computed unnecessarily distant image regions. To exploit gradient redundancy, we design an optimization scheme that reuses gradients from previous diffusion steps and skips unwarranted gradient computations. On average, FastVMT achieves 3..43 speedup without degrading the visual fidelity the temporal consistency of the generated videos.
Equal contribution Corresponding author

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Motion transfer aims to generate a novel video by transferring the dynamics of a reference video sequence to a target sequence, while preserving the target's appearance and semanties. For instance, the reference video might show an action sequence performed by an actor, which shall be transferred target subject while preserving their identity; or the reference might prescribe a particular camera path through the scene, which one would like to replicate for the target scene (see Fig. 1). In other words, motion transfer offers an intuitive interface for controllable motion synthesis, with applications ranging from movie productions and game development to digital advertising and content creation on social media platforms.
Recent advances in video motion transfer increasingly leverage large, foundational generative video models. These models typically employ the DiT architecture within a denoising diffusion loop'. They not only capable of synthesizing high-quality videos from noise, but can also be conditioned with text or image prompts to control the video style and content. A variety of motion transfer approaches have emerged that leverage these powerful visual priors, in either taaining-based and training-free fashion. Training-basee
methods (e.g., MotionDirrecoor (Zhao et al., 2023b), MOFT (Zhang et al., 2023), DeT (Shi et al., 2025)) extract the motion patterns of a specific reference video by fine-tuning the parameters of the diffusion backbone. For example, MotionDirector (Zhao et al., 2023b) and DreamMotion (Jeong al., 2024a) adopt dual-path versions of low-rank adaptation (Hu al., 2022) to disentangle the representations of motion and appearance in the diffusion DIT. Although they are capable of generating videos whose motion follows the reference, they suffer from practical limitations: overfitting to every new reference video is time-consuming (e.g., up to 2 hours on an A100 GPU) and therefore unsuitable for open-domain and real-time settings.
To achieve efficient and generally applicable motion transfer, attention has shifted to training- frameworks (Pondaven et al., 2025a; Xiao et al., 2024; Yatim et al., 2024b). They obviate the need for per-video fine-tuning and thus enable significantly faster synthesis (e.g., =100 minutes an A100 GPU). The training-free approach also exploits the gradual, iterative denoising process of contemporary video foundation models: The reference video is first inverted into the embedding space of the DiT to extract features that encode the motion. Then the output video is synthesized by denoising diffusion, guided by both text prompt and the gradient between the motion embeddings of the source and target video.
Our work is motivated by the observation that, in existing implementations of this pipeline, both the extraction of motion embedding from DjT backbone and the computation of motion gradients introduce considerable redundancy. Rather elementary properties of videos, and of the associated generative process, suggest that the computational cost of training-free free motion transfer can be reduced considerably. (i) Motion redundancy: To extract the motion embeddings from latents (Yatim et al., 2024a) or attention maps (Pondaven et al., 2025a) in the inversion stage, it is not necessary to calculate pairwise similarities between all tokens of consecutive frames. Frame-to-framd motion has limited magnitude and is locally smooth, hence motion features can be computed more efficiently, see Fig. 2(a). (ii) Gradient redundancy: In the denoising stage, there is no need to recalculate all gradients at each timestep. We find that motion transfer is a of "stable gradient optimization". Motivated by the idea of deterministic sampling to upgrade DDPM (Ho et al., 2020) to DDIM (Song al., 2020), we examine the gradient updates in consecutive optimization steps and observe that they tend to be similar, see Fig. 2(b). Consequently, gradients can be reused over multiple iterations.
Based on these observations, FastVMT makes two contributions to achieve efficient motion transfer.
(1) Instead of extracting motion embeddings token by token, as in DiTFllow (Pondaven et al., 2025b), we design widdnggwindow strategy that operates downsampled attention maps and an associated corresponding window loss, to perform a more reliable and more efficient local search for motion correspondence.
(2) We address gradient redundancy with a step-skipping gradient computation. Gradients are recalculated only at selected iteration steps, between those steps, the most recent values are reused so as to reduce the total number of gradient calculations and amortize them better.
These two tricks enable high-fidelity video generation with camera trajectories and/or object motions according to the source video, see Fig. 1. Extensive experiments and user studies confirm that
In this paper, the term "diffusion" includes flow-based interpolants (Lipmmn al., 2022; Liu al., 2022).
2

---

Published as a conference paper at ICLR 2026
redundoncy
Gradient redundency
tesn lakes need nottled with redurdantt
Devion
optiniaation dlop necessory
poLa Rold key
Corrasponded tokes will onl oppoor matesing officiens
step pap-sslipping cdlsuation musees traateppe ollicinn
Prsiiul Takar
Haps
Consaginded tahar
18)
Figure 2: Motivation of our method. Training-free video motion transfer can benefit from redun- dancies, both at the level of the DiT architecture and of the iterative diffusion process. (a) Motion redundancy: Video motion is small and locally consistent, ever match tokens in the next frame within a local neighborhood.
motion token in frame will only
(b) Gradient redundancy: Gradient updates in consecutive optimization steps mostly similar (visualized here with PCA). There is no need to recompute them at every single step.
Vast achieves state-of-the-art performance both qualitatively and quantitatively, with up to 14.91 x lower latency. Furthermore, FastVMT delivers 3.43x speedup with minimal performance degradation, preserving near-lossless quality across various evaluation metrics when compared the original training-frees free video motion transfer pipeline.
2 RELATED WORK
Text-to-video generation. fext-to-video generation aims synthesize realistic videos by precisely matching both the visual content and motion dynamics described in the input prompt. Previous works (Long al., 2025; Shen al., 2025; Chen al., 2025c; Ma et al., 2022; Feng al., 2025; Chen et al., 2024; Guo et al., 2023; Wang et al., 2023; Ma et al., 2024b; 2026; 2025e;d;a; 2024d; 2025b;e; Qiu al., 2025c:b; Xiong al., 2025; Yang et al., 2024a) introduce temporal modules in UNet architectures to generate coherent videos. To generate complex video motion, the advancement of Diffusion Transformer- based methods for text-to-video generation exhibits superior performance in both spatial quality and temporal consistency. These models (Liu et al., 2024; Yang et al., 2024b; Xu al., 2024; Kong et al., 2024; Wang al., 2025a) demonstrate the power of scaling transformers to produce highly realistic video clips from detailed prompts, unlocking potential for diverse downstream video generation tasks.
Video motion transfer. Motion transfer focuses generating novel videos while transferring motion from reference videos, differing from vidco-to-vider translation (Zhao et al., 2023a; Ma et al., 2025d; Liu et al., 2023) by decoupling spatial appearance and temporal motion. Early approaches rely explicit control signals such as poses (Ma et al., 2024a; Zhao et al., 2023a), depths (Gen, 2023; Xing al., 2024), and bounding boxes (Wang al., 2024b). Training based methods (Zhao et al., 2023b; Jeong et al., 2024a; Ren et al., 2024) employ spatial-temporal decoupled attention mechanisms by dual-path LoRA architecture. Recent works (Ren et al., 2024; Wu al., 2024) improving motion- appearance disentanglement, though they remain time-consuming and non-reusable. Training-fres methods (Hu et al., 2024; Pondaven al., 2025a: Yesiltepe et al., 2024; Ling et al., 2024; Xiao et al., 2024) extract motion embeddings during inference, with DiTFlow (Pondaven et al., 2025a) proposing attention motion flow optimization. However, existing methods suffer from computational redundancy in both the architectural and diffusion process perspectives. In contrast, we first analyze the redundancy in training-free motion transfer and design the sliding-window motion extraction and step-skipping optimization to improve efficiency.
METHOD
Given input video I [[' I".... and the prompt P describing the target video content, aim to design an efficient training-free framework to generate a novel video J [J], [J',.., J"] following the input prompt P, while preserving the same camera pose changes and object motion.
3

---

Published as a conference paper at ICLR 2026
To achieve this, propose FastVMT, efficient framework using DiT-based video generative model (Wang et al., 2025a) to transfer motion efficiently. The pipeline of our method shown in Fig. 4. We first analyze the existing redundancy in previous works and introduce our motivation in Sec. 3.1. The sliding-window motion extraction strategy is present in Sec. 3.2. To improve the motion consistency, we design the corresponding window loss in Sec. 3.3. Finally, in See. 3.4, propose the step-skipping gradient optimization to ensure gradient efficiency.
3.1 MOTIVATION
We summarize the two observed redundancies of state-of-the-ar approaches in the training-free video motion transfer task and propose the modules to address them.
Motion redundancy. In the inversion stage, existing training-free video motion transfer ap- proaches (Pondaven et al., 2025a; Xiao et al., 2024; Yatim et al., 2024b) utilize the global token similarity to obtain the reference motion flow. Specifically, for every optimization step, each token requires calculating the similarity with all tokens in the next attention map. However, we note that every motion token will only correspond with a token in nearby regions in the next attention map. As shown in Fig. 2(a), the corresponding token in the dog's s nose would only appear around nearby regions rather than on the road. Therefore, such a property about temporal consistency makes unreasonable to extract the motion flow by calculating token-by-toker similarity globally. To address this, we introduce the Blddng-windoww motion extraction strategy. Only the regional tokens are calculated for efficient motion extraction. Meanwhile, such a design enables correcting the mismatch during the motion extraction, as shown in Fig. 5, ensuring the motion consistency of generated results.
Gradient redundancy. During the optimization process of training- free motion transfer methods, significant computational bottleneck emerges from the repetitive gradient calculations per- formed each inner optimization step. Specifi- cally, for every denoising timestep, the optimiza- tion loop performs gradient computation across all inner optimization steps to update the latent representation. However, we observe that the gradient updates exhibit high similarity across consecutive optimization steps within the same denoising timestep. As shown in Fig. 2(b), the PCA analysis reveals that gradient patterns re- main relatively stable across adjacent optimiza- tion steps. Therefore, such "stable gradiens op- timization" makes it unnecessary gradients at every optimization step. To address this, we introduce the step-skipping gradient op- timization strategy. Only specific optimization steps require gradient computation, while intermediate steps reuse cached gradients for efficient optimization (in Fig. 3).
Tme If Metlen Pie
Figure 3: Illustration of siip-skipping gradient optimization. We observe that skipping some steps in the gradient optimization step does not degrade the motion transfer performance. When increase the skipping step, the optimization tra- jectory is similar until 3-step skipping.
compute
3.2 EFFICIENT ATTENTION WINDOW
Attention acquisition. We leverage the inherent attention mechanism within video Diffusion Transformers (DiTs) to extract fine-grained motion patterns, based on the premise that correlated content across video frames is naturally captured by the self-attention layer's query-key interactions.
Given an input video = ..., I"], and the prompt P of target video content, we utilize the 3D VAE encoder (Wang et al., 2025a) to obtain its latent representation āref E(Zrer). To obtain a clean motion signal, this latent is passed through a specific DiT block at a low denoising step, typically t 0. For tile-based approach,
first partition the spatial dimensions into tiles of size (th, ). For each tile, we select a representative query at the tile center and compute its attention with all keys in the target frame. For any pair of frames (i. in the video, the representative cross-frame
4

---

Published as a conference paper at ICLR 2026
Sliding window motion extraction (Sec 3.2)
Step-dkpping gradient
Inpul video
optimization 3.4)
Z
Inversion
WWNE DD
FN
Opt step
car driving
Zo
Z,
Zμ-1
2,
Cornespandence Loss (Sec 3.3)
c
2.3)
Generated video
Denoising
Arhs H2
VIE
Hogg y
running
Z'
Z
z
optimizaation
step
P Ref embedding
embedding 36 Skip step
Frozen weight
Trainable part
Figure 4: Overview our method. Left: Given a reference video, we first leverage the sliding window to extract motion embedding from attention during the inversion stage. At the denoising stage, we calculate the total loss and leverage the step-skipping gradient optimization to guide the video generation. Right: The Step-skipping gradient optimization is proposed to improve gradient redundancy. Additionally, we introduce the corresponding- window loss to boost the motion consistency of generated videos.
ro attention map Aj
is computed as:
Qrep (())T VDn .7
A softmax
R N
(I)
W. the number of tiles, S H x W the spatial token length, and the temperature parameter. From this representative attention map, we estimate the window center for each tile as:
where Nites = Fo Fo
(os(s)
where pos(s) denotes the spatial position of token s. This estimated center guides the subsequent window-constrained Attention Motion Flow (AMF) computation, enabling efficient motion extraction while maintaining spatial precision.
Sliding-window motion extraction. To enhance the computational efficiency and precision of AMF extraction, we propose a novel sliding window strategy that mitigates the redundant computa- tions inherent in prior methods. Our approach leverages the observation that long-range query-ker interactions self-attention layers yield diminished motion information, and the most relevant keys for object typically confined to local spatial window due to finite motion speeds.
extract AMF from query Q and key K., both shape (N, H, W. D), where and W denote height and width of the latent representation, and N is the number of frames. Here, Q {Q1.... IN }. with qs. E [1.....NV NN representing the query tensor for a specific frame, and follows a similar notation. Unlike prior methods that compute AMF across all q-k pairs while attending to the entire spatial dimension, our approach employs a sliding window to constrain computations both temporally and spatially:
Twindow (4:) {wj j E min(i + 88, Ny], Swindow (kin..)) (kw (h', w/) Wh,) (3)
where represents the temporal span and W/ denotes a spatial window size 1 I centered at position (h, w). To determine the optimal window center, we partition each frame into spatial blocks and select representative queries. The window center for each block is computed as:
(17) block
Block + argmax (h.wr) (Q (K())) b,
(4)

---

Published as a conference paper at ICLR 2026
where Poloc is the block center position and the argmax operation yields the displacement vector from representative query-key interactions.
Our approach significantly enhances efficiency. Temporally, it reduces the time complexity from O(F²) to ((F), where is the number of frames, enabling scalable video generation. Spatially, by constraining computations to local window containing the most relevant keys, eliminate redundant calculations, thereby achieving precise AMF extraction with minimal quality loss.
3.3 CORRESPONDING WINDOW LOss
Motivated by the observation that motion in- formation is predominantly captured by closely adjacent query-key pairs, we design a weighted AMF loss and corresponding-wing -innddooww loss to enhance motion transfer accuracy with temporal stability. The weighted AMF loss aligns the mo- tion patterns between reference and generated videos by computing the L2 distance between their respective displacement matrices, which is formulated as:
Wrong cor correspond.
X
Right correspond. V
Attn
W/o siddngg window
Ours
Figure 5: Illustration of attention motion flow extraction with sliding window. Without the slid- ing window, attention tokens are prone to incorrect correspondences (middle). Incorporating sliding window improves alignment, leading to better mo- tion consistency(right).
AMM =
Σ (FJ))T
Wbj-i|
(5) where FF represents all frame pairs within the temporal span sy, and the weights are defined
as:
1.0
d-1
if d < 8
Wd
(6)
0
otherwise
where a is set as 0.2 to provide linear decay, and d = [j represents the frame distance.
To enhance temporal consistency, introduce corresponding-wnndee loss that penalizes inconsis- tencies in key representations across adjacent frames within the sliding windows:
Lwindow 1
F-1 Σ
P
N;
K 1
K
(p) -
2
N;
where K(O), denotes the mean key representation within the sliding window w(P), for tile p when anchoring at frame and comparing with target frame j.
The total loss combines both components with appropriate weighting:
Liotal AaMP LAME + Awindow L-windows
(8)
where λAMF is to to emphasize motion alignment, and Aunekk is set balance the corresponding-wnndde loss. This dual-component loss ensures both accurate motion transfer and temporal stability, effectively addressing motion consistency challenges in video generation.
3.4 STEP-SKIPPING GRADIENT OPTIMIZATION
Despite the computational efficiency introduced by our sliding window strategy, optimizing the latent representation remains computationally intensive due the high cost of back propagation through multiple DřT blocks. Through empirical analysis, we observe a high degree of similarity in the gradients of the latent representation across consecutive optimization steps. Leveraging this insight, we propose an interval-based gradient reuse strategy that selectively computes gradients while maintaining optimization effectiveness.
Our step-skipping optimization operates with a fixed interval during the inner optimization loop. For total of J optimization steps, gradient computation occurs only when the current step j satisfies

---

Published as a conference paper at ICLR 2026
Ref
rover is driving on the moon.
A blue train is moving on the tracks.
Outrutt video Output
A leopard is running in the show.
Two Mountainsers are high -iivnniiinng in the snow.
Ref
An astronaut walking front of the magma.
dragon flying snow coww-epped mountains.
Output video
Figure 6: Gallery of our method. Given reference video, our FastVMT capable generating high-quality video clips that faithfully preserve diverse motion patterns. More visual results can be found in Appendix C.
the condition mod O, or when using the full AMF mode. The algorithm can formalized as:
V VxLuma (x²)
if j mod A mode AMF
4 Xy. geached
otherwise
where Seached represents the gradient from the most recent computation step. This strategy reduces gradient computations from J to approximately [J/A] A per guidance step, achieving theoretical
speedup of A/ [ x in the optimization phase. The cached gradient geachod is updated after each actual gradient computation:
geached gj when j mod A
(10)
This approach significantly reduces computational overhead while maintaining motion transfer quality, as the gradient similarity across consecutive steps ensures that cached gradients remain effective for optimization guidance.
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
In our experiment, we employ the open-sourced video generation model WAN-2.1 (Wan et al., 2025) the base text-to-video generation model. The denoising steps are employed for 50 for all experiments. Unless stated, the output resolution is 480 x 832 with F = 81 frames (internally rounded 4k+1). Latents are initialized as Gaussian noise of shape (1, 16, F-1 + 1, W Latent tiling is enabled with tile_size =(30, 52) and tile iel__ssridee =(15, 26) in VAE space; H this yields a per-frame token grid of = 5 by w & for the DĪT. During motion transfer, as Pondaven et al. (2024), enable our sliding-window window based AMF guidance at the first 20% outer denoising steps; each guided step runs a 10-step latent-only inner optimization with Adam and linear learning- rate decay 0.003 0.002. At each guided diffusion step t, form reference latent by adding step-consisten noise to cached clean latents and perform a forward pass with null text extract queries/keys from the 15th DiT block. More details can be found in the Appendix 5.
W.
7

---

Published as a conference paper at ICLR 2026
4.2 COMPARISON WITH BASELINES
Qualitative comparison. We compare our approach with the state-of-the-an video motion trans- fer methods visually: MOFT (Xiao et al., 2024), MotionInversion (Wang et al., 2024a), Motion- Clone (Ling et al., 2024), SMM (Yatim et al., 2024b), MotionDirector (Zhao et al., 2023b), DiT- Flow (Pondaven et al., 2024), and DeT (Shi et al., 2025). For fair comparison, we adapt the Wan-2.1 as the same backbone. Our experimental results demonstrate that FastVMT achieves superior perfor- mance and greater versatility across a wide range of motion transfer scenarios. As illustrated in Fig. 8, these works (Xiao et al., 2024; Yatim et al., 2024b; Pondaven et al., 2025a; Shi et al., 2025) have the challenge of handling complicated interaction motion. In contrast, our method enables generating videos with aligned movement patterns, preserving the spatial relationships between moving subjects.
Quantitative comparison. We compare our method with the state-of-the-art video motion transfer on on 50 high-quality videos selected from the DAVIS dataset (Perazzi et al., 2016). For fair comparison, we employ 1 as the same backbone. Previous works are constrained by the limited video length, with evaluations conducted using only 32 frames at resolution of 830 x 480. In this context, we classify the state-of-the-ar art (SOTA) methods into two categories; training-frre free and ouming-based based on whether they leverage spatial/temporal LoRA for optimizing complex motion patterns. (a) Time: We record the total time required for completing the motion transfer process, including any inference-time optimization. Leveraging proposed sliding-window motion extraction and step- skipping gradient optimization, FastVMT is the fastest method. Its runtime is faster than training-free methods, while delivering better performance. (b) Motion Fidelity: As in Yatim et al. (2024b), use motion fidelity to assess the similarity of tracklets between reference and generated vidcos. (c) Temporal Consistency: We measure frame-to-frame coherence by calculating the average feature similarity of consecutive video frames using CLIP (Radford et al., 2021). (d) Text Similarity: CLIP is used extract features from the target video, and the average cosine similarity between the input prompt and video frames is computed. (f) User Study: To account for the limitations of automatic metrics in capturing real-world preferences, we conducted a user study with 20 volunteers. They ranked methods based on motion preservation, appearance diversity, text alignment, and overall quality, using a (best) to 8 (worst) scale. The average rank per method (lower ranks are better) is presented in Appendix.5. Our method outperforms others in both automated metrics and user preferences.
In addition, we collect 40 real-world videos and 40 high-quality generated videos by advanced text-to-video gen- erative models (Kong et al., 2024; Wang et al., 2025b). For each video, we generate 5 different prompts. Four metrics in VBench (Huang et al., 2023) are employed for a more aceu- rate evaluation (in Tab. 2). (1) Sub- ject Consistency: We assess whether the identity of the subject preserved across frames. (2) Motion Smooth- ness: The metric evaluates inter- frame continuity using learned motion priors. (3) Aesthetic Quality uses LAION-trained aesthetic predictor to score visual appeal. (4) Back- ground Consistency: We evaluate the coherence of the background. Our proposed method significantly outper- forms all baseline approaches across every video quality metric, thereby showcasing the state-of- -fe perfor- mance in novel video.
Table 1: Quantitative ablation. Red and Blue denote best, 2nd.
Method
Text Sim..
Motion Fid.
Temp. Cons. f
Time(s)
w/o Sliding Wind.
0.2352
0.6912
0.9654
227
Loss
0.2345
0.5942
0.9762
183
Step Skip.
0.2317
0.7044
0.9881
302
Ours
0.2422
0.7471
0.9865
184
Motion fid.
ws cor shipping
step skipping
wes sliding wind.
Motion Smooth.
Temp. Cons.
Backgrdand
Tech-Frune
Sm.
jest Cont.
Austhetic
Figure 7: Quantitative ablation comparison on Vbench metrics. We select the seven metrics to evaluate the effec- tiveness of the proposed strategy.
8

---

Published as a conference paper at ICLR 2026
Table 2: Comparison with state-of-the-art video motion transfer methods. Red and Blue denote the best and second best results, respectively. User study scores are reported in Appendix 5.
Quantitative Merrics
Vbench Metrics
Method
Teat Sim.
Motion Fid.
Temp Cons.
Time (o..
Soh Cons.7
Back Cons.
Arc. Qual.
Motion Smooth
Tr Training-Bawee Based Methods
Monienolvvassion (Jenng ot 20144)
0.2388
0.6515
019602
632.41
0.9339
0.9322
0.4062
0.5532
MericoEFrecttrr (Zluo et 20236)
0.2336
U.5244
0.5531
W06,6 64
0.9173
0.9379
0.3443
0.9633
D&T (Sli et al, 2025)
0.2187
0.6166
0.9818
2745.00
0.9787
0.9644
0.3599
0.9598
Training- Free Methods
MOFT (Coss et al. 9114)
0.9413
Chni (Ling 2014)
(Pondaven 2025g)
Ref video
smm
MOFT
Motion Director
DeT
Motion Inversion
Motion Clone
DifFlow
Ours
A knight korseback galloping outside the castle.
spacecraft ft is moving the Space base.
Figure 8: Qualitative comparison with baselines. We perform the visual comparison with various baselines using various kinds of motions. Our method obtains better performance in various motions. More visual results can be found Appendix C.
4.3 ABLATION STUDY
Effectiveness of sliding-window based motion extraction. As shown in Tab. and Fig. 7, removing the sliding window mechanism results performance degradation across multiple metries and increased computational overhead and inference time. In Fig. 10, we present the visual results without sliding windows. is clear observe a light reduction motion fidelity and temporal consistency, confirming that our approach effectively balances computational efficiency with motion transfer quality, Additionally, we also show the visual comparison of attention motion extraction in various attention layers in DiT (see Fig. 9). The motion extraction is more accurate in the middle layer of DiT. The quantitative ablation about it provided in Appendix
Effectiveness of worresponding-window window loss. Tab. and Fig. reveal that excluding the corresponding-window loss leads to substantial degradation motion fidelity, highlighting its essential role in maintaining accurate motion transfer. As shown in Fig. 10, equipping with this loss function effectively constrains temporal inconsistencies to ensure robust motion alignment, while introducing minimal computational overhead (less than 1 1% increase in processing time), thus preserving both accuracy and efficiency.
Effectiveness of step skipping gradient upgrading. The step-skipping strategy significantly reduces computational time while preserving video generation quality. As demonstrated in Tab. and Fig. 7, this optimization achieves substantial time savings with negligible impact on motion fidelity and temporal consistency, validating the effectiveness of gradient reuse in our framework.
5 CONCLUSION
In this work, we introduced MastVMT, training-free video motion transfer framework that explicitly addresses motion redundancy in diffusion transformer architectures and gradient redundancy along the diffusion trajectory. To eliminate the motion redundancy, we propose the sliding-window window strategy associated with corresponding window loss to achieve more reliable and more efficient local search for motion correspondence. To migrate gradient redundancy, We introduce step-skipping gradient

---

Published as a conference paper at ICLR 2026
Ref vodeo
10 / 10 Leyer
15% 30 Layer
20 Layer
w/c thep alzsha
Right connespondince
Whonn cornespendence
Figure 9: Illustration token correspondence performance in various attention layers of DIT. We extract attention correspondences from different DiT layers. The middle attention lay- ers exhibit stronger token correspondence perfor- mance.
Figure 10: Qualitative ablation of the pro- posed modules. The reference video is shown at the top-left of the first column. The inference time is shown at the bottom-left of the image. The prompt is "A white cat running on the ground".
computation to ensure computational efficiency. By incorporating the proposed strategies, our method achieves x average speedup without compromising either visual fidelity or temporal consistency. We believe this line work opens new opportunities for building more efficient and practical generative video motion transfer.
ACKNOWLEDGEMENT
This research was supported by the Shanghai Science and Technology Program (Grant No. 252R1402278). Part of the compute is supported by the SwissAI Compute Grant a144 and a154.
REPRODUCIBILITY STATEMENT
All quantitative tables, qualitative images, and video results in this work are reproducible and correspond to raw model outputs without manual editing or post-hoc alteration, except for minimal format conversion and compression. After the review process, we will release partial public repository to support reproduction, including inference scripts, example data, and example videos. The datasets, configurations, and procedures used for training and evaluation are documented in Section 4. I and Appendix We will also provide fixed configuration files and random seeds so that independent runs reproduce the visual results within expected stochastic variation.
ETHICS STATEMENT
Our work studies cootion-transfer video editing. The proposed dataset contains videos of people, vehicles, and landscape camera motions. To mitigate representational bias in demonstrations, curated and displayed examples spanning different races, genders, and styles the main text and appendix. All illustrative videos shown in this paper sourced from publicly available web content; we respect the original licenses and terms of service and use the content solely for research purposes. We will not publicly release the dataset prior to completing the insertion of Al-generated watermarks and an ethics/content-saae -afety audit. We explicitly prohibit harmful or deceptive uses of our methods and data, including deepfake attacks and other malicious generative behaviors. When any portion of our code is made public, we will enforce visible and/or machine-detectable watermarking during inference to help deter misuse. Any future releases will be accompanied by usage terms that forbid impersonation, harassment, other malicious applications, and will remove or restrict content that raises privacy, legal, or safety concerns.
REFERENCES
Gen-1. https:
hatp:///mmmmmmm//eeeeeec///eeeee 2023.
Chubin Chen, Sujie Hu, Jiashu Zhu, Meiqi Wu, Jintao Chen, Yanxun Li, Nisha Huang, Chengyu Fang. Jiahong Wu, Xiangxiang Chu, al. Taming preference mode collapse via directional decoupling alignment in diffusion reinforcement learning. arXiv preprint arXiv:2512.24144 2025a.
Chubin Chen, Jiashu Zhu, Xiaokun Feng, et al. S -ggiiddanc:: Stochastic self guidance for training- enhancement of diffusion models. arXiv preprint arXiv:2508. /2880, 2025b.
10

---

Published as a conference paper at ICLR 2026
Siran Chen, Qinglin Xu, Yue Ma, Yu Qiao, and Yali Wang. Attentive snippet prompting for video retrieval. IEEE Transactions on Multimedia, 26:4348 4359, 2023.
Siran Chen, Yue Ma, Yu Qiao, and Yali Wang. M-bev: Masked bev perception for robust autonomous driving. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 1183- 1191, 2024.
Yiyang Chen, Xuanhua He, Xiujun Ma, and Yue Ma. Contextflow: Training-free video object editing via adaptive context enrichment. arXiv preprint arXiv:2509.178. 17818, 2025c.
Kunyu Feng, Yue Ma. Bingyuan Wang, Chenyang Qi, Haozhe Chen, Qifeng Chen, and Zeyu Wang. Dit4edit: Diffusion transformer for image editing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp.
2025.
Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint wwXii::307.04725, 2023.
Yuwei Guo. Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Mancesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In The Twelfth International Conference on Learning Representations, 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-683 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhn -hu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICIR, 1(2):3, 2022.
Teng Hu, Jiangning Zhang, Ran Yi, Yating Wang, Hongrui Huang, Jieyu Weng, Yabiao Wang, and Lizhuang Ma. Motionmaster: Training-fre -fre camera motion transfer for video generation. arXiv preprint arXiv:2404. 15789, 2024.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench: Comprehensive benchmark suite for video generative models. 2024 IEEE/VVE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 21807- 21818, 2023. URL https: /ppp wwwantiescho
org/ Corpus : 265506207.
Hyeonho Jeong, Jinho Chang, Geon Yeong Park, and Jong Chul Ye. Dreammotion: Space-time self-similarity score distillation for zero-shot video editing. arXiv preprint arXiv:2403. 12002, 2024a.
Hyeonho Jeong, Geon Yeong Park, and Jong Chul Ye. Vme: Video motion customization using temporal attention adaption for text-to-video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1212-9221, June 2024b.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03600 2024.
Yunlong Lin, Zixu Lin, Haoyu Chen, Panwang Pan, Chenxin Li, Sixiang Chen, Kairun Wen, Yeying Jin, Wenbo Li, and Xinghao Ding. Jarvisir: Elevating autonomous driving perception with intelligent image restoration. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22369 22380, 2025a.
Yunlong Lin. Zixu Lin, Kunjie Lin, Jinbin Bai, Panwang Pan, Chenxin Li, Haoyu Chen, Zhongdao Wang, Xinghao Ding, Wenbo Li, al. Jarvisart: Liberating human artistic creativity via an intelligent photo retouching agent. arXiv preprint arXiv:25006 / /6 2025b.
Yunlong Lin, Linqing Wang, Kunjie Lin, Zixu Lin, Kaixiong Gong. Wenbo Li, Bin Lin, Zhenxi Li, Shiyi Zhang, Yuyang Peng, et al. Jarvisevo: Towards a self-evolving photo editing agent with synergistic editor-evaluator optimization. arXiv preprint arXiv:2511. 1.23002, 2025c.
11

---

Published as a conference paper at ICLR 2026
Pengyang Ling, Jiazi Bu, Pan Zhang, Xiaoyi Dong, Yuhang Zang, Tong Wu, Huaian Chen, Jiaqi Wang, and Yi Jin. Motionclone: Training-free motion cloning for controllable video generation. arXiv preprint arXhv:2406.05330 2024. URL https: /wwpivaxiiv//..////i//.a./aa/aia org/ htte//bs/240000 05338.
Yaron Lipman, Ricky TQ Chen, Heli Ben- Gen-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.027477 2022.
Hongyu Liu, Xintong Han, Chengbin Jin, Lihui Qian, Huawei Wei, Zhe Lin, Faqiang Wang, Haoye Dong, Yibing Song, Jia Xu, et al. Human motionformer: Transferring human motions with vision transformers. Xiviv preprint arXiv:2302. 1336, 2023.
Parker Liu, Chenxin Li, Zhengxin Li. Yipeng Wu, Wuyang Li, Zhiqin Yang, Zhenyuan Zhang, Yunlong Lin, Sirui Han, and Brandon Feng. Ir3d-bench: Evaluating vision- langugge model scene understanding as agentic inverse rendering. arXiv preprint arXiv:2506.23329, 2025.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint (xXi::2209903000 2022.
Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang. Hanchi Sun, Jianfeng Gao. Lifang He, and Lichao Sun. Sora: A review on background, technology, limitations, and opportumities of large vision models. ArXiv, abs/2402.17177, 2024.
URL https: //t:://aaa /api. semant escholar org/ Corpus ID: 68032569.
Zeqian Long, Mingzhe Zheng, Kunyu Feng, Xinhua Zhang, Hongyu Liu, Harry Yang. Linfeng Zhang, Qifeng Chen, and Yue Ma. Follow-your-shape Shape-aware image editing via trajectory-guis region control. arXiv preprint arXiv:2508.00 08434, 2025.
Yue Ma, Yali Wang, Yue Wu, Ziyu Lyu, Siran Chen, Xiu Li, and Yu Qiao. Visual knowledge graph for human action reasoning in videos. In Proceedings of the 30th ACM International Conference Multimedia, pp. 4132 4141, 2022.
Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In AAAI, 2024a.
Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.
2024b.
Yue Ma. Yingqing He, Hongfa Wang, Andong Wang, Chenyang Qi, Chengfei Cai, Xiu Li, Zhifeng Li, Heung-Yeuu Shum, Wei Liu, et al. Follow-your-click Open-domain regional image animation via short prompts. arXiv preprint arXiv:2403 08268, 2024c.
Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Fine-controllable and expressive
Cai, Heung- Yeung Shum, Wei Liu, et al. freestyle portrait animation. arXiv preprint arXiv:2406.01 0/900, 2024d.
Yue Ma, Kunyu Feng, Zhongyuan Hu, Xinyu Wang, Yucheng Wang, Mingzhe Zheng, Xuanhua He, Chenyang Zhu, Hongyu Liu, Yingqing He, et al. Controllable video generation: A survey, arXiv preprint 150.::25777.1669,, 2025a.
Yue Ma, Kunyu Feng, Xinhua Zhang, Hongyu Liu, David Junhao Zhang, Jinbo Xing, Yinhan Zhang, Ayden Yang, Zeyu Wang, and Qifeng Chen. gallow-yoor-creattte Empowering 4d creation through video inpainting. arXiv preprint arXiv:2506.045590 2025b.
Yue Ma, Yingqing He, Hongfa Wang, Andong Wang. Leqi Shen, Chenyang Qi. Jixuan Ying, Chengfei Cai, Zhifeng Li, Heung- Yeung Shum, et al. Dollow-your-clice Open-domait regional image animation via motion prompts. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 6018-6026, 2025c.
Yue Ma, Yulong Liu, Qiyuan Zhu, Ayden Yang. Kunyu Feng, Xinhua Zhang, Zhifeng Li, Sirui Han, Chenyang Qi. and Qifeng Chen. Follow-your-motion Video motion transfer via efficient spatial-temporal decoupled finetuning. arXiv preprint arXiv:2506.052000 2025d.
12

---

Published as a conference paper at ICLR 2026
Yue Ma, Zexuan Yan, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yen Yeung Shum, et al. cone-contrll,able, and expressive freestyle portrait animation. arXiv preprint arXiv:2509.16630, 2025e.
Towards efficient,
Yue Ma, Zhikai Wang, Tianhao Ren, Mingzhe Zheng, Hongyu Liu, Jiayi Guo, Mark Fong, Yuxuan Xue, Zixiang Zhao, Konrad Schindler, et al. Fastymt: Eliminating redundancy in video motion transfer. arXiv preprint arXiv:2602.0555: 2026.
Federico Perazzi, Jordi Pont-Tuset, Brian Mc WiWiliiam,, and Van Gool et al. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 724-732, 2016.
Alexander Pondaven, Aliaksandr Siarohin, Sergey Tulyakov, Philip Torr, and Fabio Pizzati. Video motion transfer with diffusion transformers. arXiv preprint arXiv:4412.07776 2024. URL
https://arxiv. abrrg//bb/2112 07776.
Alexander Pondaven, Aliaksandr Siarohin, Sergey Tulyakov, Philip Torr, and Fabio Pizzati. Video motion transfer with diffusion transformers. In CVPR, 2025a.
Alexander Pondaven, Aliaksandr Siarohin, Sergey Tulyakov, Philip Torr, and Fabio Pizzati. Video motion transfer with diffusion transformers. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22911-229211 2025b.
Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. TFB: Towards comprehensive and fair benchmarking of time series forecasting methods. In Proc. VLDB Endow. pp. 2363-2377, 2024.
Xiangfei Qiu, Zhe Li, Wanghui Qiu, Shiyan Hu, Lekui Zhou, Xingjian Wu, Zhengyu Li, Chenjuan Guo, Aoying Zhou, Zhenli Sheng, Jilin Hu, Christian S. Jensen, and Bin Yang. Tab: Unified benchmarking of time series anomaly detection methods. In Proc. VLDB Endow.. pp. 2775-2789, 2025a.
Xiangfei Qiu, Xingjian Wu, Hanyin Cheng, Xvyuan Liu, Chenjuan Guo, Jilin Hu, and Bin Yang. Dbloss: Decomposition-bassed loss function for time series forecasting. In NeurlPS, 2025b.
Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo. Jilin Hu, and Bin Yang. DUET: Dual clustering
enhanced multivariate time series forecasting. In SIGKDD, pp. 1185-1196, 2025c.
Xiangfei Qiu, Yuhan Zhu, Zhengyu Li, Hanyin Cheng, Xingjian Wu, Chenjuan Guo, Bin Yang, and Jilin Hu. Dag: A dual causal network for time series forecasting with exogenous variables. arXiv preprint 2509 /4933, 2025d.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Yixuan Ren, Yang Zhou, Jimei Yang. Jing Shi, Difan Liu, Feng Liu, Mingi Kwon, and Abhinav Shrivastava. Customize-a-vided One-shot motion customization of text-to- text-to-video diffusion models. In European Conference on Computer Vision, pp. 332-349. Springer, 2024.
Yutao Shen, Junkun Yuan, Toru Aonishi, Hideki Nakayama, and Yue Ma. Towards preference-aligned image inpainting. arXiv preprint 2xXXi:2509.23088 2025.
Qingyu Shi, Jianzong Wu, Jinbin Bai, Jiangning Zhang. Lu Qi, Xiangtai Li, and Yunhai Tong. Decouple and track: Benchmarking and improving video diffusion transformers for motion transfer. arXiv preprint 2x..v:2500.222... 17350, 2025.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
Yiren Song. Cliptexture: Text-driven texture synthesis. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 3468-5476, 2022.
13

---

Published as a conference paper at ICLR 2026
Yiren Song and Yuxuan Zhang. Clipfont: Text guided vector wordart generation. In BMVC, pp. 543, 2022.
Yiren Song, Xuning Shao, Kang Chen, Weidong Zhang, Zhongliang Jing, and Minzhe Li. Clipyg: Text-guided image manipulation using differentiable vector graphics. In Proceedings the AAAI conference on artificial intelligence, volume 37, pp. 2312-2320, 2023.
Team Wan, Ang Wang, and ct.al. Wan: Open and advanced large-scale video generative models, 2025. URL https: aarxiv.vvv. wwwpxii..orr...oo.ooe www//rrrsss............ 25003 20314.
Ang Wang. Baole Ai, Bin Wen, Chaojie Mao, Chen Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang. Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li. Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang. Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang. Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.203 203 14, 2025a.
Ang Wang. Baole Ai, Bin Wen, Chaojie Mao, Chen- Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. Jianyuan Zeng. Jiayu Wang, Jingfeng Zhang. Jingren Zhou, Jinkai Wang. Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Xiaofeng Meng, Ningying Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Rui Feng. Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Weng, Tong Shen, Wei Lin, Wei Wang. Wei Wang, Wen-Chao Zhou, Wente Wang, Wen Shen, Wenyuan Yu, Xianzhong Shi, Xiaomin Huang, Xin Xu, Yan Kou, Yan- Mei Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhengbin Han, Zhigang Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. ArXiv, a03/2503.20314, 2025b. URLhttpst / /apii. semant ienntiioon icscholare org/ CorpusID: 277321639.
Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Mod- elscope text-to-video technical report. arXiv preprint arXiv:2308. 06571, 2023.
Luozhou Wang, Ziyang Mai, Guibao Shen, Yixun Liang, Xin Tao, Pengfei Wan, Di Zhang, Yijun Li, and Yingcong Chen. Motion inversion for video customization. arXiv preprint 4rX1v:2403.20193, 2024a. URL htw.t:ttt:////cctm arrxiviivvvv//a v. org httgg//b///aa/ 2403 20193.
Qinghe Wang, Baolu Li, Xiaomin Li, Bing Cao, Liqian Ma, Huchuan Lu, and Xu Jia. Characterfactory: Sampling consistent characters with gans for diffusion models. IEEE Transactions on Image Processing, 2025c.
Qinghe Wang, Yawen Luo, Xiaoyu Shi, Xu Jia, Huchuan Lu, Tianfan Xue, Xintao Wang, Pengfei Wan, Zhang, and Kun Gai. Cinemaster: 3d-aware and controllable framework for cinematic text-to-video generation. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pp. 1-10, 2025d.
Qinghe Wang, Xiaoyu Shi, Baolu Li, Weikang Bian, Quande Liu, Huchuan Lu, Xintao Wang, Pengfei Wan, Kun Gai, and Xu Jia. Multishotmaster: A controllable multi-shot video generation framework. arXiv preprint ar arX1v:2512.0304. 2025e.
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Proceedings, 2024b.
Jaazong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Motionbooth: Motion-aware customized text-to-video generation. NeurlPS, 2024.
Zeqi Xiao, Yifan Zhou, Shuai Yang, and Xingang Pan. Video diffusion models are training-fres motion interpreter and controller. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL https: //arxiv.//////////aa//a/aaarmm http//axxvv.....aaa.//// absa//////// 2240 14864.
orgg
14

---

Published as a conference paper at ICLR 2026
Jinbo Xing, Menghan Xia, Yuxin Liu, Yuechen Zhang, Yong Zhang, Yingqing He, Hanyuan Liu, Haoxin Chen, Xiaodong Cun, Xintao Wang. et al. Make-your-vides Customized video generation using textual and structural guidance. IEEE Transactions on Visualization and Computer Graphics, 2024.
Zhen Xiong, Yuqi Li, Chuanguang Yang, Tiao Tan, Zhihong Zhu, Siyuan Li, and Yue Ma. Enhancing image generation fidelity via progressive prompts. arXiv preprint arXi::2501.0777, 2025.
Jiaqi Xu, Xinyi Zou, Kunzhe Huang, Yunkuo Chen, Bo Liu, Mengl.i Cheng, Xing Shi, and Jun long video generation method based on transformer
Huang. Easyanimate: A architecture. arXiv preprint arXiv:2405. 2024.
Xiangpeng Yang, Linchao Zhu, Xiaohan Wang, and Yi Yang. Dgl: Dynamic global-local prompt tuning for texe-video retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 6540-6548, 2024a.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang. Guanyu Feng, Da Yin, Xiaotao Gu, Yuxuan Zhang, Weihan Wang, Yean Cheng, Ting Liu, Bin Xu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with an expert transformer. ArXiv, 0D//408806072 2024b. URL https: s/m.nt.
Danah Yatim, Rafail Fridman, Omer Bar-Tal Tal, Yoni Kasten, and Tali Dekel. Space-time diffusion features for zero-shot text-driven motion transfer. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition, pp. 8466-8476, 2024a.
Danah Yatim, Rafail Fridman, Omer Bar-Tal, Yoni Kasten, and Tali Dekel. Space-time diffusion features for sero-shot text-driven motion transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 8466-8476, 2024b.
Hidir Yesiltepe, Tuna Han Salih Meral, Connor Dunlop, and Pinar Yanardag. Motionshop: Zero- shot motion transfer video diffusion models with mixture of score guidance. arXiv preprint arXiv:24/2.05355 2024. URL https: / aaxiiv..ie org/ abs//b///2/ 2412. 05355.
Jusheng Zhang, Kaitong Cai, Yijia Fan, Ningyuan Liu, and Keze Wang. MAT-agent: Adaptive multi-agent training optimization. The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025a. URL https: http///ppnnneeeeeeeeeeeee htt///ooummmii YDWRTYRRSS..
Jusheng Zhang, Kaitong Cai, Yijia Fan, Jian Wang, and Keze Wang. Cf-vllm:counterfactuall vision- language fine-tuning, 2025b. URL httph::tt///ttt /ttp:///a/vv.//// org/ abs abbb//5506.1255
Jusheng Zhang. Kaitong Cai, Xiaoyang Guo, Sidi Liu, Qinhan Lv, Ruiqi Chen, Jing Yang, Yijia Fan, Xiaofei Sun, Jian Wang, Ziliang Chen, Liang Lin, and Keze Wang. Mm-cot:a benchmark for probing visual chain-of-thought reasoning in multimodal models, 2025c. URL https: // www.xv.org/o/..
2512. 08228.
Jusheng Zhang, Yijia Fan, Wenjun Lin, Ruiqi Chen, Haoyi Jiang, Wenhao Chai, Jian Wang, and Keze Wang. GAM-agent: Game-theoretic and uncertainty-aware collaboration for complex visual reasoning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025d. URL https:://// opwnreview.eeeeeeeeeeeeeeerremem net formm? id=EXKhU5ios8.
Jusheng Zhang, Yijia Fan, Zimo Wen, Jian Wang, and Keze Wang. Tri-MARF: A tri-modal multi- agent responsive framework for comprehensive 3d object annotation. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025e. URL httthhtt///peeeeeeeeeee /ww.reveee..ee.eeeeeeeee.. net/forum: id= YNIow
Jusheng Zhang, Xiaoyang Guo, Kaitong Cai, Qinhan Lv, Yijia Fan, Wenhao Chai, Jian Wang, and Keze Wang. Hybridtoken-vim: Hybrid token compression for vision-language models, 2025f. URL hwtps:tps////...... //axiivvvvv////////aa org/ abs/ 22os/2500 2512. 08240.
Jusheng Zhang. Zimeng Huang, Yijia Fan, Ningyuan Liu, Mingyan Li, Zhuojie Yang, Jiawei Yao, Jian Wang, and Keze Wang. KABB: Knowledge-awand bayesian bandits for dynamic expert coordination in multi-agent systems. In Focood International Conference on Machine Learning, 2025g. URL httpt::ss////peeeeiieeee www.wewww.e.oourn//// 1d=Akvy 4jo..
15

---

Published as a conference paper at ICLR 2026
Yinhan Zhang. Yue Ma, Bingyuan Wang, Qifeng Chen, and Zeyu Wang. Magiccolor: Multi-instance sketch colorization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15205-15217, 2025h.
Yuxin Zhang, Fan Tang, Nisha Huang, Haibin Huang, Chongyang Ma, Weiming Dong, and Chang- sheng Xu. Motioncrafter: One-shot motion customization of diffusion models. arXiv preprini
arXiv:2312.05228, 2023. URL https: //axivv wwtp://iiiorroooooooooooooooeer abs/ 05288.
Min Zhao, Rongzhen Wang. Fan Bao, Chongxuan Li, and Jun Zhu. Controlvideo: Adding conditional control for one shot text-to-video editing. arXiv preprint 1rXiv:2305.77098 2023a.
Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Junhao Zhang. Jiawei Liu, Weijia Wu, Jussi Keppo, and Mike Zheng Shou. Motiondirector: Motion customization of text-to-video diffusion models. arXiv preprint arXiv:2310.08465 2023b.
Wangbo Zhao, Yizeng Han, Jiasheng Tang, Zhikai Li, Yibing Song, Kai Wang, Zhangyang Wang. and Yang You. A stitch in time saves nine: Small vlm is a precise guidance for accelerating large vims. arXiv, 2024.
Wangho Zhao Yizeng Han, Jiasheng Tang, Kai Wang, Yibing Song, Gao Huang, Fan Wang, and Yang You. Dynamic diffusion transformer. 2025.
Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang, Hao Luo, Yibing Song, Gao Huang, Fan Wang, and Yang You. Dydit++: Diffusion transformers with timestep and spatial dynamics for efficient visual generation. IEEE Transactions Pattern Analysis and Machine Intelligence, 2026.
Chen Zhu, Wangbo Zhao, Huiwen Zhang, Yuhao Zhou, Weidong Tang, Shuo Wang, Zhihang Yuan, Yuzhang Shang, Xiaojiang Peng, Kai Wang, et al. Ea-vit: Efficient adaptation for elastic vision transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 103,-1047,2025.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
Dynamic Videos
S1
More details about Implementation
B.1
Implementation details
S1
B.2 Related work
B.3 Human Evaluation
S2
B.4 perrspondinngwiindon loss
S2
C More Comparsons
S4
D More Results
S4
D.I More Visual Results
S4
D.2 More Ablation Results
S4
D.3 More Quantitative Comparison
S4
Experiment Details
SS
Limitations and Potential Social Impact
F11
Limitation
S6
F22 Future work
S7
F3 Potential Social Impact
S7
The Usage of Large Language Models
S8
A DYNAMIC VIDEOS
We show more video motion transfer results produced by our method an MP4 file, which can be found in the file: demo mp4.
B MORE DETAILS ABOUT IMPLEMENTATION
B. IMPLEMENTATION DETAILS
Our sliding-wine window based AMF uses a tile grid of 3 x tokens, a temporal span sy = 5, a local search 1.0. The AMF loss between the reference and
window 1 21 (half-width 10), and temperature current sample is a weighted squared l2 over temporal offsets with linear weights 1.0 + 0.8 across offsets, normalized by the number of offsets and tokens; gradients update only the latent To reduce cost, within the inner loop we recompute Q/K and tile-AMF every interval= 3 steps and reuse the cached gradient on intermediate steps.
B.2 RELATED WORK
Text-fo-video generation. Text-to-video (T2V) generation targets synthesize realistic videos by precisely matching both the visual content and the motion dynamics described the prompt. Early approaches (Zhu al., 2025; Chen et al., 2024; 2023; Guo et al., 2023; Zhang et al., 20, Qiu et al., 2025d;a; Wang et al., 2023; Xiong et al., 2025; Yang et al., 2024a) introduce temporal modules in UNet architectures to generate coherent videos, while diffusion-bacce models (Guo
SI

---

Published as a conference paper at ICLR 2026
Algorithm FastVMT Algorithm
Require: Zrefi reference video latent, Zgen' generating latent
Ensure:
Optimized Zgen
1: function OPTIMIZATIONZ Zgen)
2: Align noise level: z f- MatchNoisef Zyefi Zgen)
3: Do inference: Di Zgenn
4:
Extract self attn features: Igen: Kgen; Gret Keet AttnFeatures
5:
Calculate displacement matrix: D +- CalDisplace(g, k)
6:
Computing loss: L LossFunc( gon, Dret)
genn
7:
Backpropagate and optimize: egen <- Optimization (Zgen)
8:
Output zgen
9:
end function
for 1 to n. do
11:
if < Topt then
12:
Zggn + OPTIMIZATION (Zref; Zgen)
13:
end if
14:
Zyen
(EENOISE(Zgcn)
15:
end for
Table 1: User Study Comparison for State-of-the-Art Video Motion Transfer Methods. The
results show the average rank (1=best, 8=worst) for all the methods; lower is better. Red and Blue denote the best and second best results.
Methodl
User Study
Motion Pres.]
Gen. Qual.
Text Align.
Overall
Training- Free Methods
MOPT (Xiao et al,, 2024)
5.213 5.300
4.088
4.667
MotionClone (1 (Ling al., 2024)
4.688
5.362
5.1
(Yalim al.,
5.129
DiTFlow (Pondaven et al,, 2025a)
4,713
3.200
4.088
4,000
Tuning: Based Methods
Motionlnversion (leong al., 2024h)
5,050
6.350
5.075
5.492
MotionDirecior (Zhao et al., 2023b)
5.325
5.862
5.575
5,588
DeT (Shi al., 2025)
4,350
3.175
3.825
3,783
Ours
2.562
2.400
2.225
al., 2024; Zhao et al., 2023a; Zhang al., 2025h; Ma et al., 2024c) leverage pretrained image diffusion models for temporal consistency. Recently, Diffusion Transformer-based methods exhibit superior performance, with models like Sora (Liu et al., 2024), Cog VideoX (Yang al., 2024b), EasyAnimate (Xu et al., 2024), Hunyuan Video (Kong al., 2024), and Wan2.1 (Wang et al., 2025a) demonstrating the power of scaling transformers for high-quality video generation. However, these models face computational bottlenecks in attention mechanisms and optimization processes, particularly for iterative video editing tasks. Our work identifies task-specific redundancies in video diffusion models to enable efficient motion transfer.
Video
B.3 HUMAN EVALUATION
We conducted user study via a questionnaire comprising distinct input videos spanning 4 categories: camera motion, complex human motion, single object, and multiple objects. Videos were generated using our proposed method alongside other baseline approaches. The user study interface is illustrated in Figures 1 and 2. Owing page constraints, only two generated videos are presented here.
B.4 CORRESPONDING WINDOW LOSS
We compute the read-averaged self-ateention queries and keys from fixed DiT block, denoted by
Q, K e IR Fxxxx x for F frames, an H x W spatial token grid, and channel dimension D. The spatial grid is partitioned into non-overlapping tiles ) P of size (L,,bw) bw), where P. H W For
8n f.
S2

---

Published as a conference paper at ICLR 2026
Video Evaluation Survey
Question 71 Mudipple Objects Ceepry ior nedeo dtods
Fron
pu byck Malonsee bedeeeeeeeee sakpant
003 OOGG
Figure I: Input video, target prompt, and video choices presented in the user study questionnaire. Owing to page constraints, only two videos are shown here.
motion preservatioo
dignmment
generation quality
How well the motion preserved (Tibbrt (wvent)
Ted/prompt all gnmment (1 iest, Noweret
Overall vesual qua By beed, Ni (wooe))
Video
Video
Video
Vidco
Video 3
Video 8
Video 4
Video 4
Video 4
Video
Video 5
Video
Video
Video
Videod
Vidro
Video
Figure 2: User study choices: Participants prompted to rank 8 videos in descending order of preference.
an anchor frame i E {0,..., - 1} and a temporal neighborhood J, = { 1,.., min(i + 1)} with N₁ = [., we define, for each tile p. fixed-size window W C {L...., (1...., W} on the target frame j T, (windowing rule specified in the main paper). The window- averaged key feature is
R(p) 2-
Σ
K5(u,e) ERD,
Kz(u,v) = K(jsu,,,1
[W(P) X-13
(L.))EW())
jEZ, E NoXD The tracking loss penalizes first-order temporal variations of the window means across adjacent target frames and averages over tiles and anchors:
Stacking these per-tile temporal features yields K(")
K (p) E-j
N:
AK(P)(0 (t)
k((r) - k(Pp) t I,..,, .....,,-11..
(() window
Σ
S3

---

0,9899 Published as a conference paper at ICLR 2026
Table Ablation comparison of different attention layers (30 in total). Red and Blue denote the best and second best results, respectively.
Method
Text Sim.
Temp. Cons: 1 Sub. Cons.
Back. Cons.
Acs. Qual.
Motion Smooth.
10-th layer
0.2241
0.7128
0.9730
0.9610
0,9530
0.9736 0,5595 0.5778
15th laurn((Urs)
0.2422
0,7471
0.9865
0.9809
0.9684
0.9891
20-th layer
0.2319
0.9701
0.9791
Table 3: Ablation study about temporal span. Red and Blue denote the best and second best results, respectively.
Temp. Spun
Sub. Cons,
Back. Cons.
Aes. Qual.
Motion Smooth.
span-3
0.9592
0,9461
0.5690
span-5 span-7
0.9809
0,9684
0.9891 0.5778
0,9711
0,9608
0.5522
0,9858
P-1
1
(t)
wwindow
F
window
t=0
Equivalently, writing K(") R. xxx temporal sequence, the inner is the mean L2 norm
of the finite differences K [2] k!dl [1:-]] In practice we compute R((2), +-4 in FP32 before reduction, and the overall guidance objective during latent optimization combines attention motion
flow matching and tracking:
== dam! Lami + Awindow Lwiudow:
with constants Aamf > 0 and Awindow > O.
C MORE COMPARSONS
In Fig. 5, we present additional comparisons to assess the performance of the proposed method. It is clear that previous works exhibit inconsistent motion. In contrast, our approach effectively resolves the issue of motion consistency.
D MORE RESULTS
D.1 MORE VISUAL RESULTS
We presented more visualizations in Figure 3 and where each reference video is paired with two distinct motion transferred videos. In particular, Fig. 7 presents two challenging visual cases: one featuring complex object motion and another involving complex camera motion. The first set of images shows an astronaut doing front flip off the deck into the water. The second set of images illustrates a complex camera move, where the viewpoint rises rapidly from ground level and then pushes in for close-up of the subject.
D.2 MORE ABLATION RESULTS
We conduct an ablation study to evaluate the impact of selecting different attention layers for motion extraction. As illustrated in Table our choice of the middle attention layer achieves the best performance in motion transfer.
We evaluated our ablation samples on Vbench metrics. The results are presented in Table 4.
D.3 MORE QUANTITATIVE COMPARISON
We further conducted experiment using MTBench. Our quantitative results shown in Table ??.
S4

---

Published as a conference paper at ICLR 2026
Ref video
An athlete is running in park
A lunar rover is ascending small kill
Ref Output Output video
wolf running forest
A tiger emerged from wilderness
panda walking on playground
tiger running hill
Output video
A spaceship is driving to ironman
A horse-drawe carriage goes rural path
Output video
Figure 3: More visual results. We provide more visual results to evaluate the performance.
Ref
Firefighter is running road
gorilla doing exercises
Output Output video
knight running eastle
doing exercises
Ref video
robot surfing
near owner
Output video
A horse surfing
An alpaca lies down its owner
Output video
Figure 4: More visual results. We provide more visual results evaluate the performance.
E EXPERIMENT DETAILS
All experiments presented in this study conducted utilizing NVIDIA A100-80GB GPUs for fair comparison. The reference videos were carefully curated from publicly available sources on the internet, ensuring diverse and representative dataset for evaluation purposes.
S5

---

Published as a conference paper at ICLR 2026
Table 4: Vbench metries evaluated of the ablation samples. Red and Blue denote the best and second best results, respectively.
Method
Sub.
Qual.
wlo Sliding Wind.
0,9686
0.9437
0.5628
0.9667
wio Cor. Loss
0.9753
0.9574
0.5629
0.9705
wro Step Skip.
0,9852
0.9617
0.5623
0.9887
Ours
0.9809
0.9684
0,5778
0.9891
Input
MOFT
MotronBirector
MotionInversion Motian
Ditflow
Ours
Figure 5: More qualitative comparison with baselines. We provide more visual comparison with various baselines using various kinds of motions. Our method demonstrates superior performance across a range of motion types.
F LIMITATIONS AND POTENTIAL SOCIAL IMPACT
F.1 LIMITATION
As observed in prior work (Pondaven et al., 2025a), existing frameworks are still bounded by the capacity of the pre-trained video backbone, making it challenging to handle out-of-distribution prompts motions. For instance, highly complex human actions (such as Thomas Flair) remain particularly difficult. When the generated video content and the conditioning prompt exhibit semantic inconsistency conflict, the quality of motion transfer can degrade significantly, often leading to unsatisfactory or unstable results.
A Spider-Man Man and fron man riding motorcycles
Two cars roller skating in outdoor urban
Figure 6: Failure case of proposed. Even though our method achieves good performance, still have a challenge when handling the occlusion. This failure can be mitigated by a powerful video diffusion model in the future.
S6

---

Published as a conference paper at ICLR 2026
An astronaut is doing front flip into the water.
detective is standing at the train station.
Figure 7: Complex cases of proposed. Our method also performs well in some cases with complex object motion and camera movement.
The pairwise design adopted by AME, while beneficial for capturing motion correspondences, inevitably introduces higher memory consumption compared to prior methods. Although this overhead does not critically affect short video synthesis, it may pose practical challenges when scaling long video generation. Looking ahead, we believe that this issue can be mitigated through systematic engineering optimizations, such as more efficient memory management strategies, model compression, hierarchical generation schemes.
F.2 FUTURE WORK
An important future direction is to extend our framework toward an agent-based paradigm for practical application (Lin et al., 2025a; Wang et al., 2025e;d;e; Lin al., 2025b;c; Liu et al., 2025; Zhao et al., 2024; 2026; 2025; Song, 2022; Song et al., 2023; Song & Zhang, 2022; Qiu et al., 2024; Chen et al., 2025b;a). Rather than formulating motion transfer as a single-pass motion-conditioned generation problem, we envision multi-agent architecture in which specialised agents collaboratively handle motion decomposition, structural alignment, identity preservation, and temporal stabilisation. Specifically, a Motion Parsing Agent could extract structured motion representations (c.g., skeletal trajectories, optical flow fields, or 3D pose sequences) from the driving signal; a Structure Alignment Agent could enforce geometric consistency between the source character and target motion manifold; an Appearance Preservation Agent could maintain identity-specific attributes under large pose deformation; and Temporal Consistency Agent could suppress drift and flickering across frames. Through iterative reasoning and tool invocation (e.g., pose estimation, tracking, depth inference, or diffusion-basee motion refinement modules), these agents could decompose complex motion transfer objectives into hierarchically organised sub-tasks and dynamically refine intermediate states. Such formulation enables adaptive long- long-horizon motion synthesis, mitigates cumulative structural distortion, and improves controllability under ambiguous or partially specified motion prompts.
F.3 POTENTIAL SOCIAL IMPACT
The potential social impact of FastVMT and efficient video motion transfer technologies is far- reaching, with applications spanning across multiple industries. In the entertainment sector, partic- ularly in film, gaming, and digital content creation, the ability to quickly and accurately transfer motion from one video sequence another enables faster production cycles and more dynamic storytelling, reducing costs and enhancing creativity. This could democratize high-quality video production, making accessible to smaller studios and independent creators who previously lacked the resources to produce complex motion sequences.
In the advertising industry, FastVMT offers new opportunities for creating personalized and engaging content. Brands can easily adapt their campaigns to various demographics by transferring motion from diverse sources, ensuring relevance and resonance with their audience. Additionally, this technology could be employed for real-time video adaptation in interactive applications, further improving customer experiences.
Beyond media and entertainment, the technology also holds promise in education, remote work, and healthcare. Virtual simulations and immersive training environments could benefit from enhanced motion transfer capabilities, allowing for realistic and adaptable scenarios. This could support remote learning, telemedicine, and virtual conferences, making such interactions more engaging and effective.
S7

---

Published as a conference paper at ICLR 2026
Overall, FastVMT's ability reduce computation costs and improve video synthesis efficiency drive widespread innovation, making advanced video manipulation more accessible, affordable, and impactful across various sectors, ultimately shaping the future of digital media and interaction.
G THE USAGE OF LARGE LANGUAGE MODELS
In this paper, the usage of the LLM mainly falls into the following aspects:
Grammar checking and format optimization: In the paragraphs of the paper, LLMs are used for grammar enrr checking and format checking of charts and graphs.
Language polishing: The text description part of the paper LLMs to polish and optimize the language expression.
All authors are responsible for the content generated by the LLMs.
S8


Reviews:
[{"review_id": "XTuVV08vUT", "reviewer": "Reviewer_sSov", "summary": "The paper focuses on the inefficiency issue of motion transfer in the DiT architecture and proposes a method to improve it without sacrificing quality. Specifically, the authors point out that motion redundancy arises from the neglect of motion smoothness across frames, while gradient redundancy occurs due to ignoring the slow gradient changes along the diffusion trajectory. Accordingly, they propose a sliding-window strategy that operates on downsampled attention maps and a step-skipping gradient computation strategy, which together enhance the efficiency of motion transfer.", "questions": "In the ablation study, adding the step-skipping strategy brings improvements in certain metrics such as aesthetics or text–frame similarity. However, I think the operation of reusing previous gradients at specific timesteps is essentially an approximation. Even if it does not cause a performance drop, it theoretically should not lead to improvement. Could the authors provide some explanation for this phenomenon?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well written, clearly motivated, and easy to follow.\n2. The proposed efficiency improvement strategies, involving the sliding-window and step-skipping gradient optimization, make sense to me, and the illustration of their rationale in the method section is intuitive.\n3. The experimental results demonstrate that the proposed method maintains video quality while achieving notable speedup.", "weaknesses": "1. Evaluation dataset. The authors use 50 videos selected from the DAVIS dataset, which is rather small in scale and may not cover sufficient scene and motion diversity. I notice that benchmarks used in different motion transfer papers vary—perhaps the authors follow the test set of DiTFlow? However, how does the proposed method perform on other test sets used in related works? For example, please refer to Table 1 of DeT.\n\n2. Implementation details. The authors mention that “for fair comparison, they adapt Wan-2.1 as the same backbone.” Is the 14B model or the 1.3B model used? Previous works adopt different backbones such as CogVideoX and Hunyuan. Are the authors reimplementing these methods using the Wan2.1 model? If so, there may be a risk that some methods cannot perform optimally, as they can be sensitive to hyperparameters. Overall, it would be helpful if the authors could provide more details about how each baseline is implemented.\n\n3. It would be beneficial if the authors could include and analyze some typical failure cases of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VQV6SmQ50y", "reviewer": "Reviewer_TXvo", "summary": "This paper addresses the inference speed bottleneck of Video Motion Transfer (VMT) that uses training-free Attention Motion Flow. Specifically, it employs,\n\n(1) Sliding-window motion extraction, which assumes that motion correspondences between frames are local\n\n(2) Step-skipping gradient optimization (reusing cached gradients), based on the observation that gradients across consecutive inner optimization steps are highly similar\n\nThe paper claims their proposed method achieves an average 3.43× speed-up without compromising visual fidelity or temporal consistency.", "questions": "- In Table 2, are those baselines re-implemented by the authors using the WAN-2.1 backbone, or are these numbers quoted from the original papers (potentially with different backbones)?\n- In Table 2, it looks like \"Ours\" is included under Tuning-Based Methods. Is that intentional?\n- When is the window center re-estimated along the diffusion trajectory? Fixed at (t=0), or updated progressively during the first 20% of guided steps?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses a practical problem such as inference speed improvement in training-free motion transfer\n- The proposed method also maintains (or even improves) the quality of the video\n- Clearly identifies and analyzes existing problems such as motion and gradient redundancy through experiments", "weaknesses": "- The sliding-window strategy rests on the assumption that inter-frame motion is local and small. The paper lacks analysis or discussion of performance limitation when this assumption breaks (e.g., very fast and large motions, aggressive camera movements, occlusions).\n- In Table 2, FastVMT uses WAN-2.1 as the base model, while other baselines may rely on different backbones. The paper states “fair backbone: WAN-2.1,” but it is unclear whether this means all baselines were re-implemented and re-evaluated on WAN-2.1, or merely that FastVMT used WAN-2.1. If the former, the results are very compelling. Otherwise, the quality advantage might partly stem from the newer backbone.\n- The core acceleration idea of step-skipping is a fairly common design principle in other areas, so its novelty is somewhat limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FzWDkXGMqE", "reviewer": "Reviewer_RmCa", "summary": "This paper introduces FastVMT, a training-free video motion transfer framework aiming at enhancing computational efficiency without sacrificing video quality or motion fidelity. To achieve this, FastVMT includes two key innovations: 1) sliding-window motion extraction which only computes attention with local spatial neighborhoods to better capture motion correspondences while eliminating unnecessary token interactions; 2) step-skipping gradient optimization that reuses gradients across optimization steps as the authors found that gradients change slowly along the diffusion trajectory. Experiments show that FastVMT yield an average 3.43× speedup and up to 14.9× lower latency compared to prior state-of-the-art methods (e.g., MotionDirector, DiTFlow, MOFT) without visible quality degradation.", "questions": "- How do you handle the irregular tiling that comes from the sliding window attention mechanism?\n- Does the model use Flash Attention or other memory-efficient attention operations? If not, what is the runtime or memory overhead when switching to other attention functions?\n- Could the authors provide the video outputs with longer durations or more complex motion (including both object motion and camera movement)?\n- Since GPU memory consumption is also an important metric to evaluate the efficiency, could the authors also provide the report of memory usage?\n- Could the authors include an ablation study for key hyperparameters, such as the window size, stride, temporal span, and gradient skip interval?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The proposed method is elegant and well-motivated. For example, both the ideas of sliding window motion extraction and step-skipping gradient optimization are lightweight but sounds effective and should work smoothly with existing video diffusion models. To introduce these two modules, the authors provide an insightful analysis (i.e, Figure 2), showing that motion is locally consistent and the gradient in consecutive steps are mostly similar.\n- The efficiency improvement is significantly. Specifically, the authors demonstrate 3.43× speedup and up to 14.9× lower latency compared to the existing models, which are impressive.\n- The authors provide comprehensive comparisons with the up to 7 existing models (including both training-free and finetuning-based baselines).\n- The authors also provide extensive ablation studies, which clearly show the necessity of each proposed component. The results look promising.\n- The paper is well written and easy to follow. The figures are well-plotted and informative which can make readers quickly understand the core ideas.", "weaknesses": "My first two concerns circle around the usage of sliding window attention:\n\n- First, the usage of sliding-window attention leads to irregular and non-contiguous tiling (i.e., the attention mask is no longer a uniform square but contains irregular zero-paddings to mask out non-local tokens). This irregularity would make the model incompatible with Flash Attention that requires full or casual mask. How did the authors handle this issue? If the model does not use Flash Attention, did the authors notice a speed degradation when switching to other attention functions?\n- Second, the Register Tokens paper [1] found that the transformer models tend to learn a few of register tokens which are attached by most of the tokens and used to aggregate and spread global information. However, the proposed sliding-window design restrict the receptive field of a token to its local neighborhood and further block such information exchange mechanism. This could make the model fail to handle the videos with larger dynamics or longer duration where the global information exchange is critical. However, all the video examples provided in the paper and the supplementary material are 5 seconds and only include smooth and slight motion. Could the authors provide the videos with longer sequences and larger motion to evaluate whether the proposed model can handle such cases?\n\nOther concerns:\n- The paper is lack of the report of GPU memory consumption. Since the sliding-window attention and AMF modules require storing multiple latent tensors, attention maps, and cached gradients, this may increase GPU memory usage (especially for long or high-resolution videos). However, the authors only provide runtime speedups without the profiling of memory consumption.\n- The ablation on the selection of some important hyperparameter selection is also missing. For example, the window size and the gradient skip intervals could largely affect the balance / trade-off between visual quality and runtime. Could the authors also provide such ablation?\n\n[1] \"Vision Transformers Need Registers\", ICLR 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)