Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuning
Authors: Chenjian Gao
Lihe Ding
Xin Cai
Zhanpeng Huang
Zibin Wang
Tianfan Xue
Venue: ICLR 2026 Poster (2026)
Keywords: Video Editing
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: The paper introduces a mask-based LoRA tuning method for highly flexible video editing using the pre-trained Image-to-Video model.

Abstract: Video editing using diffusion models has achieved remarkable results in generating high-quality edits for videos. However, current methods often rely on large-scale pretraining, limiting flexibility for specific edits. First-frame-guided editing provides control over the first frame, but lacks fine-grained control over the edit's subsequent temporal evolution. To address this, we propose a mask-based LoRA (Low-Rank Adaptation) tuning method that adapts pretrained Image-to-Video models for flexible video editing.
Our key innovation is using a spatiotemporal mask to strategically guide the LoRA fine-tuning process. This teaches the model two distinct skills: first, to interpret the mask as a command to either preserve content from the source video or generate new content in designated regions. Second, for these generated regions, LoRA learns to synthesize either temporally consistent motion inherited from the video or novel appearances guided by user-provided reference frames.
This dual-capability LoRA grants users control over the edit's entire temporal evolution, allowing complex transformations like an object rotating or a flower blooming. Experimental results show our method achieves superior video editing performance compared to baseline methods. The code and video results are available at our project website: https://cjeen.github.io/LoRAEdit.

Published as a conference paper at ICLR 2026
CONTROLLABLE FIRST-FRAMEGGUIDED VIDEO EDITING VIA MASK-AWARD LORA
Chenjian Gao', Lihe Ding', Xin Cai', Zhanpeng Huang ? Zibin Wang', Tianfan Xue 'Multimedia Laboratory. The Chinese University of Hong Kong; "Sense Research Shhnnhaii AT Laboratory; 'CPII under InnoHK (gc025, d1023, cx023, (xxuceeoo cubk. edu. .k; (soaree. huang, wangzb02 gemaal..@@@mmmmmmmmmmmmmm..11 com
ABSTRACT
Video editing using diffusion models has achieved remarkable results in generat- ing high-quality edits for videos. However, current methods often rely on large- scale pretraining, limiting flexibility for specific edits. First-frame guided editing provides control over the first frame, but lacks fine-gra ane-grained control over the edit's s subsequent temporal evolution. To address this, we propose mask-based LoRA (Low-Rank Adaptation) tuning method that adapts pretrained Image-to- Video models for flexible video editing. Our key innovation is using a spatiotemporal mask to strategically guide the LoRA fine-tuning process. This teaches the model two distinct skills: first, to interpret the mask as command to either preserve con- tent from the source video or generate new content in designated regions. Second, for these generated regions, LoRA learns to synthesize either temporally consis- tent motion inherited from the video or novel appearances guided by user-provided reference frames. This dual-capability LoRA grants users control over the edit's entire temporal evolution, allowing complex transformations like an object rotat- ing or a flower blooming. Experimental results show our method achieves superior video editing performance compared to baseline methods. The code and video re- sults are available at our project website: https://cjee..githu ooRooo/ooooAiiii
frame-guided
INTRODUCTION
Recent advances in diffusion models (Rombach et al. 2022 Lipman et al. 2023) have demonstrated
unprecedented improvement in high- high-guality video generation Yang al. 2025b Kong al.) 2024
Wang et al., 2025; HaCohen et al. 2024]. Based on foundational video generation models, video editing has experienced dramatic improvement (Jiang al. 2025 Hu et al. (025), now finding wide application in scientific and creative fields (Yang et al. 2024). Still, these video editing models often require computationally intensive finetuning, with large set of training data. This makes them expensive to extend to a new editing type, and less flexible for new applications. In contrast, first- frame-guided video editing (Ouyang al., 2024 Ku et al. 2024) offers a promising path toward flexible video manipulation. In this paradigm, users can edit the first frame arbitrarily, either using image AI tools or traditional editing software. These edits are then propagated the entire sequence, enabling flexible video manipulation without being constrained by dataset-specific training.
While guuiddd solutions allow flexible editing, they only provide limited control of re- maining frames. For instance, given a video of a blooming flower, the user can edit the flower in the first frame, but cannot control how the flower blooms in the following frames. Similarly, when an object rotates to a novel viewpoint, the user cannot control the disoccluded region. In addition, the first frame edits may diffuse into unedited regions, resulting in undesirable background changes. The inability to control later frames limits editing flexibility and necessitates methods that not only retain the flexibility of frst-frame-Bee trst-frame-guided editing, but support control throughout the video.
A simple solution is per-video finetuning of pre-trained image-to-video (12V) model (Kong al. 2024 Wang et al. 2025). By finetuning the model using LoRA (Hu et al. 2022 on a source video,
*Corresponding authors.

---

Published as a conference paper at ICLR 2026
Edited Region
Source Video
Edited Region
Edited Frame
First Frame Guided Editing
Edited Region
Edited Additional Frame
Guided with Additional Edited Frame
Figure I:
Given source video (top row), we achieve high-quality video editing guided by the first frame as a reference image (middle row), while maintaining flexibility for incorporating additional reference conditions (bottom row).
the model will learn content motion. This allows the edit propagate in temporally consistent way. However, this naive approach lacks finer control it cannot distinguish between regions that should change and those that should stay, nor does ensure the edited region's appearance remains controllable as it moves and deforms over time, requiring the synthesis of unseen appearance.
In this work, we build a flexible video editing model by expanding this naive edit propagation ap- proach with an additional mask, which controls which regions of the video remain unchanged and which are modified. Recent 12V models (Kong et al., 2024 Wang et al. 2025) are designed to gen- erate videos from single image, but they can also process video sequences, with built-in masking mechanism to control which regions are preserved or modified during inference. Typically, this only temporal mask, which preserves the first frame while generating the subsequent frames. However, we observe the mask has greater potential for more precise spatial control over video con- tent. To leverage this, apply LoRA fine-tune the model on the input video with the edited region masked. This allows the model to learn how to interpret flexible spatiotemporal mask as command to preserve content from the source video or generate new content designated regions. After LoRA training, the model can effectively apply the mask, leaving unedited areas unchanged.
More importantly, our mask-guided control empowers LoRA learn selectively, adapting to the specific demands of the edit. This flexibility is illustrated by the blooming flower example in Fig, [ First, by configuring the mask for motion learning, LoRA learns the flower's blooming motion from the source video (second row). Then, to control the flower's appearance reconfigured for appearance learning, directing LoRA capture the target appearance from an additional reference, such as the bloomed, vibrant red rose (third row). This dual capability allows our method to synthesize a controllable transformation. creating a video where the flower not only moves correctly but evolves into the desired state, feat unattainable with naive first-frame guidance.
it blooms, the mask
Our approach offers a simple and effective solution for video editing by leveraging LoRA's capabil- ities, without modifying the model architecture, and maintaining high flexibility through the combi- nation of different conditions. Experimental results demonstrate that our method achieves superior performance state-of-the-an approaches in both qualitative and quantitative evaluations.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORKS
Video Editing with Diffusion Models. The success of video diffusion models has spurred extensive research into video editing. Early works adapt the image diffusion network and training paradigm to video generation and editing. A-Viee- Wu al. (2023) explores the concept of one-shot tuning in video editing. Fairy Wu et al. (2024) edits keyframes utilizzig a 3D spatio-temporal self-attention extended from T2I diffusion model. VidToMe Li al. (2024) introduces image editing approaches (e.g.. ControlNet Zhang et al.] (2023) to video generation. Animatediff Guo al. (202)) decouples the appearance and motion learning during video editing. SAVE SSng et al. (2024) chooses to fine-tune the feature embeddings that directly reflect semantic information. An- other line of work manipulates the hidden features to edit a video. Video-P2P Pireo-PPP[iiuu Liu et al. (2024) and Vid2Vid-Zero Wang et al. (2023) employ cross-attention map injection and null-text inversion for video editing. TooenFlow Geyer et al. (2023) leverages motion-based feature injection, and FLAT- TEN Cong et al. (2023) further introduces optical flow for better injection. Other methods Chen et al. (2023): Yang et al. (2023) explore latent initialization and latent transition in video diffusion models. Dragvideo Deng al. (2024) achieves interactive drag-style video editing by introducing point conditioning. Recently, VACE Jiang et al. (2025) has shown promising video editing ability by large-scale conditional video diffusion training. Although large video editing diffusion models achieve impressive results, they often struggle with inaccurate identity preservation and suboptimal performance on out-of-domain test cases. In contrast, our method effectively leverages powerful video priors while efficiently learning content from both the reference image and the source video.
First-Frame Guided Video Editing. First-frame guided editing has emerged as a mainstream video editing approach, with V2V Ku et al. (2024) and 12VEdit Ouyang 22yaang et al. (2024) as representative methods. These approaches decompose video editing into two stages: (i) editing the first frame using existing image methods, and (ii) propagating edits to remaining frames using monitn-conditioned image-to-vide video diffusion models. Any V2V reconstructs motion via DDIM sampling, injecting tem- poral attention and spatial features from the original video. V2V enhances this by learning coarse motion through per-clip LoRA and refining appearance using attention difference masks. While this decoupled framework benefits from advances both image editing and video generation, the lack of explicit constraints often leads to diluted edits during propagation, manifesting as foreground inconsistencies and background leakages.
3 METHOD
In this work, we introduce a controllable fist-framm-guided video editing method based on recent image-to-video diffusion models (Wang et al.) 2025 Kong et al., 2024). In Sec. 3.1 we first tackle the issue of maintaining coherent motion of the edit by using LoRA to transfer motion patterns from the input video. In Sec. 3.2] we explore the generalization capabilities of the based conditioning mechanism in pretrained I2V models. In Sec. Sec. we demonstrate how mask-aware LoRA enables flexible video editing by leveraging the mask to control the generated content.
3.1 LORA'S FIRST STEP: A SIMPLE SOLUTION FOR VIDEO EDITING
In this section, we introduce a naive approach for edit propagation, which serves as a foundation for the subsequent improvements. Given input video Vinput 12. ... II] and an edited version of the first frame, i, the goal is generate an edited video Veditdd [I, Ig.... .. Ir] where the edits introduced in I are propagated across all subsequent frames with coherent motion.
To achieve this basic objective, we insert LoRA (Hu et al. 2022) modules dy into the self-attention and cross-attention layers of the I2V model Wang et al. (025)) and optimize them on the input video V iinput to capture its motion pattern. During training, the model is conditioned on the original first frame I and a textual prompt composed of a fixed special token p" concatenated with the caption generated for I using Florence-2 (Xiao et al. 2024) (i.e., [p"] e). The model is supervised to reconstruct the full input video Vinpant = 11, .2.... ... Ir). Following the flow matching objective of the I2V model (Wang et al. 2025). optimize the LoRA adapters by minimizing the error between
3

---

Published as a conference paper at ICLR 2026
the network velocity prediction and the target flow drift:
Loow = x0, [llva(* X,, 1 [p") + (xo x1)1l
(1)
condition
x 11=1) Lxo
where Xu ~ IO, represents the sampled Gaussian noise, and E(V impu) denotes the latent representation of the source video encoded by the VAE & Here, by is the velocity prediction network with LoRA parameters de, and /. [0, 1] is the time step. The term (xo xi) serves the flow target, guiding the interpolation from data X to noise Xo.
At inference time, the original frame I is replaced with an edited version I, and a new caption è is generated for I, using Florence-2. The prompt token p" is concatenated with è to form the inference prompt [p*] + č, which guides the generation of the edited sequence 17.
3.2 THE MASK'S S HIDDEN POWER: EXPLORING 12V MODEL CAPABILITIES
The naive edit propagation introduced above only ensures motion coherence and lacks control over the content of subsequent frames. To address this, we leverage the conditioning mechanisms in recent 12V models (Wang et al. 2025; Kong et al. 2024). Specifically, to introduce the first frame as the guidance for video generation, these models incorporate two additional conditions for the denoising network: pseudo-video V Vound and a binary spatiotemporal mask Meodd- The pseudo- video V aond E I zero-plase sero-plachhoddr frames. The binary mask Meond E {0. 1] [1xT'xxxx is designed so that indicates the preserved frame and 0 represents the frames to be generated, with the first frame set to 1 and all subsequent frames set to O.
is constructed by concatenating the first frame I E xxx1x XW with
We extend this paradigm to video-to-videe generation by replacing the pscudo video condition V with actual video frames, enabling the model to accept an entire video sequence as input. In this setting, the binary spatiotemporal mask Maadd originally designed to preserve only the first frame, can now be repurposed as more flexible mechanism that selectively controls which regions are retained and which are regenerated across space and time.
To assess the generalization capabilities of the masking mechanism, we evaluate several bi- nary mask configurations, as shown in Fig. 2 In each case, the mask Meunt is applied to the input video to construct V where regions marked as zero are regenerated and the rest are preserved. The default 12V configuration pre- serves only the first frame and leads the model to synthesize motion across the entire sequence (first row). Exploring the extremes, an all-zeros of the original content
Mask
Pseudo Video
mask that preserves forces the model to generate the appearance for the entire video (second row). Conversely, an all-ones mask aimed at preserving all content effectively maintains the video's overall structure but introduces artifacts in areas with discontinu- ous motion (third row). Finally, using spatially varying mask to preserve the background while generating the foreground reveals key challenge, the model struggles synthesize coherent foreground content (fourth row).
Figure 2:
Exploring different mask confi configurations input condition toage-to-video model.
Analysis and Motivation. frame instructions but fails at nuanced, selective editing. Our key insight is to repurpose this mech- anism, enabling the spatiotemporal mask to serve a dual purpose during LoRA tuning. First, we can use LoRA to reinforce the model's response to the mask, improving its ability to execute the preservation and generation commands defined by the mask. More critically, we can use the mask to direct what LoRA learns. By masking different content during training, we can guide LoRA to focus on either the video's 's underlying motion or a reference's 's target appearance. This interplay between LoRA and the mask is the cornerstone of our method, detailed in the following section.
The preceding cases show the raw I2V model can handle simple, full-
4

---

Published as a conference paper at ICLR 2026
Disentangling Edits Background
Appearance Control in Propagated Edits
LAKEA
Laits
Training Target
Condition
Condition
Training Target
Inference
Results
Condition
Figure 3: Our mask-guided LoRA pipeline. Training (Top): LoRA is fine-tuned learn motion from the masked source video (left) and appearance from reference frame (right). Inference (Bottom): The trained LoRA applies the learned motion and appearance to an edited first-frame, producing a temporally consistent video.
3.3 UNLOCKING EDITING FLEXIBILITY: MASK-GUIDED LORA
Building on this exploration, we modify the spatiotemporal mask to enable more flexible video edits. Combined with LoRA fine-tuning, the mask serves two complementary roles. First, it improves the 12V model's alignment with mask constraints, allowing users to limited editing to a specific region, achieving more flexible control. Second, it also acts as signal guiding LoRA learn specific patterns from the training input, such as motion pattern from video sequences or appearance of object from images. Specifically, adapt the flow matching loss in Eq. 1 to incorporate the conditioning video and mask:
C Ex,xo.x1 [Ilvo(Xe, V Vennd, M [p"] + - (Xo - x1)
x (1 t)x1 txo
where X1 = (VV tarpet) represents the latent of the target video, and X denotes the sampled noise. As shown in Fig. by configuring V M and V Vurget in different ways, enable flexible video editing through LoRA, detailed in the following sections.
3.3.1 DISENTANGLING EDITS AND BACKGROUND
Many first-frame edits alter only a part of the frame Ku al. 2024 Ouyang et al., 2024), creating conflict between two demands: the edited region must evolve, while the background must remain static. When a single generative pathway handles both, they may collide. Preserving the background can stall the edit, while propagating the edit can cause unintended background changes.
To achieve separate on control on edited regions and non-edit background, we carefully adjust the spatiotemporal mask Moond and the conditioning video V Voond during LoRA fine-tuning. The mask Meond set to ones for the first frame to preserve as the reference, and for subsequent frames, Moond is adjusted to mark unedited regions with ones (to be preserved) and edited regions with zeros (to be generated). The pseudo-vided Voond is created by applying the mask to the input video, setting the regions marked V iarget set to the input video during LoRA fine-tuning. This configuration allows the model to focus generating the edited content while locking the unedited regions. At inference time, when editing the first frame (replacing I with Í,), we the Meood during LoRA training, while V Vond has its first frame replaced by the edited version İ
in M to be empty. while leaving the rest unchanged. The objective

---

Published as a conference paper at ICLR 2026
One interesting observation is that while pre-trained 12V model struggles with selective editing, LoRA training on a single video alone learns effective mask-guided inpainting priors. We speculate this is due to the diffusion transformer processing inputs discretized tokens, with spatially varying mask sharing a similar token-level representation, making the adaptation straightfoward.
3.3.2 APPEARANCE CONTROL PROPAGATED EDITS
An edit in the first frame rarely stays static: the modified region may rotate, deform, or follow its own motion trajectory (e.g.. petals unfolding natural, the model has to infer how the edited region should appear under these evolving viewpoints and states. When the only constraint is the first frame itself, this inference is under-specified, and the edit drifts away from the user's intent. To address this, we allow users to edit any subsequent frame, providing direct guidance for how the appearance should look at specific points in time.
flower blooms). To make the subsequent frames look
During LoRA fine-tuning, we use an edited frame as the target V target- The conditioning input Vood constructed using the pre-edited frame by masking out the edited regions. The mask Moond marks the preserved background areas with ones and the edited regions with zeros. If multiple edited frames are used, we decouple appearance from motion by training on each frame as an isolated, static image, preventing the model from inferring false temporal dynamics between them. This configuration allows the model to learn how edited content should appear in context, guided by both the surrounding background and the user-provided modification.
Unlike methods that directly feed edited frames as inputs during inference (Yang et al. 2025a Jamriska 2018), we do not require the edited frame to remain the same during inference. Instead, the edited frame is used only during training to guide how edits should appear. At inference time, the model generates content based on learned patterns and context, allowing it to adapt edits smoothly across frames, even when the edits do not adhere to strict temporal alignment.
4 EXPERIMENTS
4.1
IMPLEMENTATION DETAILS
We conduct our experiments using videos consisting of 49 frames, with resolution of either 832 x 480 or 480 x 832. All main results are obtained using the Wan2. 1-12V 480P model. Ad- ditional results based on ViuyoonnVideo-12) are included in App- F Our framework is built upon the publicly available diffusion-pipe codebasq"] Details regarding our automated mask acquisition workflow are provided in App. [A] For each video editing sample, we begin by training on the input video for 100 steps described in Sec. B.3.I If additional edits are applied to later frames, continue training for another 100 steps on data that includes additional modifications as described in Sec. 3.3.2 This helps the model incorporate user-specified appearance changes. We learning rate of x 10-4 for all experiments. Training on 49-frame videos typically requires 20 GB of GPU memory. In App. C we describe a strategy that reduces GPU requirements.
4.2 COMPARISON WITH OF-EE--F-THEAARTTE
Comparison with Referedce-Guuided Video Editing. We compare method with two recent reference-guided video editing approaches: Kling1.6 (IlingA1 and VACE (Jiang et al. 2025). To evaluate the performance reference-guided video editing, clips from Pexels and YouTuube Each video is paired with a reference image representing the desired edit. We ACE++ (Mao et al. (025)) apply the edit to the first frame for our method. Figure 4 shows visual comparison results. Compared to Klingl.6 and VACE, our method better respects the intended appearance in the edited region while preserving background content and temporal consistency.
collect 20 high-quality video
Comparison with cent waraa--ggiideeee video editing approaches, including 12VEdit (Ouyang et al. (2024). Go-with- the-Flow (Burgert et al. 2025), and Any (Ku et al. 2024). All baselines take the edited first frame input and attempt to propagate the edits through the entire sequence. To ensure a fair
Video Editing. We further compare our method with re-

---

Published as a conference paper at ICLR 2026
Input
Ours
Kling 1.6
VACE
Figure 4: Comparisons with state-of-the-art reference-guided guided video editing methods.
and consistent evaluation, we adopt the test set from 12VEdit, which contains videos from diverse sources along with paired first-frame edits. Figure shows qualitative results. In the portrait ex- ample (left), our method accurately adds the necklace while preserving the facial structure, while baseline methods often distort the face or produce artifacts. In the street scene (right), our approach transfers the clothing style cleanly across frames without affecting the background, whereas baseline methods distort the clothing or introduce changes in unedited areas.
Quantitative Results. For quantitative evaluation on first-fran fisttframe-esn frame-guided video editing, we use three metrics: 1) DeQA Score You et al.) (2025), a state-of-the-art method for assessing image quality; 2) CLIP Score, which measures the semantic alignment between generated frames and edited first frame by comparing their CLIP Radford et al. 12021) embedding similarity; and 3) Input Similarity, which computes the CLIP embedding similarity between the generated frames and the input frames on per-frame basis. As shown in Tab. [ our method outperforms others across all metries. For quantitative evaluation on reference-guiee guidedd video editing, we conducted a user study with 35 partic- ipants. Each participant randomly shown 10 groups of results generated by different methods.

---

Published as a conference paper at ICLR 2026
First Frame
Source Frame
Ours
12VEdit
Go weiith-the-iie the flow
AnyV2V
Figure 5: Comparisons with state-of thrtr-rrrt wer.-frmmmgiddeee video editing methods.
Table 1: Quantitative comparison with first-frame-
Table 2: Average ranking results for
guided video editing.
comparison with reference-guided video editing.
CLIP
DEQA
Input Similarity
1
Score f
Score
Motion Consistency
Background Preservation
V2y Go-with-the-Fiin
0.8995
3.7348
0.7569
0.9047
3.5622
0.7504
Klingl .6
1.869
1.806
12VEdit
0.9128
3.4480
0.7536
VACE (14B)
2.511
2.460
Ours 0.9172
3.8013
0.7608
Ours
1.620
1.734
For each group, the participants were asked to rank the results based on motion consistency and background preservation. Tab. demonstrates the superiority of our method in both aspects.
4.3 ABLATION STUDIES
Disentangling Edits and Background. To validate the effectiveness of mask conditioning in sep- arating edited regions from preserved content, we conduct an ablation study comparing our method using a wwveground-backgromn
mask against baseline version without it. Figure O shows the re- sults. On the left, the goal is to apply a hair color change. Without mask conditioning, the edit is applied globally, altering the lighting across the frame. In contrast, with mask conditioning, the model localizes the change to the hair region while leaving the background untouched. Similarly, in the right example, clothing edits confined to the target only with mask conditioning. Appearance Control in Propagated Edits. We conduct an ablation to evaluate the impact of us- ing edited frames beyond the first frame for controlling appearance in edits propagation. Figure
8

---

Published as a conference paper at ICLR 2026
First Frame
Source Frame
w/o Mask
w/ Mask
Figure 6: Ablation of disentangling edits and background.
Input
Reference
Only
and
Figure 7: Ablation of incorporating additional reference.
compares two settings: using only the first frame as input versus adding an edited frame a later timestep. While using only the first frame can still generate reasonable results, incorporating an additional edited frame offers stronger control over the appearance, leading to more consistent and accurate propagation of the intended edit.
Mask Robustness. To evaluate the sensitivity of our method to mask quality, we conducted an ab- lation study comparing results under three mask configurations: tight mask using high-precision segmentation directly from SAM2 (Ravi et al. 2024); a noisy mask simulated by downsampling the segmentation x 7 grid to introduce significant boundary errors; and the bounding box mask used, which employs a loose rectangular region derived from the segmentation. The comparisons in Figure 8 reveal key insight: pixel-perfect precision is often unecessary and can be restrictive

---

Published as a conference paper at ICLR 2026
Mask
Result
Tight Mask
Noisy Mask
T
Bounding Box Mask (Ours)
Figure 8: Ablation on mask quality. Comparison between tight mask, noisy mask, and our bounding- box strategy. As observed, pixel-perfect precision (tight mask) restricts the generation, Using "loose" mask provides necessary spatial buffer, allowing the model to synthesize natural transitions that seamlessly integrate with the background.
for generative editing. Since the goal is to alter an object's appearance, the generated entity requires spatial buffer undergo necessary contour variations. strietly tight mask would excessively constrain the generation, forcing the new object to adhere rigidly to the original silhouette and po- tentially clipping natural details. contrast, we observe that loose masks including both the highly perturbed "Noisy Mask" and our "Bounding Box" yield better visual results than tight segmenta- tion. By conditioning on a looser region, we allow the model to utilize its strong priors to heal boundary between the edit and the frozen background. This robustness confirms that our framework relies on the mask for semantic localization rather than strict pixel clipping, validating our design choice of using automated, approximate masking workflow (See App.AA.
CONCLUSION
In this work, we present controllable fist-frrmmeggidde video editing framework leveraging mask- aware LoRA fine-tuning to achieve flexible, high-quality, and region-specific video edits without modifying the underlying model architecture. Our method enables fine-grained control over both foreground and background, supports propagation of complex edits across frames, and allows for ad- ditional appearance guidance through reference images. Experiments demonstrate that approach outperforms existing oftte--------rr methods in both qualitative and quantitative evaluations, while maintaining temporal consistency and background preservation.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work is supported in part by the Centre for Perceptual and Interactive Intelligence (CPIT) Ltd., a CUHK-led InnoCentre under the InnoHK initiative of the Innovation and Technology Commission of the Hong Kong Special Administrative Region Government. This work is also support by RGC SEEK-CUHK(22-GDDST))
Early Career Scheme (ECS) No. 24209224, No. 202550505000053. We would like to thank Yumeng Shi for her valuable suggestions the figures and demo. We also thank the reviewers for their insightful comments.
Joint Collaboration Fund
ETHICS STATEMENT
We have read and adhered the ICLR Code of Ethics. Our work, centered on controllable video editing framework, aims to advance creative, commercial, and scientific applications. We acknowl- edge, however, that generative video technologies can be misused for creating misleading or harmful content, such as deepfakes. Our research is intended to provide artists and creators with more flex- ible and precise tools, not to facilitate malicious use. We advocate for the responsible development and deployment of generative models, accompanied by robust detection mechanisms and clear con- tent provenance standards to mitigate such risks. Furthermore, our method relies on pre-trained foundation models 1-122, Hunyuan Video-121 which may inherit biases from their training data. While our approach does not introduce new data, we recognize that these biases could be re- flected in the generated outputs. Addressing and mitigating these inherited biases remains critical area for future research. The datasets used for evaluation are composed of publicly available videos from sources like Pexels, YouTube, and established academic benchmarks, respecting the data usage policies of these platforms.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our research. All implementation details re- quired to replicate our results are provided in the main paper and appendix. Our method is built upon the publicly available diffusion-pipe codebase, as cited Section We have released our specific source code. including scripts for mask-guided LoRA training and inference. The core of our experiments relies on publicly available pre-trained models, specifically Wan2.1-12V and Hunyuan Video-12V, which are clearly identified in Section 4.1 This section also details key hy- perparameters, such as learning rates (Ie-4), training steps (100+100), and hardware requirements (20GB GPU memory). Additional results and implementation strategies, including low-memory approach, are described in the Appendix.
REFERENCES
Ryan Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski, Pascal Clausen, Mingming He, Li Ma, Yitong Deng, Lingxiao Li, Mohsen Mousavi, et al. Go-with-the-fie Motion-controllable video diffusion models using real-time warped noise. arXiv preprint arXiv:2501.08331, 2025.
Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models. arXiv e-prints, pp. arXiv-2305, 2023.
Yuren Cong, Mengmeng Xu, Christian Simon, Shoufa Chen, Jiawei Ren, Yanping Xie, Juan-Manuel Perez-Rua, Bodo Rosenhahn, Tao Xiang, and Sen He. Flatten: optical flow-guided attention for consistent text-to-video editing, arXiv preprint arXiv:2310.05922, 2023.
Yufan Deng, Ruida Wang, Yuhao Zhang, Yu-Wing Tai, and Chi-Keung Tang. Dragvideo: Interactive drag-style video editing. In European Conference on Computer Vision, pp. 199. Springer, 2024.
Michal Geyer, Omer Bar- Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307. 10373, 2023.

---

Published as a conference paper at ICLR 2026
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffu- sion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023.
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00100 2024.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL /
net f
id=nZeVKeeFYF3
Teng Hu, Zhentao Yu, Zhengguang Zhou, Sen Liang, Yuan Zhou, Qin Lin, and Qinglin Lu. Hun- yuancustom: A multimodal-driven architecture for customized video generation, 2025. URL
Ondrej Jamriska. Ebsynth: Fast example-based image synthesis and style transfer. https://
github. com/ haamm//aaaiissaaaaaanallllll 2018.
Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. arXiv preprint 2rXi::2503.07599, 2025.
KlingAI. Klingai. Website, 2025. hhttp:://kingaii.cco
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603 2024.
Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. Anyv2v: A tuning-free frame- work for any video-to-video editing tasks. arXiv preprint arXiv:2403. /4468, 2024.
Xirui Li, Chao Ma, Xiaokang Yang, and Ming-Hsuan Hsuan Yang. Vidtome: Video token merging for zero-shot video editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 7486-7495, 2024.
Yaron Lipman, Ricky Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Repre- sentations, 2023. URL http:///eeeeeeooooooeeeeeeeeaean
Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiyy Jia. Video-p2p: Video editing with cross-attention control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8599-8608, 2024.
Chaojie Mao, Jingfeng Zhang, Yulin Pan, Zeyinzi Jiang, Zhen Han, Yu Liu, and Jingren Zhou. Ace++: Instruction- image creation and editing via context-aware content filling. arXiv preprint arXiv:2501. 2000022500.02487,, 2025.
Wenqi Ouyang, Yi Dong, Lei Yang. Jianlou Si, and Xingang Pan. 12vedit: First-frame-uiddes video editing via tiddeo diffusion models. arXiv preprint a051v:2405.16537, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PmLR, 2021.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.000 00714, 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
12

---

Published as a conference paper at ICLR 2026
Yeji Song, Wonsik Shin, Junsoo Lee, Jeesoo Kim, and Nojun Kwak. Save: Protagonist diversifica- tion with s tructure a gnostic v ideo e diting. In European Conference on Computer Vision, pp. 41-57. Springer, 2024.
Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. XrXiv preprint 203344 2025.
Wen Wang, Yan Jiang, Kangyang Xie, Zide Liu, Hao Chen, Yue Cao, Xinlong Wang, and Chun- hua Shen. Zero-shot video editing using off-the-shelf image diffusion models. arXiv preprint arXiv:2303. 17599 2023.
Bichen Wu, Ching- Yao Chuang, Xiaoyan Wang, Yichen Jia, Kapil Krishnakumar, Tong Xiao, Feng Liang, Licheng Yu, and Peter Vajda. Fairy: Fast parallelized instruction-gui guided video-to-video synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nilion, pp. 8261-8270, 2024.
Yao
Jay Zhangjie Wu, Yixiao Ge, Xintao Wang. Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. Proceedings of the IEEE/CVE International Conference Computer Vision, pp. 7623-7633, 2023.
Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. Florence-22 2: Advancing a unified representation for a variety of vision tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4818- 4829, 2024.
Haibo Yang, Yang Chen, Yingwei Pan, Ting Yao, Zhineng Chen, Chong- Chong-Wah Ngo, and Tao Mei. Hi3d: Pursuing high-resolution image-to-33 generation with video diffusion models. Proceed- ings of the 32nd ACM international conference multimedia, pp. 6870-6879, 2024.
Shiyuan Yang. Zheng Gu, Liang Hou, Xin Tao, Pengfei Wan, Xiaodong Chen, and Jing Liao. Mtv- inpaint: Multi-task long video inpainting. arXiv preprint arXiv:2503. 1411, 2025a.
Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender a video: Zero-shot text-guided video-to-video translation. In SIGGRAPH Asia 2023 Conference Papers, pp. 1-11, 2023.
Zhuoyi Yang. Jiayan Teng. Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng. Da Yin, Yuxuan.Zhang, Weihan Wang. Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with expert transformer. In The Thirteenth International Conference on Learning Representations, 2025b. URL https: http:::://peeeevieeeee
net / wow.mriiddoooooeoeooo LLG
Zhiyuan You, Xin Cai, Jinjin Gu, Tianfan Xue, and Chao Dong. Teaching large language models to regress accurate image quality scores using score distribution. In IEEE Conference on Computer Vision and Pattern Recognition, 2025.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3836-3847, 2023.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
A IMPLEMENTATION DETAILS: MASK ACQUISITION GUI
Se
Figure 9: Screenshot of the included GUI for mask acquisition. Users initialize the process by clicking positive (green) and negative (red) points on the first frame. The segmentation is propagated via SAM2 and automatically converted into bounding box mask for training.
To facilitate easy usage, we provide Graphical User Interface (GUT) in our codebase (see Fig. Users initialize the process by providing sparse clicks on the first frame to define the target object. The mask is then propagated automatically using SAM2.
Crucially, we utilize the bounding box derived from the segmentation for training, rather than the tight mask itself. As demonstrated Section 4.3 this loose bounding box strategy is deliberate, providing spatial buffer for natural transitions and structural changes. This workflow confirms that our method is highly automated and does not require manual frame-by-frane annotation.
B ADDITIONAL EXPERIMENTAL ANALYSIS
B.1 INPUT-LEVEL VS. FEATURE- MASKING
To verify the gain of our mask-aware fine-tuning strategy compared to feature-level strategies which utilize masks constrain the background at the latent feature level, we conducted comparison.
We established a feature-level baseline where the model is fine-tuned using the default Image-to- Video mask configuration (conditioning only on the first frame, prompting the model to learn global frame reconstruction). In this setup. background preservation is enforced as an inference-time in- tervention. Specifically, at each denoising timestep L, we restrict the generation by mechanically blending the model's predicted noisy latent =f with the noisy source latent sre using the binary segmentation mask M (where 1 indicates the constrained background regions):
2
pred (1 M) + = M
(3)
This strategy forces the background pixels to remain unchanged but ignores the semantic consistency between the edited region and the strictly constrained background.
The visual comparison in Figure 10 highlights the limitation of the feature-level constraint: fore- ground objects often appear detached. Quantitative results are reported in Table 3 Our input- conditional approach yields consistent gains across all key metrics, confirming that providing the spatiotemporal mask as a training input enables the model to synthesize more coherent and naturally integrated edits.
14

---

Published as a conference paper at ICLR 2026
Festure Level
Injection
Mask as Input
Feature Level Injection
Mask Imput
Feature Level Injection
Mark as Inpur
Figure 10: Visual comparison between Feature-Level masking (baseline) and our Input-Level mask- ing strategy. As highlighted by the white circles, the Feature-Level approach often results in discon- tinuities and sittandd-pate" artifacts. In contrast, our Input-LLevel method leverages the video prior, achieving better results.
Table 3:
Quantitative comparison of masking strategies. "Feature-Level" refers to the inference- time latent blending baseline described in Eq.
Method
CLIP Score t
DEQA Score
Input Similarity 1
Baseline (Feature-Level)
0.8936
3.5878
0.7402
Ours (Input-Levvl)
0.9172
3.8013
0.7608
B.2 EFFECTIVENESS OF MASK SCHEDULING FOR APPEARANCE LEARNING
We further validate the role of the mask during the appearance learning stage. As demonstrated in Fi using the standard I2V masking configuration (without our specific region-aware schedul- ing) fails to effectively update the object's appearance.
This failure stems from the underlying mechanism of Image-to- mnage-to-Video models: the first frame is typically provided as a strong input condition. When fine-tuning on a reference image without masking, the model encounters a task where the input condition (the reference) is identical to the target output. Consequently. the optimization finds a trivial solution: it learns to simply copy the pixel information from the condition to the output. Because the appearance information is "leaked" through the input condition, the LoRA parameters do not internalize the visual attributes of the new object. Our proposed mask scheduling resolves this by masking the target region, thereby blocking
15

---

Published as a conference paper at ICLR 2026
w/o Mask Scheduling
w/ Mask Scheduling
Additional Reference
w/o Mask
Scheduling
Scheduling
w/ Mask
Additional Reference
w/o Mask
Scheduling
w/ Mask Scheduling
Additional Reference
Figure 11: Ablation on mask conditioning during Stage 2 (Appearance Learning). We compare strategy with baseline using the default I2V mask configuration. Without specific mask scheduling, the model exploits the first-frame condition as a shorteut, simply copying the reference image the output without encoding the appearance into the learnable LoRA weights.
this trivial shortcut and acting as a striet command that forces the LoRA to explicitly learn and generate the target appearance distribution.
B.3 SCALABILITY ANALYSIS
To verify the scalability and temporal stability of our method as requested, we evaluated the editing performance across video sequences varying lengths: 13. 21, and 49 frames. We utilize the CLIP Score as the metric, which measures the semantic alignment between generated frames and edited first frame by comparing their CLIP Radford al. (202T) embedding similarity. As illustrated in Figure 12 our method demonstrates good stability.
C EFFICIENCY ANALYSIS & Low-CoST STRATEGY DETAILS
C.1 METHODOLOGICAL DESIGN: TEMPORAL WINDOWING
The high VRAM requirement (~20GB) our standard training setting primarily stems from the necessity to process the full 49-frame sequence simultaneously during training. To address this, our "Low-Cost Strategy" introduces a training-tims modification that circumvents this bottleneck with- out altering the inference pipeline. Specifically, split the training video into shorter, overlapping sliding windows and update the LoRA weights based on local 9-frame segments rather than the entire sequence once.
16

---

Published as a conference paper at ICLR 2026
0,932
0.9280
0.928
CLIP Score
0.926
0.9230
0.922
0.920
0.9172
0.916
30
Frame Count
Figure 12:
Frame-count vs. CLIP Score. The curve illustrates the consistency with the edited first frame across varying sequence lengths.
This design is theoretically grounded in the insight that motion LoRA primarily learns local dynam- ics. Since complex motions can be decomposed into continuous short-term patterns, training on 9-frame clips is sufficient capture the necessary kinematics. Crucially, this fragmentation does not compromise global consistency because of our mask-aware conditioning. Since the unmasked background fed a visible input context, the model effectively learns "how the foreground moves relative to the fixed background" within each local window. This strong contextual anchoring en- sures that the learned motion remains coherent and aligned with the environment when the full video is reassembled. As shown in Figure 13] the Low-Cost strategy maintains visual fidelity, mo- tion smoothness, and temporal consistency comparable to standard full-frame training, validating the effectiveness of our decomposition approach.
C.2 ENGINEERING OPTIMIZATION: SWAP BLOCKS
To further democratize access, we leveraged the "swap blocks" technique supported by the di f an-pipp codebase This mechanism dynamically offloads frozen base model parame- ters to the CPU, keeping only trainable LoRA weights and active transformer blocks the GPU. Table 4 quantifies the memory savings achieved by varying the blocks to swap parameter. By maximizing swapping (config 38), reduce the peak VRAM ~ GB, making the training pipeline accessible on consumer-grade GPUs.
Table 4: Effect of "Swap Blocks" optimization on Peak VRAM (using 9-frame segments).
blocks.to.swap
0 (None)
16
32 (Default)
38 (Max)
Peak VRAM (MB)
25,097
16,305
9,987
7,617
C.3 RUN-TIME VRAM COMPARISON WITH BASELINES
We benchmarked the run-time VRAM usage of our method against state-of-the-art inference-only baselines single NVIDIA RTX 4090 (832 480, 49 frames).
As shown in Table 5 our optimized Low- strategy achieves a memory footprint comparable to inference-only baselines, demonstrating broad hardware accessibility.
17

---

Published as a conference paper at ICLR 2026
Full
Low-Cost
Full
Low-Cost
Full
Low-Cost Cost
Figure 13: Visual comparison between Standard (Full) and Low-Cost strategies. Top rows: Results from standard training the full 49-frame sequence. Bottom rows: Results from optimized training using 9-frame sliding windows. The results indicate that our memory-efficient strategy preserves motion fidelity and structural details comparable to the full-frame baseline.
Table 5: Peak Run-Time VRAM comparison against inference- based methods. *For Go-with-the- flow, the first value is Sampling VRAM, second is Peak Decoding VRAM.
Method
V2V
Go-with-the-flow*
Ours (Standard)
Ours (Low-Cost)
Peak VRAM
13,680 MB
4,398 21,512 MB
21,522 MB
7,617 MB
D DIVERSE EDITING RESULTS
We present visual results of our method diverse editing tasks in Fig. 14 and Fig. 15 including object replacement, addition, and removal. We also include results for mostoo and multi-person scenario in Fig. and results for long-form video in Fig. 17
18

---

Published as a conference paper at ICLR 2026
Input
Results
Input
Results
strong enough face it all" Stoy
strong enough face Stay
strong
strong enough face. all
Input
enough
face
stay
Stay
YYou strong enough to face ell Stay
strong enough face all" Stay
strong enough faca in all stay
strong
Results
enough
face otl"
Stc
Input
Results
Figure 14: Diverse editing results (I).
E RESULTS WITH HUNYUANVIDEO-I2V MODEL.
In addition to the main results based Wan2. 1-12V, we also conducted experiments using Hunyuan Vidco-12V (See Fig. 18). demonstrates the generalization ability of our framework across different 12V architectures.
F FAILURE CASES
Figure 19 illustrates representative limitations encountered by framework. We observe specific difficulties in text generation and preservation (as seen in the top rows). Even with mask constraints, the model often degrades high-frequency semantic symbols into blurred texture, likely due to the
19

---

Published as a conference paper at ICLR 2026
Input
Results
Input
Results
Input
Results
Input
Results
Figure 15: Diverse editing results (II).
compression loss of the underlying VAE. Additionally, limitations arise complex motion preser- vation, demonstrated in the bottom where the character's hair color is modified. In scenarios featuring rapid and highly stylized abstract motion, accurately decoupling the appearance update from the original dynamics becomes challenging, potentially resulting in unnatural deformations and disruptions to the structural integrity of the motion sequence.
G DETAILS OF USER STUDY
To evaluate the performance of our method, we conducted a user study with 50 participants. Each participant was randomly shown 10 groups of results generated by different methods. For each group, the participants were asked to select the best result based on two criteria: reference similarity and visual quality. The user study interface is shown in Figure 20
20

---

Published as a conference paper at ICLR 2026
Input
Results
Input
Results
Figure 16: Diverse editing results (III).
Instructions to Participants: Participants were provided with the following instructions:
"You will be shown several video sequences with different edits applied. Each sequence will contain three options, each generated using a different method. The methods will be presented in random order avoid bias. Your task is to choose the option that best matches the reference image in terms of visual quality and how well the edit aligns with the reference image. Pay close attention to the consistency of the edits across the frames and how naturally they interact with the environment."
After viewing each video sequence, participants were asked to answer the following two questions for each video:
Which video option best matches the reference image in terms of appearance and content?
Reference Similarity:
2. Visual Quality: Which video option has the highest overall visual quality, considering both the foreground edit and the background consistency?
The demographic information of the participants, including their age distribution and professional background, shown in Figure 21 This figure demonstrates the diversity of the participant pool, ensuring a wide range of perspectives in the user study.
H BROADER IMPACTS
The work presented in this paper introduces a flexible and controllable video editing method based mask-aware LoRA fine-tuning, As with many advances in generative AI, method holds both positive and negative socictal implications.
Our video editing framework opens up possibilities in creative fields, such film production, digital art, and content creation, by enabling high-quality, real-time video edits with high degree of control. This could streamline workflows for professionals in these industries, reducing the time and effort traditionally required for manual video editing. Additionally, our approach can be used non- commercial domains, such healthcare, where controllable video editing assist in visualizing
21

---

Published as a conference paper at ICLR 2026
Input
Results
Figure 17: Diverse editing results (IV).
complex medical data, simulations, and surgical procedures. In such contexts, our method can aid improving communication and understanding of visual information.
While the benefits are substantial, the potential for misuse of generative video models also exists. High-quality video editing tools be misused generate deepfakes, spreading misinformation or altering videos in malicious ways. Given the high realism of edited videas, there is risk that such technologies could be exploited in political or social contexts, leading to challenges in verifying the authenticity of media. This raises concerns about privacy, security, and trust in iido-bbsed content.
To mitigate these risks, we advocate for responsible usage guidelines and the development of safety measures for content creation tools powered by generative models. For example, platforms using such technology could implement robust verification processes for user-generated content. Addi- tionally, we encourage future research into model interpretability and the development of tools to detect manipulated media, ensuring that generated content easily distinguishable from original footage.
I THE USE OF LARGE LANGUAGE MODELS
During the preparation of this manuscript, we utilized a large language model (LLM) to aid in polishing the writing. Specifically, it was employed improve grammar, correct spelling, enhance
22

---

Published as a conference paper at ICLR 2026
Source
Frame
Version
HunyuanVides Wan2. Version
Figure 18: Results of our method applied to Wan2. 1-12V and Hunyuan Video-12V.
clarity, and ensure the conciseness of the text. It also assisted in refining sentence structures to better adhere to formal academic style. All scientific contributions, including the core methodology. experimental design, results, and their interpretation, were solely conceived and articulated by the human authors. The authors have thoroughly reviewed and edited the final version of the text and take full responsibility for all content presented in this paper.
23

---

Published as a conference paper at ICLR 2026
Input
Results
Input
Results
Figure 19: Failure cases. Top Rows: While the primary edit (suit and tie) is applied successfully, the model fails to preserve the fine-grained text characters the watermark, leading to garbled glyphs. Bottom Rows: When modifying specific attributes (changing hair color to red) in scenarios with rapid and abstract motion, the method may struggle to align the new appearance with the original dynamics, resulting in body deformation and motion artifacts.
24

---

Published as a conference paper at ICLR 2026
Video Editing User Study
Then sou for pur puiiing oor use studs Your il looklone 20 ausupo of wideoo duch contaro orw cwrnforeooiinee and thes video voited saing ferettmattod. hou tak to select wi idhidde renol which
Welcome to the User Study
Each will contain
utted cifferent method.
The methodd will tee reeented in nn ddom ooder swold bow.
Your taskis thosse option that best motichee erernene image in termn alsus quality and how well edit algus ineeeeess ratursilyy they
consiaeiicy
Start Study
Video Evaliation
lage were comeet
coetting but
Centice
Figure 20: Screenshot of user study interface.
Age Distribution
Professional Background
50-
50
Percentage (%6
Percentage (%)
40
30-
30-
20
10
20-24
25.29
30-34
35-39
42.49
Technical
Creative
Educatiin
Busintes
Others
Figure Demographic information of the participants in the user study.
25


Reviews:
[{"review_id": "3pxp0ynF5W", "reviewer": "Reviewer_bgBu", "summary": "This paper proposes a method for controllable video editing by fine-tuning a pre-trained Image-to-Video (I2V) model using a mask-aware LoRA (Low-Rank Adaptation) strategy. The core idea is to use a spatiotemporal mask during LoRA tuning to guide the model in two ways: 1) to preserve or generate content in specific regions, and 2) to learn either motion from the source video or appearance from a reference image. The method is positioned as an improvement over first-frame-guided editing, offering more granular control over the temporal evolution of edits. Experiments show favorable results against several state-of-the-art baselines in both qualitative and quantitative evaluations.", "questions": "No Questions", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The proposed framework is relatively simple, does not require architectural changes to the base model, and offers a flexible pipeline for various editing tasks (object replacement, addition, style transfer). The inclusion of a low-memory training strategy in the appendix is a practical consideration.\n\nThe paper provides a thorough experimental section, including comparisons with both first-frame-guided and reference-guided methods, quantitative metrics, ablation studies, and a user study.", "weaknesses": "Limited Technical Novelty: The core technical components—using I2V models, LoRA fine-tuning, and spatiotemporal masks—are not novel individually. The contribution lies primarily in their specific combination and application to this problem. However, the approach feels like a natural and incremental extension of existing \"tune-a-video\" and first-frame-guided paradigms. The paper does not sufficiently establish a significant technical leap over the current state-of-the-art. The method's reliance on per-video fine-tuning, while effective, is also a limitation shared with several prior works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PrZMl36Dii", "reviewer": "Reviewer_4aXZ", "summary": "This paper addresses first-frame-guided video editing with diffusion models, proposing a training-time, mask-aware low-rank adaptation of off-the-shelf image-to-video (I2V) models. By inserting learnable LoRA modules into attention layers and conditioning the denoising network on a user-defined spatio-temporal mask, the model is taught (i) to preserve pixels where the mask is 1 and (ii) to synthesise new content where the mask is 0, either by copying motion from the source clip or by adopting appearance from an extra reference image. At inference, only the edited first frame and the same mask are needed to propagate the edit through the whole sequence. Extensive experiments against recent baselines (I2VEdit, AnyV2V, VACE, Kling1.6) show superior or comparable DEQA, CLIP-score and user-study rankings.", "questions": "- Provide a direct ablation that keeps everything else identical except mask conditioning is missing.\n- Provide frame-count vs metric curves on Benchmark to verify scalability.\n- How does performance degrade under automatic segmentation errors\n- Provide some failure case", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Two-stage mask scheduling (motion learning → appearance learning) cleanly disentangles dynamics from look\n- Solid empirical protocol: two tasks, two backbones (Wan2.1 & HunyuanVideo), three metrics plus user study, ablations for mask usage and extra reference frames.\n- Equations are minimal and directly show the change of conditioning, easing reproducibility.\n- Delivers a lightweight plug-in that practitioners can apply to any I2V model; potential to become a default extension for personalised video creation tools.", "weaknesses": "- I2VEdit already finetunes LoRA per video and uses attention masks to reduce background leakage. The paper must quantify the marginal gain of feeding the mask into the network input instead of using it only at feature level. \n- 20 self-collected clips plus an undisclosed I2VEdit test set is small. No long-form videos, fast motion, or multi-person scenes are reported. \n- Perfect masks are assumed. Evaluate with noisy masks (erosion / dilation, IoU = 0.80–0.90) or segmentation-model predictions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v2oPC2Y8cI", "reviewer": "Reviewer_eyka", "summary": "This paper proposes a controllable first-frame-guided video editing framework that adapts a pretrained image-to-video diffusion model using mask-aware LoRA. The central idea is to repurpose the I2V model’s spatiotemporal mask not just as an inference constraint but also as a learning signal that tells LoRA what to learn. The method (i) performs per-video LoRA tuning so the model learns the input video’s motion pattern while preserving unedited regions via a mask; and (ii) optionally performs a second, brief LoRA pass where edited reference frames at later timestamps teach the model the desired appearance evolution inside masked regions (e.g., a flower gradually becoming a red rose). Experiments on first-frame-guided and reference-guided editing show better results compared with AnyV2V, Go-with-the-Flow, and I2VEdit.", "questions": "See weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Simple and efficient idea**\nUsing a spatiotemporal mask as an explicit LoRA supervision signal is a clean and reusable concept that doesn’t require architectural changes. This is likely to inspire follow-up work on using conditioning signals to steer what LoRA learns. This strategy addresses two persistent failure modes in first-frame editing: (i) unwanted background drift/leakage and (ii) insufficient control of how the edited object looks as it rotates/deforms or reveals disocclusions.\n\n- **Low engineering overhead**\nPer-video LoRA is a common practice. Adding mask-aware conditioning with edited frames is easy to deploy and works across different I2V-based models.\n\n- Qualitative results look compelling on diverse manipulations (object add/replace/style, clothing/hair edits), with visibly better background preservation and temporal coherence than the chosen baselines.", "weaknesses": "- **Incremental Novelty**\nPer-video LoRA for motion propagation and mask-conditioned inpainting are known practices in video editing and inpainting. The novelty is mainly in using the mask to steer LoRA’s learning objective across space-time. This is elegant but not a large algorithmic leap.\n\n- **Mask acquisition and robustness are under-specified**\nIt is not clear how masks are obtained for training and inference. Is it automatically calculated from the first-frame edit difference, and then propagated with optical flow/segmentation? How to handle later edited references?\nRobustness to mask errors (boundary misalignment, sparsity, temporal jitter) is not evaluated. Since the method’s key promise is precise, region-specific control, an analysis with sensitivity to mask quality is critical.\n\nThe paper tackles a high-impact practical problem (controllable propagation of first-frame edits) with a minimal, effective mechanism. The idea is easy to adopt on top of modern I2V models, and qualitative outcomes are persuasive. While the novelty is incremental, the contribution is useful and timely for creators and researchers. I tend to give a positive score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uiikqbFGW2", "reviewer": "Reviewer_636m", "summary": "This paper proposes a video editing method that enhances first-frame-guided approaches with more granular temporal control. The core contribution is a mask-aware LoRA fine-tuning strategy for pre-trained Image-to-Video models. This technique uses a spatiotemporal mask to teach the model to selectively preserve background content while generating new content in specified regions. This dual approach allows the model to learn consistent motion from the source video and new appearances from user-provided reference frames, enabling complex edits like a flower blooming into a different color.", "questions": "See weaknesses, but my main questions are based on its practical applicability and efficiency.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Video results are pretty good (videos demonstrated in the supplementary and the demo in the supplementary showcase).\n\n- Application-wise it's interesting and definitely be useful. However, still, efficiency is a key problem.\n\n- Clearly outperforms previous approaches in terms of both quantitative and qualitative metrics.", "weaknesses": "- The primary limitation is the need of fine-tuning per-video= The method requires 100 training steps to learn motion and potentially another 100 steps to learn new appearance (Sec 4.1). While this is more efficient than full model training, it is still a significant computational step for every single video edit compared to zero-shot or inference-time methods. How do you compare it in terms of computataional efficiency? I think putting a comparison in term of run-time memory give some insights.\n\n- The method relies on a precise \"spatiotemporal mask\" to separate the edited region from the background (Sec 3.3.1). The paper does not specify how this mask is acquired. Does this require the user to perform manual video segmentation for all frames? If so, this is still a requirement that would make the method impractical for most users. If the mask is generated automatically, its quality would be critical to the final edit. What could be the possible ways to automatically get these masks?\n\n- The full-frame method requires 20GB of GPU VRAM (Sec 4.1), which is inaccessible to many users. The \"low-cost training strategy\" (Appendix C) is a good alternative. Can you elaborate on this, for example what ways can be done to even make it more efficient?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)