Title: DFVEdit: Conditional Delta Flow Vector for Zero-shot Video Editing
Venue: NeurIPS 2025
Accepted: no
Keywords: video editing

zero-shot

Video Diffusion Transformer
Abstract: The advent of Video Diffusion Transformers (Video DiTs) marks a milestone in video generation. However, directly applying existing video editing methods to Video DiTs often incurs substantial computational overhead, due to resource-intensive attention modification or fine-tuning. To alleviate this problem, we present DFVEdit, an efficient zero-shot video editing method tailored for Video DiTs. DFVEdit eliminates the need for both attention engineering and fine-tuning by directly operating on clean latents via flow transformation. To be more specific, we observe that editing and sampling can be unified under the continuous flow perspective. Building upon this foundation, we propose the Conditional Delta Flow Vector (CDFV) -- a theoretically unbiased estimation of DFV -- and integrate Implicit Cross Attention (ICA) guidance as well as Embedding Reinforcement (ER) to further enhance editing quality. DFVEdit excels in practical efficiency, offering at least 20x inference speed-up and 85\% memory reduction on Video DiTs compared to attention-engineering-based editing methods. Extensive quantitative and qualitative experiments demonstrate that DFVEdit can be seamlessly applied to popular Video DiTs (\emph{e.g.}, CogVideoX and Wan2.1), attaining state-of-the-art performance on structural fidelity, spatial-temporal consistency, and editing quality.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces DFVEdit, a zero-shot video editing method designed for Video Diffusion Transformers (Video DiTs). DFVEdit bypasses the need for attention modification or fine-tuning by operating directly on clean latent representations through flow transformation. The authors unify editing and sampling under a continuous flow framework and propose the Conditional Delta Flow Vector (CDFV) for estimating flow changes, enhanced by Implicit Cross Attention (ICA) and Embedding Reinforcement (ER) to improve quality. DFVEdit achieves high practical efficiency of 20× faster inference and 85% memory reduction and demonstrates state-of-the-art performance in structure fidelity, temporal consistency, and editing quality across various Video DiTs like CogVideoX and Wan2.1.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n[1] The paper correctly identifies the limitations of existing heuristic methods that require attention manipulation for diffusion-based video editing and presents a compelling alternative that achieves strong results.\n\n[2] The theoretical formulation that interprets editing as a form of sampling from a distribution flow perspective is well-grounded. I consider this to be the core contribution of the paper.\n\n[3] The visual quality of the generated videos is strong, and the authors provide a wide range of comparisons, which enhances the credibility of their results.\n\n**Weakness**\n\n[1] Since the Conditional Delta Flow Vector (CDFV) is designed to operate under a large number of diffusion steps T, it would be beneficial to provide ablation results showing how output quality varies with different values of T.\n\n[2] In some qualitative results, such as the \"moonwalking\" video, the baseline method (e.g., VideoGrain) appears to produce higher visual fidelity, especially in rendering the floor, and shows better textual alignment with the \"red fox\" edited video. But, why is there such a noticeable gap in quantitative performance compared to the proposed method?\n\n[3] While the method avoids explicit attention modification, the Implicit Cross-Attention (ICA) mechanism could still be influenced by masking and editing quality. If so, could the authors clarify the practical advantage of ICA over attention modification methods that do not rely on masks? A comparison or discussion in this regard would help clarify the benefits of the proposed approach.\n\nQUESTIONS:\n[Q1] The interpretation of editing as continuous flow sampling in Line 153 is interesting. Later in Section 3.2, this correspondence is used as a basis for introducing the Conditional Delta Flow Vector. Are the formulations in Section 3.1 entirely designed by the authors? \n\n[Q2] In Line 172, the term \"unbiased estimation\" is used. Could the authors clarify what is specifically meant by this in the context of their method? Furthermore, what would constitute a biased estimation in this setting?\n\nLIMITATIONS:\nIn Line 226, the paper states that its main objective is training-free appearance editing. However, it may be helpful to inform readers that other types of editing such as non-rigid modification and drag-based interaction also exist as below. While these approaches have been popular in image editing, they are increasingly being explored in video editing as well. A brief mention of these approaches could provide useful context and help position this work more clearly within the broader landscape of video editing. This could also be discussed as a potential limitation or direction for future work. Including such discussion in the related work or limitations section would strengthen the paper.\n\n[1] DNI: Dilutional Noise Initialization for Diffusion Video Editing\n\n[2] Drag-A-Video: Non-rigid Video Editing with Point-based Interaction\n\n[3] DragVideo: Interactive Drag-style Video Editing", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose Conditional Delta Flow Vector (CDFV) for video editing based on Video DiTs. Instead of using attention-engineering or further finetuning, CDFV operates directly on the input latents, achieving 20x faster inference speed and 85% less memory cost. CDFV achieves a SoTA performance as a zero-shot video editing method for video DiTs.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper tries to resolve a critical issue for video editing in DiTs for memory costs. It works on latent inputs instead of attention maps. The proposed method is zero-shot and training-free. The quality of the results is acceptable. My major concerns include: (a) It turns out in practice the authors still use attention maps and do attention engineering (ICA), (b) The writing of the paper is not well-organized, especially in Fig. 1 and Fig. 2, and (c) The examples shown in the supplementary material are relatively easy, with small motions and dynamics. \n\nMy current rating for the paper is Borderline Reject.\n\n## Strength \n### Idea/Motivation\n- The authors analyze why previous attention-engineering methods and finetuning methods have difficulties in video DiTs in Fig. 1. This provides a useful evidence for introducing a new method with less speed and memory overheads. \n- The authors operate directly on the latent inputs which makes sense in the context of video DiTs then. \n\n### Method\n- The authors start with DFV and CDFV with a theoretical explanation, introducing CDFV for an unbias estimate for DFV.\n- To mitigate the discrepancy between the theoretical basis and the empirical shift, the authors introduce ICA and Target Embedding Reinforcement (ER).\n\n### Experiments/results\n- The authors conduct a comprehensive comparison with previous methods in Table 1 and Fig.4, showing that DFVEdit outperforms these methods in a zero-shot fashion. \n- The ablation study on ER factor $\\gamma$ is interesting. \n- With CDFV, the temporal consistency improves a lot in the video results. \n\n## Weaknesses\n### Method\n- It is not clear to me why the proposed method is different from previous attention-engineering works conceptually because ICA and SAM (optionally though) are used. \n\n### Experiments\n- Only easy cases are shown in the paper. Does DFVEdit work for videos with larger dynamics and motions? \n- In the Zebra case in the supplementary material, it is not clear to me what the benefit of ICA is as I personally prefer the result w/o ICA. In the result with all components, the mane of the horse looks much unnatural, although the preservation of the background is achieved. \n\n### Writing/Delivery\n- Fig. 1(c) is hard for readers to understand in my opinion. Without any further explanations, readers do not understand what the meaning of \"Normalized DFV\" is and how this figure is related to the topic. It is highly recommended the authors include explanations either in the caption or in the main text, given Fig. 1 plays a critical role demonstrating the motivation of the problem. \n- The name \"DFVEdit\" comes out of nowhere in the Introduction section (L69). It needs a definition for its debut. \n- Fig. 2 lacks a clear guidance between the figure and the caption. For example, what are (1), (2), (3), (4) and (5) in the caption and how can readers associate them in the figure? \n- Also the notations in Fig.2 are messy. (a) Sometimes the authors use capitalized \"T\" but sometimes it is lower-case \"t\". (b) $\\gamma$ is not defined at all in \"Embedding Reinforcement\" in this figure. It is suggested all symbols are self-contained in a figure. (c) What is $\\otimes$ in Embedding Reinforcement?\n- Overall, Fig. 2 is not well-organized and does not help readers understand the proposed method.\n\nQUESTIONS:\n- How can you figure out which token is for stylization? (L209 and Fig. 2) By a manual segmentation? \n- This is a little minor but will the proposed methods work for other aspect ratios instead of only for squared (1:1) ones?\n\nLIMITATIONS:\nThe authors have included the limitations of the approach, including the preservation the detail fidelity and editing with large variations. However, it is not clear if the failure of preservation comes from the proposed approach or from the compression loss in the VAE.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes DFVEdit, an efficient zero-shot video editing method that operates through continuous flow transformations in a clean latent space, introducing Conditional Delta Flow Vector (CDFV) combined with Implicit Cross-Attention (ICA) and Embedding Reinforcement (ER) to enhance editing performance. Experiments demonstrate effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The overall writing logic of the paper is clear.\n2. This paper theoretically unifies generation and editing through derivation, proposing CDFV, Embedding Reinforcement, and Implicit Cross-Attention Guidance to achieve zero-shot video editing, obtaining performance superior to other methods.\n\nWeaknesses\n1. Concerns about zero-shot design. First, from a user perspective, they don't care about training resource consumption; their primary concerns are editing performance and quality, followed by inference latency. Moreover, zero-shot methods often have significantly inferior results compared to fine-tuning methods. Therefore, such methods need to provide more insights to help researchers and users understand the essence of video editing or assist in designing better training-based methods. Training-based methods only require some resource consumption during training, but the resources needed during inference after training are comparable. If the motivation is merely mentioning that training methods require some computational resources for fine-tuning, this motivation is somewhat unconvincing.\n2. Bloated method design. The framework contains quite a few small components. Besides the proposed CDFV, it also includes Embedding Reinforcement and Implicit Cross-Attention Guidance. Such complex design doesn't result in particularly good final performance, making the entire framework appear somewhat bloated with many parameters to tune.\n3. Shortcomings in experimental results. In terms of capabilities, the results shown in the paper, including the mp4 files in supplementary materials, mainly focus on style transformations, such as from \"bear\" to \"polar bear,\" \"natural scenery\" to \"ink painting\" style. I'm curious whether the method doesn't support editing of shapes, motion patterns, or actions. If so, what are possible solutions? Or said, does it also demonstrate the limited upper bound of zero-shot methods?\n4. Fairness of experimental comparisons. Other compared methods, such as FateZero, TokenFlow, VideoDirector, ControlVideo, etc., are mainly based on 2D base models like Stable Diffusion, so it's natural for them to show some deficiencies in temporal consistency. From a fairness perspective, shouldn't this paper also conduct experiments based on Stable Diffusion 1.5's image model to compare with other methods?\n\nQUESTIONS:\nAs detailed in the weakness section, my primary concerns focus on the zero-shot design motivation, the overly complex method architecture, limitations in the experimental results, and the fairness of experimental comparisons.\n\nLIMITATIONS:\nYes, discussed in the Supplementary material.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents DFVEdit - a training-free video editing framework based on pre-trained Video DiTs. \n\nAuthors unify editing and sampling under the continuous flow perspective and propose to perform video editing by integrating proposed Conditional Delta Flow Vector (CDFV) as control term within the sampling process. \n\nTo further enhance the framework, the authors present Implicit Cross Attention (ICA), which functions as a CDFV masking mechanism. ICA facilitates DFVEdit and prevents local distributional drift in unedited regions.  Additionally, the authors propose Target Embedding Reinforcement, a technique that reweights text tokens in 3D Full-Attention. This approach amplifies the importance of text tokens most relevant to the editing task, improving the framework's ability to align with user-specified instructions.\n\nDFVEdit is evaluated on real-world video datasets, including DAVIS2017 and Pexels, demonstrating superior performance compared to previous state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\n1. The method has a theoretical foundation\n\n2. It is training-free and model-agnostic, making it highly versatile and widely applicable\n\n3. The provided samples show good editing quality\n\n## Weaknesses:\n\n1. Insufficient experiments:\n\n   (1) The authors did not provide user study results which is important for editing papers since current qualitative results only provide several selected generations which is not representative.\n\n   (2) The proposed method is not specifically tied to video editing and can be tested on T2I models. I highly recommend authors to conduct image editing experiments since this field is more developed and has more developed and representative benchmarks (such as PIE-Bench [3]) as well as baseline methods (https://paperswithcode.com/sota/text-based-image-editing-on-pie-bench)\n\n   (3) The paper misses important video editing baselines: CogVideoX Video2Video pipeline [1] and VidToMe [2].\n\n   (4) Although DFVEdit achieves strong metrics, direct comparison with prior methods is complicated because they all rely on a U-Net backbone. Moreover, those baseline diffusion models produce relatively low-quality generations compared to modern DiT-based architectures (Wan2.1, CogVideoX). Therefore, a direct comparison is difficult to interpret.\n\n   (5) The paper states that DEVEdit also works on Wan2.1-14B (line 218), however results are reported for CogVideoX only.\n\n\n2. Lack of method details:\n   (1) It is not clear which version of the DFVEdit procedure is actually used in the experiments - the simplified or the full variant. Do they yield different metric results? \n\n   (2) The paper omits the analysis of the hyperparameter $\\lambda$ (equations 6 and 31) and $\\gamma$ (equation 16); how sensitive is the editing outcome to their choice?\n\n   (3) The paper does not specify which attention blocks are used to extract the mask nor how the attention maps are binarized. This opacity suggests the masking may be suboptimal—background regions are reported to degrade during editing (as noted in “Limitations”).\n\n[1] Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., ... & Tang, J. (2024). Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072.\n\n[2] Li, X., Ma, C., Yang, X., & Yang, M. H. (2024). Vidtome: Video token merging for zero-shot video editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7486-7495).\n\n[3] Ju, X., Zeng, A., Bian, Y., Liu, S., & Xu, Q. (2023). Direct inversion: Boosting diffusion-based editing with 3 lines of code. arXiv preprint arXiv:2310.01506.\n\nQUESTIONS:\n1. Please provide additional experiments that results for image editing, add results important baselines such as CogVideoX Video2Video pipeline and VidToMe, metrics for Wan model, and user study (Weakness 1).\n\n2. Please provide method details (Weakness 2).\n\n3.  In Algorithm 1, the initial latent sample is drawn only once. Could the quality of the edited video vary significantly with a different seed? What mechanism ensures stability and reproducibility across runs?\n\n4. It seems that the approach appears to struggle with replacing or transforming objects of intricate shape (e.g., swapping a jeep for a tank or an elephant), suggesting limited capacity for large-scale geometric edits. Could you please elaborate on that?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}