Title: Injecting Frame-Event Complementary Fusion into Diffusion for Optical Flow in Challenging Scenes
Venue: NeurIPS 2025
Accepted: yes
Keywords: optical flow estimation

event camera

diffusion models
Abstract: Optical flow estimation has achieved promising results in conventional scenes but faces challenges in high-speed and low-light scenes, which suffer from motion blur and insufficient illumination. These conditions lead to weakened texture and amplified noise and deteriorate the appearance saturation and boundary completeness of frame cameras, which are necessary for motion feature matching. In degraded scenes, the frame camera provides dense appearance saturation but sparse boundary completeness due to its long imaging time and low dynamic range. In contrast, the event camera offers sparse appearance saturation, while its short imaging time and high dynamic range gives rise to dense boundary completeness. Traditionally, existing methods utilize feature fusion or domain adaptation to introduce event to improve boundary completeness. However, the appearance features are still deteriorated, which severely affects the mostly adopted discriminative models that learn the mapping from visual features to motion fields and generative models that generate motion fields based on given visual features. So we introduce diffusion models that learn the mapping from noising flow to clear flow, which is not affected by the deteriorated visual features. Therefore, we propose a novel optical flow estimation framework Diff-ABFlow based on diffusion models with frame-event appearance-boundary fusion. Inspired by the appearance-boundary complementarity of frame and event, we propose an Attention-Guided Appearance-Boundary Fusion module to fuse frame and event. Based on diffusion models, we propose a Multi-Condition Iterative Denoising Decoder. Our proposed method can effectively utilize the respective advantages of frame and event, and shows great robustness to degraded input. In addition, we propose a dual-modal optical flow dataset for generalization experiments. Extensive experiments have verified the superiority of our proposed method. The code is released at <https://github.com/Haonan-Wang-aurora/Diff-ABFlow>.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an optical flow estimation method that fuses frame images and event streams, addressing the challenges posed by extreme scenarios such as high-speed motion and low-light environments. The authors argue that frame features exhibit dense appearance saturation but sparse boundary completeness, while event features are the opposite. To this end, a fusion module based on an attention mechanism is proposed to complementarily combine frame and event features. Then, based on a diffusion model, the paper introduces an iterative denoising strategy called MC-IDD to decode motion information from the fused features. Experiments on both synthetic and real datasets demonstrate the effectiveness of the approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n\n(1)The paper clearly identifies that frame features possess dense appearance saturation but weak boundary completeness, while event features can compensate for this. The fused features contain more complete information than single-modality features.\n\n(2)The authors propose an attention-based Frame-Event fusion module that enhances both appearance and boundary representation in the fused features.\n\n(3)A diffusion model is used to decode motion information from the fused features via iterative denoising, and experiments show that diffusion models are more robust than discriminative and generative models.\n\n**Weaknesses**:\n\n(1)The authors use the Sobel operator to extract boundaries from both frames and events. However, the Sobel operator typically requires a threshold hyperparameter for non-maximum suppression, and no ablation study is provided regarding this hyperparameter.\n\n(2)In lines 121–122, the authors mention constructing a 4D cost volume using the input frame features. However, according to RAFT [29], the cost volume is constructed by computing correlations between features of consecutive frames. Are these two methods different? If so, a comparison between them should be included.\n\n(3)For experiments on real-world data, the authors only select extreme scene subsets (HS-DSEC and LL-DSEC) but do not specify how these sequences are split from the original DSEC dataset. To verify generalization, the method should also be evaluated on the publicly available DSEC benchmark and compared with other methods.\n\n(4)The authors introduce two new real-world datasets: HS-FEFD and LL-FEFD, claiming them as part of their contributions. The supplementary materials mention that these datasets contain optical flow ground truth, so quantitative generalization experiments should be conducted.\n\n(5)The inference time of Attention-ABF, MIGDD, TVM-MCA, MGDD, and the overall pipeline on KITTI and DSEC datasets should be reported and compared with other optical flow methods (e.g., BFlow [8]).\n\n(6)In the TVM-MCA module, time embeddings from time step t are used. However, the meaning of \"t\" is ambiguous — is it the timestamp from the event stream and frame, or the time step in the MC-IDD denoising process? This should be clarified.\n\nQUESTIONS:\nThis paper introduces a novel idea of complementarity between frame and event data in terms of appearance and boundary information. However, the experiments are limited to selected extreme scene datasets, with no evaluation on common, non-extreme scenarios to verify generalization and applicability. Additionally, details such as hyperparameters, inference time, and module specifics are lacking. For these reasons, I give a borderline accept as my initial rating.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Diff-ABFlow, a novel diffusion-based framework for optical flow estimation in high-speed and low-light scenes using complementary fusion of frame and event data. It introduces two key modules: Attention-ABF, which enhances feature fusion by leveraging appearance boundary complementarity, and MC-IDD, an improved DDIM-based backbone tailored for optical flow tasks. Extensive experiments demonstrate that Diff-ABFlow achieves state-of-the-art performance across multiple datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper utilizes the appearance-boundary complementarity of frame and event data to obtain better visual features for optical flow estimation in degraded scenes, and the motivation is clear.\n2. The experiments in the paper demonstrate that the proposed method performs well on both synthetic and real-world data, forming an effective validation loop in the experimental setup.\n\nWeaknesses:\n1. This work shares a similar motivation with ABDA-Flow (ICLR '24), as both use RGB and event data for complementary appearance and boundary information, leading to overlap in novelty.\n2. In lines 198–199 of the paper, the authors claim to have proposed the HS-FEFD and LL-FEFD datasets. However, the visualized images in Figure 6 are identical to those in Figure 6 of ComST-Flow (CVPR '25). The authors are requested to clarify this.\n3. It is recommended that the authors include a comparison with ABDA-Flow (ICLR '24) in Table 1 and add its visualizations in Figure 5.\n\nQUESTIONS:\n1. It is recommended that the authors provide optical flow estimation results under specific weather conditions, which may help to better understand the robustness of the proposed model.\n2. Why do some comparison methods in Table 1 show performance degradation after visual enhancement? Please provide an explanation.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a framework called Diff-ABFlow for optical flow estimation in high-speed and low-light scenes. The authors propose using diffusion models combined with frame-event appearance-boundary fusion to address the challenges posed by motion blur and insufficient illumination. The framework leverages the complementary strengths of frame cameras (dense appearance saturation) and event cameras (dense boundary completeness) to improve optical flow estimation. Key components include the Attention-Guided Appearance-Boundary Fusion module and the Multi-Condition Iterative Denoising Decoder . Extensive experiments demonstrate the superiority of the proposed method over existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n+ The Attention-ABF module combines the strengths of frame and event cameras, leading to improved visual features. Dual modal features are used to guide diffusion models for optical flow estimation.\n+ Multi-Condition Iterative Denoising Decoder are used to progressively denoise the optical flow. Detailed ablation studies are conducted to demonstrate the contribution of each modules.\n+ The authors conduct extensive experiments on multiple datasets, including synthetic and real-world scenarios, to validate their method.\n\nWeaknesses:\n- The proposed framework is complex, involving multiple modules and iterative processes, which may pose challenges for implementation and computational efficiency.\n- The method is primarily tested on high-speed and low-light scenes, and its generalizability to other challenging conditions is not thoroughly explored.\n\nQUESTIONS:\n1.\tPaper states: ‘Kv, Vv and Km, Vm are obtained by flattening the visual feature xf usion and the motion feature xcv, respectively.’ Does this imply that no additional projection layer is applied—so that K_v is actually identical to V_v, and likewise K_m to V_m, with the different symbols used only for notation?\n2.\tHow does the proposed method perform in other challenging conditions, such as foggy or rainy environments? It would be valuable to test the framework in a wider range of scenarios.\n\nLIMITATIONS:\nThe authors have discussed the limitations of their work, particularly the difficulty in estimating optical flow for textureless planes. They propose incorporating LiDAR sensors in future work to address this issue. However, the paper does not explicitly discuss potential negative societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a diffusion-based method for event-guided optical flow estimation. Its main contribution is a novel feature fusion strategy that employs triple-branch cross- and self-attention to condition the DDIM process, along with innovative designs in the diffusion denoiser incorporating GRUs, memory slot, and other elements. Experimental results demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n  1. The application of diffusion models to event-based optical flow has not been widely explored.\n  2. The experimental results appear promising.\n  3. The graphical illustrations are clear and helpful.\n\nWeaknesses\n  1. There are other works that employ diffusion models for optical flow estimation, such as FlowDiffuser (CVPR 2024), which also uses a GRU-based denoiser. What distinguishes this work from FlowDiffuser? Simply adding an event modality with a cross-attention fusion mechanism may not constitute a significant innovation.\n  2. The clarity of the paper could be improved. For instance, terms like sparse/dense appearance saturation are confusing. Saturation is typically used to describe colors, and its usage here is unclear and distracting.\n\nQUESTIONS:\n1. What is meant by “appearance saturation”? Does it refer to the abundance of appearance textures within a modality? If so, what do “dense” or “sparse” appearance saturation specifically mean — do they indicate an abundance or lack of appearance textures? Although these terms do not obscure the paper’s overall motivation (i.e., using events for boundary information and frames for appearance features, which are complementary for optical flow estimation), they can be confusing upon first reading.\n\n2. Why do the authors apply cross-attention between the time embedding and the two modality features? The time embedding primarily encodes the diffusion step index and does not appear to share semantic information with the modality features. Is the intention for the time embedding to act as a bridge between the two modalities? Clarifying this design choice would be helpful.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}