Title: Dense Metric Depth Estimation via Event-based Differential Focus Volume Prompting
Venue: NeurIPS 2025
Accepted: yes
Keywords: Event Camera

Depth Estimation

Focus Volume
Abstract: Dense metric depth estimation has witnessed great developments in recent years. While single-image-based methods have demonstrated commendable performance in certain circumstances, they may encounter challenges regarding scale ambiguities and visual illusions in real world. Traditional depth-from-focus methods are constrained by low sampling rates during data acquisition. In this paper, we introduce a novel approach to enhance dense metric depth estimation by fusing events with image foundation models via a prompting approach. Specifically, we build Event-based Differential Focus Volumes (EDFV) using events triggered through focus sweeping, which are subsequently transformed into sparse metric depth maps. These maps are then utilized for prompting dense depth estimation via our proposed Event-based Depth Prompting Network. We further construct synthetic and real-captured datasets to facilitate the training and evaluation of both frame-based and event-based methods. Quantitative and qualitative results, including both in-domain and zero-shot experiments, demonstrate the superior performance of our method compared to existing approaches. Code and data will be available at https://github.com/liboyu02/EDFV/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel approach for dense metric depth estimation. In details, the proposed approach intergrades the advantages from both event and RGB images to boost the performance of dense metric depth estimation. The event images are sent to EDFV encoder to obatin EDFV features to obtain sparse depth, while the single view RGB images are passed to a pre-trained image foundation model to get the initial dense depth prediction. A sparseNet and a DenseNet are further used to reflect both outputs into feature space. FInally, a Multi-scale Cross-attention-guided Prompting Module is introudced to fuse the image and event features to obtain the final prediction results. To evaluate the proposed methods, the authors further generate and collect two synthetic datasets and two real-world datasets, and propose the first benchmark for the proposed datasets.\n\nSTRENGTHS AND WEAKNESSES:\nPro:\n1. The proposed idea is very interesting and insightful. This pioneering work can serve as an excellent inspiration for the community.\n2. The authors conduct extensive experiments to prove the effectiveness of the proposed methods.\n3. The newly collected dataset would be a big contribution for the development of the focused research direction.\n\nCons:\n1. Although the proposed idea is very interesting, the proposed framework seems lack of novelty. The main part of the proposed EDPN is the fusion model, which utlizes a cross attention block and a fusion block. However, both cross-attention fusion and conv fusion are all widely used methods in the field of AI. The rest of the proposed framework are all directly rely on existing methods.\n2. Different from existing methods, the proposed approach utlizes a pre-trained image foundation model as the image encoder, which seems unfair compared to the other methods. The authors should prove the performance gain is not caused by this.\n3. The authors use a pre-trained visual foundation model in zero-shot learning task which may causes information leak. The improvment may leads by this.\n\nOverall, I believe this paper is more suitable for the Datasets & Benchmarks track rather than the Main track.\n\nQUESTIONS:\n1. The authors should highlight their contributions on in the framework, as it currently appears that they have merely assembled a set of existing modules.\n2. The authors should discuss the implications of using a foundation model, such as the fairness of comparisons with other baselines and the potential risk of information leakage in the zero-shot setting.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach for dense metric depth estimation by fusing events with image foundation models (IFMs) via a prompting strategy.\nThe authors propose Event-based Differential Focus Volumes (EDFVs) using events triggered by focus sweeping, transforming them into sparse depth maps to prompt dense estimation via the Event-based Depth Prompting Network (EDPN) and construct synthetic and real datasets for training and evaluation. Experiments show the method outperforms state-of-the-art approaches, achieving up to 29.1% improvement in RMSE.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Innovative Modality Fusion: Proposes a novel framework fusing event cameras with image foundation models (IFMs), addressing scale ambiguity in single-image methods and low sampling rates in traditional depth-from-focus approaches.\n2. Comprehensive Dataset Construction: Creates synthetic (Blender-Syn, Sintel-Dr. Bokeh) and real-world datasets (4DLFD-Semi-Real, EDFV-Real) with ground truth, enabling rigorous evaluation of frame-based and event-based methods.\n3. Superior Performance Validation: Achieves up to 29.1% RMSE improvement over state-of-the-art methods, excelling in in-domain and zero-shot scenarios for textureless regions, scale ambiguity, and occlusion.\n\n\nWeakness:\n1. The organization of the paper could be improved. Ablation studies, as an essential part of the experimental analysis, should be presented in the main text rather than being relegated to the supplementary material. Moreover, since the construction of the two datasets is highlighted as one of the core contributions in the Introduction, a more detailed description of the dataset creation process should also be included in the main body of the paper, rather than placing the majority of this information in the supplementary material.\n2. The comparative analysis lacks baseline methods from 2025, which may limit the completeness and timeliness of the evaluation.\n3. The experimental section in the main text appears limited in scope. There are no quantitative results reported for the 4DLFD-Semi-Real setting or the proposed dataset. Additionally, the set of comparison models does not include the event-based DFF methods mentioned in the Related Work section.\n\nQUESTIONS:\nPlease see 2) and 3) in weakness.\n\nLIMITATIONS:\nPlease see the weakness.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to fuse events with depth foundation models for metric depth estimation. The main contributions include 1) a EDFV representation that encodes focus sweeping cues from events. 2) a framework that combines events and depth foundation model for metric depth estimation. 3) novel datasets for proposed task. Experimental results on Blender-Syn and Sintel-Dr. Bokeh datasets demonstrate the superiority of proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n- The exploration of encoding focus/defocus cues from events addresses an underexplored task in the field.\n- The proposed method demonstrates superior performance compared to existing approaches.\n- The introduction of new datasets is a valuable contribution to the community.\n\nWeakness\n- The frequent use of abbreviations throughout the paper may confuse readers, reducing clarity.\n- The paper’s structure needs improvement. Notably, the Ablation Study section is recommended to put in the main text.\n- The experimental evaluation lacks comparisons with state-of-the-art approaches from the past two years, which weakens the claim of superior performance.\n- Although the 4DLFD-Semi-Real and EDFV-Real datasets are introduced, no quantitative experiments are conducted on these datasets.\n- Some typo: L197, \"we two\".\n\nQUESTIONS:\n- The necessity of using events for focus/defocus cue encoding is not sufficiently justified. The data collection process still relies on focal length changing, and the authors note that event cameras are used to record focal length changes more frequently. However, since focal length changes are relatively slow, it is unclear whether such frequent recording is necessary, as standard cameras can capture more than 30 frames per second during focal length changes. This raises questions about whether a focal stack constructed from standard cameras is sufficient for accurate depth estimation. Additionally, events triggered by factors other than focal length changes may introduce noise, potentially degrading depth estimation performance. The authors should provide further analysis to demonstrate the superiority of using event cameras over traditional depth-from-focus methods.  \n- Based on the above question, if we replace the sparse depth generation step with normal depth-from-focus approaches, and keep the subsequent steps (MCPM part) unchanged, how would the performance be affected?\n-  It is suggested to indicate the type of compared method (such as DFF and Mono) in Table 1 and 2 for better clarity.\n\nLIMITATIONS:\nThe limitation is discussed in the main text.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a depth estimation framework that combines event camera data generated during focus sweeping with image foundation models via a prompting strategy. It introduces the Event-based Differential Focus Volume (EDFV) for sparse metric depth estimation and designs an Event-based Depth Prompting Network (EDPN) to refine dense depth predictions. The authors also construct synthetic and real datasets to evaluate their method.\n\n1. EDFV (Event-based Differential Focus Volumes) for extracting sparse metric depth from events.\n2. EDPN (Event-based Depth Prompting Network) that fuses sparse event depth and dense IFM predictions to produce final accurate depth.\n3. Construction of two synthetic datasets and two real/semi-real datasets with events, AIF images, and GT depth for training and evaluation.\n4. Superior in-domain and zero-shot performance compared to state-of-the-art DFF and single-image methods across multiple datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nQuality :\nThe proposed method is well-motivated and carefully designed. The derivation of the polarity reversal property in Section 3.2 shows a solid understanding of event camera properties. Extensive experiments are conducted on four datasets (two synthetic, two real/semi-real), with both in-domain and zero-shot evaluations.\n\nClarity:\nThe paper is well-organized, with clearly separated contributions and pipeline stages.\n\nSignificance:\nThe integration of event-based sparse depth with foundation model prompting addresses the critical problem of scale ambiguity in monocular depth estimation. The event-guided prompting idea may influence future depth estimation systems or even broader tasks involving hybrid sensing and prompting.\n\nOriginality:\nThe proposal of EDFV is novel in the context of depth estimation, leveraging event-triggered defocus polarity—a concept not previously exploited in this form. The method diverges meaningfully from both traditional DFF and event-based monocular methods. The use of deformable convolutions in cross-attention for misalignment correction is an innovative adaptation to the challenges of hybrid sensor data.\n\nWeaknesses:\n\nClarity:\nSome important details are left to the Supplementary Material (e.g., more mathematical derivation of event polarity behavior, ablation studies on hyperparameters). The limitations section is quite short. A more systematic reflection on assumptions (e.g., sensitivity to event noise, synchronization of sensors) would improve transparency.\n\nSignificance:\nWhile promising, the real-world applicability is still partially constrained by the hardware complexity of event-RGB hybrid systems and the assumption of focus sweeping capability. These aspects are not discussed in depth.\n\nOriginality:\nThe idea of prompting IFMs with sparse depth is increasingly common (e.g., in LiDAR prompting literature). Although the paper adapts this idea well to event data, the prompting framework (EDPN) may not be as novel as the EDFV component.\n\nQUESTIONS:\n1. While the proposed EDFV performs well overall, it's unclear how it handles textureless regions or areas with low event density. Could the authors show or describe failure cases where EDFV performs poorly? Do you apply any fallback mechanism or fusion weighting in such cases?\n\n2. In your real-world dataset (EDFV-Real), how did you ensure temporal synchronization between the event camera and RGB sensor, especially during fast focus sweeping? Were there any delays or misalignment corrections applied? Accurate depth estimation heavily depends on synchronization.\n\nLIMITATIONS:\n1. The method relies on event triggers from intensity changes, which may be sparse or absent in textureless or poorly lit regions.\n\n2. The proposed system requires precise synchronization between RGB and event cameras and a controllable focus mechanism. These requirements limit real-world usability, especially for mobile or embedded systems.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}