Title: Distil-E2D: Distilling Image-to-Depth Priors for Event-Based Monocular Depth Estimation
Venue: NeurIPS 2025
Accepted: yes
Keywords: event camera

neuromorphic vision

monocular depth estimation
Abstract: Event cameras are neuromorphic vision sensors that asynchronously capture pixel-level intensity changes with high temporal resolution and dynamic range. These make them well suited for monocular depth estimation under challenging lighting conditions. However, progress in event-based monocular depth estimation remains constrained by the quality of supervision: LiDAR-based depth labels are inherently sparse, spatially incomplete, and prone to artifacts. Consequently, these signals are suboptimal for learning dense depth from sparse events. To address this problem, we propose Distil-E2D, a framework that distills depth priors from the image domain into the event domain by generating dense synthetic pseudolabels from co-recorded APS or RGB frames using foundational depth models. These pseudolabels complement sparse LiDAR depths with dense semantically rich supervision informed by large-scale image-depth datasets. To reconcile discrepancies between synthetic and real depths, we introduce a Confidence-Guided Calibrated Depth Loss that learns nonlinear depth alignment and adaptively weights supervision by alignment confidence. Additionally, our architecture integrates past predictions via a Context Transformer and employs a Dual-Decoder Training scheme that enhances encoder representations by jointly learning metric and relative depth abstractions. Experiments on benchmark datasets show that Distil-E2D achieves state-of-the-art performance in event-based monocular depth estimation across both event-only and event+APS settings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to distill image-to-depth priors for event-based monocular depth estimation by generating dense pseudo ground truth with DepthAnythingV2. The main challenge is that DepthAnythingV2 provides relative depth, while the goal is to estimate metric depth. The authors address this by (1) training a small MLP (NDC module) to fit the dense relative depth to the sparse LiDAR metric depth; (2) estimating a confidence map with an ACE module; (3) combining pseudo ground truth loss and LiDAR ground truth loss via loss design; and (4) using a dual-decoder architecture to enhance metric-depth training with simultaneous relative-depth training. The proposed method achieves state-of-the-art performance on the DSEC and MVSEC datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The motivation of the paper is clear, and the solution is straightforward.\n- The evaluation and ablation studies are comprehensive, showing the effectiveness of each component.\n- The proposed method achieves state-of-the-art performance on the DSEC and MVSEC datasets.\n\nWeaknesses:\n- The generalization of the method is not impressive: although in-domain performance is good, it would be better if a generally trained model could be used across different datasets.\n- It is not clear how ACE works (see \"Questions\").\n\nQUESTIONS:\nObserving Equation (11) and (12), I see that if $\\mathcal{F}\\_{conf}$ learns to trivially predict all zero $w\\_t^d$, then the second term $w\\_t^d\\cdot \\mathcal{L}\\_{align}(d\\_t^m, \\hat{y}^{syn})$ of Equation (12) will be completely reduced to zero. (According to the supplementary materials, the last layer of $\\mathcal{F}\\_{conf}$ is a sigmoid layer, so exact zeros cannot be produced, but the outputs can approach zero.) How is this collapse avoided?\n\nIf this zero-collapse happens, then all effect of the calibrated depth will be lost, meaning that the model only learns from the LiDAR ground truth (through $\\mathcal{L}\\_{mde}$) and the raw DepthAnythingV2 depth (through $\\mathcal{L}\\_{rde}$).\n\nBeyond explanation of the zero-collapse prevention methodology, I would also like to see some visualizations or statistics of the confidence maps produced by the ACE module in the rebuttal stage.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the Distil-E2D, which is a framework utilizing the prior knowledge from foundational depths models. This work utilizes DepthAnythingV2 to generate pseudo labels and complements sparse LiDAR depth maps. It introduces a Confidence-Guided Calibrated Depth Loss to align the synthetic and real depths and weight the confidence. This work additionally employs a Context Transformer and a Dual-Decoder Training scheme. Experiments also prove the good performance of the Distil-E2D.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThis paper focuses on monocular depth estimation with event cameras and provides thorough consideration and design in terms of pseudo labels, loss function formulation, and network architecture. The experimental results demonstrate the superiority of the proposed approach, and the ablation studies confirm the effectiveness of each module. The writing is clear, the content is easy to understand, and the work exhibits strong originality.\n\nWeakness:\nThere are still some confusing aspects in the manuscript, which will be detailed below and require further clarification from the authors.\n\nQUESTIONS:\n1. Some parts of Figure 1 are confusing: the arrow from DepthAnythingV2 to the “MDE Prediction dtm” looks insufficient, and according to the formula, the coefficient in $L_{\\text{align}}$ between Calibrated Depth and LiDAR Depth should be $w_t^d$ rather than $W_c$.\n2. In the abstract, the first occurrence of “MDE” should be accompanied by its full form, “monocular depth estimation.” Additionally, “MDE” and “EMDE” must be used consistently throughout the paper, many places use “MDE” (e.g., line 154, line 168) when it should actually read “EMDE”. References [28] and [37] are identical, and the authors should delete the duplicate one.\n3. Why does Equation 7 use logarithmic depth instead of the raw depth? Is this based on a particular prior or intended to expand the effective depth range? Further explanation is needed.\n4. Further clarification is required on:\n   (a) the network architectures of the encoder and decoder;\n   (b) for the MVSEC and DSEC datasets, considering poorly lit scenarios such as nighttime where the APS/RGB modality degrades severely, did the training set include only well-illuminated sequences?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe missing of large volume and high-quality datasets for event-based tasks hinders the spreading of related applications such as Event Monocular Depth Estimation (EMDE).\nFor example, depth foundation models (VFM) such as Depth Anything V2 or MoGe leverage millions of high-quality labelled data to achieve astonishing results, while this kind of training is still unfeasible for the event domain, since high-quality-high-volume synthetic or real datasets are missing.\n\nIn particular with respect to EMDE, previous literature trained the models with real datasets such as MVSEC, where ground truth is collected using LiDAR, thus producing sparse and noisy labels.\nThis paper focuses on the high-quality part of the aforementioned problem, trying to overcome the problem by exploiting frame-based depth foundational models -- i.e., Depth Anything V2.\n\nThe core idea is to leverage aligned frame and event data -- e.g., using a DVS sensor that produces intrinsically aligned frame-event information -- to add an additional proxy-supervision signal from the estimations of the foundational model.\nIn other words, given an aligned frame, we can estimate an aligned proxy depth map using the foundation model and use it to complete the LiDAR information.\n\nAdditionally, the authors introduce a series of techniques to fuse LiDAR and proxy relative depth maps: 1) a MLP model to create a non linear depth mapping to convert the proxy labels to a metric scale; 2) a CNN confidence estimator model to weight the supervision of proxy labels; 3) a double head for metric and relative depth estimation.\nThe enhanced supervision is used to train a novel transformer-based architecture for both event-only monocular depth estimation and event-frame monocular depth estimation.\n\nThe authors validated the effectiveness of the proposal with two standard benchmarks -- i.e., MVSEC and DSEC -- with state-of-the-art performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- **The paper is well written and easy to follow**: the authors present their methodology in a clear and structured way, with well-organized sections and illustrative figures that guide the reader through the key concepts.\n- **Interesting idea to partially overcome to the event-data issue**: The paper addresses an important problem in the event-based literature -- i.e., missing high-quality-high-volume datasets -- with particular effort regarding the high-quality part. The proposed distillation framework is a novel paradigm to leverage the knowledge of frame-based VFM to increase the quality of event datasets.\n- **SOTA performance**: The results show the effectiveness of the proposal against several competitors using two standard benchmarks -- i.e., MVSEC and DSEC.\n\nWeaknesses\n\n- **Fairness of the E+I experiments**: Given Fig. 2, the proposed Distil-E2D (E+I) Framework uses frozen DAv2 for estimation, thus leveraging (of course indirectly) millions of labeled images, while the other competitors do not. This raises concerns regarding the fairness of the E+I evaluation protocol.\n- **Missing details about implementations**: I would like to see more details regarding the network implementation, since there are few details about it. For example, the size of the frozen Frame Encoder -- i.e., DAv2-S or DAv2-B or DAv2-L -- is missing. Also, there are no details about the encoders. I suggest that the authors include additional information, such as figures and implementation details in the supplementary.\n- **Concerns about inference times**: I tried both MDDE+ and ERE models, but I found that MDDE+ is quite faster than ERE.\n- **Usage of only one VFM for proxy-labelling**: The authors tried only one VFM for proxy supervision and Image Encoder. I suggest in future work to try other new VFM such as MoGe, DepthPro, and Lotus.\n- **Missing study regarding the high-quantity dataset issue**: The authors missed the opportunity to study the effectiveness of the proposal regarding the high-quantity problem cited in the Summary section. In particular, the authors could have tried to increase the robustness of the network with a proxy pre-training using a large amount of aligned frame-event unlabelled data and proxy supervision from VFMs (leveraging the RDE head). I suggest this as future work for the authors.\n\nGiven the Strengths and Weaknesses of the proposal, I give an initial score of \"Accept\". In my opinion, the event-based high-quality-high-quantity dataset issue is a huge problem for the event community that hinders its diffusion.\nThe authors managed to handle the problem partially with an interesting solution.\nFurthermore, I did not find any major weaknesses, thus confirming my positive score.\n\nQUESTIONS:\n- Could you provide additional information regarding the network implementation as requested in the previous section?\n- Could you confirm the inference times of MDDE+ and ERE models?\n- Instead of using relative VFM, have the authors tried to use metric foundational models?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a model trained to predict dense depth maps from event-based cameras. It works by distillating priors about geometry of dense depth maps by using zero-shot depth prediction from a foundation model (DepthAnything-V2) on co-registered grayscale or RGB images that are \"synchronized\" with the event signals, gathered within a 3d block of event voxels over a fixed time interval. \nTo transform the relative depth maps predicted by DepthAnything-V2 to metric (absolute) depth maps, a calibration model is trained, supervised by LIDAR depth projected images, to learn a non linear calibration function. The calibrated synthesized dense depth maps are then used to supervise the event-base depth prediction model.\nThe proposed architectures uses a context branch fed by the past dense depth map prediction, that is combined to the input event data through a cross attention module performed by a transformer at the end of the decoder, before going through the decoder that outputs the dense depth maps.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is clear and nicely written. The problem of estimating depth maps in nighttime or tunnel conditions is important so the subject is relevant. The proposed architecture and distillation setup make sense. The experimental part is well done, the results are good and the ablation study shows the benefits of the different components of the approach.  \nOne possible weakness of the approach is the strong dependence to the context branch. One might wonder if this method could generalize to different contexts (e.g. handheld or aerial, with fast viewpoint changes), and how the past depth maps possibly distort (blur?) the current prediction.    \nAside from this minor point I don't see major issue in this paper, except a few clarifications to add (see questions).\n\nQUESTIONS:\nCould you observe or try to analyze the possible distortions induced by the context branch in terms of excessive dependence to the past (in particular in case of accelerations)?\nPlease introduce and make clear the APS acronym and the distinction with RBG images. \nIt is not clear BTW why the APS image appears as binary (saturated?) in Figure 1.\nThere is a notation problem in equation (5), since i is the sum index, it should be \"The voxel value at (\\mathbf{u},t)...\" and \"V(\\mathbf{u},t) =\" instead of (\\mathbf{u}_i,t)\n\nLIMITATIONS:\nOne word of discussion about generalization and the possible strong dependence to the context could be added.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}