DERD-Net: Learning Depth from Event-based Ray Densities
Authors: Diego de Oliveira Hitzges
Suman Ghosh
Guillermo Gallego
Venue: NeurIPS 2025 spotlight ()
Keywords: Event-based vision
Depth estimation
Multi-view stereo
Deep learning
Disparity space images
Primary area: applications
TL;DR: We present a deep-learning–based method to predict depth from event camera data by creating disparity space images and using their ray densities.

Abstract: Event cameras offer a promising avenue for multi-view stereo depth estimation and Simultaneous Localization And Mapping (SLAM) due to their ability to detect blur-free 3D edges at high-speed and over broad illumination conditions. However, traditional deep learning frameworks designed for conventional cameras struggle with the asynchronous, stream-like nature of event data, as their architectures are optimized for discrete, image-like inputs. We propose a scalable, flexible and adaptable framework for pixel-wise depth estimation with event cameras in both monocular and stereo setups. The 3D scene structure is encoded into disparity space images (DSIs), representing spatial densities of rays obtained by back-projecting events into space via known camera poses. Our neural network processes local subregions of the DSIs combining 3D convolutions and a recurrent structure to recognize valuable patterns for depth prediction. Local processing enables fast inference with full parallelization and ensures constant ultra-low model complexity and memory costs, regardless of camera resolution. Experiments on standard benchmarks (MVSEC and DSEC datasets) demonstrate unprecedented effectiveness:
(i) using purely monocular data, our method achieves comparable results to existing stereo methods; (ii) when applied to stereo data, it strongly outperforms all state-of-the-art (SOTA) approaches, reducing the mean absolute error by at least 42\%; (iii) our method also allows for increases in depth completeness by more than 3-fold while still yielding a reduction in median absolute error of at least 30\%. Given its remarkable performance and effective processing of event-data, our framework holds strong potential to become a standard approach for using deep learning for event-based depth estimation and SLAM.

DERD-Net: Learning Depth from Event-based Ray Densities
Diego Hitzges*!
Suman Ghosh"
Guillermo Gallego 1.2
Technische Universität Berlin, Einstein Center Digital Future, Robotics Institute Germany Science of Intelligence Excellence Cluster, Germany
poses
Disparity Space Image (DSI)
depth
Back-
DERD-Nat
projection
DSI
Figure 1: Overview. We present a deep-learning based method to predict depth from event-ray densities (Disparity Space Images DDSIs) obtained by back- projecroje events using camera poses. Our deep neural network. DERD-Net. operates in parallel on local volumetric neighborhoods of the DSI data, called Sub-DSIs (in orange).
Abstract
Event cameras offer promising avenue for multi-view stereo depth estimation and Simultaneous Localization And Mapping (SLAM) due their ability to detect blur-free 3D edges at high-speed and over broad illumination conditions. However, traditional deep learning frameworks designed for conventional cameras struggle with the asynchronous, stream-like nature of event data, their architectures are optimized for discrete, image-like inputs. We propose scalable, flexible and adaptable framework for pixel-wise depth estimation with event cameras in both monocular and stereo setups. The 3D scene structure encoded into disparity space images (DSIs), representing spatial densities of rays obtained by back- back-projecting events into space via known camera poses. Our neural network processes local subregions of the DSIs combining 3D convolutions and recurrent structure recognize valuable patterns for depth prediction. Local processing enables fast inference with full parallelization and ensures constant ultra-low model complexity and memory costs, regardless of camera resolution. Experiments on standard benchmarks (MVSEC and DSEC datasets) demonstrate unprecedented effective- ness: (i) using purely monocular data, our method achieves comparable results to existing stereo methods; (ii) when applied to stereo data, strongly outperforms all state-of-the-ar (SOTA) approaches, reducing the mean absolute error by at least 42%; (iii) method also allows for increases in depth completeness by more than 3-fold while still yielding a reduction in median absolute error of at least 30%. Given its remarkable performance and effective processing of event- data, our framework holds strong potential to become a standard approach for using deep learning for event-based depth estimation and SLAMM. Project page: https://gthuub.oo//bbbppppp
Equal contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Depth estimation fundamental task in computer vision, with key applications in areas such as robotics, autonomous driving, and augmented reality. Traditional stereo vision techniques rely on synchronized cameras to capture images and infer depth by finding correspondences between them. However, these methods often struggle in low-light and fast-motion conditions. Moreover, conventional cameras produce large amounts of redundant data by capturing entire images at fixed intervals, leading to inefficiencies in both data storage and processing.
Unlike conventional cameras, event cameras operate asynchronously. detecting per-pixel brightness changes (called "events") [3). This provides high temporal resolution and robustness to motion blur, making them well suited for dynamic scenes. Their sparse output enables efficient processing of only relevant areas, making them ideal for real-time tasks such as visual odometry (VO) / SLAM.
Harnessing deep learning for depth estimation with event cameras has the potential to transform such applications. However, adapting neural networks to event data remains challenging due to its asynchronous stream-liks nature. Moreover, the scarcity of event camera datasets with ground truth depth 5 results in limited training data, which can lead to overfitting [6]. While simulating data is one way address this issue, generalizing models trained simulation to real- scenarios is far from trivial due to differences in data distribution [7H10]. Thus, instead of directly processing events (which is prone to overfitting and extensive training time) rely on intermediate event representations informed by the scene geometry.
One promising approach for 3D reconstruction is back-projecting events as rays into space and capturing their intersection densities as disparity space images (DSIs) [II]. DSIs from two or more cameras can be fused, eliminating the need for event synchronization between cameras. This reduces complexity and allows for more robust depth estimation. The
Event-based Multi- View Stereo (MC-EMVS) method [12] recently produced state-of-the-ans art (SOTA) results, outperforming other techniques in depth benchmarks across several metrics. To obtain pixel-wise depth estimates from DSI, MC-EMVS selects the disparity level with the highest ray count, effectively using an argmax operation. Ray counting is used as a proxy for finding 3D edges where the rays intersect. A selective threshold filter is applied predict depth only for pixels with sufficient ray counts.
While straightforward. this approach does not fully utilize the potential of DSIs. It is susceptible to noise and cannot effectively extract cues from surrounding pixels and more complex patterns across disparity levels. Consequently, it leads to fewer pixels obtaining depth estimation and less accurate depth predictions than the DSI might potentially allow for. Therefore, need more effective approaches for extracting depth from DSIs, that are more reliable, accurate, and produce more depth estimates, by adequately recognizing complex ray intersection patterns.
Our Contribution. We propose a novel deep learning framework for event-based depth estimation that is optimized for SL.AM scenarios and addresses the aforementioned limitations. An overview is illustrated in Fig. [ Our approach estimates pixel- depth from a DSI using a neural network with 3D convolutions and a recurrent structure. The framework is directly applicable to both monocular and stereo settings. Our key contributions include:
Learning-based Local Processing: For each selected pixel, small local subregion of the DSI (Sub-DSI) is used as input to the neural network. This novel design leverages the inherent sparsity of event data and DSIs to efficiently process and produce only relevant information for sparse depth-related tasks. (Sec. 3
Enhanced Data Utilization: Our model captures complex patterns within the DSI, increasing depth prediction accuracy and the number of pixels for which depth can be reliably estimated. Limiting the input to small local subregions around selected pixels enhances generalization by preventing the network from overfitting to dynamics specific to the training scenes and augmenting the available training set, as each Sub-DSI serves as an individual data instance.
Efficiency and Scalability: By adopting our Sub-DSI approach, we obtain small independent inputs of fixed size. This enables full parallelization and provides an ultra-light network that has the ability to handle any camera resolution with constant very short inference time. Our network architecture allows the processing of DSIs of variable depth resolution. (Sec. (.2c.
Comprehensive Experiments: We evaluate our model on both monocular and stereo data from the standard datasets MVSEC (4] and DSEC [3 using cross-validation. It outperforms the

---

state of the art by a large margin on ten figures of merit. Even using monocular data, our model achieves performance comparable to SOTA methods that require stereo data (Sec. We show downstream applicability and robustness to imperfect (noisy) camera poses.
To the best of our knowledge, our work is the first learning- based multi-view stereo method to (i) use camera poses along with events as input, which is crucial for accurate depth estimation over long intervals (as used in SLAM [5); (ii) demonstrate successful and robust depth prediction real-world event data from DSIs; (iti) report good generalization on all three MVSEC indoor flying sequences (5], even when compared to multi- -odal methods that combine stereo intensity frames with events. We provide code, trained models and video results for clarity and reproducibility.
2 Related Work
Stereo depth estimation with event cameras has been captivating problem since the invention of the first event camera by Mahowald and Mead in the 1990s [3] due to their potential for high temporal resolution and robustness to motion blur. Recent approaches have addressed stereo event-based 3D reconstruction for VO and SLAM 17 22). These methods assume a static world and known camera motion, using this information to assimilate events over longer time intervals, thereby increasing parallax and producing more accurate semi-dense depth maps. A comprehensive review is provided in [5].
MC-EMVS 12] introduced a novel stereo approach for depth estimation which does not require explicit data association, using DSIs generated from stereo events cameras. By leveraging the sparsity of events and fusing back-projected rays, they outperformed the manntt-ttching--e- based solution of [19] and thereby achieved SOTA results in 3D reconstruction and VO (14). Evidently, this DSI- based 3D reconstruction robust to imperfect poses estimated using an event camera tracking method. We advance this approach by employing a compact neural network specialized to derive explicit depth from the DSIs, creating standardized and effective framework for processing event- weeet-based data in deep learning applications that does not rely on event simultaneity matching.
Deep Learning for depth estimation from event data. Deep learning has significantly advanced depth estimation in traditional monocular and stereo camera setups, achieving remarkable re- sults (5) However, its application event camera data remains relatively limited due to the sparse asynchronous nature of event streams, which require specialized frameworks [5]. For example, in monocular vision, (26 uses synthetic data on recurrent network to capture temporal information from grid-like event inputs. Yet, mismatches between synthetic and real data degrade performance [27], and monocular depth estimation from events an ill-posed problem, making high accuracy challenging to achieve with this learning- based framework (28).
For stereo depth estimation, 6 29 present two pioneering studies. Specifically, DDES 66 introduced
the first deep-learning-basee supervised stereo-matching method, while (29 proposed the first unsupervised learning framework. Both methods use First-In First-Out queues to store events at each position, allowing for concurrent time and polarity reservation. Nevertheless, high event rates lead to greater processing demands and, consequently, increased model complexity and memory requirements, limiting the use of visual cues from both event cameras. Our method overcomes these challenges by maintaining constant input dimensions defined by the size of the DSI subregion around a selected pixel, regardless of the event rate. It thereby provides significant advancement in applying deep learning to event-based depth estimation on real-world data.
3 Methodology
In this section, we present our peeervised-lcarning--e detail, for which a general overview is provided in Fig. [ We describe the preprocessing of the data, the architecture of our network (shown Fig. 2h and the training and inference procedures.
approach and the related framework in
3.1 Framework
As an event camera with H pixels moves through a scene, triggers events eg (Ir,, yks Les +x.) and produces near continuous-time stream of data E = {ex}. Following [1] 12
this stream is
3

---

Neural Netwock
Input
Ousput
Sub-DSI
SD Canolation
GRU (dept Ityer)
Dane + ReLU Deuse
Depth
Figure 2: Network Architecture. The parameters of the network's s modules are specified in Tab.
Table 1: Details of network layers. P. and S stand for kernel-size, padding and stride.
Layer
Dimensions
Details
Sub-DSI DSI (Input)
100 x 7
Depth x Channels Width x Height K (3,3,3); P (1,0,0;; S (2,1,1) Flattens channels and frame Selects final hidden state hyo Maintains dimension
Convolution
50 x 5
ReLU Flatten
50 (4-5-5)
GRU
1x 100
Dense ReLU
100
Dense (Output)
3r x
Outputs depth value(s)
sliced into time intervals. For every time interval, a DSI is created and associated with a camera viewpoint (called Reference Viewpoint) follows: given camera poses, all events 6 in the interval are fack-projected into 3D space by casting rays from the moving camera optical center through the corresponding pixel (3*;, yk)) The depth axis is discretized into D levels, equidistant in inverse linear space, resulting in a 3D DSI of size D x W X И voxels, whose values represent the number of rays passing through each region (i.e., voxel) of space (as shown in Fig. [ Although input poses required for building a DSI, they can be obtained from tracking methods or dead-reckoning [2 14 30).
We consider stereo setup with two synchronized cameras providing scene. By leveraging parallax, this configuration enhances depth perception and allows for more accurate 3D reconstruction. For each interval, construct two DSIs (one for each camera) and fuse them by applying voxel-wise metrics (e.g., harmonic mean) as described (12). To compare performance, also apply our approach to the monocular data of the left camera only.
perspectives of the same
Since DSIs are typically large and sparse, depth estimated only for pixels with sufficient information. A confidence map is generated by projecting the DSI onto a 2D grid of size W x H, where each pixel's 's value represents the maximum ray density among all depth levels [12] (called pixel selection map in Fig. B). An adaptive Gaussian threshold (AGT) filter is then applied to this grid to select the pixels [pi..... Pa} with a sufficient maximum ray density for reliable depth estimation. For each selected pixel pæ = (I;, yi), surrounding subregion Š, is extracted from the DSI, including the ray counts of all pixels within L1 radii of rw,
S, = DSI( :, 2; - rw x, TW, Vi TH : De + rH)-
Each subregion S, is then normalized individually,
S = S mš.) E 1) (1)
and serves input neural network. Using only small subregions of the DSIs as inputs leads to an efficient and compact architecture operating independently of camera resolution. Furthermore, it reduces the risk of overfitting by encouraging the model to learn generalizable patterns of ray intersections within the Sub-DSIs instead of memorizing semantics and dynamics specific to the training scene. Such localized input processing is possible because DSIs consolidate sparse events into a structured format that preserves geometric information within small spatial regions.
The depth estimates 21,..., are computed in parallel. Since the amount of selected pixels can be controlled by the AGT filter and the dimensions of the Sub-DSIs are fixed, we achieve constant low model complexity and memory costs, regardless of the number of triggered events.
3.2 Network Architecture
The architecture of the neural network illustrated in Fig. with the dimensions of each layer listed in Tab. The network receives the normalized Sub-DSI ①1 as input. The objective is to capture local geometrical patterns in the Sub-DSI extract more relevant depth information than the SOTA argmax approach used in 12 Since established networks like U-Net (31] often include strong spatial

---

Table 2: Hyperparameters.
Dataset Details
DSI Parameters
Gauss. Filter
Training Process
Dataset
Sequences
Resolution
LiDAR A+
Span
Amase
D
Sub- DSI
Window
Butch Optimizer LR
LF
Epochs
MVSFC Indoor flying 346 x 260 px
50 ms
1s m 6.5 m 100
7x7
9x9
-10
64
AdamW
10-3 MAR
DSEC
Zurich04a
640 x 480 px
100 ms
0.2s
4m
50 m 100
7x7
9x9
-2
64
AdamW 10-3 MAB
compression and Transformers tend to impose high data demands for reliable generalization [3] we tailor an ultra-lightweight custom architecture that shares design elements with FireNet [33 adapted for very small frame sizes yet variable depth dimensions. Similarly to RAFT-Stereo [. we adopt convolutions and a Gated Recurrent Unit (GRU) to efficiently handle different depth resolutions, enabling customization of the desired depth precision without modifying architecture.
First, to capture local patterns, further reduce input size and avoid overfitting, we apply a 3D convolutional filter (3D-Conv) with ReLU activation, kernel size of 3 x 3 x 3 and no padding in the spatial dimensions (width and height). For the depth dimension, we set padding of and a stride of 2 to halve the number of depth layers. We use 4 output channels to capture different patterns simultancously, resulting in the convolved version of the Sub-DSI (T]
S = 3D-Conv(S.) E R xxx 11) (2-xx=1
(2)
To create the DSIs, rays were cast from the representative camera location into space, passing sequentially through the different depth levels. Since mapping precision requirement may change from scene to scene, we need to deal with variable depth resolution of the DSI. To efficiently and flexibly model this interdependence of consecutive depth layers for a variable D, the convolved depth layers are flattened and successively fed into GRU (35). The recurrent structure of the GRU allows us to maintain a constant ultra-low count of only 70k parameters in total. iteratively embeds each depth layer's 's information within the context of the previous layers, producing hidden state representations hy.... hp/2. We then proceed with the final hidden state hpy which condenses the relevant information from all depth layers along the depth axis:
h g = GRU(S;) E TR. 4-2w -1)-
Finally, a dense layer that preserves the dimension of hp/2 with ReLU activation and subsequent output dense layer are applied to process the hidden state. We introduce two versions of the network for custom modification of depth estimation density. In the single-pixel version, the network predicts the normalized depth for the selected pixel 2; E [0. 1], while the multi-pixel version, the output is a 3 x 3 grid Z; E [0. 1|3x3 representing the normalized depth predictions of the central pixel and its 8 neighbors. Finally, normalized depth is converted into actual depth by mapping [0, 1] to [zmin: 2max)
3.3 Training and Inference
As supervised loss function for training the neural network model we use the mean absolute error (MAE), with given ground truth depth (this is the case of standard real-world datasets used, such as MVSEC and DSEC- - see Sec. 4.. To reduce training time and further improve generalization, we additionally employ ensemble learning (EL) [36 37] For training. we initialize two identical but independent instances of our neural network with different random weights. The training set is split into two disjoint subsets, enabling parallel training. During testing or inference, each Sub-DSI S; is processed simultaneously by both networks, and the final depth estimation is obtained by averaging the individual predictions. This helps reduce variance the predictions, leading to more stable and accurate results. We also present results without EL in Tabs. II to 14] in the Appendix Sec. B
4 Experiments
In this section, we evaluate the performance and reliability of the proposed depth estimation approach. Following prior protocols, we conduct experiments on the MVSEC (4) and the DSEC [13] datasets. Ground truth (GT) depth, captured at fixed intervals using LiDAR sensors, serves as reference locations for constructing the respective DSIs over a defined time span. Pixel selection for depth estimation based on an AGT filter, where the window size determines the surrounding pixel count considered, and constant C' is subtracted from the observed ray count.
5

---

Table 3: Summarized quantitative comparison of the proposed methods with the state the art. MVSEC indoor flying and DSEC Zurich City 04 a. The full comparison over ten metrics is in the Appendix Sec.
Method
MVSEC
DSEC
Mean Err
Median Hir
had pix
#Points [million]]
Mean Err Median
had pix
#Paints [million]]
Algorithm
Modality
[em]
[em]
[S]]
[m]
[m]
[[]]
EMVS
11
monocular Fivis
33.78
14.35
3.84
1.27
5.64
2.52
13.68
1.31
EMVS
monocular Farer
50.32
20.81
11.46
4.15
7.01
3.56
24.33
6.09
ESVO
19
22.70
9.83
2.83
1.56
3.93
1.62
10.54
9.40
SGM
stereo
35.42
12.35
6.39
14.46
6.74
1.58
15.25
8.30
GTS
389.00
45,43
38.45
26.24
1.62
32.56
0.11
MC-EMVS
12
stereo Picia
20.07
9.53
1.35
0.81
3.27
0.90
10:75
1.25
MC-EMVS
sterco Famnr
28.38
12.38
3.26
2.77
4.76
1.56
17.42
4.64
MC-EMVS
MF
stereo Forig
20.64
9.72
1.43
3,00
3.51
0.96
11.81
3.83
DERD-Net
monocular Fong
23.68
11.55
2.78
1.21
3.12
1.60
5.50
2.10
DeRD-Net
monocular Flamser
28.52
4.87
6.09
DERD-Net
stereo Firiz
11.69
5,50
0.89
0.79
1.61
0.46
4.12
1.67
DeRD-Net
stereo Farer
15.24
6.68
1.70
2.77
1.80
0.54
5.04
4.64
DERD-Ne Net (multi (multi-pixel) stereo Peris
12.02
5.63
0.90
4.32
1.59
0.47
3.81
6.59
DERD-Net (multi-pixel) stereo Famnerr
15.68
6.73
1.74
11.33
1.79
0.54
4.61
14.74
We investigate the impact of DSIs derived from both monocular and stereo settings during training Prooepprovidiie and testing. Table provides an overview of the key parameters used in the datasets and the training processes of the experiments. Abbreviations are: minimum depth (zmin), maximum depth (Zmax), depth dimensions (D), filter window size (Window), subtractive constant (C), batch size (Batch), learning rate (LR), and loss function (LF).
4.1 Metrics
The performance of the networks is evaluated using ten standard metrics commonly employed in depth estimation tasks (12). We calculate both mean and median errors between the estimated and GT depths, with median errors providing robustness against outliers. Additionally, we report the number of reconstructed points, reflecting the algorithm's 's ability to generate valid depth estimations, and the number outliers (bad-pix (40)), representing the proportion of significant depth estimation errors. In the Appendix, also compute the scale-invarian logarithmic error (SILog Err) to evaluate the error while considering scale, and the sum of absolute relative differences (AErrR) to assess the relative accuracy of the depth predictions. Finally, report 5-accuracy values, which indicate the percentage points whose estimated depth falls within specified limits relative to GT [41]
4.2 Baseline Methods
We compare approach against several SOTA methods that have been benchmarked the task of long-term event-based depth estimation (5). thus evaluated under the same input conditions (events and camera poses) and output format (semi-dense depth maps) supportive of SLAM. In the absence of other deep stereo methods that learn from input camera poses, we also include comparisons against the SOTA instantaneous andtoo-end learning-base based stereo methods in Sec. S.c. for completeness. We adopt the same train-test splits established in prior work and standard benchmarks (5)
The Generalized Time-Based Stereovision (GTS) method (39 utilizes two-step process: first performing stereo matching based on a cime-consssenncy score for each event, followed by depth estimation through triangulation. The Semi-Global Matching (SGM) method (38) is adapted for event-based data by generating time images and subsequently applying stereo matching, with the depth map being refined by masking it at the locations of recent events. Another method, Event- based Stereo Visual Odometry (ESVO) [19] (ESV02 42 integrates depth estimates by employing Student-t filters, ensuring robust spatio-temporal consistency between stereo time image patches.
The two closest baseline methods performance comparison of our method are EMVS for monocu- lar vision [1] and MC-EMVS for stereo vision (12). Both methods extract pixel-wise depth from DSIs by applying the argmax function. To ensure consistency and fairness, we benchmark the methods following the procedure established in prior works [5).

---

Scene
Pixel selection map
MC-EMVS 12
MC-EMVS 12
DERD-Net (Ours)
Ground truth (GT)
with Fdenser
Figure Depth estimation. Qualitative comparison of depth estimated using the MC-EMVS method [12). applying it to the new selected pixels Faense and our method DERD-Net, for the MVSEC indoor_lying [4] (top rows) and DSEC
Sitich_iityy04__ (bottom row) sequences. Ground truth depth from LiDAR is masked by pixels with valid depth estimate. Our method estimates depth even at pixels with no GT depth. Depth maps are pseudo-colored, from blue (close) to red (far),
in the range 1-6.5m for MVSEC and 4-50m for DSEC.
4.3 Experiments MVSEC Dataset
This section describes the experiments conducted on the undoor_flying sequences 1,2,3 of the MVSEC dataset [4) evaluate the performance of the proposed depth estimation method. Most stereo methods do not evaluate on indoor foroorfflggg__ 44 (because of noisy events from the low-texture floor as the drone flies very low) and the driving sequences (because the stereo baseline is too small for the given depth range and low camera resolution). We employed three-fold cross-validation by utilizing two sequences for supervised training and reserving the remaining sequence for testing, repeating this process for all three possible combinations of sequences to ensure robustness in our evaluation.
Two pixel-selection filter settings: Forig and Fdenser- We first trained the single-pixel version of our network on monocular DSIs to compare its performance to EMVS [I]. Subsequently, we retrained it stereo DSIs fused via the harmonic mean and compared its performance MC-EMVS (12). These two baseline methods used an AGT filter Forig with a window of 5 x px and subtractive constant of Carig 14. Given that our network is designed to extract additional information from the geometrical patterns within the DSI, we hypothesized that it would still produce reliable depth estimates for pixels with lower confidence. To test this hypothesis, used a larger filter window size of 9 x 9 px and a subtractive constant of Cdenser = 10, resulting in a less strict filter F enabling depth estimation for more pixels. To ensure representative comparison, evaluated our networks as well as FMVS and MC-EMVS on both sets of pixels created by Forig and Faenser-
Multipixel vs. Morphological Filter. One apparent drawback of MC-EMVS is the limited number of pixels for which depth estimated compared to other SOTA methods. To address this, [12] presented the option of adding 4-neighbor morphological filter (MF), which dilates the depth estimation map to increase the number of septh-estimated pixels. We compare this with our framework' 's ability to further increase the number of eepth-estimated pixels by training and evaluating the multi-pixed version of our network.
The averaged results over all three sequences are displayed in the left half of Tab. 3 for all discussed modalities, while the individual results (including performance without FL.) are detailed in the Appendix, in Tabs. 12 to 14 Notably, our network's performance converges after only 3 epochs
Results.
7

---

of training. This rapid convergence is particularly advantageous for future applications where the network might be trained on more heterogeneous datasets or retrained for specific scenarios.
Monocular setup. On the monocular DSIs filtered by Fari. our single-pixel network achieves results comparable those of stereo SOTA methods and significantly outperforms EMVS (TT) by 30% in MAE. Remarkably, even on the 3.27 times larger set of pixels created by Fdenser. it still achieves better scores than EMVS at Fori for all metries, except bad-pix. Applying EMVS to the same expanded set of pixels leads to a 76% increase in MAE compared to our framework.
Stereo setup. Applying our single-pixel network to stereo DSIs filtered by Faenser allows to predict depth at significantly more pixels than any other method, except for the SGM method (38). while consistently surpassing all benchmarks across all metries. The number of pixels increases by 242%, while the MAE and MedAE reduce by 24% and 30%, compared to MC-EMVS [22 with Forig- The only exception is the bad-pix measure, where MC-EMVS performs slightly better. When both methods are compared on the same set of pixels, our approach yields a reduction in both MAE and MedAE of 42% for Forig and 46% for l'dessrr respectively. Performance remains consistent when using the multi-pixel network, which increases the amount of depth-estimated pixels by a factor of 5.47 for Forig and 4.09 for Faenser compared its single-pixel version, indicating it be superior approach to the morphological filter of MC-EMVS, which only rises the number of points by a factor of 3.70 for Forig As consequence, the multi-pixe, version of our framework estimates depth for almost as many pixels as the SGM method while delivering new SOTA results.
Qualitative comparison. To further illustrate the effectiveness of our method, Fig. 3 compares depth maps generated by our single-pixel network against those produced by SOTA method MC-EMVS. Our network not only provides denser depth estimation, which improves the recognition of contours, but also effectively eliminates the visible outliers produced by MC-EMVS. This improvement is evident when comparing our method to MC-EMVS applied both the expanded and the original set of pixels, highlighting the robustness and superiority of our approach.
4.4 Experiments on DSEC Dataset
To assess the applicability of our network architecture to different data, we retrained and tested on DSIs obtained from stereo setting in the DSEC dataset (13). This dataset presents unique challenges due to its outdoor driving scenarios, which differ significantly from the indoor environments of the MVSEC dataset, its higher spatial resolution (640 x 480 px) and different noise characteristics (Prophesee camera vs. DAVIS346 camera). Moreover, straight driving sequences are especially challenging for event-based multi-view stereo due to the little motion parallax present in them.
Setup. We select the commonly used evaluation. We split the sequence into two halves for training and testing. The DSIs were created by fusing the left and right DSIs via the harmonic mean. Analogous to Sec. 4.3 we use the original filter from MC-EMVS [1] Forig with window size of 5 x 5 px and Cong
Citt__ittaaa___ 04_ 04_a sequence to provide a focused in-depth
and denser filter Faenser with a window size of 9 x 9 px and Cdenser = 22 First, the network was trained for 3 epochs on the DSIs of the first half of the sequence and tested on those of the second. The process was then reversed and each network was used to predict in its testing half of the data sequence.
Results. The results of these experiments are displayed on the right half of Tab. Band illustrated in Fig- 3 Our approach drastically outperforms every other method across all metrics, with our multi-pixel network achieving even slightly better performance than the single-pixel network. For Forig it reduces the MAE by 55% on a 1.72x higher number of pixels compared to MC-EMVS with morphological filter. For F depth estimation density is increased by an additional factor of 2.24 while performance remains mostly stable, yielding a reduction in MAE of 62% compared to the argmax operation from MC-EMVS. Remarkably, even on purely monocular DSIs filtered by F'cnsser our framework achieved superior performance to all benchmarked methods for every metric except MedAE. These results underscore the robustness and versatility of our approach, even in complex real-world outdoor scenes.
4.5 Robustness of DERD-Net compared to other deep-learning stereo methods
Since there are no comparable learning-batt based methods that use prior camera poses, Tab. compares end-to-end lcarning-basse stereo methods, which are "instantancous" (do not take into account
8

---

camera poses) and output dense depth. In order to use their output for efficient VO/SLAM, we would need an extra step of extracting features (keypoints). Instead, DERD-Net's semi-dense depth maps help avoid unnecessary computation by outputting 3D edges for direct visual odometry, as in [4]
We use the same train-test splits established as the other learning- based methods (5). While ab- solute accuracy is not directly comparable, eval- uating the errors in the different splits relative to each other is informative about robustness: we observe that our method is the first one to gener- alize robustly across all three sequences (Tab, ( No other method reports good generalization on "split 2n of MVSEC because of the difference in dynamic characteristics of events in training and testing on that split (6 43). This is true even when compared to hybrid approaches, despite them also using stereo intensity frames ("2E+2IF" input data modality). The observed robustness of our method to such shifts may be supported by the architectural choice of processing only small subregions as input (see Tab. [ for Sub- DSI frame size), which encourages the model to learn generalizable patterns within the Sub-DSIs rather than memorizing global scene layout or sataset-specific context.
Table 4: Mean depth error [cm] of deep stereo methods on MVSEC indoor data. Values are col- lected from original sources.
Method
Modality
Split
Split
Split
DDES [6
2B:
16.7
29,4
27.8
EIT-Nel 43
2B
14.2
19.4
DTC-SPADE
2E
13.5
17.1
et 45
2B
20
25
31
StereoSpike
46
2B
16.5
18.4
ASNet 47
21:
20,46
28.74
22.15
Ghosh et
48
2B
12.1
15.6
Chen et
49
21:
13.9
14.6
StereoFlow
50
2B
EIS (ICCV 2021)
2B 2F
13.74
18.43
22.36
SCS-Net 51
2E + 2F
11.4
13.5
N. Uddin cl
29
2E 2F
19.7
26.4
Zhao al.
2B 2F
9.7
11.1
DERD--Nt
21
11.69
11.11
12.28
4.6 Sensitivity Analyses
In this section we carry experiments varying the settings in Tab. 2 Furthermore, we analyze the robustness of our method to noisy camera poses obtained from an event-based SLAM system.
Sensitivity with respect to sub-DSI size. Varying the horizontal and vertical extent of the Sub-DSIs has an impact on our method's ' performance. Our experiments show that the performance of DERD- Net can be improved by increasing the frame size of the Sub-DSIs, at the expense of increasing the network complexity (e.g.. parameter count and computational cost). See Appendix Sec. A.I
Sensitivity with respect to DSI transformations. We analyzed how DERD-Net behaves in the case previously unseen but structurally similar environments, obtained by means of horizontal and vertical flips of the DSIs Although its performance worsened slightly, it still outperformed all baseline methods. This demonstrates robustness to the aforementioned transformations. See Appendix Sec. A.2
Sensitivity with respect noisy camera poses. To assess the importance of having accurate camera poses during DSI construction, we test our framework using noisy poses with drift, mimicking real-world SLAM conditions. Instead of ground use poses estimated by the stereo event-based VO system ES- ES-P [14, which reports an Absolute Trajectory Error (ATE) of 131.62 cm over a 50 m-deep scene in the Zurich. Satrich_Ctty__ 04 sequence Running DERD-Net with these imperfect poses yields the results shown in the top rows of Tab. 5 The percentage values in parentheses denote the relative differences with respect to the performance obtained using ideal (GT) poses (Tab. 3). Remarkably, performance improved across all metrics (likely due to the slight reduction in the number of evaluated points of comparable magnitude). demonstrating strong robustness of DERD-Net to noisy poses obtained from an event-based SLAM system.
(GT) poses from LiDAR IMU odometry, we
We conduct an additional experiment where the original DERD-Net depth predictions were used to re-estimate the camera poses (in an offline manner, using the camera tracking module in (14). The resulting poses were then used to build DSIs on which DERD-Net was evaluated. This "reprojection" loop allows us to assess, using standard depth-based metrics, the robustness of our method noise in camera poses introduced by DERD-Net's own depth inaccuracies. The results are reported in the bottom rows of Tab. [5] The performance shows only minor degradation, particularly for Forig with metric worsening by more than 13%. Remarkably, even under such self-induced pose noise,

---

Table 5: Depth estimation performance on DSEC zurich zurich_city 04_004400 a using poses computed by ES- PTAM or by camera tracking on DERD-Net's output ("Reprojection" rows). Relative changes with respect to Tab. 3 which reports results obtained using GT poses, are presented parentheses.
Mean Err Median Err
bad-pix [%] +
MPoints [million]]
Algorithm
Poses
Filter
[m]
[m] 4
1.56
0.45
3.84
1.61
DERD-Net
ES-PTAM
Fong
(-3.11%)
(-2.17%)
(-6.8%)
(-3.59%)
1.74
0.52
4.84
4.49
DERD-Net ES-PTAM
Fdease
(-3.33%)
(-3.7%)
(-3.97%)
(-3.23%
DERD-Net
Reprojection
Forig
1,66
0.49
4.19
1.46
(+3.11%)
(+6.52%)
(+1.7%)
(-12.57%)
1,95
0.60
5.65
4,09
DERD-Net Reprojection
Fdeeser
(+8.33%)
(+11.11%)
(+12.1%) (-11855%)
DERD-Net's depth estimation errors remain roughly 50% lower than those prior SOTA methods using ideal poses. These results demonstrate the practical viability of deploying DERD-Net as depth-estimation module within self-sustaining SLAM system. Overall, our experiments confirm that DERD-Net remains remarkably robust even when the input poses are significantly degraded, as would be expected in real-world scenarios. See also Appendix Secs. A.3 and A.4
4.7 Runtime
Our network achieved an average inference time of only 0.37 ms per Sub-DSI on an NVIDIA RTX A6000. Since predictions parallelized on the GPU. The total inference time to estimate depth map of average density (500 pixels) from MVSEC with Fori is 1.12 ms.
made independently per pixel, inference for each Sub-DSI can be
Taking MVSEC as an example (DAVIS cameras of 346x260 pixels) and DSIs back-projecting 2 million events onto D 100 depth planes, then each DSI creation takes #:45 ms, DSI fusion takes =22 ms, and pixel selection takes =0.2 ms on an 8-core computer with Intel Xeon(R) W-2225 CPU operating at 4.10 GHz. These values are common for both the oftte--f-the-eeee method MC-EMVS and DERD-Net. It has been shown that DSI creation does not hamper real-time performance 53 because the 3D map be updated infrequently and on-demand.
Our network adds only a very small runtime compared to the DSI creation time. This ultra-fast performance, combined with its lightweight architecture, enables efficient execution, making DERD- Net ideal for real-world applications requiring low-latency depth estimation.
Conclusion
We have developed the first learning-base based multi-view stereo method for event-based depth estima- tion. Our approach combines input camera poses with events to produce intermediate geometric representations (DSIs) from which depth is estimated using deep learning. It is directly applicable to both monocular and stereo camera setups. By processing small independent subregions of DSIs in parallel, the framework operates independently of camera resolution and facilitates efficient network under MB in size with an inference time of only 0.37 ms.
Our framework consistently demonstrated superior performance across several metrics compared to other stereo methods and achieved comparable performance when using purely monocular data. is the first learning- based depth estimation approach that reports robust generalization all three indoor flying sequences of the MVSEC dataset. Adaptability to different scenes was confirmed on the outdoor driving DSEC dataset, for which it drastically outperformed benchmark approaches across all metrics. Moreover, our framework significantly increased the number of points for which depth can be robustly estimated from DSIs. It also showed strong robustness to noise in camera poses.
Given its exceptional performance, ultra-lightweight architecture, scalability and flexibility across different configurations, our method holds strong potential to become standard approach for learning depth from events and highly suitable for wal-world robotic applications requiring low latency and low memory such SLAM [1).
10

---

Acknowledgments
Funded by the Deutsche Fosschungsgemeennccaff (DFG, German Research Foundation) under Ger- many's 8 Excellence Strategy - EXC 2002/1 "Science of Intelligence" project number 390523135.
References
[1] Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck, "A 128x 128 120 dB 15 μss latency asynchronous temporal contrast vision sensor," " IEEE J. Solid-State Circuits, vol. 43, no. pp. 1p.566-55 566-576, 2008.
[2] Thomas Finateu et al., "A 1280x720 illummnnadd stacked temporal contrast event- based vision sensor with 4.86um pixels, 1.066Geps readout, programmable event-rate controller and compressive data-formatting pipeline," in IEEE Int. Solid-State Circuits Conf. (ISSCC),
pp. 112-114, 2020.
[3] Guillermo Gallego, Tobi Delbruck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew Davison, Jörg Conradt, Kostas Daniilidis, and Davide Scaramuzza, "Event-based vision: A survey," IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. pp. 154-180, 2022.
[4] Alex Zihao Zhu, Dinesh Thakur, Tolga Ozaslan, Bernd Pfrommer, Vijay Kumar, and Kostas Daniilidis, "The multivehicle stereo event camera dataset: An event camera dataset for 3D
perception." " IEEE Robot. Autom. Lett., vol. 3. pp. 2032-2039, July 2018.
[5] Suman Ghosh and Guillermo Gallego, "Event-based stereo depth estimation: A survey," IEEE
Trans. Pattern Anal. Mach. Intell., vol. 47, no. 10, pp. 9130-9149, 2025.
[6] Stepan Tulyakov, Francois Fleuret, Martin Kiefel, Peter Gehler, and Michael Hirsch, "Learning an event sequence embedding for dense event-based deep stereo," in Int. Conf. Comput. Vis. (ICCV), pp. 1527-1537, 2019.
[7] Maxim Maximov, Kevin Galim, and Laura Leal-Taixé, "Focus defocus: bridging the synthetic to real domain gap for depth estimation," in IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 1071-1080, 2020.
[8] Timo Stoffregen, Cedric Scheerlinck, Davide Scaramuzza, Tom Drummond, Nick Barnes, Lindsay Kleeman, and Robert Mahony, "Reducing the sim-to-real gap for event cameras," " in
Eur. Conf. Comput. Vis. (ECCV), pp. 534-549, 2020.
[9] Shintaro Shiba, Yoshimitsu Aoki, and Guillermo Gallego, "Secrets of event-based optical flow,"
in Eur. Conf. Comput. Vis. (ECCV), pp. 628-645, 2022.
[10] Shuang Guo, Friedhelm Hamann, and Guillermo Gallego, "Unsupervised joint learning of optical flow and intensity with event cameras," in Int. Conf. Comput. Vis. (ICCV), 2025.
[11] Henri Rebecq. Guillermo Gallego, Elias Mueggler, and Davide Scaramuzza, "EMVS: Event- based multi-view stereo-3D reconstruction with an event camera in real-time." Int. J. Comput. Vis., vol. 126, pp. 1394-1414, Dec. 2018
[12] Suman Ghosh and Guillermo Gallego, "Multi-event--ammra depth estimation and outlier rejec- tion by refocused events fusion," Adv. Intell. Syst., vol. 4, no. 12, p. 2200221, 2022.
[13] Mathias Gehrig, Willem Aarents, Daniel Gehrig, and Davide Scaramuzza, "DSEC: stereo event camera dataset for driving scenarios," IEEE Robot. Autom. Lett., vol. 6, no. 3, pp. 4947- 4954, 2021.
[14] Suman Ghosh, Valentina Cavinato, and Guillermo Gallego, "ES-PTAM: Event-based stereo parallel tracking and mapping," in Eur. Conf. Comput. Vis. Workshops (ECCVW), pp. 70-87, 2024.
[15] Guillermo Gallego. Javier Hidalgo-Carrió, and Davide Scaramuzza, "Event-based SLAM," in SLAM Handbook. From Localization and Mapping to Spatial Intelligence (Luca Carlone, Ayoung Kim, Frank Dellaert, Timothy Barfoot, and Daniel Cremers, eds.), pp. 282-302. Cambridge University Press, 2026.
[16] Misha Mahowald and Tobi Delbrück, Cooperative Stereo Matching Using Static and Dynamic Image Features, pp. 213-238. Boston, MA: Springer US, 1989.
[17] Yi Zhou, Guillermo Gallego, Henri Rebecq, Laurent Kneip, Hongdong Li, and Davide Scara- muzza, "Semi-dense 3D reconstruction with stereo event camera," in Eur. Conf. Comput. Vis. (ECCV), pp. 242-258, 2018.
[18] Alex Zihao Zhu, Yibo Chen, and Kostas Daniilidis, "Realtime time synchronized event-based stereo," in Eur. Conf. Compus. Vis. (ECCV), pp. 438-452 2018.

---

[19] Yi Zhou, Guillermo Gallego, and Shaojie Shen, Devent-based stereo visual odometry," IEEE Trans. Robot., vol. 37, no. 5, pp. 1433-1450, 2021.
[20] Anass El Moudni, Fabio Morbidi, Sebastien Kramm, and Rémi Boutteau, "An event-based stereo 3D mapping and tracking pipeline for autonomous vehicles," in IEEE Intell. Transp. Sys. Conf. (ITSC). pp. 5962-5999999 5968, 2023.
[21]
Shintaro Shiba, Yannick Klose, Yoshimitsu Aoki, and Guillermo Gallego, "Secrets of event- based optical flow, depth, and ego-motion by contrast maximization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 12. pp. 7742-7759, 2024.
[22] Sheng Zhong, Junkai Niu, and Yi Zhou, "Deep visual odometry for stereo event cameras," IEEE Robot. Autom. Lett., vol. 10, pp. 11078-10088
-1 1008, Nov. 2025.
[23] Jin Han Lee, Myung-KKu Han, Dong Wook Ko, and II Hong Suh, "From big to small: Multi- scale local planar guidance for monocular depth estimation." arXiv e-prints, 2019.
[24] Hamid Laga, Laurent Valentin Jospin, Farid Boussaid, and Mohammed Bennamoun, "A survey on deep learning techniques for stereo-based depth estimation," " IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 4. pp. 1738-1764, 2020.
[25] Fabio Tosi, Luca Bartolomei, and Matteo Poggi, "A survey on deep stereo matching in the twenties," Int. J. Comput. Vis., vol. 133, pp. 4245-4276, Feb. 2025.
[26] Javier Hidalgo- -arrió, Daniel Gehrig, and Davide Scaramuzza, "Learning monocular dense depth from events," " in Int. Conf. 3D Vision (3DV), pp. 534-542, Nov. 2020.
[27] Henri Rebecq, René Ranftl, Vladlen Koltun, and Davide Scaramuzza, "Exents-to-video: Bring- ing modern computer vision to event cameras," in IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2019.
[28] Xu Zheng, Yexin Liu, Yunfan Lu, Tongyan Hua, Tianbo Pan, Weiming Zhang, Dacheng Tao, and Lin Wang. "Deep learning for event- based vision: A comprehensive survey and benchmarks," arXiv e-prints, 2023.
(29] S. M. Nadim Uddin, Soikat Hasan Ahmed, and Yong Ju Jung, "Unsupervised deep event stereo for depth estimation," " IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 11, pp. 7489-7504,
2022.
[30] Henri Rebecq. Guillermo Gallego, and Davide Scaramuzza, "EMVS: Event-based multi-view stereo," in British Mach. Vis. Conf. (BMVC), 2016.
[31] Olaf Ronneberger, Philipp Fischer, and Thomas Brox, "U-Net: Convolutional networks for biomedical image segmentation," " in Int. Conf. Medical Image Computing and Computer- Assisted Intervention (MICCAI). pp. 234-241, 2015..
[32] Liyuan Liu, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, and Jiawei Han, "Understanding the difficulty of training transformers," in Proc. Conf. Empirical Methods in Natural Language Processing (EMNLP), 2020.
[33] Cedric Scheerlinck, Henri Rebecq, Daniel Gehrig, Nick Barnes, Robert Mahony, and Davide Scaramuzza, "Fast image reconstruction with an event camera," in IEEE Winter Conf. Appl.
Comput. Vis. ( (WACV), pp. 156-163, 2020.
[34] Lahav Lipson, Zachary Teed, and Jia Deng, "Raft-stereo: Multilevel recurrent field transforms for stereo matching." in Int. Conf. 3D Vision (3DV), pp. 218-227 2021.
[35] Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio, "Learning phrase representations using RNN encoder- decoder for statistical machine translation." in Proc. Conf. on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724-1734, ACL, 2014.
[36] Michael Opitz. Horst Possegger, and Horst Bischof, "Efficient model averaging for deep neural networks." in Asian Conf. Comput. Vis. (ACCV), pp. 205-220, 2016.
[37] Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy. Patrick Galli- nari, and Matthieu Cord, "Diverse weight averaging for out-of-distribution generalization," in Adv. Neural Inf. Process. Syst. (NeurlPS), vol. 35, pp. 10821-10836, 2022.
[38] Heiko Hirschmuller, "Stereo processing by semiglobal matching and mutual information," IEEE
Trans. Pattern Anal. Mach. Intell., vol. 30. pp. 328-341, Feb. 2008
[39] Sio-Hoi leng, Joao Carneir, Mare Osswald, and Ryad Benosman, "Neuromorphic event-based generalized time- stereovision," Front. Neurosci. vol. 12, p. 442, 2018.
[40] Andreas Geiger, Philip Lenz, and Raquel Urtasun, "Are we ready for autonomous driving? the KITTI vision benchmark suite," " in IEEE Conf. Comput. Vis. Pattern Recog, (CVPR), pp. 3354 3361, 2012.
12

---

[41] Chengxi Ye, Anton Mitrokhin, Chethan Parameshwara, Cornelia Fermüller, James A. Yorke, and Yiannis Aloimonos, "Unsupervised learning of dense optical flow, depth and egomotion with event-based sensors," in IEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), pp. 5831-58388 2020.
[42] Junkai Niu, Sheng Zhong, Xiuyuan Lu, Shaojie Shen, Guillermo Gallego, and Yi Zhou, "ESVO2: Direct visual-inertial odometry with stereo event cameras," IEEE Trans. Robot., vol. 41, pp. 2164 2183, 2025.
[43] Soikat Hasan Ahmed, Hae Woong Jang, S M Nadim Uddin, and Yong Ju Jung, "Deep event stereo leveraged by event-to-image translation," AAAI Conf. Artificial Intell., vol. 35, pp. 882-
890, May 2021.
[44] Kaixuan Zhang, Kaiwei Che, Jianguo Zhang, Jie Cheng, Ziyang Zhang, Qinghai Guo, and Luziwei Leng, "Discrete time convolution for fast event-based stereo," in IEEE Conf. Comput.
Vis. Puttern Recog. (CVPR), pp. 8666 8676, June 2022.
[45] Peigen Liu, Guang Chen, Zhijun Li, Huajin Tang, and Alois Knoll, "Learning local event- based descriptor for patch-based stereo matching," " in IEEE Int. Conf. Robot. Autom. (ICRA), pp. 412-418, 2022.
[46] Ulysse Rançon, Javier Saadrado-Anibarro Benoit R. Cottereau, and Timothée Masquelier, "Stereospike: Depth learning with a spiking neural network," IEEE Access, vol. 10, pp. 127428- 127439, 2022.
[47] Zhu Jianguo. Wang Pengfei, Huang Sunan, Xiang Cheng, and Teo Swee Huat Rodney, "Stereo depth estimation based on adaptive stacks from event cameras," in IECON 49th Annual Conf. IEEE Industrial Electr. Soc., pp. 1-6, 2023.
[48] Dipon Kumar Ghosh and Yong Ju Jung. "Two-stage cross- fusion network for stereo event-based depth estimation." " Expert Systems with Applications, vol. 241, p. 122743, 2024.
[49] Wu Chen, Yueyi Zhang, Xiaoyan Sun, and Feng Wu, "Event-based stereo depth estimation by temporal-spatial context learning." " IEEE Signal Process. Lett., vol. 31, pp. 1429-1433, 2024. [50] Hoonhee Cho, Jae- You-Young Kang. and Kuk-Jin Yoon, "Temporal event stereo via joint learning with stereoscopic flow," in Eur: Conf. Comput. Vis. (ECCV), pp. 294-314, 2024.
[51] Hoonhee Cho and Kuk-Jin Yoon, "Selection and cross similarity for event-image deep stereo," in Eur. Conf. Comput. Vis. (ECCV), pp. 470-486, 2022.
[52] Fengan Zhao, Qianang Zhou, and Junlin Xiong, "Edge-guided fusion and motion augmentation for event-image stereo," " in Eur. Conf. Comput. Vis. (ECCV), 2024.
[53] Suman Ghosh and Guillermo Gallego, "Event-based stereo depth estimation from ego-motion using ray density fusion," Eur. Conf. Comput. Vis. Workshops (ECCVW),200
13

---

A Appendix: Sensitivity Analyses
In this section we report additional experiments to assess our method's robustness to changes in hyperparameter (Tab. 6). changes in the scene (Tab. 7 and noise in camera poses (Tabs. and 9. We
also complement the evaluation on the downstream task of camera tracking (Tab. 10P
A.1 Sensitivity with respect to Sub-DSI Size
The hyperparameters used for our experiments are detailed in Tab. The LiDAR's sampling interval, the estimated minimum and maximum depths, and the number of depth layers D have all been defined to match the protocol in [12). In Sec. 4.3 we already provided a comparison between the two AGT filters Forg and Fdenser. In this section, we therefore analyze the impact of the size of the Sub-DSI.
We retrained the network for one epoch on the audoor_flying 2 and 3 sequences and compared its test performance sequence 1 using radii rw rH of 2, 3, and px, effectively creating Sub-DSI frames size x 5, x 7, and x 9 px, respectively. Layer dimensions were adapted, while the overall network architecture remained fixed.
Table 6: Sensitivity analysis of DERD-Net's performance for different Sub-DSI frame sizes after one epoch of training. MVSEC indoor flyior
Sub-DSI
Modality
Mean En
Median Err
had-pix [%]] 4
SILog Ei
AErr
log RMSE
é < 1.25 [%]]
< 1.25 [%.]
8 1.25' [8] f
#Points [million]]
[em] 4
[em]
x100
[%]
4
13.92
6.51
0.96
1.20
5.71
10.97
96.07
98.56
99.60
0.99
x 7
stereo Fass
12.13
5.82
0.85
0.97
5.04
9.93
96.81
98.85
99.66
0.98
Foig
11.58
5.54
0.82
0.92
4.85
9.64
96.97
98.91
99.68
0.98
5 x
sterzu Fdassrr 18.42
7,88
1.79
2.10
7.64
14,50
93.71
97.58
99.10
3,03
Fdaaser 15.41
12.56
95.24
99.36
9 x 9
stereo Famee
14,59
6.39
1,30
1.48
6.07
12.21
95.52
98.29
99.37
2.99
From the results reported in Tab. 6 it can be inferred that performance appears to improve as the radii increase. The network was able achieve better results after single epoch using frame size 9 x 9 than when fully trained on 7 x 7 frames (see Tab. 12 in this Appendix), highlighting its potential for further performance improvements.
Nevertheless, increasing the frame size to x 9 yielded a reduction of 5% MAE for both filters after single training epoch. In contrast that, the network had to apply 65% more 3D-convolttional operations and its total amount of parameters raised from 70k to 270k. We therefore decided for a 7 x 7 frame size for this study. Future research could explore the evident potential to further boost performance by optimizing the sub-DSI size, considering the trade-off between accuracy, parameter count and computational costs.
A.2 Sensitivity with respect to DSI Transformations
Next, we analyze the robustness of DERD-Net with respect to transformations of the DSI, in particular axis-aligned reflections the DSIs generated from the indoor_lying__ sequence of the MVSEC dataset. Specifically, we flipped the DSIs horizontally, vertically, and both horizontally and vertically. These transformations effectively generate scenes with similar geometric properties (c.g., distance ranges) but novel spatial configurations. This allows us evaluate how well the network generalizes to previously unseen, yet structurally similar environments. We therefore purposely used no data augmentation during training ensure representative assessment of the network's inherent robustness. Analogous to previous experiments, we used the single-pixt network that was trained solely the original indor__fying and for evaluation. No retraining performed.
The results of these experiments are displayed Tab. Performance worsened only slightly, with results that still significantly outperform all SOTA methods for all tested configurations, indicating that our network might effectively generalize scenes that share similar depth ranges and texture with those on which it was originally trained.
14

---

Table 7: Performance of DERD-Net when applying axis algge MVSEC indoor _aaiong__yyg____
Rellection
Modality
Meun Err
Median Err [em]
bad pix
SiLog Eir
AErr
log RMSE
1:25
11225
8< 1.25
#Points
[cm]
[%]]
X100
[%]]
x100 4
[%]]
[%]
[[]] 1
[million]
none
stereo
Fans
0.86
0.97
9.88
96.87
99.65
0.98
vertical
stereo Fang
12.81
6.27
0.92
1.06
5.43
10.34
96.56
98,80
99.64
0.98
horizontal
Fuss
14.26
6.81
1,01
1.23
5.88
11.08
95.85
98.56
99
horizontal vertical
sterco
Forg
14.28
6.79
1,02
1.23
5.92
11.11
95.79
98.53
99.60
0.98
none
sterco Faxsee 14.86
6.47
1.31
1.49
6.14
12,30
95,45
98.24
99.57
3.01
vertical
stereo Faror
1665
horzontal
steren
Fdams 18.23
7.87
1,60
1.91
7.32
13.84
93.73
97.71
99.25
3.01
horizontal vertical stereo Fareer
18.86
8.37
1,68
1.98
7.64
14.14
93,46
97.61
99.22
3.01
A.3 Sensitivity with respect Noise in Camera Poses
In Section Sec. 4.6 we summarized the sensitivity of DERD-Net with respect to noise in the camera poses used build DSIs, DSEC data. For completeness, we show results MVSEC data.
We repeat the same experiment as that in the top rows of Tab. (bb.5ppp on the MVSEC indoor indoor_flying se- quence, for which ES- PTAM reported ATE of 14.93 over a 6 m depth range. The results shown again highlight DERD-Net's
in Tab. [8] and compared to those obtained with GT poses Tab. 12 strong robustness to noisy poses estimated by went-based SLAM system: the MAE and MedAE increased only slightly, while the bad-pix -id metric even improved. The most pronounced decline was in the number of evaluated points. Nevertheless, DERD- with Faenser still predicts depth for 69% more pixels than MC-EMVS with Forig under ideal poses, while achieving a 30% lower MAE. For its multi-pixel variant, DERD-Net evaluated with noisy poses maintains superior performance state-of-the-arr methods using GT poses, while still predicting the highest number of points.
all
Table 8: Quantitative depth estimation performance on MVSEC indoor flying_lli_gg using poses com- puted downstream of ES-PTAM. Relative changes with respect to Tab. 12. which reports results obtained using GT poses, are presented in parentheses.
Mean [m]
Median [m]
bad-pix [%]
#Points [million
Algorithm
Poses
Filter
12.72
6.33
0.65
0.68
DERD-Net
ES-PTAM
Forig
(+8.81%)
(+16.79%)
(-24.42%)
(-30.61%) %)
15.76
7.36
1.17
1.62
DERD-Net
ES-PTAM Faenser
(+6.06%)
(+13,76%%
(-10.69%)
(-46.18%)
13.53
6.76
0.65
3.41
DERD-Net (mulli-pixel)
ES-PTAM Foriz
(+10.27%)
(+19.01%) %)
(-24.42%)
(-33.91%) %)
16.60
7.65
1,21
6.27
DERD-Net (multi-pixel)
ES-PTAM
Fdenser
(+6.62%)
(+16.08%)
(-11.68%) (-46.46%)
In the interest of thoroughness, we also used poses from the state-of-the-art event-based stereo visual-inertial odometry system ESVO2 (42), which is notably more accurate than ES-PTAM, to run DERD-Net on MVSEC indoor_flynng indoor_ farior_fllin__.. and indoor_flying_3. The mean results are reported in Tab. 9 Compared to Tab, DERD-Net shows only a slight decrease in depth estimation performance on Forig, while even improves on Focnser. confirming its robustness to pose noise from an event-based SLAM system integrating events and incrtial data.
Table 9: Quantitative depth estimation performance averaged over MVSEC yndoor_flying J, 2, and _3 using poses computed downstream of ESVO2. Relative changes with respect to Tab. 3 which reports results obtained using GT poses, are presented in parentheses.
Mean Err Median Er
bad-pix [%] $
#Points [million
Algorithm
Poses
Hilter
[m] 4
[m] 4
11.53
5.73
1.09
0.61
DERD-Net
ESVO2
Forig
(-1.37%)
(+4.18%)
(+22.47%)
(-22.78%)
13.69
6.32
1.46
1.45
DERD-Net
ESVO2
Faesser
(-10.17%%
(-5.39%)
(-14.12%)
(-47.65%)
15

---

A.4 Downstream Camera Tracking Performance Analysis
As intermediate results those in the bottom part of Tab. 5 we report offline camera tracking performance using the edge-alignment camera tracking module [14] acting on input events and the local maps built using DERD-Net's depth predictions (from GT poses). Camera tracking performance is given in terms of ATE and Absolute Rotation Error (ARE) on the DSEC [13) driving dataset in Tab. TO Although our model was trained only on the Zurich _ 04 aa sequence, we evaluate on all Zurich_ CitichhCCtt_uii 04 sequences to highlight its generalization capabilities.
Table 10: Camera tracking performance on DSEC zurich_city 04, without DERD-Net retraining.
Sequence
zc04a
zc04b
zc04c
zc04d
zc04e
zc04f
Duration [s]
35
13.4
53
47.8
13.6
43.1
ATE RMSE [em] J
17.07
7.85
14.00
55.64
5.71
36.11
ARE RMSE [deg] 1
0.31
0.08
0.45
0.67
0.11
0.72
The obtained pose errors in the 50 depth range scenes across all sequences show strong performance of DERD-Net for downstream tasks such as pose estimation via simple photometric edge alignment on event images, as well as its robust generalization even when trained on single sequence. Training on more diverse set of DSEC sequences would be expected to further enhance these results. These values are not comparable to online SLAM tracking results because they assume that the 3D map was pre- pre-built offline using DERD-Ne with GT poses. Therefore, the estimated camera poses reported here not accumulate drift.
B Appendix Detailed per-Sequence Results
The average results of different SOTA methods compared to DERD-Net presented in Tab. 11 In Tabs. to to 14] the individual performance on each of the respective sequences indoor_ti foyoorr 2, 3 from the MVSEC dataset are displayed. Table [1] presents the corresponding results for the Cirich_City_444 sequence from the DSEC dataset.
MVSEC Averaged
Table 11: Quantitative comparison of the proposed methods with the state of the art. MVSEC indoor_fyite (average).
Algorithm
Modality
Mean Err [mm]
Median Er
had- SILog Err
AEnR
log RMSE
5xc. 1.25 d 1.25"
à 1
#Points
[%]]]
x100
[%)
x100
[%]]
1%]1
[million]]
EMVS
11
monocular
3.84
LRI
monocular Farser
ESVD
10
steren
10.59
3.35
3.48
10.19
18.83
90.44
95.76
97.98
2.04
BSVO
mxxp
steren
22.70
9.83
2.83
3.03
9.59
17.53
91.82
96.50
98 38
1.56
SGM indep.
35.42
12.35
6.39
8.45
16.17
29.09
85.34
93.05
14.46
GTS Indep.
389.0
14.47
102.92
89.08
49.56
62.
69.36
MC-EMV'S
Fing
1,35
7.80
MC-EMVS
TP
sterco Fower
28.38
12.38
3.26
3.43
10.94
18.60
89.41
96.09
98.33
2.77
MC-EMVS
ME
steren + Fing
20.64
9.72
1.13
L.0
7.94
13.54
94.74
97.95
99.17
3.00
DERD-Net
monocular
23.68
11.55
2.78
2.62
10.18
16.20
90.25
97. 36
99
DERD-Net
Faeser
28.52
13.85
3.77
19.46
85.78
95.77
98.
Fwig
0.98
9.92
DERD-Net
sterco Faig
11.69
5.50
0.89
0.96
3,05
9.83
96,99
95.59
99.64
0.79
DERD-Net
stereo Fireer
15.24
6.68
1,70
1.54
6.41
12.44
95
9%. 19
99
2.77
DERD-Net milti-pisel
steren
12.00
5.63
0.90
0.99
5.13
9.94
96
98 83
99 63
4,32
DERD-No multi
15.68
6.75
1.74
1.59
6.54
12.61
94.75
98.10
99.36
11.33
16

---

MVSEC Indoor Flying
Table 12: Quantitative comparison of the proposed methods with the state of the art. MVSEC
indoor I
Algurithm
Modality
Mean Err [cm]
Median Ert
bad-pix [%]
SiLog Ert
AErTR
log RMSE
1.25 <:C
1.25
5 1.25
APoins
[cm] :
x100
[16]
x100
[8]] 1
[[]]
[96]
[million]
EMVS 1 EMVS
monocular Fors
39.37
14.95
3,05
4.72
13:25
22.10
82,03
93.43
97
1.21
Fiee
37.29
9.96
ESVO m.
sterco
23.39
10.03
2
2.79
9.78
16.72
91.57
96.84
98.79
1.41
SGM imdep. 1s
stereo
35,45
13.61
5,54
7.35
15.00
27.
85
91. 5
96.40
11.64
GTS indep.
stereo
700.37
38.39
52.51
79.26
111.21
91. /4
$4.27
67. 16
73. 73.30
0:03
MC-EMVS
1.30
1.94
14.11
9/.49
Faueur
20:03
88,
MC-EMVS TE4
MF
sterco+ Fara
23.33
9.90
1.39
2,08
8.12
14.61
93.16
97.
99.05
3.48
DERD-Nel
monecalr- FerK
25.76
12,60
1.89
2.62
10.39
16.19
89. 42
97 80
99 24
1,50
DERD-Net
monocaler
30.90
15.00
3.34
3.86
12.62
19.65
85.37
95. 94
98, 08.61
4.64
DERD-Nat without
5.05
0.97
4.98
96.89
999
Fera
DERD-Net
Fazuer
6.47
1.31
1,49
12.30
99.
DERD-NGt multi-pinel
seeenn Fang
12.27
5.68
0.86
0.99
5.06
9.98
96.76
98.77
99 69
5.16
DERD-Net multi gixel
stereo
15.57
6.59
1,57
1,58
6.39
12.61
95.13
98,11
99.33
11.71
MVSEC Indoor Flying
Table 13: Quantitative comparison of the proposed methods with the state of the art. MVSEC
indoor_ flying__lgg_22
Algoritm
Modality
Mcan Err [em]
Median Er
bad-pix [%]]
SiLag Ert
AErT
log RMSE
5 1.25
á 1,25
5 L.25
Atooints [million]
[cm]
x100
[96
x100
[9]]
[%6]
monocular Feex
EMVS
monocular Farr
15.69
17.96
14.56
11.74
19.35
34.81
72.69
27
94.01
3.65
ESVO
15
sterco
21.34
8.97
3.75
3.48
9.32
19.1 19.14
91. 91.60
95.5 88
97 77.86
1.89
VS Ixtpp. Is
stenco
20.42
8.63
3.50
3.24
9.14
18.35
92.03
96.19
98. 98.19
1.41
SGM indep.
32,91
8.75
8.29
9.50
15.82
31,54
8:1400
92.33
95.48
16.95
GTS indep.
167.14
MC-EMVS
stateo Forg
18:20
3.49
1.77
1.78
8.13
13.59
0.65
MC-EMVS
sienen Fascur
25.81
10:34
4.65
3,48
10.89
18:91
89.
95. 0/
98.
2.25
MC-EMVS
ME sterco + Face
18.58
3.68
1,86
1.81
8.19
13:71
95.27
98.07
99
2.42
DeRD-Net
monocular Fuce
23.37
10.43
4,98
3.31
11.07
18.41
88.30
96.23
98.46
0.98
DERD-Net
Farmx:
82.75
DERD-NAt without
Faig
5.23
1.34
1.13
5.45
10.67
96.67
Det
Geren Fuce
11.11
4.94
1.26
1.10
5.34
10.50
96.69
98, 66
99
0.58
DERD Net
steren Faueer
14.46
5.92
2.78
1,72
6.74
13.17
94.05
97.58
99 32
2.25
DERD-Not multi-pinel
steren Farg
11.29
4,88
1.28
1,14
5.30
10.66
96. 96.50
98 61
99
3.21
DERD-Net multi-pinel
Fareer
5.95
MVSEC Indoor Flying
Table 14: Quantitative comparison of the proposed methods with the state of the art. MVSEC indoor_fying.. flying 3.
Algorithm
Modality
Mean Err Median bad-pix [%]]]
Ei
AErR [%)
log RMSE
&ic 1.25 [%]]
5 1.255 [[6]
1.25 [%]]
#Points [million]]
[cm] 1
EMVS 1I
monocular Porg
30.54
15.09
2.31
3.33
11.59
18.27
88.16
96.45
98.47
1.42
EMVS
menocular Fizze
44.62
20.26
7.15
8.50
18.26
29.15
78.55
91.41
95
4.15
ESVO
29.62
12.61
3.78
4.02
11.50
20.20
88.
91. 53
97
2.29
ESVO
24.29
10.84
2.81
3.05
9.34
17.54
91.87
96.
98.
1.86
indep.
GTS indep.
Is
sterco
299,
60.66
39.75
72.24
102.77
88.87
15.04
58, $3.86
66.94
0.08
MC-EMVS
[p]
staneo + Forg
19.19
10.38
0.99
1,43
7.35
12.01
96.09
96, 60
99
0.82
MC-EMVS
staren Faew
27.89
13:65
2,01
2,83
10.25
16.85
90.997
97. 05
9% 73
3.0
MC-EMVS
MP
DeRD-Net
monocular Fire
1.46
9:07
DERD-Net
monocular Fares:
27.01
13,50
2.58
2.85
10.99
16.96
89.2
97.01
98.
4.15
DeR without EL
staren + Fasg
12.50
6.31
0.57
0.84
5.03
9.20
97.41
99. 13
99 72
0.82
DeRD-Net
strre Faie
12.28
6.13
0.55
0.82
4.91
9.11
97.41
99.15 15:15
99
0.82
DERD-Net
7.64
6.36
95.
3.04
multi-pixel
97.41
DERD-Net multi-pixel
stereo Fazeer
7.65
1.01
1.38
6.36
11.76
95.44
98.
99.47
12.77
17

---

DSEC
Table 15: Quantitative comparison of the proposed methods with the state of the art. DSEC Zurich_Ciis 04_04._5 a.
_ity_04
Algorihum
Modalit
Mean Err
Median Err
bul-pix [S]]
SiLog Ert
AErR
log RMSE
1:25
1.255 [R]
d-c 1,25*
dPvints
[m]]
[m]]
x100
[8]]
[96]
[million]]
monocular Fine
72.56
EMVS
EI
monocular Faeu
7.01
3.56
24.33
23.07
11.74
48.52
63.00
79.81
$7.7]
6.09
500
ESVO
SGM
13
serceo
stero
6.74
3.93
1.58
1.62
15.25
10.54
17.95
6.30
18.42
17.66
28: 911
42.51
84,37
800.66
890
92.
93. 16
96.05
9.40
6.30
GTS
26.24
stereo
Fute
MC-EMVS
LF!
stereo Faceer
4.76
1.56
76.37
90.97
MC-FMVS
I2!
MP
steren Fasg
3.51
0.96
11.81
8.89
18.84
29.99
81.72
90068
95.07
3.83
DERD-Net
menrcular Fare
3.12
1,00
5.50
3.96
12.19
19.92
86.06
96.29
98. 98.61
2.10
DERD-Net
sterco Jame
1.50
20.12
DERD-NG
starco Faceer
1.80
0.54
5,04
2.91
7.59
17.06
92.09
96.72
98.56
4.64
DERD-NGI multi-pisel
steren Fesg
1.59
0.47
3
2,54
6.76
15.93
93.60
97.18
98.78
6:59
DERD-NG Net multi pixel
stereo Faveer
1.79
0.54
4.51
2.76
7.46
16.62
92,31
96.82
98. 98.62
14.74
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The framework and it advantages explained in Sec. The results displayed in Sec.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We mention the requirement of available Ground Truth for training (Sec. 4 and explicitly point out that accuracy cannot be directly compared to end-to-end learning-based methods (Sec. (.5).. We state the need for camera pose in the construction of the DSIs
(Sec. (.1]. More details about limitations of DSIs are available the linked source [2]
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
19

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
No theoretical results are included.
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: See Tab. 1 Tab. 2 Sec. and Sec.
We also provide code and model
checkpoints in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers have some path reproducing verifying the results.
20

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
The supplementary material of our submission contains the full code. The datasets are publicly available.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https:/nnips.cc/pbblec
quiiessCCodeSubnissiooooiiii for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (htips: for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Tab. Tab. and Sec.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Following standard evaluation protocols used by comparable prior methods to ensure fairness through consistent experimental settings, we do not plot error bars. Instead, we test several variations of our network and report differences in the estimation errors (Tabs. II to 15 We also provide quantitative evaluation of the effects of uncertainty networks accuracy by comparing the pixel selection maps visualizations of uncertainty - to the depth maps created by the networks in Fig. Fig-3land and the video included in the supplementary material. Furthermore, we provide an analysis of how variations to the dataset (Sec. A.2 and hyperparameters (Sec. A.I. affect the network's 's performance. Guidelines:
21

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots. The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Sec. 4.7
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://eeeuisssccppppi///hhiiit
Answer: [Yes]
Justification:
Our research does not infringe the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our paper has no foresecable significant socictal impact.
22

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
We see no parts of our paper being at high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We did correctly cite the creators of MC-EMVS for the DSI approach 12 as well as the creators for the datasets of MVSEC and DSEC [3]
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
23

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assett are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paapeswitthooe wtw.wiitooodcccccaattttctttt has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The full documentation for code and models is included in the supplementary material of our submission.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No crowdsourcing experiments research with human subjects has been part of this study.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: There are no such risks to report for our study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs have not been used as an essential component in this study.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy http::/leeurr.ccccoommcree///s/s for what should or should not be described.
25


Reviews:
[{"review_id": "ZaOtHMPpQP", "reviewer": "Reviewer_PC8y", "summary": "This paper presents DERD-Net, a novel method for depth estimation from event cameras using the spatial density of back-projected event rays, represented as Disparity Space Images (DSIs). The proposed framework incorporates known camera poses to generate DSIs and processes local subregions (Sub-DSIs) with a lightweight neural network combining 3D convolutions and a gated recurrent unit (GRU). By focusing on small volumetric neighborhoods, the approach ensures constant model complexity, efficient parallel inference, and resolution independence. DERD-Net supports both monocular and stereo configurations and does not require event synchronization across views. Experiments on the MVSEC and DSEC datasets demonstrate that DERD-Net achieves superior performance compared to existing methods across multiple metrics, including mean and median error, depth completeness, and robustness. The framework also generalizes well across different scenes and modalities, and its efficient design makes it suitable for applications requiring real-time event-based depth estimation.", "questions": "1. The proposed method relies on externally provided camera poses to construct DSIs. Could the authors elaborate on the method's sensitivity to pose noise or drift, and whether any tolerance analysis has been performed?\n\n2. Given the method's stated motivation for SLAM applications, would the authors consider including evaluations on downstream tasks such as visual odometry or mapping to substantiate its practical impact?", "limitations": "The paper assumes the availability of accurate camera poses but does not analyze the method's robustness to pose estimation errors. This reliance may affect the applicability of the approach in real-world scenarios where pose inputs are noisy or unreliable.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Novel methodology: The paper introduces DERD-Net, a unique depth estimation framework that leverages event-ray densities (DSIs), representing a shift from traditional event stream processing.\n\n2. Efficient and compact design: The use of 3D convolutions and GRU on local Sub-DSIs enables accurate predictions with a lightweight model (~70k parameters) and sub-millisecond inference time, making it suitable for real-time applications.\n\n3. Comprehensive evaluation: The method is thoroughly tested on MVSEC and DSEC datasets, showing consistent improvements over state-of-the-art methods across accuracy and completeness metrics.\n\nWeakness:\n\n1. Reliance on camera poses: The method depends on accurate external pose inputs for DSI construction, but the paper does not analyze how pose noise affects performance, which could impact robustness in practice.", "was_revised": true, "final_justification": "The authors have addressed my concerns raised during the review process.", "rebuttal": "Thanks for your feedback reviewing our paper, especially for recognizing the novelty of our method, efficient design and comprehensive evaluation.\n\nThanks to the suggestion, we performed additional experiments to measure robustness of our approach against imperfect (noisy) camera poses and downstream camera tracking performance. Our experimental results showed that DERD-Net remains remarkebly robust even when the input poses are significantly degraded, as would be the case in real-world SLAM scenarios:\n\n## **Exp 1: Depth estimation using noisy poses (with drift) from event-based SLAM system**:\n\nPipeline: Stereo events --> estimated poses using event SLAM --> DSI --> Depth maps using DERD-Net\n\n(a) On DSEC zurich_city_04_a sequence, instead of GT poses (from LiDAR-IMU odometry), we used camera poses estimated by a stereo event odometry system ES-PTAM [1], which reported 131.62 cm Absolute Trajectory Error (ATE) in 50 m depth range scene, to run DERD-Net and achieved the following depth estimation performance:\n\n| Algorithm | Mean Err [m] ↓ | Median Err [m] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 1.56 (-3.11%) | 0.45 (-2.17%)  | 3.84 (-6.8%)   | 1.61 (-3.59%) |\n| Noisy DERD-Net + F_denser   | 1.74 (-3.33%) | 0.52 (-3.7%)  | 4.84 (-3.97%)    | 4.49  (-3.23%) |\n\nCompared to Table 3, while the number of points for which depth is evaluated slightly decreased, DERD-Net's depth estimation errors actually even improved. This demonstrates strong robustness to noisy poses obtained from an event SLAM system.\n\n(b) On MVSEC flying1, instead of GT poses (from motion capture), we used camera poses estimated by ES-PTAM [1] (ATE 14.93 cm in 6 m depth range scene) to run DERD-Net and achieved the following depth estimation performance:\n\n| Algorithm | Mean Err [cm] ↓ | Median Err [cm] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 12.72 (+8.81%) | 6.33 (+16.79%)  | 0.65 (-24.42%)   | 0.68 (-30.61%) |\n| Noisy DERD-Net + F_denser   | 15.76 (+6.06%) | 7.36 (+13.76%)  | 1.17 (-10.69%)   | 1.62 (-46.18%) |\n| Noisy Multi-Pixel DERD-Net + F_orig   | 13.53 (+10.27%) |  6.76 (+19.01%) | 0.65 (-24.42%) | 3.41(-33.91%) |\n| Noisy Multi-Pixel DERD-Net + F_denser   | 16.60 (+6.62%) | 7.65 (+16.08%)  |  1.21 (-11.68%) | 6.27 (-46.46%) |\n\nCompared to Table 8, the MAE and MedAE worsened only slightly, while bad-pix even improved. The strongest decrease here was in the number of points. However, note that Noisy DERD-Net + F_denser still predicts depths on 69% more pixels than MC-EMVS + F_orig using *ideal* poses, while still yielding a 30% lower MAE. For its multi-pixel version, Noisy DERD-Net shows superior performance to all SOTA methods while predicting on the most points. This again highlights DERD-Net's strong robustness to noisy poses obtained from an event SLAM system.\n\n(c)  For completeness, we also used poses from SOTA event-based stereo visual-inertial odometry system ESVO2 [2] to run DERD-Net on MVSEC flying1, flying2 and flying3 and report mean metrics. Compared to ES-PTAM, ESVO2 is notably more accurate due to the visual-inertial data fusion, producing less noisy poses, which yield an average ATE of ~8.3 cm in 6 m depth range scene. \n\n| Algorithm | Mean Err [cm] ↓ | Median Err [cm] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 11.53 (-1.37%) | 5.73 (+4.18%)  | 1.09 (+22.47%)   | 0.61 (-22.78%) |\n| Noisy DERD-Net + F_denser   | 13.69 (-10.17%) | 6.32 (-5.39%)  | 1.46 (-14.12%)   | 1.45 (-47.65%) |\n\nCompared to Table 3, this shows only a slight decrease in performance for DERD-Net's depth estimation error on f_orig, while it even improves on f_denser, thus indicating robustness to noisy poses obtained from a SLAM system using events and IMU. We observe the same trend in the number of pixels as the table in (b).\n\n## **Exp 2: Downstream camera tracking performance using depth estimated from DERD-Net**\n\nPipeline: Stereo events + GT poses (from LiDAR and IMU) → DSI → Depth and point-cloud estimated using DERD-Net → Camera pose estimated by aligning 3D map to events\n\nWe report camera tracking performance (Absolute Trajectory Errors (ATE) and Absolute Rotation Errors (ARE)) on all zurich_city_04 sequences of DSEC driving dataset, even though the DERD-Net model was trained only on the zurich_city_04_a sequence to highlight generalization capabilities. \n\n| Sequence | Duration [s] | ATE RMSE [cm] ↓ | ARE RMSE [deg] ↓ |\n|---|---|---|---|\n| zc04a  | 35 | 17.07 | 0.31  |\n| zc04b  |  13.4 | 7.85 | 0.08  |\n| zc04c  | 53 | 14.00 | 0.45  |\n| zc04d  | 47.8 | 55.64 | 0.67  |\n| zc04e  |  13.6 | 5.71 | 0.11  |\n| zc04f  | 43.1 | 36.11 | 0.72  |\n\nThese camera pose errors in the 50 m depth range scene across all sequences shows strong performance of DERD-Net for downstream tasks (pose estimation using simple photometric edge alignment on event images), as well as robust generalization even when trained on a single sequence. Training on more diverse DSEC sequences would improve performance even further.\n\n## **Exp 3: Closing the loop: Depth estimation performance on poses computed downstream of DERD-Net**\n\nPipeline: Stereo events + GT poses (from LiDAR and IMU) → DSI → Depth and point-cloud estimated using DERD-Net → Camera pose estimated by aligning 3D map to events → DSI → Depth estimated using DERD-Net\n\nTo go one step further and close the loop, we use the poses estimated above for DSEC zurich_city_04_a and re-run DERD-Net to measure robustness to noise generated by its own previous iteration. \n\n| Algorithm | Mean Err [m] ↓ | Median Err [m] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 1.66 (+3.11%) | 0.49 (+6.52%)   | 4.19 (+1.7%)    | 1.46 (-12.57%) |\n| Noisy DERD-Net + F_denser   | 1.95 (+8.33%) | 0.60 (+11.11%)   |  5.65 (+12.1%)  | 4.09 (-11.85%) |\n\nPerformance showed only minor degradation, particularly for f_orig, with no metric worsening by more than 13%. Remarkably, even under such noisy pose conditions, Noisy DERD-Net’s depth estimation errors remain around 50% better than those of prior SOTA methods using ideal poses. This demonstrates the practical viability of using DERD-Net as a depth estimation module within a self-sustaining SLAM system.\n\n# Answers to Questions:\n1. Please refer to experiments 1 and 3 above for sensitivity analysis to pose noise and drift.\n2. Please refer to experiments 2 and 3 above for downstream performance in camera tracking and re-mapping using estimated poses.\n\nReferences:\n\n[1] Ghosh et al. ES-PTAM: Event-Based Stereo Parallel Tracking and Mapping. In ECCVW 2024.\n\n[2] Niu et al. ESVO2: Direct Visual-Inertial Odometry With Stereo Event Cameras. IEEE T-RO 41 (2025), 2164–2183."}, {"review_id": "CDyqUZhWUH", "reviewer": "Reviewer_Yns5", "summary": "This paper proposes DERD-Net, a deep learning-based method for estimating depth from event camera data. The core methodology avoids direct processing of asynchronous event streams. Instead, it leverages the Disparity Space Image (DSI) representation, which is constructed by back-projecting events into a volumetric grid using known camera poses, a technique established in EMVS [9] and MC-EMVS [10]. The main contribution is a lightweight neural network that operates on small, local patches of the DS to predict pixel-wise depth. The network uses a combination of 3D convolutions and a GRU to interpret these local ray density patterns. The authors present experiments on the MVSEC and DSEC datasets, claiming state-of-the-art performance in both monocular and stereo settings, significant improvements in depth map completeness, and superior robustness compared to existing methods.", "questions": "- The performance of your method is contingent on the quality of input poses. How does DERD-Net's performance degrade as a function of increasing noise in the camera poses, simulating a realistic SLAM/VIO scenario? A sensitivity analysis is critical for understanding the practical utility of this work.\n\n- Can you justify the comparison in Table 4 against end-to-end methods that do not require camera poses as input? Given the vastly different problem setups, how can any meaningful conclusion about comparative robustness be drawn?\n\n- Could you provide a full pipeline timing analysis, including the DSI construction time for an average frame in the MVSEC dataset? This would give a more realistic picture of the method's computational cost compared to the reported sub-millisecond inference time.", "limitations": "The authors have listed several limitations, such as the requirement for ground truth depth data for training and the need for input camera poses.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n+ The paper is well-written and easy to follow.\n\n+ The idea of applying a learned function to interpret the rich information in a DSI, as opposed to a simple argmax heuristic, is a sensible and logical next step for this line of research. The local processing on Sub-DSIs is an intelligent design choice for keeping the network lightweight and potentially improving generalization.\n\n+ The authors have conducted experiments across two standard datasets, comparing their method against numerous baselines on ten different metrics.\n\nWeaknesses:\n\n- The foundational components—using event cameras for multi-view stereo, leveraging known camera poses, and the DSI data structure itself—are directly adopted from prior work, most notably EMVS [9] and MC-EMVS [10]. The proposed method, DERD-Net, essentially replaces the final argmax step of MC-EMVS with a neural network, which predictably yields better results than a non-parametric heuristic. The claim of being the \"first learning-based multi-view stereo method to use camera poses along with events as input\" is narrowly scoped and misleading, as the use of poses to create the geometric DSI representation is the main contribution of the prior work it builds upon.\n\n- The entire experimental validation rests on the assumption: the availability of high-quality camera poses. The DSI construction is acutely sensitive to the accuracy of the input camera poses. In any real-world application, such as SLAM, these poses would be estimated and contain noise. The paper provides no analysis of the method's sensitivity to pose error. It is highly probable that the performance of DERD-Net would degrade dramatically with the noisy poses from a real tracking system. Moreover, this dependency makes the comparisons, particularly in Table 4, invalid. DERD-Net is compared against end-to-end stereo methods that do not require prior camera poses as input. These are methods tackling a harder, more complete problem. The authors concede that \"absolute accuracy is not directly comparable\", yet they proceed to make strong claims about superior robustness based on this comparison. This is not scientifically rigorous. The observed \"robustness\" is more likely a property of the DSI representation itself (which integrates information over time using clean poses) than a unique feature of the DERD-Net architecture.\n\n- The paper emphasizes an \"ultra-fast\" inference time of 0.37 ms.  However, this is the time to process a single Sub-DSI. The true computational bottleneck is the construction of the DSI itself, which requires iterating through and back-projecting millions of events for each depth map. The time required for this crucial pre-processing step is not reported, leading to an incomplete and overly optimistic assessment of the method's efficiency for real-time applications.", "was_revised": true, "final_justification": "Most of my major concerns have been addressed. The new evidence has convinced me of the value and robustness of the proposed contribution. I will increase my rating.", "rebuttal": "Thanks for your insightful comments, especially for appreciating the soundness of our approach and comprehensive evaluation.\n\n## **On being the first learning-based method for multi-view stereo**\nFrom the perspective of input (events and poses) to output (depth map), our method is the first one to successfully leverage deep learning to provide high-performance depth estimation.\nApologies for the lack of clarity regarding our scope. \nAlthough DSIs for event cameras were first proposed in EMVS [9] (IJCV 2018), the concept of DSI itself is based on space-sweeping [3] which was published in 1996.\nDSIs are simply an intermediate representation of our entire framework, chosen for its strong rich geometric encoding and multi-camera scalability. \n\nEven though the event-based DSI framework was proposed in 2018, there have been no successful attempts in combining it with machine learning. Learning from DSIs without introducing additional computational overheads has substantial technical challenges since directly processing the entire DSI volume would make the network too big for on-device and real-time applications, and would also be prone to overfitting due to the limited availability of training data. \nSince the DSI is mostly empty, a full-frame model would face a high-dimensional input space with sparse supervision, making it highly susceptible to overfitting and poor generalization. \nMoreover, a network consisting of only convolutions would not be able to model the sequential nature in which projected rays pass through space when creating the DSIs, which is needed to tackle variable depth resolution.\n\nOur key contribution lies in the additional step of extracting small independent sub-regions (Sub-DSIs) around relevant pixels that can be processed in parallel. \nThis novel design takes advantage of the inherent sparsity of event data and the DSI to overcome the aforementioned challenges and provide the following advantages instead: (i) independence from camera resolution; (ii) enabling of parallel processing; (iii) extension of the otherwise limited training set (e.g. for MVSEC the dataset increases from a few thousand to a more than a million data points); (iv) yields an ultra-lightweight network with very low constant memory occupation and inference time; (v) enhanced robustness by preventing the network from remembering the scene.  \n\nTo the best of our knowledge, this work is also the first learning-based  work to tackle \"non-instantaneous\" stereo depth estimation where long-term temporal aggregation is need from multiple viewpoints. With our approach, we argue that extracting camera pose information first and then explicitly using them for depth estimation yields better results than end-to-end learning methods (e.g., Table 4).\n\n## **Sensitivity to pose noise and drift from event-based SLAM system**:\n\nWhile DERD-Net relies on input camera poses, they do not need to be of \"high quality\". \nAvailability of camera poses is a common assumption in many 3D reconstruction pipelines, including popular methods like NeRF and 3D Gaussian Splatting. \nFollowing the reviewers suggestion, we ran experiments to investigate DERD-Net's performance with noisy camera poses. \nContrary to the concern raised, our results show that DERD-Net remains robust even when the input poses are degraded by noise and drift from a SLAM system. \nBelow, we report results using input poses estimated by event-based SLAM pipelines (ES-PTAM and ESVO2).\n\nOur experimental pipeline: Stereo events --> estimated poses using event SLAM --> DSI --> Depth maps using DERD-Net.\n\n(a) On DSEC zurich_city_04_a sequence, instead of GT poses (from LiDAR-IMU odometry), we used camera poses estimated by a stereo event odometry system ES-PTAM [1], which reported 131.62 cm Absolute Trajectory Error (ATE) in 50 m depth range scene, to run DERD-Net and achieved the following depth estimation performance:\n\n| Algorithm | Mean Err [m] ↓ | Median Err [m] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 1.56 (-3.11%) | 0.45 (-2.17%)  | 3.84 (-6.8%)   | 1.61 (-3.59%) |\n| Noisy DERD-Net + F_denser   | 1.74 (-3.33%) | 0.52 (-3.7%)  | 4.84 (-3.97%)   | 4.49 (-3.23%) |\n\nCompared to Table 3, while the number of points for which depth is evaluated slightly decreased, DERD-Net's depth estimation errors actually even improved. This demonstrates strong robustness to noisy poses obtained from an event SLAM system.\n\n(b) On MVSEC flying1, instead of GT poses (from motion capture), we used camera poses estimated by ES-PTAM [1] (ATE 14.93 cm in 6 m depth range scene) to run DERD-Net and achieved the following depth estimation performance:\n\n| Algorithm | Mean Err [cm] ↓ | Median Err [cm] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 12.72 (+8.81%) | 6.33 (+16.79%)  | 0.65 (-24.42%)   | 0.68 (-30.61%) |\n| Noisy DERD-Net + F_denser   | 15.76 (+6.06%) | 7.36 (+13.76%)  | 1.17 (-10.69%)   | 1.62 (-46.18%) |\n| Noisy Multi-Pixel DERD-Net + F_orig   | 13.53 (+10.27%) |  6.76 (+19.01%) | 0.65 (-24.42%) | 3.41(-33.91%) |\n| Noisy Multi-Pixel DERD-Net + F_denser   | 16.60 (+6.62%) | 7.65 (+16.08%)  |  1.21 (-11.68%) | 6.27 (-46.46%) |\n\nCompared to Table 8, the MAE and MedAE worsened only slightly, while bad-pix even improved. The strongest decrease here was in the number of points. However, note that Noisy DERD-Net + F_denser still predicts depths on 69% more pixels than MC-EMVS + F_orig using ideal poses, while still yielding a 30% lower MAE. For its multi-pixel version, Noisy DERD-Net shows superior performance to all SOTA methods while predicting on the most points. This highlights again DERD-Net's strong robustness to noisy poses obtained from an event SLAM system.\n\n(c)  For completeness, we also used poses from SOTA event-based stereo visual-inertial odometry system ESVO2 [2] to run DERD-Net on MVSEC flying1, flying2 and flying3 and report mean metrics. Compared to ES-PTAM, ESVO2 is notably more accurate due to the visual-inertial data fusion, producing less noisy poses, which yield an average ATE of ~8.3 cm in 6 m depth range scene.\n\n| Algorithm | Mean Err [cm] ↓ | Median Err [cm] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 11.53 (-1.37%) | 5.73 (+4.18%)  | 1.09 (+22.47%)   | 0.61 (-22.78%) |\n| Noisy DERD-Net + F_denser   | 13.69 (-10.17%) | 6.32 (-5.39%)  | 1.46 (-14.12%)   | 1.45 (-47.65%) |\n\nCompared to Table 3, this shows only a slight decrease in performance for DERD-Net's depth estimation error on F_orig, while it even improves on F_denser, thus indicating robustness to noisy poses obtained from a SLAM system using events and IMU. \n\nIn all cases, depth estimation performance remains nearly unchanged.\n\n## **Runtime analysis**\nOn a PC with Intel Xeon(R) W-2225 CPU operating at 4.10GHz with 8 cores, for MVSEC (DAVIS cameras with 346x260 pixels, 100 depth planes, 2 million events), each DSI construction takes ~45 ms, DSI fusion (for stereo) takes ~26 ms, and pixel selection takes ~0.2 ms. These values are common for both the SOTA method MC-EMVS and DERD-Net.\n\nargmax for a single sub-DSI takes 0.02 ms, whereas DERD-Net inference for a single sub-DSI takes 0.37 ms. By parallelly processing sub-DSIs in GPU, this amounts to a total time of 0.06 ms for argmax and 1.12 ms for DERD-Net inference. \n\nAs rightly pointed out, the time taken by DERD-Net inference is very small compared to DSI creation time. Using a network instead of argmax thus does not create a bottleneck in this framework. \n\nIt has been shown that DSI creation does not hamper real-time performance with DAVIS cameras [4, 5] because the 3D map can be updated infrequently and on-demand. With an additional ~1 ms increase in total runtime, our method DERD-Net can also run real-time.\n\n# Answers to Questions:\n1. Please refer to our sensitivity analysis to noisy poses above. We would also refer to additional experiments done for checking robustness in our response to Reviewer PC8y.\n\n2. To the best of our knowledge, our method is the first to provide a learning-based solution to the problem of long-term depth estimation from event data, which needs ego-motion information to properly fuse event data over time (i.e., multi-view). Other learning-based methods in the literature (Table 4) are short-term (in the style of MegaDepth's single-view depth prediction), hence they do not use or leverage ego-motion information. While the setups are different, we analyze how precise the value are. In Lines 313-319, we provide a caveat against blind comparison. \nEach method in Table 4 produces three accuracy numbers (according to the data splits); we compare how similar these numbers are per method (i.e., precision). Example: our method has ratios 11.11/11.69=0.95 and 12.28/11.69=1.05 with respect to split 1, whereas other methods have larger dissimilarities (i.e., lower precision), e.g., EIS (ICCV 21): 18.43/13.74=1.34 and 22.36/13.74=1.62. That is, we use precision or \"spread\" of the results as a measure of robustness.\n\n3. Please refer to our runtime analysis above.\n\nReferences:\n\n[1] Ghosh et al. ES-PTAM: Event-Based Stereo Parallel Tracking and Mapping. In ECCVW 2024.\n\n[2] Niu et al. ESVO2: Direct Visual-Inertial Odometry With Stereo Event Cameras. IEEE T-RO 41 (2025), 2164–2183.\n\n[3] R. T. Collins, \"A space-sweep approach to true multi-image matching,\" CVPR 1996, doi: 10.1109/CVPR.1996.517097.\n\n[4] Ghosh, Gallego. \"Live 3D reconstruction using stereo event cameras\", CVPR 2023 demo.\n\n[5] Ghosh, Cavinato, Gallego. SLAM with Stereo Event Cameras. ECCV 2024 demo."}, {"review_id": "mUhAcUHLQN", "reviewer": "Reviewer_KbkG", "summary": "This paper presents DERD-Net, a neural network for depth estimation from event data captured on a stereo setup. The core idea is to first convert the event stream into a Disparity Space Image (DSI), which captures the spatial density of back-projected event rays (assuming known camera poses).\nThe two DSIs from the stereo pairs are then fused together with the harmonic mean.\nThe method then focuses on small, local subregions (Sub-DSIs) around selected pixels, where a lightweight neural network applies 3D convolutions followed by a gated recurrent unit to predict depth from these Sub-DSIs.\nExperiments are done on the MVSEC and DSEC datasets, showing that the method DERD-Net outperforms all state-of-the-art baselines. Remarkable, even only monocular input (one DSI), the method could still achieve reasonable results.", "questions": "- Could you clarify how depth can be inferred under the monocular setting?\n- How sensitive is DERD-Net to noise or drift in camera poses?\n- How is the runtime compared to other state-of-the-art methods?\n- Have you explored the performance trade-off with larger network architectures? \n- Is the performance already saturated with this small model? Or is it possible to improve accuracy by paying more computational costs?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The method outperforms existing methods on standard benchmarks (MVSEC and DSEC).\n- The method is low-latency and ideal for real-world applications.\n- Surprisingly good depth estimates even from a single event camera.\n\nWeaknesses:\n- The paper could be clearer about how the monocular setting works, and how different are the operations from the default stereo setting. The presentation clarity is not very good. More explanation or intuition about why the method still works given monocular data is needed.\n- The method's performance is fundamentally dependent on having accurate camera poses to construct the DSI. The paper acknowledges this by stating that poses can be obtained from tracking methods or dead-reckoning. However, it does not analyze the model's sensitivity to pose errors.\n- The paper also does not seem to provide runtime comparisons with other state-of-the-art methods.", "was_revised": true, "final_justification": "The clarification and additional details are satisfactory. The rating is updated to 5.", "rebuttal": "Thanks for your positive feedback and for highlighting that our approach outperforms all SOTA methods on standard benchmarks, and that it even achieves comparable performance when using monocular input only. We appreciate our method being recognized “ideal for real-world applications” due to its low latency. We address the concerns below.\n\n## **Monocular setting**\n\nThank you for raising this point. We clarify that the DERD-Net architecture and inference process remain exactly the same in both the monocular and stereo settings. In both cases, DERD-Net operates on local subregions of the respective DSI. The difference lies in how this DSI is constructed. In the monocular setting, the DSI is populated by back-projecting rays from an event camera as described in the first steps of the monocular method EMVS [9] (IJCV 2018). Such a monocular DSI contains depth information due to motion parallax from multiple viewpoints. In the stereo setting, MC-EMVS [10] (AISY 2022), two such DSIs are simply fused into a new DSI using element-wise operations (like Harmonic Mean). \nFigure 4 in MC-EMVS illustrates the differences between monocular and stereo DSIs. The stereo setup makes depth convergence faster. While argmax (in MC-EMVS) relies on a single axis of accumulation and needs stereo cameras for fast depth convergence and reliable depth estimation, DERD-Net leverages deep learning to robustly infer local 3D ray density patterns from DSIs in monocular settings, enabling effective depth inference even when ray intersections are sparser and noisier, as is typical in the monocular setting. Apologies if the explanation was not clear.\n\n## **Sensitivity to pose noise and drift from event-based SLAM system**:\n\nWhile it is true that our method depends on input poses, other 3D reconstruction pipelines like NeRFs and 3D Gaussian Splats also have the same assumption. Similar to them, our camera poses can be estimated using a parallel module. \nWe show that even when we significantly reduce the quality of input poses by using other event-based SLAM systems, our depth estimation performance remains remarkebly robust.\n\nOur experimental pipeline: Stereo events --> estimated poses using event SLAM --> DSI --> Depth maps using DERD-Net\n\n(a) On DSEC zurich_city_04_a sequence, instead of GT poses (from LiDAR-IMU odometry), we used camera poses estimated by a stereo event odometry system ES-PTAM [1], which reported 131.62 cm Absolute Trajectory Error (ATE) in 50 m depth range scene, to run DERD-Net and achieved the following depth estimation performance:\n\n| Algorithm | Mean Err [m] | Median Err [m] | bad-pix [%] | # Points [million] |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 1.56 (-3.11%) | 0.45 (-2.17%)  | 3.84 (-6.8%)   | 1.61 (-3.59%) |\n| Noisy DERD-Net + F_denser   | 1.74 (-3.33%) | 0.52 (-3.7%)  | 4.84 (-3.97%)    | 4.49  (-3.23%) |\n\nCompared to Table 3, while the number of points for which depth is evaluated slighthly decreased, DERD-Net's depth estimation errors actually even improved. This demonstrates strong robustness to noisy poses obtained from an event SLAM system.\n\n(b) On MVSEC flying1, instead of GT poses (from motion capture), we used camera poses estimated by ES-PTAM [1] (ATE 14.93 cm in 6 m depth range scene) to run DERD-Net and achieved the following depth estimation performance:\n\n| Algorithm | Mean Err [cm] | Median Err [cm] | bad-pix [%] | # Points [million] |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 12.72 (+8.81%) | 6.33 (+16.79%)  | 0.65 (-24.42%)   | 0.68 (-30.61%) |\n| Noisy DERD-Net + F_denser   | 15.76 (+6.06%) | 7.36 (+13.76%)  | 1.17 (-10.69%)   | 1.62 (-46.18%) |\n| Noisy Multi-Pixel DERD-Net + F_orig   | 13.53 (+10.27%) |  6.76 (+19.01%) | 0.65 (-24.42%) | 3.41(-33.91%) |\n| Noisy Multi-Pixel DERD-Net + F_denser   | 16.60 (+6.62%) | 7.65 (+16.08%)  |  1.21 (-11.68%) | 6.27 (-46.46%) |\n\nCompared to Table 8, the MAE and MedAE worsened only slightly, while bad-pix even improved. The strongest decrease here was in the number of points. However, note that Noisy DERD-Net + F_denser still predicts depths on 69% more pixels than MC-EMVS + f_orig using ideal poses, while still yielding a 30% lower MAE. For its multi-pixel version, Noisy DERD-Net shows superior performance to all SOTA methods while predicting on the most points. This highlights again DERD-Net's strong robustness to noisy poses obtained from an event SLAM system.\n\n(c)  For completeness, we also used poses from SOTA event-based stereo visual-inertial odometry system ESVO2 [2] to run DERD-Net on MVSEC flying1, flying2 and flying3 and report mean metrics. Compared to ES-PTAM, ESVO2 is notably more accurate due to the visual-inertial data fusion, producing less noisy poses, which yield an average ATE of ~8.3 cm in 6 m depth range scene.\n\n| Algorithm | Mean Err [cm] | Median Err [cm] | bad-pix [%] | # Points [million] |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 11.53 (-1.37%) | 5.73 (+4.18%)  | 1.09 (+22.47%)   | 0.61 (-22.78%) |\n| Noisy DERD-Net + F_denser   | 13.69 (-10.17%) | 6.32 (-5.39%)  | 1.46 (-14.12%)   | 1.45 (-47.65%) |\n\nCompared to Table 3, this shows only a slight decrease in performance for DERD-Net's depth estimation error on f_orig, while it even improves on f_denser, thus indicating robustness to noisy poses obtained from a SLAM system using events and IMU. With regards to the decreased number of pixels, the same observations apply as for the table above.\n\n## **Runtime analysis and comparison with SOTA**\nOn a PC with Intel Xeon(R) W-2225 CPU operating at 4.10GHz with 8 cores, for MVSEC (DAVIS cameras with 346x260 pixels, 100 depth planes, 2 million events), each DSI construction takes ~45 ms, DSI fusion (for stereo) takes ~26 ms, and pixel selection takes ~0.2 ms. These values are common for both  DERD-Net and the SOTA method MC-EMVS.\n\nargmax for a single sub-DSI takes 0.02 ms, whereas DERD-Net inference for a single sub-DSI takes 0.37 ms. By parallelly processing sub-DSIs in GPU, this amounts to a total time of 0.06 ms for argmax and 1.12 ms for DERD-Net inference. \n\nAs rightly pointed out, the time taken by DERD-Net inference is very small compared to DSI creation time. Using a network instead of argmax thus does not create a bottleneck in this framework. \n\nIt has been shown that DSI creation does not hamper real-time performance with DAVIS cameras [2, 3] because the 3D map can be updated infrequently and on-demand. With an additional ~1 ms increase in total runtime, our method DERD-Net can also run real-time.\n\n# Answers to Questions:\n1. Please refer to our explanation above about the monocular setting.\n\n2. Please refer to the section above on sensitivity to noisy poses. For further experiments showing robustness, please see our response to Reviewer PC8y.\n\n3. Please see our section above on runtime analysis for DERD-Net as well as the SOTA stereo event SLAM method MC-EMVS [1]. We produce comparable (real-time) performance while significantly improving accuracy.\n\n4. Please refer to Table 6 and Section A.2 in the Appendix which shows how the network performance changes with increasing sub-DSI and consequently network size. Depth accuracy improves as the size increases but at a slower rate than number of operations.\n \n5. As shown in Section A.2 in our appendix, the performance of our network is not saturated yet. Training for a single epoch with sub-DSI size 9x9 already yielded better performance than multiple epochs on the 7x7 sub-DSI. Training more epochs and an even slightly larger network size would improve performance even further. \nHowever, we expect a tipping point, where increasing network size would incentivise it to learn to over-fit to a particular scenario because it considers a large spatial window and derives semantic context, as opposed to learning about local ray intersections. Moreover, greatly increasing the sub-DSI size would at some point lead to a significant reduction of training data (due to fewer sub-DSIs). Our approach deals with limited labeled training data (which is a prevalent issue in event vision) by turning each pixel into a training sample, therefore augmenting the available dataset. Generalization performance could be further improved by data augmentation methods such as mirroring, DSIs computed with noisy poses and various depth resolutions.\n\nReferences:\n\n[1] Ghosh et al. ES-PTAM: Event-Based Stereo Parallel Tracking and Mapping. In ECCVW 2024.\n\n[2] Niu et al. ESVO2: Direct Visual-Inertial Odometry With Stereo Event Cameras. IEEE T-RO 41 (2025), 2164–2183."}, {"review_id": "oAHm6uh3re", "reviewer": "Reviewer_gXBu", "summary": "The paper addresses the problem of depth estimation (both monocular and stereo) from event-camera images. The proposed method is pertinent because it uses an important modality (event-camera outputs) to estimate depth, which is fundamental to computer vision. It uses a patch-like formulation, efficient learning based processing, and a model that is a combination of 3D Convolutions and GRUs to achieve its desired objectives.  The paper reports very convincing metrics on multiple datasets making it a significant improvement over the prior art.", "questions": "+ What is the rationale behind the patch-like formulation and why can't vanilla 3D-Convolution achieve the same effect?\n+ How sensitive is the model's performance to errors in camera pose estimation? \n+ Can this method be potentially employed for real-time SLAM?", "limitations": "It would have been better if the authors had consolidated all limitations at one place.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n+ The method uses a event-cameras, which is an important input modality for depth estimation. The overall pipeline seems  fairly novel and technically sound.\n+ The method estimates depth in both monocular and stereo settings.\n+ The paper gains considerably in terms of efficiency over the SoTA as shown by its ultra-fast inference time.\n+ The patch-like (sub-DSI) formulation seems very pragmatic.\n+  The paper presents very well-designed experiments and is overall well-written.\n\nWeaknesses:\n\n+ Talking about the robustness of this method to noisy event-camera measurements and thereby incorporating statistical significance of the metrics could benefit the overall paper quality.", "was_revised": true, "final_justification": "I was already fairly convinced about the quality of this paper and the rebuttal has convinced me further in this regard.", "rebuttal": "Thanks for the positive feedback, especially regarding our method's novelty, efficiency and well-designed experiments. Our work aims to propose learning pipelines targeting the challenges of processing sparse, asynchronous event data for efficient depth estimation. \n\n## **Robustness to noise in event camera and pose measurements**\nAll the experiments in the paper were conducted on real data with real noisy measurements. There is significant noise in the MVSEC flying sequences because it uses an older (mDAVIS346) sensor, and contains interference from the motion capture infrared lights. Thus, our method already shows robustness to realistic noise in the event sensor. Moreover, we evaluate on multiple datasets that use different sensors (DSEC uses Prophesse Gen3 cameras), and obtain SOTA performance under diverse noise characteristics. Although a more systematic noise analysis by injecting artificial random noise could be useful to gauge statistical significance, it is non-standard in 3D reconstruction and thus beyond the scope of the paper.\n\nFor robustness against noisy poses, we report results by running DERD-Net on poses estimated by other stereo event-based SLAM pipelines (ES-PTAM and ESVO2) which contain realistic noise and drift.\n\nOur experimental pipeline: Stereo events --> estimated poses using event SLAM --> DSI --> Depth maps using DERD-Net\n\n(a) On DSEC zurich_city_04_a sequence, instead of GT poses (from LiDAR-IMU odometry), we used camera poses estimated by a stereo event odometry system ES-PTAM [1], which reported 131.62 cm Absolute Trajectory Error (ATE) in 50 m depth range scene, to run DERD-Net and achieved the following depth estimation performance:\n\n| Algorithm | Mean Err [m] ↓ | Median Err [m] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 1.56 (-3.11%) | 0.45 (-2.17%)  | 3.84 (-6.8%)   | 1.61 (-3.59%) |\n| Noisy DERD-Net + F_denser   | 1.74 (-3.33%) | 0.52 (-3.7%)  | 4.84 (-3.97%)   | 4.49 (-3.23%) |\n\nCompared to Table 3, while the number of points for which depth is evaluated slightly decreased, DERD-Net's depth estimation errors actually even improved. This shows strong robustness to noisy poses obtained from an event SLAM system.\n\n(b) On MVSEC flying1, instead of GT poses (from motion capture), we used camera poses estimated by ES-PTAM [1] (ATE 14.93 cm in 6 m depth range scene) to run DERD-Net and achieved the following depth estimation performance:\n\n| Algorithm | Mean Err [cm] ↓ | Median Err [cm] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 12.72 (+8.81%) | 6.33 (+16.79%)  | 0.65 (-24.42%)   | 0.68 (-30.61%) |\n| Noisy DERD-Net + F_denser   | 15.76 (+6.06%) | 7.36 (+13.76%)  | 1.17 (-10.69%)   | 1.62 (-46.18%) |\n| Noisy Multi-Pixel DERD-Net + F_orig   | 13.53 (+10.27%) |  6.76 (+19.01%) | 0.65 (-24.42%) | 3.41(-33.91%) |\n| Noisy Multi-Pixel DERD-Net + F_denser   | 16.60 (+6.62%) | 7.65 (+16.08%)  |  1.21 (-11.68%) | 6.27 (-46.46%) |\n\nCompared to Table 8, the MAE and MedAE increased only slightly, while bad-pix even improved. The strongest decrease here was in the number of points. However, that Noisy DERD-Net on F_denser still predicts on 69% more pixels than MC-EMVS with f_orig using *ideal* poses, while still yielding a 30% decrease in MAE. For its multi-pixel version, Noisy DERD-Net shows superior performance than all SOTA methods while still predicting on most points. This shows strong robustness to noisy poses obtained from an event SLAM system.\n\n(c)  For completeness, we also used poses from SOTA event-based stereo visual-inertial odometry system ESVO2 [2] to run DERD-Net on MVSEC flying1, flying2 and flying3 and report mean metrics. Compared to ES-PTAM, ESVO2 is notably more accurate due to the visual-inertial data fusion, producing less noisy poses, which yield an average ATE of ~8.3 cm in 6 m depth range scene.\n\n| Algorithm | Mean Err [cm] ↓ | Median Err [cm] ↓ | bad-pix [%] ↓ | # Points [million] ↑ |\n|---|---|---|---|---|\n| Noisy DERD-Net + F_orig        | 11.53 (-1.37%) | 5.73 (+4.18%)  | 1.09 (+22.47%)   | 0.61 (-22.78%) |\n| Noisy DERD-Net + F_denser   | 13.69 (-10.17%) | 6.32 (-5.39%)  | 1.46 (-14.12%)   | 1.45 (-47.65%) |\n\nCompared to Table 3, this shows a slight decrease in performance for DERD-Net's depth estimation error on f_orig, while it even improves on f_denser. Again, the strongest decrease is in the number of predicted points, as observed in (b). This shows robustness to noisy poses obtained from a SLAM system using events and IMU.\n\n## **Real-time SLAM**\nOur method runs real-time with DAVIS346 cameras.\nOn a PC with Intel Xeon(R) W-2225 CPU operating at 4.10GHz with 8 cores, for MVSEC (DAVIS cameras with 346x260 pixels, 100 depth planes, 2 million events), each DSI construction takes ~45 ms, DSI fusion (for stereo) takes ~26 ms, and pixel selection takes ~0.2 ms. \nOn top of the above steps, DERD-Net inference for a single sub-DSI takes 0.37 ms. By parallelly processing sub-DSIs in GPU, this amounts to a total time of 1.12 ms for DERD-Net inference. \n\nIt has been shown that the stereo SLAM method ES-PTAM [1] which also uses DSI fusion for map can run real-time with DAVIS cameras [3] because the 3D map can be updated infrequently and on-demand. Compared to ES-PTAM, our method needs an additional ~1 ms in total runtime, and thus can also run real-time.\n\n# Answers to Questions:\n\n1. Processing sub-DSIs / patches instead of full 3D convolutions is preferable because:\n    - We can leverage sparsity of DSI by only processing a subset of the voxels, producing overall faster inference and smaller networks (lower memory, lower training time). This will otherwise be prohibitively expensive for real-time processing.\n    - A purely convolutional network would struggle to model the sequential structure of ray accumulations along the depth dimension of the DSI. This sequential aspect of how rays build up evidence for surfaces along their trajectory is critical for distinguishing true ray intersections (indicative of surfaces) from noise or spurious alignments. Our use of a GRU allows the network to capture this progression explicitly.\n   - A 3D convolution on the full DSI would lead to reduction of training data because each DSI will be a single training sample. Moreover, the network may learn to over-fit to a particular scenario because it considers a large spatial window and derives semantic context, instead of learning about local ray intersection patterns, to estimate depth. Our approach (breaking into smaller sub-DSIs) deals with limited labeled training data (which is a prevalent issue in event vision) by turning each pixel into a training sample, therefore augmenting the available dataset. For instance, each MVSEC sequence only has ~1.5k frames, which would result in only 3k data points (including cross-validation). A bigger vanilla network would make learning from less data more difficult. By creating sub-DSIs, we produce >1 million training samples per training set with only 70k parameters to train.\n   - It is independent of input camera resolution.\n\n2. Please refer to noise sensitivity analysis above. For further experiments showing robustness, please also see our response to Reviewer PC8y.\n\n3. Yes, our method can be used for real-time SLAM. Please refer to the section on Real-time SLAM above for runtime. For further details and comparison to SOTA method MC-EMVS, please refer to our Response to Reviewer Yns5. \n\nReferences:\n\n[1] Ghosh et al. ES-PTAM: Event-Based Stereo Parallel Tracking and Mapping. In ECCVW 2024.\n\n[2] Niu et al. ESVO2: Direct Visual-Inertial Odometry With Stereo Event Cameras. IEEE T-RO 41 (2025), 2164–2183.\n\n[3] Ghosh, Cavinato, Gallego. SLAM with Stereo Event Cameras. ECCV 2024 demo."}]

Author rebuttal:

Meta-review: The proposes a novel paradigm for estimating sparse depth maps from event cameras by introducing Disparity Space Images (DSIs) for events. The method achieves state-of-the-art results (PC8y, Yns5) in both monocular and stero settings, with an extremely low latency time of 1ms. 

The paper received two borderline and two strong accepts. The AC agrees with these reviews and recommends the paper for acceptance.
Decision: Accept (spotlight)