Event-based HDR Structured Light
Authors: Jiacheng Fu
Yue Li
Xin Dong
Wenming Weng
Yueyi Zhang
Zhiwei Xiong
Venue: NeurIPS 2025 poster ()
Keywords: 3D Reconstruction
HDR Structured Light
Event Camera
Primary area: machine_learning_for_sciences
TL;DR: We develop the first HDR 3D measurement framework for event-based structured light systems, enabling accurate 3D reconstruction even under extreme conditions.

Abstract: Event-based structured light (SL) systems have attracted increasing attention for their potential in high-performance 3D measurement. Despite the inherent HDR capability of event cameras, reflective and absorptive surfaces still cause event cluttering and absence, which produce overexposed and underexposed regions that degrade the reconstruction quality. In this work, we present the first HDR 3D measurement framework specifically designed for event-based SL systems. First, we introduce a multi-contrast HDR coding strategy that facilitates imaging of areas with different reflectance. Second, to alleviate inter-frame interference caused by overexposed and underexposed areas, we propose a universal confidence-driven stereo matching strategy. Specifically, we estimate a confidence map as the fusion weight for features via an energy-guided confidence estimation. Further, we propose the confidence propagation volume, an innovative cost volume that offers both effective suppression of inter-frame interference and strong representation capability. Third, we contribute an event-based SL simulator and propose the first event-based HDR SL dataset. We also collect a real-world benchmarking dataset with ground truth. We validate the effectiveness of our method with the proposed confidence-driven strategy on both synthetic and real-world datasets. Experimental results demonstrate that our proposed HDR framework enables accurate 3D measurement even under extreme conditions.

Event-based HDR Structured Light
Jiacheng Fu Yue Li Xin Dong Wenming Weng Yueyi Zhang Zhiwei Xiong"
University Science and Technology of China
j cc_ fu@mal.. ustc. edu cn zwxiongBustc.edu www.guustc.euu..com
Abstract
Event-based structured light (SL) systems have attracted increasing attention for their potential in capability of event cameras, reflective and absorptive surfaces still cause event clutter and absence, which produce overexposed and underexposed regions that degrade the reconstruction quality. In this work, we present the first HDR 3D measurement framework specifically designed for event-based SL systems. First, introduce multi-contrast HDR coding strategy that facilitates imaging of areas with different reflectance. Second, to alleviate frame- interference caused by overexposed and underexposed areas, we propose a universal confidence-driven stereo matching strategy. Specifically, we estimate a confidence map as the fusion weight for features via an mergy-guidee confidence estimation. Further, we propose the confidence propagation volume, an innovative cost volume that offers both effective suppression of inter-frame interference and strong representation capability. Third, we contribute event-based SL simulator and propose the first event-based HDR SL. dataset. We also collect real-world benchmarking dataset with ground truth. We validate the effectiveness of our method with the proposed -orfidence-driven strategy on both synthetic and real-world datasets. Experimental results demonstrate that proposed HDR framework enables accurate 3D measurement even under extreme conditions. The code and data are available at httphtt////// h/ggtttt...... www./ommaaaaaaee Coom//uma233//veea Eventt-- Ewwet-baseddt-eeeeeeeeeeeeeeaeee based- baasddHHR--3 HDR-SL
3D measurement. Despite the inherent HDR
1 Introduction
Structured light (SL) systems perform high-quality depth estimation by projecting predefined patterns onto a scene and analyzing their deformation upon interaction with object surfaces. Their high precision and dense reconstruction capabilities make them well-suited for applications in both consumer electronics and industrial applications [ 3). However, reconstructing scenes under high dynamic range (HDR) conditions remains significant challenge, conventional cameras only capture the brightness levels with limited range. The target surfaces, such as rust, glossy paint, or carbon fiber, always lead to overexposure underexposure of the projected patterns in captured images 4 S. resulting in degraded or failed reconstruction. Existing solutions typically adopt multi-exposure fusion and multi- projeiipooottion fusion techniques, but these approaches require extensive manual tuning of exposure settings and projection intensity to achieve satisfactory results M.
Recently, event cameras 6 high-speed neuromorphic sensors with HDR capability, have been integrated into SI. systems, demonstrating advantages in capturing high-speed dynamic scenes and maintaining imaging quality under complex illumination conditions. Previous work has shown that event-based SL systems offer improved HDR scene reconstruction compared to frame-based cameras, however, they still suffer from limitations of low reconsiruction quality and insufficient robustness, especially in overexposed and underexposed areas 7 Practical tests further indicate that,
"Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(a) HDR Scene
(b) Event Frame with Spockle Coding
(c) Disparity from (b)
(d) Ours
Figure I: Despite the inherent HDR capabilities of event cameras, they remain inadequate to handle HDR scenes with high reflectance variations (a), which cause event clutter and absence in the captured event frame (b). (c) shows the disparity estimated from (b) using SGBM, indicating the compromised performance of common event-based structured light systems in HDR environments. To address this, our HDR 3D measurement framework introduces multi-contrast HDR coding strategy that employs range-partitioned sensing image regions with different reflectance. As shown in (d), combined with our cconfidence-drive stereo matching strategy, our method demonstrates the ability to reconstruct high-reflectance surfaces (lamp), low-reflectance surfaces (black tripod), and regions affected by multipath interference (bowl), achieving superior reconstruction results even under extreme HDR conditions.
despite the inherent dynamic range advantages of event cameras, existing event-based SL systems remain inadequate for real-world HDR scene reconstruction, as illustrated in Fig.
In this work, we pioneer the first HDR 3D measurement framework for event-based SL., First, we propose multi-contrast HDR coding strategy that uses three pairs predefined speckle patterns with different contrast and content for event triggering and depth encoding. Leveraging the HDR capability of event cameras, different contrast enables imaging across different reflection regions without tedious projection parameter tuning, while content-wise different speckles provide coding and imaging gains, maximizing the depth information encoded in three frames.
Second, to alleviate frame-frame interference caused by overexposed and underexposed regions in individual frames, we propose universal confidence-driven stereo matching strategy. Specifically, it includes an energy energy-guided confidence estimation (ECH) module, which leverages the energy distribution of binary speckle patterns as a prior to estimate confidence map for each of the three input frames as the fusion weight. Building upon this, we innovate the confidence propagation volume (CPV) - a confidence-driven cost volume construction method. During volume construction, confidence maps are propagated along the channel and disparity dimensions, and subsequently used to weight the left and right features. This enables accurate extraction of valid coding information across the three frames, leading to a cost volume with suppressed inter-frame interference and enhanced representation capability.
Third, for model training, we contribute SL simulator based on Blender [0 and propose the first event-based HDR SI. dataset. We also build real-world benchmarking dataset with ground truth (GT) based on our built event-based SL system and a high-precision HDR SI. scanner.
We integrate the confidence- stereo matching strategy into IGEV, competing stereo network [1 to form our HDR 3D reconstruction method. Experiments on synthetic and real-world datasets demonstrate the superior performance of method, which achieves robust 3D reconstruction even under extreme HDR conditions (Fig. (d). Besides, we also demonstrate the universality of our confidence-drivee strategy, which significantly enhances the accuracy of networks across diverse paradigms.
2
Related Work
2.1
Event-based Structured Light
Existing event-based SL studies primarily focus on exploring event-triggering mechanisms and corresponding algorithms to improve acquisition speed and reconstruction accuracy 7 g 12 13
14 15
Leveraging the inherent HDR capability of event cameras, existing event-based methods offer certain advantages HDR scene perception compared frame-based cameras, yet they still struggle to handle complex HDR scenes. Matsuda al. explored line-scanning-bass reconstruction method

---

0.
Event Frames
(b) Corresgonding Confidence Maps
Figure 2: Visualization of event frames derived by multi-contrast HDR coding and corresponding
estimated confidence maps.
and tested it on shiny steel sphere 7 However, the reconstruction quality was limited by the constraints of parly-generation event sensors. Muglikar al. managed to reconstruct plaster of David under varying lighting conditions [8), however, the target surface was diffuse and did not represent typical HDR scenario, which limited the persuasiveness of the results. In [2), the proposed sinusoidal fringe-based encoding demonstrated ability in handling high-reflectance and high-contrast scenes, but still suffered from insufficient robustness, limited resolution and unsatisfactory detail recovery. To address these challenges, we propose the first HDR 3D measurement framework tailored for event-based SL systems, enabling accurate and robust reconstruction even in extreme scenarios.
2.2 Cost Volume-based Stereo Matching
Current cost
deep stereo methods fall into two categories. One is cost-filtering approaches, which build a dense 4D cost volume and aggregate matching costs via 3D CNN [6 I7
[8 [9]
These methods proposed various cost-volume paradigms: GC-Net introduces simple yet effective concatenation volume with strong representational power [16]; GweNet presents a group- wise correlation volume that combines the benefits of concatenation and correlation volume [18]
Recent attention-basee variants ACVNet and FastACV [19] employ -weigttd concatenation volumes to suppress redundancy while preserving discriminative matching cues. The other category -rrraive-optimizationn approaches (20] [1]] This category is exemplified by RAFT-Sterco [20], which introduces a deep iterative architecture using GRU-based updater to refine disparity by sampling cost values from an all all-pairs correlation volume. More recently, IGEV-Stereo [1]] combines these paradigms and achieves impressive results. However, when naively applied in our HDR setting, existing approaches would suffer from inter-framm interference that undermines cost-volume expressiveness and degrades the reconstruction quality.
3 Event-based HDR Structured Light
3.1 Multi-Contrast HDR Coding
To perceive HDR scenes using sensors with limited dynamic range, is necessary adopt a range- partitioned sensing strategy. Considering that event cameras only respond to changes in illuminance, we propose multi-contrast encoding strategy to enable the imaging of surfaces with different reflectance.
Specifically, employ N pairs of binary speckle patterns 3 consists of two sequentially projected frames with distinct contrast and content. We define the white and the rest to be the background area. In each
trigger events, where each pair
foreground speckles to constitute the foreground pair, the first frame F, assigns uniform intensity J, to the background area and to the foreground, while the second frame F2 assigns I5 to the foreground and 0 to the background. Let / (æ, y, L- At) and I (x, y, f) denote the observed intensities at pixel (Ir, 3) when projecting Fi and F2, respectively, where / Al and i the timestamps corresponding the two projections. An event is triggered when the intensity change between two projections exceeds triggering thresholds:
1(x, v,
if log
2 C
I(x, At) I U1,)
e(x, y.
(I)
if log
S C-2
T(x, - At)
where C and C2 triggering thresholds for positive and negative events, respectively.
This configuration ensures that the foreground speckle area exhibits a significant positive intensity change, thereby triggering events. In contrast, non-speckle background regions undergo negative
3

---

F1a
FLL
Fro
Cost Volume
Feutures
Captured Event Frames
Fenture Extraction
Energy- guided
Confidence
Progagation
Cost Aggregation
Confidence Estimation
Volume Construction
Confidence Mups
Energy Map
Feature Extraction
Fg2
FRI
Fgo
Feature Extraction
Right Features
Disparity
Reference Frames
Figure 3: Pipeline of the Confidence- Derivvence-Driven Stereo Matching strategy.
change. This design aims to suppress positive intensity changes in non-speckle regions, thereby reducing unwanted event triggers that may arise from high reflectance. Note that, for bandwidth efficiency, we do use negative events in our setting. which also ensures that interfering negative events will be triggered. By adjusting I, and y in each pair, we construct multiple contrast levels to enable imaging across regions with varying reflectance as shown in Fig. a).. Compared to frame- based methods that require extensive per-scene adjustment of projection settings, a key advantage of our strategy is that, leveraging the inherent HDR capability of event cameras, it can cover the dynamic range of most complex scenes using only three preset [T, Ty]. This significantly simplifies the cost system deployment.
As for the content, we adopt N mutually independent speckle frames in each pair. As shown in 22]. this strategy offers greater coding distinguishability than using one single pattern, which in turn improves reconstruction accuracy. Besides, projecting several mutually independent random- speckle frames is equivalent taking multiple independent samples of the surface micro-facee facet normal distribution, which reduces the likelihood that all frames at given pixel suffer from event clutter or absence. In practice, it is found that selecting N strikes an optimal balance between accuracy and efficiency (see the full supporting experiments in the supplement). This setting is used in all subsequent method designs and experiments.
3.2 Confidence-Driven Stereo Matching
Our proposed confidence-driven stereo matching strategy is illustrated in Fig. The inputs of the network three rectified stereo pairs: the left frames three event frames produced by the multi-contrast coding, and the right frames are corresponding projected reference frames. The Energy-Guided Confidence Estimation (ECE) processes each left image together with its energy map to predict a per-frame confidence map. These confidence maps, combined with the features from both views, subsequently fused to construct the Confidence Propagation Volume (CPV) that underpins the final disparity-estingg estimation stage.
3.2.1 Energy-Guided Confidence Estimation
Our design of the Energy-Guide Confidence Estimation (ECE) module is motivated by two key observations. On the one hand, as shown in Fig. [aa,, overexposed and underexposed areas carry no reliable coding information. When the three pairs are directly used to compute the cost as in 22). overexposed and underexposed regions not only fail provide valid matching cues, but also contaminate the correct cost computed from well-exposed regions during multi-frame cost fusion, ultimately leading to reconstruction failure. We refer to this phenomenon as inter-frame interference. To alleviate this, low-qualig, areas should contribute less to cost fusion to the construction of the matching feature, which calls for frame-wise confidence estimation mechanism. On the other hand, low- and high-reflectance regions tend to exhibit event absence and event clutter, respectively,

---

Channel Propatation
FB
Fs
C,(x,))
DCN
Penture Fisson
CNN
Enhanced Left
Left Feutures
Caloa
MC
C2lo.y)
Channel Propa apation
Confidence
Propagation
SE1
SSE
FSES
(x(,y))
Volume
Confidence Maps
(x.
Disparity Propagation
CNN
(x(x.
Pusion
Disparity Candidates
Enhanced Right Feutures
Confidence Map
Right Features
(a) Energy-Guided Confidence
(b) Confidence Propagation Volume Construction
Estimation
Figure 4:
The overview of the proposed ECE (a) and CPV (b). (a) illustrates how the ECE mod- ule processes the low-contrast input. MC and HC stand for medium contrast and high contrast, respectively. FSE1. FSE2, and FSEs are the fused features estimated from the low-, medium-, and high-contrast branches.
leading to overly low high local energy responses. Moreover, binary event frames contain only the projected speckles without background interference, making the local energy distribution a meaningful prior for speckle quality. This motivates to estimate the confidence maps using both the speckle pattern and its corresponding energy map.
Based on these insights, we propose ECE, shown in Fig. 4 a). For each left frame, features of the energy map Fg and the raw speckle data Fs are processed using deformable convolution (DCNv4), which adaptively adjusts sampling positions to accommodate the diverse and spatially irregular random speckles caused by extreme reflectance variations in HDR scenes. This flexibility improves the network's s ability to assess speckle quality, leading to more accurate confidence estimation compared to conventional CNNs with fixed sampling grids. Next, we modulate the energy map features through element-wiss multiplication enhance the speckle features, resulting in the fused feature FSEI. This operation leverages the complementary nature of the two inputs, where the raw speckle provides fine-grained pixel-level details and the energy map offers noise-robust confidence priors. Finally, since the confidence reflects the relative quality among the three frames, the confidence estimation for each frame is made jointly based all three fused features through residual structure. We visualize the learned confidence maps, as shown in Fig. b)). The visualizations show strong correspondence with the input data, demonstrating clear interpretability: regions suffering from overexposure or underexposure exhibit low confidence, while well-exposed areas yield high confidence values.
3.2.2 Confidence Propagation Volume Construction
Leveraging the estimated confidence maps, we propose a cost-volume construction strategy as shown in Fig. (b). For each spatial location (xr, y), the scalar confidences (Co(x,,) ()) C1 y). C2(2, y)) of the three input frames are individually expanded along the channel dimension to match the channel number of each frame's feature. These expanded vectors then concatenated to form the fusion weight vector C(æ, y). The confdeence-wighhee feature is computed by element-wise multiplication with the corresponding frame features:
F, Fx, (x. v) = Fr, (x. y) C(x.y).
This process weights the features of each frame according to the quality of the projected speckle. In this way, the resulting feature minimize interference caused by suboptimal imaging regions.
Although the right features are extracted from reference frames that are free from under- or over- exposure, it is still necessary to impose fusion weights them in order maintain consistency
5

---

with the confidence-weighteed left features during disparity searching, Therefore, during the disparity search of Fr, (z, y). we propagate the fusion weight C(x,y) y) along the disparity dimension weight the right features within the disparity searching range:
F FR(d; v y) = C(a, y) @ Fn(x+ddyy)
where FR(x+d,,)) y) denotes the right feature sampled at (x+d, y), and FR I, 2,3) denotes the corresponding confidence-weightee right feature. This ensures that both left and right features are modulated by the same fusion weights, enabling consistent and confidence-awee aware stereo matching.
features from both views are further refined by two independent feature fusion modules, each consisting of 1 x 1 convolutions, to reorganize information along the channel dimension, enhancing feature representation ability while reducing redundancy. Finally, the enhanced left feature is matched with the right features in disparity searching range to assemble the CPV. By integrating the proposed confidence-drived strategy into IGEV [1] HDR 3D reconstruction.
The confidence-weighiee
we derive our final method for
4 Experiment
4.1 Dataset and Implementation Details
Synthetic Dataset. To support supervised training, we design an event-bused SL simulator in Blender [10 and propose the first basee HDR SL dataset. Using the calibrated intrinsic and extrinsic parameters from our real SL system, we construct its digital twin in Blender. Since Blender lacks native event-camera sensor model, render two RGB frames, one with the projected speckle pattern and one without, and take their pixel-wise difference to approximate the imaging process of the event camera. In addition, a physically based projector is implemented using Blender's node-tree system, allowing its optical properties to be precisely configured by the intrinsics. HDR scenes are built from high-fidelity, textured meshes sourced from the OmniObject3D dataset (23). To simulate the reflectance behavior of real-world HDR surfaces, we vary each object's 's metallicity. micro-facet roughness and self-emission coefficient. The resulting scenes contain mirror-like specular peaks alongside low-reflectance regions, offering realistic dynamic range diversity. depth also rendered by Blender for supervision. Samples are shown in the supplement. We render a total of 8,550 image pairs at a resolution of 1280x 704. The dataset is split into 7,500 training samples and 1,050 test samples.
scene
World Dataset. We build a monocular event-based SL system using the Prophesee EVK4 event camera, which offers a resolution of 1280 720 and dynamic range exceeding 120 dB 22).. and the DLP6500 projector from Texas Instruments [25]. In addition, we use an HDR structured light scanner 266 capture GT point cloud. A total of 15 complex HDR scenes, including highly reflective, absorptive, transparent, and fine-grained objects, are captured. Each sample contains three event frames generated by our multi-contrast coding and corresponding GT point cloud.
Implementation Details. For multi-contrast HDR coding, the projection intensities of the three pairs set to [32,55], [32.200], and [0.255], respectively. The depth sensing rate of our system is 200 Hz. Our HDR 3D reconstruction method is implemented in PyTorch, trained on a synthetic dataset, and evaluated on both synthetic and real-world data. The disparity search range is set to 256. All experiments are conducted on the NVIDIA 3090 GPUs
4.2 Baselines and Evaluation Metrics
For comparison, we evaluate our method against several baselines. They include two traditional stereo algorithms, Block Matching (BM) [27 and Semi-Global Block Matching (SGBM) (28.. two monocular structured- light depth estimation methods, CTD [29] and GigaDepth [30), as well as three
representative cost volume-based stereo networks, RAFT-Sterco 20), FastACV 199 and IGEV (.] In our setting with three input frames, BM and SGBM are applied independently to each frame, and the final disparity map is obtained by averaging the disparities of high-confieence regions. For the deep learning-basee based methods, the three frames are concatenated along the channel dimension and jointly fed into the network. We evaluate disparity quality with three metrics: end-poin error EPE (mean absolute pixel error), Bad 7 rates for E {0.5, 1, 2, 3, 5) px (fraction of pixels whose error exceeds T), and DI (pixels whose error > 3 px and > 5% of GT). Together, they summarize average accuracy, thresholded error rates, and gross outliers.

---

Methods
EPE 1
Bad 0.5 4
Bad 1.0 4
Bad 2.0
Bad 3.0
Bad 5.0
D1 4
Time(s)
BM
31,4518
0.3919
0.3213
0.3156
0.3120
0.3053
0.3112
0.0030
BM-3
18.2216
0.3141
0.2453
0.2401
0.2349
0.2274
0.2302
0.0120
SGBM
13.3384
0.2697
0.1563
0.1421
0.1390
0.1359
0.1363
0.0060
SGBM-3
6,3327
0.2243
0.1082
0.0918
0.0874
0.0830
0.0831
0.0210
CTD
26.8375
0.3341
0.2755
0.2638
0,2598
0.2465
0.2489
0.0190
GigaDepth
8.6688
0.1425
0.1210
0.1135
0.1111
0.1082
0.1070
0.0210
FastACV
0.7112
0.1068
0.0600
0.0342
0.0243
0.0159
0.0154
0.0670
RAFT-Stereo
0.4136
0.1286
0.0507
0.0219
0.0136
0.0077
0.0077
0.8490
IGEV 0.3863
0.0548
0.0311
0.0177
0.0127
0.0084
0.0076
0,4200
Ours
0.2937
0.0359
0.0203
0.0122
0.0090
0.0063
0.0062
0,4080
Table I: Quantitative results the synthetic dataset. All metrics are error rates (4 lower is better).
Only BM and SGBM are single-frame methods, others are 3-frame methods.
256
256
Soene&GT
SGBM-3
GigaDepth
FastACV
IGEV
Ours
Figure 5: Qualitative results on the synthetic dataset. Two samples are shown, with the first row showing the disparities and the second row showing the scene and the error maps.
4.3 Synthetic Results
The results in Table demonstrate the superiority our proposed 3D measurement framework. significantly outperforming existing event-based monocular and stereo methods. The performance of BM and SGBM reflects the typical limitations of conventional event- event-based structured light systems in HDR scenes, where large reconstruction errors are observed. Their three- variants (BM-3, SGBM-3) exhibit noticeable improvements, which validates the effectiveness our proposed multi- contrast HDR coding. Among all baselines, deep learning- based stereo networks (RAFT-Stereo, FasAACV, IGEV) achieve markedly better performance. In comparison, our method achieves further improvements over the strongest baseline, IGEV, demonstrating the effectiveness of the proposed confidence-driven stereo matching strategy.
Qualitative results are shown in Fig. 5 As observed, traditional methods and monocular estimation networks produce large areas of mismatches and perform poorly near object boundaries. While deep lcarning-basee -based stereo methods partially solve these issues, however, as indicated in the error map, they still exhibit noticeable reconstruction errors regions with specular reflections or fine structures. In contrast, our method achieves superior reconstruction quality in both general and HDR-chin HDR-challenging regions, delivering more accurate disparity estimates. See more qualitative results in the supplement.
4.4 Ablation Study
To validate the effectiveness of each component in our framework, we conduct an ablation study based on method and present quantitative results in Table
Content Variation Multi-Contras HDR Coding. The first two variants (#A and #B) compare the impact of different 3- 3-frame inputs, with and without speckle content variation. As can be seen,

---

Table 2: Ablation study on the content variation in multi-contrast HDR coding (Content), confidence propagation volume (CPV), energy prior (EnP), and the energy-guided confidence estimation module (ECE).
Variants Content CPV
EnP
ECE
EPE & Bad L Bad 1.0 | Bad 2.0 L Bad 3.0 L Bad 5.0 L
D1
#A
X
0.5915
0.0776
0.0507
0.0324
0.0221
0.0141
0.0145
#B
x
X
0.3863
0.0548
0.0311
0.0177
0.0127
0.0084
0.0076
#C
/
X
X
0.3433
0.0415
0.0242
0.0145
0.0111
0.0077
0.0073
#D
V
V
X
0.3215
0.0376
0.0218
0.0131
0.0097
0.0067
0.0065
#F
V
V
0.2937
0.0359
0.0203
0.0122
0.0090
0.0063
0.0062
6.473/8.195 195
41.94 04.42
3,66X00 17U
3,0480015
2.77800.13
6,60000 172
$6.50 516
3.458/0
2.47/0.133
937 132
90.40
507.0 624
1188800321
$160 200
75
765100
KRR 633
81600
UTS
0.896.0. 120
61 01.91600.0 599
1651
118800082
103300024
GignDepth
Figure 6: Qualitative and quantitative results of real-world HDR scenes. The numbers above each result denote the EPE and Bad 2.0 error for the corresponding scene.
applying proposed content-varying coding reduces Bad 1.0 by 38.7%, indicating that introducing content diversity enhances the distinguishability of speckle coding and enables more accurate local matching. Meanwhile, Bad 5.0 decreases by 47.6%, suggesting that method not only brings imaging gains in challenging HDR regions but also provides richer cues for disparity completion.
Confidence Propagation Volume. For variant #C, we substitute the ECE with convolutions and cancel the use of the energy prior, resulting a compromised confidence estimation. Nevertheless, comparing the results of variants #B and #C, it can be found that applying CPV can still effectively suppress the inter-frame interference and provide better results.
-ecrgy-guided Confidence Estimation. We add the energy prior (EnP) to variant #C to produce variant #D. The results are improved, demonstrating the effectiveness of energy prior in confidence estimation. Finally, we substitute convolutions with the proposed ECE and achieve the best result in variant #E. can be deduced that the high-quality confidence map estimation provides better fusion weights for multi-frame information, which maximizes the suppression of the interference and enhances the representation capability.
4.5 Real-World Results
To evaluate the generalizability of our method, we test on real-world HDR scenes captured by our custom-built SL. system. For qualitative and quantitative comparison, we first convert the estimated disparity maps into depth and reconstruct corresponding 3D point clouds. The GT point cloud captured by the Photoneo MotionCam-31 M+ is then aligned each predicted point cloud using

---

Soene
High Kefleciairce
High Reffectance
Low Reftectance
Low Refhetatlee
Disparity
150
Figure Reconstruction results of the proposed method in a challenging dynamic HDR scene.
Methods
EPE 1
Bad 0.5
Bad 1.0
Bad 2.0
Bad 3.0
Bad 5.0
D1 1
FastACV
0.7112
0.1068
0.0600
0.0342
0.0243
0.0159
0.0154
Ours(FastACV)
0.4983
0.0753
0.0475
0,0261
0,0177
0.0113
0.0112
RAFT-Stereo
0.4136
0.1286
0.0507
0.0219
0,0136
0.0077
0.0077
Ours(RAFT-Stereo)
0.3248
0.0717
0.0324
0.0170
0.0117
0.0070
0.0069
Table 3: Quantitative evaluation of the universality of our confidence-driven stereo matching strategy
on the synthetic dataset. Ours(") indicates our strategy with * as the baseline.
iterative closest point (ICP) registration. Finally, the aligned GT point cloud is projected onto the image plane to generate the disparity maps used for evaluation. The qualitative and quantitative results are presented in Fig. [ Among traditional methods, SGBM-3 can recover coarse disparity but suffers from large mismatched regions. The monocular method GigaDepth performs even worse, yielding noisy and fragmented outputs, indicating inferior robustness compared to sterco-based methods. Deep learning-basee stereo networks generate more complete disparity maps. However, both FastACV and IGEV fail to reconstruct fine-scale structures. For instance, in the second row, neither method accurately recovers the legs and the left eye of the small skeleton figure, resulting missing or overly smoothed geometry. In contrast, our method successfully reconstructs these fine-grate fine-grained details, yielding complete and structurally consistent skeleton. Our method also demonstrates remarkable disparity inference and completion capabilities. As shown in the third row, our method manages reconstruct a black metallic object with extremely weak reflectance, where all other methods fail due poor speckle quality. Moreover, our method shows the capability reconstruct transparent objects, as shown in the fourth row, highlighting its strong generalization ability and robustness to nom-bambernian surfaces.
The quantitative results are consistent with the qualitative observations, confirming the superiority of methods. However, the improvements dataset. This is primarily due to the sparsity and incompleteness of the real-world ground-truth data, particularly in extreme HDR regions where even the GT fails to make accurate reconstruction.
less pronounced than those the synthetic
We further evaluate our method under dynamic HDR conditions. this end, we construct a dynamic HDR scene by placing three geometrically complex objects with diverse surface reflectance properties on a rotating platform with a controllable angular velocity. The captured dynamic scenes and their corresponding reconstructions high-quality reconstruction with high inter-frane ffte-framm consistency. Notably, for metallic surfaces whose reflected intensity varies drastically from highly saturated to extremely weak the platform rotates, our approach consistently produces accurate and stable reconstructions, clearly showcasing its robustness in dynamic HDR environments.
shown in Fig. 7 The results demonstrate that method achieves
4.6 Universality of Confidence-Driven Stereo Matching
We further validate the universality of our confidence-driven stereo matching strategy. Quantitative results are shown in Table 3 Specifically, we observe a reduction in EPE from 0.7112 to 0.4983 on FastACV, and from 0.4136 0.3248 on RAFT-Stereo, corresponding to performance gains of 29.9% and 21.5% respectively. Similar improvements are consistently observed across all Bad rates and

---

DI metric. It has been proved that incorporating our strategy can significantly improve the accuracy of stereo matching across models with different paradigms. See also the qualitative results in the supplement.
Conclusion and Discussion
Conclusion. In this work, we innovate the first framework for event-based HDR structured light. The proposed event-based HDR coding scheme encodes depth information using three speckle projections with variations in both intensity and content, enabling robust representation across challenging HDR scenes. Furthermore, the proposed vonfidence-driver stereo matching strategy fully leverages the multi-frame encoding by employing ECE and CPV suppress inter tramm interference and enhance the representation ability of the cost volume, leading to highly accurate disparity estimation. We validate the effectiveness of our method on both synthetic and real-world datasets. Our method significantly outperforms existing baselines and demonstrates robust reconstruction performance even under extreme HDR conditions.
Limitation. Our method does not explicitly address occlusions. There exist no speckles in occluded regions, leaving the disparity estimation these regions ill-posed. Due to the network's inherent disparity completion behavior, this often results in over- werrshe or expanded boundaries near the left edges in large-disparity scenes. A potential solution is to estimate a confidence score for the predicted disparity and filter out unreliable estimates based on this confidence, which we leave as future work.
Acknowledgement
This work was supported in part by the National Natural Science Foundation of China under Grants 6213 1003 and 62021001, and the Fundamental Research Funds for the Central Universities under Grant WK2100000059.
References
[1] Zhang. High-speed 3d shape measurement with structured light methods: and Lasers in Engineering, 106:119-131, 2018.
review. Optics
[2] Zhiwei Xiong, Yueyi Zhang, Feng Wu, and Wenjun Zeng. Computational depth sensing: Toward high-performance commodity depth cameras. IEEE Signal Processing Magazine, 34(3):55-68, 2017.
[3] Y. Zhang, Z. Xiong, Z. Yang, and F. Wu. Real-time scalable depth sensing with hybrid structured light illumination. IEEE Transactions Image Processing, 23(1):97-109, 2013.
[4] Pei Zhou, Yue Cheng, Jiangping Zhu, and Jialing Hu. High-dynamic-range 3-d shape measure- ment with adaptive speckle projection through segmentation-basee mapping. IEEE Transactions on Instrumentation and Measurement, 12: 2022.
[5] Ke Wu, Jie Tan, Hai Lun Xia, and Cheng Bao Liu. An exposure fusion-based structured light approach for the 3d measurement of a specular surface. IEEE Sensors Journal, 21(5):: 2314-6324 2020.
[6] Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew Davison, Jörg Conradt, Kostas Daniilidis, et al. Event- based vision: A survey. IEEE Trans. Pattern Anal. Mach. Intell., 44(1):154 1805488-0110 2020.
[7] N. Matsuda, O. Cossairt, and M. Gupta. Mc3d: Motion contrast 3d scanning. In 2015 IEEE International Conference on Computational Photography (ICCP), pages 1-10. IEEE, 2015.
[8] M. Muglikar, G. Gallego, and D. Scaramu7za. Esl: Event-based structured light. 202/ International Conference 3D Vision (3DV), pages 1165-1174, IEEE, 2021.
10

---

[9] Yuhui Li, Heng Jiang, Chen Xu, and Lilin Liu. Event-driven fringe projection structured light 3-d reconstruction based on lime-freueenyy analysis. IEEE Sensors Journal, 24(4):5097-5106, 2024.
[10] Blender Online Community. Blender 3d modelling and rendering package. https://www blender. org/ 2023. Version 3.x.
[11] Gangwei Xu, Xianqi Wang, Xiaohuan Ding, and Xin Yang. Iterative geometry encoding volume for stereo matching. Proceedings of the IEEE/CVF conference on computer vision and pattem recognition, pages
2023.
[12] X. Huang. Y. Zhang, and Xiong. High-speed structured light based 3d scanning using an event camera. Optics Express, 39(2)):5584- 35876,, 2021.
[13] Jiacheng Fu, Yueyi Zhang, Yue Li, Jiacheng Li, and Zhiwei Xiong. Fast 3d reconstruction via event-based structured light with spatio-temporal coding. Optics Express, (1(6))44588-4440 2023.
[14] Xingyu Lu, Lei Sun, Diyang Gu, and Kaiwei Wang- Sge: structured light system based on gray code with an event camera. Optics Express, 32126):46044446661 2024.
[15] Jiacheng Fu, Yue Li, Wenming Weng, Xin Dong, Yueyi Zhang, and Zhiwei Xiong. Boosting event-based structured light via cyclic spatial coding. Optics Letters, (0(20))6365-6368 2025.
[16] Alex Kendall, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, and Adam Bry. End-to-end learning of geometry and context for deep stereo regression. In Proceedings of the IEEE international conference computer vision, pages 66-75, 2017.
[17] Jia-Ren Chang and Yong-Sheng Chen. Pyramid stereo matching network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5410-5418, 2018.
[18) Xiaoyang Guo, Kai Yang, Wukui Yang, Xiaogang Wang, and Hongsheng Li. Group-wise correlation stereo network. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 3273-3282, 2019.
[19] Gangwei Xu, Yun Wang, Junda Cheng, Jinhui Tang, and Xin Yang. Accurate and efficient stereo matching via attention concatenation volume. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(4):2461-247) 2023.
[20] Lahav Lipson, Zachary Teed, and Jia Deng. Raft-stereo: Multilevel recurrent field transforms for stereo matching. In 2021 International Conference on 3D Vision (3DV), pages 218-227. IEEE, 2021.
[21] Jiankun Li, Peisen Wang, Pengfei Xiong, Tao Cai, Ziwei Yan, Lei Yang, Jiangyu Liu, Haoqiang Fan, and Shuaicheng Liu. Practical stereo matching via cascaded recurrent network with adaptive correlation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16263-16272, 2022.
[22] Q. Tang, C. Liu, Z. Cai, H. Zhao, X. Liu, and X. Peng. An improved spatiotemporal correlation method for high-accuracy random speckle 3d reconstruction. Optics and Lasers in Engineering, 110:54-62, 2018
[23] Tong Wu, Jiarui Zhang. Xiao Fu, Yuxin Wang. Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, et al. Omniobject3d: Locge-vocy 3d object dataset for realistic perception, reconstruction and generation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 803-814, 2023.
[24] Prophesee. Event Camera Evaluation Kit 4, 2024. URL https://wwk. https://www.rropeese ai/ event-camera-eev Accessed: Sep. 12, 2024.
[25] Texas Instruments. DLP6600 Digital Micromirror Device (DMD), 2023. URI,tttps //www mouser wtt.://///w/w/cacee/t//ee/e htt.:/aassaannttttttssssssitatae
Accessed: Sep. 12, 2024.

---

[26] Photoneo. Motioncam-33 3d
http:///www....totoooooooo//o// wttp://oo..oooooooucutcuccoa motioncam 3d-m fo-m-ppi -m-plus/ 2024. Accessed: 2025-05-13.
m+.
[27] Gary Bradski. Openev: Open source computer vision library. https://opency. org/ 2000. Version 4.x.
[28] H. Hirschmuller. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence,
2007.
[29] Gernot Riegler, Viyi Liao, Simon Donne, Vladlen Koltun, and Andreas Geiger. Connecting the dots: Learning representations for active monocular depth estimation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7624-7633, 2019.
[30] Simon Schreiberhuber, Jean-Baptiss Baptiste Weibel, Timothy Patten, and Markus Vineze. Gigadepth: Learning depth from structured light with branching neural networks. European Conference on Computer Vision, pages 214-229 Springer, 2022.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See the limitation section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
No theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [No]
Justification:
We will public them the manuscript is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cppppi.ccppulli//uuuia wwwiiicgguie//eeeeeeeeeic CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: See tables.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
15

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http::://///is/.eeeeeeeee httpar/ss...ppuulo/c///cccccttaa
Answer:
[Yes]
Justification:
We follow the ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is societal impact of the work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
16

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswipteeeee whworee.wthhhe....c.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
17

---

Answer:
[NA]
Justification:
Will be released once accepted.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/codd/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
18

---

The core method development in this research does not involve LLMs as any important components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non ntanstandard components.
Please refer to our LLM policy hattps://neurips... 220 for what should or should not be described.
19


Reviews:
[{"review_id": "6buaIgJ5I1", "reviewer": "Reviewer_Qbn3", "summary": "The paper proposes an event-based structured light imaging using 3 pairs of speckle structured light projected images with adjusting background and foreground intensity.\nEach image pairs not only varying light intensity, but also Speckle contents, so it is not simple HDR preprocessing.\nTo integrate the information, they suggest confidence map-based feature fusion.", "questions": "What is the frame rate of the entire system? Is it possible to capture dynamic objects like other event-based methods? If not, what are the advantages over SL methods using a general camera?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 4, "strengths_and_weaknesses": "Strength\n\nThe paper suggests a new problem setting which is HDR event-based SR. It can help to acquire varying reflectance objects simultaneously.\n\nThe approach is not naïve such as HDR preprocessing in the image acquisition step. Speckle contents vary, so it enhances accuracy. Feature level fusion and confidence maps make it able to reconstruct varying speckle contents and also improve performance.\n\nWeakness\n\nThe paper does not compare with the other event-based methods. Section 2.1 points out the problems of existing event-based methods, but the paper does not compare them at all, and the comparison targets are all general camera-based stereo methods with their event images. This comparison is a kind of an ablation study to select a basis stereo method (in this case, IGEV), not a comparison.\n\n\nMultiple acquisition for HDR may not be in line with the main purpose of using event-based methods. The advantage of using event cameras is that they can obtain higher frame rates and can capture dynamic objects, but this advantage may be diluted when using 3 pairs of images. The paper does not mention the acquisition of dynamic objects or the frame rate of the system.", "was_revised": true, "final_justification": "My concerns have been resolved, and my final score go up to weak accept.", "rebuttal": "## Response to Reviewer Qbn3\n\nThank you for your thoughtful review and constructive feedback. We sincerely appreciate the time and effort you have dedicated to evaluating our work. We are also encouraged by your recognition of both our problem setting and the proposed approach. The main concerns are addressed below.\n\n**Weakness 1**: The paper does not compare with the other event-based methods. \n\n**Reply**: Thank you for the valuable feedback. Kindly note that, we **did** compare with the existing work of event-based structured light. Specifically, BM and SGBM are our reimplementation of *Huang et al.* [1], a seminal work in the domain of event-based structured light. To improve the quality of random speckle in HDR scenes, we adopt our imaging strategy written in Lines 108-111. Moreover, to ensure fairness, we implemented three-frame extension versions (BM-3 and SGBM-3) for comparison.\n\nAdditional event-based structured light methods were not included, since existing approaches were not specially designed for HDR 3D reconstruction and thus will perform poorly under HDR conditions. This has been proved in Fig. 1. To further support this point and emphasize the strength of our approach, we implement a state-of-the-art Graycode-based method [2]. The quantitative results are shown below:\n\n**Table 1: Quantitative comparison with Graycode under HDR scenes**\n| Scene    |  Method   |  EPE   | Bad 0.5 | Bad 1.0 | Bad 2.0 | Bad 3.0 | Bad 5.0 |   D1   |\n|:--------:|:---------:|:------:|:-------:|:-------:|:-------:|:-------:|:-------:|:------:|\n| Scene 1  | Graycode  | 43.6115 | 0.7982  | 0.6407  | 0.4766  | 0.4424  | 0.4315  | 0.4182 |\n|          |   Ours    | 0.8449  | 0.2727  | 0.0781  | 0.0417  | 0.0379  | 0.0286  | 0.0249 |\n| Scene 2  | Graycode  | 38.7694 | 0.7380  | 0.5921  | 0.4387  | 0.4068  | 0.3952  | 0.3810 |\n|          |   Ours    | 0.7490  | 0.2498  | 0.0732  | 0.0383  | 0.0330  | 0.0276  | 0.0229 |\n\nAs demonstrated, [2], which is designed for general-scene reconstruction, totally fails in HDR regions. Our proposed HDR 3D reconstruction framework (only 3 frames) significantly outperforms the [2] (requiring 11 frames) in terms of both accuracy and completeness. We will add qualitative and quantitative results in the supplementary material for completeness.\n\n**Weakness 2 & Q1 & Q2**: The advantage of high speed of event cameras may be diluted when using 3 pairs of images. The paper does not mention the acquisition of dynamic objects or the frame rate of the system.\n\n**Reply**: Projecting multiple frames for depth acquisition in event-based structured light is a common practice [2,3], as the depth encoding itself is inherently multi-frame in nature. In our system, we project each set of patterns at 200Hz. By employing a time-overlapping strategy [2], our system achieves a depth sensing rate of 200 Hz. Even without the time-overlapping strategy, our system achieves a depth sensing rate of 66.6 Hz, surpassing representative event-based structured light methods [3] (50 Hz), [4] (60 Hz), and [5] (60 Hz). This rate fully supports reconstruction of dynamic scenes.\n\nIn our work, we leverage both the high-speed and high dynamic range (HDR) characteristics of the event camera, enabling dynamic HDR 3D reconstruction. We additionally supplement experiments on dynamic scenes to demonstrate our method’s capability in dynamic HDR scenes. Due to the visualization limitation of rebuttal, we will present qualitative results in the revised manuscript.\n\n\n\n[1] X. Huang, Y. Zhang, and Z. Xiong. *High-speed structured light based 3D scanning using an event camera*. Optics Express, 29(22):35864–35876, 2021.\n\n[2] X. Lu, L. Sun, D. Gu, et al. *SGE: Structured light system based on Gray code with an event camera*. Optics Express, 32(26):46044–46061, 2024.\n\n[3] Y. Li, H. Jiang, C. Xu, et al. *Event-driven fringe projection structured light 3D reconstruction based on time–frequency analysis*. IEEE Sensors Journal, 24(4):5097–5106, 2024.\n\n[4] N. Matsuda, O. Cossairt, and M. Gupta. *MC3D: Motion contrast 3D scanning*. In *Proc. IEEE Int. Conf. Computational Photography (ICCP)*, pages 1–10, 2015.\n\n[5] M. Muglikar, G. Gallego, and D. Scaramuzza. *ESL: Event-based structured light*. In *Proc. Int. Conf. 3D Vision (3DV)*, pages 1165–1174, 2021."}, {"review_id": "MTqNqRRwez", "reviewer": "Reviewer_VZsb", "summary": "This paper proposes the first HDR 3D measurement framework tailored for event-based SL systems and constructs the first datasets for this task. Experimental results show that the proposed method has achieved state-of-the-art performance in both synthetic and real-world datasets.", "questions": "Question:\n1. Supplement experiments. Refer to weakness 4 and 5.\n2. Provide theoretical explanation or relevant experiments. Refer to weakness 1 and 2.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. This paper presents a novel depth estimation framework based on event cameras and structured light, which improves the performance of current depth estimation in HDR scenes.\n2. The ablation experiments and visualization results of confidence maps demonstrate that the method can distinguish between overexposed and underexposed regions.\n3. This paper constructs a synthetic dataset and a real-world dataset, enriching the dataset resources in the field of depth estimation.\n\nWeaknesses:\n1. As in Table I, the \"multi-contrast HDR coding\" plays the most significant role and delivers the greatest performance improvement. However, no theoretical basis and further analyses are provided.\n- Why are three pairs of binary speckle patterns adopted? Will increasing the number of patterns enhance the effect?\n- In the paper, the projection intensities of the three pairs of patterns are set as [32, 55], [32, 200], and [0, 255]. Is there any experimental or theoretical basis for this?\n2. Why does the performance of structured light-based methods in Table 1 lag far behind that of stereo matching methods? It is hoped that the authors can provide further explanations.\n3. The proposed method does demonstrate remarkable performance in the newly established scenarios of this paper, yet these scenarios are specialized constructs tailored for the method. The authors should supplement experiments on a broader range of datasets, separately illustrating the performance of the proposed method in HDR scenarios and non-HDR scenarios.\n4. The paper lacks a complexity analysis of the proposed method.", "was_revised": true, "final_justification": "While acknowledging the pioneering nature of this work, the proposed method cannot be directly compared with existing approaches on existing datasets because the information they utilize is actually different. Consequently, the potential improvement claimed is difficult to definitively quantify against the existing broader state-of-the-art. Therefore, I will keep my original score (borderline accept) unchanged.", "rebuttal": "## Response to Reviewer VZsb\n\nThank you for your thoughtful review and constructive feedback. We appreciate the time and effort you have taken to evaluate our work. We are encouraged by your recognition of the novelty of our HDR 3D reconstruction framework and our contribution of datasets to the community. The main concerns are addressed below.\n\n**Q1**: As shown in Table I, the \"multi-contrast HDR coding\" plays the most significant role and delivers the greatest performance improvement. However, no theoretical basis and further analyses are provided.\n\n**Reply**: Kindly note that we have provided the theoretical basis in Lines 113–124. This significant improvement arises from two key aspects:\n1. The multiple contrast levels proposed in our multi-contrast HDR coding enable imaging across regions with varying reflectance, as shown in Fig. 2(a). \n2. The variation of the content can address the issue that a single speckle frame cannot provide a globally unique encoding, thus improving disparity estimation accuracy, as also pointed out in [1].  Moreover, in HDR scenes, projecting several mutually independent random-speckle frames is equivalent to taking multiple independent samples of the surface micro-facet normal distribution, which reduces the likelihood that all frames at a given pixel suffer from event clustering or absence. \n\n**Q2**: Why are three pairs of binary speckle patterns adopted? Will increasing the number of patterns enhance the effect?\n\n**Reply**: Thank you for your insightful question. Increasing the number of patterns N improves reconstruction accuracy but also affects efficiency. To strike a balance between the two in learning-based methods, we conducted comprehensive experiments to determine the optimal value of N. We tested reconstruction accuracy for N ranging from 1 to 7 on planes and spheres, which is a widely adopted approach for precision calibration. The results are shown below:\n\n**Table 1: Accuracy under Varying Number of Patterns (RMSE in mm)**\n\n| Num of Patterns \\(N\\) | Plane | Sphere |\n|:---------------------:|:-----:|:------:|\n| 1                     | 1.10  | 1.18   |\n| 2                     | 0.85  | 0.91   |\n| 3                     | 0.55  | 0.65   |\n| 4                     | 0.55  | 0.57   |\n| 5                     | 0.50  | 0.52   |\n| 6                     | 0.41  | 0.49   |\n| 7                     | 0.40  | 0.47   |\n\nThe results show that increasing N improves accuracy, but the improvement becomes marginal as N increases. When N=3, we achieve accuracy comparable to using more frames, i.e., N=3 is an elbow point that strikes a balance between accuracy and efficiency. We will add an explanation in the main text and include the experimental results in the supplementary material.\n\n**Q3**: In the paper, the projection intensities of the three pairs of patterns are set as [32, 55], [32, 200], and [0, 255]. Is there any experimental or theoretical basis for this?\n\n**Reply**: The projection intensities are selected based on empirical experiments conducted under various scenes. Specifically, we constructed scenes with very low reflectance for selecting the first set of parameters, typical diffuse scenes for the second set, and highly reflective scenes for the third set.\n\nThis set of parameters allows for the reconstruction of non-HDR to HDR scenes. Additionally, due to the diversity of the synthetic dataset, our method is also robust to parameter adjustments, demonstrating high flexibility.\n\n**Q4**: Why does the performance of structured light-based methods in Table 1 lag far behind that of stereo matching methods? \n\n**Reply**: Monocular structured light methods (CTD and GigaDepth) implicitly learn the projected pattern during training and directly regress the depth from the input. This enables faster inference; however, the approach is inherently vulnerable to noise and system perturbations, limiting its robustness in challenging HDR scenes.\n\nStereo methods benefit from the explicit use of projected patterns, which provide rich structured information as a reference. The pixel-wise search and matching performed directly on the projected frames lead to high accuracy and robustness.\n\n**Q5**: The authors should supplement experiments on a broader range of datasets, separately illustrating the performance of the proposed method in HDR scenarios and non-HDR scenarios.\n\n**Reply**: The HDR scenes evaluated in Fig. 6 already include non-HDR objects, such as the bookshelf in Fig. 6 (a) and the fan in Fig. 6 (c), which are nearly diffuse in reflectance.\n\nAs suggested, we have added both qualitative and quantitative results for purely non-HDR scenes. The quantitative result is shown below:\n\n\n**Table 2: Quantitative Comparison on Non-HDR Scenes**\n\n| Metric  | SGBM-3 | GigaDepth | FastACV | IGEV  | Ours  |\n|:-------:|:------:|:---------:|:-------:|:-----:|:-----:|\n| EPE     | 3.153  | 4.986     | 0.799   | 0.447 | 0.393 |\n| Bad 2.0 | 0.066  | 0.079     | 0.038   | 0.021 | 0.018 |\n\nAnd due to the visualization limitation of rebuttal, we will present qualitative results in the revised manuscript.\n\n\n**Q6**: The paper lacks a complexity analysis of the proposed method.\n\n**Reply**: We have additionally computed the inference time, and the results are shown below:\n\n**Table 3: Quantitative results on the synthetic dataset with inference time**\n\n|     **Methods**     |   **EPE**   | **Bad 2.0** | **Inference Time (s)** |\n|:-------------------:|:-----------:|:-----------:|:----------------------:|\n|        BM           |  31.4518    |   0.3156    |         0.0030         |\n|       BM_3          |  18.2216    |   0.2401    |         0.0120         |\n|       SGBM          |  13.3384    |   0.1421    |         0.0060         |\n|      SGBM_3         |   6.3327    |   0.0918    |         0.0210         |\n|        CTD          |  26.8375    |   0.2638    |         0.0190         |\n|     GigaDepth       |   8.6688    |   0.1135    |         0.0210         |\n|      FastACV        |   0.7112    |   0.0342    |         0.0670         |\n|   RAFT_Stereo       |   0.4136    |   0.0219    |         0.8490         |\n|        IGEV         |   0.3863    |   0.0177    |         0.4200         |\n|       **Ours**      | **0.2937**  | **0.0122**  |       **0.4080**       |\n\n\n[1] Q. Tang, C. Liu, Z. Cai, et al., \"An improved spatiotemporal correlation method for high-accuracy random speckle 3D reconstruction,\" *Optics and Lasers in Engineering*, vol. 110, pp. 54–62, 2018."}, {"review_id": "x1eBR75xZ8", "reviewer": "Reviewer_uctW", "summary": "The paper proposes a speckle coding for capturing high dynamic range (HDR) structured light (SL) data with an event camera, allowing for accurate disparity reconstruction of challenging surfaces such as low albedo, specular, and transmissive surfaces. For reconstruction, the paper improves on cost volume stereo matching using two custom modules: an energy guided confidence estimation which is then used to create a confidence propagation volume. The two modules aim to suppress over/under exposed pixels’ contributions to the final reconstruction. Additionally, the paper provides a synthetic dataset with an accompanying event-based SL simulator in Blender.", "questions": "- What is the inference-time compute?\n- What if an object moves within the scene?\n- Why speckle patterns and not lines of varying intensity?\n- What about glitter or objects with small holes (ie a similar scale to the speckle)? Fig 5 has the shiny corn, but the size of the corn kernels is relatively large.\n- Could including the negative events aid in the reconstruction, or are they redundant? \n- Is the pixel-wise difference of the speckled image and non-speckled image for the simulation actually a good approximation of what happens in the pixel?\n- In addition to the GT point cloud alignment, is the GT disparity map also masked in order to compare the GT with the reconstructed disparity in the real-world results?", "limitations": "Authors discussed limitations of the work, but did not discuss potential negative societal impacts even in the supplemental material. In fact, the paper checklist says, “there is societal impact of this work”. If that is true, then it begs the question: “why even research this”? Structured light is a widely used technology (3D scanners, Apple FaceID, etc), and event cameras are quickly growing in popularity due to their low latency, high temporal resolution, and HDR capabilities (among other advantages). Generating more accurate depth/disparity by combining these systems could lead to better art conservation/digitization, face recognition on darker skin, and better ground truth datasets – just to list a few downstream applications. At the same time, improving downstream applications such as those listed also has the potential to enable art theft (unauthorized use in training or reproduction), racial profiling, and training of weaponized AI.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Synthetic dataset and simulator provided\n- Real world AND synthetic data evaluation\n- Confidence-driven stereo matching also improves other baseline models (FastACV and RAFT-Stero), not only the used model. \n\nWeaknesses:\n- Results on real world data seem hard to compare quantitatively as the ground truths are very different from the reconstruction because of the shadows.", "was_revised": true, "final_justification": "The author's responses to my questions were satisfactory, hence I am keeping my score the same.", "rebuttal": "## Response to Reviewer uctW\n\nThank you for your thoughtful review and constructive feedback. We appreciate the time and effort you have dedicated to evaluating our work. We are encouraged by your recognition of our contribution of dataset and simulator, the comprehensive evaluations on both synthetic and real-world data, and the generalizability of our method. The main concerns are addressed below.\n\n**Weakness 1**: Results on real-world data seem hard to compare quantitatively as the ground truths are very different from the reconstruction because of the shadows.\n\n**Reply**: We use the Photoneo MotionCam 3D [1], a widely used industrial-grade high-precision HDR structured light scanner, to capture the ground truth. The evaluation is conducted only on regions with valid ground truth.\nThe shadowed regions are caused by: (1) Occlusions due to the large baseline between the camera and the projector, which is common in high-precision SL scanners, as a larger baseline typically leads to higher accuracy. (2) Extreme HDR regions that are difficult to reconstruct. It is worth emphasizing that our method is capable of reconstructing regions where even the HDR scanner fails, which highlights a key advantage of our approach.\n\n**Q1**:  What is the inference-time compute?\n\n**Reply**: \n\nWe add the inference time results in the table 1, as shown below:\n\n**Table 1: Quantitative results on the synthetic dataset with inference time**\n\n|     **Methods**     |   **EPE**   | **Bad 2.0** | **Inference Time (s)** |\n|:-------------------:|:-----------:|:-----------:|:----------------------:|\n|        BM           |  31.4518    |   0.3156    |         0.0030         |\n|       BM_3          |  18.2216    |   0.2401    |         0.0120         |\n|       SGBM          |  13.3384    |   0.1421    |         0.0060         |\n|      SGBM_3         |   6.3327    |   0.0918    |         0.0210         |\n|        CTD          |  26.8375    |   0.2638    |         0.0190         |\n|     GigaDepth       |   8.6688    |   0.1135    |         0.0210         |\n|      FastACV        |   0.7112    |   0.0342    |         0.0670         |\n|   RAFT_Stereo       |   0.4136    |   0.0219    |         0.8490         |\n|        IGEV         |   0.3863    |   0.0177    |         0.4200         |\n|       **Ours**      | **0.2937**  | **0.0122**  |       **0.4080**       |\n\n**Q2**: What if an object moves within the scene?\n\n**Reply**: The depth sensing rate of the system is 200 Hz, which means our system could produce 200 estimates of the scene depth in 1 second and is sufficient to support general dynamic scene reconstruction. We additionally conduct experiments on dynamic scenes, which demonstrate our method’s capability and advantage in reconstructing dynamic HDR scenes. Due to the visualization limitation of rebuttal, we will include the new results in the revised manuscript.\n\n**Q3**: Why speckle patterns and not lines of varying intensity?\n\n**Reply**: Event cameras can only perceive intensity changes and output binary data, and thus can NOT capture grayscale information in lines with varying intensity. Random speckle is a widely adopted depth encoding strategy with high encoding efficiency [2], which is a natural choice in our framework.\n\n**Q4**:  What about glitter or objects with small holes (i.e., a similar scale to the speckle)? Fig. 5 has the shiny corn, but the size of the corn kernels is relatively large.\n\n**Reply**: As indicated by the real-world results in Fig. 6, our method can reconstruct objects with sizes equal to or slightly smaller than the speckle size, but it is unable to recover pixel-level structures. As an extremely efficient encoding strategy, this limitation is inherently determined by the nature of speckle encoding [2]. \n\n**Q5**: Could including the negative events aid in the reconstruction, or are they redundant?\n\n**Reply**: It is possible. However, our multi-contrast coding is based on positive event triggering, which is also commonly adopted in many existing methods [3][4]. As a result, negative events are redundant in our setup. In implementation, we adjust the hardware configuration of the event camera to disable the generation of negative events, thereby avoiding interference and improving bandwidth efficiency.\n\n**Q6**: Is the pixel-wise difference of the speckled image and non-speckled image for the simulation actually a good approximation of what happens in the pixel?\n\n**Reply**: In implementation, we first compute the difference and then apply the logarithm, strictly following the imaging model of the event camera:\n\n$$\ne(x, y, t) =\n\\begin{cases}\n  1 & \\text{if } \\log \\left( \\frac{I(x, y, t)}{I(x, y, t - \\Delta t)} \\right) \\geq C_1 \\\\\\\\\n-1 & \\text{if } \\log \\left( \\frac{I(x, y, t)}{I(x, y, t - \\Delta t)} \\right) \\leq -C_2\n\\end{cases}.\n$$\n\nDuring simulation, we vary the value of C1 to further increase data diversity. Additionally, we use Blender’s Cycles renderer, which faithfully simulates real-world physical illumination, providing a realistic approximation of pixel-level behavior.\n\n**Q7**: In addition to the GT point cloud alignment, is the GT disparity map also masked in order to compare the GT with the reconstructed disparity in the real-world results?\n\n**Reply**: Yes, we perform the evaluation only on ground-truth regions with valid disparity values.\n\n[1] *Photoneo*. *MotionCam-3D M+*. [Online]. Available: [https://www.photoneo.com/products/motioncam-3d-m-plus/](https://www.photoneo.com/products/motioncam-3d-m-plus/). Accessed: May 13, 2025.\n\n[2] Z. Xiong, Y. Zhang, F. Wu, *et al.*, \"Computational depth sensing: Toward high-performance commodity depth cameras,\" *IEEE Signal Processing Magazine*, vol. 34, no. 3, pp. 55–68, 2017.\n\n[3] M. Muglikar, G. Gallego, and D. Scaramuzza, \"ESL: Event-based structured light,\" in *Proc. IEEE Int. Conf. on 3D Vision (3DV)*, 2021, pp. 1165–1174.\n\n[4] X. Lu, L. Sun, D. Gu, *et al.*, \"SGE: Structured light system based on Gray code with an event camera,\" *Optics Express*, vol. 32, no. 26, pp. 46044–46061, 2024."}, {"review_id": "HVuTkJ81Po", "reviewer": "Reviewer_9sYQ", "summary": "This paper proposes an event-based structured light (SL) framework for high dynamic range 3D measurement. The system introduces a multi-contrast speckle coding strategy and a confidence-driven stereo matching scheme using an energy-guided confidence estimation module and a confidence propagation volume. The authors also present a custom event-based SL simulator, a synthetic HDR dataset, and a small real-world dataset for evaluation. Experiments on both synthetic and real-world scenes demonstrate improvements over several traditional and deep stereo matching baselines.", "questions": "1. Could you clarify in more detail how your multi-contrast HDR coding strategy and confidence-driven stereo matching fundamentally differ from prior event-based structured light systems such as [7], [8], and [9]? Several elements appear conceptually incremental — what makes your approach substantially novel from an algorithmic standpoint?\n1. Given the explicit limitation regarding occlusions, did you consider incorporating an occlusion detection mechanism or confidence-based filtering of unreliable disparity regions? If so, what challenges prevented its inclusion in this work?\n1. Your real-world evaluation is based on 15 scenes. Could you describe the diversity of these scenes in terms of reflectance materials, geometry complexity, and lighting conditions? How representative are these of industrial or consumer HDR applications?\n1. Did you conduct any quantitative analysis on how your method performs at different levels of dynamic range (e.g., 60 dB, 80 dB, 120 dB)? It would be useful to understand if performance degrades gracefully or if there’s a threshold beyond which the method becomes unstable.", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n1. HDR 3D sensing under extreme lighting and reflectance variation is a well-motivated and practical problem.\n1. The combination of multi-contrast structured light coding with confidence-driven stereo matching is thoughtfully engineered.\n1. The paper provides comprehensive synthetic and real-world experiments, along with an ablation study.\n\n## Weaknesses\n1. While the paper claims to propose the first HDR 3D event-based structured light system with confidence-driven stereo matching, much of the multi-contrast coding concept and cost-volume stereo matching improvements seem to be incremental extensions of other methods: multi-contrast HDR coding seems to be a parameterization of contrast variation over a set of binary speckle projections, a technique seen in prior structured light HDR work (e.g. [7],[8],[9]). If the novelty comes primarily from combining these techniques in this specific application, it could be seen as integration rather than a fundamentally new algorithmic idea.\n1. Some sections, particularly Section 3.1 on multi-contrast HDR coding, are dense with notation and engineering descriptions without much intuitive explanation or diagrams to aid understanding.\n1. The authors openly acknowledge (Section 5) that their method does not address occlusions at all — a major shortcoming for a practical 3D reconstruction pipeline. Since occlusions are intrinsic in real-world HDR scenes, any method lacking a strategy to deal with them limits its robustness and applicability, potentially making the system unreliable for complex geometries.\n1. Although the authors collected real-world data, it is noted in Section 4.5 that the improvements are less pronounced than on synthetic data and the real-world dataset is relatively small (only 15 scenes).", "was_revised": true, "final_justification": "On clarification, the proposed multi-contrast coding and confidence-driven stereo matching are clearly motivated and technically distinct; the added experimental and qualitative evidence strengthens the case for their novelty and practical relevance. I also acknowledge the authors' plan to include additional visualizations and analysis in the final manuscript to improve clarity and accessibility of the paper. Overall, the paper seems to represent a meaningful advancement for robust HDR 3D reconstruction and thus I support its acceptance.", "rebuttal": "## Response to Reviewer 9sYQ ##\n\nThank you for your constructive feedback and for taking the time to evaluate our work. The main concerns are addressed below.\n\n**Weakness 1 & Q1**: Multi-contrast coding concept and cost-volume stereo matching improvements seem to be incremental extensions of [7][8][9].\n\n**Reply**: We respectfully disagree. Our work is the first event-based structured light work specifically designed for HDR 3D reconstruction, while [7][8][9] in the manuscript were developed for general-purpose event-based structured light applications. Therefore, [7][8][9] are fundamentally different from our proposed multi-contrast coding and confidence-driven stereo matching in both objective and technical approach. We detail the technical details below:\n\nRegarding multi-contrast coding, prior works such as [7], [8], and [9] rely on fixed-intensity scanning, which is prone to overexposure and underexposure in HDR scenes, limiting their effectiveness under such challenging conditions. In contrast, we propose a novel multi-contrast coding based on a range-partitioned sensing strategy to effectively image surfaces with varying reflectance. Besides, none of [7], [8], or [9] adopt speckle-based projection; therefore, their depth encoding principles and reconstruction pipelines differ fundamentally from ours.\n\nDue to the fundamentally different encoding strategy, both the objective and technical design of our reconstruction algorithm differ significantly from those in [7], [8], and [9]. In our work, we propose a confidence-driven stereo matching strategy based on our observation of inter-frame interference, which is caused by overexposed and underexposed regions in multi-frame fusion. To tackle this problem, we propose a universal confidence-driven stereo matching strategy. Specifically, we estimate a confidence map as the fusion weight for features via an energy-guided confidence estimation. Further, we propose the confidence propagation volume, an innovative cost volume that offers both effective suppression of inter-frame interference and strong representation capability. To the best of our knowledge, this algorithmic design has not been investigated in prior work on HDR 3D reconstruction or in [7][8][9].\n\nIn summary, our work goes beyond a simple integration of existing ideas and pioneers the first framework for event-based HDR structured light.\n\n**Weakness 2**: Section 3.1 on multi-contrast HDR coding is dense with notation and engineering descriptions without much intuitive explanation or diagrams to aid understanding.\n\n**Reply**: Thank you for the suggestion. We have created an illustrative figure to more clearly present the design of the multi-contrast coding. Due to the visualization limitation of rebuttal, we will include it in the revised manuscript. \n\n**Weakness 3**: The authors openly acknowledge (Section 5) that their method does not address occlusions at all — a major shortcoming for a practical 3D reconstruction pipeline. Method lacking a strategy to deal with them limits its robustness and applicability, potentially making the system unreliable for complex geometries.\n\n**Reply**: Occlusion detection in monocular structured light systems is an ill-posed problem. Specifically, occlusions and low-reflectance regions are fundamentally indistinguishable, as both result in the missing or absence of speckles in the captured frame. This ambiguity becomes more pronounced in HDR scenes with complex surface properties. Consequently, nearly all existing monocular structured light methods do not attempt to estimate occlusions [1][2][3], and none of the baseline methods used in our comparisons incorporate occlusion estimation. We plan to explore this challenge in future work under a multi-view setting.\n\n**Weakness 4**: Improvements of real-world data are less pronounced than on synthetic data and the real-world dataset is relatively small (only 15 scenes).\n\n**Reply**: It is important to note that the real-world dataset is not intended for training, but rather for testing, similar to the famous Middlebury stereo dataset. \n\nAs discussed in Lines 281-283, the improvements seem to be marginal due to the sparsity of the ground-truth data, which results from large occlusions and extreme HDR surfaces. Given the limited valid regions in the ground truth, the performance differences appear less significant. In particular, as indicated by the white box in Fig. 6, the ground truth in extreme HDR areas only provides sparse disparity measurements—precisely the regions where our method demonstrates clear advantages. Therefore, in terms of quantitative results, the performance of our method is likely underestimated.\n\n**Q2**: Did you consider incorporating an occlusion detection mechanism or confidence-based filtering of unreliable disparity regions? If so, what challenges prevented its inclusion in this work?\n\n**Reply**: We did consider this. We first discuss the issue of occlusions. As detailed in response to weakness 3, occlusion detection or confidence-based filtering in monocular structured light systems is an ill-posed problem. Specifically, occlusions and low-reflectance areas are essentially indistinguishable, as both lead to missing or absent speckle patterns in the captured frames. This ambiguity is further exacerbated in HDR scenes with complex surface characteristics. As a result, almost all existing monocular structured light approaches [1][2] avoid explicit occlusion estimation, and none of the baseline methods included in our comparisons address this issue. We plan to explore this challenge in future work under a multi-view setting.\n\nOn the other hand, confidence-based filtering of unreliable disparity is a separate issue from occlusion estimation. This filtering process removes not only occluded regions but also areas with low-quality speckle responses and boundary artifacts. It functions as a post-processing step and is optional in practice. It is important to emphasize that the core contribution of this work lies in proposing a general HDR 3D reconstruction strategy that is compatible with arbitrary stereo networks. Therefore, whether to apply confidence-based filtering depends on the choice of stereo network, and users can flexibly adapt the configuration to suit their specific needs.\n\n\n**Q3**: Could you describe the diversity of these scenes in terms of reflectance materials, geometry complexity, and lighting conditions? How representative are these of industrial or consumer HDR applications?\n\n**Reply**: Our real-world evaluation includes 15 scenes selected to cover a wide range of HDR challenges commonly encountered in industrial and consumer applications. These scenes contain objects with diverse reflectance properties, such as common diffuse surfaces, highly reflective metals, low-reflectance matte materials, scattering surfaces, high-contrast textured regions, partially specular objects, and translucent materials. The objects also vary in geometric complexity, including smooth surfaces, regions with abrupt depth changes, and geometric structures ranging from coarse to fine size. The lighting conditions span natural indoor lighting, strong direct illumination, and completely dark environments, simulating realistic high dynamic range scenarios. This diversity ensures that our benchmark is representative of practical use cases such as quality inspection, dental modeling, and vehicle surface reconstruction under real-world HDR conditions.\n\n**Q4**: Did you conduct any quantitative analysis on how your method performs at different levels of dynamic range (e.g., 60 dB, 80 dB, 120 dB)? \n\n**Reply**: We supplement an experiment to evaluate performance under varying dynamic ranges. The experimental setup is as follows: we first construct a complex geometric scene and keep it fixed throughout the experiment. Then, we sequentially project binary speckle patterns with varying intensities and render the corresponding 32-bit HDR images. For each rendered image, we calculate the dynamic range by extracting its brightest and darkest regions. The overall dynamic range of the scene is then defined as the maximum value among the three projected frames. To vary the scene’s dynamic range, we modify the surface materials to adjust their reflectance properties. The reconstruction results under different dynamic range conditions are summarized as follows:\n\n**Table 1: Quantitative evaluation of reconstruction accuracy under varying dynamic ranges.**\n\n| Dynamic Range (dB) | 60    | 80    | 100   | 120   | 140   |\n|:------------------:|:-----:|:-----:|:-----:|:-----:|:-----:|\n| EPE                | 0.141 | 0.147 | 0.169 | 0.209 | 0.387 |\n| BAD 2.0            | 0.005 | 0.005 | 0.007 | 0.009 | 0.019 |\n\n\nThe results show that our method maintains high accuracy across a wide range of dynamic ranges (60–120 dB), with only a modest increase in error as the dynamic range increases. At 140 dB, both high-reflectance and low-reflectance regions experience speckle loss across all three frames, leading to reduced reconstruction accuracy in those areas. Nevertheless, the network’s disparity completion capability enables reliable inference from surrounding regions, resulting in reasonable disparity estimates. Overall, our method consistently delivers stable and accurate reconstruction even under extreme HDR conditions. We will include the results above and corresponding qualitative results in the revised manuscript.\n\n\n[1] Schreiberhuber S, Weibel J B, Patten T, et al. Gigadepth: Learning depth from structured light with branching neural networks[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 214-229.\n\n[2] Riegler G, Liao Y, Donne S, et al. Connecting the dots: Learning representations for active monocular depth estimation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 7624-7633."}]

Author rebuttal:

Meta-review: With two borderline accept and two accept ratings, the perception of this work is fairly much on the positive side. All reviews and rebuttals have been carefully considered, and this AC agrees that this work meets the bar of NeurIPS.
Decision: Accept (poster)