Title: Event-based HDR Structured Light
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Reconstruction

HDR Structured Light

Event Camera
Abstract: Event-based structured light (SL) systems have attracted increasing attention for their potential in high-performance 3D measurement. Despite the inherent HDR capability of event cameras, reflective and absorptive surfaces still cause event cluttering and absence, which produce overexposed and underexposed regions that degrade the reconstruction quality. In this work, we present the first HDR 3D measurement framework specifically designed for event-based SL systems. First, we introduce a multi-contrast HDR coding strategy that facilitates imaging of areas with different reflectance. Second, to alleviate inter-frame interference caused by overexposed and underexposed areas, we propose a universal confidence-driven stereo matching strategy. Specifically, we estimate a confidence map as the fusion weight for features via an energy-guided confidence estimation. Further, we propose the confidence propagation volume, an innovative cost volume that offers both effective suppression of inter-frame interference and strong representation capability. Third, we contribute an event-based SL simulator and propose the first event-based HDR SL dataset. We also collect a real-world benchmarking dataset with ground truth. We validate the effectiveness of our method with the proposed confidence-driven strategy on both synthetic and real-world datasets. Experimental results demonstrate that our proposed HDR framework enables accurate 3D measurement even under extreme conditions.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an event-based structured light (SL) framework for high dynamic range 3D measurement. The system introduces a multi-contrast speckle coding strategy and a confidence-driven stereo matching scheme using an energy-guided confidence estimation module and a confidence propagation volume. The authors also present a custom event-based SL simulator, a synthetic HDR dataset, and a small real-world dataset for evaluation. Experiments on both synthetic and real-world scenes demonstrate improvements over several traditional and deep stereo matching baselines.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. HDR 3D sensing under extreme lighting and reflectance variation is a well-motivated and practical problem.\n1. The combination of multi-contrast structured light coding with confidence-driven stereo matching is thoughtfully engineered.\n1. The paper provides comprehensive synthetic and real-world experiments, along with an ablation study.\n\n## Weaknesses\n1. While the paper claims to propose the first HDR 3D event-based structured light system with confidence-driven stereo matching, much of the multi-contrast coding concept and cost-volume stereo matching improvements seem to be incremental extensions of other methods: multi-contrast HDR coding seems to be a parameterization of contrast variation over a set of binary speckle projections, a technique seen in prior structured light HDR work (e.g. [7],[8],[9]). If the novelty comes primarily from combining these techniques in this specific application, it could be seen as integration rather than a fundamentally new algorithmic idea.\n1. Some sections, particularly Section 3.1 on multi-contrast HDR coding, are dense with notation and engineering descriptions without much intuitive explanation or diagrams to aid understanding.\n1. The authors openly acknowledge (Section 5) that their method does not address occlusions at all — a major shortcoming for a practical 3D reconstruction pipeline. Since occlusions are intrinsic in real-world HDR scenes, any method lacking a strategy to deal with them limits its robustness and applicability, potentially making the system unreliable for complex geometries.\n1. Although the authors collected real-world data, it is noted in Section 4.5 that the improvements are less pronounced than on synthetic data and the real-world dataset is relatively small (only 15 scenes).\n\nQUESTIONS:\n1. Could you clarify in more detail how your multi-contrast HDR coding strategy and confidence-driven stereo matching fundamentally differ from prior event-based structured light systems such as [7], [8], and [9]? Several elements appear conceptually incremental — what makes your approach substantially novel from an algorithmic standpoint?\n1. Given the explicit limitation regarding occlusions, did you consider incorporating an occlusion detection mechanism or confidence-based filtering of unreliable disparity regions? If so, what challenges prevented its inclusion in this work?\n1. Your real-world evaluation is based on 15 scenes. Could you describe the diversity of these scenes in terms of reflectance materials, geometry complexity, and lighting conditions? How representative are these of industrial or consumer HDR applications?\n1. Did you conduct any quantitative analysis on how your method performs at different levels of dynamic range (e.g., 60 dB, 80 dB, 120 dB)? It would be useful to understand if performance degrades gracefully or if there’s a threshold beyond which the method becomes unstable.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a speckle coding for capturing high dynamic range (HDR) structured light (SL) data with an event camera, allowing for accurate disparity reconstruction of challenging surfaces such as low albedo, specular, and transmissive surfaces. For reconstruction, the paper improves on cost volume stereo matching using two custom modules: an energy guided confidence estimation which is then used to create a confidence propagation volume. The two modules aim to suppress over/under exposed pixels’ contributions to the final reconstruction. Additionally, the paper provides a synthetic dataset with an accompanying event-based SL simulator in Blender.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Synthetic dataset and simulator provided\n- Real world AND synthetic data evaluation\n- Confidence-driven stereo matching also improves other baseline models (FastACV and RAFT-Stero), not only the used model. \n\nWeaknesses:\n- Results on real world data seem hard to compare quantitatively as the ground truths are very different from the reconstruction because of the shadows.\n\nQUESTIONS:\n- What is the inference-time compute?\n- What if an object moves within the scene?\n- Why speckle patterns and not lines of varying intensity?\n- What about glitter or objects with small holes (ie a similar scale to the speckle)? Fig 5 has the shiny corn, but the size of the corn kernels is relatively large.\n- Could including the negative events aid in the reconstruction, or are they redundant? \n- Is the pixel-wise difference of the speckled image and non-speckled image for the simulation actually a good approximation of what happens in the pixel?\n- In addition to the GT point cloud alignment, is the GT disparity map also masked in order to compare the GT with the reconstructed disparity in the real-world results?\n\nLIMITATIONS:\nAuthors discussed limitations of the work, but did not discuss potential negative societal impacts even in the supplemental material. In fact, the paper checklist says, “there is societal impact of this work”. If that is true, then it begs the question: “why even research this”? Structured light is a widely used technology (3D scanners, Apple FaceID, etc), and event cameras are quickly growing in popularity due to their low latency, high temporal resolution, and HDR capabilities (among other advantages). Generating more accurate depth/disparity by combining these systems could lead to better art conservation/digitization, face recognition on darker skin, and better ground truth datasets – just to list a few downstream applications. At the same time, improving downstream applications such as those listed also has the potential to enable art theft (unauthorized use in training or reproduction), racial profiling, and training of weaponized AI.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the first HDR 3D measurement framework tailored for event-based SL systems and constructs the first datasets for this task. Experimental results show that the proposed method has achieved state-of-the-art performance in both synthetic and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper presents a novel depth estimation framework based on event cameras and structured light, which improves the performance of current depth estimation in HDR scenes.\n2. The ablation experiments and visualization results of confidence maps demonstrate that the method can distinguish between overexposed and underexposed regions.\n3. This paper constructs a synthetic dataset and a real-world dataset, enriching the dataset resources in the field of depth estimation.\n\nWeaknesses:\n1. As in Table I, the \"multi-contrast HDR coding\" plays the most significant role and delivers the greatest performance improvement. However, no theoretical basis and further analyses are provided.\n- Why are three pairs of binary speckle patterns adopted? Will increasing the number of patterns enhance the effect?\n- In the paper, the projection intensities of the three pairs of patterns are set as [32, 55], [32, 200], and [0, 255]. Is there any experimental or theoretical basis for this?\n2. Why does the performance of structured light-based methods in Table 1 lag far behind that of stereo matching methods? It is hoped that the authors can provide further explanations.\n3. The proposed method does demonstrate remarkable performance in the newly established scenarios of this paper, yet these scenarios are specialized constructs tailored for the method. The authors should supplement experiments on a broader range of datasets, separately illustrating the performance of the proposed method in HDR scenarios and non-HDR scenarios.\n4. The paper lacks a complexity analysis of the proposed method.\n\nQUESTIONS:\nQuestion:\n1. Supplement experiments. Refer to weakness 4 and 5.\n2. Provide theoretical explanation or relevant experiments. Refer to weakness 1 and 2.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an event-based structured light imaging using 3 pairs of speckle structured light projected images with adjusting background and foreground intensity.\nEach image pairs not only varying light intensity, but also Speckle contents, so it is not simple HDR preprocessing.\nTo integrate the information, they suggest confidence map-based feature fusion.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n\nThe paper suggests a new problem setting which is HDR event-based SR. It can help to acquire varying reflectance objects simultaneously.\n\nThe approach is not naïve such as HDR preprocessing in the image acquisition step. Speckle contents vary, so it enhances accuracy. Feature level fusion and confidence maps make it able to reconstruct varying speckle contents and also improve performance.\n\nWeakness\n\nThe paper does not compare with the other event-based methods. Section 2.1 points out the problems of existing event-based methods, but the paper does not compare them at all, and the comparison targets are all general camera-based stereo methods with their event images. This comparison is a kind of an ablation study to select a basis stereo method (in this case, IGEV), not a comparison.\n\n\nMultiple acquisition for HDR may not be in line with the main purpose of using event-based methods. The advantage of using event cameras is that they can obtain higher frame rates and can capture dynamic objects, but this advantage may be diluted when using 3 pairs of images. The paper does not mention the acquisition of dynamic objects or the frame rate of the system.\n\nQUESTIONS:\nWhat is the frame rate of the entire system? Is it possible to capture dynamic objects like other event-based methods? If not, what are the advantages over SL methods using a general camera?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-03"}