Title: Non-Line-of-Sight 3D Reconstruction with Radar
Venue: NeurIPS 2025
Accepted: yes
Keywords: Radar

NLOS Imaging

NLOS Scene Reconstruction

RF Sensing

3D Reconstruction
Abstract: Seeing hidden structures and objects around corners is critical for robots operating in complex, cluttered environments. Existing methods, however, are limited to detecting and tracking hidden objects rather than reconstructing the occluded full scene. We present HoloRadar, a practical system that reconstructs both line-of-sight (LOS) and non-line-of-sight (NLOS) 3D scenes using a single mmWave radar. HoloRadar uses a two-stage pipeline: the first stage generates high-resolution multi-return range images that capture both LOS and NLOS reflections, and the second stage reconstructs the physical scene by mapping mirrored observations to their true locations using a physics-guided architecture that models ray interactions. We deploy HoloRadar on a mobile robot and evaluate it across diverse real-world environments. Our evaluation results demonstrate accurate and robust reconstruction in both LOS and NLOS regions. Code, dataset and demo videos are available on the project website.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel around-the-corner imaging system based on millimeter-wave radar. By employing a series of deep learning models, the system processes radar heatmaps to reconstruct 3D scenes in non-line-of-sight (NLOS) scenarios. The authors collect a custom dataset involving 17 corner configurations and demonstrate that their system can accurately reconstruct scenes, even when objects are occluded from direct view.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The work is well-motivated, addressing the important problem of NLOS perception.\n- The overall system architecture is thoughtfully designed, and the inclusion of a ray tracing module contributes a novel component.\n- The dataset collected by the authors adds value and supports meaningful evaluation.\n\nWeaknesses:\n- The related work comparison is limited. References to vision transformers justify architectural choices but do not provide a relevant baseline for evaluating the core novelty of the method. It would be more appropriate to compare the method with other radar-based multi-bounce imaging approaches, which are more relevant to the core technical contribution.\n- The dataset consists of 17 corners, but additional details about the diversity of these scenes—such as the corridor widths and corner angles—would help assess the generalizability of the results under the proposed experimental setup.\n- The evaluation of hidden target imaging (i.e., human location detection without wall depth estimation) should be more prominent. While there is a brief discussion in the supplementary material, this evaluation is crucial and deserves a dedicated section in the main paper, ideally with clear metrics such as hit/miss rate.\n- The paper uses “NLOS” broadly, encompassing both behind-wall and around-the-corner scenarios. However, the method focuses specifically on around-the-corner imaging. This distinction should be emphasized early on, potentially starting from the title.\n\nQUESTIONS:\n1. Why were vision-based models (e.g., transformers) used as baselines instead of radar-focused multi-bounce imaging methods, which would provide a more meaningful comparison?\n2. Could the authors provide detailed information about the diversity of corner scenarios (e.g., widths, angles, distances) in their dataset?\n3. Why is the hidden imaging evaluation (human location only) not integrated into the main body of the paper? Would the authors consider moving it and adding clearer evaluation metrics like hit/miss rate?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
The authors present an approach to find non-line-of-sight (NLOS) 3D objects in indoor scenes by finding the multi-return range images and then reconstructing the physical scene by providing the reflected signals to their "true" locations. The HoloRadar architecture takes the RF heatmap as input and predcts the scene around corners. They show that they can recover hidden objects in the indoor environment.

STRENGTHS AND WEAKNESSES:
Strengths - The paper is well-written and well-explained. 
Weakness - 
The authors have missed a paper on exactly the same topic which deals with outdoor environments and predicts trajectory of various objects - 
Scheiner, N., Kraus, F., Wei, F., Phan, B., Mannan, F., Appenrodt, N., ... & Heide, F. (2020). Seeing around street corners: Non-line-of-sight detection and tracking in-the-wild using doppler radar. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2068-2077).
In view of this omission, the paper is weakened sufficiently since this deals with the indoor environment only and does not provide any solution for the outdoor clutter.

QUESTIONS:
Posted in weakness.

LIMITATIONS:
The limitations address the indoor aspect of the problem.
TITLE: Official Review
CLARITY: 3
IMPACT: 1
ORIGINALITY: 1
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo address the problem that the current system (e.g., LiDAR) and methods are difficult to apply in robots, the authors propose a mmWave radar-based method and system for NLOS 3D reconstruction. The proposed method integrates physics-based modeling and adopts multiple supervised terms to ensure the accuracy of the reconstructions. Experiments show that the proposed method has the capacity to effectively separate signals of each bounce and manage to reconstruct accurate 3D scenario information.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n* The paper is well constructed.\n* The authors adopt self-capturing data to train the network. Although lacking of comparable baseline methods, the authors provides resonable explanations and the results in quantitatively and qualitatively.\n\n### Weaknesses\n\n* The dataset examples provided in the paper seem to solely include indoor scenarios with limited spatial space, which suggests that the proposed method may have restricted practical applicability.\n* The paper assumes that the relay wall behaves as a specular reflector. However, light attenuation might also have a relation to occluder material, so the authors should analyze performance under more severe attenuation conditions.\n* Concerns regarding the loss function: Are the range images from the three decoders supervised jointly or separately?\n* The paper lacks a necessary ablation study. While sparse 3D convolutions reduce the computational cost, the inference time might arise due to the additional cost of computing the rulebook. An alternative method is to downsample features to trade off performance and efficiency. Further theoretical comparisons and explicit FLOPS and parameter counts would further clarify the proposed method’s advantages.\n\nQUESTIONS:\n* Include analyses and results for distinct scenario types. (e.g., outdoor scenarios).\n* Include analyses for different material types and light attenuation levels.\n* Provide a detailed description of the loss function.\n* Include the ablation study referenced in the Weaknesses.\n\nLIMITATIONS:\nSee the weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript presents HoloRadar, a method for 3D reconstruction for both Non-Line-of-Sight (NLOS) and Line-of-Sight (LOS) perceptions, using a rotating mmWave radar sensor. The work consists of a preprocessing step and a two-stage pipeline. \n\nFirst, data is collected in cycles by rotating a radar to form a cylindrical array covering the whole surroundings. For signals collected after each rotation, beamforming is applied to obtain a 3D heatmap, consisting of Elevations, Azimuths and Range bins. \n\nThe first stage of the proposed pipeline uses the 3D heat maps as input. In particular, for each Azimuth-Elevation pair, a deep model estimates a set of bounce distances, each corresponding to a reflection along a multi-return path. Through the use of an encoder/multi-decoder architecture, this multi-return imaging model predicts the range images for each bounce corresponding to the original 3D radar heatmap.\n\nThe second stage reconstructs a physical 3D scene through a ray-tracing based learned model an a deep refinement procedure. In particular, the predicted multi-return images are combined with normal estimation at each bounce, to reverse the mirroring effect of bounces. Lastly, residual geometric errors are accounted for by exploiting a voxel-based 3D UNet, which fuses predictions from different bounces and refines the reconstruction with a volumetric representations.\n\nHoloRadar has been deployed on a real-world mobile robot and it has been evaluated across multiple environments, depicting corners, showing promising and accurate results.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths.\n    1. The manuscript is well written and detailed, with all the various aspects being fully explained (from the various steps of the processing pipeline, to the description of the training procedure). This clarity allows for replicability (not considering the (un)availability of the dataset).\n    2. The presented pipeline is relatively simple, yet effective, both in terms of adopted learning models, training setup (e.g., loss functions) and mathematical formulation.\n    3. Both evaluation and ablation study show promising results in terms of metrics.\n\nWeaknesses.\n    1. The major concern about this work is related to the dataset gathered and used for the experiments. As far as it can be understood in the manuscript, all corners “captured” belong to the same building / complex, which could introduce a strong learning bias, with possible overfit. A similar discussion can be done on the number of heatmap scans gathered in total (around 1k for each corner), especially if there is no substantial difference in heatmaps associated with the same location (the paper does not give such information).\n    2. The manuscript is lacking a more in-depth analysis on the behavior of the proposed method based on the shape (e.g., T-shaped, oblique, rounded), size of corners (e.g., narrow, wide, medium), and number of perceivable entities (e.g., not just people but trash bins, furniture, or general obstacles).\n    3. There is no information in the manuscript about the processing requirements of the proposed method, i.e., if it is able to run in real-time (higher or equal frequency of data acquisition rate) and on which hardware.\n\nQUESTIONS:\n1. [SUGGESTION] According to what discussed in the weaknesses, a more detailed explanation of the experimental settings could refute the possibility of overfitting and non-adaptability to diverse scenes/structures.\n    2. [QUESTION] Why did the authors use only the F-Score to assess the reconstruction quality? One could have included Chamfer or Hausdorff distances, or Mesh quality (e.g., watertightness, manifoldness, self-intersections).\n    3. [SUGGESTION] The authors should include information about the processing time and requirements of the proposed method.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}