Title: Transient Neural Radiance Fields for Lidar View Synthesis and 3D Reconstruction
Venue: NeurIPS 2023
Accepted: yes
Keywords: neural radiance fields

3D reconstruction

single-photon lidar

computational imaging
Abstract: Neural radiance fields (NeRFs) have become a ubiquitous tool for modeling scene appearance and geometry from multiview imagery. Recent work has also begun to explore how to use additional supervision from lidar or depth sensor measurements in the NeRF framework. However, previous lidar-supervised NeRFs focus on rendering conventional camera imagery and use lidar-derived point cloud data as auxiliary supervision; thus, they fail to incorporate the underlying image formation model of the lidar. Here, we propose a novel method for rendering transient NeRFs that take as input the raw, time-resolved photon count histograms measured by a single-photon lidar system, and we seek to render such histograms from novel views. Different from conventional NeRFs, the approach relies on a time-resolved version of the volume rendering equation to render the lidar measurements and capture transient light transport phenomena at picosecond timescales. We evaluate our method on a first-of-its-kind dataset of simulated and captured transient multiview scans from a prototype single-photon lidar. Overall, our work brings NeRFs to a new dimension of imaging at transient timescales, newly enabling rendering of transient imagery from novel views. Additionally, we show that our approach recovers improved geometry and conventional appearance compared to point cloud-based supervision when training on few input viewpoints. Transient NeRFs may be especially useful for applications which seek to simulate raw lidar measurements for downstream tasks in autonomous driving, robotics, and remote sensing.

Transient Neural Radiance Fields for Lidar View Synthesis and 3D Reconstruction

Abstract

Neural radiance fields (NeRFs) have become a ubiquitous tool for modeling scene appearance and geometry from multiview imagery. Recent work has also begun to explore how to use additional supervision from lidar or depth sensor measurements in the NeRF framework. However, previous lidar-supervised NeRFs focus on rendering conventional camera imagery and use lidar-derived point cloud data as auxiliary supervision; thus, they fail to incorporate the underlying image formation model of the lidar. Here, we propose a novel method for rendering transient NeRFs that take as input the raw, time-resolved photon count histograms measured by a single-photon lidar system, and we seek to render such histograms from novel views. Different from conventional NeRFs, the approach relies on a time-resolved version of the volume rendering equation to render the lidar measurements and capture transient light transport phenomena at picosecond timescales. We evaluate our method on a first-of-its-kind dataset of simulated and captured transient multiview scans from a prototype single-photon lidar. Overall, our work brings NeRFs to a new dimension of imaging at transient timescales, newly enabling rendering of transient imagery from novel views. Additionally, we show that our approach recovers improved geometry and conventional appearance compared to point cloudbased supervision when training on few input viewpoints. Transient NeRFs may be especially useful for applications which seek to simulate raw lidar measurements for downstream tasks in autonomous driving, robotics, and remote sensing.

1 Introduction

The ability to sense and reconstruct 3D appearance and geometry is critical to applications in vision, graphics, and beyond. Lidar sensors [1] are of particular interest for this task due to their high sensitivity to arriving photons and their extremely high temporal resolution; as such, they are being deployed in systems for 3D imaging in smart phone cameras [2], autonomous driving, and remote sensing [3]. Recent work has also begun to explore how additional supervision from lidar [4] or depth sensor measurements [5] can be incorporated into the NeRF framework to improve novel view synthesis and 3D reconstruction. Existing NeRF-based methods that use lidar [4] are limited to rendering conventional RGB images , and use lidar point clouds (i.e., pre-processed lidar measurements) as auxiliary supervision rather than rendering the raw data that lidar systems actually collect. Specifically, lidars capture transient images -time-resolved picosecond- or nanosecond-scale measurements of a pulse of light travelling to a scene point and back. We consider the problem of how to synthesize such transients from novel viewpoints. In particular, we seek a method that

multiview lidar scans

lidar novel views

Transient NeRF

single-photon lidar

scanning

mirrors

pulsed

laser

TCSPC

photon count

histogram

time

returning

laser pulse

y

t

y

x

t

depth

intensity

y

x

photon count

histograms

y

x

t

takes as input and renders transients in the form of time-resolved photon count histograms captured by a single-photon lidar system 1 [8]. Lidar view synthesis may be useful for applications that seek to simulate raw lidar measurements for downstream tasks, including autonomous driving, robotics, remote sensing, and virtual reality.

The acquisition and reconstruction of transient measurements has been studied across various different sensing modalities, including holography [9], photonic mixer devices [10, 11] streak cameras [12], and single-photon detectors (SPADs) [13, 14].

In the context of SPADs and single-photon lidar, a transient is measured by repeatedly illuminating a point with pulses of light and accumulating the individual photon arrival times into a time-resolved histogram. After capturing such histograms for each point in a scene, one can exploit their rich spatiotemporal structure for scene reconstruction [15, 16], to uncover statistical properties of captured photons [17, 18], and to reveal the temporal profile of the laser pulse used to probe the scene (knowledge of which can significantly improve depth resolution [19, 20]). These properties motivate transients as a representation and their synthesis from novel views. While existing methods have explored multiview lidar reconstruction [21-25], they exclusively use point cloud data, and do not tackle lidar view synthesis.

Recently, a number of NeRF-based methods for 3D scene modeling have also been proposed to incorporate point cloud data (e.g., from lidar or structure from motion) [26, 27] or information from time-of-flight sensors [5]. Again, these methods focus on synthesizing conventional RGB images or depth maps, while our approach synthesizes transient images. Another class of methods combines NeRFs with single-photon lidar data for non-line-of-sight imaging [28]; however, they focus on a very different inverse problem and scene parameterization [29], and do not aim to perform novel view synthesis of lidar data as we do.

Our approach, illustrated in Fig. 1, extends neural radiance fields to be compatible with a statistical model of time-resolved measurements captured by a single-photon lidar system. The method takes as input multiview scans from a single-photon lidar and, after training, enables rendering lidar measurements from novel views. Moreover, accurate depth maps or intensity images can also be rendered from the learned representation.

In summary, we make the following contributions.

We develop a novel time-resolved volumetric image formation model for single-photon lidar and introduce transient neural radiance fields for lidar view synthesis and 3D reconstruction.

We assemble a first-of-its-kind dataset of simulated and captured transient multiview scans, constructed using a prototype multiview single-photon lidar system.

We use the dataset to demonstrate new capabilities in transient view synthesis and state-of-the-art results on 3D reconstruction and appearance modeling from few (2-5) single-photon lidar scans of a scene.

2 Related work

Our work ties together threads from multiple areas of previous research, including methods for imaging with single-photon sensors, and NeRF-based pipelines that leverage 3D information to improve reconstruction quality. Our implementation also builds on recent frameworks that improve the computational efficiency of NeRF training [30, 31].

Active single-photon imaging. Single-photon sensors output precise timestamps corresponding to the arrival times of individual detected photons. The most common type of single-photon sensor is the single-photon avalanche diode (SPAD). SPADs are based on the widely-available CMOS technology [32] (which we consider in this work), but other technologies such as superconducting nanowire single-photon detectors [33] and silicon photomultipliers [34], offer different tradeoffs in terms of sensitivity, temporal resolution, and cost.

In active imaging scenarios, pulsed light sources are paired with single-photon sensors to estimate the depth or reflectance of a scene by applying computational algorithms to the captured photon timestamps [19, 35, 36]. The extreme temporal resolution of these sensors also enables direct capture of interactions of light with a scene at picosecond timescales [37, 38], and by modeling and inverting the time-resolved scattering of light, single-photon sensors can be used to see around corners [28, 3941] or through scattering media [42-44]. The extreme sensitivity of single-photon sensors has made them an attractive technology for autonomous navigation [8], and accurate depth acquisition from mobile phones [2].

Our approach differs significantly from all the previous work in that we investigate, for the first time, the problem of lidar view synthesis and multi-view 3D reconstruction in the single-photon lidar regime. We introduce the framework of transient NeRFs for this task and jointly optimize a representation of scene geometry and appearance that is consistent with captured photon timestamps across all input views.

Finally, we note that while commercial lidars based on SPADs or avalanche photodiodes capture photon count histograms or time-resolved intensity, they typically pre-process these raw measurements to point cloud format before output. Hence, the raw lidar data that we use may not have been widely available to previous methods; we have publicly released our dataset of multiview photon count histograms on the project webpage to help alleviate this challenge.

3D-informed neural radiance fields. A number of recent techniques for multiview reconstruction using NeRFs leverage additional geometric information (sparse point clouds from lidar [4] or structure from motion [26, 27]) to improve the reconstruction quality or reduce the number of required input viewpoints. Similar benefits can be obtained by combining volume rendering with monocular depth estimators [45], or using data from time-of-flight sensors [5]. Other methods investigate the problem of view synthesis from few input images but leverage appearance priors instead of explicit depth supervision [46-48]. In contrast to the proposed approach, all of these methods focus on reconstructing images or depth maps rather than transients.

3 Transient Neural Radiance Fields

We describe a mathematical model for transient measurements captured using single-photon lidar and propose a time-resolved volume rendering formulation compatible with neural radiance fields.

3.1 Image Formation Model

Consider that a laser pulse illuminates a point in a scene that is imaged onto a sensor at position p ∈ R 2 (see Fig. 2). Assume light from the laser pulse propagates to a surface and back to p along

time-resolved volume rendering

optimization

ray casting

rendered

captured

laser

pulse

the same path described by a ray r ( t ) , where t indicates propagation time. The forward path along the ray is given as r ( t ) = x ( p ) + tc ω ( p ) , where x ( p ) ∈ R 3 is the ray origin, ω ( p ) ∈ S 2 is the ray direction which maps to p , and c is the speed of light. Now, let f ( t ) denote the temporal impulse response of the lidar (including the temporal profile of the laser pulse and the sensor jitter), and let α ( p ) incorporate reflectance and radiometric falloff factors [17] of the illuminated point at distance z ( p ) from x ( p ) . Then, assuming single-bounce light transport, the photon arrival rate incident on the sensor from the laser pulse is given as

where T n and P i,j indicate the temporal and spatial discretization intervals for the time bin n and pixel i, j , respectively. The term 2 z/c gives the time delay for light to propagate to a point at distance z and back.

Now, we can describe the measured transient, or the number of photon detections captured by a SPAD [17], as

where N indicates the number of laser pulses per pixel, η ∈ (0 , 1) is the detection efficiency of the sensor, and B is the total number of background (non-laser pulse) detections. Background detections in turn depend on A , the average ambient photon rate at pixel [ i, j ] , and D , number of false detections produced by the sensor per laser pulse period, also known as the dark count rate. When the number of detected photons is far fewer than the number of laser pulses, SPAD measurements can be modeled according to a Poisson process [17] where the arrival rate function varies across space and time. This model is appropriate for our measurements, which have relatively low flux ( < 5% detection probability per emitted laser pulse) [19]. The resulting measurements ˜ τ [ i, j, n ] represent a noisy histogram of photon counts collected at pixel [ i, j ] at time bin n .

3.2 Time-Resolved Volume Rendering

Using the measurements ˜ τ , we wish to optimize a representation of the appearance and geometry of the scene. To this end, we propose a time-resolved version of the volume rendering equation used in NeRF [49, 50]. Specifically, we model clean (i.e., without Poisson noise) time-resolved histograms τ [ i, j, n ] as (writing r ( t ) as r for brevity)

We denote by c the radiance of light scattered at a point r ( t ) in the direction ω , and σ represents the volume density or the differential probability of ray termination at r ( t ) . Finally, T ( t ) is the

transmittance from a distance t 0 along the ray to t , and this term is squared to reflect the twoway propagation of light [5]. We additionally explicitly account for the inverse-square falloff of intensity, through the term ( tc ) -2 . The definite integrals are evaluated over the extent of time bin n , T n = [ t n -1 , t n ] , and over p within the area of pixel [ i, j ] as in Equation 1. Note that in practice, we calculate Equation 1 using the discretization scheme of Max [51] used by Mildenhall et al. [49].

Finally, to account for the temporal spread of the laser pulse and sensor jitter, we convolve the estimated transient with the calibrated impulse response of the lidar system f to obtain

Without this step, the volumetric model of Equation 3 does not match the raw data from the lidar system and tends to produce thick clouds of density around surfaces to compensate for this mismatch.

3.3 Reconstruction

To reconstruct transient NeRFs, we use lidar measurements ˜ τ ( k ) [ i, j, n ] of a scene captured from 0 ≤ k ≤ K -1 different viewpoints. We parameterize transient NeRF using a neural network F consisting of a hash grid of features and a multi-layer perceptron decoder [30]. The network takes as input a coordinate and viewing direction, and outputs radiance and density, F ( r ( t ) , ω ) = c , σ . We use these outputs to render transients (see Fig. 2). The model is optimized to minimize the difference between the rendered transient and measured photon count histograms. We also introduce a modified loss function to account for the high dynamic range of lidar measurements, and we propose a space carving regularization penalty to help mitigate convergence to local minima in the optimization.

HDR-Informed loss function. Measurements from a single photon sensor can have a dynamic range that spans multiple orders of magnitude, with each pixel recording from zero to thousands of photons. We find that applying two exponential functions to the radiance preactivations (1) enforces non-negativity and (2) improves the dynamic range of the network output. Thus, we have c = exp(exp( ˆ c )) -1 , where the network preactivations are given by ˆ c . Following Muller et al. [30] the network also predicts density in log space.

After time-resolved volume rendering using Equation 3, we apply a loss function in log space to prevent the brightest regions from dominating the loss [52]. The loss function is given as

where the sum is over all images, pixels, and time bins.

Space carving regularization. We find that using the above loss function alone results in spurious patches of density in front of dark surfaces in a scene. Here, the network can predict bright values on the surface itself, but darkens the corresponding values of τ f by placing additional spurious density values along the ray. Since the network can predict the radiance of the density to be zero at these points, the predicted transients τ f can be entirely consistent with the measured transients ˜ τ , but with incorrect geometry. To address this, we introduce a space carving regularization

This function penalizes any density along a ray at locations where the corresponding measured transient values are less than the expected background level B . This effectively forces space to be empty (i.e., zero density) at regions where the measurements do not indicate the presence of a surface.

The complete loss function used for training is then given as

where λ sc controls the strength of the space carving regularization.

3.4 Implementation Details

Our implementation is based on the NerfAcc [31] version of Instant-NGP [30], which we extend to incorporate our time-resolved volume rendering equation. In particular, we extend the framework to output time-resolved transient measurements, to account for the pixel footprint, and to estimate depth.

Pixel footprint. Captured photon count histograms exhibit multiple peaks where the finite beam width of the laser passes over depth discontinuities. To account for this phenomenon we use a truncated Gaussian distribution to model the spatial footprint of the laser spot and SPAD sensor projected onto the scene. Specifically, we sample rays in the range of 4 standard deviations of the pixel center, weighting their contribution to the rendered transient by the corresponding Gaussian probability density function value (after rendering a histogram). We set the standard deviation of the Gaussian to 0.15 pixels for the simulated dataset and 0.10 pixels for the captured dataset.

Depth. To estimate depth we find the distance along each ray that results in the maximum probability of ray termination: argmax t T ( t ) σ ( t ) . Note that when integrating over the pixel footprint at occlusion boundaries, multiple local extrema can occur, and so taking the highest peak results in a single depth estimate without floating pixel artifacts.

Network optimization. We optimize the network using the Adam optimizer [53], a learning rate of 1 × 10 -3 and a multi-step learning rate decay of γ = 0 . 33 applied at 100K, 150K, and 180K iterations. We set the batch size to 512 pixels and optimize the simulated results until they appear to converge, or for 250K iterations for the simulated results and 150K iterations for the captured results. For the weighting of the space carving loss, we use λ sc = 10 -3 for the simulated dataset and increase this to λ sc = 10 -2 for captured data, which benefits from additional regularization. We train the network on a single NVIDIA A40 GPU. Also note that simulated results use RGB histograms, such that c ∈ R 3 . In the captured data c ∈ R because we illuminate the scene with monochromatic laser light.

4 Multiview Lidar Dataset

We introduce a first-of-its-kind dataset consisting of simulated and captured multiview data from a single-photon lidar. A description of the full set of simulated and captured scenes is included in the supplemental, and the dataset and simulation code are publicly available on the project webpage.

Simulated dataset We create the simulated dataset using a time-resolved version of Mitsuba 2 [54] which we modify for efficient rendering of lidar measurements. The dataset consists of one scene from Vicini et al. [55] and four scenes made available by artists on Blendswap ( https://blendswap.com/ ), which we ported from Blender to our Mitsuba 2 renderer. The training views are set consistent with the capture setup of our hardware prototype (described below) such that the camera viewpoint is rotated around the scene at a fixed distance and elevation angle, resulting in 8 synthetic lidar scans used for training. We evaluate on rendered measurements from six viewpoints sampled from the NeRF Blender test set [49]. The renders are used to simulate SPAD measurements by applying the noise model described in Equation 2 and setting the mean number of photon counts to 2850 per occupied pixel and the background counts to 0.001 per bin, which we set to approximate our experimentally captured data.

scanning

mirrors

pulsed

laser

beamsplitter

laser path

Hardware prototype. To create the captured dataset, we built a hardware prototype (Fig. 3) consisting of a pulsed laser operating at 532 nm that emits 35 ps pulses of light at a repetition rate of 10 MHz. The output power of the laser is lowered to < 1 mW to keep the flux low enough

Ground Truth

Instant-NGP

chair

hotdog

lego

DS-NeRF

Urban NeRF

Urban NeRF-M

Proposed

rendered

transients

2 views

5 views

3 views

4.1

4.5

3.4

3.8

3.2

3.6

3.5

3.9

t (ns)

3.4

3.8

3.4

3.8

y

x

y

x

t

L

(roughly 150 , 000 counts per second on average) to prevent pileup, which is a non-linear effect that distorts the SPAD measurements [56]. The laser shares an optical path with a single-pixel SPAD through a beamsplitter, and a set of 2D scanning mirrors is used to raster scan the scene at a resolution of 512 × 512 scanpoints. A time-correlated single-photon counter is used to record the photon timestamps with a total system resolution of approximately 70 ps.

Captured dataset. We capture multiview lidar scans of six scenes by placing objects on a rotation stage in front of the scanning single-photon lidar and capturing 20 different views in increments of 18 degrees of rotation. For each lidar scan we accumulate photons during a 20 minute exposure time to minimize noise in the transient measurements. We bin the photon counts into histograms with 1500 bins and bin widths of 8 ps (all raw timestamp data will also be made available with the dataset). We set aside 10 views sampled in 36 degree increments for testing and we use 8 of the remaining views for training. Prior to input into the network for training, we normalize the measurement values by the maximum photon count observed across all views.

Calibration. We calibrate the camera intrinsics of the system using a raxel model [57] with corners detected from two scans of checkerboard translated in a direction parallel to the surface normal. This model calibrates the direction of each ray individually, which is necessary because the 2D scanning mirrors deviate from the standard perspective projection model [58]. Extrinsics are calibrated by placing a checkerboard on a rotation stage and solving for the axis and center of rotation that best align the 3D positions of the checkerboard corners, where the 3D points are found using the calibrated ray model along with the time of flight from the lidar (see supplemental). Overall, accurate calibration is an important and non-trivial task because multiview lidar scans provide two distinct geometric constraints (i.e. stereo disparity and time of flight) that must be consistent for scene reconstruction.

5 Results

We evaluate our method on the simulated and captured datasets and use transient neural radiance fields to render intensity, depth, and time-resolved lidar measurements from novel views.

Baselines. The intensity and depth rendered from our method are compared to four other baseline methods that combine point cloud-based losses with neural radiance fields. For fairer comparison and to speed up training and inference times, we implement the baselines by incorporating their loss terms into the recently introduced frameworks of NerfAcc [31] and Instant-NGP [30] adopted by our method. We train the following baselines using intensity images (i.e., the photon count histograms integrated over time) along with point clouds obtained from the photon count histograms using a log-matched filter, which is the constrained maximum likelihood depth estimate [59].

Instant-NGP [30] is used to illustrate performance without additional depth supervision.

Depth-Supervised NeRF (DS-NeRF) [26] incorporates an additional loss term to ensure that the expected ray termination distance in volume rendering aligns with the point cloud points.

Urban NeRF [4] incorporates the ray-termination loss of DS-NeRF while also adding space carving losses to penalize density along rays before and after the intersection with a point cloud point.

Urban NeRF with masking (Urban NeRF-M) modifies Urban NeRF to incorporate an oracle object mask and extends the space carving loss to unmasked regions, providing stronger geometry regularization (additional details in supplement).

Prior to input into the network, we normalize the images and apply a gamma correction, which improves network fitting to the high dynamic range data. Finally, after training with each method, we estimate an associated depth map using the expected ray termination depth at each pixel, which is the same metric used in the loss functions of the aforementioned baselines.

5.1 Simulated Results

The method is compared to the baselines in simulation across five scenes: chair , ficus , lego , hot dog , and statue . In Fig. 4, we show RGB images rendered from novel views using the baselines and our proposed method after training on two, three, and five views. More extensive sets of results on all scenes are included in the supplemental. We find that views rendered from transient neural radiance fields have fewer artifacts and spurious patches of density, as the explicit supervision from the photon count histograms avoids the ill-posedness of the conventional multiview reconstruction problem.

Additional quantitative results are included in Table 1, averaged across all simulated scenes. For the evaluation of rendered RGB images, we normalize and gamma-correct the output of the proposed method and the ground truth in the same fashion as the baseline methods. Transient NeRF recovers novel views with significantly higher peak signal-to-noise ratio and better performance on the learned perceptual image patch similarity (LPIPS) metric [60] compared to baselines. Transient measurements provide explicit supervision of the unoccupied spaces in the scene, leading to fewer floating artifacts, and cleaner novel views.

The depth maps inferred from Transient NeRF are also significantly more accurate than baselines (see Fig. 5). One key advantage here is that we avoid supervision on point clouds obtained by potentially

Ground Truth

Urban NeRF

Urban NeRF-M

Proposed

Instant-NGP

DS-NeRF

noisy (and thus view-inconsistent) per-pixel estimates of depth. By training on the raw photon count histograms, the scene's geometry is allowed to converge to the shape that best explains all histograms across all views, resulting in much higher geometric accuracy.

Ground Truth

Instant-NGP

cinema

food

baskets

DS-NeRF

Urban NeRF

Urban NeRF-M

Proposed

rendered

transients

t (ns)

2 views

5 views

3 views

7.1

7.5

7.1

7.5

7.1

7.5

6.8

7.2

6.5

6.9

6.7

7.1

y

x

t

y

x

y

x

t

5.2 Captured Results

In Fig. 6 we show rendered novel views of intensity images from our method and baselines trained on captured data with two, three, and five views. Results are shown on the cinema , food , and baskets scenes (additional results in the supplemental). The proposed approach results in fewer artifacts and the rendered intensity images are more faithful to reference intensity images captured from the novel viewpoint. Quantitative comparisons of our method to baselines on captured data are shown in Table 2; note that we do not have access to ground truth depth for captured data and instead use depth from a log-matched filter on the ground truth transient. We find that the method outperforms the baselines in terms of PSNR and LPIPS of intensity images rendered from novel views. While performance on captured data does not improve as much as observed on simulated data with increasing numbers of viewpoints, we attribute this to small imperfections ( ≈ 1 mm) in the alignment of the lidar scans after estimating the camera extrinsics.

Since DS-NeRF trains explicitly on depth without additional regularization, it is especially sensitive to camera perturbations and can be outperformed in some cases by Instant NGP which has no additional geometry constraints. Our approach appears somewhat less sensitive to these issues, perhaps because geometry regularization is done implicitly through a photometric loss on the lidar measurements.

We notice some degradation in depth accuracy relative to simulation, likely due to imperfections in the estimated extrinsics. Sub-mm registration of the lidar measure-

y

x

t

ground truth

rendered

7.0 ns

y

x

novel viewpoint

x-t

time slice

depth

7.4 ns

ments would likely improve results, but achieving such precise registration is non-trivial and beyond the scope of our current work. Finally, in Fig. 7 we compare captured measurements to rendered transients and depth rendered for the boots scene trained on 2 viewpoints. We recover the time-resolved light propagation from a novel view, shown in x -y slices of the rendered transients over time. The depth map recovered from the novel view appears qualitatively similar to the ground truth (estimated from captured measurements using a log-matched filter [17]). We show additional 3D reconstruction results in the supplemental.

6 Discussion

Our work brings NeRF to a new dimension of imaging at transient timescales, offering new opportunities for view synthesis and 3D reconstruction from multiview lidar. While our work is limited to modeling the direct reflection of laser light to perform lidar view synthesis, our dataset captures much richer light transport effects, including multiple bounces of light and surface reflectance properties that could open avenues for future work. In particular, the method and dataset may help enable techniques for intra-scene non-line-of-sight imaging [61-64] (i.e., recovering geometry around occluders within a scene), and recovery of the bidirectional reflectance distribution function via probing with lidar measurements [65].

Our method is also limited in that we do not explore more view synthesis in more general singlephoton imaging setups, such as when the lidar and SPAD are not coaxial; we hope to explore these configurations in future work. The proposed framework and the ability to render transient measurements from novel views may be especially relevant for realistic simulation for autonomous vehicle navigation, multiview remote sensing, and view synthesis of more general transient phenomena.

Acknowledgments and Disclosure of Funding

Kiriakos N. Kutulakos acknowledges the support of the Natural Sciences and Engineering Council of Canada (NSERC) under the RGPIN and RTI programs. David B. Lindell acknowledges the support of the NSERC RGPIN program. The authors also acknowledge Gordon Wetzstein and the Stanford Computational Imaging Lab for loaning the single-photon lidar equipment.

References

Peter Seitz and Albert JP Theuwissen. Single-Photon Imaging . Springer Science & Business Media, 2011.

Ilya Chugunov, Yuxuan Zhang, Zhihao Xia, Xuaner Zhang, Jiawen Chen, and Felix Heide. The implicit values of a good hand shake: Handheld multi-frame neural depth refinement. In Proc. CVPR , 2022.

Brent Schwarz. Mapping the world in 3d. Nat. Photonics , 4(7):429-430, 2010.

Konstantinos Rematas, Andrew Liu, Pratul P Srinivasan, Jonathan T Barron, Andrea Tagliasacchi, Thomas Funkhouser, and Vittorio Ferrari. Urban radiance fields. In Proc. CVPR , 2022.

Benjamin Attal, Eliot Laidlaw, Aaron Gokaslan, Changil Kim, Christian Richardt, James Tompkin, and Matthew O'Toole. Törf: Time-of-flight radiance fields for dynamic scene view synthesis. Proc. NeurIPS , 34, 2021.

Joe C Campbell, Stephane Demiguel, Feng Ma, Ariane Beck, Xiangyi Guo, Shuling Wang, Xiaoguang Zheng, Xiaowei Li, Jeffrey D Beck, Michael A Kinch, et al. Recent advances in avalanche photodiodes. IEEE J. Sel. Top. Quantum Electron. , 10(4):777-787, 2004.

Christopher V Poulton, Ami Yaacobi, David B Cole, Matthew J Byrd, Manan Raval, Diedrik Vermeulen, and Michael R Watts. Coherent solid-state lidar with silicon photonic optical phased arrays. Opt. Lett. , 42 (20):4091-4094, 2017.

Joshua Rapp, Julian Tachella, Yoann Altmann, Stephen McLaughlin, and Vivek K Goyal. Advances in single-photon lidar for autonomous vehicles: Working principles, challenges, and recent advances. IEEE Signal Process. Mag. , 37(4):62-71, 2020.

Nils Abramson. Light-in-flight recording by holography. Opt. Lett. , 3(4):121-123, 1978.

Felix Heide, Matthias B Hullin, James Gregson, and Wolfgang Heidrich. Low-budget transient imaging using photonic mixer devices. ACM Trans. Graph. , 32(4):1-10, 2013.

Achuta Kadambi, Refael Whyte, Ayush Bhandari, Lee Streeter, Christopher Barsi, Adrian Dorrington, and Ramesh Raskar. Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Trans. Graph. , 32(6):1-10, 2013.

Andreas Velten, Di Wu, Adrian Jarabo, Belen Masia, Christopher Barsi, Chinmaya Joshi, Everett Lawson, Moungi Bawendi, Diego Gutierrez, and Ramesh Raskar. Femto-photography: capturing and visualizing the propagation of light. ACM Trans. Graph. , 32(4):1-8, 2013.

Matthew O'Toole, Felix Heide, David B Lindell, Kai Zang, Steven Diamond, and Gordon Wetzstein. Reconstructing transient images from single-photon sensors. In Proc. CVPR , 2017.

Ahmed Kirmani, Tyler Hutchison, James Davis, and Ramesh Raskar. Looking around the corner using transient imaging. In Proc. ICCV , 2009.

David B Lindell, Matthew O'Toole, and Gordon Wetzstein. Single-photon 3d imaging with deep sensor fusion. ACM Trans. Graph. , 37(4):113-1, 2018.

Jiayong Peng, Zhiwei Xiong, Xin Huang, Zheng-Ping Li, Dong Liu, and Feihu Xu. Photon-efficient 3d imaging with a non-local neural network. In Proc. ECCV . Springer, 2020.

Joshua Rapp and Vivek K Goyal. A few photons among many: Unmixing signal and noise for photonefficient active imaging. IEEE Trans. Comput. Imaging , 3(3):445-459, 2017.

Joshua Rapp, Yanting Ma, Robin MA Dawson, and Vivek K Goyal. High-flux single-photon lidar. Optica , 8(1):30-39, 2021.

Felix Heide, Steven Diamond, David B Lindell, and Gordon Wetzstein. Sub-picosecond photon-efficient 3d imaging using single-photon sensors. Sci. Rep. , 8(1):17726, 2018.

Joshua Rapp, Robin MA Dawson, and Vivek K Goyal. Dithered depth imaging. Opt. Express , 28(23): 35143-35157, 2020.

Wolfgang Hess, Damon Kohler, Holger Rapp, and Daniel Andor. Real-time loop closure in 2d lidar slam. In Proc. ICRA , 2016.

Qin Zou, Qin Sun, Long Chen, Bu Nie, and Qingquan Li. A comparative analysis of lidar slam-based indoor navigation for autonomous vehicles. IEEE Trans. Intell. Transp. Syst. , 23(7):6907-6921, 2021.

Zimo Li, Prakruti C Gogia, and Michael Kaess. Dense surface reconstruction from monocular vision and lidar. In Proc. ICRA , 2019.

Matthew J Leotta, Chengjiang Long, Bastien Jacquet, Matthieu Zins, Dan Lipsa, Jie Shan, Bo Xu, Zhixin Li, Xu Zhang, Shih-Fu Chang, et al. Urban semantic 3d reconstruction from multiview satellite imagery. In Proc. CVPR Workshops , 2019.

Stefan Lionar, Lukas Schmid, Cesar Cadena, Roland Siegwart, and Andrei Cramariuc. Neuralblox: Real-time neural representation fusion for robust volumetric mapping. In Proc. 3DV , 2021.

Kangle Deng, Andrew Liu, Jun-Yan Zhu, and Deva Ramanan. Depth-supervised NeRF: Fewer views and faster training for free. In Proc. CVPR , 2022.

Barbara Roessle, Jonathan T Barron, Ben Mildenhall, Pratul P Srinivasan, and Matthias Nießner. Dense depth priors for neural radiance fields from sparse input views. In Proc. CVPR , 2022.

Daniele Faccio, Andreas Velten, and Gordon Wetzstein. Non-line-of-sight imaging. Nat. Rev. Phys. , 2(6): 318-327, 2020.

Siyuan Shen, Zi Wang, Ping Liu, Zhengqing Pan, Ruiqian Li, Tian Gao, Shiying Li, and Jingyi Yu. Non-line-of-sight imaging via neural transient fields. IEEE Trans. Pattern Anal. Mach. Intell. , 43(7): 2257-2268, 2021.

Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph. (SIGGRAPH) , 41(4):1-15, 2022.

Ruilong Li, Matthew Tancik, and Angjoo Kanazawa. Nerfacc: A general nerf acceleration toolbox. arXiv preprint arXiv:2210.04847 , 2022.

Franco Zappa, Simone Tisa, Alberto Tosi, and Sergio Cova. Principles and features of single-photon avalanche diode arrays. Sensors and Actuators A: Physical , 140(1):103-112, 2007.

Chandra M Natarajan, Michael G Tanner, and Robert H Hadfield. Superconducting nanowire single-photon detectors: physics and applications. Superconductor Sci. Technol. , 25(6):063001, 2012.

P Buzhan, B Dolgoshein, L Filatov, A Ilyin, V Kantzerov, V Kaplin, A Karakash, F Kayumov, S Klemin, E Popova, et al. Silicon photomultiplier and its possible applications. Nucl. Instrum. Methods. Phys. Res. A , 504(1-3):48-52, 2003.

Dongeek Shin, Feihu Xu, Dheera Venkatraman, Rudi Lussana, Federica Villa, Franco Zappa, Vivek K Goyal, Franco NC Wong, and Jeffrey H Shapiro. Photon-efficient imaging with a single-photon camera. Nat. Commun. , 7(1):12046, 2016.

Julián Tachella, Yoann Altmann, Nicolas Mellado, Aongus McCarthy, Rachael Tobin, Gerald S Buller, Jean-Yves Tourneret, and Stephen McLaughlin. Real-time 3d reconstruction from single-photon lidar data using plug-and-play point cloud denoisers. Nat. Commun. , 10(1):4984, 2019.

Genevieve Gariepy, Nikola Krstaji´ c, Robert Henderson, Chunyong Li, Robert R Thomson, Gerald S Buller, Barmak Heshmat, Ramesh Raskar, Jonathan Leach, and Daniele Faccio. Single-photon sensitive light-in-fight imaging. Nat. Commun. , 6(1):6021, 2015.

David B Lindell, Matthew O'Toole, and Gordon Wetzstein. Towards transient imaging at interactive rates with single-photon detectors. In Proc. ICCP , 2018.

Joshua Rapp, Charles Saunders, Julián Tachella, John Murray-Bruce, Yoann Altmann, Jean-Yves Tourneret, Stephen McLaughlin, Robin MA Dawson, Franco NC Wong, and Vivek K Goyal. Seeing around corners with edge-resolved transient imaging. Nat. Commun. , 11(1):5929, 2020.

Shumian Xin, Sotiris Nousias, Kiriakos N Kutulakos, Aswin C Sankaranarayanan, Srinivasa G Narasimhan, and Ioannis Gkioulekas. A theory of fermat paths for non-line-of-sight shape reconstruction. In Proc. CVPR , 2019.

Wenzheng Chen, Fangyin Wei, Kiriakos N Kutulakos, Szymon Rusinkiewicz, and Felix Heide. Learned feature embeddings for non-line-of-sight imaging and recognition. ACM Trans. Graph. (SIGGRAPH) , 39 (6):1-18, 2020.

David B Lindell and Gordon Wetzstein. Three-dimensional imaging through scattering media based on confocal diffuse tomography. Nat. Commun. , 11(1):4517, 2020.

Yongyi Zhao, Ankit Raghuram, Hyun K Kim, Andreas H Hielscher, Jacob T Robinson, and Ashok Veeraraghavan. High resolution, deep imaging using confocal time-of-flight diffuse optical tomography. IEEE Trans. Pattern Anal. Mach. Intell. , 43(7):2206-2219, 2021.

Rachael Tobin, Abderrahim Halimi, Aongus McCarthy, Martin Laurenzis, Frank Christnacher, and Gerald S Buller. Three-dimensional single-photon imaging through obscurants. Opt. Express , 27(4):4590-4611, 2019.

Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger. Monosdf: Exploring monocular geometric cues for neural implicit surface reconstruction. In Proc. NeurIPS , 2022.

Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan. Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. In Proc. CVPR , 2022.

Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In Proc. CVPR , 2021.

Samarth Sinha, Jason Y Zhang, Andrea Tagliasacchi, Igor Gilitschenski, and David B Lindell. Sparsepose: Sparse-view camera pose regression and refinement. In Proc. CVPR , 2022.

Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM , 65(1):99-106, 2021.

James T Kajiya and Brian P Von Herzen. Ray tracing volume densities. ACM SIGGRAPH , 18(3):165-174, 1984.

Nelson Max. Optical models for direct volume rendering. IEEE Trans. Vis. Comput. Graph. , 1(2):99-108, 1995.

Ben Mildenhall, Peter Hedman, Ricardo Martin-Brualla, Pratul P. Srinivasan, and Jonathan T. Barron. NeRF in the dark: High dynamic range view synthesis from noisy raw images. CVPR , 2022.

Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proc. ICLR , 2015.

Diego Royo, Jorge García, Adolfo Muñoz, and Adrian Jarabo. Non-line-of-sight transient rendering. Computers & Graphics , 107:84-92, 2022.

Delio Vicini, Sébastien Speierer, and Wenzel Jakob. Differentiable signed distance function rendering. ACM Trans. Graph. , 41(4):1-18, 2022.

Joshua Rapp, Yanting Ma, Robin MA Dawson, and Vivek K Goyal. Dead time compensation for high-flux ranging. IEEE Trans. Signal Process. , 67(13):3471-3486, 2019.

Michael D Grossberg and Shree K Nayar. The raxel imaging model and ray-based calibration. Int. J. Comput. Vis. , 61(2):119, 2005.

Peter Eisert, Konrad Polthier, and Joachim Hornegger. A mathematical model and calibration procedure for galvanometric laser scanning systems. In Vision, Modeling, and Visualization , pages 207-214, 2011.

Ahmed Kirmani, Dheera Venkatraman, Dongeek Shin, Andrea Colaço, Franco NC Wong, Jeffrey H Shapiro, and Vivek K Goyal. First-photon imaging. Science , 343(6166):58-61, 2014.

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. CVPR , 2018.

Andreas Velten, Thomas Willwacher, Otkrist Gupta, Ashok Veeraraghavan, Moungi G Bawendi, and Ramesh Raskar. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. , 3(1):745, 2012.

Matthew O'Toole, David B Lindell, and Gordon Wetzstein. Confocal non-line-of-sight imaging based on the light-cone transform. Nature , 555(7696):338-341, 2018.

David B Lindell, Gordon Wetzstein, and Matthew O'Toole. Wave-based non-line-of-sight imaging using fast fk migration. ACM Trans. Graph. , 38(4):1-13, 2019.

Xiaochun Liu, Ibón Guillén, Marco La Manna, Ji Hyun Nam, Syed Azer Reza, Toan Huu Le, Adrian Jarabo, Diego Gutierrez, and Andreas Velten. Non-line-of-sight imaging using phasor-field virtual wave optics. Nature , 572(7771):620-623, 2019.

Nikhil Naik, Shuang Zhao, Andreas Velten, Ramesh Raskar, and Kavita Bala. Single view reflectance capture using multiplexed scattering and time-of-flight imaging. ACM Trans. Graph. (SIGGRAPH Asia) , 30(6):1-10, 2011.

Peer reviews:
comments: SUMMARY:
The paper brings NeRFs to a new dimension of imaging at transient timescales enabling the rendering of transient imagery. It incorporates the underlying image formation model of the lidar and recovers improved geometry and conventional appearance when training on a few input viewpoints. The paper also proposes a first-of-its-kind dataset of simulated and captured transient multiview scans from a prototype single-photon lidar and tests the transient NeRF on the dataset.

STRENGTHS:
The paper brings NeRFs to a new dimension of imaging at transient timescales and proposes a first-of-its-kind dataset of simulated and captured transient multiview scans from a prototype single-photon lidar. To deal with the new data type, HDR-Informed loss function and Space carving regularization are proposed to further improve the performance. The experiments are sound compared with other NeRF-based methods. The full paper has a clear structure and a clear narrative.

WEAKNESSES:
It only tests in a dark environment because of the single-photon Lidar. It is hardly used in broad application scenarios.

QUESTIONS:
Do you do the ablation study of HDR-Informed loss function and Space carving regularization? Can the method work under natural light?

LIMITATIONS:
n/a
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-19

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper describes a variant of NeRF specially designed for the image formation model of a time-of-flight imaging sensor. In contrast to earlier works using ToF sensors in a neural radiance field setting, the method is not designed to merely take depth maps or point clouds from the ToF sensors as input. Instead, the formulation goes one level deeper and presents a NeRF and image formation model on the basis of the time resolved photon count histograms that are used as basis for the final depth measurement by the ToF camera. To this end, the authors propose a time-resolved version of the volume rendering equation for image formation, as well as adapted HDR aware loss functions for training the neural radiance fields on photon histograms. The approach is tested on simulated data as well as real data captured with a ToF prototype setup. The authors show that in particular in a sparse view setting, the new photon count based formulation has advantages over depth-supervised or image-based NeRFs. Overall, I think this is a strong paper showing very innovative ideas. I enjoyed reading it.\n\nSTRENGTHS:\nOverall, I think this is a strong paper. It introduces quite a few clever ideas on how to adapt the neural radiance fields concept to the image formation model of a time-of-flight camera. This ranges from adapting the image formation model to a time-resolved version, to introducing a HDR aware loss and a space-carving regularization tailored to the peculiarities of the imaging properties of the employed sensor. To my knowledge, this is the first paper approaching neural radiance fields in this way.  As the authors suggest in their discussion the formulation on the basis of time resolved photon histograms may enable additional improvements in 3D reconstruction, e.g. of complex shapes, material properties etc.\n\nWEAKNESSES:\nNot much to say here. One point that could have been discussed more is from what number of viewpoints on the traditional or depth-supervised approaches catch up. At the moment, advantages of the approach are mostly shown in the sparse view setting, which is fine and showcasing an advantage of the approach.\n\nQUESTIONS:\nsee the weaknesses section\n\nOne issue with time-of-flight sensors in general is their often non-trivial noise characteristics. Would explicitly incorporating the noise model into the formulation help here ?\n\nLIMITATIONS:\nThe discussion in the paper is adequate.", "TITLE": "Official Review", "RECOMMENDATION": 9, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nNeRF is a method that has recently become popular for view synthesis. It allows estimation of a 3D scene density and 5D radiance map using a few intensity images and their camera poses. These can then be used to render intensity and depth images from any novel 2D view.\n\nThis paper extends NeRF to allow estimation of scene density and radiance from “scene transients” instead of intensity/depth images. Previous methods have tried to incorporate depth information into NeRF in the form of point clouds, but the proposed method directly uses raw scene transients (histogram of photon counts) captured by a SPAD sensor. This method can then be used to render scene transients given a novel 2D view.\n\nThe authors' main contribution is to extend the volume rendering equation of NeRF to produce time resolved measurement instead of integrated over time. Additionally their training/rendering takes into account the shape of the laser pulse, spatial footprint of the laser spot etc. And lastly they add modifications to the NeRF objective to ensure their method works well for the high dynamic range SPAD data (predict radiances in the double log space, and apply loss in the log space). \nThe authors also apply a regularization (space carving) to encourage the density estimates to be sparse (similar to what other NeRF variants have done e.g urban NeRF).\n\nThe authors also capture a real world multiview LiDAR dataset with their hardware prototype. Using this dataset and another simulation based dataset, they provide quantitative and qualitative comparison of their method with other baselines for intensity and depth reconstruction from novel views. The results from simulation data show their method’s superior performance. The results from captured data also show that their method is superior, but by a smaller margin (which they attribute to alignment imperfections in the captured dataset).\n\nSTRENGTHS:\n1. This is a nice extension to the NeRF setup that allows directly leveraging the raw lidar measurements to learn a 5D representation instead of learning from intensity/depth images which won't contain all the information available in the raw measurements. The improvement in estimation accuracy is evident in the results/experiments shown.\n2. The paper is written very well, with good attention to details (e.g. modeling assumptions, experiment setup). Background work is explained well.\n3. Authors perform exhaustive experiments with baseline methods using both simulated and captured data, and results look impressive.\n\nWEAKNESSES:\nFrom a novelty of algorithm standpoint, the contributions of the paper are limited. From what I understood, It is a straightforward application of the NeRF framework to a new domain, i.e, the authors represent the scene density/radiance just like in NeRF, the only difference is that they modify the rendering logic to produce 3D histograms instead of 2D images. The image formation model of SPADs (which is used in the rendering equation) is also not novel.\n\nQUESTIONS:\n1. You mention that you model the spatial footprint of the laser/sensor spot when rendering the transients. Is that necessary? Does performance deteriorate if you just assume an ideal spot?\n2. I'm not convinced about the utility of rendering raw lidar measurements from novel viewpoints. Other than generating simulation data, are there any practical applications?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a novel-view synthesis method for active sensors (single-photon LiDAR sensors) based on the Neural Radiance Field formulation. To this end, the volume rendering formulation for active sensors is derived, properly taking into account the measurement formation process (two-way distance, intensity fall-off) of LiDAR sensors. Along with that, a new loss functions for empty-space carving is proposed. The proposed method is implemented based on the INGP NeRF model and evaluated on both synthetic dataset of transient multiview scans (one of the contributions of this work), as well as on real-world data that was captured using a prototype single-photon LiDAR system. In these experiments, the proposed method outperforms SoTA NeRF methods (also with depth supervision) across all metrics.\n\nSTRENGTHS:\n- Original and important problem formulation. Novel-view synthesis for active sensors is an under researched, but a very important problem.\n- A volume rendering formulation for active sensors that properly takes (at least some of the) the properties of the measurement formation process into account. These include, two-way path when computing the transmittance, intensity falloff, beam divergence. \n- Simulated dataset of transient multiview scans that will be made publicly available (along with the scripts used to generate it)\n- A dataset of real-world captures that were acquired using a prototype single-photon LiDAR which, if I understand correctly, will also be made publicly available\n\nWEAKNESSES:\nThere are two main weaknesses in my view: \n- **Clarity**: The clarity of the paper could in my opinion be improved, and the main confusion stems from including the RGB images from simulated data in figure 1 and using **c** to denote number of photons (?) in Figure 2. I might have misunderstood something, but if I am not wrong, the actual LiDAR system can only measure the photon count and depth (time-of-flight) and the volumetric rendering formulation should be tied to that. However, in the Equation 3 the radiance is a vector quantity (RGB color?)  same as in L150? The simulated data seems to be full RGB (what is the reason for this?), does that mean that for the simulated data the evaluation is actually similar to the original NeRF (PSNR and LPIPS computed over the RGB images?). \n\n - **Experimental evaluation**:  In my perception, the experimental evaluation seems somewhat biased. The proposed method is compared to the following baselines *INGP*, *DS-NeRF* and *UrbanNeRF* which are supervised through color/intensity supervision (and *DS-NeRF* and *UrbanNeRF* also using the depth). However, the depth supervision only constrains the integral over the density and the supervision signal might even be very noisy (log-matched filter in case of real world data are usually very noisy). On the other hand, the proposed method is supervised with the full histograms per each ray (before integration) which provide additional supervision and helps to constrain the empty space. This is especially important in the low-view setting that was for some reason selected in the evaluation (2-5 views on synthetic data?). I am wondering why the low-view setting was selected for simulated data? In my opinion this pronounces the bias as it emphasizes the differences in the supervision signal between the methods\n\nQUESTIONS:\n- What exactly does **c** denote in Equation 2 and L150?\n- I like that the formulation tries to follow the measurement formation process, but currently only the intensity fall-off and round-trip are considered. Would it make sense to add at least the effects of the incidence angle?\n- Similar to the comment above: why does simulated data simulate RGB and why is the evaluation considering low-view setting?\n- I also like that the beam divergence is modeled through sampling multiple rays (L180), but it would be good to clarify at which point in the volume rendering formulation the contributions are averaged?\n- The depth of the proposed methods is obtained as the argmax across the histogram, however for the baselines the integral along the ray is used. Does that yield better results as taking the argmax of the volume density profile?\n\nLIMITATIONS:\nThe limitations are discussed in the conclusion section, while broader societal impacts are not (but I also don't believe that there are any significant ones).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}