Title: GL-NeRF: Gauss-Laguerre Quadrature Enables Training-Free NeRF Acceleration
Venue: NeurIPS 2024
Accepted: yes
Keywords: Neural Radiance Fields

Real-Time NeRF

Gauss-Laguerre Quadrature

Neural Rendering
Abstract: Volume rendering in neural radiance fields is inherently time-consuming due to the large number of MLP calls on the points sampled per ray. Previous works would address this issue by introducing new neural networks or data structures. In this work, we propose GL-NeRF, a new perspective of computing volume rendering with the Gauss-Laguerre quadrature. GL-NeRF significantly reduces the number of MLP calls needed for volume rendering, introducing no additional data structures or neural networks. The simple formulation makes adopting GL-NeRF in any NeRF model possible. In the paper, we first justify the use of the Gauss-Laguerre quadrature and then demonstrate this plug-and-play attribute by implementing it in two different NeRF models. We show that with a minimal drop in performance, GL-NeRF can significantly reduce the number of MLP calls, showing the potential to speed up any NeRF model. Code can be found in project page https://silongyong.github.io/GL-NeRF_project_page/.

GL-NeRF: Gauss-Laguerre Quadrature Enables Training-Free NeRF Acceleration

Abstract

Volume rendering in neural radiance fields is inherently time-consuming due to the large number of MLP calls on the points sampled per ray. Previous works would address this issue by introducing new neural networks or data structures. In this work, we propose GL-NeRF, a new perspective of computing volume rendering with the Gauss-Laguerre quadrature. GL-NeRF significantly reduces the number of MLP calls needed for volume rendering, introducing no additional data structures or neural networks. The simple formulation makes adopting GLNeRF in any NeRF model possible. In the paper, we first justify the use of the Gauss-Laguerre quadrature and then demonstrate this plug-and-play attribute by implementing it in two different NeRF models. We show that with a minimal drop in performance, GL-NeRF can significantly reduce the number of MLP calls, showing the potential to speed up any NeRF model. Code can be found in project page https://silongyong.github.io/GL-NeRF_project_page/ .

1 Introduction

Neural Radiance Fields (NeRFs) [27] have shown promising results for synthesizing images from novel views. Plenty of works extend NeRF towards different aspects applicable in the real world (see related works for details). The core component for NeRF's success is volume rendering, which requires approximating an integral by densely sampling points along the ray and evaluating volume density and radiance using neural networks for them. In practice, a dense set of points is evaluated by expensive operations like neural network inferences for a single pixel, which could be redundant. Works have been done to reduce the time needed for rendering images, aiming at providing NeRF with a real-time rendering ability [9, 44, 24, 29, 8]. Despite the promising results shown by these works, they propose different approaches for achieving real-time rendering by introducing new networks, new data structures, etc . Therefore, each individual work requires training from scratch with a specific optimization goal. In this work, we propose a novel lightweight method that could be implemented in any existing NeRF-based models that require volume rendering without further training. In contrast to existing works, our approach introduces no additional representation or neural network and is training-free. We make minimal modifications to the computation of the volume rendering integral, making it rely on much fewer samples.

Our approach arises from revisiting the volume rendering integral, the key discovery is that with a simple change of variable, we can turn the integral into a pure exponentially weighted integral of color. This specific form has a Gauss quadrature ( i.e . the Gauss-Laguerre quadrature) which best approximates it mathematically. Naturally, we propose to use the Gauss-Laguerre quadrature to directly compute the volume rendering integral, which we call GL-NeRF (Gauss Laguerre-NeRF), leading to much lower computational cost for approximating the integral and therefore lower time and memory usage. Computing the points needed for the integral requires a dense evaluation of per-point density. However, the efficiency for this step can be improved using modern techniques

OMLPcalls

OMLPcalls

NeRFuniformsamplerendering

Gauss-Laguerre Quadrature rendering (ours)

like factorized tensors [6]. Benefiting from the guarantee of the highest precision Gauss quadrature provides, only a very small number of fixed points could provide comparable results to the heavy and redundant strategy NeRF adopts, leading to free speedup.

To verify the use of the Gauss-Laguerre quadrature, we conduct an empirical study on the landscape of color function. We also analyze the relationship between our approach and other techniques that aim to reduce the sample points for NeRF. We demonstrate the plug-and-play property of our method by directly incorporating it into vanilla NeRF and TensoRF models that are already trained on NeRF-Synthetic and LLFF datasets. Furthermore, we showcase the drop in time and memory usage as a direct outcome of reducing the computational cost.

GL-NeRF provides a different perspective for computing volume rendering and has the potential to be a direct plug-in for existing NeRF-based products. Specifically, our contributions are three-fold. We propose GL-NeRF, a brand new perspective for computing volume rendering with the Gauss-Laguerre quadrature with no additional component introduced. We analyze the validity of using the GaussLaguerre quadrature for volume rendering integral and the relationship between our approach and existing sample-efficient NeRFs. We demonstrate that GL-NeRF could be incorporated into any NeRF model without further training. To the best of our knowledge, GL-NeRF is the first method that could be used without training in any NeRF models thanks to the simple formulation. We showcase that GL-NeRF reduces computational cost, time and memory usage while keeping the rendering quality.

2 Related work

Volume rendering. Volume rendering has been widely used in computer graphics and vision applications [25, 43, 7]. It maps a 3D scene onto 2D images by a weighted integral over the color of the points along the corresponding rays with a function of opacity (volume density) as weight. In practice, the integral is approximated using a finite sum over sampled points along the ray as derived in [25]. Implicit scene models like NeRF [27], Plenoxels [8] and 3D gaussians [18] and most of their follow-up all adopt this technique as the render pipeline. Since randomly sampling in space for approximating the integral may bring unnecessary information ( i.e . sampling in empty space) that may cost extra computation, plenty of works aim to address that by introducing different techniques for better approximation of the component needed for volume rendering integral ( i.e . volume density, radiance) [40, 44, 29, 23, 21, 2, 36]. PL-NeRF [40] proposes to use piecewise linear function for approximating the volume density throughout the space, leading to fewer points needed for the 'fine' stage sampling proposed by [27]. AutoInt and DIVeR [23, 44] introduce a neural network for approximating the integral of volume density instead of using Monte-Carlo sampling. DONeRF [29] reduces the sampled point needed for computing the integral by introducing a depth oracle neural network that predicts the surface position of the underlying scene and samples the points near the

Algorithm 1 Gauss Laguerre Quadrature for Volume Rendering

surface, which contributes the most to the visual effect in the images. MCNeRF [13] proposes to use Monte-Carlo rendering and denoising to do sample efficient rendering, but it still introduces a denoiser network that requires per-scene training. Different from these previous works, Our work proposes to use the Gauss-Laguerre quadrature to directly improve the precision of the volume rendering integral itself, introduces no additional neural networks or data structures and remains in the simplest version, leading to its adaptability into any existing work that relies on volume rendering integral.

NeRFs. Neural Radiance Fields (NeRFs) have proved to be a powerful tool for novel view synthesis [27]. It uses a coordinate-based multi-layer perceptron (MLP) to represent the scene and render high-fidelity images from different views. The render is done by pixel-wise volumetric rendering [25] with density and color evaluated using the MLP on hundreds of sampled points along the ray. For modeling high-frequency information in the scene, NeRF uses positional encoding to map the input coordinates onto high-frequency bands. The success of NeRF has triggered an explosive emergence of follow-up works. There are plenty of works focusing on improving or extending the ability of NeRF towards different aspects. Aliasing along xy coordinates has been tackled [3], unbounded scenes [4, 47, 39, 34], dynamic scenes [32, 22, 30] and scenes with semantic information [41, 37, 49, 19] have been well explored and demonstrated the potential of implicit scene representation with NeRF. Nonetheless, NeRF requires plenty of time for training and rendering, blocking its way of being used for real-time rendering. The bottleneck of the computation time is the MLP used. There are two main branches of work for extending NeRF towards real-time rendering. The first branch introduces different data structure [46, 9, 14, 33, 8, 6] for scene representation. Another branch, in which our method falls, improves the sample efficiency of the model [20, 29, 31, 38] to accelerate NeRF rendering process. While previous works draw their intuition from the underlying physics perspective and thus need different formulations of the sampling strategy and different neural network architecture for predicting the surface position of the underlying scenes, we propose our method based on a mathematical observation while maintaining the overall pipeline. Benefiting from this, our work could be seamlessly incorporated into any existing NeRF-related works without further training. On the other hand, despite being derived from the mathematical perspective, our method still intuitively satisfies the underlying physical constraints.

3 Preliminaries

3.1 NeRF and volume rendering

NeRF [27] is a powerful implicit 3D scene model for novel view synthesis. At the core of its rendering ability is volume rendering. NeRF uses coordinate-based MLP to encode the scene, assigning volume density (opacity) and radiance (color) to spatial points. When used for synthesizing new views, it casts a ray r ( t ) = o + t d through the pixel to be rendered, sample points along the ray and compute volume density and radiance for these points. These values are then aggregated together using Eq.1 to give the color of the pixel.

t i represents the sampled position along the ray and δ i = t i +1 -t i is the distance between two nearby sampled points. NeRF uses an MLP to represent volume density σ and color c . The loss function for training NeRF is simply the square error between rendered pixel colors and the corresponding pixel colors over batch of rays R . Variants of NeRF like TensoRF[6] use different representations for volume density and color, but the process of volume rendering remains the same.

where

3.2 Gauss quadrature

An n -point Gauss quadrature [10] is a method for numerical integration that guarantees to yield exact results for integral of polynomials of degree 2 n -1 or less, which is the highest possible precision for approximating an integral by quadrature. Intuitively, consider approximating an integral using quadrature as in Eq. 5

where p ( x ) is a polynomial of degree 2 n -1 , w ( x ) is a weight function and I is the interval for computing the integral. We first give the definition of orthogonality of two polynomials p m ( x ) and p n ( x )

̸

where p m ( x ) is of degree m , p n ( x ) is of degree n and m = n . we can use long division for p ( x ) to obtain

where L n ( x ) is a polynomial of degree n that is orthogonal to any polynomials that have degree less than n ( i.e . n degree Legendre polynomial), q ( x ) and r ( x ) are both polynomials with degree less than n . Then

Since L n ( x ) is orthogonal to any polynomials with degree less than n , the first term on the right hand side of Eq. 8 should equal to 0 . Since it doesn't contribute to the computation of the integral, we may also neglect it when computing the quadrature. Therefore, we should choose x i that satisfies L n ( x i ) = 0 [16]. With this intuition bearing in mind, carefully choosing the weights w i for computing the quadrature would help us precisely calculate Eq. 5 because we have n points to compute the second term on the right hand side of Eq. 8, which is an integral of a polynomial of degree less than n .

In general, given a function f ( x ) , Gauss quadrature computes its integral on [ -1 , 1] using

where x i , i = 1 , 2 , . . . , n corresponds to a root of the orthogonal polynomials on [ -1 , 1] . This quadrature is called Gauss-Legendre quadrature since the orthogonal polynomials on [ -1 , 1] with a weight function g ( x ) = 1 are Legendre polynomials. An n -th degree Legendre polynomial takes the form [35, 15, 17]

and w i is computed using Eq. 11 as shown in [1].

Gauss-Laguerre quadrature is an extension of Gauss quadrature for approximating integrals following the form of

In this case, the weight function is g ( x ) = e -x , the integral interval is [0 , ∞ ) . x i corresponds to the root of Laguerre polynomials

a class of polynomials that are orthogonal over the interval [0 , ∞ ) with respect to the weight function g ( x ) = e -x . The weight for computing the quadrature is computed as

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.7

0.6

x

0.5

X

0.4

X

X

0.2

0.1

0.0

Ray

While the computation for x i and w i is complicated, in practice we can use a look up table to store corresponding x i and w i for a given n .

4 GL-NeRF

We developed our algorithm based on a simple observation of the integral for volume rendering. Eq. 1 is an approximation to the integral

4.1 Volume rendering and Gauss-Laguerre quadrature

where xxxxxx

Let we have

Since σ ( r ( t )) ≥ 0 , x ( t ) is a monotonically non-decreasing function of t , therefore, x has a unique correspondence with t on increasing intervals. With this observation, we can do a change of variables for Eq. 15 to get

Volume density

Volume density (o)

0.26

1.413.607.09

12.64

xxx

Ray samples (t)

since the integral between [0 , x ( t n )) and ( x ( t f ) , ∞ ) are zero. Thus,

As can be seen from Eq. 19, the integral for volume rendering is a weighted integral of c ( r ( x ) , d ) with the weight function to be g ( x ) = e -x . We can extend the integral interval from [ x ( t n ) , x ( t f )] to [0 , ∞ ) we have a pure exponentially weighted integral with respect to the color function, which is of the exact same form as required by the Gauss-Laguerre quadrature.

4.1.1 Gauss-Laguerre quadrature for volume rendering

As discussed in Sec. 3, the Gauss-Laguerre quadrature guarantees the highest algebraic precision when computing integral over polynomials. To perform the Gauss-Laguerre quadrature for volume rendering integral calculation, a natural question arises: is the color function a polynomial, or can it be approximated by a polynomial with a satisfactory error rate?

To answer this question, we first analytically give out a fundamental theorem, and then empirically approximate the color function with polynomials.

Theorem 4.1 (Stone-Weierstrass theorem) . Suppose f is a continuous real-valued function defined on the real interval [ a, b ] . For every ϵ > 0 , there exists a polynomial p such that for all x in [ a, b ] , we have | f ( x ) -p ( x ) | < ϵ .

Since the pixel color is contributed by points that have a density larger than a threshold ( i.e . regions near the surface), we can overlook the points in the empty space and only analyze the remaining part of the color function. In Fig. 2 we plot a representative of how the color function looks like. As can be seen from the figure, it has a major region with values greater than zero while the others remain zero. When approximating the non-zero region with a 7 -th degree polynomial, we have a relative error rate lower than 6 . 5% . While the relative error is not sufficiently low, we argue that we can increase the degree to better approximate it since it's cintinuous by nature. On the other hand, this specific landscape is fluctuated and for most of the cases, the error rate could be smaller than 1% . This suggests that Theorem 4.1 holds in our case, thus the Gauss-Laguerre quadrature can be used for computing the volume rendering integral.

4.1.2 Point selection in GL-NeRF

Different from NeRF's sampling strategy, the Gauss-Laguerre quadrature enables us to use a deterministic point selection strategy for the color samples. Recall Eq. 17 is the integral variable

40

1.00

0.30

35

0.95

0.25

Vanilla

Ours

℃℃

PSNR

M

0.90

LPIPS

S 0.20

U0.85

0.15

15

S

X

0.80

x

0.10

10

5

Vanilla

Ours

0.75

X

Vanilla

Ours

0.05

0

6.4

6.6

6.87.07.27.47.6

0.70

6.4

6.6

6.87.07.27.4

7.6

0.00

6.46.66.87.07.27.47.6

Time(s)

Time(s)

Time(s)

1.2 x faster

1.2 x faster

1.2 x faster

Blender

Blender

Blender

40

1.00

0.30

Vanilla

30

35

+

0.95

X

0.25

PSNR

℃℃

X

M

0.90

S 0.20

LPIPS

U0.85

0.15

S

15

0.80

0.10

X

10

Vanilla

0.75

Vanilla

0.05

5

Ours

Ours

0

5

10

15

2025303540

0.70

5

10

15

2025303540

0.00

5

10

1520253035

40

Time(s)

Time(s)

Time(s)

2x

2 x faster

2×

2 x faster

2x

2 x faster

for Eq. 20. This means if we want to use the Gauss-Laguerre quadrature to approximate Eq. 20, we have to choose points x i that are the root of n th-degree Laguerre polynomials. Since every x i has a corresponding t i following Eq. 17, we can choose t i based on given value of x i , as depicted in Fig. 3. Specifically, we want the integral Eq. 17 to be equal to the roots of an n th-degree Laguerre polynomial. Fig. 3 gives an example of n = 5 . In the figure, the numbers in the five regions filled with different colors indicate the integral value of the volume density function from zero to the right boundary of the regions. A pseudocode for GL-NeRF rendering is shown in Algo. 1.

4.1.3 Intuitive understanding of the points selected using the Gauss-Laguerre quadrature

Since the points near the surface contribute the most to the final color of the pixel as discussed in [20, 29, 31], the optimal point selection strategy should choose points near the surface. The volume density, on the other hand, increases remarkably near the surface and remains close to zero at other areas. Therefore, the integral value of it Eq. 17 should also increases significantly near the surface and remains almost unchanged throughout the rest of the space. Therefore, most of the points chosen using GL-NeRF should lie around the surface of the underlying scene. Consider a case when n = 8 , we want to choose points t i , i = 1 , 2 , . . . , 8 such that x ( t i ) in Eq. 17 should be equal to the value x i given in the look-up table Tab. 1. Notice that the first few value for x i (say first three) are small so that they could be reached by the integral of volume density near the surface easily. These values have relatively larger weights assigned to them. Evaluating the color of these points using a neural network and summing them up using the weights w i given in Eq.

1 following Eq. 12 would contribute mostly to the pixel color. Notice that even though the last few x i are quite large and may not be reached by Eq. 17 along the ray, their corresponding weights are so small that they almost couldn't affect the final result of the pixel color. Hence, the points selected

TensoRF/+Ours

TensoRF

PSNR:20.76

PSNR:21.34

TensoRF

PSNR:35.51

using GL-NeRF also correspond to the points near the surface, like in previous works [20, 29, 31] that design different neural networks for estimating the surface position, but only without any additional neural networks. Therefore, thanks to the nice property of the Gauss quadrature, ideally we can select the optimal points for computing volume rendering integral if the volume density estimation is oracle.

5 Experiments

Datasets and evaluation metrics. We evaluate our method on the standard datasets: NeRFSynthetic and Real Forward Facing Dataset(LLFF) [26] as in [27] with two different models, i.e . Vanilla NeRF [27], TensoRF [6] and InstantNGP [28]. Since our method is training-free, we conduct render-only experiments with the vanilla volume rendering method and our method. We plot the standard render quality evaluation metrics PSNR, SSIM [42] and LPIPS [48] with respect to the average time needed for rendering one image for each scene in Vanilla NeRF. We also report the metrics with averaged color MLP calls for TensoRF and InstantNGP. For Vanilla NeRF, we use 32 points for our method while the network is trained with more than 100 points. For TensoRF and InstantNGP, the results are produced with 4 MLP calls if not otherwise mentioned. More details can be found in Sec. A.1.

5.1 Comparison with baselines

1

2

3

4

8

Ori

GT

We showcase that our method can be used for rendering novel views based on pretrained NeRF without further training. We plotted the quantitative metrics of GL-NeRF and original NeRF for an intuitive comparison in Fig. 4. It shows that our method achieves comparable results as the original NeRF while requiring less computation, leading to 1.2 to 2 times faster rendering. We also observed a drop in memory usage due to the fewer MLP calls we have. We further implement our method with TensoRF [6]. As can be seen from Tab. 5, our method significantly

reduces the number of MLP calls needed for volume rendering while the rendering quality only drops a little. We observe that the minimal drop in the performance has little effect on the quality of the image. Some qualitative comparisons can be found in Fig. 5. Other than TensoRF, we implement our method on top of InstantNGP [28] to showcase the plug-and-play attribute of our method. Our method performs similarly on Blender dataset to InstantNGP as shown in Tab. 5.

5.2 Discussion on acceleration

The reason why the speed-up in Vanilla NeRF doesn't lead to real-time performance is that it has another heavy neural network for estimating the volume density. While our method needs cheap density estimation, it can be easily achieved by recent efforts in NeRF like factorized tensors [6]. Therefore, reducing the number of color MLP calls needed could lead to real-time performance as shown by previous work [13]. We therefore follow MC-NeRF [13] and develop a real-time renderer based on WebGL

and train a small variant of TensoRF with 8 channels for each density component and color component and 32 as hidden size for the color MLP. The result on Lego in the Blender dataset is shown in Tab. 5.4. GL-NeRF is able to provide almost real-time performance in WebGL with similar quality as TensoRF running on an AMD Ryzen 9 5900HS CPU thanks to the reduced number of color MLP calls.

5.3 Ablation studies

We further study the effect of sampled points per ray. We conduct experiments using the TensoRF model on the LLFF dataset. We found that 8 points per ray already shows comparable results to the original sampling strategy that uses more than 100 points. Quantitative comparison can be found in Tab. 4. Qualitatively, in Fig. 6 we found that less number of points would lead to blurrier results. Since the points selected using GL-NeRF intuitively correspond to where the surface is, we argue that the blurriness comes from the inherent inaccuracy of piece-wise constant density estimation.

5.4 Discussion on GL-NeRF usage for training

While we mainly showcase that GL-NeRF is a general alternative to the sampling strategy for volume rendering at test time, it is also capable of being used for training. We demonstrate this by replacing the fine sample stage in Vanilla NeRF with GL-NeRF and show the result in Tab. 5 and Tab. 9. GL-NeRF is able to produce on-par results with the vanilla sampling strategy but use a much smaller number of points, i.e. 32 for GL-NeRF and more than 100 for vanilla NeRF.

6 Conclusion

In this paper, we propose GL-NeRF, a novel approach for calculating the volume rendering integral. We show that with a simple change of variable, the Gauss-Laguerre quadrature can be used for computing the volume rendering integral. Thanks to the highest algebraic precision guaranteed by the Gauss-Laguerre quadrature, GL-NeRF significantly reduces the number of MLP calls needed for the volume rendering integral. We justify the use of the Gauss-Laguerre quadrature theoretically and empirically and showcase the plug-and-play attribute of GL-NeRF in two different NeRF models. Experiments show the potential of GL-NeRF being used for accelerating any existing NeRF model. We also demonstrate that GL-NeRF can be used for training vanilla NeRF, providing a potential new direction for neural rendering research.

Limitations. While GL-NeRF shows promising results in reducing the number of MLP calls needed, it still affects the rendering quality despite the theoretical guarantee of the highest precision. How to improve the performance so that it would meet the theoretical results would be interesting.

References

Milton Abramowitz, Irene A Stegun, and Robert H Romer. Handbook of mathematical functions with formulas, graphs, and mathematical tables, 1988.

Relja Arandjelovi´ c and Andrew Zisserman. Nerf in detail: Learning to sample for view synthesis. arXiv preprint arXiv:2106.05264 , 2021.

Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5855-5864, 2021.

Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5470-5479, 2022.

Serge Bernstein. Démo istration du théoréme de weierstrass fondée sur le calcul des probabilités. Commun. Soc. Math. Kharkow , 13(1):1-2, 1912.

Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision , pages 333-350. Springer, 2022.

Robert A Drebin, Loren Carpenter, and Pat Hanrahan. Volume rendering. ACM Siggraph Computer Graphics , 22(4):65-74, 1988.

Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5501-5510, 2022.

Stephan J Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, and Julien Valentin. Fastnerf: High-fidelity neural rendering at 200fps. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 14346-14355, 2021.

Carl Friedrich Gauss. Methodvs nova integralivm valores per approximationem inveniendi . Dieterich, 1815.

Walter Gautschi. Numerical analysis . Springer Science & Business Media, 2011.

Yuan-Chen Guo. Instant neural surface reconstruction, 2022. https://github.com/bennyguo/instant-nsr-pl.

Kunal Gupta, Milos Hasan, Zexiang Xu, Fujun Luan, Kalyan Sunkavalli, Xin Sun, Manmohan Chandraker, and Sai Bi. Mcnerf: Monte carlo rendering and denoising for real-time nerfs. In SIGGRAPH Asia 2023 Conference Papers , pages 1-11, 2023.

Peter Hedman, Pratul P Srinivasan, Ben Mildenhall, Jonathan T Barron, and Paul Debevec. Baking neural radiance fields for real-time view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5875-5884, 2021.

James Ivory. V. on the figure requisite to maintain the equilibrium of a homogeneous fluid mass that revolves upon an axis. Philosophical Transactions of the Royal Society of London , (114):85-150, 1824.

Carl Gustav Jakob Jacobi. Ueber gauss neue methode, die werthe der integrale näherungsweise zu finden. 1826.

JDG Jacobi. Ueber eine besondere gattung algebraischer functionen, die aus der entwicklung der function (1-2xz+ z2) 1/2 entstehen. 1827.

Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics (ToG) , 42(4):1-14, 2023.

Abhijit Kundu, Kyle Genova, Xiaoqi Yin, Alireza Fathi, Caroline Pantofaru, Leonidas J Guibas, Andrea Tagliasacchi, Frank Dellaert, and Thomas Funkhouser. Panoptic neural fields: A semantic object-aware neural scene representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12871-12881, 2022.

Andreas Kurz, Thomas Neff, Zhaoyang Lv, Michael Zollhöfer, and Markus Steinberger. Adanerf: Adaptive sampling for real-time rendering of neural radiance fields. In European Conference on Computer Vision , pages 254-270. Springer, 2022.

Liangchen Li and Juyong Zhang. l _ 0 -sampler: An l _{ 0 } model guided volume sampling for nerf. arXiv preprint arXiv:2311.07044 , 2023.

Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6498-6508, 2021.

David B Lindell, Julien NP Martel, and Gordon Wetzstein. Autoint: Automatic integration for fast neural volume rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14556-14565, 2021.

Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. Advances in Neural Information Processing Systems , 33:15651-15663, 2020.

Nelson Max. Optical models for direct volume rendering. IEEE Transactions on Visualization and Computer Graphics , 1(2):99-108, 1995.

Ben Mildenhall, Pratul P Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Transactions on Graphics (TOG) , 38(4):1-14, 2019.

Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM , 65(1):99-106, 2021.

Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG) , 41(4):1-15, 2022.

Thomas Neff, Pascal Stadlbauer, Mathias Parger, Andreas Kurz, Joerg H Mueller, Chakravarty R Alla Chaitanya, Anton Kaplanyan, and Markus Steinberger. Donerf: Towards real-time rendering of compact neural radiance fields using depth oracle networks. In Computer Graphics Forum , volume 40, pages 45-59. Wiley Online Library, 2021.

Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, and Felix Heide. Neural scene graphs for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2856-2865, 2021.

Martin Piala and Ronald Clark. Terminerf: Ray termination prediction for efficient neural rendering. In 2021 International Conference on 3D Vision (3DV) , pages 1106-1114. IEEE, 2021.

Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10318-10327, 2021.

Christian Reiser, Songyou Peng, Yiyi Liao, and Andreas Geiger. Kilonerf: Speeding up neural radiance fields with thousands of tiny mlps. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 14335-14345, 2021.

Christian Reiser, Rick Szeliski, Dor Verbin, Pratul Srinivasan, Ben Mildenhall, Andreas Geiger, Jon Barron, and Peter Hedman. Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes. ACM Transactions on Graphics (TOG) , 42(4):1-12, 2023.

Olinde Rodrigues. De l'attraction des sphéroïdes, Correspondence sur l'É-cole Impériale Polytechnique . PhD thesis, PhD thesis, Thesis for the Faculty of Science of the University of Paris, 1816.

Gopal Sharma, Daniel Rebain, Kwang Moo Yi, and Andrea Tagliasacchi. Volumetric rendering with baked quadrature fields. arXiv preprint arXiv:2312.02202 , 2023.

Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulò, Norman Müller, Matthias Nießner, Angela Dai, and Peter Kontschieder. Panoptic lifting for 3d scene understanding with neural fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9043-9052, 2023.

Vincent Sitzmann, Semon Rezchikov, Bill Freeman, Josh Tenenbaum, and Fredo Durand. Light field networks: Neural scene representations with single-evaluation rendering. Advances in Neural Information Processing Systems , 34:19313-19325, 2021.

Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P Srinivasan, Jonathan T Barron, and Henrik Kretzschmar. Block-nerf: Scalable large scene neural view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8248-8258, 2022.

Mikaela Angelina Uy, Kiyohiro Nakayama, Guandao Yang, Rahul Krishna Thomas, Leonidas Guibas, and Ke Li. Nerf revisited: Fixing quadrature instability in volume rendering. arXiv preprint arXiv:2310.20685 , 2023.

Suhani Vora, Noha Radwan, Klaus Greff, Henning Meyer, Kyle Genova, Mehdi SM Sajjadi, Etienne Pot, Andrea Tagliasacchi, and Daniel Duckworth. Nesf: Neural semantic fields for generalizable semantic segmentation of 3d scenes. arXiv preprint arXiv:2111.13260 , 2021.

Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing , 13(4):600-612, 2004.

Lee Westover. Interactive volume rendering. In Proceedings of the 1989 Chapel Hill workshop on Volume visualization , pages 9-16, 1989.

Liwen Wu, Jae Yong Lee, Anand Bhattad, Yu-Xiong Wang, and David Forsyth. Diver: Real-time and accurate neural radiance fields with deterministic integration for volume rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16200-16209, 2022.

Lin Yen-Chen. Nerf-pytorch. https://github.com/yenchenlin/nerf-pytorch/ , 2020.

Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. Plenoctrees for real-time rendering of neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5752-5761, 2021.

Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492 , 2020.

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 586-595, 2018.

Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and Andrew J Davison. In-place scene labelling and understanding with implicit scene representation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15838-15847, 2021.

A Appendix

Here we introduce the implementation details, give a brief introduction of the Gauss-Laguerre quadrature and present our quantitative results on NeRF-Synthetic and LLFF datasets.

A.1 Implementation details

For Vanilla NeRF, our experiments are conducted based upon NeRF-PyTorch [45], a reproducible PyTorch implementation of the original NeRF [27]. We implement our method by changing the hierarchical sampling strategy into our point selection method using the Gauss-Laguerre quadrature. We follow the standard setting as done in [27] to train a 'coarse' and a 'fine' network for evaluation. We use a learning rate of 5 × 10 -4 that exponentially decays to 5 × 10 -5 over the course of optimization. Each scene is trained for 200k iterations using a single NVIDIA RTX 6000 GPU. We use 128 coarse samples and 32 fine samples to test our method. Since the density estimation for 'coarse' and 'fine' network are not aligned, we test our method using only the 'fine' network by first using 'coarse' samples to query it for an estimation of density, then use GL-NeRF to select 32 points for final rendering as discussed in Sec. 4.1.2. LLFF scenes are trained and tested with 64 coarse samples and 64 fine samples for baseline method, while NeRF-Synthetic scenes require 128 coarse samples and 64 fine samples. For TensoRF, we directly use the pretrained checkpoints in the folder VM48 provided by the authors. The qualitative results are produced by 4 neural network calls if not otherwise mentioned. For InstantNGP, we build our code on top of the public PyTorch implementation [12] and train it with the default setting. The final results for GL-NeRF are also produced by 4 neural network calls.

A.2 Gauss-Laguerre quadrature

The Gauss-Laguerre quadrature is an approximation formula for computing integrals over the semiinfinite interval [0 , + ∞ ) with the weight function e -x and reads as

Here x 0 , x 1 , · · · , x n ∈ [0 , + ∞ ) are the zeros of the Laguerre polynomial L n +1 = L n +1 ( x ) of degree ( n +1) :

for n = -1 , 0 , 1 , · · · , and the coefficients

From the Leibniz formula, it is easy to see that L n ( x ) is a polynomial of degree n and the coefficient of x n is ( -1) n n ! . In particular, we have

The fundamental property of the Laguerre polynomials is

Theorem A.1. The Laguerre polynomials L n = L n ( x ) are orthogonal with respect to the weight function e -x , that is,

̸

Proof. Assume m ≤ n and set g k ( x ) = x k e -x . From the Leibniz formula it follows that, for j < k , g ( j ) k ( x ) is a product of xe -x and a polynomial of degree ( k -1) and thereby g ( j ) k (0) = 0 = g ( j ) k (+ ∞ )

for j < k . Thus, we deduce that

By the Leibniz formula, we have

and thereby

Here the second equality is similar to that in 23 and the fourth uses

This completes the proof.

The orthogonality of the Laguerre polynomials ensures that the L n ( x ) 's are linearly independent and L n +1 ( x ) has ( n +1) distinct zeros x 0 , x 1 , · · · , x n in [0 , + ∞ ) [11]. With the zeros, the coefficients w k are chosen so that the following ( n +1) equalities

hold for j = 0 , 1 , · · · , n . This leads to a system of ( n +1) linear algebraic equations for the unknowns w k and the corresponding coefficient matrix is the Vandermonde matrix [ x j k ] ( n +1) × ( n +1) . The latter is invertible since the zeros are distinct and therefore the w k 's are uniquely determined. The specific expressions of the w k 's are given in 22 [11].

It is remarkable that all the coefficients w k are non-negative. This important property ensures the stability and convergence of the Gauss-Laguerre quadrature [11]. Moreover, we have

Theorem A.2. The algebraic precision of the Gauss-Laguerre quadrature 21 is (2 n +1) exactly. Namely, " ≈ " in 21 is " = " if f ( x ) is a polynomial of degree (2 n + 1) and is not " = " if f ( x ) is a polynomial with degree higher than (2 n +1) .

Proof. Notice that

is a polynomial of degree ( n +1) . Since

̸

the precision is less than 2( n +1) . On the other hand, for any polynomial p = p ( x ) of degree (2 n +1) there are two polynomials of degree n such that p ( x ) = q ( x ) L n +1 ( x ) + r ( x ) . Notice that q ( x ) can be written as a linear combination of L 0 ( x ) , L 1 ( x ) , · · · , L n ( x ) . Compute

Here the third equality is due to 24 (the choice of w k ) and the fourth is due to the orthogonality of L n +1 ( x ) and q ( x ) with respect to the weight function. Hence the proof is complete.

For further details on the Gauss-Laguerre quadrature and for other Gauss quadratures, the interested reader is referred to the book [11].

A.3 Proof for Theorem 4.1

Here we present a proof of the well-known Stone-Weierstrass theorem, basically from [5]. We rephrase the theorem here for readability.

Theorem A.3. Suppose f = f ( x ) : [0 , 1] → ( -∞ , ∞ ) is continuous. Then for any ϵ > 0 there is a polynomial p = p ( x ) satisfying

Namely, polynomials are dense in the Banach space C [0 , 1] .

Proof. Fix f = f ( x ) ∈ C [0 , 1] and ϵ > 0 . Since f = f ( x ) is continuous on the bounded closed interval [0 , 1] , it is bounded and uniformly continuous, meaning that there are positive numbers M > 0 and δ = δ ( ϵ ) > 0 such that

for any x, y ∈ [0 , 1] satisfying | x -y | < δ .

With δ fixed, let n ≥ M δ 2 ϵ be a positive integer. Consider the n th-order Bernstein polynomial [5]

with C k n = n ! k !( n -k )! . Notice that, for x ∈ [0 , 1] ,

Then, for any x ∈ [0 , 1] , we deduce from (25)-(28) that

This completes the proof.

A.4 Quantitative results on NeRF-Synthetic and LLFF datasets

In this part, we present our per-scene quantitative results on NeRF-Synthetic and LLFF datasets in Tab. 8 for Vanilla NeRF and in Tab. 7 for TensoRF. The full result for training using GL-NeRF is presented in Tab. 9

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We prove the claim of reducing computation in Sec. 5, the analysis and justification of using the Gauss-Laguerre quadrature are in Sec. 4.1.1 and Sec. 4.1.2.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations are discussed in Sec. 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All theories in the paper are proven in the Appendix, specifically Sec. A.2 and Sec. A.3.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Details are provided in Sec. 5 and Sec. A.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Experimental settings are discussed in Sec. 5 and Sec. A.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Our method is training-free, thus no randomness during test time.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Compute resources are introduced in Sec. 5 and Sec. A.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have read and comply to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our work is a foundational research. There is no societal impact of the work performed.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The datasets used in the paper i.e . LLFF and NeRF-Synthetic are properly cited, the codebase we use are properly cited in Sec. 5 and Sec. A.1. The license for these assets are CC-BY 3.0 for the two datasets, and MIT License for the two codebases.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: No new assets have been created in the paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
Volume rendering requires numerical integration for estimating output colors. This work proposes using the Gauss-Laguerre quadrature to reduce the number of samples and improve integration accuracy. The paper demonstrates that this method can be a plug-and-play module for any NeRF model. Experimental results show that with a limited drop in performance, the GL-NeRF can significantly reduce the number of ray samples and MLP calls.

STRENGTHS:
1. The perspective of improving quadrature for NeRF is new and interesting.
2. The formulation of using the Gauss-Laguerre quadrature looks good, and the mathematical formulation appears rigorous.

WEAKNESSES:
1. The results show a performance drop of about ~2 PSNR, while the speed improvement is not significant.
2.  The motivation is not strong. Most state-of-the-art NeRF approaches use shallow MLPs or even no MLPs, making the evaluation less expensive. Reducing ray samples does not seem to address a core issue in radiance field research.
3. How the points are selected is unclear. The method requires approximating polynomial coefficients and resolving the roots for $x$ However, since $x$ is a highly non-linear function of $t$, finding the samples  $t$ unavoidably requires root finding along the ray, which does not seem to actually improve accuracy or efficiency.

QUESTIONS:
1. Why use a look-up table? Will this lead to worse performance?
2. Why did the performance not match TensoRF? Would increasing the number of point samples help?

LIMITATIONS:
The paper mentions that it has a theoretical guarantee of the highest precision. However, there is no evidence that the current precision matches previous works.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-06-27

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a computational method for volume rendering using Gauss-Laguerre quadrature. In the context of NeRF, volume rendering is performed by evaluating MLPs (or other data structures) at a sequence of query points on a ray and integrating the weighted results. The proposed method reduces the number of evaluations without much performance degradation by computing the integrations using Gauss-Laguerre quadrature. The proposed method has been embedded in vanilla NeRF and TensoRF for validation, and the reviewer reports reductions in computation time and memory usage. The reviewer acknowledges and appreciates the effectiveness of the proposed method and looks forward to future discussions on the generality of the proposed method (other backbones and learning time applications).\n\nSTRENGTHS:\n- As emphasized in the paper, the approach proposed in this paper is a replacement of integral computations, not a learning of sampling or a change in data structure. Therefore, it is applicable and highly available for various NeRF variants that use volume rendering.\n- The position of the proposed method in NeRF research is clearly stated, especially the related work is very clearly described as an introduction to the proposed method.\n- In the introduction of the method, the reviewer's first question was l.177, and the paper answers the question. This helps the understanding of the paper and increases the credibility of the proposed method.\n\nWEAKNESSES:\n- The proposed method results in dense sampling near the surface, but can it handle translucent objects? Intuitively, there seems to be a correlation between density solidity and rendering quality. It would be desirable to discuss for which scenes the proposed method is effective and for which scenes it is not.\n- The effectiveness of the proposed method as an integration method for oracle or trained models is certain. In general, however, volume rendering is used for both training and inference. It would be better to discuss whether the proposed method is specialized for inference or whether it could be used for learning as well.\n- We expect the availability of the proposed method to be very broad. If possible, the application of the proposed method to NeRF backbones other than vanilla NeRF and TensoRF could be considered to further demonstrate the generality of the proposed method.\n- The correspondence between the plots in Figure 4 is not clear. It would be easier to compare the results if the corresponding scenes of vanilla and the proposed method were connected by a line.\n- Small Comment: Related works -> Related work\n\nQUESTIONS:\n- Can the proposed method be used for learning? If so, the proposed method could be a very powerful tool. (identical to the description in the Weaknesses section).\n- Why are the results for TensoRF not shown in Figure 4?\n- The sample points up to 8 are very small compared to the use of more than 100 sample points in vanilla NeRF. Could the performance of vanilla NeRF be exceeded with more sample points? In other words, vanilla NeRF also approximates integration by discretization, and this sampling density is constant. We believe that a comparison of vanilla NeRF and the proposed method in terms of sample points vs. image quality will more convincingly demonstrate the effectiveness of the proposed method.\n\nLIMITATIONS:\n- I agree with the limitations described in the conclusion. This reviewer believes that the proposed method contributes to real-time rendering, but it better to be validated to make this claim in the future.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper focuses on accelerating novel view synthesis using neural radiance fields (NeRF). Unlike previous works that concentrate on designing lightweight networks, this study is motivated by the specific volume rendering formula, which includes a negative exponential term in the integration function. By employing Gauss-Laguerre quadrature, the authors approximate this complex integral operation, thus improving the rendering speed of existing NeRFs. This approach is validated on two backbones: the original NeRF and TensoRF, demonstrating speed improvements ranging from 1.2X to 2X.

STRENGTHS:
1. The idea is intriguing and represents a promising exploration originating from the specific volume rendering formula.
2. This paper is highly theoretical.

WEAKNESSES:
see Questions.

QUESTIONS:
In the experiment, the authors validated the proposed method on two backbones (the original NeRF and the TensoRF). However, neither of these represents the current fastest method. It is suggested to compare the proposed method with Instant NGP, DVGO, or other faster alternatives. Such comparisons could not only better verify the plug-and-play capability but also significantly enhance the impact of the paper.

LIMITATIONS:
n/a.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-19

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method for reducing the number of color samples needed for volume rendering in Neural Radiance Fields. The method works by applying Gauss-Laguerre quadrature as a replacement for the importance sampling used by some NeRF methods to reduce the number of calls needed for their fine color MLPs. This approach is shown to yield some efficiency improvements on plain NeRF as well as TensoRF models.\n\nSTRENGTHS:\nThe mathematical aspects of the paper are detailed and well supported. It is very clear what the method is trying to achieve and how the Gauss-Laguerre formulation is being applied. Generally, the quality of explanation is good and not too hard to follow, even though the math is fairly dense.\n\nOverall, I think this is an original idea with some likely applications in volume rendering.\n\nWEAKNESSES:\nThe main issue with this paper is that the experiments do little to give an idea of how the proposed method would compare to the current state of the art, which has progressed quite significantly beyond the baselines shown here. While it is quite believable that the GL method outperforms the naive hierarchical sampling of the original NeRF, I have significant doubts about that holding when applied to something more recent like the proposal networks from Mip-NeRF 360 and Zip-NeRF.\n\nGiven the weak evaluation, I would lean towards rejecting.\n\nQUESTIONS:\nIt is strange that timings are reported for vanilla NeRF but not TensoRF, which would presumably see a larger gain. Is there a reason for this?\n\nI would suggest that the authors try to show quantifiable improvement on a more recent baseline. If there were a substantial run-time speed up for TensoRF, that would be good, but something lime Mip-NeRF 360 or Zip-NeRF would be even better as that would show how GL compares to another method which tries to draw samples near the surface.\n\nLIMITATIONS:\nIt is mentioned, but I think the writing could make it a lot more clear that only the color samples are being reduced, as this significantly affects where the method would actually be expected to provide a speedup. As it is, this is quite easy to miss and could lead to misunderstanding if one does not read the method carefully.\n\nI don't think there are any notable concerns with the paper regarding societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-19"}