GVKF: Gaussian Voxel Kernel Functions for Highly Efficient Surface Reconstruction in Open Scenes

Authors: gaochao  song
chong  cheng
hao  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 10:34:39

Abstract: In this paper we present a novel method for efficient and effective 3D surface reconstruction in open scenes. Existing Neural Radiance Fields (NeRF) based works typically require extensive training and rendering time due to the adopted implicit representations. 
In contrast, 3D Gaussian splatting (3DGS) uses an explicit and discrete representation, hence the reconstructed surface is built by the huge number of Gaussian primitives, which leads to excessive memory consumption and rough surface details in sparse Gaussian areas.
To address these issues, we propose Gaussian Voxel Kernel Functions (GVKF), which establish a continuous scene representation based on discrete 3DGS through kernel regression. The GVKF integrates fast 3DGS rasterization and highly effective scene implicit representations, achieving high-fidelity open scene surface reconstruction. Experiments on challenging scene datasets demonstrate the efficiency and effectiveness of our proposed GVKF, featuring with high reconstruction quality, real-time rendering speed, significant savings in storage and training memory consumption.

Main paper: 
 introduction: 
 3D surface reconstruction in open scenes holds great significance in various practical applications, such as autonomous driving, virtual reality, urban planning and etc. However, achieving high-fidelity and efficient open scene reconstruction has been a longstanding challenge, due to the trade-off between the rendering quality and the required resources for optimization.In pursuit of this goal, two predominant approaches are Neural Radiance Fields (NeRF) [25, 37, 1, 10, 18] and 3D Gaussian Splatting (3DGS) [14, 4, 20, 38] based methods. On one hand, NeRF-based implicit representations typically require extensive training and rendering time, which limits the practical use in large-scale scene reconstruction [10, 35, 24, 33] . On the other hand, 3DGS [14] adopts explicit representations, which enables high-quality novel view synthesis while achieving real-time rendering. This makes 3DGS more feasible for efficient scene reconstruction in the applications such as autonomous driving and virtual reality.Recently, there are studies using 3DGS technology for novel view synthesis and surface reconstruction in street scenes and urban environments [4, 20, 38, 22] . For instance, SuGaR [11] attempts to reconstruct the 3D surfaces based on Gaussian points. However, it has been noted that overly large and sparse Gaussian points can significantly affect the geometric representations of the scene, particularly in background areas. To overcome these challenges, the 2D Gaussian Splatting (2DGS) [13] proposes to use Gaussian surfaces as surfels to represent complex geometries [28] , thereby improving the surface reconstruction quality. Particularly, 2DGS faces challenges when processing large-scale scenes, as it requires the explicit representation of a large number of Gaussian primitives, leading to significant GPU memory consumption. Therefore, 2DGS still exhibits limitations in novel view synthesis capabilities and the geometric representation of large-scale scenes.In Table 1 , we summarize the comparison of 3DGS rendering and volume rendering. To fully leverage the fast rendering advantages of Gaussian alpha blending while achieving effective implicit scene representation, we propose a novel Gaussian Voxel Kernel Functions (GVKF) method. Firstly, GVKF utilizes voxelization to implicitly represent 3DGS, managing the growth and pruning of Gaussian splats. This approach retains the expressive power of explicit Gaussian splats while enabling efficient management of these splats. Secondly, we carefully analyze the intrinsic connection between Gaussian splatting alpha blending rendering and traditional volume rendering from a mathematical perspective. We establish a 3DGS-based method to represent continuous scene opacity density fields through kernel regression. This makes it possible for discrete Gaussians to represent continuous scenes. By replacing the discrete opacity values in original 3DGS rendering pipeline (which can be viewed as collapsed kernel functions) with Gaussian kernel functions, we maintain the advantages of the original 3DGS alpha blending while optimizing the representation of continuous scenes. Moreover, we demonstrate that our proposed rendering method is mathematically consistent with traditional volume rendering. Thirdly, based on our constructed scene opacity representation, which is also known as the scene opacity field, we derive the bidirectional mapping relationship between opacity and the scene surface. This enables direct mesh extraction for scene surface. In summary, our contributions are as follows:• We propose GVKF, an implicit continuous scene reconstruction method that integrates the effectiveness of implicit representation with the fast rasterization advantages of Gaussian Splatting, without the need for computationally intensive volume rendering.• Based on GVKF, we further propose implicit representation of the scene surface, achieving efficient and high-quality scene surface reconstruction.• Experiments demonstrate the usefulness of GVKF in open scenes, showcasing high-quality surface reconstruction accuracy, real-time rendering speeds, and significant savings in storage and memory consumption.2 Related Works 
novel view synthesis: 
 The introduction of Neural Radiance Fields (NeRF) [25] has significantly advanced the development of 3D reconstruction and novel view synthesis. NeRF employs volumetric rendering techniques to intricately simulate the geometric structure of scenes and viewpoint-dependent characteristics, thereby considerably enhancing the quality of image rendering. Following NeRF, variants such as Mip-NeRF [1] and Zip-NeRF [2] have addressed the aliasing issues during rendering. Additionally, UC-NeRF [6] , designed for outdoor scenes, enhances image consistency through color correction and pose refinement. InstantNGP [26] accelerates training and improves rendering efficiency by optimizing subvolume processing with grid pyramid techniques. Meanwhile, other feature grid-based scene representation methods [3, 21, 31, 5, 42] have been extensively explored to enhance the training capability and expressiveness of models. Recently, 3D Gaussian Splatting (3DGS) [14] effectively represents complex scenes using 3D Gaussian points, significantly boosting the efficiency of real-time high-resolution image rendering while maintaining rendering quality. Further research efforts like Scaffold-GS and Octree-GS [23, 30] have attempted more effective methods to organize and manage Gaussian points, which helps reduce memory usage and speed up training.Implicit SurfaceFigure 1 : Framework of Gaussian Voxel Kernel Functions (GVKF) for scene representation. In this framework, discrete Gaussian primitives G represent continuous opacity density ρ(t) on the ray via kernel regression. After slightly modifying the rasterization pipeline, the kernel function can be integrated into alpha blending rasterization without introducing dense points sampling. Additionally, we directly define the mapping relationship between the neural opacity field and the implicit surface.Traditional isosurface extraction, relying on density thresholds, often struggles with fine details due to resolution and noise constraints. Recent studies propose more complex representation methods [45] .For instance, NeuS [34] uses MLP networks for occupancy grids or SDF, improving reconstruction accuracy and noise reduction [27, 40, 19, 7, 39] . Techniques like BakedSDF [41] translate the optimization of NeRF or neural SDFs into 3D meshes, enhancing features through high-resolution grids but increasing computational load. NeuS2 [36] introduces a novel formula for second-order derivatives with multi-resolution hash encoding and CUDA-based MLP technology, significantly reducing training time. StreetSurf [10] optimizes SDF mappings in open scenes and decouples static and dynamic objects. Despite advancements, NeRF-based methods still need optimization for processing speed and real-time rendering.3DGS has gained attention for its high-quality scene reconstruction and rapid processing capabilities [14] . 3DGS uses multiple 3D Gaussian distributions with anisotropic covariance for precise control over scene attributes [46, 17] . This technology enhances surface reconstruction methods like SuGaR [11] , which employs Poisson surface reconstruction for fast and accurate mesh extraction . However, irregular Gaussian sphere distribution affects surface quality. To improve this, 2DGS [13] uses 2D Gaussian planes for better surface conformity and TSDF for accurate reconstruction, though it may cause surface fragmentation. GOF [44] directly extracts surfaces using opacity thresholds and tetrahedral mesh extraction but is limited by high VRAM requirements. GSDF [43] combines 3DGS with a NeuS-like SDF branch for optimized rendering and reconstruction, increasing training time.Despite their potential, 3DGS-based methods face challenges like managing Gaussian points, high VRAM consumption, and degraded rendering quality.As demonstrated in the ablation experiments, to balance quality and speed, we chose to downsample the initial Gaussian point cloud using a voxel size of 0.01. Within each voxel, the dimension of F is set to 32, and it stores the relative coordinates of 10 Gaussian points, indicating that the maximum number of Gaussians generated per voxel grid is 10. Gaussians with an opacity less than 0 will be hidden during each iteration. In each scene, all voxel grids share a total of four MLPs, which decode different Gaussian attributes from the corresponding voxels.Regarding voxel registration, the gradient threshold is empirically set to 2 × 10 -4 , meaning that voxel grids with an average gradient exceeding this value after each iteration will be subdivided using an octree method. The maximum recursion depth is set to 3 to control the number of Gaussians in the scene, ensuring it does not exceed a certain threshold. Voxel evaluation is performed every 500 iterations to determine which voxels should be subdivided or reclaimed. For other settings, we strive to remain consistent with the original 3DGS settings. 
methods: 
 As shown in Fig. 1 , we first introduce the implicit neural 3DGS primitives representation based on a sparse voxel grid, which offers highly efficient storage management and the fitting power of neural networks. Secondly, we present our GVKF-based continuous scene representation, to explain its rationale, we have analyzed its intrinsic connections with Gaussian alpha blending [14] and traditional volume rendering [25] from a statistical analysis perspective. Finally, we describe the relationship between the proposed continuous scene representation (a neural opacity field) and implicit surface, and derive an explicit mapping function for mesh reconstruction.To achieve orderly 3DGS management while minimizing the explicit expression of them to save training storage consumption, we use a spatial sparse voxel grid to manage Gaussian primitives.During the initialization phase, the sparse grid is generated from the downsampled SfM point clouds and dynamically grows or being eliminated during training. Each sparse grid is allowed to generate up to m Gaussian primitives, and all these primitives are limited to a small range of space centered at the voxel grid.Gaussian Generation For a particular 3D Gaussian expression, five attributes are required: p ∈ R 3 (position), α ∈ R (opacity), R ∈ R 3×3 (rotation matrix), s ∈ R 3 (scaling), and c ∈ R 3 (color). Then, a Gaussian G(x) can be generated as:EQUATIONwhere ∈ R 3×3 is covariance matrix defined as = Rss T R T . c is calculated via SH coefficients and camera direction. Different from traditional 3DGS [14] that treats them as explicit optimizable tensor, we decode them from a feature vector F ∈ R d via several MLPs:α = MLP α (F, camera), R = MLP R (F), s = MLP s (F), c = MLP c (F, camera).(EQUATIONFor alpha and color MLPs, the view camera and feature vector F are inputs, facilitating viewdependent fitting. Relative coordinates of Gaussians to the parent voxel center are stored with F, compressing explicit Gaussian components and leveraging the MLP's fitting capacity. Gaussians are dynamically generated each iteration and recycled post-update, reducing memory usage.Voxel Registration. To control Gaussian number in large open scene, we eschew the traditional adaptive density control strategy, adopting a method inspired by scaffold-Gaussian [23] and Octree-Gaussian [30] . The voxel registration is based on gradient accumulation. After each iteration, gradients from 3DGS are recorded and accumulated in their respective voxels, denoted as ∇. Voxels where ∇ exceeds a set threshold are subdivided into eight subvoxels to increase grid resolution, continuing until the maximum depth is reached. Additionally, less frequently used voxels are discarded after a specified period.Since 3DGS rasterization rendering and traditional volume rendering share some overlapping concepts, in this section, we sort them out and introduce our method from a statistical perspective while avoiding introducing redundant mathematical symbols.Continuous Scene Description. We define ρ(t) : [0, +∞] → [0, 1] as the opacity density function, which measures the probability of a ray encountering a particle at position t. We define T (t) : [0, +∞] → [0, 1] as the transmission function, which measures the probability that a ray has not encountered any particles from its origin to point t. Considering the probability that a ray does not encounter any particles at time step t + dt, denoted as T (t + dt), it is evident that T (t + dt) = T (t)(1 -ρ(t)dt). Solving this differential equation, we obtain the relationship between T (t) and ρ(t):EQUATIONTherefore, we obtain the cumulative distribution function (CDF) of the probability that a ray hits a particle over the interval [0, t]: Φ(t) = 1 -T (t), with the corresponding probability density function (PDF) being Φ ′ (t) = T (t) • ρ(t). From the perspective of volume rendering, this PDF is used as the probability of the appearance of color along the ray, ultimately taking the mathematical expectation of the color as the ray color:EQUATIONThe discrete formulation of volume rendering Eq. 4 is:EQUATIONwhere opacity α i represents the accumulated result in a sampling interval δ i of volume density σ i , hence the value of N does not influence the result as long as α i is adapted enough. Based on the similar idea of volume rendering, the PDF Φ ′ (t) = T (t) • ρ(t) can also be reasonably considered as the probability of the appearance of a surface along the ray, where the place with the highest probability density is most likely to have a surface. Correspondingly, on the CDF Φ(t), this is the place where the derivative is the largest. In this paper, we use the CDF Φ(t) to describe continuous scenes based on the camera rays, to facilitate integration with the 3DGS rasterization rendering pipeline. In section 3.3 , we will prove that under the 3DGS representation, the place where the derivative of CDF is the largest is not actually the surface, so a method to locate the surface will be introduced. In implicit scene representation methods based on volume rendering, the continuous opacity density function is directly predicted by a MLP [25] . In our approach, the continuous opacity density function ρ(t) is fitted through kernel regression via discrete Gaussian primitives after a differentiable transformation: G i (x, y, z) → K i (t -t i ) from 3D to 1D. The transformation consists of three steps: (1) . The Gaussian primitives that the ray passes through are selected as kernel functions.(2). According to the Ray-Gaussian Intersection method [15] , the ray is transformed into the local coordinate system of each 3DGS to obtain the 1D probability density alone the ray. Here, the peak of the 1D probability density, denoted as t i , is defined as the Ray-Gaussian Intersection [9, 15, 44] , indicating that the 3DGS has the greatest influence at this point alone the ray. (3). To integrate with regularization methods [13, 44] , we assume that each 3DGS fits the surface of the object. Therefore, after t i , the probability density continues to remain at its maximum value, indicating that the object is solid. Without loss of generality, the opacity density ρ(t) on camera ray can be expressed as:EQUATIONwhere N represents the number of activated kernel functions along the ray, and k i represents the summarized transform of Ray-Gaussian transform (See Appendix A.1 for details). α i represents the coefficient for each kernel function. 2Rendering. As for traditional 3DGS rasterization, the pixel color is rendered through alpha blending on N 3DGS being passed through by the ray:EQUATIONIn this scenario, α i is constant value representing the opacity of Gaussians. This point-based rendering is coherent with Eq. 5, with extremely sparse sampling points to simulate dense volume rendering. However, it is impossible to recover continuous opacity density alone the ray from such a rendering equation, as illustrated in row-3 of Fig. 2 . This is because the third row of the covariance matrix of 3DGS is discarded, and it is directly projected onto a 2D plane to evaluate the impact on the opacity of points along the ray. From the perspective of Eq. 6, this means that along the ray, the influence range of all N kernel functions that intersect with the ray collapses to an infinitesimally value, making it impossible to recover a continuous opacity density field. To solve this, Eq. 7 can be modified to [44] :EQUATIONThis equation will not affect the goal of 3DGS rendering: to approximate traditional volume rendering using sparse sampling points. And it allows for broadening the collapsed kernel functions to fit the continuous opacity function of the scene.Scene Representation. Based on the discussion at the beginning, the scene surface can be described via CDF Φ(t) on the ray in a continuous way, which can be calculated like Eq. 8 (removing color): This implicit opacity field (denoted as neural opacity field since it is represented by neural Gaussians) measures the CDF of the probability that a ray hits solid scene surface.EQUATIONIn the next section, we introduce the mapping of Φ(t) to implicit surface.We represent implicit surface with signed distance function (SDF), denoted as function D(t) on the camera ray.To recover D(t) of given Φ(t) that is calculated from well trained 3DGS, we firstly study the reverse mapping problem: ϕ :D(t) → Φ(t)Opacity Density Near the Surface. To ensure that the 3DGS aligns with the object's surface and thus reflects the object's shape, depth distortion regularization [13, 44] is introduced during the Gaussian training process. This encourages the distribution of 3DGS along the ray to aggregate together, causing the peak of the kernel functions to coincide with the object's surface. In the next discussion, the coordinate of object surface on the ray is assumed as t * with D(t * ) = 0. Considering ρ(t) at the interval t ∈ [0, t * ], we have:EQUATIONWhere M represents the number of Gaussian kernels concentrated on the surface. To facilitate calculations, we convert the opacity density system to the SDF coordinate system, with t * as the origin and letting u = -D(t), as illustrated in Fig. 3 , we have:EQUATIONEQUATIONwhereρ(u) ∼ N (0, σ 2 ), σ 2 = M i=1 1 2πα 2 i, which can be directly derived from the additive property of the normal distribution. Then letting h(u) = -ρ 2 (u) + ρ ′ (u), we have:EQUATIONIt is easy to prove that h(u) crosses a unique zero point u 0 from top to bottom on the u-axis, and u 0 < 0. This means that the peak of Φ ′ (u) will appear before the surface, so it is not reasonable to simply determine the actual intersection point of the light ray with the surface by directly evaluating the peak of Φ ′ (u). To locate the accurate surface, a transcendental equation of u is needed to be solved to get u 0 : It is impossible to directly get the analytical solution, however, numerical computation methods can be applied to solve u 0 . This may require some extra time for computation.ρ(u) = - u σ 2 , ρ(u) = 1 √ 2πσ exp(- u 2 2σ 2 ), σ 2 = M i=1 1 2πα 2 i (14)Mapping from Opacity to Surface.Based on the analysis above, we can always have exact number of u 0 via numerical computation method. However, it is hard to find out the inverse function of Φ(u) for directly building the mapping of Φ(t) to D(t). For the balance of surface smoothing while reducing the indelible error, we represent mapping relationship of u → Φ(u) via Logistic Function as follows:EQUATION)where µ represents the smooth factor. We choose Logistic Function because of its formal is concise and shares similar shape of Φ(u). More importantly, it only has one inflection point at (0, 0.5), which can be used to simulate the inflection point of Φ(u) after translation. Finally, we represent implicit SDF function via Inverse function transformation of Logistic Function, as shown in Fig. 4 : Figure 6 : Qualitative comparison on the Tanks and Temples dataset [16] shows that our method excels in reconstructing complex backgrounds with high geometric granularity. In contrast, 2DGS often results in fragmented backgrounds, while SuGaR displays uneven spherical shapes, affecting both visual and geometric quality. Datasets. To assess our method's performance against baseline methods in open scenes, we used three datasets. We first experimented with the Waymo Open Scene dataset [32] , using three cameras per scene from five available, each scene containing about 600 images. We employed LiDAR point clouds to evaluate reconstruction quality, although LiDAR data was not used as training input. We also tested on the Tank and Temple dataset [16] , which includes trajectories and ground truth for six selected scenes. Lastly, we evaluated the Mip-NeRF 360 dataset [1] ; due to the absence of ground truth, our focus was on novel view synthesis to demonstrate our method's efficacy in this aspect.EQUATIONBaselines. In terms of surface reconstruction, we presented the results on the Waymo dataset [32] in tables 2 and figures 5, comparing state-of-the-art implicit methods (such as NeuS [34] , F 2 -NeRF [35] , StreetSurf [10] ) and explicit methods (such as 3DGS [14] , SuGaR [11] , 2DGS [13] ). We utilized PSNR to evaluate the results of novel view synthesis and Chamfer distance to measure reconstruction accuracy, while also recording training time, VRAM usage, and the size of the Gaussian point files post-training. Additionally, as shown in Table 3 and Figure 6 , we conducted comparisons on the Tank [34] , Geo-NeuS [8] , Neuralangelo [19] ) and explicit methods (such as 3DGS [14] , SuGaR [11] , 2DGS [13] ). We used official scripts to evaluate F1 scores. For novel view synthesis, we compared various advanced methods on the Mip-NeRF 360 dataset, including NeRF [25] , Deep Blending [12] , Instant NGP [26] , MERF [29] , Mip-NeRF 360 [1] , BakedSDF [41] , 3DGS [14] , SuGaR [11] , and 2DGS [13] . We use evaluation metrics such as PSNR, SSIM, and LPIPS. Implementation Details Our method modifies the representation of 3DGS and slightly adjusts the opacity weights in the rendering pipeline using Gaussian kernel functions. This ensures compatibility with other components of Gaussian rasterization rendering. Similarly, we employ the same L1 loss and D-SSIM loss as 3DGS to supervise color loss, and we use the same Gaussian regularization term as 2DGS and GOF to promote alignment between the Gaussians and the surface. After training, the SDF field of the scene can be directly extracted based on Eq. 16 and exported to a mesh with the MC/MT algorithm. 
analysis: 
 Figure 5 demonstrates the superiority of our method in capturing detailed features of roadside houses, bushes, and other objects. In contrast, the 2DGS method produced more holes and fragmentation, while the StreetSurf method lost some critical geometric features. The results in Table 2 indicate that our method surpasses other methods in terms of view synthesis and reconstruction accuracy, and it requires fewer Gaussian points and VRAM for large-scale scene reconstructions. Figure 6 highlights our method's excellent performance in scene restoration. The SuGaR method generated excessive irregular protrusions, and 2DGS exhibited more fragmentation and floating debris. According to the results in Table 3 , our method outperforms all explicit methods and achieves comparable reconstruction results to implicit methods, while maintaining equivalent GPU time usage. The results in Table 4 confirm that our method leads in novel view synthesis across all compared methodologies.In this section, we evaluate the impact of varying voxel grid sizes on the neural Gaussians by conducting experiments with the Waymo datasets. We selected voxel sizes of 1, 0.1, 0.01, 0.001, as presented in Table 5 . When the voxel size is too large, the sparse neural Gaussians fail to learn the scene representation and return NaN errors. As the number of voxels increases, more Gaussians are generated for scene representation, thereby enhancing the quality of novel view synthesis. However, the improvements plateau when the voxel size is reduced to 0.001, which also requires more training time and becomes impractical. Therefore, we set the voxel size to 0.01 to balance training time and rendering quality. We further conducted ablation study on the Tanks and Temples dataset [16] to evaluate the impact of voxel representation and SDF mapping. The results are presented in Tab. 6. It can be observed that utilizing voxel representation significantly improves the PSNR for NVS tasks and reduces memory consumption dramatically compared to naive 3DGS setup. Although there is a slight decrease in the geometric quality of surface reconstruction, we consider this trade-off acceptable.Implicit methods, such as those based on NeRF [25, 34, 19] , typically utilize a global fitting approach for SDF, which allows them to fully leverage the universal approximation capabilities of MLPs. This is advantageous even in areas with sparse viewpoints. However, our current method employs a local line-of-sight-based SDF fitting, a compromise made to adapt to the 3DGS rendering style. This means that regions not covered by the training viewpoints lack fitting capability, resulting in uneven surfaces.In addition, While our method advances 3D surface reconstruction in open scenes, it faces challenges with dynamic objects and the decoupling of distant and near views, sometimes misrepresenting the sky as a surface enveloping the model. The lack of sufficient prior knowledge for optimizing complex scenes also poses limitations. 
conclusion: 
 This paper introduces GVKF, combining Gaussian splatting's rapid rasterization with the efficiency of implicit expressions to enhance reconstruction quality and speed significantly. By employing a voxelized implicit representation of 3DGS, GVKF retains the expressive power of explicit Gaussian maps while managing them effectively. We have explored the relationship between Gaussian splatting's alpha blending and traditional volume rendering, developing a GS-based method to represent continuous scene opacity density fields through kernel regression, addressing 3DGS's limitations in continuous scene representation.Experimental results demonstrate GVKF's effectiveness in open scenes, showing notable improvements in reconstruction accuracy, real-time rendering speeds, and reductions in storage and memory usage. These advancements support applications in fields like autonomous driving and virtual reality, pushing forward surface reconstruction technology.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The code and data are available in a public repository.Guidelines:• The answer NA means that paper does not include experiments requiring code. • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable). • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: All relevant training and testing details, including data splits, hyperparameters, optimizer types, and selection criteria, are clearly specified in the paper. (Sec. 4)Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: The paper reports PSNR, SSIM, F1 scores, LPIPS,and C-D values, which are commonly used as a measure of performance in image processing experiments. This approach is standard in the field and is sufficient to convey the performance of the methods under investigation.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: The paper includes a section discussing the broader impacts of the proposed method, highlighting both potential positive applications and possible negative consequences. (Sec. 5) Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).• The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A.1 Ray-Gaussian Intersection Based on Eq. 1 in the main text, the influence of 3DGS in camera space on a one-dimensional ray can be expressed as follows:EQUATIONHere, v represents the unit vector of the ray direction. This formula converts the three-dimensional influence of 3DGS into a one-dimensional function along a specific camera ray, which is a one-dimensional Gaussian function. Fig. 7 demonstrates the relationship of this transform. For ease of notation, we express it as:EQUATION)where t i denotes the point along the ray where 3DGS has the maximum impact, also known as the "ray-Gaussian intersection," which can be analytically given by:EQUATIONMore proof details can be found in "Approximate Differentiable Rendering with Algebraic Surfaces." [15] A.Our method focuses on the challenging task of open scene reconstruction. Here, we provide a comprehensive quantitative comparison with other related methods on the Mip360 dataset, as shown in Table 7 . Additionally, we have included more experimental results on the Mip360 and Tank and Temple datasets, as shown in Figures 8 and 9 . For more qualitative results, please visit the project page.Discussion on indoor scene. We observe that current methods based on 3DGS perform adequately for indoor scenes, where there is typically 360-degree viewpoint coverage. However, they underperform in outdoor scenes due to limited viewpoint coverage. Heuristic splitting and pruning strategies in original 3DGS tend to fit the training viewpoints rather than distributing evenly across the space. This leads to poorer novel view synthesis results in outdoor environments. As illustrated in Fig. 10 , without a voxel grid, heuristic Gaussian growth strategies result in uneven spatial distribution of GS, sometimes even creating holes. Conversely, using voxel grids to constrain Gaussians allows for efficient management of their spatial distribution, supporting better novel view synthesis. Table 7 : Quantitative evaluation on the Mip-NeRF 360 [1] . All scene dataset is presented.Outdoor Scene Indoor Scene Average Failure Case. As shown in Fig. 11 , in areas with sparse viewpoint coverage, the distribution of 3DGS is sparse and irregular, which hinders the fitting of smooth planes. This sparsity compromises the integrity of the surface reconstruction, resulting in models that are geometrically inaccurate. Our method still struggles to effectively address these issues.PSNR ↑ SSIM ↑ LPIPS ↓ PSNR ↑ SSIM ↑ LPIPS ↓ PSNR ↑ SSIM ↑ LPIPS ↓ NeRF 21The similar rendering equation is firstly proposed by GOF [44] , while this work provides in-depth analysis of the relationship among this rendering strategy, volume rendering and Gaussian alpha blending. Different from GOF, our scene representation is implicit, addressing the common issue of high memory consumption faced by 3D Gaussian splatting. Additionally, we developed a mapping function from opacity to SDF to alleviate the influence of directly linear transform between these fields.As shown in Tab. 8, GOF uses explicit Gaussian management, still faces high storage consumption issues, making training large scenes on a single card challenging. Our method achieves better novel view synthesis results with less storage usage. However, as shown in Tab. 9, our current implementation has some geometric precision gaps compared to GOF, the potential reasons may include:• GOF's iterative optimization extraction method achieves more precise isosurfaces than ours. (May require long meshing time ∼ 2 h) • Further adaptation of regularization term to voxel grids might be needed in our method to improve mesh quality.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Specifically, αi = βi √ k i √ π ,where βi is a constant value representing opaticy related to Gaussian primitives. 


Decision: Initially, there were issues with the writing of the paper. The rebuttal was very clear and informative. Reviewer zurJ maintained a borderline reject rating, while the other three reviewers upgraded their ratings to 2x weak accept and 1x borderline accept and agree that there is an interesting contribution and strong results. There was a big discussion on the similarity to GOF. As it was published less than 8 weeks before, it is concurrent work and should not be held against this submission.

Therefore, the decision is to accept the paper. This decision was also discussed with the SAC. The authors are requested to integrate all the feedback to improve the writing of the paper.

Reviews: review: paper_summary: paper_summary: This paper introduces a method for 3D Gaussian reconstruction and surface extraction from the 3D Gaussians. The method can render scenes with low memory usage through a voxel organization. Additionally, the authors derived a continuous implicit field on top of the 3D Gaussians and extracted the surface on this implicit field. Overall, the results look good.
strength_weakness: strength_weakness: (1) Figure 1 is too simple and abstract, lacking intuitive demonstration.

(2) I can somewhat understand what the authors want to do in Equation 5, but could the authors provide some figures for easier understanding?

(3) Equation 6 seems incorrect. There appears to be a mixed use of 'i' and 'k'.

(4) Equation 7 seems incorrect. What does the term 't' mean here? It is not defined or explained here.

(5) Why do we need to convert to an SDF value? I do not understand the reasoning.

(6) I cannot find how the authors convert this implicit field to a surface. Did you use Marching Cubes?

(7) In Equation 15, it seems the final derived SDF is defined on the ray and varies with different view directions. However, to extract a surface, the SDF should not be view-dependent.

(8) I find the writing style of the paper a bit tricky. The decrease in memory usage is inspired by ScaffoldedGS. Regarding the claimed surface reconstruction contribution (as another contribution), although the authors provide many mathematical equations, I struggle to grasp the intuition, and the motivation for these equations is not clearly provided.

Overall, while the results appear promising, I find it difficult to understand how it works, especially considering that several equations are incorrect and lack clear motivations. To me, it feels more like a mathematical wrapper. The storyline is disjointed, lacking intuitive analysis.

Minor issues:
Line 53: We -> we
Line 112: Sfm -> SfM
questions: questions: (1) In Line 292, what is the unit of “1, 0.1, 0.01, …”? 

(2) Is there any visualization results of the voxels? I am curious about its quality.
limitations: limitations: As discussed by the authors in the appendix, the limitations include the inability to reconstruct dynamic scenes and the challenge of distinguishing distant regions without priors. These are interesting for future work to explore.
paper_summary: paper_summary: The paper introduces Gaussian Voxel Kernel Functions (GVKF), a novel approach for efficient 3D surface reconstruction in open scenes, leveraging 3D Gaussian Splatting.

The approach aims to combine the strengths of both implicit representations (NeRFs, Neural SDFs) and explicit representations (2D/3D Gaussian Splatting) to achieve an accurate and fast reconstruction method with low VRAM requirements.

**Details**

1. *Compressing 3D Gaussians Using a Sparse Voxel Grid and MLPs*

The paper proposes optimizing neural 3D Gaussians stored in a sparse voxel grid, reminiscent of Scaffold-GS and Octree-GS. The voxel grid is initialized using the downsampled SfM point cloud output during camera calibration. Each voxel can generate a fixed number of Gaussians, limited to a small range around the voxel. Feature vectors are stored and optimized in the grid, alongside the positions of the Gaussians. To generate the rotation, scaling, opacity, and colors of a Gaussian bound to a voxel, the voxel feature is decoded by an MLP along with the position of the Gaussian. This helps compress the representation while leveraging the natural regularity and fitting power of MLPs. To control the number of Gaussians in the scene, the paper takes inspiration from Scaffold-GS, proposing to subdivide or prune voxels based on gradient accumulation in each voxel.

2. *Defining a Neural Opacity Field with Gaussians to Render Images* 

For rendering images, rather than using the vanilla 3DGS formula that relies solely on the opacity of the Gaussians for alpha-blending, the paper proposes explicitly computing the ray-Gaussian intersections and use them to convert Gaussian functions into small kernels. Using these kernels, the paper derives a continuous opacity density function defined along the rays. Similar to Gaussian Opacity Fields (GOF), the implicit opacity field defined with this approach allows for more geometry-accurate rendering and better alignment of the underlying neural Gaussians with the surface of the scene.

3. *Extracting Accurate Surfaces from Neural Opacity Fields* 

The paper provides mathematical descriptions of the relationship between the neural opacity field and the underlying surface of the scene, resulting in a formula that maps the opacity function to an SDF. A Marching Cubes algorithm can then be used to extract a surface mesh of the scene.

The authors conducted experiments on several challenging datasets, demonstrating that GVKF achieves high reconstruction quality, real-time rendering speeds, and efficient memory usage, outperforming existing techniques in the literature.
strength_weakness: strength_weakness: 1. The paper is extremely similar to Gaussian Opacity Fields (GOF). The rendering Equation 7 (describing the revisited alpha-blending using an increasing opacity function along rays) and opacity field Equation 8 are almost identical to Equations 8 and 9 from the GOF paper. The difference lies in the generation of Gaussians, as GOF does not use a sparse voxel grid with MLPs to decode Gaussian features. However, GOF achieves better results and shorter optimization times (see Table 1 of the GOF paper). Although the paper mentions GOF, it does not compare with it, even though the paper and code were available five weeks before the deadline. I understand that five weeks is a short period, so GOF could be considered concurrent work rather than preexisting work. However, the results from GOF raise important questions about the claims in the paper: GOF seems to perform better and faster using an almost identical approach, except that GOF does not use a sparse voxel grid representation but just a set of explicit Gaussians. Therefore, it seems the voxel grid is not needed for performance and may be useful only for compressing the representation. As no ablation in the paper compares the VRAM requirement and performance of the approach with or without a voxel grid of neural Gaussians, I would like to hear feedback from the authors about this: Does the voxel grid really help in decreasing the memory consumption of the approach? Doesn’t it just heavily decrease performance?

2. Following my previous point, some important ablations are missing in the paper to better validate several important claims. Specifically, the benefits of the voxel grid are questionable when compared with Gaussian Opacity Fields. Moreover, it is unclear why deriving an SDF from the opacity function is needed for applying Marching Cubes; wouldn’t it be possible to apply MC directly on the opacity field? Does the SDF really improve the quality of the reconstruction?

3. No limitations are provided in the main paper. I encourage the authors to move the limitations from the appendix to the main paper, as this is very important for further research.

**Comment**

I really like the main idea of the paper, as well as the overall approach. However, while being very similar to (but more complicated than) Gaussian Opacity Fields, GVKF achieves lower performance, which makes me question some claims in the paper. I believe the paper needs to better justify why each of its components is needed to support the different claims. I am very interested in hearing feedback from the authors about these points and am willing to increase my rating after more clarifications.
questions: questions: 1. How does the method perform without using a voxel grid? Is there a drop or improvement in rendering quality and surface reconstruction quality? What about the VRAM requirements?

2. Why not apply Marching Cubes directly to the opacity function? Does the SDF really improve the quality of the reconstruction?
limitations: limitations: No limitations are provided in the main paper. I encourage the authors to move the limitations from the appendix to the main paper, as this is very important for further research.
paper_summary: paper_summary: The paper presents interesting combintation of implicit and explicit representation that achieves efficient and high-fidelity open-scene reconstruction. Basiclaly, they combine the sparse voxel representation attached with per-voxel Gaussian splatting representation and proposes formulation for 3D surface reconstruction and volume rendering.
strength_weakness: strength_weakness: The geometric reconstruction is inferior to Neuralangelo, according to Table 3. Can you elaborate more on this? For example, the qualitative results are performed only against the explicit methods. I think the current method is strong enough, but the current paper does not clearly indicate when it fails, which makes the process appear weaker.

Minor comments
- Fonts in figure 2 can be larger.
questions: questions: - Even though the advantages of the proposed work will be prominent with open scenes, I think the formulation can be applied to general scenes, including objects. How will the performance gap (speed, quality, memory, etc.) be different in different settings? The results indicate that the performance is better for outdoor scenes and comparable to other works for indoor scenes (Table 6). Do you have any intuition what constraints reflect the performance gaps, and why do you emphasize reconstructing open scenes?

- line 121 function -> feature?
- Under Equation 5, $k_i$ is not clearly defined. Can you put definition of the ray-gaussian transfom?
- In equations 6, 7, and 8, is the index $i$ the same as $k$?
- Please make the embedded fonts in Figure 2 larger.
- I do not fully follow the derivation in Equation 11. Why do you put a square $-\rho^2(u)$?
- Figure 6 is hard to interprete. What are the main differences?
limitations: limitations: - The limitations are addressed in apppendix. I think the reconstruction of sky are intentionally blocked in Figure 4. I would appreciate such gap to be revealed - we can interpret the results as a trade-off to achieve more complete scene without holes.
paper_summary: paper_summary: This paper presents Gaussian Voxel Kernel Functions (GVKF) for 3D surface reconstruction. The authors establish a continuous signed distance function derived from discrete 3D Gaussians, achieving high-fidelity open-scene surface reconstruction. They claim that the proposed method has high reconstruction quality, real-time rendering speeds, and significant memory efficiency in both runtime and training time. However, these claims lack sufficient discussion or validation throughout the paper.
strength_weakness: strength_weakness: 1. In the abstract and contributions section, the paper claims significant memory savings. However, there is no discussion or validation related to memory consumption throughout the paper.

2. The statements in Table 1 lack justification and contradict the experimental results. It lists that implicit methods are good at rendering quality while explicit methods like SuGaR and 2DGS are not, which contradicts the results in Table 2. Also, stating that implicit methods have low mesh quality is inaccurate, as shown by state-of-the-art methods like Neuralangelo in Table 3.

3. The overall idea is very similar to GOF, which also establishes a transformation from 3D Gaussians to an opacity function. The regularizations are also the same as GOF, borrowed from 2DGS. The paper also adopts MC and MT for mesh extraction. Despite this, the paper does not compare to GOF or discuss it, even though the authors seem aware of it.

4. The mathematical formulation is relatively difficult to understand due to multiple misalignments in definitions. It also lacks sufficient detail on how one equation leads to another.

5. There are insufficient implementation details, making reproduction difficult. It is unclear if neural networks are used, as Figure 1 mentions a "Neural" Gaussian representation.
questions: questions: 1. In Eq (7), how is the integration over the variable $t$ performed? From my understanding, Eq (6) can be used for rasterization by indexing over $i$, but $t$ is a continuous variable along the ray in Eq (7). Do you need sample points on a ray?

2. How do you transform from Eq. (9) to Eq. (11)? Does $\rho(t)$ relate to $\rho(u)$, and if so, how?

3. In L35, the paper states that 2DGS requires a large number of Gaussian primitives, leading to significant GPU memory consumption. This seems to be a common issue in GS-based methods. How is this solved in the paper?
limitations: limitations: The paper does not include a limitation section.

Comments: title: Reply to Discussion
Discussion
Response
Answer to more Math questions
Additional comments
Response
comment: > Discussion-Q1: Memory reduction contribution

The memory reduction is resulted from our proposed voxel grids in Section 1. The voxels serve as spatial anchors, providing a more regulated, structured distribution of 3DGS. They also control the growth and splitting strategies.
1. The attributes of 3DGS such as opacity, color, rotation and scaling are encoded into voxel properties and decoded by several global MLPs when rendering, as shown in equation 2 in the main text. This implicit representation greatly saves the memory consumption of 3D gaussian primitives.
2. Voxel grids provide more structured and clearer 3DGS distributions in space. While in original 3DGS, the primitives are uncontrollable. As shown in Fig. 1 of our submitted rebuttal file.
3. Our gradient-accumulation-based voxel registration strategy controls the growth of 3D gaussian primitives effectively, compared to the heuristic growth and pruning strategy of the original 3DGS.

In addition, the Tab. 2 of main text also demonstrates the efficiency of our method in memory reduction. More in-depth analysis can be found in @epfR Q1, @sv7m Q3

> Discussion-Q2: "Taking the maximum values of each kernel function to perform discrete numerical integration for rendering" is not a novel contribution.

Currently, it appears that direct integration between 3DGS and NeRF is not feasible. If we use 3DGS, it **cannot** represent 3d continuous surface (e.g. SDF function). If we use 3d continuous surface representation, currently it must be optimized through **volume rendering**.

As we mentioned in our response to Q8, our goal is to combine the advantages of 3DGS rendering and the continuous representation in volume rendering. This requires us to solve two problems:
1. How can we represent the continuous opacity function $\rho(t)$ on a ray using discrete gaussian primitives?
2. How can we optimize this continuous opacity function $\rho(t)$ without using volume rendering?

Our contribution lies in finding a method to solve these two issues: 

**Firstly**, we consider the effects of 3D Gaussian primitives on the ray as 1D Gaussian kernel functions. Hence, the weighted sum of $\alpha_i$ of each gaussian primitive can represent the continuous opacity function $\rho(t)$ on the ray. As shown in equation 5 in the main text. This is the defination of kernel regression.

**Secondly**, to optimize the continuous function $\rho(t)$，we take the maximum values of each kernel function (including the coefficient $\alpha_i$) to perform discrete numerical integration for rendering, which is equivalent to 3DGS alpha blending. 


Please be clarified "equivalent to Gaussian splatting alpha blending" is NOT a trivial conclusion, but demonstrates our proposed method to optimize this continuous function makes sense and is mathematically correct. This indicates we propose a NEW solution, **instead of volume rendering**, to optimize continuous 3D surface.

> Discussion-Q3: Why authors refer to the method as 'kernel regression.' Kernel regression is not equivalent to 'differentiable rendering with Gaussian kernels.'

It is notable that the concept of kernel regression is used to represent continuous functions, rather than to depict the rendering process. Our aim is to represent complex continuous function $\rho(t)$ using a simple set of Gaussian kernel functions. This aligns with the process of kernel regression: for each spatial point, opacity can be calculated through the weighted sum of $\alpha_i$, where the weights come from Gaussian kernel functions.

> Discussion-Q4: Lack of analysis explaining why a 'kernel regression' is necessary and how it enhances performance.

As mentioned in Discussion-Q3, kernel regression is the concept for describing how continuous function $\rho(t)$ formed along the ray. In terms of the reason of improvement, our performance benefits from **continuous representation**.

In the original 3DGS-based methods, such as 2DGS, discrete gaussian primitives are employed to fit surfaces, and mesh extraction relies on TSDF method. This approach may lead to holes in areas with sparse gaussian primitives or occlusions. The produced meshes are often overly smooth, which hinders the representation of fine details. In contrast, with our **continuous representation of kernel regression**, we obtain more details on the surface and have less holes, as shown in Fig. 4 and Fig. 5 of main text as well as our project page.

> Discussion-Summary

We highly value your feedback on our writing style and are dedicated to enhancing the clarity and flow of our paper. 
We sincerely hope the reviewers will recognize our efforts and overall contributions. Thank you once again for your valuable time and constructive feedback.
First, I want to express my appreciation for your efforts in clarifying the equations and improving the overall clarity of the description. However, I still have concerns regarding the positioning and motivation of the paper.

- My initial question regarding the claimed memory reduction contribution in the abstract remains unaddressed.
- The statement, 'We take the maximum values of each kernel function to perform discrete numerical integration for rendering,' seems to lead to a somewhat trivial conclusion, which is 'equivalent to Gaussian splatting alpha blending.' I struggle to see this as a novel contribution.
- I am also confused as to why the authors refer to the method as 'kernel regression.' Kernel regression is not equivalent to 'differentiable rendering with Gaussian kernels.' 
- Furthermore, I personally feel that the experiments are disconnected from the overall narrative of the paper. While the experiments demonstrate that the proposed method outperforms the state-of-the-art (SoTAs), there is a lack of analysis explaining why a 'kernel regression' is necessary and how it enhances performance.

However, I would not insist on rejecting the paper. The performance is excellent, and the experimental results are strong. My concern is primarily with the writing style, which I personally do not favor.


As a result, I am inclined to maintain my original score.
I would like to thank the authors for the detailed rebuttal and very clear answers to my questions.

I recommend acceptance for the paper, as I think it brings interesting contributions to the field of mesh reconstruction using 3DGS-based approaches. However, although I appreciate the detailed mathematical framework presented in the paper to support its claims, I think the proposed approach overlaps considerably with the existing work Gaussian Opacity Fields (GOF). 

Consequently, I decide not to raise my rating, but maintain a Weak Accept.
Thank you for the detailed explanation. While I appreciate the effort, I find that I still struggle to fully grasp some of the equations. I would like to raise the score and lower my confidence level.
I read the rebuttal and other reviews. My questions are all answered, which I truly appreciate.
Thanks for your in-depth analysis, this helps us a lot for the revision.
> Q-1 Kernel Function for Ray Intersections

In general, the Gaussian kernel function does not take into account the coefficient in front of the exponential, like $e^{-\gamma t^2}$. This means that with different values of $\gamma$, the Gaussian kernel function is not a strict normal distribution. For simplicity, we have retained the general form of the kernel function in our Equation 5. When considering ray gaussian intersection along the ray, the probability density influence of the Gaussian kernel on the ray is a strict normal distribution function $\frac{\sqrt{k_i}}{\sqrt{\pi}}e^{-k_it^2}$. In addition, the Gaussian kernel itself should also have a constant opacity $\beta_i$. To be precise, we have $\alpha_i=\beta_i*\frac{\sqrt{k_i}}{\sqrt{\pi}}$. From the perspective of yours, the missing $G^{2d}$ should be $\frac{\sqrt{k_i}}{\sqrt{\pi}}$, it is already included in $\alpha_i$.


> Q-2 Volume Rendering Formulation (NeRF vs. 3DGS)

I can mainly understand your question. To clarify, please do not confuse our opacity $\rho(t)$ and volume density $\sigma(t)$ in NeRF. In the main text, we do not use any symbol or concept like $\sigma(t)$.


Firstly, we would like to clarify some basic concept for better explanation:

**Volume rendering (NeRF)**
$$
C=\sum_{i=1}^N T_i(1-\exp(-\sigma_i\delta_i))c_i, \quad T_i=\exp(-\sum_{j=1}^{i-1}\sigma_j\delta_j)
$$
which can be written as:
$$
C=\sum_{i=1}^{N}T_i \cdot \alpha_i \cdot c_i, \quad   \alpha_i=(1-\exp(-\sigma_i\delta_i)), \quad T_i=\prod_{j=1}^{i-1}(1-\alpha_i)
$$
These are illustrated in equation 1 and equation 2 of original 3DGS paper.

**3DGS rendering**

This is the rendering of 3DGS in original 3DGS paper:
$$
C=\sum_{i=1}^N=c_i \cdot \alpha_i \cdot \prod_{j=1}^{i-1} (1-\alpha_j)
$$

**Relationship of NeRF and 3DGS rendering**

As we can see, the only difference between them is: $N$ in volume rendering is larger. Hence the goal of 3DGS and volume rendering is consistent and the difference is sampling resolution. This is also the fundation of 3DGS rendering equation in original 3DGS paper. We also illustrated this in Fig 3 of rebuttal PDF file.

**Please note**
1. The opacity $\alpha_i$ is the function of volume density $\sigma_i$ and sampling interval $\delta_i$ in volume rendering, which is the **accumulated result in a sampling interval**
2. In NeRF, dense sampling is used for better record $\sigma$, rather than opacity $\alpha$
3. The concept of volume density $\sigma$ does not appear in 3DGS and our paper, and rebuttal PDF file. We directly using guassian primitives to represent opacity $\alpha$, rather than volume density $\sigma$
4.  Our goal is to find continuous opacity $\alpha$ along the ray, represented as $\rho(t)$, rather than volume density $\sigma(t)$


**Answer your question**
- *"Gaussian points are treated as sampling points for a continuous version of volume rendering"*

This is right, and as analysed above, this is the fundation of 3DGS rendering equation.

- *"The sampling interval between these points should not be omitted in the quadrature of volume rendering"*

This is not right, $\alpha_i$ already records the accumulated influence to the ray in a short or long sampling interval, hence $\alpha_i$ should not multiply any $\delta$ again; 

- *"Why omitting the $\Delta t$ factor"*

If we use kernel function to represent continuous volume density $\sigma(t)$, rather than $\rho(t)$ in our paper, the $\Delta t$ should be included to calculate opacity. However, this will make things trouble. 

> Q3 Density Along the Ray

**Contribution of preceding gaussians**

In the equation 7, the contribution of preceding gaussian kernels are reflected in the term $\prod_{j=1}^{i-1}(1-\alpha_j \cdot \mathcal{K_j}(0))$.  Hence any kernel before should not be included in current point.

**Correct sampling point**

Based on equation 5, the second sampling point should be like $K_1+K_2+...+K_N$, the linear combination of all kernel functions, rather than $K_2$ or $K_1+K_2$.

**Optimizing kernel fucntions**

Based on Equation 5, our continuous opacity function $\rho(t)$ is defined on linear combination of kernel functions, hence the best way to optimize it is to directly optimize the kernel function and linear coefficients, rather than the sampling point represented by all of the kernels. As shown in Equation 7, this way considers every kernel's contribution and the influence of preceding kernels, which is coherent with volume rendering euqation and 3DGS rendering equation in Q2
> Summarize

Thank you for your comments; they will greatly assist us in our future work. As part of our concurrent efforts, we acknowledge that GOF has achieved higher geometric quality in their rendering implementation than we have, and we recognize that this rendering approach was proposed earlier than ours. 

We will clarify in our paper that equations 5-8 are the main contributions of GOF. However, as pointed out in Q3, while this newly developed rendering method produces attractive results, it is still imperfect: the original GOF paper lacks a mathematical basis and requires extensive post-processing. We hope our analysis can better help readers understand this innovative rendering method and the improvements we have made in Section 3.3.

Thank you for your suggestions regarding Section 3.1; we will add more implementation details and provide a more in-depth analysis.

We apologize again for the misuse of mathematical formulas in our initial submission, but we assure readers that the revised formulas are correct, and we will thoroughly address your questions.

> Answer to Question

The equation 7 can be simplified as:
$$
C=\sum_{i=1}^N=c_i \cdot \alpha_i \cdot \prod_{j=1}^{i-1} (1-\alpha_j)
$$
Here we keep $\mathcal{K_i}(0)$ for the coherence with equation 5,8. This formulation also appears in the equation 3 of original 3DGS paper. Both equation 7 and equation 6 are valid, the difference is whether consider $\alpha_i$ as constant value when ray-gaussian intersection changes. For perfect aligning with equation 6, the equation 7 could be
$$
C=\sum_{i=1}^N c_i \cdot \beta_i \frac{\sqrt{k_i}}{\sqrt{\pi}} \cdot \mathcal{K_i}(0) \prod_{j=1}^{i-1}(1-\beta_j \frac{\sqrt{k_j}}{\sqrt{\pi}} \cdot \mathcal{K_j}(0)) 
$$
In this scenario, $\beta_i$ is a constant value, and does not change when ray-gaussian intersection changes. More details can be found in next comment.
After correcting some symbols, I have re-evaluated the submission and would like to share further thoughts on the current formulation. However, some concerns remain that make this paper difficult to accept in its current form.
1. **Kernel Function for Ray Intersections**: If a ray intersects with the 3D Gaussian, the kernel function should be expressed as $K_i(t)=G^{2d}\exp⁡(−k_i⋅t^2)$ instead of $K_i(t) = \exp(-k_i \cdot t^2)$. This discrepancy needs clarification. Can the authors explain why the 2D Gaussian factor was omitted, and how this impacts the accuracy of the formulation?


2. **Volume Rendering Formulation (NeRF vs. 3DGS)**: The paper attempts to establish a volume rendering approach using both a density-based (NeRF) formulation and an alpha-based (3DGS) formulation in Eq. 6 and Eq. 7. However, I believe this approach is inaccurate. Referring to Fig. 3 in the rebuttal PDF, if the Gaussian points are treated as sampling points for a continuous version of volume rendering, then the sampling interval between these points should not be omitted in the quadrature of volume rendering. The current formulation suggests that the kernel intersecting with the ray collapses to an infinitesimally small value, but this does not justify **omitting the $\Delta t$ factor**. Therefore, the transformation from Eq. 4 to Eq. 7 seems problematic. Could the authors clarify the rationale behind this omission and how it affects the overall rendering process?


3. **Density Along the Ray**: If the density along the ray is defined as $\rho(t) = \sum_{i}K_i(t - t_i)$, should the second sampling point be represented as $\rho_2(t) = K_1(t_2 - t_1) + K_2(t_2 - t_2)$ instead of just $K_2(t_2−t_2)$? The current formulation seems to *overlook the contribution of preceding Gaussian kernels* when calculating the density at subsequent points. Could the authors provide an explanation for this simplification and its implications on the accuracy of the density calculation?
Thank you for your response. I would like to share some of my thoughts on the current draft.
I have noticed that the paper bears a strong resemblance to Gaussian Opacity Fields (GOF). For instance, as pointed out by Reviewer epfR, Eq. 7 and Eq. 8 closely resemble Eq. 8 and Eq. 9 from the GOF paper. Additionally, the paper incorporates the same regularization as 2DGS (and mentions GOF in L268–L270). Furthermore, the paper adopts MC/MT for meshing, though it lacks sufficient details to ascertain whether the MC/MT implementation is the same as GOF. Based on the table in Q3, it appears that the current meshing implementation is almost identical to GOF w/o refinement (MT is a main contribution from GOF). These observations raise questions about whether the current method was developed concurrently and independently.

Although the method achieves better PSNR with significantly fewer parameters in MipNeRF 360, the author should provide more emphasis on how this gain is achieved, such as the significantly higher PSNR (25.47) achieved with 10x fewer parameters. It should also be carefully checked that there is no test-set pollution.

Moreover, there are several important details missing, such as the settings of MLP and meshing, making reproducibility difficult. Additionally, the presentation is poor, including numerous errors in symbols and tables in the initial submission, which complicates the evaluation process. Despite the author's rebuttal commitment to major revisions, I believe that if a submission requires numerous revisions and further confirmation by the reviewer, it would be better suited for resubmission. Consequently, I maintain my rating for rejecting the submission.


**Additional Questions**:

If I understand correctly, then the correct form of Equation 7 is 
$C = \sum_{i}^{N}c_i \cdot \alpha_i \prod_{j=1}^{i-1}(1-\alpha_j)$ since $K(0) = 1$? If so, how is it consistent with the form of origin 3DGS rendering (Eq 6)? Also, should Equation 8 also be corrected?