Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Set

Authors: wenyuan  zhang
yu-shen  liu
zhizhong  han
shen  liu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-06 10:01:20

Abstract: It is vital to infer a signed distance function (SDF) for multi-view based surface reconstruction. 3D Gaussian splatting (3DGS) provides a novel perspective for volume rendering, and shows advantages in rendering efficiency and quality. Although 3DGS provides a promising neural rendering option, it is still hard to infer SDFs for surface reconstruction with 3DGS due to the discreteness, the sparseness, and the off-surface drift of 3D Gaussians. To resolve these issues, we propose a method that seamlessly merge 3DGS with the learning of neural SDFs. Our key idea is to more effectively constrain the SDF inference with the multi-view consistency. To this end, we dynamically align 3D Gaussians on the zero-level set of the neural SDF, and then render the aligned 3D Gaussians through the differentiable rasterization. Meanwhile, we update the neural SDF by pulling neighboring space to the pulled 3D Gaussians, which progressively refine the signed distance field near the surface. With both differentiable pulling and splatting, we jointly optimize 3D Gaussians and the neural SDF with both RGB and geometry constraints, which recovers more accurate, smooth, and complete surfaces with more geometry details. Our numerical and visual comparisons show our superiority over the state-of-the-art results on the widely used benchmarks.

Main paper: 
 introduction: 
 3D scene representations are important to various computer vision applications, such as single or multi-view 3D reconstruction [39, 42, 67, 72] , novel view synthesis [1, 71] , and neural SLAM [22, 32, 78, 24] etc.. Mesh and point clouds are the most common 3D scene representations, and can be rendered by fast rasterization on GPUs. Instead, more recent neural radiance fields (NeRFs) [49] are continuous scene representations, but it is slow to render NeRFs due to the need of costly stochastic sampling along rays in volume rendering. More recently, 3D Gaussians with different attributes like color and opacity are used as a versatile differentiable volumetric representation [30, 77, 23] for neural rendering through splatting, dubbed 3D Gaussian Splatting (3DGS). It prompts the pros of both NeRFs and point based representations, which achieves both better quality and faster speed in rendering. Although 3D Gaussians can render plausible images, it is still a challenge to reconstruct surfaces based on the 3D Gaussians.The key challenge comes from the gap between the discrete 3D Gaussians and the continuous geometry representations, such as implicit functions. Besides the discreteness, the sparseness caused by uneven distribution and the off-surface drift make 3D Gaussians even harder to use than scanned point clouds in surface inference. To overcome these obstacles, recent solutions usually add previous volume rendering based reconstruction methods [60, 50] to 3DGS as a complement branch [44, 66, 11] , use monocular depth and normal images as priors to bypass the messy and unordered 3D Gaussians [14, 57] , or use surface-aligned Gaussians [25, 68, 14] in rasterization to approximate surfaces. However, how to learn continuous implicit representations to recover more accurate, smooth, and complete surfaces with sharp geometry details is still an open question.To answer this question, we introduce a novel method to infer neural SDFs from multi-view RGB images through 3D Gaussian splatting. We progressively infer a signed distance field by training a neural network along with learning 3D Gaussians to minimize rendering errors through splatting. To more effectively constrain the surface inference with the multi-view consistency, we dynamically align 3D Gaussians with the zero-level set of the neural SDF, and render the aligned 3D Gaussians on the zero-level set by differentiable rasterization. Meanwhile, we update the neural SDF by pulling the neighboring space onto the disk determined by each 3D Gaussian on the zero-level set, which gradually refines the signed distance field near the surface. The capability of seamlessly merging neural SDFs with 3DGS not only get rid of the dependence of costly NeRFs like stochastic sampling on rays but also enables us to access the field attributes like signed distances and gradients during the splatting process, which provides a novel perspective and a versatile platform for surface reconstruction with 3DGS. The key to the 3D Gaussian alignment and neural SDF inference is a differentiable pulling operation which uses the predicted signed distances and gradients from the neural SDF. It provides a way of imposing geometry based constraints on 3D Gaussians besides the RGB based constraints through splatting. Our numerical and visual evaluations on widely used benchmarks show our superiority over the latest methods in terms of reconstruction accuracy and recovered geometry details. Our contributions are listed below,• We propose to infer neural SDF through splatting 3D Gaussians pulled on the zero-level set, which can more effectively constrain surface inference with the multi-view consistency. This enables to recover more accurate, smooth, and complete surfaces with geometry details.• We introduce to dynamically align 3D Gaussians to the zero-level set and update the neural SDF through a differentiable pulling operation. To this end, we propose novel loss terms and training strategies to work with the discrete and sparse 3D Gaussians in surface reconstruction.• We achieve the state-of-the-art numerical and visual results in multi-view based surface reconstruction. 
related work: 
 Neural implicit representations have achieved remarkable progress in reconstructing 3D geometry with details [53, 48, 13, 45, 7, 21] . Neural implicit functions can be learned by either 3D supervisions, such as signed distances [53, 13, 41] and binary occupancy labels [48] , or 2D supervisions, such as multi-view RGB images [60] and normal images [5] . In the following, we focus on reviewing methods of learning implicit representations from 2D and 3D supervisions separately. Then we provide a detailed discussion on the latest reconstruction methods based on 3D Gaussians.Neural radiance fields (NeRFs) [49] have become an essential technology for representing 3D scene through multi-view images. Many of its applications have been explored, resulting in significant advancements in areas such as acceleration [50, 12] , dynamic scene [16, 4] and sparse rendering [58, 26] . Besides these applications, extracting accurate surfaces from NeRFs remains a challenge. Mainstream approaches typically design various differentiable formulas to transform the density in radiance fields into implicit representations for volume rendering, such as signed distance function (SDF) [60, 39, 54] , unsigned distance function (UDF) [42, 47, 15, 70] and occupancy [52] . With the learned implicit function fields, post-processing algorithms [43, 21, 48] are applied to extract the zero level set to obtain the reconstructed meshes. Following methods introduce different priors from SfM [18, 69] or large-scale datasets [67, 59, 40] to improve the reconstruction performance in large-scale scenes. Recent approaches focus on speeding up the neural rendering procedure, aiming to achieve high-quality meshes and rendering views within a short period of training time. They propose alternative data structures to replace the heavy MLP framework used in original NeRF, such as sparse voxel grid [17] , multi-resolution hash grid [50, 61] and radial basis function [12] , or design subtle differentiable rasterization pipelines to achieve real-time rendering [64, 56] . However, these methods still face the trade-off between rendering quality and training speed.Since DeepSDF [53] and OccNet [48] were proposed, learning implicit representation from point clouds has achieved remarkable results in geometry modeling. These methods use ground truth signed distances and binary occupancy labels calculated from ground truth point clouds as supervisions to learn the implicit representation of shapes. The supervisions can serve as different kinds of global priors [3, 42, 55, 36] and local priors [62, 28, 38, 6] , which enables the neural implicit function to better capture geometry details and generalize to unseen shapes during inference. Some other methods infer SDFs without 3D supervisions. They train neural networks to overfit on single point clouds. These methods introduce additional constraints [19, 76] , novel ways of using gradients [45, 74, 51, 35, 75] , specially designed priors [46, 10, 9] and normals [2, 37, 34] to estimate signed or unsigned distances and occupancy, which use point clouds as a reference.3D Gaussian Splatting [30] has become a new paradigm in neural rendering due to its fast rendering speed, intuitive explicit representation and outstanding rendering performance. However, reconstructing accurate surfaces from 3D Gaussian remains a challenge due to the messy, noisy, and unevenly distributed 3D Gaussians. To solve this problem, one kind of approaches involves combining 3D Gaussians with neural implicit surface functions [60, 50] to enhance the performance of both branches, which employs mutual supervisions between the two components [66, 11, 44] . Another kind of approaches encourage the reduction from 3D Gaussians to 2D Gaussians with a series of regularization terms, which ensures the Gaussian primitives to align with the object surfaces [25, 20, 14] . Additionally, some methods introduce additional priors from large-scale datasets [57, 14] or multi-view stereo [63] , or use elaborately designed surface extraction algorithms [68, 65] to recover 3D geometry from 3D Gaussians. Although these efforts have achieved improved reconstructions, they are still limited in capturing fine-grained geometry and lack the precise perception of continuous implicit representations. Different from all these mentioned methods, we propose to seamlessly combine 3D Gaussians with the learning of neural SDFs. Our method provides a novel perspective to jointly learn 3D Gaussians and neural SDFs by more effectively using multi-view consistency and imposing geometry constraints. 
method: 
 Overview. We aim to infer a neural SDF f from posed multi-view RGB images {v i } I i=1 , as shown in Fig. 1 . We learn 3D Gaussian functions {g j } J j=1 with their attributes like color, opacity, and shape to represent the geometry and color in the 3D scene. Meanwhile, when learning the 3D Gaussians, we introduce novel constraints to infer the continuous surfaces with the neural SDF. We rely on a differentiable pulling operation and the differentiable rasterization to bridge the gap between the discrete Gaussians and the continuous neural SDF, align 3D Gaussians on the zero-level set of the neural SDF, and back propagate the supervision signals from both the rendering errors and other geometry constraints to jointly optimize 3D Gaussians and the neural SDF.Neural Signed Distance Function. We leverage an SDF f to represent the geometry of a scene. An SDF f is an implicit function that can predict a signed distance s at an arbitrary location q, i.e., s = f (q). Recent methods usually train a neural network to approximate an SDF from signed distance supervision or infer an SDF from 3D point clouds or multi-view images. A level set is an iso-surface formed by the points with the same signed distance values. The zero-level set is a special level set, which is formed by points with a signed distance of 0. We can use the marching cubes algorithm [43] to extract the zero-level set a a mesh surface. Another character of the zero-level set is that the gradient of the SDF f at query q on the zero-level set, i.e., ∇f (q), is the normal of q. 3D Gaussian Splatting. 3D Gaussians have become a vital differentiable volume representation for scene modeling. We can learn a set of 3D Gaussians {g j } J j=1 , each of which has a set of learnable attributes including mean, variances, rotation, opacity, and color. We can render the learnable Gaussians {g j } into RGB images through the volume rendering equation below,EQUATIONwhere C ′ (u, v) is the rendered color at the pixel (u, v), c i , o i , and p i denote the color, the opacity, and the 2D projection of the j-th 3D Gaussian, respectively. At a query q = [x, y, z], the probability from the j-th 3D Gaussian is p j (q) = exp(-0.5 * (q -µ j ) T -1 (q -µ j )), where µ j is the center of the j-th Gaussian, and is the covariance matrix.We can learn these 3D Gaussian functions through a differentiable rasterization. We render 3D Gaussians {g j } into rendered RGB images v ′ i , and then, optimize the learnable attributes by minimizing the rendering errors to the ground truth observations v i , where C ′ (u, v) and C(u, v) are the rendered and the GT color values at pixel (u, v), i.e., min {gj } ||C ′ (u, v) -C(u, v)|| 2 2 . Aligning 3D Gaussians with the Zero-level Set. Since 3D Gaussian splatting is so flexible in volume rendering, it does not require 3D Gaussians to locate on the geometry surface for good rendering quality. While we expect 3D Gaussians to locate on geometry surface, so that we can more effectively leverage them and multi-view consistency as clues to infer more accurate neural SDFs for reconstruction. To this end, we introduce a differentiable pulling operation to pull 3D Gaussians on the zero-level set of the neural SDF f , and then, we render the pulled 3D Gaussians through the splatting.Specifically, inspired by Neural-Pull [45] , we rely on the gradient field of the neural SDF f during the pulling operation. We move each one of the 3D Gaussians g j using the predicted signed distance s j = f (µ j ) and the gradient ∇f (µ j ), where µ j is the mean value of the 3D Gaussian. As shown in Fig. 1 (a) , this pulling operation will turn the 3D Gaussian g j into a 3D Gaussian g ′ j that get projected onto the zero-level set of SDF f , where g ′ j shares the same attributes with g j but has a different center µ ′ j ,EQUATION)Signed Distance Inference with Pulled 3D Gaussians. We infer signed distances in the field with pulled 3D Gaussians {g ′ j }. Pulled 3D Gaussians provide a coarse estimation of the surface, which we can use as a reference. One challenge here is that the sparsity and non-uniformly distributed 3D Gaussians do not show a clear geometry clue for surface inference. Although previous methods like NeuralTPS [8] and OnSurfPrior [46] manage to learn continuous implicit functions from sparse points, it is still difficult to recover surfaces from both sparse and non-uniformly distributed points.To overcome this challenge, we introduce an approach to estimate neural SDFs from sparse 3D Gaussians. Like Neural-Pull [45] , we still use a differentiable pulling operation to pull neighboring space onto the surface but we regard the disk established by the shape of a 3D Gaussian as a pulling target, rather than a point, as shown in Fig. 2 , which aims for a larger target on surfaces. To this end, we impose constraints not only on the shape of 3D Gaussians but also on the pulling operation. Specifically, we introduce three constraints. The first one constrains 3D Gaussians to be a thin disk. The second constraint encourages the thin disk to be a tangent plane on the zero-level set. The third constraint pushes queries to get pulled onto the thin disk along the normal of the Gaussian.The first constraint adds penalties if the smallest variance among the three variances of a 3D Gaussian g j is too large, as shown in Fig. 1 (c ). Thus, the loss for a thin disk Gaussian is listed below,EQUATIONwhere r 1 , r 2 , and r 3 are variances along the three axes. Flattening a 3D Gaussian ellipsoid into a disk was first introduced in NeuSG [11] and has become a consensus in recent Gaussian reconstruction works [25, 14] . The motivation is that 2D planar disk primitives are more suitable for surface representation, making it easier to apply alignment constraints. Additionally, we can naturally use the direction pointing along the axis with the minimum variance r = min{r 1 , r 2 , r 3 } to represent the normal n j of the Gaussian g j .Based on the thin disk shape of Gaussians, the second constraint encourages the pulled Gaussians {g ′ j } to be the tangent plane on the zero-level set, as shown in Fig. 1 (d) . What we do is to align the normal n j of a Gaussian g j with the normal at the center µ ′ j of the pulled Gaussian g ′ j on the zero-level set. We use the gradient ∇f (µ ′ j ) of the neural SDF at µ ′ j as the expected normal here. Hence, we align the normal n j of a Gaussian with the normal ∇f (µ ′ j ) on the zero-level set, With the disk-like Gaussians located on the tangent plane, we introduce to sense the signed distance field by pulling randomly sampled queries on the Gaussian disks, as shown in Fig. 1 (b ). Turning the pulling target from a point [45, 8] into a plane is based on the observation that the 3D Gaussian function with a boundary can cover the surface more completely although their centers {µ 1 , ..., µ j } which are sparse and non-uniformly distributed. Thus, we expect the operation can pull a query onto a Gaussian disk plane. Fig. 2 demonstrates the improvement of pulling queries onto their nearest Gaussian disk planes over the nearest Gaussian centers. The comparisons show that pulling onto the disk plane can improve the robustness to the sparsity and non-uniformly Gaussian distribution. With learned Gaussian centers, pulling queries to centers can not recover the smooth and continuous geometry in areas where almost no Gaussian centers appear. While pulling queries to the Gaussian disk plane can recover more accurate and complete surfaces since the disk established by the learned variance of Gaussian functions can mostly cover the gap.L T angent = 1 - ∇f (µ ′ j ) |∇f (µ ′ j )| • n j (4)Specifically, at a query q, we pull it onto the zero-level set using a similar way in Eq. 2, i.e., q ′ = q -s * ∇f (q)/|∇f (q)|. To encourage the query to get pulled onto the nearest pulled Gaussian disk, we maximize its probability of belonging to its nearest pulled Gaussian ḡ which is determined in terms of the distance between q and the Gaussian center μ,EQUATIONWe minimize the negative logarithm of the probability in our implementation. Moreover, we expect the pulling can follow a direction orthogonal to the disk plane, which leads to the minimum moving distance conform to the definition of signed distances. To this end, we impose another constraint on the gradient to ensure that the pulling can follow a path with the minimum distance to the nearest pulled Gaussian disk, as shown in Fig. 1 (e) ,EQUATIONwhere the constraint aligns the gradient at query q and the normal nj of the pulled Gaussian disk ḡ.Rendering. We also render the pulled Gaussians {g ′ j } into images through splatting to add penalties on rendering errors, where {g ′ j } are Gaussians pulled onto the zero-level set from the Gaussians {g j } by the neural SDF f in Eq. 2. Each pair of g j and g ′ j shares the same attributes expect the center location. The rendering error combines an L 1 term and a D-SSIM term between rendered images {v ′ i } I i=1 and ground truth ones {v i } I i=1 , following original 3DGS [30] ,EQUATIONLoss Function. We optimize attributes of Gaussians {g j } and the parameters of neural SDF f by the following objective function, where α, β, γ, and δ are balance weights.EQUATIONImplementation Details. Our code is build upon the source code released by 3DGS [30] . Similar to [68] , we make some changes to 3DGS's densification strategy. The first one is to initialize the newly cloned Gaussians around the original Gaussians rather than at the same positions. The second one is to encourage 3DGS to split larger Gaussians into smaller ones more frequently. These strategies aim to increase the number of primitives and to avoid underfitting in textureless areas. Regularization parameters are set to α=100, β=0.1, γ=1, δ=0.1. We optimize our model for a total of 15k iterations.We stop densification and incorporate the pulling and constraints at 7k iterations. The SDF network is implemented as an MLP with 8 layers, 256 hidden units and ReLU activation function, and initialized as a sphere, following [45] . The parameters of the SDF network shares the same optimizer as that of 3D Gaussians. All the experiments are conducted on a single NVIDIA 3090 GPU. 
experiment settings: 
 Evaluation Metrics and Datasets. We evaluate the performance of our method on widely adopted datasets including both object-level and large-scale ones, including DTU [27] , Tanks and Temples (TNT) [31] and Mip-NeRF 360 (M360) [1] . To evaluate the accuracy of the reconstructed meshes, we use Chamfer Distance (CD) on DTU and F-score on TNT, using the official evaluation script. To evaluate the rendering quality in real-scene datasets, we report PSNR, SSIM and LPIPS in evaluations on M360.Baselines. We compare our geometry reconstruction accuracy with the state-of-the-art 3DGS based reconstruction methods, including SuGaR [20] , DN-Splatter [57] , GaussianSurfels [14] and 2DGS [25] . For real-world scenes which do not have ground truth meshes for evaluations, we compare the rendering quality with state-of-the-art neural rendering methods, including Instant-NGP [50] , Mip-NeRF 360 [1] and BakedSDF [64] .Surface Extraction. An advantage of our approach over the latest methods is the simplicity of extracting surfaces. Different from methods like SuGaR [20] and GauS [65] which introduce specially designed algorithms and take a long time for extracting surfaces, we adopt the marching cubes algorithm [43] to extract mesh surfaces with the learned neural SDF f . For small scale scenes, we use a resolution of 800 to extract surfaces, while we split large scale scenes into parts, each of which gets reconstructed with a resolution of 800 to bypass the limitation of our computational resources.Comparisons on DTU. We report accuracy of reconstructed meshes and training time against baselines on DTU dataset in Tab. 1. Our method outperforms all Gaussian-based reconstruction methods in terms of Chamfer Distance. Our method achieves comparable training time to the stateof-the-art Gaussian-reconstruction method 2DGS [25] but gains better reconstruction accuracy than 2DGS. The visualization results in Fig. 3 highlight the advantages of our method. By employing alignment constraints and pulling operations between the 3D Gaussians and the neural SDF field, we can reconstruct significantly smoother and more complete surfaces than the baselines.Comparisons on TNT. We further evaluate our method using more challenging large-scale unbounded scenes on TNT dataset. Numerical comparisons in Tab. 2 show that we achieve higher F-score compared to baseline methods, even surpassing NeuS, which however takes about 12 hours to fit a scene. Notably, as the scene scale increases, the number of Gaussian primitives increases rapidly, causing the adjusted CUDA rasterization kernel of 2DGS to consume more time for rendering. In contrast, since our rasterization kernel is based on 3DGS, it is less sensitive to the number of Gaussians, which enables us to learn 3D Gaussians faster than 2DGS. We provide visual comparisons in Fig. 4 . Here we crop the reconstructed meshes to show the foreground objects that are of primary interest, as captured by the cameras. Please refer to the appendix for the reconstruction results of the background regions. The visual comparisons demonstrate that we can reconstruct more complete and smooth object surfaces, such as the ground, the truck's hood and the statue's left shoulder. Comparisons on MipN-eRF 360. We further evaluate our method in neural rendering for novel view synthesis on MipNeRF 360 dataset.We report the numerical comparisons in Tab. 3. Our competitive results against the state-ofthe-art novel view synthesis methods indicate that our method is able to impose effective geometric constraints without compromising rendering quality. This provides a promising solution for learning continuous distance fields from discrete 3D Gaussians. Visual comparisons of mesh reconstructions are shown in Fig. 5 , which demonstrate that our method is able to recover more smooth and complete surface by more effectively using the multi-view consistency. In this section, we conduct ablation studies on the key techniques of our method to demonstrate their effectiveness. The full quantitative results are reported in Tab. 4, which are conducted on all scenes in DTU dataset [27] .Pulling Gaussians w/o Pulling Gaussians (a) Visualization of Gaussian centers with or without pulled onto zero-level set. We are able to obtain consistent and smooth Gaussian distributions by pulling operation. Pulling Operations. We first examine the effect of pulling Gaussians onto the zero-level set, as reported in Tab. 4 ("w/o Pull Gaussians" vs. "Ours"). The original 3DGS tends to produce floating ellipsoids near the object surfaces to overfit the training views. By pulling the Gaussians to the zero-level set of the SDF field, the Gaussians are consistently distributed on the surface. As shown in Fig. 6a , after getting pulled onto the zero-level set, the Gaussian centers are distributed on a thin layer of the object surface, thus achieving an accurate geometry estimation. Meanwhile, we pull neighboring space onto Gaussian disks to learn neural SDFs. Comparing to NeuralPull [45] which pulls query points to centers, we innovatively pull query points to Gaussian disks, which bridge the gap between continuous SDF field and sparse Gaussian distributions, as highlighted in Fig. 2 and Tab. 4 ("Pulled to centers" vs. "Ours").Constraint Terms. We further explore the effect of our constraint terms, as reported in Tab. 4 ("Constraint Terms"). Our full model provides the best performance when applying all constraint terms. The orthogonal loss helps to learn a more regularized SDF field, while the thin loss and tangent loss provide constraints to align the orientation of Gaussian disks with the gradient of neural SDF on the zero-level sets, resulting in a good normal field and a reconstructed mesh, as shown in Fig. 6b, 6c .Mesh Extraction. We also report the reconstruction accuracy using TSDF fusion and screened Poisson reconstruction [29] , as shown in Fig. 6d and Tab. 4 ("Mesh Extractions").For TSDF fusion, we render depth maps and fuse them using a voxel size of 0.004 and truncation threshold as 0.02, the same as 2DGS [25] . For screened Poisson, we use the Gaussian centers and normals as input. Unlike 2DGS [25] and GSurfels [14] which incorporate rendered depth into the differentiable rasterization pipeline, we do not directly optimize depths, resulting in noisy depth maps and unsatisfactory reconstruction results. However, since the positions and normals of the Gaussians are well optimized through our approach, screened Poisson reconstruction can achieve relatively good results. Gradient Constraint. We follow Neural-Pull [45] to use normalized SDF gradient for pulling operation. We report the result with an additional Eikonal term [19] to explicitly constrain the gradient length, as shown in Fig. 7 .The result is significantly degenerated because that Neural-Pull depends on both predicted SDF values and gradient directions to optimize the SDF field. It makes the optimization even more complex when adding additional constraint on the gradient length.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: We provide the detailed information in reproducing our methods in Section 3 of the main paper. 
conclusion: 
 We propose a method to learn neural SDFs for multi-view surface reconstruction with 3D Gaussian splatting. Our results show that we can more effectively leverage multi-view consistency to recover more accurate, smooth, and complete surfaces with geometry details by rendering 3D Gaussians pulled on the zero-level set. To this end, we dynamically align 3D Gaussians to the zero-level set and update neural SDFs through both differentiable pulling and splatting for both RGB and geometry constraints. Our methods successfully refine the signed distance field near the surface in a progressive manner, leading to plausible surface reconstruction. Our ablation studies justify the effectiveness of our novel modules, loss terms, and training strategies. Our evaluations show our superiority over the latest methods in terms of accuracy, completeness, and smoothness.• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
acknowledgement: 
 This work was supported by National Key R&D Program of China (2022YFC3800600), and the National Natural Science Foundation of China (62272263, 62072268), and in part by Tsinghua-Kuaishou Institute of Future Media Data. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the applications and potential impacts of our method in the introduction.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification:We use the open-sourced datasets and codes under their licenses. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: A.1 Background Reconstruction Since our SDF field was learned by fitting Gaussian ellipsoids, it can infer implicit surfaces at any location where Gaussians are distributed. Therefore, our method has the same capability to reconstruct backgrounds as methods like TSDF fusion, as shown in Fig. 8 . Current works generally utilize screened Poisson or TSDF fusion to reconstruct meshes [25, 20] and tend to reconstruct large sky spheres in the background. Our method learns neural SDFs and utilize marching cubes to reconstruct mesh, which avoid such bad cases. We provide a theoretical analysis here to demonstrate the advantage of pulling queries onto disks compared to pulling queries onto centers. We provide a visual comparison of the two strategies in Fig. 9 , showcasing the changes of the loss function and the loss gradients as the query point approaches the Gaussian center. As the query point gets closer to the Gaussian center, the loss function of "pulling to centers" decays at a constant rate, and the gradient of the loss remains constant. In contrast, for "pulling to disks", the loss function decreases quadratically, and the gradient of the loss gradually diminishes. This means that under the influence of the disk loss, as the query point approximates the center, the received gradient becomes smaller, reducing the driving force that pushes the query point towards the center. In other words, the disk loss has a higher "tolerance" for the query point not being pulled to the center. This explains why we can learn a continuous field from a sparse and non-uniformity distribution of Gaussian ellipsoids using the disk loss, whereas the center loss would lead to the SDF field overfitting to every Gaussian center.Reconstructed mesh Reference Image While our method successfully recovers accurate appearance and geometry reconstruction for a wide range of objects and scenes, it also has several limitations. Firstly, the neural SDF is seamlessly integrated with Gaussian ellipsoids, making it difficult to avoid the inherent drawbacks of original 3D Gaussians, such as the lack of transparent objects and areas with strong specular reflections. Secondly, although we address the issue of learning a continuous SDF field from sparse and non-uniformly distributed Gaussian ellipsoids by pulling query points to disks, our method shows limited performance in extremely sparse areas. In very distant regions of unbounded scenes or areas with colors similar to the background color, where 3DGS reconstructs no ellipsoids or only a few ellipsoids, our method tends to produce holes. Thirdly, due to the continuous and smooth characteristics of MLPs, our SDF tends to capture the low-frequency features of objects, making it difficult to reconstruct high-frequency details. A failure case is shown in Figure . 10, where we can reconstruct the very smooth tablecloth but fail to recover the details of the lego. There are two potential solutions for this issue in the future: one is to enhance the representation capability of the SDF by integrated with latest implicit representation learning methods, such as BACON [41] and GridPull [7] ; the other one is to dig into the capabilities of TSDF fusion and screened Poisson in reconstructing our SDF field, which have the ability to reconstruct arbitrary resolution details.We provide additional reconstruction results in Fig. 11 , 12, 13, which further justifies the superiority of our method. We notice that there are some holes on the flowerbed area in Fig. 13 . This is due to the overly complex geometric structures and a lack of view covering, thus emitting a significant under-fitting issue. This results in a set of extremely sparse, huge, and unevenly distributed Gaussians, which makes Gaussians are thick ellipsoid like shape rather than relatively thin plans, leading to poor sense of surface. Although these huge Gaussians may work well in rendering, but they cannot recover any geometry covered by them. How to control the size of Gaussians for SDF inference could be an interesting future work direction.We also visualize the error maps on meshes obtained by 2DGS and ours in Fig. 14 , which highlights our superiority in terms of the accuracy of extracted surfaces. The surfaces learned by 2DGS are usually fat and a little bit drift away from ground truth surfaces, although their meshes seem to show more details. Our method is able to capture more accurate surfaces by using 3D Gaussians pulled onto the zero-level set and pulling query points onto Gaussian disks at the same time, leading to much more accurate zero-level set. Large Error Small Error Figure 14 : Error maps between 2DGS and our method on TNT dataset.Learning SDF Learning UDF Justification: We discussed why pulling queries onto disks is better than pulling queries onto centers in the appendix. There is no theoretical theorems or lemmas in it, but We tried our best to give a theoretical analysis for this discussion.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 


Decision: This paper is a clear accept. It receives 2x accepts and 2x borderline accepts. The reviewers agreed that the proposed method is simple but effective with well-thought losses, the proposed method demonstrated state-of-the-art reconstruction results on the DTU and TNT datasets, with each component of the approach well-ablated. This work proposes an interesting method of combining a differentiable pulling operation with Gaussian Splatting for surface reconstruction to learn neural SDF for GS. The paper is also well-written and easy to follow. Although Reviewer SW8k initially raised several issues on lack of explanation, unsatisfactory results on TNT, inconsistency of quantitative results, lack of citation, etc, these concerns are well-addressed by the authors. The reviewer finally raised the score to borderline accept.

Reviews: review: paper_summary: paper_summary: The paper proposes a extension to 3D Gaussian Splatting by making it consistent with a neural SDF that is learnt along with the 3D Gaussians. To make it consistent, the Gaussians are projected to the zero level set, and the neural SDF is optimized to represent the SDF of the surface implied by the Gaussians. As the latter requires the Gaussians to be thin (essentially discs), they encourage this with a loss, and further encourage other properties that hold when the two representations are consistent.
strength_weakness: strength_weakness: - The proposed method uses a loss to make the Gaussians to be flat, why not directly use 2D Gaussians/Surfels? Those papers mention that direct parameterisation is much better than using a loss to flatten Gaussians, why does your method not suffer from issues from not having an exactly flat Gaussian? It would be interesting to have a comparison of your method with 2D Gaussians. However, I would expect at least some discussion of this (whether it is actually important to have exactly flat Gaussians or not). 
- While I like this way of constraining the Gaussians better than previous methods, the novelty is a bit limited since there are similar methods (improving GS using SDFs and having Gaussians be flat)
questions: questions: - For $L_{Tangent}$, how do you determine orientation of the normal?
- Ablations study results don't line up with each other and with TNT results (0.41 vs 0.42 vs 0.43 vs 0.46)? A little confused/concerned here.
- End of page 5, should $n_j$ be $\bar{n}_j$?
limitations: limitations: Addressed in the appendix/supplemental
paper_summary: paper_summary: This paper focuses on the challenge of inferring a signed distance function (SDF) for multi-view surface reconstruction from 3D Gaussian splatting (3DGS), which is hindered by the discreteness, sparseness, and off-surface drift of the 3D Gaussian. To overcome these challenges, the authors propose a method that seamlessly integrates 3DGS with the learning of neural SDFs. This approach constrains SDF inference with multi-view consistency by dynamically aligning 3D Gaussians on the zero-level set of the neural SDF and then rendering the aligned 3D Gaussians through differentiable rasterization. Through the utilization of both differentiable pulling and splatting, the approach jointly optimizes 3D Gaussians and neural SDFs with both RGB and geometry constraints, resulting in the generation of more accurate, smooth, and complete surfaces. Extensive experimental comparisons on various datasets demonstrate the superiority of the proposed method.
strength_weakness: strength_weakness: - It seems that the authors did not incorporate the Eikonal loss, which is commonly used to regularize SDF values in space during optimization. Could the authors provide specific reasons for this omission? Additionally, it is not entirely clear to me how the pull loss ensures that the optimized field satisfies the properties of SDF.

- In Eq8, the weight of the splatting loss appears to be substantially larger than the others. Could the authors explain how these weights are determined?

- I noticed that the densification operation and the pull/constraint losses are employed at different stages. How does the simultaneous use of densification and pull loss affect the final result? Stopping the densification operation in the second stage seems unreasonable because it may limit the performance of 3DGS.
questions: questions: See weakness for details.
limitations: limitations: The authors have thoroughly discussed the limitations of the proposed method. I believe that incorporating monocular priors during optimization would be beneficial to the learning of SDF. Additionally, I think that further exploration of the relationship between densification and SDF values would be an interesting and valuable research pursuit.
paper_summary: paper_summary: This paper combines 3D Gaussian Splatting and NeuralPull to extract surface. By this way, it can utilize the existing extracting method Marching Cubes algorithm to extract the zero-level set as a mesh surface.
strength_weakness: strength_weakness: 1. Lack of explanation of how this method can work. As we all know, NeuralPull needs point clouds as ground truth to let a neural SDF learn surface, while the pseudo ground truth provided by 3D Gaussians is noisy, so how can your method learn? In addition, based on your description, you jointly train the 3D Gaussian and the neural SDF and use Eq.2 to pull the 3D Gaussian onto the zero-level set. However, at the beginning of training, the neural SDF will not provide good guidance and even provide a poor direction leading to making the 3D Gaussian away from the surface, leading to catastrophe.

2. Unsatisfactory visual results on TNT. Although your F1-score is better than 2DGS, in Fig.14 your results look over smooth and lack details. This figure only shows your results, can you provide more comparisons with other methods, like 2DGS? Also, in Fig.15, why are there many holes in the mesh in the middle?

3. The results of 2DGS are worse than the original paper. For example, on the TNT dataset, 2DGS is 0.3 in the original paper, while it is 0.26 in Fig.2 of your paper. Also, the qualitative results of 2DGS are better than yours. In detail, the mesh shown in Fig.10 of 2DGS's paper has more details, which are better than yours, and 2DGS's results are shown in Fig.4 of your paper.

4. Your quantitive results are not consistent between Tab.2 and Tab.4 on TNT. In detail, in Tab.2, the full model F1-score is 0.43, while in Tab.4 it is 0.46, leading to suspicion of the accuracy of your results. Can you explain it?

5. Lack of correct citation. For example, your first constraint $L_{Thin}$ is actual $L_s$ in NeuSG, but you don't illustrate the origin.
questions: questions: Some questions have been listed in the weaknesses. I still have other questions here:
1. According to [1,2], SDF with Neural Implicit Functions (NIFs) can only reconstruct closed surfaces. The limitation prevents NIFs from representing most real-world objects. However, on the TNT dataset, some scenes are not closed surfaces, like 'Barn' with the ground, 'Meetingroom' with windows. How do you solve this problem using SDF with NIFs?

2. For Eq.4 and Eq.6, you use L1 loss to regulate vectors, however, it is more common to use the cosine distance function to regulate vectors. Can you explain why you use L1 loss instead of cosine distance?

3. For rendering, you only use L2 loss, however, D-SSIM is also used to optimize Gaussian splatting. Can you explain why you don't use it in your method?

4. The method is based on NeuralPull, which learns a neural SDF from ground truth point clouds, while this method optimizes a neural SDF from 3D Gaussian splatting. Therefore, the original NeuralPull is the upbound of this method. Can you provide an experiment that trains NeuralPull on the TNT dataset and compares it with it to show the upbound of the proposed method?

5. How do you deal with the surface of 'ground'? For example, in Fig.4 (right, 'Truck') and Fig.14 (left, 'Barn'), the ground is built. However, in Fig.14 (middle, 'Caterpillar'), the ground is missing. Do you manually delete it or something else? In your evaluation, do you also delete it? As I know, on the TNT dataset, the extracted meshes are not modified manually for evaluation.


[1] CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization.

[2] Neural unsigned distance fields for implicit function learning.
limitations: limitations: In Fig.12, the paper shows this method can not reconstruct the details of 'lego'. Can the authors show the comparsion with this method and SuGar as well as 2DGS?
paper_summary: paper_summary: This paper aims to infer neural signed distance functions (SDF) for Gaussian Splatting. To this end, this paper introduce an MLP to represent  the SDF. To learn SDF from sparse and non-uniform Gaussian points, this paper introduces a differentiable pulling operation from Neural-Pull to align Gaussians on the zero-level set of the SDF and updates the SDF by pulling neighboring space to the pulled 3D Gaussians. This paper designs tangent loss, pull loss and orthogonal loss to encourage the above operations. Experiments on DTU and Tanks and Temples datasets show that the proposed method achieves SOTA reconstruction performance.
strength_weakness: strength_weakness: * The proposed method cannot reconstruct geometric details well as shown in Figures 4, 13 and 14.
* The ablation results in Table 4 are not consistent with the results in Table 2.
* The ablation study is bad and confusing. The quantitative results are conducted on Tanks and Temples but the qualitative results are conducted on DTU.
* The rendering evaluation setting is not consistent with prior works, such as 2DGS and GOF. Using lower resolution training images can improve rendering results. However, the training setting for NeRF-based methods still use the original setting.
questions: questions: * Gaussian splatting will generate many points that are far away from surfaces, how does the pulling operation tackle these points?
* When encouraging the thin disk to be a tangent plane on the zero-level set, why not make $f(\mu'_j)\approx0$?
* For Table 4, can you explain the difference from Table 2 in terms of the performance of full model? Moreover, can you explain the detailed operations of w/o Pull GS? I wonder how to train the SDF without Pull GS.
* As the proposed method uses the densification from GOF, it is better compare the rendering performance of proposed method with GOF.
* How many Gaussian points and random points are sampled to train SDF during each optimization?
limitations: limitations: The authors have discussed the limitations and their future works.

Comments: title: Thanks for the accept recommendation
Thanks for your accept recommendation
We will be happy to take questions
comment: Thank you very much for your feedback. I have increased my score!
We tried two implementations to add high-frequency information. The first one is appending positional encoding to the input XYZ coordinates, like the original NeRF. The other is replacing the input 3D points with multi-resolution hash encoding, like Instant-NGP. All parameter settings are consistent with those of the original implementations. The experiment is conducted on the Ignatius scene of the TNT dataset as requested. Using the same experimental setting, as reported in the following table, both adding positional encoding and hash feature grid from instant-ngp can slightly improve the reconstruction visually and numerically. We do not have time to try more parameter options like frequencies in pe and grid resolution in hash feature grid, but we will report that in our revision.

Ours: 0.71

+positional encoding: 0.73

+hash encoding: 0.75
Thank you very much for your comprehensive ablation studies. Since I am curious about the potential of the method on reconstruction details, I am looking forward to your experimental results on the reconstruction details using Instant-NGP or high-frequency positional encoding.
3. More thorough ablation studies

We have completed the ablation studies for all scenes on DTU and MipNeRF360. The results are listed in the tables below. We will add these results in our revision. The conclusions drawn from this version of the ablation study, which is conducted on all scenes, are largely in agreement with the results from the previous version of the ablation study, which was conducted on a subset of scenes.

**Ablation studies on DTU dataset across all scenes**

| | Pulling            |               | Constraints            |               |               | Mesh  |      |            |
|---------|--------------------|---------------|---------------|---------------|---------------|------------------|------|------------|
|         | **Pulled to centers**  | **w/o Pull GS**   | **w/o $L_{Thin}$** | **w/o $L_{Tan}$** | **w/o $L_{Oth}$** | **TSDF**             | **Poisson** | **Full model** |
| **CD↓**     | 0.90               | 0.85          | 0.78          | 0.83          | 0.79          | 1.41             | 0.79    | **0.74**       |


**Ablation studies on MipNeRF360 dataset across all scenes**

|            | **Indoor**       |           |            | **Outdoor**      |           |            |
|------------|------------------|-----------|------------|------------------|-----------|------------|
|            | **PSNR↑**        | **SSIM↑** | **LPIPS↓** | **PSNR↑**        | **SSIM↑** | **LPIPS↓** |
| **w/o Pull GS**  | 29.97            | 0.898     | 0.191      | 22.47            | 0.684     | 0.293      |
| **w/o $L_{Thin}$**| 30.57         | 0.922     | 0.185      | 23.63            | 0.699     | 0.279      |
| **w/o $L_{Tan}$** | 29.65         | 0.897     | 0.207      | 22.05            | 0.674     | 0.300      |
| **w/o $L_{Oth}$** | 30.69         | 0.923     | **0.182**      | 23.74            | 0.702     | **0.278**      |
| **Full model**  | **30.78**           | **0.925**     | **0.182**      | **23.76**            | **0.703**     | **0.278**      |
Thanks for your comments. We are pleased to discuss more about the details.

1. Ours (Screened Poisson) in Fig. F in our rebuttal PDF was obtained using points back projected from rendered depth rather than 3D Gaussians.

Reviewer ZhFP misunderstood how our mesh reconstruction was produced by the screen Poisson. As we mentioned in our response “1. Reconstruction details” posted at 14:53 on Aug.7, we used points back projected from rendered multi-view depth maps to reconstruct the mesh surface with Poisson, rather than the raw Gaussian points. 

Points from depth maps have been used by SuGaR and GaussianSurfels to reconstruct surfaces with Poisson reconstruction, and we have not seen methods directly using Gaussian positions to reconstruct surfaces so far. The reasons are twofold. One is that Gaussians are not guaranteed to be dense everywhere in a scene since they are just responsible for good rendering quality, some poor reconstruction caused by sparse Gaussians can be found in the large holes in the middle of Fig.15 in the outdoor scene in our paper. Another key reason, we believe, is that Gaussian positions do not accurately represent the surface, even if previous methods and we have tried various constraints, such as our pulling, to locate Gaussian positions on the estimated surface. However, splatting makes a difference. When we splat these Gaussian on a 2D plane and render them, the Gaussians in the view frustum will work together to approximate more accurate geometry and also surface details inferred from multi-view photometric consistency, just like what we show in Fig. F in our rebuttal PDF. Thus, Gaussian positions do not represent geometry details quite well, but depth maps rendered by splatting Gaussians do.

We believe pulling works well in recovering geometry details, since it has been justified by high fidelity reconstructions in NeuralPull[1], and its following works, such as [2-4].

2. Why not Instant-NGP

As we explained in our previous post, using Instant-NGP like hash encoding and high-frequency positional encoding will be our future work. We did not use them in this project, since we have seen pretty stunning reconstructions obtained by previous methods using pulling with MLP. Although most of these works are working with point clouds (not learnable), we thought this mature framework can also recover geometry details from 3D Gaussians (learnable) with proper other designs. Thus, this is a straightforward and intuitive attempt, but definitely a good start for us to keep working in this direction.

We will start working with integrating hash encoding with the SDF representation in our current method immediately, and try our best to post results as you requested before the reviewer-author discussion period ends. Our implementation may take some time. If we are unable to complete it before the discussion period ends, we guarantee that we will further discuss this point in the revision.

[1]. Ma B, Han Z, Liu Y S, et al. Neural-Pull: Learning Signed Distance Function from Point clouds by Learning to Pull Space onto Surface. International Conference on Machine Learning. PMLR, 2021: 7246-7257.

[2]. Ma B, Liu Y S, Zwicker M, et al. Surface reconstruction from point clouds by learning predictive context priors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6326-6337.

[3]. Ouasfi A, Boukhayma A. Few-Shot Unsupervised Implicit Neural Shape Representation Learning with Spatial Adversaries. Forty-first International Conference on Machine Learning. 

[4]. Chou G, Chugunov I, Heide F. Gensdf: Two-stage learning of generalizable signed distance functions. Advances in Neural Information Processing Systems, 2022, 35: 24905-24919.
Thank you very much for your response. However, the answer on the **Reconstruction details** cannot address my concern. For many detail regions, the Gaussian points are with small scales. However, the pulling operation cannot reflect these details. As shown in the Figure F, when applying Poisson reconstruction to extract meshes from Guassion points, the dotted details on the front of the Lego can be reconstructed well. This demonstrates the Guassian points can better represent these details. However, the proposed method cannot reconstruct these details. In addition, the high frequency positional encoding and the Instant-NGP are commonly used with MLP to reconstrcut surfaces. I wonder why the proposed method does not leverage these techniques. Can you combine these tehniques with the proposed method to test the Ignatius scene of the TNT dataset and show the evaluation results?
Dear reviewer 5rKf,

Thanks for your time and expertise. We are glad to know that our rebuttal addressed your concerns and you are willing to raise your rating. We really appreciate the accept recommendation. 

We will also follow your advice to revise our paper accordingly.

Best,

The authors
Thanks for the efforts of the authors. Most of my concerns have been addressed. I am willing to improve my rating.
Dear AC,

We are writing to report an issue of openreview today. It seems that it does not send emails to reviewers and authors when we submit our comments. We tried many times, and all of our authors did not receive emails like before. We are afraid that the reviewers may miss our comments, since the discussion period will end in one day. 

If you can see this message by any chance, can you inform reviewer ZhFP this issue and ask reviewer ZhFP to check our latest reply using personal emails?

Really appreciate your help.

Thanks,

The authors
Yes, we will release the code upon paper acceptance.
Will you release your code in the future?
Dear reviewer ZhFP,

Thanks for your questions.   

1. Reconstruction details 
 
Since our method infers signed distances from 3D Gaussians, we believe the 3D Gaussians are the key to recover geometry details. Although 3D Gaussians are dense (for most time but we still have some sparse cases), they are not dense enough like point cloud scans to recover fine geometry. This is because 3D Gaussian is not a point but a sphere or a plane which covers some space. This is fine for rendering, since the pixel color is usually produced by several overlapped Gaussians, but obviously using one Gaussian is hard to infer fine geometry in the area it covers, especially when the Gaussian is very large, which makes them much different from points back projected from the rendered depth. Just like the extreme case raised up by reviewer SW8k, the large holes in the middle of Figure 15 in the outdoor scene, we observed huge Gaussians in these areas. Although these huge Gaussians may work well in rendering, but they cannot recover any geometry covered by them. Thus, how to control the size of Gaussians for SDF inference could be an interesting future work direction.

Another reason is like what we mentioned in the limitation. The MLP we use to estimate an SDF prefers low-frequency information, which also affects the performance of recovering fine geometry. This can also be resolved in our future work by capturing high-frequency geometry either using high frequency positional encoding or without using MLP.

We will add this discussion in our revision.


2. MipNeRF360 ablation

We conduct additional ablation studies on MipNeRF360 dataset, as shown in the following table. Due to the time limit, we select two scenes from indoor and outdoor scenes, respectively. The terms that are irrelevent to rendering are omiited. We are trying our best to report the results on all scenes along with the ablation results on DTU before the discussion ends. We would like to clarify in advance that ablations related to rendering metrics do not show as clear distinctions as those related to reconstruction metrics. This is because the rendering metrics are usually more stable than reconstruction metrics, which is evident in previous works.

The term "w/o $L_{Tan}$" has the most significant impact on rendering quality because we align the Gaussian normals with the surface normals to obtain consistent orientated Gaussians, which enhances the rendering quality. The term "w/o Pull GS" has the second most significant impact on rendering metrics, because we optimize the Gaussian positions by pulling them towards the zero-level set to achieve better rendering quality. The other terms "w/o $L_{Thin}$", and "w/o $L_{Oth}$" are mainly designed for learning SDF and have a minor improvement on the rendering metrics.

|             | **Indoor**           |           |           | **Outdoor**          |           |           |
|-------------|----------------------|-----------|-----------|----------------------|-----------|-----------|
|             | **PSNR↑**            | **SSIM↑** | **LPIPS↓**| **PSNR↑**            | **SSIM↑** | **LPIPS↓**|
| **w/o Pull GS** | 29.74              | 0.920     | 0.207     | 23.97                | 0.730     | 0.255     |
| **w/o $L_{Thin}$**  | 30.07              | 0.919     | 0.190     | 24.11                | **0.747**     | **0.237**     |
| **w/o $L_{Tan}$**   | 29.35              | 0.913     | 0.213     | 22.75                | 0.702     | 0.261     |
| **w/o $L_{Oth}$**   | 30.18              | 0.927     | 0.190     | 24.11                | **0.747**     | 0.239     |
| **Full model**  | **30.19**              | **0.929**     | **0.189**     | **24.12**                | **0.747**     | **0.237**     |



Please feel free to let us know if you have any more questions.


Best,

The authors
Thanks for your further response. I still have the following questions.  
**Reconstruction details.** As you explained, the proposed method cannot recover potential details like 2DGS, although these details are not accurate. The current SOTA methods achieve ~0.6 performance on DTU (lower is better) and ~0.5 performance on TNT dataset (higher is better) as they can recover accurate details. Since Gaussian points are usually dense in details, this should help recover potential details, I wonder what limits the proposed method to recover the potential details.  
**Comparison with GOF.** Thanks for your explanation. Can you also conduct a rendering ablation study on MipNeRF 360 for the proposed losses like Table B? I wonder what component influences the rendering.
Dear reviewer SW8k,

Thanks a lot for your comments and suggestions. We sincerely appreciate your accept recommendation.

Best,

The authors
Thank you for your response. I'm willing to raise my rating to borderline accept.
Dear reviewer ZhFP,

Thanks for your questions.

1. Reconstruction Details

If compared methods can recover comparable surface positions, then more geometry details definitely produce better numerical results. However, if one method fails to recover accurate surface positions, then more geometry details could not improve the numerical results a lot. If a reconstructed surface does not come from an accurate zero-level set, it would have inaccurate surface position which drifts away a lot from the GT surface, which hurts the numerical results.  To show this, we draw a figure below to illustrate the comparison. This figure shows that our surface (▲) is nearer to the ground truth (*), although the geometry details recovered by the marching cubes algorithm are not many. In contrast, the other surface (■) has much more geometry details which also look quite similar to the ground truth, but fails to recover accurate surface position, drifting far away from the ground truth, which hurts the numerical results a lot. This is also indicated quite clearly in Fig.C in our rebuttal PDF, where the error map across the whole surface obtained by 2DGS looks much darker (bigger errors) than ours.

```plane text
*: GT surface, ▲: Our surface, ■: 2DGS surface
---------------------------------------------------
Our surface vs. GT
   *        ***
  *▲*      *▲  *▲      
▲▲▲ ▲▲  ▲▲▲▲ ▲▲▲ ▲▲ ▲▲▲▲
      ▲▲ *        *▲
       **
---------------------------------------------------
2DGS surface vs. GT
   *        ***
  * *      *   **
**   *    *      *  ****
      *  *        **
       **
             ■
            ■ ■
  ■■■      ■   ■■    ■
■■   ■    ■      ■  ■ ■
      ■  ■        ■■   ■
       ■■
---------------------------------------------------
```



Moreover, we seriously checked our code, especially the evaluation part, we did not see bugs. For fair evaluations on TNT, we use the official github code released with the dataset of the original paper “Tanks and Temples: Benchmarking Large-Scale Scene Reconstruction”.

8. Comparison with GOF

We achieved comparable rendering quality to GOF in indoor scenes. We also noticed that our rendering quality is worse than GOF in outdoor scenes. The main reason is that outdoor scenes include lots of huge Gaussians, each of which covers a large area. This may be caused by the lack of view coverage. These huge Gaussians usually struggle to meet all the constraints that we imposed for SDF inference. For example, they can not be aligned well with the gradient at the zero-level set, and their large size amplifies little normal adjustment to a large impact on the rendering, since it can not take all area it covers into account. This also raises up a future work direction, that is how to control the size of Gaussians during our inference, so that we can maintain good rendering quality of GS.
Dear reviewer SW8k,

Thanks for raising your rating. We are glad to discuss more details here.

1. About "Performance of NeuralPull". 

In our rebuttal, in order to intuitively compare the pure pulling operation between NeuralPull and our method, we use a set of sparse ground truth point clouds, which were downsampled at a factor of 2000 from the ground truth points provided by TNT dataset to conduct the experiments. We did not use the sparse point clouds estimated by COLMAP, because we want to exclude the potential misguidance of the inaccurate points. We initialize the points as flat spheres to simulate the 3D Gaussians, and fix the points during training. The parameters to be optimized are only the SDF network parameters. This experiment highlights the differences between NeuralPull and our proposed method. 

Moreover, in the main paper, the initial point clouds are from COLMAP, which initialize the learnable Gaussians and is the basic procedure in 3DGS. After that, we do not use the initial point cloud any more. Then, we learn Gaussians through splatting, simultaneously the positions and the shapes of Gaussians which are used to estimate the SDF. Obviously, this is a dynamic optimization procedure, both Gaussians and SDF are updated iteration by iteration, which is not like what NeuralPull can do using a fixed set of points as pulling targets. Here, we got a coarse SDF by pulling randomly sampled queries to their nearest Gaussian disks on the zero-level set, meanwhile, we also pull Gaussians onto the zero-level set, and directly impose constraints on these pulled Gaussians, including the normal alignment, minimal rendering errors, and flat plans, etc.. Thus, we use pulled Gaussians (not fixed) as the pulling targets. Moreover, with these pulling targets, queries are pulled onto the Gaussian disks rather than the centers, as shown in Eq.5 in our paper, which aims to handle the sparsity of Gaussians and also differentiates our method from the original NeuralPull.

2. About the results

As we explained in our response “2. Visual results. ” in the rebuttal above, their numerical results are not as good as their visual results, since the reconstructed surfaces do not come from an accurate zero-level set. To show this, we draw a figure below to illustrate the comparison. 

This figure shows that our surface (▲) is nearer to the ground truth (*), although the geometry details recovered by the marching cubes algorithm are not many. In contrast, the other surface (■) has much more geometry details which also look quite similar to the ground truth, but fails to recover accurate surface position, drifting far away from the ground truth, which hurts the numerical results a lot. This is also indicated quite clearly in Fig.C in our rebuttal PDF, where the error map across the whole surface obtained by 2DGS looks much darker (bigger errors) than ours.

```plane text
*: GT surface, ▲: Our surface, ■: 2DGS surface
---------------------------------------------------
Our surface vs. GT
   *        ***
  *▲*      *▲  *▲      
▲▲▲ ▲▲  ▲▲▲▲ ▲▲▲ ▲▲ ▲▲▲▲
      ▲▲ *        *▲
       **
---------------------------------------------------
2DGS surface vs. GT
   *        ***
  * *      *   **
**   *    *      *  ****
      *  *        **
       **
             ■
            ■ ■
  ■■■      ■   ■■    ■
■■   ■    ■      ■  ■ ■
      ■  ■        ■■   ■
       ■■
---------------------------------------------------
```
Thanks for your response. I still have the following questions.  
**1. Reconstruction details.** I cannot agree that enhancing geometry details does not significantly improve the numerical results. On DTU dataset, many methods have achieved the reconstruction performance of around 0.6, such as Neuralangelo [1]. These method can reconstruct details well, leading to their better performance. In fact, the visualization of the proposed method on TNT dataset is relatively poor, making the good quantitative results a bit unconvincing. As mentioned in your main paper (L249-250), you have split large scale scenes into parts to extract meshes, however, the proposed method cannot still recover details on TNT. In addition, how do you evaluate your reconstructed meshes on TNT dataset? 
[1] Li, Zhaoshuo, et al. "Neuralangelo: High-fidelity neural surface reconstruction." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.  
**8. Comparison with GOF.** As shown in the complete Table, in fact, the method degrades the rendering quality a lot. Can you explain it?
Your answers solve most of my concerns. I'm happy to raise my rating. However, I still have some questions here.
1. About "Performance of NeuralPull": How do you train both your method on sparse ground truth point clouds, instead of COLMAP point clouds? I think in your paper, you train your method on sparse ground truth point clouds from colmap, right?
2. I'm still confused that your f1 score is not consistent with your visualization on TNT dataset. I think 0.43 F1 score is high, but your visualization on TNT dataset is really poor.
Dear reviewer CCgx,

Thanks for your comments and valuable advice. We really appreciate your accept recommendation.

Best,

The authors
**Discussion about NeuSG**: While I agree that NeuSG optimizing a neural radiance field (specifically NeuS) makes it different from yours (where it is pure Gaussian splatting), I still think that they are related enough to warrant more discussion in the paper. I definitely agree that the extension to neural pull is novel and interesting, and overall think the proposed method is quite elegant (especially in relation to NeuSG). And of course the time performance in relation to NeuSG is amazing. I am glad the authors are willing to add more clarifications in the paper.

**Normal loss**: glad you are happy to add this in the paper/supplement.

**2DGS results**: It seems I was looking at v2 of their paper on arXiv (which came out after the submission deadline for NeurIPS), where their result on on TnT is a mean of 0.32 and their mean calculation seems fine. I see now that the mean calculation is wrong in v1 of their paper.

**Overall**: The authors have addressed all my concerns and am glad for their efforts in the discussion period. I hope they add the clarifications and ablations that they promised as well as release their code. I wish to maintain my previous recommendation of accept (7).
Dear reviewer CCgx,

Thanks for the comments. It is a pleasure to further clarify some points.

**1. We are much different from NeuSG**

*i) The task*

Our method directly learns to capture the geometry inherently represented by 3D Gaussians, while NeuSG aims at utilizing Gaussians as guidance to improve the effect of neural implicit reconstruction.


*ii) The framework*

Our method is based on 3DGS (learning radiance fields from multi-view images) and NeuralPull (learning SDF from points), and we train SDFs by unsupervisedly pulling query points onto the Gaussian disk. In contrast, NeuSG is based on 3DGS (learning radiance fields from multi-view images) and NeuS (learning NeRF and SDF from multi-view images), therefore it trains SDFs through classical neural rendering techniques, including ray marching, point sampling and volume rendering. This makes the whole framework so different from each other.

*iii) Losses*

We would like to clarify that, we introduce a variant of NeuralPull, which pulls queries on Gaussian plans but rather on points like the original NeuralPull. To achieve this, our pulling loss is different from the NeuralPull loss. In addition, the idea of Gaussian normal constraints comes back from the era of Surface Splatting[1], and the idea of constraining gradients of SDF is also not new since NeuralPull[2] came out, methods like DiGS[3] and Towards Normal Consistency[4] also followed this idea. Thus, manipulating gradients and Gaussian normals has become a commonly used strategy, which does not originally come from NeuSG. Our reference to NeuSG in Line 175 is just for the flatten of Gaussians.

Since we focus on Gaussians on the zero-level set, the implementations of our constraints on Gaussian normals and SDF gradient are much different from NeuSG. For example, we need to pull Gaussians onto the zero-level set, calculate SDF gradients at the pulled Gaussians on the zero-level set, and align Gaussian normal to these gradients (our Tangent loss in Eq.4 of the original paper). Moreover, we also pursue alignment across different level-sets by aligning gradients at queries (may appear on any level sets in the field) to the normal of Gaussians on the zero-level set (our orthogonal loss in Eq.6 of the original paper), which makes the gradients on other level sets orthogonal to the corresponding Gaussian disk on the zero-level set.

We will make sure to include these discussions and NeuSG in our revision.

**2. We are also more efficient than NeuSG**

Since NeuSG is based on NeuS, which needs to sample points along a ray for integration, it inherits the drawback of NeuS, i.e., poor efficiency. On a scene, we just need about half an hour, while NeuSG needs about 16 hours. Obviously, our solution can better leverage the advantages of 3DGS than NeuSG.

We will make sure to include these discussions and NeuSG in our revision.

**3. Normal loss**

Using either cosine or L1 to constrain normal is also widely used in point cloud normal estimation. As we stated in our response “7. Normal Loss” to reviewer SW8k, we did not see any differences on the performance for these two constraints on normal. We will include this ablation study in our revision.

**4. 2DGS results**

There is a numerical error in the mean operation. According to Table 2 in 2DGS paper, if we average the per-scene result provided in the table, we can get (0.36+0.23+0.13+0.44+0.16+0.26)/6=0.26, but 2DGS reported the mean value as 0.30 instead of 0.26. 


[1]. Zwicker M, Pfister H, Van Baar J, et al. Surface Splatting. Proceedings of the 28th annual conference on Computer graphics and interactive techniques. 2001: 371-378.

[2]. Ma B, Han Z, Liu Y S, et al. Neural-Pull: Learning Signed Distance Function from Point clouds by Learning to Pull Space onto Surface. International Conference on Machine Learning. PMLR, 2021: 7246-7257.

[3]. Ben-Shabat Y, Koneputugodage C H, Gould S. DiGS: Divergence guided shape implicit neural representation for unoriented point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 19323-19332.

[4]. Ma B, Zhou J, Liu Y S, et al. Towards better gradient consistency for neural signed distance functions via level set alignment. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 17724-17734.
Dear reviewer ZhFP, 
Thanks for your comments. We are happy to answer your questions. Unfortunately, we are not allowed to either 1) show you any reconstruction visualization or visual comparison during this reviewer-author discussion period or 2) provide you any URL towards these visual results, according to the rebuttal policy.

1. Reconstructing details.

We are sorry for missing the details of 2DGS, since we confused 2DGS and GaussianSurfels, one uses TSDF to extract the zero-level set while the other uses Poisson surface reconstruction. We just ran TSDF fusion on our results to compare, and got comparable results to our results obtained by Poisson surface reconstruction. This is because we use the same depth as input, and the resolution used either in Poisson or TSDF fusion is similar.

Indeed, we can reconstruct details at a higher resolution, but enhancing these details does not significantly improve the numerical results. As illustrated in Figure C in our rebuttal PDF, though 2DGS contains more geometric details, it achieves a lower numerical results than our method. Our superiority results come from the way that we estimate a more accurate zero-level set.

3. How we improve accuracy on TNT.

As we mentioned in the reply to “2. Visual results.” for reviewer SW8k, we improve the accuracy on TNT by estimating a more accurate zero-level set. Although current methods can reconstruct geometry details, the reconstructions are usually fat, which means their surfaces usually drift away from the real zero-level set. Our method can pull 3D Gaussians onto the zero-level set, which enables us to impose constraints directly on the zero-level set. This really helps us estimate a more accurate zero-level set. Please refer to Figure C in the rebuttal PDF for comparisons of error maps between 2DGS and our method. Additionally, we will provide more quantitative ablation results on TNT dataset in our revision.

6. Constraints on pulled Gaussians.

We did not find a proper way to directly implement the constraint of "approximately equal to" in the code.
When we were trying verify if “equal to” helps, we tried different weights on this loss term. When the weight is small, we can achieve the same effect as “approximately equal to”, while a large weight can turn the loss into a hard constraint "equal to". But both of them can not work with the uncertainty in the signed distance field during optimization. If you have any better ideas on how to impose a constraint of "approximately zero," please let us know, as we are willing to undertake further attempts.

Additionally, our results in Fig.G in our rebuttal PDF were produced with removing far away Gaussians using the same way explained in "5.About far away points".

8. Comparison with GOF.

Results in Tab.D are averaged results over indoor scenes, not the averaged results over both indoor and outdoor scenes. The full comparison is provided in the following.


|       | **Indoor Scene**    |             |             | | **Outdoor Scene**    |             |             |
|-------|---------------------|-------------|-------------|-|----------------------|-------------|-------------|
|       | **PSNR$\uparrow$**  | **SSIM$\uparrow$** | **LPIPS$\downarrow$** | | **PSNR$\uparrow$**  | **SSIM$\uparrow$** | **LPIPS$\downarrow$** |
| **GOF**   | 30.79              | 0.924       | 0.184       | | 24.82                | 0.750       | 0.202       |
| **Ours**  | 30.78              | 0.925       | 0.182       | | 23.76                | 0.703       | 0.278       |



As we mentioned in the rebuttal, we can not report the results on all scenes in DTU due to the time limit. We are trying our best to report the results on all scenes before the discussion ends. We are limited to a handful computational resources to finish so many results in the rebuttal PDF.
Dear reviewer SW8k, Thanks for your comments. We
are happy to answer your questions.

1. About 2DGS numerical results.

According to Table 2 in 2DGS paper, if we average the per-scene result provided in the table, we can get (0.36+0.23+0.13+0.44+0.16+0.26)/6=0.26, but 2DGS reported the mean value as 0.30 instead of 0.26. Thus we say '2DGS wrongly reported the mean result on TNT in its original paper'.

2. About "Performance of NeuralPull".

We trained both NeuralPull and our method on sparse *ground truth* point clouds, instead of COLMAP point clouds in our rebuttal (see our response "9. Performance of NeuralPull"). Following your advice, we additionally conduct experiments on all of the six scenes of TNT dataset, following the same setting used in our response "9. Performance of NeuralPull." The numerical results in terms of F-score are reported in the following table.

| Methods    | Barn | Caterpillar | Courthouse | Ignatius | Meetingroom | Truck | Mean |
|------------|------|-------------|------------|----------|-------------|-------|------|
| NeuralPull | 0.70 | 0.40        | 0.51       | 0.51     | 0.56        | 0.73  | 0.57 |
| Ours       | **0.78** | **0.58**   | **0.66**   | **0.66** | **0.65**    | **0.76**  | **0.68** |

The comparison indicates that our method is a variant of NeuralPull, but not exactly the same as NeuralPull. Thus, we can produce much better accuracy on points than the upper bound of NeuralPull, due to the ability of pulling queries on plans rather than points.

3. About the holes.

Due to overly complex geometric structures and a lack of view covering, there is a significant under-fitting issue in the flowerbed area of the middle scene. This results in a set of extremely sparse, huge, and unevenly distributed Gaussians, which makes Gaussians are thick ellipsoid like shape rather than relatively thin plans, leading to poor sense of surface. While this issue does not occur in other scenes.

4. About learning UDF.

Following your advice, we also additionally conduct experiments of learning UDFs on all of the six scenes in TNT dataset, as reported in the following table, in terms of F-score. The superiority results of learning SDF beyond learning UDF demonstrates the drawbacks of UDF.

| Methods    | Barn | Caterpillar | Courthouse | Ignatius | Meetingroom | Truck | Mean |
|------------|------|-------------|------------|----------|-------------|-------|------|
| NeuralPull | 0.55 | 0.34        | 0.15       | 0.60     | 0.17        | 0.46  | 0.38 |
| Ours       | **0.60** | **0.37**   | **0.16**   | **0.71** | **0.19**    | **0.52**  | **0.43** |

The comparison in the above table indicates that our method can learn not only SDF but also UDF, which also show dvantages over NeuralPull in learning of UDFs.
My weaknesses/questions have been sufficiently addressed. However, looking at the other reviewer's comments and the authors rebuttal to them, I have the following concerns:
- I was not aware of NeusG, and after looking into it, their method seems highly related. NeusG is essentially GS losses + IGR losses + thin loss + consistency losses, while the proposed method is GS losses + NP losses + thin loss + consistency losses. However NeusG is barely mentioned: the authors respond that it is mentioned in line 175 but that is a citation about the general trend of flattening Gaussians. There should be at least a sentence only about NeusG given it is very similar. Either the work is considered concurrent work and should be stated as such, or it is considered a work that inspired your work (including having first introduced the thin loss which you borrow from them) and this should be made clear in the related work. Furthermore, for the latter case NeusG should also compared to in the results!! Their main results are on TnT and it looks like your method performs better, but can you check they use the same settings and then put it into your paper?
- As for the other comments, while some seem quite concerning, the authors have addressed most of them sufficiently. The things that are left are: 
	- Normal loss: since cosine sim is what NeusG does and you change it, there should be an ablation on it (in the final version of the paper, not suggesting you get results in time to show us)
	- '2DGS wrongly reported the mean result on TNT in its original paper': also want explanation about this statement, did you consult the authors and figure this out, or are you saying there is a numerical error in the mean operation (doesn't seem to be?)
Thank you for your rebuttal. I still have some questions:
1. Why do you say '2DGS wrongly reported the mean result on TNT in its original paper'? Can you explain it or give some proof?
2. About 'Performance of NeuralPull': The Fig. E in your rebuttal PDF only shows one scene. Can you give a table across six scenes from TNT dataset to show your method is better? Also, the NeuralPull used in your experiment is trained on sparse point clouds from colmap, right? Since TNT has the ground truth point cloud, can you use the ground truth point cloud to train the NeuralPull. I just want to see the upbound of your method, which is the NeuralPull trained with ground truth. It's okay that the neuralpull trained with ground truth is better than your method.
3. About 'the holes in the middle of Figure 15 are due to the ill-conditioned distribution of Gaussians'. However, why is there not a hole in the right figure in Fig. 15 of your paper? What's the difference between the middle and the right?
4. About 'UDF': The Fig. D in your rebuttal PDF only shows one scene. Can you give a table across six scenes from TNT dataset to show your method is better?
Thank you for the response. I have also read the other comments and responses. I have some following questions.  
**1. Reconstruction details.** 2DGS uses TSDF fusion to reconstruct meshes. However, the method combines Poisson surface reconstruction to reconstruct meshes from Gaussian points to show the details. It is unfair. Moreover, it is easy to extract higher resolution meshes from SDF, as shown in MonoSDF [1]. Can you show higher-resolution meshes on DTU or TNT dataset? In fact, DTU is a small-scale object-centric dataset. However, the proposed method cannot reconstruct details well on DTU dataset.  
Yu, Zehao, et al. "Monosdf: Exploring monocular geometric cues for neural implicit surface reconstruction." Advances in neural information processing systems 35 (2022): 25018-25032.  
**3. About ablation study.** Since TNT dataset is more challenging, it is better to show ablation qualitative results on this dataset.  Since the the proposed method cannot handle details well, I would like to know how the proposed strategies improve surface reconstruction on the TNT dataset.  
**6. Why not make $f(\mu'_j)\approx0$?** In fact, I wonder if the **$f(\mu'_j)\approx0$** operation can help reconstruction instead of $f(\mu'_j)=0$. Moreover, can you try this using the filtered Gaussian points as you mentioned in *5. About far away points.*    
**8. Comparison with GOF.** The results shown in Table D are not consistent with those in the main paper. Can you check them carefully?  
In addition, since the proposed method is efficient, why only using 4 scenes to conduct ablation studies on DTU dataset? I am curious whether the results in Table C are convincing.
Dear AC,

Thanks for initiating the discussion. We are also looking forward to having a discussion with reviewers. It would be great if reviewers could let us know if their concerns are addressed after reading our rebuttal materials. We appreciate that a lot.

Best,

The authors
Hi reviewers,

Thank you for your hard work in reviewing the paper!
Please check out the authors' responses and ask any questions you have to help clarify things by Aug 13.

--AC
Dear reviewers,

We appreciate your comments and expertise. Please let us know if there is anything we can clarify further. We would be happy to take this opportunity to discuss with you.

Thanks,
The authors