Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Set
Authors: Wenyuan Zhang
Yu-Shen Liu
Zhizhong Han
Venue: NeurIPS 2024 poster (2024)
Keywords: Gaussian Splatting
Signed distance function
3D reconstruction
Primary area: machine_vision

Abstract: It is vital to infer a signed distance function (SDF) for multi-view based surface reconstruction. 3D Gaussian splatting (3DGS) provides a novel perspective for volume rendering, and shows advantages in rendering efficiency and quality. Although 3DGS provides a promising neural rendering option, it is still hard to infer SDFs for surface reconstruction with 3DGS due to the discreteness, the sparseness, and the off-surface drift of 3D Gaussians. To resolve these issues, we propose a method that seamlessly merge 3DGS with the learning of neural SDFs. Our key idea is to more effectively constrain the SDF inference with the multi-view consistency. To this end, we dynamically align 3D Gaussians on the zero-level set of the neural SDF, and then render the aligned 3D Gaussians through the differentiable rasterization. Meanwhile, we update the neural SDF by pulling neighboring space to the pulled 3D Gaussians, which progressively refine the signed distance field near the surface. With both differentiable pulling and splatting, we jointly optimize 3D Gaussians and the neural SDF with both RGB and geometry constraints, which recovers more accurate, smooth, and complete surfaces with more geometry details. Our numerical and visual comparisons show our superiority over the state-of-the-art results on the widely used benchmarks.

Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Set
Wenyuan Zhang', Yu-Shen Liu'*, Zhizhong Han?
School of Software, Tsinghua University, Beijing, China
zhangwen21 @mails tsinghua. edu. en, Liuyushen@tsinghua. edu, cn Department of Computer Science, Wayne State University, Detroit, USA h31 h312h@wayne.ed edu
Abstract
It is vital to infer signed distance function (SDF) in multi-view based surface reconstruction. 3D Gaussian splatting (3DGS) provides novel perspective for volume rendering, and shows advantages in rendering efficiency and quality. Al- though 3DGS provides a promising neural rendering option, it still hard to infer SDFs for surface reconstruction with 3DGS due to the discreteness, the sparseness, and the off-surface drift of 3D Gaussians. To resolve these issues, we propose method that seamlessly merge 3DGS with the learning of neural SDFs. Our key idea is to more effectively constrain the SDF inference with the multi-view consistency. To this end, we dynamically align 3D Gaussians the zero-level set the neural SDF using neural pulling, and then render the aligned 3D Gaussians through the differentiable rasterization. Meanwhile, pulling neighboring space to the pulled 3D Gaussians, which progressively refine the signed distance field near the surface. With both differentiable pulling and splatting, we jointly optimize 3D) Gaussians and the neural SDF with both RGB and geometry constraints, which recovers more accurate, smooth, and complete surfaces with more geometry details. Our numerical and visual comparisons show our superiority over the state-of-the-art results on the widely used benchmarks. Project page: https: //en//ennggg yuan- mann-zang,... github. io/GS-Pull.
update the neural SDF by
Introduction
3D scene representations are important to various computer vision applications, such as single or multi-view 3D reconstruction [39, 42, 67, 72], novel view synthesis [1, 71], and neural SLAM [22, 32, 78, 24] etc.. Mesh and point clouds are the most common 3D scene representations, and can be rendered by fast rasterization on GPUs. Instead, more recent neural radiance fields (NeRFs) [49] are continuous scene representations, but it is slow to render NeRFs due to the need of costly stochastic sampling along rays in volume rendering. More recently, 3D Gaussians with different attributes like color and opacity used as a versatile differentiable volumetric representation [30, 77, 23] for neural rendering through splatting, dubbed 3D Gaussian Splatting (3DGS). It prompts the pros of both NeRFs and point based representations, which achieves both better quality and faster speed in rendering. Although 3D Gaussians can render plausible images, it is still a challenge to reconstruct surfaces based the 3D Gaussians.
The key challenge comes from the gap between the discrete 3D Gaussians and the continuous geometry representations, such as implicit functions. Besides the discreteness, the sparseness caused by uneven distribution and the off-surface drift make 3D Gaussians even harder use than
"The corresponding author is Yu-Shen Liu.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

scanned point clouds in surface inference. To overcome these obstacles, recent solutions usually add previous volume rendering based reconstruction methods [60, 50] to 3DGS complement branch [44, 66, 111, use monocular depth and normal images as priors to bypass the messy and unordered 3D Gaussians [14, 57], or use surface-aligned Gaussians [25, 68, 14] in rasterization to approximate surfaces. However, how to learn continuous implicit representations to recover more accurate, smooth, and complete surfaces with sharp geometry details is still an open question.
To answer this question, we introduce a novel method to infer neural SDFs from multi-view RGB images through 3D Gaussian splatting. We progressively infer a signed distance field by training a neural network along with learning 3D Gaussians to minimize rendering errors through splatting. To more effectively constrain the surface inference with the multi-view consistency, we dynamically align 3D Gaussians with the zero-level set of the neural SDF, and render the aligned 3D Gaussians on the zero-level set by differentiable rasterization. Meanwhile, we update the neural SDF by pulling the neighboring space onto the disk determined by each 3D Gaussian on the zero-level set, which gradually refines the signed distance field near the surface. The capability of seamlessly merging neural SDFs with 3DGS not only get rid of the dependence of costly NeRFs like stochastic sampling on rays but also enables us to access the field attributes like signed distances and gradients during the splatting process, which provides novel perspective and versatile platform for surface reconstruction with 3DGS. The key to the 3D Gaussian alignment and neural SDF inference is a differentiable pulling operation which uses the predicted signed distances and gradients from the neural SDF. It provides a way of imposing geometry based constraints on 3D Gaussians besides the RGB based constraints through splatting. Our numerical and visual evaluations widely used benchmarks show our superiority over the latest methods in terms of reconstruction accuracy and recovered geometry details. Our contributions are listed below,
We propose infer neural SDF through splatting 3D Gaussians pulled on the zero-level set, which can more effectively constrain surface inference with the multi-view consistency. This enables to recover more accurate, smooth, and complete surfaces with geometry details.
We introduce to dynamically align 3D Gaussians to the zero-level set and update the neural SDF through differentiable pulling operation. To this end, we propose novel loss terms and training strategies to work with the discrete and sparse 3D Gaussians in surface reconstruction.
We achieve the state-of-the-an numerical and visual results in multi-view based surface reconstruction.
2
Related Work
Neural implicit representations have achieved remarkable progress in reconstructing 3D geometry with details [53, 48, 13, 45, 7, 21]. Neural implicit functions can be learned by either 3D supervisions, such as signed distances [53, 13, 41] and binary occupancy labels [48], or 2D supervisions, such as multi-view RGB images [60] and normal images [5]. In the following, we focus on reviewing methods of learning implicit representations from 2D and 3D supervisions separately. Then we provide a detailed discussion on the latest reconstruction methods based on 3D Gaussians.
2.1 Learning Implicit Representation from Multi-view Images
Neural radiance fields (NeRFs) [49] have become an essential technology for representing 3D scene through multi-view images. Many of its applications have been explored, resulting in significant advancements in areas such as acceleration [50, 12], dynamic scene [16, 4] and sparse rendering [58, 26]. Besides these applications, extracting accurate surfaces from NeRFs remains a challenge. Mainstream approaches typically design various differentiable formulas transform the density in radiance fields into implicit representations for volume rendering, such as signed distance function (SDF) [60, 39, 54], unsigned distance function (UDF) [42, 47, 15, 70] and occupancy [52]. With the leamed implicit function fields, post-processing algorithms [43, 21, 48] are applied to extract the zero level set to obtain the reconstructed meshes. Following methods introduce different priors from SfM [18, 69] large-scale datasets [67, 59, 40] to improve the reconstruction performance in large-scale scenes. Recent approaches focus on speeding up the neural rendering procedure, aiming to achieve high-quality meshes and rendering views within short period of training time. They

---

propose alternative data structures to replace the heavy MLP framework used in original NeRF, such as sparse voxel grid [17], multi-resolution hash grid [50, 61] and radial basis function [12], or design subtle differentiable rasterization pipelines to achieve real-time rendering [64,56) However, these methods still face the trade-off between rendering quality and training speed.
2.2 Learning Implicit Representation from Point Clouds
Since DeepSDF [53] and OceNet [48] were proposed, learning implicit representation from point clouds has achieved remarkable results in geometry modeling. These methods use ground truth signed distances and binary occupancy labels calculated from ground truth point clouds as supervisions to learn the implicit representation of shapes. The supervisions can serve as different kinds of global priors [3, 42, 55, 36] and local priors [62, 28, 38, 6]. which enables the neural implicit function to better capture geometry details and generalize to unseen shapes during inference. Some other methods infer SDFs without 3D supervisions. They train neural networks to overfit on single point clouds. These methods introduce additional constraints [19, 76], novel ways of using gradients [45, 74, 51, 35, 75], specially designed priors [46, 10, 9] and normals [2, 37, 34] to estimate signed or unsigned distances and occupancy, which use point clouds as reference.
2.3 Surface Reconstruction with 3D Gaussians
3D Gaussian Splatting [30] has become new paradigm in neural rendering due to its fast rendering speed, intuitive explicit representation and outstanding rendering performance. However, reconstruct- ing accurate surfaces from 3D Gaussian remains a challenge due to the messy, noisy, and unevenly distributed 3D Gaussians. To solve this problem, one kind of approaches involves combining 3D Gaussians with neural implicit surface functions [60, 50] to enhance the performance of both branches, which employs mutual supervisions between the two components [66, 11 1, 44]. Another kind of ap- proaches encourage the reduction from 3D Gaussians to 2D Gaussians with series of regularization terms, which ensures the Gaussian primitives to align with the object surfaces [25, 20, 14]. Addi- tionally, some methods introduce additional priors from large-scale datasets [57. 14] or multi-view stereo [63]. or use elaborately designed surface extraction algorithms [68, 65) to recover 3D geometry from 3D Gaussians. Although these efforts have achieved improved reconstructionss they are still limited in capturing fine-grained geometry and lack the precise perception of continuous implicit representations. Different from all these mentioned methods, we propose to seamlessly combine 3D Gaussians with the learning of neural SDFs. Our method provides a novel perspective to jointly learn 3D Gaussians and neural SDFs by more effectively using multi-vies view consistency and imposing geometry constraints.
3 Method
Overview. We aim infer a neural SDF f from posed multi-view RGB images (vi) as shown in Fig. 1. We learn 3D Gaussian functions {g, J/ with their attributes like color, opacity, and shape to represent the geometry and color in the 3D scene. Meanwhile, when learning the 3D Gaussians, we introduce novel constraints to infer the continuous surfaces with the neural SDF. We rely differentiable pulling operation and the differentiable rasterization to bridge the gap between the discrete Gaussians and the continuous neural SDF, align 3D Gaussians neural SDF, and back propagate the supervision signals from both the rendering errors and other geometry constraints jointly optimize 3D Gaussians and the neural SDF
the zero-level set of the
Neural Signed Distance Function. We leverage an SDF ff to represent the geometry of a scene. An SDF is implicit function that predict signed distance s at arbitrary location q, ie., s = f(g). Recent methods usually train a neural network to approximate an SDF from signed distance supervision or infer an SDF from 3D point clouds multi-view images. A level set is an iso-surface formed by the points with the same signed distance values. The zero-level set a special level set, which is formed by points with signed distance of O. We can use the marching cubes algorithm [43] to extract the zero-level set a a mesh surface. Another character of the zero-level set is that the gradient of the SDF f at query g on the zero-level set, i.e., V is the normal of q.
3D Gaussian Splatting. 3D Gaussians have become vital differentiable volume representation for scene modeling. We can learn set of 3D Gaussians {g, 1 each of which has a set of learnable
3

---

(a) Pulling Caussians
(b) Pulling Quectes
Aligned Caussian Disks
Pulling
Vfia)
Densification Rasterization
Learued Newra SDF Field
Fly
Regalarizing
Image
(c) Loss
(d) Tangent Loss
(e) Orthorgnal Loss
Figure I: Overview our method. We (a) pull 3D Gaussians onto the zero-level set for splatting, while (b) pulling the neighboring space onto the Gaussian disks for SDF inference. To better facilitate this procedure, we introduce three constraints: (c) push the Gaussians to become disks; (d) encourage the disk to be tangent plane the zero-level set; (e) constrain the query points to be pulled along the shortest path.
attributes including mean, variances, rotation, opacity, and color. We can render the learnable Gaussians {gy} into RGB images through the volume rendering equation below,
i-1
C'(u, = Σ 9j 00 (s
I
(1 on (x(
where Culu, v) is the rendered color at the pixel (u, v), C₁ oi and p, denote the color, the opacity, and the 2D projection of the j-th 3D Gaussian, respectively. At a query = [x, z|, the probability from the j-th 3D Gaussian is (g) exp(-0.5 (q my) Σ (q-1))). where j4y is the center of the j-th Gaussian, and Σ is the covariance matrix.
We can learn these 3D Gaussian functions through a differentiable rasterization. We render 3D Gaus- sians {gj) } into rendered RGB images v/2 and then, optimize the learnable attributes by minimizing the rendering errors to the ground truth observations v, where C" v) and Clu, v) are the rendered and the GT color values at pixel (u, v), ie.,, min(g,) (g,} [[C((u,)) v) Clu, v)..
Aligning 3D Gaussians with the Zero-level Set. Since 3D Gaussian splatting is so flexible in volume rendering, it does not require 3D Gaussians to locate on the geometry surface for good rendering quality. While we expect 3D Gaussians to locate on geometry surface, so that we can more effectively leverage them and multi-vies consistency as clues infer more accurate neural SDFs for reconstruction. To this end, we introduce a differentiable pulling operation to pull 3D Gaussians on the zero-level set of the neural SDF f, and then, we render the pulled 3D Gaussians through the splatting.
Specifically, inspired by Neural-Pull [45], we rely on the gradient field of the neural SDF f during the pulling operation. We move each one of the 3D Gaussians using the predicted signed distance s, f(1,) and the gradient V (u;)), where (1, the mean value of the 3D Gaussian. As shown in Fig. 1 (a), this pulling operation will turn the 3D Gaussian gj into 3D Gaussian 9'J that get projected onto the zero-level set of SDF f. where 9¹ shares the same attributes with gj but has different center n'e
Vl(g)) {
M/ = - 8j
(2)
Signed Distance Inference with Pulled 3D Gaussians. We infer signed distances in the field with pulled 3D Gaussians {g]} Pulled 3D Gaussians provide a coarse estimation of the surface, which we can use as reference. One challenge here that the sparsity and non-uniformly distributed 3D Gaussians do not show clear geometry clue for surface inference. Although previous methods like Neu [8] and OnSurfPrior [46] manage to learn continuous implicit functions from sparse points, it is still difficult to recover surfaces from both sparse and non-uniformly distributed points.

---

To overcome this challenge, we introduce an approach to estimate neural SDFs from sparse 3D Gaussians. Like Neural-Pull [45], we still use a differentiable pulling operation to pull neighboring space onto the surface but we regard the disk established by the shape of a 3D Gaussian as pulling target, rather than point, as shown in Fig. 2, which aims for larger target on surfaces. To this end, we impose constraints not only on the shape of 3D Gaussians but also on the pulling operation. Specifically, we introduce three constraints. The first one constrains 3D Gaussians to be a thin disk. The second constraint encourages the thin disk to be tangent plane the zero-level set. The third constraint pushes queries to get pulled onto the thin disk along the normal of the Gaussian.
The first constraint adds penalties if the smallest variance among the three variances of a 3D Gaussian Jj is too large, as shown in Fig. (c). Thus, the loss for thin disk Gaussian listed below,
Lrhin |I|
where TL, 72, and are variances along the three axes. Flattening a 3D Gaussian ellipsoid into a disk was first introduced in NeuSG [11] and has become consensus in recent Gaussian reconstruction works [25, 14]. The motivation is that 2D planar disk primitives are more suitable for surface representation, making it easier to apply alignment constraints. Additionally, we can naturally use the direction pointing along the axis with the minimum variance F = min F2, ra} to represent the normal ny of the Gaussian Øj-
Based on the thin disk shape of Gaussians, the second constraint encourages the pulled Gaussians {oz) to be the tangent plane the zero-level set, shown in Fig. (d). What do is align the normal ny of Gaussian 9j with the normal the center μú, of the pulled Gaussian gj on the zero-level set. We the gradient ((()) of the neural SDF at μl the expected normal here. Hence, we align the normal nj of a Gaussian with the normal V f (m()) on the zero-level set,
(S(K)) [[(a
LTangent
nl
(4)
With the disk-like Gaussians located the tangent plane, we introduce to sense the signed distance field by pulling randomly sampled queries on the Gaussian disks, as
Quary Points Zeru Pilling dinetions
shown in Fig. (b). Turning the pulling target from a point [45, 8] into a plane is based on the observation that the 3D Gaus- sian function with boundary can cover the surface more completely although their centers {MI,..., } which are sparse and non-uniformly distributed. Thus, we ex- pect the operation can pull a query onto a Gaussian disk plane. Fig. 2 demonstrates the improvement of pulling queries onto their nearest Gaussian disk planes over the nearest Gaussian centers. The comparisons show that pulling onto the disk plane can improve the robustness to the sparsity and non-uniformly Gaussian distribution. With learned Gaussian centers, pulling queries to centers can not recover the smooth and continuous geometry in areas where almost no Gaussian centers appear. While pulling queries to the Gaussian disk plane can recover more accurate and complete surfaces since the disk established by the learned variance of Gaussian functions mostly cover the gap.
Gaussian Centers Pulled to Centers Pulled to Disks Figure 2: Comparison of pulling Gaussians to centers and to disks. The former tends to overfit sparse Gaussian centers, resulting in incomplete meshes. We address this issue by pulling queries onto disk planes.
Specifically, at a query q, we pull it onto the zero-level set using a similar way in Eq. 2, i.e., z= 4-=-0a Vl(g)//V/t disk, we maximize its probability of belonging to its nearest pulled Gaussian ğ which is determined in terms of the distance between g and the Gaussian center p.
To encourage the query to get pulled onto the nearest pulled Gaussian
5

---

Lpwild:p)) e-122xlg'-ai c
- g = arg min lla] all.
(5)
(
We minimize the negative logarithm of the probability in implementation. Moreover, expect the pulling can follow a direction orthogonal to the disk plane, which leads to the minimum moving distance conform to the definition of signed distances. To this end, impose another constraint the gradient to ensure that the pulling can follow a path with the minimum distance to the nearest pulled Gaussian disk, as shown in Fig. (c).
Vg(g) [ nl
Lothorgnal
(6)
where the constraint aligns the gradient at query g and the normal ni of the pulled Gaussian disk g. Rendering, We also render the pulled Gaussians {9r} into images through splatting to add penalties on rendering errors, where {9') } are Gaussians pulled onto the zero-level set from the Gaussians {gr } by the neural SDF f in Eq. Each pair of gj and g' shares the same attributes expect the center location. The rendering error combines L1 term and D-SSIM term between rendered images {u and ground truth ones {wili= following original 3DGS [30].
L Splatting 0.8 (1%, + 0.2 LD- LD-SSIM (v/, (1)
Loss Function. We optimize attributes of Gaussians {gi} } and the parameters of neural SDF / by the following objective function, where a, B, y, and 8 are balance weights.
min Spiotting + aLThin + BBL ongentent + 7LPul LLothorgnal- to
(8)
Implementation Details. Our code is build upon the source code released by 3DGS [30]. Similar to [68], we make some changes to 3DGS's s densification strategy. The first one is to initialize the newly cloned Gaussians around the original Gaussians rather than the same positions. The second one is to encourage 3DGS to split larger Gaussians into smaller ones more frequently. These strategies aim to increase the number of primitives and to avoid underfitting in textureless areas. Regularization parameters are set to o=100, 880011, 7=1, 6=0.1. We optimize our model for a total of 15k iterations. We stop densification and incorporate the pulling and constraints 7k iterations. The SDF network is implemented as an MLP with 8 layers, 256 hidden units and ReLU activation function, and initialized sphere, following [45]. The parameters of the SDF network shares the same optimizer as that of 3D Gaussians. All the experiments are conducted on a single NVIDIA 3090 GPU.
4 Experiments
4.1 Experiment Settings
Evaluation Metrics and Datasets. We evaluate the performance of our method on widely adopted datasets including both object-level and large-scale ones, including DTU [27], Tanks and Temples (TNT) [31] and Mip-NeRF 360 (M360) [1]. To evaluate the accuracy of the reconstructed meshes, we use Chamfer Distance (CD) on DTU and F-score on TNT, using the official evaluation script. To evaluate the rendering quality in real-scene datasets, report PSNR, SSIM and LPIPS in evaluations on M360.
Baselines. We compare our geometry reconstruction accuracy with the statc-of-the-art 3DGS based reconstruction methods, including SuGaR [20], DN-Splatter [57]. GaussianSurfells [14] and 2DGS [25]. For real-world scenes which do not have ground truth meshes for evaluations, we compare the rendering quality with ffute-o--the-ar neural rendering methods, including Instant-NGP [50]. Mip-NeRF 360 [1] and BakedSDF [64].
Surface Extraction. An advantage of our approach over the latest methods is the simplicity of extracting surfaces. Different from methods like SuGaR [20] and GauS [65] which introduce

---

3DGS
SuGaR
DN- DNlSttater
2DGS
Ows
Reference Image
Figure 3: Visual comparisons on DTU dataset.
Table 1: Numerical comparisons in terms of CD on DTU dataset. Best results are highlighted as
1st
2nd
and 3rd
Methods
24
37
55
63
69
83
105
106
110
114
118
122
Mean
Time
NeuS [60]
1.90
1.37
0.93
0.43
1.10
0.65
0.57
1.48
1:09
0.83
0.52
1.20
0.35
0:49
0.54
0.84
-9h
3DGS [30]
2.14
1.53
2,08
1.68
3.49
2.21
1.43
2.07
2.22
1.75
1,79
2.55
1.53
1.52
1.50
1.96
15.1m
SuGaR [20]
DN DN-Splatter [57]
1.50
2.03
1.42
1.44
2.37
2.11
1.62
1.95
1.88
1.48
1.63
1.82
1.20
1.50
1.40
1.70
31.2m
GSarfels [14]
0.66
0.93
0.54
0.41
1.06
1.14
0.85
1:29
1.53
0.79
0.82
1.58
0.45
0.66
0.53
0.88
10.9m
2D0S
0.39
Ours
0.51
0.56
0.46
0.39
0.82
0.67
0.85
1.37
25
0.73
0.54
1.39
0.35
0.88
0.42
0.75
21.8m
Table Numerical comparisons on Tanks And Temples dataset. Best results are highlighted as
1st
2nd and 3rd
Methods
Barn
Caterpillar
Courthouse
Ignatius
Meetingroom
Truck
Mean
Time
NeuS [60]
0.29
0.29
0.17
0.83
0.24
0.45
0.38
-12h
3DGS [30]
0.13
0.08
0.09
0.04
0.01
0.19
0.09
20.5m
SuGaR [20]
0.14
0.16
0.08
0.33
0.15
0.26
0.19
2.1h
DN-Splatter [57]
0.15
0.11
0.07
0.18
0.01
0.20
0.12
54.9m
GSurfels [14]
0.24
0.07
0.24
0.21
15.1m
2DGS [25]
0.41
0.23
0.16
0,51
0.17
0.45
0.32
39.44 99.4m
Ours
0.60
0.37
0.16
0.71
0.22
0.52
0.43
37.6m
specially designed algorithms and take a long time for extracting surfaces, we adopt the marching cubes algorithm [43] to extract mesh surfaces with the learned neural SDF f. For small scale scenes, we use a resolution of 800 to extract surfaces, while we split large scale scenes into parts, each of which gets reconstructed with resolution of 800 to bypass the limitation of our computational resources.
4.2 Comparisons
Comparisons on DTU. We report accuracy reconstructed meshes and training time against baselines on DTU dataset in Tab. 1. Our method outperforms all Gaussian-based reconstruction methods terms of Chamfer Distance. Our method achieves comparable training time to the state- of-the-art Caassian-reconstruutioo method 2DGS [25] but gains better reconstruction accuracy than 2DGS. The visualization results in Fig. 3 highlight the advantages of our method. By employing alignment constraints and pulling operations between the 3D Gaussians and the neural SDF field, can reconstruct significantly smoother and more complete surfaces than the baselines.
Comparisons on TNT. We further evaluate our method using more challenging large-scale unbounded scenes on TNT dataset. Numerical comparisons Tab. show that we achieve higher F-score compared to baseline methods, even surpassing NeuS, which however takes about 12 hours to fit scene. Notably, as scene scale increases, the number of Gaussian primitives increases rapidly, causing the adjusted CUDA rasterization kernel of 2DGS to consume more time for rendering. In contrast, since our rasterization kernel is based on 3DGS, it is less sensitive to the number of Gaussians, which enables us to Iearn 3D Gaussians faster than 2DGS. We provide visual comparisons

---

3DGS
SuGaR
2DGS
Ours
Reference Image
Figure 4: Visual comparisons on Tanks and Temples dataset.
3DGS
SuGaR
2DGS
Owrs
Reference Image
Figure 5: Visual comparisons on Mip-NeRF 360 dataset.
in Fig. 4. Here we crop the reconstructed meshes to show the foreground objects that are of primary interest, as captured by the cameras. Please refer the appendix for the reconstruction results of the background regions. The visual comparisons demonstrate that we can reconstruct more complete and smooth object surfaces, such the ground, the truck's ss hood and the statue's left shoulder.
Comparisons on MipN- eRF 360. We further eval- uate our method in neural rendering for novel view synthesis on MipNeRF 360 dataset. We report the numerical comparisons in Tab. 3. Our competitive results against the state-of- the-art novel view synthe- sis methods indicate that our method able to im- pose effective geometric constraints without compro- mising rendering quality. This provides promising solution for learning continuous distance fields from discrete 3D Gaussians. Visual comparisons of mesh reconstructions are shown in Fig. 5, which demonstrate that our method is able to recover more smooth and complete surface by more effectively using the multi-view consistency.
Table 3: Quantitative evaluations of rendering quality on Mip-NeRF
360 [1] dataset. Best results are highlighted as 1st
2nd and 3rd.
Indoor Scene
Outdoor Scene
PSNR1
SSIM
LPIPS
PSNRT
SSIMI
LPIPS
NeRF[49]
26.84
0.790
0.370
21.46
0.458
0.515
Instant-NGP [50]
29.15
0.880
0.216
22.90
0.566
MipNeRF 360 [1]
31.72
0.917
0.180
24.47
0.691
0.283
BakedSDF [64]
27.06
0.839
0.258
22.47
0.585
0.349
3DGS [30]
30.99
0.926
0.199
24.24
0.705
0.283
SuGaR [20]
29.44
0.911
0.216
22.76
0.631
0.349
2DGS [25]
30.39
0,924
0.183
24.33
0.709
0.284
Ours
30.78
0.925
0.182
23.76
0.703
0.278

---

Table 4: Ablation studies on DTU dataset.
Pulling
Constraint Terms
Mesh Extractions
Methods
Pulled to centers
w/o Pull GS
w/o Leyhon
w/o Lyan
w/o Losh
TSDF
Poisson
Full model
CDL
0.85
0.90
0.78
0.82
0.79
1.41
0.79
0.75
4.3
Ablation Studies
In this section, we conduct ablation studies on the key techniques of our method demonstrate their effectiveness. The full quantitative results are reported in Tab. 4, which are conducted on all scenes in DTU dataset [27].
Pulling Operations. We first examine the effect of pulling Gaussians onto the zero-level set, as reported in Tab. 4 ("w/o Pull Gaussians" vs. "Ours"). The original 3DGS tends to produce floating ellipsoids near the object surfaces overfit the train- ing views. By pulling the Gaussians to the zero-level set of the SDF field, the Gaus- sians are consistently distributed on the sur- face. As shown in Fig. 6a, after getting pulled onto the zero-level set, the Gaussian centers are distributed on thin layer of the object surface, thus achieving ac- curate geometry estimation. Meanwhile, pull neighboring space onto Gaussian disks to learn neural SDFs. Comparing to NeuralPull [45] which pulls query points centers, we innovatively pull query points to Gaussian disks, which bridge the gap between continuous SDF field and sparse Gaussian distributions, highlighted in Fig. and Tab. ("Pulled to centers" vs. "Ours").
weo Pullin Geassian
Pulling Gaesias
(a) Visualization Gaussian centers with or without pulled onto zero-level set. We are ahle to obtain consistent and smooth Gaussian distributions by pulling operation.
Originel JDGS
CAtS
(b) Comparisons between Gaussian ellipsoids learned by orig- inal 3DGS and Gaussian disks learned by our method.
Constraint Terms. We further explore the effect of our constraint terms, reported in Tab. ("Constraint Terms"). Our full model provides the best performance when applying all constraint terms. The orthog- onal loss helps to learn more regularized SDF field, while the thin loss and tangent loss provide constraints to align the orienta- tion of Gaussian disks with the gradient neural SDF on the zero-level sets, resulting in good normal field and a reconstructed mesh, as shown in Fig. 6b, 6c.
W/O Langent
w/ Ayungent
Reconstructod Mush
(c) Qualitative ablation studies for Tangent loss.
TSDF Fusion CD-241
Screened Poisson
Marching Cubes CD=0,88
CD=0.98
(d) Comparisons of different mesh extraction methods.
Mesh Extraction. We also report the re- construction accuracy using TSDF fusion
and screened Poisson reconstruction [29], as shown in Fig. 6d and Tab. 4 ("Mesh Extractions")
For TSDF fusion, render depth maps and fuse them using voxel size of 0.004 and truncation threshold as 0.02, the same as 2DGS [25]. For screened Poisson, we use the Gaussian centers and normals input. Unlike 2DGS [25] and GSurfels [14] which incorporate rendered depth into the differrnntiable rasterization pipeline, we do not directly optimize depths, resulting in noisy depth maps and unsatisfactory reconstruction results. However, since the positions and normals of the Gaussians are well optimized through our approach, screened Poisson reconstruction can achieve relatively good results.

---

Gradient Constraint. We follow Neural- Pull [45] use normalized SDF gradient for pulling operation. We report the result with additional Eikonal term [19] to explicitly con-
strain the gradient length, as shown in Fig. 7. The result significantly degenerated because that Neural-Pull depends on both predicted SDF values and gradient directions to optimize the SDF field. makes the optimization even more complex when adding additional constraint on the gradient length.
wo Eikonal loss
w/ Fikonal lass
Figure 7: Visualization of the effect of eikonal loss.
Conclusion
We propose a method to learn neural SDFs for multi-view surface reconstruction with 3D Gaussian splatting. Our results show that more accurate, smooth, and complete surfaces with geometry details by rendering 3D Gaussians pulled the zero-level set. To this end, we dynamically align 3D Gaussians the zero-level set and update neural SDFs through both differentiable pulling and splatting for both RGB and geometry constraints. Our methods successfully refine the signed distance field near the surface in progressive manner, leading to plausible surface reconstruction. Our ablation studies justify the effectiveness of our novel modules, loss terms, and training strategies. Our evaluations show our superiority over the latest methods in terms of accuracy, completeness, and smoothness.
more effectively leverage multi-view consistency recover
Acknowledgement
This work was supported by National Key R&D Program of China (2022YFC3800000, and the National Natural Science Foundation of China (62272263, 62072268), and in part by Tsinghua- Kuaishou Institute of Future Media Data.
References
[1] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip- NeRF 360: Unbounded anti-aliased neural radiance fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022.
[2] Alexandre Boulch, Pierre-Alas Langlois, Gilles Puy, and Renaud Marlet. NeeDrop: Self- supervised shape representation from sparse point clouds using needle dropping. In International Conference 3D Vision, 2021.
[3] Alexandre Boulch and Renaud Marlet. POCO: Point convolution for surface reconstruction. In IEEE Conference Computer Vision and Pattern Recognition, 2022.
[4] Ang Cao and Justin Johnson. HexPlane: A fast representation for dynamic scenes. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130-141, 2023.
[5] Xu Cao and Takafumi Taketomi. Supernormal: Neural surface reconstruction via multi-view normal integration. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 20581-20590, 2024.
[6] Chao Chen, Zhizhong Han, and Yu-Shen Liu. Learning local pattern modularization for point cloud reconstruction from unseen classes. European Conference on Computer Vision, 2024.
[ Chao Chen, Yu-Shen Liu, and Zhizhong Han. GridPull: Towards scalability in learning implicit representations from 3d point clouds. Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 18322-18334, 2023.
10

---

[8] Chao Chen, Yu-Shen Liu, and Zhizhong Han. Unsupervised inference of signed distance functions from single sparse point clouds without learning priors. Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 17712-17723, 2023.
[9] Chao Chen, Yu-Shen Liu, and Zhizhong Han. Inferring neural signed distance functions by overfitting on single noisy point clouds through finetuning data-driven based priors. In Advances in Neural Information Processing Systems, 2024.
[10] Chao Chen, Yu-Shen Liu, and Zhizhong Han. NeuralTPS: Learning signed distance functions without priors from single sparse point clouds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[11] Hanlin Chen, Chen Li, and Gim Hee Lee. NeuSG: Neural Implicit Surface Reconstruction with
3D Gaussian Splatting Guidance. arXiv preprint arXiv:23/2.00846, 2023.
[12] Zhang Chen, Zhong Li, Liangchen Song, Lele Chen, Jingyi Yu, Junsong Yuan, and Yi Xu. NeuRBF: neural fields representation with adaptive radial basis funetions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4182-41944 2023.
[13] Gene Chou, Ilya Chugunov, and Felix Heide. GenSDE: Two-Stage Learning of Generalizable Signed Distance Functions. In Advances in Neural Information Processing Systems, 2022.
[14] Pinxuan Dai, Jiamin Xu, Wenxiang Xie, Xinguo Liu, Huamin Wang, and Weiwei Xu. High- quality surface reconstruction using gaussian surfels. In SIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024.
[15] Junkai Deng, Fei Hou, Xuhui Chen, Wencheng Wang, and Ying He. 2S-UDF: A Novel Two- stage UDF Learning Method for Robust Non- don-tttertight Model Reconstruction from Multi-view Images. 2024.
[16] Sara Fridovich- Keil,, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-Planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12479-12488, 2023.
[17] Sara Fridovich- Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5501-5510, 2022.
[18] Qiancheng Fu, Qingshan Xu, Yew-Soon Ong, and Wenbing Tao. Geo-Neus: Geometry- consistent neural implicit surfaces learning for multi-view reconstruction. Advances in Neural Information Processing Systems, 2022.
[19] Amos Gropp. Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. In International Conference Machine Learning, pages 3789-3799. PMLR, 2020.
[20] Antoine Guédon and Vincent Lepetit. SuGaR: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[21] Benoit Guillard, Federico Stella, and Pascal Fua. MeshUDF: Fast and differentiable meshing of unsigned distance field networks. In European Conference on Computer Vision, pages 576-592. Springer, 2022.
[22] Yasaman Haghighi, Suryansh Kumar, Jean-Philippe Thiran, and Luc Van Gool. Neural Implicit Dense Semantic SLAMM. arXiv preprint arXiv:2304. 14560, 2023.
[23] Liang Han, Junsheng Zhou, Yu-Shen Liu, and Zhizhong Han. Binocular-guided 3d gaussian splatting with view consistency for sparse view synthesis. Advances in Neural Information Processing Systems, 2024.
11

---

[24] Pengchong Hu and Zhizhong Han. Learning neural implicit through volume rendering with attentive depth fusion priors. In Advances in Neural Information Processing Systems, 2023.
[25] Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. In SIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024.
[26] Han Huang, Yulun Wu, Junsheng Zhou, Ge Gao, Ming Gu, and Yu-Shen Liu. NeuSurf: On- surface priors for neural surface reconstruction from sparse input views. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 2312-2320, 2024.
[27] Rasmus Jensen, Anders Dahl, George Vogiatzis, Engil Tola, and Henrik Aanæs, Large scale multi-view stereopsis evaluation. In IEEE Conference Computer Vision and Pattern Recog- nition, pages 406 413, 2014.
[28] Chiyu Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, Thomas Funkhouser, et al. Local implicit grid representations for 3D scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6001-6010, 2020.
[29] Michael Kazhdan and Hugues Hoppe. Screened poisson surface reconstruction. ACM Transac- tions on Graphics (ToG), 32(3):1-13, 2013.
[30] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 4214 2023.
[31] Arno Knapitsch, Jaesik Park, Qian Yi Zhou, and Vladlen Koltun. Tanks and Temples: Bench- marking large-scale scene reconstruction. ACM Transactions on Graphics, 36(4), 2017.
[32] Xin Kong, Shikun Liu, Marwan Taher, and Andrew J Davison. vMAP: Vectorised object mapping for neural field slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 952-961, 2023.
[33] A. Laurentini. The visual hull concept for silhouette-based image understanding. IEEE Trunsactions on Puttern Analysis and Machine Intelligence, 16(2):150-162 1994.
[34] Qing Li, Huifang Feng, Kanle Shi, Yue Gao, Yi Fang, Yu-Shen Liu, and Zhizhong Han. Learning signed hyper surfaces for oriented point cloud normal estimation. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2024.
[35] Shengtao Li, Ge Gao, Yudong Liu, Ming Gu, and Yu-Shen Liu. Implicit filtering for learning neural signed distance functions from 3d point clouds. European Conference on Computer Vision, 2024.
[36] Shengtao Li, Ge Gao, Yudong Liu, Yu-Shen Liu, and Ming Gu. GridFormer: Point-grid transformer for surface reconstruction. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 3163-3171, 2024.
[37] Shujuan Li, Junsheng Zhou, Baorui Ma, Yu-Shen Liu, and Zhizhong Han. NeAF: Learning neural angle fields for point normal estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1396-1404, 2023.
[38] Shujuan Li, Junsheng Zhou, Baorui Ma, Yu-Shen Liu, and Zhizhong Han. Learning continuous implicit field with local distance indicator for arbitrary point cloud upsampling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 3181-3189, 2024.
[39] Zhaoshuo Li, Thomas Müller, Alex Evans, Russell Taylor, Mathias Unberath, Ming- Yu Liu, and Chen-Hsuan Lin. Neuralangelo: High-fideltyy neural surface reconstruction. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognûtion, pages 8456-8465, 2023.
12

---

[40] Zhihao Liang, Zhangjin Huang, Changxing Ding, and Kui Jia. HelixSurf: A robust and efficient neural implicit surface learning of indoor scenes with iterative intertwined regularization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 13165-13174, 2023.
[41] David B Lindell, Dave Van Veen, Jeong Joon Park. and Gordon Wetzstein. Bacon: limd-limited coordinate networks for multiscale scene representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16252- 162622 2022.
[42] Xiaoxiao Long, Cheng Lin, Lingjee Liu, Yuan Liu, Peng Wang, Christian Theobalt, Taku Komura, and Wenping Wang. Neu Learning unsigned distance fields for multi-view the
reconstruction of surfaces with arbitrary topologies. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20834 20843, 2023.
[43] William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3D surface construction algorithm. Computer Graphics, 21(4):163-169, 1987.
[44] Xiaoyang Lyu, Yang-Tian Sun, Yi-Hua Huang. Xiuzhe Wu, Ziyi Yang. Yilun Chen, Jiangmiao Pang, and Xiaojuan Qi. 3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting. arXiv preprint arXiv:2404.000 00409, 2024.
[45] Baorui Ma, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. Neural-Pull: Learning signed distance function from point clouds by learning to pull space onto surface. In International Conference Machine Learning, pages 7246-7257. PMLR, 2021.
[46] Baorui Ma, Yu-Shen Liu, Matthias Zwicker, and Zhizhong Han. Reconstructing surfaces for sparse point clouds with on-surface priors. In IEEE Conference Computer Vision and Pattern Recognition, 2022.
[47] Xiaoxu Meng, Weikai Chen, and Bo Yang- NeAT: Learning neural implicit surfaces with arbitrary topologies from multi-view images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 248-258, 2023.
[48] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3D reconstruction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4460-4470, 2019.
[49] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV), pages 405-421. Springer, 2020.
[50] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with multiresolution hash encoding. ACM Transactions Graphics (ToG), 41(4):1- 15, 2022.
[51] Takeshi Noda, Chao Chen, Weiqi Zhang, Xinhai Liu, Yu-Shen Liu, and Zhizhong Han. MultiP- ull: Detailing signed distance functions by pulling multi-level queries at multi-step. In Advances in Neural Information Processing Systems, 2024.
[52] Michael Oechsle, Songyou Peng. and Andreas Geiger. UNISURF: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5589-5599, 2021.
[53] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pages 165-174 2019.
[54] Minyoung Park, Mirae Do, Yeon Jae Shin, Jaeseok Yoo, Jongkwang Hong, Joongrock Kim, and Chul Lee. H20-SDF: Two-phase learning for 3d indoor reconstruction using object surface fields. In The Twelfth International Conference Learning Representations, 2023.
13

---

[55] Songyou Peng, Michael Niemeyer, Lars Mescheder, Mare Pollefeys, and Andreas Geiger. Convolutional occupancy networks. In European Conference on Computer Vision, pages 523-540. Springer, 2020.
[56] Christian Reiser, Rick Szeliski, Dor Verbin, Pratul Srinivasan, Ben Mildenhall, Andreas Geiger, Jon Barron, and Peter Hedman. MeRF: Memory-efficiet radiance fields for real-time view synthesis in unbounded scenes. ACM Transactions on Graphics (TOG), 42(4):1 2023.
[57] Matias Turkulainen, Xugian Ren, Iaroslav Melekhov, Otto Seiskari, Esa Rahtu, and Juho Kannala. DN-Splatter: Depth and normal priors for gaussian splatting and meshing. arXiv preprint arXiv:2403. 17822, 2024.
[58] Guangcong Wang. Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. SparseNeRF: Distilling depth ranking for few-shot novel view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9065-9076, 2023.
[59] Jiepeng Wang, Peng Wang, Xiaoxiao Long, Christian Theobalt, Taku Komura, Lingjie Liu, and Wenping Wang. NeuRIS: Neural reconstruction of indoor scenes using normal priors. In European Conference on Computer Vision, 2022.
[60] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. Advances Neural Information Processing Systems, 34, 2021.
[61] Yiming Wang, Qin Han, Mare Habermann, Kostas Danilidiss, Christian Theobalt, and Lingjie Liu. NeuS2: Fast learning of neural implicit surfaces for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 3295-3306. 2023.
[62] Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. Deep geometric prior for surface reconstruction. IEEE Conference on Computer Vision and Puttern Recognition, 2019.
[63] Yaniv Wolf, Amit Bracha, and Ron Kimmel. Surface reconstruction from gaussian splatting via novel stereo views. European Conference on Computer Vision, 2024.
[64] Lior Yariv, Peter Hedman, Christian Reiser, Dor Verbin, Pratul P Srinivasan, Richard Szeliski, Jonathan T Barron, and Ben Mildenhall. Bakedsdf: Meshing neural sdfs for real-time view synthesis. ACM SIGGRAPH 2023 Conference Proceedings, pages 1-9, 2023.
[65] Chongjie Ye, Yinyu Nie, Jiahao Chang, Yuantao Chen, Yihao Zhi, and Xiaoguang Han. GauStudio: A Modular Framework for 3D Gaussian Splatting and Beyond. arXiv preprint arXiv:2403. 19632, 2024.
[66] Mulin Yu, Tao Lu, Linning Xu, Lihan Jiang, Yuanbo Xiangli, and Bo Dai. GSDF: 3DGS Meets SDF for Improved Rendering and Reconstruction. arXiv preprint arXiv:2403. 16964, 2024.
[67] Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger. MonoSDF: Exploring monocular geometric cues for neural implicit surface reconstruction. Advances in Neural Information Processing Systems, 2022.
[68] Zehao Yu, Torsten Sattler, and Andreas Geiger. Gaussian Opacity Fields: Efficient and compact surface reconstruction in unbounded scenes. arXiv preprint arXiv:2404. 10772, 2024.
[69] Jingyang Zhang. Yao Yao, Shiwei Li, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. Critical regularizations for neural surface reconstruction in the wild. In Proceedings of the IEEF/CVF Conference on Computer Vision and Patlern Recognition, pages 6279,,
2022.
[70] Wenyuan Zhang, Kanle Shi, Yu-Shen Liu, and Zhizhong Han. Learning unsigned distance functions from multi-view images with volume rendering priors. European Conference on Computer Vision, 2024.
14

---

[71] Wenyuan Zhang, Ruofan Xing, Yunfan Zeng, Yu-Shen Liu, Kanle Shi, and Zhizhong Han. Fast learning radiance fields by shooting much fewer rays. IEEE Transactions on Image Processing. 32:2703-2718 2023.
[72] Junsheng Zhou, Yu-Shen Liu, and Zhizhong Han. Zero-shot scene reconstruction from single images with deep prior assembly. In Advances in Neural Information Processing Systems, 2024.
[73] Junsheng Zhou, Baorui Ma, Shujuan Li, Yu-Shen Liu, and Zhizhong Han. Learning a more continuous zero level set in unsigned distance fields through level set projection. In Proceedings of the IEEE/CVF international conference on computer vision, 2023.
[74] Junsheng Zhou, Baorui Ma. Yu-Shen Liu, Yi Fang, and Zhizhong Han. Learning consistency- aware unsigned distance functions progressively from raw point clouds. In Advances Neural Information Processing Systems, 2022.
[75] Junsheng Zhou, Baorui Ma, Yu-Shen Liu, and Zhizhong Han. Fast learning of signed distance functions from noisy point clouds via noise to noise mapping. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[76] Junsheng Zhou, Baorui Ma, Wenyuan Zhang. Yi Fang, Yu-Shen Liu, and Zhizhong Han. Differentiable registration of images and lidar point clouds with exclpoint-to-pixxt matching. Advances in Neural Information Processing Systems, 36, 2024.
[77] Junsheng Zhou, Weiqi Zhang, and Yu-Shen Liu. DiftGS: Functional gaussian splatting diffusion. In Advances in Neural Information Processing Systems, 2024.
[78] Zihan Zhu, Songyou Peng, Viktor Larsson, Zhaopeng Cui, Martin R Oswald, Andreas Geiger, and Mare Pollefeys. Nicer-SLAM: Neural implicit scene encoding for rgb slam. In 2024 International Conference on 3D Vision (3DV), pages 42-52. IEEE, 2024.
15

---

A Appendix
A.1 Background Reconstruction
SuGaR
Ours
Reference Image
Figure 8: Visualization of reconstructed backgrounds.
Since SDF field learned by fitting Gaussian ellipsoids, it can infer implicit surfaces any location where Gaussians are distributed. Therefore, our method has the same capability to reconstruct backgrounds as methods like TSDF fusion, shown in Fig. 8. Current works generally utilize screened Poisson or TSDF fusion to reconstruct meshes [25, 20] and tend to reconstruct large sky spheres in the background. Our method learns neural SDFs and utilize marching cubes to reconstruct mesh, which avoid such bad cases.
A.2 Theoretical Analysis
We provide theoretical analysis here to demonstrate the advantage of pulling queries onto disks compared to pulling queries onto centers. We provide visual compari- son of the two strategies in Fig. 9, showcasing the changes of the loss function and the loss gradients as the query point approaches the Gaussian center. As the query point gets closer to the Gaussian center, the loss function of "pulling to centers" decays at a constant rate, and the gra- dient of the loss remains constant. contrast, for "pulling to disks", the loss function decreases quadratically, and the gradient of the loss gradually diminishes. This means that under the influence of the disk loss, as the query point approximates the center. the received gradient becomes smaller, reducing the driving force that pushes the query point towards the center. In other words, the disk loss has higher "tolerance" for the query point not being pulled to the center. This explains why we can learn a continu- ous field from sparse and non-uniformity distribution of Gaussian ellipsoids using the disk loss, whereas the center loss would lead to the SDF field overlitting to every Gaussian center.
Plunter
Lonter
Lask
....
[basx]]
distance center
Figure 9: Visualization of loss and gra- dient between Pulling to centers and Pulling to disks with the distance of query point to the Gaussian center.
A.3 Limitations & Future Works
While method successfully recovers accu- rate appearance and geometry reconstruction for wide range of objects and scenes, it also has several limitations. Firstly, the neural SDF is seamlessly integrated with Gaussian ellipsoids, making it difficult to avoid the inherent draw- backs of original 3D Gaussians, such as the lack of transparent objects and areas with strong spec- ular reflections. Secondly, although we address the issue of learning continuous SDF field from sparse and non-uniformly distributed Gaus- sian ellipsoids by pulling query points to disks,
mmn
Reconstructed mash
Refereace Image
Figure 10: Failure case. This is because the SDF network cannot accurately capture highffrequency details due to the smooth characteristic of MLPs.
16

---

our method shows limited performance in ex-
tremely sparse areas. very distant regions of unbounded scenes or areas with colors similar the background color, where 3DGS reconstructs to produce holes. Thirdly, due to the continuous and smooth characteristics of MLPs, our SDF tends to capture the low-frequency features of objects, making it difficult to reconstruct high-frequeng details. failure case is shown in Figure. 10, where we can reconstruct the very smooth tablecloth but fail to recover the details of the lego. There are two potential solutions for this issue in the future: one to enhance the representation capability the SDF by integrated with latest implicit representation learning methods, such as BACON [41] and GridPull [7]; the other one is to dig into the capabilities of TSDF fusion and screened Poisson in reconstructing our SDF field, which have the ability to reconstruct arbitrary resolution details.
ellipsoids or only few ellipsoids, our method tends
A.4 More Results
We provide additional reconstruction results in Fig. 11, 12, 13, which further justifies the superiority of our method. We notice that there are some holes on the flowerbed area in Fig. 13. This is due to the overly complex geometric structures and a lack of view covering, thus emitting a significant under-fitting issue. This results in a set of extremely sparse, huge, and unevenly distributed Gaussians, which makes Gaussians are thick ellipsoid like shape rather than relatively thin plans, leading to poor sense of surface. Although these huge Gaussians may work well in rendering. but they cannot recover any geometry covered by them. How to control the size of Gaussians for SDF inference could be an interesting future work direction.
We also visualize the error maps on meshes obtained by 2DGS and ours in Fig. 14, which highlights our superiority in terms of the accuracy of extracted surfaces. The surfaces learned by 2DGS are usually fat and a little bit drift away from ground truth surfaces, although their meshes seem to show more details. Our method is able to capture more accurate surfaces by using 3D Gaussians pulled onto the zero-level set and pulling query points onto Gaussian disks at the same time, leading to much more accurate zero-level set.
Figure 11: More visualization results on DTU dataset.
Figure 12: More visualization results on TNT dataset.
17

---

Figure 13: More visualization results on MipNeRF 360 dataset.
2DGS
Ours
Large Emor
Small Error
Figure 14: Error maps between 2DGS and our method on TNT dataset.
A.5 Discussion
About open surfaces. Since there are lots of non-closed surfaces in large- scale scenes, a natural solution is to learn unsigned distance field to re- construct open structures [70, 73, 33]. However, extracting the zero-level set from UDF as a mesh surface is still challenge, resulting in artifacts and outliers on the reconstructed meshes. We report the result of learning UDFs instead SDFs in Fig. 15, which shows the shortcomings of UDF learning. To avoid the influence of double-layer
Leeming SDF
Learing UDF
Figure 15: Visualization of surfaces reconstructed by SDF and UDF.
surfaces on evaluation accuracy under the SDF settings, we practically delete the back faces according to the visibility of each face under each camera view. Through this way, we can accurately reconstruct open structures with single-layer surfaces.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations and failure cases in the Appendix A.3.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

We discussed why pulling queries onto disks is better than pulling queries onto centers in the appendix. There is no theoretical theorems or lemmas in it, but We tried our best to give theoretical analysis for this discussion.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide the detailed information in reproducing our methods in Section 3 the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
20

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We provide our demonstration code as part of our supplementary materials. We will release the source code, data and instructions upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
ceabbbiissioooooccyy. for more details. httee//uiddee/oeeuueeeeeeeeeeeee
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ httppsscccppbpiccccceeceeecl geides/ CodeeSbbm ssi aasonnooicyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the training and testing details in the experiment section (Section
4).
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[No]
Justification:
We report the average performance the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
21

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report our training time with baseline methods in the experiment part.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttpas//////us////eeee cc httpppuulllthicccccceee csCuidel ines?
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
We discuss the applications and potential impacts of our method in the introduction.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
22

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We use the open- sourced datasets and codes under their licenses.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi wwwode.hhoe...c. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
23

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.


Reviews:
[{"review_id": "VasqJ38Q11", "reviewer": "Reviewer_CCgx", "summary": "The paper proposes a extension to 3D Gaussian Splatting by making it consistent with a neural SDF that is learnt along with the 3D Gaussians. To make it consistent, the Gaussians are projected to the zero level set, and the neural SDF is optimized to represent the SDF of the surface implied by the Gaussians. As the latter requires the Gaussians to be thin (essentially discs), they encourage this with a loss, and further encourage other properties that hold when the two representations are consistent.", "questions": "- For $L_{Tangent}$, how do you determine orientation of the normal?\n- Ablations study results don't line up with each other and with TNT results (0.41 vs 0.42 vs 0.43 vs 0.46)? A little confused/concerned here.\n- End of page 5, should $n_j$ be $\\bar{n}_j$?", "limitations": "Addressed in the appendix/supplemental", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Simple but effective method with well thought out losses\n- Good visualisations to explain the benefits of components (Figs 2,6-9)", "weaknesses": "- The proposed method uses a loss to make the Gaussians to be flat, why not directly use 2D Gaussians/Surfels? Those papers mention that direct parameterisation is much better than using a loss to flatten Gaussians, why does your method not suffer from issues from not having an exactly flat Gaussian? It would be interesting to have a comparison of your method with 2D Gaussians. However, I would expect at least some discussion of this (whether it is actually important to have exactly flat Gaussians or not). \n- While I like this way of constraining the Gaussians better than previous methods, the novelty is a bit limited since there are similar methods (improving GS using SDFs and having Gaussians be flat)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Comparison between flat loss and surfels.** Our method requires calculating the inverse of a three-dimensional covariance matrix (Eq.(5) in our paper) to deterimine the distribution probability of a query point within its nearest Gaussian ellipsoid. This allows us to maximize the probability for pulling query points towards nearest Gaussians. The surfel setting, however, merely provides two dimensional of scaling factors, which does not meet our requirement. In general, pushing Gaussians to be flat or setting Gaussians as surfels serves a similar purpose in mitigating the bias of rendering depth and determining Gaussian normals, thus the flatting loss is a reasonable choice. As an evidence, we replace our flat loss (Eq.(3) in our paper) with the surfel setting in GaussianSurfels[1], as shown in **Fig. A** in the rebuttal PDF. The optimization procedure fails at the start with surfel setting, which demonstrates that it is not a good choice in the differentiable pulling operation.\t\n\n**2. About L_tangent.** We consider the direction of the smallest scaling factor as Gaussian normals, following previous works[2,3]. The loss pushes the normal to get aligned with the gradient on the zero-leve set. \n\n**3. About $n_j$.** It should be $\\bar{n_j}$. We will correct it in our version.\n\n\n\n[1]. Dai P, Xu J, Xie W, et al. High-quality Surface Reconstruction using Gaussian Surfels. ACM SIGGRAPH 2024 Conference Papers. 2024: 1-11.\n\n[2]. Guédon A, Lepetit V. SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 5354-5363.\n\n[3]. Chen H, Li C, Lee G H. NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance. arXiv preprint arXiv:2312.00846, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jn9VDvLdDb", "reviewer": "Reviewer_5rKf", "summary": "This paper focuses on the challenge of inferring a signed distance function (SDF) for multi-view surface reconstruction from 3D Gaussian splatting (3DGS), which is hindered by the discreteness, sparseness, and off-surface drift of the 3D Gaussian. To overcome these challenges, the authors propose a method that seamlessly integrates 3DGS with the learning of neural SDFs. This approach constrains SDF inference with multi-view consistency by dynamically aligning 3D Gaussians on the zero-level set of the neural SDF and then rendering the aligned 3D Gaussians through differentiable rasterization. Through the utilization of both differentiable pulling and splatting, the approach jointly optimizes 3D Gaussians and neural SDFs with both RGB and geometry constraints, resulting in the generation of more accurate, smooth, and complete surfaces. Extensive experimental comparisons on various datasets demonstrate the superiority of the proposed method.", "questions": "See weakness for details.", "limitations": "The authors have thoroughly discussed the limitations of the proposed method. I believe that incorporating monocular priors during optimization would be beneficial to the learning of SDF. Additionally, I think that further exploration of the relationship between densification and SDF values would be an interesting and valuable research pursuit.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The approach of optimizing the neural Signed Distance Function (SDF) using only the regularization from 3DGS, without any monocular geometry supervision, is both novel and compelling. It effectively circumvents the time-consuming point sampling process for volume rendering, commonly utilized in prior studies for SDF learning. The method's natural balance between 3DGS and the neural SDF is simple and intuitive, showcasing the potential for seamless integration of explicit 3DGS and implicit neural fields.\n\n- The proposed method has demonstrated state-of-the-art reconstruction results on the DTU and TNT datasets, with each component of the approach undergoing thorough examination in ablation studies.\n\n- This paper is well-structured and easy to follow.", "weaknesses": "- It seems that the authors did not incorporate the Eikonal loss, which is commonly used to regularize SDF values in space during optimization. Could the authors provide specific reasons for this omission? Additionally, it is not entirely clear to me how the pull loss ensures that the optimized field satisfies the properties of SDF.\n\n- In Eq8, the weight of the splatting loss appears to be substantially larger than the others. Could the authors explain how these weights are determined?\n\n- I noticed that the densification operation and the pull/constraint losses are employed at different stages. How does the simultaneous use of densification and pull loss affect the final result? Stopping the densification operation in the second stage seems unreasonable because it may limit the performance of 3DGS.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Eikonal loss.** We believe pulling based methods may not need an Eikonal loss to guarantee the learned distance field is an SDF. This is because we use a normalized gradient and a predicted signed distance when pulling a query. It has been proved by NeuralPull[1] that adding the Eikonal loss will significantly degenerate the learning of SDF, as reported by ``Gradient constraint'' in Table 8 in the original paper of NeuralPull. The reason is that NeuralPull depends on both predicted SDF values and gradient directions to optimize the SDF field. It makes the optimization even more complex when adding additional constraint on the gradient length. We also additionally report a result with an Eikonal loss in **Fig. B** in our rebuttal PDF. The comparison indicates that the results degenerate significantly with an Eikonal loss.\n\n**2. SDF property.** It has been proved that an MLP which is trained using a pulling loss can converge to a signed distance function. The reason is that the sign of distances needs to turn over when across the surface points so that the pulling loss can be minimized on both sides of the surface points. Please see Theorem 1 in the original paper of NeuralPull for more details.\n\n**3. Weight of splatting loss.** We set the weight of splatting loss to 1.0, following all 3DGS baseline methods. And we set the weights of thin loss, tangent loss, pull loss, orthogonal loss as 100, 0.1, 1, 0.1, respectively. The weight of thin loss is larger than others because the scaling factor is usually small and we want the smallest scaling factor to be as small as possible, which is the same as NeuSG[2]. A smaller weight of thin loss will cause Gaussians to be not flat enough, leading to an inaccurate alignment of Gaussians to the zero-level set.\n\n**4. Densification operation and pull loss.** There are three reasons why we separate the densification operation and pulling operation. **(1)** The 3D Gaussians are sparse and unstable at the early stage of training. Current works usually start to add additional losses after certain epochs of training 3DGS, such as the distortion loss (Eq.13) in 2DGS[3] and the regularization loss (Eq.8) in SuGaR[4]. **(2)** The densification operation causes the number of Gaussians to grow rapidly, leading to a collapse of memory overflow. Therefore, current works generally stop densification after certain epochs of training. For example, original 3DGS stops densification at 15000 epoch while SuGaR stops it at 7000 epoch. Here we follow the setting of SuGaR. **(3)** The pulling operation needs to sample query points for each Gaussian, which is time-consuming. Therefore, we only sample query points for all Gaussians once before adding pulling loss for efficiency. If the number of Gaussians are changing during densification, we need to re-sample query points, which will bring additional time consumption. We conduct an ablation study on different stages of stopping densification or start pulling, on DTU scan37 scene, as reported in **Tab. A** in our rebuttal PDF. The first one is to start pulling loss from beginning, which significantly degenerates the performance and slows down the convergence. Another one is to add densification operation until end of training, which takes much longer training time and degenerates the performance.\n\n[1]. Ma B, Han Z, Liu Y S, et al. Neural-Pull: Learning Signed Distance Function from Point clouds by Learning to Pull Space onto Surface. International Conference on Machine Learning. PMLR, 2021: 7246-7257.\n\n[2]. Chen H, Li C, Lee G H. NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance. arXiv preprint arXiv:2312.00846, 2023.\n\n[3]. Huang B, Yu Z, Chen A, et al. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. ACM SIGGRAPH 2024 Conference Papers. 2024: 1-11. \n\n[4]. Guédon A, Lepetit V. SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 5354-5363.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lWQonp9Y1m", "reviewer": "Reviewer_SW8k", "summary": "This paper combines 3D Gaussian Splatting and NeuralPull to extract surface. By this way, it can utilize the existing extracting method Marching Cubes algorithm to extract the zero-level set as a mesh surface.", "questions": "Some questions have been listed in the weaknesses. I still have other questions here:\n1. According to [1,2], SDF with Neural Implicit Functions (NIFs) can only reconstruct closed surfaces. The limitation prevents NIFs from representing most real-world objects. However, on the TNT dataset, some scenes are not closed surfaces, like 'Barn' with the ground, 'Meetingroom' with windows. How do you solve this problem using SDF with NIFs?\n\n2. For Eq.4 and Eq.6, you use L1 loss to regulate vectors, however, it is more common to use the cosine distance function to regulate vectors. Can you explain why you use L1 loss instead of cosine distance?\n\n3. For rendering, you only use L2 loss, however, D-SSIM is also used to optimize Gaussian splatting. Can you explain why you don't use it in your method?\n\n4. The method is based on NeuralPull, which learns a neural SDF from ground truth point clouds, while this method optimizes a neural SDF from 3D Gaussian splatting. Therefore, the original NeuralPull is the upbound of this method. Can you provide an experiment that trains NeuralPull on the TNT dataset and compares it with it to show the upbound of the proposed method?\n\n5. How do you deal with the surface of 'ground'? For example, in Fig.4 (right, 'Truck') and Fig.14 (left, 'Barn'), the ground is built. However, in Fig.14 (middle, 'Caterpillar'), the ground is missing. Do you manually delete it or something else? In your evaluation, do you also delete it? As I know, on the TNT dataset, the extracted meshes are not modified manually for evaluation.\n\n\n[1] CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization.\n\n[2] Neural unsigned distance fields for implicit function learning.", "limitations": "In Fig.12, the paper shows this method can not reconstruct the details of 'lego'. Can the authors show the comparsion with this method and SuGar as well as 2DGS?", "rating": 5, "confidence": 5, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "1. With a neural SDF network, this paper can utilize the Marching Cubes algorithm to extract the surface instead of TSDF fusion or Poisson.\n2. The paper uses three datasets to validate the proposed method.", "weaknesses": "1. Lack of explanation of how this method can work. As we all know, NeuralPull needs point clouds as ground truth to let a neural SDF learn surface, while the pseudo ground truth provided by 3D Gaussians is noisy, so how can your method learn? In addition, based on your description, you jointly train the 3D Gaussian and the neural SDF and use Eq.2 to pull the 3D Gaussian onto the zero-level set. However, at the beginning of training, the neural SDF will not provide good guidance and even provide a poor direction leading to making the 3D Gaussian away from the surface, leading to catastrophe.\n\n2. Unsatisfactory visual results on TNT. Although your F1-score is better than 2DGS, in Fig.14 your results look over smooth and lack details. This figure only shows your results, can you provide more comparisons with other methods, like 2DGS? Also, in Fig.15, why are there many holes in the mesh in the middle?\n\n3. The results of 2DGS are worse than the original paper. For example, on the TNT dataset, 2DGS is 0.3 in the original paper, while it is 0.26 in Fig.2 of your paper. Also, the qualitative results of 2DGS are better than yours. In detail, the mesh shown in Fig.10 of 2DGS's paper has more details, which are better than yours, and 2DGS's results are shown in Fig.4 of your paper.\n\n4. Your quantitive results are not consistent between Tab.2 and Tab.4 on TNT. In detail, in Tab.2, the full model F1-score is 0.43, while in Tab.4 it is 0.46, leading to suspicion of the accuracy of your results. Can you explain it?\n\n5. Lack of correct citation. For example, your first constraint $L_{Thin}$ is actual $L_s$ in NeuSG, but you don't illustrate the origin.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Explanation of the method.** When training 3DGS, 3D Gaussians are progressively approaching to the zero-level set. At the same time, we pull Gaussians to align with the zero-level set of SDF. Under the joint optimization of these two process, along with our novel pulling operation and constraints, the Gaussian point clouds that get pulled on the zero-level set gradually turn out to be clean, which can be used as the pulling target to infer signed distance in the field, leading to surfaces fitted by the SDF that gradually recovers detailed and accurate geometry.\n\nWe agree that the learned SDF field is coarse at the very beginning, which is not a good guidance for Gaussians. Therefore, we firstly train 3DGS for 7000 epochs, leading to relatively stable Gaussians, and then start to pull queries onto Gaussians to estimate a rough SDF, and finally we pull Gaussians on to the zero-level set, and queries are also following to get pulled onto the pulled Gaussians since then.\n\n**2. Visual results.** The surfaces learned by 2DGS are usually fat and a little bit drift away from ground truth surfaces, although their meshes seem to show more details. Our method is able to capture more accurate surfaces by using 3D Gaussians pulled onto the zero-level set and pulling query points onto Gaussian disks at the same time, leading to much more accurate zero-level set. We additionally report comparisons of error maps on meshes obtained by 2DGS and ours in **Fig. C** in rebuttal PDF, which highlights our superiority in terms of the accuracy of extracted surfaces. Additionally, the holes in the middle of Figure 15 are due to the ill-conditioned distribution of Gaussians, making it difficult to distinguish geometric structures.\n\n**3. 2DGS numerical results.** 2DGS wrongly reported the mean result on TNT in its original paper. The calculated average value is indeed 0.26.\n\n**4. Results between Tab.2 and Tab.4.** Thanks for pointing this out. Before the deadline, we updated our results frequently, but we forgot to update the latest results everywhere in the paper. We wrongly reported an older version of ablation study, which was conducted on the first four scenes in TNT dataset. We will update the ablation study with the final version, which was conducted on all of the TNT scenes, as reported in **Tab. B** in the rebuttal PDF.\n\n**5. $L_{Thin}$.** We did mention NeuSG in Line 175. We'll make it more clear that the thin loss is inspired by NeuSG in our revision.\n \n**6. Closed Surfaces.** SDF can also represent open structures. As a result, it will reconstruct double-layer surfaces. You can refer to the second and third row of “NeuS” in Figure 4 in the paper of NeuralUDF (CVPR2023), where the SDFs successfully distinguish the collar and cuffs, but the reconstructed cloths have tight double-layer surfaces. To avoid the influence of double-layer surfaces on evaluation accuracy, we practically delete the back faces according to the visibility of each face under each camera view. Through this way, we can accurately reconstruct open structures with single-layer surfaces. Additionally, although UDF can reconstruct open surfaces, extracting the zero-level set from UDF as a mesh surface is still a challenge, resulting in artifacts and outliers on the reconstructed meshes. Our method can also learn UDF, and we additionally conduct an experiment using the same setting to learn UDF and compare surfaces extracted from the SDF and the UDF, as shown in **Fig. D** in the rebuttal PDF, which is a corner of “Barn”, showing the shortcomings of UDF learning.\n\n**7. Normal loss.** We also tried cosine similarity as the normal loss. We did not see any difference on the performance for these two constraints on normal.\n\n**8. D-SSIM loss.** We also tried the D-SSIM loss. But we did not see any difference on the performance. For simplicity, we do not include it in the loss function.\n\n**9. Performance of NeuralPull.** In this paper, we resolve the problem of learning SDF from multi-view images through sparse and noisy Gaussians by innovatively pulling queries onto Gaussian disks on the zero-level set of the SDF. Although we are using pulling, our pulling is much different from NeuralPull. The difference lies in that NeuralPull pulls a query to a point while we pull a query onto a Gaussian disk. 3D Gaussians are constrained to be a disk-like shape which covers an area rather than a point during optimization, which inspires us to introduce this pulling variant. Therefore, the performance of our method is not limited to the bottleneck of NeuralPull. Figure 2 in our paper shows that merely pulling queries to the point cloud, like NeuralPull, does not work well with 3D Gaussians, due to the sparsity. Additionally, we conduct an experiment that using a sparse ground truth point cloud of “Ignatius'” scene to train both NeuralPull and our pulling operation. The point clouds are initialized as spheres to simulate the Gaussians used by our method. The results in **Fig. E** in our rebuttal PDF highlights the superiority of our proposed method over NeuralPull.\n\n**10. Ground of surface.** The ground truths of TNT dataset have provided bounding box for culling the reconstructed meshes. For example, the ground truth points of “Truck” and “Barn” contain the ground while the points of “Caterpillar” do not contain the ground. We first crop the meshes using the provided bounding box and then visualize them.\n\n**11. Reconstructing details.** Since we use the marching cubes algorithm to extract the zero-level set as a mesh surface, our results is limited by the resolution used by the marching cubes. While 2DGS uses multi-view depth images to reconstruct meshes with Poisson surface reconstruction, which achieves higher resolution and recover more geometry details. We additionally use the same way of reconstruction as 2DGS, and show visual comparison in **Fig. F** in our rebuttal PDF, where we can see that our reconstruction show more geometry details than 2DGS.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xhHSoyl95t", "reviewer": "Reviewer_ZhFP", "summary": "This paper aims to infer neural signed distance functions (SDF) for Gaussian Splatting. To this end, this paper introduce an MLP to represent  the SDF. To learn SDF from sparse and non-uniform Gaussian points, this paper introduces a differentiable pulling operation from Neural-Pull to align Gaussians on the zero-level set of the SDF and updates the SDF by pulling neighboring space to the pulled 3D Gaussians. This paper designs tangent loss, pull loss and orthogonal loss to encourage the above operations. Experiments on DTU and Tanks and Temples datasets show that the proposed method achieves SOTA reconstruction performance.", "questions": "* Gaussian splatting will generate many points that are far away from surfaces, how does the pulling operation tackle these points?\n* When encouraging the thin disk to be a tangent plane on the zero-level set, why not make $f(\\mu'_j)\\approx0$?\n* For Table 4, can you explain the difference from Table 2 in terms of the performance of full model? Moreover, can you explain the detailed operations of w/o Pull GS? I wonder how to train the SDF without Pull GS.\n* As the proposed method uses the densification from GOF, it is better compare the rendering performance of proposed method with GOF.\n* How many Gaussian points and random points are sampled to train SDF during each optimization?", "limitations": "The authors have discussed the limitations and their future works.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "* This work combines a differentiable pulling operation with Gaussian Splatting for surface reconstruction, which helps to learn neural SDF for GS. This is interesting.\n* The approach designs the tangent loss to encourage the pulled Gaussians to be the tangent plane on the zero-level set.\n* To tackle the sparsity and non-uniform Gaussian distribution, the method proposes to pull randomly sampled points to disks.\n* The method achieves promising surface reconstructions on Tanks and Temples dataset.", "weaknesses": "* The proposed method cannot reconstruct geometric details well as shown in Figures 4, 13 and 14.\n* The ablation results in Table 4 are not consistent with the results in Table 2.\n* The ablation study is bad and confusing. The quantitative results are conducted on Tanks and Temples but the qualitative results are conducted on DTU.\n* The rendering evaluation setting is not consistent with prior works, such as 2DGS and GOF. Using lower resolution training images can improve rendering results. However, the training setting for NeRF-based methods still use the original setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**1. Reconstructing details.** Since we use the marching cubes algorithm to extract the zero-level set as a mesh surface, our results is limited by the resolution used by the marching cubes. While 2DGS uses multi-view depth images to reconstruct meshes with Poisson surface reconstruction, which achieves higher resolution and recover more geometry details. We additionally use the same way of reconstruction as 2DGS, and show visual comparison in **Fig. F** in our rebuttal PDF, where we can see that our reconstruction show more geometry details than 2DGS.\n\n**2. Results between Tab.2 and Tab.4.** Thanks for pointing this out. Before the deadline, we updated our results frequently, but we forgot to update the latest results everywhere in the paper. We wrongly reported an older version of ablation study, which was conducted on the first four scenes in TNT dataset. We will update the ablation study with the final version, which was conducted on all of the TNT scenes, as reported in **Tab. B** in the rebuttal PDF.\n\n**3. About ablation study.** The TNT scenes are large and complex, therefore we believe the quantitative results on TNT dataset are more convincing than that of DTU dataset. However, the qualitative differences are more distinct on small object scenes such as DTU dataset, therefore we chose DTU dataset for visualization. Due to the time limit, we select the first four scenes in DTU dataset to conduct additional ablation study to demonstrate the effectiveness of our method, as shown in **Tab. C** in our rebuttal PDF. We will update the results with all scenes of DTU in our revision.\n\n**4. Rendering evaluation setting.** For fair comparison, we directly borrowed the results of all baseline methods from Table 4 in 2DGS paper. Due to the limit of memory, current methods, including NeRF-based and 3DGS-based, coherently train outdoor scenes with a down-sampling factor of 4 and train indoor scenes with a factor of 2. Our experiments keep the same settings with baseline methods. We will clarify this in our revision.\n\n**5. About far away points.** Indeed, most of the Gaussian points are distributed around the ground truth surfaces, which can be seen as a noisy point cloud. The Gaussians that are significantly far away from the surfaces usually have small opacity values. In our implementation, we practically utilize a filtering strategy before pulling operation to filter out the Gaussians that have too small opacities, similar as what SuGaR does at 9000 epoch. Through this strategy, almost all of the Gaussian points are distributed around the surfaces. If there are several points that are still far away, the loss gradient of these points will be averaged by other points and they will be ignored by the learned SDF.\n\n**6. why not make $f(\\mu_j’)=0$.** Our preliminary experiments show that directly constraining the Gaussians to be the zero-level set will hurt the learning of SDF, as shown in **Fig. G** in our rebuttal PDF. The reason is that the signed distance field has large uncertainty during the optimization, and the pulled queries produced by inaccurate pulling will lead to inaccurate zero-level set. Directly using a hard constraint will make the optimization inefficient. Our results show that implicitly aligning Gaussian disks to the zero-level set resolves this problem.\n\n**7. About w/o Pull GS.** We train SDF by pulling randomly sampled queries to their nearest Gaussians on the zero-level set. The setting of “w/o Pull GS” means that we do not pull Gaussians onto the zero-level set and just pull randomly sampled queries onto their nearest Gaussians. The full model with pulling GS means that we align the Gaussians with zero-level set of SDF, which means that the positions of Gaussians are encouraged to move towards the zero-level set.\n\n**8. Comparison with GOF.** Our method shows comparable rendering results with GOF, as reported in **Tab. D** in our rebuttal PDF.\n\n**9. Number of Gaussian points.** We sample 100,000 points to train SDF in each epoch. Because we need to calculate the nearest Gaussian for each sampling point, the number of sampled Gaussian points per epoch is also 100,000.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank reviewers for comments and highlighting our ***simple and interesting idea*** (Reviewer CCgx, 5rKf, ZhFP), ***good performance and visualization*** (Reviewer CCgx, ZhFP), ***well-written manuscript*** (Reviewer 5rKf). We have provided detailed responses to each reviewer's comments. All the figures and tables referenced are included in the rebuttal PDF. We gratefully thank the reviewers for taking the time to review our paper and for their valuable suggestions. We are looking forward to the feedback.

Meta-review: This paper is a clear accept. It receives 2x accepts and 2x borderline accepts. The reviewers agreed that the proposed method is simple but effective with well-thought losses, the proposed method demonstrated state-of-the-art reconstruction results on the DTU and TNT datasets, with each component of the approach well-ablated. This work proposes an interesting method of combining a differentiable pulling operation with Gaussian Splatting for surface reconstruction to learn neural SDF for GS. The paper is also well-written and easy to follow. Although Reviewer SW8k initially raised several issues on lack of explanation, unsatisfactory results on TNT, inconsistency of quantitative results, lack of citation, etc, these concerns are well-addressed by the authors. The reviewer finally raised the score to borderline accept.
Decision: Accept (poster)