Inferring Neural Signed Distance Functions by Overfitting on Single Noisy Point Clouds through Finetuning Data-Driven based Priors
Authors: Chao Chen
Yu-Shen Liu
Zhizhong Han
Venue: NeurIPS 2024 poster (2024)
Keywords: point cloud
SDF
3D reconstruction
Primary area: machine_vision

Abstract: It is important to estimate an accurate signed distance function (SDF) from a point cloud in many computer vision applications. The latest methods learn neural SDFs using either a data-driven based or an overfitting-based strategy. However, these two kinds of methods are with either poor generalization or slow convergence, which limits their capability under challenging scenarios like highly noisy point clouds. To resolve this issue, we propose a method to prompt pros of both data-driven based and overfitting-based methods for better generalization, faster inference, and higher accuracy in learning neural SDFs. We introduce a novel statistical reasoning algorithm in local regions which is able to finetune data-driven based priors without signed distance supervision, clean point cloud, or point normals. This helps our method start with a good initialization, and converge to a minimum in a much faster way. Our numerical and visual comparisons with the stat-of-the-art methods show our superiority over these methods in surface reconstruction and point cloud denoising on widely used shape and scene benchmarks. The code is available at https://github.com/chenchao15/LocalN2NM.

Inferring Neural Signed Distance Functions by Overfitting on Single Noisy Point Clouds through Finetuning Data-Driven based Priors
Chao Chen' Yu-Shen Liu Zhizhong Han
School of Software, Tsinghua University, Beijing, China
*Department of Computer Science, Wayne State University, Detroit, USA chenchao (eterhhhha...iingaau... org. cn liuyushen@mailss taspehhenmmmilsssssnnnnuaaa edu. cn h312h@wayne edu
Abstract
It important to estimate an accurate signed distance function (SDF) from point cloud in many computer vision applications. The latest methods learn neural SDFs using either a data-driven based overfitting-bast based strategy. However, these kinds of methods are with either poor generalization or slow convergence, which limits their capability under challenging scenarios like highly noisy point clouds. To resolve this issue, we propose a method to promote pros of both data-driven based and overfitting-based methods for better generalization, faster inference, and higher accuracy in learning neural SDFs. We introduce a novel statistical reasoning algorithm in local regions which able finetune data-driven based priors without signed distance supervision, clean point cloud, or point normals. This helps our method start with good initialization, and converge to a minimum in much faster way. Our numerical and visual comparisons with the state state-of-thee oo-tthe-rrt methods show our superiority over these methods in surface reconstruction and point cloud denoising on widely used shape and scene benchmarks. The code is available at https: h/
/ /ggtttb...../.. www.coocccco.c.o cchechhal15/ooca1/2..
1 Introduction
an important task to estimate an implicit function from a point cloud in computer graphics, computer vision, and robotics. An implicit function, such as signed distance function (SDF), describes a continuous 3D distance field indicate distances to the nearest surfaces at arbitrary locations. Since point clouds easy obtain, they widely used estimate SDFs, particularly without using normals that are not available for most scenarios. The challenge for SDF estimation mainly comes from the difficulty of bridging the gap between the discreteness of point clouds and the continuity of implicit functions.
information source to
Recent methods [62, 64, 29, 14, 95, 80, 58, 74] overcome this challenge using either a data-driven based or an overfitting-bant based strategy. To map a point cloud to a signed distance field, the data-driven based methods [60, 27, 36, 45, 81, 79, 22, 42, 92, 83] rely on a prior learned with signed distance supervision from a large-scale dataset, while the overfitting- 50, 18, 88] do not need signed distance supervision and just use the point cloud to infer signed distance field. However, both of the two kinds of methods have pros and cons. The data-driven based methods can do inference fast but suffers from the need of large-scale training samples and poor
methods [28, 1, 102, 2, 99, 4, 21,
"The corresponding author is Yu-Shen Liu. This work was supported by National Key R&D Program of China (2022YFC380600) and the National Natural Science Foundation of China (62272263, 62072268), and part by Tsinghua-K Kuaishou Institute of Future Media Data.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

generalization to instances that are unseen during training. Although the overfitting- based methods have a better generalization ability and do not need the large-scale signed distance supervision, they usually require a much longer time to converge during inference. The cons of these two kinds of methods dramatically limit the performance of learning neural SDFs under challenging scenarios like highly noisy point clouds. Therefore, beyond pursuing higher accuracy of SDFs, how to balance the generalization ability and the convergence efficiency is also a significant issue.
To resolve this issue, we propose to learn an SDF from single point cloud by finetuning data-driven based priors. Our key idea is to promote the advantages of both the data-driven based and the overfitting- based strategy to pursue better generalization, faster inference, and higher accuracy. Our method overfits a neural network on a single point cloud estimate an SDF with a novel loss without using signed distance supervision, clean point, or point normals, where the neural network was pretrained as data-driven based prior from large-scale signed distance supervision. With finetuning priors, our method can generalize better on unseen instances than the data-driven based methods, and also converge much more accurate SDFs in much faster way than the overfitting-bat -based methods. Moreover, our novel loss for finetuning the data-driven based prior can conduct a statistical reasoning in a local region which can recover more accurate and sharper underlying surface from noisy points. We report numerical and visual comparisons with the state-of-the-ar methods and show our superiority over these methods in surface reconstruction and point cloud denoising on widely used shape and scene benchmarks. Our contributions summarized below,
We introduce a method which is capable of funetuning a data-driven based prior by minimiz- ing an overfitting based loss without signed distance supervision, leading to neural SDFs with better generalization, faster inference, and higher accuracy.
The proposed overfitting- based loss can conduct a novel statistical reasoning in local regions, which improves the accuracy of neural SDFs inferred from noisy point clouds.
Our method produces the state-of-the-art results in surface reconstruction and point cloud denoising on the widely used benchmarks.
2 Related Works
Learning implicit functions has achieved promising performance various tasks [62, 64, 29, 14, 95, 80, 58, 74, 30, 33]. We can learn neural implicit representations from different supervision including 3D supervision [61, 69, 59, 17], multi-view images [78, 44, 38, 101, 46, 94, 63, 41, 98, 97, 25, 86, 100, 89, 84, 85], and point clouds [92, 43, 60, 27]. We briefly review the existing methods related to point clouds below.
2.1 Data-driven based Methods
In 3D supervision, many techniques utilize a data-driven approach to learning priors, and then apply these learned priors to infer implicit models for unseen point clouds. Some strategies focus acquiring global priors [60, 27, 36, 45, 81, 79, 22, 42] at the shape level, whereas others aim to boost the generalization of these priors by learning local priors [92, 83, 11, 37, 6, 51] at the component or patch level. These learned priors facilitate the marching cubes algorithm [47] to reconstruct surfaces from implicit fields. The effectiveness of these methods often rely on extensive datasets, but they may not generalize well when facing with unseen point clouds that significantly deviate in geometry from training samples.
2.2 Overfitting-based Methods
In an effort to enhance generalization, some methods concentrate on precisely fitting neural networks
to single point clouds. These methods incorporate innovative constraints [28, 1, 102, 2, 99, 4, 21], utilize gradients [50, 18, 88), employ differentiable Poisson solvers [70], or apply specially tailored priors [51, 54] to learn either signed [50, 28, 1, 102, 2, 15, 56, 13] or unsigned distance functions [18, 104, 103]. Despite achieving significant advances, these approaches typically require clean point
clouds to accurately determine distance occupancy fields around the point clouds.

---

Training
Inferncce
Query Points
Noisy Point Cloud M
Cleen D Dataset
KNN
KNN
Local Querys
Neural Implicit Function
Local Denoising
Lorol
Query q
Avesage Code C
Initiaizaaion
f
D
Neural
Dala- Driven based Prior
sdf
Implictt
sdf
Function
Initialization
C
LMSE
Parameter
Marching Cubes
GT
Figure I: The overview of our method, We learn the data-driven based prior by learning neural implicit function I' with a condition c' on a clean dataset. During inference, we employ novel statistical reasoning algorithm to infer neural SDF f for noisy point cloud M with learned prior (average code and learned parameter).
2.3 Learning from Noisy Point Clouds
The key to accurately reconstructing surfaces on noisy point clouds is to minimize the effect of noise in inferring implicit functions. PointCleanNet [73] was developed to filter out noise from point clouds through a data-driven approach. GPDNet [72] incorporated graph convolution based on dynamically generated neighborhood graphs to enhance noise reduction. Some other methods leveraged point cloud convolution [6], alternating latent topology 57], semi-supervised strategy [106, 19], dual and integrated latent [76], or neural kernel field [91, 35] to reduce noise from point clouds. On the unsupervised front, TotalDenoising [10] adopts principles similar to Noise2Noise [40], utilizing spatial prior suitable for unordered point clouds. DiGS [3] employs a soft constraint for unoriented point clouds. Noise2NoiseMapping [52] leverage statistical reasoning among multiple noisy point clouds with specially designed losses. Some methods using downsample-upsanpl frameworks [48], gradient fields [49, 9, 16, 68, 65], convolution-fres intrinsic occupancy network [67], intra-shape regularization [66]. eikonal equation [96, 23), neural Galerkin [34] and neural splines [93] have heen implemented to further diminish noise in point clouds. Our method falls in this category, but we aim to promote the advantages of both the data-driven based and the overfitting beasettiin-bbedd strategy to pursue better generalization, faster inference, and higher accuracy.
3 Method
Overview. We aim to infer neural SDF f from a single point cloud with noises M. Our method includes two stages shown in Fig. one is to learn a prior f' in data-driven manner, the other is to infer a neural SDF f on unseen noisy point cloud M. At the first stage, we learn a prior by training neural SDF using ground truth signed distances of clean meshes indicated by embeddings cy. At the second stage, we finetune the learned prior f' to infer neural SDF f of M using our proposed local noise to noise mapping, where the embedding e indicating M is also learned. We can use the marching cubes algorithm [47] to extract the zero-level set of f as the mesh surface of M.
Neural Signed Distance Function. We leverage an SDF f to represent the geometry of a shape. An SDF f is an implicit function that can predict a signed distance s for arbitrary location g. i.e., s = f()). The latest methods usually train neural network to approximate an SDF from signed distance supervision or infer an SDF from 3D point clouds multi-view images. A level set is an iso-surface formed by the points with the same signed distance value. For instance, zero-level set is a special level set, which is formed by points with signed distance of O. On the zero-level set, the gradient f)(g) of the SDF if at an arbitrary location q is also the surface normal at q.
As shown in Fig. 1, we employ an auto-decoder similar to DeepSDF [69] for learning a prior I' in data-driven manner and inferring neural SDF for single point clouds with noises, respectively. We employ a data-driven strategy to learn a prior f' from clean meshes first. Specifically, we learn f' with an embedding e' as a condition of queries. For each shape, we sample queries q around a shape represented by cj,, and establish the signed distance supervision by recording the signed distance to the ground truth mesh. Thus, we learn the prior f' by minimizing the prediction errors to the ground truth signed distances,
Data-driven Based Prior.
3

---

J ||s ΣΣ Sit
min
f'a,, c)1
(I)
where df is learnable condition for the i-th training shape, qj is the j-th query that is randomly sampled around the i-th shape, and si the ground truth signed distance. We also add regularization term on the learned embeddings c,, and is the balance weight.
Signed Distance Inference. With the learned prior f", we infer neural SDF f for single point cloud with noises M. We do not require ground truth signed distances, clean point clouds, or even point normal during the inference of Specifically, we infer by finetuning parameters of f' with learnable embedding с indicating the single point cloud with noises. The finetuning relies on a novel statistical reasoning algorithm on local regions.
The advantage of our method lies in the capability of conducting the statistical reasoning in local regions. Comparing to the global reasoning method [52], our method is able to not only infer more accurate geometry but also significantly improve the efficiency. Our method starts from randomly sampling a local region m, on the shape M. We randomly select one point on M as the center of Ma, and set up its K nearest noisy points as local region m,, Then, we randomly sample U queries {ūu)a= around Ma, and also randomly select U noisy points {po ==1 out of my for statistically reasoning the surface in each iteration.
Our key idea of inferring neural SDF f is to estimate a mean zero-level set that is consistent to all points in the local region m,, To this end, we use the U sampled queries {Gu} to represent the zero-level set in this area using f. and minimize the distances of the U noisy points {pr} to the zero-level set in each iteration. Statistically, the expectation of the zero-level set should have the minimum distance to all the noisy point splitting in region m...
Specifically, we first project the sampled queries {īu} onto the zero-level set of f using a differentiable pulling operation [50]. For each query Tu. its projection on the zero-level set is,
d₂ = s D Vf c)llcc////VV c)l,
where Vh is the projection of q. on the zero-level set, f(q,, c). VS( c) is the gradient of f at the location Ju. and is the learnable embedding that represents the noisy point cloud M.
With the pulling operation, we can use projections {Ø", of queries {ēx, } to approximate the zero-level set in region my. With a coarse zero-level consistent to various subsets of noises {pr) sampled from man Thus, we minimize the errors between the (d.)== and subset of points {po)az on area m, in each optimization iteration,
estimation, we expect this zero-level set can be
min Em, Mau ma mm., EMD(Hdd) (p.)) + 3||c|12 f.e
where we learn f through finetuning the prior I' and learning the embedding e representing the noisy point cloud M. The expectation is over the local regions m, that randomly sampled from the noisy point cloud M, and the subset patch Pa randomly sampled from each m,, We follow the method [52] to use the EMD to evaluate the distance between the two sets of points, which leads the neural SDF to converge on the specific noisy point cloud M.
Initialization.
The network architecture of f is the same to the one of prior f'. We learn ff with the parameters of f' as the initialization, representing the prior that we learned. For the embedding e that represents M, we initialize e. as the center of the embedding space learned by the prior I' in Eq. 1, i.c., 1/1 E/- c/. This initialization is important for the accuracy and efficiency of learning for single noisy point cloud M. This finetuning of parameters of f' also shows advantages over the auto-decoding [69] in terms of generalization and efficiency. We will justify these advantages in experiments.
Implementation Details. We randomly select one point from noisy point cloud M as center, and select its K 1000 nearest points to form a local region m,, We also randomly sample 1000 queries around the K noisy points for statistically reasoning. Specifically, adopt method

---

Mesnics
PSK
PSG [2]]
K2N2T 120]
Cdse [1]
SAP [0]
OCNNTSA]
IMLS [5S]
POCOT
ALTO
N2NM [2]
Ours
CDa
0,259
0.147
0.173
0.044
0004
D.067
0.051
0,030
0.02X
0.022
6,023
NC
0.772
0.715
0.938
0.94
0,932
0.944
0.950
0.955
0.962
0.973
0.612
0.259
0.400
0.992
Table Numerical Comparisons on ShapeNet dataset in terms CD11 X 10, NC and F-Score.
introduced by NeuralPull [50] to sample queries around each one of the K noisy points. We use
a Gaussian distribution centered at each point and set the standard deviation as the distance to the 51th nearest neighbor in the point cloud. We run the marching cubes for surface reconstruction at resolution 256 for shapes, and 512 for large-scale scenes.
The length of the embedding с or c' is set to 256. We use Adam optimizer for learning neural implicit network, which is an auto-decoder similar to DeepSDF [69]. For training, we use an initial embedding learning rate of 0.0005 for updating embeddings and an auto-decoder learning rate of 0.001 for optimizing the prior network. Both learning rates are decreased by 0.5 for every 500 epochs. We train the prior network f' for 2000 epochs. For inference, we finetune the network f for each noisy point cloud in 4000 iterations with a learning rate of 0.0001.
4 Experiments and Analysis
We compare our method with the latest methods in terms of numerical and visual results on synthetic point clouds and real scans in surface reconstruction.
Datasets and Metric. We use eight datasets including shapes and scenes in the evaluations. For shapes, conduct experiments under five datasets including ShapeNet [12]. ABC [22], FA- MOUS [22], Surface Reconstruction Benchmark (SRB) [92] and D-FAUST [5]. For scenes, we conduct experiments under three real sean datasets including 3D Scene [105], KITTI [26], Paris-rue- Madame [75], and nuScenes [8]. We leverage L1 Chamfer Distance (CDLi). L2 Chamfer Distance (CD12) to evaluate the error between the reconstructed surface and ground truth. We also use Normal Consistency (NC) [59] and F-Score [82] with a threshold of 1% to evaluate the normal accuracy of the reconstructed surface. In the ablation study, we also report time consumption highlight the superiority of our data-driven based prior. For KITTI and Paris-rueMMadane datasets, due to their lack of ground truth meshes, we only report visual comparisons.
4.1 Surface Reconstruction for Shapes
Evaluation on ShapeNet. We first report our results on shapes from ShapeNet. We report evaluations by comparing our method with the latest prior-based and overfitting-based methods
in Tab 1. For prior-based methods, we compare our
method with PSG [24],
R2N2 [20]. COcc [71],
OCNN [87], IMLS [45],
POCO [7], and ALTO [90].
All of these methods are pretrained to learn priors using shapes with noises in training set of ShapeNet. We also follow these meth- ods to use the same set of training shapes to learn our prior. For overfitting- based methods, compare our method with PSR [39], SAP [70], and N2NM [52]. These methods did not need to learn a prior, and have the ability of inferring neural implicit functions on each shape in the testing set. We also follow these methods and report our results by finetuning our prior through overfitting on each testing shape. All the shapes for testing are corrupted with noises with variance of 0.005.
Noka lnput ConrCoo
POOD
ALTO
NZNM
Ous
GI
Comparison in surface reconstruction on ShapeNet. More visual results are provided in the appendix.
Figure 2:
5

---

The comparisons in Tab. I indicate that our method can infer much more accurate neural implicit functions than the prior-based methods. The improvement comes from the ability of conducting test time optimization with the learned prior and inferring signed distances using the local noise to noise mapping. Moreover, our local statistical reasoning not only achieves better ability of recovering geometry from noisy points than overfitting-br baaaeddnnn-asedddd methods but also significantly reduces the time complexity during the test time overfitting procedure with our prior. Different from prior-based methods, our ability of conducting test- time optimization with our local statistical reasoning loss significantly improve the generalization ability on unseen shapes. Tab. shows that our method can infer neural implicit functions on single shapes much faster than the overfitting baasei methods. We also demonstrate our advantages in visual comparisons in Fig. 2.
Metnes
SAP[.0]
N2NM [52]
Olurs
Time
14 min
46
5
Table 2: Time consumption
on ShapeNet dataset with
overfitting-ban methods.
Evaluation on ABC. We also report our evaluations on ABC dataset in Tab. 3. We learn priors from shapes in training set, and finetune this prior for each single shape in the testing set. The numerical comparisons are conducted the testing set of ABC dataset released by P2S [22]. It in- cludes two versions with different noise levels. Similarly, we also report comparisons with prior-based methods and overfitting- based methods. With local noise to noise mapping,
Input Figure 3: Comparison in surface reconstruction ABC. More visual results are provided in the appendix.
IMLS
P2S
NeuralPull Ours
achieve the best performance over all baselines.
Compared to prior- methods, such as P2S [22]. COcc [71], and POCO [7]. our loss can infer more accurate geometry during the test time overfitting procedure. Also, the ability of finetuning the prior can also provide a coarse estimation and good start for inferring neural implicit from single noisy points. Besides the accuracy, also observe improvements on efficiency. Fig. 3 demonstrates the improvements over the baselines in terms of surface completeness and edge sharpness.
Evaluation SRB. We report previous experiments using man-made objects in ShapeNet and ABC dataset, We also port our results on real seans on SRB dataset [92]. Since there is training sam- ples on SRB, we use the prior learned from the ShapeNet as the prior for real scans. Al- though the shapes in ShapeNet are not sim- ilar to shapes in SRB, we found the prior can also work well with the scans on SRB. Different from the man man- objects, real scans have unknown noises. We report the evaluations with the rrior-based and overfitting-bn show that our method achieves the best performance in implicit surface reconstruction. Under the same experimental settings, our method can infer more accurate geometry details with our local noise to noise mapping.
Ours Figure 4: Comparison in surface reconstruction SRB. More visual results are provided in the appendix. methods in Tab. and Fig. The comparisons
Input Point2Mesh SIREN GriPPull ALTO
Evaluation FAMOUS. We re- port evaluations on more complex shapes on FAMOUS dataset. Sim- ilar to SRB. we also use the prior learned from ShapeNet. We evalu- ate the performance on two kinds of noises in Tab. We can see that our method can recover more geometry details and achieve higher accuracy and smoother surfaces. We also report visual comparisons in Fig. 5, which also highlights our improvements in
Ingut
IMLS
LPI
GodPul
CUuS
GT
Figure 5: Comparison in surface reconstruction on FAMOUS. More visual results are provided in the appendix.

---

Datavel
PSR[E))
124]
COcc [7]]
KPT50]
IMLS
PCPT55]
POCO
N2NM
ABC var
5.29
2.14
0.89
0.72
0.57
0.49
2001
3.52
0.113
0,099
ABC max
1.45
2.50
0.139
Table 3: Numerical Comparisons on ABC dataset in terms of CDL2 x 100.
Metries
IGR [2s]
Poin2Mesh [22]
PSR PSR [39]
SIREN [27]
GP [P6)
ALTO [90]
Steik [96]
SAP [70]
NKSR [35]
NINM [52]
Ours
0.178
0.T16
0,252
0.22
1186
0,0R9
0.076
0009
01097
F-Score
0:648
0.738
0.766
0.772
0,822
0.830
0.829
0.860
Table 4: Numerical Comparisons on SRB dataset in terms of CDL1 x 10 and F-Score.
terms of accuracy, smoothness, com- pleteness, and recovered sharp edges.
Evaluation on D-FAUST. Finally, we report our results non-rigid shapes, i.e., humans. Different from rigid shapes in the previous experiments, humans are with more complex poses. We learn prior from the training set, and finetuning the prior unseen hu- mans with different poses. We mainly compare our method with overfitting- based methods in Tab. We can see that our method achieves the best per- formance in CD, F-Score, and com- parable performance to N2NM [52] but with faster inference speed. We further show the visual comparison in Fig. 6. We can see that our method recover more accurate geometry and poses.
GT Figure 6: Comparison in surface reconstruction on D-FAUST. More visual results are provided in the appendíx.
Imput.
IGR
Point2Mesh
SAP
Ours
4.2 Surface Reconstruction for Scenes
Since we have a limited number of scenes for training, we use the prior learned from ShapeNet as the pretrained prior in experiments scenes. Specifically, conduct experiments on four different scene datasets: 3D Scene [105], KITTI [26], Paris-rue-Madam [75] and nuScenes [8], where the results nuScenes reported in the appendix.
Evaluation on 3D Scene. We further evaluate our method surface reconstruction for scenes in 3D Scene [105]. We follow previous methods LIG [37] to randomly sample 1000 points per m²
We compare our method with the latest methods including COce [71] and LIG [37], Deepl.S [11],
NeuralPull (NP) [50] and Noise2Noisee
(N2NM) [52]. For rrserd methods COcc [71]
and LIG [37], we leverage their released pretrained models to produce the results, and we also provide them with the ground truth point normals. For overfitting-base
methods DeepLS [1]], NP [50] and
N2NM [52], we overfit them to produce results with the same noisy point clouds. We follow LIG [37]
to report CD11, CD and NC for evaluation. We report the comparisons in Tab. 7. The results demonstrate that our method outperforms both kinds of methods with learned priors such as LIG [37] and werfitting-basel N2NM [52]. The visual comparisons in Fig. show that method reveal more geometry details on real scans, which justifies our capability of handling noise in point clouds.
ConvOoc
DeepLS Figure 7: Comparison in surface reconstruction on 3D Scene.
GridPull
N2NM
Ours
GT

---

Dataset
PSR [39]
NP [50]
IMLS [45]
LP [15]
PCP [SS]
POCO
OnSurf [53]
GP [16]
N2NM [52]
Ours
F-var
1.80
0.28
0.80
0.19
0.07
1.50
0.59
0.13
0.033
0.029
F-max
3.1
0.39
0.26
0.30
2.75
0.21
0.117
0.105
Table 5: Numerical Comparisons on FAMOUS dataset in terms of CDr22 x 100.
Metrics
IGR [28]
Point2Mcsh [32]
PSR [39]
SAP [70]
N2NM [52]
Ours
CDra x 10
0.235
0.071
0.044
0.043
0,037
0.034
F-Score
0.805
0.855
0.966
0.966
0.966
0.973
NC
0.911
0.905
0.965
0.959
0.970
0.968
Table 6: Accuracy of reconstruction D-FAUST dataset in terms of CDL1. NC and F-Score.
Evaluation on KITTI. Following GridPull [16]. we further evaluate our method on KITTI [26] odometry dataset (Sequence 00, frame 3000 to 4000), which contains about 13.8 million points, which are split into 15 chunks. We reconstruct each of them and concatenate them together for visualization. We compare our method with the latest methods SAP [70] and GridPull [16]. As shown in Fig. 8, our method is robust to noise in real scans, successfully generalizes to large-scale scenes, and achieves visual-appealing reconstructions with more details.
Evaluation on Paris-rue-Madame. Following N2NM [52], we further evaluate our method on Paris-rue-Me Med-aae [75], which contains much noises. We split the 10 million points into 50 chunks each of which is used to learn a neural implicit function. We compare our method with LIG [37] and N2NM [52]. For LIG [37]. we produce the results for each chunk with released pretrained models. For N2NM [52], we overfit on all chunks until convergence. As shown in Fig. 9, we achieve better performance over LIG [37] and N2NM [52] in large-scale surface reconstruction, which highlight our advantages in reconstructing complete and detailed surfaces from noisy scene point clouds.
4.3 Ablation Studies
We conduct ablation studies on the ABC dataset [22] to justify each module of our method.
Embedding Size, We evaluate our performance on different sizes of embedding We try several sizes {128, 256, 512) to infer the signed distance functions from noisy point cloud. The numerical comparison in Tab. 8 shows that the optimal result is obtained with a size of 256. Deviations from this value, either longer or shorter dimensions, leads to worse results with the current number of training samples.
Metric
128
256
512
CDr, x 100
0.102
0.096
0.114
Table 8: Effect of the embedding size.
We conduct experiments to ex- plore the importance of data-driven based prior. We first replace our learned embedding and parameter with randomly initialized embedding and parameter, or only replace e with randomly initialized embedding. As shown in Tab. 9, The degenerated result of "Without Prior" and "Without Embed" indicates that directly inferring implicit functions without our prior or learned embedding makes it difficult to accurately learn the surfaces of the noisy point clouds, and also slows the convergence. Then we fix the learned parameters and only optimize the embedding c, similar to auto-decoding. The results also get worse, as shown in "Fixed Param".
Prior.
Metric
Without Prior
Without Embed
Fixed Param
With Prior
C x 100
0.108
0.103
0.144
0.096
Time
12min
30min
8 min
Table Effect of the prior.
Local Region Splitting, We further vali- date the effectiveness of local region split- ting strategies. We employ three different
Metric
Voxel
Sphere (Fixed Size)
Sphere (KNN)
CD²z 100
0.314
0.101
0.096
Table 10: Effect of splitting strategies.
splitting strategies in Tab. 10. We first
split the whole space where the noisy point cloud is located uniformly into multiple voxel blocks, as shown by the result of "Voxel". The severely degenerated results indicate that this splitting strategy is even worse than the global method N2NM [52], as it results in many empty voxel blocks. Then we randomly select point from the noisy point cloud as a center sample all points within radius of 0.1 as a local region. The result of "Sphere (Fixed Size)" slightly degenerates due to some of the spheres containing too few points. In contrast, our splitting strategy, as shown by the result of "Sphere
(KNN)", ensures that each local region has enough points to help achieve superior performance.
Metrics
COce [71]
LIG [37]
DeepLS[1T]
NP [50]
N2NM [52]
Ours
CDus 1000
14.10
6. 190
1.607
2.115
0.507
0.389
CD1
0.052
0.048
0.025
0.034
0.019
0.016
NC
0.908
0.849
0.915
0.900
0.929
0.942
Table 7: Numerical Comparisons on 3D Scene dataset in terms of CDLi, CD122 and NC. Detailed
comparisons for each scene are provided in the appendix.
8

---

SAP
GridPall
Outs
Figure 8: Comparison in surface reconstruction on KITTI.
LIG
NZNM
Ours
Figure Comparison surface reconstruction on Paris-rue-Madanee
Global and Local. With learned prior, compare performance in global and local mappings with finetuning the priors. We report results obtained with the local noise to noise mapping or the global one during the finetuning. As shown in Tab. 11, the numerical comparison shows that the global mapping struggles to infer local details from noisy point clouds. Moreover, our local prior also converges faster than the global statistical reasoning.
Metric
Global
Local
CD x 100
0.106
0.096
Time
21 min
8 min
Table 11: Effect of local mapping.
Local Region Size. further validate the effec-
Metrie
500
1000
3000
5000
tiveness of local region sizes (points number in local region) in Tab. 12. We use different local re-
CD22 x 100
0.102
0.096
0.111
0.114
gion sizes including (500, 1000, 3000, 5000}. The
Table 12: Effect of local region size.
results show that 1000 is the best.
SDF initialization. We further validate the effectiveness of different SDF initial- izations in Tab. 13 and Fig. 10, including random initialization, geometry initializa- tion [1], initialization to simple square shape, and ours.
Metric
Random
Square
Sphere (SAL)
Ours
Time
8.3min
7.1min
5.5min
5.0min
Table 13: The effect of SDF initialization.
We can see our prior can reconstruct more accurate surfaces from single noisy point clouds in much shorter time than any other initializations.
Random
Square
Sphere
Figure 10: Comparison with different SDF initializations.
Noise
Half Noise
GT
Figure 11: Visual results with nonuniform noises.
Noise Type. We report our performance with various noise types, i.e., impulse noise, quantization noise, Laplacian noise, and Gaussian noise. Visual comparison in Fig. 12 justifies that we can also handle other types of noise quite well. More- over, we also tried more challenging cases with nonuniform noises which do not have zero expectation across a shape, like a shape with only half of points having noises or shape with several patches hav- ing noises. The result in Fig. 11 shows that our method can also handle nonuniform noises well.
Impulse nosse
Quantizaion noise
Laplace
Goussian
Figure 12: Visual results with different noise types.

---

Noise Level. We report our performance on point clouds with different lev- els of noise. As shown in Tab. 14, the noise lev- els of middle and max come from the ABC dataset [22]. The middle indicates noises with variance of 0.01L, where L is the longest edge of the bounding box. The max indicates noises with a variance of 0.05L. Our extreme noise comes with a variance of 0.07L.
Naisy lnput
OuES
0.5%
1%
3%
5%
7%
GT
Figure 13: Visual comparison with different noise levels.
The CD22 comparison shows that our results slightly de- generate with max and extreme noise, but still outperform N2NM [52]. The visual results in Fig. 13 indicates that our method more robust noises even when the noise variance as large 7%.
Method
Middle
Max
Extreme
[22M[
0.113
0.139
0.156
Ours
0.096
0.113
0.125
Table 14: Effect of noise level.
Sparsity. We report the effect of the sparsity of noisy point clouds. We downsample the noisy point clouds to 25% and 50% of their original size to validate the impact of sparsity. The CD results in Tab. 15 and visual comparisons in Fig. 14 indicate that our method can handle sparsity in noisy point
Method
25%
50%
100%
N2NM [52]
0.154
0.133
0.113
Ours
0.121
0.107
0.096
Table 15: Effect of sparsity.
clouds better than N2NM [52]. Since our data-driven based prior can help to learn more complete surface and reduce the impacts brought by the sparsity.
10%
70%
100%
GT
Figure 14: Visual comparison with different point numbers.
Time Consumption. Since our method can handle sparsity and require less time
Metrie Time
10%
30%
50%
70%
100% (3k)
3.1min
3.6min
4.0min
4.5min
5.0min
the point number decreases, we conduct an experiment with downsampled noisy points in Tab. 16. Fig. 14 indicates that we can work well on much fewer points, and also provide an alternative of improving efficiency.
Table 16: The comparison of time consumption with different point numbers.
Optimization.
We visualize the optimization process Fig. 15. We reconstruct meshes using the neural SDF f learned differ- ent iterations. We
1000
1500
2000
2500
3000
3500
4000
Figure 15: Optimization during inference.
see that the shape is updated progressively to the ground truth shapes.
Conclusion
We propose a method resolve the key problem inferring SDFs from single noisy point cloud. Our method can effectively use a data-driven based prior as an initialization, and infer neural SDF by overfitting on a single noisy point cloud. The novel statistical reasoning successfully infers an accurate and smooth signed distance field around the single noisy point cloud with the data-driven based prior. By finetuning data-driven based priors with statistical reasoning, our method significantly improves the robustness, the scalability, the efficiency, and the accuracy in inferring SDFs from single point clouds. Our experimental results and ablations studies show our superiority and justify the effectiveness of the proposed modules.
10

---

References
[1] Matan Atzmon and Yaron Lipman. SAL: Sign agnostic learning of shapes from raw data. In IEEE Conference Computer Vision and Pattern Recognition, 2020.
[2] Matan Atzmon and yaron Lipman. SALD: sign agnostic learning with derivatives. International Conference on Learning Representations. 2021.
[3] Yizhak Ben Sha-aa, Chamin Hewa Koneputugodage. and Stephen Gould. DiGS: Divergence guided shape implicit neural representation for unoriented point clouds. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.
[4] Yizhak Ben-Shabat, Chamin Hewa Koneputugodage, and Stephen Gould. DiGS: Divergence guided shape implicit neural representation for unoriented point clouds. CoRR, abs/2106. 10811, 1, 2021.
[5] Federica Bogo, Javier Romero, Gerard Pons-Moll, and Michael J. Black. Dynamic FAUST: Registering
human bodies in motion. In IEEE Computer Vision and Pattern Recognition, 2017.
[6] Alexandre Boulch and Renaud Marlet. POCO: Point convolution for surface reconstruction. Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6302-6314 June 2022.
[7] Alexandre Boulch and Renaud Marlet. Poco: Point convolution for surface reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.
[8] Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong. Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijhom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[9] Ruojin Cai, Guandao Yang. Hadar Averbuch-Flow Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan. Learning gradient fields for shape generation. In European Conference on Computer Vision, 2020.
[10] Pedro Hermosilla Casajus, Tobias Ritschel, and Timo Ropinski. Total denoising: Unsupervised learning of 3d point cloud cleaning, In IEEE International Conference on Computer Vision, pages 52 60, 2019.
[11] Rohan Chabra, Jan Eric Lenssen, Eddy 1lg. Tanner Schmidt. Julian Straub, Steven Lovegrove, and Richard A. Newcombe. Deep local shapes: Learning local SDF priors for detailed 3D reconstruction. In European Conference Computer Vision, volume 12374, pages 608-625 2020.
[12] Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An Information-Ritn Rich 3D Model Repository. Technical Report 1rXiv:1512.03000 [cs.GR], Stanford University Princeton University- Toyota Technological Institute Chicago, 2015.
[13] Chao Chen, Zhizhong Han, and Yu-Shen Liu. Unsupervised inference of signed distance functions from single sparse point clouds without learning priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17712-17723, 2023.
[14) Chao Chen, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. Unsupervised learning of fine structure generation for 3D point clouds by 2D projections matching. In IEEE International Conference on Computer Vision, 2021.
[15] Chao Chen, Yu-Shen Liu, and Zhizhong Han. Latent partition implicit with surface codes for 3d representation, In Furopean Conference on Computer Vision, 2022.
[16] Chao Chen, Yu-Shen Liu, and Zhizhong Han. GridPull: Towards scalability in learning implicit represen- lations from 3d point clouds. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 18322-18334, 2023.
[17] Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[18] Julian Chibane, Aymen Mir, and Gerard Pons-Moll. Ncural unsigned distance fields for implicit function learning, arXiv, 2010.139333 2020.
[19] Gene Chou, Ilya Chugunov, and Felix Heide. GenSDF: Two-stage learning of generalizable signed distance functions. In Advances in Neural Information Processing Systems, pages 24905-24919, 2022.
[20] Christopher B. Choy, Danfei Xu, Jun Young Gwak, Kevin Chen, and Silvio Savarese. 3D-r2n2: A unified approach for single and multi-view object reconstruction. Bastian Leibe, Jini Matas, Nicu Sebe, and Max Welling, editors, European Conference on Computer Vision, volume 9912, pages 628-644 2016.
[21] Angela Dai and Matthias NieBner. Neural Poisson: Indicator functions for neural fields. arXiv preprint arXiv:2211.14245 2022.
[22] Philipp Erler, Paul Guerrero, Stefan Ohrhallinger, Niloy J. Mitra, and Michael Wimmer. Points2Surf: Learning implicit surfaces from point clouds. In European Conference Computer Vision, 2020.
11

---

[23] Miguel Fainstein, Viviana Siless, and Emmanuel Iarussi. DUDF: Differentiable unsigned distance fields with hyperbolic scaling. arXiv preprint arXiv:2402.08876 2024.
[24] Haoqiang Fan, Hao Su, and Leonidas J. Guibas. A point set generation network for 3D object reconstruc- tion from single image. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, pages 2463-2471, 2017.
[25] Qiancheng Fu, Qingshan Xu, Yew-Soon Ong, and Wenhing Tao. Geo-Neus: Gicometry-coniisten neural implicit surfaces learning for multi-view reconstruction. 2022.
[26] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Computer Vision and Pattern Recognition, 2012.
[27] Kyle Genova, Forrester Cole, Daniel Vlasic, Aaron Sarna, William Freeman, and Thomas Funkhouser. Learning shape templates with structured implicit functions. In International Conference on Computer Vision, 2019.
[28] Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. In International Conference on Machine Learning, volume 119 of Proceedings of Muchine Learning Research, pages 3789-3799, 2020.
[29] Zhizhong Han, Chao Chen, Yu-Shen Liu, and Matthias Zwicker. DRWR: A diffcrentiable renderer without rendering for unsupervised 3D structure learning from silhouette images. In International Conference on Machine Learning, 2020.
[30] Zhizhong Han, Chao Chen, Yu-Shen Liu, and Matthias Zwicker. ShapeCaptioner: Generative caption network for 3D shapes by learning a mapping from parts detected in multiple views to sentences. In ACM International Conference on Multimedia, 2020.
[31] Zhizhong Han, Xiyang Wang, Yu-Shen Liu, and Matthias Zwicker. Hierarchical view predictor: Unsuper- vised 3d global feature learning through hicrarchical prediction among unordered views. In Proceedings of the 29th ACM International Conference Multimedia, pages 3862 3862-3871, 2021.
[32] Rana Hanocka, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. Point2mesh: self-prior for deformable meshes. ACM Transactions on Graphics, 39(4):126, 2020.
[33] Pengehong Hu and Zhizhong Han. Learning neural implicit through volume rendering with attentive depth fusion priors. In Advances in Neural Information Processing Systems, 2023.
[34] Jiahui Huang, Hao-Xiang Chen, and Shi-Min Hu. A neural galerkin solver for accurate surface recon- struction, ACM Trans. Graph., (6)),, 2022.
[35] Jiahui Huang. Zan Gojeie, Matan Atzmon, Or Litany, Sanja Fidler, and Francis Williams. Neural kernel surface reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4369 4379, 2023.
[36] Meng Jia and Matthew Kyan. Learning occupancy function from point clouds for surface reconstruction. arXiv, 1137,, 2020.
[37] Chiyu Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nicßner, and Thomas Funkhouser. Local implicit grid representations for 3D scenes. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[38] Yue Jiang. Dantong Ji. Zhizhong Han, and Matthias Zwicker. SDFDiff: Differentiable rendering of signed distance fields for 3D shape optimization. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[39] Michael M. Kazhdan and Hugues Hoppe. Screened poisson surface reconstruction. ACM Transactions on Graphics, (2(3):29:1-29111 2013.
[40] Jaakko Lehtinen. Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras. Mika Aittala, and Timo Aila. Noisc2noise: Learning image restoration without clean data, In Jennifer G. Dy and Andreas Krause, editors, International Conference on Machine Learning, volume 80, pages 19900, 2018
[41] Chen-Hsuan Lin, Chaoyang Wang, and Simon Lucey. SDF-SRN: Learning signed distance 3D object reconstruction from static images. In Advances in Neural Information Processing Systems, 2020.
[42] Cheng Lin, Changjian Li, Yuan Liu, Nenglun Chen, Yi-King Choi, and Wenping Wang. Point2Skeleton: Learning skeletal representations from point clouds. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 4277-4286, 2021.
[43] Minghua Liu, Xiaoshuai Zhang, and Hao Su. Meshing point clouds with predicted intrinsic-extrinsic ratio guidance. In European Conference on Computer vision, 2020.
[44] Shaohui Liu, Yinda Zhang, Songyou Peng. Boxin Shi. Mare Pollefeys, and Zhaopeng Cui. DIST: Rendering deep implicit signed distance function with differentiable sphere tracing. In IEEE Conference Computer Vision and Pattern Recognition, 2020.
12

---

[45] Shi-Lin Liu, Hao-Xiang Guo, Hao Pan, Pengshuai Wang. Xin Tong, and Yang Liu. Deep implicit moving least-squares functions for 3D reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[46] Shichen Liu, Shunsuke Saito, Weikai Chen, and Hao Li. Learning to infer implicit surfaces without 3D supervision. In Advances Neural Information Processing Systems, 2019.
[47] William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3D surface construction algorithm. Computer Graphies, 2144):633-169, 1987.
[48] Shitong Luo and Wei Hu. Differentiaalle manifold reconstruction for point cloud denoising. In ACM International Conference on Multimedia, pages 1330- 1330, ACM, 2020.
[49] Shitong Luo and Wei Hu. Score-based point cloud denoising. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 4583-4592, 2021.
[50] Baorui Ma, Zhizhong Han, Yu-Shen I Liu, and Matthias Zwicker. Neural-pull: Learning signed distance functions from point clouds by learning to pull space onto surfaces. International Conference on Machine Learning, 2021.
[51] Baorui Ma, Yu-Shen Liu, and Zhizhong Han. Reconstructing surfaces for sparse point clouds with on-surface priors. In IEEE Conference on Computer Vision and Pattern Recognition, pages 6315,, 2022.
[52) Baorui Ma, Yu-Shen Liu, and Zhizhong Han. Learning signed distance functions from noisy 3D point clouds via noise to noise mapping. In International Conference on Machine Learning, pages 23338-233 23357. PMLR, 2023.
[53] Baorui Ma, Yu-Shen Liu. Matthias Zwicker, and Zhizhong Han. Reconstructing surfaces for sparse point clouds with on-surface priors. In IEEE Conference on Computer Vision and Puttern Recognition, 2022.
[54] Baorui Ma, Yu-Shen Liu, Matthis Zwicker, and Zhizhong Han. Surface reconstruction from point clouds by learning predictive context priors. In IEEE Conference on Computer Vision and Pattern Recognition, pages 6316-6327, 2022.
[55] Baorui Ma, Yu-Shen Liu, Matthias Zwicker, and Zhizhong Han. Surface reconstruction from point clouds by learning predictive context priors. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.
[56] Baorui Ma, Junsheng Zhou, Yu-Shen Liu, and Zhizhong Han. Towards better gradient consistency for neural signed distance functions via level set alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 17724-17734, 2023.
[57] Aihua Mao, Biao Yan. Zijing Ma, and Ying He. Denoising point clouds latent space via graph convolution and invertible neural network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5768-5777, June 2024.
[58] Julien N. Martel, David Lindell. Connor Lin, Eric R. Chan, Marco Monteiro, and Gordon Wetzstein. ACORN: adaptive coordinate networks for neural scene representation. CoRR, abs/2105.0275 2021.
[59] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occu- pancy networks: Learning 3D reconstruction in function space. In IEEE Conference on Computer Vision and Pattern Recognition, 2019,
[60] Zhenxing Mi, Yiming Luo, and Wenbing Tao. SSRNet: Scalable 3D surface reconstruction network. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[61] Mateusz Michalkiewicz, Jhony K. Pontes, Dominic Jack, Mahsa Baktashmotlagh, and Anders Eriksson. Deep level sets: Implicit surface representations for 3D shape inference. CoRR, abs/1901 ,06802, 2019.
[62] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren
Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, 2020.
[63] Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Differentiable volumetric rendering: Learning implicit 3D representations without 3D supervision. In IEEE Conference on Computer Vision and Puttern Recognition, 2020.
[64] Michael Oechsle, Songyou Peng, and Andreas Geiger. UNISURF: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In International Conference on Computer Vision, 2021.
[65] Amine Ouasfi and Adnane Boukhayma. Few 'zero level set' shot learning of shape signed distance functions in feature space. In European Conference on Computer Vision, 2022.
[66] Amine Ouasfi and Adnane Boukhayma. Robustifying generalizable implicit shape networks with a tunable parametric model. In Advances in Neural Information Processing Systems, 2023.
13

---

[67] Amine Ouasfi and Adnane Boukhayma. Mixing-denoisiin generalizable occupancy networks. In International Conference on 3D Vision, 2024.
[68]
Amine Ouasfi and Adnane Boukhayma. Unsupervised occupancy learning from sparse point cloud. In IEEE Conference on Computer Vision and Pattern Recognition, 2024.
[69] Jeong Joon Park, Peter Florence, Julian Straub, Richard Neweombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[70] Songyou Peng, Chiyu "Max" Jiang. Yiyi Liao, Michael Niemeyer, Mare Pollefeys, and Andreas Geiger. Shape points: A differentiable poisson solver. In Advances in Neural Information Processing Systems, 2021.
[71] Songyou Peng. Michael Niemeyer, Lars M. Mescheder, Mare Pollefeys, and Andreas Geiger. Convolu- tional occupancy networks. In European Conference Computer Vision, volume 12348, pages 523-540, 2020.
[72] Francesca Pistilli, Giulia Fracastoro, Diego Valsesia, and Enrico Magli. Learning graph-convolutional representations for point cloud denoising. In European Conference on Computer Vision, volume 12365. pages 103-118, 2020.
[73] Marie-Julie Rakotosaona, Vittorio La Barbera, Paul Guerrero, Niloy J. Mitra, and Maks Ovsjanikov. Pointcleannet: Learning to denoise and remove outliers from dense point clouds. Computer Graphics Forum, (9())188--003, 2020.
[74] Konstantinos Rematas, Ricardo
and Vittorio Ferrari. Sharf: Shape-conditioned radiance fields from single view. In International Conference on Machine Learning, 2021.
[75] Andrés Serna, Beatriz Marcotegui, François Goulette, and Jean-Em Emmanuel Deschaud. maris-rue-madame database- A 3D mobile laser scanner dataset for benchmarking urban detection, segmentation and classification methods. In International Conference on Pattern Recognition Applications and Methods, pages 819-824, 2014.
[76] Jaehyeok Shim and Kyungdon Joo. DITTO: Dual and integrated latent topologies for implicit 3d reconstruction. arXiv preprint arXiv:2403.00 05005, 2024.
[77] Vincent Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In Advances Neural Information Processing Systems, 2020.
[78] Vincent Sitzmann, Michael Zollhöfer, and Gordon Wetzstein. Scene representation networks: Continuous 3D-structure-aware neural scene representations. In Advances in Neural Information Processing Systems, 2019.
[79] Peng Songyou, Niemeyer Michael, Mescheder Lars, Pollefeys Marc, and Geiger Andreas. Convolutional occupancy networks. In European Conference Computer Vision, 2020.
[80] Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, and Sanja Fidler. Neural geometric level of detail: Real-time rendering with implicit 3D shapes. In JEEE Conference on Computer Vision and Pattern Recognition, 2021.
[81] Jiapeng Tang, Jiabao Lei, Dan Xu, Feiying Ma, Kui Jia, and Lei Zhang. SA-ConvONet: Sign-agnostic optimization convolutional occupancy networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
[82] Maxim Tatarchenko. Stephan R. Richter. Rene Ranfil, Zhuwen Li, Vladlen Koltun. and Thomas Brox. What do single-view 3D reconstruction networks learn? In The IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[83] Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Carsten Stoll, and Christian Theobalt. PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations. European Conference Computer Vision, 2020.
[84] Delio Vicini, Sébastien Speierer, and Wenzel Jakoh. Differentiable signed distance function rendering. ACM Transactions on Gruphics, 11:4)::25:11125:::::: 2022.
[85] Jiepeng Wang. Peng Wang, Xiaoxiao Long. Christian Theobalt, Taku Komura, Lingjie Liu, and Wenping Wang. NeuRIS: Neural reconstruction of indoor scenes using normal priors. In European Conference on Computer Vision, 2022.
[86] Peng Wang, Lingjie Liu, Yuan Liu, Christian Thcobalt, Taku Komura, and Wenping Wang. NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In Advances in Neural Information Processing Systems, pages
2021.
[87] Peng-Sin Shuai Wang, Yang Liu, and Xin Tong. Deep octree- based cans with output-guided skip connections for 3d shape and scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 2666-267,20 2020.
14

---

[88] Ruian Wang. Zixiong Wang, Yunxiao Zhang, Shuangmin Chen, Shiqing Xin, Changhe Tu, and Wenping Wang. Aligning gradient and hessian for neural signed distance function. In Advances in Neural Information Processing Systems, volume 36, pages 63515-63528, 2023.
[89] Yiqun Wang, Ivan Skorokhodov, and Peter Wonka. HF-NeuS: Improved surface reconstruction using high-frequency details. 2022.
[90] Zhen Wang. Shijie Zhou, Jeong Joon Park, Despoina Paschalidou, Suya You, Gordon Wetzstein, Leonidas Guibas, and Achuta Kadambi. Alto: Alteraaing latent topologies for implicit 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 259-270, 2023.
[91] Francis Williams, Zan Gojeic, Sameh Khamis, Denis Zorin. Joan Bruna, Sanja Fidler, and Or Litany. Neural fields as Icarnable kemels for 3D reconstruction. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 18500-18510, 2022.
[92] Francis Williams, Teseo Schneider. Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. Deep geometric prior for surface reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[93] Francis Williams, Matthew Trager, Joan Bruna, and Denis Zorin. Neural splines: Fitting 3D surfaces with wnde neural networks. In IEFF Conference on Computer Vision and Pattern Recognition, pages 9949,9958,20 2021.
[94] Yunjie Wu and Zhengxing Sun. DFR: diffcrentiable function rendering for learning 3D generation from images. Computer Graphics Forum, 39(5)22412252 2020.
[95] Peng Xiang, Xin Wen, Yu-Shen Liu, Yan-Pei Cao, Pengfei Wan, Wen Zheng, and Zhizhong Han. SnowflakeNet: Point cloud completion by snowflake point deconvolution with transfornffrr. In JEEE International Conference on Computer Vision, 2021.
[96] Huizong Yang, Yuxin Sun, Ganesh Sundaramoorthi, and Anthony Yezzi. StEik: Stabilizing the optimiza- tion of neural signed distance functions and finer shape representation. In Advances Neural Information Processing Systems, 2023.
[97] Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. Volume rendering of neural implicit surfaces. In Advances in Neural Information Processing Systems, 2021.
[98] Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lipman. Multiview neural surface reconstruction by disentangling geometry and appearance. Advances in Neural Information Processing Systems, 33, 2020.
[99] Wang Yifan, Shihao Wu, Cengiz Oztireli, and Olga Sorkine-Homunn, Iso- Iso-Points: Optimizing neural implicit surfaces with hybrid representations. CoRR, 0tss22012064433 2020.
[100] Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger. MonoSDF: Exploring monocular geometric cues for neural implicit surface reconstruction. 2022.
[101] Sergey Zakharov, Wadim Kchl, Arjun Bhargava, and Adrien Gaidon. Autolabeling 3D objects with diffcrentiable rendering of sdf shape priors. In IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[102] Wenbin Zhao, Jiabao Lei. Yuxin Wen, Jianguo Zhang, and Kui Jia. Sign-agnostic implicit learning sur- face self-similarities for shape modeling and reconstruction from raw point clouds. CoRR, abs/2012.07498, 2020.
[103] Junsheng Zhou, Baorui Ma, Shujuan Li, Yu-Shen Liu, and Zhizhong Han. Learning more continuous zero level set unsigned distance fields through level set projection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3181-3192, 2023.
[104] Junsheng 7hou, Baorui Ma, Yu-Shen Liu, Yi Fang, and Zhizhong Han. Learning consistency-awar unsigned distance functions progressively from raw point clouds. In Advances in Neural Information Processing Systems, 2022.
[105] Qian- Yi Zhou and Vladlen Koltun. Dense scene reconstruction with points of interest. ACM Transactions Graphies, 12:111112 2013.
[106] Runsong Zhu, Di Kang, Ka-Hei Hui, Yue Qian, Shi Qiu, Zhen Dong, Linchao Bao, Pheng- Ann Heng, and Chi- Wing Fu. Ssp: Semi-signed prioritized neural fitting for surface reconstruction from unoriented point clouds. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 7769-3778 2024.
15

---

A Appendix
A.1
Limitations
Our method is still limited to too large noises. For noises that corrupted shapes too much, our method still produces bad results. One direction for our future work is improve our prior, so that we could have a better sense of shape even under large noises.
A.2 Detailed Comparisons on 3D Scene
We detail evaluations each scene in 3D dataset in Tab. 17. The comparisons highlight
our advantages in each scene.
Name
Metrics
COce [71]
LIG [37]
DeepLS[11]
NP [50]
N2NM [52]
Ours
CD x 1000
27.46
3.055
0.401
1.204
0.504
0.429
Burghers
CD11
0.079
0.045
0.017
0.031
0.020
0.016
NC
0.907
0.835
0.920
0.905
0.925
0.939
CD12 x 1000
9.540
9.672
6.103
1.079
0.602
0.333
Lounge
CD11
0.046
0.056
0.053
0.019
0.016
0.014
NC
0.894
0.833
0.848
0.910
0.923
0.935
CD x 1000
3.610
0.609
5.795
0.442
0.389
Copyroom
CDLi
0.045
0.036
0.021
0.036
0.016
0.016
NC
0.892
0.810
0.901
0.862
0.903
0.916
CD12 X 1000
20.46
5.032
0.320
0,983
0.330
0.313
Stonewall
CD11
0.069
0.042
0.015
0.029
0.020
0.015
NC
0.905
0.879
0.954
0,930
0,951
0.961
CD12 1000
2.054
9.580
0.601
1.513
0.657
0.482
Totepole
CDL1
0.021
0.062
0.017
0.054
0.023
0.020
NC
0.943
0.887
0.950
0.893
0.945
0.957
Table 17: Numerical Comparisons on 3D Scene dataset terms of CDD1. CD12 and NC.
A.3 More Results
We visualize more surface reconstruction results under ShapeNet [12], ABC [22], Surface Recon- struction Benchmark (SRB) [92], FAMOUS [22], D-FAUST [5] and nuScenes [8] Fig. 16, Fig. 17, Fig. 18, Fig. 19, Fig. 20 and Fig. 21.
16

---

Noisy Input.
Con/Occ
POCO
ALTO
N2NM
Ours
GT
Figure 16: Comparison in surface reconstruction on ShapeNet.
Input
ConvOcc
IMLS
P2S
NeuraPPull
OnSurf
N2NM
Ouns
GT
Figure 17: Comparison in surface reconstruction on ABC.
17

---

117
1
1
Input Poinl2Mesh
PSR
SIREN
GridPull
ALTO
Steik
NKSR
N2NM
Ours
Figure 18: Comparison in surface reconstruction on SRB.
Input
IMLS
Nuura
LPI
OnSunf
GridPull
NENM
Ours
GT
Figure 19: Comparison in surface reconstruction on FAMOUS.
18

---

Input
IGR
Point2Mesh
SAP
N2NM
Ours
GT
Figure 20: Comparison in surface reconstruction on D-FAUST.
N2NM
Ours
Figure 21: Comparison in surface reconstruction nuScenes.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the Appendix A.I.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
20

---

We describe in Method one of the core contributions of the local noise-to- noise mapping, and although there is no theory or theorem it, we verify its validity and reasonableness in our experiments.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed information in reproducing our methods in Implementation Details of Section 3.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
21

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We provide our demonstration code as part of our supplementary materials. We will release our source code, data and sufficient instructions upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. cc/
Sttpi//gguidd//ooeeuiiei www.imimssiooooicyyy... for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / httpppscccppbllccceeeeeeeellllet semblic/guudes.... gwwgguuiee/eee
Seeiiidee//oeubnnnnn Codeeubmi ssi asionPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all the training and test details for shapes and scenes in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We report the average performance terms of several metrics as the experi- mental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
22

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report our inference time with other methods in the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: httprr/////////....////////ss cc htcp//ullcc//eee Apublic/Ethii cl ines?
Answer: [Yes]
Justification: The research conducted in this paper conforms in respects to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the application and potential positive impact of our method in the introduction.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
23

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
There is no such risk to the paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We use open-source datasets and code under their licence.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi wwwowitetcc....c com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
24

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "U3H3iVkLWs", "reviewer": "Reviewer_AdmT", "summary": "The paper proposes 3D shape reconstruction given a noisy point cloud using a test-time optimization approach. The paper proposes a new loss function that can work directly on noisy point clouds without the need of ground truth normals or SDF. The approach involves learning a network (DeepSDF) to predict SDFs and later fine-tune the network on a single shape during test. The learned priors reduce the test-time fitting and leads to better reconstruction results.", "questions": "1. Please improve the notations in the method section. The vectors should be bold. Use different symbols for parameterized function and output.\n2. The motivation of the loss function is not clear. Perhaps a diagram would enhance the understanding for a reader.", "limitations": "1. Limitations regarding pose variations and incomplete data (holes) are not discussed.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The paper proposes a simple and easy to implement approach to 3D reconstruction that works on a bit noisy point clouds.\n2. The better quality results and fast inference has a broad significance in 3D reconstruction.", "weaknesses": "1. In the introduction, the test-time optimization is hailed as a novel thing, however previous methods (including DeepSDF) has already proposed it.\n2. The main contribution is the loss function defined using \"pulling\" operation, which forces the surface defined by the projected points close to the input points be similar to the input point clouds. However, why this loss function should work in a noisy point cloud setting is not well motivated and well explained.\n3. The writing of the method section can be significantly improved. For example, the symbol $f$ seems to represent both the neural network that predicts the signed distance field and the field it self. More about it in the Questions section.\n4. The use of the phrase \"statistical reasoning\"is quite vague. The phrase is used throughout the manuscript but does not convey anything meaningful.\n5. It is not clear whether the baselines in the experiment section are also trained on noisy data to make them robust.\n6. Comparison with Points2Surf [21] method is missing. I think it work quite well on shapes in Figure 4.\n7. What does \"extreme noise\" mean in the Table 13? What does it translate to in percentage of the scale of the scene.\n8. How do you achieve 100% sparsity in Table 14? Does it imply no points?\n9. Noise level of 0.5% is quite low to claim that the method is robust.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Why pulling works\n\nPulling queries towards surface points has been widely used to estimate a signed distance function from a point cloud. We can use a neural network to infer the SDF by pulling randomly sampled queries to their nearest surface points using the predicted signed distances and gradients. We train this network by minimizing the distance between the pulled queries and queries’ nearest surface points. \n\nHowever, it is not a good way to infer an SDF through pulling queries on a noisy point cloud, since queries’ nearest surface points are not accurate to find due to noise corruption. To resolve this issue, we got inspiration from [1] and [2], which infers the clean 2D and 3D information from noisy observation respectively. Our statistical reasoning in a local region managed to estimate an SDF by minimizing the distance between different sets of pulled queries that are randomly sampled around and their commonly shared noisy patch. The rationale behind this is that the different sets of pulled queries are forced to be as near to the same noisy point patch as possible, when we do this over different noisy point patches that overlap with each other, the optimization converges to a consistent zero-level set.\n\n[1] Noise2noise: Learning Image Restoration without Clean Data\n\n[2] Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping\n\n2. Notation\n\nWe will revise the notation to make them more specific and clear.\n\n3. Statistical Reasoning\n\nPlease read “G5. Statistical Reasoning” in the rebuttal above.\n\n4. Baselines\n\nData-driven based methods such as IMLS and ConOcc were pre-trained on point clouds with noise of different variances, so that they can work with various noises during inference. While overfitting based methods directly learn SDFs on single shapes in the testing set. Both data-driven based and overfitting based methods use the same set of noisy point clouds as ours for evaluations.\n\n5. Comparison with Points2Surf\n\nWe compared the data-driven based method Points2Surf on ABC in Tab. 3 (“P2S”), Fig. 3 (“P2S”), and Fig. 12 (“P2S”). Numerical and visual comparisons indicate that our method significantly outperforms Points2Surf. Since Points2Surf is a relatively old method that was proposed back in 2020, and its following methods like PCP [3] and OnSurf [4] have reported much better performance on FAMOUS dataset, we just list PCP and OnSurf as baselines in Tab. 5. Moreover, Points2Surf did not report its results on SRB in the original paper, and we failed to reproduce plausible results on SRB by running its code as well, because of its poor generalization ability on SRB. Thus, we did not compare Points2Surf in Fig.4.\n\n[3] Surface reconstruction from point clouds by learning predictive context priors\n[4] Reconstructing surfaces for sparse point clouds with on-surface priors\n\n6. Extreme Noise in Tab. 13\n\nThe noise levels of middle and max come from the ABC dataset released by Points2Surf. The middle indicates noises with a variance of 0.01L, where L is the longest edge of the bounding box. The max indicates noises with a variance of 0.05L. Our extreme noise comes with a variance of 0.07L.\n\n\n7. 100\\% sparsity in Tab. 14\n\nAs stated in Line 361-362, the percentages shown in Tab. 14 are ratios of left over points after the downsampling. Thus, 100\\% indicates the results without downsampling. We will clarify this in the revision.\n\n8. 0.5\\% noise variance\n\nWe use 0.5\\% noise variance on ShapeNet for fair comparisons with previous methods. As shown on ABC in Tab. 3, FAMOUS in Tab. 5, shapes with extreme noise in Tab. 13, and the additional results in Fig. 1 in the rebuttal PDF, our method can handle point clouds with noise variance as large as 7\\%.\n\n9. Understanding of Our Loss\n\nThe diagram of our loss is shown as L_{Local} on the right in Fig. 1. For a noisy point patch, we randomly sample a set of queries within the noisy point patch. Then, we use the learned SDF to pull them onto the zero-level set, and we minimize the EMD distance between the set of pulled queries and the noisy point patch.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A2MSz8ci4c", "reviewer": "Reviewer_fwt4", "summary": "The authors propose an implicit surface reconstruction method from point clouds that uses a learned prior to initialize the optimization of a neural SDF. First, a neural SDF generator based on DeepSDF [66] is trained on a dataset of shapes to learn a prior over shapes. Given a point cloud, both the shape code used as input for the generator, as well as the parameters of the neural SDF are then optimized to fit the point cloud, using an approach based on Noise to Noise Mapping (N2NM) [50], but focusing on local patches instead of the full global point cloud. The authors show that this approach performs better than N2NM as well as several state-of-the-art methods in an extensive evaluation on several datasets.", "questions": "- A clarification of the unclear details regarding local patches would help.\n- A clarification of how timings for methods were chosen would help as well.", "limitations": "The authors have not discussed clear limitations unfortunately.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Using a learned prior as initialization for an optimization-based surface reconstruction method like N2NM has not been done before as far as I can tell.\n- Using local patches also seems like a (smaller) contribution to me (the authors show that this contributes to the performance, but it is not very clear to me at this point why it improves performance significantly - this could be improved).\n- Results show consistently better reconstruction accuracy than the state-of-the-art.\n- The evaluation is extensive and mostly well-done, and convincingly shows and advantage over the state-of-the-art.", "weaknesses": "- While the idea of using a learned prior as initialization of an optimization based method has not been done for surface reconstruction as far as I can tell, it is not very surprising that this can be done and is also relatively straight-forward to do. Both stages are essentially the same as existing work [66] and [50], with some deviation from [50] in using local patches. So all-in-all, the technical contribution does not seem huge, although possibly still large enough for acceptance.\n- While local patches do seem to improve performance, it is not fully clear why this is the case, and the exact details of using local patches are missing some details in the exposition, making it unclear exactly how they are implemented.\n- A few recent methods are missing both from the related work and the comparisons (see below).\n- The exposition could use some additional details and clarifications in some parts (see below).\n\nMost of the weaknesses, apart from the first one, can be addressed with text changes to some extent. Given the good performance of the method and the extensive evaluation, I lean towards acceptance.\n\nDetails:\n- The field of surface reconstruction from point clouds is quite vast, so the authors missed a few works:\n  - Neural-Singular-Hessian: Implicit Neural Representation of Unoriented Point Clouds by Enforcing Singular Hessian, Wang et al., TOG 2023\n  - Iterative Poisson Surface Reconstruction (iPSR) for Unoriented Points, Hou et al., TOG 2022\n  - 3DShape2Vecset: A 3D Shape Representation for Neural Fields and Generative Diffusion Models, Zhang et al., TOG 2023\n  - Geoudf: Surface reconstruction from 3d point clouds via geometry-guided distance representation, Ren et al., TOG 2023\n\n- Given good normals, surface reconstruction becomes much easier - the normal computation could be followed by Poisson reconstruction based on the normals for example. Therefore papers to compute oriented normals are relevant, such as:\n  - Orienting Point Clouds with Dipole Propagation, Metzer et al., TOG 2021\n  - Globally Consistent Normal Orientation for Point Clouds by Regularizing the Winding-Number Field, Xu et al., TOG 2023\n  - SHS-Net: Learning Signed Hyper Surfaces for Oriented Normal Estimation of Point Clouds, Li et al., CVPR 2023\n\n- The exposition could use some additional details and clarifications:\n  - The approach for using local patches is missing some details: How many patches are used during optimization? In which order are patches evaluated? One after the other? Or is a random patch chosen in each iteration? This should be clarified.\n  - The difference of the second stage to [50] could use some more discussion. The authors describe the application of [50] to local patches as main difference (as shown in Eq. 3). But since the expectancy in Eq. 3 is taken over random samples near random patches, how is this different than taking the expectancy over random samples near the whole shape (as in [50])? This could use some  discussion. (Or is the SDF actually optimized separately per patch? From reading I did not get this impression.)\n  - [50] also uses a geometric consistency constraint as regularization, although such a regularization is not described by the authors. If it is not used, a discussion why the authors removed it might be useful.\n  - c seems to be initialized to a fixed constant vector: the average of all embeddings in the training set, while the original DeepSDF paper uses a c randomly sampled from a Gaussian. Why did the authors use a different initialization for c here? Did the average empirically perform better? This could be discussed.\n  - In the paragraph starting at line 129, It should be stated clearly that the method described here is the method proposed in [50] (although evaluated at random samples near random local patches, rather than random samples near the whole shape).\n\n- The evaluation is quite extensive, but would benefit from a few improvements:\n  - In the evaluation, it would be good to hear more about how the time used to fine-tune each optimization-based method was chosen. Was each result optimized until convergence? Or otherwise, how was the number of iterations/optimization time chosen?\n  - It would be good to add DeepSDF to the comparison, as it is the closest to the initialization used for the optimization stage, and would show how much the fine-tuning  improves accuracy.\n  - In the ablation, using a large local region seems worse than using a global approach without local regions. This should be discussed.\n\n- References [52] and [53] are duplicates", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Contribution and Novelty\n\nOur method is not a simple combination of DeepSDF and noise2noise. Please see G1-G3 in our rebuttal above.\n\n2. Why Local Patches Work Better\n\nPlease read “G4. Why Local Patches Work Better” in our rebuttal above for the analysis.\n\n3. We will add references you mentioned\n\nWe will add these references. The main difference between these methods and ours lies in the ability of reasoning on noisy point clouds in an overfitting manner. This ability significantly improves the generalization performance on unseen noisy point clouds.\n\n4. Local Patches in Optimization\n\nWe randomly sample a noisy patch and a set of queries within the patch in each iteration. Thus, there is no order needed. We use the loss difference between two successive iterations as a metric to determine the convergence, which can be used to conduct time comparisons in ablation studies. The optimization is usually converged in 4000 iterations, and no more than 4000 patches are used. We will make these details more clear in our revision.\n\n5. Difference to the Global Mapping [50]\n\nPlease read “G4. Why Local Patches Work Better” in our rebuttal above before going ahead.\n\nBesides the inaccuracy and inefficiency in inference, another downside of [50] is the requirement of additional constraints on the pulling, i.e., each travel distance should be as small as possible, which makes sure the learned distance is the minimum to a surface, as shown in Fig. 3 in the original paper of [50]. The reason why [50] needs a constraint like this is two fold. One is that the optimization of SDF is still not converged well during inference, and it produces lots of uncertainty, thus queries can not get pulled at the right place. The other is that the loss of noise2noise just pushes the pulled queries to be as near to the noisy patch as possible, but does not constrain how the pulling will be conducted. Thus, our local patches resolve this issue with a patch limit, which also gets rid of the regularization term. Our preliminary results showed that the regularization term does not improve the performance but slows down the optimization a lot. We will make this more clear in our revision.\n\n6. Shape Embedding c\n\nPlease read “G3. Traditional test-time optimization vs ours” in our rebuttal above before going ahead.\n\nUsing averaged shape code as an initialization is a key point to generalize our prior with an overfitting loss.  As shown in our ablation studies in Tab. 9, using randomly initialized shape code (“Without Embed”) makes a negative impact on the accuracy and efficiency. Our averaged shape code takes a good advantage of the prior space, and provides an object-like SDF to start the inference with uncertainty of SDF using the overfitting loss.\n\n7. Convergence Metric\n\nWe use the loss difference between two successive iterations as a metric to determine the convergence, as shown in Tab. 9 and Tab. 11. The optimization is usually converged in 4000 iterations, as stated in Line 164. Each result was recorded when the optimization converges. We will make these details more clear in our revision.\n\n8. Comparisons with DeepSDF\n\nWe compare it with DeepSDF in Tab. 9. As stated in Line 329-330 and the result (“Fixed Param”) in ablation studies in Tab. 9, the auto-decoding introduced in DeepSDF can not work well with the overfitting loss on single noisy point clouds. In almost all cases, it does not reconstruct a plausible shape. As analyzed in “G3. Traditional test-time optimization vs ours”, the signed distances at the same locations inferred by the overfitting loss from a single noisy point cloud are fluctuating iteration by iteration, which is not like the GT signed distances, i.e., constant values, used in DeepSDF. This makes the prior generalization different a lot.\n\n9. Why Larger Patches Do Not Work Well\n\nPlease read “G4. Why Local Patches Work Better” in our rebuttal above going ahead.\n\nWith larger patches, more queries and noisy points that are far away from each other are paired to infer the SDF, which is quite similar to the global strategy [50].  This reduces the reconstruction accuracy and efficiency.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6sm3fYqNBe", "reviewer": "Reviewer_mU7t", "summary": "The presented work tackles the problem of reconstructing a shape from a noisy point cloud (PC) into an implicit representation, using the signed distance function (SDF). Recent methods are categorised into i) data-driven approaches that learn a shape prior with a dataset of training shapes, with poor generalization but generally faster, or ii) approaches that overfit on a single PC at a time, with better generalization but slow convergence. The paper proposes a middle ground: finetuning a pre-trained shape prior model to a single PC, using a novel approach that performs \"local statistical reasoning\": supervising local parts of the shape with multiple sampled noisy PC. The method is experimentally validated against multiple baselines, either data-driven or overfitting-based, and on multiple datasets showing better reconstruction accuracy at lower convergence times. A final ablation study shows that the prior and locality of the loss play a role in the method's performance.", "questions": "1. What is the reasoning behind using the locality in the \"statistical reasoning\" and its better performance?\n2. For the ShapeNet dataset, is it per-category and then averaged or trained on multiple categories combined? (and which ones?) Same questions regarding the SRB/FAMOUS experiments with ShapeNet prior.\n3. What are the PC sizes? It's hard to appreciate the other parameters without this info. What are the datasets sizes in number of training and test shapes?\n4. How does the prior help if it apparently does not need to be from the same dataset as the test shapes (e.g., SRB/FAMOUS experiments)? Is it actually pre-training on any valid SDFs that helps? If so, how do you think it compares to the simple SAL [2] geometric initialization?\n5. Why are the metrics (CD-L2/L1, NC, F1-score) not used consistently through the datasets and results?\n\n[2] Atzmon and Lipman, SAL: Sign Agnostic Learning of Shapes from Raw Data, CVPR 2020.", "limitations": "Limitations are briefly mentioned in the appendix regarding noise level. What kind of noise level is considered too big? It might also be worth commenting on the noise types (see Weakness 3.), is the method working only for Gaussian noises?", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Experiments are extensive: the PC reconstruction is evaluated against multiple baselines (both data-driven or overfitting-based) on various datasets (ShapeNet, ABC, FAMOUS, SRB, and D-FAUST). In nearly all cases and presented metrics, the proposed approach is the best.\n2. I found the locality approach to solving the problem interesting; supervising parts of the shape at a time instead of the full one.\n3. Using a shape prior that is unrelated to the target object (such as ShapeNet prior for FAMOUS reconstruction) also works. This suggests that this approach is applicable even if the target shape is from an unknown category (thus helps for generalization).\n4. The method is fairly simple and clear, which should make it easily usable.", "weaknesses": "1. The main pipeline is fairly simple (fine-tuning a pre-trained model). I believe the locality of the \"statistical reasoning\" is perhaps the main novelty, and the ablation shows that it improves accuracy and convergence time (§ at L.342), but it is not discussed or explained why using a local approach instead of a global one improves convergence and accuracy.\n2. The writing could be improved. More specifically, the experimental results on the different datasets all make more or less the same points and so could be condensed into something more compact. This can free up some space to expand other parts and increase their clarity, e.g., the related works and the \"Neural Signed Distance Function.\" paragraph (L.101).\n3. Throughout the work, the noise is assumed (additive) Gaussian. While this is not a weakness in itself, this still is an assumption that should be explained and made clear:\n   1. Why is a Gaussian noise model good? Are Lidar data, for instance, usually with gaussian noise and what real use-cases does it apply to? Generally speaking, the motivation of the work could be expanded beyond \"reconstructing a PC to an SDF\".\n   2. \"Statistically, the expectation of the zero-level set should have the minimum distance to all the noisy point splitting.\" (L.131-132) This is true assuming Gaussian noise with expectation 0. With other noise types the loss may not perform as well (e.g., \"Text removal\" in Noise2Noise [1]).\n\n[1] Lehtinen et al., Noise2Noise: Learning Image Restoration without Clean Data, ICML 2018.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Contribution and Novelty\n\nOur novelty is not only the loss but also the way of generalizing the prior. Please see G1-G3 in our rebuttal above.\n\n2. Why Local Patches Work Better\n\nPlease read “G4. Why Local Patches Work Better” in our rebuttal above for the analysis.\n\n3. Writing\n\nWe will follow your advice to polish our writing in our revision.\n\n4. Gaussian Noise\n\nPlease read ”G6. Gaussian Noise” in our rebuttal above for more details.\n\n\n5. What is Statistical Reasoning and Why Local Patches Work Better\n\nPlease read “G5. Statistical Reasoning” and “G4. Why Local Patches Work Better” in our rebuttal above.\n\n6. Training Data-driven Priors\n\nThe prior is learned on multiple shape categories on ShapeNet. We also tried to learn the prior in each category separately, the performance can get even better than we reported in the current submission. \n\n7. Priors Used on SRB/FAMOUS\n\nAs stated in Line 237-239 and Line 253-255, we use the prior learned from ShapeNet, and generalize the learned prior on SRB/FAMOUS for inference. Either SRB or FAMOUS is much different from ShapeNet. Our good performance on these real scans justify the good generalization ability of our method.\n\n8. Point Cloud Size and Training and Testing Sets\n\nFor point clouds for training, we used the data from [50] for fair comparison on ShapeNet, where each point cloud has 3k points. We used over 27k shapes from 13 categories in ShapeNet for learning a data-driven prior, and generalize this prior on SRB, FAMOUS, D-FAUST, 3D scene, KITTI, and Paris-rue-Madame datasets which do not contain a training set. We used nearly 5k shapes from the training set in ABC to learn a prior which is generalized to the testing set, where each shape has 5k to 12k points. \n\n\nFor testing shapes, the point number of point clouds is determined by datasets. We used about 7k shapes from 13 categories on ShapeNet, and each point cloud has 3k points. On SRB, there are 5 shapes, and each shape contains about 57k to 95k points with noise. On ABC, the testing set we used include 100 shapes, each shape has 5k to 12k points. On FAMOUS, there are 22 shapes, and each shape has 20k points. On D-FAUST, there are 5 shapes for testing, and each shape has 200k points. On 3D scene, there are 5 scenes, and each scene has about 3900k points. On KITTI, we use one scene which contains 13720k points. And on Paris-rue-Madame, we use one scene which contains 10000k points.\n\nWe will make this more clear in our revision.\n\n\n9. Difference to SAL geometry initialization\n\nGeometry initialization introduced by SAL uses analytically determined parameters to initialize an SDF like a sphere, which requires to set a proper radius and just determines the outside and inside. While ours can produce an object like SDF, which produces more details in the initialization, and more importantly, our prior can adjust the SDF fast according to what it saw during the training. \n\nWe do not think any valid SDF can help. Since simple shapes just provide an SDF with a boundary indicating the inside or outside, but can not adjust the SDF with any experience.\n\nWe conduct an additional experiment to compare with different initializations in Tab. 2 and Fig. 4 in the rebuttal PDF, including random initialization, geometry initialization, simple square, and ours. We can see our prior can reconstruct more accurate surfaces from single noisy point clouds in much shorter time than any other initializations.\n\n10. Metrics on Different Benchmarks\n\nFor fair comparisons, we follow previous methods to report the results in terms of the same metrics they used, which leads to inconsistent metrics on different benchmarks.\n\n11. Big Noise Level\n\nThere is no standard to define what noise level is a big noise level. But we can handle pretty large noises like what we show in Fig. 1 in the rebuttal PDF. Noises with a variance of 5% are usually used as a large noise by previous methods, but we can handle noises with a 7% variance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9U8lUKOjva", "reviewer": "Reviewer_76DM", "summary": "The paper presents a method for learning neural signed distance functions (SDFs) from noisy point clouds. This approach integrates the advantages of data-driven and overfitting-based methods to enhance generalization, accuracy, and inference speed. They employ statistical reasoning within local regions to refine the estimation of SDFs without requiring clean point clouds or signed distance supervision.", "questions": "1. How does the method perform under limited computational resources? When scaling to larger datasets, is there a significant trade-off between accuracy and computational efficiency?\n2. Can the method maintain its performance across different types of noise or corruption that were not part of the training set?\n3. How does the method perform when such priors are unavailable, or how sensitive is it to the initial conditions set by these priors?", "limitations": "See the \"Questions\" and \"Weaknesses\" sections above.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. This paper successfully combines data-driven and overfitting-based approaches, leveraging their strengths while mitigating their weaknesses. \n2. This paper shows that the method enhances robustness against noise, a common real-world challenge. They effectively use local statistical reasoning to adjust the neural SDFs, improving accuracy in surface reconstruction and denoising tasks, as evidenced by the benchmark results.", "weaknesses": "1. There are a lot of typos in the manuscript, e.g., In the Abstract and Introduction, \"stat-of-the-art\" should be \"state-of-the-art\"; Line 57: \"Learning implicit functions have achieved\" should be \"Learning implicit functions has achieved\"; Line 39: \"overfits an neural network\" should be \" a neural network\", \"Querys\", and so on. In the Abstract, the \"prompt\" might be \"promote\". It would be best to check out all the typos.\n2. Dependency on initial conditions: The performance might heavily rely on the quality of the initial data-driven priors, which might limit the method's effectiveness when such priors are not well-tuned or applicable.\n3. The paper does not extensively discuss the computational demands or scalability when applied to larger datasets (e.g., nuScenes and Waymo) or more complex scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Typo\n\nWe will correct these typos and proofread the paper more carefully.\n\n2. Impact on Performance by the Prior\n\nSince we use ground truth signed distances and a mature neural network to learn an implicit function as a prior, the network usually converges quite well. Imperfect priors should not be a concern because of our robust loss. To justify this, we conduct an experiment with a prior that is trained in different iterations, such as 500, 1000, 1500, and 2000 (used in our paper). As shown in Fig. 5 in the rebuttal PDF, we did not see a significant difference between these reconstructions.\n\n3. Time Complexity\n\nWe reported time comparisons in Tab. 9 and Tab. 11. Using a prior can provide a good object-like SDF and adjusts the field accordingly fast. Moreover, our loss that infers information in local regions can also be more efficient than the global strategy, since we avoid points and queries that are far away from each other, please read “G4. Why Local Patches Work Better” above for more details. Thus, our method is much more efficient than globally overfitting based methods. The time is recorded on a single RTX3090 GPU card.\n\n4. Results on nuScenes\n\nWe did report results on complex scenarios like the results on KITTI in Fig. 8 and Paris-rue-Madame in Fig. 9. Both of these results are produced on large scale real scans. Similarly, we additionally report a result on a scene from nuScenes in Fig. 6 in the rebuttal PDF, where our reconstructed mesh surface is nearer to the point and not fat like the N2NM’s result. All these results indicate that our method can handle very large scale and complex real scans.\n\n5. Running with Limited Resources\n\nLimited computational resources will make a negative impact on the speed but not the accuracy. As we explained in “G4. Why Local Patches Work Better” in the rebuttal above, we train each patch in a batch, which is a good way to run our method on multiple GPU cards in parallel. Another trade-off between accuracy and computational efficiency is to downsample the noisy point cloud. We additionally conducted an experiment with downsampled noisy points in Fig. 2 and Tab. 1 in the rebuttal PDF. We can see that we can work well on much fewer points, and also save some time.\n\n6. Noise Types\n\nPlease read ”G6. Gaussian Noise” in the rebuttal above for more details.\n\n7. Corruptions\n\nSince our method is targeting on inferring SDF from single noisy shapes, the shapes we used are mainly corrupted by noises, where the prior is learned on clean shapes that are not corrupted at all. But we found our method can generalize the prior on unseen shapes or scenes quite well. Like the objects in SRB or FAMOUS in Fig. 4 and Fig. 5, humans in D-FAUST in Fig. 6 and scenes in 3D scene in Fig. 7, our prior did not see these kinds of data during training. Similarly, our method also did not see any corruptions like some missing structures on real scans like KITTI in Fig. 8, Paris-rue-Madame in Fig.9, and nuScene in Fig. 6 in the rebuttal PDF, but our method also handles these corruptions quite well. This is because our statistical reasoning loss can infer information in local regions in an overfitting manner.\n\n8. Results without Our Prior\n\nUsing data-driven prior is one of our contributions to combine the data-driven based prior with the overfitting based strategy. Our ablation studies in Tab. 9 justify the effect of the prior. Using either random parameters in the network (“Without Prior”) or random initialized shape code (“Without Prior”, “Without Embed”) indicates no prior is used in the experiments, which either lowers the accuracy and slows down the convergence.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the reviewers' valuable comments, which highlighted our simple and interesting method (AdmT, mU7t), strong performance and extensive experiments (fwt4, mU7t, 76DM), and the broad significance and usefulness of our work (mU7t, AdmT).

G1. Our Novelty
 
Our novelty lies in how to combine data-driven based priors with the overfitting based strategy but rather the test-time optimization. Our framework is much different from the test-time optimization introduced in DeepSDF, since DeepSDF uses ground truth (GT) signed distances during both training and testing, which is relatively easy to generalize the prior learned during training. While we use different kinds of supervision during training and testing, which brings challenges for prior generalization. Specifically, during training, we use GT signed distances to learn what shapes look like as a prior, while using a loss to infer plausible signed distances from noisy point clouds fast based on the prior during testing. The signed distances inferred in iterative optimization may be slightly fluctuating iteration by iteration,  which brings significant uncertainty and makes it hard to optimize the code with fixed parameters like auto-decoder, while GT signed distances are constant. Thus, this challenge requires a novel way of generalizing a prior in an overfitting strategy.

G2. Our Contribution

Our main contributions are two folds. One is the noise to noise mapping in local regions, the other is how to leverage the prior as an initialization for inferring a SDF from a single noisy point cloud. Our first contribution leads to a loss that can infer more accurate and sharper geometry from a single noisy point cloud. The second contribution leads to a novel way of using the prior, which is the key to the fast convergence and good generalization of the prior. This way is not a simple variation of DeepSDF and noise2noise, please read “G3. Traditional test-time optimization vs ours” for more details.

G3. Traditional test-time optimization vs ours

The test-time optimization introduced in DeepSDF just optimizes a learnable code with fixed network parameters (a prior). It works well if the supervision used during testing is the same kind as the one used during training, such as GT signed distances. The difference to ours lies in that we do not access GT signed distances during testing, but just noisy point clouds, so we leverage a loss to infer signed distances. The inferred supervision is different from GT signed distances, since ground truth signed distances are constant values which keeps exactly the same in different iterations, while the signed distances inferred by pulling are fluctuating iteration by iteration, even at the same locations, which makes the learned prior hard to generalize according to the inferred supervision. We tried the following options at very beginning, but none of them can make the prior work with an overfitting based loss.

a)Randomly code + fixed network parameters ( DeepSDF)

b)Randomly code + learnable network parameters (prior initialization)

c)Randomly code with normalization + learnable network parameters (prior initialization)

Ours shown below work well with an overfitting based loss quite well.

d)averaged shape code + learnable network parameters (prior initialization) 

This initialization provides us a good object-like SDF that is robust enough to the information fluctuation inferred by the overfitting loss. Moreover, the prior can adjust the SDF fast.

G4. Why Local Patches Work Better

As stated in Line 122-132, we randomly sample a point patch from a noisy point cloud, and a set of queries within this patch, and pair these two sets with each other to learn a SDF. Different from the global method which samples both noisy points and queries all over a shape, we have a clear boundary each time, which makes the SDF inference more targeted and efficient. This is because we avoid noisy points and queries that are too far away from each other, which makes no sense to pair them together for inference. Our ablation studies in Tab. 11 justify that our local strategy can significantly improve the reconstruction accuracy and reduce the inference time.

G5. Statistical Reasoning

As stated in Lines 129-132, we aim to reason a neural SDF f from a noisy point cloud. Since we randomly sample queries and noisy points, we statistically minimize the expectation of the distance between different sets of pulled queries and noisy points. Statistically, the expectation of the zero-level set should have the minimum distance to all the noisy points. We regard this inferring process from a noisy point cloud as statistical reasoning. We will make this more clear in our revision.

G6. Gaussian Noise

Current studies have shown that the distribution of real noises from scanning sensors is very close to Gaussian distribution, please see Fig.7 in [1]. Vast amounts of literatures on point clouds also commonly use Gaussian distribution to add noises. Although our optimization minimizes the expectation, our method can work beyond Gaussian noises.  We also perform quite well on real scans with unknown noise types, such as our results on SRB in Tab.4 and Fig.4, D-FAUST in Tab.6 and Fig.6, 3D scene in Tab.7 and Fig.7, KITTI in Fig.8, and Paris-rue-Madame in Fig.9.

We additionally conduct an experiment to show our performance with various noise types, i.e., impulse noise, quantization noise, Laplacian noise, and Gaussian noise. Visual comparison in Fig. 3 in the rebuttal PDF justifies that we can also handle other types of noise quite well. Moreover, we also tried more challenging cases with nonuniform noises which do not have a zero expectation across a shape, like a shape with only a half point having noises or a shape with several patches having noises. The result in Fig. 7 in the rebuttal PDF shows that our method can also handle nonuniform noises well.

[1] Noise characterization of depth sensors for surface inspections

Meta-review: Initially, the reviewers had mixed responses, but after the rebuttal and the discussions, the majority of the reviewers voted in favour of acceptance. Except for Reviewer 76DM, who never returned after the initial review despite both the authors and the AC trying to reach the reviewer, all reviewers recommended accepting the paper. Reviewer AdmT's rating currently shows up as borderline reject, but in the discussions recommended acceptance, hence this seems to be just that the reviewer did not update the rating after the discussions.

In short, there were concerns about the limited novelty, but reviewers agreed that the claims made in the paper have been adequately verified, and the paper also shows a significant performance gain. The AC agrees with this recommendation and recommends accepting the paper. The authors are, however, strongly encouraged to incorporate the discussions during the rebuttal to the main paper, as it played a pivotal role in the paper's acceptance.
Decision: Accept (poster)