Title: Learning Unsigned Distance Fields from Local Shape Functions for 3D Surface Reconstruction
Venue: NeurIPS 2024
Accepted: no
Keywords: Surface Reconstruction

Implicit Fields

Unsigned Distance Fields
Abstract: Unsigned distance fields (UDFs) provide a versatile framework for representing a diverse array of 3D shapes, encompassing both watertight and non-watertight geometries. Traditional UDF learning methods typically require extensive training on large datasets of 3D shapes, which is costly and often necessitates hyperparameter adjustments for new datasets. This paper presents a novel neural framework, LoSF-UDF, for reconstructing surfaces from 3D point clouds by leveraging local shape functions to learn UDFs. We observe that 3D shapes manifest simple patterns within localized areas, prompting us to create a training dataset of point cloud patches characterized by mathematical functions that represent a continuum from smooth surfaces to sharp edges and corners. Our approach learns features within a specific radius around each query point and utilizes an attention mechanism to focus on the crucial features for UDF estimation. This method enables efficient and robust surface reconstruction from point clouds without the need for shape-specific training. Additionally, our method exhibits enhanced resilience to noise and outliers in point clouds compared to existing methods. We present comprehensive experiments and comparisons across various datasets, including synthetic and real-scanned point clouds, to validate our method's efficacy.

Abstract

Unsigned distance fields (UDFs) provide a versatile framework for representing a diverse array of 3D shapes, encompassing both watertight and non-watertight geometries. Traditional UDF learning methods typically require extensive training on large datasets of 3D shapes, which is costly and often necessitates hyperparameter adjustments for new datasets. This paper presents a novel neural framework, LoSF-UDF , for reconstructing surfaces from 3D point clouds by leveraging local shape functions to learn UDFs. We observe that 3D shapes manifest simple patterns within localized areas, prompting us to create a training dataset of point cloud patches characterized by mathematical functions that represent a continuum from smooth surfaces to sharp edges and corners. Our approach learns features within a specific radius around each query point and utilizes an attention mechanism to focus on the crucial features for UDF estimation. This method enables efficient and robust surface reconstruction from point clouds without the need for shape-specific training. Additionally, our method exhibits enhanced resilience to noise and outliers in point clouds compared to existing methods. We present comprehensive experiments and comparisons across various datasets, including synthetic and real-scanned point clouds, to validate our method's efficacy.

1 Introduction

3D surface reconstruction from raw point clouds is a significant and long-standing problem in computer graphics and machine vision. Traditional techniques like Poisson Surface Reconstruction[1] create an implicit indicator function from oriented points and reconstruct the surface by extracting an appropriate isosurface. The advancement of artificial intelligence has led to the emergence of numerous neural network-based methods for 3D reconstruction. Among these, neural implicit representations have gained significant influence, which utilize signed distance fields (SDFs) [2-8] and occupancy fields [9-12] to implicitly depict 3D geometries. SDFs and occupancy fields extract isosurfaces by solving regression and classification problems, respectively. However, both techniques require internal and external definitions of the surfaces, limiting their capability to reconstructing only watertight geometries. Therefore, unsigned distance fields [13-20] have recently gained increasing attention due to their ability to reconstruct non-watertight surfaces and complex geometries with arbitrary topologies.

Reconstructing 3D geometries from raw point clouds using UDFs presents significant challenges due 31 to the non-differentiability near the surface. This characteristic complicates the development of loss 32 functions and undermines the stability of neural network training. Various unsupervised approaches 33 [17, 14, 19] have been developed to tailor loss functions that leverage the intrinsic characteristics 34 of UDFs, ensuring that the reconstructed geometry aligns closely with the original point clouds. 35 However, these methods suffer from slow convergence, necessitating an extensive network training 36 time to reconstruct a single geometry. As a supervised method, GeoUDF [15] learns local geometric 37

priors through training on datasets such as ShapeNet [21], thus achieving efficient UDF estimation. 38

Nonetheless, the generalizability of this approach is dependent on the training dataset, which also 39 leads to relatively high computational costs. 40

In this paper, we propose a lightweight and effective supervised learning framework, Losf-UDF , to 41 address these challenges. Since learning UDFs does not require determining whether a query point is 42 inside or outside the geometry, it is a local quantity independent of the global context. Inspired by the 43 observation that 3D shapes manifest simple patterns within localized areas, we synthesize a training 44 dataset comprising a set of point cloud patches by utilizing local shape functions. Subsequently, we 45 can estimate the unsigned distance values by learning local geometric features through an attention46 based network. Our approach distinguishes itself from existing methods by its novel training strategy. 47 Specifically, it is uniquely trained on synthetic surfaces, yet it demonstrates remarkable capability 48 in predicting UDFs for a wide range of common surface types. For smooth surfaces, we generate 49 training patches (quadratic surfaces) by analyzing principal curvatures, meanwhile, we design simple 50 shape functions to simulate sharp features. This strategy has three unique advantages. First, it 51 systematically captures the local geometries of most common surfaces encountered during testing, 52 effectively mitigating the dataset dependence risk that plagues current UDF learning methods. Second, 53 for each training patch, the ground-truth UDF is readily available, streamlining the training process. 54 Third, this approach substantially reduces the costs associated with preparing the training datasets. 55 We evaluate our framework on various datasets and demonstrates its ability to robustly reconstruct 56 high-quality surfaces, even for point clouds with noise and outliers. Notably, our method can serve as 57 a lightweight initialization that can be integrated with existing unsupervised methods to enhance their 58 performance. We summarize our main contributions as follows. 59

60

61

We present a simple yet effective data-driven approach that learns UDFs directly from a synthetic dataset consisting of point cloud patches, which is independent of the global shape.

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

Our method is computationally efficient and requires training only once on our synthetic dataset. Then it can be applied to reconstruct a wide range of surface types.

Our framework achieves superior performance in surface reconstruction from both synthetic point clouds and real scans, even in the presence of noise and outliers.

2 Related Work

Surface reconstruction. Reconstructing 3D surfaces from point clouds is a classic and important topic in computer graphics. The most widely used Poisson method [1, 22] fits surfaces by solving PDEs. These traditional methods involve adjusting the gradient of an indicator function to align with a solution derived from a (screened) Poisson equation. A crucial requirement of these methods is the input of oriented normals. The Iterative Screened Poisson Reconstruction method[23] introduced an iterative approach to refine the reconstruction process, improving the ability to generate surfaces from point clouds without direct computation of normals. The shape of points [24] introduced a differentiable point-to-mesh layer by employing a differentiable formulation of PSR [1] to generate watertight, topology-agnostic manifold surfaces.

Neural surface representations. Recently, the domain of deep learning has spurred significant advances in the implicit neural representation of 3D shapes. Some of these works trained a classifier neural network to construct occupancy fields [9-12] for representing 3D geometries. Poco [12] achieves superior reconstruction performance by introducing convolution into occupancy fields. Ouasfi et al. [25] recently proposed a uncertainty measure method based on margin to learn occupancy fields from sparse point clouds. Compared to occupancy fields, SDFs [2-8] offer a more precise geometric representation by differentiating between interior and exterior spaces through the assignment of signs to distances. Some recent SOTA methods, such as DeepLS [3], using volumetric SDFs to locally learned continuous SDFs, have achieved higher compression, accuracy, and local shape refinement.

Unsigned distance fields learning. Although Occupancy fields and SDFs have undergone signif86 icant development recently, they are hard to reconstruct surfaces with boundaries or nonmanifold 87 features. G-Shell[26] developed a differentiable shell-based representation for both watertight and 88 non-watertight surfaces. However, UDFs provide a simpler and more natural way to represent 89 general shapes [13-20]. Various methods have been proposed to reconstruct surfaces from point 90 clouds by learning UDFs. CAP-UDF [17] suggested directing 3D query points towards the surface 91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

…

Synthetic Dataset

UDF

LoSF-UDF

LoSF-UDF

DCUDF

Supervised

Training

Mesh Extraction

Training

Reconstruction

n

P

with a consistency constraint to develop UDFs that are aware of consistency. LevelSetUDF [14] learned a smooth zero-level function within UDFs through level set projections. As a supervised approach, GeoUDF [15] estimates UDFs by learning local geometric priors from training on many 3D shapes. DUDF [19] formulated the UDF learning as an Eikonal problem with distinct boundary conditions. UODF [20] proposed unsigned orthogonal distance fields that every point in this field can access to the closest surface points along three orthogonal directions. Instead of reconstructing from point clouds, many recent works [27-30] learn high-quality UDFs from multi-view images for reconstructing non-watertight surfaces. Furthermore, UiDFF [31] presents a 3D diffusion model for UDFs to generate textured 3D shapes with boundaries.

3 Method

Motivation. Distinct from SDFs, there is no need for UDFs to determine the sign to distinguish between the inside and outside of a shape. Consequently, the UDF values are solely related to the local geometric characteristics of 3D shapes. Furthermore, within a certain radius for a query point, local geometry can be approximated by general mathematical functions. Stemming from these insights, we propose a novel UDF learning framework that focuses on local geometries. We employ local shape functions to construct a series of point cloud patches as our training dataset, which includes common smooth and sharp geometric features. Fig. 1 illustrates the pipeline of our proposed UDF learning framework.

3.1 Local shape functions

Smooth patches. From the viewpoint of differential geometry [33], the local geometry at a specific point on a regular surface can be approximated by a quadratic surface. Specifically, consider a regular surface S : r = r ( u, v ) with a point p on it. At point p , it is possible to identify two principal direction unit vectors, e 1 and e 2 , with the corresponding normal n = e 1 × e 2 . A suitable parameter system ( u, v ) can be determined such that r u = e 1 and r v = e 2 , thus obtaining the corresponding first and second fundamental forms as

where κ 1 , κ 2 are principal curvatures. Without loss of generality, we assume p corresponding to u = v = 0 and expand the Taylor form at this point as

Decomposing r uu (0 , 0) , r uv (0 , 0) , and r vv (0 , 0) along the tangential and normal directions, we can 119 formulate Eq.(2) according to Eq.(1) as 120

where o ( u 2 + v 2 ) ≈ 0 is negligible in a small local region. Consequently, by adopting { p , e 1 , e 2 , n } 121 as the orthogonal coordinate system, we can define the form of the local approximating surface as 122

123

124

125

126

Planar

Parabolic

Hyperbolic

V-Saddle

Crease

Corner

(a) Smooth patches

(b) Sharp patches

y

= 0

which exactly are quadratic surfaces z = 1 2 ( κ 1 x 2 + κ 2 y 2 ) . Furthermore, in relation to Gaussian curvatures κ 1 κ 2 , quadratic surfaces can be categorized into four types: ellipsoidal, hyperbolic, parabolic, and planar. As shown in Fig. 2, for differentiable points on a general geometry, the local shape features can always be described by one of these four types of quadratic surfaces.

Sharp patches. For surfaces with sharp features, they are not differentiable at some points and cannot 127 be approximated in the form of a quadratic surface. We categorize commonly seen sharp geometric 128 features into four types, including creases, cusps, corners, and v-saddles, as illustrated in Fig. 2(b). 129 We construct these four types of sharp features in a consistent form z = f ( x, y ) like smooth patches 130

131

132

133

134

135

136

137

138

139

140

141

142

143

where h can adjust the sharpness of the shape, and k can control the direction of the crease. Fig 3 illustrates various smooth and sharp patches with distinct parameters.

Synthetic training dataset. We utilize the mathematical functions introduced above to synthesize a series of point cloud patches for training. As shown in Fig. 3, we first uniformly sample m points { ( x i , y i ) } i m =1 within a circle of radius r 0 centered at (0 , 0) in the xy -plane. Then, we substitute the coordinates into Eq.(4-5) to obtain the corresponding z -coordinate values, resulting in a patch P = { p i m =1 } , where p i = ( x i , y i , z ( x i , y i )) . Subsequently, we randomly collect query points { q i } n i =1 distributed along the vertical ray intersecting the xy -plane at the origin, extending up to a distance of r 0 . For each query point q i , we determine its UDF value U ( q i ) , which is either | q ( z ) i | for smooth patches or 1 -| q ( z ) i | for sharp patches. Noting that for patches with excessively high curvature or sharpness, the minimum distance of the query points may not be the distance to (0 , 0 , z (0 , 0)) , we will exclude these patches from our training dataset. Overall, each sample in our synthetic dataset is specifically in the form of { q , P , U ( q ) } .

Query points

Point cloud patch

1 = 1, K2 =1

K1=1,K2=1

K1 =1, K2 = 0

d=0.5

d=0.5

d=0.5

d=0.5

K =1, K2=5

K=-5,K2=5

1 =5,K2=0

d=1

d=1

d=1

d=1

Q:

K =10,K2=10

K=-10,K2=10

K1=10,K2=0

d=2

d=2

d=2

d =1.5

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

3.2 UDF learning

We perform supervised training on the synthesized dataset which is independent of specific shapes. The network learns the features of local geometries and utilizes an attention-based module to output the corresponding UDF values from the learned features. After training, given any 3D point clouds and a query point in space, we extract the local point cloud patch near the query, which has the same form as the data in the training dataset. Consequently, our network can predict the UDF value at that query point based on this local point cloud patch.

3.2.1 Network architecture

For a sample { q , P = { p i } i m =1 , U ( q ) } , we first obtain a latent code f p ∈ R l p related to the local point cloud patch P through a Point-Net [34] F p . To derive features related to distance, we use relative vectors from the patch points to the query point, V = { p i -q } i m =1 , as input to a VectorsNet F v , which is similar to the Point-Net F p . This process results in an additional latent code f v ∈ R l v . Subsequently, we apply a cross-attention module [35] to obtain the feature codes for the local geometry,

where we take f p as the Key-Value (KV) pair and f v as the Query (Q). In our experiments, we set l p = l v = 64 , and l G = 128 . Based on the learned geometric features, we aim to fit the UDF values from the distance within the local point cloud. Therefore, we concatenate the distances d ∈ R m induced from V with the latent code f G , followed by a series of fully connected layers to output the predicted UDF values U Θ ( q ) . Fig. 4 illustrates the overall network architecture and data flow.

Point Cloud Patch

K

, V

Q

Vectors Pointing to Query

Points-Net

Vectors-Net

Cross-Attn

Full-Connected

Layers

Denoising

Module

Points-Net

Self-Attn

Full-Connected

Layers

Denoising Module

Denoising module. In our network, even if point cloud patches are subjected to a certain degree of noise or outliers, their representations in the feature space should remain similar. However, distances induced directly from noisy vectors V will inevitably contain errors, which can affect the accurate prediction of UDF values. To mitigate this impact, we introduce a denoising module that predicts displacements ∆ d from local point cloud patches, as shown in Fig. 4. We then add the displacements ∆ d to the distances d to improve the accuracy of the UDF estimation.

3.2.2 Training and evaluation

Data augmentation. During the training process, we scale all pairs of local patches P and query points q to conform to the bounding box constraints of [ -0 . 5 , 0 . 5] , and the corresponding GT UDF values U ( q ) are scaled by equivalent magnitudes. Given the uncertain orientation of local patches extracted from a specified global point cloud, we have applied data augmentation via random rotations to the training dataset. Furthermore, to enhance generalization to open surfaces with boundaries, we randomly truncate 20% of the smooth patches to simulate boundary cases. To address the issue of noise handling, we introduce Gaussian noise N (0 , 0 . 1) to 30% of the data in each batch during every training epoch.

Loss functions. We employ L 1 loss L u to measure the discrepancy between the predicted UDF 178 values and the GT UDF values. Moreover, for the displacements ∆ d output by the denoising module, 179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

we employ L 1 regularization to encourage sparsity. Consequently, we train the network driven by the following loss function,

where we set λ d = 0 . 01 in our experiments.

Evaluation. Given a 3D point cloud P for reconstruction, we first normalize it to fit within a bounding box with dimensions ranging from [ -0 . 5 , 0 . 5] . Subsequently, within the bounding box space, we uniformly sample grid points at a specified resolution to serve as query points. Finally, we extract the local geometry P p for each query point by collecting points from the point cloud that lie within a sphere of a specified radius centered on the query point. We can obtain the predicted UDF values by the trained network U Θ ∗ ( q , P q ) , where Θ ∗ represents the optimized network parameters. Note that for patches P p with fewer than 5 points, we set the UDF values as a large constant. Finally, we extract meshes from the UDFs using the DCUDF model [32].

4 Experiments

4.1 Experiment setup

Datasets. To compare our method with other state-of-the-art UDF learning approaches, we tested it on various datasets that include general artificial objects from the field of computer graphic. Following previous works [30, 17, 14], we select the "Car" category from ShapeNet[21], which has a rich collection of multi-layered and non-closed shapes. Furthermore, we select the real-world dataset DeepFashion3D[36] for open surfaces, and ScanNet[37] for large outdoor scenes. To assess our model's performance on actual noisy inputs, we conducted tests on real range scan dataset [38] following the previous works[17, 14].

Baselines. For our validation datasets, we compared our method against the state-of-the-art UDF learning models, which include unsupervised methods like CAP-UDF[17], LevelSetUDF[14], and DUDF[19], as well as the supervised learning method, GeoUDF[15]. We trained GeoUDF independently on different datasets to achieve optimal performance. Table. 1 shows the qualitative comparison between our methods and baselines. To evaluate performance, we calculate the Chamfer Distance (CD) and F1-Score (setting thresholds of 0.005 and 0.01) metrics between the ground truth meshes and the meshes extracted from the UDFs out by our model and each baseline model. For a fair comparison, we test all baseline models using the DCUDF[32] method. All experimental procedures are executed on NVIDIA RTX 4090 and A100 GPUs.

Table 1: Qualitative comparison of different UDF learning methods. 'Normal' indicates whether the method requires point cloud normals during learning. 'Feature Type'' refers to whether the information required during training is global or local. 'Noise' and 'Outlier' indicate whether the method can handle the presence of noise and outliers in point clouds.

4.2 Experimental results

Synthetic data. For general 3D graphic models, ShapeNetCars, and DeepFashion3D, we obtain dense point clouds by randomly samping on meshes.

Considering that GeoUDF [15] is a supervised method, we retrain it on ShapeNetCars, and DeepFashion3D, which are randomly partitioned into training (70%), testing (20%), and validation subsets (10%). All models are evaluated in the validation sets, which remain unseen by any of the UDF learning models prior to evaluation. The first three rows of Fig. 5 show the visual comparison of recon-

struction results, while Tab. 2 presents the quantitative comparison results of CD and F1-score. We

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

test each method using their own mesh extraction technique, as shown in the inset figure, which display obvious visual artifacts such as small holes and non-smoothness. We thus apply DCUDF [32] , the state-of-art method, to each baseline model , extracting the surfaces as significantly higher quality meshes. Since our method utilizes DCUDF for surface extraction, we adopt it as the default technique to ensure consistency and fairness in comparisons with the baselines. Our method achieves stable results in reconstructing various types of surfaces, including both open and closed surfaces, and exhibits performance comparable to that of the SOTA methods. Noting that DUDF[19] requires normals during training, and GeoUDF utilizes the KNN approach to determine the nearest neighbors of the query points. Although DUDF and GeoUDF achieve better evaluations, they are less stable when dealing with point clouds with noise and outliers.

Clean

Noise

CD

↓

F1

↑

CD

↓

F1

↑

CD

↓

F1

↑

method

F

1

0

.

005

F

1

0

.

01

F

1

0

.

005

F

1

0

.

01

F

1

0

.

005

F

1

0

.

01

ShapeNetCars [21]

CAP-UDF [17]

2.432

0.523

0.888

2.602

0.194

0.381

4.982

0.183

0.314

LevelSetUDF [14]

1.534

0.561

0.908

2.490

0.209

0.401

4.177

0.199

0.363

1.257

0.571

0.889

1.232

0.351

0.873

4.870

0.187

0.346

DUDF [19]

0.568

0.903

0.991

3.180

0.312

0.527

4.235

0.168

0.308

Ours

1.085

0.510

0.938

1.114

0.427

0.922

1.272

0.485

0.771

DeepFashion3D [36]

CAP-UDF [17]

1.660

0.417

0.818

1.892

0.336

0.542

4.941

0.172

0.430

LevelSetUDF [14]

1.500

0.403

0.856

1.488

0.453

0.729

4.328

0.203

0.468

0.652

0.864

0.977

1.258

0.380

0.957

4.463

0.147

0.300

DUDF [19]

0.381

0.991

0.998

1.894

0.334

0.535

4.970

0.144

0.272

Ours

0.932

0.652

0.983

1.150

0.361

0.976

1.029

0.549

0.973

Noise & outliers. To evaluate our model with noisy inputs, we added Gaussian noise N (0 , 0 . 0025) to the clean data across all datasets for testing. The middle three rows in Fig. 5 display the reconstructed surface results from noisy point clouds, and Tab. 2 also presents the quantitative comparisons. It can be observed that our method can robustly reconstruct smooth surfaces from noisy point clouds. Additionally, we tested our method's performance with outliers by converting 10% of the clean point cloud into outliers, as shown in the last three rows of Fig. 5. To further demonstrate the robustness of our method, we conducted experiments on point clouds with higher percentage of outliers. Our framework is able of reconstructing reasonable surfaces even with 50% outliers. We also tested the task on point clouds containing both noise and outliers. Please refer to Fig. 9 in the Appendix for the corresponding results.

Real-world scanned data. Dataset [38] provide several real-world scanned point clouds, as illustrated in Fig. 6 (Left), we evaluate our model on the dataset to demonstrate the effectiveness. Our approach can reconstruct smooth surfaces from scanned data containing noise and outliers. However, our model cannot address the issue of missing parts. This limitation is due to the local geometric training strategy, which is independent of the global shape. Additionally, we conduct tests on large scanned scenes to evaluate our algorithm, as shown in Fig. 6 (Right).

4.3 Analysis & ablation studies

Efficiency . As a supervised UDF learning significant improvement in training efficiency As shown in the insert table, we calculate the data storage space required by GeoUDF when using ShapeNet as a training dataset. This includes the GT UDF values and point cloud data needed during the training process. Our

synthetic point cloud patches training dataset occupies under 1GB, which is merely 0.5% of the storage needed for GeoUDF. Our network is very lightweight, with only 653KB of trainable parameters and a total parameter size of just 2MB. Additionally, we highlight time-saving benefits. The provided table illustrates the duration required to produce a single data sample for dataset preparation ('Data-prep'), as well as the total time for training ('Training').

Patch radius. During the evaluation phase, the radius r used to find the nearest points for each query point determines the size of the extracted patch and the range of effective query points in the space. As shown in Fig. 7, we analyzed the impact of different radii on the reconstruction results. An excessively small r will generate artifacts, while an overly large r will lose many details. In our experiments, we generally set r to 0.018.

(a) Input

(b) CAP-UDF

(c) LevelSetUDF

(f) Ours

(g) GT

好好好好好好

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

Missing

GT

r

= 0

.

08

r

= 0

.

10

r

= 0

.

20

r

= 0

.

30

Denoising module. Our framework incorporates a denoising module to handle noisy point clouds. 263 We conducted ablation experiments to verify the significance of this module. Specifically, we set 264 λ d = 0 in the loss function Eq. (7) to disable the denoising module, and then retrained the network. 265 As illustrated in Fig. 8, we present the reconstructed surfaces for the same set of noisy point clouds 266 with and without the denosing module, respectively. 267

Noise

W/O

W/O

W/

W/

Outliers

CD = 0.927,

CD = 0.896,

CD = 0.886,

CD = 0.796,

F10.005  0.656,

F10.005  0.630,

F10.005  0.697,

F10.00  0.767,

F10.01  0.929

F1001 = 0.997

F10.01  0.930

F10.01  0.982

5 Conclusion

In this paper, we introduce a novel and efficient neural framework for surface reconstruction from 3D point clouds by learning UDFs from local shape functions. Our key insight is that 3D shapes exhibit simple patterns within localized regions, which can be exploited to create a training dataset of point cloud patches represented by mathematical functions. As a result, our method enables efficient and robust surfaces reconstructions without the need for shape-specific training. Extensive experiments on various datasets have demonstrated the efficacy of our method. Moreover, our framework achieves superior performance on point clouds with noise and outliers.

Limitations & future work. Owing to its dependence solely on local geometric features, our approach fails to address tasks involving incomplete point cloud reconstructions. However, as a lightweight framework, our model can readily be integrated into other unsupervised methods to combine the global features with our learned local priors. Furthermore, in our future work, we intend to design a method that dynamically adjusts the radius based on local feature sizes [39] of 3D shapes when extracting local point cloud patches for queries, aiming to improve the accuracy of the reconstruction.

283

284

285

286

287

288

References

MKazhdan. Poisson surface reconstruction. In Eurographics Symposium on Geometry Processing , 2006.

Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019.

Rohan Chabra, Jan Eric Lenssen, Eddy Ilg, Tanner Schmidt, Julian Straub, Steven Lovegrove, 289 and Richard Newcombe. Deep local shapes: Learning local sdf priors for detailed 3d recon290 struction, 2020. 291

292

293

294

Ma Baorui, Han Zhizhong, Liu Yu-Shen, and Zwicker Matthias. Neural-pull: Learning signed distance functions from point clouds by learning to pull space onto surfaces. In International Conference on Machine Learning (ICML) , 2021.

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

Peng-Shuai Wang, Yang Liu, and Xin Tong. Dual octree graph networks for learning adaptive volumetric shape representations. ACM Transactions on Graphics , 41(4):1-15, July 2022.

Hao Pan Pengshuai Wang Xin Tong Yang Liu Shi-Lin Liu, Hao-Xiang Guo. Deep implicit moving least-squares functions for 3d reconstruction. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021.

Zixiong Wang, Pengfei Wang, Pengshuai Wang, Qiujie Dong, Junjie Gao, Shuangmin Chen, Shiqing Xin, Changhe Tu, and Wenping Wang. Neural-imls: Self-supervised implicit moving least-squares network for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics , pages 1-16, 2023.

Ma Baorui, Liu Yu-Shen, and Han Zhizhong. Reconstructing surfaces for sparse point clouds with on-surface priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022.

Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) , 2019.

Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll. Implicit functions in feature space for 3d shape reconstruction and completion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, jun 2020.

Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. Convolutional occupancy networks. In European Conference on Computer Vision (ECCV) , 2020.

Alexandre Boulch and Renaud Marlet. Poco: Point convolution for surface reconstruction, 2022.

Julian Chibane, Aymen Mir, and Gerard Pons-Moll. Neural unsigned distance fields for implicit function learning. In Advances in Neural Information Processing Systems (NeurIPS) , December 2020.

Junsheng Zhou, Baorui Ma, Shujuan Li, Yu-Shen Liu, and Zhizhong Han. Learning a more continuous zero level set in unsigned distance fields through level set projection. In Proceedings of the IEEE/CVF international conference on computer vision , 2023.

Siyu Ren, Junhui Hou, Xiaodong Chen, Ying He, and Wenping Wang. Geoudf: Surface reconstruction from 3d point clouds via geometry-guided distance representation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 14214-14224, 2023.

Jianglong Ye, Yuntao Chen, Naiyan Wang, and Xiaolong Wang. Gifs: Neural implicit function for general shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022.

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

Junsheng Zhou, Baorui Ma, Yu-Shen Liu, Yi Fang, and Zhizhong Han. Learning consistencyaware unsigned distance functions progressively from raw point clouds. In Advances in Neural Information Processing Systems (NeurIPS) , 2022.

Qing Li, Huifang Feng, Kanle Shi, Yi Fang, Yu-Shen Liu, and Zhizhong Han. Neural gradient learning and optimization for oriented point normal estimation. In SIGGRAPH Asia 2023 Conference Papers , 2023.

Miguel Fainstein, Viviana Siless, and Emmanuel Iarussi. Dudf: Differentiable unsigned distance fields with hyperbolic scaling, 2024.

Yujie Lu, Long Wan, Nayu Ding, Yulong Wang, Shuhan Shen, Shen Cai, and Lin Gao. Unsigned orthogonal distance fields: An accurate neural implicit representation for diverse 3d shapes. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2024.

Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR], Stanford University - Princeton University - Toyota Technological Institute at Chicago, 2015.

Michael Kazhdan and Hugues Hoppe. Screened poisson surface reconstruction. Acm Transactions on Graphics , 32(3):1-13, 2013.

Fei Hou, Chiyu Wang, Wencheng Wang, Hong Qin, Chen Qian, and Ying He. Iterative poisson surface reconstruction (ipsr) for unoriented points. ACM Transactions on Graphics , 41(4):1-13, July 2022.

Songyou Peng, Chiyu "Max" Jiang, Yiyi Liao, Michael Niemeyer, Marc Pollefeys, and Andreas Geiger. Shape as points: A differentiable poisson solver. In Advances in Neural Information Processing Systems (NeurIPS) , 2021.

Amine Ouasfi and Adnane Boukhayma. Unsupervised occupancy learning from sparse point cloud, 2024.

Zhen Liu, Yao Feng, Yuliang Xiu, Weiyang Liu, Liam Paull, Michael J. Black, and Bernhard Schölkopf. Ghost on the shell: An expressive representation of general 3d shapes. 2024.

Junkai Deng, Fei Hou, Xuhui Chen, Wencheng Wang, and Ying He. 2s-udf: A novel two-stage udf learning method for robust non-watertight model reconstruction from multi-view images, 2024.

Xiaoxu Meng, Weikai Chen, and Bo Yang. Neat: Learning neural implicit surfaces with arbitrary topologies from multi-view images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , June 2023.

Xiaoxiao Long, Cheng Lin, Lingjie Liu, Yuan Liu, Peng Wang, Christian Theobalt, Taku Komura, and Wenping Wang. Neuraludf: Learning unsigned distance fields for multi-view reconstruction of surfaces with arbitrary topologies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20834-20843, 2023.

Yu-Tao Liu, Li Wang, Jie Yang, Weikai Chen, Xiaoxu Meng, Bo Yang, and Lin Gao. Neudf: Leaning neural unsigned distance fields with volume rendering. In Computer Vision and Pattern Recognition (CVPR) , 2023.

Junsheng Zhou, Weiqi Zhang, Baorui Ma, Kanle Shi, Yu-Shen Liu, and Zhizhong Han. Udiff: Generating conditional unsigned distance fields with optimal wavelet diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024.

Fei Hou, Xuhui Chen, Wencheng Wang, Hong Qin, and Ying He. Robust zero level-set extraction from unsigned distance fields based on double covering. ACM Trans. Graph. , 42(6), dec 2023.

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

Manfredo P Do Carmo. Differential geometry of curves and surfaces: revised and updated second edition . Courier Dover Publications, 2016.

Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems , 30, 2017.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023.

Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016.

Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE , 2017.

Matthew Berger, Joshua A. Levine, Luis Gustavo Nonato, Gabriel Taubin, and Claudio T. Silva. A benchmark for surface reconstruction. ACM Trans. Graph. , 32(2), apr 2013.

Yulan Guo, Mohammed Bennamoun, Ferdous Sohel, Min Lu, Jianwei Wan, and Ngai Ming Kwok. A comprehensive performance evaluation of 3d local feature descriptors. International Journal of Computer Vision , 116:66-89, 2016.

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our abstract and introduction accurately describe our technical contributions to Unsigned Distance Fields learning.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations in the conclusion section (Sec.5).

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the differential geometry theory employed by our method in the main text (Sec.3).

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide the most detailed algorithmic details possible in the main text and appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: We will definitely make our code publicly available one day, but not at this moment.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We introduce all the training and test details in the main text and appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We provide various evaluation metrics about our method.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide the related information in the experimental section.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We strictly adhere to the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Our method may be applied to 3D reconstruction in daily life, demonstrating significant social value.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The original owners of all code, data, and models in our paper are properly credited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: There is no new assets attached to our paper. We will make our code and data public once paper is accepted.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

703 704 705 706 707

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

A Appendix 708

709

710

711

712

713

714

715

716

717

718

A.1 Network details

The two PointNets used in our network to extract features from point cloud patches P and vectors V consist of four ResNet blocks. In addition, the two fully connected layer modules in our framework consist of three layers each. To ensure non-negativity of the UDF values output by the network, we employ the softplus activation function.

A.2 Robustness to outliers

Our method can reconstruct relatively accurate geometry from point clouds with 10% added outliers and reasonably smooth surfaces from point clouds with even higher outlier ratios. Furthermore, our approach can reconstruct high-quality geometry from point clouds containing both noise and outliers, as shown in Fig. 9.

GT

Outliers=10%

Outliers =25%

Outliers=50%

Outliers =10%+Noise =10%

A.3 More results 719

As shown in Fig. 10 and Fig. 11, we provide more visual comparisons on the DeepFashion3D and 720 ShapeNetCars dataset, using point clouds containing noise and outliers. 721

(a) Input

(b) CAP-UDF

(c) LevelSetUDF

(f) Ours

(g) GT

(a) Input

(b) CAP-UDF

(c) LevelSetUDF

(f) Ours

(g) GT

小小小

#小小

小小

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new method to learning unsigned distance functions. Specifically, the method leverages local shape priors which brings geometry priors and is also able to handle noises and outliers. The results demonstrate that the proposed method outperforms previous baselines, especially in the corrupted situations.\n\nSTRENGTHS:\n1. The idea of learning local shape functions is widely explored in the SDF or occupancy based methods, but is not yet introduced to the field of UDF-based reconstruction. This paper combines the strengths of local functions and UDF which can represent shapes with arbitrary typologies.\n2. The paper is overall easy to follow.\n3. The comparison are comprehensive by introducing recent works as the baselines. \n4. The method can better handle the noises and outliers compared to the previous prior-free methods.\n\nWEAKNESSES:\n1. The idea is straightforward to learn local shape functions as the prior for global shape reconstruction. Similar ideas are proposed in SDF-based methods. I would like to see more insight in the differences to the previous methods and also the importance of introducing the local geometric priors for UDF learning. \nLocal Implicit Grid Representations for 3D Scenes (CVPR 2020)\nSurface Reconstruction from Point Clouds by Learning Predictive Context Priors (CVPR 2022)\nDeep local shapes: Learning local sdf priors for detailed 3d reconstruction (ECCV 2020)\n2. The visualization is not quite convincing in the geometry details. For example, the reconstructions of real-scans in Fig.6 seems worse than the results of other methods in their papers. The local shape functions is expected to produce reconstruction with more details and sharper edges, but the scene reconstructions lack details. \n3. Also, some reconstructions are too fat, which in my opinion, is caused by the inaccurate UDF near the zero-level set, since the method use DCUDF for UDF meshing.\n\nQUESTIONS:\nMore references on the local shape functioning and UDF learning will be helpful.\n\nLocal Implicit Grid Representations for 3D Scenes (CVPR 2020)\nCAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization (TPAMI 2024)\n\nCan the proposed method handle large scale scenes like KITTI?\n\nLIMITATIONS:\nPlease refer to the strengths and weaknesses above.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an approach to reconstruct 3D surfaces from point clouds, using unsigned distance fields. The proposed approach consists in training a specific neural network architecture to predict UDF values from local point cloud patches, which can be triangulated using UDF meshing methods. The paper mathematically analyses the possible local patches that can appear, smooth and sharp, and trains the proposed architecture with them. Once trained, the neural network is queried for UDF values from a point cloud, and it computes them by extracting a local point cloud patch for each query point and applying the information learned during training. A denoising module is employed to reduce the impact of noise and outliers. Experiments are carried out on ShapeNet cars and DeepFashion3D, synthetically sampling point clouds from the dataset meshes and reconstructing surfaces using a number of baselines, an extracting meshes using DCUDF. Results show that the performance of the proposed method is not the strongest on clean data, but it is so in the presence of noise and/or outliers. Experiments on real-world point clouds are shown qualitatively.\n\nSTRENGTHS:\nThe analysis of the possible kinds of local surface patches is well thought out and significative, enabling the possibility to treat the problem of surface reconstruction locally instead of globally, with an extensive training set that covers virtually all possible cases. I believe this is a good contribution, and the main strength in the paper.\nThe performance of the method on noisy data, albeit the noise was introduced synthetically, is good; the performance with synthetic outliers looks impressive.\nThe writing quality is generally good, with some exceptions on clarity (see weaknesses).\n\nWEAKNESSES:\nI believe there are a few weaknesses in the paper, mostly regarding clarity, architecture validation and experiments. I list them in no particular order.\n1) The paper proposes a complex architecture, consisting of two branches, a cross-attention module, fully connected layers and a denoising module. The latter is only qualitatively justified in Fig.8, but the rest of the architecture is not validated in any way. A (quantitative) experimental evaluation of why this architecture is suitable for the task would be needed, in my opinion, as well as the intuition that led the researchers to arrive to this final architecture.\n2) The cross-attention mechanism is unclear to me: the two branches (points and vectors net) output a latent code. What is the meaning of a cross-attention module on a single input token (actually one as K-V, one as Q)?\n3) In the experiments, a few details are not specified, for example the query grid resolution and the number of points used for the Chamfer distance.\n4) A time evaluation is completely missing. The paper claims \"Our method is computationally efficient\" in the introduction, which is partially justified by the comparatively better training times and storage requirements with respect to GeoUDF, but there is no evaluation of the time required by the method to reconstruct a surface, compared to the other methods. Notice also that DCUDF is an extremely slow meshing algorithm, so the evaluation should be performed both with it and without it.\n5) The paper claims \"superior performance in surface reconstruction from both synthetic point clouds and real scans, even in the presence of noise and outliers\". The experiments however show no superiority in the absence of noise and outliers. Moreover, there are no quantitative experiments on real scans, making it impossible to evaluate whether the noise robustness on synthetic noise also translates to real data. Additionally, the qualitative experiments shown on real data lack comparison with the baselines. Thus, in general, I find the experimental section incomplete and not fully convincing.\n\nQUESTIONS:\nRelating to some of the weaknesses, I would ask the authors:\n1) Could you clarify the role and mechanism of the cross-attention module in the architecture?\n2) Could you specify the missing details? (grid resolution, number of points for Chamfer distance)\n3) If you already have them in the logs or if time allows, could you provide the timings for your methods and the baselines?\n4) Why are surfaces from real data not evaluated quantitatively?\n5) In the conclusions, addressing the limitation of the proposed method on incomplete point clouds, you mention the possibility to readily integrate your framework with other methods, but it is unclear to me how this could be possible. Could you provide a brief explanation?\n\nLIMITATIONS:\nA limitation of the method on completing incomplete surfaces has been correctly addressed and disclosed. \nThe limited performance on clean data is shown in the experiments, but not acknowledged in the claims in the introduction.\nA quantitative assessment of the performance on real scans and of the time required to reconstruct the surface are missing, making it harder to fully evaluate the possible limitations of the method with respect to the claims.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2024-07-09"}

comments: SUMMARY:
The paper proposes a novel training strategy to learn Unsigned Distance Fields from local shapes. The idea is to train the model on a dataset of point cloud patches characterized by mathematical functions representing a continuum from smooth surfaces to sharp edges and corners. Although trained only on synthetic surfaces, it demonstrates a remarkable capability in predicting UDFs for a wide range of  surface types. The method is evaluated on “Car” category of ShapeNet dataset in addition to DeepFashion3D, and ScanNet datasets. Furthermore, the paper shows results  on scans from real range scan dataset , in addition to ablation studies highlighting the robustness of the method to noise and outliers.

STRENGTHS:
- The paper is well written and easy to follow.
- **Novelty**: While training on local patches is not new, the design of the local shapes and network architecture is novel and effective.
- **Performance**: The paper shows good generalization results while being only trained on local synthetic patches and more robustness to noise and outliers in the input pointcloud.

WEAKNESSES:
- **Inference time**: While the method is better in terms of data storage space, data preparation time and training time, no comparison regarding the  inference time is provided.
- **Patch radius**: The patch radius used at test time is a crucial hyper-parameter for the method. A discussion about how to set this parameter and how it depends on the point cloud density/size would strengthen the paper.

QUESTIONS:
-  What the inference time of the method compared to other baselines ?
- How to set the patch radius for new shapes with different pointcloud densities and sizes ?

LIMITATIONS:
The authors adequately addressed the limitations of the method.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method for open surface reconstruction from 3D point clouds. They train a network to predict unsigned distance functions (UDFs) from point cloud patches using only synthetic data of quadratic surfaces. Evaluation shows that the trained network generalizes well to other complex patterns and is more resilient to noise when reconstructing 3D surfaces from point clouds.\n\nSTRENGTHS:\nThe idea of training a UDF regression network using only synthetic data of quadratic surfaces is quite intriguing. This approach allows for a controlled and systematic way to generate training data, which can be more consistent and free from the imperfections and variability found in real-world data. \n\nUsing quadratic surfaces as a basis for synthetic data is quite interesting where the authors argue that quadratic surfaces can approximate various local geometries. I have some doubts about this but it is reasonable and novel to a certain extend.\n\nWEAKNESSES:\nI have three main concerns: potential biases with the synthetic training data, the evaluation scheme, and the applicability in practice due to the patch radius.\n\n\n## Potential Biases with Synthetic Training Data: \n\nThe use of primitive geometrical patches as training data for a UDF regressor might introduce biases. The observation that any local geometries can be approximated by quadratic surfaces is only valid at a very fine resolution, which requires dense point clouds to observe reliably. It is unclear how many patches have been synthesized and whether they provide a good approximation of universal geometrical primitives. Some analyses would be helpful here: for example, using the ShapeNet car dataset, cropping all local patches from each car, and finding the closest synthesized one to check for approximation errors. Are there any patterns with sufficiently high approximation errors? Can we perform these analyses at different resolutions and see how they correlate with surface reconstruction?\n\n## Evaluation Scheme\n\nThe testing data is simulated to match the scenarios the method is designed for: the point cloud is quite dense, and artificial noise is added similarly to the training data. There is no quantitative evaluation for real-world scanned data. \n\n## How sensitive is the method to different patch radii?\n\nThe value of 0.018 is oddly specific. I suspect that with a larger value of r, the method will generate overly smooth surfaces (as shown in Figure 6 - right), and with a smaller value of r, it will generate holes due to the point cloud not being dense enough. Overall, there is an inherent issue with this trade-off that may not be resolvable with this approach. Detailed experiments varying the patch radius and analyzing the impact on reconstruction quality would be helpful.\n\n## Additional Concerns\n\n- Ablation studies on the network architecture are missing. I am unsure about the roles of the two branches and the cross-attention mechanism. There is a potential issue with the reliance on Point-Net for embedding computation. Is this network pre-trained on other datasets? If so, we should be careful with the claim of using only synthetic data, as pre-training on real data could influence the results.\n\n- It also seems that the method could be quite slow. The authors should include a speed test to provide insights into the computational efficiency of the proposed approach. Evaluating the method's runtime on different hardware setups and for varying point cloud sizes would give a clearer picture of its practical applicability.\n\nQUESTIONS:\nMain questions I would like the answers for:\n- Analyses on using the synthetic patches to approximate the real local 3D geometries.\n- Speed test.\n- Quantitative results on real-world scanned data, such as 3D-Scene dataset.\n\nLIMITATIONS:\nThe authors acknowledged that the proposed method cannot handle incomplete point cloud. However, it is unclear what how resilient it is when dealing with this.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}