SlowGuess/NeurIPS_2023_OCR huggingface dataset:
ABC and Thingi10K. The ABC dataset [24] contains a variety of CAD meshes, while Thingi10K [57] comprises shapes with rich geometric details. We follow [16] to perform splitting for each dataset and randomly sample 10K points from each mesh. Each dataset contains 100 shapes. The baseline methods include Screened Poisson Surface Reconstruction (SPSR) [23], SAL [1], IGR [19], SIREN [40], Neural-Pull [5], Shape as Points (SAP) [38], DiGS [6], iPSR [20], and PGR [27]. Note that we found that the supervised version of SAP does not generalize well to shapes not present in the training set (ShapeNet) when using a global PointNet-based encoder. Therefore, we compare our approach against the unsupervised version of SAP. We also include the supervised method POCO [8] for comparison. To assess the generalization ability of supervised methods, we retrain them using 10K points on ShapeNet. The quantitative comparison results are presented in Tab. 3. Furthermore, the visual comparison on ABC [24] and Thingi10K [57] (see Fig. 4) demonstrates that our method can effectively recover CAD features such as small holes and thin plates and has ability to accurately recover high-fidelity geometric details.
3D Scene. Finally, we conducted tests on a 3D scene dataset [56] by sampling 20K points for each model. The quantitative comparison results are presented in Tab. 4 and visual comparison in Fig. 5. Our method, when combined with either SIREN [40] or Neural-Pull [5], yields significant improvements even with only 20K points for the scene-level reconstruction.
Normal C.	Chamfer	F-Score
mean↑	std.↓	mean↓	std.↓	mean↑	std.↓
SIREN [40]	84.26	3.46	7.38	4.47	53.94	29.13
Ours (SIREN)	88.82	3.33	7.14	4.12	58.37	36.97
Neural-Pull [5]	82.05	5.34	33.96	17.14	20.99	25.46
Ours (Neural-Pull)	87.32	5.11	12.04	9.79	35.82	28.86
Table 4: Quantitative comparison on 3D Scene [56].
Notably, our method effectively captures both the details and overall shape of these challenging scenes.
4.2 Shape Space Learning
The D-Faust [7] dataset contains high-resolution raw scans (triangle soups) of 10 humans with various poses. We follow DualOctreeGNN [46] to perform splitting that 6K scans are used for training and
ABC	Thingi10K
Normal C. mean↑	std.↓	Chamfer mean↓	std.↓	F-Score mean↑	std.↓	Normal C. mean↑	std.↓	Chamfer mean↓	std.↓	F-Score mean↑	std.↓
SPSR* [23]	95.16	4.48	4.39	3.05	74.54	26.65	97.15	2.95	3.93	1.79	77.03	23.71
SAL [1]	86.25	8.39	17.30	14.82	29.60	18.04	92.85	5.01	13.46	7.97	27.56	14.64
IGR [19]	82.14	16.12	36.51	40.68	43.47	40.06	90.20	10.61	27.80	34.8	54.28	39.99
SIREN [40]	82.26	9.24	17.56	15.25	30.95	22.23	88.30	6.53	17.69	13.47	26.20	19.74
Neural-Pull [5]	94.23	4.57	6.73	5.15	42.67	10.75	96.15	2.80	5.89	1.12	46.44	8.53
SAP [38]	81.59	10.61	15.18	16.60	45.88	33.67	92.60	7.03	10.61	13.84	53.32	31.91
DiGS [6]	94.48	6.12	6.91	6.94	66.22	32.01	97.25	3.30	5.36	5.59	74.45	27.11
iPSR [20]	93.15	7.47	4.84	4.06	71.59	24.96	96.46	3.57	4.41	2.94	74.88	22.72
PGR [27]	94.11	4.63	4.52	2.13	68.91	27.86	96.80	3.25	4.22	2.01	72.86	22.98
POCO+ [8]	92.90	7.00	6.05	6.80	68.29	26.05	95.16	5.00	5.61	9.42	73.92	25.79
Ours (SIREN)	96.50	3.95	3.72	2.48	83.50	19.28	97.92	2.90	3.34	3.14	89.68	17.11
Nerual-Pull
Figure 5: Visual comparison of our method to other methods under 3D Scene [56].
Ours (Nerual-Pull)
SIREN
Ours (SIREN)
IGR*
DiGS
SAL
IGR
DualOctreeGNN
Figure 6: Visual comparison on DFAUST [7].
SALD\*\*
DualOctreeGNN*
Ours
Table 3: Quantitative comparison on ABC [24] and Thingi10K [57]. Each raw point cloud has 10K points.
Normal C. mean↑	std.↓	Chamfer mean↓	std.↓	F-Score mean↑	std.↓
IGR* [19]	92.02	3.34	29.01	33.61	73.32	14.05
DualOctreeGNN* [46]	97.65	0.34	1.78	3.70	97.48	1.03
SAL+ [1]	96.77	0.81	2.82	4.67	91.35	9.15
SALD** [2]	97.04	0.92	3.06	1.32	88.56	12.73
IGR [19]	57.93	3.40	48.56	2.35	6.54	0.11
DiGS [6]	87.60	1.91	11.87	4.56	37.77	7.11
DualOctreeGNN [46]	92.42	0.45	3.02	2.38	85.77	3.50
Ours (SIREN)	95.43	0.41	2.95	2.21	88.75	2.51
2K scans for testing. In the training phase, we adopt the encoder following Convolutional Occupancy Networks [41]. Besides, we adopt the FiLM conditioning [11] that applies an affine transformation to the network's intermediate features as SIREN is weak in handling high-dimensional inputs [11, 33]. The baseline approaches include IGR [19], SAL [1], SALD [2], DualOctreeGNN [46] and DiGS [6]. Also, we further demonstrate the results of IGR, as well as the results of DualOctreeGNN trained without normals. As shown in Fig. 6, although IGR can generate well details, it yields spurious planes away from the input. SAL, supervised with unsigned distance, can only produce smooth results. Additionally, SALD, with the support of normal supervision, can generate more details, but its reconstruction accuracy is even worse than SAL since it suffers from a large systematic misalignment that does not respect input poses.
By contrast, DualOctreeGNN gets the most impressive results since the well-designed octree network can capture local prior for details. However, the performance of either IGR or DualOctreeGNN is compromised without normals.
Table 5: Quantitative comparison on DFAUST [7]. All methods marked with  ${}^{* * }$  ’ means it leverages with normals in the training stage and marked with  ${}^{+ + }$  ’ means supervised.
Normal C.	Chamfe	F-Score
mean↑	std.↓	mean↓	std.↓	mean↑	std.↓
α = 0.06	91.02	4.34	4.55	4.10	76.82	29.41
α = 0.6	92.65	3.95	4.17	4.20	80.44	23.46
α = 6	96.52	3.31	3.38	3.64	88.71	18.28
α = 60	93.04	1.03	2.19	0.73	82.91	24.73
Table 6: Effect of weights of  ${L}_{\text{align }}$  .
DiGS cannot yield reliable results though it is also based on SIREN. To summarize, our
method can learn shape space without requiring input normals or additional supervision but still produces faithful shapes. It's important to note that the resulting surfaces of DualOctreeGNN for point clouds without input normals are not watertight, despite the small Chamfer distances.
Our ablation studies were conducted on the ShapeNet dataset [13], which comprises 13 categories of shapes, with 20 shapes per category for a total of 260 shapes, each represented by 3K points. We use SIREN as the activation function for our baseline.
Comparison with smoothness energies In Figure 7, we increased the weighting coefficient of the regularization term to  $1 \times 10^{3}$ ,  $1 \times 10^{5}$ , and  $1 \times 10^{7}$ , respectively, for both our approach and those based on smoothing. It's evident that even with a large weight, our regularization term consistently produces high-fidelity surfaces, in contrast to the smoothing-based approaches, whereas their reconstruction results may become overly smooth or even fail (Note that double layers for Dirichlet energy).
A similar trend is observed in the 3D context. As illustrated in Figure 9, our method excels in reconstructing faithful and high-fidelity shapes from input data with 100K points, while the other approaches tend to produce over-smoothed results.
We also conducted an ablation study to compare the quantitative performance of our energy term with that of Dirichlet Energy [28] and Hessian Energy [6]. The results presented in Tab. 8 demonstrate that our energy term  $E_{\mathrm{align}}(f_{\theta})$  significantly improves the performance. It's worth noting that the comparison between DiGS [6] (with a Laplacian energy) and ours has been made in previous subsections, which shows that ours has a superior performance. Therefore, in this subsection, we do not include Laplacian energy for comparison.
Figure 7: Ablation study on the weighting coefficient of the regularization term.
Effect of weights of  $L_{\mathrm{align}}$ . We conducted an ablation study to evaluate the effect of the weight  $\alpha$  of  $L_{\mathrm{align}}$ . By selecting different values, i.e.,  $\alpha = \{0.06, 0.6, 6, 60\}$ , we keep the statistics in Tab. 6, which demonstrates that the best performance was achieved when the weight was set to 6.
Effect of adaptive per points weights. We demonstrate the effectiveness of the adaptive weight  $\beta_{q}$  for each query in Tab. 7. We also compare the performance of different decay parameters  $\delta$  used to compute  $\beta_{q}$  in Eq. 8, and find that  $\delta = 10$  yields the best results.
Normal C.	Chamfer	F-Score
mean↑	std.↓	mean↓	std.↓	mean↑	std.↓
δ = 0	91.85	7.22	4.73	3.91	79.95	22.38
δ = 1	91.85	7.23	4.72	3.52	79.94	22.38
δ = 10	96.52	3.31	3.38	3.64	88.71	18.28
δ = 100	90.65	6.93	4.81	3.77	80.12	25.41
Table 7: Effect of adaptive per points weights.
Comparison to LSA. Recently, LSA [31] proposed enforcing alignment between the gradient of each level set and the gradient of the 0-level set to account for the direction of the gradient. However, it is important to note that the gradient of the 0-level set may not align with the true surface normals for its results. As such, using
the gradient of the 0-level set as a reference may not be particularly effective. We present results for SIREN [40] using both our loss and LSA under six shapes from the Stanford Scanning dataset with 2 million points same as the settings of LSA in its main paper. As shown in Fig. 8, our method effectively eliminates ghost geometry in empty areas due to incorrect gradient direction, while LSA does not. Tab. 9 presents the comparison statistics for three different approaches: 1) our methods combined with SIREN [40], 2) original SIREN and 3) LSA.
Normal C.	Chamfer	F-Score
mean↑	std.↓	mean↓	std.↓	mean↑	std.↓
Dirichlet Energy	92.91	7.48	4.30	3.75	77.67	23.83
Hessian energy	93.28	7.21	4.38	3.85	81.12	25.41
Ours	96.52	3.31	3.38	3.64	88.71	18.28
Figure 8: Visual comparison with SIREN [40] and LSA [31] on Stanford Scanning dataset.
Table 8: Quantitative comparison to Smooth Energy.
Normal C.	Chamfer	F-Score
mean↑	std.↓	mean↓	std.↓	mean↑	std.↓
SIREN	93.37	5.58	7.19	7.57	74.82	29.21
LSA (SIREN)	96.40	1.04	4.10	1.19	86.38	7.37
Ours (SIREN)	98.25	1.03	2.19	0.73	97.66	4.15
Table 9: Quantitative comparison to LSA [31].
Figure 9: Comparison with smoothness energies in 3D context.
Figure 10: Our method encountering challenges while handling sparse inputs.
In this paper, we propose a novel approach for surface reconstruction from unoriented point clouds by aligning the gradient and the Hessian of the Signed Distance Function (SDF). Unlike existing smoothness energy formulations that minimize SDF volatility, our approach offers more effective control over gradient direction, allowing the implicit function to adapt to the inherent complexity of the input point cloud. Comprehensive experimental results demonstrate that our approach effectively suppresses ghost geometry and recovers high-fidelity geometric details, surpassing state-of-the-art methods in terms of reconstruction quality.
Our current algorithm still has a few drawbacks. Firstly, it struggles with processing super sparse inputs, such as sketch point clouds or LiDAR data (as shown in Figure 10). The inherent challenge arises from completing extensive missing parts and closing the gaps between the input stripes. Secondly, it faces difficulties when dealing with large-scale scenes, like those from Matterport3D [12], mainly due to the neural networks' catastrophic forgetting issue. Representing large-scale scenes within a single network becomes exceedingly challenging. In our future work, we plan to address these challenges. To handle sparse inputs more effectively, we are considering implementing a supervision mechanism. Additionally, we aim to adopt a sliding window strategy, similar to approaches like DeepLS [10] and BlockNeRF [43], to mitigate the catastrophic forgetting issue.
The authors would like to thank the anonymous reviewers for their valuable comments and suggestions. This work is supported by the National Key R&D Program of China (2022YFB3303200), the National Natural Science Foundation of China (62002190, 62272277, 62072284), and the Key Research and Development Plan of Shandong Province of China (2020ZLYS01).
[1] Matan Atzmon and Yaron Lipman. SAL: sign agnostic learning of shapes from raw data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[2] Matan Atzmon and Yaron Lipman. SALD: sign agnostic learning with derivatives. In International Conference on Learning Representations (ICLR), 2021.
[3] Ma Baorui, Liu Yu-Shen, Zwicker Matthias, and Han Zhizhong. Surface reconstruction from point clouds by learning predictive context priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[4] Ma Baorui, Liu Yu-Shen, and Han Zhizhong. Reconstructing surfaces for sparse point clouds with on-surface priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[5] Ma Baorui, Han Zhizhong, Liu Yu-Shen, and Zwicker Matthias. Neural-Pull: learning signed distance functions from point clouds by learning to pull space onto surfaces. In Proceedings of the International Conference on Machine Learning (ICML), 2021.
[6] Yizhak Ben-Shabat, Chamin Hewa Koneputugodage, and Stephen Gould. DiGS: Divergence guided shape implicit neural representation for unoriented point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[7] Federica Bogo, Javier Romero, Gerard Pons-Moll, and Michael J. Black. Dynamic FAUST: Registering human bodies in motion. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017.
[8] Alexandre Boulch and Renaud Marlet. Poco: Point convolution for surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[9] Fatih Calakli and Gabriel Taubin. SSD: smooth signed distance surface reconstruction. Computer Graphics Forum, 30(7):1993-2002, 2011.
[10] Rohan Chabra, Jan E Lenssen, Eddy Ilg, Tanner Schmidt, Julian Straub, Steven Lovegrove, and Richard Newcombe. Deep Local Shapes: Learning local sdf priors for detailed 3d reconstruction. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIX 16, pages 608-625. Springer, 2020.
[11] Eric Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, and Gordon Wetzstein. pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
[12] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv:1709.06158, 2017.
[13] Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: an information-rich 3d model repository. Technical Report arXiv:1512.03012 [cs.GR], 2015.
[14] David Cohen-Steiner and Frank Da. A greedy delaunay-based surface reconstruction algorithm. The visual computer, 20:4-16, 2004.
[49] Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. Deep geometric prior for surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019.
[50] Francis Williams, Matthew Trager, Joan Bruna, and Denis Zorin. Neural splines: fitting 3d surfaces with infinitely-wide neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[51] Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent sitzmann, and Srinath Sridhar. Neural fields in visual computing and beyond. Computer Graphics Forum, 41(2):641-676, 2022.
[52] Rui Xu, Zhiyang Dou, Ningna Wang, Shiqing Xin, Shuangmin Chen, Mingyan Jiang, Xiaohu Guo, Wenping Wang, and Changhe Tu. Globally consistent normal orientation for point clouds by regularizing the winding-number field. ACM Transactions on Graphics (TOG), 2023.
[53] Wang Yifan, Lukas Rahmann, and Olga Sorkine-hornung. Geometry-consistent neural shape representation with implicit displacement fields. In International Conference on Learning Representations (ICLR), 2021.
[54] Jingyang Zhang, Yao Yao, Shiwei Li, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. Critical regularizations for neural surface reconstruction in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6270-6279, 2022.
[55] Junsheng Zhou, Baorui Ma, Liu Yu-Shen, Fang Yi, and Han Zhizhong. Learning consistency-aware unsigned distance functions progressively from raw point clouds. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[56] Qian-Yi Zhou and Vladlen Koltun. Dense scene reconstruction with points of interest. ACM Trans. Graph., 32(4), 2013.
[57] Qingnan Zhou and Alec Jacobson. Thingi10k: A dataset of 10,000 3d-printing models. arXiv preprint arXiv:1605.04797, 2016.