SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 1: RMSE of oriented normals on datasets PCPNet and FamousShape. Our method achieves SOTA performance even when compared to supervised baselines. * means the code is uncompleted.
Category	PCPNet Dataset	FamousShape Dataset
Noise	Density	Average	Noise	Density	Average
None	0.12%	0.6%	1.2%	Stripe	Gradient	None	0.12%	0.6%	1.2%	Stripe	Gradient
PCA [26]+MST [26]	19.05	30.20	31.76	39.64	27.11	23.38	28.52	35.88	41.67	38.09	60.16	31.69	35.40	40.48
PCA [26]+SNO [67]	18.55	21.61	30.94	39.54	23.00	25.46	26.52	32.25	39.39	41.80	61.91	36.69	35.82	41.31
PCA [26]+ODP [53]	28.96	25.86	34.91	51.52	28.70	23.00	32.16	30.47	31.29	41.65	84.00	39.41	30.72	42.92
LRR [79]+MST [26]	43.48	47.58	38.58	44.08	48.45	46.77	44.82	56.24	57.38	45.73	64.63	66.35	56.65	57.83
LRR [79]+SNO [67]	44.87	43.45	33.46	45.40	46.96	37.73	41.98	59.78	60.18	45.02	71.37	62.78	59.90	59.84
LRR [79]+ODP [53]	28.65	25.83	36.11	53.89	26.41	23.72	32.44	39.97	42.17	48.29	88.68	44.92	47.56	51.93
AdaFit [86]+MST [26]	27.67	43.69	48.83	54.39	36.18	40.46	41.87	43.12	39.33	62.28	60.27	45.57	42.00	48.76
AdaFit [86]+SNO [67]	26.41	24.17	40.31	48.76	27.74	31.56	33.16	27.55	37.60	69.56	62.77	27.86	29.19	42.42
AdaFit [86]+ODP [53]	26.37	24.86	35.44	51.88	26.45	20.57	30.93	41.75	39.19	44.31	72.91	45.09	42.37	47.60
HSurf-Net [40]+MST [26]	29.82	44.49	50.47	55.47	40.54	43.15	43.99	54.02	42.67	68.37	65.91	52.52	53.96	56.24
HSurf-Net [40]+SNO [67]	30.34	32.34	44.08	51.71	33.46	40.49	38.74	41.62	41.06	67.41	62.04	45.59	43.83	50.26
HSurf-Net [40]+ODP [53]	26.91	24.85	35.87	51.75	26.91	20.16	31.07	43.77	43.74	46.91	72.70	45.09	43.98	49.37
PCPNet [24]	33.34	34.22	40.54	44.46	37.95	35.44	37.66	40.51	41.09	46.67	54.36	40.54	44.26	44.57
DPGO* [73]	23.79	25.19	35.66	43.89	28.99	29.33	31.14	-	-	-	-	-	-	-
SHS-Net [39]	10.28	13.23	25.40	35.51	16.40	17.92	19.79	21.63	25.96	41.14	52.67	26.39	28.97	32.79
Ours	10.60	18.30	24.76	33.45	12.27	12.85	18.70	16.57	19.28	36.22	50.27	17.23	17.38	26.16
In addition, we pursue better gradient consistency across iterations, i.e., make the gradient directions in each move step parallel. To achieve this, we align the gradient  $\nabla f_{i}$  after the first step with the gradient  $\nabla f_{1}$  of the first step. Meanwhile, we also consider the confidence of each point with respect to the underlying surface. Thus, we introduce confidence-weighted cosine distance to evaluate the gradient consistency at points  $\{Q,G\}$ , with the form of
$$
\mathcal {L} _ {\text {c o n}} (\boldsymbol {\theta}) = \sum_ {i = 2} ^ {\mathcal {I}} w \cdot \left(1 - \left\langle \nabla f _ {1}, \nabla f _ {i} \right\rangle\right), \text {a n d} \nabla f _ {i} = \left\{\nabla f _ {i} ^ {Q}, \nabla f _ {i} ^ {G} \right\}, \tag {8}
$$
where  $\langle \cdot \rangle$  denotes cosine distance of vectors.  $w = \exp (-\rho \cdot |f_1|)$  is an adaptive weight that indicates the significance of each input point based on the predicted distance, making the model pay more attention to points near the surface. In summary, our method combines an implicit representation and a gradient approximation with respect to the underlying geometry of  $P$ . We not only consider the gradient consistency at  $q$  during iterations, but also unify the adjacent regions of  $q$  to compensate for the inaccuracy of  $P$ . This is important because the input point cloud may be noisy and some points are not on the underlying surface.
Loss Function. We optimize global surface approximation and consistent gradient learning in an end-to-end manner by adjusting the parameter  $\theta$  of  $f$  using the following loss function
$$
\mathcal {L} (\boldsymbol {\theta}) = \mathcal {L} _ {v} (\boldsymbol {\theta}) + \lambda_ {1} \mathcal {L} _ {\text {c o n}} (\boldsymbol {\theta}) + \lambda_ {2} \mathcal {L} _ {d} (\boldsymbol {\theta}) + \lambda_ {3} \mathcal {L} _ {\text {r e g}} (\boldsymbol {\theta}), \tag {9}
$$
where  $\lambda_{1} = 0.01$ ,  $\lambda_{2} = 0.1$  and  $\lambda_{3} = 10$  are balance weights and are fixed across all experiments.
Neural Gradient Computation. The above losses require incorporating the gradient  $\nabla f(\pmb{x};\pmb{\theta})$  in a differentiable manner. As in [22, 7], we construct  $\nabla f(\pmb{x};\pmb{\theta})$  as a neural network in conjunction with
Figure 4: Visual comparison of oriented normals on point clouds with density variations and complex geometries. Point color indicates normal RMSE and red  $(180^{\circ})$  indicates reversed orientation.
Table 2: RMSE of unoriented normal on datasets PCPNet and FamousShape. Our method achieves SOTA performance compared to unsupervised baseline methods.
Category	PCPNet Dataset	FamousShape Dataset
Noise	Density	Average	Noise	Density	Average
None	0.12%	0.6%	1.2%	Stripe	Gradient	None	0.12%	0.6%	1.2%	Stripe	Gradient
Supervised
PCPNet [24]	9.64	11.51	18.27	22.84	11.73	13.46	14.58	18.47	21.07	32.60	39.93	18.14	19.50
Nesti-Net [9]	7.06	10.24	17.77	22.31	8.64	8.95	12.49	11.60	16.80	31.61	39.22	12.33	11.77
Lenssen et al. [36]	6.72	9.95	17.18	21.96	7.73	7.51	11.84	11.62	16.97	30.62	39.43	11.21	10.76
DeepFit [8]	6.51	9.21	16.73	23.12	7.92	7.31	11.80	11.21	16.39	29.84	39.95	11.84	10.54
Zhang et al. [80]	5.65	9.19	16.78	22.93	6.68	6.29	11.25	9.83	16.13	29.81	39.81	9.72	9.19
AdaFit [86]	5.19	9.05	16.45	21.94	6.01	5.90	10.76	9.09	15.78	29.78	38.74	8.52	8.57
GraphFit [38]	5.21	8.96	16.12	21.71	6.30	5.86	10.69	8.91	15.73	29.37	38.67	9.10	8.62
NeAF [41]	4.20	9.25	16.35	21.74	4.89	4.88	10.22	7.67	15.67	29.75	38.76	7.22	7.47
HSurf-Net [40]	4.17	8.78	16.25	21.61	4.98	4.86	10.11	7.59	15.64	29.43	38.54	7.63	7.40
Du et al. [19]	3.85	8.67	16.11	21.75	4.78	4.63	9.96	6.92	15.05	29.49	38.73	7.19	6.92
SHS-Net [39]	3.95	8.55	16.13	21.53	4.91	4.67	9.96	7.41	15.34	29.33	38.56	7.74	7.28
Unsupervised
Boulch et al. [11]	11.80	11.68	22.42	35.15	13.71	12.38	17.86	19.00	19.60	36.71	50.41	20.20	17.84
PCV [78]	12.50	13.99	18.90	28.51	13.08	13.59	16.76	21.82	22.20	31.61	46.13	20.49	19.88
Jet [15]	12.35	12.84	18.33	27.68	13.39	13.13	16.29	20.11	20.57	31.34	45.19	18.82	18.69
PCA [26]	12.29	12.87	18.38	27.52	13.66	12.81	16.25	19.90	20.60	31.33	45.00	19.84	18.54
LRR [79]	9.63	11.31	20.53	32.53	10.42	10.02	15.74	17.68	19.32	33.89	49.84	16.73	16.33
Ours	7.89	9.85	18.62	24.89	9.21	9.29	13.29	13.74	16.51	31.05	40.68	13.95	13.17
$f(\pmb{x};\pmb{\theta})$ . Specifically, each layer of the network has the form  $\pmb{y}^{\ell +1} = \phi (\pmb {W}\pmb{y}^{\ell} + \pmb {b})$ , where  $\pmb {\theta} = (W,b)$  is the learnable parameter of  $\ell$  layer with output  $\pmb{y}^{\ell +1}$ , and  $\phi :\mathbb{R}\to \mathbb{R}$  is a non-linear derivable activation function. According to chain-rule, the gradient is given by [22]
$$
\nabla \boldsymbol {y} ^ {\ell + 1} = \operatorname {d i a g} \left(\phi^ {\prime} (\boldsymbol {W} \boldsymbol {y} ^ {\ell} + \boldsymbol {b})\right) \boldsymbol {W} \nabla \boldsymbol {y} ^ {\ell}, \tag {10}
$$
where  $\phi^{\prime}$  is the derivative of  $\phi$ , and  $\mathrm{diag}(z)$  arrange its input vector  $z\in \mathbb{R}^{m}$  on the diagonal of a square matrix  $\mathbb{R}^{m\times m}$ . In practice, we use the automatic differentiation module of PyTorch [58] to implement the computation of the gradient  $\nabla f(\pmb {x};\pmb {\theta})$  in the forward pass.
Architecture. To learn the neural gradient function, we use a simple neural network similar to [6, 46, 22]. It is composed of eight linear layers and a skip connection from the input to the intermediate layer. Except for the last layer, each linear layer contains 512 hidden units and is equipped with a ReLU activation function. The last layer outputs the signed distance for each point. The parameters of the linear layer are initialized using the geometric initialization [6]. After optimization, we use the network to derive pointwise gradients from the data  $P$  (see Fig. 2).
Figure 5: The PGP curves of oriented normal on the FamousShape dataset. It is plotted by the percentage of good point normals (PGP) for a given angle threshold. Our method has the best value at large thresholds.
Distribution  $\mathcal{D}$ . We make the distribution  $\mathcal{D}$  concentrate in the neighborhood of  $P$  in 3D space. Specifically, it is set by uniform sampling points  $p$  from  $P$  and placing an isotropic Gaussian  $N(p,\sigma^2)$  centered at each  $p$ , with standard deviation parameter  $\sigma$  is adaptively set to the distance from the  $l$ -th nearest point to  $p$  [6, 7]. We set  $l$  to 25 for the PCPNet dataset and 50 for the FamousShape dataset.
Implementation. We choose the neighborhood scale set  $\{K_s\}_{s=1}^{N_K}$  as  $\{1, K/2, K\}, K = 8,$  and set the parameter  $\rho$  of adaptive weight to 60. We select  $N_Q = 5000$
points from the distribution  $\mathcal{D}$  and  $N_G = 2500$  points from  $P$  to form the input  $\{Q,G\}$ . The number of steps is set to  $\mathcal{I} = 2$  for all evaluations. As for metric, we use normal angle Root Mean Squared Error (RMSE) to evaluate the estimated normals and Percentage of Good Points (PGP) to show the normal error distribution [40, 39].
Figure 6: Comparison with surface reconstruction methods using point movement strategy, e.g., Neural-Pull [46], CAP-UDF [85], OSP [47] and PCP [48]. The input point clouds have different levels of noise (low, medium and high).
4.1 Performance Evaluation
Evaluation of Oriented Normal. In this evaluation, we not only compare with one-stage baseline methods, including PCPNet [24], DPGO [73] and SHS-Net [39], but also compare with two-stage baseline methods that combine representative algorithms based on different design philosophies. Specifically, we use the various combinations of unoriented normal estimation methods (PCA [26], AdaFit [86] and HSurf-Net [40]) and normal orientation methods (MST [26], SNO [67] and ODP [53]). Note that PCPNet, DPGO, SHS-Net, AdaFit and HSurf-Net rely on supervised training with ground truth normals. We report quantitative evaluation results on datasets PCPNet and FamousShape [39] in Table 1. Our method has better performance under the vast majority of data categories (noise levels and density variations), and achieves the best average results. It is worth
SNO
PCPNet
SHS-Net
GCNO
Figure 7: The reconstructed surfaces on wireframe point clouds with sparse and non-uniform sampling. There are less than 1000 points in each input point cloud. The initial normals of MST, ODP and SNO are provided by PCA. GCNO [77] performs similarly to our method.
mentioning that our method achieves a large performance improvement on the FamousShape dataset, which contains different shapes with more complex geometries than the PCPNet dataset. We provide a visual comparison of the estimated normals in Fig. 3, which shows normal vectors of a column section with a gear-like structure. In Fig. 4, we show the normal error map on point clouds with uneven sampling and multi-branch structure. The normal error distribution is shown in Fig. 5. It is clear that our unsupervised method has advantages over baselines in all categories when the angle threshold is larger than  $50^{\circ}$ . The evaluation results demonstrate that our method can consistently provide good oriented normals in the presence of various complex structures and density variations.
Evaluation of Unoriented Normal. In general, estimating unoriented normals is easier than estimating oriented normals as it does not need to explore more information to determine the orientation, but only focuses on finding the perpendicular of a local plane or surface according to the input patch. Moreover, existing deep learning-based methods for unoriented normal estimation all rely on supervised training with ground truth normals. In contrast, our method is designed for estimating oriented normal in an unsupervised manner. To evaluate the unoriented normals, we compare with the baselines using our oriented normals and ignore the orientation of normals. In Table 2, we report the quantitative evaluation results of supervised and unsupervised methods on datasets PCPNet and FamousShape. We can see that, compared to unsupervised methods, our method achieves significant performance gains on almost all data categories of these two datasets and has the best average results. The supervised methods learn from ground truth normals, and their supervision provides perfect surface perpendiculars, leading to superior results. Our better results than PCPNet highlight the strong learning ability of our method without labels.
Surface Reconstruction. For surface reconstruction from point clouds, we compare our method with the latest methods in two different ways. (1) We determine the zero level set of the learned implicit function  $f$  via signed distances, and use the marching cubes algorithm [42] to extract the global surface representation of the point cloud. A visual comparison of the extracted surfaces on point clouds with different noise levels is shown in Fig. 6, where our surfaces are much cleaner and more complete under the interference of noise. (2) Based on the estimated oriented normals, we employ
Table 3: Comparison of oriented normal estimation. We use implicit representation methods to estimate oriented normals.
Noise	Density	Average
IGR [22]	54.77	75.90	65.33
SAP [60]	57.56	41.32	49.44
SAL [6]	46.69	43.78	45.24
Neural-Pull [46]	48.48	26.22	37.35
Ours	36.92	26.08	31.50
the Poisson reconstruction algorithm [33] to generate surfaces from wireframe point clouds, and the results in Fig. 7 show that our method can handle extremely sparse and uneven data. The reconstructed surfaces of unsupervised methods on LiDAR point clouds of the KITTI dataset [20] are shown in Fig. 8, where our undistorted surface exhibits a more realistic road scene of real-world data.
Moreover, we repurpose some implicit representation methods, which are designed for surface reconstruction, to estimate oriented normals from point clouds. The comparison of normal RMSE on some selected point clouds in the datasets PCPNet and FamousShape is reported in Table 3. A visual comparison of the reconstructed surfaces on noisy point clouds is shown in the supplementary material. We can see that our method has clear advantages, especially on noisy point clouds. We provide more evaluation results on different data in the supplementary material.
Table 4: Ablation studies for unoriented and oriented normal estimation on the FamousShape dataset.
Category	Unoriented Normal	Oriented Normal
Noise	Density	Average	Noise	Density
None	0.12%	0.6%	1.2%	Stripe	Gradient	None	0.12%	0.6%	1.2%	Stripe	Gradient
(a)	w/o Lreg	13.76	16.45	31.32	42.56	13.72	12.87	21.78	16.63	19.15	36.77	55.22	16.89	16.35	26.84
w/o Ld	13.98	16.66	30.88	40.70	14.01	13.30	21.59	16.74	19.30	35.72	49.98	17.21	17.39	26.06
w/o Lcon	13.93	17.13	33.81	45.95	13.95	13.02	22.97	23.97	26.70	40.49	69.06	23.26	17.87	33.56
w/o Lv	28.41	31.34	49.04	49.65	30.60	26.47	35.92	46.97	43.00	73.64	75.61	43.10	35.33	52.94
w/o w	13.76	16.56	31.45	41.61	14.00	12.83	21.70	16.81	19.40	37.06	52.97	18.12	16.11	26.74
w/ Lv·w	13.44	16.14	31.35	41.80	13.71	12.90	21.56	16.20	18.99	36.66	51.47	17.61	18.60	26.59
(b)	w/o G	13.86	16.73	31.10	40.73	14.02	13.30	21.62	16.99	21.66	36.70	50.30	17.37	20.35	27.23
(c)	I = 1	14.03	17.05	33.42	45.97	13.94	13.36	22.96	23.39	26.34	40.10	63.89	19.47	27.89	33.51
I = 3	13.94	16.61	30.63	39.82	13.88	13.06	21.32	16.91	19.69	35.65	50.08	17.64	16.46	26.07
(d)	{1}	14.11	17.25	31.62	41.36	13.72	12.85	21.82	23.87	23.14	36.47	53.32	17.82	16.35	28.50
{K}	14.31	16.67	30.43	40.12	14.45	13.37	21.56	17.26	19.36	35.34	49.13	18.33	16.88	26.05
{1, K}	13.72	16.62	30.77	40.28	13.92	13.06	21.39	16.54	19.40	35.65	51.98	17.02	16.55	26.19
K = 4	13.60	17.34	30.90	40.43	13.70	12.56	21.42	16.66	23.60	36.17	49.44	16.86	14.72	26.24
K = 16	14.32	16.69	30.86	41.25	14.40	13.40	21.82	17.26	19.42	35.85	50.98	17.43	17.17	26.35
K = 32	15.02	17.18	30.61	41.15	15.44	14.26	22.28	17.82	19.79	35.70	50.64	19.28	18.22	26.91
Final	13.74	16.51	31.05	40.68	13.95	13.17	21.52	16.57	19.28	36.22	50.27	17.23	17.38	26.16
We aim to achieve the best performance on average in both oriented and unoriented normal estimation. We provide the ablation results on the FamousShape dataset in Table 4 (a)-(d), which are discussed as follows.
(a) Loss. We do not use one of the four loss terms in Eq. (9) and the weight  $w$  in Eq. (8), respectively. We also try incorporating the weight  $w$  into  $\mathcal{L}_v(\pmb{\theta})$  of Eq. (7). These ablations do not provide better performance in both oriented and unoriented normal estimation. We observe that the performance drops a lot without using  $\mathcal{L}_v(\pmb{\theta})$ , and that  $\mathcal{L}_{con}(\pmb{\theta})$  has a larger impact on performance. Note that  $\mathcal{L}_d(\pmb{\theta})$  plays a more important role on the PCPNet dataset than the FamousShape dataset in our experiments.
Figure 8: The reconstructed surfaces on the KITTI dataset using estimated normals of unsupervised methods. Points are colored with their heights.
(b) Input. We do not sample a subset  $G$  from  $P$ , and take  $Q$  as the input instead of  $\{Q, G\}$ . The results verify the positive effect of  $G$  on explicitly indicating the surface and improving performance.
(c) Iteration. We set the number of steps  $\mathcal{I}$  to 1 and 3, respectively. More iterations bring better performance, but we still choose  $\mathcal{I} = 2$  after weighing the running time and memory usage.
(d) Scale Set. The scale set  $\{K_s\}_{s=1}^{N_K}$  in Eq. (6) is chosen to be  $\{1, K/2, K\}$ ,  $K = 8$  in our implementation. Here we change the scale set to different sizes, e.g.,  $\{1\}$ ,  $\{K\}$  and  $\{1, K\}$ ,  $K = 8$ , or different values of  $K$ , e.g., 4, 16 and 32. Some of these settings have advantages in a single evaluation, but do not give better results for both oriented and unoriented normal estimation, such as  $\{K\}$  and  $\{1, K\}$ .
In this work, we propose to learn neural gradient functions from point clouds to estimate oriented normals without requiring ground truth normals. Specifically, we introduce loss functions to facilitate query points to iteratively reach the moving targets and aggregate onto the approximated surface, thereby learning a global surface representation of the data. Meanwhile, we incorporate gradients into the surface approximation to measure the minimum signed deviation of queries, resulting in a consistent gradient field associated with the surface. Extensive evaluation and ablation experiments are provided, and our excellent results in both unoriented and oriented normal estimation demonstrate the effectiveness of innovations in our design. Future work includes adding more local constraints to further improve the accuracy of normals.
[79] Jie Zhang, Junjie Cao, Xiuping Liu, Jun Wang, Jian Liu, and Xiquan Shi. Point cloud normal estimation via low-rank subspace clustering. Computers & Graphics, 37(6):697-706, 2013.
[80] Jie Zhang, Jun-Jie Cao, Hai-Rui Zhu, Dong-Ming Yan, and Xiu-Ping Liu. Geometry guided deep surface normal estimation. Computer-Aided Design, 142:103119, 2022.
[81] Haoran Zhou, Honghua Chen, Yidan Feng, Qiong Wang, Jing Qin, Haoran Xie, Fu Lee Wang, Mingqiang Wei, and Jun Wang. Geometry and learning co-supported normal estimation for unstructured point cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13238-13247, 2020.
[82] Haoran Zhou, Honghua Chen, Yingkui Zhang, Mingqiang Wei, Haoran Xie, Jun Wang, Tong Lu, Jing Qin, and Xiao-Ping Zhang. Refine-Net: Normal refinement neural network for noisy point clouds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):946–963, 2022.
[83] Jun Zhou, Hua Huang, Bin Liu, and Xiuping Liu. Normal estimation for 3D point clouds via local plane constraint and multi-scale selection. Computer-Aided Design, 129:102916, 2020.
[84] Jun Zhou, Wei Jin, Mingjie Wang, Xiuping Liu, Zhiyang Li, and Zhaobin Liu. Improvement of normal estimation for point clouds via simplifying surface fitting. Computer-Aided Design, page 103533, 2023.
[85] Junsheng Zhou, Baorui Ma, Yu-Shen Liu, Yi Fang, and Zhizhong Han. Learning consistency-aware unsigned distance functions progressively from raw point clouds. Advances in Neural Information Processing Systems, 35:16481-16494, 2022.
[86] Runsong Zhu, Yuan Liu, Zhen Dong, Yuan Wang, Tengping Jiang, Wenping Wang, and Bisheng Yang, AdaFit: Rethinking learning-based normal estimation on point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6118-6127, 2021.