SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Note that if  $u$  is a signed distance function, it satisfies the eikonal equation and thus the mean curvature is the Laplacian of the SDF, i.e.,  $H = \Delta u$ . Hence, for arbitrary shapes, the Laplacian of an SDF is the mean curvature of the level sets, which is not always close to zero. If we would like to represent shapes with fine detail and complex curvatures, penalizing the Laplacian of  $u$  in the loss would not necessarily be beneficial, although the term stabilizes the eikonal loss.
Iter. 0
Iter. 1.5k
Iter. 3k
Figure 2: Illustration of the ability of our new regularization to capture fine-scale details of shape while still stabilizing the optimization. Without a penalty on the mean curvature, our directional divergence term restores the shape more quickly and captures fine details (bottom). On the other hand, the full divergence term (top) unnecessarily penalizes the mean curvature of the level sets, resulting in over-smoothness. Note the dark black lines represent the zero level set (lighter ones indicate other level sets). Ground truth is a snow-flake like shape (dotted gray). Note that both divergence terms avoid instabilities, but our proposed one recovers the correct curvatures of the shape.
However, we note that there is a component of the Laplacian of a SDF that is zero. Indeed, if we compute the gradient of both sides of the eikonal equation  $(|\nabla u(x)| = 1)$ , we obtain that
$$
0 = D ^ {2} u (x) \cdot \frac {\nabla u (x)}{| \nabla u (x) |} = D ^ {2} u (x) \cdot \nabla u (x), \tag {15}
$$
where  $D^2 u(x)$  indicates the Hessian of the SDF, and the dot indicates matrix-vector multiplication. Note that the above quantity dotted with  $\nabla u$  is the second derivative of  $u$  in the normal direction of the level sets, which is a component of the full Laplacian of  $u$ . Hence, we introduce a new loss term as a replacement for the penalty on the full Laplacian, which we refer to as Laplacian normal regularization or directional divergence:
$$
L _ {\mathrm {L . n .}} (u) = \int_ {\Omega} | \nabla u (x) ^ {T} D ^ {2} u (x) \cdot \nabla u (x) | \mathrm {d} x. \tag {16}
$$
This loss enforces the constraint in SDFs that the second derivative in the normal direction is zero, without enforcing unwanted smoothness by penalizing the fine detail (points of high mean curvature) of the level sets. This will lead to a fourth-order (non-linear) PDE for its gradient descent. The gradient descent PDE includes a term that is  $-\Delta[\Delta u]$ , an isotropic fourth order term, which from the previous analysis would stabilize the lower order eikonal instability. Although the full flow only regularizes in the normal direction, over the evolution it regularizes over other directions as the normal vector changes direction, killing the eikonal instability.
New Training Loss: We combine the new stabilizing term with the loss function used in SIREN [12] without the normal constraint (for more applicability) to form our proposed training loss:
$$
L = \alpha_ {e} L _ {\mathrm {e i k}} + \alpha_ {m} L _ {\text {m a n i f o l d}} + \alpha_ {n} L _ {\text {n o n m a n i f o l d}} + \alpha_ {l} L _ {\mathrm {L . n .}},
$$
$$
L _ {\text {m a n i f o l d}} = \int_ {\Omega_ {0}} | u (x) | \mathrm {d} x, \quad L _ {\text {n o n m a n i f o l d}} = \int_ {\Omega \backslash \Omega_ {0}} \exp (- \alpha | u (x) |) \mathrm {d} x, \tag {17}
$$
where  $\alpha, \alpha_{e}, \alpha_{n}, \alpha_{m}, \alpha_{l} > 0$  are hyper-parameters, and  $\Omega_0$  are known points on the surface of interest (e.g., point cloud data).  $L_{\mathrm{manifold}}$  penalizes surface points away from the zero level set.  $L_{\mathrm{non~manifold}}$  penalizes points not on the surface of interest from being close to the zero level set. We use  $p = 1$  for the Eikonal loss, the same as in SIREN [12] and DiGS [14]. For  $\alpha_{l}$  we use the annealing strategy as in DiGS [14]. See supplementary for details.
4.2 A New Representation for Finer Shape Representation in Neural SDFs
We now introduce a new neural network representation for SDFs, motivated by our result that allows stabilizing the eikonal loss even when the representational power of the network increases. Note in a ReLu MLP, the network represents a piecewise-linear function. Activations partition the domain where various linear approximations are used. To capture finer details of shape (without resorting to heavy linear networks), it is natural to leverage more general Taylor series (quadratic and beyond) approximations to capture the curvature of the shape. Motivated by this observation, we propose to use quadratic layers rather than linear layers. Notice that the composition of a quadratic with a quadratic function is a quartic function, and thus composing quadratic layers many times can approximate any desired order of a Taylor series, even without the use of activations. We still use activations, however, to partition the domain into regions where different Taylor approximations are used. Without stabilizing the eikonal term in the optimization, such finer-scale representations become unstable; thus, our regularization plays a crucial role. Note quadratic layers have been proposed for neural networks [30]; however, proposing them for shape representation in neural SDFs is novel to the best of our knowledge. Note also that SIREN [12] uses a sinusoidal activation to obtain a representation beyond piecewise linear in ReLu MLPs; in that representation, however, the activation serves to both partition the domain in pieces, and represent each piece with more complex (polynomial) functions. Quadratic layers allow more complex (polynomial) functions in the pieces, without overloading the activation with both partitioning the domain and more complex function representation.
As in [30], we define a quadratic layer using the following representation:
$$
\mathbf {a} (\mathbf {x}) = \left(W _ {1} \mathbf {x} + \mathbf {b} _ {1}\right) \circ \left(W _ {2} \mathbf {x} + \mathbf {b} _ {2}\right) + W _ {3} \mathbf {x} ^ {2} + \mathbf {b} _ {3}, \tag {18}
$$
where  $W_{j} \in \mathbb{R}^{m_{1} \times m_{2}}$ ,  $\mathbf{x} \in \mathbb{R}^{m_{2}}$  is the input vector,  $\mathbf{x}^2$  is the element-wise square,  $\mathbf{b}_{\mathrm{j}} \in \mathbb{R}^{m_{2}}$  are biases, and  $\circ$  denotes the element-wise product. We replace the linear neurons in the SIREN [12] network with quadratic neurons to obtain a high-order expression for the signed distance function. For implementation, we use the combination of three linear layer modules in PyTorch.
We now demonstrate the effectiveness of our method on the task of surface reconstruction from point clouds. For all the experiments in this section, we follow the same mesh generation procedure and evaluation setting as the state-of-the-art method DiGS [14]. We experiment on three benchmarks: Surface Reconstruction Benchmark (SRB) [32], ShapeNet [33], and the Scene Reconstruction Benchmark [12]. We use a network with 5 hidden layers and 128 hidden channel for SRB and ShapeNet, and we use 8 hidden layers, and 256 channels for scene reconstruction. The number of training iterations is the same as in DiGS [14], 10k for SRB and ShapeNet, and 100k for scene reconstruction. We provide all of the training details in the supplementary.
5.1 Surface Reconstruction Benchmark (SRB)
SRB consists of 5 noisy range scans and each contains point cloud and normal data. We compare our method against the current state-of-the-art methods on this benchmark without using normal data (as in [14] as normal data may be difficult to obtain). Results are shown in Table 1. We report the Chamfer  $(d_{C})$  and Hausdorff  $(d_{H})$  distances between the reconstructed meshes and the ground truth meshes. Furthermore, we report their corresponding onesided distances  $(d_{\vec{C}}$  and  $d_{\vec{H}})$  between the reconstructed meshes and the input noisy point cloud, which measures how much the reconstruction overfits noise in the input. Results show that StEik is better than SoTA methods on
Method	GT	Scans
dC	dH	dC	dH
IGR wo n	1.38	16.33	0.25	2.96
SIREN wo n	0.42	7.67	0.08	1.42
SAL[1]	0.36	7.47	0.13	3.50
IGR+FF[17]	0.96	11.06	0.32	4.75
PHASE+FF[17]	0.22	4.96	0.07	1.56
DiGS[14]	0.19	3.52	0.08	1.47
Our StEik	0.180	2.800	0.096	1.454
the ground truth metrics, but can slightly overfit the noisy input due to the fine representation property of our method. The improvement is not so dramatic compared to DiGS [14] because this SRB is a relatively easy task without many thin structures and complex structures, and DiGS [14] already has a good performance. However, we still achieve a better result with  $25\%$  fewer parameters than DiGS.
5.2 ShapeNet
We evaluated our method on a preprocessed subset [44], [45] of ShapeNet [33], which consists of 20 shapes in each of 13 categories with only surface point data. Note points are sampled from the shapes (as in [44]) to simulate point clouds. We compare StEik against the current state-of-the-art methods on this dataset without using normal data and report the results in Table 2. As criteria for the benchmark, we consider the Intersection over Union (IoU) and Chamfer Distance between the reconstructed shapes and the ground truth shapes. The Intersection over Union (IoU) captures the accuracy of the predicted occupancy function, while the Chamfer Distance captures the accuracy of the predicted surface. Under both metrics, StEik outperforms all other methods by a large margin. This demonstrates that StEik is particularly effective for reconstructing thin structures. A visual example is shown in Figure 3 (see supplementary for more).
Table 1: Quantitative results on the Surface Reconstruction Benchmark[32] using only point data (no normals).
Method	squared Chamfer ↓	IoU ↑
mean	median	std	mean	median	std
SIREN wo n	3.08e-4	2.58e-4	3.26e-4	0.3085	0.2952	0.2014
SAL[1]	1.14e-3	2.11e-4	3.63e-3	0.4030	0.3944	0.2722
DiGS[14]	1.32e-4	2.55e-5	4.73e-4	0.9390	0.9764	0.1262
Ablation (of Regularizations & Linear vs Quad Layers)
Lin+L L n.	1.71e-4	1.23e-5	1.20e-3	0.9586	0.9809	0.0993
Qua+L div	5.45e-5	1.05e-5	3.60e-4	0.9593	0.9852	0.1130
Our StEik (Qua+L L n.)	6.86e-5	6.33e-6	3.34e-4	0.9671	0.9841	0.0878
Table 2: Quantitative results on the ShapeNet[33] using only point data (no normals).
Ablation Study: Below the middle line in Table 2, we study the effectiveness of each of our novel contributions (the Laplacian normal regularization and quadratic layers). On 5 out of 6 metrics,
our normal Laplacian regularization using linear networks outperforms DiGs (which uses linear networks), showing the utility of the new regularization. On 4 out of 6 metrics, our normal Laplacian regularization out-performs the standard Laplacian regularization using quadratic networks, again showing the utility of our new regularization separately. Note that in both cases the metrics where the normal Laplacian normal performs worse are just slightly worse compared to the amount of increase in the other metrics. On all 6 metrics, the quadratic network using the same Laplacian regularization as DiGs out-performs DiGs, showing the utility of quadratic networks alone. Note that each of our contributions, i.e., Laplacian normal regularization and quadratic layers, separately show increase performance against DiGs (except one metric in the linear case) even though the hyper-parameters were not optimized in the approaches compared against DiGS.
DiGS
Figure 3: Example Visual results on ShapeNet [33]: We manifest the effectiveness of the new regularization and the new representation of Neural SDFs independently. Furthermore, the combination of two modules demonstrates an extra improvement. See supplement for more visual results.
Linear  $+L_{\mathrm{L.n.}}$
Quadratic  $+L_{\mathrm{div}}$
Our StEik
5.3 Scene Reconstruction
In Figure 4, we show the reconstruction of a room scene point cloud from roughly 10M points and compare our method with DiGS[14], the current SoTA method without normals. This is the same scene used in [12] and contains many thin features that are difficult to reconstruct. The surface produced by DiGS is over-smoothed so that the thin structures like picture frames and sofa legs are not recovered, while in StEik those fine details are recovered.
DiGS
Figure 4: Visual results on the Scene Reconstruction Benchmark using only point data (no normals).
5.4 Timing Performance and Model Size
Table 3 compares the training time of one iteration and the number of parameters of DiGS [14] and our method. The evaluation is performed on a single Nvidia Tesla A100 GPU. The setting above the line is for SRB [32] and ShapeNet [33] experiments. The setting below the line is for the scene reconstruction experiment [12]. We achieve better performance than DiGS [14] with only  $3/4$  the number of parameters. There is an increase in training time (per iteration) for StEik compared to DiGS due to the extra computation cost introduced by quadratic neurons. Note that our  $L_{L,n}$  regularization is computationally less expensive
Method	Structure	Runtime	Parameters
DiGS	5×256	37.86ms	0.26M
Lin+L L.n.	5×256	32.52ms	0.26M
Qua+L div	5×128	50.92ms	0.20M
Our StEik	5×128	42.20ms	0.20M
DiGS	8×512	63.28ms	1.84M
Lin+L L.n.	8×512	50.90ms	1.84M
Qua+L div	8×256	100.27ms	1.39M
Our StEik	8×256	80.62ms	1.39M
Table 3: Network structure, speed, and size comparison.  $5 \times 256$  means 5 layers with 256 neurons.
than  $L_{div}$  (proposed in DiGS) as  $L_{L,n}$  can be expressed as  $\frac{1}{2}\nabla_x[\langle \nabla u(x),\nabla u(x)\rangle ]$  , which can be computed with a single back-propagation call. In contrast there is no such simpler expression for the Laplacian.
5.5 Limitations
Due to the lack of efficient implementation of quadratic layers in the deep learning libraries, the increase in training time is not negligible. Indeed, there is no native Pytorch implementation of a quadratic layer, and so we currently stack linear layers. As quadratic layers become more frequently used, we expect that a native Pytorch implementation become available, which would address the current drawback. In addition, there is still improvement space for reconstruction results, as in some cases the surface is not perfectly recovered.
We showed that stability is an important consideration in the design of neural SDF representations. We showed that the eikonal loss can result in instabilities that can cause artifacts in both the optimization and the recovered shape, or even converge to sub-optimal local minima. Our theory allows for understanding the instability and existing methods for neural SDFs in a common framework. Our framework enabled the construction of a new regularization term for neural SDFs that stabilizes the instability while avoiding over-regularization. The regularization enabled us to consider finer shape representations with neural SDFs that are piecewise polynomial while stabilizing the eikonal term. Empirical results validated our theoretical findings. This work opens up the possibility of exploring a broader range of geometric regularizations that naturally arise from PDEs, and the possibility of exploring new finer-scale network representations.
Supported in part by Army Research Office (ARO) W911NF-22-1-0267 and by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/ Interior Business Center (DOI/IBC) contract number 140D0423C0075. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DOI/IBC, or the U.S. Government.
[1] M. Atzmon and Y. Lipman, Sal: Sign agnostic learning of shapes from raw data, 2020. arXiv: 1911.10414 [cs.CV].
[2] Y. Lipman, Phase transitions, distance functions, and implicit neural representations, 2021. arXiv: 2106.07689 [cs.LG].
[3] M. Niemeyer, L. Mescheder, M. Oechsle, and A. Geiger, Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision, 2020. arXiv: 1912.07372 [cs.CV].
[4] L. Yariv, J. Gu, Y. Kasten, and Y. Lipman, Volume rendering of neural implicit surfaces, 2021. arXiv: 2106.12052 [cs.CV].
[5] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, Nerf: Representing scenes as neural radiance fields for view synthesis, 2020. arXiv: 2003.08934 [cs.CV].
[6] J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualla, and P. P. Srinivasan, Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields, 2021. arXiv: 2103.13415 [cs.CV].
[7] K. Zhang, G. Riegler, N. Snavely, and V. Koltun, Nerf++: Analyzing and improving neural radiance fields, 2020. arXiv: 2010.07492 [cs.CV].
[8] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, *Mip-nerf* 360: Unbounded anti-aliased neural radiance fields*, 2022. arXiv: 2111.12077 [cs.CV].
[28] P. Mantini and S. K. Shah, "Cqnn: Convolutional quadratic neural networks," in 2020 25th International Conference on Pattern Recognition (ICPR), 2021, pp. 9819-9826. DOI: 10.1109/ICPR48806.2021.9413207.
[29] G. G. Chrysos, S. Moschoglou, G. Bouritsas, Y. Panagakis, J. Deng, and S. Zafeiriou, II-Nets: Deep polynomial neural networks, 2020. arXiv: 2003.03828 [cs.LG].
[30] F. Fan, J. Xiong, and G. Wang, Universal approximation with quadratic deep networks, 2019. arXiv: 1808.00098 [cs.LG].
[31] Z. Xu, F. Yu, J. Xiong, and X. Chen, *Quadratic*: A performant quadratic neural network library for architecture optimization and design exploration*, 2022. arXiv: 2204.01701 [cs.LG].
[32] M. Berger, J. A. Levine, L. G. Nonato, G. Taubin, and C. T. Silva, “A benchmark for surface reconstruction,” ACM Trans. Graph., vol. 32, no. 2, Apr. 2013, ISSN: 0730-0301. DOI: 10.1145/2451236.2451246. [Online]. Available: https://doi.org/10.1145/2451236.2451246.
[33] A. X. Chang, T. Funkhouser, L. Guibas, et al., Shapenet: An information-rich 3d model repository, 2015. arXiv: 1512.03012 [cs.gr].
[34] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[35] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
[36] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708.
[37] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258.
[38] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "Mobilenetv2: Inverted residuals and linear bottlenecks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510-4520.
[39] N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, "Shufflenet v2: Practical guidelines for efficient cnn architecture design," in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 116-131.
[40] L. N. Trefethen, "Finite difference and spectral methods for ordinary and partial differential equations," 1996.
[41] L. C. Evans, *Partial differential equations*. American Mathematical Society, 2022, vol. 19.
[42] M. Spivak, A comprehensive introduction to differential geometry. Publish or Perish, Incorporated, 1970, vol. 4.
[43] L. Ambrosio and H. M. Soner, "Level set approach to mean curvature flow in arbitrary codimension," Journal of differential geometry, vol. 43, no. 4, pp. 693-737, 1996.
[44] F. Williams, M. Trager, J. Bruna, and D. Zorin, “Neural splines: Fitting 3d surfaces with infinitely-wide neural networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 9949–9958.
[45] L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger, Occupancy networks: Learning 3d reconstruction in function space, 2019. arXiv: 1812.03828 [cs.CV].