VisCo Grids: Surface Reconstruction with Viscosity and Coarea Grids
Abstract
Surface reconstruction has been seeing a lot of progress lately by utilizing Implicit
Neural Representations (INRs).
Despite their success, INRs often introduce hard to control inductive bias (i.e., the solution surface can exhibit unexplainable behaviours), have costly inference, and are slow to train.
The goal of this work is to show that replacing neural networks with simple grid functions, along with two novel geometric priors achieve comparable results to INRs, with instant inference, and improved training times.
To that end we introduce VisCo Grids: a grid-based surface reconstruction method incorporating Viscosity and Coarea priors.
Intuitively, the Viscosity prior replaces the smoothness inductive bias of INRs, while the Coarea favors a minimal area solution.
Experimenting with VisCo Grids on a standard reconstruction baseline provided comparable results to the best performing
INRs on this dataset.
Introduction
Figure 1: The VisCo prior (left) is incorporating viscosity and Coarea; Middle and right shows ablations on each.
Reconstructing 3D surfaces from sparse point clouds is a long standing problem in both computer vision and graphics [7].
Methods tackling this problem aim to estimate 3D surfaces given as input unordered point sets (point clouds), with or without corresponding normals.
Surface representations can be divided to two groups: parametric and implicit.
Parametric methods represents the surface using some parametric domain, while implicit methods represent the surface as some level-set,
S = (cid:8) p ∈ R 3 | f ( p ) =
c (cid:9) , of a volumetric function f : R 3 → R .
While parametric methods can easily sample the surface, implicit methods can readily adapt to topological changes of the reconstructed surface.
Parametric methods include, e.g., meshes and spline surfaces, while implicit methods use, e.g., volumetric data structures such as voxel grids,
Radial Basis Functions (RBFs), or (recently) neural networks.
Implicit Neural Representation (INR) [29, 35, 13, 3, 16, 18, 4, 37] is categorized as an implicit method using a neural networks to define the implicit function f .
INRs build upon the inherit inductive bias in neural networks and their optimization process to provide smooth yet flexible and expressive surface reconstructions.
INRs have several disadvantages: First, the neural inductive bias is hard to control, often introducing undesired or unexplained surface behaviours.
In fact, a considerable research effort is dedicated to fix/change/control this bias [45, 43, 27, 26].
Second, INRs have an increased deployment cost, requiring many network evaluations for surface contouring, e.g., with marching cube based methods [29, 35], or direct rendering [49, 31].
Lastly, although using high optimized solvers, INRs are still slow to train.
The goal of this work it show that network-free grid-based implicit representations can achieve
INR-level reconstructions when incorporating suitable priors.
To that end, we present VisCo Grids: a grid-based surface reconstruction algorithm that incorporates well-defined geometric priors: Viscosity and Coarea.
In short, VisCo, see Figure 1, right.
The viscosity loss, is replacing the Eikonal loss [18, 43] used in INRs for optimizing Signed Distance Functions (SDF).
The Eikonal loss posses many bad minimal solution that are avoided in the INR setting due to the network’s inductive bias, but are present in the grid parametrization, see e.g., Figure 1, middle.
The viscosity loss, uses the notion of vanishing viscosity to regularize the Eikonal loss and provide well defined smooth solution that converges to the "correct" viscosity SDF solution.
The viscosity loss provides smooth SDF solution but do not punish excessive or "ghost" surface parts, see e.g., Figure 1 (right).
Therefore, a second useful prior is the coarea loss, directly controlling the surface’s area, and encourages it to be smaller.
The coarea loss is defined using a "squashing" function applied to the viscosity SDF making it approximately an indicator function, and then integrates its gradient norm over the domain.
Integrating the gradient norm of a function is called the Total Variation loss [12, 27] and is measuring the perimeter of indicator functions, which in our case approximates area( S ) .
VisCo grids (as other grid methods) have instant inference, and even with our current rather naive implementation are faster to train than INRs.
Considerable training time improvement are expected with a more efficient implementation.
We tested VisCo Grids on a standard 3D reconstruction dataset, and achieved comparable accuracy to the state-of-the-art INR methods.
Through ablations, we demonstrate the properties and benefit in the
VisCo prior.
Related Work
3D Surface Reconstruction Classical approaches for surface reconstruction from point clouds [7] are either parametric [2] or implicit with mostly linear function bases, e.g., grids or radial basis functions [10, 24].
Recent works have developed methods for surface reconstruction using neural networks, which consist of a non-linear function space, making these methods non-convex.
Those methods differ by how they choose to represent the 3D reconstruction.
[20, 46, 21] employ a parametric point of view.
Such discretizations do not yield watertight reconstruction, and/or lack topological detail.
A more flexible solution is the Implicit Neural Representation (INR).
INRs based methods [35, 29, 3, 13, 18, 43, 27, 6] show great progress in leveraging the inductive bias of MLPs to represent smooth surfaces, using additional losses and regularizers.
For example, [27] introduce a perturbed Dirichlet loss (i.e., norm of gradient) to push for a unique and regular occupancy solution; [6] incorporates a Divergence loss (i.e., absolute value of the divergence of the gradient of trained distance field) for encouraging the learned field to resemble a gradient field of a true distance functions.
Neural Spline [47] does not use neural networks directly, rather derive a kernel-based formulation arising from infinitely-wide shallow networks.
Shape
As points (SAP) [37] represent the surface using a differentiable Poisson solver and contouring process.
Grid-based representations Recent works suggested to reduce, completely or partially, the use of neural networks in implicit representations and replacing it with a grid-based data structure.
This is due to the heavy computational resources required in optimizing and evaluating neural networks.
Plenoxels [1] propose a view-dependent sparse voxel model and show comparable results to NeRF [31] and a speedup of two orders of magnitude.
Neural Geometric Level of Detail [44] uses an octree-based feature volume and a small MLP to represent SDF.
[32] shows fast training of INR’s using a small neural network augmented by a multiresolution hash table with trainable features.
Similar to DeepSDF [35], both work used 3D supervision for learning the SDF.
Method
We consider the 3D euclidean space R 3 with points p
= ( x, y, z ) ∈ R 3 .
We discretize the unit cube
C = [0 , 1] 3 with a 3D voxel grid G = { p I } , with nodes p I indexed by I =
( i, j, k ) , i, j, k ∈ [ n ] = { 1 , . . . , n } , i.e., p I = ( x ijk , y ijk , z ijk ) .
We denote by h = n − 1 , and by N = n 3 the total number of nodes.
We represent our reconstructed surface as a zero level of a scalar function f defined over the cube C .
f is defined by prescribing its values at the grid’s nodes f I ∈ R and trilinear interpolating in each voxel.
We will denote by f ( p ) the interpolated value at point p .
Given an input point cloud consisting of m points q k ∈ R 3 with or without (unit norm) normals n k ∈ R 3 , k ∈ [ m ] , our goal is to compute f so that its zero level set approximates the unknown surface, i.e.,
[Equation]
=
[Equation] [Equation]
Our approach to compute f is to minimize a loss function of the form
[Equation]
where
[Equation]
where ∥·∥ is the standard euclidean norm in R 3 , ∇ f ( p ) ∈ R 3 is the gradient of f sampled at point p .
Note that ∇ f is defined in interior of voxels, which is generically where the input points q k resides.
L data is the standard data loss encouraging the zero level to pass through the input points q k , and its normals (defined by gradients of f ) to coincide with input normals n k .
The prior, L prior , is the main contribution of this work, where we combine two novel losses,
[Equation]
Intuitively, the viscosity loss optimizes for a smooth Signed Distance Function (SDF) solutions, avoiding auxiliary bad minima of the Eikonal equation, while the coarea loss strives to minimize the area of the zero level surface.
Our loss has 4 hyper-parameters λ p , λ n , λ v , λ c .
We provide more details on these priors next.
3.1 Viscosity Loss
The goal of the viscosity loss is to make f approximate an SDF over C .
Given boundary conditions asking f to vanish on some closed compact surface S , the SDF solves the Eikonal equation PDE, i.e., ∥∇ f ( p ) ∥
= 1 , in a certain well defined sense (viscosity).
This motivated some previous work to directly optimize the Eikonal loss [18, 43]
[Equation]
Unfortunately, the Eikonal loss has many undesirable minima which are not SDFs.
Figure 2 shows a 1D example: both depicted solutions (denoted f ) vanish at the input points q 1 , q 2 (black points) and globally minimize the Eikonal loss over the grid (grid points are shown in blue).
The INR works mentioned above use neural networks for representing f which injects an inductive bias avoiding these bad minima, however on grids, minimizing equation 5 cannot avoid these solutions.
See, e.g., middle column in Figure 1.
More classical Eikonal solvers do work with grids however use mostly fast marching or sweeping methods [33, 41, 50, 11].
Namely, use a special discretization of the Eikonal equation favoring the viscosity solution of the Eikonal [40], and update node values according to a moving front [41].
Since this discretization is up-wind (will only propagate values in one direction) and requires choosing the maximal among its solution, its success in adaptation to a loss is not clear.
We use a different approach to build a loss favoring SDF solutions over grids motivated by the vanishing viscosity method [15].
Namely, adding to the Eikonal PDE a small perturbation of the
Laplacian of f (denoted by ∆ f ), i.e., ∥∇ f ( p ) ∥− 1 − ϵ ∆ f ( p ) = 0 , makes the PDE semi-linear elliptic [9], and hence with suitable boundary conditions it is uniquely solvable inside S with a smooth solution, approaching the viscosity positive distance function to the boundary as ϵ → 0 .
Similarly, for 1 − ∥∇ f ( p ) ∥
− ϵ ∆ f ( p ) =
the solution approaches the negative distance function inside the domain.
Motivated by the vanishing viscosity principle we suggest the following viscosity loss:
[Equation]
We discretize this loss over the grid G by replacing the first order derivatives and second order derivatives with symmetric finite differences, i.e.,
[Equation]
and similarly for D y and D z .
We use these discrete operators to approximate the gradient (cid:98) ∇ f ( p I ) =
( D x f I , D y f
I , D z f I ) and Laplacian [Equation]
[Equation]
f I .
The discretized viscosity loss now takes the form
[Equation]
3.2 Coarea loss
The coarea loss is approximating the area of the zero level set, and therefore incorporating it in the optimization pushes the reconstructed surface to be economic in area.
First, simiarly to [48] we use the centered Laplace CDF
[Equation]
to transform the SDF f to a smooth approximation of the indicator function:
[Equation]
As β → 0 , χ β converges to an indicator function leading to 1 inside S and 0 outside.
The coarea loss is defined as
[Equation]
To understand why this loss approximates the area of S we can use the coarea formula [39]:
[Equation]
where χ − 1 β ( s )
= { p | χ β ( p ) =
s } is the preimage of the value s .
Since χ x ( p ) ∈ [0 , 1] the r.h.s.
integral can be restricted to the interval [0 , 1] , and therefore the coarea loss averages the area of the level sets of χ β .
Next,
[Equation]
which shows that the level set s ∈ (0 , 1) of χ β is the level set − Ψ β − 1 ( s ) of the SDF f .
As β → 0 , − Ψ β − 1 ( s ) → 0 for all s ∈ (0 , 1) (and uniformly in ( ϵ, 1 − ϵ ) for fixed ϵ > 0 ).
Therefore the average of the level set area (i.e., the r.h.s.
of equation 11) converges to the area of f − 1 (0) =
S .
Figure 1 (right) shows how removing the coarea loss introduces an extraneous zero level set, and hence results in an undesired surface part.
Figure 3 shows a comparison of a reconstruction of semisphere with and without coarea.
In the experiments section we provide more ablation tests with the coarea and viscosity losses.
To discretize the coarea loss we let w I denote the centers of grid’s voxels, and note that ∇ χ β ( w I ) =
Φ β ( − f ( w I )) ∇ f ( w I ) , where
[Equation]
is the PDF of the Laplace distribution, and ∇ f ( w I ) is computed as a linear combination of the voxel’s corner values f I 1 , . . .
, f I 8 , see more details in the Appendix.
We end up with the discretized loss:
[Equation]
This loss is usually incorporated with a small hyper-parameter λ c with the purpose of eliminating redundant surface parts.
Experiments
In this section we extensively evaluate VisCo grids.
First, we evaluate on two standard surface reconstruction benchmarks [46, 22] (Sec. 4.1) against a large variety of state-of-the-art methods:
Poisson Surface Reconstruction [24], DGP [46], IGR [18], SIREN [43], FFN [45], NSP [47],
PHASE [27], GD [14], BPA [8], SPSR [25], RIMLS [34], SALD [5], IGR [19], OccNet [30],
DeepSDF [36], LIG [23], Points2Surf [17], DSE [38], IMLSNet [28] and ParseNet [42].
We then perform an ablation study (Sec. 4.2), and conduct a detail examination of the main components of the model, namely the viscosity and coarea losses.
Finally, we discuss the model’s ability to reconstruct sparse point clouds (Sec. 4.3) using scans from Stanford 3D Scanning Repository.
4.1 Surface reconstruction benchmarks
We next evaluated our model on two benchmarks: Surface Reconstruction Benchmark [46] and
Surface Reconstruction from Real-Scans [22].
Each containing challenging object with complex shape and topology.
Importantly, we use same hyper-paramenters for all meshes of all benchmarks with no extensive hyper-parameter search.
Surface Reconstruction Benchmark
This benchmark [46] consists of 5 noisy range scans, each containing point cloud and normal data.
We evaluate our method against current state of the art methods on this benchmark:
Deep Geometric Prior (DGP) [46], Implicit Geometric Regularization (IGR) [18], SIREN [43], Fourier Feature Networks (FFN) [45], NSP [47] and PHASE [27].
We additionally compare to the classical method of Poisson Surface Reconstruction [24].
Quantitative results are summarized in Table 1.
We report the Chamfer ( d C ) and Hausdorff ( d H ) distances between the reconstructed meshes and the ground-truth point clouds.
Furthermore, we report their corresponding one sided distances ( d → H and d → C ) between the reconstructed meshes and the input noisy point cloud.
Representative qualitative results are shown in Figure 5.
Note that we achieve comparable results to the current state-of-the-art INR methods.
Surface Reconstruction from Real-Scans This benchmark [22] consists of 21 noisy range scans of real objects.
We evaluate our method against: GD [14], BPA [8], SPSR [25], RIMLS [34], SALD [5],
IGR [19], OccNet [30], DeepSDF [36], LIG [23], Points2Surf [17], DSE [38], IMLSNet [28] and
ParseNet [42].
Quantitative results are summarized in Table 2.
We report Chamfer Distance ( d C ),
F-score, Normal Consistency Score (NCS) [30], and Neural Feature Similarity (NFS)
[22] distances between the reconstructed meshes and the ground-truth point clouds.
Furthermore, we report the one sided distances ( d → H and d → C ) between the reconstructed meshes and the input noisy point cloud.
Representative qualitative results are shown in Figure 5.
Note that we achieve 1-st to 3-rd place across all categories, with top F-score.
4.2 Ablation study
We provide an ablation study of the main components of our model in Table 3 and Figure 6.
Specifically we compared with the following alternatives: i) Eikonal loss without the viscosity term that prevents undesirable non-SDF solutions, i.e., ϵ = 0 in equation 6; ii) removing the coarea loss enforcing minimal surface area, i.e., λ c = 0 ; and iii) removing the normal loss, i.e., λ n = 0 . Note
Poisson
DGP
IGR
SIREN
FFN
NSP
PHASE
Ours (30mins)
Ours (8mins)
Anchor d C 0.
d H 14.
d → C 0.
d → H 14.
Daratech d C 0.
d H 7.
d → C 0.
d → H 7.
DC d C 0.
d H 3.
d → C 0.
[Table]
that without coarea and viscosity the reconstruction tends to have holes and discontinuities near the surface boundaries.
Only combination of all the components results in a good surface reconstruction.
Learning with viscosity.
We further provide a more in depth discussion of the proposed viscosity loss.
Figure 7 compares reconstructions with different levels of the viscosity parameter, i.e., ϵ in equation 6.
As can be inspected from this figure, viscosity affects the smoothness of the reconstructed surface.
For a low viscosity parameter the zero level sets become noisy.
This can be explained by the viscosity eikonal loss (i.e., equation 6) becoming numerically too close to the eikonal loss in
[Table]
grouped according to surface geometry priors, as originally defined in the benchmark.
Our method achieves top F-score and 1-st to 3-rd place in all scores.
equation 5 and the solution deviates from the viscosity SDF solution.
This leads to artifacts across the surface, similar to the limit case ( ϵ = 0 ) where only the eikonal loss is used, see the second column from the left.
For a high viscosity parameter, and as expected with the addition of a non-vanishing
Laplacian term, the surface becomes over-smoothed.
Learning with coarea.
Similarly, we also provide an analysis of the proposed coarea loss.
In
Figure 8 we show the effect of changing the parameter weight of the coarea loss, λ c .
As can be observed in the figure, a low coarea weight leads to the presence of excessive surface area in the reconstruction.
In contrast, a high weight will strive to minimize the surface area.
In a sense, the coarea serves a surface tension parameter; stronger tension will ignore points, weaker tension will overfit.
Baseline
[Table]
Grids.
Baseline is the full method.
The remaining columns correspond to optimizing without normal loss, viscosity loss and coarea loss, respectively.
We show results for each mesh of the benchmark [46].
The results justify the use of the different components in VisCo Grids.
In contrast, setting it too low leads to noise and discontinuities in the surfaces, similarly to removing it by setting ϵ = 0 .
4.3 Reconstructing from sparse point clouds
We now evaluate our model ability to reconstruct surfaces in the challenging case of sparse input point clouds.
For this experiment we use point clouds from the Stanford 3D Scanning Repository and downsample them at different levels: 1%, 10%, and 25%.
In Figure 9 we visualize the VisCo reconstructions.
Note that VisCo can reconstruct the shapes even with sparse input.
This provides a further validation for the proposed geometrical priors.
Conclusions
We introduced VisCo Grids, a novel grid-based surface reconstruction approach that leverages two novel geometrically motivated priors: viscosity and coarea.
We advocate VisCo’s
prior for grid functions as an alternative to the implicit inductive bias of implicit neural representations for the task of surface reconstruction.
One important limitation of our method, shared by all grid methods, is that its degrees of freedom, namely nodes’ location, are set a-priori.
In contrast, using non-linear function spaces, such as neural networks, allows a more flexible usage of the degrees of freedoms in the model and can adjust to areas with more detail.
Nevertheless, we still believe that grid functions and direct priors, incorporated with modern computing power, are valuable add-ons to the surface reconstruction toolbox, providing few clear benefits over neural networks, such as a well-understood control over surface properties, instant inference time, and faster training.
Acknowledgements.
We thank the anonymous reviewers for their constructive comments.
LY was supported by a grant from Israel CHE Program for Data Science Research Centers.
Social impact.
We don’t see any immediate negative societal impact from our work.
However, we acknowledge that high quality 3D reconstruction in general can be used in malicious settings.