VisCo Grids: Surface Reconstruction with Viscosity and Coarea Grids
Anonymous Author(s)
Affiliation
Abstract
Surface reconstruction has been seeing a lot of progress lately by utilizing Implicit
Neural Representations (INRs).
Despite their success, INRs often introduce hardto control inductive bias (i.e., the solution surface can exhibit unexplainable behaviours), have costly inference, and are slow to train.
The goal of this work is toshow that replacing neural networks with simple grid functions, along with twonovel geometric priors achieve comparable results to INRs, with instant inference,and improved training times.
To that end we introduce VisCo Grids: a grid-basedsurface reconstruction method incorporating Viscosity and Coarea priors.
Intuitively, the Viscosity prior replaces the smoothness inductive bias of INRs, whilethe Coarea favors a minimal area solution.
Experimenting with VisCo Grids on astandard reconstruction baseline provided comparable results to the best performing
INRs on this dataset.
Introduction
Reconstructing 3D surfaces from sparsepoint clouds is a long standing problem in both computer vision and graphics [5].
Methods tackling this problem aim to estimate 3D surfaces givenas input unordered point sets (pointclouds), with or without correspondingnormals.
Surface representations canbe divided to two groups: parametricand implicit.
Parametric methods represents the surface using some parametric domain, while implicit methodsrepresent the surface as some level-set,
S = (cid:8) p ∈ R 3 | f ( p ) =
c (cid:9) , of a volumetric function f : R 3 → R .
While parametric methods can easily sample the surface, implicit methods can readily adaptto topological changes of the reconstructed surface.
Parametric methods include, e.g., meshes andspline surfaces, while implicit methods use, e.g., volumetric data structures such as voxel grids,
Radial Basis Functions (RBFs), or (recently) neural networks.
Implicit Neural Representation (INR) [19, 23, 10, 3, 12, 13, 4, 24] is categorized as an implicit methodusing a neural networks to define the implicit function f .
INRs build upon the inherit inductive bias in neural networks and their optimization process to provide smooth yet flexible and expressivesurface reconstructions.
INRs have several disadvantages: First, the neural inductive bias is hardto control, often introducing undesired or unexplained surface behaviours.
In fact, a considerableresearch effort is dedicated to fix/change/control this bias [30, 28, 18, 17].
Second, INRs have anincreased deployment cost, requiring many network evaluations for surface contouring, e.g., withmarching cube based methods [19, 23], or direct rendering [34, 20].
Lastly, although using highoptimized solvers, INRs are still slow to train.
The goal of this work it show that network-free grid-based implicit representations can achieve
INR-level reconstructions when incorporating suitable priors.
To that end, we present VisCo Grids: agrid-based surface reconstruction algorithm that incorporates well-defined geometric priors: Viscosityand Coarea.
In short, VisCo, see Figure 1, right.
The viscosity loss, is replacing the Eikonal loss[13, 28] used in INRs for optimizing Signed Distance Functions (SDF).
The Eikonal loss possesmany bad minimal solution that are avoided in the INR setting due to the network’s inductive bias,but are present in the grid parametrization, see e.g., Figure 1, middle.
The viscosity loss, uses thenotion of vanishing viscosity to regularize the Eikonal loss and provide well defined smooth solutionthat converges to the "correct" viscosity SDF solution.
The viscosity loss provides smooth SDFsolution but do not punish excessive or "ghost" surface parts, see e.g., Figure 1 (right).
Therefore,a second useful prior is the coarea loss, directly controlling the surface’s area, and encourages it tobe smaller.
The coarea loss is defined using a "squashing" function applied to the viscosity SDFmaking it approximately an indicator function, and then integrates its gradient norm over the domain.
Integrating the gradient norm of a function is called the Total Variation loss [9, 18] and is measuringthe perimeter of indicator functions, which in our case approximates area( S ) .
VisCo grids (as othergrid methods) have instant inference, and even with our current rather naive implementation arefaster to train than INRs.
Considerable training time improvement are expected with a more efficientimplementation.
We tested VisCo Grids on a standard 3D reconstruction dataset, and achieved comparable accuracy tothe state-of-the-art INR methods.
Through ablations, we demonstrate the properties and benefit in the
VisCo prior.
Related Work
3D Surface Reconstruction Classical approaches for surface reconstruction from point clouds[5] are either parametric [2] or implicit with mostly linear function bases, e.g., grids or radial basisfunctions [7, 16].
Recent works have developed methods for surface reconstruction using neuralnetworks, which consist of a non-linear function space, making these methods non-convex.
Thosemethods differ by how they choose to represent the 3D reconstruction.
[14, 31, 15] employ aparametric point of view.
Such discretizations do not yield watertight reconstruction, and/or lacktopological detail.
A more flexible solution is the Implicit Neural Representation (INR).
INRs basedmethods [23, 19, 3, 10, 13, 28, 18] show great progress in leveraging the inductive bias of MLPs torepresent smooth surfaces, using simple losses and regularizers.
Neural Spline [32] does not useneural networks directly, rather derive a kernel-based formulation arising from infinitely-wide shallownetworks.
Shape
As points (SAP) [24] represent the surface using a differentiable Poisson solver andcontouring process.
Grid-based representations Recent works suggested to reduce, completely or partially, the use ofneural networks in implicit representations and replacing it with a grid-based data structure.
Thisis due to the heavy computational resources required in optimizing and evaluating neural networks.
Plenoxels [1] propose a view-dependent sparse voxel model and show comparable results to NeRF[20] and a speedup of two orders of magnitude.
Neural Geometric Level of Detail [29] uses anoctree-based feature volume and a small MLP to represent SDF.
[21] shows fast training of INR’susing a small neural network augmented by a multiresolution hash table with trainable features.
Similar to DeepSDF [23], both work used 3D supervision for learning the SDF.
Method
We consider the 3D euclidean space R 3 with points p
= ( x, y, z ) ∈ R 3 .
We discretize the unit cube
C = [0 , 1] 3 with a 3D voxel grid G = { p I } , with nodes p I indexed by I =
( i, j, k ) , i, j, k ∈ [ n ] = { 1 , . . . , n } , i.e., p I = ( x ijk , y ijk , z ijk ) .
We denote by h = n − 1 , and by N = n 3 the total number ofnodes.
We represent our reconstructed surface as a zero level of a scalar function f defined over thecube C .
f is defined by prescribing its values at the grid’s nodes f I ∈ R and trilinear interpolating ineach voxel.
We will denote by f ( p ) the interpolated value at point p .
Given an input point cloud consisting of m points q k ∈ R 3 with or without (unit norm) normalsn k ∈ R 3 , k ∈ [ m ] , our goal is to compute f so that its zero level set approximates the unknownsurface, i.e.,
[Equation] [Equation]
Our approach to compute f is to minimize a loss function of the form
[Equation]
where
[Equation]
where ∥·∥ is the standard euclidean norm in R 3 , ∇ f ( p ) ∈ R 3 is the gradient of f sampled at point p .
Note that ∇ f is defined in interior of voxels, which is generically where the input points q k resides.
L data is the standard data loss encouraging the zero level to pass through the input points q k , and itsnormals (defined by gradients of f ) to coincide with input normals n k .
The prior, L prior , is the main contribution of this work, where we combine two novel losses,
[Equation]
[Equation] Intuitively, the viscosity loss optimizes for a smooth Signed Distance Function (SDF) solutions,avoiding auxiliary bad minima of the Eikonal equation, while the coarea loss strives to minimizethe area of the zero level surface.
Our loss has 4 hyper-parameters λ p , λ n , λ v , λ c .
We provide moredetails on these priors next.
3.1 Viscosity Loss
The goal of the viscosity loss is to make f approximate an SDF over C .
Given boundary conditionsasking f to vanish on some closed compact surface S , the SDF solves the Eikonal equation PDE,i.e., ∥∇ f ( p ) ∥
= 1 , in a certain well defined sense (viscosity).
This motivated some previous work todirectly optimize the Eikonal loss [13, 28]
[Equation]
Unfortunately, the Eikonal loss has many undesirable minima whichare not SDFs.
Figure 2 shows a 1D example: both depicted solutions(denoted f ) vanish at the input points q 1 , q 2 (black points) and globallyminimize the Eikonal loss over the grid (grid points are shown in blue).
The INR works mentioned above use neural networks for representingf which injects an inductive bias avoiding these bad minima, howeveron grids, minimizing equation 5 cannot avoid these solutions.
See,e.g., middle column in Figure 1.
More classical Eikonal solvers do work with grids however use mostlyfast marching or sweeping methods [22, 27, 35, 8].
Namely, use aspecial discretization of the Eikonal equation favoring the viscositysolution of the Eikonal [26], and update node values according toa moving front [27].
Since this discretization is up-wind (will onlypropagate values in one direction) and requires choosing the maximalamong its solution, its success in adaptation to a loss is not clear.
We use a different approach to build a loss favoring SDF solutions over grids motivated by thevanishing viscosity method [11].
Namely, adding to the Eikonal PDE a small perturbation of the
Laplacian of f (denoted by ∆ f ), i.e., ∥∇ f ( p ) ∥− 1 − ϵ ∆ f ( p ) = 0 , makes the PDE semi-linear elliptic[6], and hence with suitable boundary conditions it is uniquely solvable inside S with a smoothsolution, approaching the viscosity positive distance function to the boundary as ϵ → 0 .
Similarly, for 1 − ∥∇ f ( p ) ∥
− ϵ ∆ f ( p ) =
the solution approaches the negative distance function inside thedomain.
Motivated by the vanishing viscosity principle we suggest the following viscosity loss:
[Equation]
We discretize this loss over the grid G by replacing the first order derivatives and second orderderivatives with symmetric finite differences, i.e.,
[Equation]
and similarly for D y and D z .
We use these discrete operators to approximate the gradient (cid:98) ∇ f ( p I ) =(
D x f I , D y f
I , D z f I ) and Laplacian (cid:98) ∆
[Equation]
( p [Equation] = [Equation]
+
[Equation]
f
[Equation]
+ D 2 z f I .
The discretized viscosity lossnow takes the form
[Equation]
3.2 Coarea loss
The coarea loss is approximating the area of the zero level set, and therefore incorporating it in theoptimization pushes the reconstructed surface to be economic in area.
First, simiarly to [33] we use the centered Laplace CDF
[Equation]
) to transform the SDF f to a smooth approximation of the indicator function:
[Equation]
As β → 0 , χ β converges to an indicator function leading to 1 inside S and 0 outside.
The coarea lossis defined as
[Equation]
To understand why this loss approximates the area of S we can use the coarea formula [25]:
[Equation]
where χ − 1 β ( s )
= { p | χ β ( p ) =
s } is the preimage of the value s .
Since χ x ( p ) ∈ [0 , 1] the r.h.s.
integral can be restricted to the interval [0 , 1] , and therefore the coarea loss averages the area of the level sets of χ β .
Next,
[Equation]
which shows that the level set s ∈ (0 , 1) of χ β is the level set− Ψ β − 1 ( s ) of the SDF f .
As β → 0 , − Ψ β − 1 ( s ) → 0 for alls ∈ (0 , 1) (and uniformly in ( ϵ, 1 − ϵ ) for fixed ϵ > 0 ).
Therefore theaverage of the level set area (i.e., the r.h.s.
of equation 11) convergesto the area of f − 1 (0) =
S .
Figure 1 (right) shows how removing thecoarea loss introduces an extraneous zero level set, and hence resultsin an undesired surface part.
Figure 3 shows a comparison of a reconstruction of semisphere with and without coarea.
In the experimentssection we provide more ablation tests with the coarea and viscositylosses.
To discretize the coarea loss we let w I denote the centers of grid’svoxels, and note that ∇ χ β ( w I ) =
Φ β ( − f ( w I )) ∇ f ( w I ) , where
[Equation]
is the PDF of the Laplace distribution, and ∇ f ( w I ) is computed as a linear combination of the voxel’scorner values
f I 1 , . . .
, f I 8 , see more details in the Appendix.
We end up with the discretized loss:
[Equation]
[Equation]
This loss is usually incorporated with a small hyper-parameter λ c with the purpose of eliminatingredundant surface parts.
Experiments
In this section we evaluate VisCo grids.
We provide a comparison of our method against currentmethods: Poisson Surface Reconstruction [16], DGP [31], IGR [13], SIREN [28], FFN [30], NSP [32] and PHASE [18] (Sec. 4.1) on a standard surface reconstruction benchmark [31].
We thenperform an ablation study (Sec. 4.2), and conduct a detail examination of the main components of themodel, namely the viscosity and coarea losses.
Finally, we discuss the model’s ability to reconstructsparse point clouds (Sec. 4.3) using scans from Stanford 3D Scanning Repository.
4.1 Surface reconstruction benchmark
We next evaluated our model on the Surface Reconstruction Benchmark [31].
This benchmarkconsists of 5 noisy range scans, each containing point cloud and normal data. Each scan correspondsto a challenging object with complex shape and topology.
We evaluate our method against currentstate of the art methods on this benchmark: Deep Geometric Prior (DGP) [31], Implicit Geometric Regularization (IGR) [13], SIREN [28], Fourier Feature Networks (FFN) [30], NSP [32] and PHASE [18].
We additionally compare to the classical method of Poisson Surface Reconstruction[16].
Quantitative results are summarized in Table 1.
We report the Chamfer ( d C ) and Hausdorff( d H ) distances between the reconstructed meshes and the ground-truth point clouds.
Furthermore, wereport their corresponding one sided distances ( d → H and d → C ) between the reconstructed meshes andthe input noisy point cloud.
Representative qualitative results are shown in Figure 4.
Note that weachieve comparable results to the current state-of-the-art INR methods.
4.2 Ablation study
We provide an ablation study of the main components of our model in Table 2 and Figure 5.
Specifically we compared with the following alternatives: i) Eikonal loss without the viscosityterm that prevents undesirable non-SDF solutions, i.e., ϵ = 0 in equation 6; ii) removing the coarealoss enforcing minimal surface area, i.e., λ c = 0 ; and iii) removing the normal loss, i.e., λ n = 0 .
Notethat without coarea and viscosity the reconstruction tends to have holes and discontinuities near thesurface boundaries.
Only combination of all the components results in a good surface reconstruction.
Learning with viscosity.
We further provide a more in depth discussion of the proposed viscosityloss.
Figure 6 compares reconstructions with different levels of the viscosity parameter, i.e., ϵ inequation 6.
As can be inspected from this figure, viscosity affects the smoothness of the reconstructedsurface.
For a low viscosity parameter the zero level sets become noisy.
This can be explained bythe viscosity eikonal loss (i.e., equation 6) becoming numerically too close to the eikonal loss inequation 5 and the solution deviates from the viscosity SDF solution.
This leads to artifacts across thesurface, similar to the limit case ( ϵ = 0 ) where only the eikonal loss is used, see the second columnfrom the left.
For a high viscosity parameter, and as expected with the addition of a non-vanishing
Laplacian term, the surface becomes over-smoothed.
Learning with coarea.
Similarly, we also provide an analysis of the proposed coarea loss.
In
Figure 7 we show the effect of changing the parameter weight of the coarea loss, λ c .
As can beobserved in the figure, a low coarea weight leads to the presence of excessive surface area in thereconstruction.
In contrast, a high weight will strive to minimize the surface area.
In a sense, thecoarea serves a surface tension parameter; stronger tension will ignore points, weaker tension willoverfit.
[Table] [Figure]
.3 Reconstructing from sparse point clouds
We now evaluate our model ability to reconstruct surfaces in the challenging case of sparse inputpoint clouds.
For this experiment we use point clouds from the Stanford 3D Scanning Repositoryand downsample them at different levels: 1%, 10%, and 25%.
In Figure 8 we visualize the VisCoreconstructions.
Note that VisCo can reconstruct the shapes even with sparse input.
This provides afurther validation for the proposed geometrical priors.
[Table]
Conclusions
We introduced VisCo Grids, a novel grid-based surface reconstruction approach that leverages twonovel geometrically motivated priors: viscosity and coarea.
We advocate VisCo’s
prior for gridfunctions as an alternative to the implicit inductive bias of implicit neural representations for the taskof surface reconstruction.
One important limitation of our method, shared by all grid methods, isthat its degrees of freedom, namely nodes’ location, are set a-priori.
In contrast, using non-linearfunction spaces, such as neural networks, allows a more flexible usage of the degrees of freedoms inthe model and can adjust to areas with more detail.
Nevertheless, we still believe that grid functionsand direct priors, incorporated with modern computing power, are valuable add-ons to the surfacereconstruction toolbox, providing few clear benefits over neural networks, such as a well-understoodcontrol over surface properties, instant inference time, and faster training.
Social impact.
We don’t see any immediate negative societal impact from our work.
However, weacknowledge that high quality 3D reconstruction in general can be used in malicious settings.
[-1 , λ n = 10 − 5 , λ v = 1 e − 4 , λ c = 1 e − 6 , ϵ = 
-]
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
[No] (b) Did you specify all the training details (e
g., data splits, hyperparameters, how they were chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experi- ments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]