N EURAL I MPLICIT S HAPE E DITING USING B OUNDARY S ENSITIVITY
Anonymous authors
Paper under double-blind review
A BSTRACT
Neural fields are receiving increased attention as a geometric representation due to their ability to compactly store detailed and smooth shapes and easily undergo topological changes.
Compared to classic geometry representations, however, neural representations do not allow the user to exert intuitive control over the shape.
Motivated by this, we leverage boundary sensitivity to express how perturbations in parameters move the shape boundary.
This allows to interpret the effect of each learnable parameter and study achievable deformations.
With this, we perform geometric editing : finding a parameter update which best approximates a globally prescribed deformation.
Prescribing the deformation only locally allows to deform the rest of the shape according to some prior, such as semantics or deformation rigidity.
Different to previous efforts, our method is model-agnostic and can be applied to any pre-trained NN and update it in-place.
Furthermore, we show how boundary sensitivity helps to optimize and constrain objectives (such as surface area and volume), which are difficult to compute without first converting to another representation, such as a mesh.
I NTRODUCTION
A neural field is a neural network (NN) mapping every point in a domain of interest, typically of 2 or 3 dimensions, to one or more outputs, such as a signed distance function (SDF), occupancy probability, opacity or color.
This allows to represent smooth, detailed, and watertight shapes with topological flexibility, while being compact to store compared to classic implicit representations (Davies et al., 2020).
When the NN is trained not on a single shape but instead an entire collection, each shape is encoded in a latent vector, which is an additional input to the NN (Park et al., 2019;
Chen & Zhang, 2019; Mescheder et al., 2019).
As a result, neural fields are receiving increased interest as a geometric representation in numerous applications, such as shape generation (Park et al., 2019), shape completion (Chibane et al., 2020), shape optimization (Remelli et al., 2020), scene representation (Sitzmann et al., 2020), and view synthesis (Mildenhall et al., 2020).
Some pioneering works have also investigated geometry processing, like smoothing and deformation, on neural implicit shapes (Yang et al., 2021;
Remelli et al., 2020; Mehta et al., 2022; Guillard et al., 2021), but these can be computationally costly or resort to intermediate mesh representations.
In part, this difficulty stems from the shape being available only implicitly as the sub-level set of the field.
While classic explicit or parametric representations (like meshes, splines, or subdivision schemes) often have intuitive (often synonymous with local) geometric control as a key design principle, it is not trivial to edit even classic implicit representations, especially ones with global functions (Bærentzen & Christensen, 2002).
Previous works on neural implicit shape editing have focused on the shape semantics , i.e. changing part-level features based on the whole shape structure, but achieve this through tailored training procedures or architectures.
We instead propose a framework which unifies geometric and semantic editing and which is agnostic to the model and its training and modifies the given model in-place akin to classic representations.
To treat the geometry, not the field, as the primary object we consider boundary sensitivity to relate changes in the function parameters and the implicit shape.
This allows us to express and interpret a basis for the displacement space.
In this framework, the user supplies a target displacement on (a part) of the shape boundary in the form of deformation vectors or normal displacements at a set of surface points. Employing boundary sensitivity we find the parameter update which best approximates the prescribed deformation.
In geometric editing, we prescribe an exact geometric update on the entirety of the boundary.
Akin to local control in classic representations, we especially study the case where the prescribed displacement is local and the rest of the boundary is fixed.
In semantic editing only a part of the boundary is prescribed a target displacement.
The remaining unconstrained displacement is determined by leveraging the generalization capability of the model as an additional prior, producing semantically consistent results on the totality of the shape.
Another prior often used in shape editing is based on deformation energy, such as as-rigid-as-possible (Sorkine & Alexa, 2007) or as-Killing-as-possible (Solomon et al., 2011), which generates physically plausible deformations minimizing stretch and bending.
This is not trivially applicable to implicit surfaces, as there is no natural notion of stretch due to ambiguity in tangent directions.
We discuss a few options to resolve this ambiguity and demonstrate that boundary sensitivity can be leveraged to optimize directly in the space of expressible deformations.
Lastly, we use level-set theory and boundary sensitivity to constrain a class of functionals, which are difficult to compute without first converting to another representation, such as a mesh.
As a specific use-case, we consider fixing the volume of a shape to prevent shrinkage during smoothing.
R ELATED W ORK
Implicit Shape Representations and Manipulation Implicit shape representations or level-sets have been widely used in fields such as computer simulation (Sethian & Smereka, 2003), shape optimization (van Dijk et al., 2013), imaging, vision, and graphics (Vese, 2003; Tsai & Osher, 2003).
Classically, an implicit function is represented as a linear combination of either a few global basis functions, such as in blobby models (Muraki, 1991), or many local basis functions supported on a (potentially adaptive) volumetric grid (Whitaker, 2002).
While few global bases use less memory, the task of expressing local displacements is generally ill-posed (Whitaker, 2002).
Hence, methods for interactive editing of implicit shapes are formulated for grid-supported level-sets (Museth et al., 2002; Bærentzen & Christensen, 2002).
These methods use a prescribed velocity field, for which the level-set-equation – a partial differential equation (PDE) modelling the evolution the surface – is solved using numerical schemes on the discrete spatiotemporal grid.
Neural Fields In neural fields , a NN is used to represent an implicit function.
Different from classic implicit representations, these are non-linear and circumvent the memory-expressivity dilemma
Davies et al.
In addition, automatic differentiation also provides easy access to differential surface properties, such as normals and curvatures, useful for smoothing and deformation (Yang et al., 2021; Mehta et al., 2022; Atzmon et al., 2021) or more exact shape fitting (Novello et al., 2022).
Early works propose to use vanilla multilayer perceptrons (MLPs) to learn occupancy probability (Mescheder et al., 2019; Chen & Zhang, 2019) or signed distance functions (Park et al., 2019; Atzmon & Lipman, 2020), whose level-sets define the shape boundary.
Conditioning the NN on a latent code as an additional input allows to decode a collection of shapes with a single NN (Chen & Zhang, 2019;
Mescheder et al., 2019; Park et al., 2019).
Later works build upon these constructions by introducing a spatial structure on the latent codes using (potentially adaptive) grids, affording more spatial control when generating novel shapes (Ibing et al., 2021) and allowing to reconstruct more complex shapes and scenes (Jiang et al., 2020; Peng et al., 2020; Chibane et al., 2020). In this work, we develop a method to interactively modify shapes generated by any of these methods.
Neural Shape Manipulation Although there are many previous works on deformation of shapes with NNs, we focus only on methods that use neural implicit representations.
These can roughly be sorted into two groups based on their guiding principle.
Methods in the first group manipulate shapes based on a semantic principle.
Hertz et al.
(2022) create a generative framework with part-level control consisting of three NNs decomposing and mixing shapes in the latent space.
Elsner et al.
(2021) encourage the latent code to act as geometric control points, allowing to manipulate the geometry by moving the control points.
Chen et al.
(2021) demonstrate how to interpolate between shapes, while balancing their semantics by choosing which layer’s features to track.
Similar to our method, this is agnostic to the training and the model.
Hao et al.
(2020) learn a joint latent space between an SDFs and its coarse approximation in the form of a union of spherical primitives.
This way modifications of the spheres can be translated to the best matching change of the high-fidelity shape.
In our work, we allow a similar editing approach, but through insights from boundary sensitivity, we are able to apply such manipulation to arbitrary NNs
Methods in the second group are based on some well defined energy .
Yang et al.
(2021) demonstrate shape smoothing using a curvature-based loss.
Similarly, shape deformation is performed by optimizing the deformation energy for which another invertible correspondence NN is needed.
Niemeyer et al.
(2019) train an additional NN as a spatio-temporally continuous velocity field, along which the points of a shape are integrated.
Boundary Sensitivity Boundary sensitivity has already been introduced in this context in several previous works.
Atzmon et al.
(2019) use it to translate geometric losses defined on points sampled from a classifier or SDF level-sets to parameter updates during training.
Atzmon et al.
(2021) use approximate Killing vector fields (AKVFs) and boundary sensitivity during training to encourage latent space interpolation to act as physically plausible deformation.
Neural implicit shape manipulation has also been achieved by leveraging meshes as an intermediate representation to benefit from the well studied geometry processing operations on those.
Remelli et al.
(2020) propose differentiable mesh extraction to then propagate gradients from a differentiable operation on the mesh through to the implicit NN.
Mehta et al.
(2022) studies this link in more detail using classic theory of level-sets, using boundary sensitivity to translate several mesh-based operations to SDF updates. Guillard et al. (2021) build upon differentiable mesh extraction for sketch-based editing by translating an image-based error to shape updates.
We do not use boundary sensitivity during training, being in fact agnostic to the training procedure, but instead modify already existing models. While our method also generalizes to mesh-based operations, we focus on requiring only points due to simplicity. We further show how to restrict the editing process by introducing explicit constraints or by using the NN’s semantic prior.
B OUNDARY S ENSITIVITY
Let f be a differentiable function mapping a spatial coordinate x ∈ D ⊂ R D to a scalar f ( x ; Θ ) ∈ R on a domain of interest D .
Let the vector Θ ∈ R P gather the parameters of f .
For a given Θ , the sign of f implicitly defines the shape Ω( Θ ) =
{ x ∈ D| f ( x ; Θ ) ≤ 0 } and its boundary Γ( Θ ) = { x ∈ D| f ( x ; Θ ) = 0 } .
Consider the variation of Ω( Θ ) by a displacement field (also known as velocity field )
δ x : D (cid:55)→ R D , denoted by Ω δ x ( Θ ) =
{ x + δ x ( x ) ∈
D| f ( x + δ x ( x ); Θ ) ≤ 0 } .
If δ x is sufficiently small, the initial and the perturbed shapes are diffeomorphic (Allaire et al., 2004), meaning there is a one-toone correspondence between the points of both.
Consider the displacement field δ x induced by a sufficiently small parameter perturbation δ Θ and the perturbed shape Ω δ x ( Θ )
=
Ω( Θ + δ Θ ) .
To compute δ x , consider the total derivative of the boundary condition f
( x ; Θ ) = 0 :
[Equation]
We can replace the infinitesimal increments d x and d Θ with sufficiently small perturbations ∇ x f ⊤ δ x + ∇ Θ
f ⊤ δ Θ = 0 .
Assuming bounded gradients ∇ x f = ∇ x f ( x ; Θ ) ∈
R D and ∇ Θ f = ∇ Θ f ( x ; Θ ) ∈ R P allows to express δ x = δ x ( x ; Θ , δ Θ ) as
[Equation]
This is what we refer to as boundary sensitivity – the movement of the boundary δ x caused by the parameter perturbations δ Θ .
Equation 2 consists of normal and tangent components δ x = δ x n + δ x t = n [Equation] n + t δx t .
With n = ∇ x f/ ∥∇ x f ∥ the outward facing normal, we rewrite the normal part as the weighted sum of the basis b =
b ( x ; Θ ) ∈ [Equation] P
[Equation]
For each positive parameter perturbation δ Θ p , p ∈ { 1 ..P } , a positive gradient ∂f/∂ Θ p implies that the boundary moves inward along the normal, since the value of f at x increases.
The total movement caused by all parameter perturbations is their superposition.
We show some basis functions in Figures 2 and 3.
The tangential component δ x t is ambiguous since per definition ∇ x f ⊤ δ x t = 0 .
This is transport of the surface along itself which has no effect on the implicit representation.
However, this ambiguity can be solved to recover δ x t based on an additional assumption.
In the context of classic implicit representations two such assumptions considered are that normals of points do not change during the deformation (Jos & Schmidt, 2011) and that they are nearly-isometric (Tao et al., 2016).
Editing Let δ ¯ x : ¯Γ (cid:55)→ R D be a prescribed deformation on (a part of) the boundary ¯Γ ⊆ Γ .
In practice, this deformation will be provided as a finite set of surface vectors.
E C = (cid:82) ¯Γ ||
δ ¯ x − δ x
||
d s is the energy associated with the deviation from the prescribed displacement.
The task in editing is to find a parameter update δ Θ inducing the displacement δ x which minimizes this energy.
Since a parameter update can cause movement only in the normal direction, it can only approximate the normal component of the target.
[Equation]
In practice, we estimate this integral with discrete samples at locations { x i } i =
I ⊂ ¯Γ :
[Equation]
= argmin
[Equation]
This is a linear-least-squares problem B δ Θ = δ ¯ y with B =
[ b ( x 1 ) | · · · | b ( x I )]
⊤ ∈ R I × P and δ ¯ y =
[ δ ¯ x n ( x 1 ) , · · · , δ ¯ x n ( x I )] ∈ R I .
To improve the numerical stability one often uses Tikhonov regularization which penalizes the norm of the solution min δ Θ E C + λ ∥ δ Θ ∥ 2 for some small positive regularization constant λ .
In our setting,
Tikhonov regularization serves an additional purpose: small δ Θ are necessary for the valididity of the linear expansion in Equation 1.
Furthermore, especially in semantic editing, we might sample less points than parameters I < N , which would lead to an underdetermined system if regularization is not used.
Lastly, regularization can also control how similar the final shape is to the source shape, as indicated in Figure 7.
Target Deformation
We sample points x i on the boundary ¯Γ via iterative rejection sampling on the domain of interest or near the farthest point samples of the existing points.
This is a sufficiently efficient method for our needs, although more advanced methods exist (Hart et al., 2002).
Target deformations are then prescribed on the sampled points.
If these are not given as the magnitude along the normal deformation δ ¯ x n , but as a vector δ ¯ x , we project them
δ ¯ x n = n ⊤ δ ¯ x .
With partially prescribed targets, one must be careful about the target being unintentionally restrictive if the normals at the sampled points are nearly orthogonal to the target vector.
This can be remedied by additionally filtering the points based on their normals.
Large Displacements Despite the boundary sensitivity in Equation 1 being valid only for small displacements due to the assumption of locally constant gradients, large displacements can be achieved via several iterations.
The initially sampled surface points are moved by the computed δ x in order to obtain the samples of the next iteration.
To obtain the respective target deformations we split the initially user-provided target (either scalar along normal or vector) into equal parts.
If the target is a vector, for each iteration we project the divided target vector onto the current normal. In the demonstrated geometric and semantic editing applications a small number of iterations (<15) is sufficient. Note, that the number of iterations will in general scale with the extent of the desired deformation. Constraints Furthermore, we demonstrate the use of boundary sensitivity to fix a value of a functional during parameter updates.
If the functional is expressed as a surface or volume integral, this can be done without computing the integrals themselves.
An example of this is constant area, which can simulate the behaviour of unstretchable, but bendable materials, such as rope in 2D or textile in 3D. Similarly, volume preservation is characteristic to incompressible materials (Desbrun &
Gascuel, 1995) and is desirable in smoothing to prevent shrinkage (Taubin, 1995).
To this end, we loosely introduce shape derivatives and refer to Allaire et al.
(2021) for more detail.
H ′ (Ω)( δ x ) denotes a shape derivative of the functional H (Ω) ∈ R if the expansion H (Ω δ x ) =
H (Ω) + H ′ (Ω)( δ x ) holds for small δ x .
We rewrite this using perturbations as δH (Ω)( δ x ) =
H (Ω δ x ) − H
(Ω) .
For several functionals the shape derivatives are known analytically.
For the functional H (Ω) = (cid:82) Ω h d x defined as a volume integral of h = h ( x ) ∈
R the shape derivative is
[Equation]
where the last equality is due to the divergence theorem on a bounded and Lipschitz domain Ω .
Inserting the boundary sensitivity from Equation 2 we arrive at
[Equation]
where b H := (cid:82) Γ h ( x ) b ( x ) d x ∈ R P again acts as a basis, but for the perturbed integral quantity.
{+δH = 0 can now be enforced either as soft constraint or as a hard constraint by projecting any parameter update δ Θ onto the R P − 1 linear subspace where the integral stays constant δ Θ δH =
+}
= ( I − b H b ⊤ H ) δ Θ .
Analogously, for a functional G (Ω) =
(cid:82) Γ g d x defined as a surface integral of g = g ( x ) ∈ R the shape derivative as a perturbation is δG (Ω)( δ x ) =
(cid:82) Γ ( ∂g/∂ n + κg ) δ x ⊤ n d x , where ∂g/∂ n is the directional derivative of g along n and κ = κ ( x ) = div( n ( x ))
∈ R is the mean-curvature.
The corresponding basis for δG =
b ⊤ G δ Θ is b G =
(cid:82) Γ ( ∂g/∂ n + κg )
b d x .
A PPLICATIONS
Having established the framework for editing implicit shapes through boundary sensitivity, we consider its applications.
We emphasize geometric and semantic editing since our framework unifies both, while being agnostic to the architecture and training of the model.
We also address deformation-rigidity-based editing since it classically is a widely used approach to shape editing.
Lastly, we demonstrate the introduction of constraints on shape functionals on the example of volume preservation during smoothing.
In all cases, we include Tikhonov regularization with λ = 0 .
1 (see Appendix A), but note that the regularization behaviour depends on the number of points I and the magnitude of the prescribed displacement and in turn influences the number of iterations required and the approximation residual.
4. G EOMETRIC E DITING
In geometric editing, the displacement is prescribed on the entirety of the boundary.
However, we focus on studying the case where the prescribed displacement is local and the rest of the boundary is fixed, akin to local control in classic representations.
Deformation in Section 4.3 and smoothing in
Section 4.4 are indirect examples of the target displacement being global.
For each considered shape, we train a separate network to fit the value and surface normals of the SDF. All networks share the same architecture: sin activation functions and 3 hidden layers ofneurons each.
In total, there are P = 2273 learnable parameters, all of which are manipulated during editing.
We demonstrate several examples of geometric editing in Figure 1 where we displace parts of both man-made and organic neural implicit shapes.
When the displacements are, loosely speaking, natural to the shape, the approximations recover the target well.
However, not arbitrarily complex displacements can be recovered as illustrated by the inscribed letters.
The characteristic length of the detailed target is much finer than that of the shape, especially in the relevant region.
We hypothesize that the good memory-expressiveness trade-off of neural fields requires the NN to allocate geometric complexity where it is needed during training.
To illustrate this, some basis functions are shown in Figure 2.
As can be seen, their characteristic length is of a similar size as the shape. As all possible edits are a combination of these basis functions, it is unlikely that modifications in a much smaller scale can be faithfully reconstructed.
4. S EMANTIC E DITING
In representation learning the aim is to explain observations with a small number of latent variables (Bengio et al., 2013).
Generative models, such as (variational) auto-encoders or generative adversarial networks, attempt to map novel latent codes to novel outputs within the same distribution as the observations.
When interpolating between two latent codes, the appearance of the generated output changes continuously (Shen et al., 2020a).
We investigate the basis of the latent space as described by Equation 3, illustrating some of these basis functions for the decoder of the IM-Net architecture (Chen & Zhang, 2019) in Figure 3.
The decoder is trained as part of an auto-encoder reconstructing the entire ShapeNet dataset (Chang et al., 2015) and is implemented as a MLP with 8 layers of 1024 neurons each and leaky-ReLU activations. As input it receives in addition to the three spatial coordinates a l ∈ R 256 latent code, representing the individual shapes.
We use a pretrained model available at https://github.com/czq142857/IM-NET-pytorch .
These basis functions can be observed to encode semantics, such as rotation and reflection symmetries, and segment semantically meaningful parts, such as windscreen on the car, rim of the lamp and different surfaces on chairs.
For similar parts, such as the two chairs, the bases show similar behaviour.
However, it is known that not all directions in the latent space are (equally) viable (Chen et al., 2022; Vyas et al., 2021), hence, a similar argument can be made for the bases.
Furthermore, while each basis function might cause meaningful deformation, they are not necessarily disentangled, i.e. each basis function does not explain a single generative factor, unless latent disentanglement is used in training (Bengio et al., 2013; Shen et al., 2020b; Tschannen et al., 2018), which would further increase their interpretability.
Despite this, with our method we can intuitively traverse the latent space by considering the link between geometric and latent code changes.
We present several examples in Figure 4, where the highlighted areas ¯Γ of an initial shape are prescribed a local, high-level displacement, such as shortening a leg of a chair or squeezing the sides of the boat.
We sample roughly 100 points on these areas, prescribe the same target vector at each point, and leave the remaining boundary unconstrained.
After projecting the target onto the current normal and finding the best fit parameter update according to Equation 5, we repeat this process for a few ( < 15 ) iterations to achieve visually obvious changes.
User input is provided only at the beginning.
Note that we only consider the latent parameters for our basis and leave all network parameters unchanged.
Despite only prescribing local geometric deformation, we observe global and semantically consistent changes.
Not only are obvious symmetries preserved, but also the morphology of the shape can change significantly, such as with the boat.
Compared to geometric editing, semantic editing has the computational advantage of sampling points on just a part of the boundary and the number of latent parameters being much smaller than number of NN parameters, leading to a very small least-squares system.
Per iteration The method only requires a single forward- and backward-pass through the model, altogether being fit for interactive use.
4. R IGID E DITING
As a last approach to editing, we briefly address deformation-rigidity since it classically is a widely used approach to shape editing.
On the one hand, deformation energy is one of the many alternative priors to semantic prior discussed previously.
A simple approach is to consider pure bending energy, since it can be computed only from the implicit representation itself.
On the other hand, typical applications, such as as-rigid-as-possible (Sorkine & Alexa, 2007) or as-Killing-as-possible (Solomon et al., 2011), also include the stretching energy.
This is not trivially applicable to implicit surfaces, as there is no natural notion of stretch due to ambiguity in the tangent directions.
As discussed in Section 3, there are several choices for recovering tangential displacements, but we consider perhaps the simplest one – using another independent function to provide the tangential component δ x t ( x ; Ξ) = (cid:0) I − n ( x ) n ( x ) ⊤ (cid:1) f t ( x ; Ξ) as the tangential projection of a sufficiently smooth NN f t ( x ; Ξ) : R 3 (cid:55)→ R 3 parameterized in Ξ .
We leverage Killing vector fields (KVFs) which generate isometric deformations.
δ x is a KVF if it has an anti-symmetric Jacobian J ( δ x )
=
J ( δ x )( x ) ∈ R 3 × 3 everywhere on the boundary
J ( δ x ) + J ( δ x ) ⊤ = 0 ∀ x ∈ Γ (Solomon et al., 2011).
A vector fields’ deviation from being Killing can be measured with its Killing energy
[Equation]
Classic works compute the Jacobians using discrete operators on spatial discretizations while Atzmon et al.
(2021) formulates AKVF on a neural implicit directly.
We follow this approach and seek an
AKVF respecting the prescribed boundary deformations δ ¯ x ( x i ) weighted by some small α >
[Equation]
Boundary sensitivity allows to search for the energy minimizing deformation directly in the space of expressible deformations and directly optimize the normal component over δ Θ using Equation 2.
The E C in Equation 4 is expressed trivially but can now also accommodate the tangential component.
To express E K , we can leverage J being a linear operator J ( δ x ) =
J ( δ x n ) + J ( δ x t ) .
Here
J ( δ x n ) =
J ( b ⊤ δ Θ ) =
J ( b ⊤ ) δ Θ with J ( b ⊤ ) ∈ R 3 × 3 × P and J ( δ x t ) =
J ( f t ) straight-forward.
Figure 5 illustrates the results of AKVF deformation.
4. V OLUME P RESERVING S MOOTHING
As an example for fixing the value of a surface or volume integral, we demonstrate the simple case of fixing the volume itself while smoothing the shape.
Fixing the volume of an implicit shape is difficult without going to an intermediate representation, such as a mesh (Remelli et al., 2020; Mehta et al., 2022), since there is no trivial way to differentiably compute the volume [-δ Θ δV =
 = ( I − b V b ⊤ V ) δ Θ .-]of an implicit shape.
Smoothing on neural fields has been previously formulated using gradient descent on an objective penalizing the deviation from a specified mean-curvature (Yang et al., 2021) and as mean-curvature flow computed on an intermediate mesh (Mehta et al., 2022).
We also use mean-curvature flow without resorting to the intermediate mesh, but the proposed volume-constraint is agnostic to the choice of the smoothing method.
For that, we first compute the mean curvature by expanding κ ( x ) = div( n ( x )) .
From there, we can reuse the geometric editing framework with the globally prescribed target displacement δx n ( x ) = − κ ( x ) ∀ x ∈ ¯Γ = Γ .
Note, that this approach induces the correct geometric flow as discussed by Mehta et al. (2022).
To fix the volume V while smoothing, we simply set h
= 1 in the bases introduced in Equation 7.
We enforce volume preservation by projecting the parameter update onto the isochronic subspace.
Figure 6 compares smoothing with and without this constraint.
The volume after 86 iterations decreases by 1 . 6% with the constraint and 80% without.
Note, that without a volume constraint the shape ultimately converges to a singular point.
C ONCLUSIONS
With implicit neural shapes becoming a widespread representation, we have demonstrated a unifying approach to perform geometric and semantic editing without the need for tailored training or architectures, while being simple to implement, and, especially in the case of semantic editing, fast and fit for interactive use.
While we touched upon optimizing directly in the deformation space with a rigidity-prior, using other priors for unconstrained and tangential deformations remains an interesting problem.
We used signed-distance fields and occupation probabilities, but the editing framework can be also extend to other neural fields, where NeRFs especially provide tantalizing options.
We hope that formulating a basis for the deformation space allows future work to further study and build models with desirable properties, such as interpretability, tailored degrees-of-freedom, linear-independence, compactness or use them for segmentation or symmetry detection.
A E FFECT OF T IKHONOV REGULARIZATION ON SEMANTIC EDITING