Neural Implicit Shape Editing using Boundary Sensitivity
Authors: Arturs Berzins
Moritz Ibing
Leif Kobbelt
Venue: ICLR 2023 poster (2023)

Abstract: Neural fields are receiving increased attention as a geometric representation due to their ability to compactly store detailed and smooth shapes and easily undergo topological changes. Compared to classic geometry representations, however, neural representations do not allow the user to exert intuitive control over the shape. Motivated by this, we leverage boundary sensitivity to express how perturbations in parameters move the shape boundary. This allows us to interpret the effect of each learnable parameter and study achievable deformations. With this, we perform geometric editing: finding a parameter update that best approximates a globally prescribed deformation. Prescribing the deformation only locally allows the rest of the shape to change according to some prior, such as semantics or deformation rigidity. Our method is agnostic to the model and its training and updates the NN in-place. Furthermore, we show how boundary sensitivity helps to optimize and constrain objectives (such as surface area and volume), which are difficult to compute without first converting to another representation, such as a mesh.

Published as a conference paper at ICLR 2023
NEURAL IMPLICIT SHAPE EDITING USING BOUNDARY SENSITIVITY
Arturs Berzins
Moritz Ibing & Leif Kobbelt Visual Computing Institute RWTH Aachen University
Department of Mathematics and Cybernetics
SINTEF
arturs www.weessssntee.....
ABSTRACT
Neural fields are receiving increased attention as a geometric representation due their ability compactly store detailed and smooth shapes and easily undergo topological changes. Compared to classic geometry representations, however, neural representations do allow the user to exert intuitive control over the shape. Motivated by this, we leverage boundary sensitivity to express how perturbations in parameters move the shape boundary. This allows to interpret the effect of each learnable parameter and study achievable deformations. With this, we perform geometric editing: finding a parameter update that best approximates globally prescribed deformation. Prescribing the deformation only locally allows the rest the shape to change according to some prior, such as semantics or deformation rigidity. Our method is agnostic to the model its training and updates the NN in in-place. Furthermore, we show how boundary sensitivity helps to optimize and constrain objectives (such surface area and volume), which difficult compute without first converting to another representation, such as a mesh.
INTRODUCTION
neural fiel is neural network (NN) mapping every point in a domain of interest, typically of 2 or 3 dimensions, to one or more outputs, such signed distance function (SDF), occupancy probability, opacity or color. This allows represent smooth, detailed, and watertight shapes with topological flexibility, while being compact to store compared to classic implicit representations (Davies et al., 2020). When the NN is trained not on a single shape but instead an entire collection, each shape is encoded in latent vector, which is additional input to the NN (Park et al., 2019; Chen & Zhang, 2019; Mescheder et al., 2019). As a result, neural fields are receiving increased interest geometric representation in numerous applications, such as shape generation (Park et al., 2019), shape completion (Chibane et al., 2020), shape optimization (Remelli et al., 2020), scene representation (Sitzmann et al., 2020), and view synthesis (Mildenhall et al., 2020). Some pioneering works have also investigated geometry processing, like smoothing and deformation, on neural implicit shapes (Yang et al.. 2021: Remelli al., 2020: Meeht al., 2022: Guillard al., 2021), but these can be computationally costly or resort to intermediate mesh representations. In part, this difficulty stems from the shape being available only implicitly the sevelevel set of the field.
While intuitive (often synonymous with local) geometric control is a key design principle of classic explicit or parametric representations (like meshes, splines, or subdivision schemes), it is not trivial to edit even classic implicit representations, especially ones with global functions (Bærentzen & Christensen, 2002). Previous works on neural implicit shape editing have focused on the shape semantics, i.c. changing part-level features based on the whole shape structure, but achieve this through tailored training procedures or architectures or resort to intermediate mesh representations.
We instead propose framework which unifies geometric and semantic editing and which is agnostic to the model and its training and modifies the given model in in-place akin to classic representations. To treat the geometry, not the field, as the primary object consider boundary sensitivity to relate changes in the function parameters and the implicit shape. This allows us to express and interpret a basis for the displacement space.

---

Published as a conference paper at ICLR 2023
In this framework, the user supplies a target displacement (a part) of the shape boundary in the form of deformation vectors or normal displacements at a set of surface points. Employing boundary sensitivity we find the parameter update which best approximates the prescribed deformation. In geometric editing, we prescribe an exact geometric update on the entirety of the boundary. Akin to especially study the case where the prescribed displacement
local control in classic representations, is local and the rest of the boundary is fixed. In semantic editing only a part of the boundary is prescribed a target displacement. The remaining unconstrained displacement is determined by leveraging the generalization capability of the model as an additional prior, producing semantically consistent results on the totality of the shape. Another prior often used in shape editing is based on deformation energy, such as as-rigid-as-possibel (Sorkine & Alexa, 2007) or as- Kkilling-as-possiiee (Solomon et al., 201 1), which generates physically plausible deformations minimizing stretch and bending. This is not trivially applicable to implicit surfaces, as there is no natural notion of stretch due to ambiguity in tangent directions. We discuss a few options to resolve this ambiguity and demonstrate that boundary sensitivity can be leveraged to optimize directly in the space of expressible deformations. Lastly, we use level-set theory and boundary sensitivity to constrain a class of functionals, which are difficult to compute without first converting to another representation, such as mesh. As a specific use-case, we consider fixing the volume of a shape to prevent shrinkage during smoothing.
2 RELATED WORK
Implicit Shape Representations and Manipulation Implicit shape representations or level-sets have been widely used in fields such as computer simulation (Sethian & Smereka, 2003), shape optimization (van Dijk et al., 2013), imaging. vision, and graphics (Vese, 2003; Tsai & Osher, 2003). Classically, an implicit function is represented as a linear combination of either a few global basis functions, such in blobby models (Muraki, 1991), or many local basis functions supported (potentially adaptive) volumetric grid (Whitaker, 2002). While few global bases use less memory, the task of expressing local displacements is generally ill-posed (Whitaker, 2002). Hence, methods for interactive editing of implicit shapes are formulated for grid-supported level-sets (Museth et al., 2002; Bærentzen & Christensen, 2002). These methods use prescribed velocity field, for which the level-set-equation - a partial differential equation (PDE) modelling the evolution the surface - is solved using numerical schemes the discrete spatiotemporal grid.
Neural Fields In neural fields, a NN is used to represent an implicit function. Different from classic implicit representations, these are non-lineat and circumvent the memoory-xpressivviyy dilemma Davies et al. (2020). In addition, automatic differentiation also provides easy access to differential surface properties, such as normals and curvatures, useful for smoothing and deformation (Yang et al., 2021; Mehta al., 2022; Atzmon al., 2021) more exact shape fitting (Novello al., 2022). Early works propose to use vanilla multilayer perceptrons (MLPs) to learn occupancy probability (Mescheder et al., 2019; Chen & Zhang, 2019) or signed distance functions (Park et al., 2019; Atzmon & Lipman, 2020), whose level-sets define the shape boundary. Conditioning the NN on a latent code as an additional input allows to decode a collection of shapes with a single NN (Chen & Zhang, 2019; Mescheder et al., 2019; Park et al., 2019). Later works build upon these constructions by introducing a spatial structure on the latent codes using (potentially adaptive) grids, which affords more spatial control when generating novel shapes (lbing et al., 2021) and allows to reconstruct more complex shapes and scenes (Jiang et al., 2020; Peng et al., 2020; Chibane et al., 2020). In this work, we develop a method to interactively modify shapes generated by any of these methods.
Neural Shape Manipulation Although there are many previous works on the deformation of shapes with NNs, we focus only on methods that use neural implicit representations. These can roughly be sorted into two groups based on their guiding principle. Methods in the first group manipulate shapes based on a semantic principle. Hertz et al. (2022) create generative framework with leve- control consisting of three NNs decomposing and mixing shapes in the latent space. Elsner et al. (2021) encourage the latent code to act as geometric control points, allowing to manipulate the geometry by moving the control points. Chen et al. (2021) demonstrate how to interpolate between shapes while balancing their semantics by choosing which layer's 's features to track. Similar to our method, this is agnostic to the training and the model. Hao et al. (2020) learn joint latent space between an SDFs and its coarse approximation in the form of a union of spherical primitives. This way modifications of the spheres can be translated to the best matching change of the high-fidelit shape.
2

---

Published as a conference paper at ICLR 2023
In our work, we allow a similar editing approach, but through insights from boundary sensitivity, are able to apply such manipulation to arbitrary NNs.
Methods in the second group based some well defined energy. Yang et al. (2021) demonstrate shape smoothing using a curvature- loss. Similarly, shape deformation is performed by optimiz- ing the deformation energy for which another invertible correspondence NN is needed. Niemeyer et al. (2019) train an additional NN as a spatio-temporally continuous velocity field, along which the points of shape are integrated.
Boundary Sensitivity Boundary sensitivity has already been introduced in the context of implicit neural shapes in several previous works. Atzmon al. (2019) use translate geometric losses defined points sampled from classifier or SDF level-sets to parameter updates during training. Atzmon et al. (2021) use approximate Killing vector fields (AKVEs) and boundary sensitivity during training encourage latent space interpolation to act physically plausible deformation. Neural implicit shape manipulation has also been achieved by leveraging meshes as an intermediate representation to benefit from the well studied geometry processing operations those. Remelli et al. (2020) propose differentiable mesh extraction to then propagate gradients from a differentiable operation on the mesh through the implicit NN. Mehta et al. (2022) study this link in more detail using classic theory of level-sets, using boundary sensitivity to translate several mesh-based operations SDF updates. Guillard al. (2021) builds upon differentiable mesh extraction for editing by translating an image- maage-based error to shape updates. Similar to our work, Sketch2Mesh uses boundary sensitivity only for the editing, being model and training agnostic. While our method trivially generalizes to meshes, we require only points sampled from the surface. We further show how restrict the editing process by introducing explicit constraints or by using the NN's semantic prior.
3 BOUNDARY SENSITIVITY
Let f be differentiable function mapping spatial coordinate x D R to scalar f(x; θ) R on a domain of interest 'D. Let the vector θ E R.. gather the parameters of f. For a given O, the sign of f implicitly defines the shape ?((@) {x E D) +) 0} and its boundary 03.
r(0) = (x E
Consider the variation of (2@) by a displacement field (also known velocity field) ix D R.D. denoted by ( {x + 6x(x) E Dx (x()):+) ) S 0}. If ōx is sufficiently small, the initial and the perturbed shapes are diffeomorphic (Allaire al., 2004), meaning there is a one-to- one correspondence between the points of both. Consider the displacement field 8x induced by sufficiently small parameter perturbation 80 and the perturbed shape Ωgx (+) = ?(( 86). To
compute ix, consider the total derivative of the boundary condition f(x; + 0:
df(x: θ) Vx dx Vol" de v
We can replace the infinitesimal increments dx and de with sufficiently small perturbations
Vxf 8x + Vef T0 O. Assuming bounded gradients Vxf = Vxf(x; θ) E R and Vef vof(x: R allows to express dx 5x(x;; θ 80)
dx -V /V/ TO
+ ox,
?
This is what we refer boundary sensitivity the movement of the boundary 8x caused by the parameter perturbations d0. Equation consists of normal and tangent components 8x = ox, + ôx, nôœ, + tôæg, With n Vxf V the outward facing normal, we rewrite the normal part as the weighted sum of the basis b = b(x; θ) E R.
Vef
6x = b T 50
b
(3)
|
For each positive parameter perturbation 80,,. E (1..P). positive gradient Of 00/ implics that the boundary moves inward along the normal, since the value of increases. The total movement
3

---

might sample Published as a conference paper at ICLR 2023
caused by all parameter perturbations is their superposition. We show some basis functions in Figures 2 and
The tangential component ôx, is ambiguous since per definition Vxf ox, O. This is transport of the surface along itself which has no effect on the implicit representation. However, this ambiguity can be resolved recover ox₃ based representations two such assumptions considered are that normals of points do not change during the deformation (Jos & Schmidt, 201 I) and that they are nearly-isometric (Tao et al., 2016).
additional assumption. In the context of classic implicit
Editing Let ox : I + IR be a prescribed displacement on (a part of) the boundary I' S r. Ec = Jr. ||xx - | ds is the energy associated with the deviation from the prescribed displacement. The task in editing is to find a parameter update . inducing the displacement d' which minimizes this energy. Since a parameter update can cause movement only in the normal direction, it can only approximate the normal component of the target.
J 166ā ôor 8.. 8eri 112 cs
80 argmin Ee argmin 86
.6
(4)
/ [165an - b 8011 do /1065-- 6x
argmin s
In practice, the displacement is provided as a finite set of surface vectors at the locations {x:)==1.1 C and the integral is estimated with the sum:
i
argmin Σ E1ld0n((x)) b(xx) T80112.
5
This mnear-least-squares problem BJO 5y with B (x:)(x:)] E R
[b [x:)] E RXP and dy
To improve the numerical stability one often uses Tikhonov regularization which penalizes the norm of the solution minse Ec + X||d@|| 2 for some small positive regularization constant X. In our setting, Tikhonov regularization serves an additional purpose: small 8e are necessary for the valididity of the linear expansion in Equation 1. Furthermore, especially in semantic editing, less points than parameters < P, which would lead to an underdetermined system if regularization is not used. Lastly, regularization can also control how similar the final shape to the source shape, as indicated in Figure 9.
Target Deformation We sample points x; on the boundary I' via iterative rejection sampling on the domain of interest or near the farthest point samples of the existing points. This is sufficiently efficient method for our needs, although more advanced methods exist (Hart al., 2002). Target deformations are then prescribed on the sampled points. If these are not given as the magnitude along the normal deformation dæ,, but as a vector dx, we project them Sz, = 8x. With partially prescribed targets, one must be careful about the target being unintentionally restrictive if the normals at the sampled points are nearly orthogonal to the target vector. This can remedied by additionally filtering the points based their normals.
Large Displacements Despite the boundary sensitivity in Equation 1 being valid only for small displacements due the assumption of locally constant gradients, large displacements can be achieved via several iterations. The initially sampled surface points are moved by the computed 6x in order obtain the samples of the next iteration. To obtain the respective target deformations we split the initially user-prov -rovided target (either scalar along normal or vector) into equal parts. If the target vector, for each iteration we project the divided target vector onto the current normal. In the demonstrated geometric and semantic editing applications a small number of iterations (<15) is sufficient. Note, that the number of iterations will in general scale with the magnitude of the desired deformation.
Constraints Furthermore, we demonstrate the use of boundary sensitivity to fix a value a functional during parameter updates. If the functional is expressed surface volume integral,
4

---

Published as a conference paper at ICLR 2023
this can be done without computing the integrals themselves. An example of this is constant area, which can simulate the behaviour of unstretchable, but bendable materials, such as rope in 2D or textile in 3D. Similarly, volume preservation is characteristic to incompressible materials (Desbrun & Gascuel, 1995) and is desirable in smoothing to prevent shrinkage (Taubin, 1995).
To this end, we loosely introduce shape derivatives and refer Allaire al. (2021) for more detail. (22) (6x) denotes a shape derivative of the functional (Ω)) E R if the expansion (S2xx) = H(22) H' holds for small dx. We rewrite this using perturbations
(S))(xx H( - (S2). For several functionals the shape derivatives are known analytically. For the functional H(2) Sa dz defined as a volume integral of h(x) E R the shape derivative
(5))(6) = -So div (hox) dr = S. hōx TTxTnn dæ
(6)
where the last equality is due to the divergence theorem on bounded and Lipschitz domain Ω Inserting the boundary sensitivity from Equation 2 we arrive at
8H = -0 J.
b bbdd=
where by Jr hxx)b(xx dæ R again acts basis, but for the perturbed integral quantity. 5H = 0 can now be enforced either as soft constraint or as a hard constraint by projecting any parameter update 30 onto the R P-1 linear subspace where the integral stays constant 80 5 - [-brbb
Analogously, for functional G(Ω) f ( g dx defined surface integral of g 9xx) E R the shape derivative as a perturbation is (6x) = Sr (ag/ôn + kg)) 5x dæ, where ög/ōn the directional derivative of y along n and x(x) div(n(x)) R is the mean-curvature. The corresponding basis for 5G = b is b. = fr (Øg/On kg) b dr.
4 APPLICATIONS
Having established the framework for editing implicit shapes through boundary sensitivity, consider its applications. We emphasize geometric and semantic editing since our framework unifies both while being agnostic to the architecture and training of the model. We also address deformation- rigidity- editing since it classically is a widely used approach to shape editing. Lastly, we demonstrate the use of constraints with an example of volume-preserving smoothing.
In all cases, we include Tikhonov regularization with X = 0.1 (see Appendix B). However, the behavior of regularization depends the number of points J and the magnitude of the prescribed displacement. In turn, À influences the number of required iterations and the residual.
4.1 GEOMETRIC EDITING
In geometric editing, the displacement is prescribed on the entirety of the boundary. In this section, we focus on studying the case where the prescribed displacement local and the rest of the boundary is fixed, akin to local control in classic representations. Deformation in Section 4.3 and smoothing in Section 4.4 are examples of geometric editing with global target displacement.
For each considered shape, we train a separate network to fit the value and surface normals of the SDF, All networks share the same architecture: hidden layers of 32 neurons each with sin activations. In total, there are P = 2273 learnable parameters, all of which are manipulated during editing. We demonstrate several examples of geometric editing in Figure where we displace parts of both man-made and organic neural implicit shapes. In addition, we quantify and plot the relative geometric error between the computed shape and prescribed target normalized by the largest target displacement (dz, / maxxer 8.. I. When the displacements are, loosely speaking, natural to the shape, the approximations recover the target well. However, not arbitrarily complex displacements can be approximated as in the example of inscribing letters. The characteristic length of the target is much smaller than that of the shape, especially in the relevant region. We hypothesize that the good memory-expred expressiveneess trade-off of neural fields requires the NN to allocate geometric complexity
5

---

Published as a conference paper at ICLR 2023
13.00
33.00 20.355
Figure Geometric editing. Top: non-zero target displacement prescribed on the highlighted regions and the remaining boundary is fixed with o%, O. Bottom: the resulting shape and normalized error. Natural displacements are approximated well. counter-example is given in the last column: the prescribed displacements (imprinting letters) too complex and outside the limits of expressible deformations, resulting in a coarse dent in the general area.
Figure Basis functions of several neural implicit shapes trained single geometry. Each column depicts a weight from each layer increasing in depth left-to-right. Red/blue denote positive/negative deformation w.r.t. the outward normal. These bases do not strongly encode semantic or geometric meaning, different from latent basis functions in generative models in Figure 3.
where it needed during training. To illustrate this, some basis functions are shown in Figure As be seen, their characteristic length similar to that of the shape features. As all possible edits a combination of these basis functions, it is unlikely that modifications on a much smaller scale can be reconstructed faithfully.
4.2 SEMANTIC EDITING
In representation learning the aim is to explain observations with a small number of latent variables (Bengio al., 2013), which we will denote by Generative models f(x;; O), such as (variational) auto-encoders generative adversarial networks, attempt to map novel latent codes to novel outputs within the same distribution as the observations. When interpolating between two latent codes, the appearance of the generated output changes continuously (Shen al., 2020a).
For a generative neural implicit shape model, the basis for the boundary movement is locally described by b Vif / Vxfll (Equation 3). Figure 3 illustrates few of such basis functions for the generative decoder of the IM-Net model (Chen Zhang, 2019). The decoder is trained as part of auto-encoder reconstructing the entire ShapeNet dataset (Chang al., 2015) from E R226

---

Published as a conference paper at ICLR 2023
Figure 3: Basis functions of IM-Net. They partially encode semantics, such as segmentation and symmetries. Each column represents the same basis function. For similar shapes, such as the two chairs, the bases are qualitatively similar. Red/blue denote positive/negative deformation w.r.t. the normal.
latent variables. We use a pretrained model available at https: //github. com/ com/czql 42857/ IM-NET-pytorch, which is implemented MLP with 8 layers of 1024 neurons each and Leaky-ReLU activations.
These basis functions can be observed to encode semantics, such as rotation and reflection symmetries, and segment semantically meaningful parts, such as windscreen on the car, rim of the lamp and different surfaces chairs. For similar parts, such as the two chairs, the bases show similar behaviour.
However, it is known that not all directions in the latent space are (equally) viable (Chen et al., 2022; Vyas et al., 2021), hence, similar argument be made for the bases. Furthermore, while each basis function might cause meaningful deformation, they are not necessarily disentangled, i.e. each basis function does not explain single generative factor, unless latent disentanglement is used in training (Bengio et al., 2013; Shen et al., 2020b; Tschannen et al., 2018). This could further increase the interpretability of the basis.
Despite this, with our method we can intuitively traverse the latent space by considering the link between geometric and latent variable changes. We present several examples in Figure 4, where the highlighted areas I of an initial shape are prescribed local, high-level displacement, such as shortening a leg of a chair or squeezing the sides of the boat. We sample roughly 100 points in these areas, prescribe the same target vector at each point, and leave the remaining boundary unconstrained. After projecting the target onto the current normal and finding the best fit parameter update according to Equation 5, repeat this process for few (< 15) iterations to achieve visually obvious changes. User input is provided only at the beginning. Note that we only consider the latent parameters for our basis and leave all network parameters unchanged. Despite only prescribing local geometric deformation, we observe global and semantically consistent changes. Not only are obvious symmetries preserved, but also the morphology of the shape can change significantly, such with the boat.
In Appendix A, we repeat the set of experiments with the DualSDF (Hao et al., 2020) architec- ture. Each model is trained on single ShapeNet category. This gives better generative results and a more semantically pronounced basis.
Compared to geometric editing, semantic editing has the computational advantage of sampling points on just small part of the boundary I'. Furthermore, the number of latent parameters much smaller than the number of NN parameters, leading to very small least-squares systems. Per iteration, the method only requires a single forward- and backward- -asss through the model, altogether being fit for interactive use.
4.3 RIGID EDITING
As final approach to editing, briefly address deformation-rigiann rigidity since it classically is widely used approach to shape editing. On the one hand, deformation energy one of the many alternative priors to semantic prior discussed previously. A simple approach is consider pure bending energy
7

---

Published as a conference paper at ICLR 2023
Figure 4: Semantic editing. Top: non-zero target displacement is prescribed the highlighted regions I'T and the rest is unconstrained. Bottom: the result after < 15 iterations. A semantically plausible result is produced from plausible locally prescribed displacement, A counter-example is given in the last column: prescribing each wing to move in opposite directions is not a semantically viable displacement and the method fails to approximate the displacements and produce meaningful results.
since it can be computed only from the implicit representation itself. On the other hand, typical applications, such as as-rigid-as as-posiible (Sorkine & Alexa, 2007) or as- us-Killing--aa as as-posiible (Solomon al., 201 I), also include the stretching energy. This not trivially applicable to implicit surfaces, as there is no natural notion of stretch due to ambiguity in the tangent directions. As discussed in Section 3, there are several choices for recovering tangential displacements, but we consider perhaps the simplest one using the tangential projection 8x, (x; E)
(x))xx))x) fx(x; E) of separate sufficiently smooth NN f; Ex; 3 + Rd parameterized in E.
To quantify the deformation energy, we leverage Killing vector fields (KVFs) which generate isometric deformations (Solomon et al., 2011). 8x is a KVF if it has an anti-symmetric Jaco- bian [ōx) = (xx)(xx) E R333 everywhere on the boundary: J(xx) + J(8x) = vx E T. A vector fields' deviation from being Killing can be measured with its Killing energy (xxxx, r) = J [J (6x) + ((x) T dx.
Classically, the Jacobian is computed using discrete operator spatial discretization, while Atzmon et al. (2021) formulates AKVF on a neural implicit directly. We follow this approach and seek an AKVF respecting the prescribed boundary deformations dx (xx) weighted by some small > O: minsx EK + aEc.
Boundary sensitivity allows to search for the energy minimizing deformation directly in the space of expressible deformations and to directly optimize the normal component over 80 using Equation 2. The Ec in Equation is expressed trivially but can now also accommodate the tangential component. To express Ex. we can leverage J being a linear operator J(xx) = J(xx,) + J (6x,). Here (óx.) J(b (0) J(b T )&O with J(bT) ) E R3x3xP and J J[(x;) = (f,,) straight-forward.
Figure 5 illustrates the results of AKVF deformation. Comparing these results with mesh- based method, after sufficient iterations we achieve qualitatively similar results and comparable energies: Expesh 254, Ex 320 for the bunny and Emgss = 11, Ex = 6 for the cactus. However, due to the need to train an additional NN and perform second-order differentiation, our approach about an order of magnitude slower than the mesh-based LSTSQ solver.
4.4 VOLUME PRESERVING SMOOTHING
As an example of fixing the value of a surface or volume integral, we fix the volume itself during smoothing. Fixing the volume of an implicit shape difficult without resorting to an intermediate representation, such as mesh (Remelli et al., 2020; Mehta et al., 2022), since there is no trivial way to differentiably compute the volume of an implicit shape. Smoothing on neural fields has been objective penalizing the deviation from specified
previously formulated using gradient descent curaature (Yang et al., 2021) and as mean-curvatur flow computed on an intermediate mesh (Mehta al., 2022). We also mean-curvature flow without resorting to the intermediate mesh, but
8

---

Published as a conference paper at ICLR 2023
44
Figure 5: Rigid editing. Target displacement is prescribed in the highlighted regions I and the rest is
unconstrained. In both cases, the bottom of the shapes is anchored.
Figure 6: Select iterations of smoothing via mean-curvature flow without and with volume constraint. The volume after 86 iterations decreases by 80% and 1.6%, respectively. The constraint is achieved by projecting the parameter update onto the locally linear isochronic subspace found using boundary sensitivity.
the proposed volume-constraine agnostic to the choice of the smoothing method. We first compute the mean curvature by expanding x(x) div(n(x)), From there, framework with the globally prescribed target displacement 8x, (x) = -x(x) V x [ -. Note, that this approach induces the correct geometric flow as discussed by Mehta al. (2022). To fix the volume V while smoothing, we simply set = in the bases introduced in Equation 7. We enforce volume preservation by projecting the parameter update onto the isochronic subspace. Figure 6 compares smoothing with and without this constraint. After 86 iterations, the volume decreases by 1.6% with and 80% without the constraint. Ultimately, the shapes converge sphere and a singular point, respectively.
the geometric editing
CONCLUSIONS
With implicit neural shapes becoming widespread representation, we have demonstrated unifying approach to perform geometric and semantic editing without the need for tailored training or archi- tectures, while being simple to implement, and, especially in the case of semantic editing, fast and fit for interactive use. While we touched upon optimizing directly in the deformation space with a rigidity-prior, using other priors for unconstrained and tangential deformations remains interesting problem. We used signed-distane extended other neural fields, where NeRFs especially provide tantalizing options. We hope that formulating a basis for the deformation space allows future work to further study and build models with desirable properties, such as interpretabilityy tailored degrees-of offrees,, ineer-iideeeedeecee and compactness or use them for segmentation or symmetry detection.
and occupancy fields, but the editing framework can also be
ACKNOWLEDGMENTS
This was supported by the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement number 860843.

---

Published as a conference paper at ICLR 2023
REFERENCES
Grégoire Allaire, François Jouve, and Anca-Maria Toader. Structural optimization using sensitivity analysis and a level-set method. Journal of Computational Physics, 194(1):363-393, 2004. ISSN 0021-9991. doi: https://doi.org/ 1ttp://oiirr//1..1110010.. 10.1/10.00........ jop.
Grégoire Allaire, Charles Dapogny, and François Jouve. Chapter 1 shape and topology optimization. In Andrea Bonito and Ricardo H. Nochetto (eds.), Geometric Partial Differential Equations Part II, volume 22 of Handbook of Numerical Analysis, pp. 1-132. Elsevier, 2021. doi: https: //doi.org/100 1016/016..b..... hww.a020..0........ 10.004.
Matan Atzmon and Yaron Lipman. Sal: Sign agnostic learning of shapes from raw data. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2020.
Matan Atzmon, Niv Haim, Lior Yariv, Ofer Israelov, Haggai Maron, and Yaron Lipman. Controlling neural level sets. In Advances in Neural Information Processing Systems. pp. 2032-2041, 2019.
Matan Atzmon, David Novotny, Andrea Vedaldi, and Yaron Lipman. Augmenting implicit neural shape representations with explicit deformation fields. arXiv preprint arXiv:2108.089) 08931, 2021.
A. Bærentzen and N. Christensen. Volume sculpting using the level-set method. In International Conference on Shape Modelling and Applications (SMI) 2002, may 2002. URL http: /www2. compute. dtu. . dk putd// pubs/ 7004 full.html.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:1798-1822 2013.
Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An Information-Rich 3D Model Repository. Technical Report 111.:0151200302 [cs.GR], Stanford University Princeton University Toyota Technological Institute Chicago, 2015.
Nutan Chen, Patrick van der Smagt, and Botond Cseke. Local distance preserving auto-encoders using continuous k-nearest neighbours graphs. ArXiv, abs/2206.05900 2022.
Yunlu Chen, Basura Fernando, Hakan Bilen, Thomas Mensink, and Efstratios Gavves. Neural feature matching in implicit 3d representations. In Marina Meila and Tong Zhang (eds.). Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Muchine Learning Research, pp. 1582-1593. PMLR, 18-24 Jul 2021.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939 5939-59999 5948, 2019.
Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll, Implicit functions in feature space for 3d shape reconstruction and completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6970-695 2020.
Thomas Davies, Derek Nowrouzezahrai, and Alec Jacobson. On the effectiveness of weight-encoded neural implicit 3d shapes. arXiv preprint arXiv:2009, 09808, 2020.
Mathieu Desbrun and Marie-Paule Gascuel. Animating soft substances with implicit surfaces. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '95, pp. 287-290, New York, NY, USA, 1995. Association for Computing Machinery. ISBN 0897917014. doi: 10. 10.1145/218880.218444 218380. 218456.
URL /
Tim Elsner, Moritz Ibing, Victor Czech, Julius Nchring-Wirxel,, and Leif Kobbelt. Intuitive shape editing in latent space, 2021.
Benoit Guillard, Edoardo Remelli, Pierre Yvemay, and Pascal Fua. Sketch2mesh: Reconstructing and editing 3d shapes from sketches. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 13023-13032, 2021.
10

---

Published as a conference paper at ICLR 2023
Zekun Hao, Hadar Everbuch-Elow Noah Snavely, and Serge Belongie. Dualsdf: Semantic shape manipulation using a two-level representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
John C. Hart, Ed Bachta, Wojeiech Jarosz, and Terry Fleury. Using particles sample and control more complex implicit surfaces. In SMI '02: Proceedings of the Shape Modeling International 2002 (SMP022, pp. 129, Washington, DC, USA, August 2002. IEEE Computer Society. doi: 10/dfw2ss.
Amir Hertz, Or Perel, Raja Giryes, Olga Sorkine Sorkine-Horuung, and Daniel Cohen-Or. Spaghetti: Editing implicit shapes through part aware generation. arXiv preprint arXiv:2201.13166 2022.
Moritz lbing,
Isaak Lim, and Leif Kobbelt. 3d shape generation with grid- aad-based implicit functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 15559-13568, 2021.
Chiyu Max Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, and Thomas Funkhouser. Local implicit grid representations for 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
Stam Jos and Ryan Schmidt. On the velocity of implicit surface. ACM Transactions on Graphics, 30:1-7, 2011. ISSN 15577368. doi: 10. (0.1145/1 145/1166394. 1966400.
Ishit Mehta, Manmohan Chandraker, and Ravi Ramamoorthi. A level set theory for neural implicit evolution under explicit flows. Xrviv preprint arXiv:2204.07159, 2022.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction function space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4460-4470 2019.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Shigeru Muraki. Volumetric shape description of range data using "blobby model".. In Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '91, pp. 227-235, New York, NY, USA, 1991. Association for Computing Machinery. ISBN 0897914368. doi: (0.1145/122718.)) 1227433....
Ken Museth, David E. Breen, Ross T. Whitaker, and Alan H. Barr. Level set surface editing operators. ACM Trans. Graph., 21333330-333, jul 2002. ISSN 0730-0301. doi: 10. 10.11455566654.56558.
Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Occupancy flow: 4d reconstruction by learning particle dynamics. In International Conference Computer Vision, October 2019.
Tiago Novello, Guilherme Schardong, Luiz Schirmer, Vinicius da Silva, Helio Lopes, and Luiz Velho. Exploring differential geometry in neural implicits, 2022. URL https: //appiii////iv..////////a/// org/ /bs/ 2201. 09263.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 165-174, 2019.
Songyou Peng, Michael Niemeyer, Lars Mescheder, Mare Pollefeys, and Andreas Geiger. Convolu- tional occupancy networks. In European Conference on Computer Vision, pp. 523-540. Springer, 2020.
Edoardo Remelli, Artem Lukoianov, Stephan Richter, Benoit Guillard, Timur Bagautdinov, Pierre Baque, and Pascal Fua. Meshsdf: Differentiable iso-surface extraction. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.). Advances in Neural Information Processing Systems, volume 33, pp. 22468-22 22478. Curran Asso- ciates, Inc., 2020. URL htttppp:/ppooeeeaaiiiieeeoooeeee www.innuri.... wwttrrsssccccpaape cctpppeeppppc./////c///pppp file/ fe40fb94 4ere4444400999eeeeeeeeeddddeeede
.dd.
11

---

Published as a conference paper at ICLR 2023
J. A. Sethian and Peter Smereka. Level set methods for fluid interfaces. Annual Review of Fluid Mechanics, 35(1):341-372, 2003. doi: 10.1 14w////nree..fuii...5.... www.:ffu.......1111111166.1111
Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. Interpreting the latent space of gans for semantic face editing. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9240-9249, 2020a.
Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. Interpreting the latent space of gans for semantic face editing. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9243-9252, 2020b.
Vincent Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In Proc. NeurIPS, 2020.
Justin Solomon, Mirela Ben-Chen, Adrian Butscher, and Leonidas Guibas. vector fields for planar deformation. Computer Graphics Forum, 30(5):1543-1552, 2011. doi: https://doi.org/10... 14678847778699911 11111477666992211.02222 201.00228.x
Olga Sorkine and Mare Alexa. As-Rigid-As-Possinn Surface Modeling. In Alexander Belyaev and Michael Garland (eds.), Geometry Processing. The Eurographics Association, 2007. ISBN
978-3-905673-46-3. doi: 10.23 0..211//GG//SGP07100991
Michael Tao, Justin Solomon, and Adrian Butscher. Near-isometric level set tracking. In Proceedings of the Symposium Geometry Processing, SGP 16, pp. 65-77. Furographics Association, 2016.
G. Taubin. Curve and surface smoothing without shrinkage. In Proceedings of IEEE International Conference on Computer Vision, pp. 552-857, 1995. 10. 111.///C....99.46666000
Richard Tsai and Stanley Osher. Level set methods and their applications in image science. Commu- nications mathematical sciences, 12 2003. doi: 10.43 10.4310/CMS.2000 w0....000..ll.....
Michael Tschannen, Olivier Bachem, and Mario Lucic. Recent advances autoencoder-based representation learning. arXiv preprint arXiv: 18/2.05069 2018.
Nico van Dijk, Kurt Maute, Matthijs Langelaar, and Fred Keulen. Level-set methods for structural topology optimization: A review. Structural and Multidisciplinary Optimization, 48, 09 2013. doi: 10.1007/<00155--011 -801009--....
Luminita Vese. Multiphase Object Detection and Image Segmentation, pp. 175-194. Springer New York, New York, NY, 2003. ISBN 978-0-387-21810-6 doi:
10. URL
https: //t::///oii.. org / 1111111000000 1007 / -0000-88----810 218011110----- 60.
Shantanu Vyas, Ting-Ju Chen, Ronak R. Mohanty, Peng Jiang, and Vinayak R. Krishnamurthy. Latent embedded graphs for image and shape interpolation. Computer-Aided Design, 140:103091, 2021. ISSN 0010-4485, doi: http:///cceecctrtaaaa///////////
URL https: /www.
sciencedirect com/ wtt.m//ciienneeettieeeeeeeeeeeee
Ross Whitaker. Isosurfaces and level-set surface models. School of Computing, University of Utah,
2002.
Guandao Yang. Serge Belongie, Bharath Hariharan, and Vladlen Koltun. Geometry processing with neural fields. In -hirty-Fifth Conference on Neural Information Processing Systems, 2021.
12

---

Published as a conference paper at ICLR 2023
A SEMANTIC EDITING WITH DUALSDF
We repeat the experiments from Section 4.2 with a different architecture, namely, DualSDF (Hao al., 2020). DualSDF learns joint latent space for fine-scale shape and its coarse approximation as a union of spheres. When a sphere is manipulated, an optimization process is run to find a latent code that explains the new sphere configuration, i.e. coarse shape. From the updated latent code the fine-scale shape can be generated.
Different to IM-Net, DualSDF trained on individual ShapeNet categories. We use pretrained models on planes and chairs available at https: / / thub. com/ zekunhaol 995 90/Dua SDF, In addition, we train another model on cars following the provided training procedure on all 3515 car shapes available after preprocessing the ShapeNet category.
Figure shows a few select basis functions of the three different DualSDF models. Figure show the semantic editing results with our method, compared with the method described in DualSDF. For both methods, the shapes are generated with the same generative network. The difference lies only in how the edit is prescribed. In our approach prescribe the movement directly the sampled surface, while in DualSDF we move the spherical primitives, selecting them according to a similar design intent.
Both Du and our method achieve plausible, though different, results. Both methods fail to adhere to the semantically implausible updates prescribed in the last column. However, the main benefit of our approach is that we are able to apply such manipulation to arbitrary NNs, whereas DualSDF needs second NN for the coarse approximation and task-specific training.
Figure 7: Select basis functions of three different DualSDF models. The semantics in most basis functions are not as prominent as the ones shown here. The two sets of chairs show the same respective basis functions, which can be seen have the same semantic quality.
13

---

Published as a conference paper at ICLR 2023
Figure 8: Semantic editing. The first two rows are similar Figure 4, but use the generative network of the DualSDF architecture. The third and fourth rows show the prescribed deformation and the result using the editing procedure described by Hao et al. (2020).
B EFFECT OF TIKHONOV REGULARIZATION ON SEMANTIC EDITING
Figure 9: Effect of Tikhonov regularization on semantic editing. Left: source shape is prescribed an inward displacement the highlighted regions while the rest is unconstrained. The three figures on the right have decreasing amount of Tikhonov regularization X = 10 10 10-3 Stronger regularization better preserves similarity to the source shape, which is especially noticeable on the unconstrained front of the boat remaining wider. Stronger regularization also requires much more iterations converge to similar result (860 for À = 10 compared to 10 for the other two) since the the constraint violation Eg is weighted less heavily than regularization.
14

---

Published as a conference paper at ICLR 2023
C
SPLITTING LARGE DEFORMATIONS
YYMMY
Figure 10: Effect of splitting a large target deformation in geometric editing. Displayed are the results after splitting the target into 1,2,4,8, 16 equal parts. As the figure illustrates, this helps accurately recover large deformations which violate the first-order approximation. For comparison, the black silhouette in the background shows the target.
15


Reviews:
[{"review_id": "tGlf00Y3G4R", "reviewer": "Reviewer_VBEG", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a method for interactively editing 3D shapes defined as the level sets of neural implicit functions. Specifically, the approach quantifies the distribution of changes to this level set (the shape boundary) w.r.t. the parameters governing the neural implicits (either the network weights for a model regressed to fit a single shape, or the latent variables of a generative model). This is referred to as \"boundary sensitivity\". Based on this, the authors demonstrate several example of shape editing guided by high-level sparse user input, as well as introducing deformation constraints (rigidity, volume preservation...) into the framework. While this sort of inverse control has been demonstrated for other representations, this is, to the best of my knowledge, a novel contribution for neural implicits.\n", "summary_of_the_review": "I think the paper develops a nice mathematically justified approach for tying implicit field boundary changes to parameter updates, using this to solve various inverse control tasks. I think it can be better positioned w.r.t. evaluations and prior work.", "strength_and_weaknesses": "The paper is well-written and develops an interesting method for computing the boundary sensitivity of an implicit field. Various applications and examples are presented to highlight the utility of the approach. In general, I am positive about this paper.\n\nOne weakness is that the paper does not really go into how and where the method fails. E.g. for what magnitude of displacements does the linearity assumption lead to incorrect results? If the network is very complex (very high-dimensional), are the geometric edits induced by perturbing network weights still smooth enough to look plausible? How does this latter aspect play with the explicit constraints induced in Sections 4.3 and 4.4?\n\nAnother weakness is that it does not sufficiently compare to baselines, as pointed out in other reviews.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "t7DFf7b3ypd", "reviewer": "Reviewer_mGb1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This submission proposes a mathematical framework for editing 3D represented as deep implicit surfaces. Given a target displacement on the shape, the corresponding modifications of network parameters is computed and applied.\nThis can either be applied to a network representing a single shape (shape editing) or to the latent code of a multi-shape network. In the latter case, the network acts as a learned prior and allows semantic editing.\n", "summary_of_the_review": "Nicely executed paper, but I don’t see the added value compared to ([1] Controlling Neural level sets, Atzmon et al., NeurIPS 2019 and [2] MeshSDF, Remelli et al., NeurIPS 2020) or even [3] Sketch2Mesh, Guillard et al., ICCV 2020", "strength_and_weaknesses": "The theory supporting the framework is very sound, clear, and easy to follow. As opposed to previous works deriving similar formulas ([1] Controlling Neural level sets, Atzmon et al., NeurIPS 2019 and [2] MeshSDF, Remelli et al., NeurIPS 2020), authors here explicitly write the the tangential component before zeroing it out.\n\nAs experiments demonstrate, it effectively works for shape modifications.\nThe provided visualization of “basis functions” (ie. ~gradients at intermediate layers) are nice and shed a light on what the network has learned, but cannot be used for any practical purpose.\n\n\nMy main concern is that this whole idea of finding gradients for network parameters corresponding to some shape modifications is not novel. [1] and [2] derived the exct same gradient, and [3] Sketch2Mesh, Guillard et al., ICCV 2020 already used it for interactive shape manipulation. The proposed method loses some generality compared to these frameworks which express gradients and chain rule for any downstream gradient, not only displacements.\n\nThe only 2 minor differences compared to [1] and [2] are:\n- 1: [1] and [2] use gradient descent to update parameters, here a linear least square is used.\n- 2: [1] and [2[ always zero out the tangential component of the gradient, here it can be non zero.\n\nTo convince me of the relevance of their approach over [1,2], authors would need to argue and demonstrate at least one advantage. It could be in terms of:\n- Speed? Would linear least square faster than gradient descent?\n- Using a non zero tangential component? This is only used in one experiment, not very well explained (I could not reproduce), and not ablated.\n- Not having to reconstruct the explicit surface? But I think this is a misleading argument, because to pick a place where displacements must be applied, one must get an explicit representation of the surface first. And the reprojection of points sounds very much like an adaptive remeshing algorithm. Even more so if that was true, what would be the point of avoiding an explicit mesh in the loop? Speed? Marching Cubes is very fast with coarse to fine grids. Precision? Marching Cubes precision far exceeds the accuracy of the upstream MLP network…\n\nAnother minor concern is that it is not very clear to me how the displacement is indicated on the shape.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "V5auiUXrnYQ", "reviewer": "Reviewer_n2KN", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a unified story for shape derivatives of neural implicit surfaces. The method enables projection onto deformations that preserve volume or are approximately rigid, and it suggests a path toward enforcing other kinds of constraints. Results are limited and only qualitative.", "summary_of_the_review": "The paper's fully-implicit approach to editing neural fields seems valuable. More careful evaluation and explanation is required to make this a great paper.", "strength_and_weaknesses": "## Strengths\n- The paper is mostly clear and concise\n## Weaknesses\n- The paper has no quantitative empirical results. As such, it is unclear how well, e.g., volume preservation works. Since the method is only preserving volume to first order by using step-projections, this should be evaluated.\n- Ditto for the AKVF approach to rigidity", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "O5k-j2Li0w", "reviewer": "Reviewer_TumX", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The key idea behind this work is to leverage boundary sensitivity to express\nhow the perturbations in the parameters of a previously trained implicit-based\nmodel alter the shape boundary. The authors consider two types of editing\noperations: (i) geometric editing, which applies a geometric update on the\nentire shape boundary and (ii) semantic editing, which applies a target\ndisplacement only to a part of the boundary. The paper does not compare to any\nbaseline, nor reports any quantitative metrics to evaluate the quality of the\nediting operations.", "summary_of_the_review": "While I really like the idea of this paper, I am a bit concerned regarding the experimental evaluation, since the authors do not compare to other methods nor report any quantitative metrics. Moreover, since various details are unclear I am leaning towards rejecting the paper. Next, I have included various comments and questions regarding the paper:\n\n1. In the 4th paragraph of Section 1.0, the authors state \"the user supplies a\ntarget displacement on (a part) of the shape boundary\". This sentence is a bit\nunclear. How is the target displacement defined/parametrized? While additional\ndetails are provided later in the text, I think it is important to add one/two\nmore sentences at this point to elaborate more on this.\n\n2. The Related Work section can be improved. For the first two groups of\npapers, i.e. Implicit Shape Representations and Manipulation and Neural Fields,\nI think it would be nice to clearly state that these works are not related work to the\nproposed method. Instead the proposed method can edit implicit shapes generated\nfrom these models by applying perturbations on the NN parameters. Regarding the\nNeural Shape Manipulation section, I believe it would helpful to clearly\nexplain how boundary sensitivity is deployed in the works of Atzmon et al.,\nRemelli et al. and Mehta et al., as stated in the last sentence of this\nsection. This analysis is important in order to better understand what is novel\nin this work and what was already possible in prior research.\n\n3. For the bottom right experiment in Figure 1, what is the editing operation\nthat is considered? Moreover, I don't fully understand why the ABC letters\ndisappear?\n\n4. I think that the experiment in Figure 2, is not properly discussed. Can the\nauthors explain what was the message they intended to convey with this\nexperiment? \n\n5. I think it would be great if the authors could also show results on the\ngeometric shape editing task on the IM-NET model that is used in the\nexperiments of Section 4.2. Being able to edit implicit shapes of a model\ntrained on multiple objects is a more challenging setup.\n\n\nTypos:\n-----\n- Caption in Figure 2 has typo: depthg -> depth\n", "strength_and_weaknesses": "## Strengths:\n-------------\n\n1. I really like the fact that the proposed method is model-agnostic and can be\napplied to a pre-trained Neural Network to deform a shape in place.\n\n2. The idea of employing boundary sensitivity to associate changes in the\nparameters of a previously trained model with the implicit shape is novel and\nhas many potential applications to various tasks.\n\n3. The proposed idea is very simple and potentially can be applied for editing\nvarious implicit-based shapes.\n\n\n## Weaknesses:\n-------------\n\n1. While I really like the idea of this paper, unfortunately the experimental\nevaluation is quite weak. No baselines are considered and no quantitative\nmetrics are reported, hence making it quite hard to evaluate of final edits.\nWhile qualitatively the results are very impressive, I believe it is necessary\nto also include some quantitative metrics that measure the realism of the final\nshape. One potential baseline to consider, which is not 100% comparable with\nthe proposed method is the Deformation Handles. While Deformation Handles\ncannnot operate on the implicit shape it might be interesting to see whether\nthey can yield similar results when applied to the reconstructed shape\n\n2. Various details from the Applications section are missing. For example, in\nsection 4.1, the authors state \"The considered models are trained to fit the\nSDF and surface normals a shape and share the same architecture: sin activation\nfunctions and 3 hidden layers of 32 neurons each.\" This sentence is a bit\nunclear and contains some typos so I am not really sure what is the\nexperimental setup in this section. Do the authors train their model on each one of\nthe shapes in Figure 1? What is the input? How are the displacements\nparametrized? Αre they defined only on the normal direction?\n\n3. Another point that is a bit unclear from the text is whether the editing\noperations are defined in relation to some target shape. I think that the\nauthors only define a target deformation rather than a target shape. Can the\nauthors please clarify this?\n\n4. Another limitation of this work is that the authors do not experiment with\nvarious network architectures. I think it would be useful to showcase that\ntheir model can be applied on various implicit-based architectures such as\nOccNet, DeepSDF, NeUS etc.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster