Reduced Representation of Deformation Fields for Effective Non-rigid Shape Matching
Authors: Ramana Subramanyam Sundararaman
Riccardo Marin
Emanuele Rodolà
Maks Ovsjanikov
Venue: NeurIPS 2022 Accept (2022)
Keywords: non-rigid 3D shape correspondence
3D shape deformation
3D Vision
Computer Graphics

Abstract:  In this work we present a novel approach for computing correspondences between non-rigid objects, by exploiting a reduced representation of deformation fields. Different from existing works that represent deformation fields by training a general-purpose neural network, we advocate for an approximation based on mesh-free methods. By letting the network learn deformation parameters at a sparse set of positions in space (nodes), we reconstruct the continuous deformation field in a closed-form with guaranteed smoothness. With this reduction in degrees of freedom, we show significant improvement in terms of data-efficiency thus enabling limited supervision. Furthermore, our approximation provides direct access to first-order derivatives of deformation fields, which facilitates enforcing desirable regularization effectively. Our resulting model has high expressive power and is able to capture complex deformations. We illustrate its effectiveness through state-of-the-art results across multiple deformable shape matching benchmarks. Our code and data are publicly available at: https://github.com/Sentient07/DeformationBasis.

Reduced Representation of Deformation Fields for Effective Non-rigid Shape Matching
Ramana Sundararaman', Riccardo 2arin Emanuele Rodolà², and Maks Ovsjanikov!
LIIX Ecole Polytechnique, IP Paris University of Tübingen Sapienza University of Rome
Abstract
In this work we present novel approach for computing correspondences between non-rigid objects, by exploiting reduced representation of deformation fields. Different from existing works that represent deformation fields by training general- purpose neural network, we advocate for approximation based on mesh-free methods. By letting the network learn deformation parameters at a sparse set of positions in space (nodes), reconstruct the continuous deformation field in a closed-form with guaranteed smoothness. With this reduction in degrees of freedom, we show significant improvement in terms of data-efficiency thus enabling limited supervision. Furthermore, our approximation provides direct access to first-order derivatives of deformation fields, which facilitates enforcing desirable regularization effectively. Our resulting model has high expressive power and is able capture complex deformations. We illustrate its effectiveness through state- of of-the-art art results across multiple deformable shape matching benchmarks. Our code and data are publicly available at: https:/ //gi thub. com/ SSntiinntt/ Deformatio onBasis. orrooriiinnaaii....
Introduction
Shape correspondence is central problem in computer vision and computer graphics as it facilitates many downstream tasks, such as tracking [1], texture transfer [2] and statistical modeling [3] name a few. Due to its ubiquitous applicability. wide range of techniques have been developed over the past several years [4]. While early approaches relied on axiomatic modeling, recent methods follow data-driven techniques based on different input signals [5, 6, 7, 8] within shape collection.
key question in this context is the choice of representation used to model the non-rigid shape matching problem. Approaches based on intrinsic or pose invariant representations have established gold standard in the context where surfaces are well-defined [9, 10, 11]. Such methods, however, strongly rely on the presence of clean shapes and struggle when acquisition comes from noisy and non-uniform discretization [12]. In contrast, extrinsic techniques which directly operate Euclidean space (R3) show strong resilience to artifacts. Unfortunately, this robustness of extrinsic methods often comes the cost of relying on significant amounts of annotated training data [7, 5]. The main limiting factor arises in the representation of the deformation fields. The standard approach is to use general- purpose MLPs to learn deformation fields that can fit an arbitrary shape deformation [13, 14, 5]. However, given the fact that MLPs are general-purpose networks, they require significant amounts of training data learn both coarse and fine details [S].
To overcome this limitation, we propose to learn a coarse representation of deformation parameters fixed positions in space called "nodes". By learning reduced representation of deformation fields, intuitively, we restrict the learning process to global patterns of the input signal. Then, to recover
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

04
Examples showing generalization of our deformation field representation. Our approach allows to compute correspondences across wide range of shape categories: (a) human articulation, (b) physically- deformation from real-world scans, (c) shapes undergoing topological changes, and (d) shapes represented via implicit functions.
Figure 1:
finer details, we reconstruct the continuous deformation field function in closed- using class of mesh-free approximation techniques [15]. This allows us scale our approach to arbitrary resolution with guaranteed smoothness and across different object classes as shown in Figure 1.
Apart from being theoretically grounded and simple in practice, two key advantages. First, it is significantly more data-efficient and can learn to capture complex deformations given only small number of examples. Secondly, it is more amenable regularization, since it provides explicit access to first-order derivatives of the deformation field in closed form. This especially useful in imposing geometric priors such us local rigidity and volume preservation.
reduced representation has
Our contributions can be summarized as follows: (a) We propose to learn a compact representation of deformation parameters, that is data-efficient, resolution agnostic, and facilitates regularization through direct access to deformation gradients (b) We show an efficient way of incorporating desirable regularization to promote well-structured deformation space. (c) Through extensive experiments across real-world and synthetic datasets, we demonstrate the generalization ability of our method over different down-stream applications such non-rigid shape matching, registration, unsupervised part segmentation and interpolation.
2
Related Works
2.1 Non-rigid shape Correspondence and Registration
Shape correspondence is very well-studied area of computer vision and computer graphics and we refer interested readers to the recent survey [4] for a comprehensive overview. Notable ax- iomatic approaches in this category are hased on the functional maps paradigm [9, 16, 17, 18], that aims to compute a near-isometric mapping by estimating a linear transformation between func- tions represented in reduced basis. This framework has been successfully adapted by learning techniques [6, 19, 20, 7, 10] which demonstrate near-perfect accuracy [1 I] on several shape corre- spondence benchmarks. However, these approaches can be prone errors in the presence of noisy point clouds or significant acquisition artefacts. Although registration-basee techniques [21, 22, 23] present relatively more robust option, they are often based on human- priors or require significant training data.
2.2 Template-based and Template-free Methods
Deforming a template shape to match a target geometry is a long-standing and well studied problem [24, 25, 26]. Such template can be polygonal mesh [27, 28], possibly parame- terised [29, 30, 31] or an unordered point-set [32, 33, 34, 35, 36] or implicitly defined through zero-level set of a Neural Field [13, 14, 37]. In the recent years, learning based model-free defor- mation techniques [32, 33, 23] have emerged
viable option for registration and correspondence tasks given copious amount of training data [32, 38]. Among them, the closest to our approach 3D-CODED [32] which learns deformation fields through point- wise MLPs. However, since this

---

approach fits a general-pune purpose MLP and treats each point on the shape equally likely, it requires abundant training data achieve optimal performance.
2.3 Deformation Field Representation and Shape Interpolation
Deformation between a pair of shapes can be represented as a simple displacement field at every vertex. However, such representation can be unnecessarily complex, and costly optimize. As result, several alternatives have been proposed. The most prominent ways to parameterize the space of deformations include handle-based [39, 40, 41, 42] cage-based [43, 44, 45, 46] representations (see also [47, 48, 49] for an overview). More recently, a common approach is to construct a reduced representation via learned latent embedding [50, 51, 52, 53].
Deformation Field regularization Several geometric constraints have been proposed with the aim of preserving desirable properties of the shape by the deformation field, including imposing elasticity [54, 55, 56] and volume preservation [57, 58, 40, 59, 60]. Recently, these constraints have been successfully adapted by data-driven methods [61, 62, 37, 63] and more relevantly through the differential of the map [51, 63, 63]. Distinct from such approaches,
approximation via mesh-free free method enables evaluating this map differential at fixed points a closed-fone form, which significantly simplifies the deformation field regularization without additional computational overhead.
Shape Interpolation Shape interpolation refers to sime-parameteriaae deformation, where source shape continuously deformed to target shape. Our work is related efforts which aim to enforce intermediate shapes to preserve certain intrinsic properties [52, 53, 61, 64, 65, 51]. Among them closest to our approach is LIMP [52] which disentangles the latent space based on style and pose to preserve geodesic distance. In contrast, our approach does not require such a priori information, which can be costly in terms of annotation efforts.
2.4 Reduced representations and Approximations
In this work we use mesh-free function approximation method [66, 67, 15], to approximate deforma- tion fields. Mesh-free methods have been successfully adapted in Smoothed Particle Hydrodynamics (SPH) modeling [68, 69], image processing [39], animation [40, 70, 71] and more recently in data-driven framework [41]. Differently from [41]. instead of learning the weights of the least squares function, we instead learn deformation values at nodes and demonstrate our method to be applicable in wide-range of downstream tasks. Alternatively, Eisenherger el al. [60] have proposed to use compact representation of deformation fields using the first & eigenfunctions of the Laplace Beltrami Operator (LBO). While their approach provides volume preserving deformation, does not facilitate other regularizations such as se-rigid-as-possiie deformation fields without requiring correspondence at inference time [72].
3 Motivation, Background and Notation
3.1 Motivation:
Parametric models such as SMPL [29] have been tremendously useful the recent years in digitizing and processing human models. This success can largely be attributed to their expressive power, allowing to generate a wide range of styles and poses using a small fixed set of deformation parameters. While this efficacy with such a compact representation is remarkable, it also raises an inspiring question: what is the optimal amount of learnable parameters necessary to represent general deformations? Today, general-purpose MLPs form the conventional way of representing deformation fields due to their simplicity and potential of being universal functional approximators [73, 74]. Unfortunately, the generic power of MLPs also comes at cost of copious training efforts [32, 23]. Furthermore, representing a deformation field using a neural network makes access to certain quantities such Jacobian matrices of deformation fields cumbersome. For these reason, we propose to learn a reduced set of deformation parameters from which we approximate the deformation field function using mesh-freee method.
3

---

3.2 Mesh-free Approximation
Mesh-free methods are class of approximation techniques which constructs a continuous function based on independent, potentially sparse and irregular observations. Assume that our domain of interest IR is equipped with K fixed points qi R along with some observations u, at qi and choice of polynomial basis p(.). We refer to fixed points qi "nodes" (or, alternatively, "deformation nodes"). Our main goal to construct a continuous approximation of some real-valued function uu() in some subdomain & C R of interest. We let Ω to be arbitrary point in our region of interest. The key idea behind this approximation fitting (also referred to as Moving Least Squares) approach [15]. Specifically, we first build compactly supported weighting function (x(x) in the neighborhood of qo via:
use local weighted least-squares
|x
r2 )
9:113
3
X q
1 ST
wi (x)
otherwise
The compactness of this weighting function is useful in preserving the local characteristics of approximation. From this, Shape Function 4, associated with each node i, is constructed as:
$ (x) = p" (x)(M(x))
(q:).
Here (x) is the Moment Matrix associated with the approximation, and defined as:
M (x)
Σ wx(x)p (q.)
The shape function @; is a continuous function that describes how each node qi influences the approximation of u--) across points x E f2. Jointly the the set of D 'ss enable the reconstruction of arbitrary functions to nh order consistency [67], where, n is the order of the polynomial p(-). Specifically, a smooth local approximation of u(x) is given as:
K
u(x) Σ (x)x
(3)
As the construction of & involves computing M- (c.f Eq. (2)). it is a sufficient condition for each point be compactly supported by 4 non-planar nodes q, for M to be non-singular. It is important to note that Eq. (3) is approximating and not interpolating, i.e u, * u(q:). For instance, owing to the compact nature of (xx). it possible that u(q:) is undefined if q: & Q. For this reason, we sample the nodes a priori to have well-supported domain Ω C RS where w(x) is well-defined.
Furthermore, important advantage of using mesh-free approximations comes from exact analyt- ical expression for the gradient function of (xx). To the scope of our current discussion, considering u(x) to be the approximation of deformation field function, the Jacobian of this deformation field only depends on evaluation point and is independent of observed deformation parameters ui
K
04.(x) (x)(x) Ox
(x) Oy
V s.y. (())
Σ
u;
Oz
This nice property helps us characterize the deformation fields with desired first-order regularization an efficient manner. We refer interested readers [67, 66] for a detailed summary.
3.3 Notation:
As our training set, we consider collection of shapes {SI... Sy) with ground truth correspondences Is,s, between them. Shapes be represented as triangular meshes S, (V,E) or simply unordered sets of points (point clouds) S, (V). We pick one shape from the collection template T. and let [T] be the volume enclosed by the boundary ØT. We refer to Q E k as

---

Encoder PoimtNet
Decoder MLP
Meshfree approximation Equation 5
Point Cloud or a Mesh
Learnt
Target Mapping
Deformation Parameters
Figure 2: Overview of our approach. First, learn the deformation parameters at nodes using an Auto-Encoder. Then, we use mesh-free approximation to obtain continuous deformation mapping.
nodal positions, which are fixed points in space sampled from the template volume [7]. We let D(-) R - R be the deformation mapping, which, intuitively maps points in the deformation volume to points on target shapes. We refer to Uj as the nodal deformation parameters corresponding to the jth shape and analogously define Dy(-). Each node q E Q has a support radius r; and associated deformation parameter u4. We use lower-case notation Vi to refer to the value of the deformation field at node q: corresponding to shape j. We denote E Ω C R as points in space which are supported by at least four non-planar nodes. We refer to Ux(x) as the continuous approximation of the deformation field, constructed from deformation parameters Uj using Eq. (3). We re-iterate that 4, * (,)). The relation between a deformation field and deformation mapping is given by (x) = x UX((, For the sake of consistency, index nodes using shape collection using j, and points within shape using k.
Method: Learning Nodal Deformation-Field
Overview. Our network is based on PointNet [75] auto-encoder shown in Figure 2. Our network Fo(-) predicts nodal deformation parameters Uj for each training shape Sj, i. Uj = Fo) where U, R Kx3 As mentioned before, the nodes are fixed a priori. From the predicted Uj. we can compute the shape-specific deformation mapping (z) and its Jacobian Jj, via:
K
Dj(x) x
4 (x)x)
(5)
I I V xU,,
Where, V U)(x, is given in Equation 4.
4.1 Training
Intuitively, would like train network that S, (x)xx) T), subject appropriate regularization. Although (x) can be approximated at an arbitrary x, which is supported by four non-planar nodes, we restrict ourselves
for the ease of learning. As Fo(-) represents auto-encoder, it can be decomposed as Fo(S,) = Dec( (enn(E(())))) = Dec(Z,) where Zj denotes the latent embedding. Leveraging this fact, provide novel way promote plausible latent deformation spaces Z, by enforcing first-order constraints over the intermediate shapes as well. The overall optimization objective of our network given as:
Lret Xi AgCool >aharap XgZz
(6)
For the unsupervised case, we replace Lcor with LOD which denotes the Chamfer's distance.
Correspondence Loss Given a set of C of corresponding points Xt), where x TT, E Sj our correspondence loss given
Leor Σ
Σ
Dy (xi)
(xi)
x
j=1 xx)
Where oi. denotes the th point in jth shape.
5

---

Volume Preserving Field A deformation field is volume preserving iff its Jacobian has unit determinant over the entire shape. Consequently, our local volume preservation regularization is given as follows:
N
K
Loot
(T)(()))))) - -1
(8)
j=
We empirically observe poor convergence when this objective is enforced over the entire shape due to its stringent nature. This because not all deformations are strictly volume preserving. Thus, we restrict this regularization only at nodes.
As Rigid As Possible (ARAP) Deformation Since rigid motions preserve pairwise distances, a deformation field associated with such a transformation is characterized by orthonormal Jacobian matrix. Thus, in order to promote locally rigid deformation field at the deformation nodes, we define ARAP regularization
Curop
JJ (q.),, (q.) -I
Structuring Latent Deformation Space A well-known advantage of an auto-encoder architecture is the construction of the latent space, where each shape has an embedding Zj RR Then, parameterized path this latent space between two shapes Zi,((a) = aZi + (1- a) Z, continuously deforms S, to S/ with rate of change controlled by a. This allows constructing sequence of shapes, often referred to as interpolated shapes. Since each Dec(Z,) = U,, we can further require our network Fg to produce a plausible deformation between each pair of training shapes. To that end, we introduce our latent smoothness loss as follows:
[S]
[S|
L2
Curap (dec((1-a))) a)zz + az))) +
Σ Idj
Lool (dec(( a)z, + azi))
(10)
bÍj
Where Coraps Loot defined in Eqs. (8), (9) and (0, 1) are sampled randomly.
4.2 Inference
At test-time, given pair of unseen shapes (X.Y) we follow three-step procedure to obtain the correspondence xyy.. First, we separately reconstruct (Dxx Dy) by deforming the fixed template T. Second, we enhance the respective reconstructions by optimizing the latent vector Z independently for shapes (X, The objective for this optimization is to minimize the bi-directional Chamfer Distance [32] while also enforcing first-order constraints as follows:
Z argmin AjLeD + AzCarop AgCvol- Z
(11)
As Dx is the reconstruction of X', the correspondence between Dx. X' can be computed via simple nearest neighbor search in 3D (analogously for J). Since Dx, Dy are deformed versions of a template they enjoy natural correspondence (by vertex ordering). Finally, the correspondence between (X' a composition of two nearest neighbour searches xx == (NN(Dx, 2)), NN(Dy,))).
4.3 Extending sparse dense Correspondence
An added advantage of our representation is the ability to retrieve dense shape correspondence between a shape pair, given few sparse key-point correspondences (x), Vx, E X, Vуι V.
First, we estimate the deformation parameter at the nodes by solving an optimization:
u; = argmii A1
Σ
Dx (xi) yxll2 1 + de AllIsf -11 f-= + Aa det(Jxx) 113
(12)
V(XLYE)
Then, dense mapping can be computed by approximating the deformation field (c.f Equation 5).

---

Source
12]
[321
23
Ours
GT
Figure 3: Color-coded correspondences on the SCAPE (PC+N) dataset. "Twist" is challenging articulation as wrong deformation can lead to large geodesic error (see Cheese-Pull effect [76]. Figure 11), We hypothesize that our approach, which learns "global" sense of the articulation, does not suffer from such artefacts since the fine (local) details are computed closed form.
Method
Conespondence Frror
Type
FAUST(NI)
SCAPE(PC*N)
Spectral
GeoFMap [7]
1.7
11.2
20.1
27.7
Dift-FMap [12]
1.0
15.1
5.4
26.0
Pair-wise
ConNer33 [8]
15.0
9.6
25.9
38.0
3D-CODED [S]
23.0
10,3
7.0
18.7
Template based
TransMatch
Ours
0.1
4.8
5,3
6.6
Table 1: We report correspondence eтrоr as geodesic distortion (in cm) scaled by square root of shape area. #Tr data denotes number of training shapes scaled by 10 4
4.4 Implementation details
Analytical Gradients and Timing advantages We leverage the advantage of inexpensive access to Jacobians as mentioned in Equation 4. Because evaluation points are known priori, due the use of a fixed template T, the matrix can be pre-computed and re-used at training and evaluation. In practice, we observe 10x x speed-up training time when enforcing and a 350x speed-up incorporating the latent constraints (c.f. Eqn 10). We provide more timing details in the supplementary.
first-order constraints
Node Sampling: Since the deformation field at point is determined by the nodes within the radius, it is important to limit the influence of node which close in a Euclidean sense but geodesically far. For instance, it is counter-intuitive to have node in the trunk of the human influencing the deformation of a point in the arm. Bearing this in mind, our node sampling strategy divided into three main steps. First, we construct dense sampling of points in the volume and around the boundary of the template ÖT. Second, we use rejection sampling exclude a node that exerts its influence in semantically different regions [29]. Finally, perform Farthest Point Sampling (FPS) until each surface point covered by 4 non-planar nodes. We emphasize that this step is performed only the template shape and using SMPL [29] segments is one many possible ways to perform segmentation. An in-depth ablation study is provided in the supplementary material.
5 Experiments
The reduced representation for deformation field which we have discussed so far is conducive to produce naturally smooth deformation while significantly reducing the amount of supervision needed to facilitate learning. We empirically show the efficacy of our proposed representation of deformation fields across four main tasks, namely Non-rigid 3D shape correspondence, Shape registration, Unsupervised part segmentation and Shape interpolation.
7

---

Sre
[82]
Ours
Sa 191033 Phycoelly Bewed
63
Oatee +435
0.8
8.18
033
Figure 4: Quantitative and qualitative results on SHREC'20. Our approach predicts smooth corre- spondences across highly-granlle surface-level deformation.
5.1 Shape Correspondence
We consider three challenging benchmarks, namely, SHREC'19, FAUST (PC), SCAPE (PC+N).
SHREC' 19 [77] is a standardised benchmark consisting of 430 evaluation pairs with significant variations in mesh resolution and connectivity. FAUST (PC) denotes more recent Non-Isometric Point Cloud variant [12] of the FAUST dataset consisting of 1000 points with large variance in point sampling density. Third, we evaluate on a variant of the recent SCAPE-Remesh dataset [78] consisting of 20 shapes of the same human in 20 distinct poses. We further augment the challenge by adding random Gaussian noise and refer to as SCAPE (PC+N). We evaluate correspondence error following the Princeton benchmark protocol [79]. We train our method on subset of 1000 SURREAL shapes [80] for 1000 epochs with data-augmentation along Y-axis.
Baselines We compare method against data-driven correspondence methods broadly classified into Spectral, Pairwise and Template based. We use GeoFMap [7] with the more robust feature extractor Diffusion-Net [1]] wise baseline. For our template based baselines, we use 3D-CODED [32] and TransMatch [32]. For the evaluation of baselines our proposed SCAPE (PC+N),
spectral bascline, Diff-FMaps [12] and CorrNet3D [8] as pair-
the author-prooidde pre-trained models, and apply consistent preprocessing to the input shapes across all methods.
Discussion Our approach consistently outperforms baselines summarised in Table While quantitative correspondence results are persuasive, it is remarkable to note that our method requires order of magnitude less training data in comparison to competing methods. This supports premise that characterizing typical deformations requires far fewer parameters than what is leveraged by existing data-driven methods. We show qualitative correspondence results through color transfer for a challenging pair with "twisted" motion in Figure 3.
5.2 Shape Registration
Shape registration is special case of correspondence, where our goal is to find optimal deformation between the scan and a fixed template. For this, we consider the recent SHREC'20 benchmark [8] consisting of 11 partial scans of stuffed toy rabbits to be registered single scan. This benchmark is particularly challenging due to granulated surface deformation, scanning artefacts, and limited data and supervision.
Experiment We split this dataset into 7 training shapes and 4 shapes for evaluation. Shapes in our test are made of "chickpea" material, which exhibits the largest magnitude granular surface de- formation. We compare our method with 4 baselines namely, FARM [21], BCICP [78]+ZoomOut [83], 3D-CODED [32] and TransMatch [23]. Since the two data-driven baslines are not designed for training with key- tey-point supervision, we use Equation 12 to generate dense-groed ground truth for training. In fairness, we report two variants of our method trained- one trained with key-point and the other with dense supervision denoted as "Ours" and "Ours++" respectively. We stress that this additional supervision is used only at training time while we maintain the test set to be fully-blind. We summa- rize our guantitative and qualitative results in Figure 4. It is remarkable that our approach outperforms

---

Plane
Table
CD
IoU
CD
IoU
(xle4)
(96
(x1e4)
(%)
DIT [14]
24.6
69.1
26.7
68.9
DIF [13]
15.0
78.0
11.2
79.3
Ours
(w/o Con)
88.8
Source
Con)
Ours
Ours
0.6
89.3
3.0
90.0
Figure 5: Ours (w/o Con) is a variant of our approach without any deformation constraints. Both of variants show significant improvement over baseline that models volumetric deformation fields. Qualitative result demonstrates color coded segmentation transfer across significant shape variability.
axiomatic and competing data-driven baselines by at least a two-fold margin. Importantly, despite our network sharing the same encoder [75] as 3D-CODED [32], there is a striking difference in performance. We attribute this to our well-regularized deformation space.
5.3 Unsupervised Segmentation Transfer
In this section, we demonstrate the generalization ability of our approach to model deformation between shapes with considerable topological differences. To that end, we consider the task of part-level segmentation over point clouds consisting of table and plane categories from ShapeNet [84] dataset. Apart from topological differences and large structural variance, the absence of ground truth annotations exacerbates the challenge. In this setting, we compare our method with two Deep Implicit networks, namely DIF-Net [13] and DIT [14], which model a volumetric deformation field between learned template and training shapes. Our choice of baseline endows with fair ground of comparison between the two representations of the deformation field MLP-based and ours.
Experiment and Discussion We train our approach using the unsupervised loss mentioned in Section 4.1 over 1000 objects sampled at random from each category. We consider 190 evaluation pairs per-category from the prescribed validation set and measure the segmentation accuracy by the IoU metric [84]. In addition, we also measure the bi-directional Chamfer's distance of reconstructed geometries. We summarize our quantitative observation along with a qualitative example Figure We remark that while deformation fields between aforementioned categories are not strictly volume- preserving, we still observe a noticeable improvement over the baseline. This is because our deformation priors help in structuring the space of deformations, which explicitly avoids degeneracy such as collapsing shape parts. This remark is corroborated by a lower (preferred) Chamfer distance while there is decline in the accuracy when regularization is applied.
Shapellow
Surpel fan- 3EL
Searllogy
4300
Source
Targtt
0.00
6l6
cls
6.2
4.5
Hear 4026 segh diparty
Figure 6: Quantitative and qualitative comparison of interpolation. While ShapeFlow [51] enforces volume preservation prior, its latent deformation space not distortion-free. By better structuring the latent-space (c.f Eqn 10), our sampled intermediate shapes are near distortion-free.

---

5.4 Shape Interpolation
notable characteristic of well-structured latent space is the ability to produce plausible intermedi- ate shapes given source and target. This task is commonly referred to shape interpolation. Since there exists no canonical path, interpolation sequences are gauged by the extent to which intrinsic met- rics are preserved, in particular isometric distortion [85]. For this setting, consider the FAUST [3] dataset, where, we train our method on the first 80 shapes and evaluate over the last 20 shape pairs. We compare method against four baselines namely 3D-CODED [5], NeuroMorph [61], LIMP [52] and ShapeFlow [5 I. Since LIMP employs a fixed-size decoder and NeuroMorph uses a separate interpolation module involving
explicit computation of correspondence matrix, both of these approaches are limited by shape resolution. On the other hand, our approach is resolution agnostic and outperforms the baselines by discemnibee margin as summarized in Figure 6. This improvement over the baseline is due to the incorporation of our latent deformation priors in a computationally feasible manner, which will be justified through an extensive ablation study in the supplementary.
Additional Results
In addition to the results shown above, we also present qualitative correspondence results between neural implicit fields and real-world data the supplementary. More specifically, in Section 5 of the supplementary. we show qualitative interpolation and correspondence results between implicitly defined surfaces. Then, in Section 6. of supplementary, we show qualitative correspondence results in the form of texture transfer between pair of shapes from the CMU-Panoptis dataset [86] consisting of point clouds acquired from from Kinect RGB-D sensor. Finally, in Section 6.2, we also show the versatility of our representation in modelling deformation field between shapes that have more freedom regarding such as meshes of the human heart [87].
6 Conclusion, Limitations and Future Work
We presented an effective representation of deformation fields, which allows learning a reduced of shape-specific deformation parameters while constructing the continuous deformation field using mesh-free approximation. A key observation behind our method is that in many settings, the space of realistic deformations is well-constrained and expressed with a small set of parameters. To that end, we demonstrated that our approach can achieve significant improvement upon existing baselines across challenging downstream applications and remarkably reduce the dependence on training data. Moreover, this representation also endowed us with access to first-order derivatives in closed form, thereby facilitating the use of strong first-order regularization. Our approach still has some limitations and leads to possible exciting future work. Firstly, while our approach produces a smooth deformation field in principle, there is no guarantee of bijectivity or invertability. Second, instead of fixed nodal positions, optimizing with respect to our approximation function would also be an interesting direction to study.
Acknowledgments: Parts of this work were supported by the ERC Starting Grants No. 758800 (EXPROTEA) and No. 802554 (SPECGEO), the ANR AI Chair AIGRETTE, an Alexander von Humboldt Foundation Research Fellowship. We thank Robin Magnet and Gautam Pai for their feedback in improving our manuscript.
References
[1] Matthias Innmann, Michael Zollhöfer, Matthias Nießner, Christian Theobalt, and Marc Stam- minger. Volumedeform: Real-time volumetric non-rigid reconstruction. In European Conference on Computer Vision, pages 362-379. Springer, 2016.
[2] Davide Boscaini, Jonathan Masci, Emanuele Rodolà, and Michael Bronstein. Learning shape correspondence with anisotropic convolutional neural networks. Advances in neural information processing systems, 29, 2016.
[3] Federica Bogo, Javier Romero, Matthew Loper, and Michael J Black. Faust: Dataset and evaluation for mesh registration. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3794-3801, 2014,
10

---

[4] Yusuf Sahillioglu. Recent advances in shape correspondence. The Visual Computer, 36(8):1705- 1721, 2020.
[5] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 3d- coded: 3d correspondences by deep deformation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 230-246, 2018.
[6] Or Litany, Tal Remez, Emanuele Rodola, Alex Bronstein, and Michael Bronstein. Deep functional maps: Structured prediction for dense shape correspondence. In Proceedings of the IEEE international conference computer vision. pages 5659-5667, 2017.
[7] Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. Deep geometric functional maps: Robust feature learning for shape correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8592-8601, 2020.
[8] Yiming Zeng, Yue Qian, Zhiyu Zhu, Junhui Hou, Hui Yuan, and Ying He. Correet3d: Un- supervised end-to-end learning of dense corresoondence for 3d point clouds. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[9] Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Functional maps: a flexible representation of maps between shapes. ACM Trunsactions on Graphics (ToG), (1): 2012.
[10] Marvin Eisenberger, Aysim Toker, Laura Leal-Taixé, and Daniel Cremers. Deep shells: Un- supervised shape correspondence with optimal transport. Advances in Neural Information Processing Systems, 33:10491-10502, 2020.
[11] Nicholas Sharp, Souhaib Attaiki, Keenan Crane, and Maks Ovsjanikov. DiffusionNet: Dis- cretization agnostic learning surfaces. ACM Transactions Graphics, 41(3):1 June 2022.
[12] Riccardo Marin, Marie-Julie Rakotosaona, Simone Melzi, and Maks Ovsjanikov. Correspon- dence learning via linearly-invariant embedding. ArXiv, abs/2010. 0.000.1.000 13136, 2020.
[13] Yu Deng, Jiaolong Yang, and Xin Tong. Deformed implicit field: Modeling 3d shapes with learned dense correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10286-10296, 2021.
[14] Zerong Zheng, Tao Yu, Qionghai Dai, and Yebin Liu. Deep implicit templates for 3d shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1429 1439, 2021.
[15] Peter Lancaster and Kestutis Salkauskas. Surfaces generated by moving least squares methods. Mathematics of Computation, 37:141-158, 1981.
[16] Artiom Kovnatsky, Michael M Bronstein, Alexander M Bronstein, Klaus Glashoff, and Ron Kimmel. Coupled quasi-harmonic bases. In Computer Graphics Forum, volume 32, pages 439-448. Wiley Online Library, 2013.
[17] Yonathan Aflalo and Ron Kimmel. Spectral multidimensional scaling. Proceedings of the National Academy of Sciences, 110(45): 2013.
[18] Emanuele Rodolà, Luca Cosmo, Michael M Bronstein, Andrea Torsello, and Daniel Cremers. Partial functional correspondence. In Computer Graphics Forum, volume 36, pages 222-236. Wiley Online Library, 2017.
[19] Oshri Halimi, Or Litany, Emanuele Rodola, Alex M Bronstein, and Ron Kimmel. Unsupervised learning of dense shape correspondence. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4370-4379, 2019.
[20] Jean-Michel Roufosse, Abhishek Sharma, and Maks Ovsjanikov. Unsupervised deep learning for structured shape matching. Proceedings of the IEEF/CVE International Conference on Computer Vision, pages 1617-1627, 2019.

---

[21] R. Marin, S. Melzi, E. Rodolå, and U. Castellani. Farm: Functional automatic registration method for 3d human bodies. Computer Graphics Forum, 39(1):160-177 2020.
[22] D. Hirshberg, M. Loper, E. Rachlin, and M.J. Black. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In European Conf. on Computer Vision (ECCV), LNCS
7577, Part IV, pages 242-255. Ppringer-Verlage October 2012.
[23] Giovanni Trappolini, Luca Cosmo, Luca Moschella, Riccardo Marin, Simone Melzi, and Emanuele Rodolà. Shape registration in the time of transformers. In A. Beygelzimer, Y. Dauphin, Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[24] Brett Allen, Brian Curless, and Zoran Popović. Articulated body deformation from range scan data. ACM Trans. Graph., 21(3)x612-619, jul 2002.
[25] Brett Allen, Brian Curless, and Zoran Popović. The space of human body shapes: Reconstruction and parameterization from range scans. 22(3):587-594, jul 2003.
[26] Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '99, page 187-194, USA, 1999. ACM Press/Addison-Weent
Publishing Co.
[27] Venkat Krishnamurthy and Marc Levoy. Fitting smooth surfaces to dense polygon meshes. Proceedings of the 23rd annual conference Computer gruphics and inteructive techniques, 1996.
[28] Matthias Eck, Tony DeRose, Tom Duchamp. Hugues Hoppe, Michael Lounsbery, and Werner Stuetzle. Multiresolution analysis of arbitrary meshes. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '95, page 173-182, New York, NY, USA, 1995. Association for Computing Machinery.
[29] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 446))244:1-2248
October 2015.
[30] Silvia Zuffi, Angjoo Kanazawa, David Jacobs, and Michael J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), July 2017.
[31] N. Hasler, C. Stoll, M. Sunkel, Rosenhahn, and H.-P. Seidel. A statistical model of human pose and body shape. Computer Graphics Forum, 28(2)3337-346, April 2009.
[32] Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan Russell, and Mathieu Aubry, 3d-coded: 3d correspondences by deep deformation. In ECCV, 2018
[33] Theo Deprelle, Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, and Mathieu Aubry. Learning elementary structures for 3d shape generation and matching. In NeurlPS, 2019.
[34] Thibault Groueix, Matthew Fisher, Vova Kim, Bryan Russell, and Mathieu Aubry. Unsupervised cycle-consisten deformation for shape matching. In Symposium on Geometry Processing (SGP), 2019.
[35] Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann. 3dn: 3d deformation network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[36] Thibault Groueix. Matthew Fisher, Vladimir G. Kim, Bryan Russell, and Mathicu Aubry. AtlasNet: Papier-Mäché Approach to Learning 3D Surface Generation. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018.
[37] Matan Atzmon, David Novotny, Andrea Vedaldi, and Yaron Lipman. Augmenting implicit neural shape representations with explicit deformation fields. arXiv preprint arXiv:2 08.00891, 2021.
12

---

[38] Silvia Zuffi, Angjoo Kanazawa, and Michael J. Black. Lions and tigers and bears: Capturing non-rigid, 3D, articulated shape from images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 2018
[39] Scott Schaefer, Travis McPhail, and Joe D. Warren. Image deformation using moving least squares. ACM Trans. Graph., 25:533- 1433-500220 2006.
[40] Bart Adams, Maks Ovsjanikov, Michael Wand, Hans-Peter Seidel, and Leonidas J. Guibas. Meshless modeling of deformable shapes and their motion. In Proceedings of the 2008 ACM JGGGRAPH/Euroyaphicc Symposium on Computer Animation, SCA '08, page 77-86, Goslar, DEU, 2008. Eurographics Association.
[41] Meitar Shechter, Rana Hanocka, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. DeepMLS: geometry-aware control point deformation. Proc. Eurographics Short Papers, 2022.
[42] Alec Jacobson, Ilya Baran, Jovan Popović, and Olga Sorkine. Bounded biharmonic weights for real-time deformation. ACM Trans. Graph., 30(4), jul 2011.
[43] Yaron Lipman, David Levin, and Daniel Cohen-Or. Green coordinates. ACM Trans. Graph., 2703):1-10, aug 2008.
[44] Michael S. Floater, Géza Kós, and Martin Reimers. Mean value coordinates in 3d. Computer Aided Geometric Design,
October 2005.
[45] Pushkar Joshi, Mark Meyer, Tony DeRose, Brian Green, and Tom Sanocki. Harmonic coordi- nates for character articulation. ACM Trans. Graph., 26(3):71 jul 2007.
[46] Wang Yifan, Noam Aigerman, Vladimir G Kim, Siddhartha Chaudhuri, and Olga Sorkine- Hornung, Neural cages for preseeigg 3d deformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 75-83, 2020.
[47] Dominique Bechmann. Space deformation models survey. Computers & Graphics, 18(4):571- 586, 1994.
[48] James Gain and Dominique Bechmann. A survey of spatial deformation from user-centered perspective. ACM Transactions on Graphics (TOG), 27(4):1-21, 2008
[49] Jesús R Nieto and Antonio Susín. Cage based deformations: a survey. In Deformation models, pages 75-99. Springer, 2013.
[50] Mikaela Angelina Uy Vladimir G. Kim, Minhyuk Sung, Noam Aigerman, Siddhartha Chaud- huri, and Leonidas Guibas. Joint learning of 3d shape retrieval and deformation. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2021.
[51] Chiyu Jiang, Jingwei Huang, Andrea Tagliasacchi, and Leonidas Guibas. Shapeflow: Learnable deformations among 3d shapes. Advances in Neural Information Processing Systems, 2020.
[52] Luca Cosmo, Antonio Norelli, Oshri Halimi, Ron Kimmel, and Emanuele Rodolà. LIMP: Learning latent shape representations with metric preservation priors. In Computer Vision ECCV 2020, pages 19-35. Springer International Publishing, 2020.
[53] Marie-Julie Rakotosaona and Maks Ovsjanikov. Intrinsic point cloud interpolation via dual latent space navigation. In Computer Vision ECCV 2020, pages 655-672. Springer International Publishing, 2020.
[54] Olga Sorkine and Mare Alexa. As-Rigid-As-Poosiblle Surface Modeling. In Alexander Belyaev and Michael Garland, editors, Geometry Processing. The Eurographics Association, 2007.
[55] Eitan Grinspun, Anil N. Hirani, Mathieu Desbrun, and Peter Schröder. Discrete shells. In Proceedings of the 2003 ACM SICGRAPH//ungraphen Symposium on Computer Animation, SCA '03, page 62-67, Goslar, DEU, 2003. Eurographics Association.
[56] José A. Iglesias, Martin Rumpf, and Otmar Scherzer. Shape-aware matching of implicit surfaces based on thin shell energies. Foundations of Computational Mahematics, 18(4))891-99) 1977, June 2017.
13

---

[57] A. Rappoport, A. Shefferr, and M. Bercovier. Volume- preserving free-form solids. IEEE Transactions on Visualization and Computer Graphics, 2(1):19-27, 1996.
[58] Kun Zhou, Jin Huang, John Snyder, Xinguo Liu, Hujun Bao, Baining Guo, and Geungg-Yenng Shum. Large mesh deformation using the volumetric graph laplacian. ACM Trans. Graph., 24(3):496 503,, jul 2005.
[59] G. Hirota, R. Maheshwari, and M.C. Lin. Fast volume-preserringg free-form deformation using multi-level optimization. Computer-Aided Design, 32(8-9):49--51,, August 2000.
[60] Marvin Eisenberger, Zorah Lähner, and Daniel Cremers. Divergence- shape correspondence by deformation. Computer Graphics Forum, 38, 2019.
[61] Marvin Eisenberger, David Novotny, Gael Kerchenbaum, Patrick Labatut, Natalia Neverova, Daniel Cremers, and Andrea Vedaldi. Neuromorph: Unsupervised shape interpolation and correspondence in one go, 2021.
[62] Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. Nerfies: Deformable neural radiance fields. ICCV, 2021.
[63] Guandao Yang, Serge Belongie, Bharath Hariharan, and Vladlen Koltun. Geometry processing with neural fields. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[64] B. Heeren, C. Zhang, M. Rumpf, and W. Smith. Principal geodesic analysis in the space of discrete shells. Computer Graphics Forum, 37(5):173-1844 August 2018.
[65] Qingyang Tan, Zherong Pan, Lin Gao, and Dinesh Manocha. Realtime simulation of thin-shell deformable materials using CNN-based mesh embedding. IEEE Robotics and Automation Letters, 5(2):2325-2332 April 2020.
[66] T. Belytschko, Y. Krongauz, D. Organ, M. Fleming, and P. Krysl. Meshless methods: An overview and recent developments. Computer Methods in Applied Mechanics and Engineering, 139(1-4)33-47, December 1996.
[67] P. Fries and Hermann G. Matthies. Classification and overview of meshfree methods. 2004.
[68] I. Hammani, Marrone, A. Colagrossi, G. Oger, and D. Le Touzé. Detailed study on the extension of the SPH model to multi-phase flow. Computer Methods in Applied Mechanics and Engineering, 368: 13189, August 2020.
[69] S. Marrone, M. Antuono, A. Colagrossi, G. Coliechio, D. Le Touzé, and G. Graziani. Delta SPH model for simulating violent impact flows. Computer Methods in Applied Mechanics and Engineering, 200(13-16):152-11
March 2011.
[70] M. Muller, R. Keiser, A. Nealen, M. Pauly, M. Gross, and M. Alexa. Point based animation of elastic, plastic and melting objects. In Proceedings of the 2004 ACM SIGGRAPPH/Eurorruphis symposim on Computer animation SCA '04. ACM Press, 2004.
[71] Matthias Müller, Bruno Heidelberger, Matthias Teschner, and Markus Gross. Meshless de- formations based on shape matching, ACM Transactions on Graphics, 24(3):471-478, July 2005.
[72] Marvin Eisenberger and Daniel Cremers. Hamiltonian dynamics for real-world shape interpola- tion. In ECCV, 2020.
[73] Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks universal approximators. Neural Networks, 2(5):359- -36,, January 1989.
[74] Allan Pinkus. Approximation theory of the MLP model in neural networks. Acta Numerica, 8:143-195, January 1999.
[75] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. arXiv preprint arXiv:1612.00593 2016.
14

---

[76] Marvin Eisenberger, Zorah Lähner, and Daniel Cremers. Smooth shells: Multi-scale shape registration with functional maps. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12262-12271, 2020.
[77] Melzi, R. Marin, Rodolà, U. Castellani, J. Ren, A. Poulenard, P. Wonka, and M. Ovsjanikov. Matching humans with different connectivity. Eurographics Workshop on 3D Object Retrieval, 2019.
[78] Jing Ren, Adrien Poulenard, Peter Wonka, and Maks Ovsjanikov. Continuous and orientation- preserving correspondences via functional maps. ACM Transactions on Graphics, 37(6):1-11, December 2018.
[79] Vladimir Kim, Yaron Lipman, and Thomas Funkhouser. Blended intrinsic maps. ACM Transactoon on Graphics (Proc. SIGGRAPH), 30(4), July 2011.
[80] Gil Varol, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, and Cordelia Schmid. Learning from synthetic humans. In CVPR, 2017.
[81] Roberto M. Dyke, Feng Zhou, Yu-Kun Lai, Paul L. Rosin, Daoliang Guo. Kun Li, Riccardo Marin, and Jingyu Yang. SHREC 2020 Track: Non-rigid shape correspondence physically- based deformations. In Tobias Schreck, Theoharis Theoharis, Ioannis Pratikakis, Michela Spagmuolo, and Remco C. Veltkamp, editors, Eurographics Workshop on 3D Object Retrieval. The Eurographics Association, 2020.
[82] Riccardo Marin, Simone Melzi, Emanuele Rodola, and Umberto Castellani. Farm: Functional automatic registration method for 3d human bodies. In Computer Graphics Forum, volume 39, pages 160-173. Wiley Online Library, 2020.
[83] Simone Melzi, Jing Ren, Emanuele Rodolà, Abhishek Sharma, Peter Wonka, and Maks Ovs- janikov. ZoomOut. ACM Transactions on Graphics, 38(6):1-14, December 2019.
[84] L. Yi, Lin Shao, Manolis Savva, Haibin Huang, Yang Zhou, Qirui Wang, Benjamin Graham, Martin Engelcke, Roman Klokov, Victor S. Lempitsky, Yuan-Zhu Gan, Pengyu Wang. Kun Liu, Fenggen Yu, panpan Shui, Bingyang Hu, Yan Zhang, Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Minki Jeong, Jachoon Choi, Changick Kim, Angom Geetchandra. Narasimha Murthy, Bhargava Ramu, Bharadwaj Manda, M. Ramanathan, Gautam Kumar, P. Preetham, Siddharth Srivastava, Swati Bhugra, Brejesh Lall, Christian Häne, Shubham Tulsiani, Jitendra Malik, Jared Lafer, Ramsey Jones, Siyuan Li, Jie- Yan Lu, Shi Jin, Jingyi Yu, Qixing Huang. Evangelos Kalogerakis, Silvio Savarese, Pat Hanrahan, Thomas A. Funkhouser, Hao Su, and Leonidas J. Guibas. Large-scale 3d shape reconstruction and segmentation from shapenet core55. ArXiv, abs/1710.06104, 2017.
[85] Martin Kilian, Niloy J. Mitra, and Helmut Pottmann. Geometric modeling in shape space. ACM Transactions on Graphics (SIGGRAPH), 26(3):#64, 1-8, 2007.
[86] Hanbyul Joo et al. Panoptic studio: A massively multiview system for social interaction capture. TPAMI, 2017.
[87] Marina Strocchi, Christoph M. Augustin, Matthias A. F. Gsell, Elias Karabelas, Aurel Neic, Karli Gillette, Orod Razeghi, Anton J. Prassl, Edward J. Vigmond, Jonathan M. Behar, Justin Gould, Baldeep Sidhu, Christopher A. Rinaldi, Martin J. Bishop, Gemot Plank, and Steven A. Niederer. A publicly available virtual cohort of four-chamber heart meshes for cardiac electro- mechanics simulations. PLOS ONE, 15())e00231155 June 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 6
15

---

(c) Did you discuss any potential negative societal impacts of your work? [N/A] We do see any negative societal impact of our work.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A] We do not include theoretical results; all assumptions are reported Section 3
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [No]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 5.1
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount of compute and the type of resources used (e.g.,
type of GPUs, internal cluster, or cloud provider)? [Yes] Details reported in the supplementary
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 5.1
(b) Did you mention the license of the assets? [N/A] The involved assets are publicly available for research purpose; check the related references for further details.
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [N/A] The involved data are publicly available for research purpose; check the related references for further details.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] The involved data are publicly available for research purpose; check the related references for further details.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
16


Reviews:
[{"review_id": "nkHPZvzLwF0", "reviewer": "Reviewer_86v4", "summary": "The paper computes the correspondences between two non-rigid 3d objects with point cloud or mesh representations. From given 3D inputs with only limited supervision, the proposed method effectively understands the correspondences and matches suitable parts between two different 3D objects. The authors proposed how to represent a parametrized compact deformation in the problem of 3D matching. In the experimental section, the authors have shown the effectiveness of their proposed method in terms of non-rigid shape matching, registration, unsupervised part segmentation, and interpolation while comparing to each baseline method. The paper is also positive to publicly share their codes and models proposed in the paper.\n", "questions": "- In the end, does the matching be done by 3D Point Cloud instead of mesh? In case of mesh input, will the conversion must be processed in order to translate 3D Point Cloud?\n\n- Why does the compact representation reduce the number of training samples? It would be better to have a discussion in the experimental section, even if it is just a citation of related studies and their discussion.\n", "limitations": "- I can understand the paper limitation due to 9-page restriction, however, the discussion part is not sufficient in order to judge the effectiveness in the proposed method. At the current revision, the authors list the fact of results from performance rates and qualitative results in figures. More detailed discussion should be required such as \"why is the proposed method effective in the category 'twisted'\", and \"why can the proposed method train with a limited labeled 3D data\". I believe that a more detailed discussion improves the paper quality. \n", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "[Strengths]\nS1) The MLP-based representation approximates fixed-length parameters. The compact 3D representation matches a high-level matching between two different objects, despite the small number of training samples are worthy of evaluation. By using the proposed method, the authors have achieved good performance rates in comparison with conventional methods and is expected to become a good baseline in this field.\n\nS2) Experimental comparisons have been made with the respective baseline methods in diverse tasks (non-rigid shape matching, registration, unsupervised part segmentation, and interpolation) to demonstrate their superiority. These comparisons show the effectiveness of the proposed method in wide-range of tasks.\n\nS3) The code distribution in itself has a positive impact to the 3D matching community. In terms of reproducing a 3D matching method that is compact yet achieves high performance, it is easier for readers to try the proposed method. Additionally, the code would make the paper more understandable.\n\n[Weaknesses]\nW1) While the proposed paper has very good properties - a compact feature representation, small amount of training samples, and higher matching performance - there exists weaknesses that the detailed parameters are not well demonstrated through experimental section. \n\nW2) Although the number of training samples are suppressed which is a good aspect in the paper, the lack of a relationship between the #training-sample and their performance is insufficient. There is no guarantee as to how much training sample is required in order to reach a sufficient level of 3D matching performance. The authors can appeal to the research community if the proposed method enables to train a well-deserved 3D matcher from extremely limited number of training samples.\n\nW3) Eq. (6) shows the objective functions, however, the mathematical equation is not validated in terms of their combination and contribution to the training at each loss function. It would be better if the loss functions of importance are clarified.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "QSyi5OX5RWC", "reviewer": "Reviewer_u8B8", "summary": "The paper describes a method to compute correspondences from shapes with application for shape registration, shape interpolation and unsupervised Segmentation Transfer. The authors use an implicit (meshless) shape reconstruction based on the moving least squares as a shape representation. An autoencoder provides a way to create a compact representation of shapes which is then used to solve correspondence problems.", "questions": "- The choice of the control points seems to be essential to obtain a compact representation that captures the important details. Explain how to properly choose the number K of those points ? Why not make it as large as possible ? How the local radius r_i are chosen ?\n\n- It is not clear where the section \"Extending sparse to dense Correspondence\" and Eq(12) is used in the result section. Make this more explicit  or remove the section.\n\n- In the test time refinement, it is difficult to understand what is being done without reading the 3D-CODED paper. Improve this section. Why is the optimization taking place only for Z_y and not for Z_X ? This breaks the symmetry between the 2 shapes.\n", "limitations": "- The authors have cited 2 limitations (non diffeomorphic deformations + optimization of control points) which are relevant", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths : \ni) Novel shape representation for shape encoding inspired from meshless representations.\nii) extensive experiments showing the ability of the proposed approach to solve  shape registration, shape interpolation and unsupervised segmentation transfer problems.\niii) The paper is fairly well written with a lot of implementation details in the supplementary material\n\nweaknesses :\ni) the considered problem assumes that the correspondences between shapes are known at training time \nii) Only displacement fields are estimated that may lead to non diffeomorphic deformations. This may be a limitation \niii) the main concept of the paper is borrowed from 3D-CODED but the shape representation is original ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "6YxAVTezUtX", "reviewer": "Reviewer_DTPf", "summary": "This work addresses the issue of attaining correspondence between non-rigid objects. \n\nConventionally, methods using intrinsic representation require accurate mesh for each object, whereas extrinsic methods require various annotated data for training to achieve accurate correspondences. The authors propose to take an intermediate approach and use ''nodes'', or fixed control points that determine the deformation of the object.\n\nTo achieve this goal, they define the influence of each nodes using a shape function based on moving least squares, an established deformation model, and propose to learn the corresponding deformation parameters that determine the shape. One of the shapes is selected as a template shape, and the deformation parameters required from the template to each target is learnt.\n\nTo constrain the learnt deformation, the paper proposes to add various constraints using the Jacobian of the deformation field, which can be precomputed for the template shape, making the calculation significantly efficient. Furthermore, they add a constraint on the latent space so that intermediate shapes between the template and the target can be generated by traversing a continuous path in the resulting latent space.", "questions": "What kind of poses would be difficult to be used as the template?\n\nI am curious as to why the data-heavy baselines such as 3D-CODED cannot achieve high correspondence accuracy compared to the proposal. The authors mention deformation requiring less parameters, but the deformation model used in the method is rather conventional and presumably less flexible compared to the learnt one. What was the key difference? Is the lack of constraints in the prior methods the main source of weakness?\n\nIn Section 4.2, the authors mention that correspondence between unknown shapes $(\\mathcal{X}, \\mathcal{Y})$ is found by extracting mappings $(\\mathcal{D}_\\mathcal{X},\\mathcal{D}_\\mathcal{Y})$ and then finding the closest neighbors. However, if I understood correctly, doesn't Dx and Dy represent deformation required for template $\\mathcal{T}$ to deform to $\\mathcal{X}$ and  $\\mathcal{Y}$? If so, the end poses are different and nearest point would unlikely be true correspondences. I think the inverse transformation is required to establish correspondence. I would like some clarification regarding the test phase. Or, does this mean that $\\mathcal{D}_\\mathcal{X}$and $\\mathcal{D}_\\mathcal{Y}$ are templates transformed to \\mathcal{X} and \\mathcal{Y}, and closest points corresponding to each point on the deformed templates in $\\mathcal{D}_\\mathcal{X}$and $\\mathcal{D}_\\mathcal{Y}$ are considered correspondences? This would make sense but is not very obvious from the text.\n\nIn many cases involving human figures, the head does not seem to be turning. Is there any particular reason for this phenomenon?\n\nWhat would happen if the nodes were more evenly distributed, without removing any from near the segment boundaries? Would the experimental results be worse?", "limitations": "The authors do disucss the limitation of the proposal. One is the bijectivity of the correspondence, and the other deals with the fixation of the nodes. \n\nI would also like to see what would happen for shapes whose parts are difficult to determine (e.g. human organs, etc.), as the term ''non-rigid'' includes such objects as well. In other words, the title seems to suggest broader application, whereas the paper rather mainly targets articulated shapes.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\nI was very intrigued by how the authors utilized well-established algorithms and models in conventional non-rigid shape analysis works, such as moving least squares and constraints using the Jacobian of deformation fields, and focus the learning on the corresponding deformation parameters, and not the point coordinates themselves. This enables the proposed method to effectively recover deformation parameters around the fixed nodes, which can be diffused to the surrounding regions to estimate the deformation field around the entire shape. Various experiments show that the resulting correspondence is highly accurate, indicating that the deformations were properly conducted.\n\nThe overall accuracy, especially with respect to achieving proper correspondence among different poses is very impressive. I presumed that data-driven extrinsic methods would perform rather well, however, according to the manuscript, the proposal is able to achieve highly accurate correspondence while using less data for training. This approach seems useful in practice, as it does not require accurate meshes.\n\nThe continuous constraint on the latent space is also interesting, as it enabled smooth transition between each of the source pose and the template. The intermediate poses in Fig. 5 demonstrate the effectiveness of such constraint. The authors also thoroughly demonstrated the role of each constraint in the supplementary material, which further supported the necessity of each element.\n\n### Weaknesses\n\nThe node sampling strategy in Section 4.4 seems to imply that the deformation is mainly articulation-based, rather than a fully non-rigid one. I was actually curious about why this segmentation-based sampling is required, as I presumed that the deformation function based on moving least squares would nicely blend the influence from each node based on the distance away from it. I think this results in a rather curved corners, as can be seen from the purple figure in Fig. 5. The elbows seem rather rubbery, as the deformation parameters in those regions presumably had to be approximated from afar.\n\nExperiments using shapes that have topological differences, e.g. in Fig. 4, seem to be random and do not support the true contributions of the proposed method. As the constraints and losses are designed to match corresponding shapes there is not much point in applying the method to shapes where only semantic correspondence exist, despite seemingly providing robust correspondence where available.\n\nThe detail about the order of the polynomial basis $p$ used in the experiments seems to be missing.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "0NJH5JT9ll", "reviewer": "Reviewer_6kov", "summary": "This paper presents an efficient approximation of deformation fields using nodes plus local weighted least-square fitting. The proposed representation enables easy computation of Jacobian and thus facilitates first-order regularizations. Experiments on multiple tasks show that it performs well compared to recent existing methods.", "questions": "Questions:\n- In the node sampling in Section 4.4 it says the method \"exclude(s) a node that exerts its influence in semantically different regions\" (line 223), does it mean that semantic segmentations/annotations are implicitly used? -- although it seems to be a minor issue as the sampling may not be the core of the proposed method, using this information still sounds improper as they can be strong hints to correspondences (or in other words, if you already have access to this information, why not also using them in the correspondence computation...). I wonder how the method works if the node samplings involve no semantics? Or correct me with how \"semantically different regions\" are defined -- my understanding is that the segmentations are from SMPL[30] ground-truth annotations.\n\nSuggestions:\n- In Section 3.2, it can be made clearer which contributions are from prior works and which are proposed in this paper. For instance, the Moving Least Squares are from [16], but for the following shape representations, are they simply existing formulations, or are there any adaptations to the specific settings here?\n\nOther minor suggestions:\n- (line 3) \"Differently from\" -> \"Different from\"\n- The plot in Figure 5 left is too small. It's a bit hard to read the caption and legends.", "limitations": "The authors have addressed the limitations of their work.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper introduces a deformation field representation for more efficient learning of shape correspondences. It also enables easy computations of multiple regularization losses based on its accessibility to the Jacobian.\n- Good experimental results show the effectiveness of the proposed method compared to prior works. The authors also presented multiple applications such as interpolation and unsupervised segmentation.\n\nWeaknesses:\n- I feel the point of preserving \"high-frequency components\" (claimed in line 32-37) is not well addressed in the experiments. Apart from the generic tasks, specific experiments demonstrating the method's superiority on this can make the argument stronger.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept