GenCorres: Consistent Shape Matching via Coupled Implicit-Explicit Shape Generative Models
Authors: Haitao Yang
Xiangru Huang
Bo Sun
Chandrajit L. Bajaj
Qixing Huang
Venue: ICLR 2024 poster (2024)
Keywords: Joint shape matching;shape generator;implicit representation;geometric regularization
Primary area: generative models

Abstract: This paper introduces GenCorres, a novel unsupervised joint shape matching (JSM) approach. Our key idea is to learn a mesh generator to fit an unorganized deformable shape collection while constraining deformations between adjacent synthetic shapes to preserve geometric structures such as local rigidity and local conformality. GenCorres presents three appealing advantages over existing JSM techniques. First, GenCorres performs JSM among a synthetic shape collection whose size is much bigger than the input shapes and fully leverages the datadriven power of JSM. Second, GenCorres unifies consistent shape matching and pairwise matching (i.e., by enforcing deformation priors between adjacent synthetic shapes). Third, the generator provides a concise encoding of consistent shape correspondences. However, learning a mesh generator from an unorganized shape collection is challenging, requiring a good initialization. GenCorres addresses this issue by learning an implicit generator from the input shapes, which provides intermediate shapes between two arbitrary shapes. We introduce a novel approach for computing correspondences between adjacent implicit surfaces, which we use to regularize the implicit generator. Synthetic shapes of the implicit generator then guide initial fittings (i.e., via template-based deformation) for learning the mesh generator. Experimental results show that GenCorres considerably outperforms state-of-the-art JSM techniques. The synthetic shapes of GenCorres also achieve salient performance gains against state-of-the-art deformable shape generators.

Published as a conference paper at ICLR 2024
GENCORRES: CONSISTENT SHAPE MATCHING VIA COUPLED IMPLICIT-EPPLICT SHAPE GENERATIVE MODELS
Haitao Yang', Xiangru Huang", Bo Sun', Chandrajit Bajaj 1, Qixing Huang"
Thhe University of Texas Austin
MIT CSAIL
ABSTRACT
This paper introduces GenCorres, novel unsupervised joint shape matching (JSM) approach. Our key idea is to learn mesh generator to fit unorganized deformable shape collection while constraining deformations between adjacent synthetic shapes to preserve geometric structures such as local rigidity and local conformality. GenCorres presents three appealing advantages over existing JSM techniques. First, GenCorres performs JSM among synthetic shape collection whose size is much bigger than the input shapes and fully leverages the data- driven power of JSM. Second, GenCorres unifies consistent shape matching and pairwise matching (i.e., by enforcing deformation priors between adjacent synthetic shapes). Third, the generator provides concise encoding of consistent shape correspondences. However, learning a mesh generator from an unorganized shape collection is challenging, requiring a good initialization. GenCorres addresses this issue by learning an implicit generator from the input shapes, which provides intermediate shapes between two arbitrary shapes. We introduce novel approach for computing correspondences between adjacent implicit surfaces, which we use to regularize the implicit generator. Synthetic shapes of the implicit generator then guide initial fittings (i.e., via template-based deformation) for learning the mesh generator. Experimental results show that GenCorres considerably outperforms state-of-the-ard JSM techniques. The synthetic shapes of GenCorres also achieve salient performance gains against state-of-the-a deformable shape generators.
INTRODUCTION
Shape matching a long-standing problem with rich applications in texture transfer (Schreiner et al., 2004), compatible remeshing (Kraevoy & Sheffer 2004), shape morphing (Eisenberger et al. 2022]) deformation transfer (Sumner & [opović[[20xx to name just few. It also provides foundations for analyzing and processing shape collections (Kim et al. 2012 Huang al. [014 2019a). As the sizes and variations of geometric shape collections continue to grow, there are fundamental challenges in formulating and solving the shape matching problems. Pairwise approaches work for similar shape pairs and become less effective on less similar shapes. The real difficulties lie in developing suitable matching potentials (that factor out usually unknown inter-shape variations) and non-convexity the induced non-convex optimization problems.
In contrast to pairwise matching, joint shape matching (JSM) simultaneously optimizes consistent
correspondences among a shape collection (Nguyen et al. 2011 Huang et al. 2012; Kim et al.) 2012, Huang & Guibas 2013 Wang & Singer 2013 Huang et al. 2014: 2019b 2020b a). These techniques bypass the difficulty of matching two different shapes through paths similar shape pairs. Despite significant advances on this topic, existing approaches present three challenges. The first is to obtain sufficiently large dataset so that each shape has neighboring shapes where shape matching succeeds. The second is that pairwise inputs are usually detached from joint matching. Third, encoding consistent dense correspondences is costly for large shape collections.
This paper presents GenCorres for solving the JSM problem. GenCorres takes motivations from recent advances in neural shape generators. Given a collection of shapes with no inter-shape correspondences, GenCorres seeks to learn a mesh generator to fit the input shapes while constraining deformations between adjacent synthetic shapes to preserve geometric structures such as local rigidity and conformality (See Figure T). Interestingly, this simple framework addresses all the challenges of JSM. GenCorres performs JSM among synthetic shapes, whose size is much larger than the number of input shapes. Second, shape matching is done among neighhoring shapes through the local

---

Published as a conference paper at ICLR 2024
rigidity and local conformality potentials, bypassing the difficulty of crafting non-linear objective function between less similar shapes. In addition, GenCorres unifies pairwise matching (i.e., through deformation priors between adjacent shapes) and consistent matching (i.e., through the generator). Furthermore, the mesh generator provides an efficient encoding of shape correspondences.
However, learning the mesh generator directly from the input shapes is challenging as it requires good initializations. Moreover, optimization pro- cedures, e.g., that minimize the earth-mover dis- tances between synthetic shapes and training shapes (Fan al., 2017; AAhhiiptas et al. 2018). can easily get trapped into local minmumm. Gen- Corres addresses this issue by learning an im- plicit shape generator from the input shapes. The formulation builds on novel approach for estab- lishing dense correspondences between adjacent implicit surfaces defined by the shape genera- tor. GenCorres enforces these correspondences to preserve local rigidity and conformality be- tween pairs of adjacent shapes and satisfy the cy- cle consistency constraint among adjacent shape triplets. These constraints modeled as regu- larization terms for learning the implicit shape generator. GenCorres then converts the learned implicit generator into an explicit mesh generator. The implicit generator offers initial consistent correspondences by guiding template-basee reg- istration.
w/o Regu.
w/ Regu.
Implicit Generator
Explicit Generator
Figure I: GenCorres performs consistent shape match- ing by Icarning a coupled implicit and mesh (explicit) generator to fit a shape collection without pre pre-defined correspondences. (Left) Interpolation between a pair of shape in the shape space. Constraining deformations between adjacent synthetic shapes with the regulariza- tion loss improves the shape space. (Right) The mesh generator provides consistent correspondences between pairs of shapes.
We have evaluated GenCorres on various deformable shape collections, including humans and animals. Experimental results show that GenCorres outperforms stateoofthhe-ae JSM approaches and implicit and point cloud shape generators, making GenCorres a universal framework for computing JSM and learning deformable implicit shape generators. An ablation study justifies the importance different components of GenCorres.
2 RELATED WORK
We discuss relevant work under five topics, which are described below.
Pairwise shape matching, Pairwise shape matching has been studied extensively in the litera- ture (Sahillioglu. 2020 van Kaick et al. 20100 Kim et al., 2011 Ovsjanikov et al. 2012. Aigerman
al. 2014; Maron al. 2016 Melzi al.) 2019; Bednarik et 2021). recent line papers
establishe ishes learning Iramework under the functional map representation (Litany el al. 2017 Halimi
ct al. 2019 Donati et al. 2020 Sharma & Ovsjanikov] 2020 Cao et al. 2003). However, existing
techniques still do not work well for less sumilar shape pairs, where it challenging to learn suitable matching objective functions.
NeuroMorph (Eisenberger et al., 2021) combines a correspondence module with a shape interpolation module. The network is trained in an unsupervised manner. Several other methods (Eisenberger et al., 2019; Eisenberger & Cremers 2020) also optimize interpolation paths to establish correspondences. While GenCorres is relevant these approaches, GenCorres is data-driven approach that uses an implicit generator with learned representations from all input shapes to drive pairwise matching.
Joint shape matching. The underlying principle of joint shape matching (JSM) techniques (Nguyen 2011 Huang et al., 2012) Kim et al.] 2012) Huang & Guibas 2013 Wang & Singer 2013 2214 Chen et 2014 2014 Huang al. 2019b 2020b
et al.
1s cycle-consistency. State-ot- the-art JSM techniques use the equivalence between the cycle- consistenn constraint and the data
Huang al.
matrix's low-rank property, which encodes pairwise maps in blocks (c.f. (Huang & Guibas )).3)).
This leads to constrained low-rank matrix recovery approaches (Huang & Guibas Wang &
Singer 2013 Chen al. 2014 Huang al. 2017 Bajaj et al. 2018 Huang et al. 2019b), which
possess strong theoretical guarantees.
GenCorres advances JSM in multiple ways. First, JSM's performance improves when the input
collection size increases, as each shape can have more similar shapes for pairwise shape match- ing (Nguyen et al. 2011 Huang al. 2012 Kim et al. 2012 Huang & Guiiba 2013). The advantage
2

---

Published as a conference paper at ICLR 2024
of GenCorres is that it utilizes large collection of synthetic shapes and fully leverages the data-driven behavior of JSM. Second. in prior methods, joint matching and pairwise matching are typically decou- pled. CZO Huang et al. (020a) is an exception, yet it still requires good initializations, e.g. (Kim) et al. 2011). In contrast, GenCorres unifies pairwise matching and joint matching under a simple tormulation. Cycle consistency is automatically enforced through the generator. Moreover, JSM performs pairwise matching among neighboring shapes through simple geometric regularizations. Finally, JSM still requires storing consistent matches across the input shape collection (Huang et al..
2012: Kim et al. 2012 Huang et al. 2014). GenCorres addresses this issue using a shape generator compress consistent correspondences effectively.
Generative model based correspondences. Generative models under explicit representations provide inter-shape correspondences, making them appealing for practical applications. However, existing methods are sub-optimal for high-fidelity correspondence computation. Most mesh-based generators (Tan et al. 20188] Verma et al.) [2018] 2018,Litany et al. 2018: Tretschk et al. 2020 Rakotosaonal & Ovsjanikov] 2020 Murahkrishhan et al. 2022) require consistent dense correspondences as input. In contrast to mesh- based generators, point point-based generators (Fan et al. 201 (017] Achlioptas et al., 2018) Yang et al. 2018 Li ct al. b0119abb do not require inter-shape correspondences. The downside is that point cloud vermutation-mnaante
Therefore, the point indices in a point cloud do not always reflect meaningful correspondences. GenCorres addresses these limitations by performing shape matching under implicit representations using seape-preeervig potentials.
3D-CODED (Groueix et al., 2018) adopts an auto-encoder to deform a template shape for shape matching. The tramiig combines the Chamfer distance for shape alignment and regularizations on Laplacian operators and edge lengths. As the regularizations of 3D-CODED are designed for isometric deformations, and the Chamfer distance drives training, it mainly applies to minor inter- shape variations. In contrast, GenCorres applies to shape collections under large deformations.
Matching under implicit surfaces. A fundamental problem for neural implicit shape representation defining inter-shape correspondences. The technical challenge is that there is only one constraint along the normal direction at each surface point, c.f., (Stam & Schmidt, 2011). GenCorres solves a constrained optimization problem to obtain inter-shape correspondences. A relevant formulation has been studied in (Tao et al. 2016). Under the implicit representation, popular way to regularize local rigidity is the Killing vector field approach (Ben-Chhn et al., 2010 Solomon et al., 2011] Tao ct al. 2016, Slavcheva et al. 2017), which is correspondence-ree In contrast, the correspondences computed by GenCorres allow us to introduce the cycle-come sice--consistency regularization.
Neural implicit representations. Neural implicit representations have received significant interest on modeling 3D shapes, including man-made objects (Park et al. (2019] Mescheder el al. 2019) Chen
Zhang 2019; Deng et al. (02T) and deformable objects (Satto et al. 2019] 2020 Alldieck et al.
2021 Peng et al. 1021). Unlike developing novel implicit network architectures, GenCorres locuses on regularization losses that enforce geometric priors for deformable objects.
Developing regularization losses for training implicit neural networks has also been studied re- cently (Gropp et al. 2020 Atzmmnn et al. 202I). GenCorres is most relevant to (Atzmon et al.) 2021). I. which uses an os-killing-as-possbbe regularization loss to preserve global rigidity. In contrast, GenCorres focuses on maintaining local rigidity and conformality.
GenCorres is also relevant to ARAPReg (Huang et al. 2021). However, defining suitable loss under the implicit representation has to address the fundamental challenge of determining inter-shape correspondences. GenCorres also enforces the cycle-consistency constraint among induced shape correspondences to enhance the implicit generator.
PROBLEM STATEMENT AND APPROACH OVERVIEW
Problem statement. The input to GenCorres is shape collection S (S,,... ,.)) C C, where S is the underlying shape space. Each shape S, can be a raw mesh or a raw point cloud.
GenCorres seeks to learn a mesh generator m Z + S. where 0 are the network parameters, 2 : Rd is the latent space. Our goal is to align each input shape S, with the corresponding synthetic shape m (z;) where z; E IRd is the latent code of S, The mesh generator then provides consistent inter-shape correspondences.
Approach overview. As illustrated in Fig. 2 GenCorres proceeds in three stages. The first two stages provide initializations for the third stage, which learns the mesh generator. Specifically, the first stage adopts variational auto-encoder (VAE) to lear implicit generator f RR3 Z IR and an
3

---

Published as a conference paper at ICLR 2024
fo
Mesh Template
Input Shapes
Unorganized
Shapes
Latent Space Interpolation
eshh G Gnneatorr Initialization
Mesh G Generator Refinement
Implicit Shape G Generator
Figure 2: GenCorres has three stages. first stage learns an implicit shape generator
input shapes.
The training loss regularizes the induced correspondences between adjacent implicit shapes of the generator. second stage uses the implicit generator imitialize a mesh generator through latent space interpolation and template matching. The third stage then refines mesh generator with ACAP energy.
encoder h from the input shapes:
min IvAR (1"11 + >genTTgoa (c0 + AeyeTeye
where & and to are the network parameters, !VAE is a VAE loss on the training shapes, Ageo and Aeye are the weights the regularization terms. (f and rfyc(f%), which are key contributions of this paper, build on induced correspondences between adjacent implicit shapes defined by f Specifically,
T (f%) enforces that the induced correspondences preserve local geometric structures. Teye (f) enforces that the induced correspondences are cycle-consistent. other words, ()) and raye ( perform pairwise matching and consistent matching, respectively. The second stage of GenCorres fits template mesh to all input shapes along paths of interpolated shapes provided by the implicit shape generator. The resulting correspondences are used to learn an initial mesh generator m The third stage of GenCorres refines the mesh generator by solving another optimization problem:
min deep (m" () + >aTa(m²)
where deep (m²,3) aligns the explicit generator with the input shape collection; ra(m²) enforces ---conformall-a-poosiitle deformation prior among adjacent shapes; À is the weight of ra(m)).
4 STAGE I: IMPLICIT SHAPE GENERATOR
This section introduces how learn the implicit shape generator f We begin with novel approach for computing dense correspondences between adjacent implicit surfaces in Section [.1 Based on the induced correspondences. we introduce two regularization terms (f) and (f in Section 4.2
and Section 4.3 respectively. Finally, Section 4.4 elaborates on the implementation details.
4.1 INDUCED SHAPE CORRESPONDENCES
Our goal to compute the dense correspondences between the implicit surface fd 2) = and an adjacent implicit surface f (x, + cu) = 0, where æ E I e E Rd is a direction in the unit ball g and is an infinitesimal value. The computation is nontrivial because of the difficulties in representing correspondences for the implicit surfaces. To this end, we first discretize (0x (x. 2) = 0 using a mesh
with n vertices g (z) E Raá, e.g., via Marching cube (Lorensen & Cline 1987). We then formulate
the corresponding vertices of g (z) on (ax, z ev) as (z
g ((z) + d d=(2) E 3n
With this formulation, computing correspondences between two implicit surfaces is reduced to the computation of d' (z). As discussed in (Stam & Schmidt 2011 Tao et al. 2016)), for each vertex
gf (z) 3 the implicit representation offers one constraint on its corresponding d, (x) IR3 along the normal direction:
0 (g) (z), (x) + 0f Oz (g) (z), (z), 0.
(3)
dx
To introduce extra constraints on d' (2), we enforce that the displacements of the 1-ring patch at each vertex gf(2 (2) as-rigid-as possible (ARAP) (Alexa al. 2000 Huang al. 2009
2021
4

---

Published as a conference paper at ICLR 2024
0.05
(a)
(d)
(1)
Figure 3: (Ieft) Effects of the geometric deformation regularization loss (f ). We compute 30 interpolated shapes between a source shape (a) and a target shape (b) via linear interpolation between their latent codes. All the interpolated shapes visualized the same coordinate system. (c) Interpolation results without rgeo ( " (d) With (f)). (Right) Effects of the cycle-consistency regularization loss Tey( ). We color-code errors of propagated correspondences through path of intermediate shapes between cach source-target shape pair. The error visualized on the target mesh. (e) Without reyc (f=). (f) With reye( (.²))
and as-conformal-as possible (ACAP) Yoshiyasu et al. approximate the latent rotation at g. (z) as I3 + xx This leads an ARAP potential on d' d2) as
2014). In the infinitesimal regime, we
min Σ l|e x (g) - (z)=) (df di (=))] d (g) (4)
JEN
where the expression of L arup (g) (z)) is the supp. material.
Similarly, we can parameterize the latent similarity transformation at gf (z) as (1 + 8 + cxx and define the ACAP potential
Σ min Σ 11(sTT +1(1ss+ 99 (d) d) (=)) TT (g (z)) JEN;
(5)
where the expression of Iacap (g (z)) is in the supp. material.
Denote (z) a (g)(2)) (z)) + I ( (z)) where a is tradeoff parameter (a 10 in our experiments). We compute d' (z) via linearly constrained quadratic programming:
d" (z) lim argmin +00 d
d'I + #lld]l s.t. C (2F
(6)
where (2)d - FFF is the matrix representation of 3.. (* E R= is block diagonal sparse matrix, (z) E Radd p is used to avoid degenerate cases, e.g., rotating sphere. The expressions of (z) and (z) (z) are in the supp. material. It is easy to check that
d= -e
(z) C (C) (
(z) {(())))((()))))
(7)
where 1+ denotes the Moore-Penrose inverse of A.
4.2 GEOMETRIC DEFORMATION REGULARIZATION LOSS
We proceed to introduce the first regularization loss rooo (f²), which penalizes local rigidity and local conformality distortions of the induced correspondences from f (x, x) = to fo + +vv 0:
(f
I Fgeu (z, v) = d (z) (z)d
E (z)x
(8)
(=) (C* (x) (z)L((z)))))))))) (*)))()))))))))))))))))))))))))) (z)
Integrating a over the unit ball Bdd in Rdd (Huang al. 2021 and omitting the constant e², we define
(a) Ex~Na [ Joes, o7 E (z) Ez-NG
Vol( Tr(E Tr))
d
Figure 3 (Left) shows that rgen (f*) can improve the quality of the implicit shape generator. The
interpolated shapes are smoother and more shape-preserving. leading better shape space.
5

---

Published as a conference paper at ICLR 2024
4.3 CYCLE- CONSISTENCY REGULARIZATION LOss
The induced correspondences defined enable to compute correspondences between two shapes by composing induced correspondences along path of intermediate shapes. An additional regularization we can enforce is that the induced correspondences are wycle-consistent. To this end, constrain 3-cycle consistency (Huang & Guibas 2013) among three neighboring synthetic shapes f (xx, 2) 0, (x, + cv) = 0. and {z, + co') = 0, where v and v' are two different displacement vectors. Formally speaking, model 3-cycle distortion as
r n (2) = d' (2) d (2 (2 (v) - d (2) (z) 2 0
Oz (v (1-
(10)
Based on 10 we define the cycle-consistency regularization term as
/
0G
Taye (J¹) Eg
do
(11)
VEB
Oz
where - is the tensor Frobienus norm. We use finite-difference to compute Teye ( Specifically,
we compute 1 +|G (z + Ecyce;)
(z)113 as an approximation of rfyc(f%), where e, is a random we quantitatively show that reyo further enhances the shape
standard basis in Rdd In Section
space.
4.4 IMPLEMENTATION DETAILS
We use the VAE network proposed in SALD (Atzmon & Lipman 2021). where the encoder h a modified PointNet QQ et al. 2017) and the decoder f is an 8-layer MLP. The data loss IvAE is the VAE loss of SALD. We set Àgeo le-3 Auye le-1, and le-3, We use autograd in PyTorch (Paszke et al. 2019) compute (2) and C* For other derivative computations, use finite-difference for approximation. More details are deferred to the supp. material.
5 STAGE II: MESH GENERATOR INITIALIZATION
The second stage initializes the mesh generator m using the implicit shape generator f obtained in the previous stage. GenCorres uses the same mesh generator as ARAPReg (Huang al. (021), which maps the latent code to displacement vectors associated with vertices of template mesh M. We use the learned encoder htth to find the latent code ztemp of .. Let 2; = h (S;) be the latent code of the input shape S,. We generate T intermediate shapes g ( ) S < T (T 10 in Sump, We then apply non-rigid
experiments) by linearly interpolating ztemp and z; z = Ztemp + T+1
registration align the template mesh M with each intermediate shape (z in order, i.e., the alignment of one intermediate shape provides the initialization for aligning the next intermediate
shape. Non-rigid alignment adopts an ARAP deformation energy, and the details are deferred to the supp. material.
After propagating the correspondences along the interpolation path in the shape space, we obtain the deformed template mi!i for each input shape S, We then initialize the mesh generator m using the standard regression loss:
ginit argmin
m init mz (z.)
(12)
6 STAGE III: MESH GENERATOR REFINEMENT
The third stage refines the mesh generator m (z) between the mesh generator and the input shape collection as
) by solving
2
To this end, we define the distance
deep (m' ) = - (m
(13)
where m ) is the i-th generated mesh, LCD is the Chamfer loss. The loss can be optimized robustly thanks to the good initialization of the mesh generator from the first two stages. As (3

---

Published as a conference paper at ICLR 2024
(a)
(b)
(c)
(c)
Figure 4: The mesh generator improves the ihter-shape correspondences by learning better shape generation. (a) the deformed template from stage II. (b) the shape generated by the mesh generator. (c) the input raw mesh.
only constrains that vertices of the mesh generator lie on the surface, merely minimizing it does not avoid drifting. To address this issue, we define the regularization term ra ra(m² mm) to enforce that the deformations between meshes with similar latent codes preserve geometric structures. We enforce the deformations be ACAP, which allows the mesh generator to capture large non-rigid deformations. Based on (5). we define
/
dm o²
(z) Oz
ra(m²) EgzNaa
L ( (z))
vdv.
(14)
vEBa
Oz
We then substitute 113 and (14) into (2) to refine the mesh generator. As shown in Figure 4 the mesh generator can improve the shape quality from the implicit generator. Higher shape generation quality implies better inter-shape correspondences since the mesh generator directly provides consistent correspondences.
7 EXPERIMENTAL EVALUATION
This section presents an experimental evaluation of GenCorres. We begin with the experimental setup in Section 7.1 Section 7.2 evaluates the shape generation quality of GenCorres. We proceed to compare GenCorres with safe--oftth-aar JSM approaches in Section 7.3 Section 7.4 evaluate
GenCorres on FAUST (Bogo et al. 2014 Ren et al. 2018). Section 7.5 presents an ablation study.
7.1 EXPERIMENTAL SETUP
Datasets. We evaluate GenCorres on two categories of deformable shape collections, i.e., Human and Animal. The Human category considers DFAUST (Bogo et al. (017) and FAUST (Bogo et al.) 2014). We use the registered SMPI. model from the original T' Since there is low variety between the adjacent shapes, subsample 2000 meshes from the original dataset. For FAUST, use the re-meshed version (Ren al., (018). Animal category has one dataset of 383 shapes (Huang let al. 2021), which is generated from SMAL (Zuffi et al. 2017). Due to space constraints, we defer the details of dataset processing to the supp. material.
Evaluation protocols. We evaluate the quality of shape generation by measuring the reconstruction errors testing shapes, i.e., using the Chamfer distance between the reconstructed mesh and the original testing shape. For correspondence evaluation, we report the mean and median geodesic errors of the predicted correspondences between involved shape pairs.
7.2 EVALUATION ON SHAPP GENERATION QUALITY
We compare with the state-of-the-a art shape generation approaches that do not rely pre-defined ground-trufe -ouun correspondences. Those include implicit shape generators DeepSDF (Park al. 2019) and SALD (Atzmon & Lipman 2021), point-based generators, such as LGF (Cal et al. 2020) and DPM (Luo & Huj 2021). For Human category, we train the shape generator from 1000 shapes and evaluate them on the remaining 1000 shapes. For the Animal category, we 289 shapes for training and shapes for testing.
Table provides quantitative comparisons between GenCorres and baseline shape generators. Figure 3 shows the qualitative results. More comparisons are in the supp. material. GenCorres is superior to all baselines in terms of both reconstruction errors and plausibility of synthetic shapes. Quantitatively, the reductions in mean/median reconstruction errors are 2.7 2.7%/10.6%. 2.3% %%%%%0%% on DFAUST and SMAL,, respectively. Qualitatively, GenCorres provides much better interpolation results compared to SALD, especially in preserving the rigidity of arms and legs of the humans. These performance gains mainly come from the geometric deformation regularization loss employed by GenCorres.
7

---

Published as a conference paper at ICLR 2024
(a)
(b)
(c)
(b)
(d)
Figure 5: The comparison shape interpolation between SALD (Atzmon Lipman 2021 and our method on the DFAUST dataset. (a) source shape. (b) target shape. (c) interpolatron results of SALD. (d) our results.
Table 2:
Evaluations of JSM DFAUST and SMAL. using geodesic errors of the predicted corresnon- dences (in cm). Baselines described in Section 7.3
Table 1: Evaluations of shape generation quality. For each method, report the and median reconstruction errors (em) of the testing shapes. Baselines are described Section
DFAUST
SMAL
DFAUST
SMAL
median
median
mcan median mean median
CZO
3.71
3.68
1.19
1.12
MIM
3.42
3.40
1,30
1.29
LGF
4.62
2.30
9.13
8.15
Point
DPM
3.80
2.00
8.07
7.44
NeuroMorph
2.49
2.47
1.59
1.43
GenCorres
1.30
1.13
1.02
0.47
Implicit DeepSDF
2.03
1.98
7.84
7.59
Gen CenCorres-NoCyeee
1.41
1.11
0.51
SALD
1.88
1.79
7.66
7.32
7.65
7.34
6,28
5.09
GenCorres
1.83
1.75
6.85
6.81
1.62
1.42
1.07
0.49
GenCorres-Ini
2.62
1.81
1.24
0.53
7.3 EVALUATION ON JOINT SHAPE MATCHING
Table 2 reports statistics of GenCorres for JSM on DFAUST and SMAL. For baseline comparison, we choose consistent zoom out (C70) (Huang et al., 2020a) and multiple isometric matching (MIM) (Gao al. (02T), T), which are two State-of-the-art JSM approaches. We evaluate the methods by computing correspondence error between template shape to rest the shapes. We also report the performance of the top-perforning pair-wise matching approach NeuroMorph (Eisenberger et al. 202T) these pairs. Note that NeuroMorph is originally not designed for JSM problem. Overall, GenCorres outperforms both JSM baselines by large margins. Specifically, GenCorres reduces the mean/median errors by 62.0% 62.0%/66.8 5% and 14.3%/58.00 on DFAUST and SMAL, respectively. The performance gains come from two aspects. First, enforcing ARAP and ACAP deformations in the shape space locally is superior to applying sophisticated deformation models between pairs of shapes directly. Second, GenCorres performs map synchronization on synthetic shapes of the generator whose size is much larger than the input shape collection used by JSM baselines.
7.4 EVALUATION ON PAIR-WISS SHAPE MATCHING
Most the shape matching approaches are evaluated on the pairwise benchmark FAUST (Bogo et al. 2014) with 20 testing shapes. We apply JSM on these shapes using GenCorres. However,
Table 3: Evaluations of pair-wise matching on FAUST dataset (in cm). Baselines are described in Section 7
Both NM and our method apply nonrigid registration as post-processing step.
Axiomatic
Spectral Learning
Template Based
BCICP
70 S-Shells
GeoPM
AFmap D-Shells
ULRSSM NM 3D-CODED IFS Ours
error
6.4
6.1
2.5
1.9
1.9
1.7
1.6
1.6
2.5
2.6 1.6
8

---

Published as a conference paper at ICLR 2024
directly applying it 20 imput shapes of FAUST does not offer satisfactory results since learning deformable shape space from few shapes is very difficult. To show the advantage of GenCorres, we augment the input shapes with unorganized shapes from FAUST and DFAUST dataset, resulting in 1 100 shapes in total. The inter-shape correspondences between two input shapes are given by the correspondences induced from the template model.
Quantitative results shown in Table 3 We mainly compare with state-of-the-art template based approaches, including 3D-CODED (Groueix et al. 2018)) and IFS (Sundararaman et al. 2022). For completeness, we also provide the results of the axtomatic methods, inc including BCICP (Ren el al. 2018), Zo (Melzi et al. 2019), and S-Shells (Eisenberger al. 22200a;; and the spectral learning methods, including GeoFM (Donati et al. 2020), Aap (Li al. 2022), ULRSSM (Cao al. 2023). D-Shells (Eisenberger et al. 2020b) and NM (Eisenberger et al. 202T). Note that template based methods do not utulize intrinsic Ieatures, thus they usually have worse performance compared to spectral learning methods, especially in the region of ceff-interscction. GenCorres (Ours) outperforms all template based methods. It also achieves comparable performances to spectral learning methods.
How to incorporate intrinsic features into our pipeline is left for future research.
7.5 ABLATION STUDY
This section presents an ablation study on different components of GenCorres. As the main purpose of GenCorres is inter-shape correspondences, we focus on how the correspondence quality changes when varying different components of GenCorres (See Table 2}.
Without the cycle-consistency regularization. Dropping this term hurts the implicit generator. Quantitatively, the correspondence errors increase by 8.4% 97.%% and 8.8%/8.5% in mean/median on DFAUST and SMAL.
Without the geometric regularization. The performance of GenCorres drops considerably when removing the geometric deformation regularization term. The mean/median geodesic errors increase by 488%/549% and 515%/982% on DFAUST and SMAL. This shows that even the cycle-consistency constraint is enforced on the correspondences computed from optimizing ARAP and ACAP losses, constraining that these correspondences minimize ARAP and ACAP losses is critical.
ACAP versus ARAP. SenCorres-NoACAN replaces ACAP regularization with the ARAP regular- ization. As shown in Table 2 the performance of GenCorres slightly decreases. In particular, on DFAUST that exhibit large inter-shape deformations, i.c., thin versus fat and low versus tall, the performance drops are noticeable. Such performance gaps show that the ACAP regularization loss important for modeling large non-isometric inter-shape deformations.
No explicit generator. Finally, we drop the explicit generator and use the implicit shape gener- ator to propagate correspondences computed along linearly interpolated intermediate shapes, i.c. GenCorres-Imp The correspondences of the explicit generator (GenCorres) are superior prop- agated correspondences of the implicit generator, i.e., 50.4%/37.6% and 17.7%/11.39% of error reductions on DFAUST and SMAL. respectively. Such improvements are expected as propugated correspondences between shapes that undergo large deformations may drift.
CONCLUSIONS, LIMITATIONS, AND FUTURE WORK
This paper shows that learning shape generators from a collection of shapes leads to consistent inter-shape correspondences that considerably outperform state-of-the-art JSM approaches. The key novelties of GenCorres are the idea of using a mesh generator to formulate JSM and two regularization losses that enforce geometric structures are preserved and induced correspondences are cycle-consistent. We present extensive experimental results to justify the effecctivnnes of these two regularization terms. Besides high-quality inter-shape correspondences, GenCorres also outperforms Matteoof-the-an deformable shape generators trained from unorganized shape collections.
One limitation of GenCorres is that it requires a reasonably large input shape collection to learn the shape generator and does not work with few input shapes. In this latter regime, learning pairwise matching has the advantage over GenCorres. This issue may be partially addressed by using more advanced implicit generator for deformable shapes, which is
for future research.
There are ample future directions. So far, the regularization terms are based on discretizing implicit surfaces into meshes. An interesting question is how to define them without mesh discretization. Another direction is to explore regularization terms for man-made shapes, e.g., to enhance topological generalization and promote physical stability.
9

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
We would like to acknowledge NSF IIS-2047677,
and CCF-2019844.
REFERENCES
Panos Achlioptas, Olga Diamanti, Ioannis Milliagkas, and Leonidas J. Guibas. Learning representa- tions and generative models for 3d point clouds. In ICML, volume 80 of Proceedings of Machine Learning Research, pp. 40-49, Stockholm, Sweden, 2018. PMLR.
Noam Aigerman, Roi Poranne, and Yaron Lipman. Lifted bijections for low distortion surface mappings. ACM Trans. Graph., 33(4), jul 2014. ISSN 0730-0301. doi: 10.1145/26000972260.11 158. 10. 111.0.14554400
URL http:tttt//.o..oooo
2601158
Mare Alexa, Daniel Cohen-Or, and David Levin. As-rigid-as-possieee as-possible shape interpolation. In Judith R. Brown and Kurt Akeley (eds.). Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2000, New Orleans, LA, USA, July 23-28, 2000, pp- 157-164. ACM, 2000.
Thiemo Alldieck, Hongyi Xu, and Cristian Sminchisescu. imghum: Implicit generative models of 3d human shape and articulated pose. In ICCV, pp. 5441-,,50, Montreal, Canada, 2021. IEEE.
Matan Atzmon and Yaron Lipman. Sal: Sign agnostic learning of shapes from raw data. In IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), June 2020.
Matan Atzmon and Yaron Lipman. SALD: sign agnostic learning with derivatives, In 9th International Conference Learning Representations, ICLR 2021, 2021.
Matan Atzmon, David Novotný, Andrea Vedaldi, and Yaron Lipman. Augmenting implicit neural shape representations with explicit deformation fields, 2021.
Chandrajit Bajaj, Tingran Gao, Zihang He, Qixing Huang, and Zhenxiao Liang. SMAC: simultaneous mapping and clustering using spectral decompositions. Proceedings of the 35th International Conference on Machine Learning, 2018,018, pp. 334-343, Stockholmsmäässan, Stockholm, Swe- den, 2018. PMLR. URtpttt::/proceeeeee
m wpress/vv//baaaa...
html
Jan Bednarik, Vladimir Kim, Siddhartha Chaudhuri, Shaifali Parashar, Mathiuu Salzmann, Pascal Fua, and Noam Aigerman. Temporally-coherent surface reconstruction via metric-consistent atlases. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 10458-10467, October 2021.
Mirela Ben-Chen, Adrian Butscher, Justin Solomon, and Leonidas J. Guibas. On discrete killing vector fields and patterns surfaces. Comput. Graph. Forum, 29(5):1701-1711, 2010.
Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. FAUST: Dataset and evaluation for 3D mesh registration. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Piscataway, NJ, USA, June 2014. IEEE
Federica Bogo, Javier Romero, Gerard Pons-Moll, and Michael J. Black. Dynamic FAUST: registering human bodies in motion. In CVPR, pp. 5573-5582, Washington, DC, USA, 2017. IEBE Computer
Society.
Ruojin Cai, Guandao Yang, Hadar Averbuch-Elar Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan. Learning gradient fields for shape generation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pp. 364-381. Springer, 2020.
Dongliang Cao, Paul Roetzer, and Florian Bernard. Unsupervised learning of robust spectral shape matching. ACM Trans. Graph. 42(4)::32:1-132:1 2023. doi: 10.1145/3592107. URL
hittps://dol .rg. 1t.g//....1//3211110001
Yuxin Chen, Leonidas J. Guibas, and Qi-Xing Huang. Near-optimal joint object matching via convex relaxation. In ICML., volume 32 of JMLR Workshop and Conference Proceedings, pp. 100-108. Beijing, China, 2014. JMLR.org.
10

---

Published as a conference paper at ICLR 2024
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, Long Beach, CA, USA, 2019. Computer Vision USA, June 16-20, 2019, pp. (WWCCVRRR.01..0000 000.00000000000 URL htLphttp:/// //ttppp//ppnnccees. / thecvf Foundation / IEEE. doi: 10. 109//RR coam/conent. eeppeettt________ h001/html//e/ /hhen Learning wwwlimippiii.............miiiiie rieedsseeae for Sinerratvve_hhae___SM Model iig__nn_eee_ee 2019 wwwpppappprp....
Yu Deng, Jiaolong Yang, and Xin Tong. Deformed implicit field: Modeling 3d shapes with learned dense correspondence. In CVPR, pp. 10286-10296, Virtual, 2021. Computer Vision Foundation
IEEE.
Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. Deep geometric functional maps: Robust feature learning for shape correspondence. In CVPR, pp. 8589-8598, Washington, DC, NY, 2020.
Computer Vision Foundation/ IEEE.
Marvin Eisenberger and Daniel Cremers. Hamiltonian dynamics for real-world shape interpola- tion. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michate Frahm (eds.), Com- puter Vision ECCV 2020 16th European Conference. Glasgow, UK. August 23-28, 2020, Proceedings, Part IV, volume 12349 Lecture Notes in Computer Science, pp. 179-196, Virtual, 2020. Springer. doi: 1001990977 000--0300--888888 111888811.110 978 978-3-030-
10C7/
11
Marvin Eisenberger, Zorah Lähner, and Daniel Cremers. Divergence-free shape correspondence by deformation. Comput. Graph. Forum, 38(5):1-12, 2019.
Marvin Eisenberger, Zorah Lahner, and Daniel Cremers. Smooth shells: Multi-scale shape registration with functional maps. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020a.
Marvin Eisenberger, Aysim Toker, Laura Leal-Taixé, and Daniel Cremers. Deep shells: Un- supervised shape correspondence with optimal transport. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neu- ral Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, pp. 10491-10502, Virtual, 2020b. Cur- ran Associates, Inc. URL
wtt.:/owcceed.oonc.eeeeeeee.epee neurips cc httpc/ppper20000 /assh/ Ab--:e2222-bbtataac html
Marvin Eisenberger, David Novotny, Gael Kerchenbaum, Patrick Labatut, Natalia Neverova, Daniel Cremers, and Andrea Vedaldi. Neuromorph: Unsupervised shape interpolation and correspon- dence in one go. In Proceedings of the IEÉE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7473- 7783,, Virtual, June 2021. Computer Vision Foundation IEEE.
Haoqiang Fan, Hao Su, and Leonidas Guibas. A point set generation network for 3d object reconstruction from a single image. In CVPR, pp. 2463-2471, Honolulu, Hawaii, 2017. IEEE Computer Society.
Maolin Gao, Zorah Lahner, Johan Thunberg, Daniel Cremers, and Florian Bernard. Isometric multi- shape matching. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), pp. 14183-14193, Virtual, June 2021. Computer Vision Foundation / IEEE.
Michael Garland and Paul S. Heckbert. Surface simplification using quadrie error metrics. In Proceedings of the 24th Anmual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997, Los Angeles, CA, USA, August 3-8, 1997, pp. 209-216. ACM, 1997.
Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regulariza- tion for learning shapes. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 3789-3799. PMLR, 2020. URL http:://ppttt http:://pooeedings milr.press/vll gropp20a.htmm html
Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, and Mathieu Aubry. 3d- coded: 3d correspondences by deep deformation. In ECCV (2), volume 12006 Lecture Notes in Computer Science, pp.
Virtual, 2018. Springer.
Oshri Halimi, Or Litany, Emanuele Rodolà, Alexander M. Bronstein, and Ron Kimmel. Unsupervised learning of dense shape correspondence. In CVPR, pp. 4370 Long Beach, CA, 2019. Computer Vision Foundation IEEE.
11

---

Published as a conference paper at ICLR 2024
Qi- Di-Xing Huang and Leonidas J. Guibas. Consistent shape maps via semidefinite programming Comput. Graph. Forum, 32(5):177-186, 2013. doi: 10.111 12184. /
bxq/10.1111/ce
Qi-Xing Huang. Martin Wicke, Bart Adams, and Leonidas Guibas. Shape Decomposition using Modal Analysis. Computer Graphics Forum, 28(2):407-416, 2009, ISSN 1467-8659, doi:
Qi- Di-Xing Huang, Guo-Xin Zhang, Lin Gao, Shi-Min Hu, Adrian Butscher, and Leonidas Guibas. An optimization approach for extracting and encoding consistent maps in shape collection.
ACM Trams. Graph. (6), nov 2012. ISSN 0730-0301.
doi: 10.11452366614522666668 URL
Qixing Huang, Bart Adams, Martin Wicke, and Leonidas J. Guibas. Non-rigid registration under isometric deformations. Comput. Graph. Forum, 27(5):1449-1453 2008.
Qixing Huang, Fan Wang, and Leonidas J. Guibas, Functional map networks for analyzing and exploring large shape collections. ACM Trans. Graph., 33(4):36:1-36:11 2014.
Qixing Huang. Zhenxiao Liang. Haoyun Wang, Simiao Zuo, and Chandrajit Bajaj. Tensor maps for synchronizing heterogeneous shape collections. ACM Trans. Graph., 38(4), jul 2019a. ISSN 0730-0301. doi: 10. 00.1145/330644433329444 URL /ttp:::/ddi..rooo 11111111111133336610 3322944
Qixing Huang, Xiangru Huang, Bo Sun, Zaiwei Zhang, Junfeng Jiang, and Chandrajit Bajaj. Arapreg: An as-rigid-as possible regularization loss for learning deformable shape generators. In Proceedings of the IEEE/CVF international conference computer vision, pp. 5815-5825, 2021.
Ruqi Huang, Jing Ren, Peter Wonka, and Maks Ovsjanikov. Consistent zoomout: Efficient spectral map synchronization. Comput. Graph. Forum, 39(5):65-22) 278. 2020a.
Xiangru Huang, Zhenxiao Liang, Chandrajit Bajaj, and Qixing Huang. Translation synchronization via truncated least squares. In Guyon, U. V. Luxburg. S. Ben- gio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 1459-1468. Curran Associates ec/paper/ synchronizationn via www.cateudee.................... -east squares.
Inc.. Red Hook, NY, USA. 2017. URL 2744-Erans.a pdf
Xiangru Huang, Zhenxiao Liang, Xiaowei Zhou, Yao Xie, Leonidas J. Guibas, and Qixing Huang. Learning transformation synchronization. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 8082-8091, Virtual, 2019b. Computer Vision Foundation IEEE.
Xiangru Huang, Zhenxiao Liang, and Qixing Huang. Uncertainty quantification for multi-scan registration. ACM Trans. Graph., (9)4), July 2020b. ISSN 0730-0301. doi: 10.1 10/3355/3366.9. 3392402. URL hht.thttp/./. :ttpps://doft orgg 0. 011553386569.33922
Vladimir G. Kim, Yaron Lipman, and Thomas Funkhouser. Blended intrinsic maps. ACM Trans. Graph. 30(4), jul 2011. ISSN 0730-0301. doi: 10. 29944///00032411949974 URL https
7/doi. warli.org/ai 10. 1145/ 201002222 10324. 11444.96444 1964974]
Vladimir G. Kim, Wilmot Li, Niloy J. Mitra, Stephen DiVerdi, and Thomas Funkhouser. Exploring collections of 3d models using fuzzy correspondences. ACM Trans. Graph., 31(4), jul 2012. ISSN 0730-0301. doi: 10. 11114/221852022185555555 URL UPL_:ttpps///0o...///00111155 2185550
Vladislav Kraevoy and Alla Sheffer. Cross- paramterrizaioo and compatible remeshing of 3d models. ACM Trans. Graph. 23(3):861 869, aug 2004. ISSN 0730-0301. doi: 10.1145/1015706.1011 101588
URL httph:ttp//// doi www.doo....oooooooooooooooooo..o 11.....11111111111 1015811
Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabás Póczos, and Ruslan Salakhutdinov. Point cloud GAN, 2019a.
Lei Li, Nicolas Donati, and Maks Ovsjanikov. Learning multi-resolution functional maps with spectral attention for robust shape matching. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information P Processing Systems 2022, NeurlPS 2022, 8-14 4 December 2022, Vancouver, BC, Canada, pp. 1-10, 2022.
12

---

Published as a conference paper at ICLR 2024
Ruihui Li, Xianzhi Li, Chi- Whig Fu, Daniel Cohen- Or, and Pheng-Ann Heng. PU-GAN: A point cloud upsampling adversarial network. In ICCV, pp. 7202-7211, Scoul, Republic of Korea, 2019b. IEEE.
Or Litany, Tal Remez, Emanuele Rodolà, Alexander M. Bronstein, and Michael M. Bronstein. Deep functional maps: Structured prediction for dense shape correspondence. In ICCV, pp. 5660-5668, Washington, DC, USA, 2017. IEEE Computer Society.
Or Litany, Alexander M. Bronstein, Michael M. Bronstein, and Ameesh Makadia. Deformable shape completion with graph convolutional autoencoders. In 2018 IEEE Conference Computer Vision and Pattern Recognitiio, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 1886-1895, Salt Lake City, UT, USA, 2018. IEEE Computer Society. doi: 10. 109/ 2018.00202. URL httptt:::///pecccessss..s thecvf..c... com htt.://ccoooooottoeeeeeeeeo http://tttttetcccceeeee Jttany_Deformall Baareaalle_hh_p_ee_ee wwtp:eeooomeeeeooeeoeoeeeooooee 201 paper. .html
201 8 /
William Lorensen and Harvey E. Cline. Marching cubes: high resolution 3d surface construction algorithm. In Maureen C. Stone (ed.). Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1987, Anaheim, California, USA, July 27-31, 1987, pp. 163,169, New York, NY, 1987. ACM. doi: 10.1145/37401.37423 URL https //doi. [tdoi.org/I0 10. 1ww.1114111111331 37422
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 2837-2845, 2021.
Haggai Maron, Nadav Dym, Itay Kezurer, Shahar Z. Kovalsky, and Yaron Lipman. Point registration via efficient convex relaxation. ACM Trans. Graph., 35(4):73:1-73:12, 2016.
Simone Melzi, Jing Ren, Emanuele Rodolà, Abhishek Sharma, Peter Wonka, and Maks Ovsjanikov. Zoomout: spectral upsampling for efficient shape correspondence. ACM Trans. Graph. 38(6): 155:1-155:14 14, 2019. doi: 0..1145/335508933356533 URL hRthts::::///o 0ttp://////o0.0000001110113 3355089 3356524
Lars M. Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In CVPR, pp. 4470,
Long Beach, California, 2019. Computer Vision Foundation IEEE.
Sanjeev Muralikrishnan, Siddhartha Chaudhuri, Noam Aigerman, Vladimir G. Kim, Matthew Fisher, and Niloy J. Mitra. Glass: Geometric latent augmentation for shape spaces. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pallern Recognition (CVPR), pp. 18552-1856), June 2022.
Andy Nguyen, Mirela Ben-Chen, Katarzyna Welnicka, Yinyu Ye, and Leonidas J. Guibas. An optimization approach improving collections of shape maps. Comput. Graph. Forum, 30(5): 1481- 2011.
Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Func- tional maps: A flexible representation of maps between shapes. ACM Trans. Graph., (1). jul 2012. ISSN 0730-0301. doi: 10. 10.1145/216559221155566 URL 14552185520 2185526
Jeong Joon Park, Peter Florence, Julian Straub, Richard A. Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In IEEE Conference on Computer Vision and Patlern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 165 165-174, 174, Long Beach, California, 2019. Computer Vision Foundation IEEE. doi: 10. 10.19//CVPRR 2019 00025. URL Cett:://penaccess: thecvf. com/ content CVPR 2019 2tw9/htmlmeaaeeee Functions_ for Shape Representation Setpettnnonnn____eeeeeeee CVPR_ 1ww.111pppepe_eeeeeeeee html
DeepSDF Learning Continuous Signed Distance
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Ed- ward Z. Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 8024-8035, 2019.
13

---

Published as a conference paper at ICLR 2024
Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In CVPR. pp. 0054-9063, Washington, DC, 2021. Computer Vision Foundation IEEE.
Charles R. Qi Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
Marie-Julie Rakotosaona and Maks Ovsjanikov. Intrinsic point cloud interpolation via dual latent space navigation. In Andrea Vedaldi. Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23- 28, 2020, Proceedings. Part II, volume 12347 of Lecture Notes in Computer Science, pp. 655-
672, New York, NY, 2020, Springer. doi: 10. 00077977-33000055566666
URL https
//doi.org/10.
www.ro0............ 1000g/1..110///000000 1007/978 3 030 58536 5_39
Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J. Black. Generating 3d faces using convolutional mesh autnencoders. In Vittorio Ferrari, Martial Hehert, Cristian Sminchisescu, and Yair Weiss (eds.). Computer Vision ECCV 2018- 15th European Conference, Munich, Germany, September 14, 2018, Proceedings, Purt III, volume 1 1207 of Lecture Notes in Computer Science, URL
pp. 725-741, New York, NY, 2018. Springer. doi: 10.
https: /tt:::/doiioo www.i.oorg/000 1007 978 3 030 000111111111 9-19999_433 9_43
Jing Ren, Adrien Poulenard, Peter Wonka, and Maks Ovsjanikov. Continuous and orientation- preserving correspondences via functional maps. ACM Trans. Graph., 37(6), dec 2018. ISSN 0730-0301. doi: 10.1145/3272127.32755000 URL 3275040
Yusuf Sahillioglu. Recent advances in shape correspondence. Vis. Compur., 7651):177--111 2020.
Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Hao Li, and Angjoo Kanazawa. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In ICCV, pp. 2304-2314, 231,, Scoul, Republic of Korea, 2019. IEEE.
Shunsuke Saito, Tomas Simon, Jason M. Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high-resolutiinn 3d human digitization. In CVPR, pp. 81-90, Washington, DC, 2020. Computer Vision Foundation IEEE.
John Schreiner, Arul Asirvatham, Emil Praun, and Hugues Hoppe. Inter-surface mapping. In ACM SIGGRAPH 2004 Papers, SIGGRAPH '04, pp. 870-877 New York, NY. USA, 2004. Association for Computing Machinery. ISBN 978 9781450378239 doi: 14445/1116622111558558
URL http:/ttt////oooooo owoplooogg/000 10. 01144/1116565 1015812
Abhishek Sharma and Maks Ovsjanikov. Weakly supervised deep functional maps for shape matching. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020. NeurIPS 2020, December 6-12, 2020, virtual, pp. 19264-19275. Red Hook. NY. USA, 2020. Curran Associates Inc. URL https: http::/:////ccceciingee neurips cc/ http://pee///////aaaaa... -bstt html
Miroslava Slavcheva, Maximilian Baust, Daniel Cremers, and Slobodan Ilie. Killingfusion: Non- rigid 3d reconstruction without correspondences. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21- 2017, pp. 5474 5483, Washington, DC, 2017. IEEE Computer Society. doi: 00.1009/VPP..2017..55 URL https! 7/doi.
1119/1109// 2017.581
Justin Solomon, Mirela Ben-Chen, Adrian Butscher, and Leonidas J. Guibas. As-killing-as-possibeee vector fields for planar deformation. Comput. Graph. Forum, 30(5):1543-1552, 2011.
Olga Sorkine and Mare Alexa. As-rigid-as serigid-s-poosiee -sosssiille surface modeling. In Proceedings of the Fifth Eurographics Symposium on Geometry Processing, Barcelona, Spain, July 4-6, 2007, volume 257, pp. 1069-116,00 2007.
Jos Stam and Ryan M. Schmidr. On the velocity of an implicit surface. ACM Trans. Graph., 30(3): 21:1-21:7, 2011.
14

---

Published as a conference paper at ICLR 2024
Robert W Sumner and Jovan Popović. Deformation transfer for triangle meshes. ACM Transactions on graphies (TOG), 23(3):399 405, 2004.
Ramana Sundararaman, Gautam Pai, and Maks Ovsjanikov. Implicit field supervision for robust non-rigid shape matching. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner (eds.), Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part III, volume 13666 of Lecture Notes in Computer Science, pp. 362. Springer, 2022. doi: 10. 0000/9988-3-01122006633 20. URL httpss:/ddp .rggorggg.. /1/./11.1100 1007 978 3 0000003002222-2 20
Qingyang Tan, Lin Gao. Yu-Kun Lai, and Shihong Xia. Variational autoencoders for de- forming 3d mesh models. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT. USA, June 18-22, 2018, pp. 5841-5850, Salt Lake City, UT. USA, 2018. IEEE Computer Society. doi: 10.1 10.1109/CVPR.2015 00612. URL/nttp: / hptp///cppcccc.ccooooe/ee thevvf com/ www.coo//////eeee content evpr 2018/html//Tt /Tan Variational Aut oencoders for CVPR 2018 paper.html
Michael Tao, Justin Solomon, and Adrian Butscher. Near-isometric level set tracking. Comput. Graph. Forum, 35(5):65-77, 2016.
Edgar Tretschk, Ayush Tewari, Michael Zollhöfer, Vladislav Golyanik, and Christian Theobalt.
DEMEA: deep mesh autoencoders for non-rigidly deforming objects. In Andrea Vedaldi, Horst
Bischof, Thomas Brox, and Jan-Michael Frahm (eds.), Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IV, volume 12349 of Lecture Notes in Computer Science, pp. 601-617 Glasgow, 2020. Springer. doi: 10.1007/ 978-3 778-3-030-58548-8-333 URL https: /tcps::/doi..ore wt..or//o........ 100.0.00011110000 35
030 58548
Oliver van Kaick, Hao Zhang, Ghassan Hamarneh, and Daniel Cohen-Or. A survey on shape correspondence. In Helwig Hauser and Erik Reinhard (eds.), 31st Annual Conference of the European Association for Computer Graphics. Eurographics 2010 State of the Art Reports, Norrköping, Sweden, May 3-7, 2010, pp- 61-82, 9105 Salley Street Montreal, Quebec H8R 2C8 CANADA, 2010. Eurographics Association. doi: 10.23 2e23322/gst.20101002 URL https 2112110..312egsee.. http:////sss.211000...
Nitika Verma, Edmond Boyer, and Jakob Verbeek. Feastnet: Feature-steered graph convo- lutions for 3d shape analysis. In 2018 IEEE Conference on Computer Vision and Pat- tern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 2598- 2606, Salt Lake City, UT. USA. 2018. IEEE Computer Society. doi: 10.1 10.//0R/ 2018.00275. URLhttp: / http://ppppcccs...//p/ theevf. weteevffccoo/eee wwwm/oottet....mmomooooooomm...e cvpr 2018 Verma FeaStNet Feetteeeeearreeeeeeee Steered Graph CVPR_20111 pallppapeere.e html
Lanhui Wang and Amit Singer. Exact and stable recovery of rotations for robust synchronization. Information and Inference: Journal of the IMA, 2:145-199 December 2013.
Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In CVPR, pp. 206-215, Washington, DC, 2018. Computer Vision Foundation / IEFE Computer Society.
Yusuke Yoshiyasu, Wan-Chun Ma, Eicchi Yoshida, and Fumio Kanehiro. As-conformallas-posiiii surface registration. In Proceedings of the Symposium on Geometry Processing, SGP 14, pp. 257-267, Goslar, DEU, 2014. Eurographics Association. doi: 10.1 124. URL https //doi.org/ 10. 11w11 12451.
Yi Zhou, Chenglei Wu, Zimo Li, Chen Cao, Yuting Ye, Jason M. Saragih, Hao Li, and Yaser Sheikh. Fully convolutional mesh autoencoder using efficient spatially varying kernels. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florins Balcan, and Hsuan- Tien Lin (eds.). Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurlPS 2020, December 12, 2020. virtual, pp. neurips. cc/paper/ http:///oooeeeeeeeee/e/oe wtw.:/eeooeeeteaaaaaaaat.tt.t...
Red Hook, NY,
USA, 2020. Curran Associates Inc. http////000011111002232300000000
Silvia Zuffi, Angjoo Kanazawa, David W. Jacobs, and Michael Black. 3d menagerie: Modeling the 3d shape and pose of animals. In CVPR, pp. 5524-5532, Washington, DC, USA, 2017. IEFF Computer Society.
15

---

Published as a conference paper at ICLR 2024
Appendices
The supplementary materials provide more details of implementing the regularization loss used in the implicit shape generator in Section and details of mesh generator initialization in Section B Section
Section[givvr Ggives more details the dataset preprocessing. SectionDaan and Section show more results on shape space learning and shape matching, respectively.
A DETAILS OF REGULARIZATION Loss
A.1 EXPRESSION OF L L (z))
T. (g -2.. & 1 Buaupp (g (z)) Damp (g) (g
Banp (z))²
where is the graph Laplacian of the mesh, and Bara (z)) E R3nx3n a sparse block matrix defined as
Σ (z)
i=j
(z))
KEA;
(22(2)
E else
where ez)(2) = -g,(2) g (z) and Duna (g E 3nxxxn is a diagonal block matrix defined as
Dama (g)
E -))
is
JEN:
A.2 EXPRESSION OF L aeap (z))
L neup (g) -2L 8 la Buup (z) Ducp (g)(2)) Baap (g Baa (x)))
where Bacup (g (z)) E IR 323xxx An is sparse block matrix defined as
Σ ( (-e (x) ex2(
i=
Bacap
(g (z))
KEN
(2)x
jE M else
and Dacap (g) E I 1nxin is diagonal block matrix defined as
D (g
Σ
(Ilef diag(0, diag(0,e (e)(0,))(())e (=))))
JEN.
A.3 EXPRESSION OF (z(2) AND (=)
Considering all vertices, the matrix representation of
0f ( (z), (d)
Oz (g)
(z),2)² T)T
d
can be written as (²(2) (F where
d (z)
(2)
d -
E Rn
d
16

---

Published as a conference paper at ICLR 2024
85 2x 0 (z),
o (x),
Rax3n
81
(g)(2)) 2)²
05 g 22)
dE
of (z),
[²)
RRnd
01 (z),
8z
A.4 IMPLEMENTATION DETAILS
Both the geometric deformation regularization rgen( and the cycle-consistency regularization
Taye
( (f*) rely on the mesh with vertices that is discretized from f ))- = 0. We use Marching Cube for discretization. For the human category, we use a voxel grid with size 64 77 64. For the animal category, the size of the voxel grid is 82 x 50 71. The output mesh from the Marching Cube algorithm typically contains more than 5000 vertices. To reduce the computation complexity, we simplify the output mesh into 2000 faces (Garland & Heckbert 1997 before computing goo (f and (
The number of vertices around 1000, thus the size of L (z) is around 3000 x 3000. Computing (I'(2) (z))+ only takes about 40ms in PyTorch (Paszke et al. 2019
B DETAILS OF MESH GENERATOR INITIALIZATION
B. TEMPLATE- REGISTRATION
In order to register the template mesh M the input shape S;, we first generate T intermediate shape go (2?), where 2 Ztemp + j T+1 1 S j S T. Instead of directly register M to S,, we
first register M to (z()) with ARAP deformation energy (Sorkine & Alexa 2007 Huang et al. 2008). Since M and (zx) ) are very close, we directly apply nearest neighbor search to compute the correspondence for the data term. The registration gives the resulting deformed template MM, We
then register Mj to (z?) and get the deformed template M2. register M2 to gd (x*") and get the deformed template M3,, and well-aligned with S..
and forth. Finally get the deformed template Mm, which is
The interpolation-guidee
registration typically works well but might fail when the template M is too far from S,, i.e. the two shapes have very different poses. The reason is that the intermediate shapes on the interpolation path might not have good quality. The make full use of the learned shape space, add shapes from the input shape collection to the interpolation path in these cases. First, we compute the distance between each pair of shape S; and S, using the distance of their embedded latent codes ||z,- zj|l. Based this distance metric, we build K-NN graph among the input shape collection. We set K 25 for the human dataset and = 40 for the animal dataset. We then perform interolation-guidee
registration on each edge (i.j) of the graph and obtain the correspondences between S; and S,. We use the distortion of the mapped edges (Huang et al.) 2008) as the weights in the K-NN graph. Finally compute the shortest path from the template each shape S, and get the correspondences by composing the correspondences along the shortest path.
B.2 MESH GENERATOR ARCHITECCUREE
The network architecture of m follows from that in (Huang et al. 2021), of vertex positions of the template mesh M. We sample 4 resolutions of the mesh connections of the template mesh. The network architecture stacks 4 blocks of convolution + up-sampling layers. The convolution layer employs Chebyshev convolutional filters with 6 Chebyshev polynomials (Ranjan) et al. 2018). Similar to (Zhou et al. 2020), there is fully connected layer between the latent code and the input the first convolution layer.
which outputs displacements
17

---

Published as a conference paper at ICLR 2024
Figure 6: We use FPS to further select more diverse and challenging subset of 1000 shapes from the training split (Atzmon & Lipman| 2021: 2020; Gropp al.) 2020). In this example, among 10 10 100 shapes from the training split (Atzmon & Lipman 2021: [020 Gropp al. 2020), only the red shapes are selected because the gray shapes have similar poses.
C DETAILS OF DATASETS
There are approximately 41k shapes in the original DFAUST dataset. Since there low variety between the adjacent shapes, recent works (Atzmon & Lipman 2021 [020 Gropp et al.) 2020) create new training/testing split by uniformly sample 20% shapes from the original dataset. However, we notice that there still many similar shapes in the splits. For example, almost all motion sequences start from shapes with very similar rest pose. In order make the dataset more challenging, we further select 1000 shapes from the training split. Specifically, we first learn a VAE (Atzmon & Lipman 2021) to embed all the shapes from the training split into different latent vectors. We find that the latent vectors of similar shapes are typically closer. Then we apply farthest point sampling (FPS) to the latent vectors and select the first 1000 shapes. An example shown in Figure 6 We apply the same approach to the testing split and select another 1000 shapes for testing.
The original SMAL dataset from (Huang et al. 2 contains 300 training shapes and 100 testing shapes. We filter out the shapes that have unreasonable self-intersection, leading to 289 training shapes and 94 testing shapes.
For both DFAUST and SMAL dataset, we evaluate the correspondences from a template shape to the remaining shapes.
D MORE RESULTS OF SHAPE SPACE LEARNING
We show the shape interpolation results of the sate-offthheart implicit generator (Atzmon & Lipman 2021) and our method in Figure / and Figure g We show 30 interpolated shapes. By adding the proposed geometric deformation regularization loss and cycle-consistency regularization loss, our generator gives more meaningful interpolation results, which are important to the interpolation-guide registration.
E MORE RESULTS OF SHAPE MATCHING
We show the correspondence results of NeuroMorph (Eisenberger et al. 202T and our method in Figure and and Figure I0 Our method has lower errors compared to Nouro.Mrrph.
18

---

Published as a conference paper at ICLR 2024
(a)
(b)
(d)
(b)
(c)
(d)
Figure 7: The comparison of shape interpolation between SALD (Atzmon & Lipman 2021 and our method on the DFAUST dataset. (a) source shape. (b) target shape. (c) interpolation results of SALD (Atzmon & Lipman 2021). (d) our results.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
Figure 8: The comparison of shape interpolation between SALD (Atzmon & Lipman 202 and our method on the SMAL dataset. (a) source shape. (b) target shape. (c) interpolation results of SALD Atzmon & Lipman 2021). (d) our results.
19

---

Published as a conference paper at ICLR 2024
0.15
Figure 9: The comparison of the correspondences between method and NeuroMorph (Eisenberger al.] 2021) on the DFAUST dataset. We show the correspondence errors on the target shapes. For each group of shapes, the left is from NeuroMorph, the right is our result.
0.15
Figure 10: The comparison of the correspondences between our method and NeuroMorph (Eisen- berger et al., (02T) on the SMAL dataset. We show the correspondence errors on the target shapes. For each group of shapes, the left is from NeuroMorph, the right is our result.
20


Reviews:
[{"review_id": "zThhv63Oob", "reviewer": "Reviewer_WbPs", "summary": "This paper proposes a new algorithm for joint matching of a set of 3D shapes represented as polygon meshes or surface point clouds. The core idea is to learn a generative model for meshes, based on deformations of a base template, that can produce the input shapes and thereby infer correspondences between them. To achieve this in a robust and accurate way, the authors propose several innovations, including learning an initial implicit generator that informs the eventual mesh generator, and techniques to regularize the generator output by imposing distortion-minimizing/consistency-maximizing losses in epsilon-neighborhoods of the synthesized shape space (not just around the input landmarks).", "questions": "Please see questions inline in Strengths and Weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "I really like this paper. I think it addresses an important, general problem by bringing together a bunch of relevant ideas in a well-justified way: using generative models for discriminative problems, physically-inspired regularization, etc. The core insights are simple and compelling. The individual technical contributions are theoretically meaningful and elegant, well-presented, and show significant improvements over baselines in experiments. I particularly like the computation of optimal correspondence fields between similar implicit shapes in 4.1. And not being an expert in this precise area, I am impressed that this can be done both differentiably (is this where finite differences come in as mentioned in 4.4?) and fast (how long does the training take?)\n\nI am provisionally recommending acceptance and am open to revising my opinion further upwards (if the authors can address the critiques in Weaknesses) or downwards (if other reviewers find serious flaws).", "weaknesses": "This is a fairly complex system. I am not 100% certain that every design decision is fully justified since there are too many possible ablations (though the authors do study several obvious ones). In particular, the authors claim that the implicit approach is necessary since learning a mesh generator from scratch is too difficult and error-prone. While this may be true, it is not actually demonstrated within the ambit of the proposed pipeline. There is the 3D-CODED comparison, but that is an entirely different pipeline. I do understand that this would be an \"ablation\" that's at least half a research project by itself, but still... Maybe the authors have already done experiments to verify this which are not included in the paper?\n\nAlso, code (and preprocessed data) would be really helpful so that others can understand and verify the claimed contributions. Will it be released?\n\nA couple of relevant papers may be worth mentioning since they are in the overall spirit of this paper:\n\nMuralikrishnan et al., \"GLASS: Geometric Latent Augmentation for Shape Spaces\", CVPR 2022\n(learning generative models from sparse landmarks guided by ARAP regularization -- does require input with correspondences though so it's sort of complementary to this paper)\n\nBednarik et al., \"Temporally-Coherent Surface Reconstruction via Metric-Consistent Atlases\", ICCV 2021\n(using metric distortion energies for consistent reconstruction, and hence joint matching, of time-varying shape sequences)\n\nMinor:\np3: Chamber --> chamfer (spelling, and should be lowercase)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bOgFdQRmfu", "reviewer": "Reviewer_MqJa", "summary": "This paper presents GenCorres, a method to solve the Joint Shape Matching (JSM) problem for a collection of unorganized shapes.  It is based on fitting the input shapes with a mesh generator that is constrained to preserve the local structure (conformal or isometric) of the shapes. Particularly, GenCorres first learns an implicit generator from the input shapes using a VA, which produces intermediate shapes between arbitrary pairs. The paper introduces a novel approach for computing correspondences between adjacent implicit surfaces, which is used to regularize the implicit generator. Second, synthetic shapes generated by the implicit generator guide the initial fittings (template-based deformations) for learning the mesh generator. Experimental results demonstrate that GenCorres outperforms state-of-the-art JSM techniques in collections of articulated body shapes.", "questions": "How well does GenCorres perform with man-made shapes or other shapes that do not correspond to articulated objects? \n\nWhat is the influence of the hyperparameters (lambda and epsilon) in the final result? Are the values specified in the paper valid for other sets of input shapes?\n\nIf the number of input shapes is a critical factor, I suggest the authors to include an experiment to establish the limit number for which the method significantly degrades.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "GenCorres is a well-crafted method divided into three stages. Stage 1 learns an implicit generator from unorganized sets using a VAE, where local (ACAP) and cycle consistency is imposed between surfaces that are close along the embedding dimension, defining the geometric deformation and cycle-consistency regularization losses. Stage 2 learns the explicit mesh generator from the VAE encoder-decoder. Stage 3 refines and fits the mesh generator to the input set of meshes and enforces again local structure consistency with ACAP. The three stages are well constructed and contain novel contributions to this field, especially the regularization losses in Stage 1. The results in the paper reveal that the GenCorres' generative method produces high-quality meshes and the correspondences across the input set significantly improve the state-of-the-art.\nIn terms of the quality, the methodology is well described and motivated. The results offer enough evidence that GenCorres improves the state-of-the-art of JSM in collections of shapes describing articulated body shapes. The article also includes an ablation study to assess the importance of each individual step.\nIn terms of significance, this method offers an interesting solution to a difficult and open problem that has important applications in computer graphics.", "weaknesses": "The main weakness of GenCorres, which is revealed by the experiments and commented by the authors, is the need of a relatively large set of input shapes to learn the shape generators properly. Solving this issue is a difficult task that requires further research.\n\nGenCorres seems to be especially suitable for a particular type of shapes (articulated body or animal shapes) to which ACAP and ARAP represent good deformation constraints. Other shape collections, such as man made objects, probably represent a challenge for this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1FdjZepMxS", "reviewer": "Reviewer_pFHc", "summary": "The paper presents a method for computing correspondences between target shapes from a collection of shapes. Namely, the method first learns an implicit shape generator from the input collection, enforcing several useful shape-preserving constraints. It then evaluates the learned implicit generator to interpolate between target shapes, and uses the interpolation to guide the learning of the explicit shape generator. After a final refinement step, the target shapes are corresponded with high quality and appropriate in-between interpolations. The main shortcoming of the approach is the need for a substantially large collection of shapes to begin with.", "questions": "The main title in the PDF seems to be misspelled: \"CenCorres\" instead of \"GenCorres\".\n\nIn general, there are many symbols used in the mathematical notation, which need to be clearly mentioned and described.\n  * What are the different fonts for g (looks like implicit expressions use lowercase letters, while explicit use bold letters)\n  * What is Theta in g^Theta in the \"Problem statement\" paragraph (page 3)?\n  * What is Phi in \"Approach overview\" (page 3) and what does it map from and to? Why R^3 x Z? Does it compute the distance field value at the specific 3D coordinate for the shape defined by a latent code?\n  * What is Psi in h^Psi in equation (1)? Also mention lambdas for equations (1) and (2).\n  * Symbol x is introduced in section 4.1 without describing what it is.\n  * What does d stand for in equation 3, some displacement between a vertex and an infinitesimally close corresponding vertex? Is d of unit length, then scaled down by epsilon? Or should d be the small displacement that doesn't need to be further multiplied by epsilon?\n  * At some point we get exposed to several capital letter symbols (C, F, G, E) whose meaning is hard to follow.\n\nTable 2 caption should mention that the metric is mean and median geodesic distance between correspondences.\n\nAcronyms should be spelled out the first usage.\n  * First mention of ACAP is not accompanied by description and reference, maybe just remove it from section 3 until it's properly introduced in section 4.\n  * MP-pseudo inverse should just be Moore-Penrose inverse.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The high level idea is simple and sound: learn a shape generator with useful geometric priors, then apply its interpolation capabilities to guide explicit correspondence. The results are convincing, with rigid limbs bending appropriately between corresponded shapes, and the ablation study is informative.", "weaknesses": "The paper is hard to follow, both in text and in math, which makes it difficult to clearly understand and implement the presented ideas. A few rounds of editing should help the flow. Much of the mathematical detail (especially in section 4.1) should be relegated to the appendices (with full derivations and descriptions), and only clearly readable final formulas should be shown in the main text (I'd carefully pick which symbols represent which concepts).\n\nThe demonstrated use case is pretty contrived -- all collections come from well parameterized datasets. I would love to see results using 3D body scans with various clothes, hairstyles, etc.\n\nThe human shape collection seems to include a multitude of body types, and is clearly able to correspond/interpolate between them. The animal collections seem to only have shapes of the same animal. The paper would be stronger if you show an interpolation between say a lion and a horse. It is not clear whether this method would blend well between very different quadruped shapes.\n\nA more suitable ARAP reference should be: As-rigid-as-possible shape interpolation by Alexa et al. 2000", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ae3vGKf8ii", "reviewer": "Reviewer_sJwu", "summary": "The paper proposes joint shape matching (JSM), an unsupervised training schema based on three steps: the training of a generative model for implicit shapes; a template fitting to the shape generated by interpolation in the latent space; a final refinement provided by a chamfer fitting. The method is tested on humans and animals on relatively small datasets, and shows promising performance w.r.t. previous methods, comparable to ones that use implicit information.", "questions": "Following the Weaknesses above:\n1) How would linear interpolation behave in the presence of significant diverse geometrical topology data? Is the template triangulation the main limitation in this case?\n2) Could you provide some measure of the computational cost of the method? I think would be important to understand the scalability of the method, and I assume that the method would require a significant computational effort. How does it compare with NeuroMorph?\n3) Would it be possible to test the method on a class for which a dense correspondence is not provided; for example, some classes of ShapeNet (e.g., cars, airplanes)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The method is unsupervised and extrinsic but provides good performance, competitive with intrinsic approaches that uses the surface as regularization \n- The method seems novel, and the idea of enhancing the learning representation by interpolation in the latent space seems straightforward and reasonable", "weaknesses": "- The central part of the method is not easy to grasp. I find the overall principle clear, and it is probably possible to replicate the work in principle. Still, the technique is not well explained in detail, and the notation is often confusing. For example, the letter \"g\" is used both for the shape generator and for the mesh generator, and requires some back-and-forth to get used to it. I suggest revisiting the method explanation and clarifying the methodological details.\n- By my understanding, the method highly relies on the target shapes belonging to a given distribution that should not only bounded in terms of structure and class but in particular, should be possible to express by the registered template. Many methods rely on template registration, and it is not a weakness per se, but the proposed approach makes use of techniques that aim to be general and flexible (e.g., implicit representation, unsupervised learning), but the paper does not show any out-of-distribution results (and also, results only on shapes for which data are available and can be even generated synthetically). I also believe that the topological constraint given by the template limits the generality of the method. The limitations barely mention this, and it should be emphasized more.\n- Experiments are performed on a relatively small set of data. The datasets are outdated and do not deal with the literature's more recent and real challenges (e.g., partiality, noise, clutter, ...). In this sense, I suggest stressing the method further; I believe that they would be interesting to investigate the latent interpolation when the learned space represents more diverse shapes (i.e., I wonder if relying on implicit representation may help in the context of limited topological variation during the interpolation, or if the relying on a template do not let to generalize to these cases)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)