Learning Multi-resolution Functional Maps with Spectral Attention for Robust Shape Matching
Authors: Lei Li
Nicolas Donati
Maks Ovsjanikov
Venue: NeurIPS 2022 Accept (2022)
Keywords: Non-rigid shape matching
functional map
multi-resolution
spectral attention

Abstract: In this work, we present a novel non-rigid shape matching framework based on multi-resolution functional maps with spectral attention. Existing functional map learning methods all rely on the critical choice of the spectral resolution hyperparameter, which can severely affect the overall accuracy or lead to overfitting, if not chosen carefully. In this paper, we show that spectral resolution tuning can be alleviated by introducing spectral attention. Our framework is applicable in both supervised and unsupervised settings, and we show that it is possible to train the network so that it can adapt the spectral resolution, depending on the given shape input. More specifically, we propose to compute multi-resolution functional maps that characterize correspondence across a range of spectral resolutions, and introduce a spectral attention network that helps to combine this representation into a single coherent final correspondence. Our approach is not only accurate with near-isometric input, for which a high spectral resolution is typically preferred, but also robust and able to produce reasonable matching even in the presence of significant non-isometric distortion, which poses great challenges to existing methods. We demonstrate the superior performance of our approach through experiments on a suite of challenging near-isometric and non-isometric shape matching benchmarks.

Learning Multi-resolution Functional Maps with Spectral Attention for Robust Shape Matching
Li
Nicolas Donati
LIX, École Polytechnique, IP Paris 11i0lix, wol.ypocniiue......... fr
LIX, École Polytechnique, Paris nicolas. wonati1@poll.eeeciiuue.e..o edu
Maks Ovsjanikov
LIX, École Polytechnique, IP Paris
maks@lix.polll pow.eeecii...tteciiiuueee.ee fr
Abstract
In this work, we present novel non-rigid shape matching framework based on multi-resolution functional maps with spectral attention. Existing functional map learning methods all rely the critical choice of the spectral resolution hyperparameter, which can severely affect the overall accuracy or lead overfitting, if not chosen carefully. In this paper, show that spectral resolution tuning can be alleviated by introducing spectral attention. Our framework is applicable both supervised and unsupervised settings, and we show that it is possible to train the network so that it can adapt the spectral resolution, depending on the given shape input. More specifically. we propose to compute resliirresoltion functional maps that characterize correspondence across range of spectral resolutions, and introduce spectral attention network that helps combine this representation into a single coherent final correspondence. Our approach is not only accurate with near- isometric input, for which high spectral resolution is typically preferred, but also robust and able to produce reasonable matching even in the presence of significant non-isometric distortion, which poses great challenges to existing methods. We demonstrate the superior performance of our approach through experiments on a suite of challenging near-isometric and non-isometric shape matching benchmarks.
Introduction
Shape matching is a critical task in 3D shape analysis and has been paramount to a broad spectrum of downstream applications, including registration, deformation, and texture transfer [1, 2], to name few. The algorithmic challenge of robust shape matching primarily lies in the fact that shapes may undergo significant variations, such as arbitrary non-rigid deformations. Earlier works to tackle non-rigid shape correspondence conventionally build upon hand-crafted features and pipelines [3], while with the advent of deep learning, the research focus has largely shifted to data-driven and learning-based approaches for improved matching robustness and accuracy [4].
To learn for non-rigid shape matching, growing body of literature [7, 9, 10, 11, 12] advocates the use of spectral techniques, in particular, the functional map representation [13]. which compactly encodes correspondences
small-sized matrices using reduced spectral basis. A number of advances have been made to the functional map- based networks in terms of probe feature learning [5, 14], differentiable map regularization [5], supervised [7] and unsupervised learning [9, 8, 15], among many others. Despite this progress, existing works nearly always learn functional maps in a single spectral resolution (the number of basis functions used), which often set empirically. However, the functional map resolution plays crucial role in the non-rigid shape matching performance,
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Source
Ground Truth
GeomFmaps
Ours
Matcling with GeomFmaps on SMAL
12.
Main Gootak Ense (9000
10-
130
Specioal Raakkize
(a)
(b)
Figure (a) The unstable matching performance of GeomFmaps [5] w.r.t the critical Spectral Resolution hyperparameter
animal shape dataset SMAL [6]. (b) Correspondence visualization by texture transfer for near-isometric (top) and non-isometric (bottom) shapes. GeomFmaps is trained with ground truth supervision, while our approach is not.
observed in existing literature [9]. As a concrete example, Fig. 1-(a) shows that the matching performance of state-of-the-art supervised learning method GeomFmaps [5] fluctuates significantly when trained with different map resolution (i.e., size of the functional map). Therefore, to improve robustness, it is desirable to enable networks to adaptively change the resolution in data-dependent manner: for near-isometric shapes, adopting higher spectral resolution allows high-frequency details to be leveraged for more precise matching; while for non-isometric shapes, adopting a lower spectral resolution is advantageous obtain approximate but potentially more robust maps.
Motivated by the above discussion, in this work, we propose a novel learning-baad based functional map framework that learns to adaptively combine multi-resolution maps with mechanism that we call spectral attention, which can accommodate both near-isometric non-isometric shapes at the same time. Specifically, our framework consists of two novel components (Fig. 2): (1) multi-resolution functional maps and (2) the spectral attention module.
Given input pair of non-rigid shapes, first use functional map network to estimate series of maps with varying spectral resolution. Next, we feed the obtained functional maps to a spectral attention network to predict weight for each map. The attention weights are used to combine all the intermediate maps into a final coherent map. To enable such an assembly, we design a differentiable spectral upsampling module that transform the intermediate maps to the same spectral resolution within a learnable network. Finally, to train our network, we propose to impose penalties the intermediate multi-resolution functional maps as well the final map. This is different from existing approaches, e.g., [7, 9, 5, 10, 16], which work with and penalize a single hand-picked spectral resolution. Our method can be trained in both supervised and unsupervised settings and can directly benefit from other advances in deep functional map training, such as improved architectures or regularization. To evaluate our model, we perform a comprehensive set of experiments on several challenging non-rigid shape matching datasets, where our model achieves superior matching performance over existing methods.
In a nutshell, the main contributions of our work are as follows: (1) We introduce a powerful non-righ non-rigid shape matching framework equipped with multi-resolution functional maps with spectral attention for handling diverse shape inputs. (2) We propose a novel spectral attention network and a differentiable spectral upsampling module for robust functional map learning. (3) We demonstrate the superior performance of our model compared to existing approaches through extensive experiments on challenging non-rigid shape matching benchmarks. Our code and data publicly available'.
2 Related Work
In shape analysis, the field of non-rigid shape matching is both extensive and well-studied. In the following paragraphs, we review the works that are most closely related to our approach. A more complete overview can be found in recent surveys [17, 18], and more recently [4] (Section 4).

---

Functional Maps Our method is based on the functional maps framework, first introduced in [13], and extended in various works such as [19, 20, 21, 22, 23, 24] among others (see [25] for an overview). This general approach is based on encoded maps between shapes using reduced basis representation. Consequently, the problem of map optimization becomes both linear and more compact. Besides, this framework allows to represent natural constraints such as near-isometry or bijectivity as linear-algebraic regularization. It has also been extended to the partial setting [26, 27].
One of the bottlenecks of this framework is the estimation of so-called "descriptor functions" that are key to the functional map computation. Early methods have relied on axiomatic features, mainly based on multi-scale diffusion-based descriptors, e.g., HKS and WKS [28, 29].
Learning- based methods Several approaches have proposed to learn maps between shapes by formulating it as dense segmentation problem, e.g., [30, 31, 32, 33, 34, 35]. However, these methods (1) usually require many labeled training shapes, which can be hard to obtain, and (2) tend to overfit to the training connectivity, making the methods unstable to triangulation change.
Closer our approach are deep shape matching methods that also rely on the functional map framework, pioneered by FMNet [7]. In this work, SHOT descriptors [36] are given as input to the network, whose goal is refine these descriptors in order to yield a functional map close the ground-truth as possible. The key advantage of this approach is that it directly estimates and optimizes for the map itself, thus injecting more structure in the learning problem. FMNet introduced the idea of learning for shape pairs, using the same feature extractor (in their case, a SHOT MLP-based refiner) for the source and target shapes in Siamese fashion produce improved output descriptors for functional map estimation. However, later experiments conducted in [5] have highlighted that SHOT-based pipelines suffer greatly from connectivity overfitting. Thus, in more recent works, the authors in [S, 15, 14] ad for learning directly from shapes' geometry, while exploiting strong regularizers for functional map estimation.
The major upside of using the functional map framework for deep shape matching is that it relies on the intrinsic information of shapes, which results in overall good generalization from training to lesting, especially across pose changes, which involve minimal intrinsic deformation.
Unsupervised spectral learning The methods described above supervised deep shape matching pipelines. While these methods usually give good correspondence prediction, they need ground- truth supervision training time. Consequently, other methods have focused on training for shape matching using the functional map framework, withoul ground-truth supervision. This was originally performed directly on top of FMNet by enforcing either geodesic distance preservation [8, 37], or natural properties on the output functional map [9], as well as by promoting cycle consistency [38].
To disambiguate symmetries present in many organic shapes, some works choose to rely so-called "weak surers-suprvisiin.". by rigidly aligning all shapes (on the same three axes) as a pre-processing step [15, 391, and then use the extrinsic embedding information resolve the symmetry ambiguity. This, however, limits their utility to correspondences between shapes with the same rigid alignment as the training set. Another solution is use input signals that are independent to the shape alignment, such as SHOT [36] descriptors as done in the original FMNet. One of these recent methods [11], makes use of optimal transport on top of this SHOT-refiner align the shapes different spectral scales. This method, like ours, computes the functional map at different scales via progressive upsampling, but they only keep the last map as the output whereas we propose to let the network learn the best combination of different resolutions. Additionally, this method is dependent on the SHOT input, which makes it unstable towards change triangulation. In-network refinement is also performed in DG2N [40], but not in the spectral space.
Attention- based spectral learning The attention mechanism was originally introduced in deep learning for natural language processing, and consists in putting relative weights on different words of an input sentence [41]. This mechanism can be applied in different contexts, including that of shape analysis. Indeed, attention learned in the feature domain can be used to focus on different parts of a 3D shape, for instance in partial shape matching, as done in [10]. As we show in this paper, attention can also be used in the spectral domain by letting the network focus on different levels of details depending on the input shapes and their resulting functional maps at different spectral resolutions. Indeed, the utility of considering different resolutions of functional map, e.g., via upsampling of its size, has been highlighted in [42, 43). Here we propose to let the network learn to adaptively combine all the intermediate functional maps into a final coherent correspondence.
3

---

3 Background
Our work proposes learning-based framework for non-rigid shape matching by building upon the functional map representation [13], and especially its learning-band based variant GeomFmaps, introduced in [5]. Before describing our approach in Sec. first briefly describe the basic learning pipeline with functional maps for shape correspondence. We refer the interested reader to relevant works [25, 8, 9, SJ for more technical details.
Deep Functional Map Pipeline We consider a pair of shapes S₁ and S2, represented as triangle meshes with ni and n2 vertices, respectively. The goal is to compute high quality dense correspon- dence between these shapes in an efficient way. The basic learning pipeline estimates a functional map between S and S2 using the following four steps [25].
Compute the first l eigenfunctions of the Laplace-Beltrami Beltrami operator [44] on each shape, which will be used as a basis for decomposing smooth functions on these shapes. The Laplacian is discretized as S where S is the diagonal matrix of lumped area elements for mesh vertices and W is the classical cotangent weight matrix [45]. The eigenfunctions are stored as columns in matrices Φ1 E IR x and 42 E R=2*
Second, set of descriptors (also known as probe, or feature functions) on each shape are extracted by feature extractor network [7, 5] here denoted by Fe with learnable parameters O These feature functions are expected to be approximately preserved by the unknown map. We denote the learned feature functions as Fa(S G1 E IR" xd and Fe((2) = G2 E R=axd, where d is the number of descriptors. After projecting them onto the respective eigenbases, the resulting coefficients are stored as columns of matrices A1, A2 E k respectively.
Next, we compute the optimal functional map C E k by solving:
C = arg min || ||CA| - A2ll² + a CA1 - A2C12
(1)
where the first term promotes preservation of the probe functions, and the second term regularizes the map by measuring its commutativity with the Laplace-Beltramt operators [13, 5], which in the reduced basis become diagonal matrices of the eigenvalues A and Ag.
As last step, the estimated map C can be converted a point-to-point map commonly by nearest neighbor search between the aligned spectral embeddings & C and Φ,, with possible post-refinement applied [46, 22, 42, 47].
To train the feature extractor network Fe. one defines a set of training shape pairs, and another set of shape pairs for testing. As shown in [5], the solution to Eq. (1) can be obtained in closed form within a neural network in differentiable manner, and constitutes what is called "FMReg" in Fig. 2. Using this insight, during training time, the network aims at reducing a loss L(C), defined on the output functional map (()) Fo(S)) estimated from the learned descriptors using the closed form solution of Eq. (1). Through backpropagation, the parameters O are then updated to make the network produce better features for the next pair of shapes.
We stress that this pipeline, the size of the functional map a critical non-learned hyperparameter, which can strongly affect matching results (as highlighted in Fig. I and in existing literature [9]).
4 Method
The main goal of our work is to robustly and adaptively estimate functional maps for shape pairs with diverse geometric properties, including both near-isometric and non-isometric transformations. In this section, describe the technical details of our proposed non-rigid shape matching framework. We illustrate the whole pipeline in Fig. 2. Our framework has two main stages: multi-resolution functional map learning (Sec. 4.1) and spectral attention learning (Sec. 4.2).
4.1 Multi-resolution Functional Maps
In the first stage of our framework, given the input shapes S and S2, we follow the basic learning pipeline, described in Sec. 3, to infer multi-resolution functional maps for extensively characterizing

---

Diff. Upsample
C
DiffusionNet FMRee
AttentionNet
B
Fi
Ag
S2
C
K"
Input Shapes
Multi-res Functional Maps
Cinter
Spectral Attention
Figure 2: Illustration our non-rigid shape matching pipeline based on multi-resolution functional
maps with spectral attention. Our network takes input a pair of shapes S and Sg. First, estimate a set of intermediate functional maps [C'] J==1 with varying resolution, using DiffusionNet Fe as the feature backbone and the differentiable FMReg module for the map computation. Next, design spectral attention network Av to predict a set of attention weights {a') )_ for combining the functional maps at different resolutions, transformed by differentiable spectral upsampling. into the final map C. We impose losses Limer and Linal during training.
shape matching in the spectral domain. As mentioned above, this differs from existing saariin--basee functional map works that typically compute single map specific hand-picke resolution, which may not be universally applicable due to the rich diversity of shapes, thus putting limit on the generalization of networks across datasets.
To learn multi-resolution maps, we adopt a recent surface feature learning network DiffusionNet [14] as the backbone to learn the d probe functions. DiffusionNet is shown to be robust to varying mesh sampling and widely applicable to non-rigid shape analysis tasks [14, 12]. Following the notation in Sec. 3, let Fe denote this feature backbone. We feed the input shapes to the network in a Siamese manner and obtain G₁ = (e(S)) and G2 = (S2).. These Iearned probe functions are shared in the subsequent computation of multi-resolution maps.
To solve for a functional map with Eq. (1) in differentiable manner, we employ the FMReg module proposed in [5]. With slight abuse of notation, let C = {C*} f denote series of n estimated functional maps with varying resolution (Fig. 2-1eft), where the 7 map C' obtained via Eq. (1) is of size ka x k.. Adjacent maps differ in their resolution by rows and columns, i.e., k++1 k + for C++1 meaning that r more eigenfunctions are included in Φ and Φ for the map optimization.
Acceleration In the above approach, FMReg is repeatedly used optimize Eq. (1) for each C' E C, incurring noticeable computation cost. To address this issue, we point out an acceleration scheme based on principal submatrices to significantly simplify the computation of C. Specifically, we observe that for i < n. the principal k' x k submatrix of C" can be treated as approximation of C' computed in the standard way with Eq. (1). Accordingly, we can first employ FMReg once to compute the largest map C" and then construct the multi-resolution functional maps C as set of principal submatrices of C", thus avoiding the optimization for [Ci) Ji=l In practice, we find this acceleration scheme to work reasonably well, showing comparable matching performance with the above standard procedure (Sec. 5).
4.2 Spectral Attention
Given the estimated multi-resolution functional maps C, there prioritize the maps, and (2) how to address the resolution difference for combining the maps together. Thus in the second stage of our framework, instead of performing hard assignment, we propose to learn for each map C E C soft attention weight a' E [0, 1], representing its contribution to the final map assembly. To enable the assembly, also introduce a differentiable spectral upsampling module to align the map resolution within the network.
main questions: (1) how
Intuitively, our spectral attention network takes as input, the set functional maps C of different sizes, and predicts, for each functional map C in C, a scalar weight a², which represents the confidence associated with C To predict {a - the attention network needs to jointly assess the multi-resolution functional maps with their associated spectral information (Fig. 2-right). Directly
Network
5

---

using the cigenfunctions as input signals to the network can be problematic due to the known issues of sign flipping and order changes [13]. In this work, we opt for spectral alignment residual as pointwise features and build the network upon PointNet-based architecture [48].
Specifically, let क and @ denote the spectral embeddings associated with C², L..., matrices of the first k" eigenfunctions on SJ and S2, respectively, as defined in Sec. 3. For a point E S2, we define its spectral alignment residual r as:
min 6:p:
8e bp = (1( (*)
PES1
where & [p] denotes the 7d row the matrix Vi similarly for Pi[g). We gather the residuals across into feature vector r = [..., / r //t1///tT, ...JT for the point where the scaling factor Vi is to counteract the dimensionality difference across the spectral embeddings. The feature rag can be interpreted
form of confidence of each map C' the point q-
The vectors of spectral alignment residuals {ry)ess, collectively form an unordered set. We then feed this n-dimensional point set to our spectral attention network, denoted as Ay with learnable parameters V. We build As with the classification architecture of PointNet [48] with feature transformations and global feature pooling. We forward the pooled global features through multi- layer perceptrons and the softmax function to obtain spectral attention weights a' s.t. E a² 1.
Note that the spectral alignment residuals, as input to our spectral attention network Ay, are computed using the Laplacian eigenbasis. Nevertheless, as we prove in the supplementary material, the residuals, and thus our network, are invariant under the changes of the Laplacian eigenbasis. In practice, we have also observed that the network Ay is stable and trains well.
Differentiable Spectral Upsampling Due to the dimensionality difference, the functional maps in C cannot be directly combined together with the learned spectral attention weights. Inspired by [42], we propose a differentiable spectral upsampling module to transform all the maps to the same spectral resolution. The main idea consists of two differentiable steps: (1) Convert the k x k--size functional map C' to soft pointwise map; (2) Convert the pointwise map to a ke x k-=size functional map. Specifically, we first compute a soft pointwise map, denoted as IT' E R from C' as:
exp(-62 /L)p
LI
(3)
En exp(- fppr() /1)
where p.p' E S and 9 E S2, and is a learnable temperature parameter. Next, to compute an upsampled map Et of size hnth x k", we project IT$ onto the corresponding spectral basis:
=²= (1
(4)
where f denotes the Moore- Peoors-eenroseae inverse. After this differentiable upsampling, we obtain a set of functional maps غ = (C+,, that are all of size k" x k². Moreover, since they all represent maps between the same shape pair S and S2, they can be directly compared to each other and linearly combined.
Map Assembly With the estimated spectral attention {a and the upsampled functional maps C, the last step of our framework is assemble the intermediate maps together into final coherent map output for shapes Si and S2 (Fig. 2-rightmost). We denote the final map C, which is computed by simple linear combination as follows:
C Σ C.
(5)
4.3 Training
Loss To train our network, we impose losses on both the intermediate multi-resolution functional maps C= [C'} Fi=1 and the final map output C, as shown Fig. 2. Following prior works [5, 9], our network can be trained in a supervised or unsupervised way, and we present comprehensive results of both training paradigms in See. 5. In both the supervised and unsupervised settings. the loss Liner for the intermediate maps C and the loss Lúnal for the final map C are formulated as:
n
1m 2 L(C'),
Linter
Linal = L(C),
(6)

---

where L(C) is the penalty for a given map C depending on the training paradigm and described below. The map penalty Limer is multiplied by a scaling factor to counterbalance the spectral resolution difference. The overall training loss combines the two losses as Listal Lineer + Limat We note that Lisal remains unchanged regardless of whether the acceleration scheme (Sec. 4.1) is used.
For supervised training, we define the map penalty L(C' = ||C where the Frobenius norm is used. The ground-truth functional map C has the same spectral resolution as C and can be obtained by projecting the point-to-point map onto the reduced spectral basis. For unsupervised training, we follow [9] to define the map penalty as L(C) = | C I|| which promotes orthogonality of the given map. Note that a Laplacian commutativity penalty [9] is not necessary because of the regularization imposed in Eq. (1).
Implementation We implement network with PyTorch [49]. The feature backbone consists of four DiffusionNet blocks [14] and produces d = 256 dimensional features as output. We use = 1003 in Eq. (1). During pre-processing, we perform one-time cigendecomposition of the Laplacian on each shape and take the first 200 eigenfunctions, which are ordered by the corresponding eigenvalues and reused in all subsequent functional map computations. We compute resolution functional maps with sizes ranging from 10 x 10 to 200 x 200 with a step size r = 10. We set the batch size to I and use ADAM [50] for optimization. We use the WKS [29] descriptor as input to the network for all experiments. except for the SMAL dataset [6] in Sec. 5.2, where the 3D coordinates are used input. This X signal input is augmented with rotations around the up axis (here Y), since the shapes in SMAL are randomly rotated around this axis. This input signal can better address overfitting in the hard case of different animal species than WKS. More implementation details are provided in the supplementary material.
20 multi-
5 Experiments
In this section, we present extensive experimental results on a wide range of challenging non-rigid shape correspondence benchmarks to demonstrate the superior performance of our approach. We perform evaluations
widely adopted human shape matching datasets such FAUST [2] and SCAPE [5]] as well as a non-isometric dataset SHREC' 19 [52]. We also evaluate on an animal shape matching dataset SMAL [6] to stress-test generalization. Moreover, we test on a recent humanoid shape dataset constructed from Deforming ThinningTTTiggDDDD [53, 54] for both near-isometric and highly non-isometric correspondence. We use remeshed variants for these datasets, introduced in [22] and used in recent learning-based methods [5, 15, 11]. This ensures that the shapes do not share identical mesh connectivity.
5.1 Matching on FAUST, SCAPE, and SHREC'19
Following [5, 11], the FAUST dataset, which has 100 human shapes, split into 80 and 20 shapes for training and testing, respectively. For the SCAPE dataset consisting of 71 human shapes, the training and testing split 51/20. The SHREC'19 dataset has 44 human shapes and is used only as test set. Shape 40 in SHREC' 19 is a partial shape and removed from the dataset, because the Laplacian basis used has global support [13, 26] and special architecture designs like [10] are required for partial shape matching, which is outside the scope of this work.
Datasets
Baselines We perform comparison with existing non-rigid shape matching works categorized as follows: (1) Axiomatic approaches including BCICP [22], ZoomOut [42], and Smooth Shells [55]; (2) Supervised learning approaches including FMNet [7], 3D-CODED [56], HSN [35], ACSCNN [57],
TransMatch [58], and GeomFmaps [5]: (3) Unsupervised learning approaches including SURFM-
Net [9], Unsup [8]. NeuroMorph [39], DeepShells [11], and GeomFmaps [5].
3D-CODED and TransMatch take point clouds as input, while the other methods work on meshes.
Match is a recent transformer- based method typically requiring volumious training data, thus in experiments we initialized the training of TransMatch with its released network weights pretrained on the SURREAL dataset [59] consisting of 10,000 shapes.
GeomFmaps is a strong baseline closely related to our work. For a fair comparison, we also use DiffusionNet as the feature backhone in GeomFmaps, with the same input signal as ours, for improved performance [14]. The supervised and unsupervised training losses for GeomFmaps mmps are the same as
7

---

Table Mean geodesic error 100) on FAUST, SCAPE, and SHREC'19 19
Train
F
S
F
S
Test
F
S
S19
F
S
S19
S
S19
FMNet
sup
11.0
30.0
33.0
30.0
3D-CODED
sup
2.5
31.0
33.0
31.0
HSN
sup
3.3
25,4
16.7
3.5
ACSCNN
sup
2.7
8.4
6.0
3.2
TransMatch
2.7
33.6
21.0
18.6
18.3
38.8
2.7
18.6
16.7
TransMatch Refine
sup
1.7
30.4
14.5
15.5
12.0
37.5
1.6
11.7
10.9
GeomFmaps
sup
2.6
3.6
9.9
2.9
2.9
12.2
2.6
2.9
7.9
Ours-Fast
sup
1.3
2.9
7.1
1.8
1.8
11.7
1.3
1.8
7.1
Ours
sup
1.4
2.2
9.4
1.7
1.8
12.2
1.3
1.8
6.2
ВCICP
6.1
11.
ZoomOut
6.1
7,5
SmoothShells
2.5
4.7
SURFMNet
unsup
15.0
32.0
32.0
12.0
33.0
29.0
UnsupFMNet
unsup
10.0
29.0
22.0
16.0
11.0
13.0
NeuroMorph
unsup
8.5
28.5
26.3
18.2
29.9
27.6
9.1
27.3
25.3
DeepShells
unsup
1.7
5.4
27.4
2.7
2.5
23.4
1.6
2.4
21.1
GeomFmaps
unsup
3.5
4.8
8.5
4.0
4.3
11.2
3.5
4.4
7.1
Ours-Fast
unsup
1.9
2.6
5.8
1.9
2.1
8.1
2.3
6.3
Ours
unsup
1.9
2.6
6.4
2.2
2.2
9.9
1.9
2.3
5.8
The best results are highlighted separately for supervised and unsupervised methods.
the map penalties I. defined in Sec. 4.3. GeomFmaps uses 30 eigenfunctions, recommended in the original work [5].
Results Tab. reports the matching performance in terms of mean geodesic error on unit-area shapes [60]. Ours-Fast refers our method with the acceleration scheme (Sec. 4.1). We observe that our method outperforms all baselines in both the supervised and unsupervised settings. The slightly better correspondences obtained on FAUST by DeepShells [11] can be explained by the fact that they use refinement as post-processing step, which our method does nol require. Our method directly produces high-quality maps by choosing the best weighted combination of functional maps obtained from matching at different spectral resolutions. In particular, notice that how improve upon the closest competitor, GeomFmaps [5], by enabling the network to optimize the spectral resolution.
Besides, the fast version of our method gives almost the same results as the complete approach, and even better results in some cases. We attribute this partly to the fact that Ours-Fast avoids solving many linear systems inside the network. While Ours is more principled, it also relies on solving multiple linear systems with differentiable matrix inversion for each intermediate functional map [5]. Numerically this can lead to more instabilities, especially at the early training stage when descriptors are not fully trained, which ultimately can lead drop performance certain cases. Nevertheless, the results indicate that the acceleration scheme used in Ours-Fast works well in practice.
5.2 Matching on SMAL
Dataset The SMAL dataset has four- legged animal shapes of eight species. To stress-test the generalization of learning- based approaches, we use five species for training and three species for testing, resulting a 29/20 split of the shapes. Thus no shapes sim- ilar to the testing data are seen during train- ing, and the correspondence across species is highly non-isometric, presenting great chal- lenges to learning- approaches.
Table 2: Results (x 100) SMAL and DT4D-H
SMAL
DT4D-H
intra-class
inter-class
GeomFimaps
sup
8.4
2.1
4.1
Ours-Fast Ours
sup
4.7
sup
5.3
1.8
4.6
DeepShells GeomFimaps
unsup unsup
31.1
7.6
22.6
Ours-Fast
unsup
5.8
1.2
14.6
Ours
unsup
11.6
8

---

Source
Ground Truth
DeepShells
GeomFmaps
Ours
Intra Intra-class cara-class
Jeaton
Speccinr HoulMM
Inter-cian class
Specta Haube
Special alauuitton
Figure 3: Correspondence visualization by color transfer for shapes from the DT4D-H dataset, where the methods are trained in the unsupervised setting. The rightmost plots are the learned spectral attention by method.
Results Tab. 2-1eft reports the matching performance. Again, our method significantly improves upon its predecessor GeomFmaps both supervised and unsupervised settings. Besides, DeepShells fails to predict accurate maps in this hard setting, due to its tendency to overfit to the training set, since they use SHOT input which is very triangulation-sensitie Our method thus has strong generalization power, being able to predict correspondence between two animals whose species are never encountered during training.
5.3 Matching on DeformingThhingsee
Dataset We also test recent non-rigid shape matching benchmark. introduced in [54]. This benchmark consists of shapes from a large-scale animation dataset EeffrminngThingseee [53] with dense ground truth correspondences. The inter-class correspondences were obtained by non-rigid ICP using manually selected landmarks [54]. We use nine classes humanoid shapes for our evaluation. In total, there 198/95 non-rigid, and often non-isometric, shapes for training/testing, significantly more than the previous datasets. We denote this humanoid shape benchmark as DT4D-H.
Tab. 2-right presents the performance for both intra and inter class matching. In the supervised setting, we obtain performance comparable to GeomFmaps. This may be due to local artefacts in the inter-class ground truth computed by non-rigid ICP across strongly non-isometric shape classes, with matching accuracy below 5 being saturated. Nevertheless, the improvement becomes remarkably noticeable the unsupervised setting. DeepShells suffers from overfitting, is shown to succeed in intra-class matching but not in inter-class matching. Meanwhile, our approach the only one give reasonable overall results in this challenging unsupervised case.
Results
Visualization In addition to the SMAL results in Fig. 1-(b), in Fig. we show qualitative corre- spondence results on the DT4D-H dataset, across methods trained the unsupervised setting, We observe that method is able to produce high-quality correspondences for both intra and inter class shapes, compared to other methods.
In Fig. 3-right, we also plot the spectral attention (i.e., {a') defined in Sec. 4.2) estimated by our network with the unsupervised training. The plots demonstrate that, for near-isometric (intra-class) shape pairs, the network puts most of the attention weights on functional maps of resolution in [150, 200]; while for more difficult non-isometric (inter-class) shape pairs, the network assigns higher weights to functional maps of resolution in [50, 150]. For the non-isometric case, this suggests that the smaller functional maps are more reliable and can more accurately estimated than the larger ones. In contrast, for near-isee -sommiitttetic shapes, higher-resolution maps be estimated directly (and thus,

---

Table Ablation results (x 100)
Table 4: Backbone ablation (x 100)
Ours-Fast
Ours
DiffusionNet
KPConv
Full model
5.8
5.4
GeomFmaps
8.4
9.4
w/o Liner
13.1
13.4
Ours-Fast
5.8
6.8
Ag uni.
56.2
5.9
w/o Ag w/ avg. res.
11.1
5.6
r
5.1
7.2
6.4
6.9
lower-resolution ones as well, since those are given as principal submatrices of the larger functional maps). This highlights the importance of choosing the spectral resolution in data-dependent manner.
5.4 Ablation Study
We perform ablation studies w.r.t. our network components and show the results Tab. 3, where unsupervised training SMAL is used. First, we train our network without the loss on intermediate multi-resolution functional maps and denote it as w/o Linter in Tab. 3. We observe that using Limer can greatly improve the matching performance owing to the better intermediate representations produced. Next, to validate the spectral attention network Ay, we remove it and compute spectral attention with two non-learning schemes. One option is to as w/o Av w/ uni. in Tab. 3. The other is to use mean spectral alignment residual (i.e. softmax n Eges, ri for a²), denoted w/o Av w/ avg. res. The results show that the matching performance degrades without the spectral attention network, in particular for Ours-Fast, which fails to converge with the uniform weight during training, strongly indicating the robustness brought by learned spectral attention. Moreover, we also test different values for the spectral step size t (Sec. 4. and
uniform weight (i.e., a' 1/n), denoted
10 is balanced choice for both Ours- Fast and Ours.
To show the generality of our approach, we compare different feature backbones on SMAL where supervised training is used. Specifically, we replace DiffusionNet (Fig. 2) with KPConv [61], which another advanced architecture working on point clouds. Tab. 4 shows that our method brings noticeable improvement across various backbones, indicating the wide generality of our approach.
Conclusion, Limitations, and Societal Impacts
To conclude, we introduced a robust non-rigid shape correspondence framework applicable in both supervised and unsupervised settings. We leverage a multi-resolution functional map representation and propose to learn spectral attention for final coherent correspondence estimation, resulting in powerful deep model that can adaptively accommodate near-isometric and non-isometric shape input. We demonstrate the improved matching performance of our model through extensive experiments on challenging non-rigid shape matching benchmarks.
One limitation of our approach is that, similarly to existing functional map works, we need to pre- compute spectral decomposition for each input shape. While the pre-computation is efficient for moderately sized shapes, down-samme down-sampling or remeshing may be needed for large-size input. Besides, we focus on full shape matching and assume input shapes represented triangle meshes. It would be interesting to investigate an extension of our approach to partial shapes or point clouds as input.
Lastly, do not any immediate ethical issue with the proposed method, but note that considering the superior performance on human shape matching, unintended uses, such as surveillance, may potential negative societal impact of our work.
Acknowledgements
The authors thank the anonymous reviewers for their valuable comments and suggestions. Parts of this work were supported by the ERC Starting Grant No. 758800 (EXPROTEA) and the ANR AI Chair AIGRETTE.
10

---

References
[1] Huong Quynh Dinh, Anthony Yezzi, and Greg Turk. Texture transfer during shape transforma- tion. ACM TOG, 24(2):89-310, 2005.
[2] Federica Bogo, Javier Romero, Matthew Loper, and Michael Black. FAUST: Dataset and evaluation for 3d mesh registration. In CVPR, 2014.
[3] Oliver Van Kaick, Hao Zhang, Ghassan Hamarneh, and Daniel Cohen-Or. survey on shape correspondence. In CGF, volume 30, pages 1681-1707 Wiley Online Library, 2011.
[4] Bailin Deng, Yuxin Yao, Roberto M. Dyke, and Juyong Zhang. A survey of non-rigid 3d registration, 2022.
[5] Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. Deep geometric functional maps: Robust feature learning for shape correspondence. In CVPR, 2020.
[6] Silvia Zuffi, Angjoo Kanazawa, David W. Jacobs, and Michael J. Black. 3D Menagerie: Modeling the 3d shape and pose of animals. CVPR, 2017.
[7] Or Litany, Tal Remez, Emanuele Rodola, Alex Bronstein, and Michael Bronstein. Deep functional maps: Structured prediction for dense shape correspondence. In ICCV, 2017.
[8] Oshri Halimi, Or Litany, Emanuele Rodola, Alex M. Bronstein, and Ron Kimmel. Unsupervised learning of dense shape correspondence. In CVPR, 2019.
[9] Jean-Michel Roufosse, Abhishek Sharma, and Maks Ovsjanikov. Unsupervised deep learning for structured shape matching. In ICCV, 2019.
[10] Souhaib Attaiki, Gautam Pai, and Maks Ovsjanikov. DPFM: Deep partial functional maps. In 3DV, 2021.
[11] Marvin Fisenberger, Aysim Toker, Laura Leal-Taixé, and Daniel Cremers. Deep Shells: Unsu- pervised shape correspondence with optimal transport. NeurlPS, 2020.
[12] Nicolas Donati, Etienne Corman, and Maks Ovsjanikov. Deep orientation-aware functional maps: Tackling symmetry issues shape matching. In CVPR, 2022.
[13] Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Functional maps: a flexible representation of maps between shapes. ACM TOG, 31(4);1-11, 2012.
[14] Nicholas Sharp, Souhaib Attaiki, Keenan Crane, and Maks Ovsjanikov. DiffusionNet: Dis- cretization agnostic learning surfaces. ACM TOG, 41(3):1-16, 2022.
[15] Abhishek Sharma and Maks Ovsjanikov. Weakly supervised deep functional maps for shape matching. NeurlPS, 33, 2020.
[16] Lei Li, Souhaib Attaiki, and Maks Ovsjanikov. SRFeat: Learning locally accurate and globally consistent non-rigid shape correspondence. In 3DV, 2022.
[17] Silvia Biasotti, Andrea Cerri, Alex Bronstein, and Michael Bronstein. Recent trends, applica- tions, and perspectives in 3d shape similarity assessment. In CGF, volume 35, pages 87-119, 2016.
[18] Yusuf Sahillioglu. Recent advances in shape correspondence. The Visual Computer, 36(8): 1705- 1721, 2020.
[19] Davide Eynard, Emanuele Rodola, Klaus Glashoff, and Michael M Bronstein. Coupled func- tional maps. In 3DV, pages 399-407. IEEE, 2016.
[20] Dorian Nogneng and Maks Ovsjanikov. Informative descriptor preservation via commutativity for shape matching. CGF, 36(2):259-267, 2017.
[21] Oliver Burghard, Alexander Dieckmann, and Reinhard Klein. Embedding shapes with green's functions for global shape matching, Computers & Graphics, 68:1-10, 2017.

---

[22] Jing Ren, Adrien Poulenard, Peter Wonka, and Maks Ovsjanikov. Continuous and orientation- preserving correspondences via functional maps. ACM TOG, 37(6):1-16, 2018.
[23] Anne Gehre, Michael Bronstein, Leif Kobbelt, and Justin Solomon. Interactive curve constrained functional maps. In CGF, volume 37, pages 1-12, 2018.
[24] Meged Shoham, Amir Vaxman, and Mirela Ben-Chen. Hierarchical Functional Maps between Subdivision Surfaces. CGF, 2019.
[25] Maks Ovsjanikov, Etienne Corman, Michael Bronstein, Emanuele Rodolà, Mirela Ben-Chen, Leonidas Guibas, Frederic Chazal, and Alex Bronstein. Computing and processing correspon- dences with functional maps. In SIGGRAPH ASIA Courses, 2016.
[26] Emanuele Rodolå, Luca Cosmo, Michael M Bronstein, Andrea Torsello, and Daniel Cremers. Partial functional correspondence. In CGF, volume 36, pages 222-236, 2017.
[27] Or Litany, Emanuele Rodolà, Alex M Bronstein, and Michael M Bronstein. Fully spectral partial shape matching. In CGF, volume 36, pages 247-258. Wiley Online Library, 2017.
[28] Jian Sun, Maks Ovsjanikov, and Leonidas Guibas. A concise and provably informative multi- scale signature based on heat diffusion. CGF, volume 28, pages 1383-1392, 2009.
[29] Mathieu Aubry, UIrich Schlickewei, and Daniel Cremers. The wave kernel signature: A quantum mechanical approach shape analysis. In ICCV workshops, pages 1626 1626-1633, 2011.
[30] Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. Geodesic convolutional neural networks on riemannian manifolds. In ICCV Workshops, pages 37-45, 2015.
[31] Davide Boscaini, Jonathan Masci, Emanuele Rodolà, and Michael Bronstein. Learning shape correspondence with anisotropic convolutional neural networks. NeurIPS, 29, 2016.
[32] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In CVPR, pages 5115-5124, 2017.
[33] Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Heinrich Müller. Splinecnn: Fast geometric deep learning with continuous b-spline kernels. In CVPR, pages 869 869-877, 2018.
[34] Adrien Poulenard and Maks Ovsjanikov. Multi-directional geodesic neural networks via equivariant convolution. ACM TOG, 37(6);1-14, 2018.
[35] Ruben Wiersma, Elmar Eisemann, and Klaus Hildebrandt. CNNs on surfaces using rotation- equivariant features. ACM TOG, 39(4):92-1, 2020.
[36] Samuele Salti, Federico Tombari, and Luigi di Stefano. SHOT: Unique signatures histograms for surface and texture description. CVIU, 125, 2014.
[37] Mchmet Aygün, Zorah Lähner, and Daniel Cremers. Unsupervised dense shape correspondence using heat kernels. In 3DV, pages 573-582. IEEE, 2020.
[38] Dvir Ginzburg and Dan Raviv. Cyclic functional mapping: Self-supervised correspondence between non-isometric deformable shapes. In ECCV, pages 36-52. Springer, 2020.
[39] Marvin Eisenberger, David Novotny, Gael Kerchenbaum, Patrick Labatut, Natalia Neverova, Daniel Cremers, and Andrea Vedaldi. NeuroMorph: Unsupervised shape interpolation and correspondence in one go. In CVPR, 2021.
[40] Ginzburg Dvir and Raviv Dan. Dual geometric graph network (DG2N) zero-shot refinement for dense shape correspondence. CoRR, abs/2011.14723, 2020.
[41] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0477 2014.
12

---

[42] Simone Melzi, Jing Ren, Emanuele Rodolà, Abhishek Sharma, Peter Wonka, and Maks Ovs- janikov. ZoomOut: Spectral upsampling for efficient shape correspondence. ACM TOG, 38(6), November 2019.
[43] Jing Ren, Simone Melzi, Peter Wonka, and Maks Ovsjanikov. Discrete optimization for shape matching. In SGP, 2021.
[44] Bruno Vallet and Bruno Lévy. Spectral geometry processing with manifold harmonics. CGF, volume 27, pages 251-260, 2008.
[45] Mark Meyer, Mathieu Desbrun, Peter Schröder, and Alan H Barr. Discrete differential-geomeery operators for triangulated 2-manifolds. In Visualization and mathematics III, pages 35-57. Springer, 2003.
[46] Danielle Ezuz and Mirela Ben-Chen. Deblurring and denoising of maps between shapes. In CGF, volume 36, pages 165-174, 2017.
[47] Gautam Pai, Jing Ren, Simone Melzi, Peter Wonka, and Maks Ovsjanikov. Fast Sinkhorn Filters: Using matrix scaling for non-rigid shape correspondence with functional maps. In CVPR, pages 384-393, 2021.
[48] Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. PointNet: Deep learning point sets for 3d classification and segmentation. In CVPR, 2017.
[49] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang. Z. Vetioo, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, Bai, and S. Chintala. PyTorchh: An imperative style, high-performance deep learning library. NeurlPS, 2019.
[50] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
[51] Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. SCAPE: shape completion and animation of people. In SIGGRAPH, pages 408-416. 2005.
[52] Simone Melzi, Riccardo Marin, Emanuele Rodolà, Umberto Castellani. Jing Ren, Adrien Poulenard. Peter Wonka, and Maks Ovsjanikov. Matching humans with different connectivity. In 3DOR, pages 121-128, 2019.
[53] Yang Li, Hikari Takehara, Takafumi Taketomi, Bo Zheng, and Matthias Nießner. 4DComplete: Non-rigid motion estimation beyond the observable surface. In ICCV, 2021.
[54] Robin Magnet, Jing Ren, Olga Barkine-Hornung and Maks Ovsjanikov. Smooth non-rigid shape matching via effective Dirichlet energy optimization. In 3DV, 2022.
[55] Marvin Eisenberger, Zorah Lahner, and Daniel Cremers. Smooth shells: Multi-scale shape registration with functional maps. In CVPR, 2020.
[56] Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, and Mathieu Aubry. 3D-CODED: 3d correspondences by deep deformation. In ECCV, 2018.
[57] Qinsong Li, Shengjun Liu, Ling Hu, and Xinru Liu. Shape correspondence using anisotropic Chebyshev spectral cnns. In CVPR, June 2020.
[58] Giovanni Trappolini, Luca Cosmo, Luca Moschella, Riccardo Marin, Simone Melzi, and
Emanuele Rodolà. Shape registration in the time of transformers. NeurIPS, 34, 2021.
[59] Gul Varol, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, and Cordelia Schmid. Learning from synthetic humans. In CVPR, 2017.
[60] Vladimir G Kim, Yaron Lipman, and Thomas Funkhouser. Blended intrinsic maps. ACM TOG, 30(4):1-12, 2011.
[61] Hugues Thomas, Charles R. Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, Francois Goulette, and Leonidas J. Guibas. KPConv: Flexible and deformable convolution for point clouds. In /CCV, 2019.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (cither the supplemental material or as a URL)? [Yes] Please see Section 1.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [Yes] More details can be found in the supplementary material.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you're you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "r0XtbfdqFum", "reviewer": "Reviewer_PHU8", "summary": "The manuscript introduces shape matching framework based on multi-resolution functional maps. the idea is that the Laplace-Beltrami operators and their spectra are computed for shapes at various resolution, and for each resolution the corresponding functional map is computed. In order to make eigenfunctions and functional maps at various resolution comparable, a spectral upscaling scheme is adopted.\nWith he functional maps to hand, an attention mechanism based on mapped-point residual is adopted. The output of the attention network is a set of weight used to linearly combine the the maps.", "questions": "Does the approach assume uniform (possibly unequal) sampling of the shapes? I see no indication of the use of the area elements in the spectral decomposition and the multi-resolution approach would probably need to adapt to non-uniform sampling.\n\n", "limitations": "I believe that the negative societal impacts have been assessed correctly. As for Limitations, the performance issue with spectral decomposition is common to all spectral approaches, but they also incur other, arguably more severe limitations regarding their robustness to noise, extreme partiality, and topological deformation. That being said, the present proposal is affected by those limitations more than other spectral approaches because it needs to compute multiple spectral decomposition (one per scale) and then perform the upscaling since in the end it needs to work at the highest resolution.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The addition of the multi-resolution aspect o functional maps is sound and interesting, and the attention mechanism provides a smart data-driven way to select the resolutions. However, what it lacks is locality: different scales can behave differently in different regions of the shape, especially in conjunction to non-uniform sampling.\n\nI quite like the fact that the approach can work both in a supervised and unsupervised setting, but I believe that a supervised setting where the map is known is a bit unrealistic. Perhaps a setting where known corresponding functions over the two shapes are given would form a better use-case (incidentally, that was the original use-case for functional maps).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "PM3oDztoQJ_", "reviewer": "Reviewer_6wTY", "summary": "The paper proposes a neural network architecture that exploits multi-resolution functional maps to predict a pointwise correspondence between two shapes. The architecture is composed of two blocks. The first block follows the idea of Deep Functional Maps (REF) and consists of multiple networks each predicting a functional map at a different resolution. The second block takes as input these functional maps and the residual (i.e. the eigenbases alignment error per each point of the two shapes) and computes an attention score with which to derive the final functional map as a weighted sum of all the functional maps. To make it possible to sum the maps at different resolutions, the authors propose to adopt a differentiable upsampling scheme inspired by ZoomOut. The method is validated on common 3D non-rigid datasets composed of humanoid and animal shapes.", "questions": "I’m a bit confused about the use of this attention scheme with the fast version of the network, where lower resolutions are derived as submatrices. In this case, it seems to me that the only difference between the different resolution FMs is given solely by the upsampling strategy. Are the upsampled matrices improving the final result or is most of the attention put on the larger FM? If for ZoomOut it is reasonable that, with few corresponding functions, solving for a smaller optimization problem and then upsampling can lead to better results, in this case, the method solves the least-square problem for the wider basis-function and then takes a submatrix. It is not obvious to me that this leads to the same result.\n\nAlso, following the previous point, is  L_{inter} for the fast version computed only for the wider FM? If so, why the huge difference in tab 3?\n\nMinor:\n- Maybe I missed it in the paper, but how are the point-wise correspondences computed?\n- Description of deep functional map refers to a course. Isn’t there any reference to the original paper proposing this method?\n- I’m also not sure of understanding the ablation of \\tau. In tab 3 results with 5 and 20 are reported, while the full model should use 10, justifying this choice as a trade-off between speed and performance. Looking at the results I do not see this trend, in fact, both 5 and 20 are way worse than the Full model (with tau=10). Am I missing something?\n- In the comparison with other methods, it is worth mentioning which kind of date each method works. Some of the methods take as input point clouds, while the proposed method requires triangular meshes (at least in the shown experiments). Also, even if not critical, it would be worth adding recent SOTA methods for shape registration with transformers (Trappolini et al., Shape registration in the time of transformers, NeurIPS 2021).\n", "limitations": "The paper clearly states the main limitations of the proposed approach, including the need for triangled meshes and the lack of experiments on partial ones. (eg. SHREC’16: Partial matching of deformable shapes, Matching deformable objects in clutter)", "rating": 7, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper addresses a common problem in the design and application of Functional Map-based methods, which consists of the choice of suitable size of the truncated eigenbasis for the problem at hand. The proposed data-driven approach is for sure an interesting solution. \n\nUnfortunately, an analysis of the obtained attention scores in different scenarios (e.g. near-isometric vs non-isometric) is missing and would have been a strong contribution to the paper. Indeed, as a motivation for this work, the authors suggest that few basis functions are prefered for highly non-isometric shapes while near-isometric shapes can benefit from the use of a wider eigenbasis that includes also higher frequencies. Even if this sounds like a reasonable premise, there isn’t any analysis supporting this claim.\n\nAs a last note, I found the paper generally well written, but some details on the method implementation are missing, or at least not perfectly clear.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "On4k4-3Y2zb", "reviewer": "Reviewer_da7e", "summary": "The paper proposes a learning-based framework for non-rigid shape matching by building upon the functional map representation, and especially its learning-based variant GeomFmaps. The major contribution of the paper is learning the soft thresholding weights to combine functional maps obtained from matching at different spectral resolutions, which allows the network to optimize the spectral resolution. The paper illustrates the effectiveness of the combination on various benchmarks, on which it achieves state-of-the-art or comparable performance. ", "questions": "1) line 47 -  \"our framework consists **in** two novel..\"  -- Did you mean \"consists in\" or \"consists of\"? \n2) \"Shape 40 in SHREC’19 is a partial shape and removed from the dataset, since it is outside the scope of this work\" - it is not clear why partial shape is out of scope of this work, please explain (line 123 actually mentions that attention has been previously applied to partial shape matching) \n3) Please explain why the fast method yields improved results in many cases (Tables 1 and 2).", "limitations": "To the best of my understanding there is no negative societal impact to the paper. The authors mention human surveillance, which does not  seem to be a direct application of the shape matching.\nOtherwise the authors discuss general limitations -  computationally non-efficient computation of spectral decomposition for each input shape. (lines 360 - 363 \"While the pre-computation is efficient for moderately sized shapes, down-sampling or remeshing may be needed for large-size input\"). ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nExtensive experiments and state-of-the-art performance - the authors perform evaluations on widely adopted human shape matching datasets such as FAUST and SCAPE and a non-isometric dataset SHREC’19. Table 1 shows that on these datasets the paper outperforms the GeoFmaps (the method extended in this paper) and also outperforms all the other methods (presented), with a few exceptions. \nThe authors further evaluate on an animal shape matching dataset SMAL and show (Table 2) considerable improvement on GeomFmaps (e.g., in supervised setting: 5.3 vs. 8.4 mean geodesic error X 100) and DeepShells (unsupervised setting 5.3 vs. 29.3 ). Additional evaluation is provided on the challenging humanoid shape dataset from DeformingThings4D  for both near-isometric and\nhighly non-isometric correspondence. The authors show improved performance intra-class in supervised setting and intra and inter- class in unsupervised setting (Table 2). The authors also provide an ablation study Table 3, that shows the importance of learned spectral attention (linear combination weights of the functional maps).\n\n\nWeaknesses:\n1) Limited novelty of the idea:  lines (54 -  57) \"Finally, to train our network, we propose to impose penalties on the intermediate multi-resolution functional maps as well as the final map. This is different from existing approaches [7, 9, 5, 10], to name a few, that penalize a single hand-picked spectral resolution\" --Essentially the only novelty of this paper is learning of the soft-thresholding of the / best weighted combination of functional maps obtained from matching at different spectral resolutions. While the benefit to the performance is substantial, the idea is not essentially novel (it makes sense that a learned combination of weights would work better).\n2) Extension of an existing work: The authors claim that their method is generic enough to directly benefit from other advances in deep functional map training, such as improved architectures or regularization. Note that this claim seems reasonable based on the architecture introduced in Figure 2 (the blue block can be probably replaced with a different architecture), however it is not supported by experiments. This makes their work seem like an extensions of one specific approach. \n3) There is little deep dive into the results, for example the authors do not elaborate on why the more efficient technique named \"Ours-FAST\" sometimes outperforms \"Ours\", which is not intuitive.  The authors only say  lines 293 - 295: \"Besides, the fast version of our method gives almost the same results as the complete approach, and  even better results in some cases. This underlines the fact that the approximation made in this version of our model works well in practice.\". There is no insight as to why this happens.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept