Shape Non-rigid Kinematics (SNK): A Zero-Shot Method for Non-Rigid Shape Matching via Unsupervised Functional Map Regularized Reconstruction
Authors: Souhaib Attaiki
Maks Ovsjanikov
Venue: NeurIPS 2023 poster (2023)
Keywords: shape matching

Abstract: We present Shape Non-rigid Kinematics (SNK), a novel zero-shot method for non-rigid shape matching that eliminates the need for extensive training or ground truth data.SNK operates on a single pair of shapes, and employs a reconstruction-based strategy using an encoder-decoder architecture, which deforms the source shape to closely match the target shape. During the process, an unsupervised functional map is predicted and converted into a point-to-point map, serving as a supervisory mechanism for the reconstruction. To aid in training, we have designed a new decoder architecture that generates smooth, realistic deformations. SNK demonstrates competitive results on traditional benchmarks, simplifying the shape-matching process without compromising accuracy. Our code can be found online: https://github.com/pvnieo/SNK

Shape Non-rigid Kinematics (SNK): A Zero-Shot Method for Non-Rigid Shape Matching via Unsupervised Functional Map Regularized Reconstruction
Souhaib Attaiki
Maks Ovsjanikov
LIX, École Polytechnique, IP Paris attaiki@lix wow.ypocttttttte..........
LIX, École Polytechnique, IP Paris maks@lix. www.iiopoyyccceuu.......
Abstract
We present Shape Non-rigid Kinematics (SNK), novel zero-shot method for non-rigid shape matching that eliminates the need for extensive training or ground truth data. SNK operates on single pair of shapes, and employs reconstruction- based strategy using an encoder-decoder architecture, which deforms the source shape to closely match the target shape. During the process, an unsupervised functional map predicted and converted into a point-to-poing map, serving as a supervisory mechanism for the reconstruction. To aid in training, we have designed a new decoder architecture that generates smooth, realistic deformations. SNK demonstrates competitive results on traditional benchmarks, simplifying the shape- matching process without compromising accuracy. Our code can found online: https://ge gigghu........ cww www/pvnieo////cooccoccotcccowwoo /VNi
Introduction
Shape matching, specifically non-rigid shape matching, is fundamental problem in Computer Vision, Computational Geometry, and related fields. plays a vital role in a wide range of applications, including statistical shape analysis [1. 2], deformation [3], registration [4], and texture transfer [5], among others. This problem involves finding a correspondence between two shapes that may differ due to non-rigid deformations, such bending, stretching, compression.
Traditional methods for addressing this problem have predominantly been axiomatic, offering several benefits, such theoretical guarantees [6] and their application in different scenarios [7, 8, 9, 10]. Some of these methods include intrinsic isometries-basee methods [11, 12], parametrization domain- based methods [13, 14], and functional map methods [15, 16, 17). However, these methods often heavily rely on the availability of good initialization [18, 19, 20] or the use of handcrafted descriptors [21, 22,232 23]. This dependence can be limiting factor, constraining the performance of these methods and making them less effective in situations where these requirements cannot be adequately met.
Learning-bai based approaches have emerged as alternative to these traditional methods, with notable examples including Deep Functional Maps methods [24, 25, 26, 27, 28, 29], dense semantic seg- mentation methods [30, 31, 32, 33], or template- based methods [34] which have shown significant promise in solving the shape-matching problem. The strength these methods lies in their capacity to learn features and correlations directly from data, outperforming handcrafted descriptors. However, these techniques introduce significant challenge. Even when the task match just single pair of shapes, these methods necessitate the collection and training of thousands [34]. The demand for these extensive data collections, coupled with the long training times that can extend into days [35, 36], can be particularly onerous.
large data set, often in the tens
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Target
C22 Cz fla
Feature extractor
FM solver
S1
F2
Loss function
L
Latent code
Shape encoder
Reconstruction
Source
Shape decoder
S
Figure 1:
Method overview. Given two shapes as input, we deform source shape to resemble the target. Our network can be trained using just single pair of shapes and uses combination of spatial and spectral losses, as well as a regularization on the decoder.
Our proposed method, Shape Non-rigid Kinematics (SNK), positions itself uniquely in this landscape, combining the benefits of both axiomatic and learning- based approaches while addressing their limitations.
In particular, SNK works directly on a given pair of shapes, without requiring any prior training. This removes the need for large data sets or pre-existing examples. Unlike axiomatic methods. however, that rely on fixed input features such as HKS or SHOT, we employ a deep neural network for feature computation. As we demonstrate below, this of neural network introduces significant prior for our method, setting it apart from purely geometric approaches. We thus refer to our method "zero-shot" to highlight its ability be trained single pair of shapes, while using neural network for feature learning.
SNK is applied to each pair individually using a reconstruction-baat strategy that employs an encoder-decoder architecture. By doing so, SNK effectively deforms the source shape to closely match the target shape. A point-to-point map is then extracted by simple nearest neighbor between the target and the deformed source.
At the heart of our method lies the functional map paradigm. In our approach, a functional map generated in an unsupervised manner and then converted into a point-to-poing map. This map subsequently used as supervision for the decoder, ensuring the accuracy of the shape reconstruction process. To confine the deformation to plausible transformations, we introduce new decoder architecture that is grounded the PriMo energy [37]. Lastly, our method is trained using losses in both the spatial (reconstruction) and spectral (functional map) domains, ensuring comprehensive performance improvement.
Overall, our contributions can be summarized follows:
We have devised decoder architecture rooted in the PriMo energy concept. This architecture facilitates the production of deformations that appear smooth and natural.
We have shown that a loss function, which imposes penalties on both spatial and spec- tral quantities, is adequate for deriving matches single pair of shapes without any prerequisite training.
We have developed method for zero-shot shape matching that attains state-of-ther thee-of-thhertttt results among methods that are trained on shape collections. Furthermore, it competes with. and frequently outperforms, several supervised training-based approaches.

---

2 Related Works
Non-rigid shape matching is a widely researched subject, and providing a comprehensive overview is beyond the scope of this paper. However, we will briefly discuss methods closely related to ours. For more detailed exploration, we encourage interested readers to refer to recent surveys [38, 39, 40, 41].
Functional Map Our method is built upon the functional map paradigm, which since its introduction
[15], has been widely used for nonrigid shape matching [27, 19, 18, 17, 16, 7, 9, 42, 29, 43, 44, 45, 46]. It offers distinct advantage by transforming the complexity of pointwise maps optimization, originally quadratic relative to the vertex count, into a more manageable process of optimizing a functional map (which consists of small quadratic matrices), making the optimization process feasible, see [47] for an overview.
In the pursuit of identifying the functional map, initial studies depended on handcrafted feature functions defined on both source and target shapes, such as HKS [22]. WKS [23], and SHOT [21] features. Subsequent research enhanced this pipeline by incorporating extra regularization [16, 8, 48], adjusting it to accommodate partial shapes [9. 49, 10, 7], and suggesting efficient refinement techniques [19, 50, 51]. Contrarily, while these techniques often rely on an initial map for refinement or handcrafted features that lack robustness [25], our approach distinctively learns features by precisely fitting the source and target shapes using a neural network.
Learning-bass based Methods for Shape Matching The field of shape matching has recently seen a surge in success with Marning-based methods. These approaches take on various forms, including framing the issue as segmentation problem, which involves training a network to segment each vertex on the source shape into a class corresponding to vertex ID) on a template shape [30, 31, 32, 33]. Other methods include template fitting, which deforms template align with source and target shapes [34], and the use of functional maps [24, 28, 52, 25].
The latter method involves a neural network that extracts features from source and target shapes to predict the functional map, which is then transformed into point-to-pon to--on map. This innovative line of research was pioneered by FMNet [24] and subsequently expanded in numerous studies [27, 29, 25, 26]. These studies used ground-truth functional maps for supervision.
Simultaneously, parallel line of research focuses unsupervised learning, which proves to be beneficial when ground truth correspondences are unavailable. These methods ensure structural properties like bijectivity and orthonormality functional maps in the reduced basis [28, 29], penalize the geodesic distortion of the predicted maps [53, 54], or harmonize intrinsic and extrinsic shape alignment [35, 55].
However, unlike axiomatic approaches, these methods are typically not competitive when applied on single pair without prior training. In contrast, method, though it is based on a neural network, can be directly applied to a single pair without any prerequisite training.
Shape Reconstruction Our method employs an encoder-decoder framework to deform the source shape to fit the target shape. One significant challenge in this domain is crafting a shape decoder architecture capable of generating a 3D shape from a latent code. Early efforts in this field, such as those by Litany er al., Ranjan et al., and Bouritsas et al. [56, 57, 58], built on graph architec- ture, defining convolution, pooling, and unpooling operations within this structure. However, they presumed uniform triangulation across all shapes, rendering them ineffective for shape matching.
Other methods, such as Zhang et al. [59], predict occupancy probabilities on a 3D voxel grid, but the resource-intensive nature of dense, volumetric representations restricted resolution. Different approaches decod shapes as either raw point clouds [60, 61, 62] or implicit representations [63, 63]. Nevertheless, these methods overlooked the point ordering, making them unsuitable for matching.
In our work, we propose novel decoding architecture, which leverages the architecture of Diffu- sionNet [64] as a foundational element. Our approach computes rotation and a translation matrix for every face on the source shape to align with the target shape. While previous methods relied on energies such as As-Rigid-As-Posse As-Pssbbee (ARAP) [65], Laplacian [66], or edge contraction [34] to restrict the decoder, our approach is adept at using the PriMo energy [37]. This energy encourages smooth, natural deformations that effectively preserve local features, particularly in areas of bending [67], providing more constrained and intuitive approach to shape deformation.
3

---

3 Notation & Background
Our work adopts the functional map paradigm [68] conjunction with the PriMo energy [37]. In the following section, we briefly introduce these concepts, where we will use consistent notation throughout the paper.
3.1 Notation
Given a 3D shape St composed of ny vertices and represented as a triangular mesh, we perform a cotangent Laplace-Beltrami decomposition [69] it. The first l eigenvalues of S; are captured in the matrix 2; E IR We also create diagonal matrix A, E p where the diagonal elements hold the first eigenvalues of S,.
Next, the diagonal matrix of area weights, M; E R is defined. It's worth mentioning that Φ; is orthogonal in relation to M,, and that D, M, is equivalent to the & identity matrix, denoted as 4. Lastly, we represent D D M, using the (left) Moore-Penrose pseudo-inverse symbol, -t,
3.2 Functional Map Pipeline
Denoting source and target shapes as S and S2, we define a pointwise map, T12 : S + S2. encapsulated in binary matrix 112 E [R X22, To manage its quadratic growth with vertex count, employ the functional map paradigm [15], yielding (k k) size functional map C21 x 122 with & usually around 30, making the optimization process feasible.
Feature functions F1 and F2 then derived from S1 and S₂ respectively, with their spectral coefficients calculated as A, D. This enables the formulation of an optimization problem:
arg |CA, Azll A||CA| A2ClI,
(1)
where C the desired functional map and the second term is regularization promote structural correctness [25]. Finally, the point map 1l E {0, 1} is derived from the optimal C12, based on the relationship Φ 122 z II 1 using methods such nearest neighbor search or other post-processing techniques [18, 50].
3.3 PriMo Energy: Prism Based Modeling
The PriMo approach, introduced in [37], is prominent method in shape deformation [67], employing a model that mimics the physically plausible behavior thin shells and plates [70, 71]. In this model, volumetric layer is formed by extruding the input mesh along vertex normals, thus generating rigid prisms for each mesh face (refer to Fig. 2). These prisms are interconnected by elastic joints, which stretch under the extent of this stretch effectively determines the deformation energy. Interestingly, this formulation permits a unique global minimum the energy in the undeformed state, with any form of bending, shearing, twisting, stretching causing increase in this energy.
We elaborate on the elastic joint energy using the notation in Fig. Two adjacent prisms, denoted P. and Pj. are considered. In their undeformed state, they share common face that might no longer match after deformation. The face of P; adjacent Pj is defined as a rectangular bi-linear patch fitj(u, v). which interpolates its four corner vertices. The corresponding opposite face is denoted as fi (u, v) C Py. The energy between P, and P, is thus defined as:
Eig =
| v) fi 2
(2)
10,11
|e F. + F3]
E Wij
Fajz
Wij
(+
Here, E is the deformation energy of the whole mesh, and the energy contribution from each pair P.. P3 is influenced by the areas of their corresponding mesh faces F., Fj. and the squared length of their shared edge eij. In the original PriMo paper, users are allowed to dictate the positions and orientations of some prisms, while an optimization technique is deployed to minimize the total

---

h
1. Initial state
3. Bending
+i
fj
Pj
Pi
6
2. Notation
4. Stretching
Figure 2: PriMo construction. The mesh vertices are represented white circles. The elastic joints are colored yellow.
deformation energy. In contrast, in our approach, the deformation is the result of the decoder's output and the PriMo energy is utilized to regularize it.
Method
In this section. delve into our proposed method for zero-shot matching, which consists of two primary modules. The initial (functional map) module, is responsible for predicting functional maps between the source and target shapes and converting these into point-to-poins (p2p) maps. The second module, an encoder-decoder, encodes the target shape into latent code. Subsequently, the decoder utilizes this latent code to morph the source shape align with the target shape.
Diverging from previous studies, we incorporate losses in both the spatial and spectral domains. Spatially, utilize the p2p map, as predicted by the functional map module, ensure the reconstruc- tion closely resembles the target. Spectrally, we enforce structural properties such as bijectivity and orthogonality on the functional map. Furthermore, facilitate the learning process, we minimize the PriMo energy on the decoder, which ensures it generates natural-looking and smooth deformation.
In the subsequent sections, elaborate each stage of proposed model, and will use the same notation as Sec. 3.
4.1 Functional Map Module
The functional map module is designed with a two-fold objective to predict both fmap and p2p maps reciprocally between the source and target shapes. This module is composed of two key elements: the feature extractor and the fmap solver, as illustrated in Fig. 1.
The feature extractor's purpose is extract features from the raw geometry that are robust to shape deformation and sampling, which are later employed for predicting the fmap. To achieve this, we use the DiffusionNet architecture [64], following the recent approaches in the domain [27, 52, 26, 72]. The feature extractor processes both the target and the source shapes, S and Sn respectively, and extracts pointwise features in Siamese manner, denoted as Fi and F2.
Following the feature extraction, the fmap solver accepts the features F₁ and Fg, and predicts an initial approximation of the fmaps Co and C2 utilizing Eq. (1), These fmaps are matrix solutions to the least square optimization, and thus do not necessarily originate from p2p maps. To rectify this, we draw inspiration from recent methodologies [43, 18, 45, 73, 74] and predict new set of fmaps that stem from p2p maps. Initially, we transform the preliminary fimps into early estimation of p2p maps by applying:

---

Source 2
Per-vertex features
Per-face features
Source faces transformed
Reconstruction Sq
Latent code
Feature extractor
MLP
Per- face rotation and translation prediction R1, f
Figure 3: Prism Decoder. Our novel architecture, built upon DiffusionNet, initiates the process by extracting per-vent vertex features, which are subsequently consolidated into per-face features. These features are then processed by Multilayer Perceptron (MLP) to generate per-face rotation and translation, which are used to rigidly transform the input faces. The transformed faces are then consolidated to produce the final reconstruction.
exp (G G/)/T) Ek= k= exp (G2 Gf)/T)
n2 (1,5)
is a soft p2p map predicted in a differentiable manner, (s represents the scalar product, and r denotes temperature parameter. We predict ff'2 in comparable way, followed by predicting the final fmaps C and C21 that arise from p2p maps using the definition Cy = 2 ji 2.. Ultimately, we estimate the final p2p map ÎL,, from Cý by applying Eq. (4) for second time. This procedure aligns with two iterations of the ZoomOut algorithm [18].
Here, = 41 $ fog
4.2 Shape Reconstruction Module
The goal of the shape reconstruction module is deform the source shape match the target shape. It is composed of a encoder that extract latent code that summarize the target shape, and decoder that takes the latent code as well as the source shape and deform it to match the target.
The encoder a DiffusionNet network [64], that takes the 3D coordinates of the target shape, and produce di dimentiooal vertex-wise features. A max pooling is applied in the vertex dimension to obtain a latent code / E [Rd:
The prism decoder We introduce novel architecture for the decoder, utilizing DiffusionNet at its core. overview of this architecture is presented in Fig. 3. The decoder receives two inputs: the 3D coordinates of the source shape and the latent code 1. The latent code is duplicated and combined with the XYZ coordinates, which are subsequently inputted into the DiffusionNet architecture. This process generates vertex-wise features D RR of dimension d2.
f where These vertex-wise features D are then transformed into face-wise features G f2 denotes the number of faces of the source shape. This transformation is achieved via mean pooling, where the three features corresponding the vertices constituting each face averaged: G. Σjer, Dj, with F; representing the t face of the target shape.
These features are further refined by a compact network composed of pointwise linear layers and ReLU nonlincarities, producing outputs Hl 1/2x12 The first three features are interpreted translation vector t; and the final nine features are reshaped into a 3 x matrix R. This matrix is subsequently transformed into rotation matrix R, by solving the Orthogonal Procrustes problem [75]:
R = arg min ||
RR I|
subject to
TTQ 1z.
(5)
9

---

With & and R2 defined for each face, we transform the original source shape's s faces F, resulting in new of faces F'. The vertices of the decoded shape are then constructed by averaging the new face coordinates' positions for all the faces they belong to.
4.3 Training Procedure
Our method operates in a zero-shot manner, meaning that it is trained each pair of shapes independently. This approach merges the benefits of axiomatic methods, which do not rely on large training sets and extended training periods, and the advantages of learning methods, which facilitate matching through learned features, transcending the limitations of handcrafted features used in axiomatic methods.
The process of our method unfolds as follows: the target and source shapes, designated S₁ and Sg. processed through the functional map module. The output of this stage comprises the fimp and p2p maps, represented as Cjt and iI.- Concurrently, a latent code, is extracted via the encoder and employed to deform the source shape to align with the target shape. The resulting deformed shape, S3, maintains the same triangulation the source shape. Utilizing all these outputs, loss, denoted as L. is computed, which subsequently guides the optimization of the parameters of the various networks through gradient descent. This process iteratively executed until either the loss ceases to decrease or the maximum number of iterations is achieved. The output that yields the lowest loss is ultimately chosen for matching.
In this context, a p2p map between the source and target shape, known as T2 is derived by performing nearest neighbor search between the reconstructed shape Sy and the target shape S1. As suggested in previous works [27, 52, 25, 29, 64, 26], we refine this map by applying the ZoomOut algorithm [18]. This method enhances the map by iterating between the spectral and spatial domains, gradually augmenting the number of spectral basis functions. The refined map, T21, is then employed for evaluation.
Loss function In training our network, we employ multiple loss functions. Contrary to prior works [29, 28, 36, 35] that solely utilize either spatial spectral losses, our method synergistically integrates both types for enhanced performance. The overall loss function is as follows:
L = Ausee + Minap
AcycleCeycle + Aprimoopriime
(6)
We denote S; as the XYZ coordinates of shape S;. The Mean Square Error (MSE) loss, Cyser is employed to ensure that the reconstructed shape closely resembles the target shape using the predicted p2p map, it's calculated as Lise || S
S31l2
The fmap loss, Limap. is used to regularize the structural properties of the predicted fimaps, such as bijectivity and orthogonality [28]: Limap ||C,, J 1 + |C 11. The bijectivity constraint ensures that the map from S, through Sy and back S, the identity map, while the orthogonality constraint enforces local area preservation of the map.
The cycle loss, Leycle, enforces cyclic consistency between predieted p2p maps: Leycle || - 11 Lastly, the primo energy, Cprime. is employed encourage the decoder to yield a coherent shape and to ensure the deformation appears smooth and natural, facilitating the training process. calculated as the energy from Eq. (3).
Results
In this section, we evaluate our method on different challenging shape matching scenarios.
5.1 Datasets
Our experimental setup spans two tasks and involves four distinct datasets referenced in previous literature, incorporating both human and animal representations.
For near-isometric shape matching, we employed several standard benchmark datasets, namely, FAUST [2]. SCAPE [76], and SHREC'19 [77]. To enhance the complexity, we selected the remeshed versions of these datasets, suggested in [29, 25, 48], instead of their original forms. The FAUST
7

---

Table Quantitative Results on Near-Isometric Benchmarks. The best and second-best results are highlighted. Our method comes out on top among axiomatic methods and matches or even surpasses some trained methods.
METHOD
FAUST
SCAPE
SHREC
Ini BCICP [19]
6,4
11
Axion
Ini ZoomOut [18]
6.1
7.5
Smooth Shells [55]
2.5
4.7
12.2
DiscreteOp [43]
5.6
13.1
3D-CODED [34]
2.5
31
FMNet [24]
11
17
Sup
GeoFMNet [25]
3.1
4.4
9.9
DifiGeoFMMNet [64]
2.6
2.9
8.5
Trans.atch [80]
2.7
18.3
14.5
Unsup. FMNet [53]
10.0
16.0
SurFMNet [28]
15.0
12.0
30.1
Unup
Cyclic FMaps [54]
14.8
12.7
36.5
WSupFMNet [29]
3.3
7.3
11.3
Deep Shells [35]
1.7
2.5
21.1
NeuroMorph [36]
8.5
29.9
26.3
SNK (Ours)
1.8
4.7
5.8
dataset contains 100 shapes, depicting 10 individuals in 10 different poses each, with our evaluation focused on the last shapes. The SCAPE dataset comprises distinct poses of a single person, and based assessment the final 20 shapes. The SHREC dataset, with its notable variations in mesh connectivity and poses, presents a more significant challenge. It consists 44 shapes and total of 430 evaluation pairs. For unsupervised methods, we used the oriented versions of the datasets as detailed in [29]. Moreover, for training methods, we showcase their optimal performance on SHREC using model trained either FAUST SCAPE.
In the context of non-isometric shape matching, we conducted our experiments using the SMAL animal-based dataset [78, 79], hereinafter referred to as SMAL. This dataset includes 50 distinct, non-rigid 3D mesh models of organic forms. We divided these into two equal sets of for training and testing, ensuring that the specific animals and their poses used for testing were not seen during the training phase.
5.2 Near-Isometric Shape Matching
In this section, evaluate method's performance in near-isometric, non-rigid shape matching. We draw comparisons with several contemporary approaches the field of non-rigid shape matching. These be broadly divided into three categories. The first category is axiomatic approaches and it includes BCICP [48], ZoomOut [18], Smooth Shells [55], and DiscreteOp [43]. The second category is supervised learning approaches and it comprises methods such 3D-CODED [34], FMNet [24], GeomFmaps [25], DiffGeoFMNet [64], and Mrthh [80]. Finally the third category unsupervised learning approaches and it includes Unsupervised FMNet [53], SURFMNet [28],
Cyclic FMaps [81], Weakly Supervised FMNet [29]. DeepShells [35], and NeuroMorph [36].
To evaluate all correspondences,
adopt the Princeton protocol, method widely used in recent works [24, 25, 27, 29, 35]. Specifically, we calculate the pointwise geodesic error between the predicted maps and the ground truth map, normalizing it by the square root of the target shape's 's total surface area.
Results are presented in Table 1. As can be seen, our method outperforms axiomatic approaches and comparable or superior to some training-based methods, particularly on the FAUST and SCAPE datasets. When it comes to the SHREC dataset, our method achieves state-of-the-art results across all methods. Most learning- been trained on many training examples. This is because of the wide mix of poses and identities in the
methods stumble over this challenging dataset, even though they have

---

Source
Target
Smooth Shells
Deep Shells
Dift
Ours
Figure 4: Qualitative Results from the SHREC dataset. The correspondences are visualized by transferring a texture through the map. Our method results in visually superior outcomes.
Table 2: Quantitative results on the non-isometric benchmark. We highlight the best and the second best results. Our method achieves competitive results with trained methods.
METHOD
SMAL
FMNet [24]
47.1
GeoFMNet [25]
8.8
Sup
DiffGeoFMNNet [64]
6.6
LIE [82]
20
WSupFMNet [29]
13.3
Axiom Unup
Deep Shells [35]
15.2
NeuroMorph [36]
23.1
Smooth Shells [55]
16.3
SNK (Ours)
9.1
dataset. This highlights how our approach, which focuses on overfitting a single pair, is particularly effective such challenging cases.
In Fig. 4, we showcase point-to- tonttto--pint maps generated by our method on the SHREC dataset. We compare these maps those produced by the bert-performing baseline method across each category axiomatic, unsupervised, and supervised. Upon inspection, it is apparent that our method attains highly accurate correspondences. In contrast, the baseline methods either fail to achieve precise matching or suffer from symmetry flip issues.
5.3 Non-Isometric Shape Matching
In the non non-isometric, non-rigid context, we adhered to the same evaluation protocol and present our findings Table It can be seen from these results that our method outperforms both the axiomatic and unsupervised approaches, and competes strongly with, or even surpasses, some supervised techniques. This further demonstrates the efficacy and versatility of our method across diverse mhape-matching scenarios.

---

Source
Target
Source
Target
Non isometric
Non homeomorphic
Figure 5: Examples of non-isometric and non-homeomorphie matching on the SMAL and SHREC datasets, respectively.
In Fig. 5, we present qualitative examples of non-isometric and non-homeomorphic matching from the SMAL and SHREC datasets, respectively. The results further highlight the resilience of our approach to various deformations and topological changes.
5.4 Ablation Studies & Additional Details
In the supplementary material, we present an ablation study that underscores the efficacy of each component introduced in our methodology. Additionally, we provide comprehensive details regarding the implementation process, the computational specifications involved, as well as some qualitative results.
Importance of the neural prior In the supplementary we also evaluate the importance of using neural networks for training method. In particular, we show that when unconstrained optimization replaces neural networks, the results degrade significantly. This illustrates the importance of "neural prior" [83, 84] employed by our method and further highlights the difference between our approach and purely axiomatic methods.
Conclusion & Limitations
In this work, introduced SNK, novel approach zero-shot shape matching that operates independently of prior training. Our method hinges on the precise alignment of a pair of shapes, enabling the source shape to be deformed to match the target shape without any intervention. By integrating both spatial and spectral losses, and architecting a new decoder that encourages smooth and visually pleasing deformations, our method has proven its effectiveness. Extensive experiments reveal that our approach competes with, and often surpasses, many supervised techniques, even achieving state-of-the-art results the SHREC dataset.
Despite its success, SNK has certain limitations we aim to address in future research. Firstly, our method is currently tailored towards complete shape correspondence, necessitating adaptation for partial shape-matching scenarios. Secondly, while our method's convergence time of approximately 70 seconds per shape outpaces many competitors, exploring ways to further expedite this process would be beneficial. Finally, our method utilizes DiffusionNet as a backbone, which theoretically could accommodate both point clouds and triangular 3D meshes. Consequently, extending approach to encompass other types of representations could prove useful.
The authors would like thank the anonymous reviewers for their valuable suggestions. Parts of this work were supported by the ERC Starting Grant No. 758800 (EXPROTEA) and the ANR AI Chair AIGRETTE.
Acknowledgements
10

---

References
[1] Leonid Pishchulin, Stefanie Wuhrer, Thomas Helten, Christian Theobalt, and Bernt Schiele. Building statistical shape spaces for 3d human modeling. Pottern Recognition, 67:276-286, July 2017. 1
[2] Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. FAUST: Dataset and evaluation for 3d mesh registration. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, June
2014. 7,
[3] Ilya Baran, Daniel Vlasic, Eitan Grinspun, and Jovan Popović. Semantic deformation transfer. ACM Transactions on Graphics, 28(3): 6, July 2009. 1
[4] Qian- Yi Zhou, Jaesik Park, and Vladlen Koltun, Fast global registration. In Computer Vision ECCV 2016, pages 766-782. Springer International Publishing. 2016.
[5] Huong Quynh Dinh, Anthony Yezzi, and Greg Turk. Texture transfer during shape transformation. ACM Transactions on Graphics, 24(2)288-310,, April 2005.
[6] Yaron Lipman and Thomas Funkhouser. Möbius voting for surface correspondence. ACM Transactions on Graphics (Proc. SIGGRAPHI, 28(3), August 2009.
[7] Emanuele Rodolà, Luca Cosmo, Michael M Bronstein, Andrea Torsello, and Daniel Cremers. Partial functional correspondence. In Computer Graphics Forum, volume 36, pages 222-236. Wiley Online Library, 2017. 3,
[8] Artiom Kovnatsky, Michael M Bronstein, Alexander M Bronstein, Klaus Glashoff, and Ron Kimmel. Coupled quasi-harmonic bases. In Computer Graphics Forum, volume 32, pages 448. Wiley Online Library, 2013. 3,
[9] O. Litany, E. Rodolà, A. M. Bronstein, M. M. Bronstein, and D. Cremers. Non-rigid puzzles. Computer Graphics Forum, 35(5): 1355):335, August 2016. 1,3
[10] O. Litany, F. Rodolà, A. M. Bronstein, and M. M. Bronstein. Fully spectral partial shape matching. Computer Graphics Forum, 36(2):247-258, May 2017. 1,3
[11] Facundo Mémoli and Guillermo Sapiro. thcoretical and computational framework for isometry invariant recognition of point cloud data. Foundations of Computational Mathematics, 5(3):313-344 June 2005. I
[12] Alexander M. Bronstein, Michael M. Bronstein, and Ron Kimmel. Generalized multidimensional scaling:
A framework for isometry- invariant partial surface matching. Proceedings of the National Academy of Sciences, 103(5):1 166-1112,, January 2006.
[13] Noam Aigerman and Yaron Lipman. Hyperbolic orbifold tutte embeddings. ACM Transactions on Graphics, 35(6):1-14, November 2016.
[14] Noam Aigerman and Yaron Lipman. Orbifold tutte embeddings. ACM Trunsuctions on Graphics, 34(6):1-
12, November 2015. 1
[15] Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Functional
maps. ACM Transactions on Graphics, 31(4):1-11, August 2012. 1, 3,4
[16] Dorian Nogneng and Maks Ovsjanikov. Informative descriptor preservation via commutativity for shape matching. Compuder Gruphics Forum, 36(2):259-26) May 2017. 1,3
[17] Davide Eynard, Emanuele Rodola, Klaus Glashoff, and Michael M Bronstein. Coupled functional maps. In 3D Vision (3DV), pages 399-407. IEEE, 2016. 1,3
[18] Simone Melzi, Jing Ren, Emanuele Rodolå, Abhishek Sharma, Peter Wonka, and Maks Ovsjanikow. ZoomOut: Spectral upsampling for efficient shape correspondence. ACM Transactions on Graphics, 38(6):1-14, Nov 2019. 1, 3, 4, 5, 6, 7,
[19] Jing Ren, Adrien Poulenard, Peter Wonka, and Maks Ovsjanikov. Continuous and orientation- preserving correspondences via functional maps. ACM Transactions on Graphics (ToG), 37 2018. 1, 3, 8, 17
[20] Gautam Pai, Jing Ren, Simone Melzi, Peter Wonka, and Maks Ovsjanikov. Fast sinkhorn filters: Using matrix scaling for non-rigid shape correspondence with functional maps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 384-393, June 2021.
[21] Samuele Salti, Federico Tombari, and Luigi Di Stefano. SHOT: Unique signatures of histograms for surface and texture description. Computer Vision and Image Understanding, 125:251-264 August 2014.
1, 3
[22] Jian Sun, Maks Ovsjanikov, and Leonidas Guibas. A concise and provably informative multi-scale signature based heat diffusion. In Computer graphics forwn, pages 1383- 13922. Wiley Online Library, 2009. 1,
[23] Mathieu Aubry, UIrich Schlickewei, and Daniel Cremers. The wave kereel signature: A quantum mechanical approach to shape analysis. In 2011 IEEE international conference on computer vision workshops (ICCV workshops). pages 1626 1633. IEEE, 2011. 1,
11

---

[24] Or Litany. Tal Remez, Emanuele Rodola, Alex Bronstein, and Michael Bronstein. Deep functional maps: Structured prediction for dense shape correspondence. In Proceedings of the IEEE international conference on computer vision, pages 5659-5667, 2017. 8,9 9,
[25] Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. Deep geometric functional maps: Robust feature learning for shape correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8592 8601, 2020. 1, 4, 7, 8,9
[26] Lei Li, Souhaib Attaiki, and Maks Ovsjanikow. SRFcat: Learning locally accurate and globally consistent non-rigid shape correspondence. In 2022 International Conference on 3D Vision (3DV). IEEE, September 2022. 3, 7
[27] Souhaib Attaiki, Gautam Pai, and Maks Ovsjanikov. DPFM: Deep partial functional maps. In 2021 International Conference on 3D Vision (3DV). IEEE, December 2021. 1,3, 7,
[28] Jean-Michel Roufosse, Abhishek Sharma, and Maks Ovsjanikov. Unsupervised deep learning for structured shape matching. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 1617-1627, 2019. 1, 3. 7, 8, 18
[29] Abhishek Sharma and Maks Ovsjanikov. Weakly supervised deep functional maps for shape matching. In
H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information P Processing Systems, volume 33, pages 19264-19275. Curran Associates, Inc., 2020. 7, 7,8,9 9
[30] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model cans. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5115- 5-244, 2017. 1.3
[31] Davide Boscaini. Jonathan Masci, Emanuele Rodoià, and Michael Bronstein. Learning shape correspon- dence with anisotropic convolutional neural networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 3197-3208 2016. 1,3
[32] Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. Geodesic convolutional neural networks on riemannian manifolds. Proceedings of the IEEE international conference on computer vision workshops, pages 37-45, 2015. 1,3
[33] Adrien Poulenard and Maks Ovsjaníkov. Multi-directional geodesic neural networks via equivariant convolution. ACM Transactions on Graphics (TOG), 37(6):1-14, 2018. 1,3
[34] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 3d-coded: 3d correspondences by deep deformation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 230-246, 2018. 3,
[35] Marvin Eisenberger, Aysim Toker. Laura shape correspondence with optimal transport. In H. Larochelle, M. Ranzato. R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Prucessing Systems, volume 33, pages 10491-10502. Curran Associates, Inc., 2020. 1, 3. 7, 8, 9, 18
and Daniel Cremers. Deep shells: Unsupervised
[36] Marvin Eisenberger. David Novotný, Gael Kerchenbaum, Patrick Labatut, Natalia Neverova, Daniel Cremers, and Andrea Vedaldi. Neuromorph: Unsupervised shape interpolation and correspondence one go. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7469 7479, 2021. 1,7,8,9, 9, 18
[37] Mario Botsch, Mark Pauly, Markus Gross, and Leif Kobbelt. PriMo: Coupled Prisms for Intuitive Surface Modeling. In Alla Sheffer and Konrad Polthier, editors, Symposium on Geometry Processing. The Eurographics Association, 2006. 2, 3, 4
[38] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for point clouds: A survey. IEEE transactions on pattern analysis and machine intelligence, 2020.
[39] Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-44 2017.
[40] Wenming Cao. Zhiyue Yan, Zhiquan He, and Zhihai He. A comprehensive survey on geometric deep learning. IEEE Access, 8:35929-35949, 2020. 3
[41] Yulan Guo, Mohammed Bennamoun, Ferdous Sohel, Min Lu, Jianwei Wan, and Ngai Ming Kwok. comprehensive performance evaluation of 3d local feature descriptors. International Journal of Computer Vision, 116(1):66 89, 2016. 3
[42] Adrien Poulenard, Primoz Skraba. and Maks Ovsjanikov. Topological Function Optimization for Continu- ous Shape Matching. Computer Graphics Forum, 37(5):13-25, 2018. 3
[43] Jing Ren, Simone Melzi, Peter Wonka, and Maks Ovsjanikov. Diserete optimization for shape matching. Computer Graphics Forum, 40(5):81-90 August 2021. 5,8
[44] Jing Ren, Simone Melzi, Maks Ovsjanikov, and Peter Wonka. Maptree. ACM Transactions on Graphics 39(6):1 Dec 2020. 3
12

---

[45] Souhaib Attaiki and Maks Ovsjanikov. Understanding and improving features learned in deep functional maps. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2023. 3,5
[46] Vincent Mallet, Souhaib Attaiki, and Maks Ovsjanikov. Atomsurf: Surface representation for learning on protein structures, 2023. 3
[47] Maks Ovsjanikov, Etienne Corman, Michael Bronstein, Emanuele Rodolà, Mirela Ben-Chen, Leonidas Guibas, Frederic Chazal, and Alex Bronstein. Computing and processing correspondences with functional maps. In ACM SIGGRAPH 2017 Courses. ACM, July 2017.
[48] Jing Ren, Adrien Poulcnard, Peter Wonka, and Maks Ovsjanikov. Continuous and oriennation-presereee preservving correspondences via functional maps. ACM Transactions Graphics (TOG), 37(6), 2018. 3, 7,8
[49] Luca Cosmo, Emanuele Rodola, Michael M Bronstein, Andrea Torsello, Daniel Cremers, and Y Sahillioglu. Shrec' 16: Partial matching of deformable shapes. Proc. 3DOR, 2(9):12, 2016. 3, 20
[50] Matthias Vestner, Roce Litman, Emanuele Rodola, Alex Bronstein, and Daniel Cremers. Product manifold filter: Non-rigid shape correspondence via kemee density estimation in the product space. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul 2017. 3,4
[51] Danielle Fzuz and Mirela Ben-Chen. Deblurring and denoising of maps hetween shapes. In Computer Graphics Forum. Wiley Online Library. 2017. 3
[52] Nicolas Donati, Etienne Corman, and Maks Ovsjanikov. Deep Orientation-Aware Functional Maps: Tackling Symmetry Issues in Shape Matching. CVPR "12: IEEE Conference on Computer Vision and Pattern Recognition, 2022. 3, 5,7
[53] Oshri Halimi, Or Litany, Emanuele Rodola, Alex M Bronstein, and Ron Kimmel. Unsupervised learning of dense shape correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4370-4377 2019. 3,8
[54] Dvir Ginzburg and Dan Raviv. Cyclic functional mapping: Self-supervised correspondence between non-isometric deformable shapes. In Computer Vision - ECCV 2020, pages 36-52. Springer International Publishing, 2020. 3,8 8
[55] Marvin Eisenberger, Zorah Lähner, and Daniel Cremers. Smooth shells: Multi-scale shape registration with functional maps. 2020 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 12262-12271, 2020. 3, 8, 9, 17, 19
[56] Or Litany, Alex Bronstein, Michael Bronstein, and Ameesh Makadia. Deformable shape completion with graph convolutional autoencoders. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1886-1895. TEFF, 6 2018. 3
[57] Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J. Black. Generating 3d faces using convolu- tional mesh autoencoders. Proceedings of the European Conference on Computer Vision, pages 704-720, 2018. 3
[58] Giorgos Bouritsas, Sergiy Bokhnyak, Stylianos Ploumpis, Stefanos Zafeiriou, and Michael Bronstein. Neural 3d morphable models: Spiral convolutional networks for 3d shape representation learning and generation. Proceedings of the IEEE International Conference on Computer Vision, 201 0019-0ctob::212- 7221, 2019. 3
[59] Yang Zhang, Zhen Liu, Tianpeng Liu, Bo Peng. and Xiang Li. Realpoint3d: An efficient generation network for 3d object reconstruction from a single image. IEEE Access, 7:57539-515 57549, 2019. 3
[60] Panos Achlioptas, Olga Diamanti, loannis Mitliagkas, and Lconidas Guibas. Learning representations and generative models for 3d point clouds. 35th International Conference on Machine Learning, ICML 2018, 1:67-85, 2018. 3
[61] Can Qin, Haoxuan You, Lichen Wang, C.-C. Jay Kuo, and Yun Fu. Pointdan: A multi-scale 3d domain adaption network for point cloud representation. In H. Wallach, H. Larochelle, A. Beygelzimer, F, d'Alché- Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. 3
[62] Yongheng Zhao, Tolga Birdal, Haowen Deng, and Federico Tombari. 3d point capsule networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 3
[63] Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for leamning shapes. International Conference on Machine Learning, pages 3789-3799. PMLR, 2020.
[64] Nicholas Sharp, Souhaib Attaiki, Keenan Crane, and Maks Ovsjanikov. Diffusionnet: Discretization
agnostic learning on surfaces. ACM Trans. Graph., 01()), 2022. 3, 5, 7, 9, 16
[65] Olga Sorkine and Mare Alexa. As As-Rigid-Ar As Possiilee Surface Modeling. In Alexander Belyaev and Michael Garland, editors, Geometry Processing. The Eurographics Association, 2007. 3, 18
13

---

[66] Mario Botsch and Olga Sorkine. On linear variational surface deformation methods. IEEE Transactions Visualization and Computer Gruphics, 14(1):213-230, 2008. 3
[67] Zohar Levi and Craig Gotsman. Smooth rotation enhanced as-rigid-a--possible mesh animation. IEEE
Transactions on Visualization and Computer Graphics, 21 21(2):66--777 2015. 3, 4, 18
[68] Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Functional
Maps: A Flexible Representation of Maps Between Shapes. ACM Transactions on Graphics (TOG), (4):330, 2012. 4
[69] B. Vallet and Levy. Spectral Geometry Processing with Manifold Harmonics. Computer Graphics Forum, 2008. 4
[70] George Celniker and Dave Gossard. Deformable curve and surface finite-elements for free-form shape design. ACM SIGGRAPH Computer Graphics, 25(4):257-266 July 1991.
[71] Demetri Terzopoulos, John Platt. Alan Barr. and Kurt Fleischer. Elastically deformable models. ACM SIGGRAPH Computer Graphics, 21 21(4):205-214, August 1987. 4
[72] Souhaib Attaiki, Lei Li, and Maks Ovsjanikov. Generalizable local feature pre-training for deformable shape analysis. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2023.
[73] Mingze Sun, Shiwei Mao, Puhua Jiang, Maks Ovsjanikov, and Ruqi Huang. Spatially and spectrally consistent deep functional maps. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14497-14507, 2023. 5
[74] Dongliang Cao, Paul Roetzer, and Florian Bernard. Unsupervised learning of robust spectral shape matching, ACM Trans. Gruph., 42(4), jul 2023. 5
[75] Peter H. Schönemann. generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1):1-10, March 1966. 6, 16
[76] Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis.
SCAPE. ACM Transactions on Gruphics. 24(3):408 416, July 2005.
[77] S. Melzi, R. Marin, E. Rodolà, U. Castellani, J. Ren, A. Poulenard, P. Wonka, and M. Ovsjanikov. Matching
humans with different connectivity. Eurographics Workshop on 3D Object Retrieval, 2019. 7
[78] Silvia Zuffi, Angjoo Kanazawa, David Jacobs, and Michael J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In IEEE Conf. Computer Vision and Pattern Recognition (CVPR), July 2017.
8
[79] Riccardo Marin, Souhaib Attakii, Simone Melzi, Emanuele Rodolà, and Maks Ovsjanikov. Why you should learn functional basis, 2021. 8
[80] Giovanni Trappolini. Luca Cosmo, Luca Moschella, Riccardo Marin, Simone Melzi, and Emanuele Rodolà. Shape registration in the time of transformers. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors,Advancee, in Neural Information Processing Systems, volume 34, pages 5731-5744. Curran Associates, Inc., 2021. 8
[81] Dvir Ginzburg and Dan Raviv. Cyclic functional mapping: Self-supervised correspondence between non-isometric deformable shapes. 2019. 8
[82] Riccardo Marin, Marie-Julie Rakotosaona, Simone Melzi, and Maks Ovsjanikov. Correspondence learning via dincarly-invarian embedding. ArXiv, 0///20100131366 2020, 9
[83] Souhaib Attaiki and Maks Ovsjanikov. NCP: Neural correspondence prior for effective unsupervised shape matching. Advances Neural Information Processing Systems, 2022. 10, 18,
[84] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior.
2017. 10.
18, 19
[85]
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. 16
[86] Berthold K. P. Horn. Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A, 4(4):299, April 1987. 17
[87] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf. Edward Yang. Zachary De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024 8035. Curran Associates, Inc., 2019. 17
[88] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 165- 174, 2019. 18
14

---

[89] Michal Edelstein, Danielle Ezuz, and Mirela Ben-Chen. ENIGMA. ACM Transactions on Gruphics, 39(4). August 2020. 19
[90] A. Gehre, M. Bronstein, L. Kobbelt, and J. Solomon. Interactive curve constrained functional maps. Computer Graphics Forum, 37(5):1-12 12, August 2018, 19
[91] D. Ezuz, B. Heeren, Azencot, M. Rumpf, and Ben-Chen. Elastic correspondence between triangle meshes. Computer Graphics Forum, 38(2):211-344 May 2019. 19
15

---

Supplementary Materials for: Shape Non-rigid Kinematics (SNK)
Souhaib Attaiki LIX, École Polytechnique, IP Paris attaiki@lix. www..oliww.....ttttiiuuuuuue.... fr
Maks Ovsjanikov
LIX, École Polytechnique, IP Paris maks@lix, wol.teeciicceeeiu.u......
In this document, we have gathered all the results and discussions that, due to page limitations, were not included in the main manuscript.
More precisely, we initially detail the specifics of our implementation in Appendix Following this, we present comprehensive analysis of the components introduced in our methodology in Appendix B. Further examination of our architecture s components can found in Appendix Lastly, we include some qualitative results in Appendix D.
A Implementation Details
A.1 Network Architecture
In Section 4 of the main paper, we elaborated on our architecture, which is comprised of two primary modules: the functional map module and the shape reconstruction module.
For the functional map module, we utilized DiffusionNet [64] as a feature extractor, using the authors' original implementation It consists of four blocks, each with width of 128, and the feature dimension is likewise 128. The functional map block possesses a dimension of A: = 30, and we used À = 10-3 in Equation 1, and r 0.07 in Equation 4 from the main paper.
The shape reconstruction module employs an encoder via DiffusionNet, comprising four blocks and possessing width of 128, with an output dimension dj = 512. We derive the latent code / by performing max-pooling over the vertex dimension.
In terms of PriMo modeling, we construct a volumetric layer by (virtually) extruding the input mesh along the vertex normals. This process generates rigid prisms for each mesh face, with each prism having a height, h, set to 0.02 (see Figure of the main text).
Our novel decoder, the prism decoder, relies on DiffusionNet at its core. Given the source shape and the latent code we duplicate the latter, concatenate with the XYZ coordinates of the source shape, and then process them with a DiffusionNet composed of four blocks with width of 128. The output is pointwise feature E I Xd2 with de = 512. We transform these features into face-wise features G, described in the main paper. These features are subsequently refined with a pointwise MLP, which consists of four layers Linear layer and ReLU layer; however, the last layer does contain ReLU, The width of the MLP gradually narrows from 512 12 in the final layer. We obtain the matrix R in Equation from the main paper by initially performing an SVD decomposition [75] of Ro such that Ro UDVT The solution is then obtained by R USVT with S = diag (1, 1, ded((U)dee (V)). Employing the predicted translation and rotations matrices I, and R2 for each face (considered prism in the PriMo design), we rigidly deform the source mesh faces. The deformed surface mesh is finally derived from the resulting prisms by updating the position of each vertex using the average transformation of its incident prisms.
For all our experiments, we utilized the Adam optimizer [85], with learning rate preset to 0.001. With respect the losses in Equation 6 from the main text, we used Xmse Xfmap Aeycle
Aprimo 1.
'https:// httt:::://gttab...o/////serr/// www.///wwhatrr/ttfffffoooo.ooo
16

---

Table Runtime comparison. comparison of the runtime of various baseline methods with our method on the FAUST dataset. Time is presented in seconds.
METHOD
Runtime (s)
BCICP [19]
780
Smooth Shells [55]
430
SNK (Ours)
70
For every new pair, we trained our network over the course 1000 iterations, terminating the training if no improvement in the loss was observed for over 100 consecutive iterations.
A.2 PriMo Loss Implementation
In the main paper, Equations and 3 introduced the formulation for the PriMo energy. this section, we further detail the practical computation of this energy.
Given two functions a(u) and b(u). defined by bi-linear interpolation of four values faoo. A10 ao1, )11} and {bro, 61o, b01 b11) respectively, we define the scalar product [86]:
(a, 2) = f
10.11 a(u) b(u)du
0ij bui 2(-1i-M1-15-1)
(7)
k.j.k.I00
Next, referring Figure the main paper, the energy Eig is simply defined as:
Eig (fi fi pi f)-")/2
(8)
Here, fi
(fo0 1o
fo ( represents the four corners of the face of prism P, that is
adjacent prism Pj-
A.3
Computational Specifications
We carry out all our experiments using Pytorch [87] on a 64-bit machine equipped with an Intel(R) Xeon(R) CPU ES-2630 v4 operating at 2.20GHz and an RTX 2080 Ti Graphics Card. For comparison, we use the original code provided by the authors each competing method and adhere to the optimal parameters reported in their respective papers.
A.4 Runtime Comparison
Our method's runtime performance, compared to various baselines, is analyzed in Tab. 3. Specifically, we assess the overall runtime for all the baselines on the FAUST dataset, which consists of shapes averaging around 5000 vertices. The average time per pair, reported in seconds, demonstrates that our method outpaces the competition in terms of speed.
B Ablation Study
In Section 4 of the main text, we introduced the components our methodology. this section, we investigate the impact of each component independently. For illustrative purposes, utilize the near-isometric matching experiment on the FAUST dataset (as referenced Section 5.2 of the main text). Nevertheless, analogous conclusions be drawn in other scenarios.
Initially, we examine the novel decoder architecture we proposed. this case, we maintain other aspects of our method and solcly modify the decoder. We experiment with decoder based
17

---

Table 4: Ablation Study. This table displays an ablation study of the various components of our method, conducted on the FAUST dataset. The study shows that every component is crucial to the overall performance. The best result highlighted.
METHOD
FAUST
Encoder Decoder
MLP
35,9
DiffusionNet
13.4
DiffusionNet ARAP
14.8
Latent Code
4.0
Chamfer loss
80.7
Losses
Spectral only
8.7
Ours (without Lprima)
6.5
Ours (without Leycle)
2.7
SNK (Ours)
1.9
Multilayer Perceptron (MLP) network (m), and one based on the DiffusionNer network (+), both possessing comparable number of parameters to our decoder. Furthermore, given that several methods use the ---Smooth-As-Posiie (ARAP) [65, 35, 36] energy for deformation regularization, also compare our approach with DiffusionNet decoder supplemented with ARAP loss (x).
Concerning the encoder, besides the shape encoder design we employed, we also explored the option of directly learning the latent code (marked with e) in way similar to DeepSDF [88].
With respect to the loss functions, we compare our method against an approach that exclusively overfits the spectral losses [28], and then report the result of converting the predicted functional maps into point-to- to-pootttt (0). Additionally, we also report the outcome of training our network with the unsupervised Chamfer loss as substitute for the Mean Squared Error (MSE) loss Lmse that we used (0). Moreover, we experiment with our network trained without the PriMo loss Cprimo (a) and finally without the cycle loss Leyele
The results are summarized in Tab. illustrating that all the components we incorporated are crucial for optimal performance. Specifically, the choice of the decoder is vital. Both the MLP and DiffusionNet decoders failed match the results of our decoder, even when all our loss functions were used. The ARAP loss, in particular, hindered the performance of the DiffusionNet decoder, contrast its typical application network training on dataset.
Regarding the encoder, the shape encoder design outperforms the latent code design. This superior performance can be attributed to the shape encoder's ability to utilize the input shape added information source. This advantage is further enhanced by the intrinsic regularization provided by the network architecture, often referred to the neural bias [83, 84].
In terms of the loss functions, training our architecture with the Chamfer loss was completely unsuccessful as it does not preserve the order of the points and is therefore unsuitable for matching. Using only the spectral loss yields decent results but falls short of state-of-the illustrates the importance of the interplay between spectral and spatial losses for optimal performance. The PriMo Energy plays a key role as, without it, the decoder fails to produce smooth, natural- looking deformations [67] that facilitate learning. Finally, the cycle loss proves critical in refining the point-to-point maps and achieving state-of-the-art results.
performance. This
C
In-Depth Exploration of SNK'S Components
C.1 The Feature Extractor
The feature extractor plays a pivotal role in our architecture. The features it extracts are crucial for predicting both the fmap and the p2p maps, which turn influence consistency between the features of the two shapes is essential. To clucidate this, we have provided
the loss. Consequently,
18

---

Features
Features
Figure 6: Consistency between extracted features for source and target shapes.
visualization in Fig. 6 that highlights the congruence of features produced by the feature extractor for a shape pair. this representation, we reduced the feature dimensions using PCA and then portrayed them as RGB values. The visualization readily demonstrates that the features from both shapes align closely, highlighting congruent areas.
C.2 SNK's Priors
In this section, detail the priors employed by method and highlight the unique neural prior that sets our approach apart from axiomatic methods.
Elastic Energy as a Prior: PRIMO being an clastic energy, it penalizes non-linee stretching and bending. This directs our optimization towards smooth and realistic deformations.
Cyclic Bijectivity via the cycle consistency loss.
Near-isomeery: Our method weakly enforces near-isonee pear-isometry in fmap predictions through 30) functional basis. should be noted that using elastic energy and weakly promoting isometries with low-frequency functional maps for non-isometric matching has precedents in works like [89, 90, 91, 55].
Laplacian commutativity in a reduced
Neural Networks: Finally, subtle yet essential prior in our method in our use of a neural network for feature extraction. Our approach follows studies like Deep Image Prior [84] and Neural Correspondence Prior [83] which show that neural networks can act as strong regularizers, generating features that enable plausible mappings even without pre-training
To demonstrate the neural prior empirically, we conducted two additional experiments:
HKS-based Features: We replaced our feature extractor with the more traditional HKS features.
Free Variables: We treat Fi and F&z free variables, optimized via gradient descent without using a neural network.
Our findings (see Tab. 5) show that without the neural network regularization, the free variable model fails converge. The IKSS-based approach performs, but significantly worse than ours. This suggests neural networks offer extra regularization, surpassing traditional handcrafted features.
Overall, our method's 's priors are synergistic and versatile, making our approach applicable to broad range of shapes.
D Qualitative Results
In this section, we present selection of visual results derived from our proposed method. Firstly, Fig. 7, we present the reconstructed shapes from the SCAPE dataset as produced by our method when overfitting shape pair. This reconstruction is subsequently used to perform matching
19

---

Table Ablation on the neural prior.
METHOD
FAUST
HKS features
20.3
Free Variables for Features
64.0
SNK (Ours)
1.9
Target
Source
Reconstruction
Figure 7: Shape Reconstruction from the SCAPE dataset: The figure showcases the results of deforming the source shape to match the target shape using our architecture. It's noteworthy how well the shape has been reconstructed, a process that was completed in entirely unsupervised manner.
using the nearest-neighhor approach. The high quality of the reconstruction is evident, even when dealing with challenging input pairs where the source and target pose exhibit significant variance.
Secondly, inspect the quality of the shapes reconstructed by our novel decoder architecture during training. For this experiment, we deform a source shape to match a target shape using only the Luse loss. Instead of utilizing the map generated by the functional map module, employ the ground truth map, visualizing various shapes produced during the optimization process. We compare our decoder to DiffusionNet-basee decoder and a decoder utilizing DiffusionNet with ARAP loss applied, aligning with the approach in Appendix B. These results are illustrated in Fig. 8 using shape pair from the FAUST dataset. Our method is the only one to produce natural-looking, smooth deformations, which in turn results in superior matching outcomes.
Lastly, showcase geared toward complete shape matching, but in Fig. 9, we showcase our method's performance on three partial pairs from the Shrec 16' Cuts and Holes subsets [49]. Our method performs satisfactorily when dealing with moderate partiality.
failure cases of method the partial setting. In fact, method is
20

---

Source
Interpolation
Target
Prism Decoder
DiffusionNet
DiffusionNet + ARAP
Figure Shape Interpolation using FAUST dataset: Various decoders are employed to interpolate between source and target shape. Compared to the baselines, method yields smooth and natural- looking deformations.
Target
Source
Target
Source
Target
Source
Figure 9: Our method is tailored toward complete shape matching. Here we showcase the results of our method on some partial pairs from the Shrec partial benchmark.
21


Reviews:
[{"review_id": "B9ZPJrnleR", "reviewer": "Reviewer_osbV", "summary": "This paper works on learning a point-to-point mapping between two sets of deformable mesh vertices in a self-supervised manner. To this end, the authors extract features of two meshes from a DiffusionNet, solve for functional maps by an optimization problem, and finally convert the functional maps to a point-to-point map iteratively. \n\nIn the meantime, a per-face rigid transformation is generated from a shape decoder and is used to transform the source shape to the target shape. To demonstrate the effectiveness of the proposed method, the authors conduct experiments on both near-isometric and non-isometric datasets, and achieve reasonable performance as a self-supervised method.", "questions": "The shape encoder might be redundant, and similar to DeepSDF the authors can directly learn the latent code jointly with other module parameters.", "limitations": "The limitations have been adequately addressed.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well organized and written. The problem is well defined in Sec.1, while the previous works and preliminary knowledge are also well introduced in Sec.2 and Sec.3.\n\n- The two-stream (implicit shape transformation decoder and explicit functional map) pipeline is a reasonable design:\n  - The low-rank functional map estimation is not only efficient but also facilitates the learning of feature extraction.\n  - The MLP shape decoder and the PriMo energy regularize the predicted transformation by both network structure and training losses.\n\n- As a self-supervised method, the performance is strong on both near-isometric and non-isometric data.", "weaknesses": "I do not see major weaknesses except for some details:\n- The feature extractor should have more discussion, since it is key for the functional map prediction. Specifically, the author could either visualize or quantitatively measure the consistency of the extractor features. In addition, it would be better to explicitly indicate that the DiffusionNet is not pre-trained (in spite of lines 36~37) to avoid confusion with existing models.\n\n- The pipeline figures could be better illustrated, such as the module blocks can be colored based on whether they are learned or have explicit formulation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed and thoughtful comments and suggestions. We invite the reviewer to consider the general feedback that we provided to all reviewers in the \"Author Rebuttal\" answer. Below are our answers to the reviewer's questions.\n\n\n**Q1: Further Insights on the Feature Extractor**\n\nThank you for this question. Indeed, the features computed by our feature extractor are generally consistent with respect to the underlying maps. To illustrate this, we've included a visualization that showcases the consistency of the features generated by the feature extractor across a pair of shapes (See Figure 7 in the \"Author Rebuttal\" answer). In this figure, we reduced the dimension of the extracted features with PCA and then visualized them as RGB values. From this visualization, it's clear that the features from both shapes match well and emphasize the same areas. We will be happy to provide other similar illustrations in the final version. \n\n\n**Q2: Direct Latent Code Learning vs. Using an Encoder**\n\nWe appreciate the reviewer's insightful observation. Indeed, we considered the choice of directly learning the latent code in a manner akin to DeepSDF. However, our empirical results indicated a superior performance when employing the encoder architecture. Specifically, for the Faust dataset, using a latent code gave a geodesic error of 4.0 versus our encoder's 1.9. For the Scape dataset, the errors were 9.9 and 4.7 respectively.\n\nA crucial distinction between our method and DeepSDF is the context of the optimization. Our SNK optimizes over a single shape pair in an **unsupervised** manner, whereas DeepSDF is trained on a substantial dataset using a supervised paradigm. This stronger training signal in DeepSDF can effectively assist in deriving robust latent codes. Moreover, our encoder offers the benefit of harnessing the input shape as an additional source of information, further enriched by the intrinsic regularization introduced by the network architecture  (as discussed in our response to reviewer **j61z**, Q3). \n\nWe recognize the importance of this comparison and will ensure to integrate this ablation study in the revised version of the paper for clarity. \n\n\n\n**Q3: Exposition / Enhancements to the Pipeline Figure**\n\nWe thank the reviewer for finding the paper \"well organized and written\". In our revised version, we will improve the visualization by color-coding the pipeline blocks to differentiate between optimized components, like the feature extractor, shape encoder, and parts of the decoder, compared to those with explicit formulation, such as the FM solver and the rotation and translation predictions in the decoder. Furthermore, to prevent any confusion, we will explicitly indicate that our approach *does not* rely on any pre-training, including the DiffusionNet feature extractor."}, {"review_id": "zFREmNlPu7", "reviewer": "Reviewer_E6YV", "summary": "The paper proposes a novel zero-shot method for non-rigid shape matching that eliminates the need for extensive training or ground truth data. The proposed method, Shape Non-rigid Kinematics (SNK), operates on a single pair of shapes and employs a reconstruction-based strategy using an encoder-decoder architecture, which deforms the source shape to match the target shape closely. SNK demonstrates competitive results on traditional benchmarks, simplifying the shape-matching process without compromising accuracy.\n", "questions": "1. For the Shape reconstruction module, why not directly use the vertex feature to predict the deformation on the vertex rather than using the features on the faces to predict the rotation of the faces? BTW, I think the optimization of eq. 5 is not trivial and needs more time to get optimal.\n\n2. The proposed method is currently tailored towards complete shape correspondence. Have you thought of adapting it for partial shape-matching scenarios?\n\n3. In the limitations section, you mentioned exploring ways to further expedite the training process. Could you provide more details on what you mean?\n\n4. In the loss function, how do you determine the weight for each term and evaluate the importance of each term for the target? \n", "limitations": "The paper adequately addresses the limitations of the proposed method but could benefit from a detailed discussion of the potential negative societal impact. Besides, I think the running time should be evaluated. I think the optimization of Eq. 5 is too slow and takes other steps. The paper also mentions it in the last section.\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper introduces a novel method for non-rigid shape matching that does not require extensive training data, making it more practical for many applications. The proposed method also combines the benefits of axiomatic and learning-based approaches and addresses their limitations. Moreover, the paper introduces a new decoder architecture that facilitates the generation of smooth and natural-looking deformations. The proposed method is thoroughly evaluated and achieves state-of-the-art results on the SHREC dataset.\n", "weaknesses": "While the paper highlights the strengths of the proposed method, it lacks a thorough comparison with closely related work. For instance, although several learning-based methods are presented, there is no detailed comparison, such as a visual comparison and evaluation of other datasets (recommend moving some results to the main paper). Also, the limitations of the proposed method are mentioned but need to be adequately discussed.\n\nFor performance, the proposed method can not achieve the best average, especially for some unsupervised methods. For the SOTA - Deep Shells, why do the proposed methods have the similar performance on Faust and Scape datasets instead of Shrec? Could there be some analysis and discussion of the results?\n\nThe description of the Prism decoder is not clear, and I cannot find the corresponding components in Figure 3. I suggest the author re-illustrate the figure and add more details for easier understanding.\n\nSome mathematical notations are repeatable:\n> F_i and F_j in eq3, F_1 and F_2 at line 1130\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed and thoughtful comments and suggestions. We invite the reviewer to consider the general feedback that we provided to all reviewers in the \"Author Rebuttal\" answer. Below are our answers to the reviewer's questions.\n\n**Q1: Enhancing Comparisons with Related Work**\n\nWe value your feedback. A visual comparison with three leading methods is in Figure 4 of the supplementary material, and quantitative comparisons across several well-known datasets [53, 25, 62, 35, 29, 77, 46] are provided in Tables 1 and 2 of our paper. We will be happy to relocate the visual comparison to the main content and add results from any other dataset you suggest, either during discussions or in the finalized manuscript.\n\n\n\n**Q2: Performance Comparison with Unsupervised Methods**\n\nWe appreciate your observation. While our method may not top the average scores for Faust and Scape against certain unsupervised methods, these comparisons might not be apples-to-apples. The unsupervised methods train on extensive datasets for extended periods, sometimes for days [35, 36]. Our approach, in contrast, processes each shape pair without prior training, highlighting its zero-shot nature. Nevertheless, it still remains competitive with both supervised and unsupervised techniques and even achieves SOTA performance on the Shrec dataset.\n\n\n**Q3: Variability in DeepShells Performance Across Datasets**\n\nThanks for noting this. DeepShells requires extensive training to exhibit proficient performance, as seen in its results on Faust and Scape datasets. However, Shrec is smaller, with 44 shapes just for evaluation. Usually, the best model from Faust or Scape is used for Shrec evaluation, as mentioned in several studies [26,70,50] and addressed in L263 of our paper. It's also crucial to recognize that DeepShells uses SHOT descriptors, which might be affected by remeshing changes [25, 70], a feature of the Shrec dataset. In contrast, our method evaluates each shape pair independently and is unaffected by prior data or remeshing.\n\n\n**Q4: Clarifying the Prism Decoder Illustration and Notation**\n\nThanks for highlighting the ambiguities around the Prism decoder and repetitions in notation. In our revision, we will improve the Prism decoder illustration, capturing all details from Section 4.2, and include visuals for the latent code and the MLP's output. Additionally, we will address all repetitive notations for consistency across the paper.\n\n\n**Q5: Vertex vs. Face Features for Deformation Prediction**\n\nWe appreciate your question about deformation prediction. We did a test using only DiffusionNet's output for direct vertex deformations, as detailed in the Supplementary Material's Table 4 (see the `DiffusionNet` row). Our results show that using the Prism decoder, which predicts each face's rotation and translation, offers about seven times better performance.\n\n\n**Q6: Addressing Concerns on the Optimization of Equation 5**\n\nWe appreciate your feedback on Equation 5. To clarify, this equation is solved in closed form, as mentioned in L27 of the Supplementary Material. The matrix R is obtained from the SVD decomposition of a 3x3 matrix, which is very efficient.\n\n\n**Q7: Extending the Method to Handle Partial Shapes**\n\nWe value your insight on partial shape correspondence. As mentioned, our method is geared towards complete shape correspondence. We believe that adapting to partial shape-matching is possible in principle, and for this, we would consider a strategy similar to DPFM [27]. This would require using the partial functional map estimation approach and potentially predicting a pointwise mask, indicating source shape regions aligning with the target, to adjust the MSE loss. This adaptation isn't straightforward, but a promising direction for future work.\n\n\n**Q8: Strategies to Accelerate the Optimization Process**\n\nWe appreciate your query on optimization speed. As outlined in our limitations, we're actively seeking ways to expedite this:\n\n - **Convergence Speed**: Adapting our learning rate or using a more advanced gradient descent method could hasten convergence. We used the Adam optimizer, but newer techniques like Adan optimizer [79] offer quicker convergence.\n\n- **Network Implementation**: With our networks (DiffusionNet and Prism-decoder's MLP) being MLP-based, a \"fully fused\" implementation [80] could significantly boost speed.\n\nWe're optimistic that these changes can improve our optimization speed and plan to explore them in future work.\n\n\n**Q9: Determining and Evaluating the Weights of Loss Terms**\n\nThanks for your question on loss term weights. As mentioned in L34 of the supplementary materials, we've uniformly weighted all loss components across datasets. Optimizing these weights could indeed enhance results.\n\n\n**Q10: Run Time Evaluation Concerns**\n\nWe appreciate your input on run time evaluation. Please refer to Table 3 in the supplementary materials, where we've provided a detailed run-time assessment for our method.\n\n\n**Q11: Societal impact**\n\nShape matching and analysis are crucial in fields like medical imaging, archaeology, and computational biology. The proposed methodology in this paper is highly applicable across these sectors, especially where data annotation is challenging or infeasible, and training datasets are scarce. Our approach reduces the need for extensive data labeling, aiding smaller research groups and leading to cost-efficient research. Moreover, it offers increased accuracy vital in areas like biology, where intricate shape analyses inform health conditions. While our method addresses a core issue in computer graphics and vision, and we anticipate mostly positive outcomes, it's essential to note the potential misuse in areas like surveillance. We strongly discourage such applications."}, {"review_id": "mcsT1QZ78P", "reviewer": "Reviewer_j61z", "summary": "A zero shot method for computing correspondences between meshes is proposed. The core idea is to use DiffusionNet to produce features which are then used to produce a functional map from which a p2p mapping is produced. Using it, a deformation module using Primo is employed to deform the meshes to one another. This deformed shape is then used to reevaluate the functional maps and the process continues, training an autoencoder to predict the features that drive the correspondence and deformation.", "questions": "how is the use of the low eigenspectrum of the laplacian justified without assuming near-isometry?\nwhy is the autoencoder needed? why couldn't the functional maps and deformation module be optimized in block-descent manner directly?", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "Zero-shot correspondence computation is an important problem to solve in cases where no object categories are known. The combination of an extrinsic deformation module with an intrinsic functional maps approach is appealing and novel as far as I can tell. The paper is written in a clear manner for the most part (see below). ", "weaknesses": "I have several concerns:\n* In this context, \"zero shot\" essentially means \"no use of semantic information\" (as opposed to, e.g., zero shot stemming from using some existing module that has semantic knowledge and applying it to the correspondence problem without further training). Thus, the term used in the paper for \"axiomatic\" methods simply means ones using geometric priors rather than semantic ones. As such, I am uncertain why use the word \"zero shot\" at all -- this is simply an optimization algorithm for shape correspondence. As such it lies within a different research area than learning which encompasses many other works. Two recent ones are ENIGMA and Neural Surface Maps. The advantage of this approach of theirs is not immediately apparent and they should be discussed at least, if not compared to.\n* It is unclear to me what is the exact prior being enforced here. At first I thought it's a an isometric-distortion minimization prior but the authors show results on non-isometric datasets, so it is unclear to me what is the prior here and what is the justification for it. Without a justified prior nor semantics, it seems concerning the method solely relies on empirical success. It is further unclear what specific class of deformations is Primo regularizing for. It does seem to be for reducing isometric distortion here, thus I do not fully understand the claim to work on a non-isometric case. I suspect that the method is still working on rather-isometric cases - SMAL's 4-legged animals are still rather isometric. Again in this context, ENIGMA shows success and seems like a good comparison.\n* Additionally, the method is currently limited to matching meshes that are homeomorphic and are made up of one connected component - this is not mentioned in the paper and is somewhat limiting. \n* in terms of novelty, the method is simply a mix of several techniques - DiffusionNet, Functional Maps, Primo, which indeed work well together, but there does not seem to be a great novel insight beyond combining them to have a method to predict correspondences and then update a deformation prediction, which is a very classic idea (in essence, a non-rigid ICP approach where the closest point is replaced with a functional map prediction). ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed and thoughtful comments and suggestions. We invite the reviewer to consider the general feedback that we provided to all reviewers in the \"Author Rebuttal\" answer. Below are our answers to the reviewer's questions.\n\n**Q1: The Use of \"Zero-Shot\"**\n\nThank you for your remark on our terminology. We use \"Zero-Shot\" for two main reasons: (1) Instead of relying on fixed input features like HKS or SHOT, we use a deep neural network for feature computation, and (2) Our use of a neural network provides an important prior for our method, compared to purely geometric `axiomatic methods` (please see also our response to Q3 below). Nevertheless, we recognize the potential ambiguity and will articulate this more clearly in our revised manuscript.\n\n**Q2: ENIGMA & NSM**\n\nThank you for referencing ENIGMA and Neural Surface Maps (NSM). In our comparison, we focused on the most recent SOTA techniques that evaluate their results on standard dense non-rigid benchmarks, such as FAUST, SCAPE, SHREC’19, etc. Nevertheless, we agree that these are relevant prior methods and will include a discussion in the final version. Briefly,\n\n1. ENIGMA: While ENIGMA is indeed a relevant baseline, its code isn't publicly accessible, making direct comparison non-trivial. We've requested the authors for the implementation and will update accordingly.\n\n2. NSM: NSM's main objective is 3D mesh representation rather than shape matching. When applied to shape-matching, NSM necessitates input keypoints, making it semi-supervised. Furthermore, NSM's cited correspondence computation between a shape pair ranges from 6-10 hours.\n\n**Q3: Underlying Priors**\n\nWe're grateful for the insightful queries. Here's a concise overview of our method's key priors:\n\n1. Elastic Energy as a Prior: PRIMO being an elastic energy, it penalizes non-linear stretching and bending. This directs our optimization towards smooth and realistic deformations.\n\n2. Cyclic Bijectivity via the cycle consistency loss.\n\n3. Near-isometry: Our method *weakly* enforces near-isometry in fmap predictions through Laplacian commutativity in a *reduced* (k=30) functional basis.  It should be noted that using elastic energy and weakly promoting isometries with low-frequency functional maps for non-isometric matching has precedents in works like [81,82,53] and ENIGMA.\n\n4. Neural Networks: Finally, a subtle yet *essential* prior in our method is in our use of a neural network for feature extraction. Our approach follows studies like Deep Image Prior [83] and Neural Correspondence Prior [84] which show that neural networks can act as strong regularizers, generating features that enable plausible mappings even without pre-training.\n\nTo demonstrate this empirically, we conducted two additional experiments:\n\na. HKS-based Features: We replaced our feature extractor with the more traditional HKS features.\n\nb. Free Variables: We treat F1 and F2 as free variables, optimized via gradient descent *without using a neural network*.\n\nOur findings (see Table 5 in the \"Author Rebuttal\") show that without the neural network regularization, the free variable model fails to converge. The HKS-based approach performs, but significantly worse than ours. This suggests neural networks offer extra regularization, surpassing traditional handcrafted features.\n\nOverall, our method's priors are synergistic and versatile, making our approach applicable to a broad range of shapes. We'll highlight this analysis in our revised manuscript.\n\n**Q4: Novelty Concerns**\n\nThank you for feedback on our method. While building on prior work, our contributions are distinct:\n\n1. Primo Energy Adaptation: We've reimagined Primo Energy's usage. Contrary to the original paper's user-constrained approach, ours is fully automatic, in a *deep learning setting*, marking a departure from its initial intent. Moreover, our deformation model uses a face-based representation, with strong regularization, which distinguishes our method from most previous works in this domain that use direct vertex-based deformations.\n\n2. Novel approach and loss integration: (a) Our use of a neural network for feature extraction in the Zero Shot setting, as well as (b) our combination of extrinsic and intrinsic losses in a deep unsupervised setting, are both novel. Furthermore, we believe that demonstrating that the resulting approach can perform on par or even better than data-driven methods, but in a zero-shot setting, constitutes an important contribution to the domain.\n\nIn essence, our novelty stems from the innovative integration and adaptation of existing components, bringing significant progress to shape matching.\n\n**Q5: Isometry, Topology, & Multiple Components**\n\n * Isometry: The SMAL dataset is diverse, including animals like lions, cats, and deer in different poses. Our method's successful matches here demonstrate its robustness to non-isometric challenges.\n\n * Homeomorphism: Our approach is not restricted to homeomorphic shapes. The SHREC dataset, which tests among other *topological* changes, proves our method's adaptability. Please see Figure 8 for visual examples of our method from SMAL and SHREC.\n\n * Multiple Components: We recognize DiffusionNet's limitation here. Considering alternative feature extractors or viewing shapes as point clouds might help. However, this challenge isn't unique to us; other recent SOTA methods using DiffusionNet [27,83,25,62,26] share it.\n\n**Q6: Sequential Optimization**\n\nThe deformation module relies on the fmap module's outputs, so training it first isn't viable. Our ablation study (Table 4) shows an 8.7 error rate on Faust when only training the fmap, versus 1.9 with our full method. This quality wouldn't effectively train the deformation module, evidenced by a 9.2 error when we tried. The synergy between intrinsic and extrinsic modules is vital for optimal results.\n\n**Q7: Autoencoder Necessity**\n\nDue to length limits, refer to our response to Q2 from reviewer **osbV**."}, {"review_id": "lJtIjYsFAt", "reviewer": "Reviewer_nxSf", "summary": "The paper proposes a zero-shot method for non-rigid shape matching based on \nunsupervised functional map regularisation and non-rigid shape deformation regularisation.\nThe proposed method combines both intrinsic information from the functional map regularisation and\nthe extrinsic information from shape deformation regularisation (i.e. PriMo energy) leading to a better matching performance compared to methods solely based on either intrinsic or extrinsic information from the 3D shapes.\nIn the standard shape matching benchmarks, the proposed method demonstrates better matching performance \ncompared to axiomatic approaches and competitive results in comparison to both supervised and unsupervised learning-based methods. \n", "questions": "1. Even though in the limitation part, the paper claims that the proposed method is specifically tailored for complete shape matching, it is also recommended to include some (potential failure) matching results of partial shapes in the supplementary material. This will benefit follow-up works addressing partial matching.\n2. What is the number of the LBO eigenfunctions for functional map computation? Prior methods like Smooth Shells/Deep Shells optimise the matching with gradually increasing number of the LBO eigenfunctions - is the proposed method also robust to the choice of the number of LBO eigenfunctions?  ", "limitations": "1. The proposed method is specifically tailored for complete shape matching, so more adaptations should be taken for partial shape matching.\n2. The runtime of the proposed method takes 70s for shapes with ~5000 vertices, which is much slower compared to existing learning-based methods.\n3. The proposed method is tailored to shapes represented as triangle mesh, so the extension to other data representation like point cloud could be the future work.", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-written and easy to follow. Both the motivation and the proposed method are well explained in the main paper. The technical details are also well illustrated in the supplementary material.\n2. Unlike most existing shape matching methods solely based on functional map regularisation, the proposed method also considers spatial alignment and adds the shape deformation regularisation to incorporate also the extrinsic/spatial information leading to a better matching performance.\n3. Instead of using common deformation regularisation such as ARAP, Laplacian, the paper utilises the PriMo energy leading to a more smooth and natural shape deformation, as demonstrated in the supplementary material.", "weaknesses": "1. The paper claims that the method is a zero-shot method for non-rigid shape matching. But the main difference between \nthe proposed method and the previous unsupervised methods is not well discussed. In theory, all previous unsupervised\nmethods can also be used as a zero-shot method to optimise only for a single shape pair. More explanation and experiments should be provided.\n2. In the supplementary material, it shows the average runtime on the FAUST dataset is about 70s, which is much slower compared to existing learning-based methods. Prior unsupervised works (e.g. Cao et al. Unsupervised Deep Multi-Shape Matching, ECCV 2022) typically train on the training data and utilise test-time-adaptation for a few iterations to reduce runtime. More discussion and experiments about the runtime is expected to be provided.\n3. Unlike purely intrinsic/spectral methods, the proposed method also aims to spatially align two shapes. \nTherefore, more insights about the rotation robustness of the proposed method should be given. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed and thoughtful comments and suggestions. We invite the reviewer to consider the general feedback that we provided to all reviewers in the \"Author Rebuttal\" answer. Below are our answers to the reviewers’ questions. \n\n\n**Q1: Difference between the proposed method and other unsupervised methods**\n\nThank you for pointing this out. In the introduction (L29-33), we highlighted that while unsupervised techniques can work without labeled data, their best performance often requires training on numerous shape examples for extended periods [35, 36]. Such methods, once trained, can exhibit biases towards their training datasets, as seen with DeepShells' strong results on Faust and Scape but poorer performance on Shrec. Unlike them, our zero-shot approach requires no prior extensive training, allowing immediate use on new shape pairs and avoiding dataset-specific biases.\n\nOn using unsupervised methods as zero-shot techniques, we agree. Our ablation study in the supplementary material's Table 4 (specifically row 5) shows that while possible (training an unsupervised approach individually on each shape pair), these techniques don't rival the efficacy of our method, underlining our approach's uniqueness and benefits.\n\n\n**Q2: Runtime Comparison Between the Proposed Method and Some Unsupervised Methods**\n\nWe appreciate your observation. While our method may seem slower at test time compared to some unsupervised techniques, a direct comparison might be skewed. Our method is zero-shot, meaning it handles new shape pairs without prior training. A more apt comparison might be with axiomatic methods, which also work without training. This is the comparison we have shown in Table 3 of the supplementary material.\n\nOn the other hand, unsupervised methods require significant datasets and long training times for optimal performance. Comparing them to our approach without accounting for their training context might not give a full perspective.\n\n**Q3: Rotation robustness of the proposed method**\n\nThank you for highlighting this aspect. As referenced in L262, both our approach and unsupervised methods utilize datasets aligned in accordance with previous studies [29,35,62,36,53]. To delve deeper into rotation robustness, we conducted an experiment. We randomly selected 10 shape pairs from the Faust dataset and computed the point-to-point maps with our method. Concurrently, we progressively rotated one shape around the up-axis from 0 to 45°. The outcomes, depicted in Figure 9 of the \"Author Rebuttal\" answer, indicate a slight performance drop as rotation increases. It's pivotal to note that combining intrinsic and extrinsic properties is what empowers our approach to deliver robust results. We'll clarify this further in our revised manuscript.\n\n**Q4: Matching results on partial shapes**\n\nThank you for your input. In Figure 10 of the \"Author Rebuttal\" answer, we showcase our method's performance on three partial pairs from the Shrec 16' Cuts and Holes subsets. Our method performs satisfactorily when dealing with moderate partiality. \n\n\n**Q5: Robustness to the number of eigenfunctions**\n\nThanks for highlighting this. As noted in the supplementary material (L12), we use 30 eigenfunctions, aligning with many recent studies using deep functional maps, such as [25,27,26,43,29,62,75]. We also tested with 50 eigenfunctions, yielding comparable results (1.9 geodesic error on the Faust dataset). This will be clarified in the revised version.\n\n\n**Q6: The proposed method is tailored towards complete shapes**\n\nWe acknowledge your point. While our method is primarily built for complete shapes, adapting it for partial shape-matching is a viable next step. We consider employing strategies like in DPFM [27], by refining functional map estimation for partial contexts and predicting a pointwise mask to highlight aligning regions. This mask could then refine the MSE loss. However, such adaptations are non-trivial and present an exciting direction for future work.\n\n\n\n**Q7: The proposed method is tailored towards triangular meshes**\n\nThank you for noting this. Our method is primarily designed for triangular meshes. Adapting it to point clouds would entail using feature extractors like DGCNN [85] or PointMLP [86] tailored for such data. We see this as a promising direction for future exploration and plan to delve into it in subsequent research."}]

Author rebuttal:
We thank the reviewers for their constructive comments. We find the suggestions to be beneficial for improving the quality of our work, making it clearer and more convincing.

Before responding to individual concerns, we stress the following contributions of our work:

 * We have devised a new decoder architecture rooted in the PriMo energy concept [**osbV**, **nxSf**]. This architecture facilitates the production of deformations that appear smooth and natural [**osbV**, **E6YV**, **nxSf**].

 * We have shown that a loss function, which imposes penalties on both spatial and spectral quantities [**osbV**, **j61z**, **nxSf**], is adequate for deriving matches on a single pair of shapes without any prerequisite training [**E6YV**].

 * We have developed a method for zero-shot shape matching that attains state-of-the-art results among methods that operate on individual pairs [**E6YV**, **nxSf**]. Furthermore, it competes with, and frequently outperforms, several supervised training-based approaches [**osbV**, **E6YV**].


We believe that all of the suggested changes can be easily done within a minor revision and we will make sure to address all of the comments and concerns in the final version. We will also release our code and data for full reproducibility of our results and to facilitate future work in this area.

---
In response to the reviewers' queries, we've added new references. For clarity, we'll enumerate them below.

[79] Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models. X. Xingyu et. al. 2022.

[80] Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. T. Muller et. al. ToG 2022.

[81] Interactive curve-constrained functional maps. A. Gehre et. al. CGF 2018.

[82] Elastic Correspondence between Triangle Meshes. D. Ezuz et. al. CGF 2019.

[83] Deep Image Prior. D. Ulyanov et. al. CVPR 2018

[84] NCP: Neural Correspondence Prior. S. Attaiki et. al. Neurips 2022

[85] Dynamic Graph CNN for Learning on Point Clouds. Y. Wang et. al. TOG 2019

[86] Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework. X. Ma et. al. ICLR 2022

Meta-review: This paper introduces a zero-shot approach for unsupervised learning of inter-shape correspondences. The key idea is to combine two state-of-the-art correspondence paradigms, i.e., the functional map framework which is intrinsic and the Primo deformation module which is extrinsic. Experimental results show that the approach combines the strength of both modules and achieves state-of-the-art results on common benchmark datasets. The submission received all positive reviews. The hybrid approach presented in this paper can stimulate future research. Please incorporate comments from the reviewers to prepare the final version.
Decision: Accept (poster)