Non-Rigid Shape Registration via Deep Functional Maps Prior
Authors: Puhua Jiang
Mingze Sun
Ruqi Huang
Venue: NeurIPS 2023 poster (2023)
Keywords: shape registration; functional maps; unsupervised learning

Abstract: In this paper, we propose a learning-based framework for non-rigid shape registra- tion without correspondence supervision. Traditional shape registration techniques typically rely on correspondences induced by extrinsic proximity, therefore can fail in the presence of large intrinsic deformations. Spectral mapping methods overcome this challenge by embedding shapes into, geometric or learned, high- dimensional spaces, where shapes are easier to align. However, due to the dependency on abstract, non-linear embedding schemes, the latter can be vulnerable with respect to perturbed or alien input. In light of this, our framework takes the best of both worlds. Namely, we deform source mesh towards the target point cloud, guided by correspondences induced by high-dimensional embeddings learned from deep functional maps (DFM). In particular, the correspondences are dynamically updated according to the intermediate registrations and filtered by consistency prior, which prominently robustify the overall pipeline. Moreover, in order to alleviate the requirement of extrinsically aligned input, we train an orientation regressor on a set of aligned synthetic shapes independent of the training shapes for DFM. Empirical results show that, with as few as dozens of training shapes of limited variability, our pipeline achieves state-of-the-art results on several benchmarks of non-rigid point cloud matching, but also delivers high-quality correspondences between unseen challenging shape pairs that undergo both significant extrinsic and intrinsic defor- mations, in which case neither traditional registration methods nor intrinsic methods work. The code is available at https://github.com/rqhuang88/DFR.

Non-Rigid Shape Registration via Deep Functional Maps Prior
Puhua Jiang 1.2#
Mingze Sun 1#
Ruqi Huang!*
1. Tsinghua Shenzhen International Graduate School, China 2. Peng Cheng Lab, China
Abstract
In this paper, propose learning-based framework for non-rigid shape registra- tion without correspondence supervision. Traditional shape registration techniques typically rely on correspondences induced by extrinsic proximity, therefore can fail in the presence of large intrinsic deformations. Spectral mapping methods overcome this challenge by embedding shapes into, geometric learned, high- dimensional spaces, where shapes are easier to align. However, due to the depen- dency on abstract, non- non-linear embedding schemes, the latter can be vulnerable with respect to perturbed or alien input. In light of this, our framework takes the best of both worlds. Namely, we deform source mesh towards the target point cloud, guided by correspondences induced by high-dimensional embeddings learned from deep functional maps (DFM). In particular, the correspondences are dynamically updated according to the intermediate registrations and filtered by consistency prior, which prominently robustify the overall pipeline. Moreover, in order to alleviate the requirement of extrinsically aligned input, we train orientation regressor on set aligned synthetic shapes independent of the training shapes for DFM. Empirical results show that, with as few as dozens of training shapes of limited variability, our pipeline achieves state-of-the-ane results on several benchmarks of non-rigid point cloud matching. but also delivers high-quality correspondences between unseen challenging shape pairs that undergo both significant extrinsic and intrinsic defor- mations, in which neither traditional registration methods intrinsic methods work. The code is available at https: : //gt github. com/ wwwc rg/.angggunnDDRR
1 Introduction
Estimating correspondences between deformable shapes is a fundamental task in computer vision and graphics, with a significant impact on an array of applications including robotic vision [50], animation [40], 3D reconstruction (58), to name a few. In this paper, tackle the challenging task of estimating correspondences between unstructured point clouds sampled from surfaces undergoing significant non-rigid deformations.
In fact, this task has attracted increasing interest (5 26 35 S in shape matching community. These methods all follow a data-driven approach to learn embedding schemes, which project point clouds into certain high-dimensional spaces. By lifting to nigh-dimensional spaces, the non-rigid deformations are better characterized than in the ambient space, R33 The optimal transformations and therefore the correspondences are then estimated in the embedded spaces. Despite the great progresses, such approaches suffer from the following limitations: 1) Their performance unseen shapes is largely unwarranted; 2) The learned dige--imnnsional embeddings lack intuitive geometric meaning, and the induced correspondences are difficult to evaluate and analyze without ground-truie labels.
# indicates equal contribution. Corresponding author: ruqihuang @ ww tsingssssinuaaduu. edu.cc
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Rogistration
Template
NDP
AMM
Ours
Target
Shape registration methods like NDP [3 and AMM [55 estimate intermediate correspon- dences via extrinsic proximity, therefore suffering from large intrinsic deformations. In contrast, by incorporating DFM- point feature extractor, our method successfully deforms a FAUST template (the left-most mesh) to another individual of a different pose (the right-most point cloud).
Figure 1:
On the other hand, the above limitations are less of a concern from the perspective of shape registration, either axiomatic (55 or learning- based [31] Unlike the former, the latter deform source shape to non-rigidly align with a target explicitly in the ambient space R In consequence, the mapping quality can be directly related to the alignment quality. Given a deformed source shape and target, one can evaluate both qualitatively (by visual comparison) and quantitatively (by computing Chamfer distance), and, more importantly, further optimize the registration outcome via the correspondence- label-free quantitative metric. However, shape registration itself does not serve as a direct rescue to our problem of interest, as the previous approaches typically rely on the premise that the undergoing non-rigid deformation can be approximated by a set of local, mall-to-moderatte rigid deformations, which severely hinders their performance in the presence of large deformations (see Fig. I and heterogeneity.
Motivated by the above observations, we propose framework to take the best of classic shape registration and learning-based embedding techniques. In a nutshell, we leverage the estimated correspondence from the latter guide shape deformation via the former iteratively. Our key insight is to enforce similarity between the deformed source and target in both ambient space and learned high-diimessionl space. Intuitively, in the presence of large deformation, the learned correspondence is more reliable than that obtained by proximity in the ambient space. On the other hand, properly deforming source mesh to target point cloud increases spatial similarity in the ambient space, leading to higher similarity in the embedded space and therefore more accurate correspondences. In the end, we can compute correspondences between raw point clouds with a shared source mesh as a hub. As will be shown in Sec. 4 our method allows for choosing source shape during inference, which can be independent of the training data.
While being conceptually straightforward, we strive to improve the performance, robustness and practical utility of our pipeline by introducing several tailored designs. First of all, the key component of our pipeline is an embedding scheme for accurately and efficiently estimating correspondences between the deformed source shape and the target point cloud. particular, to take advantage of Deep Functional Maps (DFM) [29 7 49), the current sate-of-theear approach on matching triangular meshes, we treitrain an unsupervised DFM 49 on meshes as a teacher net and then learn point- based feature extractor (i.e., an embedding scheme for points) as a student net on the corresponding vertex sets with a natural self-supervision. Unlike the approach taken in [ which heavily relies on DiffusionNet (47) to extract intricate structural details (e.g.. Laplace- Beltrami operators) from both mesh and point cloud inputs, the teacher-student paradigm allows us to utilize a more streamlined backbone DGCNN [53] Secondly, in contrast to prior works that either implicitly RI (55) or explicitly [46: 8 24 require rigidly aligned shapes for input/iittializatin, or demand dense correspondence labels (35). We train orientation regressor on large set of synthetic, rigidly aligned shapes to automatically pre-process input point clouds in arbitrary orientations. Last but not least, we propose a dynamic correspondence updating scheme, bijectivity- correspondence filtering, two-stage registration in our pipeline. We defer the technical details to Sec.
Laplace-Beltrami
Overall, our framework enjoys the following properties: 1) Due to the hybrid nature of our framework, can handle point clouds undergoing significant deformation and/or heterogeneity; 2) As our feature extractor is self-supervised by some deep functional maps network, our framework is free of correspondence annotation throughout; 3) Our core operation performed in the ambient space, enabling more efficient, faithful, and straightforward analysis of registrraiin/mmaping results than the

---

Deep Shells
Trach
Source
Figure 2: We estimate correspondences between heterogeneous shapes from SHREC'07 with four learning-based methods, all trained on the SCAPE_r dataset. Our method outperforms the competing methods large margin. Remarkably, our method manages deform SCAPE template shape to heterogeneous shapes, as indicated by the blue shapes.
purely learning- basnid approaches; 4) Based on a data-driven orientation regressor, we achieve an automatic pipeline for estimating correspondences between deformable point clouds.
As shown Fig. trained on the SCAPE 2 dataset, our framework generalizes well and outperforms the competing methods by large margin in the challenging SHREC07-H dataset, which presents prominent variability in both extrinsic orientation and intrinsic geometry. We conduct a set of experiments to verify the effectiveness of pipeline. We highlight that it achieves state-of-thee afetth--rtt results in matching non-rigid point clouds in both near-isometric and heterogeneous shape collections. More remarkably, it generalizes well despite the distinctiveness between the training and test set. Moreover, our method, being point-based, is more robust with respect to topological perturbations within input shapes than the mesh-based baselines.
2 Related Works
Non-rigid Shape Registration: Unlike the rigid counterpart, non-rigidly aligning shapes is more
challenging because of the perplexity of deformation models. In general, axiomatic approaches M assume the deformation of interest can be approximated by local, mmall-to-moderate rigid deforma- tions, therefore suffer from large intrinsic deformations. In fact, recent advances such as 660 28 55
mainly focus improving the efficiency and robustness regarding noises and outliers. On the other hand, there exists as well a trend incorporating deep learning techniques (,1 Among them, NDP 31 is similar to our method in the sense that the non-learned version follows an optimization- based approach based on some neural encoding. However, shape registration is purely guided by proximity in the ambient space, hindering its performance in our problem of interest. Perhaps the most relevant approach with our method along this line is TransMatch [51], which follows supervised Icarning scheme and learns transformer to predict directly 3D flows between input point clouds. As demonstrated in Fig. 2 and further in Sec. as a supervised method, TransMatch generalizes poorly to unseen shapes.
Non-rigid Point Cloud Matching:
Unlike the above, several recent approaches 26 30 59 directly
establish correspondence between a pair of point clouds. In general, these methods embed point clouds into a canonical feature space, and then estimate correspondences via the respective proximity. Leapard 30 learns such feature extractor under the supervision of ground-truth correspondences between partial point clouds with potentially low overlaps. On the other hand, DPC [26] and CornetNet3D [59] leverage point cloud reconstruction proxy task to learn embeddings without correspondence label. Since intrinsic information not explicitly formulated in these methods, they suffer from significant intrinsic deformations and often generalize poorly to unseen shapes.
Deep Functional Maps: Stemming from the seminal work of functional maps (39 and a series of follow-ups [38] 22 44 37: 23), spectral methods have achieved significant progress in axiomatic non-rigid shape matching problem, and consequentially laid solid foundation for the more recent advances on Deep Functional Maps (DFM), which is pioneered by Litany et al. 32] Unlike axiomatic functional maps frameworks that aim at optimizing for correspondences represented in spectral basis (so-called functional maps) with hand- cand-crafted features prior. DFM takes an inverse
3

---

Stage
Stage 1
A
IIsT
Fs
VS
V'S
A
FT
T
Ground truth
Vr
Orientation
Point Feature Extractor
High-dimensiona embedding
Coordinates feature
Regressor
F
Permutation matrix
The schematic illustration of pipeline. A is pre-trained orientation regressor for aligning imput shapes. Then a pre-trained feature extractor embeds them into a high-dimensional canonical space. During the iterative optimization procedure of registration, correspondences dynamically updated according to learned features (Stage-I) and coordinates (Stage-II) of the inter- mediate shapes. See more details in the text.
Figure 3:
viewpoint and aims to search for the optimal features, such that the induced functional maps satisfy certain structural priors well possible. Recent findings along this direction empirically suggest that the structural priors suffice to train an effective feature extractor, without any correspondence However, because of the heavy dependence of eigenbasis of Laplace- Beltrami operators, DFM is primarily designed for shapes represented by triangle meshes and can suffer notable performance drop when applied to point clouds without adaptation [
supervision 12 29
7 3
In fact, inspired by the success of DFM, several approaches [24] S have been proposed to leverage intrinsic geometry information carried by meshes in the training of feature extractors tailored for non- structural point clouds. Among these, NIE (24] and SSMSM [8] are closely related to our approach. per-point feature extractor by leveraging meshes during training. However, unlike the former two, our method incorporates optimization-bant based shape registration techniques, which effectively alleviates the limitation of purely relying on an abstract high-dimensional embedding scheme (i.e., the learned feature extractor).
As all three works attempt to train effective, intrinsic- geometry-
Unlike the above, our method operates in both extrinsic and intrinsic spaces. From this point of view, the most relevant work Smooth Shells 14, which proposes to perform shape registration under a mixed representation of shapes, including cigenbasis, coordinates, and normal. Though achieving considerable high-quality correspondence on standard benchmarks, it heavily relies on the availability of mesh representation, but also is sensitive to the original extrinsic alignment of input shapes. These limitations are inherited by Deep Shells [16] the learning version of the former. Similarly, NeuralMorph [[55] jointly performs interpolation and correspondence estimation between a pair of shapes, which inject the intrinsic information by imposing geodesic preservation loss. As result, it is non-trivial lift the requirement of mesh input.
Combination of Extrinsic and Intrinsic Methods:
Methodology
Given pair of shapes S, T. our target is to deform S non-rigidly align with T. We assume that represented as a triangle mesh so that we can effectively regularize the deformed shape by preserving local intrinsic geometry. On the other hand, we require structural information on T and generally assume to point cloud. Our pipeline can also be extended to compute correspondences between two raw point clouds Tı, Tz. To this end, we fix template mesh S, perform respective shape registration between S and the target point clouds, and finally compute the map by composition Ti2 = T.2 Tis-
Our pipeline is shown in Fig. which consists of three main components: 1) orientation regressor, A, for extrinsically aligning input shapes, either mesh point cloud; 2) A point feature extractor, F,

---

trained under deep functional maps scheme; 3) A registration module that iteratively optimizes for deformations non-rigidly aligning S with In particular, takes the rigidly aligned shapes from as input, and leverages 2) to update correspondences during optimization.
Though our pipeline leverages pre-trained feature extractor as registration prior, highlight that neither the source nor the target is necessarily within or close to the respective training set. We provide throughout experimental evidence in Sec. 4 showing the generalizability of our pipeline.
3.1 Orientation Regressor
Our first objective is to align input point clouds with arbitrary orientations into a canonical frame. We take data-driven approach by training an orientation regressor [9] on 5000 synthetic SURREAL shapes from [52], which are implicitly aligned to canonical frame by the corresponding generative codes. We refer readers to Supplementary Materials for more details.
3.2 Point Feature Extractor
In order to efficiently and accurately estimate correspondences between deformed source shape and the target point cloud during registration, our next goal is to train a feature extractor for point clouds, which is intrinsic-geonee geometry aware. The authors of [8] propose a multi-modal feature extractor based on DiffusionNet [47]. which can process point clouds with an extra step of graph Laplacian construction [48.. Though it has demonstrated satisfying performance accommodating point cloud representation in (8) the explicit graph Laplacian construction computationally heavy. Hence, we adopt the modified DGCNN proposed in [24] as our backbone, which is lightweight and robust regarding the sampling density and sizes of point clouds.
In particular, our training follows teacher-student paradigm. Namely, we first train deep functional maps (DFM) network, G, on a collection of meshes. Then we train DFM network on the corresponding vertex sets, with an extra self-supervision according to the inclusion map between meshes and their vertex sets. other words, the feature produced by F is point-wisely aligned with that produced by G.
Training DFM Meshes: Our training scheme in principle follows that of 49 which delivers both accurate map estimation and excellent generalization performance.
1) We take pair of shapes S₁, S2, and compute the leading l eigenfunctions of the Laplace-Beltrami operator on each shape. The cigenfunctions are stored as matrices 2 E RR x i = 1,2.
2) We compute G GSS)). G2 G(S2), projected into the spaces spanned by 4,, 2 respectively, and leverage the differential FMReg module proposed in [3] to estimate the functional maps C12, C21 between the two shapes, then by regularizing the structure of C12, C21, say:
bbjj (G) = 121 T|E Frttho (G) 1|C12C12 7|| 11 7||-
(1)
3) To enhance G, we estimate the correspondences between S₁, S2 via the proximity among the rows of G1, G2 respectively. Namely, compute
exp(-adij) vi [[]]].j [[S2]].
(1i(( ΣS exp(-ad;x)
where d; = (i.: :2(i.)) and a is the temperature parameter, which is increased during training 49 Then 11 the soft map between the two shapes. Ideally, it should be consistent with the functional maps estimated in Eqn. giving rise to the following loss:
E aalign GG ||C12 21 Illl + |C21
2 111222l
where denotes to the pseudo-inverse To sum up, G arg min (G), which is defined as:
(G) Ebj (G) Aorth Forth (G) + FaliinnFalign
(4)
Training DFM on Point Clouds: Now we are ready for training F with a modified DGCNN 24 as backbone. We let Fi = F(SI), F2 F(S2). On top of EDFM in Eqn. 4). we further enforce the
5

---

extracted feature from F to be aligned with that from G" via PointInfoNCE loss [54]:
R2
(P((F
log
log
GTG.)))
exp((Fa(t.)) GU.)))
where (.) refers to the dot products, 7 is the temperature parameter, and is the number of points of
S, So the final training loss defined as:
E(F) (FF + ANCEEECEE(E, (F, G
(6)
which is optimized on all pairs of shapes within the training set.
3.3 Shape Registration
Now ready to formulate our registration module. We first describe the deformation graph construction our pipeline. Then we formulate the involved energy functions and finally describe the iterative optimization algorithm. During registration,
denote the deformed source model by S {V* V7 = (vf i I,.... N). where indicates the iteration index and of is the position of the i-th vertex at iteration k. The target point cloud is denoted by T {u, | = 1,...,,AA M).
Deformation Graph: Following 201 we reduce the vertex number on S to H = [N/2] with QSlim algorithm [17]. Then an embedded deformation graph DG is parameterized with axis angles and translations A E # X {), The vertex displacements are then computed
θ E IR xx from the corresponding deformation nodes. For given X we can compute displaced vertices via:
Dk+ D V*).
Dynamic correspondence update: Thanks to our point-based feature extractor F, we can freely update correspondences between deforming source mesh and target point clouds. In practice, for the sake of efficiency, we update every 100 iterations in optimization (see Alg.
Correspondence Filtering via Bijectivity: One key step in shape registration is to update cor- respondences between the deforming source and the target. It is then critical to reject erroneous corresponding points to prevent error accumulation over iterations. In particular, we propose a filter based on the well-known bijectivity prior. Given S and T, we first compute maps represented as permutation matrices IlsT, IITs. either based on the learned feature or the coordinates. For vf E S we compute the geodesic on S between of and its image under permutation IIsTlITs. then reject all of if the distance is above some threshold. For the sake of efficiency, we pre-compute the geodesic matrix of S and approximate that of & with it. Finally, we denote the filtered set of correspondences by C* {(u,, un ), (v uyz)....
In the following, we introduce the energy terms regarding our registration pipeline.
Correspondence Term measures the distance of filtered correspondences between S and T, given
as:
Ecorr
|
U
(8)
C
(ul Jeck
Chamfer Distance Term has been widely used
31
30
to measure the extrinsic distance between
Sk and T:
Eed
min min JE[M]
||
-
Σ min
<E[N] ||o
N
M
KE[N
NEIN
jE[M]
serigid-as--ossiibee Term reflects the deviation of estimated local surface deformations from rigid transformations. We follow (20 [27 and define it as:
Farap =
Σ
Σ
(14,,(()1 (((R(O))) - R (O)))
(10)
HE H] (EW())

---

Algorithm 1: Shape registration pipeline.
Input: Source mesh (V. 8 and target point cloud T Trained point feature extractor F Output: X converging to a local minimum of Fiotal: Deformed source model [V*,E};
Correspondence y Ts between S and
Initialize deformation graph DG and X with identity transformations: FT F(T);k 0:
2 while True do
Update source vertex (A) by Eqn. if 1%100 = 0 and Flag- Stage-1 then
L FF = F(V(K)); ); ss = NN $55 FT): 1 (F
k%100 0 and Flag Stage-ll then ST = NN(((6) T) 11 - NN(T, v(k)); Compute the set of filtered correspondences C*: X(++1) arg min by Bq
if converged and Flag == Stage-I then Flag Stage-Il; converged and Flag == Stage-Il then return 1sssill V";
11
k=k+1;
- = dn, = (#x) -n) + A, + 9h. - (9) + A1).
(11)
Here, y E R are the original positions of the nodes in the deformation graph DG, and (h) is the 1-ring neighborhood of the h-th deformation node. R(-) is Rodrigues' rotation formula that computes a rotation matrix from an axis-angle representation and a is the weight the smooth rotation regularization term.
Total cost function:
The total cost function Eioul combines the above terms with the weighting factors Xed, Acorr, and Xarap to balance them:
Etotal >ed Acor Ecorr + Aarap
(12)
Now we are ready to deseribe our algorithm for minimizing Fiotal. which shown in Alg.
Two-stage Registration: Finally, we observe that solcly depending on learned features to infer correspondence is sub-optimal. At the converging point, the deformed source shape is often the right pose but has deficiencies in shape style. To this end, we perform a second-stage registration, based the coordinates of deforming source and target. As shown in Sec. 4.2 such a design is clearly beneficial.
4 Experiments
We evaluate our method and several state-of-the-art techniques for estimating correspon- dences between deformable shapes
Datasets:
array of benchmarks as follows: FAUST r: The remeshed version of FAUST dataset [4), which consists of 100 human shapes (10 individuals performing the same 10 actions). We split the first 80 as training shapes and the rest as testing shapes; SCAPE r:: The remeshed version SCAPE dataset (2). which consists human shapes (same individual in varying poses). We split the first 51 as training shapes and the rest as testing shapes; SHREC19 The remehsed version of SHREC19 dataset [36), which consists 44 shapes different identities and poses. We use it solely in test, and follow the test pair list provided by [36]; SHREC07-H: A subset of SHREC07 dataset [8] which consists of 20 heterogencous human shapes the varying number of vertices. We use it solely in test, and use the accompanied sparse landmark annotations to quantify all the pairwise maps among them; DT4D-H: of 10 categories of heterogeneous humanoid shapes. We use it solely in testing, and evaluating the inter-class maps split [29]: TOPKIDS: This is challenging dataset [25] consisting of shapes of kid in different poses, which manifest significant topological perturbations in meshes.
dataset proposed in [34 which consists
Baselines: We compare our method with an array of competitive baselines, including axiomatic shape registration methods: Smooth Shells [14, NDP [31], AMM [55); learning- based registration

---

methods: 3D-CODED 19). Deep Shells [[]]. TransMatch [ST]. SyNoRiM [2]]: deep functional maps
frameworks: SURFMNet 45 WSupFMNNet 46], GeomFMaps [T3, DiffFMaps [35, NIE [24]. ConsistFMaps 77. SSMSM g
According to input requirements, we put those relying on pure mesh input on the top, and the rest, suitable for point clouds, in the bottom of both tables.
Train/Test Cases: Throughout this section, we consider a highly challenging scenario: for each learning-based method, we train two models respectively with FAUST_ r and SCAPE_ r dataset, and then we run tests in a range of test cases including FAUST_r, SCAPE r, SHREC19_, SHREC07-H, DT4D-H and TOPKIDS. The test pairs of each case have been described above. In the following. A/B means we train on dataset A and test on dataset B.
Datasets Alignment:
There exist extrinsically aligned versions for all the involved datasets but SHREC07-H. We equally feed in aligned datasets to all the baselines when available. On the other hand, the original version of SHREC07-H manifests significant variability of orientations. For the sake of fairness and simplicity, we apply our orientation regressor to it and provide baselines for our automatically aligned data. Note that, all the aligned datasets, as well as the synthetic dataset on which we train our orientation regressor, roughly share the same canonical orientation, which is defined by the SMPL [3] model. Finally, we always apply automatic alignment before inputting shapes into our pipeline, whatever the original orientation they are.
Choice of Source Shape: As mentioned at the beginning of Sec. 3 we compute correspondences between two raw input point clouds by associating them via a common source mesh S. In Tab. [and Tab. 2 we indicate the source mesh. For simplicity, we fix the source mesh from each dataset and visualize them in the appendix. On the other hand, when implementing axiomatic shape registration methods, we only consider deforming the same template we pick from FAUST_ (resp. SCAPE_r) to every test point cloud in the test set of FAUST_ (resp. SCAPE_r), and establish correspondence by map composition, in the same manner as ours.
choice of source mesh by Ours-name, where name indicates the origin of
Though we primarily focus on matching point clouds, we adopt the commonly used geodesic error normalized by the square root of the total area of the mesh, to evaluate all methods, either for mesh point cloud.
Metric:
Papaee-arammtter:: We remark that all the hyper-parameters are fixed for all experiments this section. In particular, we settle them by performing grid search with respect to the weights used in the final optimization (quantitatively in terms of Chamfer distance and qualitatively by visual inspection) few training shapes. We provide more detailed discussion and ablation of the choice hyper- in the appendix.
seek for the combination that leads to the the best registration results
4.1 Experimental Results
Near-isometric Benchmarks: As shown in Fig. [ in the presence of large deformation between the template and the target, NDP[]]] and AMM[55 fail completely while our method delivers high- quality deformation. Moreover, as illustrated in Table I our method achieves the best performance in 5 out of 6 test cases. Remarkably, in the two most challenging tasks, CAUST_//HRECII _r and FAUST r/SHREC19 _r, our method indeed outperforms all of the baselines, including the of the-art methods that take meshes input. Regarding posnt-based methods, SSMSM [8] performs well in the standard case and outperforms ours in AASUT_r/FAUST_r but generalizes poorly to unseen shapes. Another important observation is that our method manifests robustness with respect the choice of template shapes. In fact, the above observations remain true no matter which template we select.
Non-isometric Benchmarks: We stress test our method on challenging non-isometric datasets including SHREC07-H H and DT4D-H. We emphasize that these test shapes are unseen during training.
1) SHREC07-H contains 20 heterogeneous human shapes, whose number of vertices ranges from 3000 to 15000. Moreover, there exists some topological noise (e.g., the hands of the rightmost shape Fig. 2 is attached to the thigh in mesh representation). As shown in Fig. 2 SSMSM barely delivers reasonable results, which might be due to the sensitivity of graph Laplacian construction on point clouds. Topological noise, on the other hand, degrades mesh-based methods like Deep Shells [16]. Meanwhile, as shown in Tab. 2 our method achieves performance improvement of
8

---

Table 1: Mean geodesic (x100) FAUST_ SCAPE_ and SHREC19_r. The best is
highlighted.
Train Test
FAUST_ SCAPE_
SCAPE FAUST_
Method
FAUST_
SHREC19_
SCAPE_
SHREC19_
Smooth Shells
2.5
4.7
SURFMINe(U)
+
15.0
32.0
12.0
32.0
NeuraMorph((U)
15
8.5
29.0
30.0
18.0
WSupFMNe((W)
3.3
7.3
G
13
mesh
3.1
11.0
9.6
4.4
6.0
11.4
Deep Shells(W)
[6
1.7
26.6
2.5
2.7
21.4
ConsisFFMapo(()
AtuntiveFFapss()
1,9
2,6
6.2
2.2
2.2
9,3
NDF 31
20.4
16.2
AMM
14.2
13.1
(U)Net3D(()
63.0
58.0
3D CODED(S) 19
2,5
31.0
31.0
33,0
SyNoRiM(S)
7.9
21.9
9.5
24.6
TransMatch(S)
2.7
33.6
18.6
38.8
DPC(S) 26
11.1
17.5
31.0
17.3
11.2
28.7
DillPMaps(S)
35
3.6
19.0
16.4
12.0
17.6
NIE(W)
5.5
11.0
SSMSM(W)
2.4
11.0
9.0
4.1
8.5
7.3
SCAPE
3.4
2.6
5.1
Outs-FAUSST
3.0
29
Table 2: Mean geodesic errors (x100) on SHREC'07 and DT4D-H. The best is highlighted.
Train Test
F
SCAPE r
Method
SHRECO7-H(")
DT4D-H
SHREC07-H(*
DT4D-H
GeomFMaps 13
30.5
38.5
28.9
28.6
Deep Shells 16
30.6
35.9
31.3
25.8
CansistFMaps
mesh
36.2
33.5
37.3
38.6
AttentiveFMaps
29
16.4
11.0
21.1
21.4
TransMatch 51
25.3
26.7
31.2
25.3
DiffFMaps
16.8
18.5
15.4
15.9
NIE [24]
15.3
13.3
13.4
12.1
SSMSM 8
ped
42.2
11.8
37.7
8.0
SCrs-SCAPE
9.3
9.8
5,9
5.7
Ours-FAUST
8.5
6.1
Ours Curs-CRYPTO
6.9
5.7
over 40% compared the previous SOTA approaches (8.5 vs. 15.3; 5.9 vs. 13.4), which again confirms the robustness of our approach. 2) Regarding DT4D-H, We follow the test setting of AttentiveFMaps (29), and only consider the more challenging inter-class mapping. Interestingly, our method outperforms the state-of-the-at point cloud-based approach by approximately 30% and exceeds the performance of baseh state-of-the-arr methods by over 70%. Furthermore, even in the worst case, training on FAUST and using SCAPE template, our error is still the lowest compared to external baselines.
Topologically Perturbed Benchmark: The shapes in TOPKIDS present various adhesions hand/arm adhered abdomen/thigh (see Fig. in the appendix). Given the ground-truth maps from each topologically perturbed shape to the reference shape, we evaluate all 25 maps (excluding the trivial one from reference itself) with models trained on FAUST and SCAPE r respectively. Since the orientation of the TOPKIDS dataset is not originally agrecable with the aligned version of the training sets, we align the input with our orientation regressor and feed them into the Regarding the template shape, we adopt the reference shape as our source mesh in registration. We report the quantitative comparisons in Tab. 3 It is obvious that topological noise poses great challenges for methods based on pure intrinsic information
baselines depending on extrinsic information [6 B5
24, g
[13 29], while the counterparts perform relatively well. Especially, among the point-based methods, our method outperforms the latest SOTA method by a large margin, namely, over 40% relative error reduction (7.1 vs. 12.3). We refer readers to the appendix for a qualitative comparison, which agrees with the quantitative results above.

---

Table 3: Mean geodesic errors (x100) TOPKIDS which trained on FAUST and SCAPE The best is highlighted.
Moom
Deep Shells
[66]
MonsistMMpps
AteutivcFMaps 29
DiriMMaps [1
NEE 24
SSMSM g
Ous
FAUST ATOOKIDS
35.9
20.5
189
14.2
SCAPE_ATOPKIDS
21.7
15,3
33.1
39.4
18.0
162
123
7.
Table 4: Mean geodesic errors (x100) on different ablated settings, the models are all trained on SCAPE and test on SHREC'07. w/o Registration w/o Stage
w/o Stage II
w/o updating corre.
w/o cons. filter
Full
11.5
10.6
10.1
8.1
7.2
5,9
Table 5: Mean geodesic errors (x100) Ours, NDP, AMM based on the same initial maps.
SCAPE
SHREC19.
SHREC07-H
Ini. NDP
5.5
8.1
11.5
5,4
11.4
8.9
AMM
11.4
10.7
8.8
5.1
5.9
4.2 Ablation Study
We report ablation studies in Tab. in which we verify the effectiveness of each core design formulated in Sec. 3 Throughout this part, we train on SCAPE_r and test on SHREC07-H. In the registration stage, we use the SCAPE_r template as the source mesh and deform it each point cloud from SHREC07-H. It is evident that each ablated module contributes to the final performance. In particular, both stages of registration play an equally important role in our framework.
Finally, to validate the superiority of our registration scheme, we report in Tab. 5 the average errors of the initial maps computed by our point-based DFM, and that of output maps of Ours, NDP (I], AMM [55], which are all based on the former. It is evident that, across three different test sets. our method consistently improves the initial maps, while NDP and AMM can even lead to deteriorated maps than the initial input.
5 Conclusion and Limitation
In this paper, we propose a novel learning-based based shape registration framework without correspondence supervision. Our framework incorporates learning- based shape matching and optimization- shape registration. Based on the former, our framework can perform registration between shapes undergoing significant intrinsic deformations; On the other hand, thanks to the latter, our method exhibit superior generalizability over the learning- bbaeid competitors. Apart from several designs tailored for our intuitive pipeline, we also introduce a data-driven solution to facilitate the burden of extrinsically aligning non-rigid points. We verify our framework through a series of challenging tests, in which it shows superior performance but also remarkable robustness.
Limitation Future Work We identify two main limitations of our method: 1) Our optimization procedure involves iteratively updating correspondences between the intermediate point clouds and the target one, leaving sufficient room for improvement for efficiency; 2) Our pipeline is designed for full shapes, while it is applicable to partial targets, the registration quality is sub-optimal. In the future, we also plan to incorporate more existing techniques (e.g., various regularization terms) from shape registration to further enhance the overall performance.
Acknowledgement The authors thank anonymous reviewers for their constructive comments and suggestions. This work was supported in part by the National Natural Science Foundation of China under contract No. 62171256. 62331006. in part by Shenzhen Key Laboratory of next-generation interactive media innovative technology (No. ZDSYSS20200023392000000
References
[1] Brian Amberg, Sami Romdhani, and Thomas Vetter. Optimal step nonrigid icp algorithms for surface registration. 2007.
10

---

[2] Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis.
SCAPE: Shape Completion and Animation of People. 2005.
[3] Souhaib Attaiki and Maks Ovsjanikov. Understanding and improving features learned in deep functional maps. In CVPR, 2023.
[4] Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. FAUST: Dataset and evaluation for
3D mesh registration. In CVPR, 2014,
[5] Aljaz Bozic, Pablo Palafox, Michacl Zollhöfer, Angela Dai, Justus Thies, and Matthias Nießner. Neural non-rigid tracking. In NewrlPS, volume 33, pages 18727-18737 2020.
[6] Aljaz Bozic, Michael Zollhofer. Christian Theobalt, and Matthias Nicßner. Deepdeform: Learning non-rigid rgb-d reconstruction with semi-supervised data. In CVPR, pages
2020. [7] Dongliang Can and Florian Bemard. Unsupervised deep multi-shape matching. In ECCV, 2022.
[8] Dongliang Cao and Florian Bernard. Self-supervised learning for multimodal non-rigid shape matching.
In CVPR. 2023.
[9] Jiayi Chen, Yingda Yin, Tolga Birdal, Baoquan Chen, Leonidas Guibas, and He Wang. Projective manifold gradient layer for deep rotation regression. In CVPR, 2022.
[10] Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas Guibas. Vector neurons: general framework for so(3) equivvaraa networks. In ICCV, 2021.
[11] Jiacheng Deng, Chuxin Wang, Jiahao Lu, Jianfeng Hc, Tianzhu Zhang, Jiyang Yu, and Zhe Zhang. Sc-ornet: Self-ensemblng orientation-awarr network for unsupervised point cloud shape correspondence. arXiv
preprini arXiv:2304, 05395, 2023.
[12] Nicolas Donati, Etienne Corman, and Maks Ovsjanikov, Deep orientation aware functional maps: Tackling symmetry issues in shape matching. In CVPR, pages 742-751 2022.
[13] Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. Deep geometric functional maps: Robust feature learning for shape correspondence.
CVPR, 2020.
[14] Marvin Eisenberger, Zorah Lahner, and Daniel Cremers. Smooth shells: Multi-scalc shape registration
with functional maps. In CVPR, 2020.
[15] Marvin Eisenberger, David Novotny, Gael Kerchenbaum, Patrick Labatut, Natalia Neverova, Daniel Cremers, and Andrea Vedaldi. Neuromorph: Unsupervised shape interpolation and correspondence in one
go. In CVPR, 2021.
[16] Marvin Fisenherger, Aysim Toker, Laura Leal-Taixé, and Daniel Cremers. Deep shells: Unsupervised
shape correspondence with optimal transport. In NewrIPS, 2020.
[17] Michael Garland and Paul S Heckbert. Surface simplification using quadric error metrics. In Proceedings Computer graphics and interactive techniques, pages 209-216, 1997. [18] Daniela Giorgi, Silvia Biasotti, and Laura Paraboschi. Shape retrieval contest 2007: Watertight models
of the 24th annual conference
track. SHREC competition, 2007.
[19] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 3d-coded: 3d correspondences by deep dcformation. In ECCV, 2018.
[20] Chen Guo. Xu Chen, Jie Song, and Olmar Hilliges. Human performance capture from monocular video in the wild. In 3DV, 2021.
[21] Jiahui Huang. Tolga Birdal, Zan Gojeie, Leonidas J. Guibas, and Shi-Min Hu. Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization. JEEE Transactions on Pattern Analysis
and Machine Intelligence, pages 1-1, 2022.
[22] Ruqi Huang and Maks Ovsjanikov. Adjoint map representation for shape analysis and matching. In Computer Graphics Forum, volume 36, pages 151-163. Wiley Online Library, 2017.
[23] Ruqi Huang, Jing Ren, Peter Wonka, and Maks Ovsjanikov. Consistent zoomout: Efficient spectral map synchronization. In Computer Graphics Forum, volume 39, pages 265-278. Wiley Online Library, 2020.
[24] Puhua Jiang. Mingze Sun, and Ruqi Huang. Neural intrinsic embedding for non-rigit point matching. In CVPR, 2023.
[25] Zorah Lähner, Emanuele Rodolà, Michael M Bronstein, Daniel Cremers, Oliver Burghard, Luca Cosmo, Andreas Dieckmann, Reinhard Klein, and Yusuf Sahillioglu. Shree' 16: Matching of deformable shapes with topological noise. Proc. 3DOR, 2(10.2312), 2016.
[26] Itai Lang, Dvir Ginzburg. Shai Avidan, and Dan Raviv. Dpc: Unsupervised deep point correspondence via cross and self construction. In 2021 International Conference on 3D Vision (3DV), pages -4511.
IEFF, 2021.
[27] Zohar Levi and Craig Gotsman. Smooth rotation enhanced wwwiiidda--ppssiuie mesh animation. IEEE
transactions
visualization and computer graphics. 21(2):264-277 2014.
[28] Kun Li, Jingyu Yang, Yu-Kun Lai, and Daoliang Guo. Robust non-rigid registration with reweighted posi- tion and transformatin sparsity. IEEE transactions visualization and computer graphics, 25(6):2255- 2269, 2018
[29] Lei Li. Nicolas Donati, and Maks Ovsjanikov. Learning multi-resolution functional maps with spectral attention for robust shape matching. In NewrIPS, 2022.
[30] Yang Li and Tatsuya Harada. Lepard: Learning partial point cloud matching in rigid and deformable scenes. CVPR, 2022.
[31] Yang Li and Tatsuya Harada. Non-rigid point cloud registration with neural deformation pyramid. In NewrIPS, 2022.
[32] Or Litany, Tal Remez, Emanuele Rodolà, Alexander M. Bronstein, and Michael M. Bronstein. Deep functional maps: Structured prediction for dense shape correspondence. In ICCV, 2017.
11

---

[33] Matthew Loper, Naureen Mahmood, Javier Romero. Gerard Pons-Moll, and Michael J. Black. SMPL: A
skinned multi-person linear model. TOG, 34(6):248:1 1-24:::1128:16666 2015.
[34]
Robin Magnet, Jing Ren, Olga Sorkine-Homunng, and Maks Ovsjanikov. Smooth non-rigid shape matching
via effective dirichlet energy optimization. In 3DV, 2022.
[35] Riccardo Marin, Marie-Julie Rakotosaona. Simone Melzi, and Maks Ovsjanikov. Correspondence learning
via linearly-invariant invariant embedding. Advances in Neural Information Processing Systems, 33:1608-16200 2020.
[36] S. Melzi, R. Marin, E. Rodolå, U. Castellani, J. Ren, A. Poulenard, P. Wonka, and M. Ovsjanikov. Matching Humans with Different Connectivity, In Eurogruphics Workshop on 3D Object Reirieval, 2019.
[37] Simone Melzi, Jing Ren, Emanuele Rodolå, Peter Wonka, and Maks Ovsjanikov. Zoomout: Spectral
upsampling for efficient shape correspondence. Proc. SIGGRAPH Asia, 2019.
[38] Dorian Nogneng and Maks Ovsjanikov. Informative descriptor preservation via commutativity for shape matching. Computer Graphics Forum. 36(2):259-22 267. 2017.
[39]
Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. Functional Maps: A Flexible Representation of Maps Between Shapes. ACM Transactions on Graphics (TOG),
34))330, 2012.
[40] Gianluca Paravati, Fabrizio Lamberti, Valentina Gatteschi, Claudio Demartini, and Paolo Montuschi. Point
cloud-based automatic assessment of 3d computer animation courseworks. IEEE Transactions on Learning
Technologies. 10(4):532-54) 54:53333,33,222 2016.
[41]
Adrien Poulenard, Maric-Julic Rakotosaona, Yann Ponty, and Maks Ovsjanikov. Effective rotation-invarian point cnn with spherical harmonics kemels.. In 3DV, 2019.
[42] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d
classification and segmentation. In CVPR, 2017.
[43] Charles R. Qi, Li Yi, Hao Su, and Lconidas J. Guibas. Pointnct++:: Deep hierarchical feature learning on
point sets in metric space. In NeurlPS, 2017.
[44] Jing Ren, Adrien Poulenard. Peter Wonka, and Maks Ovsjanikov. Continuous and orientation- prreeervnnnneeeiingggg. correspondences via functional maps. ACM Trans. Graph., 24::):24: 16, Dec. 2018.
[45] Jean-Michel Roufosse, Abhishek Sharma, and Maks Ovsjanikov. Unsupervised deep learning for structured
shape matching. In ICCV, 2019.
[46] Abhishek Sharma and Maks Ovsjanikov. Weakly supervised deep functional maps for shape matching. In
NeurlPS, 2020.
[47] Nicholas Sharp. Souhaib Attaiki, Keenan Crane, and Maks Ovsjanikov. Diffusionnet: Discretization
agnostic learning on surfaces. ACM Transactions on Gruphicx. 2022.
[48] Nicholas Sharp and Keenan Crane. A laplacian for nonmanifold triangle meshes. Computer Graphics
Forum, 2020.
[49] Mingze Sun, Shiwei Mao, Puhua Jiang, Maks Ovsjanikov, and Ruqi Huang. Spatially and spectrally
consistent deep functional maps. In JCCV, 2023.
[50] Carlos Sánchez- Belenguer Simone Ceriani, Picrluigi Taddei, Erik Wolfart, and Vitor Sequeira. Global matching of point clouds for scan registration and loop detection. Robotics and Autonomous Systems, 123:103324, 2020.
[51] Giovanni Trappolini, Luca Cosmo, Luca Moschella, Riccardo Marin, Simone Melzi, and Emanuele Rodolà. Shape registration in the time of transformers. Advances in Neural Information Processing Systems,
34:5731-5744, 2021.
[52] Güil Varol, Javier Romero, Xavier Martin, Naurcen Mahmood, Michael J. Black, Ivan Laptev, and Cordelia
Schmid. Learning from synthetic humans. In CVPR, 2017.
[53] Yue Wang. Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph can for learning on point clouds. ACM Transactions on Graphics (TOG), 2019. [54] Saining Xie, Jiatao Gu, Demi Guo, Charles R Qi, Leonidas Guibas, and Or Litany. Pointcontrast:
Unsupervised pre-training for 3d point cloud understanding. In ECCV, 2020.
[55] Yuxin Yao, Bailin Deng, Weiwei Xu, and Juyong Zhang. Fast and robust non-rigid registration using accelerated
cewpiizatioommmimizztiiio IEEE Transactions on Pattern Analysis and Machine Intelligence,
2023.
[56] Hao Yu, Ji Hou, Zheng Qin, Mahdi Saleh, Ivan Shugurow, Kai Wang. Benjamin Busam, and Slobodan Ilie. Riga: Rotation-in invariant and globally aware descriptors for point cloud registration. arXiv preprint arXiv:2209. 13252, 2022.
[57] Hao Yu, Zheng Qin, Ji Hou, Mahdi Salch, Dongsheng Li, Benjamin Busam, and Slohodan Tlie. Rotation- invariant transformer for point cloud matching. arXiv preprint arXiv:2303.082:: 2023.
[58] Tao Yu, Zerong Zheng, Kaiwen Guo, Jianhui Zhao, Qionghai Dai, Hao Li, Gerard Pons-Moll, and Yebin Liu. Doublefusion: Real-time capture of human performances with inner body shapes from a single depth sensor. In CVPR, 2018
[59] Yiming Zeng, Yue Qian, Zhiyu Zhu, Junhui Hou, Hui Yuan, and Ying He. Cormet3d: Unsupervised end-to-end learning of dense correspondence for 3d point clouds. In CVPR, pages 6052-6061, 2021. [60] Qian- Yi Zhou, Jacsik Park, and Vladlen Koltun, Fast global registration. ECCV, 2016.
12

---

In this appendix, we provide more technical details and experimental results. In Sec. A we provide
1) A detailed description of our orientation regressor in Sec. A.I 2) More details on the registration method presented in Alg. 1 of the main text in Sec. A.3 3) Implementation details of the whole pipeline in Sec. A.44
In the second part, we provide more experimental results. We first provide more qualitative results in Sec. B.I] as well quantitative results on training on large dataset and on animal dataset in Sec. B.2 Then we demonstrate the robustness of our method with respect to several perturbations in Sec. B.3
In Sec. B.4 we showcase the scalability of our method in matching real scans. Finally, we report run-time decomposition and analysis in Sec. B.5
Note that also include short video showcasing the registration process of our pipeline in the Supplemental Material.
A Technical Details
A.1 Orientation Regressor
Orientation plays an important role in point cloud processing. Classic point- boint- feature extrac- tors 42 43 [53] are not rotation-invarianteene invariant/equivariant by design and, therefore are limited. A common practice is to leverage data augmentation [42]. However, such can only robustify the model under small perturbations, while falling short of handling arbitrary rotations in (3))). On the other hand, recent advances in designing light on alleviating alignment requirements in non-rigid point matching. However, we empirically observe that such a design generally degrades the expressive power of the feature extractor, leading to inferior performance of the trained DFM.
iaation-invariant/eee
feature extractor [4] [O] seem to shed
This issue also attracts attention from the community of shape matching. In [.] the rotation prediction, instead of directly estimating the transformation, is formulated as classification problem. However, this method can only address the rotation around a fixed axis and exhibits relatively low accuracy. Other works [56 57 transform the input point cloud into handcrafted features. However, it should be noted that this approach often increases the computational complexity due to the additional feature extraction and processing steps involved.
In this paper, we propose a data-driven solution to this problem, accompanied by a registration module that is robust with the estimated alignment (see Tab.3 in the main text). As mentioned in the main text, we adopt the regressor model proposed in ( We use the synthetic SURREAL shapes [52), which are implicitly aligned by the corresponding generative codes to train our orientation regressor. We follow [9] and use PointNet++ [43] as the feature extractor. As we assume to be given full shape, either mesh or point cloud, for each input cloud, we translate it so its mass lies at the origin point. Then we estimate the orientation deviation, R, of it from the orientation implicitly defined by the SURREAL dataset. Finally, we apply the inverse rotation Rd the input to obtain the aligned shape. We emphasize again that all the shapes, either train or test, go through this alignment in our pipeline.
A.2 Deep Functional Maps (DFM)
We illustrate the DFM, both for meshes and point clouds in Fig.
A.3 Registration Algorithm
We provide a more detailed version of our registration algorithm in Alg. 2 given a source mesh S, target point cloud T and trained point feature extractor F, we first initialize the deformation graph DG with QSlim method [17] and set (0) with identity transformations. After that, we compute the target point feature Fy. During registration, we first deform the source model with a deformation graph, then we update the correspondence by computing the nearest neighbor between the source feature and target feature in stage-l or the source point cloud and target point cloud in stage-Il. After we compute the set of filtered correspondences C²., we update the total cost function Frotal with the ADMM optimizer. Finally, we follow [3]] to implement an early stop strategy in both stages.
13

---

4
S
G
G₁
A
Aland wiihht
FMreg
Ci
S2
G
G2
A2
1(G.Co)
C2
42
(4)
S1
G
Gj
@1
V4
F
F1
E
wrigfes
FMreg
2/2
F2
E2
L(C2,C2)
π
C2
S2
G2
42
(b)
Figure (a) DFM architecture for pre-training on meshes; (b) Our proposed architecture for training a point-based DFM, which is self-supervised by its mesh-based counterpart.
Algorithm 2: Shape registration pipeline.
Input: Source mesh S {V, } and target point cloud T Trained point feature extractor F Output: X converging a local minimum of Etotal: Deformed source model (V", E): Correspondence IIşT Il between S and T.
Initialize deformation graph DG and (() with identity transformations; Fy F(T):k = O:
2 eps = 100 8: count = 0;
while True do
y Dx* 1)-;
1%100 = 0) and Flag- == Stage-/ then L F( (()(()); ! NN(F ,Fr) 11 NN,FF,FF
(1))
if 1%%00 = 0 and Flag- == Stage-Il then [ = NN(V(K), T) / NN(T, v());
Compute the (()
filtered correspondences ch
total Add (Dd((k)) T) + Ecorr Tex) Aarap Farap (DQ); :')):
(() arg min (xk)) 11
'otal'
if (k ((k-1)
12
total total < eps then
13
count count + 1;
14
if count > 15 then
15
if Flag = Stage-I then Flag Stage-Il; count = 0; eps 10-7;
16
Flag = Stage-Il then return IIl>: V";
17
k=k+
14

---

Figure 5: More non-rigid shape registration results. The original source mesh and its deformed counterpart are represented in blue, while the target point cloud and the ground truth mesh are depicted in orange.
A.4
Implementation Details
We implement our framework in PyTorch. We modified DGCNN (24
the backbone of feature extractor. The output feature dimension is set to 128. In the training related to deep functional maps, always use the first 50 eigenfunctions of Laplace-Beltrrmi operators. For training the DFM network, in (6n. of the main text, we empirically set Abjj = 1.0, orth = 1.0, >align = le-4, ÀNCE 1.0. We train feature extractor with the Adam optimizer with learning rate equal to 2c-3. The batch size is chosen to for all datasets.
Regarding the registration optimization, in Eqn. of the main text, we empirically set A 0.01, Acorr 1.0, Aarap 20 in Stage-I and > = 1.0, λcoι 0.01, Aarap in Stage-II. For a in (9n. of the main text, we set = 0.2. Note that the roles of Aed and λcorr are reversed from Stage-I to Stage-II. In fact, Stage-I, the initial shapes are likely to differ by large deformation, therefore it is more reliable to estimate correspondences via the learned embeddings. Once Stage-I converges, we empirically observe that the deformed source shape admits a similar pose with the target (see the video in Supplemental Material), therefore it is reasonable to further deform the source by extrinsic embeddings, i.e., Chamfer distance on the coordinates. We stress that both losses are critical in both stages and perform the following ablation experiment setting Add and λcorr to be zero independently the two stages. Tab, reports the scores. We find that most of the time, turning off leads worse results, especially when the test shapes heterogeneous (see, e.g., SHREC07-H),
Table 6: Mean geodesic errors (x100) of Ours setting Aot and Acoer to be zero independently in the two stages
SCAPH_r
SHREC19
SHREC07-H
Aed Stage-I
2.5
5.5
Acorr 0 Stage-11
2.7
5.6
6.8
Ours
2.6
5.1
5.9
B Experiments
B.1
Further Qualitative Results
Visualization of Registration Resultsin this section, show more qualitative results of registration regarding different templates and different test shapes in Fig Our method is capable of registering shapes with significant deformations across different templates.
15

---

Source
ALL Faps
Deep Shells
SSMSM
Ours
GT
Figure 6: We present the qualitative results of testing the TOPKIDS trained on SCAPE r. Our method demonstrates robustness in regions of mesh topology with noise, maintaining stable performance.
Figure 7: We showcase some examples of failed registrations.
we qualitatively visualize maps obtained by different methods tested in the TOPKIDS benchmark. It is obvious that our results outperform all the competing methods. Especially, purely intrinsic method, AttentiveFMaps seriously fails due the topological perturbation within the mesh structure.
Qualitative Comparison in TOPKIDS: In Fig.
Failure Cases: On the other hand, also identify some failure cases, which are shown in Fig. 7 The two cases both suffer prominent heterogeneity between the template and the target. In the left where ideal reference is given, adapt in registration instead of enforcing the hard highly non-isometric deformation with templates from SCAPE_r or FAUST_r (see Sec. B.3). While in the right case, is not obvious how to choose an easier template for establishing correspondences between the two shapes, we. therefore, settle down at the setting described in Sec.4.2. It is worth noting that, though, we can easily identify that the problematic regions, i.e., head, and hands from the visualization, which is unavailable for methods only return maps.
Templates: In the Fig. we visualize the templates used in different experiments. We select a pose close to the T-pose from the training sets of SCAPE_, r, DT4D-H, and FAUST respectively, as our templates.
Figure 8: We respectively use template from SCAPE (left), CRYPTO (middle), and FAUST (right) our experiments.
16

---

B.2 Further Quantitative Results
We have followed the setting of SSMSM to train our feature extractor on the SURREAL 5k dataset [f] and to test it on the FAUST SCAPE I, and SHREC19 datasets. We select one training shape from SURREAL 5k as the template in registration. As shown in Table 7 our method outperforms the competing non-rigid point cloud matching methods by noticeable margin. We also refer the readers to Table 3 in g for a complete table including methods utilizing mesh input, and we highlight that our method is even comparable with the latter. In particular, ours vs. SOTA of mesh-based techniques over the three test datasets: 3.2 vs. 2.3, 2.5 vs. 3.3, 4.6 vs. 4.7. That is, achieved the best overall result in of the three test datasets.
Trained on Large-Scale Dataset:
Table Mean geodesic errors (x100) trained on SURREAL
Train
SURREAL
Method
Datal
Test
FAUST_
SCAPE_r
SHREC19 r
DatfFMaps
26.5
34.8
42.2
230k
DPC
13.4
15.8
17.4
230k
SSMSM
3.5
3.8
6.6
5k
SURR-UURREAL
3.2
2,5
4.6
5k
Trained on Animal Dataset: We also perform experiments on the remeshed SMAL dataset. We first randomly generate 5000 shapes with SMAL model to train the alignment module. Then we train a DFM with the remeshed SMAL dataset. The template shape is 'dog_00 06' from the training set. The quantitative results are reported in Table 8 Remarkably, our method achieves more than a 40% performance improvement than the second- best baselines.
Table 8: Mean geodesic errors (x100) on SMAL
Tracs
DPC
DiffF
NIE
SSMSM
Ours
S
20.7
21
17.1
16.3
7.2
4.2
B.3 Robustness
Robustness with respect to Input Perturbation: We evaluate the robustness of different methods with respect to noise and orientation perturbations in Tab. [9] In the former, perturb the input point clouds with Gaussian noise. the latter, we randomly rotate the input point clouds within +30 optimization- approach and orientation regressor, in
degrees along v, y, axis. Thanks both tests we achieved the best accuracy, but also the lowest standard deviation over three runs of experiments.
Robustness with respect Missing Parts: Tab. I of main text, we report shape matching results on SHREC19 dataset, where the thth shape is commonly excluded in the test phase of prior works 20 The main reason that this shape is incomplete, consisting of two main connected components (see the abdomen of the source shape in Fig. 99) Therefore, for intrinsic methods depending on mesh eigenbasis, such disconnectedness makes spectral alignment extremely difficult, if not impossible. On the other hand, extrinsic methods show more robust outcome. In particular, we show Fig. that our framework can reasonably deform the template mesh (from SCAPE_r) to
Table Mean geodesic errors (x100) on under different perturbations. Noisy PC means the input point clouds are perturbed by Gaussian noise. Rotated PC means the input point clouds randomly rotated within +30 degrees along x, y, 2 axises respectively. The standard deviation value is shown
in parentheses.
Unperturbed PC
Noisy PC
Rotated PC
Daps [35
12.0
14.9(2.57)
26.5(3.35)
NIE 2
11.0
11.5(0.32)
19.9(1.29)
SSMSM g
4.1
5.4(0.11)
9.2(1.01)
Ours
2.6
2.9(0.06)
3.6(0.96)
17

---

the incomplete point cloud with a significantly isolated portion, which leads to superior matching results compared the baselines.
Our registration
Source
Att. FMaps
Deep Shelks
SSMSM
Ours
GT
Figure 9: challenging case from SHREC19_ dataset, see the text for details.
Robustness of Orientation Regressor: demonstrate that our learned orientation regressor (see S of the main text), we visualize its outcome on shapes from SHREC07-H, which manifests notable diversity in sampling density, size, and initial orientation.
Aligned
Raw
Figure 10: Bottom row: The input shapes of SHREC'07 exhibit various orientations; Top row: The shapes aligned by our orientation regressor. Note that there exists significant differences in sampling density.
B.4 Scalability
Our approach is scalable w.r.t input size. Thanks to the fact that we non-rigidly align shapes in R23 we can in theory freely down- and up-sample both the template mesh and the target point clouds. Note this is non-trivial for methods based on mesh or graph Laplacian, deducing dense maps with landmark correspondences over graph structures is difficult task on its own. In Fig. [1] we show matching results on real scans from the FAUST challenge, each of which consists around 160,000 vertices. In contrast, S can handle at most 50,000 vertices without running out of 32G memory on a V100 GPU. We visualize matching results on the subsampled (to 40,000 vertices) point clouds for comparison.
18

---

SSMSM
Ours
Source (40k veris)
Target (40k verts)
Source (160k verts)
Target (200k & 180k verts)
Figure 11:
Qualitative results on matching MPI-FAUST raw scan data. Our method can match the point clouds at the original resolution, while SSMSM g can not.
B.5 Run-time Analysis
In this part, we analyze the time cost of our optimization- registration module. In particular, we separate it into two main parts, initialization, and iterative optimization. For the former, we in general fix source mesh template, and then we construct deformation graph on it as described in Sec.3.2 2 in the main text. Since the deformation graph can be pre-computed and cached for further use of the same template, we put more emphasis on the latter.
In particular, we perform registration between the template shape from SCAPE_ r and the 20 test shapes therein and compute the average run-time of each core step in the optimization. As illustrated in Tab. 10 our method achieves an average optimization time of approximately 13.4 seconds per shape, with about 90% of the total time being dedicated Stage-I. Analysis depicted in Fig. T2 reveals that the most time-consuming operation is the ADMM backward step, while the combined time cost of our correspondence filtering and updating operations accounts for less than 3% of the overall execution time. Though the reported run-time is not quite satisfying, we emphasize that evaluate maps across point clouds, we only need register them to the same template shape (O(n)), which is different from the pair-wise (O(n²) mapping and post-pnn post-processing approaches (7].
Table 10: Average time and iteration cost for each shape.
Total
Stage-I
Stage-II
Time cost(s)
13.4
12.0
1.4
Iterations
1274.
1130.
144.
14%
Correspondence Filtering
Mesh deforming
18%
Energy Computing
Correspondeuce Updating
1%
ADMM Backward
Figure 12: Run-time decomposition of our registration module.
19


Reviews:
[{"review_id": "lrTQ4F6B7h", "reviewer": "Reviewer_iB8H", "summary": "This paper studies the problem of non-rigid shape matching. The problem is first decomposed into learning (rigid) orientation of shapes and   then learning shape matching on aligned shapes. For the later, it proposes to learn it with a  sequential pipeline, consisting of various modules, that is also optimized in a two stage manner.  the main idea is to facilitate learning by similarity in both ambient space (R^3) as well as high dimensional learned feature space. To this end, it trains a DGCNN feature extractor based on combination of several loss functions (DFM prior loss, ARAP loss, Chamfer/cosine loss). The method is validated on  isometric as well non isometric benchmarks.\n\nPost Rebuttal:\n\nThe rebuttal provided additional experiments to support some of its claim as mentioned below. I lean towards rejection since the work is a sequential concatenation of existing individual modules (also noted by reviewer CNuX and xFDL ) without any conceptual justification to why those modules need to be combined e.g. why do we need to combine a DFM and ARAP loss. The rebuttal also contains several unsubstantiated statements detailed below. Combining two lines of research is not a technical contribution or contribution to community if there no conceptual justification given. Majority of the introduction section needs to be rewritten and contextualised *correctly* wrt prior work. Besides, submission had several hyper-parameters whose values were missing and there is no mention of model sensitivity towards these hyper-parameters even though the approach is unsupervised.", "questions": "- for Non-isometric, SMAL is considered the main benchmark and there is already a large literature that benchmark their results on SMAL. why ignore such standard benchmark?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Experimental results on various near isometric benchmark look promising (Table 1) and compares extensively with existing work.\n\n- the paper identifies the generalization problem of some embedding based approach [30,8] to unseen shape data.", "weaknesses": "- Presentation: Section 1 contains several factually incorrect or unsupported claims. Moreover, the motivation of the work is misplaced since their is no validation of it later in the experimental section. Please support all the intuitive claims either with a prior reference or by explicitly mentioning results from this paper. \n\na)  why do we need to decompose the shape matching problem into learning an alignment first and then matching aligned shapes? Is it \n because [8], [22] do so or is there a scientific justification behind it as shown empirically in rigid shape matching or non-rigid shape \n matching with DFM. e.g. it is shown to provide an extrinsic supervision that helps to disambiguate symmetry issues in DFM. Please motivate the problem/solution accordingly.\n\nb)  Line 46 < prior work with resulting high dimensional embedding lack intuitive geometric meaning> Since this is used as one of 3 \nreasons behind this paper/formulation, please demonstrate this on an example where in contrast, this paper obtains an intuitive \nembedding with geometric meaning.\n\nc)  Line 54-57: Please provide a reference to support these claim or prove it later with visualization and examples in experimental section.\n\nd) [30] requires shapes to be pre-aligned at train or test time. This is not true. You can train and test [30] without such alignment and it makes no such assumption on input requirements in the paper[30].\n\n\n-  Novelty: the main conceptual idea/key insight (Line 52) in this work is to enforce similarity in both ambient space as well as learned feature space. DPC [b] proposed the exact same idea with DGCNN, cosine/chamfer loss in the most simplest possible way for non-rigid shape matching. The idea is already well known that others have even built upon it for non-rigid shape matching e.g. [a]. So claiming this as a key insight is not justified IMO.\n\n-  Missing references and comparison of a very similar work[b]:  authors should also compare their work with DPC on their benchmarks. this will show what gains (if any) are brought by DFM prior or ARAP loss.\n\na. Learning Canonical Embeddings for Unsupervised Shape Correspondence with Locally Linear Transformations\nb. DPC: Unsupervised deep point correspondence via cross and self construction, 3DV 2021.\n\n-  formulation without a principled approach: the network consisting of (DGCNN, DFM prior, ARAP loss, chamfer loss, pointinfo loss etc) combines modules from different frameworks without any principled reason. e.g. DFM prior from DFM literature, DGCNN feature extractor, chamfer & cosine loss from DPC etc. in a two stage optimization procedure. why do we need a DFM prior when we are deforming a source and a target shape with a ARAP loss ? \n\n-  Too many hyperparameters in an unsupervised approach : The paper should mention in the main body how many hyperparameters  overall does this approach has and how were their values chosen? I count more than 10. Moreover, how is it justified to choose different values for the same hyperparameter  (weighing scalars) in an algorithm and call the resulting approach *unsupervised*?  there is a two order magnitude of difference in hyperparameter values ($\\lambda_{cd}$ and $\\lambda_{corr}$) between different runs of the algorithm (Stage 1 and Stage 2).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the constructive comments on our motivation and novelty. Below we address the comments. \n\n**Novelty, especially compared with DPC:** We argue that our approach is *fundamentally* different from DPC in the following perspectives: \n\n1.\tOur feature extractor for point clouds is learned in an intrinsic-aware way, while such information is absent in learning the counterpart of DPC; \n\n2.\tOur registration component explicitly deforms the template towards the target point cloud. In DPC, the cross-reconstruction is essentially a re-indexing of the target point cloud via soft maps induced by the latent proximities. For instance, Eqn. (3) of DPC amounts to a point permutation when the weights deduced in Eqn. (2) approximate a delta distribution; \n\n3.\tFinally, like DFM-based approaches [8, 22], DPC infers correspondences via high-dimensional embeddings, while our approach does so in the ambient space, which is more intuitive and easier to analyze.\n\nMoreover, we compare our method with DPC on the near-isometric benchmarks. As shown in Table 1 & 2 in the Rebuttal Mat., our method outperforms DPC by a *significant* margin when trained on both small-scale and large-scale datasets. We refer the readers to Table 3 of [8] for more detailed results. \n\n**Principle of the proposed formulation:** We refer the readers to the Author rebuttal for detailed motivation of our framework design. \n\n**Motivation of alignment and matching:** We emphasize that our approach performs extrinsic shape registration on *raw* point clouds, in a direct fashion without any explicit mesh/graph construction. It is then clear that our approach is sensitive with respect to position and orientation of the input. Many relevant works either implicitly (NDP [27], AMM [45]) or explicitly (SSMSM [8], NIE [22]) require rigid aligned shapes as input/initialization. Another way out is to leverage dense correspondence labels (DiffFMaps [30]). We remark the latter is not by construction robust w.r.t SO(3) perturbations. As shown in Table 3 of our main submission, it is sensitive w.r.t rotations when trained on aligned point clouds, without the effective data augmentation in the original implementation. \n\nIn contrast, we leverage the generative model and propose a principled solution to this challenge, which lifts the rigid alignment in inference or dense correspondence labels in training. \n\n**Geometric meaningful embeddings:** We clarify that our high-dimensional embeddings, similar to that of [8, 22], are obtained by an uninterpretable learned network, which also lacks geometric meaning. In general, correspondences induced by such embeddings are difficult to evaluate and analyze without ground-truth maps. In contrast, our formulation leverages the learned embeddings to deform a template shape explicitly towards a given target point cloud, which provides a more geometrically intuitive mapping/registration procedure. We particularly refer the readers to the video in the Supp. Mat. As a result, one can perform both qualitative (by visually comparing the deformed template and the target) and quantitative (by Chamfer distance or RMSE) analysis on the output maps, even without ground-truth maps. \n\n**Claim made in Line 54-57:** The regarding claim has been justified in Fig. 3 and Table 1 of the main submission. Especially, we compare our method with NDP [27] and AMM [45], which depend on proximities in the ambient space to iteratively estimate correspondences and can fail significantly in the presence of large deformation. In Fig. 3, we deform the FAUST template (the 3rd shape from left) to the right-most shape. It is obvious that ambient proximities would lead to erroneous maps in the beginning, making it difficult to guide the right deformation (to rise arms significantly). The quantitative results in Table 1 also validate this observation. Our method achieves at least 78% matching error reduction compared to them on the two standard benchmarks. \n\n**Motivation of using DFM prior when ARAP loss is used:** The ARAP loss in general serves as a regularizer for shape registration, which prevents the deformed template from being overly distorted. Solely using ARAP loss does not lead to successful registration, as it could not provide any cue for matching. We refer the readers to Table 4 of our main submission: Comparing *w/o Stage-I* and *Full*, we can see the DFM prior (i.e., Stage-I) contributes significantly. \n\n**Too many hyper-parameters:** The hyper-parameters used in the pre-trained models (orientation regressor and DFM) follows the regarding prior works. As for those for the registration optimization, we search for the optimal hyper-parameters with *2 pairs of training shapes* of SCAPE w.r.t the registration loss, which does not depend on any correspondence label. We remark that our hyper-parameters are fixed across different template meshes, training sets, and test sets\n. \n**Experiments on SMAL:** As requested, we have performed experiments on the *remeshed* SMAL dataset [d]. We first randomly generate 5000 shapes with SMAL model [e] to train the alignment module. Then we train a DFM with the remeshed SMAL dataset [d]. The template shape is ‘dog_06’ from the training set. The quantitative results are reported in Table. 3 of the Rebuttal Mat. Remarkably, our method achieves more than a 40% performance improvement than the second-best baselines. \n\n[d] Complex functional maps: a conformal link between tangent bundles, N. Donati, E. Corman, S. Melzi, M. Ovsjanikov, CGF 2022. [e] https://smal.is.tue.mpg.de\n"}, {"review_id": "ZlWwq4v4wO", "reviewer": "Reviewer_xFDL", "summary": "This paper proposes an unsupervised framework for non-rigid shape registration. The proposed method deforms source mesh towards the target point cloud, guided by correspondences induced by high-dimensional embeddings learned from deep functional maps. Empirical results show that the proposed method achieves state-of-the-art results on several benchmarks for non-rigid point cloud matching.", "questions": "1. Why is the proposed method unsupervised? It seems the orientation regressor requires the ground-truth pose to the canonical space. And the PointInFoNCE loss (Eq. 4) also requires correspondence labels? Or the authors call the generalization from one dataset to the other datasets the unsupervised learning?\n\n2. How good is the orientation regressor? Can it deal with large translations? e.g., the translation introduced by a walking human. From my experiences, this kind of pose regressor can only overfit to a specific model and hard to generalize. Also, it is usually not very accurate (correct me if I was wrong);\n\n3. Why all the models are called \"pre-trained\"? Should it be generalizing a model trained on the same task to new data?", "limitations": "Limitations have been discussed in the main paper.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The overall writing is fluent;\n\n2. The proposed method outperforms state-of-the-art approaches on several benchmarks;", "weaknesses": "1. The organization can be further improved (the organization of Methodology does not follow Fig. 1), which makes the paper sometimes hard to follow (probably due to the complexity introduced by assembling 3 stages for tackling the problem);\n\n2. The novelty of this paper would be a major concern. It seems the proposed algorithm in this paper simply assembles three stages, each with an existing method. The orientation regressor is from [9]. The feature extractor is the modified DGCNN proposed in [22]. The non-rigid registration mainly follows [18].\n\n3. The submission and main paper have different titles. Please fix it.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for all the insightful comments. First of all, we would be happy to improve the presentation as suggested, as well as to fix the inconsistent titles in the future revision. Below we address the comments: \n\n**Novelty:** We acknowledge that many components in our pipeline are inspired by existing works. However, we would like to emphasize that the way they are integrated are novel, which leads to a simple yet effective solution to the challenging problem of matching **unstructured** point clouds undergoing significant deformations. Beyond that, to the best of our knowledge, our solution to train a point-based feature extractor that respects intrinsic geometry, but without ground-truth correspondence labels is novel. We kindly refer the readers to the Author Rebuttal for a more detailed response.\n\n**Why is the method unsupervised?**  A more precise description of our supervision could be **correspondences label-free**. Given the opportunity, we would be happy to clarify this in the future revision. We acknowledge that the orientation regressor indeed relies on the ground-truth pose, which is obtained by simply fixing certain parameters in the SMPL generative model. On the other hand, we emphasize the fact that our method requires **no** ground-truth correspondence, either sparse or dense, throughout the pipeline. Obviously correspondence labels are much more difficult to obtain in practice. \n\n**PointInfoNCE requires labels:** The PointInfoNCE loss is introduced to enforce the output features of DGCNN (for point clouds) to be *point-wisely* close to that of DiffusionNet (for mesh). The features are computed on the same shape (without and with mesh connectivity, respectively). Therefore, we simply use the identity map between the same set of vertices (regarded as mesh vertices and point clouds respectively) on each shape to formulate Eqn. (4), which does *not* require any non-trivial correspondence label. \n\n**Performance of orientation regressor and its robustness regarding translation:** Since we assume the completeness of input point clouds in the submission, deviations induced by large translations can be resolved by moving the mass centers of all point clouds to a fixed point, which is also a standard practice in shape registration. \n\nRegarding the quality of orientation regressor, as shown in Fig. 3 of the Rebuttal Mat., though it is trained on a set of generated shapes sharing the same mesh connectivity, it can handle heterogenous shapes of varying number of vertices (2,000~20,000) well and delivers reasonable rigid alignments. \n\nMoreover, we empirically observe that our registration component enjoys certain robustness regarding the imperfect initial orientation. In particular, we *turn off* the orientation regressor, and *directly* perform registration on the rotated point clouds (see experiments reported in Table 3 of the main submission) and get **4.7(1.03)**/mean(std) in mean geodesic errors. New results show that our method still produces better and more stable results than the baselines. We attribute such robustness to the fact that non-rigid shape registration also involves deformations with respect to extrinsic orientation deviations. \n\n**Why all models are called \"pre-trained\"?** We call the orientation regressor and the point-based DGCNN *pre-trained* to emphasize that our key module, the final registration component, is optimization-based, leading to a geometrically meaningful procedure (see, for example, the video clip in the Supp. Mat.). The learning-based models are pre-trained and frozen during registration.\n"}, {"review_id": "07guOqlIG4", "reviewer": "Reviewer_1VPz", "summary": "The paper describes a method for corresponding a 3D triangle mesh to a point cloud of a similar (possibly articulated) shape.  The two-stage process first corresponds the two shapes in a high-dimensional feature space, then corresponds them again using geometric features while deforming the source closer to the target.", "questions": "Could this approach be implemented without learned functions? The intro mentions that spectral mapping can be geometric (rather than learned), and the second stage is akin to non-rigid ICP.\n\nWould it make sense to run Stage-I again after Stage-II, and repeat both stages a few times?", "limitations": "The method requires a triangle mesh for the source shape, which is not discussed in the limitations section. The exposition would be stronger if the authors took the source shape as a point cloud and ran some automatic meshing on it for their experiments.\n\nAnother limitation that the authors mention is that the method only works on \"full shapes\". I assume this means that the whole source and target surfaces are adequately sampled. It would help if the authors suggested some ideas on how one would go about lifting this requirement.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The approach makes sense, and the results seem strong compared to previous work, both qualitatively and quantitatively.", "weaknesses": "In general, the presentation could be improved.\n\nThere should be a figure analogous to Figure 1, but showing qualitative correspondences for cases where train/test datasets do match. Only showing severe failures of competing methods on very disparate shapes does not convey the full picture.\n\nThe supplementary video showing the progress of the correspondence algorithm is very helpful. It would be even more informative if the mesh was texture mapped with the checkerboard as in Figure 1.\n\nEquations, Figures, Tables need more thorough descriptions.\n\n  Equation 1:\n          Even though the text says that one can optimize the feature function F when it introduces equation 1, it is unclear how the function F is optimized. If not including an equation involving F, you could at least point to a specific equation number in [11].\n  Equation 2: Parameter alpha should be defined after this equation, rather than after equation 4.\n  Equation 3: What does the cross symbol represent?\n  Equation 4: Parameter n2 is not used in the expression. Should the denominator summation go up to n2?\n  Table 1 & 2:\n          What are the two sections separated by a horizontal line (they each have a bold set of numbers)?\n          Train/Test column contains no values.\n  Table 2:  What is \"Ours-CRYPTO\", this acronym is not discussed/referenced in the text?\n  Table 3: What is \"Ideal PC\"?\n\nAlgorithm should also initialize \"Flag\" (presumably to Stage-I).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the constructive comments and the recognition of our contributions. Below we address the comments: \n\n**Improve paper presentation:** \nThank you for the suggestions on improving the presentation of the paper, we would be happy to incorporate all of them in the future revision. Regarding Fig. 1, our intension is to illustrate the heterogeneity presented in the SHREC07 dataset, as well as our stronger generalization capacity than the competing baselines. The more thorough evaluation results are reported in Table 2, which also agree with the qualitative ones in Fig. 1. Below are responses to the minor comments: \n\n1. We would be happy to clarify Eqn. (1)-(4) in the future revision. On the other hand, we would like to refer the readers to Sec. 1 of the Supp. Mat. for a more self-contained description; \n\n2. The cross symbol (\\dagger) in Eqn. (3) indicates the pseudo inverse of a matrix; \n\n3. The PointInfoNCE loss is introduced to enforce the output features of DGCNN (for point clouds) to be *point-wisely* close to that of DiffusionNet (for mesh). The features are computed on the same shape (without and with mesh connectivity, respectively). A similar loss regarding S_2 is used as well; \n\n4. In Table 1 and 2, the methods above the horizontal lines are *designed to* take meshes as input. The rest methods take point clouds directly; \n\n5. Ours-CRYPTO in Table 2 indicates that a shape belongs to the category ‘Crypto’ of the DT4D-H dataset is used as template; \n\n6. Ideal PC in Table 3 means the clean, aligned point clouds. \n\n**Implement our approach with non-learned functions:** \nWe highlight that our pipeline requires no pre-processing or pre-alignment on the input point clouds. It is possible to implement with axiomatic spectral method. However, it would require to build graph Laplacian on-the-fly, since during registration the template is dynamically deformed. Such approach can be less efficient, and may suffer from scalability issues. \n\n**Repeat Stage-I and -II for multiple times:**\nThank you for the suggestion. We have performed the whole registration procedure twice on the Scape dataset. The registration error decreased from 2.57 to 2.49, i.e., an improvement of 3.1%. It is worth noting, though, such approach on average introduces an over 60% computational overhead (convergence steps: 1274 vs. 1931). \n\n**Perform meshing on the source point cloud:**\nThank you for the suggestion for improving the utility of our approach, which we believe is surely feasible. We think exploring it in a principled way can help to lift the need of template mesh in the future. \n\n**Applicability on partial point clouds:** \nIn Fig. 1 and Fig. 2 in the Rebuttal Mat., we demonstrate some preliminary results on extending our pipeline to matching partial, even noisy point clouds. Essentially, we train a DFM tailored for partial-view point clouds generated on SCAPE_r dataset, and replace the two-way Chamfer distance with a one-way one. Note that in this experiment we assume the partial point clouds are rigidly aligned. Nevertheless, we believe the results have sufficiently shown the potential of our general scheme.\n\n"}, {"review_id": "kocWccQ3xG", "reviewer": "Reviewer_CNuX", "summary": "In this paper an unsupervised non-rigid shape registration method is proposed. The proposed method combines intrinsic spectral mapping (i.e. based on the deep functional map framework) together with  extrinsic deformable shape registration (i.e. a deformation graph) to enable unsupervised 3D deformable shape matching. In many challenging benchmark datasets, the proposed method demonstrates competitive matching performance, better cross-dataset generalisation ability, and robustness against noise and rotation of input shapes.", "questions": "1. Since the proposed method is based on test-time optimisation, what is the convergence speed of the proposed method? \n2. Since the deep functional map can be fully intrinsic (i.e. with intrinsic input features), what if we use it to obtain the initial correspondences and rigidly align the shapes into the canonical frame without using rotation regressor?\n", "limitations": "1. The proposed method is tailored to complete shape matching, so it cannot achieve desirable matching results for partial shapes.\n2. The proposed method is based on iterative optimisation to align two shapes, so the runtime is slower than other learning-based methods.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow. The main contribution and methodology are well illustrated.\n2. The paper integrates shape matching and shape registration into the same framework. The shape matching part is based on the deep functional map framework to obtain point-wise correspondences. The shape registration part is based on the deformation graph to non-rigid align two shapes to refine the final correspondences.   \n3. In order to enable matching shapes with different orientation, the paper proposes an orientation regressor to align shapes into a canonical frame.", "weaknesses": "1. The novelty of the method is limited. The proposed method consists of three components (rotation regressor, feature extractor, shape registration) and each component is derived from prior works without large modifications.\n2. The interconnection of the components in the method is missing. The first stage is to train a rotation regressor to align shapes into the canonical frame. The second stage is to train a feature extractor to obtain correspondences. The third stage is to optimise the deformation graph to align two shapes. Every stage is somehow separated during the training stage, while it is desired to see the connection of shape matching and shape registration like Deep Shells or NeuroMorph.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the constructive comments and the recognition of our contributions. Below we address the comments: \n\n**Modules are separated in the pipeline and lack connection:** \nThank you for the constructive comment. We agree that integrating shape matching and shape registration in a more associated manner can be desirable. However, we would like to highlight that Deep Shells and NeuroMorph fully leverage the structural information within the meshes. More specifically, Deep Shells extensively computes the first 500 Laplacian eigenpairs, and NeuroMorph uses mesh connectivity in graph neural network training and dense geodesic distance matrices on both input shapes for training loss. Such intrinsic geometric information is critical for the respective non-rigid shape registration. In contrast, our method expects nothing but raw point clouds during inference (the template mesh is fixed), which can be challenging for extracting intrinsic information (see, e.g., shapes in Fig. 3 of the supplemental material). \n\nTo overcome this difficulty, we propose to learn a teacher network (DiffusionNet) on a small number of training meshes (80 in all of our experiments), and then train a student network (DGCNN) that consumes points but infers intrinsic features by mimicking the teacher network. Finally, the student network is frozen and used to estimate correspondences dynamically during the final registration part. If we were to integrate shape matching and registration as an associated pipeline, it would make more sense to put the teacher network, the student networks, as well as the registration component all trainable. However, it would violate our main goal -- performing shape registration on raw point clouds.\n\nAs a proof of concept, we have tried to unfreeze and fine-tune the point feature extractor during the registration process on the SCAPE_r dataset, in the following two ways: 1) Updating point-wise correspondences per 100 iterations as in Alg. 1, leading to a slight performance drop (from 2.6 to 2.8); 2) Updating per iteration, leading to a failure of convergence. \n\n**Convergence speed:**\nWe have reported the average convergence steps as well as a full running-time decomposition in our Supp. Mat. (see Table 2 and Fig.6 therein). Especially, our method converges within 1274 iterations (1130+144) on average, tested in the Scape test set.  \n\n**Use intrinsic methods to obtain rigid alignment:**\nWe emphasize again that our main target is to perform *directly* shape matching/registration on raw point clouds. In particular, our pipeline, once trained, can deform a given template mesh to target point clouds without any pre-processing on the latter. Note that pre-processing may be slow and parameter sensitive, and is beyond the scope of this paper. \n\n**Applicability on partial point clouds:**\nIn Fig. 1 and Fig. 2 in the Rebuttal Mat., we demonstrate some preliminary results on extending our pipeline to matching partial, even noisy point clouds. Essentially, we train a DFM tailored for partial-view point clouds generated on SCAPE_r dataset, and replace the two-way Chamfer distance with a one-way one. Note that in this experiment we assume the partial point clouds are rigidly aligned. Nevertheless, we believe the results have sufficiently shown the potential of our general scheme."}, {"review_id": "oxZni49omg", "reviewer": "Reviewer_GUi9", "summary": "The method proposes an unsupervised pipeline to solve 3D shape-to-shape registration. Firstly, the two shapes are aligned by a pre-trained orientation regressor. Then, a soft-correspondence is obtained by a Point Feature extractor, optimized using a Deep Functional Maps schema, and considering several unsupervised regularizations (e.g, bijectivity, orthogonality). Finally, the source shape is iteratively deformed using a Deformation graph (using Chamfer distance, the learned correspondence, and as-rigid-as-possible), and the correspondence is updated every 100 iterations. The method is tested on humanoid datasets (FAUST_r, SCAPE_r, SHREC19_r, SHREC07-H, DT4D-H),  and extensively compared against many other approaches, both  supervised and unsupervised. obtaining interesting results.\n\n== FINAL RATING ==\n\nIn the discussion phase, authors provided much further evidence and clarified my concerns. I notice, however, that while a majority is leaning toward acceptance, there is not a consensus among the reviewers. Going through the reviewers that assigned negative scores, I do not find significant ground to change my rating. In particular, Reviewer xFDL mainly criticizes the novelty of the work but does not point to other works to support this claim. While I can understand that the work does not seem particular novel in its component (and the general principle of registration + correspondence in feature embedding can be found in previous works (e.g., more SmoothShell, DeepShell than DPC, while both of them are designed for meshes), I do not find itself a proper ground of rejection (as also reported in the Reviewing Guidelines; also combinations of existing techniques are valuable). The experimental evaluation is appreciable, the performance convincing, and I do not see this as incremental w.r.t. any other previous work. I find this an interesting contribution to a research field that counts only a limited amount of work. \nInstead, I see the detailed review and discussion of Reviewer iB8H, and I think it contains many valuable observations that could improve the paper. However, summarizing the reported criticisms seems to be: \n\nA) novelty w.r.t to DPC; which in the last comment looks solved, or at least significantly tuned down \n\nB) general motivation of the work; while I see that the underlying principle and positioning of the work in literature can be improved, overall, I do not see other works that perform similarly, and I struggle to see this as a \"follow-up\" of a specific methodology to consider this an incremental contribution of something in particular. Also, the obtained results seem to be already a reasonable justification for the proposed approach (since I think we all agree that the paper effort is beyond engineering work, and hence, they communicate a promising research direction)\n\nC) Other details (e.g., clarifying the role of hyper-parameters, missing citations, rephrasing); that I think make sense, but they can be easily addressed in the camera ready, and I do not consider a sufficient ground for rejection.\n\nFor these reasons, I lean toward acceptance. I suggest authors incorporate the suggestions (especially about suggested experiments and paper positioning in literature), and I wish them the best of luck with their work.", "questions": "1) In Table 1 some results are not reported. Why?\n2) The orientation regressor module resembles the input transform module of PointNet. Wouldn't a rotation augmentation for the feature extractor work (and remove the need for a further module)? Another possible alternative is given by the method proposed in [A].\n3) Given that overall the pipeline assumes some degrees of bijectivity, how would it perform in the presence of a significantly different number of vertices between the two shapes (e.g., by a factor of x100)? Even considering two complete shapes which have a bijectivity on the surface, the losses are defined vertex wise, and might lead to a degradation of the regularization impact.\n\nMinor:\n1) I see that reference [30] is incorrect in the bibliography, and [31] has names compressed. I suggest double-checking the references.\n2) Relevant references not discussed: [B], [C], [D]\n\n[A]: Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes, Zhou et al., 3DV 2022 \\\n[B]: NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go, Eisenberger et al., ECCV 2021 \\\n[C]: NCP: Neural Correspondence Prior for Effective Unsupervised Shape Matching, Attiki et al., NeurIPS 2022 \\\n[D]: 3d-coded: 3d correspondences by deep deformation, Groueix et al., ECCV 2018", "limitations": "Limitations are discussed in the main paper.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1) Not many methods are available to solve shape-to-shape correspondence in an unsupervised way; the proposed approach smartly combines existing techniques and well-established methods to obtain good results.\n\n2) The proposed approach outperforms existing direct competitors, even some supervise approaches. I am sure it would be of interest for the shape-matching community.\n\n3) The paper is well presented, clear and direct, I enjoyed the reading, and I had no problem understanding the main components. I am sure the method can be re-implemented with limited effort.", "weaknesses": "1) The general applicative context of the method is unclear. From the introduction and the pipeline figure, I had the impression that the method is designed to obtain correspondence between point cloud. However, the method is about registering a mesh to a point cloud, which can be extended to the point cloud case, using the source mesh as a bridge. This, however, is not properly tested: while other approaches also show noisy point clouds with different levels of noise (e.g., [30]), in this case, only point clouds sampled from meshes are reported, with a gaussian noise that is not detailed nor in the main manuscript or in the supp.mat. . Given the nature of the used feature extractor (which I expect to be quite sensitive to real noise and clutter), I suggest including more details on the considered Gaussian noise, and test on real raw pointclouds (which could also be another failure case, but would point the reader to interesting future directions)\n\n2) While I appreciate the simplicity of the proposed approach and the combination, I struggle to see a general message or insight. Also, I do not see the method as directly applicable since it is trained only on a few shapes, and we do not know how it would scale on larger datasets (e.g., AMASS), and I guess it would fail to solve registrations in real contexts (e.g., noisy scans), since it would ruin the learning on the underlying surface. It is also tested only on a domain (humans) with a significant amount of labelled data. So, without these elements, I wonder if the paper could be of significant impact or if it is just a carefully designed pipeline that will be just another matching method without much influence in the field. I suggest discussing the main message, analysing the scalability of the approach, and testing on domains in which labelling data are much more complicated to obtain (e.g., chairs, which also would make stronger the claim about topological noise in the supplementary)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the constructive comments and the recognition of our contributions. Below we address the comments:\n\n**Applicative context of the proposed method:**\nWe actually follow the same scheme of adding noise perturbation as DiffFmaps [30]. \nWe highlight our new results reported in Rebuttal Mat., which demonstrate that our method can be *directly* applied to matching large-scale real scans, as well as be extended to perform matching and registration regarding partial, even noisy point clouds. \n\n**Scalability in terms of training set:**\nFirst of all, we consider the ability of our method to efficiently and effectively learn from a small-scale training set as an advantage. That being said, we have followed the setting of SSMSM [8] to train our feature extractor on the SURREAL 5k dataset [f] and to test it on the FAUST_r, SCAPE_r, and SHREC19_r datasets. We select one training shape from SURREAL 5k as the template in registration. As shown in Table 2 of the rebuttal material, our method outperforms the competing non-rigid point cloud matching methods by a noticeable margin. We also refer the readers to Table 3 in [8] for a complete table including methods utilizing mesh input, and we highlight that our method is even comparable with the latter. In particular, ours vs. SOTA of mesh-based techniques over the three test datasets: 3.2 vs. 2.3, **2.5** vs. 3.3, **4.6** vs. 4.7. That is, we achieve the best *overall* result in two of the three test datasets. \n \n**Extension to more challenging datasets, such as chairs:**\nWe highlight that our unsupervised matching network is essentially trained with the prior that the underlying maps among training shapes are isometric and bijective. Especially, the former plays a pivotal role in the development of DFM frameworks. In the case of chairs, such prior can be violated significantly (e.g., chairs can be with or without arms, back, etc.), making direct extension challenging. We believe that exploring effective matching priors for such data is an interesting future direction to investigate.\n\n**Missing scores in Table 1:**\nThere are three axiomatic (optimization-based) methods in Table 1 -- Smooth Shells [12], NDP [27], and AMM [45]. Obviously, they do not have generalization results. Regarding the rest missing scores, we apologize for not being able to perform all the tests upon the deadline. On the other hand, we would like to point out that the regarding methods do not perform strongly in the accomplished, typically simpler, tests, which therefore has little impact on the overall comparison and analysis. \n\n**Alternatives to orientation regressor:**\nWe emphasize that our current pipeline can in general take arbitrarily rotated point clouds as input (See Fig. 3 in the Rebuttal Mat.), performing data augmentation in SO(3) can be quite heavy, and might not be optimal (see Table 1, 2 in [g] for reference).\n\n**Dependence on bijectivity:**\nFirst of all, one simple solution is to down-sample the target point clouds. Note that our pipeline is primarily designed for matching organic shapes like humans and animals (see the latter in the rebuttal material), down-sampling the underlying smooth surfaces in general would not lose many high-frequency signals. Once the template mesh is deformed to fit the down-sampled point clouds, then one can easily infer the maps regarding the full-resolution point cloud, since all the shapes are explicitly, non-rigidly aligned. \n\nIn practice, we also observe the robustness of our pipeline regarding the number of vertices. For instance, the number of vertices of the shapes in SHREC07 ranges from 2,000 to 16,000, while the template shape has around 5,000 points. And our method achieves decent registration results (see Fig. 1) and outperforms the baselines by a large margin (see Table 2) under such perturbation, beyond the heterogeneity. \n\n**References issues:**\nWe thank you for pointing out the relevant prior works and would be happy to include and discuss them in the future revision. Of course, we will fix the typos in the current references as well. \n\n[f] Learning from synthetic humans, G. Varol, et al., CVPR 2017. \n[g] Vector Neurons: A General Framework for SO(3)-Equivariant Networks, C. Deng, et al., ICCV 2021."}]

Author rebuttal:
We would like to thank all the reviewers for their time, effort, and insightful comments on our manuscript. We are glad that all the reviewers recognize our promising results on various benchmarks. We are encouraged by the recognition that our formulation is reasonable and that our writing is clear and easy to follow (Reviewer xFDL, CNuX, GUi9). We appreciate the recognition of our method's motivation and design by Reviewer CNuX, GUi9. We extend our gratitude to Reviewer GUi9 for acknowledging the difficulty of our task of interest and recognizing the contributions of our work to the shape matching community. 

Before we clarify our main contributions and address some common concerns, we would like to kindly refer all the reviewers to our **Supp. Mat.**, in which we put a decent amount of extra experimental results and analysis, as well as a video clip demonstrating intuitively our pipeline. We also report several experimental results per reviewers' requests in the attached document, which we refer as the **Rebuttal Mat.** in the following. 

### **Problem statement and key challenges:**
In this paper, we propose a hybrid pipeline for computing dense correspondences between a pair of unstructured point clouds, which can undergo significant non-rigid deformations. In particular, we encounter the following challenges: 

1.	It is difficult to directly infer intrinsic structure (i.e., geometric structure regarding the underlying surface) from raw point clouds. A common practice is to perform meshing or to build certain graph Laplacian on top of them [8], which can be inefficient and unscalable; 

2.	Non-rigid shape registration techniques [27, 45] can handle small to moderate non-rigid deformations via proximities in the ambient space R^3. However, in the presence of large deformation, they can fail significantly since the intermediate maps induced by Euclidean proximities do not necessarily respect the intrinsic, non-rigid deformations;

3.	Except for the purely intrinsic methods, most of the existing works either require the input point clouds to be extrinsically aligned [8, 22], or depend on heavy correspondence labels during training [30].

### **Novelty:**
To address the above challenges, we propose a systematic pipeline, which combines the learning-based shape matching and the optimization-based shape registration techniques. We conclude our key novelties as follows: 

1.	To the best of our knowledge, we are the first to address the problem of **unstructured** point cloud matching via a hybrid approach. Relevant works such as Deep Shells, NeuroMorph strictly require meshes as input. We believe that lifting this assumption is novel and non-trivial. On the other hand, mesh-based methods can be sensitive to missing parts and topological noise (see Fig. 2, 3 & Table 1 in the Supp. Mat.);

2.	Our solution to the first challenge above is novel. We consider the most relevant work as SSMSM [8], which also follows a self-supervised approach. However, [8] essentially depends on the fact that DiffusionNet can be trained jointly over meshes and the inherent vertices. In fact, [8] explicitly constructs graph Laplacian during training and inference, which is less efficient. For instance, it takes more than one minute to pre-process an input of 40,000 points with a V100 GPU. In contrast, our self-supervised scheme is without pre-process and can in principle take any pair of backbones tailored for meshes and point clouds respectively. The natural and simple teacher-student learning scheme effectively learns a point-based feature extractor by mimicking the mesh-based counterpart. Unlike [8], which is adhered to DiffusionNet, our scheme is more general, flexible, and can be extended in the future (e.g., replacing DGCNN with a rotation-invariant point feature extractor).

### **Potential contributions to the community:**

We sincerely thank all the reviewers for their constructive comments, which help us to re-think and position our approach in a clearer way. In particular, we would like to highlight the following features: 

1.	The idea of combining DFM prior and optimization-based shape registration techniques is simple and of great potential. Our approach is simply a first step along this line of exploration, which can undoubtedly benefit from the rapid advances from both directions; 

2.	Our approach is **scalable** w.r.t input size. Thanks to the fact that we non-rigidly align shapes in R^3, we can in theory freely down- and up-sample both the template mesh and the target point clouds. Note this is non-trivial for methods based on mesh or graph Laplacian, as deducing dense maps with landmark correspondences over graph structures is a difficult task on its own [a]. In Fig. 4 of our Rebuttal Mat., we show the matching results on the real scans from the FAUST challenge [b], each of which consists of around 160,000 vertices. In contrast, [8] can handle at most 50,000 vertices without running out of 32G memory on a V100 GPU. We visualize its matching results on the subsampled (to 40,000 vertices) point clouds for comparison; 

3.	As a natural follow-up, we in fact have managed to get some preliminary results on extending our pipeline to matching **partial** point clouds (see Fig. 1 in the Rebuttal Mat.). Essentially, we train a DFM tailored for partial-view point clouds generated on SCAPE_r dataset, and replace the two-way Chamfer distance with a one-way one. We also test our pipeline with noisy partial scans from [c] (see Fig. 2 in the Rebuttal Mat.). Though currently, it remains a challenge to deal with partial point clouds of arbitrary orientation and position, we believe the results sufficiently show the potential of our general scheme.

[a]: Weighted averages on surfaces, D. Panozzo, I. Baran, O. Diamanté, O. Sorkin-Hornung, SIGGRAPH 2013. [b]: https://faust-leaderboard.is.tuebingen.mpg.de [c]: http://domedb.perception.cs.cmu.edu


Meta-review: The paper tackles unsupervised deformable shape matching and registration (alignment), an open and challenging problem across several communities. It received overall positive reviews initially, identifying key strengths in the extensive and convincing experimental results, and in the quality of the writing. Some potential weaknesses were reported on the potentially limited novelty of the approach, but these were resolved with an extensive rebuttal that was evaluated positively. We believe the paper to make further steps forward for the registration problem, and its quality meets the NeurIPS standards; we recommend acceptance, with the understanding that the suggested revisions should be incorporated in the camera ready version.
Decision: Accept (poster)