SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Regarding the registration optimization, in Eqn.(12) of the main text, we empirically set  $\lambda_{\mathrm{cd}} = 0.01$ ,  $\lambda_{\mathrm{corr}} = 1.0$ ,  $\lambda_{\mathrm{arap}} = 20$  in Stage-I and  $\lambda_{\mathrm{cd}} = 1.0$ ,  $\lambda_{\mathrm{corr}} = 0.01$ ,  $\lambda_{\mathrm{arap}} = 1$  in Stage-II. For  $\alpha$  in Eqn.(9) of the main text, we set  $\alpha = 0.2$ . Note that the roles of  $\lambda_{\mathrm{cd}}$  and  $\lambda_{\mathrm{corr}}$  are reversed from Stage-I to Stage-II. In fact, in Stage-I, the initial shapes are likely to differ by large deformation, therefore it is more reliable to estimate correspondences via the learned embeddings. Once Stage-I converges, we empirically observe that the deformed source shape admits a similar pose with the target (see the video in Supplemental Material), therefore it is reasonable to further deform the source by extrinsic embeddings, i.e., Chamfer distance on the coordinates. We stress that both losses are critical in both stages and perform the following ablation experiment on setting  $\lambda_{\mathrm{cd}}$  and  $\lambda_{\mathrm{corr}}$  to be zero independently in the two stages. Tab. 6 reports the scores. We find that most of the time, turning off leads to worse results, especially when the test shapes are heterogeneous (see, e.g., SHREC07-H).
Table 6: Mean geodesic errors (×100) of Ours on setting  $\lambda_{cd}$  and  $\lambda_{corr}$  to be zero independently in the two stages
SCAPE_r	SHREC19_r	SHREC07-H
λcd=0 in Stage-I	2.5	5.5	6.9
λcorr=0 in Stage-II	2.7	5.6	6.8
Ours	2.6	5.1	5.9
B.1 Further Qualitative Results
Visualization of Registration Results In this section, we show more qualitative results of registration regarding different templates and different test shapes in Fig 5. Our method is capable of registering shapes with significant deformations across different templates.
Figure 6: We present the qualitative results of testing the TOPKIDS trained on SCAPE_r. Our method demonstrates robustness in regions of mesh topology with noise, maintaining stable performance.
Figure 7: We showcase some examples of failed registrations.
Qualitative Comparison in TOPKIDS: In Fig. 6, we qualitatively visualize maps obtained by different methods tested in the TOPKIDS benchmark. It is obvious that our results outperform all the competing methods. Especially, a purely intrinsic method, AttentiveFMaps seriously fails due to the topological perturbation within the mesh structure.
Failure Cases: On the other hand, we also identify some failure cases, which are shown in Fig. 7. The two cases both suffer prominent heterogeneity between the template and the target. In the left case, where an ideal reference is given, we adapt it in registration instead of enforcing the hard highly non-isometric deformation with templates from SCAPE_r or FAUST_r (see Sec. B.3). While in the right case, it is not obvious how to choose an easier template for establishing correspondences between the two shapes, we, therefore, settle down at the setting described in Sec.4.2. It is worth noting that, though, we can easily identify that the problematic regions, i.e., head, and hands from the visualization, which is unavailable for methods only return maps.
Templates: In the Fig. 8 we visualize the templates used in different experiments. We select a pose close to the T-pose from the training sets of SCAPE_r, DT4D-H, and FAUST_r, respectively, as our templates.
Figure 8: We respectively use template from SCAPE (left), CRYPTO (middle), and FAUST (right) in our experiments.
B.2 Further Quantitative Results
Trained on Large-Scale Dataset: We have followed the setting of SSMSM [8] to train our feature extractor on the SURREAL 5k dataset [f] and to test it on the FAUST_r, SCAPE_r, and SHREC19_r datasets. We select one training shape from SURREAL 5k as the template in registration. As shown in Table 7, our method outperforms the competing non-rigid point cloud matching methods by a noticeable margin. We also refer the readers to Table 3 in [8] for a complete table including methods utilizing mesh input, and we highlight that our method is even comparable with the latter. In particular, ours vs. SOTA of mesh-based techniques over the three test datasets: 3.2 vs. 2.3, 2.5 vs. 3.3, 4.6 vs. 4.7. That is, we achieved the best overall result in two of the three test datasets.
Table 7: Mean geodesic errors  $\left( {\times {100}}\right)$  trained on SURREAL
Method	Train Test	FAUST_r	SURREAL SCAPE_r	SHREC19_r	|Datal|
DiffFMaps	26.5	34.8	42.2	230k
DPC	13.4	15.8	17.4	230k
SSMSM	3.5	3.8	6.6	5k
Ours-SURREAL	3.2	2.5	4.6	5k
Trained on Animal Dataset: We also perform experiments on the remeshed SMAL dataset. We first randomly generate 5000 shapes with SMAL model to train the alignment module. Then we train a DFM with the remeshed SMAL dataset. The template shape is 'dog_06' from the training set. The quantitative results are reported in Table 8. Remarkably, our method achieves more than a  $40\%$  performance improvement than the second-best baselines.
Table 8: Mean geodesic errors  $\left( {\times {100}}\right)$  on SMAL_r.
TransMatch	DPC	DiffFMaps	NIE	SSMSM	Ours
SMAL_r	20.7	21	17.1	16.3	7.2	4.2
B.3 Robustness
Robustness with respect to Input Perturbation: We evaluate the robustness of different methods with respect to noise and orientation perturbations in Tab. 9. In the former, we perturb the input point clouds with Gaussian noise. In the latter, we randomly rotate the input point clouds within  $\pm 30$  degrees along  $x, y, z$ -axis. Thanks to our optimization-based approach and orientation regressor, in both tests we achieved the best accuracy, but also the lowest standard deviation over three runs of experiments.
Robustness with respect to Missing Parts: In Tab.1 of the main text, we report shape matching results on SHREC19_r dataset, where the  $40^{\mathrm{th}}$  shape is commonly excluded in the test phase of prior works [29; 8]. The main reason is that this shape is incomplete, consisting of two main connected components (see the abdomen of the source shape in Fig. 9). Therefore, for intrinsic methods depending on mesh eigenbasis, such disconnectedness makes spectral alignment extremely difficult, if not impossible. On the other hand, extrinsic methods show a more robust outcome. In particular, we show in Fig. 9 that our framework can reasonably deform the template mesh (from SCAPE_r) to
Table 9: Mean geodesic errors  $(\times 100)$  on under different perturbations. Noisy PC means the input point clouds are perturbed by Gaussian noise. Rotated PC means the input point clouds are randomly rotated within  $\pm 30$  degrees along  $x, y, z$  -axes respectively. The standard deviation value is shown in parentheses.
Unperturbed PC	Noisy PC	Rotated PC
DiffFMaps [35]	12.0	14.9(2.57)	26.5(3.35)
NIE [24]	11.0	11.5(0.32)	19.9(1.29)
SSMSM [8]	4.1	5.4(0.11)	9.2(1.01)
Ours	2.6	2.9(0.06)	3.6(0.96)
the incomplete point cloud with a significantly isolated portion, which leads to superior matching results compared to the baselines.
Figure 9: A challenging case from SHREC19_r dataset, see the text for details.
Robustness of Orientation Regressor: To demonstrate that our learned orientation regressor (see Sec.3.1 of the main text), we visualize its outcome on shapes from SHREC07-H, which manifests notable diversity in sampling density, size, and initial orientation.
Figure 10: Bottom row: The input shapes of SHREC'07 exhibit various orientations; Top row: The shapes aligned by our orientation regressor. Note that there exists significant differences in sampling density.
B.4 Scalability
Our approach is scalable w.r.t input size. Thanks to the fact that we non-rigidly align shapes in  $\mathbb{R}^3$ , we can in theory freely down- and up-sample both the template mesh and the target point clouds. Note this is non-trivial for methods based on mesh or graph Laplacian, as deducing dense maps with landmark correspondences over graph structures is a difficult task on its own. In Fig. 11, we show the matching results on the real scans from the FAUST challenge, each of which consists of around 160,000 vertices. In contrast, [8] can handle at most 50,000 vertices without running out of 32G memory on a V100 GPU. We visualize its matching results on the subsampled (to 40,000 vertices) point clouds for comparison.
SSMSM
Source (40k verbs)
Target (40k verbs)
Ours
Source (160k verts)
Figure 11: Qualitative results on matching MPI-FAUST raw scan data. Our method can match the point clouds at the original resolution, while SSMSM [8] can not.
Target (200k & 180k verts)
B.5 Run-time Analysis
In this part, we analyze the time cost of our optimization-based registration module. In particular, we separate it into two main parts, initialization, and iterative optimization. For the former, we in general fix a source mesh as a template, and then we construct a deformation graph on it as described in Sec.3.2 in the main text. Since the deformation graph can be pre-computed and cached for further use of the same template, we put more emphasis on the latter.
In particular, we perform registration between the template shape from SCAPE_r and the 20 test shapes therein and compute the average run-time of each core step in the optimization. As illustrated in Tab. 10, our method achieves an average optimization time of approximately 13.4 seconds per shape, with about  $90\%$  of the total time being dedicated to Stage-I. Analysis depicted in Fig. 12 reveals that the most time-consuming operation is the ADMM backward step, while the combined time cost of our correspondence filtering and updating operations accounts for less than  $3\%$  of the overall execution time. Though the reported run-time is not quite satisfying, we emphasize that to evaluate maps across  $n$  point clouds, we only need to register them to the same template shape  $(O(n))$ , which is different from the pair-wise  $(O(n^{2}))$  mapping and post-processing approaches [7].
Table 10: Average time and iteration cost for each shape.
Total	Stage-I	Stage-II
Time cost(s)	13.4	12.0	1.4
Iterations	1274.	1130.	144.
Figure 12: Run-time decomposition of our registration module.