Shape Registration in the Time of Transformers
Authors: Giovanni Trappolini
Luca Cosmo
Luca Moschella
Riccardo Marin
Simone Melzi
Emanuele Rodolà
Venue: NeurIPS 2021 Poster (2021)
Keywords: 3D Registration
Point Clouds
Deep Learning
Transformers
Shape Matching

Abstract: In this paper, we propose a transformer-based procedure for the efficient registration of non-rigid 3D point clouds. The proposed approach is data-driven and adopts for the first time the transformers architecture in the registration task. 
Our method is general and applies to different settings. Given a fixed template with some desired properties (e.g. skinning weights or other animation cues), we can register raw acquired data to it, thereby transferring all the template properties to the input geometry. 
Alternatively, given a pair of shapes, our method can register the first onto the second (or vice-versa), obtaining a high-quality dense correspondence between the two.
In both contexts, the quality of our results enables us to target real applications such as texture transfer and shape interpolation.
Furthermore, we also show that including an estimation of the underlying density of the surface eases the learning process. By exploiting the potential of this architecture, we can train our model requiring only a sparse set of ground truth correspondences ($10\sim20\%$ of the total points). 
The proposed model and the analysis that we perform pave the way for future exploration of transformer-based architectures for registration and matching applications. Qualitative and quantitative evaluations demonstrate that our pipeline outperforms state-of-the-art methods for deformable and unordered 3D data registration on different datasets and scenarios.

Shape registration in the time of transformers
Giovanni Trappolini
Luca Cosmo DAIS
Department of Computer Engineering
Sapienza University of Rome
Ca' Foscari University of Venice luca. cosmolunive.it
giovanni.. trappol ini Guniromal.
Luca Moschella
Riccardo Marin Department of Computer Science Sapienza University of Rome
Department of Computer Science Sapienza University of Rome
marin@di. uniromal.it
Simone Melzi
Emanuele Rodolà
Department of Computer Science Sapienza University of Rome s simone. melziellle Quniroma1. it
Department of Computer Science Sapienza University of Rome emanuele rodola@uniromal it
Abstract
In this paper, we propose transformer- procedure for the efficient registration of non-rigid 3D point clouds. The proposed approach data-driven and adopts for the first time the transformer architecture in the registration task. Our method is general and applies different settings. Given a fixed template with some desired properties (e.g. skinning weights or other animation cues), we can register raw acquired data to it, thereby transferring all the template properties the input geometry. Alternatively, given pair of shapes, our method can register the first onto the second (or vice-versa), obtaining a high-quality dense correspondence between the two. In both contexts, the quality of our results enables us to target real applications such as texture transfer and shape interpolation. Furthermore, we also show that including estimation of the underlying density of the surface eases the learning process. By exploiting the potential this architecture, we can train our model requiring only sparse set of ground truth correspondences (10 20% of the total points). The proposed model and the analysis that we perform pave the way for future exploration of transformer-ban baaee architectures for registration and matching applications. Qualitative and quantitative evaluations demonstrate that state-of-the of-the-art methods for deformable and unordered 3D data registration on different datasets and scenarios.
pipeline outperforms state-of
Introduction
Recent technological advancements of 3D acquisition pipelines have produced an abundance of available data. The direct consequence is the non-standardization of the acquisition process. Such technological democratization brings along a disparate amount of different representations, dis- cretization, arbitrary resolution. Given so, the request to align such data has become urgent. Furthermore, data-driven statistical approaches require aligned data to relate feature changes across the population, inferring underlying patterns.
The Computer Vision community has devoted an extraordinary effort in the last decades to address 3D objects analysis. common way to approach this problem to align the geometry of one
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Figure I: Some results in real application targeted by our method. In the first row, three examples of texture transfer on human shape pairs. In the second and third row, examples of interpolation between intra-class and inter-class shapes from ShapeNet (one for each row).
known shape to an incoming one. Such methodology is referred to as registration. Many different axiomatic pipelines have been proposed that address different kind of objects and domains. While many methods rely on the assumption that the shapes used in the registration task differ just by a rigid transformation, the non-rigid domain is far more complex and interesting. This category pertains organic objects (c.g.. humans, animals, internal organs), which are particularly interesting as well. Non-rigid registration aims align two geometries that may differ by bending and stretching of the geometry, which may also significantly modify its metric. This problem is even more complicated if the geometry representation given just by a sparse point cloud.
However, an emerging field merges data with classical algorithmic problems, exploiting such statistics regularization. Among the different learning approaches, recently, the use of the Attention mechanism has become significantly popular in NLP domain, being later transferred to Computer Vision applications. Such architecture is called Transformers, and they represent one of the most significant groundbreaking methodological advancement since the introduction of CNNs.
In this work, aim to let non-rigid registration meet the transformers. Intuitively, we aim transformer as a geometrical translator between two non-rigid point clouds. As the first element, modified the attention mechanism, proposing make it aware of the underlying density of the geometry. Hence, we apply such a mechanism in an autoencoder- like architecture, which takes a template point cloud as input and aims to modify its geometry fit the target point cloud.
the
The proposed method achieves better results than several state-of-the-an competitors in the shape matching task. We show results on humans case, but also inter-class objects. In this second case, our method is trained in an unsupervised manner, showing the power of our attention mechanism to infer the underlying geometry. Also, thanks to the attention mechanism, the network considers relevant for the registration. Finally, we can target texture transfer and shape interpolation showing applicability in real tasks as demonstrated in [
able to interpret what
Our contributions could be summarized follows: (a) We propose the first transformer for non-rigid registration task, showing the advantages of translation paradigm; (b) We modified the attention mechanism make it aware of the point cloud density and of the underlying geometry of shape; (c) We significantly improve the state-of-the-art performances different datasets and challenging scenarios.
All code and data is publicly available

---

2 Related work
Shape matching is a problem with tradition of decades. For a complete overview, we refer to the surveys [54] 51 below we cover the literature that more closely relates to our work.
Surface matching Early attempts to match non-rigid objects work under the assumption of near isometry. Such is the case, for instance, of blended intrinsic maps [27]. which combine multiple conformal mappings with an additional penalty to preserve local areas. Similarly, several variants and applications of the functional maps framework [41] implicitly assume near isometries by requiring
special structure of the functional representation, or by means of dedicated regularizers [42 40 18
38 49]
also designed for handling partial shapes (50 451. A common drawback to these works is that they do not disambiguate the intrinsic shape symmetries; further, the surface connectivity introduces a structural bias which may affect the performance, as recently shown in [36]. An attempt to overcome these issues was proposed in [37]. but with the extra assumption that the shapes to match are in the same pose. More recently, SmoothShells [T5] proposes an iterative algorithm recover dense correspondences by an alignment of intrinsic information. While these methods do not address 3D shape registration directly, they rely on the general idea that a correspondence can be recovered by aligning specialized, possibly high-dimensional embeddings of the shapes at hand.
Template-free registration A popular approach to solve for a matching between 3D objects is to align their geometries extrinsically via ICP-like procedures (5] matching are intimately related problems with different goals. While the matching problem aims to find a combinatorial solution, which indicates for each point its image on the target shape, registration looks for a spatial transformation of the geometry. If the two shapes have significantly different discretization, the latter problem less ambiguous than the former. ICP-based approaches iteratively solve the two subproblems in an alternating fashion, by finding a point-to to-ttoooottt correspondence and the best transformation that adheres to such correspondence. These methods do not convergence to a good solution if the input shapes are significantly misaligned. Similarly, Coherent Point Drift [39 and variants 226 22 rephrase the registration problem as an alignment of probability densities.
22 2]. In fact, registration and
Template-based registration
different family of approaches make use of a given template, which is known a priori and is possibly parametric, toward which a given input shape is to be matched. Taking as an example the case of human bodies, it is common to model the surface deformation related to the subject identity using while recent advancements in statistical data-science suggested that non-linear methods are more expressive to catch fine details of humans [48] 59 9]. Similarly, the pose can be
PCA 13 31
modeled by simple paradigms like Linear-Bi Blend Skinning 31 44 triangle deformations (3) 23), but also learning methods [59]. Efforts to register such templates to arbitrary target models have been However, the requirement of a template is
carried out extensively by the community 23 61 32
not always easy to satisfy.
Learning methods With the rise of learning methods, several attempts have been made to introduce a statistical prior to the matching and registration process. For example, several extensions have been proposed to bring the functional maps formalism into a learning paradigm (30 13 53) SmoothShells has also been extended to be data-driven [16.. The point cloud representation has received comparably less attention, mainly in rigid alignment setting 24 52 In the non-rigid domain, a seminal work is 3DCoded [19 that proposes proper registration using an autoencoder architecture. However, having a fixed template forbids inter-class operations and limits the use of the point cloud structure. The constraint of having a fixed template has been relaxed in (1 but it still requires couple of isomorphic shapes in training. Recently, it has been proposed to learn a linearly-invariant embedding separate networks and relies on simple PointNets [46]
(34, but the method requires training which are not able to catch the fine details of the objects. Finally, a recent trend in geometric deep learning suggests that implicit representations may also be used for shape matching [6].
Transformer-basee architectures Transformers have been first introduced in the context of neural machine translation by the pioneering work (56), and later procedeed to revolutionize the field of natural language processing I2 47 The success obtained in NI. inspired further work to employ transformers in computer vision 14 where they managed to outperform convolutional networks.
Exploiting the input invariance property characterising the attention mechanism, many works have
3

---

naturally extended transformers to handle point clouds 22 60, 17]. These works show promising results, but work only in the context of object classification and segmentation. Instead, [57] proposes a network to find the rigid alignment between two point clouds imitating ICP, and using transformer architecture to infer the residual term. Differently, we aim solve for non-rigid registration, which more general case.
3 Notation and general objective
3D shapes We model 3D shapes as compact 2-dimensional Riemannian manifolds M. possibly with boundary Ü.M. In the discrete setting, we represent the manifold M as an unorganized point cloud of n.M vertices embedded in R2, and encoded in coordinate matrix X.M RR=M*
Shape registration The main objective of this paper is to intro- duce a data-driven approach to perform shape registration. Given source shape S and a larger shape T, respectively represented by the sets of vertices Xs s and XT, our goal is to find a corre- sponding 3D position for each point in S the surface of T. An example is shown in the inset figure, where the underlying surface is visualized for reference only.
source
target registration
Our method does not assume the source be a fixed template, but can generalize arbitrary shape pairs S and T; this is in contrast, e.g., with [19], where the objective is to learn how to deform fixed known template into another shape.
Attention One of the most influential ideas in the recent years, attention originated in the realm of natural language processing but has since gained traction in other fields, such as computer vision and signal processing, due the vast increase in performance and interpretability exhibited in several tasks. At its heart, the attention mechanism allows learning models to encode latent relations between inputs, assigning higher importance, or "attention". to the parts deemed more relevant. In this sense, attention allows to efficiently capture context information as well as higher order dependencies.
Formally, given two generic input sequences X1 Rundd and X₂ E Im (for clarity of exposition we assume a constant embedding dimension d, but it is not a necessary assumption), the attention mechanism models the linearly encoded representation of the inputs triplets of query, key, and value matrices, respectively Q E Raxd K E Rm=xd and V mx The attention score W is then defined as W softmax (QKTd resulting in an output feature matrix A = WV. We refer to self-attention whenever X, and X₂ are the same object, and use the term Pross-attention otherwise.
and used to compute the weighted mean of the value vectors,
Method
Our method takes as input two point clouds Xy and Xs. with n. and ni points respectively, and deforms the points of the source shape Xs to fit the geometry described by the target point cloud Xy. To do so, we rely on novel attention mechanism which considers the underlying geometry conveyed by the point cloud, rather than treating the points simply as elements of set.
Surface Attention The classic attention definition, as introduced in Section 3 looks at the input data points simply elements of a set, and uses the computed attention scores perform weighted sum of the value vectors. When the value vectors represent sampling of signal over a surface, however, the natural domain for the integration should be the surface itself, of which the weighted sum is just an approximation highly sensitive to the specific surface sampling (depending, for instance, on the acquisition method).
To overcome this limitation we propose to modify the attention mechanism to consider the portion of surface represented by each point, weighting the attention score by estimated local area element.
In practice, for each point w; E X, we estimate its area contribution as the inverse of the local point density: A(X); = (Mær E X s.l. ||æ, z 2| < <r)))_ where | denotes the cardinality of a set, and is a local radius (r = 0.05 our experiments). Note that, we are not interested in the absolute value of the area elements, rather the relative contribution of each point. The surface attention

---

Pastional
LST
Probes
Atantion
Gel Adeentonn
Foruard
Nome
Enooding
Cross
Ace
Feed
Nom
Horn
XT
Xs
Figure 2: The proposed transformer-bat architecture for 3D point cloud registration. The latent probes capture the geometry of the input target shape X through the encoder layers. The resulting latent vectors drive the deformation of an input source shape Xs in the decoding layers, resulting in deformation of the points of Xg to fit the geometry of XT.
score is thus defined as:
WiJ ((X) Σ, A.....A))
As in the classical attention, output features are computed as WV. Figure 3 shows that the surface attention mechanism results in a more stable localization of the attention scores across different samplings of the surface.
In our architecture, each time attention is computed point clouds,
such formulation.
Architecture
The architecture we propose, portrayed in Figure 2 is an iteratively conditioned autoencoder. The two main ingredients are an encoder, that maps target point cloud X into a latent space L,7; and a decoder, that deforms source point cloud Xs to resemble the geometry of the target point cloud Xy.
The encoder draws inspiration from 225 featuring a set of learnable parameters, we refer to them as latent probes LP. It presents multiple layers of cross attention which iteratively condition the latent probes LP with the embedding of Xy. After each conditioning, the resulting latent space is further transformed by layers of self attention, feed forwards and residual connections. The encoder output a set of latent vectors LST containing relevant information collected from the target point cloud Xy.
The decoder is analogous the encoder but the relationship between the latent space and the point cloud in the cross attention is reversed. That is, the embedding of the source point cloud Xs is transformed and iteratively conditioned with the latent space L.Sr produced by the encoder. This procedure induces deformation of Xss that, after a final MLP layer, aligns to the points of Xy.
Training Given training dataset equipped with ground-truth correspondence, we train our net- work in supervised setting. Starting from a set of shapes in correspondence, we minimize the reconstruction error using the standard reconstruction loss:
(Xu, = ||X - D(Xs: E(X+))
Furthermore, in case a ground-truth correspondence is not available, our network can be trained in an unsupervised way using the Chamfer distance, between XT and Y7 D(Xs, E(XT)), as defined below:
Σ min Is SEXT
+11 KEY
Σ KEYT
min $11
(3)
Testing At test time, network register point cloud to another instantaneously, with single forward pass. In the following experiments that involve the computation of matching, the correspondence be obtained just by looking for the Euclidean nearest-neighhor between X and the output D(Xs, E(XT)) in the 3D space.
Refinement The peculiar structure of our architecture allows to refine the output during the testing procedure. This is achieved by minimizing the energy function chamfer(X7, D(Xs. E(XT))) with
5

---

Table 1: Ablation study. We report the relative average matching error (lower is better) with respect the baseline architecture used in all the experiments (for which the error is set to 1).
Test Set
# of latent space vectors
# E/D layers
latent space dimension
64
128
SURREAL
1.00
1.00
0.93
0.97
0.91
1.04
1.00
0.93
1.36
1.00
1.45
FAUST
1.00
1.01
1.24
1.30
1.29
1.45
1.00
1.22
0.82
1.00
0.94
Average
1.00
1.01
1.08
1.13
1.10
1.24
1.00
1.07
1.09
1.00
1.19
Classic attention
Surface attention
Original
(7K)
Norual deviation (IK) (183
Quadricc
Figure 3: Comparison between the classic (left) and surface (right) attention mechanism behavior with differently sampled surfaces. The surface attention is more stable across different sampling strategies. The two attention mechanism have been trained separately, so there no correspondence on the attention localization between the two. Surfaces are shown just for visualization purposes.
Figure 4:
The colormap shows the attention value given to each point of the shape by each input latent vector (only of 32 shown) at the first (left half) and last (right half) layers of the decoder.
respect to the latent vectors L.ST. In practice, we use the latent vectors produced by a single forward pass as initial guess for I.S,, and minimize the previous energy function using Adam optimizer.
As we show in Section 5 this step can significantly improve the shape matching results. We remark that this is possible only thanks to the registration formulation of our approach. Shape alignment in 3D space can be optimized continuously, while a point-to-point assignment is non-differentiable due to the combinatorial nature of the problem.
Experiments & Results
We evaluate the effectiveness of our architecture on a number of challenges. We begin by analyzing the key components our model, motivating our architectural choices through ablation studies. Finally, present our results in the context of matching, registration, and class registration.

---

5.1 Experimental settings
Training data For all our experiments in the humans domain, we trained our method on the same shapes from the SURREAL dataset (53) used in 34 It consists of 10000 point clouds for training. Each point cloud has 1000 points, which simulate significantly sparse sampling of the original shape. During training we augment the data by randomly rotating shapes along the second axis. We also trained our model on the ShapeNet dataset [8]. This dataset is composed of 16881 I point clouds representing 3D shapes from 16 different objects categories(from chairs to airplanes). that also do not share a ground truth correspondence. Hence, we trained our method in an unsupervised manner for this particular data. We sample 1024 points for each object and use the same train/test split as in [58]
We also compare our model on other human datasets, thus assessing the ability to generalize to data of the training distribution. A popular dataset to analyze real identities and poses is FAUST (7]. which is composed by ten subjects in ten different poses. We also used 1000 points version of it, which we refer as FAUSTIK. To simulate the noise produced by 3D acquisition pipeline, we considered the same data from 34 in which the points are perturbated by a Gaussian noise. Also, we challenge our method on SHREC' 19 36). Such dataset composed by 44 shapes which have different connectivities, poses and densities. In all the experiments we refer to our method as Our.
In our comparisons we train our model for 10000 epochs using Adam optimizer (2 We use 32 latent probes of dimension 64, and 8 layers for both the encoder and the decoder.
Competitors We consider 3DCoded [9] (3DC) as our principal competitor. Similarly to us, it aims at deforming shape into another using an autoencoder architecture. However, it assumes one of the two shapes to be a predefined template, limiting its generalization capability. Further, we consider the Linearly-Invarian embedding in which the shapes can be aligned by a linear transformation. It is based on PointNet, considered the Geometric Functional
Embedding approach of [4] (LinInv), which learns an high-dimensional
and do not exploit any local structural mechanism. Finally, Maps definition [13), using DiffusionNet [53] as feature extractor (DiffNer). Similarly to (34), it learns to embed each shape point into common higher- digeee-dimmnsional..
Both our method and 3DC can improve the registration with a post-processing refinement, we refer to these as 3DCR and Ourn-
5.2 Ablation and analysis
Here we justify our choices on hyperparameters through an ablation study, and we present some insight on the key properties of our method with an in depth analysis.
Ablation study To investigate the different components of the proposed architecture, we run a batch of experiments in a reduced version of the SURREAL dataset, using the first 1000 shapes for training, and a different set of 2700 shapes for testing. We also test on FAUSTIK.
We report the results in Table I. We test for three possible ablations: number of latent space vectors, their dimension, and the number of encoder-decoder blocks. We remark that the while SURREAL share discretization and density with the training data, FAUSTIK K does not. This may explain the difference performance we observe, and their almost inverse relationship. In fact, it seems that a low number of latent vectors tend to overfit the specific sampling seen during training, while a higher number seems to provide better generalization. A similar behavior can be observed for the other hyperparameters considered. Finally, to reach a good trade off between bias and variance, we choose the configuration that produces the minimum mean error on these two evaluation sets.
Analysis of the Surface attention Quantitative and qualitative results showing the importance of the surface attention mechanism compared to the standard point attention. This novel kind of attention we propose has the merit of being much more agnostic the particular discretization and density of the given point cloud. We can observe this behavior clearly in Figure 3 Here we visualize the attention across different densities and discretization strategies and two different settings, one with surface attention and one with regular attention. With the regular attention mechanism the part of the point cloud attended shows erratic behavior, with different intensities and often even different part that gets attended. Surface attention completely solves this issue, enabling the architecture to achieve
7

---

we can Figure 5: Interpolation examples of two shapes. Left group: left and right-most shapes are the registration of source shape S to two different target shapes Ti, T2, the central shape is obtained passing as input to the decoder latent vectors obtained by linearly interpolating the latent vectors of the two target shapes. In the second and third groups the interpolation is performed only on a subset of the vectors: we fix the latent vectors of the first shape which attention, on the last layer of the decoder, focuses respectively on the upper part of the body (center) and on the legs (right), while interpolating the remaining ones.
greater generalization capacity and enjoying increased robustness, and allowing us to decrease the eтror on the full version of FAUST of more that 50%.
Multivariate Latent Space
Inspecting the cross attention of the decoder layer we can seize the impact of the latent probes in our learning. In the top row of Figure 4we can see how the attention in the first layer of the decoder captures global information of the shape, while in the last layer (bottom row), attention puts its focus on small details of the shape.
The possibility of directly visualizing attention maps and the multivariate nature of the latent space permits further analysis. In Figure in particular, we explore the structure of our latent space and how attention influences it. Similarly recent non-rigid shape autoencoders [ 35 4
manipulate the latent vectors to generate new shapes. Specifically, we grab the two latent spaces output of the encoder by registering a source shape from the left triad, the left. most and the right most shape represents the registration of S to Ti and To respectively; the one in the middle is obtained by linearly interpolating the two latent spaces. In the middle triad we add twist to this procedure, namely we locate the latent vectors attending the most to the upper body of the shape and keep them fixed. A similar procedure is undertaken in the right triad, but this time focusing on the legs. From this experiments, it follows that our latent space not only linearly navigable, meaning that linear interpolation the shapes encoded the latent space preserve reasonable semantics, but also, and most interestingly might say, attention characterizes this space and directly allow for meaningful alteration, or preservation, of selected chosen characteristics.
two different target shapes Ti, T2. Starting
5.3 Results and Applications
In this section we present results and application of our method. In particular, we show state of the art performance on the shape matching task well shape registration.
Matching One of the task we consider is that of matching. Given two generic point clouds we want to find correspondences between them. Our model approach this task in natural and elegant way, by registrating one shape onto the other. It becomes trivial then to obtain correspondences through nearest point search. Results are reported in Table 2 Our method consistently outperforms the state of the art by solid margin. Furthermore, we notice that our method is endowed with much greater ability to generalize. This can be noted on the SHREC' 19 dataset, as visualzied in Figures The quality of matching enable to achieve high quality texture transfer shown in Figures and
Template Registration A classical problem in Computer Graphics is to register a given template, usually a triangular or polygonal mesh to some acquired point cloud. This setup a special instance our method, in which the point cloud to be given as input to the decoder remains constant. Regarding our competing methods, even if DiffusionNet and LinInv are not proper registration algorithm, we can move template point to the corresponding point (as found by the matching algorithm) on the input point cloud. This partially explains why, even though performing the worst overall, achieve lower chamfer distance, since their error is somewhat bound. On the other hand, 3DC is trained

---

Table 2:
Comparison of the average geodesic eтrоr on different datasets. FAUST(IK) is obtained from FAUST sampling points, FAUST(1K-nok)) obtained FAUST(IK) but perturbing each vertex with Gaussian noise. FAUST 7 and SHREC19 (36] have very different sampling densities, with point clouds ranging from
200 thousand points.
Method
FAUST
FAUST(IK)
FAUST((k-noise)
SHREC19
3DC
0,0776
0,0542
0,0712
0.2138
DiffNet
0.0656
0,0534
0,0985
0,1509
LinInv
0.0942
0.0471
0.0618
0.1284
Our 0.0513
0.0419
0.0510
0.0802
3DCR
0.0485
0.0367
0.0526
0.1935
Oure
0.0369
0.0263
0.0410
0.0615
100
3DC
3DC
Linlnv
Our
Our
0.16
Source
3DCR
DiffNet
LinIny
Our
Figure 6:
Comparison of different methods on SHREC19 [36]. Left: Each curve shows the
percentage of points (y-axis) with at most geodesic error (x-axis). Right: Qualitative comparison. From left to right, the source shape S, the ground truth color transfer to the target geometry T, the results of the competitors and result. The color transfer predictions paired with the corresponding error visualizations, from white (error=0) to black (error>0.75).
Table 3: Comparison on the registration task on FAUST (7]. Left: Each curve shows the percentage of points (y-axis) with at most that geodesic error (x-axis). Right: Table showing for each method: the mean geodesic eтror (MGO) of the resulting matching; the Chamfer distance, the maximum and the mean Euclidean distance (Max EU, Mean EU) between the registered template and the target.
Method
Chamfer
Max EU
Mean EU
MGO
3DC
0.0409
0.2231
0.0723
0,0463
DiftNet
0.0164
1.2942
0.1023
0.0761
3DC
LinInv
0.0177
0.3314
0.1044
0.0692
3DCR
Our
0.0333
0,2299
0.0650
0.0434
DifNNet
Linlnv
3DCR
0.0214
0.1705
0.0445
0.0293
Our
Our
Our
0.0129
0.1626
0.0306
0.0275
0.16
exactly this fashion. Note also that 3DC is the only method that sees the template shape during the training phase. Even though we train our method on different task, we manage to improve on the state the art as can be seen in Table without the need for any, altough possible, fine-tuning.
5.4 Unsupervised Registration and Interpolation
One of our main advantages is that we do not require template. Fixing common template is not trivial, if not possible at all, when dealing with very different objects. To show ability of our method of dealing with this challenging scenario we trained model to register pair of shapes belonging to possibly different object categories of ShapeNet, using the chamfer loss defined in Section

---

Source
3DC
3DCs
DiffNet
LinInv
Our
OurR
Figure 7: Qualitative comparison of texture transfer on the SHREC19 [36 dataset. From left to right, the source shape S, the texture transfer to the target shape 'I of competitors and our results.
We show in Figure (2nd and 3rd row) two interpolation sequences between two airplanes and between chair and table, showing that our method is able to register different objects preserving meaningful correspondence, represented by similar colors. The interpolated reconstructions are obtained by embedding the outermost shapes in the latent space through the encoder and then using linearly interpolated latent vectors and left-most shape as input to the decoder.
Conclusion
We propose the first transformer based architecture to tackle the problem of non-rigid registration. We introduce a novel surface attention mechanism better suited to exploit the local geometric priors of the underlying structure. Our method reaches state of the art performance in shape matching and shape registration without assuming any fixed template, and generalizes also to different and complex geometries, e.g. handling multiple classes of ShapeNet [8] simultaneously. The attention mechanism at the core of our architecture has the potential to enforce local control of the interpolation, as seen in Figure
Limitations and future works. Our method shares a common drawback with most transformed- based architectures, requiring long training and post-processing time due to the nature the refine- ment procedure. Further investigation needed to explore the possibility to introduce additional priors on the attention to force a semantically meaningful localization and interpolation behavior.
Acknowledgment
This work is supported by the ERC Grant No. 802554 (SPECGEO) and the MIUR under grant "Dipartimenti di eccellenza 2018-2022".
10

---

References
[1] Allen, B., Curless, B., Popović, Z.: The space human body shapes: reconstruction and parameterization from range scans. ACM transactions graphies (TOG) 22(3), 587-594 (2003)
[2] Amberg, B., Romdhani, S., Vetter, T.: Optimal step nonrigid iep algorithms for surface regis- tration. In: 2007 IEFE conference on computer vision and pattern recognition, pp. 1-8. IEEE (2007)
[3] Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., Davis, J.: Scape: shape completion and animation of people. In: ACM SIGGRAPH 2005 Papers, pp. 408 416 (2005)
[4] Lumentado-Armstuenn
T., Tsogkas, S., Jepson, A., Dickinson, S.: Geometric disentanglement for generative latent shape models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8181-8190 (2019)
[5] Besl, P.J., McKay, N.D.: A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 14(2),
(Feb 1992)
[6] Bhatnagar, B.L.,, Sminchisescu, C., Theobalt, C., Pons-Moll, G.: Loopreg: Self-supervised learning of implicit surface correspondences, pose and shape for 3d human mesh registration. In: Advances Neural Information Processing Systems (NeurIPS) (December 2020)
[7] Bogo, F., Romero, J., Loper, M., Black, M.J.: FAUST: Dataset and evaluation for 3D mesh registration. In: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). IEEE, Piscataway, NJ, USA (Jun 2014)
[8] Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al.: Shapenet: An information-rich 3d model repository, arXiv preprint arXiv:1512.03011 (2015)
[9] Cheng, S., Bronstein, M., Zhou, Y., Kotsia, I., Pantic, M., Zafeiriou, S.: Meshgan: Non-linear 3d morphable models of faces. arXiv preprint arXiv: 1903:1100.100000 10384 (2019)
[10] Cosmo, L., Norelli, A., Halimi, O., Kimmel, R., Rodolà, E.: Limp: Learning latent shape repre- 2020: 16th European
sentations with metric preservation priors. In: Computer Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16. pp. 19-35. Springer (2020)
[11] Cosmo, L.,, Rodolà, E., Masci, J., Torsello, A., Bronstein, M.M.: Matching deformable objects in clutter. In: International Conference on 3D Vision (3DV), pp. 1-10. IEEE (2016)
[12] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers). pp. 4171-4186. Association for Computa- tional Linguistics, Minneapolis, Minnesota (Jun 2019). https://doi.org/10.. 11110//0100000111111111111122220 N
https://www.aclveb.co wwwprrrggngnngggggrrrrrrreeoe/ee
[13] Donati, N., Sharma, A., Ovsjanikov. M.: Deep geometric functional maps: Robust feature learning for shape correspondence. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)
[14] Dosovitskiy, A., Beyer, I.., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale (2020)
[15] Eisenberger, M., Lahner, Z., Cremers, D.: Smooth shells: Multi-scale shape registration with
functional maps. In: Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition. pp. 12265-1222 (2020)
[16] Eisenberger, M., Toker, A., Leal-Taixé, L.,, Cremers, D.: Deep shells: Unsupervised shape
correspondence with optimal transport. arXiv preprint arXiv:2010.15220 (2020)
[17] Engel, N., Belagiannis, V., Dietmayer, K.: Point transformer (2020)
[18] Ezuz, D., Ben-Chen, M.: Deblurring and denoising maps between shapes. Computer Graphics
Forum 36(5), 165-174 (2017)

---

[19] Groueix, T., Fisher, M., Kim, V.G., Russell, B.C., Aubry. M.: 3d-coded: 3d correspondencess by deep deformation. In: Proceedings of the European Conference on Computer Vision (ECCV).
pp. 230-246 (2018)
[20] Guo, M.H., Cai, J.X., Liu, Z.N., Mu, T.J., Martin, R.R., Hu, S.M.: Pet: Point cloud transformer. Computational Visual Media pp. 187-99 (June 2021). https://doi..or/10...00 1ttp:/oo.....11.00000/0010995500 1017000190900001--29575
5
[21] Halimi, O., Imanuel, I., Litany, O., Trappolini, G., Rodolå, E., Guibas, L.,, Kimmel, R.: Towards precise completion of deformable shapes. In: European Conference on Computer Vision (ECCV). pp. 359-377 (2020)
[22] Hirose, O.: A bayesian formulation of coherent point drift. IEEE Transactions on Pattern Analy- sis and Machine Intelligence pp. 1-1 (2020). https://doi.orge 10.g//0011111 109/TPAMM.2020299777 2977199670.277777777
[23] Hirshberg, D.A., Loper, M., Rachlin, E., Black, M.J.: Coregistration: Simultaneous alignment and modeling of articulated 3d shape. In: European conference on computer vision. pp. 242-255. Springer (2012)
[24] Huang, X., Mei, G., Zhang, J.: Feature-metric registration: A fast semi-supervised approach for robust point cloud registration without correspondences. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)
[25] Jaegle, A., Gimeno, F., Brock, A., Zisserman, A., Vinyals, O., Carreira, J.: Perceiver: General perception with iterative attention (2021)
[26] Jian, B., Vemuri, B.C.: Robust point set registration using gaussian mixture models. IEEE transactions on pattern analysis and machine intelligence 33(8), 1633-1645 (2010)
[27] Kim, V.G., Lipman, Y., Funkhouser, T.: Blended intrinsic maps. ACM Transactions Graphies (TOG) 30(4), 79 (2011)
[28) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015), (tt01)..tttt://///. http:://rxii..oo hate//bs///ee /142
6980
[29] Li. H., Sumner, W.W. Pauly, M.: Global correspondence optimization for non-rigid registration of depth scans. Computer graphies forum 27(5), 1421-1430 (2008)
[30] Litany, O., Remez, T., Rodolà. E., Bronstein, A., Bronstein, M.: Deep functional maps: Structured prediction for dense shape correspondence. In: Proceedings of the IEEE International
Conference on Computer Vision. pp. 5659-5667 (2017)
[31] Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: Smpl: A skinned multi- person linear model. ACM transactions on graphies (TOG) 34(6), 1-16 (2015)
[32] Marin, R., Melzi, S., Rodolå, E., Castellani, U.: Farm: Functional automatic registration method for 3d human bodies. Computer Graphics Forum 39(1,, 160-173 (2020)
[33] Marin, R., Melzi, S., Rodolà, E., Castellani, U.: High-resolution augmentation for automatic
template-based matching human models. In: 2019 International Conference on 3D Vision
(3DV). pp. 230 239. IEEE (2019)
[34] Marin, R., Rakotosaona, M.J., Melzi, S., Ovsjanikov, M.: Correspondence learning via linearly- invariant embedding. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances Neural Information Processing Systems. vol. 33, pp. 1608-1620. Curran Associates,
Inc. (2020)
[35] Marin, R., Rampini, A., Castellani, U., Rodola, E., Ovsjanikov, M., Melzi, S.: Instant recovery of shape from spectrum via latent space connections. In: 2020 International Conference on 3D Vision (3DV), pp. 120-129. IEEE (2020)
[36] Melzi, S., Marin, R., Rodolà, E.,, Castellani, U., Ren, J., Poulenard, A., Wonka, P., Ovsjanikov, M.: Matching Humans with Different Connectivity. In: Biasotti, S., Lavoué, G., Veltkamp, R.
(eds.) Eurographies Workshop on 3D Object Retrieval. The Eurographies Association (2019)
[37] Melzi, S., Marin, R., Musoni, P., Bardon, F,, Tarini, M., Castellani, U.: Intrinsic/extrinsic embedding for functional remeshing of 3d shapes. Computers & Graphics 88, 1 12 (2020)
12

---

[38] Melzi, S., Ren, J., Rodolà, E., Sharma, A., Wonka, P., Ovsjanikov. M.: Zoomout: Spectral upsampling for efficient shape correspondence. ACM Transactions on Graphics (TOG) 38(6), 155 (2019)
[39]
Myronenko, A., Song, X.: Point set registration: Coherent point drift. IEEE transactions on pattern analysis and machine intelligence 32(12), 2262-2275 (2010)
[40] Nogneng, D., Ovsjanikov, M.: Informative deseriptor preservation via commutativity for shape
matching. Computer Graphics Forum 36(2), 259-267 (2017)
[41] Ovsjanikov, M., Ben-Chen. M., Solomon, J., Butscher, A., Guibas, L.: Functional maps: flexible representation of maps between shapes. ACM Transactions on Graphics (TOG) 31(4), 30:1-30:11 (2012)
[42] Ovsjanikov, M., Corman, E., Bronstein, M., Rodolà, E.,, Ben-Chen, M., Guibas, L., Chazal, F., Bronstein, A.: Computing and processing correspondences with functional maps. In: SIG- GRAPH 2017 Courses (2017)
[43] Pais, G.D., Ramalingam, S., Govindu, V.M., Nascimento, J.C., Chellappa, R., Miraldo, P.: 3dregnet: A deep neural network for 3d point registration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 7193-7203 (2020)
[44] Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A.A., Tzionas, D., Black, M.J.: Expressive body capture: 3d hands, face, and body from a single image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10975-10985 (2019)
[45] Postolache, E., Fumero, M., Cosmo, L., Rodolà, E.: parametric analysis of discrete hamil- tonian fuunctional maps. In: Computer Graphics Forum. vol. 39, pp. 103-118. Wiley Online Library (2020)
[46] Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 652 (2017)
[47] Radford, A., Narasimhan, K.. Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)
[48] Ranjan, A., Bolkart, T., Sanyal, S., Black, M.J.: Generating 3D faces using convolutional mesh
autoencoders. In: European Conference on Computer Vision (ECCV), pp. 725-741 (2018)
[49] Ren, J., Melzi, S., Ovsjanikov, M., Wonka, P.: Maptree: Recovering multiple solutions in the
space of maps. ACM Trans. Graph. 39(6) (Nov 2020)
[50] Rodolå, E., Cosmo, L., Bronstein, M.M., Torsello, A., Cremers, D.: Partial functional cor-
respondence. In: Computer Graphics Forum. vol. 36, pp. 222-236. Wiley Online Library (2017)
[51] Sahillioglu, Y.: Recent advances in shape correspondence. The Visual Computer 36(8), 1705- 1721 (2020)
[52] Sarode, V., Li, X., Goforth, H., Aoki, Y., Srivatsan, R.A., Lucey, S., Choset, H.: Pernet: Point cloud registration network using pointnet encoding. arXiv preprint arXiv: weXiv:1908.000 1908.07906 (2019)
[53] Sharp, N., Attaiki, S., Crane, K., Ovsjanikov, M.: Diffusion is all you need for learning on surfaces. arXiv preprint trXiv:2002.00888 (2020)
[54] Van Kaick, O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. Computer graphies forum 30(6), 1681-1707 (2011)
[55] Varol, G., Romero, J., Martin, X., Mahmood, N., Black, M.J., Laptev, I,, Schmid, C.: Learning from synthetic humans. In: CVPR (2017)
[56] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Proc. NIPS (2017)
[57] Wang, Y., Solomon, J.M.: Deep closest point: Learning representations for point cloud regis- tration. In: Proceedings of the IEEE/VV International Conference on Computer Vision. pp. 3523-3533 (2019)
[58] Wang, Y., Sun, Y., Liu, 7.,, Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph cnn for learning on point clouds. ACM TOG 38(5), 146 (2019)
13

---

[59] Xu, H., Bazavan, E.G., Zanfir, A., Freeman, W.T., Sukthankar, R., Sminchisescu, C.: Ghum ghuml: Generative 3d human shape and articulated pose models. In: Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 6184 6193 (2020)
[60] Zhao, H., Jiang, L., Jia, J., Torr, P., Koltun, V: Point transformer (2020)
[61] Zuffi, S., Black, M.J.: The stitched puppet: A graphical model of 3d human shape and pose. In: Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp.
3537-3546 (2015)
14


Reviews:
[{"review_id": "shcf3Lutjxk", "reviewer": "Reviewer_VAnV", "summary": "This paper proposes a Transformer-based point cloud registration scheme. Given two point clouds, the target and an initial latent feature go through the encoder of a Transformer-like network, and the resulting latent vector is combined with the source and goes through the decoder to yield a transformed point cloud (that resembles the target). In the Transformer-like structure, iterative cross attention is used, which is inspired by [21]. There are some other details to handle the particular problem efficiently, e.g., density aware attention for handling the density differences in point clouds. The decoder and the loss function can also be used to refine the result with further optimization. The experiments show that the proposed method mostly achieves state-of-the-art results (sometimes with large gaps) for well-known datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "+- Somewhat derivative, but appropriately designed: The main ideas of the paper are inspired by other works, such as the recently popular Transformer structures and especially the iterative cross attention structure [21]. Architecture-wise, one might say that the work is somewhat derivative from [21]. But at the same time, some additional effort (e.g., density-adaptive attention) is appropriately made to handle the problem effectively. The method yields state-of-the-art performance, and sometimes, the gaps are quite large, which suggests that the proposed method is indeed effective.\n\n** Why the Transformer for cross-modality?: The iterative cross attention inspired by [21] is originally for cross-modality. Point clouds are rather homogeneous in this sense. In my understanding, the authors are using this to easily pass on the latent feature (which can be smaller and contain essential information) from the target to the source. This can be indeed an efficient way to handle the problem. But at the same time, I think that a regular Transformer-like structure could also be used and there is a chance that the performance can be better. Why is the proposed method a better choice? More discussion is needed.\n\n** How is the initial latent feature (latent probe) handled?: One thing that is not clearly explained in the paper is how the latent probe, which is one of the inputs of the network, is handled. The authors have provided a code as supplementary material, but providing this in the paper or in the supplementary document will be good for completeness.\n\n[After rebuttal]\nI have read all the other reviews and the authors' responses. All my previous questions have been answered. I stay with my original score.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "** What happens if we use somewhat heterogeneous point clouds for source and target? In my understanding, most of the experiments are about more-or-less homogeneous data. Discussing this for exploring the limitations of the method can be interesting.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "rodpPsM13rg", "reviewer": "Reviewer_eCbC", "summary": "The authors propose a novel transformer based approach for 3D shape registration for non-rigid objects.\nTransformers have been very successful in language modelling and recently also in vision applications. This paper introduces transformers for 3D shape registration which is an interesting step forward.\nThe approach also outperforms competing baselines.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "+ The approach is technically sound and makes intuitive sense.\n+ Quantitative results look good.\n+ Authors will release their code and data, which is appreciated.\n\nI have minor questions regarding the work:\n\n1. Surface attention, Eq. 1. The whole buzz around the transformer idea is centered around the fact that we no longer need to provide inductive bias to the network regarding how different components in the input are related. Eg: with CNNs the inductive bias takes the form of predefined kernel shapes that indicate to the network that some correlation exists within the receptive field of the kernel. Whereas with transformers no such assumption needs to be made. Despite this fact, why does the proposed method need explicit normalization for point density? Since SURREAL is a synthetic dataset, why can't it be augmented such that network learns to generalise to different sampling strategies. This is interesting because despite the proposed normalisation, there is a performance gap between performance on SURREAL and FAUST, which authors attribute to different surface distributions.\n\n2. Authors mention that their method can be used to obtain correspondences between two shapes. Why is this not evaluated on the FAUST challenge?\n\n3. Table 2: Are the baselines trained on the same data as the proposed method?\n\n4. Authors currently restrict their problem setting to low resolution pointclouds (1000 points as input). How do the transformers perform with denser inputs?\n\n5. Current limitations stated in the conclusion section are generic. Since this is a novel direction, it will be helpful to add a dedicated limitation and future works section with more detail. It would be good to also discuss other directions in 3D vision where transformer architectures would be beneficial.\n\nOverall, I like the idea of using transformers for 3D shape registration and the authors also show good quantitative results.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors very briefly mention some limitations of their work but this can be further improved (see comments above).", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "YETvlApWCE", "reviewer": "Reviewer_NiyT", "summary": "Summary:\n\na) This paper proposes a transformer-based procedure to register non-rigid 3D point clouds or shapes;\nb) Given a source point cloud, this paper aims to deform its coordinates to align to target point cloud;\nc) Authors modifies the attention mechanism to make it aware of the point cloud density, by adding weight to the original attention score;\nd) Experiments show the advantages of this paper over previous methods.\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\ni) Strong points.\n\na) The use of the Transformer architecture for the task of non-rigid registration is new,  and seems to be the first as claimed by the authors;\nb) For complete shape scans, it makes sense to modify the attention mechanism to make it aware of the point cloud local density;\nc) Overall, this paper is well-written, and easy to follow.\n\nii) Weak points.\n\na) Architecture.\n\nAs noted by authors in line 109, the deep closest point [50] also use a transformer to register rigid shapes. In light of this,  it is highly beneficial if the author can provide the  following comparisons:\n\n  * Trim the architecture of [50], add MLP to \\Phi_X or \\Phi_Y (Figure 2 in [50]) to estimate the deformed points. \n\n  * Use the above baseline to compare with your proposed architecture.\n\nb) Partial scans.\n\nThis paper lacks experiments of partial--to-partial registration of non-rigid point clouds. Adding noise in experiment is good (Line 207).  I would expect this partial-to-partial registration are more  realistic in many real-world applications.   How will your method handle  significant portion of non-matchable outliers? \n\nFurthermore, the above experiment would also check the effectiveness and robustness of the proposed modified attention mechanism (Equation 1).\n\nc) Quantitative comparisons between original attention and modified attention.\n\nGiving qualitative comparisons in Figure 3 is good. However, I would like to see full quantitative comparisons besides the 50% claim on line 245.\n\nd) Computation time.\n\nPlease specify the details of computation times, and compare them with respect to other methods (in Table 2).\nAs indicated on Line 301-302, the proposed method is computationally expensive/time consuming.  Why do you claim it is an \"efficient registration\" method (c.f. Line 1) ? \n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes. ", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "BA3LXsFbeKc", "reviewer": "Reviewer_pM1i", "summary": "This paper introduces the use of transformers architecture for shape matching and the method is applied to 3D point clouds. The main innovation with respect to standard attention architecture on point clouds is the weight introduced in the attention mechanism which accounts for the local density of the shape. Experiments show a clear gain of the method compared to other state of the art method, both quantitatively and qualitatively.", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "On the positive side, the proposed method seems to outperform state of the art methods in the field, although I am not an expert in this field.\nThe method does not require the use of a template which makes the method directly applicable.\n\nMy main concern is the writing style of the paper which is too loose and not accurate enough in my opinion. The first two sections are nicely written overviews discussing common generalities on attention and shape registration. Beginning of section 4 loosely describes the architecture with inaccuracies in formula (1). It is not possible to reimplement the method from reading out the main text.\nThe authors do not discuss the problem of long training time of transformers in their situation.\n\nQuestions: \n- Is the surface attention a particular case of classic attention in a particular formulation? Is it an inductive bias that could be learnt from enough data?\n- How is the bottleneck of attention (quadratic in the input) dealt with in the context of hundreds of thousand of points ? it is not detailed enough.\n\nComments: \n- it seems that there are multiple typos in formula 1. The index summation $t$ is not present elsewhere in the formula, and the notations $\\exp^{(x)}y$ and $\\exp^{xy}$ are quite unusual. I would suggest to replace it with more standard notations.\n- The running times and computational resources are difficult to find in the main text although the authors replied positively that it is stated in section 5. I have hard time to find it. It is not discussed in this section and only touched upon in the conclusion. It needs more discussion.\n- Chamfer loss is said to be defined in section 4, but it is a pointer to another publication.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are not discussed enough, in particular the training time of transformers and ways to get around it.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers agree that the idea of using a transformer-type architecture for (non-rigid) 3D point cloud registration is interesting and novel. While reviewers have raised questions or pointed out concerns about certain issues (e.g., being derivative), I am of the opinion that the authors sufficiently addressed those points in their answers (despite little or no serious discussion happening). I do, however, encourage the authors to follow the reviewer's advice on improving the writing style in some places and including all required clarifications as promised in the rebuttal. 
Decision: Accept (Poster)