Adaptive Multi-step Refinement Network for Robust Point Cloud Registration
Authors: Zhi Chen
Yufan Ren
Tong Zhang
Zheng Dang
Wenbing Tao
Sabine Susstrunk
Mathieu Salzmann
Venue: Accepted by TMLR ()

Abstract: Point Cloud Registration (PCR) estimates the relative rigid transformation between two point clouds of the same scene. Despite significant progress with learning-based approaches, existing methods still face challenges when the overlapping region between the two point clouds is small. In this paper, we propose an adaptive multi-step refinement network that refines the registration quality at each step by leveraging the information from the preceding step. To achieve this, we introduce a training procedure and a refinement network. Firstly, to adapt the network to the current step, we utilize a generalized one-way attention mechanism, which prioritizes the last step's estimated overlapping region, and we condition the network on step indices. Secondly, instead of training the network to map either random transformations or a fixed pre-trained model's estimations to the ground truth, we train it on transformations with varying registration qualities, ranging from accurate to inaccurate, thereby enhancing the network's adaptiveness and robustness. Despite its conceptual simplicity, our method achieves state-of-the-art performance on both the 3DMatch/3DLoMatch and KITTI benchmarks. Notably, on 3DLoMatch, our method reaches 80.4% recall rate, with an absolute improvement of 1.2%.

Published in Transactions on Machine Learning Research (02/2025)
Adaptive Multi-step Refinement Network for Robust Point Cloud Registration
Zhi Chen* HUST
ahennhhut.edd..cm
Yufan Ren" IVRL, EPPL
yufan.ren@epffl.ch
Tong Zhang IVRL, EPFL
tong.zhang
Zheng Dang CVLab, EPFL
zheng. Wepfl.ch
Wenbing Tao HUST
wenbingtao@hhust.cde
Sabine Süsstrunk IVRL, EPFL
sabine.usstruukBeepf..cc
Mathieu Salzmann CVLab, EPFL
wadhieu..almmaanneeppph
Reviewed on OpenReview: https: // openreview. net/ forum? id- M3P
Abstract
Point Cloud Registration (PCR) estimates the relative rigid transformation between two point clouds of the same scene. Despite significant progress with learning-based approaches, existing methods still face challenges when the overlapping region between the two point clouds small. In this paper, we propose an adaptive multi-step refinement network that refines the registration quality at each step by leveraging the information from the preced- ing step. To achiove this, we introduce a training procedure and a refinement network. Firstly, to adapt the network to the current step, utilize a generalized one-way at- tention mechanism, which prioritizes the last step's estimated overlapping region, and we condition the network step indices. Secondly, instead of training the network to map either random transformations or a fixed pre-trained model's estimations to the ground truth, we train it on transformations with varying registration qualities, ranging from ac- curate to inaccurate, thereby enhancing the network's adaptiveness and robustness. De- spite its conceptual simplicity, our method achieves state-of-the-art performance on both the 3DMatch Dallatcch3DDLohee M3toh and KITTI benchmarks. Notably, on 3DLoMatch, our method reaches 80.4% recall rate, with an absolute improvement of 1.2%. Our code is available at htttps:/giithub... whw.:o///hhhhhhhhh/hhea
1
Introduction
Point Cloud Registration (PCR) estimates the optimal rigid transformation between two point clouds. The classical Iterative Closest Point (ICP) algorithu (Besl & McKay 1992) pairs points two overlapping point
"Equal contribution

---

Published in Transactions on Machine Learning Research (02/2025)
clouds and minimizes the pairwise Euclidean distances. ICP employs iterative optimization process and, due to its non-convex nature, requires precise initialization. Some ICP variants (Yang et al. 2015b Campbell & Petersson 2016) introduce global optimizations to improve robustness, at the cost of low efficiency (Fu
ct al. 2023).
Recently, learning- beanenng-based methods, particularly feature-matching techniques, have become predominant PCR (Choy et al. 2019 Huang et al. |021] Qin et al.
2022 Yang et al.) 2022] Yu et al. 2021 2023b
Ao
et al.
2023). These methods employ neural networks to extract features of key points (down-sampled point
cloud), establish point correspondences (Yu et al. 2021 Qin et al.
2023
Yang et al.
2022
and utilize robust
ostimators to determine the relative transformation (Choy et al. 2020 Bai ct al.
2021
Fischler & Bolles,
1981 Chen et al.] 2022ap. While these learning- based methods have shown impressive performance, they encounter challenges in low overlapping scenarios, typically < 30%. The primary issue is during the feature extraction phase, similar structures and shapes may produce ambiguous key point features. Consoquently, featuro-based methods may crroncously match key points that do not belong to the overlapping region, thereby rntrodugng inaccuracies, as is depicted in Fig. (a) + The source point cloud, colored in yellow, represents the left side of kitchen scene, while the target point cloud, in blue, represents the right side of the room. The overlapping region between these two point clouds is small, as can be seen on the right-most of Fig. ! (c), where the two point clouds are almost perfectly aligned. As shown by the red lines in Fig. [ (a), the seminal work Geo ReeoTransformer(Qin et al. 2022) erroneonsly matches one point in the source point cloud to points outside the overlapping region.
The recent work PEAL (Yu et al., 2023b) introduces a post-processing method to refine the registration results. It utilizes the overlap estimation of a pre-trained PCR network, i.c., Geo Thoorranffrmeree certormmeriinnnne et al. 2022), auxiliary information (also referred to as prior), and incorporates this information in the refinement network to improve the estimate. By employing the same network repeatedly, PEAL establishes multi- step approach (see Fig. (b)) and demonstrates enhanced performance. However, despite being utilized in multiple steps, the refinement network is only trained for a fixed mapping from prior to ground truth, not considering the fact that the priors become increasingly accurate Uhroughout the refinement steps.
In this work, our intuition is enhance multi-step registration by making the refinement process adaptive. Our first contribution is a training procodure that, rather than mapping random transformation pre-trained model's estimate (i.e., GeoTrranffrmmr) to the ground Iruth, trains the network with priors of varying accuracy levels, from accurate to poor alignments. We design degradation function that interpo- lates between the prior and ground truth, enabling a smooth transition from accurate to inaccurate priors. Adjusting the interpolation ratio lets us manipulate the quality of the priors. Furthermore, we propose to explicitly condition the network on the current refinement step index, with each step's network trained sepa- rately, reflecting the accuracy of the prior. Moreover, we propose a generalized version of one-way attention focusing on the overlapping regions in both the source and target point clouds. Considering the non-linearity of the transformation space, we define the degradation function in the spherical linear space (Yew & Lee
2022).
Our extensive experiments demonstrate our method's effectiveness; we achieve state-of-the-arl recall rates on the 3DMatch/3DDD Mathh and KITTI benchmarks. Notably, our method yields 80.4% recall rate on the 3DLoMatch benchmark, with an absolute improvement of 1.2% compared with the previous state of the art. In summary, the effectiveness of our work stems from the following contributions:
We introduce a novel adaptive multi-step refinement method tailored for the low-overlap challenge in PCR, achieving state-of of-tt---rttttttto registration recall rates on the 3DMatch/3DLoMate and KITTI benchmarks.
We propose a training procedure that makes the network adaptive to the improving prior quality during refinement and consequently improving registration.
We propose generalized one-way atlention mechanism that focuses on the pre-aligned overlapping regions of the two point clouds, facilitating improved feature learning.
We manually adjust the position and rotation of the two point clouds for better visualization of matches, same for Fig. (b).
2

---

Published in Transactions on Machine Learning Research (02/2025)
Source point cloud
Target point cloud
K times
Estimated overlapping
Model
Correct match Wrong matches
RE: 14.14 1 TE 46 8cm (Final step)
(a) Single-step matching w/o overlapping prior (GeoTransformer)
(b) Iterative single model matching w/ overlapping prior (PEAL)
RE:53.0 TE:287 6em
6E 3E:35
RE.2.9 TE:5. Iem
Model K
Model
Model
Step (GeoTransformer)
Step
Final Estimation
(c) Per-step aligned point cloud visualization (Ours)
Illustration of the proposed framework. (a) Matching visualization. Similar structures and shapes result in similar features, lcading to the erroncous matches by the GooTransformer, where one key point in the source point matched lo key points in the target point cloud outside the overlapping region, indicated by red lines. The ground truth match is shown as a green line. (b) Utilizing rough overlapping information, indicated as the shaded regions, PEAL successfully filters out many incorrect matches outside the overlap area, visualized similarly to (a). (c) Per-step aligned point cloud visualization. refinement approach employs multiple models, each progressively trained with Our adaptive multi-step accurate priors as the model index increases. By adapting to the progressively increasing accuracy of priors during the refinement steps, our method surpasses PEAL in performance. RE and TE stand for rotation error (L) and translation error (L) (Best viewed on screen when zoomed in).
Figure 1:
2 Related Work
Traditional point cloud registration (PCR) methods estimate the rigid transformation (rotation and translation) between two point clouds using geometric or appearance information. The most widely used PCR methods are Iterative Closest Point (ICP) (Besl & McKay 1992) and its variants (Chen & Medioni 1992: Segal al., 2009). Despite their widespread application in various real-world scenarios due to their straightforward formulation, ICP algorithms are sensitive to initial conditions and are prone to convergence at local minimum. To address this, some approaches, such as Go-ICP (Yang et al. 2015b) and GOGMA (Campbell & Petersson 2016), integrate Branch-and- Bound optimization into ICP. However, the global optimization process inherent. these methods significantly increases computational time, thus limiting their practicality.
An alternative methodology in PCR is the feature- feature-matching- based approach. The methods in this category form correspondences by creating and matching point feature descriptors, and then robust estimators to remove outlicrs and recover the relative pose. Based on how fcature descriptors are obtained, traditional feature matching can be calegorized into Local Reference Frame (LRF) based and non-LRRFbaseed descrip- tors. LRF- based descriptors initially typically begin with covariance analysis (Novatnack Nishino, 2008 Tombari et al.] 2010)) to create a local coordinate system, which then serves as the reference for generat- ing feature descriptors. Techniques for deriving descriptors from LRF include coordinate plane projection Zaharescu al. 2009) and rotational projection statistics (Guo et al. 2013 By contrast, non-LRFF-based
3

---

Published in Transactions on Machine Learning Research (02/2025)
(1) Feature Extraction
(2)
Feature Regression
(3) Coarse Matching
(4) Fine Matching
(S) Estimation
Injecting overlapping prior
X2
MiLA
MitA
Milh
Registration Result
P
1E0D
MILE
MIA
MHAA
MHA
MHA
Pateh-wise Correspondence map
KPCinn
Potches
Viniiti Ceea Creas-ittatio
Chocoeay Choss No0eattoo
Super
Self-Artention
Cross-Amention
MEDI
Points
Points
Figure 2: Refinement Network step index k. (1) For two overlapping point clouds P and Q, we extract super points (F$, FE) and fine points (F$, F5) using KPConv. (2) We derive matchable superpoint features using Transformers, where MHA stands for multi-head attention. Instead of standard self and cross-attention, we employ our generalized one-way attention, integrating the overlapping information from the previous step - (3) We compute the patch-wise correspondence map. (4) We propagate patch-wise correspondences to fine correspondences. (5) We obtain the final estimate through a robust estimator. (Best. viewed on a screen when zoomed in)
descriptors directly extract features from the raw point cloud. The most notable non-LRF-basse met are histogram-basee computing shape (Frome et al., 2004p geometry (Rusu et al. 2009 Wu et al. 2017) histograms, and integrating these histograms from key points and their neighbors to form descriptors.
Learning-basee based PCR methods utilize the powerful data-driven ability of deep neural nel for PCR.
Several learning-based methods (Choy et al. 2019 Ao et al. 2021 | design local extractors to enhance the feature representations. The recent integration
Transformer architectures (Vaswani et al., 2017) has
inspired methods (Huang
2021
Yu et al.
2021
Qin et al.
2022
Yang et al.
2022 Chen et al. 2023a
Yu et al. 2023a Ao et al. 2023) that incorporate attention mechanisms within
PCR networks for robust
results. In addition, some methods Choy et al. 2020 Bai ct al.
2023b Zhang al. 2023) introduce robust est.imators to improve the alignment. from feature correspondences. Other studies
2021 Chen et al.
2022a
propose deep learning-bass based end-to-end registration frameworks Aoki et al. 2019 Wang & Solomon, 2019
Yew & Lee 2020 Yuan et al. 2020 Yew & Loe 2022 Chen et al., 2022b which streamline the registration
process. However, these approaches ollen struggle in challenging scenarios, such as low-overlapping point clouds. Our work multi-step registration pipeline, augmenting registration with auxiliary information of estimated overlapping regions. Furthermore, tailor the registration models each refinement step, significantly improving the recall rate and robustness.
3 Method
Point cloud registration defined as the process of aligning two partially overlapping point clouds. These are denoted as the source point cloud (P {p. R NJ) and the target point cloud (Q { R
1, 1...., M)). The goal is to accurately recover the ground-truth relative rigid transformation, 2 (R, t). which aligus their overlapping region.
We propose a novel multi-step registration framework, F = [fald (.2. } = L...,.K,, to adaptively refine the registration as
X = f
(1)
where each step fs utilizes the transformation estimate from the precoding step as auxiliary information.
It important to note that when all the steps are identical (fi f2 fK = f), the framework F simplifies to repetitive iterative method, akin to PEAL (Yu al. 2023b
4

---

quat quati (t)) Published in Transactions on Machine Learning Research (02/2025)
Leariing largel
Augmented data
Learned nelwork
0
Prior Estimation (reoTransormmer
PEAL
Ground truth
Ours
Figure 3: ID toy example. PEAL trains the network map prior ground truth. We propose to sample new prior transformations by interpolating between the GeoTransformer estimate and the ground truth. Upper row: Illustration of the point cloud aligument transitioning from the Transfosorre estimate to the ground truth. (Best viewed on screen when zoomed in)
The proposed framework for point cloud registration consists of three parts. 1) Prior (input alignment) creation (Section 3.1,, which uses a degradation function, denoted as = [T)"", to generate a series of rigid transformations with varying levels of accuracy. Here, E [1, ..... 7] the accuracy level, that higher indicates more accurate prior, and T is the number of the accuracy levels. 2) Training (Section 3.2), which employs the obtained rigid transformation, represented by Xx-1 <(X", k), to train the registration models F across different step indices. 3) Inference (Section 3.3 which utilizes the adaptive refinement network to obtain the final estimate &.
3.1 Creating Point Cloud Prior Transformations
As illustrated in Fig. PEAL Yu et al. 2023b) trains the network, f to map prior to ground truth. This, however, has limitation: The model does not adjust to the improved quality of the prior during the multi- step refinement process. To adapt the multi-step refinement network to each step, one should specify the refinement f& that each step should perform.
Consequently, we introduce a degradation function e(X"", (T) that generates point cloud prior transformations based on the accuracy level 7. Notably, the number of accuracy levels T is much larger than the number of models K, producing diverse samples. The degradation function should meet two criteria. Firstly, higher accuracy level should correspond to more accurate prior transformation, result. from the previous step. Secondly, the initial prior should be coarse alignment, as we do not expect the network to learn to utilize totally random overlapping information.
each step aims to refine the
Taking these requirements into account, we propose to sample priors for each accuracy level 7 by interpolating between the Geo Transforreer prior and the ground truth. Given the non-linearity of the transformation space, we define the degradation within the spherical linear space (Yew & Lee 2022
Formally, define the interpolation, we first define discrete time step range [0, 1, .... T7. Then, define the interpolation for both rotation and translation with an accuracy level T E [0, 1,.,, T] as
R, = Sleep R prior R.g
(2)
t, (1 a,) - tprior + a, tgt
where indicates the accuracy level of the generated prior aligument and ar 7 Slerp( is the quat are the quaternion represenlation of the prior and ground-truth rotation; purm and tx are the prior and ground-truth translation. As 7 decreases, a, approaches 0, bringing R,, t, closer to the prior rigid transformation. Conversely, larger 7 shifts R,, t, towards the ground truth. The Slerp function facilitates a smooth transition rotation between these states.
spherical linear interpolation (Shoemake) 1985); R and R prior
5

---

Published in Transactions on Machine Learning Research (02/2025)
Source point cloud
Target point cloud
Overlagping prior
FA
F2
Key Value
Key Value
II
H
MIL
MHA
Miz
MHA
Query
Oulput
Fp
Query
Ouiput
(a) way-aattttnniin
(b) Cross-one way-attention
Figure 4: One-way attention (taking the source point cloud P as an example). (a) The self-one- self-onewway-atiene way-attentionn uses the feature of anchor points (FF) Key and Value in the attention operation to compute the feature of non-anchor points, reducing the impact of non-overlapping regions on the learned features (F ). (b) The proposed cross-one-way way-attntonnn utilizes the anchor point features in the target point cloud (FQ) as Key and Value, further considering interactions between the two point clouds in the wn--aat-tttn
3.2 Adaptive Multi-step Refinement
Network backbone. We adopt Geo'Transformer QQn et al. 2022 as the backbone of the refinement process, illustrated in Fig. 2 The network comprises five stages which we now describe. (1) Feature extraction employs KP-Conv (Thomas al., 2019 point-based convolutional network to extract features. This stage gradually down-samples the point cloud in its encoding part to extract sparse points, called superpoints (PS, 0*), and features (FS, F$). It then contains decoding part, which up-samples the superpoints to yield the final fine points (PF, Q²)) and features (FF, F5). Using the point-to-node strategy (Li et al. 2018), each fine point is associated with a superpoint, and each superpoint. forms a point patch. (2) Feature interaction leverages multi-heat attention operations on the two sets of superpoints to learn matchable features, with L interleaved self and cross-attention layers (sce Fig. 2 The self-attention and rros-atteention module contain both vanilla attention and the proposed one-way attention (Fig. 4 (3) Coarse matching determines the correspondence map through feature similarity, matching the superpoints by finding the top-k entries. (4) Fine matching refines this aligument at the point level within the patch correspondences established by coarse matching, resulting in precise correspondences. (5) Rigid transformation estimation utilizes the fine correspondences with a robust estimator, such as RANSAC (Fischler & Bolles 1981 or LGR (Qin et al. 2022), to estimate the rigid transformation.
Registration process. After generating the prior data T different levels of accuracy, we linearly divide this data into K (K < T) groups (see Fig. 3). Each group will be utilized to train specific refinement network, while all models sharing a Tranffooorerrr backbone. For each accuracy level, the model index is k= ((7TT) * K7, where [.] denotes the ceiling function.
Moreover, unlike the standard GeoTransformer, which processes only two point clouds, our refinement net- work integrates the previous estimates. More specifically, we incorporate overlapping information through a one-way attention mechanism, akin the masking technique prevalent Transformer models (Cheng 2022 One-way attention gives higher weights to specific regions determined by priors. PEAL Yu eet al. b023b introduces one-way altention PCR and augments the features by focusing on the overlapping regions in the point cloud itself. Inspired by this, propose more general one-way attention that attends to both intra and inter-point cloud overlapping regions, arguing that the two overlapping point sets could be complementary, as visualized in Fig. This allows our model to implicitly take the registration result of the last step into account.
ctt al.
To be specific, the input of the one-way attention is the feature of the points in the middle of the network, and one-way attention attempts to establish interactions between point features to update the features. To formulate the one-way attention, given transformation estimate X, (Rr,47) determine the

---

Published in Transactions on Machine Learning Research (02/2025)
overlapping points in source point cloud P
Px = (P,) max(O 2,)) > 03, 03 0
where (P()) transforms the patches corresponding to P by X2, and ((:-)) computes the overlap between two point patches. We call these points anchor points. The anchor points in the target point cloud Q are defined similarly. We name the remaining points non-anchor points, denoted as P and O
Then, we perform one-way attention (OA) between the anchor points and the non-anchor points shown in Fig. 4 Formally, it can be written
OA(FA FM) FN MIP(FNN Value"),
w = sof softmax BealftmaaQuurrr"" (Query (K
where FA and J. are the features of the anchor points and the non-anchor points, respectively. Here, we omit the source point and target point subscripts for brevity. Query", Key and Value produced by applying linear transformations to F and F., Instead of taking the anchor points and non-anchor points from the same point cloud, as in one way self-attention, we propose to take the anchor points and non-anchor points from the counterpart point cloud, resulting in one-way cross-attention (see Fig. 4
3.3 Inference
After obtaining the K models trained on the data groups with progressively higher accuracy, use these models for inference. In our method, the initial model's rigid transformation is set to a prior rigid trans- formation from GeoTransSormer (Qin et al.] 2022). Importantly, this initial rigid transformation ion can be any prior or even pure random noise, and we validate the effect of the imput in the experiment section and supplementary material. During the multi-step optimization process, the rigid transformation output by the current model serves as prior for the subsequent model. Finally, the result of the last model is considered the output of our method.
3.4 Loss Functions
To train the refinement model, we adopt the same training losses as GroTransformer QQin et al. 2022 which consist of an overlap-aware circle loss foe and point matching loss Cp: i.c.,
L= + Cx-
(5)
The overlap-aware circle loss extends the circle loss (Sun et al. 2020 emphasizing the positive samples with high overlap. The point-matching loss formulated as negative log- liceliooldood loss on the fine point correspondences.
4
Experiments
4.1
Implementation Details
For fair comparison with the baselines, we adhere the same implementation details and experimental setup as Georransfrmmrr (Qin al. 2002).. We preprocess the point clouds by down-sampling them using voxel grids, setting the voxel size to 2.5cm for the EDMatch/3DLoMatee benchmarks and 30cm for the KITTI benchmark. Given that the KITTI benchmark typically contains more points per point cloud than DTMaathh/DLLMMateee
we follow Geo Transforrmr'ss approach by using 5-stage KP-Conv for KITTI and a 3-stage KP-Conv for 3DMatch 3DMatchh3DLLoMili /3LooMach. For the data degradation process, we set the accuracy levels T to 1000 to create more diverse data, and this data is divided into K = 5 groups for training models (Section 3.2 We train the KITTI model for epochs and the JDMatch/3DLoMMM taking around 48 hours on a single V100 GPU. Within each refinement network, the attention module is applied three times (i.e.,
models for 20 epochs,
in Fig. 2

---

Published in Transactions on Machine Learning Research (02/2025)
3DMatch
RR (%)
TR. (%)
f
FMR (%)
1
Method
Reference
5000
2500
1000
500
250
5000
2500
1000
500
250
5000
2500
1000
500
250
Deseriptor
FOGF
ICCV2019
Chov et al.
2019
$5.1
84.7
83.3
81.6
71.4
56.8
54.1
48.7
42.5
34.1
97.4
97.3
97.0
96.7
96.6
D3Fest
CVPR2020
Bai et al
A720
91.6
84.5
83.4
62.4
77.9
39.0
38.8
40.4
41.5
41.8
95.6
95.4
94.5
94.1
93.1
SpiuNet
CVPR2021
al.
88.6
86.6
85.5
83.5
70.2
47.5
11.7
39.1
33.9
27.6
97.6
97.2
95.5
91.3
YOHO
ACM MM2022
Wang
2022
90.8
90.3
89.1
88.6
84.5
64.4
00.7
55.7
46.4
41.2
98.2
97.6
97.5
97.7
96.0
REGTR
CVPR2022
Yew & Lee
2022
92.0
Predator
CVPR2222
Huang et al
2021
89.0
89
90.6
88.5
86.6
38.0
58.
57.1
51.1
49.3
96.6
96..6
96.5
96.3
96.5
Transformer- bused
CoFINet
NeurPS20221
Yu et al
2021
89.3
88.9
98.4
87.4
87.0
49.8
51.2
51.9
52.2
52.2
98.
98.
98.1
98.2
98.3
GeoTrranformer
CVPR2022
Qin al
92.0
91.8
91.8
91.1
91.2
71.9
75.2
76.0
82.2
85.1
97.6
OIF-Net
NeurIP22022
Yang el
20124
924
91.9
91.8
92.1
91.2
62.3
65.2
66.8
67.1
67.5
98.1
98.1
97.9
98.4
96.4
RolTY
CVPR2023
Yu cl ml
202
91.9
91.7
91.8
91.4
91.0
82.6
82.8
83.0
83.0
83.0
98.0
98.0
97.9
96..0
97.9
BUFFER
CVPR2023
Ao dl.
20/23
92.9
PEAL
CVPR2023
21233
94.4
94.1
94.1
93.9
93.4
81.3
87.9
89.2
98.5
98.6
98.7
SIRA-PCR
ICCV2023
21/25
93.6
93.9
93.9
92.7
92.4
70.8
78.3
83.7
85.9
87.4
98.2
98.4
918.4
98.5
98.5
DHEPOR
Arxiv2024
Wi mi ml.
2023
94.2
55.4
97.4
Ours
94.4
94.3
94.5 94.0 93.9
75.0
81.6
86.3
88.2 89.4
98.3
98.33
98.3
98..3 98.3
3DLoMatch
RR (%) 1
IR.
(%)
f
FMR (%) 1
Meshod
5000
2500
1000
500
250
5000
2500
1000
500
250
5000
2500
1000
Deseriptrr
FOGF
ICCV2019
Choy es
2019
40.1
41.7
38.2
35.4
26.8
21.4
20.0
17.2
14.8
11.6
76.6
75.4
74.2
71.7
07.3
D3Fest
CVPR2020
2120
37.2
42.7
16.9
13.8
39.1
13.1
14.0
11.6
15.0
67.3
06.7
67.0
66.7
66.5
SpinNet
CVPR2021
Ao of nl.
4721
59.8
54.9
48.3
39.8
26.8
20.5
19.0
16.3
13.8
11.1
75.3
74.9
72.5
70.0
63.6
YOHO
ACM MM2022
Wang et al 2022
65.2
65.5
63.2
56.5
48.0
26.9
23.3
22.6
18.2
15.0
79.4
78.1
76.3
73.8
69.1
REGTR
CVPR2022
Yew & Lee
2022
61.88
Predator
CVPR2021
Huang et
2021
59.8
61.2
62.4
00.8
58.1
26.7
28.1
28.3
27.5
25.8
78.6
77.4
76.3
75.7
75.3
Transformer-h based
CoFINet
NeueIPS2021
67.5
66.2
61.2
61.0
25.9
26.7
26.8
26.9
83.1
83.3
82.6
GeoTransformer
CVPR2022
Qin el al.
2122
75.9
718
74.2
74.1
73.5
43.5
45.3
46.2
52.9
57.7
88.3
88.6
88.8
88.6
86.3
OIF-Net
PS2222
Yang ct 21122
76.1
75.4
75.1
74.4
73.6
27.5
30.0
31.2
32.6
33.1
84.6
85.2
85.5
86.6
87.0
RoTTr
CVPR2023
el ml
20233
74.7
71.
74.8
71.2
73.6
54.33
54.6
55.1
55.2
55.3
89.6
89.6
89.5
89.4
89.3
BUFFER
CVPR2222
el al.
21255
71.8
PEAL
CVPR2023
Yu et ml
21/44
79.2
79.0
78.8
78.5
77.9
49.1
54.1
60.5
63.6
65.0
89.1
89.2
89.0
89.0
88.8
SIRA-PCR
ICCV2023
( hen ct ml
2723
73.5
73.9
73.0
73.4
71.1
43.3
49.0
55.9
58.8
60.7
88.8
89.0
88.9
88.6
87.7
DiEPCR
Anxiv2024
Wu et al.
2023
73.4
22.5
80.6
Ours
80.4
79.2 78.8
49.7
55.4
61.8
64.5
68.2
86.3
85.9
86.0 86.1
Table 1: Results on indoor datasets. The results of the compared methods are taken from their paper. The best scores are in red, the second best in yellow.
4.2 Indoor Scenes: 3DMatch & 3DLoMatch
Datascts. Following previous research (Yu et al. 2021 Qin et al. 2022 Yang et al. 2022), we evaluate our
method using the 3DMatch [Zeng et al. 2017) and Match (Huang el al. 2021) benchmarks. These two datasets are created from 62 RGB-D scenes, with 46 scenes used for training, 8 for validation, and for testing. A key difference is that 3DLoMatch features a lower overlapping ratio (10% 30%) than 3DMatch, which has overlapping ratio greater than 30%.
Metrics. We adopt the evaluation metrics from Predator's Huang et al. 2021 p, reporting Registration Recall (RR), Feature Matching Recall (FMR) and Inlier Ratio (IR) across varying numbers of correspondences. RR measures the percentage of point cloud pairs aligned within a specified RMSE (Root Mcan Square Error), i.e., RMSE < 0.2m). TR quantifies the ratio of correspondences that fall within a residual threshold under the true transformation, while FMR assesses the percentage of point cloud pairs with an IR exccoding 5%.
Registration results. Tab. I compares our method with recent deep learning beasedng-based baselines, including 4 local descriptors (FCGF, D3Feat, SpinNet, and YOHO), and 10 Transformer- based methods (REGTR, Predator, CoFiNet, GeoTransfrrmer, OIF-Net RoITR, BUFFER, PEAL, SIRA-PCR, and DiffPCR). Wo adhere to the evaluation protocol of Predator (Huang el al., 2021). sampling 5 different numbers of cor- respondences (5000, 2500, 1000, 500, 250) for each method and evaluating the results. As REGTR and BUFFER directly output the final rigid transformation, they are excluded from correspondenco metrics. PEAL can use either 2D or 3D information as prior. Since the official 3DMatch and 3DLoMatch datasets do not give the 2D information, PEAL first generates the 2D prior. Since the remaining methods do not need
8

---

Published in Transactions on Machine Learning Research (02/2025)
3DMatch
3DLoMatch
RR
IR
FMR
RR
IR
FMR
Geu'Transformeer
92.5
70.9
96.2
74.0
43.5
87.1
PEAL-3D
91.2
73.3
98.5
79.0
19.0
87.6
Ours-SD
94.4
73.4
98.3
80.0
49.6
87.0
PEAL-20
91.3
72.4
99.0
81.2
15.0
91.7
Ours-DD
95.3
73.9
98.5
50.4
87.7
Table 2: Registration results with the LGR estimator. The best scores are in red, the second best in yellow,
the 2D information, in Tab. 1 report the results of PEAL using 3D prior input for fair comparison.
We compare PEAL-2D with our method in a separate table (Tab. 2b. Note that PEAL and our method both adopt iterative optimization, and we use the same number of steps (set to 5) in our experiments.
2021 which directly measures the In PCR, RR is the most critical metric (ang et al. 2022 Huang et al.
registration success rate (RMSE smaller than a threshold). As shown in Tab. 1 our method achieves the highest RR across different numbers of correspondences. Our method outperforms all one-pass techniques, which provide an estimate with a single-step network. Compared with the multi-step method PEAL, our method still achieves notably better results. We attribute this improvement to our adaptive refinement design and our one-way-cros--tteniio mechanism. Notably, our method with five iterations achieves an RR of 80.4%, which surpasses PEAL's performance with ten itcrations (78.8%) on 3DLoMatch. Our method also oulperforms DisPCR (Wu et al. 2023), which is a Diffusion-based PCR network conditioned on the matching matrix, especially on the 3DLoMatch dataset.
In terms of IR, we achieve the best performance in most configurations and slightly worse than one bascline, RolTr, with 5000 correspondences. It means our approach can establish better correspondences through multi-step optimization, which is the key foundation for the improvement of the final registration perfor- mance, as reflected by RR. Feature Matching Recall (FMR) measures whether correspondences' inlier ratio is above threshold, indicating the percentage of point clouds that are likely to be registered. Notably, FMR does not test. if the actual transformation can be determined (Huang et al. 2021). Tn terms of the FMR, benchmarks. Given our higher
our technique is inferior to some baselines on the JDMatch/3DLoMattee registration success rate, i.c., RR, we speculate that the worse FMR is caused by point cloud pairs where both the baseline methods and ours fail to register, where our method yields lower IR.
Geo Transformrr QQin et al. 2023 proposes LGR estimator to compute the rigid transformation for coarse-to-fine based methods. We compare the performance of our method with GeoTrausforeer and PEAL when combined with LGR. Here, we also compare the results of using 2D prior as imput, as done in PEAL. As shown in Tab. - our method still yiclds the highost registration recall on both 3DMatch and 3DLoMatch, no matter whether with a 2D prior or 3D prior.
Additionally, we compare our method with SOTA 3D outlier removal methods, including PSintDSC (Bai et al. 2021), SC2-PCR (Chen et al. 2022a) and MAC (Zhang et al. 2023). In contrast to our method, these methods use the rolation and translation error, instead of the RMSE, as the threshold for computing the registration recall. For fair comparison, we follow the evaluation strategy of MAC to re-compute the registration recall of our method Q and present the results in Tab. 3 Following the best result they report, the RR of compared methods obtained by combining it with the GeoTransformer to establish the correspondences. Our method demonstrates substantial improvements over these methods on both the 3DMatch and 3DLoMatch datasets.
Fig. 5 showcases qualitative results of our method applied to point cloud pairs with exceedingly low- overlapping regions. For comparison, we also show the alignment results of our baseline Geo Transffrrrrr CoFiNet, PEAL, and the ground truth. Our method accurately aligns the point clouds in these challenging settings, while Geo Transforoerr and PEAL get completely wrong results.
We the evaluation protocol in MAC's official code released at http:///shhibboom/////ggtttt1.1 Maximal-Cliquee
with
9

---

Published in Transactions on Machine Learning Research (02/2025)
3DMatch
3DLoMateh
5000
2500 1000 500
250
5000
2500
1000 500
250
Geu'rransforner + PSCntDDC Bai et al.
2021
95.4
95.4
95.2
95.2
94.5
77.8
77.9
77.2
76.9
75.7
Transfooree SC2-PCR Chen Tron + MAC Zhang Al. Ours
95.3
78.3
78.4
2023
96.7
96.7
95.2
95.3
94.6
78.9
78.7
78.2
77.7 76.6
96.9
96.9 97.0
96.6
96.3
814
814
83.8
83.0
82.5
Table 3: Comparison with the recent outlier removal basclines. Registration Recall is reported as the evaluation metric. The best scores are in red, the second best in yellow.
CoFiNet
GeoTransformer
PEAL
Ours
Ground truth
Figure 5: Qualitative results of CoFiNet, Tranffrffrrrr PEAL, and our method compared with the ground-truth alignment. The overlapping areas are highlighted by the red boxes. (Best viewed screen when zoomed in)
Despite method effectively improving registration recall, present two failure cases in Fig. 6 In both examples, the overlap between the two point clouds is substantially low, and a large portion consists of planes with no distinctive features. Future work on more effective feature extractors may mitigate this issue.
4.3 Outdoor Scenes: KITTI Odometry
Dataset. KITTI odometry (Geiger et al. 2012 consists of 11 sequences of driving scenes scanned using L.DAR. Following the methodology of previous studies (Qin et al. 2022 Choy et al. 2019 use sequences 0-5 for training, 6-7 for validation, and 8-10 for testing. We use the optimized ground-truth poses with ICP and use only point cloud pairs that are at least 10m away for evaluation.
Metrics. We follow (Huang et al. 2021 and evaluate the methods by three metrics: (1) Relative Rotation Error (RRE), which the geodesic distance between the estimated and ground-truth rotation matrices; (2) Relative Translation Error (RTE), which measures the Euclidean distance between the estimated and -ound-truth translation vectors; (3) Registration Recall (RR), which encodes the fraction of point cloud pairs whose RRE and RTE are both below certain thresholds (RRE< 5 and RTEK 2m).
Registration results. We compare our network with 9 recent baselines, including FCGF, D3Feat, SpinNet, Predator, CoFiNet, GeoTransformer, OIF-Net, PEAL, and MAC. PEAL does not have open-source code for the KITTI dataset, so we report the result of our implementation, The coarsc-to-fine approaches, which inchude CoFiNet, GeoTransformer, OIF-Net, PEAL, and our method, are compatible with the LGR estimator QQn et al. 2022). Therefore, present their results combined with L.G.. For the other methods.
10

---

Published in Transactions on Machine Learning Research (02/2025)
Ours
Ground truth
Ours
Ground truth
(a)
(b)
Figure 6: Failure cases of our method. (Best viewed on screen when zoomed in)
Methods
Reference
RTEE 4 RRE
RR f
FCGF
ICCV2019
Chov et al.
2019
9.5
0.30
96.6
D3Feat
CVPR2020
[663 a.
2020
7.2
0.30
99.8
SpiuNet
CVPR2021
Ao
2021
9.9
0.47
99.1
Predator
CVPR2021
Huang 0t a..
2021
6.8
0.27
99.8
CnFiNet.
NeurIPS2021
YD
8.5
0.41
99.8
GeoTrans
CVPR2022
Cm et at
2024
6.8
0.24
99.8
OIF-Net
NeurIPS2022
Yang
PEAL
CPPR2023
Yu 6l
255
6.8
0.23
99.8
MAC
CVPR2023
Zhaug 8l.
2023
8.5
0.40
99.5
Ours
6.3
0.23
99.8
Table 4: Results KITTI odometry. The best scores are in red, the second best in yellow.
94.5
Registration Recall
80
94
93.5
78
93
Geerransforee
TenTronsrr
92.5
Prodator
76
Predator
CoFiNet
CoFiNel
92
No
Prior
Gaussian
Caussian
91.5
74
(a) 3DMatch
step
(b) 3DLoMatch
step
Figure 7: Iteration results using different priors than the training one.
we use RANSAC as their post-processing step. For PEAL and our method, we just perform two steps of iterative optimization because the KITTI dataset is relatively simple. As can be seen in Tab. 4 our method yiclds 99.8% RR, which oquals the best performance recent methods. Furthormore, our method achieves the best RRE and RTE.
4.4 Ablation Studies
In Tab. analyze design choices by ablating different. components in framework. All the experiments use LGR as the post-processing method.
Overlap (%) RRR <
15
15
20
20
25
25
30
30
GraTransformer
57.6
72.9
78.2
84.2
92.3
PEAL
61.9
80.4
87.6
86.2
95.7
96.5
Table 5: Regitration recall in the scenes of different overlap rates. The best scores are in red, the second best in yellow.
11

---

Published in Transactions on Machine Learning Research (02/2025)
3DMatch
3DLoMatch
RR
IRt
FMR
RR
IR
FMR
Full setting
94.4
73.4
98.3
80.0
49.6
87.0
w/o cross
94.3
72.6
98.3
79.5
48.5
86.7
w// self
93.9
72.5
98.2
79.4
48.7
86.6
w/o self, cross (baseline)
92.5
70.9
98.2
74.0
43.5
87.1
Only prior (baseline)
93.9
70.9
98.6
78.7
47.4
86.6
Add noise
92.7
67.6
97.3
76.9
44.5
85.8
w/o Slerp
73.0
98.2
49.2
86.5
Table 6: Ablations on network components and degradation schemes. Full setting: The final version of our method. Only prior: Directly using the prior as input without degradation scheme. Add noise: Using Gaussian noise as degradation function. The best scores are in red, second best in yellow.
Evaluation in different overlap rates. An important aspect of evaluating network is to measure whether it is robust to noise. For the point cloud registration lask, the overlap rate reflect the noise level. Thus, in Tab. 5 we provide a detailed evaluation by dividing 3DLoMatch into different groups by overlapping ratio, similarly to (Chen et al. 2022a). As reported in the table, our method consistently outperforms PEAL and Geonransfrrmrr, demonstrating our method's robustness, particularly in scenarios with smaller overlaps.
Initialization with different pretrained models. In our method, use the results obtained by the pretrained Tronffoffrrrrr input. To illustrate generalization ability, test other networks to generate the prior for our method, including Predator (Huang et al.] 2021) and CoFiNet Y et al. (002).. Furthermore, we also design two control experiments: 1) providing only an identity matrix, not an actual prior; 2) randomly sampling a rigid transformation from a Gaussian distribution as prior. In Fig. 7 we report the results at different iterations. Since Predator and CoFiNet perform worse than GeoTranstormer, the performance of using their results as prior is close to using the identity matrix. Through multi-step optimization, even without prior, our method yields near 93.5% RR on 3DMatch and 3DLoMatch, respectively, which (220%%/5500%) This showcases our method's strong
is much better than the results of Geo generalization to different types of priors.
Design of the refinement network. Our method's refinement network for each timestep leverages both one-way-self-attente
aratacccss--ttttttionn and our novel one-way-cross attention. To analyze the effectiveness of these two mod- ules, we remove them separately and summarize the results in Tab. The results indicate that integrating either module enhances the overall performance, with the best results achieved when both are employed.
Degradation schemes. In our method, propose novel degradation scheme for rigid transformation. Different from the commonly used strategy in generation tasks that relies random noise as the initial step for training, we interpolate between prior information and ground truth at different time steps as the noisy training data for the refinement network. To validate the importance of the degradation scheme, we also use two different degradation schemes to generate the training inputs. The baseline consists of directly using the prior the input, akin to PEAL, and a naive strategy is to add noise to the ground truth. As shown in Tab. G simply adding noise as degradation scheme yields even worse results than the baseline. The proposed deterministic degradation scheme considers the distribution of the results, and (Full) leads to the highest results in all three metrics on both the 3DMatch and 3DLoMatch benchmarks.
Finally, we remove the Slerp interpolation function and directly use the Euler angle as the interpolation function, and the performance also decreases, because the Slerp function can ensure the lincar during the data degradation process.
5 Discussions
Efficiency overhead. Similar to our baseline PEAL Yu et al. (202bb). multi-step methods are generally slower than single-step methods. As shown in Supp-Tab, g our method consistently more efficient than PFAL with equivalent or better registration results. A promising future direction could be to employ smaller
12

---

Published in Transactions on Machine Learning Research (02/2025)
networks and compression strategies, such network pruning, to strike balance between recall rate and speed.
Storage overhead. Our pipeline requires storing multiple models and we tested converting the pretrained models from Float32 to Fl which reduced the storage size by 50% yet maintained an RR of 80. 80.0% on 3LooMatch, outperforming the PEAL (Yu et al. 2023b) baseline without compression (79.0%).
6 Limitation and Conclusion
Our network exhibits dataset bias, necessitating the training of specific models for each application domain, such indoor scenes and driving scenes, which is a common issue for sorming-based methods.
In summary, we propose an adaptive multi-step refinement network for robust point cloud registration in this paper. We introduced technical innovations of conditioning the refinement, network on different steps, and we leveraged a novel training strategy to create point cloud pairs with varying levels of accuracy, Despite being conceptually simple and having minimal overhead, compared to our baseline, our method achieves srate-of-the-ar registration recall on the 3DMatch/ (bMlattch/DLoMMatel and KITTI benchmarks, with a notable absolute registration recall improvement of 1.2% on the challenging 3DLoMatch benchmark. This work also demonstrales the potential of the iterative refinement idea from classical point cloud registration methods in learning- based systems.
7
Acknowledgements
We appreciate that Dr. Junle Yu and Prof. Wenhui Zhou provide us with the code PEAL and 2D prior data. Zhi Chen supported by China Scholarship Council (CSC). This work was supported in part by the Swiss National Science Foundation via the Sinergia grant CRSI15-180359.
References
Sheng Ao, Qingyong Hu, Bo Yang, Andrew Markham, and Yulan Guo. Spinnet: Learning general surface descriptor for 3d point cloud registration. In Proceedings of the IEEE/CVP International Conference on Computer Vision, pp. 11753 11762, 2021.
Sheng Ao, Qingyong Hu, Hanyun Wang, Kai Xu, and Yulan Guo. Bulfer: Balancing accuracy, elficiency, and generalizability in point cloud registration. Tn Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1255 1264, 2023.
Yasuhiro Aoki, Hunter Goforth, Rangaprasad Arun Srivatsan, and Simon Lucey. Pointnetlk: Robust & efficient point cloud registration using pointnet. In Proccedings of the IEEE/CV Conference on Computer Vision and Pattern Recognition, pp. 7163 7172, 2019.
Xuyang Bai, Zixin Luo, Lei Zhou, Hongbo Fu, Long Quan, and Chiew-Lan Tai. D3feat: Joint learning of dense detection and description of 3d local features. In Proceedings of the IEEE/CVI Conference Computer Vision and Pallern Recognition, pp. 6359 6367, 2020.
Xuyang Bai, Zixin Luo, Lei Zhou, Hongkai Chen, Lei Li, Zeyu Hu, Hongbo Fu, and Chiew-Lan Tai. Pointdsc: Robust point cloud registration using deep spatial consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15859-15800 2021.
Paul J Besl and Neil D McKay. Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, volume 1611, pp. 586-606. Spie, 1992.
Dylan Campbell and Lars Petersson. Gogma: Globally-optimal gaussian mixture alignment. In Procoedings the IEEE conference on computer vision and pattern recognition, pp. 5685 5694, 2016.
Suyi Chen, Hao Xu, Ru Li, Guanghui Liu, Chi-Wing Fu, and Shuaicheng Liu. Sira-per: Sim-to-real adap- tation for 3d point cloud registration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14394-144444 14405, 2023a.
13

---

Image and Published in Transactions on Machine Learning Research (02/2025)
Yang Chen and Gérard Medioni. Object modelling by registration of multiple range images. vision computing, 10(3):145-155 1992.
Zhi Chen, Kun Sun, Fan Yang, and Wenbing Tao. Sc2-per: A second order spatial compatibility for elficient and robust point cloud registration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13221-13231, 2022a.
Zhi Chen, Fan Yang, and Wenbing Tao. Detarnet: Decoupling translation and rotation by siamese network for point cloud registration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 401-409, 2022b.
Zhi Chen, Kun Sun, Fan Yang, Lin Guo, and Wenbing Tao. Se4 Rethinking the generation and selection for efficient and robust point cloud registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023b.
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attentiion mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1290-1299, 2022.
Christopher Choy, Jaosik Park, and Vladlen Koltun. Fully convolutional geometric features. In Proceedings of the LEEE/CVF International Conference on Computer Vision, pp. 8958 8966, 2019.
Christopher Choy, Wei Dong, and Vladlen Koltun. Deep global registration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2514 2523, 2020.
Martin A Fischler and Robert C Bolles. Random sample consensus: paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395, 1981.
Andrea Frome, Daniel Huber, Ravi Kolluri, Thomas Bilow, and Jitendra Malik. Recognizing objects in range data using regional point descriptors. In European conference on computer vision, pp. 224-237. Springer, 2004.
Kexue Fu, Jiazheng Luo, Xiaoyuan Luo, Shaolei Liu, Chenxi Zhang, and Manning Wang. Robust point cloud registration framework based on deep graph matching. IEEE Transactions on Pattern Analysis & Machine Intelligence, (5()))61183 1195, 2023.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition, pp. 3354-3361. IEEE, 2012.
Yulan Guo, Ferdous Sohel, Mohammed Bennamoun, Min Lu, and Jianwei Wan. Rotational projection statistics for 3d local surface description and object recognition. International journal of computer vision, 105:63 86, 2013.
Shengyu Huang, Zan Gojcic, Mikhail Usvyatsov, Andreas Wieser, and Konrad Schindler. Predator: Regis- tration of 3d point clouds with low overlap. In Procoodings of the IEEE/CV Conference on computer vision and pallern recognition, pp. 4267 4276, 2021.
Jiaxin Li, Ben M Chen, and Gim Hee Lec. So-net: Self-organizing network for point cloud analysis. In
Proceedings of the IEFE conference on computer vision and pattern recognition, pp- 1997-9406, 2018.
Nicolas Mellado, Dror Aiger, and Niloy J Mitra. Super 4pes fast. global pointeloud registration via smart indexing. In Computer graphics forum, volume 33, pp. 205-215 Wiley Online Library, 2014.
John Novatnack and Ko Nishino. Scale-dependenn//invviiia local 3d shape descriptors for fully automatic 2008: 10th European Conference
registration of multiple sets of range images. In Computer on Computer Vision, Marscille, France, October 12-18, 2008 Procoedings, Part III 10, pp. 440-453. Springer, 2008
14

---

Published in Transactions on Machine Learning Research (02/2025)
Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, and Kai Xu. Geometric transformer for fast and robust point cloud registration. In Proceedings of the IEFE/CVT conference on computer vision and pattern recognition, pp. 11143 11152, 2022.
Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, Slobodan Ilic, Dewen Hu, and Kai Xu. Geotransformer: Fast and robust point cloud registration with geometric transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Radu Bogdan Rusu, Nico Blodow, and Michael Beetz. Fast point feature histograms (fpfh) for 3d registration. In 2009 IEEE international conference on robotics and automation, pp.
IEEE, 2009.
Aleksandr Segal, Dirk Haehnel, and Sebastian Thrun. Generalized--cpp. In Robotics: science and systems, volume 2, pp. 435. Seattle, WA, 2009.
Ken Shoemake. Animating rotation with quaternion curves. In Proceedings of the 12th annual conference on Computer graphics and interactive techniques, pp. 245 254, 1985.
Yilan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, and Yichen Wei. Circle loss: A unified perspective of pair similarity optimization. In Proceedings of the LEEE/CVF conference on computer vision and pattern recognition, pp. 6398-6407, 2020.
Hugues Thomas, Charles R Qi, Jean-Emmannel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proccedings of the CEEF/CVF international conference on computer vision, pp.
2019.
Federico Tombari, Samuele Salti, and Luigi Di Stefano. Unique signatures of histograms for local surface description. In European conference on computer vision, pp. 356-369. Springer, 2010.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Haiping Wang, Yuan Liu, Zhen Dong, and Wenping Wang. You only hypothesize once: Point cloud regis- tration with rotation-equivarrant descriptors. In Proceedings of the 30th ACM International Conference on Multimedia, pp. 1630-1641, 2022.
Yue Wang and Justin M Solomon. Deep closest point: Learning representations for point cloud registration. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 3523 3532, 2019.
Kanzhi Wu, Xiaoyang Li, Ravindra Ranasinghe, Gamini Dissanayake, and Yong Liu. Risas: A novel rotation, illumination, scale invariant appearance and shape feature. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 4008 4015. IEEE, 2017.
Qianliang Wu, Haobo Jiang, Yaqing Ding, Lei Luo, Jin Xie, and Jian Yang. Diff-per: Diffusion- correspondence searching in doubly stochastic matrix space for point cloud registration. arXiv preprint arXiv:2401.00436, 2023.
Fan Yang, Lin Guo, Zhi Chen, and Wenbing Tao. One-inlier is first: Towards efficient position encoding for point cloud registration. Advances in Neural Information Processing Systems,, 35:6982-666 -9222, 2022.
Jiaolong Yang, Hongdong Li, Dylan Campbell, and Yunde Jia. Go-iep: A globally optimal solution to 3d icp
point-set registration. IEEE transactions pattern analysis and machine intelligence, 38(1)):2241 -2544, 2015a.
Jiaolong Yang, Hongdong Li, Dylan Campbell, and Yunde Jia. Go-iep: A globally optimal solution to 3d iep point-set registration. IEEE transactions pattern analysis and machine intelligence, 38(11):2241-2254. 2015b.
Zi Jian Yew and Gim Hee Lee. Rpm-net: Robust point matching using learned features. In Proccedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11824- 11833, 2020.
15

---

Published in Transactions on Machine Learning Research (02/2025)
Zi Jian Yew and Gim Hee Lee. Regtr: End-to-end point cloud correspondences with transformers. arXiv preprint ar MxXiv2220.145111 2022.
Hao Yu, Fu Li, Mahdi Saleh, Benjamin Busam, and Slobodan llie. Cofinet: Reliable coarse-to-line corre- spondences for robust pointcloud registration. Advances in Neural Information Processing Systems, 34, 2021.
Hao Yu, Zheng Qin, Ji Hou, Mahdi Saleh, Dongsheng Li, Benjamin Busam, and Slobodan Ilie. Rotalion- invariant transformer for point cloud matching. In Proceedings of the IEEE IEEE/CVE /CVF Conference on Computer Vision and Pattern Recognition, pp. 5384-5393, 2023a.
Junle Yu, Luwei Ren, Yu Zhang, Wenhui Zhou, Lili Lin, and Guojun Dai. Peal: Prior-embedded explicit attention learning for low-overlap point cloud registration. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17702 17711, 2023b.
Wentao Yuan, Benjamin Eckart, Kihwan Kim, Varun Jampani, Dieter Fox, and Jan Kantz. Deepgmr: Learning latent gaussian mixture models for registration. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part V 16, pp. 753-750. Springer, 2020.
Andrei Zaharescu, Edmond Boyer, Kiran Varanasi, and Radu Horaud. Surface feature detection and de- scription with applications to mesh matching. In 2009 IEEE conference on computer vision and pattern recognition, pp. 373-380 TEEE, 2009.
Andy Zeng, Shuran Song, Matthias Nicßner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3dmatch: Learning local geometric descriptors from rgb-d reconst ructions. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1802-1811, 2017.
Xiyu Zhang, Jiaqi Yang, Shikun Zhang, and Yanning Zhang. 3d registration with maximal cliques. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17745- 17754, 2023.
Qian-Yi Yi Zhou, Jaesik Park, and Vladlen Koltun. Fast global registration. In European Conference Computer Vision, pp. 766-782. Springer, 2016.
A Overview of this Supplementary
In the following sections, we first provide a comparison with classical hon-learning-based PCR algorithms, followed by more specific comparisons with PEAL, and then present more qualitative results.
B Comparison with non-Learning-basee Methods
Methods ICP
Go-ICP Super4PCS FGR Ours
RR
6.59
22.9
21.6
42.7 94.4
Table 7: Comparison of registration recall with classic non-learning methods on the 3DMatch dataset. All values are sourced from the official leaderboard of 3DMatch. The highest scores are highlighted in bold.
For interested readers, we present a comparison with classic non-learning based methods (Tab 7 including ICP Besl & McKay 1992), Go-ICP (Yang al.) (0155a), Super4PCS (Mellado al., (2014). and FGR (Zhou) et al. 2016). Our method significantly outperforms non-learning methods because of effective learned fea- tures.
16

---

Published in Transactions on Machine Learning Research (02/2025)
C Per-step Comparison with PEAL
We present step-wise registration recall comparison with PEAL. As shown in Fig. g method yields consistently better results. Notably, our method with two steps achieves better results than PEAL with more than 6 steps, highlighting our method's efficacy.
94.5
94.3
93.9
PEAL
Ours
93.7
step
(a) 3DMatc
80.5
80.0
79.5
79.0
78.5
PEAL
78.0
Ours
77.5
2
3
4
6
(b) 3DLoMatch
step
Figure 8: Multi-step registration recall comparison with PEAL. In both the 3DMatch and 3DLoMatch benchmarks, our method consistently surpasses PEAL. Notably, our method with two steps outperforms PEAL with more than steps.
Computation Time
We present the time cost of our method in Tab. and compare with two baselines (Geo Transforrer and PEAL). As a reference, we also put the registration recall (RR) of the methods Our method has a higher computational cost than Grorransfrrerr 1-step inference, due additional network components of one-way allentions, while being comparable with PEAL. Note that since our method boosts performance with more steps, In addition, our method can get a higher RR with 2-step optimization than PEAL with 5 steps.
the number of steps to run as parameter for different application scenarios.
3DMatch
3DLoMatch
RR
Time (sec.)
RR
Time (sec.)
GeoTransformer
92.0
0.296
74.0
0.284
Geo Transfooormer + PEAL 1-step
93.7
0.663
77.8
0.642
Geo Transformer + PEAL 5-step
94.0
2.131
78.5
2.074
Geo Transforrrre Ours 1-step
93.9
0.625
78.2
0.620
Gro'rranform Ours 2-step
94.2
0.954
79.4
0.956
GeoTransformr Ours 5-step
94.4
1.939
80.0
1.964
Table 8: Comparison of inference time efficiency between Gennrrrsorr,rr,, PEAL and our method. The best scores are in bold.
For Gee and PFAL, we their official code with default settings the same form as ours.
17

---

Published in Transactions on Machine Learning Research (02/2025)
CoFiNet
raansfomme
PEAL
Ours
Ground truth
Figure Qualitative comparison of the proposed method with recent methods. From left right are CoFiNet, Geo GeoTTrasffomee PEAL, ours, and ground truth alignment. Our method best aligns these chal- lenging low-overlapping point cloud pairs. (Best view screen when zoomed in)
E Qualitative Results
We present more qualitative comparisons with baselines in Fig. g Low-overlapping PCR poses typical challenge, non-overlapping regions might be mistakenly matched, leading to incorrect alignments. In contrast, our method effectively aligns these point clouds.
18


Reviews:
[{"review_id": "VjuRlKsJPn", "reviewer": "Reviewer_Wm98", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Designing a specialized model for each intermediate refinement step is reasonable, as the input registration qualities vary significantly across different steps.\n2. The introduction of step indices and the estimated overlapping region from the last steps, along with training multiple models with transformations of varying qualities, allows the model to focus more effectively on the current stage.\n3. There are improved results on both indoor and outdoor datasets.\n\nWeaknesses:\n1. My primary concern is aligning the multiple registration models trained during the training stage with the different stages during testing. Specifically, during training, varying qualities are defined by formula (2), where the ground truth $\\alpha_{\\tau}$ is available. However, in the inference stage, we cannot obtain the registration qualities of the current stage, thus it is unclear which registration model to use. I suspect the authors assume that as iteration steps increase, registration qualities also improve, thus progressively adopting models corresponding to higher qualities during testing. However, there are two issues with this approach: (a) the specific registration qualities are still unknown, making it difficult to precisely determine which model would perform better, and (b) qualities may not improve progressively with each step, as shown in Figure 7. Overall, my question is, how do we determine which model to use at each step during testing?\n2. During the testing phase, what would be the impact of using the same model across all steps? Assuming there are 6 steps, if the same model is used for each step, either the first or a middle one, instead of using a different model for each step, how much would the results decline? Specific analysis is requested.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The article proposes an adaptive multi-step refinement method to tackle point cloud registration, especially when the overlapping region between two point clouds is small. It builds on the current multi-step refinement methods, introducing improvements by designing a customized model for each step. Specifically, the authors incorporate the estimated overlapping region from the last step and the step indices, and train the model with varying registration qualities. Experiments are conducted on standard benchmark datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weaknesses.", "broader_impact_concerns": "Na", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "SuRZbzEye3", "reviewer": "Reviewer_s9A7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. Addressing a practical challenge: The paper effectively tackles a challenge of PCR when the overlapping region is small, a scenario where existing methods often struggle.\n2. Experimental studies\n   - Comprehensive design: The experimental setup is well-designed, comparing the proposed method against a sufficient number of baseline algorithms. Furthermore, detailed ablation studies validate the effectiveness of each component of the proposed method.\n   - Performance: The algorithm achieves state-of-the-art results on benchmark datasets, outperforming recently developed methods.\n\n\n**Weaknesses and Questions**\n1. Limited novelty: The proposed framework seems to be a multi-step variation of PEAL, with the only difference being the use of the target point cloud in attention calculation. Is there any significant novelty compared to PEAL that I might have missed?\n2. Explanation about why the proposed algorithm works well: Why does the proposed approach (the multi-step network and the modified attention mechanism in the registration process) outperform existing methods when the overlapping region is small? An intuitive explanation should clarify this connection.\n3. Behavior with accurate priors: How does the method perform when the prior transformation from GeoTransformer is nearly identical to the ground-truth? In such cases, dividing the refinement into multiple steps may seem unnecessary.\n4. (Question) Are there optimal transport-based methods applicable to PCR? If so, how do they differ intuitively from ICP? If not, why is optimal transport unsuitable for PCR?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "First of all, I would like to clarify that I am not an expert in this topic (point cloud registration).\n\n- This paper aims to solve Point Cloud Registration (PCR) by developing a new multi-step refinement network with a modified attention mechanism for the registration process.\n- When the overlapping region between the point clouds is small, recent PCR methods struggle due to the generation of ambiguous features, leading to incorrect matching of non-overlapping points.\nThe authors aim to resolve this issue by introducing a multi-step refinement network, which is a modified version of PEAL, an existing PCR method. \n- Empirical results show that training the proposed network outperforms existing baselines, particularly in cases where the overlap between point clouds is small, improving PCR performance.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- See Weaknesses and Questions above.\n- Presentation: The paper would benefit from clearer formulation. The paper could benefit from clearer formulation. For instance, what are the network’s inputs and outputs? What is the output in Figure 4? Is the cross-one-way-attention in Figure 4 part of the overall process in Figure 2?\n- (Minor) The notation \"K\" is used ambiguously, denoting both the number of models and the key vector in the transformer network. This conflict should be clearly revised.", "broader_impact_concerns": "I do not think any ethical concerns exist.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eqdf9saRJ1", "reviewer": "Reviewer_MdNU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "In general, I believe this paper is polished and well-executed. It presents an interesting idea for iteratively predicting transformations to achieve fine-grained point cloud registration. It demonstrates better performance compared to existing methods on multiple datasets. More importantly, the improvements are more pronounced when the initial overlap is small. The authors also provide a comprehensive analysis of the proposed method, which I find particularly helpful.\n\nSome design choices require clarification or further experimentation. I will detail my questions in the next “Requested Changes” section, but I will briefly summarize them here:\n* The method for dividing accuracy levels needs more clarification and exploration.\n* More analysis of inference time is needed.\n* Ablation experiments with respect to PEAL should be made easier to understand.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a new architecture for point-cloud registration. It trains multiple models, with each model responsible only for predicting the relative transformation at different coarse-to-fine levels. To achieve this, it first generates data by interrelating inputs and the ground truth across different levels. Each model is then trained with only a subset of the dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I’m curious about the different design choices for dividing accuracy levels. In the current setting, the method generates 1,000 different pairs of (input, ground-truth) data by interpolating the raw inputs and ground truth. However, since the raw inputs have varying overlap rates, the generated pairs will also have different overlap rates even if they are in the same accuracy level. For example, if data A has a 0.1 overlap rate and data B has a 0.5 overlap rate, the inputs for the most fine-grained model will be [0.0 - 0.02] for data A and [0.0 - 0.1] for data B. This might be sub-optimal. Have you considered setting accuracy levels according to overlap rates? Specifically, after augmenting the data, you could divide all the data according to their overlap rates and train models for each overlap rate.\n\nRegarding inference time, did you simply apply the five models sequentially? In this case, after applying the first model, the data might be in the optimal input region for model 3 instead of model 2. How do you address this issue?\n\nSorry if I missed this experiment: Are there any ablation studies on the number of K (e.g., how many models you used for training)?\n\nFigure 6 is quite interesting. I’m surprised that the performance remains good even with random Gaussian or identity inputs. What are the average overlap rates between inputs and ground truth for identity, random Gaussian, and using GeoTransformer?\n\nTable 6 presents important ablation experiments. What is the difference between “only prior” and PEAL? Why does this entry have lower performance compared to PEAL?\n\nWhat do the failure cases of the proposed method look like?\n\nMinor question: I’m confused about the sentence, “Notably, our method with five iterations achieves an RR of 80.4%, which surpasses PEAL’s performance with ten iterations (78.8%) on 3DLoMatch.” I only found 80.4 in RR2500 and 78.8 in RR1000. Is this a fair comparison?", "broader_impact_concerns": "Not applicable.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper works on the classical problem of point cloud registration, addressing the challenging case when the two point clouds overlap little. To this end, the paper proposes an adaptive multi-step refinement procedure where a novel training mechanism (i.e.,  training on transformations with varying registration qualities) and a generalized one-way attention are proposed. Experiments show the efficacy of the proposed technical contributions.

Reviewers initially raised questions mostly about better clarifying the paper, including difference of the proposed method from a a multi-step variation of PEAL, clarification on details of setting about dividing accuracy levels, the unknown issue of specific registration qualities and correspondingly the determination of which models to use during testing stages. The authors give convincing clarifications and present additional experiments including designing an early exit mechanism into the proposed framework. All three reviewers are satisfied with the revision. AE agrees with the reviewers.
Decision: Accept (poster)