One-Inlier is First: Towards Efficient Position Encoding for Point Cloud Registration
Authors: Fan Yang
Lin Guo
Zhi Chen
Wenbing Tao
Venue: NeurIPS 2022 Accept (2022)
Keywords: Point cloud registration
Position encoding
One-inlier
Joint optimization

Abstract: Transformer architecture has shown great potential for many visual tasks, including point cloud registration. As an order-aware module, position encoding plays an important role in Transformer architecture applied to point cloud registration task. In this paper, we propose OIF-PCR, a one-inlier based position encoding method for point cloud registration network. Specifically, we first find one correspondence by a differentiable optimal transport layer, and use it to normalize each point for position encoding. It can eliminate the challenges brought by the different reference frames of two point clouds, and mitigate the feature ambiguity by learning the spatial consistency. Then, we propose a joint approach for establishing correspondence and position encoding, presenting an iterative optimization process. Finally, we design a progressive way for point cloud alignment and feature learning to gradually optimize the rigid transformation. The proposed position encoding is very efficient, requiring only a small addition of memory and computing overhead. Extensive experiments demonstrate the proposed method can achieve competitive performance with the state-of-the-art methods in both indoor and outdoor scenes.

One-Inlier is First: Towards Efficient Position Encoding for Point Cloud Registration
Fan Yang
Lin Guo Zhi Chen
Wenbing Tao"
School of Artificial Intelligence and Automation
Huazhong University of Science and Technology, Wuhan 430074, China {fanyang, linguo, z_ chen, xenbingtao) Shhuttttteeeeeeee edu.
Abstract
Transformer architecture has shown great potential for many visual tasks, including point cloud registration. As an order-aware module, position encoding plays an important role in Transformer architecture applied to point cloud registration task. In this paper, we propose OIF-PCR, a one-inlier based position encoding method for point cloud registration network. Specifically, we first find one correspondence by a differentiable optimal transport layer, and use normalize each point for position encoding. It can eliminate the challenges brought by the different reference frames of two point clouds, and mitigate the feature ambiguity by learning the spatial consistency. Then, we propose a joint approach for establishing correspon- dence and position encoding, presenting an iterative optimization process. Finally, we design progressive way for point cloud alignment and feature learning gradually optimize the rigid transformation. The proposed position encoding is very efficient, requiring only small addition of memory and computing overhead. Extensive experiments demonstrate the proposed method can achieve competitive performance with the state-of-ther of-the-art methods in both indoor and outdoor scenes.
Introduction
Point cloud registration a fundamental research topic many applications including scene recon- struction, autonomous driving, robotics, etc. The goal of it is to align two partially overlapped point clouds by estimating rigid transformation between them. The commonly used feature-band -bsse point cloud registration involves two aspects: establishing point correspondences and estimating the rigid transformation. Although it has been widely studied in past decades, it still challenging due the difficulties such as low-overlap, repetitive patterns.
In recent years, learning-based works [28, 44, 50, 8] have made great advances in 3D representation. In addition to utilizing deep neural networks to learn better local feature descriptions [1, 9, 11, recent methods [18, 48, 29, 47] try to introduce Transformer [41] architecture to establish correspondences. One of the most important strategies of Transformer is position encoding, which has been proven crucial when applying Transformer to many computer vision tasks [22, 34, 35]. However, how to adopt the position encoding to the domain of 3D point cloud registration is still challenging. Different from other one-instance tasks (e.g., classification, segmentation and detection), which handle one image point cloud in inference, point cloud registration is two-instance task. It needs to learn features of two point clouds simultaneously and match them. Since the two point clouds are under uncorrelated reference frames, the straightforward position encoding is not good idea [24].
Recently, some methods attempt to design specific position encoding techniques [26, 24, 29] for point cloud registration. Geotransformer [29] calculates the relative positions between all points, then combines the position encoding with self-attention. This approach is equivalent to treating all points as reference points. It works well, but adds a lot of computational overhead due to the additional
Corresponding author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

calculations about the distance and the angle similarity between points. DoPE [26] presents an efficient manner learn a joint- origin as the reference point for position encoding. However, it depends on the centroid of point cloud to initialize the joint-origin, which is susceptible to noise in the low-overlapping scenes.
y
X
In this paper, we propose an efficient one-inlier based position encoding method for point cloud registration. We utilize a rig- orously defined property of rigid transformation, i.e., does not change the length of vector (isometric isomorphism). As illus- trated in Fig. 1, (11, 311 and (Oz, O²) are two correct correspondences. and the lengths of vector æi (2) + O²) and yii (V1 + Oÿ) are equal. Suppose Oz and Og are selected as the reference points respectively, then we can find that (I1,3) 91) is more likely to be a correct correspondence than (#2, /1) only by position information without local feature, hecause the length of zi is more close than 222 to that of yi- In this way, position information can benefit the establishing of correspondences when the local feature is ambiguous. This shows that only one inlier is enough to preserve the spatial consistency [2. 7] of correct correspondences and ensure the distinctiveness of point-wise features. Based on this observation, we only expect to find one inlier for position encoding. Specifically, we adopt the two points of this inlier to normalize each point cloud respectively, then use the normalized points for position encoding enhance the feature of each point. The selected inlier plays two roles in the position encoding: 1) The two points in the inlier see the same spot of the same scenario, so positional differences caused by reference frames can be compensated based on their normalization. 2) It is feasible to do so to ensure spatial consistency, i.e., the relative spatial correlation between arbitrary point and the reference point is maintained across the two point clouds.
Figure 1: One-inlier based posi- tion learning illustration.
However, it is not easy to find an accurate inlier in advance as the reference point, which is a chicken-and-ege problem because determining the reference point depends on discriminative point- wise features. Thus, we propose a joint learning approach to optimize both position encoding and correspondences. It performs an iterative optimization process for finding good correspondences and position encoding. Specifically, we select mullti-nniiers by performing a differentiable optimal transport layer to produce virtual correspondence instead of directly finding one correspondence, avoiding the unstable caused by the arbitrary selection. Then the two points corresponding to the correspondence are considered the reference points. Next, the reference points are utilized to further encode point-wise geometric position features, and generate more discriminative features for point cloud. The updated point cloud features are further used for finding one virtual correspondence and performing position encoding again. Such circular optimization leads to more explicit position encoding. Furthermore, we use a progressive approach to gradually align the two point clouds (reference points), which reduces dependency on initial settings. During each step of the above optimization process, an estimated transformation recovered from some correspondences to align the two point clouds. The position gap between reference points is narrowed in the process of alignment. We repeat the above step several times, which presents a progressive optimization process. Fig. 2 shows the overview of the proposed method.
Compared to the previous works the position encoding for point cloud registration [26, 24, 29], this kind of strategy mainly has three advantages: 1) It is more reasonable for finding a correct correspondence than centroid estimation for position encoding, especially in the low-overlapping scenarios. 2) It is succinct and efficient, and competitive in terms of GPU memory usage and speed, which has been verified in the subsequent experiments. We only need find an inlier efficiently enhance the distinctiveness of features. 3) It is robust, which reduces the impact of outlier through constructing a virtual correspondence and avoids the dependency on the initialization through joint optimization.
In summary, our main contributions follows:
An efficient position encoding with few correspondences for point cloud registration, which is light-weight and only relies on a few points for accurate position encoding.
A joint optimization strategy that, optimizes the correspondence establishment and position encoding with point-wise feature encoding as the agent. By the way, the point cloud features are also continuously optimized.
A progressive point cloud alignment approach, which progressively updates the point cloud positions so as to reduce dependency on initialization.
2

---

Coarse Comespondence
Fine Corespondecee
Progressive Alignment
Cherse Chase Marinn
Mutual Top-
Y
Inlier
Position
Leaming
Embedding
Local Patch
Decoder
Encoder
Cruss Allentio Self Lentang
XL.
Optimal Transport
M Matching
Refrence Points
similartl malais
PIVCR
Comee Peeeprrdee
Waighted SVD
Motchel
(R
Vector
Ox
dustbun
1. Inlier Leaming
2 Position Embodding
Progressive Allgument
Figure 2: Overview of the proposed method. (#: Feature reconstruction: 3: Point cloud normaliza-
tion; (C): Matched correspondences). The whole network is designed in a coarse-to-fing architecture for establishing correspondences by the proposed one-inlier based position encoding. The proposed position encoding mainly contains the following parts: 1. inlier learning approach proposed to establish correspondences and output two reference points ox and öy (Sec. 3.3). 2. We utilize the predicted reference points 0x and oy to normalize the two point clouds for position embedding (Sec. 3.3). 3. We leverage the established correspondences to iteratively align point clouds (reference points Ox and ōy) (Sec. 3.4). Finally, we repeat the above operations I. times through continual feature reconstruction, which presents a progressive joint optimization process (Sec. 3.5).
2 Related Work
Local 3D feature descriptors. Early traditional handcrafted descriptors widely summarize the local geometry in the form of histogram. Histograms usually contain statistical information such as the spatial distribution and geometric properties of points [16]. The descriptors represented with the 3D spatial distribution include Spin Image [19]. 3D Shape Context [13], etc. Another category of descriptors includes Fast Point Feature Histograms (FPFHs) [30], SHOT [39], etc. Recent works attempt to encode 3D local descriptors a data-driven way. Zeng et propose the pioneering work 3DMatch [49], which utilizes 3D Convolutional Neural Networks to encode local descriptors from local volumetric patches. PPFNet [11] introduces rotation-invariant Point-Pair Features (PPF) [12] to encode local descriptors. Follow-up work PPF-FoldNet [10] incorporates FoldingNet [45] as backbone for unsupervised descriptor learning. 3DSN [15] leverages the voxelized Smoothed Density Value (SDV) to learn rotation-ins rotation-invarian local descriptors. FCGF [9] first utilizes the sparse 3D fully convolutional network to extract dense feature descriptors. SpinNet [1] proposes a 3D cylindrical convolution layer for learning rotation-invarian features.
Learning-based point cloud registration. Deep Closest Point [43] utilizes graph neural network to extract point cloud features and pointer network to predict correspondences. IDAM [23] proposes an iterative distance-aware similarity matrix convolution module to integrate feature and Fuclidean space into the point matching- Predator [18] leverages the attention mechanism to detect the overlapping region for sampling salient feature points. CoFiNet [48] utilizes a coarse- to-fine strategy for correspondence finding. Lepard [24] utilizes the Rotary positional encoding [34] to encode 3D positional information for feature learning. NgeNet [51] introduces multi-scale architecture to extract features with multiple neighborhood sizes for adaptive selection of feature scale. REGTR [47] uses transformer to directly predict set point correspondences and estimates the rigid transformation using variant of the Kabsch- -uescmaa [20, 40] algorithm.
Position embedding. The idea relative positional encodings first appeared in machine translation [32] and music generation [17]. They produce learned positional embeddings in the self-attention. DETR [6] applies fixed positional encodings
[27, 4] the input of the transformer architecture for object detection. Swin Transformer [25] adds relative position biases in similarity computation

---

of self-attention improving performance over models without these biases. LoFTR [35] uses the 2D extension of the position encoding to produce position-dependers features for image matching. DoPE [26] proposes to calculate joint-origin of all points for point cloud position encoding. Geotransformer [29] treats all points as reference points to calculate the relative positions between point pairs and combines them with self-attention to embed positional information into the features.
3 Approach
3.1 Problem Statement
M) and target point cloud Y = (y3 E IR3 _ I,..., N). where M and N are the point numbers of X and Y, respectively. Our goal is to recover the rigid transformation T = {R, 1] that best aligns the two point clouds, where R E SO(3) represents the rotation component and E IR represents the translation component, as
Suppose there are two point clouds: source point cloud X = (
follows:
arg min R,t
Σ
|| (R xx 2
(L,)ECC
where C is set of predicted correspondences between the source and target point clouds.
3.2
Pipeline
Fig. illustrates our overview framework. Following [35, 48, 36), our method employs coarse-to- fine manner to find correspondences. In the coarse stage, we first use the KPConv [38] to down-sample raw input points to uniformly distributed nodes X TRA and Y R and learn the associated features Fx E R RM and Fy E IR Nx Then coarse correspondences are generated by matching the nodes. After obtaining the coarse correspondences from the previous stage, we refine them in fine level. Specifically, we assign each point in the fine level to its nearest nodes in geometry space, Each node will be expanded into a local patch by the point-to-node grouping strategy [21, 48], and the patch correspondences can be established by the matching of nodes. For each patch correspondence, utilize an optimal transport layer [31] to calculate soft assignment matrix between the fine-level points within this patch correspondence, then obtain the fine point correspondences by the mutual top-k selection. The final point correspondences are collected from all coarse correspondences.
Since the results of fine correspondences mainly depend on the coarse results, our method focuses on boosting the accuracy of the coarse results. Specifically, an inlier- based position learning approach is proposed to find an inlier and normalize the point clouds for position embedding (Sec. 3.3). Next, we leverage the matched correspondences (the byproduct of the "one-inlier" establishing) to progressively align point clouds (Sec. 3.4). After that, we propose to jointly optimize correspondence establishment and position embedding bridged by feature reconstruction (Sec. 3.5). Finally, we introduce the loss function for training the network (Sec. 3.6).
3.3 Inlier-based Position Learning
Inlier learning. As mentioned before, once a correspondence is an inlier, its corresponding points in the two point clouds can be considered as the reference points respectively. Since the two reference points are mapped from the same 3D point in different views, the matching points satisfied the ground-truth rigid transformation can preserve the spatial consistency (e.g., the spatial distance and orientation) relative to the respective reference points across the two point clouds. Thus, only one inlier is desired to compensate for the difference in 3D spatial position because the two point clouds in different reference systems can be normalized by the two points of the inlier. Given that the inlier is preemptively agnostic, our approach is mainly devoted to the mining of one correct correspondence. The attentinn [41, 31] can encode the geometric consistency across the two point clouds by integrating the features of the other point cloud. We use it to enhance the distinctiveness of features. Formally, given the coarse features F and F Fy. we use learnable matrices Wg to project the x to query q, and use (Wa, W.) to project Fy. to key and value (k, v) as follows:
4. = F Fx, W k, = F2, WR. Vj = Fy, W..
(2)
where WG. Wt W. E DDD The coarse feature Fx updated by F'x,
F
MLP 1=1 any v,), where ag softmax(q,k ////xx/( is the feature similarity score between
4

---

Fx, and Fy, MLP(-) denotes a fully connected network. The cross-attention features for Fy are computed and denoted in similar way. Following the cross-attention use self-attention to aggregate the global context, which updates features a similar way like cross-attention, but is performed inside the source and target point cloud independently.
After obtaining the strengthened features F'x and Fyy. we leverage a differentiable optimal transport to find good correspondences. We compute the similarity matrix Š E RM*N between F's and Fy::
S F F T
then we append a new row and a new column as in [31], filled with a learnable dustbin parameter a. The Sinkhorn algorithm [33] is adopted on S for searching the best solution of the optimal transportation problem. Finally, we obtain the soft matching scores S E ÆM between the X and Ý by dropping the last row and the last column of S. Intuitively, the higher the matching score, the higher the probability that the associated correspondence is an inlier. But is not always true due to the widely existing repetitive structures, textureless structures, etc. In order to enhance the robustness of the proposed algorithm and reduce the bias caused by the above issues, we propose to pick multiple reliable correspondences Ctopk with higher matching scores via top-k selection strategy. We use the selected correspondences to produce virtual correspondence by an averaging operation of the point coordinates. The corresponding points of the virtual correspondence are regarded as the reference points o {ox: ōy), where the Ox, Oy denote the reference point in source and target point cloud, respectively.
Position embedding. After obtaining the reference points o, they are utilized to normalize the two point clouds respectively. Specifically, we use them to calculate point-wise position vector relative to the respective reference point for each node in the two point clouds:
V(x,) = x; ox, v(r,) = 8, ōy
(4)
Once the reference points and (x, vy) are inliers, i.e., they both meet the ground-truth rigid transfor- mation between the point clouds, then the two vector V(X;). V(i,) satisfy the isometric isomorphism property. It means that the isomorphism respective reference point in different reference frames. Based on this property, the point-wise vectors normalized by the reference points are fed into position embedding layer for feature reconstruction. Specifically, use an MLP with 4 hidden layers of feature size being 256 each individual vector, and output an embedded geometric position feature. Formally, for vector V(K,) and V(ÿ,), ). the corresponding feature is denoted as P(V().)) and P(V(Š,) )), separately. Then we use the embedded geometric position features to enhance the distinctiveness of the node features. In a simple and effective manner, we strengthen the node features by directly adding the geometric position features to the primal features, i..., x, = Fxx P((((x,)) Hy, =F', P(((())))
ã; or ý, maintaining constant distance with respect to the
3.4 Progressive Point Cloud Alignment
As mentioned before, the main challenge of position encoding is that the two point clouds are generally under irrelevant reference frames. We further solve this challenge by progressively aligning the point clouds coordinate space. We expect to recover a rigid transformation with reliable correspondences Coopk to make the position encoding easier.
Position update. After obtaining the coarse correspondences Cropk with relatively higher matching
scores, we use them to estimate the current optimal transformation T,. Specifically, given the coarse correspondences Ctopk and corresponding matching scores, weighted SVD [5] to estimate the current optimal transformation T, Then we use the estimated T. (R, i, to update the relative position between two point clouds:
X;, (R, x; + (.), x; E X, 1,... M.
(5)
Naturally, the reference points, i.e., the centroids ō of the selected patches in the two point clouds have also been updated:
ox. = RR ox + +.),
(6)
where ox, represents the transformed reference point of the source point cloud. As the two point clouds are roughly aligned by the estimated transformation, the position vectors of the two points in
5

---

each correct correspondence tend to be consistent. Thus, the encoded position feature can provide more positive effect for establishing correspondences.
Progressive alignment. We repeat the above operation leveraging the enhanced features during the joint optimization (see Sec. 3.5) to progressively update the relative positions between the two point clouds. Specifically, every time before the position encoding operation, we perform an alignment between the two point clouds. We utilize the enhanced features to rebuild some reliable correspondences Cropk and re-estimate the current optimal transformation Tr for the alignment. Intuitively, the enhanced features are more distinctive than original features, so the latter estimated transformation is more accurate than that in previous stage. Thus, multiple transformations enable the two point clouds to be progressively aligned coordinate space. In this way, the position encoding can also be optimized in the progressive alignment process.
3.5 Jointly Optimizing Position Encoding and Correspondences
Inlier learning and position encoding of the point clouds is chicken-and-ege problem: the proposed position encoding for learning distinctive node features would require the knowledge about at least one correct correspondence; but meantime, establishing one accurate correspondence also relies on the correct position encoding of the point clouds for feature reconstruction. Either of them hard to be solved advance. order to tackle this circular problem, we propose to jointly optimize the correspondences and position encoding by multiple feature rebuilding. As shown in Fig. 2, we incorporate the position encoding into the process of the node feature learning. We construct it together with inlier learning a joint optimization task, with view to improving the accuracy of coarse correspondences by enhancing the discrimination of position encoding. Specifically, the joint optimization is performed by enhancing the features of point clouds with 3D positional information. The updated more accurate vectors V(x.), V(J,)) with respect to the selected reference points facilitate better node feature encoding, which contributes to finding one more accurate correspondence. Meanwhile, the more accurate correspondence means better reference points o, which enables that the updated vectors V( V(ý,) gradually satisfy the isometric isomorphism property. It will result in more discriminative node-wise features. During the above process, the features of the point clouds are continuously optimized.
3.6 Loss Functions
The training loss function L= L. + Ly is composed of the coarse correspondence loss Lg and fine correspondence loss Lj.
Coarse correspondence loss. To supervise the patch-wise feature descriptors for coarse correspon- dences, we follow [18, 29] and use a variant of the circle loss [37]. Considering again an point cloud pair and Y, we first pick the patch P.N (denotes the patch corresponding to the node x; E X) that has at least one positive patch in Y to form a collection of patches P. For each patch P. E P. the set of its positive (share at least 10% overlap) and negative (do not share overlap) patches in Y are denoted as ep and en,, respectively. The loss of coarse correspondence on X is then calculated as:
log +
Σ Map (d)
-
Σ PVee
B (A.. (An--))). -dt),
P
PV Ey
H represents the distance in feature space, and x =of) and of denotes the overlap ratio between P. and Pj. Ap, An are positive and negative margins, respectively. The weights for each positive and negative sample calculated individually: 8 7dd Ap) and gi.k = >A,, - )²), with the empirical margins Ap = 1, A,, = 1.4. 7 a hyper-parameter. The loss C on Y is computed in the same way. The total loss L. = (C. + //2
where di
H x,
Fine correspondence loss. For fine correspondence, correspondences using the ground-truth relative transformations. Then we extract a set of ground- truth fine correspondences M, in each correspondence patches. We label the unmatched points in this two patches as I, and Jr. The fine correspondence loss is computed as:
first determine the ground-truth coarse
f =
Σ (..)).M
log
P' a.y
Σ EEI,
log pd
2,n++1 +
Σ log P' pm)tyy BEJ.
(8)

---

under the where C' the number of the patch pairs. P' E () is the soft assignment matrix of the i-th patch correspondence. m, and n, represent the number of the fine level points in this patch pair.
Table I: Evaluation results on 3DMatch and 3DLoMatch datasets.
3DMatch
RR (%)
f 1000
FMR (%) 1
IR
(%) f
# Samples
5000
2500
500
250
5000
2500
1000
500
250
5000
2500
1000
500
250
3DSN [15]
78.4
76.2
71.4
67.6
50.8
95.0
94.3
92.9
90.1 82,9
36.0
32.5
26.4
21.5
16.4
descriptor
FCGF [9]
85.1
84.7
83.3
81.6
71.4
97.4
97.3
97.0
96.7
96.6
56.8
54.1
48.7
42.5
34.1
D3Feat [3]
81.6
84.5
83.4
82.4
77.9
95.6
95.4
94.5
94.1
93.1
39.0
38.8
40.4
41.5
41.8
SpinNet [1]
88.6
86.6
85.5
83.5
70.2
97.6
97.2
96.8
95.5
94.3
47.5
44.7
39.4
33.9
27.6
YOHO [42]
90.8
90.3
89.1
88.6
84.5
98.2
97.6
97.5
97.7
96.0
64.4
60.7
55.7
46.4
41.2
REGTR [47]
92.0
end-to-end
Predator [18]
89.0
89.9
90.6
88.5
86.6
96.6
96.6
96.5
96.3
96.5
58.0
58.4
57.1
54.1
49.3
CoFiNet [48]
89.3
88.9
88.4
87.4
87.0
98.1
98.3
98.1
98.2
98.3
49.8
51.2
51.9
52.2
52.2
Teans [29]
92.0
91.8
91.8
91.4
91.2
97.9
97.9
97.9
97.9
97.6
71.9
75.2
76.0
82.2
85.1
Ours
92.4
91.9
91.8
92.1
91.2
98.1
98.1
97.9 98.4
98.4
62.3
65.2
66.8
67.1
67.5
3DLoMatch
RR (%) f
FMR (%) f
IR (%) f
Samples
5000
2500
1000
500
250
5000
2500
1000
250
5000
2500
1000
500
250
descriptor
3DSN [15]
33.0
29.0
23.3
17.0
11.0
63.6
61.7
53.6
45.2
34.2
11.4
10.1
8.0
6.4
4.8
FCGF [9]
40.1
41.7
38.2
35.4
26.8
76.6
75.4
74.2
71.7
67.3
21.4
20.0
17.2
14.8
11.6
D3Feat [3]
37.2
42.7
46,9
43.8
39.1
67.3
66.7
67.0
66.7
66.5
13.2
13.1
14.0
14.6
15.0
SpinNet [1]
59.8
54.9
48.3
39.8
26.8
75.3
74.9
72.5
70.0
63.6
20.5
19.0
16.3
13.8
11.1
YOHO [42]
65.2
65.5
63.2
56.5
48.0
79.4
78.1
76.3
73.8
69.1
25.9
23.3
22.6
18.2
15.0
REGTR [47]
64.8
end-to-end
Predator [18]
61.2
62.4
60.8
58.1
78.6
77.4
76.3
75.7
75.3
26.7
28.1
28.3
27.5
25.8
CoFiNet [48]
67.5
66.2
64.2
63.1
61.0
83.1
83.5
83.3
83.1
82.6
24.4
25.9
26.7
26.8
26.9
Trans [29]
75.0
74.8
74.2
74.1
73.5
88.3
88.6
88.8
88.6
88.3
43.5
45.3
46.2
52.9
57.7
Ours
76.1
75.4
75.1
74.4
73.6
84.6
85.2
85.5
86.6
87.0
27.5
30.0
31.2
32.6
33.1
4 Experiments
We evaluate our approach on indoor 3DMatch [49] and 3DLoMatch [18] benchmarks (Sec. 4.1), and outdoor KITTI odometry [14] benchmark (Sec. 4.2). The point cloud pairs in 3DMatch have 30% overlap, while those in 3DLoMatch have low overlap of 10% 30%. KITTI odometry is an outdoor driving scenario sparse point cloud dataset acquired by LiDAR. Details about the datasets and implementation are provided in the supplementary material.
4.1 3DMatch and 3DLoMatch
Metrics. Our main metric, related to the practical purpose of point cloud registration, is Registration Recall (RR), i.e., the percentage of point cloud pairs whose transformation error (RMSE) below 0.2 m. It is the most important metric in point cloud registration task [18, 48], because it directly measures the final registration performance. Following [3, 18), we also report Feature Matching Recall (FMR), i.e., the fraction of point cloud pairs whose Inlier Ratio is larger than 5%, and Inlier Ratio (IR), defined as the fraction of correct correspondences (residuals are less than ground truth transformation) among the putative matches.
Comparisons to the state-of-the-ar approaches. We compare the registration results of our method with recent state-of the approaches including: 3DSN [15], FCGF [9], D3Feat [3], SpinNet [1], YOHO [42], Predator [18]2, CoFiNet [48], REGTR [47] and Geo' GronTranfformer (GeoTrans in the table) [29] in Tab. 1. We report the results under different numbers sampled correspondences. Since REGTR [47] needs to take all the final downsampled points for transformation estimation instead of sampling, we only provide the results the original paper. Following [3, 18), we run RANSAC- 50k to estimate the rigid transformation. When compared with local 3D descriptors, our method outperforms all the approaches on both datasets. For end-to-end methods, the proposed method and
'The Predator had bug its initial version, we provides the updated results in its corrected version.
7

---

BeoTTrasffrmerr [29] outperform other methods by large margin on all the metrics, especially in the low-overlap scenarios (3DLoMatch). This is because our method and Transfosfree both consider the position information in the network, which demonstrates the importance of position encoding for point cloud registration.
It is worth noting that our method performs worse than Geo Transforrrremr [29] on Inlier Ratio (IR). However, when referring to the most important metric Registration Recall (RR) which better reflects the final performance on point cloud registration, the proposed method achieves the state-of-the-an performance. can be explained by the fact that the inlier rate is not total positively- correllate with the registration recall. As pointed out in [18, 48, 29], besides the inlier rate, the other important issue for registration is the sparsity of the correspondences. Aggregated inliers may improve the inlier rate, but are not necessarily beneficial for the registration. Since our position encoding can introduce spatial consistency, it is possible to find matching pairs that cannot be found by local features alone. Thus, although the inlier rate of our method is not the highest, our method still achieves the best performance in terms of Registration Recall.
3DMatth
3DLoMatch
10%
Infialzztion Optimizated
Intta Izoton
00%
12%
O miaatttedd
Patcentage
50%
Percantage
10%
40%
3%
30%
4%%
10%
2%
0%
0.2
0.4
0.6
0%
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.8
1.0
Inlier Ratio of Capk
Ratior of Cupk
Figure 3: The scene distribution in different inlier ratios of Clopk on 3DMatch and 3DLoMatch.
Initialization and optimization. Our method performs an iterative optimization for finding refer- ence points and reconstructing features. In order to clearly show the effectiveness of the optimization process, we count the scene fiequency of different inlier ratios in the initial and optimized Cropk (described in Sec. 3.3) and provide the histograms of scene distributions in Fig. 3. Specifically, we compute the inlier ratio of the initial and optimized Czapk respectively for each scene on 3DMatch and 3DLoMatch datasets. Then we count the number of scenes with different inlier ratios of Czopk on both datasets and present them in percentage form. In the initial stage. we only use the features learned by the backbone to find Czopk- On 3DMatch dataset, the inlier ratio is between 0.7 and 1.0 in most scenarios, and that of 3DLoMatch dataset is about 0 to 0.8. This shows that the initial backbone features in most scenes are enough to provide acceptable initial reference points. After the joint optimization, the node-wise features are reconstructed with the normalized 3D positional information. The inlier ratio increased to 0.9 ~ 1.0 on the 3DMatch dataset, and the inlier ratio in most scenes of 3DLoMatch is raised to 0.6 ~ 1.0. It demonstrates that the proposed optimization method can significantly improve the matching accuracy of the reference points. Furthermore, we visualize the iterative alignment of the reference points (red and green patches) in Fig. 4. This indicates that the alignment error between two reference points gradually decreases during the optimization process.
Computing overhead and runtime. We evaluate the parameters, GPU memory us- age and runtime of our method. Tab. 2 shows the comparison results with other methods. Considering the practical appli- cation, we provide the peck of GPU mem- ory usage. On parameters, GeoTransformer [29] is 27% more than ours, but our method still achieves a competitive registration ac- curacy compared to it and significantly out- performs other methods. On GPU memory usage, our method ranks 2nd only behind CoFiNet [48], and is only about 40% of Grensfasformer'ss [29]. With such low memory usage, our method still achieves the state-of-the-an registration performance. On runtime, our method achieves the fastest registration speed. This demonstrates the superiority of our method in both accuracy and speed.
Table 2:
Evaluation of parameters [M], GPU memory
[GB] and runtime [s] on 3DMatch and 3DLoMatch.
Methods
Params Mem.
Time
Model
Pose
Total
D3Feat [3]
27.3
7.26
0.0449
1.832
1.877
Predator [18]
7.43
7.78
0.0468
2.666 2.713
CoFiNet [48]
5,48
3.72
0.218
0.705 0,923
Geo Transformer [29]
9.83
12.73
0.247
1.743 1.99
Ours
7.75
4.85
0.141
0.414 0.555
8

---

Initial
Istittration
2nd iteration
Ground truth
Figure 4: Qualitative process of the iterative alignment.
4.2 KITTI
Metrics, Following [18], we evaluate our method with three metries: 1) Relative Rotation Error (RRE), the geodesic distance between esti- mated and ground-trutt rotation matrices, 2) Rel- ative Translation Error (RTE), the Euclidean dis- tance between estimated and ground-trute -tuth trans- lation vectors, and 3) Registration Recall (RR),
Table 3: Evaluation results KITTI odometry.
Methods
RTE(cm) + RRE (*) RR(%)
3DFeat-Net [46]
25.9
0.25
96.0
FCGF [9]
9.5
0,30
96.6
D3Fcat [3]
7.2
0.30
99.8
SpinNet [1]
9.9
0.47
99.1
which has mentioned before, the thresholds are set as RRE < 5" and RTE < 2m.
Predator [18]
6.8
0.27
99.8
CoFiNct [48]
8.5
0.41
99.8
Registration results. In Tab. 3, we compare our method with state-of-the-arr approaches, in-
GeoTransformer [29]
6.8
0.24
99.8
Ours
6.5
0.23
99.8
cluding: 3DFeat-Net [46], FCGF [9], D3Feat [3]. SpinNet [1], Predator [18], CoFiNet [48] and GeaTransformer [29]. Our method performs the best for all metrics, which proves that our method is scene- agenstigooticc and maintains good registration accuracy in strongly differing scenarios.
4.3 Ablation study
Table 4 shows the results of the ablation studies the 3DMatch and 3DLoMatch datasets. We remove one component at each time from the complete pipeline (Full) to measure the corresponding contribution. Removing the progressive alignment leads to lower Registration Recall both datasets, which means that aligning reference points in 3D space is beneficial for more accurate positional embedding. Without the joint optimization, the eиrо of the estimated rigid transformation drastically increases and the matching inlier ratio decreases significantly. The joint optimization plays an important role in learning discriminative features. It avoids lot of incorrect matches arising from 3D structural ambiguity. Finally, we replace the proposed one-inlier based position encoding with the centroid based position encoding, i.e. using the centroids reference points instead of learning an inlier. Since the centroids do not constitute correct correspondence for pair of partially overlapping point clouds, this alternative results in worse performance.
Table 4: Ablation study for each component, tested with # Samples 5000.
3DMatch
3DLoMaich
Methods
RR(%)
FMR(%) f IR(%)
RR(%)
FMR(%)
IR(%) f
Full
92.4
98.1
62.3
76.1
84.6
27.5
w/o progressive alignment
91.4
98.2
61.8
74.2
84.1
27.1
w/o joint optimization
91.1
97.5
58.9
72.6
83.6
25.7
w/o associated reference points
89.8
97.2
57.1
69.8
82.7
24.2
4.4 Relationship between the registration and correspondences
In order to analyze the relationship between the registration results and the established correspon- dences, we count the scene frequency of different inlier ratios for Geotrranffrrmer and our method, and present the curve charts of scene distributions in Fig. 5. Specifically, we count the number of scenes with different inlier ratios on 3DMatch and 3DLoMatch datasets, and present them in frequency form. Besides, we also calculate the scene registration recall of different inlier ratios and provide the curve charts in Fig. 6. On 3DMatch dataset, the inlier ratio of our method is between 0.2 and 0.9 most scenarios, while the inlier ratio of Geotransformer is about 0.5 1.0. On 3DLoMatch. the inlier ratio of most scenarios for our method is between 0.05 and 0.6. For Geotransformer, it has more scenarios where the inlier ratio 0.5 0.9. fact, the inlier ratio is counted by averaging all the point cloud pairs. Geotransformer [29] has higher inlier ratio in some scenarios, which causes the mean inlier ratio to be higher than ours. However, higher inlier ratio not a necessary condition for high registration recall. As show in the Fig. 6, we can observe that our method can achieve better

---

registration recall than Geotransformer with the same inlier ratio (especially the interval [0.15, 0.5] on 3DLoMatch dataset), which shows that the inliers of our method may be better distributed for model estimation.
3DMatch
Pupatage
0.04
Ouss
0.03
0.02
0.01
0
0.2
0.5
0.8
0.9
Inlier Ralio
3DLoMatch
Cenl Ouss
pergelagg
0.006
0.04
0.02
0.1
02
0.3
0.4
0.5
0.7
0.8
0.9
Inller Ratio
Figure 5: The scene distribution of different inlier ratios 3DMatch and 3DLoMatch datasets.
3DMath
Registration Rosall
Gopi
fams
0.6-
0.2-
%
0.7
d.9
Inller
3D1 oaach
Registratinn Ravvil
Geol QUsS
0.8-
0.6
0.4
0.2
0.5
0.6
d.
0.9
Ratio
Figure 6: The scene registration recall of different inlier ratios on 3DMatch and 3DLoMatch datasets.
5 Conclusion
In this paper, we have introduced a simple but efficient position encoding for point cloud registration. In contrast to previous methods that rely lot of computing resources need to meet stringent conditions, our method light-weight and easy to implement. Our method only relies on a few correspondences: we show that one inlier is enough to realize accurate position encoding for learning distinguishable features. We utilize a differentiable optimal transport layer to find a good correspondence, which normalizes each point for position encoding. It builds reliable bridge between point clouds in different reference frames, and preserves spatial consistency to reduce feature ambiguity. Besides, propose joint optimization between correspondence establishment and position encoding, and along with that, the point cloud features are also iteratively optimized. Furthermore, progressive alignment approach is proposed update point cloud positions and features alternately. The proposed method achieves a competitive registration performance with more complex state-of-the-ar pipelines indoor and outdoor datasets.
Limitations. Our method adopts a coarse-to-fine manner to find correspondences. However, the final point correspondences are largely subject to the coarse correspondences. Once patch correspondence is incorrect, it would result in incorrect point correspondences. These incorrect point correspondences would have potential negative impact on the final registration accuracy. Besides, method would fail in the case the overlapping region between two point clouds too small or the overlapping region is ambiguous structure. A possible solution is expand the overlapping region via point cloud completion.
Acknowledgment
This work was in part supported by the National Natural Science Foundation of China under Grants 62 17606.
10

---

References
[1] Sheng Ao, Qingyong Hu, Bo Yang, Andrew Markham, and Yulan Guo. Spinnet:
Learning a general surface descriptor for 3d point cloud registration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11753-11762, 2021.
[2] Xuyang Bai, Zixin Luo, Lei Zhou, Hongkai Chen, Lei Li, Zeyu Hu, Hongho Fu, and Chiew-Lan Tai. Pointdse: Robust point cloud registration using deep spatial consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15859-15869, 2021.
[3] Xuyang Bai, Zixin Luo, Lei Zhou, Hongbo Fu, Long Quan, and Chiew-Lan Tai. D3feat: Joint learning of dense detection and description of 3d local features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 1359-6367, 2020.
[4] Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, and Quoe V Le. Attention augmented convolutional networks. In Proceedings of the IEEF/CVF international conference on computer vision, pages 1286-3295, 2019.
[5] Paul Besl and Neil McKay. Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, volume 1611, pages 586-606. Spie, 1992.
[6] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213-229. Springer. 2020.
[7] Zhi Chen, Kun Sun, Fan Yang, and Wenbing Tao. Sc" 2-per: A second order spatial compatibility for efficient and robust point cloud registration. arXiv preprint arXiv: /4453, 2022.
[8] Zhi Chen, Fan Yang, and Wenbing Tao. Detarntt: Decoupling translation and rotation by siamese network for point cloud registration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 401-409 2022.
[9] Christopher Choy, Jaesik Park, and Vladlen Koltun. Fully convolutional geometric features. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9958-8966 2019.
[10] Haowen Deng, Tolga Birdal, and Slobodan Ilie. Ppf-foldnet: Unsupervised learning of rotation invariant 3d local descriptors. In Proceedings of the European Conference on Computer Vision (ECCV), pages 602-618, 2018
[11] Haowen Deng. Tolga Birdal, and Slobodan Ilie. Ppfnet: Global context aware local features for robust 3d point matching. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 195-205, 2018
[12] Bertram Drost, Markus Ulrich, Nassir Navab, and Slobodan Ilie. Model globally, match locally: Efficient and robust 3d object recognition. In 2010 IEEE computer society conference computer vision and pattern recognition, pages 998-1005. leee, 2010.
[13] Andrea Frome, Daniel Huber, Ravi Kolluri, Thomas Bülow, and Jitendra Malik. Recognizing objects in range data using regional point descriptors. In European conference computer vision, pages 224 237. Springer, 2004.
[14] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference computer vision and pattern recognition, pages 3354-3361. IEEE, 2012.
[15] Zan Gojcic. Caifa Zhou, Jan D Wegner, and Andreas Wieser. The perfect match: 3d point cloud matching with smoothed densities. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2019.
[16] Yulan Guo, Mohammed Bennamoun, Ferdous Sohel, Min Lu, Jianwei Wan, and Ngai Ming Kwok. comprehensive performance evaluation of 3d local feature descriptors. International Journal of Computer Vision, 116(1):66-89) 2016.

---

[17] Cheng-ZZ Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, lan Simon, Curtis Hawthorne, Andrew M Dai, Matthew D Hoffman, Monica Dinculescu, and Douglas Eck. Music transformer. arXiv preprint arXiv: 1809.04281, 2018.
[18] Shengyu Huang, Zan Gojeic, Mikhail Usvyatsov, Andreas Wieser, and Konrad Schindler. Predator: Registration of 3d point clouds with low overlap. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4267-4276, 2021.
[19] Andrew E Johnson and Martial Hebert. Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Transactions on pattern analysis and machine intelligence. 21 21(5):433 449, 1999.
[20] Wolfgang Kabsch. A solution for the best rotation to relate two sets of vectors. Acta Crystal- lographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography, 32(5):922-923, 1976.
[21] Teuvo Kohonen. The self-organizing map. Proceedings of the IEEE, 78(9):1444-144) 1990.
[22] Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly, Thomas Unterthiner, and Xiaohua Zhai. An image is worth 16x16 words: Transformers for image recognition at scale. 2021.
[23] Jiahao Li, Changhao Zhang, Ziyao Xu, Hangning Zhou, and Chi Zhang, Iterative distance-aware similarity matrix convolution with mutual -upeerispe point climination for efficient point cloud registration. In European conference on computer vision, pages 578-394. Springer, 2020.
[24] Yang Li and Tatsuya Harada. Lepard: Learning partial point cloud matching in rigid and deformable scenes. arXiv preprint arXiv:2111.125999 2021.
[25] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10022-10022, 2021.
[26] Taewon Min, Chonghyuk Song, Eunseok Kim, and Inwook Shim. Distinetiveness oriented posi- tional equilibrium for point cloud registration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5490-5498, 2021.
[27] Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. Image transformer. In International Conference on Machine Learning, pages 4055 4064. PMLR, 2018
[28] Charles R Qi, Hao Su. Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.
[29] Zheng Qin, Hao Yu, Changjian Wang, Yulan Guo, Yuxing Peng, and Kai Xu. Geometric transformer for fast and robust point cloud registration. arXiv preprint arXiv:2202.066 06688, 2022.
[30] Radu Bogdan Rusu, Nico Blodow, and Michael Beetz. Fast point feature histograms (fpfh) for 3d registration. In 2009 IEEE international conference on robotics and automation, pages 3212-3217. IEEE, 2009.
[31] Paul-Fdouard Sarlin, Daniel DeToone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEÉF/CVF conference on computer vision and pattern recognition, pages 4938 4938, 2020.
[32] Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position repre- sentations. arXiv preprint arXiv:1803.02155. 2018.
[33] Richard Sinkhorn and Paul Knopp. Concerning nonnegative matrices and doubly stochastic matrices. Pucific Journal of Mathematics, 21(2):343-348, 1967.
[34] Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. arXiv preprint arXiv:2104.09866 2021.
12

---

[35] Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector- free local feature matching with transformers. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 19922-89311 2021.
[36] Kun Sun, Wenbing Tao, and Yuhua Qian. Guide to match: Multi-layer feature matching with hybrid gaussian mixture model. IEEE Transactions on Multimedia, 22(9):224-22611, 2019.
[37] Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, and Yichen Wei. Circle loss: A unified perspective of pair similarity optimization. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 6398-6407, 2020.
(38) Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6411-6420, 2019.
[39] Federico Tombari, Samuele Salti, and Luigi Di Stefano. Unique signatures of histograms for local surface description. In European conference computer vision, pages 356-369, Springer, 2010.
[40] Shinji Umeyama. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence, 13((4))376-38)) 1991.
[41] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[42] Haiping Wang, Yuan Liu, Zhen Dong, Wenping Wang, and Bisheng Yang. You only by- pothesize once: Point cloud registration with rotation-equivariant descriptors. arXiv preprint arXiv:2109.00182 2021.
[43] Yue Wang and Justin M Solomon. Deep closest point: Learning representations for point cloud registration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3523-3532, 2019.
[44] Yue Wang, Yonghin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph enn for learning on point clouds. Acm Transactions On Graphics (tog), 38(5):1-12, 2019.
[45] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 206-215, 2018.
[46] Zi Jian Yew and Gim Hee Lee. 3dfeat-net: Weakly supervised local 3d features for point cloud registration. In Proceedings of the European Conference on Computer Vision (ECCV), pages 607-623, 2018.
[47] Zi Jian Yew and Gim Hee Lee. Regtr: and-to-end point cloud correspondences with transformers. arXiv preprint arXiv:2203. 14517, 2022.
[48] Hao Yu, Fu Li, Mahdi Saleh, Benjamin Busam, and Slobodan Ilic. Cofinet: Reliable coarse- to-fine correspondences for robust pointeloud registration. Advances in Neural Information Processing Systems, 34, 2021.
[49] Andy Zeng. Shuran Song, Matthias Nießner. Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1802- 1811, 2017.
[50] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 16259- 16268, 2021.
13

---

[51] Lifa Zhu, Haining Guan, Changwei Lin, and Renmin Han. Neighborhood-aware geometric encoding network for point cloud registration. arXiv preprint arXiv:2201. 2022.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See the supplemental material.
(c) Did you diseuss any potential negative societal impacts of your work? [Yes] See the supplemental material.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code. data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)? [Yes] See the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See the supplemental material.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See the supplemental material.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 4.
(b) Did you mention the license of the assets? [Yes] See the supplemental material.
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "szX9PZcpCO", "reviewer": "Reviewer_VUqY", "summary": "This paper proposes a progressive point cloud registration method that estimates an initial transformation between two point clouds before registration estimation. The authors claim that one inlier is enough to estimate a good enough initial transformation between two point clouds, which boosts the downstream point cloud registration process, including positional encoding and progressive alignment. The main contributions of this paper are a) the one inlier finding process which is light-weight, b) the process can be jointly optimized with other steps, and c) the progressive alignment approach reduces the dependency on initialization.", "questions": "#### Q1 (ad W2): Inaccurate or Unrelated Citations\nline 32: \"... the straightforward position encoding is not a good idea [23].\"  \nIn [23] it shows up to 3.8% improvement on 4DLoMatch and 1.5% higher RR on 3DMatch and 2.3% on 3DLoMatch.\nThis does not imply straightforward position encoding is not a good idea.\nTo make this statement true, one should show the comparison with/without direct position encoding, which is not provided in the paper.\n\nline 51: \"This shows that only one inlier is enough to preserve the spatial consistency [2, 7] ...\"  \nI am not sure why those two papers are cited here. Do they mention anything related to this statement?\n\n#### Q2 (ad W3): Claims Without Proof:\nline 286: \" This is because ... both consider the position information ...\"  \nTo justify this statement, one should provide an ablation study on a model with only the difference with and without positional encoding. However, this is not provided in the evaluation section.\n\nline 342: \"our method is scene-agnostic and maintains good registration accuracy in strongly differing scenarios.\"  \nWithout a description of how the network was trained in those experiments, it is impossible to know whether the written sentence is true. \n\n#### Q3: Additional questions\n- In Figure 3, the inlier ratio for 3DLoMatch. Why does the 0.0 percentage increase after optimization?", "limitations": "The authors do not mention limitations in the main paper. No text in the main paper indicates a limitation section in the supplementary material.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths:\n#### S1. Core Idea.\nThe idea is novel and outperforms other methods in two public benchmarks.\n#### S2. Impact on Applications.\nThe method is lightweight and can potentially be applied in many other pipelines / with other methods, which benefits the community.\n#### S3. The paper is well written and easy to follow.\n\n### Weaknesses:\n#### W1. Limited theoretical / technical Novelty\nAlthough the paper is novel regarding the idea, the theoretical and technical novelty is limited.\n#### W2. Inaccurate Claims\nThis paper has made some claims with some cited papers that are inaccurate or unrelated (details below).\n#### W3. Missing Justification\nSome claims are made without proper proof (see details below).\n#### W4. Minor Typos / Writing\n* line 134: \"C is ...\" -> \"where C is ...\"\n* Eq 5: \"..., M,\" -> \"..., M.\"", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "s2pcVnRoimF", "reviewer": "Reviewer_2PZE", "summary": "This paper studies correspondence-based rigid point cloud registration problem and proposes a new position encoding method for point clouds to establish better correspondences. A virtual corresponding point pair, which is regarded as the one inlier, is first constructed from a set of real correspondences, and then the two point clouds are normalized by using the two points in the correspondence as reference and the position embedding is extracted from the normalized point clouds. Point features and position embedding are added together for establishing correspondences. Based on the above method of establishing correspondences, an iterative strategy is adopted to align two point clouds gradually. Experiments are conducted on 3DMatch, 3DLoMatch and KITTI, and sota registration recall are achieved on these datasets.", "questions": "1. In section 4.1, especially for the experiments on 3DLoMatch, the proposed method consistently has higher RR but lower FMR and IR. I agree that FMR and IR are not total positively-correlated to RR, but more analysis is needed to show why the proposed method has higher RR with a far lower IR, and how this is related to the proposed position encoding strategy. \n2. For the “Progressive alignment”, how many iterations are performed?\n3. For Fig. 3, the inlier ratio of every iteration should be given, instead of only the initialization and the final optimized one.  \n4. In Table 2, what constitutes the “Pose” Time? Is it the runtime of the RANSAC 50k? If so, why is the proposed method much faster than GeoTransformer with lower IR?\n5. What’s the number of sample correspondences in the runtime experiment in Table 2?\n6. The quality of the “one-inlier” for position encoding  should be given in each iteration.\n7. RRE and RTE should be reported for the experiments on 3DMatch and 3DLoMatch since the objective of this paper is registration. \n", "limitations": "The authors talk about the limitation of the proposed method in supplementary material. I suggest address the issue about if the proposed method is applicable to registration of objects, such as models in ModelNet40. ", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1.The proposed one-inlier based position encoding method is novel. The method is straightforward and effective. \n2.Experiments are conducted on three typical datasets and comparison are done to latest methods. High performance is achieved.\n\nWeakness:\n1. A good position encoding should be helpful to find more and better correspondences and a better registration is only a by-product of better correspondences, but the experimental results show that the IR and FMR of the proposed method are far lower than that of GeoTrans. Analysis are needed about this issue.\n2. The last two contributions are not significant. Especially, iterative optimization is widely used in recent learning-based registration methods, such as PRNet, RPM-Net.\n3. The proposed position-encoding idea is not fully analyzed. See question No.1. \n\nMinor issue:\n1. The part “one-inlier is enough” in title is a little misleading.  \n2. In Fig.2, it should be the Matched Correspondences instead of {R, t}, that are transferred from the Inlier Learning module to the Progressive Alignment module.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "8oEPpglFeng", "reviewer": "Reviewer_fXaq", "summary": "This paper proposes a position encoding method by using optimal transport and use it to normalize each point. The benefit is that the  optimal transport based position encoding can mitigate the feature ambiguity. Then, an iterative optimization process is introduced to establish the correspondence and transformation matrix. ", "questions": "Why the inlier ratio is relatively low?", "limitations": "NO", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. The idea is reasonable and the overall performance is good.\n2. The presentation is easy to understand.\n\nWeakness:\n1. I am a bit worried about the robustness of the proposed algorithm since the inlier ratio is significantly dropped.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "31SJOF-G23t", "reviewer": "Reviewer_dDv9", "summary": "This paper presents a learning-based algorithm that establishes reliable correspondences for registering a pair of point clouds.\nPoint cloud registration has been a long studied problem in robotics and computer vision, and there exist many papers that propose learning-based methods to perform feature learning and establish correspondences.\nIn my view, there are two unique ideas about this paper that differ it from previous papers:\n- The proposed pipeline predicts an \"anchor\" point for each point cloud (in the paper this is called \"one inlier\") and uses the \"anchor\" point to \"normalize\" the rest of the point cloud (in the paper, the normalization is basically subtracting the anchor point from each other point in the cloud). After normalization, an additional feature learning step is performed to augment the coarse features (learned by KPConv).\n- The proposed pipeline does correspondence learning and transformation estimation in an iterative manner, similar to the classical ICP algorithm. In the classical ICP, correspondences are established via nearest neighbor search. But in this paper, correspondences are established from learning, using the features learned from the \"anchoring\" idea mentioned above.\n\nThis paper then tests the proposed pipeline in the 3DMatch and 3DLoMatch dataset, and demonstrates the proposed pipeline delivers similar or better performance compared to several previous learned features for point cloud registration.", "questions": "Since you know the groundtruth pose, could you add into the loss function a pose error and see how it affects the performance?", "limitations": "Limitations not mentioned.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The paper is overall well written and easy to understand.\n+ The idea of predicting or voting an \"anchor\" point and using it to normalize the point cloud seems to be relatively new (though I feel this idea is similar to Hough voting).\n+ Experiments are well conducted and convincing.\n\nWeaknesses:\n- I feel the paper is a little bit out of fashion/date, given that the community has designed many **supervised** learned point cloud feature descriptors and moved to studying **unsupervised** or \"self-supervised\" feature descriptors. The proposed descriptor, though outperforming others, is still supervised and requires mining ground-truth correspondences for supervision. [Ref1] and others, for example, have already shown the possibility to learn features in a self-supervised way. Therefore, I think the paper is making an incremental contribution rather than a major contribution.\n- The title of this paper, to me, is deceiving and jargoning. One inlier correspondence is NOT enough to register a pair of point clouds, and at least THREE inlier correspondences (with noncollinear points) are required to define a unique rigid transformation (see [Ref2]). What the paper really means is to predict a \"anchor\" point (sort of like the centroid of the point cloud) and use the anchor point to subtract all other points. I think the name of the paper needs to be better worded.\n\n\n[Ref1] Yang H, Dong W, Carlone L, Koltun V. Self-supervised geometric perception. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2021 (pp. 14350-14361).\n[Ref2] Horn BK. Closed-form solution of absolute orientation using unit quaternions. Josa a. 1987 Apr 1;4(4):629-42.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept