Title: A robust inlier identification algorithm for point cloud registration via $\mathbf{\ell_0}$-minimization
Venue: NeurIPS 2024
Accepted: yes
Keywords: Point cloud registration

Inlier identification

Optimization
Abstract: Correspondences in point cloud registration are prone to outliers, significantly reducing registration accuracy and highlighting the need for precise inlier identification. In this paper, we propose a robust inlier identification algorithm for point cloud registration by reformulating the conventional registration problem as an alignment error $\ell_0$-minimization problem. The $\ell_0$-minimization problem is formulated for each local set, where those local sets are built on a compatibility graph of input correspondences. To resolve the $\ell_0$-minimization, we develop a novel two-stage decoupling strategy, which first decouples the alignment error into a rotation fitting error and a translation fitting error. Second, null-space matrices are employed to decouple inlier identification from the estimation of rotation and translation respectively, thereby applying Bayesian theory to $\ell_0$-minimization problems and solving for fitting errors. Correspondences with the smallest errors are identified as inliers to generate a transformation hypothesis for each local set. The best hypothesis is selected to perform registration. We demonstrate that the proposed inlier identification algorithm is robust under high outlier ratios and noise through experiments. Extensive results on the KITTI, 3DMatch, and 3DLoMatch datasets demonstrate that our method achieves state-of-the-art performance compared to both traditional and learning-based methods in various indoor and outdoor scenes.

A robust inlier identification algorithm for point cloud registration via ℓ 0 -minimization

Abstract

1 Introduction

Recent works in point could registration with outliers can generally be categorized into three groups: learning-based, geometry-only, and optimization-based methods. Learning-based methods [1, 8, 19]

On the other hand, optimization-based methods [4, 5, 18, 40, 46] solve the registration problem by formulating some non-convex objectives [18]. The Branch-and-Bound (BnB) algorithm is widely used to solve non-convex objectives [4, 5, 40] due to its ability to guarantee global optimality. However, the efficiency of BnB is affected by the dimensions of search space and the bounds on objectives [18], which may lead to worst-case exponential time [18, 39]. An alternative approach is to relax the non-convex registration problem into a convex semidefinite program [3, 39]. However, semidefinite relaxation is computationally expensive and may introduce outliers or noise, leading to poor estimation results. Therefore, achieving robust and efficient registration in scenarios with high outlier ratios and noise remains a challenging problem.

A novel robust inlier identification algorithm is proposed by reformulating the conventional registration as an alignment error ℓ 0 -minimization problem, which can effectively identify inliers and perform accurate registration under high outlier ratios and noise.

A two-stage decoupling strategy is designed for the proposed ℓ 0 -minimization problem. This strategy first decouples rotation and translation, and then decouples inlier identification from rotation or translation estimation.

A robust Bayesian-based approach is proposed to solve the decoupled ℓ 0 -minimization problem and identify inliers, enhancing the algorithm's performance on noisy data.

2 Related Work

Learning-based methods. Inspired by the success of deep learning in 3D perception [34, 32, 37, 29], recent works have adopted learning networks for point cloud registration [1, 8, 19, 20, 44]. Deep

3 Methods

3.1 Conventional Point Cloud Registration Problem Statement

Given the source point cloud P = { p i ∈ R 3 | i = 1 , . . . , N } and target point cloud Q = { q i ∈ R 3 | i = 1 , . . . , M } , the objective of point cloud registration is to align these two point clouds by estimating an optimal rigid transformation T = { R , t } , where R ∈ SO(3) denotes the rotation matrix and t ∈ R 3 denotes the translation vector. The transformation is then solved by the following registration problem [5, 25]:

3.2 Inlier Identification via ℓ 0 -minimization

<latexi sh

1_b

64="RBAmGEjoYVJKSO

FNk

WdZ

I

>

D

n

c

HL

w

835

2

y7

0C

T

u

M

z

p

r

gP

Q

9

/

Xq

v

U

f

+

<latexi sh

1_b

64="W

S

Jg2

j

FHw

r

/k

>

Cz3

c

LT

D

U

+I

m

Z

Q

PB

p

G

M5

Nu9

f

o

v

7

K

y

n

Y

8

R

E

<latexi sh

1_b

64="yB2m

Ffj

0

JYRwO8N

g

o

>A

Cz3

c

VHLT

D

U

+I

d

kZ

q

S

Q

P

p

G

M5

u9

/

vW7

r

X

K

n

E

<latexi sh

1_b

64="Y

G5pB02

HXD3I

C

Q

y

7

E

>A

z

cjV

LT

J

F

U

+

d

m

kZ

q

R

S

Nu9

f

/

o

O

vW

r

K

n

w

P

8

<latexi sh

1_b

64="0kFjm

AX

T

ud+B

fZMQ

N

>

Cz3

c

VHL

J

D

U

I

q

v

G

S

Y

7

g

2

OPW

r

8

/

w

nR

5

p

E

o

9

y

K

<latexi sh

1_b

64="Ij/q zv

V3XwcO

Y

ukp8

>A

Cy

HLS

N

FD2

r

f

Zd

g

R

E

U

9o

T

M

m

W

K

P

+

0Q

5

n

G7

J

B

<latexi sh

1_b

64="Vn

MrcG

g

C

ILOH

798

05

T

Q

>A

z3

j

J

FD

U

+

d

m

kZ

q

R

S

pB

2

Nu

f

/

o

vW

X

K

y

Y

w

PE

<latexi sh

1_b

64="uFc57

Vf

Hn

g

vkAr

0

8

>

Cz3

j

LT

J

D

U

+I

d

m

Z

q

R

E

S

M

pB

G

2

N

9

/

o

O

W

X

K

Q

y

Y

w

P

<latexi sh

1_b

64="

qc wC8

YX

M2

DNPL

3+VQ

>A

z

j

H

T

J

F

U

I

d

m

kZ

o

S

p

G

9

f

/0

R

5

W

E

u

v

B

g

n

7

K

r

y

O

<latexi sh

1_b

64="qd

/ZHC5

RN8JPWL3oG

>A

0

cjV

S

FD2

r

pf

u

k

w

n

O

K

X

+

9

vE

M

Y

yQ

7

z

IT

B

m

Input Correspondences

Inlier Identification

via

ℓ

𝟎

-minimization

min

[𝐨

!

, 𝐨

"

, 𝐨

#

, 𝐨

$

]

ℓ

!

min

[𝐨

&

, 𝐨

'

, 𝐨

(

, 𝐨

)

]

ℓ

!

𝐨

!

𝐨

"

𝐨

#

𝐨

$

ℓ

*

-minimization problem

. . .

𝐨

%

𝐨

&

𝐨

'

𝐨

(

C

=

{

c

i

|

i

= 1

, . . . , N

c

}

𝐑

+

∗

𝐑

!

∗

⋯

Stage 1

: Decouple alignment error into

translation

fitting error

𝐨 ,

.

Stage 2

:

Bayesian-based

inlier identification

Stage 2

:

Bayesian-based

inlier identification

𝐨 "

𝟐

𝐨 "

𝟕

⋯

⋯

⋯

𝐭

+

∗

𝐭

!

∗

⋯

Two-stage

Decoupling Strategy

Hypothesis Selection

Inlier Identification via

ℓ

𝟎

-minimization

Two-stage Decoupling Strategy

⋯

c

2

c

4

c

5

c

3

c

6

c

7

c

8

c

9

𝐨 #

"#

𝐨 #

"!

𝐨 #

&(

Stage 1

: Decouple alignment error into

rotation

fitting error

𝐨 .

.0

Define alignment error

𝐨

.

for

correspondence

𝐜

.

in each local set

an inlier ( i.e., it satisfies | Rp k i + t -q k i | ≤ ξ k i with the Gaussian noise ξ k i ), o k i should ideally be a zero vector. Consequently, the indices of zero vectors in the solution O ∗ k of Eq. (2) correspond to the inlier indices in the k -th set. The formulations for other local sets are defined in a similar way.

A key insight into our approach is the use of ℓ 0 norm to optimize alignment errors. This is based on the principle that only inliers can be fitted by the same transformation [5], and the optimal transformation is estimated as the one that fits the largest number of inlier correspondences. Therefore, our optimization objective is to maximize the count of zero vectors in the alignment error. Compared to the common formulations for point cloud registration [18], such as consensus maximization [5, 7] and truncated least-squares [39], our formulation reduces the impact of outliers through ℓ 0 norm. The focus of this norm is to minimize the number of non-zero vectors rather than their magnitudes, thereby enhancing the robustness of our method to outliers and noise.

3.3 Two-stage Decoupling Strategy

To resolve the proposed ℓ 0 -minimization problem, we design a two-stage decoupling strategy. The solution process is described for the k -th local set and similarly applied to other local sets.

Decoupling the alignment error into rotation and translation fitting errors. Simultaneously estimating the rigid transformation with 6 degrees of freedom (DOF) is time-consuming due to the high-dimensional parameter space [5, 39]. To effectively resolve the ℓ 0 -minimization problem proposed in Eq. (2) for each local set, we decouple the 6-DOF transformation into 3-DOF rotation and 3-DOF translation by computing the relative positions between point pairs. For any two given points p k i and q k j in the k -th local set, the translation vector t k cancels out in the subtraction [39]:

Based on Eq. (3), we define ¯ q k ij = q k j -q k i and ¯ p k ij = p k j -p k i as the relative positions. ¯ o k ij = o k j -o k i represents the rotation fitting error to minimize, unaffected by translation. ¯ ξ k ij = ξ k j -ξ k i is the Gaussian noise. If both the i -th and j -th correspondences are inliers, ¯ o k ij should ideally be a zero vector. Therefore, the rotation fitting error ¯ o k ij for the correspondence pair c k i = ( p k i , q k i ) and c k j = ( p k j , q k j ) is formulated as:

Having decoupled rotation from translation, we can now formulate the ℓ 0 -minimization problem for the rotation fitting error ¯ O k in the k -th local set, focusing on the 3-DOF rotation R k :

Once obtaining the rotation estimate R ∗ k by solving Eq. (5), we can substitute it back into Eq. (2) to estimate the translation. The ℓ 0 -minimization problem for the translation fitting error ˆ O k is formulated as follows, focusing on the 3-DOF translation t k :

where ¯ Q k ∈ R ¯ N × 3 and ¯ P k ∈ R ¯ N × 3 are relative positions between all point pairs in Q k and P k , respectively. Here, ¯ N = N 2 ( N 2 -1) 2 is the number of relative point pairs in a local set. The Gaussian noise ¯ Ξ k is modeled as N (0 , λ R I ) , where λ R indicates the variance.

where Ξ k is modeled as N (0 , λ t I ) , where λ t indicates the variance of Gaussian noise. The translation t ∗ k is estimated by solving Eq. (6).

Decoupling rotation estimation from ℓ 0 -minimization. Optimizing the estimation of rotation while simultaneously identifying inliers is a chicken-and-egg problem, because reliable identification of inliers depends on the precise rotation estimation (as shown in Eq. (5)). To address this, we further decouple inlier identification from the estimation of rotation. The inliers that can be fitted by the same rotation are identified through Bayes Theorem and used for the subsequent rotation estimation.

We incorporate a robust Bayesian approach to solve Eq. (5), improving the algorithm's robustness to noisy data [42]. The key step is to define a null-space matrix ¯ Θ k , whose rows form a basis for the left null space of ¯ P k . By left-multiplying each term in the constraint of Eq. (5) with ¯ Θ k , the component associated with the rotation R k is eliminated:

where ¯ Θ k ¯ O k represents the transformed rotation fitting error. Given that ¯ Ξ k is Gaussian noise and the left-multiplication by ¯ Θ k is a linear operation, ¯ Θ k ¯ Ξ k also follows a Gaussian distribution with a covariance matrix of λ R ¯ Θ k ¯ Θ T k . The likelihood is formulated as:

where ˜ ¯ Q k = ¯ Θ k ¯ Q k and ¯ Θ k ¯ Θ T k = ¯ Π k . Based on the Bayes Theorem and Maximum A Posteriori (MAP) estimate, the unconstrained optimization for rotation fitting error ℓ 0 -minimization in Eq. (5) is redefined as:

where λ R is the regularization parameter that trades off the fitting error and model complexity. However, since the formulation incorporating the ℓ 0 norm is known to be computationally expensive, we use the following convex relaxation:

where ∥ · ∥ F is the Frobenius norm ( F norm), which is both differentiable and convex. To find the optimal solution, we set the gradient of the objective function with respect to ¯ O k to zero:

The optimal explicit solution ¯ O ∗ k can be directly calculated as:

Based on ¯ O ∗ k , we identify topK R correspondences with minimal rotation fitting error for accurate rotation estimation. These correspondences, indexed by I R , provide the basis for estimating rotation from the SVD decomposition of the matrix H = U Σ V T ∈ R 3 × 3 [23]. For the k -th local set, the rotation hypothesis is estimated as [5, 2]:

Decoupling translation estimation from ℓ 0 -minimization. Employing a strategy similar to that used for rotation estimation, we utilize a null-space matrix Θ k that satisfies Θ k 1 = 0 to isolate the translation. By applying Θ k to the transpose of the constraint in Eq. (6), we eliminate the components associated with translation t k :

Incorporating the Bayes Theorem, we formulate the following convex relaxation for the unconstrained optimization problem:

where X k = Θ k ( Q T k -( P k R ∗ k ) T ) and Π k = Θ k Θ T k . The explicit solution ˆ O ∗ k is obtained by solving the gradient of the objective function with respect to ˆ O k :

where I denotes the identity matrix. Using ˆ O ∗ k , topK t correspondences with the smallest errors are identified as inliers for translation estimation. Their index set is denoted as I t . The translation hypothesis t ∗ k for the k -th local set is estimated based on these inliers ( p k i , q k j ) , with ( i, j ) ∈ I t .

3.4 Hypothesis Selection

Finally, we evaluate and select the best estimation from the transformation hypotheses computed for all local sets:

where N c is the number of input initial correspondences and τ is a predefined error threshold. For each transformation hypothesis, we quantify its effectiveness by counting the number of correspondences that satisfy the constraints within τ . The transformation with the highest inlier count is selected for registration.

4 Experiments

4.1 Datasets and Experimental Setup

Synthetic dataset. We evaluate the accuracy, robustness, and efficiency of our algorithm using the Bunny point cloud from the Stanford 3D Scan Repository [10]. Similar to [5, 39], the Bunny model is downsampled to N c points and resized to fit a [0 , 1] 3 cube, creating the source point cloud P . To generate the target point cloud Q , a random transformation ( R , t ) is applied to P and then Gaussian noise ϵ i ∼ N (0 , σ 2 I 3 ) is added. A pair of the original and moved points defines an inlier. The inliers are contaminated with outliers generated by random transformations.

Outdoor scenes. For evaluations on outdoor scenes, we conduct experiments on the KITTI dataset [13]. Following [5, 6], we use 555 pairs of point clouds from scenes 8 to 10 for testing. We construct a 30 cm voxel grid to downsample point clouds and form correspondences using handcrafted FPFH [26] and learned FCGF [9] descriptors.

Indoor scenes. We conduct experiments on the 3DMatch dataset [44] to evaluate performance on indoor scenes. Following [5, 6, 45], we use RGB-D scans from 8 real indoor scenes for testing. The

0

1

2

3

4

5

10

20

30

40

50

60

70

80

90

FGR

RANSAC

TEASER++

MAC

Ours

0

3

6

9

12

15

18

21

10

20

30

40

50

60

70

80

90

FGR

RANSAC

TEASER++

MAC

Ours

Rotation Error (degrees)

Translation Error (cm)

1

10

100

91

92

93

94

95

96

97

98

99

FGR

RANSAC

TEASER++

MAC

Ours

Rotation Error (degrees)

0.1

1

10

100

FGR

RANSAC

TEASER++

MAC

Ours

91

92

93

94

95

96

97

98

99

Translation Error (cm)

0

50

100

150

200

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

FGR

RANSAC

TEASER++

MAC

Ours

Noise

Rotation Error (degrees)

0

100

200

300

400

500

FGR

RANSAC

TEASER++

MAC

Ours

Translation Error (cm)

Noise

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

point clouds are downsampled using a 5 cm voxel grid. We use the hand-crafted FPFH [26] along with two learned descriptors, FCGF [26] and 3DSmoothNet [14], for feature extraction. To evaluate our method in more challenging scenarios, we conduct experiments on 3DLoMatch [17] (overlap rate between scenes < 30% ). Following [5, 19], the Predator descriptor [17] is used in 3DLoMatch.

Evaluation criteria. Following [5, 39], we use the rotation error (RE), translation error (TE), and registration recall (RR) as evaluation metrics. The registration is considered successful when the RE ≤ 15 ◦ , TE ≤ 30 cm on 3DMatch & 3DLoMatch datasets, and RE ≤ 5 ◦ , TE ≤ 60 cm on KITTI dataset. Average RE and TE are computed only on the successfully registered pairs [5, 6].

Implementation details. We implement our method in PyTorch [24]. All the experiments are conducted on a machine with an Intel Xeon Gold 6134 CPU and a single NVIDIA GTX3090.

4.2 Evaluation on Synthetic Dataset

Robustness to outliers. We evaluate the robustness to outliers by increasing the outlier ratio from 10% to 90% . The Bunny point cloud is downsampled to N c = 500 . We add zero-mean Gaussian noise with a standard deviation set to σ = 0 . 01 . For each outlier ratio, we conduct 50 independent trials and report the average rotation error (RE) and translation error (TE). We compare our method with state-of-the-art traditional methods [4, 12, 39, 45, 46]. As shown in the first row of Fig. 2, the rotation and translation errors of FGR [46] increase sharply as the proportion of outliers increases. RANSAC [12] and GORE [4] start failing at an outlier ratio of 60% . Our method remains robust to outliers up to 90% and produces more accurate estimates than all other methods. We further

1

10

100

FGR

RANSAC

TEASER++

TDS

Rotation Error

(degrees)

0

4

8

12

16

FGR

RANSAC

TEASER++

TDS

Translation

Error (cm)

(a) Efficiency comparisons

(b) Effectiveness of TDS

Time (s)

0.0001

1

10000

FGR

RANSAC

TEASER++

MAC

Ours

250

500

1000

2500

5000

Number of correspondences

compare the performance of different methods under extreme outlier ratios, i.e., when the outlier ratio increases from 91% to 99% . The second row of Fig. 2 shows that even with outlier ratios as high as 99% , our method continues to perform well, consistently producing lower transformation errors than other methods.

Robustness to noise. We further evaluate the robustness against Gaussian noise with different variances. As the noise standard deviation increases from σ = 0 . 01 to σ = 0 . 1 , the geometric structure of the Bunny model is completely destroyed [39] (refer to Appendix A.5). Fig. 3 shows the comparison results as σ increases from 0 . 01 to 0 . 09 . When the noise variance reaches 0 . 03 , the translation errors of geometric-only method MAC [45] significantly increase. Both FGR [46] and RANSAC [12] show large rotation errors when σ increases to 0 . 05 . In contrast, our method achieves the lowest rotation and translation errors under high noise, demonstrating its robustness to noise.

Efficiency and accuracy. We increase the number of correspondences N c from 250 to 5000 to compare efficiency and accuracy. We set the noise standard deviation σ to 0 . 01 and the outlier ratio to 50% . The comparison results are shown in Fig. 4(a). As the number of correspondences increases, the running time of GORE [4] and TEASER++ [39] increases significantly. Notably, when N c grows to 2500 , the running time of GORE is about 10 4 times longer than that of our method. Our method solves the ℓ 0 -minimization problems with explicit solutions, significantly enhancing efficiency through parallel matrix computations and GPU execution. The curves of FGR, RANSAC, MAC, and our method in Fig. 4(a) are flat and difficult to visually distinguish, indicating the efficiency of these methods. However, as shown in Appendix A.6, our method outperforms FGR [46], RANSAC [12], and MAC [45] in registration accuracy. Therefore, our inlier identification algorithm via ℓ 0 -minimization is efficient while maintaining high accuracy.

Effectiveness of the two-stage decoupling strategy. We evaluate the effectiveness of the twostage decoupling strategy (TDS) by formulating the ℓ 0 -minimization problem directly on the Bunny data instead of local sets and estimating the final rotation and translation. Specifically, we set N c = 100 and σ = 0 . 01 . As shown in Fig. 4(b), we compare the TDS with optimization-based

(c) SC

2

-PCR

(d) Ours

(b) MAC

(e) Ground-truth

(a) Input

4.3 Evaluation on Outdoor Scenes

To evaluate our algorithm on real outdoor scenes, we conduct experiments on the KITTI dataset [13]. The comparison results with state-of-the-art traditional [5, 6, 12, 18, 39, 45, 46] and learningbased [1, 8, 19] methods are reported in Table 1. We first use the FPFH [26] descriptor to generate initial correspondences. As shown in the left column of Table 2, our method outperforms traditional and learning-based methods on all metrics. For the most important criterion of registration recall (RR), our method improves by about 2% over the nearest competitor MAC [45]. Following [6], the average RE and TE are only calculated on successfully registered pairs, which makes methods with high registration recall more likely to have larger average errors. Nonetheless, our method still achieves the best results on RE and TE. Besides, we report the comparison results with the FCGF [9] descriptor in the right column of Table 2. Our method achieves the highest RR and the lowest RE due to its effective inlier identification algorithm. The superior performance demonstrates the ability of our method to align sparse and non-uniformly distributed data in outdoor scenes. In addition to its high accuracy, our method also achieves comparable efficiency, making it highly competitive for practical applications. The visualizations of registration results on KITTI are provided in Appendix A.12.

4.4 Evaluation on Indoor Scenes

We further conduct experiments on the 3DMatch [44] and 3DLoMatch [17] datasets to evaluate the performance in real indoor scenes. The comparison results are reported in Table 2 and Table 3.

Combined with FPFH, FCGF, and 3DSmoothNet descriptors. As shown in the left column of Table 2, compared to both traditional and learning-based methods, our method achieves the highest RR with the handcrafted FPFH [26] descriptor. The middle column of Table 2 reports the comparison results with the learned FCGF [9] descriptor. Our method achieves the lowest RE and TE. Compared to SC 2 -PCR [6], our method improves the RR, RE, and TE by 0 . 13% , 0 . 97% , and 0 . 77% respectively,

Alignment

True: 385 False: 31

True: 326 False: 90

True: 45 False: 371

True: 358 False: 58

TEASET

++

SC

2

-PCR

MAC

Ours

which benefit from our ℓ 0 -minimization formulation for inlier identification. Since TR-DE [5] and TEAR [18] have not made their code or results for FPFH and FCGF publicly available, their results are excluded in the left and middle columns of Table 2. Following [18], we also compare the registration accuracy using the learned 3DSmoothNet [14] to extract features. The results in the right column of Table 2 show that our method outperforms all other methods across all evaluation metrics, demonstrating the robustness of our method to different local descriptors. We show the results of qualitative comparisons in Fig. 5 and Appendix A.12. Methods such as MAC may fail in scenes with ambiguous features or limited overlap. Our algorithm still achieves satisfactory alignment and is close to the ground truth.

Robust to lower overlap ratios. Furthermore, we report results for a more challenging dataset: 3DLoMatch [17] (overlap rate < 30% ). Following [5, 19], we use the Predator [17] descriptor to generate the initial correspondences. We compare the registration recall (RR) under different numbers of correspondences. As shown in Table 3, the proposed method improves the average RR by 7% over TR-DE [5], demonstrating the effectiveness of our method in dealing with

low-overlap scenarios. In Fig. 6, we compare the output inlier ratio with traditional methods [6, 39, 45] in the low overlap scenario. Our method is more effective with more correct predicted inliers.

5 Conclusion

In this paper, we propose a robust inlier identification algorithm by reformulating the conventional registration problem as an alignment error ℓ 0 -minimization problem. For each local set, we resolve the ℓ 0 -minimization problem using a designed two-stage decoupling strategy. First, the alignment error is decoupled to a rotation fitting error and a translation fitting error, formulating two decoupled ℓ 0 -minimization problems. Second, null-space matrices are introduced to decouple the inlier identification from the estimation of rotation or translation respectively, there by applying a robust Bayesian approach to decoupled ℓ 0 -minimization problems and solving for fitting errors. Correspondences with the smallest errors are identified as inliers to generate a transformation hypothesis for each local set. We experimentally demonstrate that the proposed algorithm is robust to high outlier ratios and noise. Extensive results on the KITTI, 3DMatch, and 3DLoMatch datasets also demonstrate that our method achieves state-of-the-art registration accuracy while being comparable in efficiency in both indoor and outdoor scenes. Limitations and broader impact are discussed in Appendix A.10.

References

Bai, X., Luo, Z., Zhou, L., Chen, H., Li, L., Hu, Z., Fu, H., Tai, C.L., 2021. Pointdsc: Robust point cloud registration using deep spatial consistency, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15859-15869.

Besl, P.J., McKay, N.D., 1992. Method for registration of 3-d shapes, in: Sensor fusion IV: control paradigms and data structures, Spie. pp. 586-606.

Briales, J., Gonzalez-Jimenez, J., 2017. Convex global 3d registration with lagrangian duality, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4960-4969.

Bustos, A.P., Chin, T.J., 2017. Guaranteed outlier removal for point cloud registration with correspondences. IEEE transactions on pattern analysis and machine intelligence 40, 2868-2882.

Chen, W., Li, H., Nie, Q., Liu, Y .H., 2022a. Deterministic point cloud registration via novel transformation decomposition, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6348-6356.

Chen, Z., Sun, K., Yang, F., Tao, W., 2022b. Sc2-pcr: A second order spatial compatibility for efficient and robust point cloud registration, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13221-13231.

Chin, T.J., Suter, D., 2022. The maximum consensus problem: recent algorithmic advances. Springer Nature.

Choy, C., Dong, W., Koltun, V., 2020. Deep global registration, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2514-2523.

Choy, C., Park, J., Koltun, V., 2019. Fully convolutional geometric features, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 8958-8966.

Curless, B., Levoy, M., 1996. A volumetric method for building complex models from range images, in: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 303-312.

Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., Theobalt, C., 2017. Bundlefusion: Real-time globally consistent 3d reconstruction using on-the-fly surface reintegration. ACM Transactions on Graphics (ToG) 36, 1.

Fischler, M.A., Bolles, R.C., 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24, 381-395.

Geiger, A., Lenz, P., Urtasun, R., 2012. Are we ready for autonomous driving? the kitti vision benchmark suite, in: 2012 IEEE conference on computer vision and pattern recognition, IEEE. pp. 3354-3361.

Gojcic, Z., Zhou, C., Wegner, J.D., Wieser, A., 2019. The perfect match: 3d point cloud matching with smoothed densities, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5545-5554.

Guo, J., Wang, Q., Park, J.H., 2020. Geometric quality inspection of prefabricated mep modules with 3d laser scanning. Automation in Construction 111, 103053.

Halber, M., Funkhouser, T., 2017. Fine-to-coarse global registration of rgb-d scans, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1755-1764.

Huang, S., Gojcic, Z., Usvyatsov, M., Wieser, A., Schindler, K., 2021. Predator: Registration of 3d point clouds with low overlap, in: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 4267-4276.

Huang, T., Peng, L., Vidal, R., Liu, Y.H., 2024. Scalable 3d registration via truncated entry-wise absolute residuals. arXiv preprint arXiv:2404.00915 .

Jiang, H., Dang, Z., Wei, Z., Xie, J., Yang, J., Salzmann, M., 2023. Robust outlier rejection for 3d registration with variational bayes, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1148-1157.

Jiang, Y., Zhou, B., Liu, X., Li, Q., Cheng, C., 2024. Gtinet: Global topology-aware interactions for unsupervised point cloud registration. IEEE Transactions on Circuits and Systems for Video Technology .

Lai, K., Bo, L., Fox, D., 2014. Unsupervised feature learning for 3d scene labeling, in: 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE. pp. 3050-3057.

Mises, R., Pollaczek-Geiringer, H., 1929. Praktische verfahren der gleichungsauflösung. ZAMM-Journal of Applied Mathematics and Mechanics/Zeitschrift für Angewandte Mathematik und Mechanik 9, 58-77.

Papadopoulo, T., Lourakis, M.I., 2000. Estimating the jacobian of the singular value decomposition: Theory and applications, in: Computer Vision-ECCV 2000: 6th European Conference on Computer Vision Dublin, Ireland, June 26-July 1, 2000 Proceedings, Part I 6, Springer. pp. 554-570.

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al., 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32.

Qin, Z., Yu, H., Wang, C., Guo, Y., Peng, Y., Xu, K., 2022. Geometric transformer for fast and robust point cloud registration, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11143-11152.

Rusu, R.B., Blodow, N., Beetz, M., 2009. Fast point feature histograms (fpfh) for 3d registration, in: 2009 IEEE international conference on robotics and automation, IEEE. pp. 3212-3217.

Rusu, R.B., Blodow, N., Marton, Z.C., Beetz, M., 2008. Aligning point cloud views using persistent feature histograms, in: 2008 IEEE/RSJ international conference on intelligent robots and systems, IEEE. pp. 3384-3391.

Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., Fitzgibbon, A., 2013. Scene coordinate regression forests for camera relocalization in rgb-d images, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2930-2937.

Sun, J., Xie, Y., Chen, L., Zhou, X., Bao, H., 2021. Neuralrecon: Real-time coherent 3d reconstruction from monocular video, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15598-15607.

Turk, G., Levoy, M., 1994. Zippered polygon meshes from range images, in: Proceedings of the 21st annual conference on Computer graphics and interactive techniques, pp. 311-318.

Valentin, J., Dai, A., Nießner, M., Kohli, P., Torr, P., Izadi, S., Keskin, C., 2016. Learning to navigate the energy landscape, in: 2016 Fourth International Conference on 3D Vision (3DV), IEEE. pp. 323-332.

Wang, Y., Pan, Z., Li, X., Cao, Z., Xian, K., Zhang, J., 2022. Less is more: Consistent video depth estimation with masked frames modeling, in: Proceedings of the 30th ACM International Conference on Multimedia, pp. 6347-6358.

Wang, Y., Shi, M., Li, J., Hong, C., Huang, Z., Peng, J., Cao, Z., Zhang, J., Xian, K., Lin, G., 2024. Nvds + : Towards efficient and versatile neural stabilizer for video depth estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence doi: 10.1109/TPAMI.2024.3476387 .

Wang, Y., Shi, M., Li, J., Huang, Z., Cao, Z., Zhang, J., Xian, K., Lin, G., 2023. Neural video depth stabilizer, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9466-9476.

Wu, Y., Ding, H., Gong, M., Qin, A.K., Ma, W., Miao, Q., Tan, K.C., 2022. Evolutionary multiform optimization with two-stage bidirectional knowledge transfer strategy for point cloud registration. IEEE Transactions on Evolutionary Computation 28, 62-76.

Xiao, J., Owens, A., Torralba, A., 2013. Sun3d: A database of big spaces reconstructed using sfm and object labels, in: Proceedings of the IEEE international conference on computer vision, pp. 1625-1632.

Yang, B., Wen, H., Wang, S., Clark, R., Markham, A., Trigoni, N., 2017. 3d object reconstruction from a single depth view with adversarial learning, in: Proceedings of the IEEE international conference on computer vision workshops, pp. 679-688.

Yang, F., Guo, L., Chen, Z., Tao, W., 2022. One-inlier is first: Towards efficient position encoding for point cloud registration. Advances in Neural Information Processing Systems 35, 6982-6995.

Yang, H., Shi, J., Carlone, L., 2020. Teaser: Fast and certifiable point cloud registration. IEEE Transactions on Robotics 37, 314-333.

Yang, J., Li, H., Jia, Y., 2013. Go-icp: Solving 3d registration efficiently and globally optimally, in: Proceedings of the IEEE International Conference on Computer Vision, pp. 1457-1464.

Yew, Z.J., Lee, G.H., 2022. Regtr: End-to-end point cloud correspondences with transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6677-6686.

Yuan, Y., Tang, X., Zhou, W., Pan, W., Li, X., Zhang, H.T., Ding, H., Goncalves, J., 2019. Data driven discovery of cyber physical systems. Nature communications 10, 4894.

Yuan, Y., Wu, Y., Fan, X., Gong, M., Ma, W., Miao, Q., 2023. Egst: Enhanced geometric structure transformer for point cloud registration. IEEE Transactions on Visualization and Computer Graphics .

Zeng, A., Song, S., Nießner, M., Fisher, M., Xiao, J., Funkhouser, T., 2017. 3dmatch: Learning the matching of local 3d geometry in range scans, in: CVPR, p. 4.

Zhang, X., Yang, J., Zhang, S., Zhang, Y., 2023. 3d registration with maximal cliques, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17745-17754.

Zhou, Q.Y., Park, J., Koltun, V., 2016. Fast global registration, in: Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, Springer. pp. 766-782.

A Appendix

In the appendix, we first provide the detailed construction for local sets (Sec. A.1), the rigorous definitions of evaluation metrics (Sec. A.2), then describe the pseudocode for key parts (Sec. A.3) and the hyper-parameter selection (Sec. A.4). We further provide additional experimental results (Sec. A.5, Sec. A.6, Sec. A.7, and Sec. A.8), ablation studies on parameters (Sec. A.9), and discuss the limitations (Sec. A.10) and scalability (Sec. A.11) of our work. Finally, we show more qualitative results of registration on 3DMatch, 3DLoMatch, and KITTI (Sec. A.12) and provide the detailed information for these datasets (Sec. A.13).

A.1 Local set construction

In this section, we provide the detailed construction for local sets. We first construct a global compatibility graph for input correspondences. Specifically, we calculate the Euclidean distance between the correspondence pair ( c i , c j ) as follows:

where p i and p j denote points in the source point cloud and q i and q j are the corresponding points in the target point cloud. The first order compatibility score for each pair ( c i , c j ) is calculated based on the Euclidean distance, as follows:

where d t is the threshold for distance. When the distance difference between two correspondences is less than d t , they are considered compatible due to the length consistency of rigid transformations [6]. The hard compatibility matrix can be formulated as:

However, the first order compatibility measure suffers from outliers due to locality and ambiguity [6]. Following [45], we calculate the second order compatibility scores [6] as edges in the graph. The second order compatibility score between the correspondence pair ( c i , c j ) is computed based on the hard compatibility matrix:

where N c is the number of input correspondences. Based on the compatibility graph, we select K reliable correspondences as seeds and construct local sets for each seed. Specifically, following [1, 6], we use first-order compatibility scores to compute the leading eigenvectors via the power iteration method [22]. These leading feature vectors serve as confidence scores for reliable seed selection. For each seed, we explore its topN f neighbors in the second order measure space. Then, within each neighbor set, we recompute the second-order compatibility score and select the topN 2 ( N 2 < N f ) correspondences as the local set for the i -th seed [6].

A.2 Evaluation Metrics

Rotation Error (RE) measures the geometric distance in degrees between the estimated and groundtruth rotation matrices:

where R denotes the estimated rotation matrix and R gt denotes the ground-truth rotation matrix.

Translation Error (TE) measures the Euclidean distance between the estimated and ground-truth translation vectors:

where t denotes the estimated translation vector and t gt denotes the ground-truth translation vector.

Registration Recall (RR) measures the fraction of correctly registered point cloud pairs whose RE and TE are both below certain thresholds:

Following [1, 5, 6, 39], we compute the mean RE and TE only with the correctly registered point cloud pairs .

A.3 Pseudocode for key parts of our algorithm

There are two key parts in our algorithm: Inlier Identification via ℓ 0 -minimization and two-stage decoupling strategy.

Algorithm 1: Inlier Identification via ℓ 0 -minimization

Data: Source point cloud P = { p i ∈ R 3 | i = 1 , . . . , N } and target point cloud

Result: ℓ 0 -minimization problem of alignment error for each local set

1 Establish correspondences C through feature matching

2 Calculate the compatibility score S 2 ( c i , c j ) according to Eq. (18)-Eq. (21)

3 Select N 1 seed correspondences with high compatibility scores.

4 For i = 1 , 2 , . . . , N 1

5 Construct i -th local set, which containing N 2 compatible correspondences.

6 Formulate the ℓ 0 minimization problem of alignment error for the i -th local set according to Eq. (2)

7 EndFor

Algorithm 2: Two-stage decoupling strategy

Data: Given point pairs in the k -th local set P k = { p k i } K 2 i =1 and Q k = { q k i } K 2 i =1 , parameters λ R , λ t , K R , K t

Result: Estimated rotation R ∗ k and translation t ∗ k hypothesis for the k -th local set

1 % Decoupling the alignment error into a rotation fitting error and a translation fitting error

2 Calculate relative positions ¯ P k and ¯ Q k for all pairs: ¯ p k ij = p k j -p k i and ¯ q k ij = q k j -q k i

3 % Decoupling the inlier identification from the rotation estimation

4 Formulate the ℓ 0 -minimization problem for the rotation fitting error ¯ O k :

5 % Bayesian-based inlier identification and rotation estimation

6 Construct ¯ Θ k from the left null-space of ¯ P k : ¯ Θ k ¯ P k = 0

7 Eliminate the components related to R k in the constraints of rotation fitting error ℓ 0 -minimization: ¯ Θ k ¯ O k = ¯ Θ k ¯ Q k -¯ Θ k ¯ Ξ k

8 Define ˜ ¯ Q k = ¯ Θ k ¯ Q k and ¯ Π k = ¯ Θ k ¯ Θ T k and formulate the unconstrained problem for rotation fitting error:

9 The explicit solution can be calculated directly:

10 Solve R ∗ k using SVD on the identified top K R pairs with the smallest error ¯ P I R and ¯ Q I R

11 % Decoupling the inlier identification from the translation estimation

12 Based on the estimated R ∗ k , the ℓ 0 -minimization problem for the translation fitting error ˆ O k is formulated as:

13 Define Θ satisfying Θ1 = 0 . Eliminate the components associated with translation:

14 % Bayesian-based inlier identification and translation estimation

15 Define X k = Θ k ( Q T k -( P k R ∗ k ) T ) and Π k = Θ k Θ T k . Formulate the unconstrained optimization problem:

16 The explicit solution can be calculated directly:

17 Solve t ∗ k based on the identified topK t correspondences with the smallest error P I t and Q I t .

A.4 Hyper-parameter selection

The K 1 and K 2 are set to 30 and 20 for all experiments. For other hyper-parameters ( λ R , λ t , K R and K t ), we employ a grid search strategy with criterion of maximizing inliers. For a given set of

parameters ( λ R , K R ), the optimization criterion for R k is shown as:

where I R is the index set of the inlier correspondence pairs and the operation ⌊·⌋ denotes the cardinality of the set. Similarly, the selection criterion for the translation vector t k is established as follows:

where I R is the index set of inlier correspondences.

A.5 Impact of noise standard deviation

In this section, we visually illustrate the impact of noise standard deviation on the point cloud. As shown in Fig. 7, compared with (a) the clean Bunny model, when the noise standard deviation is increased to 0.01, the geometric structure of the model in (b) remains mostly recognizable. Therefore, 0.01 is often used as the noise standard. However, as the noise standard deviation increases up to 0.1, the geometric structure of the Bunny is severely degraded, going beyond the noise levels typically encountered in robotics and computer vision applications.

(a) Bunny dataset

(b) Bunny dataset

! = 0.01

(c) Bunny dataset

! = 0.1

(d) Bunny dataset

, 99% outliers

! = 0.1

A.6 Efficiency and accuracy.

In this section, we report the inference time, rotation error, and translation error by increasing the corresponding number N c from 250 to 5000 . The curves of FGR, RANSAC, MAC, and our method are flat and difficult to distinguish visually, demonstrating their efficiency. In addition, when there are fewer inputs, the influence of outliers is more obvious. The registration accuracy of FGR and RANSAC decreases significantly as the number of points decreases. The rotation and translation errors of our method are less affected by the number of correspondences. Compared with other methods, our method achieves the most accurate and fast registration for each number of input correspondences.

A.7 Effectiveness of the two-stage decoupling strategy.

We evaluate the effectiveness of our proposed two-stage decoupling strategy (TDS) by formulating the alignment error ℓ 0 -minimization problem directly on the Bunny data instead of local sets. The rotation and translation are estimated without hypotheses. We provide a comparison with other optimization-based methods [46, 12, 4, 39] as the outlier ratio increases from 0% to 90% . Our TDS consistently achieves the highest registration accuracy, demonstrating its inlier identification capability.

0

5

10

15

20

250

500

1000

2500

5000

FGR

RANSAC

TEASER++

MAC

Ours

0

1

2

3

250

500

1000

2500

5000

FGR

RANSAC

TEASER++

MAC

Ours

Number of correspondences

Number of correspondences

Rotation Error (degrees)

Translation Error (cm)

Time (s)

0.0001

1

10000

FGR

RANSAC

TEASER++

MAC

Ours

250

500

1000

2500

5000

Number of correspondences

1

10

100

1000

10

20

30

40

50

60

70

80

90

FGR

RANSAC

TEASER++

TDS

0

4

8

12

16

10

20

30

40

50

60

70

80

90

FGR

RANSAC

TEASER++

TDS

Translation Error (cm)

Rotation Error (degrees)

A.8 Additional comparisons.

We also provide a comparison with learning-based registration method EGST [43], we re-evaluate our method under the same dataset settings and metrics as EGST. The comparison results on KITTI and 3DMatch are shown in Table 4 below. The results of EGST reported in the table follow its published paper. Our method shows better performance in rotation error and comparable results in translation error.

A.9 Sensitivity to parameters.

Weconduct ablation studies on the KITTI dataset to evaluate the sensitivity of our algorithm to various parameters. Firstly, we ablate the number of local sets N 1 and correspondences N 2 in each local set. As shown in tables below, our method is insensitive to N 1 and N 2 , achieving high registration rates (RR) and low errors (RE and TE). Then, we evaluate the impact of rotation estimation threshold K R and translation estimation threshold K t . As shown in Fig. 10, the curves of registration metrics (RR, RE and TE) remain stable when K R and K t increase, indicating the insensitivity of our method to these parameters.

0.1

1

10

100

RR(%)

RE(°)

TE(cm)

𝐾

!

5

10

15

20

25

0.1

1

10

100

RR(%)

RE(°)

TE(cm)

𝐾

"

3

9

12

15

18

(a)

(b)

The results demonstrate that our method is parameter insensitive, making it reliable in practical implementations.

A.10 Limitations and broader impact.

We propose a ℓ 0 -norm based method to solve the point cloud registration problem. The method is robust to high outlier ratios and noise, and effective for different numbers of correspondences. It introduces a novel perspective to achieve accurate point cloud registration in practical applications. Our algorithm is most likely to be used in quality inspection and autonomous driving. It can provide fast and accurate alignment between workpieces and templates, as well as enhance localization and scene perception for autonomous vehicles. Furthermore, we wish to test the effectiveness of our method in other areas involving registration tasks, including multimodal medical image registration. One possible situation is a quality inspection scenario, where our algorithm may fail when dealing with large workpiece surfaces without obvious features. Future research will focus on enhancing the robustness of our algorithm to featureless data.

A.11 Scalability of our algorithm.

Exploring the scalability of our algorithm and its suitability for real-time applications is important for practical deployment. Existing algorithms struggle to achieve both fast speed and high accuracy. Our experiments demonstrate that our algorithm not only achieves high accuracy and robustness but also remains competitive in terms of efficiency, highlighting its potential for real-time applications. The

speed of our method can be further improved through techniques such as parallel computing and C++ implementation. Notably, the two-stage decoupling strategy (TDS) consumes most of the running time (95% of the total), and thus, it could particularly benefit from parallelization. In the first stage of TDS, we compute the relative positions for all point pairs. In the second stage, the computation of null-space matrices also requires substantial processing time. Therefore, these two components are the primary targets for acceleration. Regarding scalability, the proposed two-stage decoupling strategy is a crucial step for inlier identification and can be flexibly combined with other methods to improve accuracy.

A.12 Qualitative results.

We show qualitative results on 3DMatch [44] and 3DLoMatch [17] in Fig. 12. The yellow and blue point clouds represent the source and target point clouds, respectively. The first column represents the input point clouds and the second column represents the aligned point clouds transformed with the ground-truth transformations. Compared to other methods [6, 45], our approach achieves better alignment results. We also provide registration results on the KITTI [13] dataset in Fig. 13. The input source and target point clouds are in different poses, and the point clouds transformed using our estimated transformations are successfully registered.

The visualization of failure cases is provided in Fig. 11 . We observe that when there are repeated patterns (e.g., similar chairs appearing in different locations) or textureless structures (e.g., walls, floors), failure cases may occur due to the feature matching ambiguity. These remain challenging problems in point cloud registration and have not yet been effectively addressed. Potential solutions include improving feature extraction or applying point cloud completion based on the scene context.

(a) Input

(c) Ours

(a) Input

(c) Ours

A.13 Datasets.

Stanford Bunny The Bunny model from the Stanford 3D Scanning Repository [10] is scanned with a Cyberware 3030 MS scanner, with licensing restrictions against commercial use.Each scan takes the form of a range image, described in the local coordinate system of the scanner. These range images are merged using a modified ICP algorithm [30].

Odometry KITTI KITTI [15] is published under the NonCommercial-ShareAlike 3.0 License. It contains 11 sequences scanned by a Velodyne HDL-64 3D laser scanner in outdoor driving scenarios. Following [5, 6], we use sequences 8-10 for testing.

3DMatch and 3DLoMatch 3DMatch [44] comprises 62 scenes from SUN3D [36], 7-Scenes [28], RGB-D Scenes v.2 [21], Analysis-by-Synthesis [31], BundleFusion [11], and Halbel et al. [16] (Table. 7). These scenes are captured from diverse indoor environments using different sensors like the Microsoft Kinect and Intel Realsense, and are processed into point cloud fragments by fusing 50 consecutive depth frames using TSDF volumetric fusion [10]. The dataset contains 46 scenes for training, 8 scenes for validation and 8 scenes for testing. The original 3DMatch [44] only considers point cloud pairs with > 30% overlap. In addition to this benchmark (3DMatch),

(d) SC

2

-PCR

(e) Ours

(c) MAC

(b) Ground-truth

(a) Input

3DMatch

3DLoMatch

3DLoMatch

3DMatch

3DMatch

3DLoMatch

we follow [17] to include point cloud pairs with overlaps between 10% and 30% to form another benchmark (3DLoMatch).

Input

Ours

Input

Ours

(a) With the FCGF descriptor

(b) With the FPFH descriptor

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The contribution and scope of the paper are accurately stated in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper discusses the limitations of the work in Appendix A.10.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper provides theorems and formulas in Sec. 3 and theoretical results in Sec. 4.2. However, the paper does not include a full set of assumptions and proofs.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The main experimental results of the paper are reproducible, and we will release our code after the paper is accepted.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: We will make the complete code public following the acceptance of the paper.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The experimental setup and implement details are provided in Sec. 4.1, as well as in Appendix A.4.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We perform 50 independent trials for each experiment in Sec. 4.2 and report the average results. Our experiments are stable across multiple runs.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide information on the compute workers and model efficiency in Sec. 4.1 and Sec. 4.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: This article complies in all respects with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The potential positive and negative societal impacts of the work are discussed in Appendix A.10.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The paper uses publicly available datasets and code for training and comparative evaluation, adhering to all protocol restrictions attached to the publication. Detailed licenses for the datasets used are provided in the appendix A.13.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We will release the our code under the CC BY-NC-SA 4.0 license after the acceptance of the paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of point cloud registration, namely the challenge of outliers that can negatively affect registration quality.  The proposed algorithm first constructs a compatibility graph from which local sets of possible correspondences are extracted. The method further involves decoupling the alignment error into rotation and translation fitting errors and then using null-space matrices to decouple inlier identification from the rotation and translation estimation. The method demonstrates robustness under large outlier ratios and noise, achieving state-of-the-art performance on multiple datasets. \n\nOverall, the paper introduces an innovative approach to a longstanding problem in Computer Vision, and I am willing to raise my score if my comments and questions are addressed.\n\nSTRENGTHS:\n1. The proposed method seems novel and effective. While I am not very familiar with the literature on the topic, the l0 optimization scheme, rotation and translation decoupling, and solving the whole system in an analytical way contain sufficient novelty. The experiments show that the proposed method is more robust to outliers and noise in the point clouds than previously proposed methods.\n\n2. I appreciate the thorough empirical validation of the approach. By studying the method's robustness on simpler datasets and comparing real-world datasets like KITTI, the paper makes a compelling argument for its strength. \n\n3. Overall, the paper is well-motivated and written, with some small errors that can be corrected to increase its readability.\n\nWEAKNESSES:\n1. The proposed method involves complex mathematical formulations and a multi-stage process that seems computationally intensive, either in speed or memory. The paper would benefit from a more thorough discussion of the computational complexity and efficiency of the proposed algorithm, especially in terms of how time complexity affects the practicability of the proposed method in real-world scenarios.\n\n2. The evaluation of the real-world datasets (Tables 1, 2, 3) shows that all analyzed methods fall tightly together, suggesting that point cloud registration performance is saturating. There seem to be multiple methods that perform almost similarly. This would suggest that on real-world data, one does not need such high levels of robustness to noise or outliers as suggested in the paper's motivation. I suggest the paper restate additional advantages of the method (such as time complexity or practicality in real-world applications) in the experiment section by adding time complexity as a metric to these tables.  \n\n3. Figure 1 is quite small. I suggest increasing the size of these figures so that the reader can grasp the concept of the approach more easily.\n\n4. I would exchange Figure 5 for a more representative figure showing the proposed methods' qualitative results. It isn't easy to see the proposed method's advantage qualitatively. Maybe adding a case where other methods fail could highlight the proposed method's strengths. Additionally, it would be better to show the ground truth alignment in the same pose as the different baselines. \n\nMinor points:\n* I suggest mentioning the dataset used in the captions of Figures 2-4.\n* Line 22: overlapped -> overlapping\n* Lines 25-27: This is a hard-to-understand sentence; maybe rewrite it.\n\nQUESTIONS:\n* What is meant by Bayesian Theory? Do you mean Bayes Theorem? Properly introducing the concept to the reader would be beneficial. \n* Line 123: What is K2? This seems to be an undefined variable.\n\nLIMITATIONS:\nLimitations are discussed in Appendix A.8", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study presents a framework for robust inlier identification in multi-model fitting tasks, which is a significant problem in computer vision. The proposed method and its evaluation show promising results.\n\nSTRENGTHS:\nOriginality:\n-Novel Framework: The paper introduces a new framework for robust inlier identification in multi-model fitting tasks, which addresses a significant problem in computer vision. This two-stage decoupling strategy (TDS) is a unique approach that separates the initial inlier identification and fine-tuning phases, enhancing robustness and accuracy.\n-Innovative Approach: The combination of graph model construction and optimization algorithms represents a creative integration of existing techniques, providing a fresh perspective on solving the inlier identification problem. This method stands out by effectively reducing the impact of noise and outliers, which is a persistent challenge in this domain.\nQuality\n-Thorough Evaluation: The paper conducts comprehensive experiments on multiple public datasets, including KITTI, 3DMatch, and 3DLoMatch, ensuring the proposed method's performance is rigorously tested across diverse scenarios.\n-Multiple Metrics: The use of various evaluation metrics such as Rotation Error (RE), Translation Error (TE), and Registration Rate (RR) provides a well-rounded assessment of the method’s effectiveness and robustness.\nClarity\n-Structured Presentation: The paper is well-organized with clear sections delineating the problem, methodology, experimental setup, and results. Each section builds logically on the previous one, facilitating understanding.\n-Detailed Results: The presentation of experimental results is clear, with well-labeled tables and figures that effectively communicate the performance of the proposed method compared to baseline techniques.\nSignificance\n-Addressing a Key Challenge: The problem of robust inlier identification in the presence of high noise and outliers is crucial for various computer vision tasks. The proposed method’s ability to handle such challenging conditions has significant implications for improving the reliability of multi-model fitting applications.\n-Broader Applicability: The methodology's potential for application across different domains (e.g., 3D reconstruction, object recognition, and scene understanding) underscores its broad significance. By improving inlier identification, the paper contributes to advancing the state-of-the-art in these areas.\n\nWEAKNESSES:\n-The abstract should succinctly summarize the problem, methodology, and key results. Currently, it lacks clarity in explaining the significance of the proposed method and its comparative advantage over existing techniques.\n-While the paper outlines the methodology, some parts lack detailed explanations. Specifically, the algorithmic steps should be elaborated with clear justifications for each step. Including pseudo-code or flowcharts could improve understanding.\n-The paper should clearly state any assumptions made during the development of the methodology and discuss potential limitations. This transparency helps in assessing the validity and applicability of the proposed method.\n\nQUESTIONS:\nHow sensitive is your method to the choice of parameters in the initial inlier identification and the fine-tuning stages? It would be helpful to see an analysis or discussion on parameter sensitivity and its impact on performance.\n\nLIMITATIONS:\n-Discuss the scope within which the proposed method is expected to perform well. Are there specific types of data or noise levels where the method might not be as effective? Providing this context will help readers understand the boundaries of the method's applicability.\n-The current comparative analysis in the paper is somewhat limited and does not include enough recent and advanced baseline methods. For a more thorough evaluation, it is recommended to compare the proposed method with a wider array of baseline algorithms. Specifically, the inclusion of EGST (EGST: Enhanced geometric structure transformer for point cloud registration), EMFPCR (Evolutionary multiform optimization with two-stage bidirectional knowledge transfer strategy for point cloud registration) and EMTR-SSC (Evolutionary Multitasking with Solution Space Cutting for Point Cloud Registration) would provide a more comprehensive benchmark.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents an inlier identification algorithm for point cloud registration by reformulating the conventional registration problem as an alignment error ℓ0-minimization problem. This approach focuses on enhancing the accuracy of point cloud registration under conditions with high outlier ratios and noise. The proposed method involves a novel two-stage decoupling strategy that first separates the alignment error into rotation and translation fitting errors, and then employs null-space matrices to decouple inlier identification from the estimation of these errors. Bayesian theory is applied to solve the decoupled ℓ0-minimization problems, identifying correspondences with the smallest errors as inliers. Extensive experiments on the KITTI, 3DMatch, and 3DLoMatch datasets demonstrate that the proposed algorithm is effective in various scenarios, both indoor and outdoor, compared to traditional and learning-based methods.\n\nSTRENGTHS:\nS1. Registration as Alignment with Decoupling. The paper introduces a novel approach to point cloud registration by reformulating the problem as an alignment error ℓ0-minimization problem and applying a two-stage decoupling strategy, offering a fresh perspective on tackling the issue of outliers in point cloud registration. \n\nS2. Method Details and Experimentation. The proposed algorithm is meticulously described and tested across multiple datasets, including KITTI, 3DMatch, and 3DLoMatch. The comprehensive experimental results demonstrate the robustness and effectiveness of the algorithm under various challenging conditions, including high outlier ratios and noise.\n\nWEAKNESSES:\nW1. Efficiency Considerations Unclear. While the paper demonstrates the robustness and accuracy of the proposed method, it does not extensively address the computational complexity and efficiency of the algorithm quantitatively. E.g. in Fig.7 and Fig.4, it is hard to distinguish the flat curves, it would be better if the author can show the running time as numbers in the paper for better comparison or change the axis scaling.\n\nW2. Parameter Ablation Missing. The sensitivity of the algorithm to various parameters, such as the number of local sets and the thresholds used for inlier identification is not thoroughly investigated. Understanding the impact of these parameters on the performance could provide valuable insights for practical implementation. \n\nW3. Questionable Performance on Small Scale Data. This paper conducts experiments on large scale scene-level datasets, which contains somehow distinctive features, however, it might fail when handling some small-scale object-level scens which may contain less distinctive feature, or some ambiguous scene. Can it be anticipated how the model would work in these cases / is the experimental evidence for it?\n\nQUESTIONS:\nQ1. Runtime Performance. Can this method achieve real-time performance? What is the exact running time for this method?\n\nLIMITATIONS:\nL1. Failure Cases. It could be helpful to understand the method better if the author can provide some failure cases analysis.\n\nL2. Scalability Considerations. Exploring the scalability of the algorithm and its suitability for real-time applications would enhance its practical value. Providing insights into potential optimizations for faster execution could also be beneficial. Can the authors comment on limitations in this regard?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}