3D Focusing-and-Matching Network for Multi-Instance Point Cloud Registration
Authors: Liyuan Zhang
Le Hui
qi liu
Bo Li
Yuchao Dai
Venue: NeurIPS 2024 poster (2024)
Keywords: Point Cloud Registration
Focus and Matching
Primary area: machine_vision

Abstract: Multi-instance point cloud registration aims to estimate the pose of all instances of a model point cloud in the whole scene. Existing methods all adopt the strategy of first obtaining the global correspondence and then clustering to obtain the pose of each instance. However, due to the cluttered and occluded objects in the scene, it is difficult to obtain an accurate correspondence between the model point cloud and all instances in the scene. To this end, we propose a simple yet powerful 3D focusing-and-matching network for multi-instance point cloud registration by learning the multiple pair-wise point cloud registration. Specifically, we first present a 3D multi-object focusing module to locate the center of each object and generate object proposals. By using self-attention and cross-attention to associate the model point cloud with structurally similar objects, we can locate potential matching instances by regressing object centers. Then, we propose a 3D dual-masking instance matching module to estimate the pose between the model point cloud and each object proposal. It performs instance mask and overlap mask masks to accurately predict the pair-wise correspondence. Extensive experiments on two public benchmarks, Scan2CAD and ROBI, show that our method achieves a new state-of-the-art performance on the multi-instance point cloud registration task.

3D Focusing-and-Matchhng Network for Multi-Instance Point Cloud Registration
Liyuan Zhang, Le Hui", Liu, Bo Li, Yuchao Dai*
School of Electronics and Information, Northwestern Polytechnical University
Shaanxi Key Laboratory Information Acquisition and Processing zhangliyuannpudunn .ww.u...iii.u....mmmmm. .wpu. edu. en, fhuile, liugi, libo, deiyuchaol@hspuu.cc edu.
Abstract
Multi-instance point cloud registration aims to estimate the pose of all instances of model point cloud in the whole scene. Existing methods all adopt the strategy first obtaining the global correspondence and then clustering to obtain the pose of each instance. However, due to the cluttered and occluded objects in the scene, it is difficult to obtain an accurate correspondence between the model point cloud and instances the scene. To this end, we propose a simple yet powerful network for multi-instance point cloud registration
3D mocusing-and-matenn by learning the multiple pair-wise point cloud registration. Specifically, we first present 3D multi-object focusing module to locate the center of cach object and generate object proposals. By using self-attention and cross-attention to associate the model point cloud with structurally similar objects, we can locate potential matching instances by regressing object centers. Then, we propose a 3D dual- masking instance matching module to estimate the pose between the model point cloud and each object proposal. It performs instance mask and overlap mask masks accurately predict the pair-wist wise correspondence. Extensive experiments public benchmarks, Scan2CAD and ROBI, show that our method achieves a new state-of-the-ar performance the multi-instance point cloud registration task. The project page is at htthps:: chttps::/nppccae //ppuc/r.../..//// github. 1o/3DFMNet/
Introduction
Point cloud registration, fundamental process in computer vision, involves aligning two point clouds through estimating a rigid transformation. In practical applications like robotic bin picking, multi-instance registration emerges critical need, demanding the alignment of a model's point cloud with multiple instances within the scene. This task presents heightened complexity compared to single-point cloud registration, primarily due to challenges such as the uncertain number of instances and inter-instance occlusions. These complexities are particularly pronounced in cluttered environments, where precise alignment becomes pivotal for effective robotic operations. Therefore, how to improve the accuracy of multi-instance point cloud registration is still a challenging issue.
There are few efforts for tackling multi inlti-iinttacee point cloud registration. Existing pipelines can be roughly divided into two types: two-stage and one-stage. For the two-stage process, we first extract point correspondences between the model point cloud and scene point clouds, and then recover per-instance transformations through multi-model fitting 24 36 48 Although two-stage methods are simple and feasible, the success of these methods largely depends on the quality of the
*Corresponding authors.
Liyuan Zhang, Le Hui, Qi Liu, Bo Li, and Yuchao Dai are with the Key Lab of Shaanxi Key Laboratory of Information Acquisition and Processing, School of Electronics And Information, Northwestern Polytechnical University, China.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

object center
CAD Model
CAD Model
Pair-wise Registrtaion
Sceno Point Cloud
Scene Point Cloud
Object Proposals
Existing Methods
Methed
Figure 1: Comparison between our method and existing methods in multi-instance point cloud registration. Our method decomposes the multi-instance point cloud registration into multiple pair- wise point cloud registration.
correspondence. Furthermore, due to cluttered and occluded objects, it is still difficult to accurately cluster the correspondences into individual instances for subsequent pair-wise registration. For the one-stage process, it takes the model point cloud and scene point cloud as inputs, and directly outputs pose. As a representative one-stage work, Yu et al. [46] proposed a coarse-to-fine framework, which learns to extract instance-aware correspondencee for estimating transformations without multi-model fitting. Due to the consideration of instance-level information in correspondence, it can obtain fine- sine-grained features, thereby boosting the performance. However, for the scene with multiple objects. obtaining accurate instance-level correspondence is very difficult, especially for the cluttered and occluded objects. Therefore, alleviate the difficulty of learning correspondence between the model point cloud and multiple objects in the scene, as shown in Figure [ we consider first focusing on the object centers, and then learning the matching between the object proposal and the model point cloud.
In this paper, we propose simple yet powerful 3D point cloud registration. The core idea of our method is decompose the multi-instance point cloud registration into multiple pair-wise point cloud registrations. Specifically, we propose a 3D multi-object focusing module to localize the potential object centers and generate object proposals. To associate the object with the input CAD model, we use self-attention and cross-attention to learn the structurally similar features, thereby improving the accuracy of prediction for object centers. Based on the learned object center, we incorporate the radius of the CAD model to generate object proposals through ball query operation. After that, we propose 3D dual mual-masiing instance matching module to learn accurate pair-wise registration between the CAD model and object proposal. It adopts an instance mask filter the background points in the object proposal and uses an overlap mask to improve the pair-wise partial registration of incomplete objects.
network for multi-instance
In summary, contributions lie in three aspects:
1. Our primary contribution does not lie in the network architecture but rather in proposing a new pipeline to address the multi-instance point cloud registration problem. Existing methods (such PointCLM 48 and MIRETR (46)) mainly learn correspondence between the one CAD model and multiple objects (one-to-many paradigm), while our method decompose the one-to-many paradigm into multiple pair-wise point cloud registration (multiple one-to-one paradigm) by first detecting the object centers and then learning the matching between the CAD model and each object proposal.
2. Our new pipeline is simple yet powerful, achieving the new state-of-the-art both Scan2CAD (3) and ROBI [43] datasets. Especially on the challenging ROBI dataset, our method significantly outperforms the previous SOTA MIRETR by about 7% in terms of MR, MP, and MF.
3. The progressive decomposition approach of transforming multi-instanee point cloud regis- tration into multiple pair-wise registrations, as proposed our paper, also holds significant insights for other tasks, such as multi-targel tracking and map construction.

---

Related Work
Point Cloud Registration is a crucial task in fields such as robotics and autonomous driving, which usually involving three stages: point matching, outlier rejection, and pose estimation. Acquiring accurate point correspondences is critical for successful registration, making the first two stages particularly important. Accurate point matching deeply relies on features that are descriptive and rotation invariant. Many researchers have made efforts on this, including handcrafted descriptors 32 6 and learning- based descriptors
[9] 20]41]2 2). Some recent coarse-to coursctto-fing ro-fine frameworks [88 31 bypass keypoint detection and achieve accurate correspondences in large-scale scenes. To handle the problem of outliers, RANSACI [5] and its variants follow the sypothesis-and-verffee process to reject outliers. And some learning-bast methods 11 T6 (4] for eliminating outliers have also been proposed for robust pose estimation. On the other hand, several methods(39) 17 40
[1]]]9
44 38
45] directly estimate the transformation with neural network in an end-to-end manner. However, these point cloud registration methods almost focus the one-to-one problem which only need to solve the transformation between two point clouds. So they cannot directly work when faced with the challenge of large number of instances in multi-instance registration tasks and heavy intra-instance occlusion.
Multi-Instance Point Cloud Registration, which aligns source point cloud to its various instances within a target point cloud, has received relatively less attention. Unlike multi-way registration [35], which aims create globally consistent reconstruction from multiple fragments through pairwise registration 30J, multi-instance registration involves not only rejecting outliers from noisy corre- spondences but also identifying the inlier for each individual instance. This makes it even more challenging than the traditional registration problem. Early methods of multi-model fitting were used for this task. In the early stages of this task's ' development, various methods of multi-model followed a hypothesis verification
fitting were employed. RANSAC-based approaches 24 5 13 22
47], entailed sampling an extensive set of hypotheses and clustering the correspondences based on their residuals under these hypotheses. Recently, [46] proposed an instance-awe aware correspondence extrac- tion method for end-to-end multi-instance pose estimation. However, existing methods are often affected by outliers from other instances and highly rely on speccfs global feature extraction. In this approach, the multi-instance registration problem is addressed by detecting matching instance centers in the scene and subsequently splitting the instances, effectively transforming into pairwise point cloud registration problem.
approach to multiple models, while another method, based clustering 266 27 25 36
N-*3 Point Offict
WIX
Encodeer
N=C
cluster
Scene Point Cloud P
H
Self-Atication
Cross-Ateation
N₂11
Iestance Mask
times
Y
3D Multi-Object Focusing
CAD Model
matched Instances
T=3
pair-wise matching
Overłap Mask
Instance Mask
Tx3
Query
Object
Sp
Tx3
Registration
Object Proposals
3D Dual-Masking Instance Matching
The framework of our 3D focusing-ande -aatthhnnnm network for multi-instaned instance pint cloud registration. Given the scene point cloud and the CAD model, we first present the 3D multi-object focusing module to localize the centers the potential objects in the scene. Then, we design the 3D dual-masking instance matching module learn pair-wise point cloud registration from the localized object proposals.
Figure 2:
3

---

3 Method
The overall pipeline is illustrated in Figure 2 Our method is two-stage framework, which first localizes the center of each object and then performs pair-wise correspondence. For the first stage, we present 3D multi-object focusing module (Sec. 3.1) for detecting the potential instance centers by learning the correlation between the input model point cloud and the scene point cloud. For the second stage, we design a 3D dual masking instance matching module (Sec. for predicting pair-wise correspondence between the input model point cloud and the localized region of each object center. At last, we introduce the loss functions of our method (Sec. 3.3
3.1 3D Multi-Object Focusing
As the first stage of our method, the 3D multi-object focusing module aims to regress the center of the potential objects for generating high-quality proposals for pair-wise correspondence. Compared to predicting the bounding box or mask of an instance, directly regressing the center of the object is much easier, especially in cluttered and occluded scenes. In order to accurately detect the object center, we first learn the correlation between the model point cloud and the scene point cloud. Then, we predict the object center by learning the offset of each point. Finally, we introduce how to construct 3D object proposals for subsequent pair-wise point cloud registration.
Feature correlation learning. We design a simple yet efficient feature extraction structure to learn the correlation between the scene point cloud P E g2xx3 and the model point cloud Q E IR Mx3 where N and M are the numbers of their points, respectively. Note that for fair comparison, do not use RGB information of point cloud. Before learning correlation, we adopt the encoder of KPConv 37 to extract multi-scale point features of P and Q, respectively. The output feature maps are denoted by F2 E RN..CC XC and F, E M, where N₂ and M, are the numbers of points after using grid subsampling (37 After that, we simply use self-attention and cross-attention to build the correlation between the model point cloud and the scene point cloud, which is written as:
Fp SelfAttn Fp. Fp), = SelfAttn Fg. Fy. Fq),
(1)
Hp = CrossAttn Fp. Fg. Fgy)
where Hp RNV is the output feature map of the scene point cloud P that embeds the relationship of the model point cloud Q. Note that in the experiments, we stack three cross-attention layers, in which the output of the previous layer will be sent to the next layer as the input. For simplicity, we use the same symbol H, to represent the feature map of the final output. By learning feature correlations, it is desired that the potential instances will be enhanced in the background, making them easier to detect.
Object center prediction. After feature correlation learning, we regress the object center from the whole scene. Given the feature map H, E N.C, we directly predict the offset vector and the instance mask for each point. The point offset vector means the displacement to its instance center, which is given by:
V MLP(H,)
where V, E R.N.3 is the offset matrix for N₂ points in the scene. To identify whether a point belongs to an instance rather than the background, we predict the point mask, which is written as:
Y, = MTLP(Concat(FF Gp))
where Y, R.Nxx1 is the mask score. G, is the geodesic distance embedding from 46 If the mask score is larger than 0.5, this point is identified as belonging to the point on the object. To obtain an accurate object center, we first displace each point to its potential instance center by adding the original coordinates of each point to its learned point offset (P + Vp). Then, we use the learned mask Y, to filter out background points and leave points on the instance. Subsequently, we employ DBSCAN [1] to group the offset points into K clusters. Finally, by averaging the points in each cluster, we can obtain the center of each instance, which is formulated by:
Sp = Avg(Y, Vp))
where S, E R K is the center of K objects.
Object proposal generation. Based on the obtained object centers, we construct the object proposals through ball query operation. Based on the object center, we use radius to draw a three-dimensional

---

sphere and collect the points that fall within the sphere as the object proposal. Note that the radius parameter is equal to the radius of the model point cloud. is desired that the constructed spherical regions should include the entire object as much possible. Compared with directly learning multi-instance point cloud registration, we can learn pair-wise point cloud registration between each object proposal and the model point cloud, thereby reducing the difficulty of registration.
3.2 3D Dual-Masking Instance Matching
Once we obtain object proposals, we employ a 3D dual-masking instance matching module to learn pair-wise point cloud registration. Specifically, we first learn the instance mask to segment the instance from the object proposal. Then, we learn the overlap mask to segment the common area between the instance and the model point cloud. Finally, based on the instance mask and overlap mask, we learn the pair-wise instance matching.
Instance mask. Since we cannot obtain the ideal object proposal, we need to filter out the background points from the object proposal to obtain the mask of the instance. Given the point cloud of object proposal O E R²'x3 (T is the number of points in object proposal), we employ small encoder structure of KPConv [37] to extract the feature of object proposal. Therefore, we can obtain the
feature map E, E TCC where T3 is the number of points after grid subsampling. The instance mask Y, is formulated by:
Y, = MLP ALL((Concati (E, G.))
(5)
where Y. E R is the mask score and G. is the geodesic distance embedding from [46]. It worth noting that in the first stage, we learn point masks for all instances from the entire scene, making it difficult to obtain accurate point masks for each instance. Here, we learn point masks from the object proposal, so we can obtain more accurate instance masks.
Overlap mask. Generally, due to object occlusion, there are large number of incomplete objects. Therefore, we consider learning the overlap mask between the incomplete object and the complete model point cloud. We feed the model point cloud into the designed small KPConv encoder to obtain feature map E, E R²',xC, Similarly, we use atff-attenionn and cross- cooss-attention to learn the correlation between the object proposal and the model point cloud, which is formulated as:
E, SelfAttn E., E.), Eg = SelfAttn Eg Eg Eg).
(6)
Z. Attn EE,, Ey, Eg)
where Z, E RT.xC is the enhanced feature map of the object proposal. After that, we use MLP to predict the overlap mask, which given by:
Yo = MLP(Connct( Zo: Go))
where Yop RTxx1 is the obtained overlap mask. To upsample the overlap mask to the original resolution, we use small KPConv decoder generate feature map Yop Tx
For subsequent matching steps, we follow 31 to match the dense points within the local patches of two matched sampled points with an optimal transport (3). However, the local correspondences extracted in this manner often cluster closely, which results in unstable pose estimation, as noted in 20). Since the local area extracted from the focus network will contain some scene noise, and the intercepted instance point cloud will be incomplete, this problem will be more serious. To mitigate this problem, we suggest extracting a dense of point correspondences within the instance boundaries by utilizing instance masks and overlap masks. For each points correspondences Cj (p:, q,), we collect their neighboring points N," and NQ. The points out of instance is removed from N!" based on the instance mask. In order to solve the problem of incomplete point clouds, we further use overlap masks to eliminate non-overlapping parts within patch. Then the clear pair-wise correspondences extracted with optimal transport layer and mutual top-k selection, and using the local-to-global registration followed by 31
3.3 Loss Function
Loss focusing. For the 3D multi-object focusing module, we need to learn better shape features for localization. Therefore, we follow [3]] and use circle loss Leirete to learn fine interactive features,
5

---

as:
17 cirele
log[1+ +
Σ EfEE
Ap)
(8)
A
EAA
where and Q are source and target points-set and G is the anchor patches of each set. df = | h is the distance in feature space, X = (ot) and of is the overlap ratio between o and The gf. The weights 3. = 7(d) Ap) and 8 = 7A,, dt) are determined individually for each positive and negative example, using the margin circle loss on P is calculated in the same way.
Ap 0.1 and A 1.4. The For each sampled points in the scene, we constrain their learned offsets O fo1, ON Rx3 from their nearest target instance center using an LI regression loss as follows:
1 Lreg E.P. Elo, (a p.ll
where & is the centroid of the nearest instance that points i belongs to. Considering the varying object sizes across different categories, it is challenging for the network to accurately regress precise offsets, especially for boundary points of large objects, as these points are relatively far from the instance centroids. To tackle this problem, introduce direction loss constrain the direction of the predicted offset vectors. We define this loss, following the method i2[2]], as measure of the negative cosine similarities, i.e.,
E.Ps Σ Oj
è Pt
Ldir
(10)
10l112 1
Pill2
The overall focusing loss is computed as: focusing Leivete Lreg
Ldir-
Loss in matching. For coarse points feature learning, we employ the circle loss, as mentioned earlier in the Focus Network. As for point matching, we following [31] use a negative log-likeldhoo loss on the assignment matrix Z; of each ground-truth points correspondence E just as following:
Iull,i
Σ (r,))ec
log zf
Σ ZET,
log
zd +,m
Σ NET.
log fnt+ly
(11)
where I,andJ; are the unmatched points in the two matched patches. The final loss the average of the loss over all points matches: Ln = N, >> Lp.i-
Regarding the prediction of instance masks and overlap masks, we follow the methodology outlined in (28) The mask prediction loss composed binary cross-entropy (BCE) loss and dice loss with Laplace smoothing, defined as follows:
m. gf
+
L Lmask, BCE(m, mn") + -
(12)
|mg] + mm
0t
+
where m, and m prediction loss is the average loss over all. The total matching loss function Ematching is defined as:
l are the predicted and the ground-truth instance masks, respectively. The final mask
Imatching
Leiretee + Lull + Loverlapmaek + Liistancemmask
(13)
Experiments
4.1 Datasets and Evaluation Metrics
For multi-instance point cloud registration, We train and evaluate our method on two public bench-
marks: Scan2CAD 3 and ROBI 43
Scan2CAD. As pioneering dataset the realm of aligning scenes with CAD models, it leverages the resources of ScanNet [12 and ShapeNet [9] to form multi-instance registration dataset. With corpus comprising 1,506 scenes sourced from ScanNet, meticulously annotated with 14,225 CAD models from ShapeNet alongside their spatial orientations within the scenes, Scan2CAD sets new standard in scene-CAD alignment. For the total of 2,184 pairs of point clouds, it allocates 70% of pairs for training, 10% for validation, and reserves 20% for testing.

---

Table 1: Results comparison different methods on the test sets of both the Scan2CAD and ROBI datasets. The best results are highlighted in bold. Please note that "3DFMNet" indicates the upper bound of our method.
Scan2CAD
ROBI
Methods
MR (%)
MP (%)
MF (%)
MR (%)
MP (%)
MF (%)
T-Linkage
26
77.12
46.04
57.65
12.04
10.47
11.20
RansaCov
27
84.78
71.34
77.48
14.14
26.29
18.38
PointCLM
91.85
91.08
91.46
18.68
40.11
25.48
ECC 36
96.52
89.03
92.62
24.65
34.85
28.91
MIRETR 46
95.70
91.21
93.40
38.51
41.19
39.80
3DFMNet (ours)
95.4 95.44
94.15
94.79
46.81
50.61
48.63
3DFMNet" (ours)
97.68
94.63
96.14
52.59
63.13
57.38
ROBI. It is a dataset tailored specifically for industrial bin-picking applications. ROBI collects 7 reflective metallic industrial objects and 63 meticulously crafted bin-picking scenes. Each point cloud pair is meticulously crafted, with the scene point cloud generated through back projection from depth images, while the model point cloud is meticulously sampled from the CAD model corresponding to its industrial counterpart. There a total of 4,880 pairs of ROBI, divided into 70% for training, 10% for validation, and 20% for testing.
Metrics. We adopt three registration metrics to evaluate the methods, including Mean Recall (MR), Mean Precision (MP), and Mean Fl score (MF). We refer to the settings used in MIRETR and previous work to determine whether an instance is recognized correctly registered based on RTE and RRE 48 36). Specifically, we consider match successful when RTE S vorelsize and RRE < 15.. Following existing methods, such as MIRETR [6), the voxel sizes of Scan2CAD and ROBI dataset are set to 0.025m and 0.0015m, respectively. MR quantifies the proportion of registered instances relative to the total number of ground-truth instances, while MP measures the ratio of registered instances against the entirety of predicted instances. MF score is the harmonic mean of both MP and MR. Additionally, we present the pair-wise inlier ratio (PIR), elucidating the proportion inliers from a single instance amidst all extracted correspondences, considering one of the most important challenges of multi-instance registration is identifying the set of inliers for individual instances.
4.2 Implementation Details
Our method is trained on NVIDIA RTX 4090 GPUs and uses the Pytorch deep learning platform... We employ the Adam optimizer for 60 epochs. In initial learning rate and weight decay are set to 0.001 and 0.0001, respectively. We use a KPConv-FPN[37] backbone followed by (46) for feature extraction. We utilize voxel subsampling approach to reduce the resolution of the point clouds, resulting in the creation sampled points and dense points, which are then inputted into the network. The initial step involves downsampling the input point clouds using voxel-grid filter with a size of 2.5cm for Scan2CAD and 0.15cm for ROBI. Subsequently, we employ a 4-stage backbone architecture in both the multi-object focusing and sub-matching network. Following each stage, the voxel size is increased twofold to further reduce the resolution of the point clouds. The initial and final (coarsest) levels of downsampled points correspond to the dense points and sampled points, respectively, which are used for follow-up process. In the multi-object focusing network, we employ ball query approach akin the one detailed in [2), enabling us to retrieve the local neighborhood surrounding the regression center. The search radius is set to 1.2 times the size of the CAD model. Specifically, we randomly sample 4096 points from the dense points obtained through voxel subsampling in both Scan2CAD and ROBI datasets. During training, we use the ground truth center as supervision to train
3D multi-object focusing module and use the point cloud around the ground truth center as the training data to train the matching network. During testing, we use the center predicted by the 3D multi-object focusing module and its surrounding point cloud as the input to the 3D dual-maue hual-masking instance matching module to regress the final pose.
7

---

4.3 Results
Quantitative results. For a comprehensive comparison, we compare our 3DFMNet with five recent advanced multi-instancce point cloud registration approaches. including T-Linkage [26]. RansaConv [27], PointCLM [48). ECC [36], and MIRETR [46.. As shown in Table [ we re- port the results of different methods on Scan2CAD 3 and ROBI 43,, respectively. Note that except for the results of our method, the results of all other methods are taken from the official paper of MIRETR, as their results are obtained using the same backbone network. From the table, it can be observed that the performance of our method is superior to all other methods on the ROBI datasets. The density of objects in ROBI is higher than that in Scan2CAD, making ROBI more challenging. It can be observed that our method improves performance by about 7% in terms of MR. MP, and MF on the ROBI dataset. Since we use the focus-and-matching strategy to learn multiple pair-wise registrations, our method can learn more accurate matches from object proposals compared to the whole scene.
Since our method first localizes the object and then executes pair-wise correspondence, the precision of object localization has a significant impact on subsequent matching. In Table we provide the upper bound on the performance of our method, denoted by "3DFMNet"". Specifically, we utilize the ground truth object center to generate the object region, and use to evaluate the performance upper bound of our method. For the Scan2CAD dataset, the upper bounds of MR and MP are about 97% and 94%, respectively. However, due to the cluttered and incomplete objects in the ROBI dataset, the upper bounds for MR and MP are only about 52% and 63%. Nonetheless, the performance upper bound of our method is still higher than previous methods. In word, the theoretical upper bound indicates that the strategy of focusing first and then matching can ensure that our method can achieve excellent performance.
To evaluate the 3D multi-object focusing module, we computed the mean recall (MR), mean precision (MP), and root mean square error (RMSE) for detected object centers. We defined successful detection as cases where the predicted center lies within distance I 0.1 X l'nstaacee (where Iinstance is the radius of instance) of the ground truth center. As shown in Table 2 the module achieves strong results across MR, MP, and RMSE, confirming its effectiveness in identifying object centers in the first stage. the second stage, the 3D dual-masking instance matching module refines detections by applying instance and overlap mask learning to reduce false positives. Analyzing the Scan2CAD dataset, we found that 1.05% of objects were initially misdetected (25 instances), but 22 were successfully filtered out by the masking module. These misdctections have minimal impact, as few matching points are generated for falsely detected objects, preventing SVD from calculating a relative pose and limiting their influence final results.
For unseen scene experiments, we follow MIRETR by using the ShapeNet dataset 9 (55 categories in total) to evaluate generalization to novel categories. Specifically, we train on CAD models from the first 30 categories and test on the remaining 25. To address class imbalance, we sample up to 500 models per category, as in MIRETR. Each point cloud pair includes randomly selected CAD model and a scene model with 4-16 random poses applied, resulting in 8,634 pairs for training, 900 for validation, and 7,683 for testing. Table 3 shows that our method demonstrates strong generalizability on unseen scenes compared to MIRETR.
Table 2: Result related to the 3D multi-object focusing module.
Table 3: The generalizability of the 3D multi- object focusing module to unseen scenes.
Datasets
MR (%)
MP (%)
RMSE (m)
Methods
MR (%)
MP (%)
MF (%)
Scan2CAD
98.14 80.30
98.85
0.0814
MIRETR [46]
94.95
93,94
94.44
ROBI 43
99.99
0.0065
3DFMNet (ours)
95.12
94.23
94.67
Visualization. In Figure 3 we show the visualization of chairs of our method with the previous state-of state-of-the-art MIRETR [46] on the test set of the Scan2CAD dataset. From the left to right, the figure shows the results of ground truth, MIRETR, and 3DFMNet (ours), respectively. Note that we transform the chair's CAD model to the corresponding scenes. By comparing the ground truth with our method, it can be observed that our method can successfully match the multiple chairs in the cluttered scene. In addition, we also show the wiir correspondences on the test set of the Scan2CAD datasct in Figure 5 In Figure 4 we also show the visualization of the incomplete objects
8

---

on the test set of the ROBI dataset. It can be observed that our method can effectively obtain the correspondences between the challenging incomplete objects and the model point cloud.
Time costs. Here we report the inference time of different methods the ROBI dataset for a compre- hensive comparison. Table 4 shows model inference time for feature extraction and pose estimation time for transformation. "Total" represents the sum of "Model" time and "Pose" time. While our two-stage method has a slightly higher total time than the one- stage MIRETR [46], runs faster than the two-stage PointCLM [48]. Nonetheless, compared with Point- CLM and MIRETR, our 3DFMNet achieves higher performance.
Table 4: Per scene time the ROBI dataset.
Methods
Model
Pose
Total
T-Linkage 26
0.30
3.07
3.34
RansaCov
127
0.30
0.17
0.47
PoinCLLM 488
0.30
0.33
0.63
ECC [36
0.30
0.21
0.51
MIRETR [46
0.30
0.10
0.40
3DFMNet (ours)
0.26
0.28
0.54
#Inst: 20
#Inst:
#Inst: 20
# #Inst: 26
#Inst: 12 (b) MIRETR
#Inst: 25
(a) Ground Truth
(c) 3DFMNet (ours)
Figure 3: Registration results on the test of the Sacn2CAD dataset. We visualize the successfully registered instances of MIRETR [46] in (b) and ours in (c). "# Inst" means the number of registered instances. Note that for better view, we draw the green boxes for the ground truth and the red boxes for the predict correspondences.
4.4 Ablation Study
Impact of the number of sampled points. The grid sizes influence the number of sampled points.
We conduct experiments on the Scan2CAD dataset to verify the impact of different numbers of points. The results of MR and MP are 83.76% and 92.50% (1024 points), 95.44% and 94.15% (default 4096 points), and 95.20% and 93.75% (8192 points), respectively. points hinder information gathering and reduce accuracy. Conversely, too many sampling points do not improve accuracy and instead decrease it due to redundancy.
is evident that too few sampling
Effective of dual-masking, We evaluate the necessity of our dual-masking structure under the fair setting of the Scan2CAD dataset. When taking turns removing components, the results of MR and MP are 94.76% and 93.30% (only removing the overlap mask), 91.82% and 93.53% (only removing the instance mask), 90.01 % and 90.90% (removing both), and 95.44% and 94.15% (using both). The ablation study results can demonstrate the effectiveness of our dual- mual-masking structure.

---

#Inst: 14
#Inst:2
#Inst:6
# #Ins::26
#Inst:1 10 (c) MIRETR
#Inst:19
(a) Input
(b) Ground Truth
(d) 3DFMNet (ours)
Figure 4: Registration results on the test set of the ROBI dataset. We visualize the successfully registered instances of MIRETR (46) in (c) and ours in (d).
(a) Ground Truth
(b) Input Point Cloud
(c) Instance Level Matching
Figure 5: Visualization results of pair- wise correspondences the test set of Scan2CAD dataset.
4.5 Limitation
Our 3D goousing-and-matchine network is two-stage framework for the multi-instance point cloud registration task. The localization accuracy the first stage will affect the pair-wise correspondence in the second stage. We have analyzed the upper bound of our method in 4.3 In addition, due to the two-stage process, the inference time our method is slightly lower than the previous MIRETR [46).
In future work, we will strive to improve the performance and speed of our method.
Conclusion
In this paper, we proposed a 3D boousing-and-maceen network (3DFMNet) for multi-instance point cloud registration. Specifically, we first presented a 3D multi-object focusing module that learns to localize the center of the potential target the scene by considering the correlation between the model point cloud and the scene point cloud. Then, designed a 3D dual-masking instance matching module to learn the pair-wise correspondence between the model point cloud and the localized object. Extensive experiments two public benchmarks, Scan2CAD and ROBI, show that our method achieves a new state-of-the-art performance on the multi-instance point cloud registration task.
Acknowledgments
The authors would like thank reviewers for their detailed comments and instructive suggestions. This work was supported by the National Science Fund of China (Grant Nos. 62306238, 62271410, 62001394) and the Fundamental Research Funds for the Central Universities.
10

---

References
[1] Matthias Niessner Matthew Fisher Jianxiong Xiao Andy Zeng. Shuran Song and Thomas Funkhouser. 3DMatch: Learning local geometric descriptors from RGB-D reconstructions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[2] Sheng Ao. Qingyong Hu. Hanyun Wang. Kai Xu, and Yulan Guo. Buffer: Balancing accuracy, efficiency, and generalizability in point cloud registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[3] Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X Chang, and Matthias Nießner. Scan2CAD: Leamning CAD model alignment in RGB-D D scans. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2019.
[4] Daniel Barath and Jiri Matas. Graph-cut RANSAC. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[5] Daniel Barath and Jiri Matas. Progressive-X X: Efficient, anytime, multi-model fitting algorithm. In Proceedings of the IEEE International Conference Computer Vision (ICCV), 2019.
[6] Nassir Navab Bertram Drost, Markus Ulrich and Slobodan Ilic. Model gobally, match locally: Efficient and robust 3D object recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010.
[7] Eric Brachmann and Carsten Rother. Neural-guided RANSAC:
Learning where to sample model hypothe- ses. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
[8] Xinyue Cao, Xiyu Zhang, Yuxin Cheng, Zhaoshuai Qi, Yanning Zhang, and Jiaqi Yang. Instance by instance: An iterative framework for multi-instance 3D registration. arXiv preprint arXiv:2402.0419) 2024.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang. Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. ShapeNet: An information-rich 3D model repository. arXiv preprint 1rXiv:1512.0353 2015.
[10] Jaesik Park Christopher Choy and Vladlen Koltun. Fully convolutional gometric features. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
[11] Wei Dong Christopher Choy and Vladlen Koltun. Deep global registration. In Proceedings the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[12] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias NieBner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[13] Ivan Eichhardt Levente Hajder Daniel Barath, Denys Rozumny and Jiri Matas. Progressive X+: Clustering in the consensus space. In arXiv preprint arXiv:2103.1 10800....177...
2021. 2021.
[14] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density- based algorithm for discovering clusters in large spatial databases with noise. In kdd, 1996.
[15] Martin A Fischler and Robert C Bolles. Random sample consensus: paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 1981.
[16] Venu Madhav Govindu Jacinto C Nascimento Rama Chellappa G Dias Pais, Srikumar Ramalingam and Pedro Miraldo. 3DRegNet: A deep neural network for 3D point registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2020.
[17] Guangfu Wang Guanghui Liu Hao Xu, Shuaicheng Liu and Bing Zeng. Omnet: Learning overlapping mask for partial-to-pan -orrttaa point cloud registration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021.
[18] Mahdi Saleh Benjamin Busam Hao Yu, Fu Li and Slobodan Ilce. CofiNet: Reliable coarse-to- fine correspondences for robust point cloud registration. In Proceedings of the Advances in Neural Information Processing Systems (NeurlPS), 2021.
[19] Tolga Birdal Haowen Deng and Slobodan Ilie. PPFNet: Global context aware local features for robust 3D point matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
11

---

[20] Shengyu Huang. Gojeie Zan, Usvyatsov Mikhail, Wieser Andreas. and Schindler Konrad. Predator: Registration of 3D point clouds with low overlap. In Proceedings of the IECE Conference Computer Vision and Pattern Recognition (CVPR), 2021.
[21] Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi Wing Fu, and Jiaya Jia. Pointgroup: Dual-set point grouping for 3D instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[22] Yasushi Kanazawa and Hiroshi Kawakami. Detection of planar regions with uncalibrated stereo using distributions of feature points. In Proceedings of British Machine Vision Conference(BM 2004.
[23] Xiaoyuan Luo Kexue Fu, Shaolei Liu and Manning Wang. Robust point cloud registration framework based on deep graph matching. In Proceedings of the IEFF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[24] Florian Kluger, Eric Brachmann, Hanno Ackermann, Carsten Rother, Michael Ying Yang, and Bodo Rosenhahn. Consac: Robust multi-model fitting by conditional sample consensus. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition (CVPR), 2020.
[25] et al Luca Magri, Fusiello Andrea. Robust multiple model fitting with preference analysis and low-rank approximation. In Proceedings of British Machine Vision Conference BMVC)) 2015.
[26] Luca Magri and Andrea Fusiello. T-Linkage: A continuous relaxation of J-Linkage for multi-model fitting. In CVPR, 2014.
[27] Luca Magri and Andrea Fusiello. Multiple model fitting set coverage problem. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition (CVPR), 2016.
[28] Fausto Milletari, Nassir Navab, and Seyed-Ahman Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of 2016 fourth international conference on 3D vision (3DV), 2016.
[29] Charles Ruizhongtai Qi, Yi, Hao Su, and Leonidas Guibas. PointNet+++: Deep hierarchical feature learning on point sets in a metric space. 2017.
[30] Jaesik Park Qian Yi Zhou and Vladlen Koltun. Fast global registration. In Proceedings of the European Conference on Computer Vision (ECCV), 2016.
[31] Zheng Qin, Hao Yu, Changjian Wang. Yulan Guo, Yuxing Peng, and Kai Xu. Geometric Transformer for fast and robust point cloud registration. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2022.
[32] Nico Blodow Radu Bogdan Rusu and Michael Beetz. Fast point feature histograms for 3D registration. 2009.
[33] Paul-Ede Buul-Edouard Sarlin, Daniel DeTone, Tomasz Malisicwiez, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[34] Yaqi Shen, Le Hui, Haobo Jiang, Jin Xie, and Jian Yang. Reliable inlier evaluation for unsupervised point cloud registration. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2022.
[35] Qian Yi Zhou Sungjoon Choi and Vladien Koltun. Robust reconstruction of indoor scenes. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition (CVPR), 2015.
[36] Weixuan Tang and Danping Zou. Multi-instance point cloud registration by efficient correspondence clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2022.
[37] Hugues Thomas, Charles Qi, Beaannmma Deschaud, Beatriz Marcotegui. François Goulette, and Leonidas Guibas. KPConv: Flexible and deformable convolution for point clouds. In Proceedings of the IEFE International Conference on Computer Vision (ICCV), 2019.
[38] Yue Wang and Justin Solomon. PRNet: Self-supervised learning for partial-to-partill registration. In P Proceedings of the Advances in Neural Information Processing Systems (NeurIPS). 2019.
[39] Yue Wang and Justin M Solomon. Deep closest point: Learning representations for point cloud registration.
In Proceedings of the IEEE International Conference Computer Vision (ICCV) 2019.
12

---

[40] Guofeng Mei Xiaoshui Huang and Jian Zhang. ceature-metrit registration: A fast semi supervise approach for robust point cloud registration without correspondences. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2020.
[41] Lei Zhou Hongbo Fu Long Quan Xuyang Bai. Zixin Luo and Chiew-Lan Tai. D3Feat: Joint learning of dense detection and description 3D local features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[42] Lei Zhou Hongkai Chen Lei Zeyu Hu Hongbo Fu Xuyang Bai, Zixin Luo and Chiew Tai. Robust point cloud registration using deep spatial consistency. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[43] Jun Yang. Yizhou Gao, Dong Li, and Steven L Waslander. ROBI: A multi-view dataset for reflective objects in robotic bin-picking. In Proceedings of the International Conference on Intelligent Robots and Systems (IROS), 2021.
[44] Rangaprasad Arun Srivatsan Yasuhiro Aoki, Hunter Goforth and Simon Lucey. PointnetLK: Robust and efficient point cloud registration using pointnet. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition (CVPR), 2019.
[45] Zi Jian Yew and Gim Hee Lec. RPM-Net: Robust point matching using learned features. In Proceedings of IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2020.
[46] Zhiyuan Yu, Zheng Qin, Lintao Zheng, and Kai Xu. Learning instance-Aware correspondences for robust multi-Instance point cloud registration cluttered scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[47] Zhiyuan Qin Zheng, Chenyang Zhu, and Kai Efficient accurate Multi- InstaiInnttacee point cloud registration with iterative main cluster detection. 2024.
[48] Mingzhi Yuan, Zhihao Li, Qiuye Jin, Xinrong Chen, and Manning Wang. PointCLM: contrastive learning-based based framework for multi-instance point cloud registration. In Proceedings of the European Conference on Computer Vision (ECCV), 2022.
[49] Jan D. Wegner Zan Gojeic, Caifa Zhou and Andreas Wieser. The perfect match: 3D point cloud matching with smoothed densitics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
13

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Please refer to the abstract and introduction.
Guidelines:
The answer NA means that abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match thcorctical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Please refer to the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper limitations, but those are not discussed the paper.
The authors are encouraged create separate "Limitations" scction their paper.
The paper should point out any strong assumptions and how robust the results these assumptions (e.g.. independence assumptions, noiseless settings, model well well-specifiennt specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
violations of
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets with few runs. In general, empirical results often depend implicit assumptions, which should he articulated.
The authors should reflect the factors that inffuence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: Our Paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
14

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by
formal proofs provided in appendix or supplemental material.
Theorems and Lemmas the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please refer to the experimental part.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (c.g., in the case of a large language model), releasing of model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g.. with an open-source dataset or instructions for how to construct
dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome to describe the particular way they provide for reproducibility. In the of cosed-source models, it may be that access to the model is limited some way (e.g.. to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open faithfully reproduce the main experimental results, described supplemental material?
the data and code, with sufficient instructions
Answer: [Yes]
Justification:
We will release the codes.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc.sspps.ccppeeee cc/public/ quides/CodeSubbis sei.bboiiioi for more details.
While we encourage the release of code and data, we understand that this might not be possihle, so "Na" an acceptable answer. Papers cannot rejected simply for not including code, unless this is central to the contribution (e.g.. for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: : :/nniss//.///////////e:::eeee cc/public/ qu..des//ooeeSubbllsssooool..... for more details.
15

---

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) recommeneed, but including URLs to data and code is permitted.
6. Experimental Setting/Detan Details
Question: Does the paper specify all the training and test details (c.g,, data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer to the experimental part.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [No]
Justification: We didn't 't report еттor bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should given (e-g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error than state that they not verified.
CL, if the hypothesis of Normality of errors
asymmetric distributions, authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). crror bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, docs the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Please refer to the experimental part.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
16

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform. every respect, with NeurIPS Code Ethics
Answer:
[Yes]
Justification:
We have ensured anonymity.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e-g., if there is a special consideration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [No]
Justification: Our work is only for academic rescarch purpose.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations. (e.g.. deploy- of technologies that could make decisions that unfairly impact specific groups), privacy considcrations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated relcase of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible relcase of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: Our is only for academic research purpose.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
17

---

12. Licenses for existing assets
Question: Are creators or original owners assets (e.g., code, data, models), used the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected"'
Answer: [Yes]
Justification: Please refer to the reference part.
Guidelines:
The answer NA means that the paper does not use existing assets.
authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, ppaperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged reach out asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification:
We don't include any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training. license, limitations.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, applicable, as as details about compensation (if any)?
Answer: [NA]
Justification: We didn't use crowdsourcing or conduct research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: We didn't use crowdsourcing or conduct research with human subjects.
Guidelines:
18

---

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this the paper.
recognize that the procedures for this may vary significantly between institutions and Iocations, and we expect authors adhere to the NeurIPS Code of Ethics the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review,
19


Reviews:
[{"review_id": "k9UF2iGPP4", "reviewer": "Reviewer_8vwC", "summary": "For multi-instance point cloud registration, the authors proposed a 3D focusing-and-matching network by learning multiple pair-wise point cloud registration. Specifically, a 3D multi-object focusing module is proposed to locate the center of each object and generate object proposals. In addition, a 3D dual-masking instance matching is introduce to estimate the pose between the model point cloud and each object proposal. Extensive experiments on two popular datasets, Scan2CAD and ROBI, show that the proposed method achieves new state-of-the-art performance on the multi-instance point cloud registration task.", "questions": "Please refer to weaknesses.", "limitations": "The authors did not discuss the computational complexity of method", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- From the experimental results, it can be seen that decomposing multi-instance registration into multiple pair-wise registrations is very simple and effective.\n- The method achieves new state-of-the-art on Scan2CAD and ROBI. Especially in the challenging ROBI dataset, the proposed method is significantly better than the previous SOTA (+9%).\n- It is very convincing that the authors analyzed the upper bound of the method.", "weaknesses": "- Since the overall architecture is a two-stage structure for multi-instance point cloud registration, its inference time is lower than one-stage MIRETR (0.54s vs. 0.40s per scene in Table 2). Why the inference time is slightly lower than MIRETR? Considering the two-stage strategy, the author should conduct a detailed analysis.\n- The authors did not provide detailed training and testing strategies for its two-stage approach.\n- In Table 1, it can be observed that the proposed method did not achieve SOTA in terms of MR on the Scan2CAD dataset. What are the possible reasons behind this phenomenon?\n- In Figure 4, it can be observed that both MIRETR and the proposed method 3DFMNet cannot successfully match all parts on the ROBI dataset, especially on dense scenes with lots of parts.\n- As mentioned in the limitation, the proposed method is a two-stage method, and its inference time is slightly lower than MIRETR. It is better to discuss a potential plan to solve this issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "&nbsp;  \n\nWe thank the reviewer for the detailed comments to improve the paper.   \n&nbsp;  \n\n**Q1:** Detailed analysis of why the inference speed is slightly lower than MIRETR.  \n\n**A1:** In the main paper, we have measured the total inference time of our method (0.540s) and MIRETR (0.400s) per scene on the ROBI dataset. Since our method is a two-stage method, we supplement the detailed inference time of each stage in the following table:\n\n|Focusing Model Time (first stage) | Matching Model Time (second stage) |Total Time|\n|---|---|---|\n|0.145s|0.405s|0.540s|\n\nIt can be observed that the first stage (Focusing Model) tasks up a few time while the second stage (Matching Model) takes up a larger proportion of the time. Compared with the inference speed 0.400s of MIRETR, our inference speed 0.540s is slightly lower than MIRETR.  \n&nbsp;  \n\n**Q2:** Details of training and testing strategies.  \n\n**A2:** During training, we use the ground truth center as supervision to train the 3D multi-object focusing module and use the point cloud around the ground truth center as the training data to train the matching network. During testing, we use the center predicted by the 3D multi-object focusing module and its surrounding point cloud as the input to the 3D dual-masking instance matching module to regress the final pose.  \n&nbsp;  \n\n**Q3:** Possible reasons behind the phenomenon of MR not SOTA on the Scan2CAD.  \n\n**A3:** In Table 1 of the main paper, it can be seen that in the Scan2CAD dataset, our method improves the mean precision (MP) by about 3% while reducing the mean recall (MR) by about 1%. For Scan2CAD dataset, it may be due to some objects in the scene being too close to each other and relatively small in scale compared to the scene, which cause the multiple objects to be detected as a single object, leading to a decrease in MR. In order to better evaluate MP and MR, we also adopt the mean F1 score (MF), which is the harmonic mean of both MP and MR. In Table 1 of the main paper, it can be observed that our method improve MF by about 1.3%, which further demonstrate the effectiveness of our method.  \n&nbsp;  \n\n**Q4:** Analyze the reasons why current methods cannot achieve good results on the ROBI dataset.   \n\n**A4:** Due to the fact that the ROBI dataset is generated from monocular images, occlusions and other factors can cause the instance point clouds to be quite incomplete, making it more challenging. This is the main reason that the performance of all methods is not ideal. Nonetheless, our method significantly outperforms the previous SOTA MIRETR by about 7% in terms of MR, MP, and MF.  \n&nbsp;  \n\n**Q5:** Potential plan to solve the problem of inference time.  \n\n**A5:** In the future, we plan to use multiple GPUs for parallel computation to address this issue. We believe that the time taken to process a single matching proposal will be the upper limit of our optimization.  \n&nbsp;  \n\n**Q6:** Discuss the computational complexity.  \n\n**A6:** We provide the inference time and the number of parameters in the ROBI dataset in the following table:\n\n| Methods | Inference Time | Parameters (MB) | MR (%) | MP (%) | MF (%) |\n|---|---|---|---|---|---|\n| MIRETR | **0.400s** | **11.31M** | 38.51% | 41.19% | 39.80% | \n| 3DFMNet (ours) | 0.540s | 21.15M | **46.81% (+8.3%)**  | **50.61% (+9.42%)** | **48.63% (+8.83%)** |\n\nIt can be our running time is slightly higher than MIRETR. Although the number of parameters is about 2$\\times$ of MIRETR, the performance has been greatly improved by about 8% in terms of MR, MP, and MF on the ROBI dataset.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LxPa7xmx9K", "reviewer": "Reviewer_uLBB", "summary": "This paper introduces a novel focusing-and-matching technique for addressing the multi-instance point cloud registration challenge. Instead of fitting multiple models from a set of incorrect correspondences, this method initially detects potential instance regions and subsequently performs standard pairwise point cloud registration. The approach was tested on two benchmarks, where it outperformed existing state-of-the-art methods in terms of recall and precision metrics.", "questions": "I don't have any questions.", "limitations": "I can see the limitation is the performance of the 3D focusing module, espeecially its generalizability to unseen scene. Once it wrongly detects object proposals, the following pairwise registration module will fail to register the instance as well.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The primary contribution of this paper is the establishment of a new approach to solving the multi-instance registration task. Unlike previous methods that rely on multi-model fitting (using RANSAC-like methods) from a set of spurious correspondences, which are computationally expensive and unreliable due to the large matching space, this method uses learned scene priors to narrow down the matching space within individual regions of interest before performing standard pairwise point cloud registration.\n\n2. The writing is clear and easy to follow. In particular, the methodology section is well-structured. The concept of each variable is connected and explained fluently.", "weaknesses": "1.\tCertain details in Section 3.2 need clarification. Specifically, when the first module predicts K object centers but there are only (K-2) ground truth instances, how does the pair-wise registration model manage these two falsely detected objects? From my understanding, it appears that the model only predicts the transformation parameters without providing additional confidence scores.\n\n2.\tThe metrics require explanation or references. For MR, MP, and MF, how is a “registered instance” defined? What criteria are used to consider a predicted pose successful, such as RMSE, chamfer distance, relative translation error, or relative rotation error? Additionally, what are the thresholds?\n\n3.\tMore experimental results related to the 3D focusing module are needed, as this module determines the number of pairwise registrations performed, which sets the upper bound for the number of “successfully registered instances.” Results such as the number of detected objects, correctly detected objects, and wrongly detected objects are necessary, as these significantly impact the metrics, including MR, MP, and MF.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "&nbsp;  \n\nWe thank the reviewer for the diligent comments to improve the paper.   \n&nbsp;  \n\n**Q1:** How does the pair-wise registration model manage these falsely detected objects?  \n\n**A1:** Our method is a two-stage approach, so we analysis the falsely detected objects in both two stage.  \n(1) For the 3D multi-object focusing model (first stage), we compute the mean recall and mean precision of detect object centers. Specifically, we usex the distance between the predicted center and the ground truth center ($distance \\leq 0.1 * r_{instance}$, $r_{instance}$ is the radius of instance) as a metric to determine the success of the 3D multi-object focusing module. The results are as following table: \n\n| Datasets | Mean Recall (%) | Mean Precision (%) |\n|---|---|---|\n| Scan2CAD | 98.14% | 98.85% |\n| ROBI | 80.30% | 99.99% |\n\nOn Scan2CAD and ROBI, it can be observed that our method achieve very high precision (98.85% and 99.99%), which means that our method can successfully detect the object centers.  \n(2) For the 3D dual-masking instance matching module (second stage), the instance mask learning and overlap mask learning further filter the falsely detected objects. Specifically, we study the influence of the false detected objects on the Scan2CAD dataset. According to the above table, it can be observed that there are 1.05% objects are falsely detected, i.e., there are 25 cases of failed detected objects on the Scan2CAD dataset. After the instance mask learning and overlap mask learning, the 22 cases are filtered. Since instance mask learning and overlap mask learning cannot effectively achieve the matching points between the falsely detected objects and the CAD model, there are few matching points obtained for falsely detected objects, resulting in SVD being unable to solve for the relative pose. Therefore, falsely detected objects slightly affect our final results.  \n&nbsp;  \n\n**Q2:** Details of the metrics.  \n\n**A2:** We refer to the settings used in MIRETR and previous work to determine whether an instance is recognized as correctly registered based on RTE and RRE [1][2]. Specifically, we consider a match successful when $RTE \\leq voxelsize * 4$ and $RRE \\leq 15^\\circ$. Following existing methods, such as MIRETR, the voxel sizes of Scan2CAD and ROBI dataset are set to 0.025m and 0.0015m, respectively.\n\n[1] Yuan M, Li Z, Jin Q, et al. PointCLM: A Contrastive Learning-based Framework for Multi-instance Point Cloud Registration. In ECCV, 2022.  \n[2] Yu Z, Qin Z, Zheng L, et al. Learning Instance-Aware Correspondences for Robust Multi-Instance Point Cloud Registration in Cluttered Scenes. In CVPR, 2024.  \n&nbsp;  \n\n**Q3:** Experimental results related to the 3D multi-object focusing module are needed.  \n\n**A3:** To evaluate 3D multi-object focusing module, we compute the mean recall (MR) and mean precision (MR) and root mean square error (RMSE) of detect object centers. Specifically, we usex the distance between the predicted center and the ground truth center ($distance \\leq 0.1 * r_{instance}$, $r_{instance}$ is the radius of instance) as a metric to determine the success of the 3D multi-object focusing module. The results are as following table: \n\n| Datasets | MR (%) | MP (%) | RMSE (m) |\n|---|---|---|---|\n| Scan2CAD | 98.14% | 98.85% | 0.0814m |\n| ROBI | 80.30% | 99.99% | 0.0065m |\n\nIt can be observed that our 3D multi-object focusing module has good results in terms of MR, MP, and RMSE, which ensures that our method can successfully detect the object centers in the first stage.  \n&nbsp;  \n\n**Q4:** The generalizability of the 3D multi-object focusing module to the unseen scenes.  \n\n**A4:** For the unseen scenes, we follow MIRETR and use ShapeNet dataset [1] (total 55 categories) to conduct experiments on invisible semantics. Specifically, we use CAD models of ShapeNet dataset from the first 30 categories for training and the remaining 25 categories for testing to assess generalization to new categories. To prevent class imbalance, we follow MIRETR to randomly sample up to 500 models per category. Each point cloud pair consists of a randomly selected CAD model and a scene model created by applying 4–16 random poses. This results in 8,634 pairs for training, 900 for validation, and 7,683 for testing. The results of testing set are as following table: \n\n| Methods | MR (%) | MP (%) | MF (%) |\n|---|---|---|---|\n| MIRETR | 94.95% | 93.94% | 94.44% |\n| 3DFMNet (ours) | **95.12%** | **94.23%** | **94.67%** |\n\nIt can be observed that our method show good generalizability on unseen scenes compared to previous SOTA MIRETR. We will add the results in the revised manuscript.\n\n[1] Chang A X, Funkhouser T, Guibas L, et al. ShapeNet: An Information-Rich 3D Model Repository. arXiv preprint arXiv:1512.03012, 2015.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Glld59UiDA", "reviewer": "Reviewer_RCTj", "summary": "This paper proposes FMNet, an end-to-end deep learning approach for multi-instance point cloud registration. The key novelty is an attention-weighted feature matching module that can adaptively focus on reliable point correspondences during matching. Unlike traditional two-step methods, FMNet jointly learns feature extraction and matching within a unified network using an attention mechanism. The attention module computes matching probabilities between point features through scaled dot-product, allowing the network to dynamically highlight meaningful matches. Integrated with geometric and cycle consistency losses, FMNet achieves state-of-the-art registration performance on multiple datasets, demonstrating robust handling of noise, occlusions and point density variations. This work pioneers applying attention for solving the point cloud registration problem in a unified deep learning framework.", "questions": "What is the approximate percentage of time spent finding the center point and the registration, respectively?", "limitations": "Yes, the author fully addresses the limitations of his proposed methodology.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed an attention-weighted feature matching module that can adaptively focus on important point pairs, enhancing robust matching. This innovative design surpasses many previous hand-crafted or geometry-constrained feature matching strategies.\n2. The proposed FMNet end-to-end framework is well-designed, with clear mathematical explanations for the attention module. Extensive evaluations on public datasets and robustness analysis against noise/missing data provide strong empirical evidence.", "weaknesses": "1. The author's contribution is not summarized in points, which seems rather unclear.\n2.Even though your network is clearly expressed in the article, I think it is overly structured, which is why I find it less than innovative.\n3. In contrast to previous work PointCLM utilized density information for clustering relationships, it seems to me that this article does not effectively extract and utilize density information or other bases used for clustering. I think this information is more important for multi-instance alignment.\n4. Compared to previous work PointCLM and ECC, the article seems to be less theoretical, with a large number of formulas used to describe the structure of the network.\n5. I think this method of finding the center point first and then doing multi-instance point cloud registration is time consuming.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "&nbsp;  \n\nWe appreciate your diligent review and valuable feedback to help improve our paper.   \n&nbsp;  \n\n**Q1:** The contribution and innovation of the method.  \n\n**A1:** Our contributions lie in three aspects:  \n(1) Our primary contribution does not lie in the network architecture but rather in proposing a new pipeline to address the multi-instance point cloud registration problem. Existing methods (such as PointCLM and MIRETR) mainly learn correspondence between the one CAD model and multiple objects (**one-to-many paradigm**), while our method decompose the one-to-many paradigm into multiple pair-wise point cloud registration (**multiple one-to-one paradigm**) by first detecting the object centers and then learning the matching between the CAD model and each object proposal.  \n(2) Our new pipeline is simple yet powerful, achieving the new state-of-the-art on both Scan2CAD and ROBI datasets. Especially on the challenging ROBI dataset, our method significantly outperforms the previous SOTA MIRETR by about 7% in terms of MR, MP, and MF.   \n(3) The progressive decomposition approach of transforming multi-instance point cloud registration into multiple pair-wise registrations, as proposed in our paper, also holds significant insights for other tasks, such as multi-target tracking and map construction.  \n&nbsp;\n\n**Q2:** Reasons for not using density information (such as PointCLM) for clustering.  \n\n**A2:** PointCLM uses the feature density of contrastive learning to filter out matching pairs that do not belong to the instance before clustering. However, our method decomposes the one-to-many paradigm into multiple one-to-one paradigm by first detecting the object centers and then learning the matching between the CAD model and each object proposal. For the first detection stage, we can filter lots of background points by detecting the object centers and generating object proposals. For the second matching stage, the instance mask learning and overlap mask learning further filter the noisy points. To this end, we count the inlier ratio of points falling on objects and the total correspondence on the ROBI dataset. The following table show the results: \n\n| Methods | Inlier Ratio (%) |\n|---|---|\n| PointCLM | 36.27% |\n| MIRETR | 56.59% |\n| 3DFMNet (ours) | **59.84%** |\n\nIt can be observed that our method achieves the highest inlier ratio, which further demonstrate that our multiple one-to-one paradigm can effectively improve inlier ratio.  \n&nbsp;\n\n**Q3:** The article seems to be less theoretical, with a large number of formulas used to describe the structure of the network.  \n\n**A3:** As mentioned in A1, our contribution primarily lies in proposing a new paradigm for solving the multi-instance point cloud registration problem. Our method is simple yet powerful, achieving new state-of-the-art performance on both Scan2CAD and ROBI dataset.  \n&nbsp;\n\n**Q4:** I think this method of finding the center point first and then doing multi-instance point cloud registration is time consuming.  \n\n**A4:** Generally, two-stage methods are time-consuming than one-stage methods. In Table 2 of the main paper, we compare our method (3DFMNet) with existing methods (such as PointCLM, MIRETR):\n\n| Methods |  | Total Time (s) |\n|---|---|---|\n| PointCLM | Two-Stage |0.63s|\n| MIRETR | One-Stage | **0.40s** |\n| 3DFMNet (ours) | Two-Stage | 0.54s |\n\nOur inference speed of our two-stage 3DFMNet (0.54s) is lower than one-stage MIRETR (0.40s), while our inference speed is higher than two-stage PointCLM (0.63s). Although each method has different inference speeds, the inference speeds of these methods are all at the same level. In order to reduce the overall time consumption, one possible strategy for reducing the inference time of our two-stage method is to use parallel optimization to simultaneously match multiple pair-wise registrations. In the future, we will further consider reducing the inference time of our method.  \n&nbsp;\n\n**Q5:** The time spent of the center point detection and the registration.  \n\n**A5:** We supplement the detailed inference time of each stage in the following table:\n\n|Focusing Model Time (first stage) | Matching Model Time (second stage) |Total Time|\n|---|---|---|\n| 0.145s | 0.405s | 0.540s |\n\nIt can be observed that the first stage (Focusing Model) tasks up a few time while the second stage (Matching Model) takes up a larger proportion of the time. Compared with the inference speed 0.400s of MIRETR, our total inference speed 0.540s is slightly lower than MIRETR.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces an interesting approach for multiple pair-wise point cloud registration. The reviewers appreciated the proposed idea and its impressive registration accuracy. Minor concerns regarding computation overhead, missing details, and additional analysis were raised during the review phase. The authors provided solid feedback to relieve the concerns, and all reviewers appreciate the idea of predicting matching confidence scores or accuracy to handle wrongly detected objects, which considers time consumption. As a result, reviewer 8vwC gave 'weak acceptance', reviewer uLBB gave 'weak acceptance', and reviewer RCTj upgraded the score to 'borderline acceptance'. AC notes that all reviewers reached a clear consensus, and the idea and the proposed approach have several merits in the field.
Decision: Accept (poster)