RAR: Region-Aware Point Cloud Registration
Authors: Yu Hao
Yi Fang
Venue: ICLR 2022 Submitted (2022)

Abstract: This paper concerns the research problem of point cloud registration to find the rigid transformation to optimally align the source point set with the target one. Learning robust point cloud registration models with deep neural networks has emerged as a powerful paradigm, offering promising performance in predicting the global geometric transformation for a pair of point sets. Existing methods firstly leverage an encoder to regress a latent shape embedding, which is then decoded into a shape-conditioned transformation via concatenation-based conditioning. However, different regions of a 3D shape vary in their geometric structures which makes it more sense that we have a region-conditioned transformation instead of the shape-conditioned one. With this observation, in this paper we present a \underline{R}egion-\underline{A}ware point cloud \underline{R}egistration, denoted as RAR, to predict transformation for pairwise point sets in the self-supervised learning fashion. More specifically, we develop a novel region-aware decoder (RAD) module that is formed with an implicit neural region representation parameterized by neural networks. The implicit neural region representation is learned with a self-supervised 3D shape reconstruction loss without the need for region labels. Consequently, the region-aware decoder (RAD) module guides the training of the region-aware transformation (RAT) module and region-aware weight (RAW) module, which predict the transforms and weights for different regions respectively. The global geometric transformation from source point set to target one is then formed by the weighted fusion of region-aware transforms. Compared to the state-of-the-art approaches, our experiments show that our RAR achieves superior registration performance over various benchmark datasets (e.g. ModelNet40).

Under review as a conference paper at ICLR 2022
RAR: REGION-AWARD POINT CLOUD REGISTRATION
Anonymous authors
Paper under double-blind review
ABSTRACT
This paper concerns the research problem of point cloud registration to find the rigid transformation to optimally align the source point set with the target one. Learning robust point cloud registration models with deep neural networks has emerged as a powerful paradigm, offering promising performance in predicting the global geometric transformation for pair of point sets. Existing methods firstly leverage an encoder to regress a latent shape embedding, which is then de- coded into shape-conditioned transformation via concatenation- based condition- ing. However, different regions of a 3D shape vary in their geometric structures which makes it more sense that we have region-conditioned transformation in- stead of the shape-conditioned one. In this paper we present a Region- Aware point cloud Registration, denoted as RAR, to predict transformation for pairwise point sets in the self-supervised learning fashion. More specifically, we develop a novel region-aware decoder (RAD) module that is formed with an implicit neural region representation parameterized by neural networks. The implicit neural region rep- resentation is learned with a self-supervised 3D shape reconstruction loss without the need for region labels. Consequently, the region-aware decoder (RAD) module guides the training of the region-awart transformation (RAT) module and region- aware weight (RAW) module, which predict the transforms and weights for differ- ent regions respectively. The global geometric transformation from source point set target one is then formed by the weighted fusion of region-aware transforms. Compared to the state-of-the approaches, our experiments show that our RAR achieves superior registration performance over various benchmark datasets (e.g. ModelNet40).
Point set registration is a challenging but meaningful task, which has wide application in many fields Bai et al.] (2007); Bai & Latecki] (2008); Myronenko & Song (2009); Ma et al. (2016): Wu et al. 22112; Klaus et al. (2006); Maintz& & Viergever (1 (1998); Besl & McKay (1992) Raguram al. (2008); Yuille & Grzywaez (1988); Sonka al. (2014). Most existing non-learning methods solve the registration problem through an iterative optimization process to search the optimal geo- metric transformation to minimize a pre-defined alignment loss between transformed source point set and target point set Myronnnno et al. (2007): Ma et al. (2013 2014); Ling & Jacobs (2005). The geometric transformation can be modeled by specific type of parametric transformation (c.g. rota- tion, translation, thin-plate spline, and so on) Besl & McKay (1992.. For example, one of the most commonly applied methods, iterative closest point (ICP) Best & McKKa (1992), estimates the rigid transformation based on set of corresponding points. The ICP model, however, strongly depends on the initialization and has limited performance in choosing corresponding points. Moreover, itera- tive methods usually treat registration as independent optimization process for each given pair of source and target point sets, which cannot transfer knowledge from registering one pair to another.
In recent years, deep-learning- based algorithms have been implemented in various industries and achieved great success, researchers are increasingly interested in bringing deep-learning-bast based so- lutions to the field of point set registration. Instead of directly optimizing the transformation matrix towards minimization of alignment loss in non-learning-ban based methods, learning- based methods usually leverage modern feature extraction technologies for feature learning and then regress the transformation matrix based the mutual information and correlation defined on the extracted features of source and target shapes. The most recent model, deep closest point (DCP) Wang & Solomon 2219), leverages DGCNN Wang et al. (2019) for feature learning and pointer network to perform soft matching. To refine the soft matching results to predict the final rigid transformation, the DCP model further proposes singular value decomposition layer for fine-tuning. However, it is

---

Under review as a conference paper at ICLR 2022
Previous Methods: Shape-conditioned Transformation
Transformation
Global
Network
Transformation
Our Method: Region-condittoneed Transformation
RATNN
RATN2
Global Transformation
RAIIN's S
INRR: icit Neural Region Representation
Weighted Fusion
RATN: Region-Aware Transformation Network
Figure I: Comparison between the shape-conditioned transformation and region-conditioned trans- formation. The shape-conditioned transformation predicts one global transformation for point sets alignment whereas the rggion-conditioned transformation predicts set of transformations (i.c. Tn) for different implicit regions, which are then weighted fused form a global transformation. Tr denotes the n-th region-specifie transformation.
still challenging to design an explicit module for learning both the features from unstructured point clouds and their "geometric relationship" Wang et al. (2018). Existing works developed various models to compute the spatial correlation feature. For example, FlowNet3D Liu et al. (019) tied to
concatenate two global descriptors of source and target point sets; Balakrishhn et (2018) used
U-Net-based structure to mix the source and target volumetric shapes; Rocco et al. (2017) proposed correlation tensor calculated from source and target feature map and on. The learning of robust point cloud registration models with deep neural networks has emerged as a powerful paradigm, offering promising performance predicting the global geometric transformation for a pair of point sets. Those methods share similar pipeline by firstly leveraging encoder to regress latent shape embedding, which is then decoded into a shape-conditioned transformation via concatenation-base conditioning.
In this paper, we observe that different regions a 3D shape vary in their geometric structures which makes it more sense that have region-conditioned (in contrast to shape-conditioned) transformation decoder via concatenation-bart based conditioning. As shown in Figure I the shape- conditioned transformation predicts global transformation for point sets alignment whereas the region-conditioned transformation predicts a set of transformations for different implicit regions, which then weighted fused to form global transformation. With this observation, illus- trated in Figure 2] we present a region-aware point cloud registration, denoted as RAR, to predict transformation pairwise point sets a self-supervised learning fashion. Our proposed RAR framework contains three main components. The first component is region-aware decoder (RAD) module that is formed with an implicit neural region representation parameterized by neural net- works conditioned on a shape embedding. The implicit neural region representation is learned with a self-supervised 3D shape reconstruction loss without the need for region labels. The second com- ponent is region-aware transformation (RAT) module which decodes shape embedding features to regress set of region-specific transformations. The third component is the region-aware weight (RAW) module which generates the weights for different regions of the 3D shape be aligned. The global geometric transformation from source point set to target one is then formed by weighted fusion of region-aware transforms. Our contribution is follows:
2

---

Under review as a conference paper at ICLR 2022
"recoptrucctonn
Forward
Backward Flow
Enceder
Module
Module
RAD
stared wndht
trooodding Mrx (M
Global Iransfo Transformaioo mation
Enceder
Module
Stape trocodding
RAW
1xk
RAT
MI
Modale
2M
Figure 2: Our pipeline. Our proposed RAR framework contains three main components. The first component is region-aware decoder (RAD) module that is formed with an implicit neural region representation parameterized by neural networks conditioned a shape embedding. The implicit neural region representation is learned with self-supervised 3D shape reconstruction loss without the need for region labels. The second component region-awe aaaree transformation (RAT) module which decodes shape embedding features to regress set of region-specific transformations, The third component is the region-awand aware weight (RAW) module which generates the weights for different regions of the 3D shape to be aligned. The global geometric transformation from source point set to target one then formed by weighted fusion region-aware transforms. Laligument denotes alignment loss. Creconstruction denotes the reconstruction loss which allows to reconstruct the input shape. denotes the region probability score and T denotes the region-specifer transformation.
We introduce a new concept of region-conditioned transformation that contributes to a novel region-aware point cloud registration (RAR) the learning approach for robust point set alignment. Our RAR models are realized with the development of three new mod- ules: region-aware decoder (RAD) module, region-aware transformation (RAT) module, and region-aware weight (RAW) module.
Our RAR novel unsupervised learning model for point cloud registration without the need of training on labeled datasets.
Experimental results demonstrate the effectiveness of the proposed method for point set registration, our RAR achieved superior performance compared to unsupervised and pervised state-of-the-arr approaches even without labeled data for training.
RELATED WORKS
1.1 ITERATIVE REGISTRATION METHOOSS
The development of optimization algorithms estimate rigid and non-rigid geometric transforma- tions in an iterative routine has attracted extensive research attention in past decades. Assuming that pair of point translation and rotation parameters in the iterative search routine, therein aiming to minimize a dis- tance metric between two sets of points. The iterative closest point (ICP) algorithm Besl & McKay (1992) is one successful solution for rigid registration. It initializes an estimation of rigid function and then iteratively chooses corresponding points refine the transformation. However, the ICP algorithm is reported to be vulnerable to the selection of corresponding points for initial transforma- tion estimation. Go-ICP Yang et al. (2015) was further proposed by Yang et al. to leverage the BnB scheme for searching the entire 3D motion space to solve the local initialization problem brought by partially proposed by Chui and Rangarajan Chui &
related by rigid transformation, the standard approach is estimate the best
ICP. Zhou et al. proposed fast global registration Zhou et al. (2016) for the registration
overlapping 3D surfaces. The TPS-RSM algorithm Rangarajam (2000) to estimate parameters of non-rigid transformations with a penalty on second- order derivatives. Existing classical algorithms have achieved great success the registration task.
3

---

Under review as a conference paper at ICLR 2022
Although the independent iterative optimization process limits the efficiency of registering large number of pairs, inspiring us to design a learning- based system for this task.
1.2 LEARNING-BASED REGISTRATION METHODS
In recent years, learning -ased methods have achieved great success many fields of computer vision Su et al. (2015); Sharma et al. ) Maturana & Scherer (2015); Bai et et. (2016); 0i et al.
(2017): Verma et al. (2018): Masci et al. (2015); Zeng et al. (2017). In particular, recent works have started trend of directly learning geometric features from cloud points (especially 3D points),
which motivates us to approach the point set registration problem using deep neural networks Rocco
et al. (2017) Balakrishnan al. (2018) Zeng ct al. (017); Qi al. (017);; Verma al. (2018);
Masci et al. (2015). PointNetL. Aoki et al. (2019) was proposed by Aoki et al. to leverage the
newly proposed PointNet algorithm for directly extracting features from the point cloud with the
classical Lucas & Kanade algorithm for the rigid registration of 3D point sets. Liu et al. proposed Dlo al. (2019) to treat 3D point cloud registration as motion process between points. Wang et al. proposed a deep closest point Wang & Solomon (2019) model, which first leverages the DGCNN structure to exact the features from point sets and then regress the desired transformation based on it. Balakrishnan et al. /Balakrishnan et al. (2018) proposed a voxeIMorph CNN architecture to learn the registration field to align two volumetric medical images. For the learning- regis- tration solutions listed above, the main challenge concerns how to effectively model the "geometric relationship" between source and target objects in a learning-basse approach. For example, Rocco et al. (2017) proposed a correlation tensor between the feature maps of source and target images. Balakrishnan et al. (2018) leveraged a L-Net-based structure to concatenate features of source and target voxels. Liu al. (2019); Aoki et al. (2019) used PointNet-based structure, and Waag & Solomon (019) used a DGCNN structure to learn the features from a point set for further registra- tion decoding. In contrast, we introduce region-aware point cloud registration, denoted RAR, to predict transformation for pairwise point sets in the self-supervised learning fashion.
2 METHODS
We introduce our approach in the following sections. The problem statement of our method is intro- duced in section 2.1 We explain the learning shape descriptor section 2.2 Section 2.3 illustrates the network structure of our region-aware decoder module. The region- region-aware aware weight module is de- fined in section 2.4 In section 2.5 we describe the region-aware transformation module. The loss
function is also discussed in section 2.6
2.1 PROBLEM STATEMENT
We define the optimization task of the deep learning-based methods which directly use unordered point clouds as input at first. Giving training dataset denotes the input source point clouds and G, denotes the input target point clouds. We aim to obtain a parametric function (S G3) using a neural network structure that can predict the rotation matrix R SO(3) and translation vector & E TR33 that can deform the source point cloud towards the target point cloud. A pre-defined alignment metric between the transformed source point cloud and the target point cloud can be defined the objective loss function to update the parameters For given dataset D, a stochastic gradient-descent based algorithm can usually be utilized to optimize the parameters 0 by minimizing the pre-defined loss function:
{S,, Gj)}. where St, G, R S,
0" = arg min mig
(Ds G. (o(; (i, Gt))
where L represents the pre-defined loss function.
2.2 LEARNING SHAPE EMBEDDING
For the input point clouds, the learning shape embedding is a non-linear multi-layer perceptron (MLP)-based function neural network that can extract shape features and capture the geometric information. Formally, let P. denotes the input point clouds and fz C RT denotes the feature of
x, Vr P., where m is the dimension of output layer. Our Learning Shape Descriptor includes
4

---

Under review as a conference paper at ICLR 2022
two components: encoding network and feature information. We define the encoding network 91 : R + RR which uses multi-layer perceptrons (MLP) with ReLu activation function for feature extraction:
f= = 91 (x)
(2)
The feature information is combined by extracted feature and point coordinates. Specifically, Var E Pt, we concatenate the learned feature fz with the coordinates æ as the combined feature æ| E (3m++)), thes,the shape descriptor input point cloud P. is: {lfa.
2.3 REGION-AWARF TRANSFORMATION
In this section, introduce the network structures for region-aware transformation module. In order to learn the relation information between the source point cloud S; and the target point cloud G; based on the learning shape descriptor, we define the transformation network 92 (2(2++))) - R; where с is the dimension of output layer. 92 is a set of non-linear MLP-based functions. We have the predicted rigid transformation matrix 0i,k as:
i (92 Jy: DE
where [.] denotes the operation of concatenation, denotes the kth transformation function in Region-Aware Transformation.
2.4 AEARON-AWARE DECODER
We aim to use the neural networks to form the implicit neural region representation the shape, For the learning shape embedding [fe, x| E R(m+3), we use the MLP-based function Ja ((m+3) > g to learn the implicit region field for each point. Specifically, probability score that indicates the likelihood that the given point is inside the particular region:
network outputs the region
L Sof (maaa(gs((aa.
After we obtained the region probability score of points, we use the MLP-based function 9a R + I' with max pool to groups the multiple regions output to form the final implicit field de which indicates the onside-outside status of the points and allow us to reconstruct the given shape with the reconstruction loss La will describe in [
4x Maxpool (lz)
(5)
2.5 REGION-AWARE WEIGHT
Based on the RAD output features [/z, (x] of each point, we use a MLP-based function 95 R(m+k+3) + T with average pool layer and a MI.P-based function R - I to learn
the region-aware weight c for the corresponding region transformation function in region-aware transformation module:
C₁ 96 (Is
After we get the region-aware weight, we use it as the weight to balance among the multiple ML.s obtained from the region-aware transformation module. We define the final transformation ma- trix Φ as:
K
We further compute the transformed source point set, defined as S,
Š, = To, (5:)
where To, denotes the desired geometric transformation with parameters Φ, Based on the trans- formed source point set S; and the target point set Gü, further introduce the loss function.
5

---

Under review as a conference paper at ICLR 2022
2.6 LOss FUNCTION
We define the loss function in this section. As for unsupervised learning-basee based method, do not have the transformation ground truth for supervision. Therefore, we adopt Chamfer Distance, a simple but effective distance metric defined tion loss function. Given the transformed source point cloud S, and the target point cloud G; The loss function is defined as:
non-corresponding point clouds, registra-
LcD(S.,G.) E rES,
min ||
vll +
| - 111l VEG. =ES;
Σ
min
DEG;
As for the region-aware weight module, we leverage a mean square loss to compute the inside- outside status of the sampled points. which allows us reconstruct the shape in the output layer. Following the sampling method in Chen & Zhang (2019), we sampled points in the 3D space sur-
rounding the input shape and the inside-outside status of the sampled points. The unsupervised reconstruction loss defined as:
Ln(S.) (f(p) -f
(10)
where (p) is the output value of the region-aware decoder module with maxpool operation for each point in the source point cloud, and f (p) is the ground truth inside-outside status for a point p.
We define the overall loss function as:
C = Ccp(Si, (LR(S))
(11)
where denotes the weight that balances between the registration loss and reconstruction loss.
EXPERIMENTS
In this section, we conduct experiments to demonstrate the effectiveness of our pro- posed method on the ModelNet40 bench- mark dataset. In section 3.I we describe the preparation of the dataset in our exper- iment. Section 3.2 shows our experimental settings. In section 3.3] we compare different settings and initialization of our model on the ModelNet40 dataset to demonstrate the su- periority of our proposed method. We per- form the study the resistance to Data In- completeness((..) noise, Point Drifts (P.D.) noise, and Data Outliers (D.O.) noise in sec- tion 3.4, Further comparison with other state- of-the-art methods is shown in section 3.5
Figure 3: Qualitative results. Random selected
qualitative results of region estimation.
3.1 DATASET PREPARATION
We test the performance of our method 3D point cloud registration on the ModelNet40 bench- mark dataset Wu et al. (215)). The ModelNet40 dataset is an open-source collection, which consists of 12311 preprocessed CAD models from 40 categories. To generate the data for training and eval- uation for a fair comparison, shape, we generate the target shapes by applying a random rigid transformation: the rotation matrix characterized by three rotation angles along the zyz axes, where each value is uniformly sampled from [0, 45] unit degree, and the translation is uniformly sampled from [0.5, 0.5]. To ensure a fair comparison, we follow the official split to preprocess dataset Wu et al. (0155).
randomly sample 1024 points each shape. As for each source

---

Under review as a conference paper at ICLR 2022
Table 1: Quantitative result. We conduct the ablation study on the ModelNet40 dataset.
Models
MSF(R)
RMSF(R)
MAF(R)
MSF()
RMSF(()
MAE(()
Baseline(one decoder)
1.6750888
1.289137
1.001619
0.000289
0.017017
0.012868
With RAT
1.442023
1.200014
0.961432
0.000202
0.014229
0.010829
With RAD, RAW and RAT
0.305423
0.551856
0.422876
0.000048
0.006938
0.005477
3.2 EXPERIMENTAL SETTINGS
In our proposed method, we use Adam optimizer as our optimizer. The learning shape descriptor includes 5 MLPs with dimensions of (64, 128, 128, 512, 1024). For the region-aware transformation module, we use 8 decoders for each region. Each decoder includes ML.Ps with dimensions of (256, 128, 64), one fully connected layers with dimensions of (3) for decoding the rotation matrix, and 3 MLPs with dimensions of (256, 128, 64), one fully connected layers with dimensions of (3) for decoding the transformation matrix. The region- aawaree decoder module includes fully connected layers with dimensions of (1024, 256, 8) to reconstruct the shape. The region- aggion-aware weight module includes 3 MLPs with dimensions of (256,256, 128) and a max pool layer. Following the recent paper BAE-NET Chen et al. (2019), we set the number of pre-defined region categories as 8 and utilize one fully connected layer with dimensions of (8) to generate the final region- region-aware aware weight for region-aware transformation module. We use the ReLU activation function and implement batch normalization for every MLP layer except the last output layer. We set the loss weight a as I and the learning rate 0.001.
For evaluation of point cloud registration performance, We use the mean squared error (MSE), root mean squared error (RMSE), and mean absolute error (MAE) to measure the performance of our model and all comparing methods. Lower values indicate better alignment performance. All angular measurements in our results in units of degrees. Note that the ground truth labels are only used for the performance evaluation process.
3.3 ABLATION STUDY
Experiment setting: For 12311 shapes from 40 categories in the ModelNet40 dataset, We sample 1024 points from each point cloud. To show the efficiency of our proposed method, we test three settings of our model for comparison. the first setting, without our regional-aware network, the baseline uses the classical shape-conditiones transformation network for point cloud registration. In the second setting, we leverage the RAT module which is able to predict the desired rotation and transformation matrix for multiple regions. In the third setting, use the RAD and RAW module to predict weights for different regions respectively which is further used to weighted fuse the multiple transformations of the RAT module get the final global transformation.
Results: We list the quantitative experimental results in Table [ In this table, we examine and evaluate the performances of different models based on the prediction errors of translation vectors and rotation angles. As the process of incremental improvements that data describes, the RAD module and RAW module play significant roles in model since they will guide the training of the RAT module. Its combination would generate outstanding performance beyond other settings, the MSE(R) values are only 0.30. Figure indicates that our method can divide the shape into different meaningful regions.
3.4 STUDIES ON RESISTANCE TO NOISE
Experiment setting: In this experiment, we conduct the experiments to verify our model's perfor- mance using Data Incompleteness (D.I.) Noise, Point Drifts (P.D.) Noise and Data Outliers (D.O.) Noise on 3D shapes. As for D.I. noise, we randomly remove a certain amount of points from the entire point set. As for P.D. noise, we randomly add Gaussian noise the entire shapes, which is randomly sampled from N (0, 0.01) and clipped to [0.05, 0.05]. As for D.O. noise, we first remove certain amount of points and randomly add the same amount of points generated by zero-mean Gaussian to the entire point clouds. Note that we add the noise on the target shape. We compare model with other representative registration methods: DCP and PR-NET. We follow exactly

---

Under review as a conference paper at ICLR 2022
Input Shape
Ours
Iuput Shape
Ours
Input Shape
Ours
D.I. Noise
Noise
D.O.
Qualitative results. Randomly selected qualitative results in presence of D.I., P.D., and D.O. noises on the ModelNet40 dataset.
Figure 4:
Table 2:
Quantitative result. Comparison using shapes with D.I. noise on the ModelNet40 dataset.
Noise
Models
MSE(R)
RMSE(R
MAE(R)
MSE()
RMSE())
MAE(1)
PR-NET
16.998832
3,930473
2,969286
0.000528
0.022995
0.017540
D.L. noise
DCP
35.178237
5,605362
4.414905
0,000961
0,031015
0,023442
Ours
5.272810
2.286349
1888420
0.000393
0.019824
0.014672
PR-NET
7.700405
2.743103
2.150551
0.000503
0.022428
0.017721
P.D. noise
DCP
17.120563
3,990559
3.010491
0.000484
0.022016
0.017079
Ours
1.734638
1.306637
1.013553
0.000242
0.015587
0.012142
PR-NET
24.027700
4.638125
3.559682
0.000612
0.024748
0.018945
D.O. noise
DCP
53.339375
7.103211
5.779260
0.001221
0.034955
0.027130
Ours
6.301775
2.493354
1.897488
0.000584
0.024166
0.017890
the source code provided by DCP, PR-NET, and their default settings for training and testing their model.
Results: We list the quantitative experimental results about comparison using shapes with several noises in Table The table presents that our method achieves remarkably better performance than PR-NET and DCP models regarding the translation prediction and rotation angle results for the data in the ModelNet40 dataset. As for D.I. noise, model consistently has greater accuracy far beyond using which error measurements. In comparison to DCP and PR-NET under P.D. noise, for all the shown metries of alignment performance our method achieves much better results than DCP and PR-NET. Especially for the rotation matrix estimation, our method achieves an MSE of 1.67 in comparison 15.04 achieved by DCP and 7.84 achieved by PR-NET. Note that our method trained in a purly unsupervised manner. For the dataset in presence of D.O. noise, the quantitative result achieved by our model is better than the results of DCP and PR-NET. In comparison to the method DCP, for the precision of rotation matrix estimation, our method achieves 7.54 MSE(R) in comparison 48.5 56 achieved by DCP and 23.74 74 achieved PR-NET. In addition, from the qualitative results shown in Figure 4 we notice that our model achieves better alignment result with D.I. noise, P.D. noise and D.O. noise for most cases.
others, regardless
3.5 COMPARISONS WITH STATE-OF-THEEART METHODS
Experiment setting: this experiment, we evaluate the overall registration performance of our proposed model on multiple shape categories and yield the performance compared to well applied and current state-of-the the-hhh-rt methods. For a fair comparison, for the 12,311 CAD models from the ModelNet40, following exactly DCP's setting, 2,468 models for testing. Note that our model is trained without using any ground-truth information and our model does not require the SVD-hased fine-tuning processes.
split the dataset into 9,843 models for training and
8

---

Under review as a conference paper at ICLR 2022
Table 3: Quantitative result. Comparison with SOTA using shapes the ModelNet40 dataset.
Models
MSF(R)
RMSF(R)
MAF(R)
MSF(()
RMSF(1)
MAF(1)
ICP
894.897339
29.914835
23.544817
0.084643
0.2909935
0.248755
GO-ICP
140.477325
11.852313
2.588463
0.000659
0.025665
0.007092
FGR
87.661491
9.362772
1.999290
0.000182
0.013939
0.002839
PointNetLK
227.870331
15.095374
4.225304
0.000487
0.022065
0,005404
DeepGMR
7.910688
2.712874
2.812594
0.000100
0,009588
0,010010
DCPvI+SVD
6.480572
2.545697
1.505548
0.000003
0.001763
0.001451
DCPv2+SVD
1.307329
1,143385
0.770573
0,000003
0.001786
0.001
Ours
0.371577
0.609398
0.475871
0.000070
0.008409
0,006345
Inputs:
Outputs:
Inputs:
Outputs:
Inputs:
Outputs:
Inputs:
Outputs:
Figure 5: Qualitative results. Randomly selected qualitative results of point clouds registration multiple categories of the ModelNet40 dataset.
We list the quantitative experimental results in Table The data demonstrate that model, as an unsupervised method, possesses excellent generalization ability on unseen data. Even though our approach does not require label information for training purposes and an additional SVD layer for fine-tuning, our model still has significantly better performances than DCPv2+SVD (super- vised) version. Also, our model is more robust to random point sampling of source and target shapes by adopting the Chamfer Distance loss, whereas the DCP would have severe degradation since by default assumes the same sampling of points. One may note that the translation vector prediction performance of our model is inferior to that of DCPv1+SVD, DCPv2+SVD, The reason for this gap is that DCPv2+SVD adopts an additional attention mechanism in its network for enhancement. 1/DCPV2+SD leverage SVD as an additional fine-tuning the process to refine their results. Comparing to other unsupervised algorithms, like ICP and FGR, the strength and accuracy of our model could be clearly observed since ICP has an MSE(R) of 894.89 and we only have 0.37. As shown in Figure 5 the registration results indicate our model achieves remarkable performance in aligning the source and target point sets.
Results:
CONCLUSION
In this paper, we present a region-aware point cloud registration, denoted as RAR, to predict trans- formation for pairwise point sets in the self-supervised learning fashion. Compared with previous shape-conditioned transformation methods, with the proposed region-aware transformation network, our model can learn the desired geometric transformations for multiple regions in particular shape which makes the model has great generalization ability and more robust to the noises. Our proposed method is trained in unsupervised manner without any ground-truth labels. Also, experi- mentally verified the effectiveness of our model and achieved superior registration results on the ModelNet40 3D point cloud registration dataset.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Yasuhiro Aoki, Hunter Goforth, Rangaprasad Arun Srivatsan, and Simon Lucey, Pointnetlk: Robust & efficient point cloud registration using pointnet. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7163-7172, 2019.
Song Bai, Xiang Bai, Zhichao Zhou, Zhaoxiang Zhang, and Longin Jan Latecki. Gift: A real-time and scalable 3d shape search engine. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2016.
Xiang Bai and Longin Jan Latecki. Path similarity skeleton graph matching. IEEE transactions pattern analysis and machine intelligence, 30(7): 1282-129, 2008
Xiang Bai, Longin Jan Latecki, and Wen-Yu Liu. Skeleton pruning by contour partitioning with discrete curve evolution. IEEE transactions on pattern analysis and machine intelligence. 29(3), 2007.
Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. An unsu- pervised learning model for deformable medical image registration. In Proceedings of the IEEE Conference Computer Vision and Pattem Recognition, pp. 9252-92600 2018.
Paul J Besl and Neil D McKay. Method for registration of 3-d shapes. In Sensor Fusion IV: Control Paradigms and Data Structures, volume 1611, 1, pp. 586-607. International Society for Optics and Photonics, 1992.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939-5948 2019.
Zhiqin Chen, Kangxue Yin, Matthew Fisher, Siddhartha Chaudhuri, and Hao Zhang. Bac-net: Branched autoencoder for shape co-segmentation. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), October 2019.
Haili Chui and Anand Rangarajan. A new algorithm for non-rigid point matching. In Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, volume 2, pp. 44-51. IEEE, 2000.
Andreas Klaus, Mario Sormann, and Konrad Karner. Segment- based stereo matching using belief propagation and a self-adapting dissimilarity measure. In Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, volume 3, pp. 15-18. IEFF, 2006.
Haibin Ling and David W Jacobs. Deformation invariant image matching. In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, volume 2, pp. 1466-1473. IEEE, 2005.
Xingyu Liu, Charles R Qi, and Leonidas J Guibas. Flownet3d: Learning scene flow in 3d point clouds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 529-537, 2019.
Jiayi Ma, Ji Zhao, Jinwen Tian, Zhuowen Tu, and Alan L Yuille. Robust estimation of nonrigid transformation for point set registration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2147-2154, 2013.
Jiyy Ma, Ji Zhao, Jinwen Tian, Alan L Yuille, and Zhuowen Tu. Robust point matching via vector field consensus. IEEE Trans. image processing, 1764)1170-1122,, 2014.
Jiayi Ma, Ji Zhao, and Alan L Yuille. Non-rigid point set registration by preserving global and local structures. IEEE Transactions on image Processing, 25(1):53-64 64, 2016.
JB Antoine Maintz and Max Viergever. survey of medical image registration. Medical image analysis, 2-1):1-36, 1998.
Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. Geodesic volutional neural networks on riemannian manifolds, In Proceedings of the IEEE international conference on computer vision workshops, pp. 37-45, 2015.
10

---

Under review as a conference paper at ICLR 2022
Daniel Maturana and Sebastian Scherer. Voxnet: A 3d convolutional neural network for real-time object recognition. In Intelligent Robots and Systems (IROS), 2015 JEEE/RSJ International Con- ference on, pp.
IEEE, 2015.
Andriy Myronenko and Xubo Song. Image registration by minimization of residual complexity. 2009.
Andriy Myronenko, Xubo Song, and Miguel A Carreira- Carreira-Perpinán. Non-rigid point set registration: Coherent point drift. In Advances in Neural Information Processing Systems, pp. 1009-1016, 2007.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 12):4, 2017.
Rahul Raguram, Jan-Michael Frahm, and Mare Pollefeys. A comparative analysis ransac tech- niques leading adaptive real-time random sample consensus. In European Conference on Com- puter Vision, pp. 500-513. Springer, 2008.
Ignacio Rocco, Relja Arandjelovic, and Josef Sivic. Convolutional neural network architecture for geometric matching. In Proc. CVPR, volume 2, 2017.
Abhishek Sharma, Oliver Grau, and Mario Fritz. Vconv-dae: Deep volumetric shape learning with- out object labels. In Computer Eisvon-ECCV 2016 Workshops, pp. 236-250. Springer, 2016.
Milan Sonka, Vaclav Hlavac, and Roger Boyle. Image processing, analysis, and machine vision. Cengage Learning, 2014.
Hang Su. Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Mill Multi-view convo- lutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision, pp. 945-955 953, 2015.
Nitika Verma, Edmond Boyer, and Jakob Verbeek. Feastnet: Feature-steered graph convolutions for 3d shape analysis. In CVPR 2018- 8-EEEE Conference on Computer Vision & Pattern Recognition, 2018.
Shenlong Wang, Simon Suo, Wei-Chiu Ma, Andrei Pokrovsky, and Raquel Urtasun. Deep paramet- ric continuous convolutional neural networks. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2018.
Yue Wang and Justin M Solomon. Deep closest point: Learning representations for point cloud registration. In Proceedings of the IEEE International Conference on Computer Vision, pp. 3523- 3532, 2019.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael Bronstein, and Justin M Solomon. Dynamic graph cnn for learning point clouds. ACM Transactions on Graphics (TOG), 38(5): 1-12, 2019.
Yi Wu, Bin Shen, and Haibin Ling, Online robust image alignment via iterative convex optimization. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1808-1814. IEEE, 2012.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912-1920, 2015.
Jiaolong Yang, Hongdong Li, Dylan Campbell, and Yunde Jia. Go-iep: A globally optimal solution to 3d icp peint-set registration. IEEE transactions on pattern analysis and machine intelligence, 38(1 38(11):2244 (2(11))224122254 2015.
Alan L Yuille and Norberto M Grzywacz. A computational theory for the perception of coherent visual motion. Nature, 333(6168)))) 1988.
11

---

Under review as a conference paper at ICLR 2022
Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In Com- puter Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pp. 199- 208. IEEE, 2017.
Qian- Yi Zhou, Jaesik Park, and Vladlen Koltun. Fast global registration. In European Conference on Computer Vision, pp. 766-782. Springer, 2016.
12


Reviews:
[{"review_id": "vY_A22sSm3Ba", "reviewer": "Reviewer_tsa1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: It seems they have got a big improvement over the DeepGMR and DCP method.\n\nWeakness: The writing is rather unclear and the paper is difficult to follow. First, how is the region or part information is predicted without any supervised signal. I don't think the network can learn this in a fully unsupervised way.  For example, how is it possible to get this segmented results in Figure 3. How could we distinguish the chair back/ chair legs, airplane body/wings without training an instance or semantic segmentation network.\n\nSecond, what is the purpose of defining the reconstruction loss and how will it help for the registration or region segmentation.\n\nThird, what is the region aware means? segmentation or something else? Since the region aware is the key point for the paper, detailed explanation should be added.  \n\nFurther, there are many more recent works on this topics and I belive it is necessary to compare with, For example,\n\"RPM-Net: Robust Point Matching using Learned Features\"\n\nThere are many problems regarding the technical details, for example what is the number of the region, will it get learned from the network? \n\nI think another problem is this proposed method cannot generalize to other scene-level datasets, like ICL-NUIM Dataset and the SUN3.\n\nFinally, clearly the author has missed the section number for Introduction which makes the format looks weird.", "summary_of_the_paper": "This paper addresses the problem of global rigid registration which is a fundemental problem in computer vision or graphics and they proposed a region-aware decoder and fuse the transformation predicted for all those regions. ", "summary_of_the_review": "The major problem is it is hard to understand the underlying principle of why this network can encode the region feature and how is the region feature gets learned without any supervision. Overall, I feel this paper is not ready for submission and is below the level of this conference. But I hope the authors could address my questions, I want to make sure this is no misunderstanding.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "rvdD4Ik79nk", "reviewer": "Reviewer_bd2E", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper presents an interesting idea, i.e., adopting a region-aware estimation for the registration task. However, the authors failed to mention why such a region-aware approach can facilitate the registration task, a task that estimates transformation performed on the whole shapes. The statement \"different regions of a 3D shape vary in their geometric structures which makes it more sense that we have a region-conditioned transformation instead of the shape-conditioned one\" is too vague to support this motivation. Besides, this paper suffers from the following issues:\n\nMethod：\n1. The proposed RAT is claimed to \"predict the transforms for different regions respectively.\" However, as the method described in Section 2.3, it just adopts k MLP functions on the global features to obtain k transformations. Those transformations are then combined into one transformation that is performed on the whole shape instead of different regions. Thus, these k transformations are totally irrelevant with the k regions estimated in Section 2.4 and the main statement of this paper cannot be proven.\n2. In Section 2.5, there are no constraints on the region-aware weights. Thus, they can be negative, which is meaningless.\n3. This paper fails to mention how to obtain the translation t. Strangely, the translation errors vary in the ablation study shown in Table 1.\n\n\nExperiments:\n1. This method should be compared with the SOTA method RPM-Net: Robust Point Matching using Learned Features (CVPR2020).\n2. The authors did not provide visual comparisons.\n3. It is better to provide an ablation study for the region number k.\n4. One possible advantage for this region-aware registration is to perform registration on partial shapes. However, this paper lacks such a validation.\n\nAnother major problem in this paper is the unreadable expression, including the notations, equations, descriptions, and flowchart visualization.\n\n1. In Section 2.1, the authors defined Si and Gi as the pair of input point clouds. However, in Section 2.2, the authors suddenly used Pi, which I think the authors may refer to Si or Gi.\n2. According to the flowchart in Figure 2, the RAD module is applied to Si, which failed to be implied in Section 2.4. Besides, if the region-awarer decoder is applied on Si, which is the input point cloud, then \"q_x which indicates the inside-outside status\" should be zero directly. If so, why do we need to calculate q_x? Moreover, the authors should state clearly the relationship between f(p) in Eqn. (10) and q_x in Eqn. (5).\n3. The flowchart in Figure 2 is too vague and omits some crucial information, such as the dimensions in the RAD module. Moreover, the first MLP in the RAW module seems to be incorrect. Based on the description in Section 2.5, it is a shared MLPs that are similar to MLPs in RAD and RAT modules.\n\nBesides, the description for this cited paper is wrong: “FlowNet3D Liu et al. (2019) tied to concatenate two global descriptors of source and target point sets.” Please check their network architecture in Figure 3 of FlowNet3D.\n", "summary_of_the_paper": "This paper aims to solve the rigid registration of 3D point clouds using a deep neural network. The key difference from previous methods is that this paper proposes a region-conditioned transformation. Specifically, this method first estimates k transformation matrices and then adopts a region segmentation module to divide the shape, which is further utilized to estimate the region-aware weights to combine the k transformations", "summary_of_the_review": "This paper presents a region-aware deep learning framework for the point cloud registration task. It can achieve good performance\nand is robust to noise. However, the authors failed to state the motivation of this approach clearly and the implementation details\ncannot fulfill the claimed idea. This paper also has many flaws, such as suspicious implementations, insufficient experiments and\nunclear descriptions. Therefore, I tend to reject this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ZbfA43fFPQ", "reviewer": "Reviewer_EHWV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is understandably written and the method is presented such that it can likely be reproduced based on the paper. The discussed related work is comprehensive, however, it is notable that no papers from 2020 or 2021 are cited (this includes relevant prior art that might outperform the presented method).\n\n\nClarity of method\n*****************\n\nIn Sec. 2.2, it is unclear to me if or how the proposed MLP uses the neighborhood information of points. Existing point-cloud specific networks, such as PointNet, PointNet++, DGCNN and others, that successfully tackle that problem, were not further evaluated. From the notation alone, it appears that each point is fed into the MLP on its own, without taking spacial neighborhood into account, something that would not be very effective.\n\nIn Sec. 2.3, what does the \"i\" index of \\phi refer to in Eq. (3)?\n\nIn Sec. 2.4, what is the Maxpool operation in (5) performed over? \n\nWhat is N, M in Fig. (2) in the Shape Embeddings; is M the same as m in Sec. (2.3)?\n\nOverall it remains unclear to me how the supervision signal of Eq.(10) is built, and what the \"regions\" are that the method is to be aware of. Does the method use some decomposition of the ModelNet40 models? How is the decomposition into regions for Fig. (3) performed? From how I understand Sec. (2.6), the model predicts the inside-outside status of its input points, which on itself makes no sense, since all input points are exactly on the shape of the object.\n\n\n\nExperiments\n***********\n\nThe evaluation is performed on a standard dataset with a mostly usual evaluation protocoll. However, there is a significant weakness, which is that the source and target point clouds are *identical* and only rotated and translated. Usually, one would sample two different sets of N points from the original shape, then transform one of those, to simulate the fact that two scans are typically independent samplings of a common shape. This significantly weakens the conclusions drawn from this setup.\n\nThe paper cites none of the papers from 2020 and 2021. Among those are methods that significantly outperform DCP, and which might thus outperform the presented work. Just two examples:\n\n    Choy, Christopher, Wei Dong, and Vladlen Koltun. \"Deep global registration.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.\n\n    Wu, Bingli, et al. \"Feature Interactive Representation for Point Cloud Registration.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.\n    \nWhich method is \"PR-NET\"? Iit is only mentioned in the experiments, and there is no citiation.\n\nPlease also compare PR-NET and DCP in Table 1 (on the noise-free point clouds).\n\nFor the \"D.I.\" noise, if points are randomly removed and yet 1024 points are sampled from the cloud, then this kind of noise would be expected to have very little to no effect. A better way to model DI noise would be to completely remove parts of the model in either source or destination (such as the backside when observed from a certain direction, thus emulating self-occlusion when scanning objects).\n\n\nMinor\n*****\n\nTypos etc. (no need to respond).\n\n- p.6, before Eq. (9), \"cloud G_iThe loss\"\n- p.6, \"translation is uniformly sampled from [0.5, 0.5]\" should probably be \"[-0.5, 0.5]\"\n- p.7: \"regional-aware\" -> \"region-aware\"\n", "summary_of_the_paper": "This paper proposes a method for the registration of 3D point clouds using a trainable cascade of MLPs. The pipeline first computes per-point features, then proposes a transformation based on those features. The transformations are weighted using a predicted weight and combined into a final global transformation. The method is evaluated on ShapeNet40, showing good results w.r.t. prior art. The impact of some building blocks is validated in ablation studies.", "summary_of_the_review": "While the method proposes an interesting idea, it is not yet ready for publication. First, the method description is unclear, especially in Sec. (2.2) and (2.6). Second, the experimental setup is not convincing, as identically sampled points are used for source and target shapes. Finally, several important related work from 2020 and 2021 is not discussed or compared against.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "PMkkbihhGLQ", "reviewer": "Reviewer_7prb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this paper is not in a ready shape and should be revised much before getting accepted. Many proposed modules are not technically sound and are not clear to follow. Below, I summarize the strengths and weaknesses. \n\nStrengths:\n\n1. The motivation to predict per-region transformations and to ensemble the transformations is interesting. Indeed, global registration is prone to local minima. Using an ensemble method might alleviate this issue and make the final prediction robust to poor feature matching. \n\nWeaknesses:\n\n1. The paper is not well organized and presented. A lot of technical details are hard to follow. \n\n2. How is the region prediction module trained without supervision? It seems this paper proposes a pure self-supervised region prediction module. How does this module fit into this end-to-end pipeline? Specifically, I am extremely confused about Eq (4) & Eq (5) & Eq (6).\n \n3. In Figure 3, the visualization of region segmentation seems very accurate. How is it possible to learn such a good region segmentation model without any supervision? Does this visualization use the ground-truth region labels? \n\n4. In Eq (3), what's the consideration to get a feature descriptor of all points from both shapes? It seems this PointNet (or MLP) does not consider local features at all. \n\n5. Why does the reconstruction help? For the reconstruction loss Eq (10), what does the \"inside outside status\" mean? \n\n6. This paper only conducts comparisons to the original DCP and PRNet, which were presented two years ago. There are many recent papers achieving much better performance such as [a], [b], and [c].\n\n[a] \"PointNetLK Revisited\"\n[b] \"DeepGMR: Learning latent Gaussian mixture models for registration\"\n[c] \"RPM-Net: Robust point matching using learned features\" ", "summary_of_the_paper": "This paper studies point cloud registration with deep neural networks. The key insight is learning a region-based transformation is more robust than learning a per-shape transformation even though shapes are rigid. The proposed method follows a DCP pipeline except that it performs point cloud reconstruction as well as self-supervised region segmentation. First, it uses a PointNet to lift points into a high-dimensional space and tag each point with a pseudo label. Then, points are clustered based on the region labels. Next, a per-region transformation is predicted based on the features of points in each region. Finally, the global transformation is an ensemble of region transformations. ", "summary_of_the_review": "This paper is not straight-forward to follow. Many technical details are either not clear or not sound. So it's hard to understand and justify the contributions and novelty introduced by this paper. In addition, another issue is the correctness of the region segmentation module. I don't think it can be trained without any supervision. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3TZCw6MinrW", "reviewer": "Reviewer_mwXA", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The submission is not well prepared. For example:\n(1) Missing title of section 1.\n(2) The module names in Fig.1 and Fig.2 do not correspond to each other;\n(3) The loss representations Fig.2 are not consistent with those in Sec. 2.6.\n(4) In sec2.1, S_i and G_i represent the source and target point cloud respectively, while in Sec 2.2, there is a new symbol P_i is introduced while without any illustration of its relationship to S_i and G_i. \n(5) Symble \"f\" in Eq.(2) represents features, while the symbol \"f\" in Eq.(10) represents a function. Also, the definition of f in Eq.(10) is not clear. Does it denote Eq.(4)~Eq. (5)?\n(6) There should be space in \"(256,256,128)\" in Sec. 3.2. \n\nConfusions regarding the method:\n(1) How many subregions are selected in each point cloud, is the number pre-defined and fixed for all point clouds, or determined dynamically? If dynamically, according to what? \nThough the visualizations in Fig.3 are quite beautiful, I am wondering how to determine the subregions? By the inside-outside status of the sampled points, indicated by Eq. (10)? What is the definition of \"inside-outside\" status, are they binary values, or continually formulated like signed distance function?\n(2) Which dimension does the softmax in Eq.(4) operate on, the number of points, or the feature dimension k? Similarly, does the Maxpoo l in Eq.(5) operate on the number of points, since the input to Maxpool has a feature dimension of 1? \n(3) The authors claim that their method is fully self-supervised, while Eq. (10) uses the ground truth inside-outside status. \n\nRegarding the experiments: \n(1) The content of Sec.3.2 is implementation details, while the title is \"experimental settings\". \n(2) There are separate settings in Sec. 3.3, 3.4, and 3.5. While in the first experiments section (Sec. 3.3), there is no illustration of the training and testing set. Are the training and testing sets in the three subsections not the same?\n(3) Too few comparison algorithms; missing citations of comparison algorithms; and the basic comparison algorithm DCP is out of data (published in 2019); \n(4) What's the difference between DCPv1, DCPv2 and DCP? Why the comparison algorithms in Tab.2 is less than that in Tab.3?\n(5) In Sec.3.5, the authors emphasize that they do not use SVD as a fine-tuning step and thus they achieve inferior results on translation estimation. What if the SVD is also applied to the proposed method?\n(6) Experiments are only conducted on one dataset. ", "summary_of_the_paper": "This paper addresses the problem of point cloud registration, namely, given two point clouds with the same rigid shape, they aim to determine the relative pose (R and T) between them. For this purpose, they propose to predict several regional poses (determined by some subregions of the point cloud), weighted, and fuse them together to form the global pose estimation. ", "summary_of_the_review": "The paper is not well-organized. There are many conflict symbols and illustrations which introduce confusion. \nThe overall idea is just-so-so and it has less impact on the community from my point of view. The experiments settings are not clear, comparison algorithms are too old, and there is only one evaluation dataset. All those degrade the quality of this paper and I am not convinced by the results. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a learning-based method for shape registration that conditions on regions of the shape rather than learning from the entire point cloud in one shot.  The reviewers point out several questions about the method, thanks to expository issues as well as missing comparisons/ablation studies.  As the authors have chosen not to submit a rebuttal, I will refer them to the original reviews for details here for additional points of improvement.
Decision: Reject