RAR: Region-Aware Point Cloud Registration

Point set registration is a challenging but meaningful task , which has wide application in many fields Bai et al . ( 2007 ) ; Bai & Latecki ( 2008 ) ; Myronenko & Song ( 2009 ) ; Ma et al . ( 2016 ) ; Wu et al . ( 2012 ) ; Klaus et al . ( 2006 ) ; Maintz & Viergever ( 1998 ) ; Besl & McKay ( 1992 ) ; Raguram et al . ( 2008 ) ; Yuille & Grzywacz ( 1988 ) ; Sonka et al . ( 2014 ) . Most existing non-learning methods solve the registration problem through an iterative optimization process to search the optimal geometric transformation to minimize a pre-defined alignment loss between transformed source point set and target point set Myronenko et al . ( 2007 ) ; Ma et al . ( 2013 ; 2014 ) ; Ling & Jacobs ( 2005 ) . The geometric transformation can be modeled by a specific type of parametric transformation ( e.g . rotation , translation , thin-plate spline , and so on ) Besl & McKay ( 1992 ) . For example , one of the most commonly applied methods , iterative closest point ( ICP ) Besl & McKay ( 1992 ) , estimates the rigid transformation based on a set of corresponding points . The ICP model , however , strongly depends on the initialization and has limited performance in choosing corresponding points . Moreover , iterative methods usually treat registration as an independent optimization process for each given pair of source and target point sets , which can not transfer knowledge from registering one pair to another . In recent years , deep-learning-based algorithms have been implemented in various industries and achieved great success , researchers are increasingly interested in bringing deep-learning-based solutions to the field of point set registration . Instead of directly optimizing the transformation matrix towards minimization of alignment loss in non-learning-based methods , learning-based methods usually leverage modern feature extraction technologies for feature learning and then regress the transformation matrix based on the mutual information and correlation defined on the extracted features of source and target shapes . The most recent model , deep closest point ( DCP ) Wang & Solomon ( 2019 ) , leverages DGCNN Wang et al . ( 2019 ) for feature learning and a pointer network to perform soft matching . To refine the soft matching results to predict the final rigid transformation , the DCP model further proposes a singular value decomposition layer for fine-tuning . However , it is still challenging to design an explicit module for learning both the features from unstructured point clouds and their “ geometric relationship ” Wang et al . ( 2018 ) . Existing works developed various models to compute the spatial correlation feature . For example , FlowNet3D Liu et al . ( 2019 ) tied to concatenate two global descriptors of source and target point sets ; Balakrishnan et al . ( 2018 ) used a U-Net-based structure to mix the source and target volumetric shapes ; Rocco et al . ( 2017 ) proposed a correlation tensor calculated from source and target feature map and so on . The learning of robust point cloud registration models with deep neural networks has emerged as a powerful paradigm , offering promising performance in predicting the global geometric transformation for a pair of point sets . Those methods share a similar pipeline by firstly leveraging an encoder to regress a latent shape embedding , which is then decoded into a shape-conditioned transformation via concatenation-based conditioning . In this paper , we observe that different regions of a 3D shape vary in their geometric structures which makes it more sense that we have a region-conditioned ( in contrast to shape-conditioned ) transformation decoder via concatenation-based conditioning . As shown in Figure 1 , the shapeconditioned transformation predicts one global transformation for point sets alignment whereas the region-conditioned transformation predicts a set of transformations for different implicit regions , which are then weighted fused to form a global transformation . With this observation , as illustrated in Figure 2 , we present a region-aware point cloud registration , denoted as RAR , to predict transformation for pairwise point sets in a self-supervised learning fashion . Our proposed RAR framework contains three main components . The first component is a region-aware decoder ( RAD ) module that is formed with an implicit neural region representation parameterized by neural networks conditioned on a shape embedding . The implicit neural region representation is learned with a self-supervised 3D shape reconstruction loss without the need for region labels . The second component is a region-aware transformation ( RAT ) module which decodes shape embedding features to regress a set of region-specific transformations . The third component is the region-aware weight ( RAW ) module which generates the weights for different regions of the 3D shape to be aligned . The global geometric transformation from source point set to target one is then formed by weighted fusion of region-aware transforms . Our contribution is as follows : • We introduce a new concept of region-conditioned transformation that contributes to a novel region-aware point cloud registration ( RAR ) as the learning approach for robust point set alignment . Our RAR models are realized with the development of three new modules : region-aware decoder ( RAD ) module , region-aware transformation ( RAT ) module , and region-aware weight ( RAW ) module . • Our RAR is a novel unsupervised learning model for point cloud registration without the need of training on labeled datasets . • Experimental results demonstrate the effectiveness of the proposed method for point set registration , our RAR achieved superior performance compared to unsupervised and supervised state-of-the-art approaches even without labeled data for training . 1 RELATED WORKS . 1.1 ITERATIVE REGISTRATION METHODS . The development of optimization algorithms to estimate rigid and non-rigid geometric transformations in an iterative routine has attracted extensive research attention in past decades . Assuming that a pair of point sets are related by a rigid transformation , the standard approach is to estimate the best translation and rotation parameters in the iterative search routine , therein aiming to minimize a distance metric between two sets of points . The iterative closest point ( ICP ) algorithm Besl & McKay ( 1992 ) is one successful solution for rigid registration . It initializes an estimation of a rigid function and then iteratively chooses corresponding points to refine the transformation . However , the ICP algorithm is reported to be vulnerable to the selection of corresponding points for initial transformation estimation . Go-ICP Yang et al . ( 2015 ) was further proposed by Yang et al . to leverage the BnB scheme for searching the entire 3D motion space to solve the local initialization problem brought by ICP . Zhou et al . proposed fast global registration Zhou et al . ( 2016 ) for the registration of partially overlapping 3D surfaces . The TPS-RSM algorithm was proposed by Chui and Rangarajan Chui & Rangarajan ( 2000 ) to estimate parameters of non-rigid transformations with a penalty on secondorder derivatives . Existing classical algorithms have achieved great success on the registration task . Although the independent iterative optimization process limits the efficiency of registering a large number of pairs , inspiring us to design a learning-based system for this task . 1.2 LEARNING-BASED REGISTRATION METHODS . In recent years , learning-based methods have achieved great success in many fields of computer vision Su et al . ( 2015 ) ; Sharma et al . ( 2016 ) ; Maturana & Scherer ( 2015 ) ; Bai et al . ( 2016 ) ; Qi et al . ( 2017 ) ; Verma et al . ( 2018 ) ; Masci et al . ( 2015 ) ; Zeng et al . ( 2017 ) . In particular , recent works have started a trend of directly learning geometric features from cloud points ( especially 3D points ) , which motivates us to approach the point set registration problem using deep neural networks Rocco et al . ( 2017 ) ; Balakrishnan et al . ( 2018 ) ; Zeng et al . ( 2017 ) ; Qi et al . ( 2017 ) ; Verma et al . ( 2018 ) ; Masci et al . ( 2015 ) . PointNetLK Aoki et al . ( 2019 ) was proposed by Aoki et al . to leverage the newly proposed PointNet algorithm for directly extracting features from the point cloud with the classical Lucas & Kanade algorithm for the rigid registration of 3D point sets . Liu et al . proposed FlowNet3D Liu et al . ( 2019 ) to treat 3D point cloud registration as a motion process between points . Wang et al . proposed a deep closest point Wang & Solomon ( 2019 ) model , which first leverages the DGCNN structure to exact the features from point sets and then regress the desired transformation based on it . Balakrishnan et al . Balakrishnan et al . ( 2018 ) proposed a voxelMorph CNN architecture to learn the registration field to align two volumetric medical images . For the learning-based registration solutions listed above , the main challenge concerns how to effectively model the “ geometric relationship ” between source and target objects in a learning-based approach . For example , Rocco et al . ( 2017 ) proposed a correlation tensor between the feature maps of source and target images . Balakrishnan et al . ( 2018 ) leveraged a U-Net-based structure to concatenate features of source and target voxels . Liu et al . ( 2019 ) ; Aoki et al . ( 2019 ) used a PointNet-based structure , and Wang & Solomon ( 2019 ) used a DGCNN structure to learn the features from a point set for further registration decoding . In contrast , we introduce a region-aware point cloud registration , denoted as RAR , to predict transformation for pairwise point sets in the self-supervised learning fashion . 2 METHODS . We introduce our approach in the following sections . The problem statement of our method is introduced in section 2.1 . We explain the learning shape descriptor in section 2.2 . Section 2.3 illustrates the network structure of our region-aware decoder module . The region-aware weight module is defined in section 2.4 . In section 2.5 , we describe the region-aware transformation module . The loss function is also discussed in section 2.6 . 2.1 PROBLEM STATEMENT . We define the optimization task of the deep learning-based methods which directly use unordered point clouds as input at first . Giving a training dataset D = { ( Si , Gi ) } , where Si , Gi ⊂ R3 . Si denotes the input source point clouds and Gi denotes the input target point clouds . We aim to obtain a parametric function gθ ( Si , Gi ) using a neural network structure that can predict the rotation matrix R ∈ SO ( 3 ) and a translation vector t ∈ R3 that can deform the source point cloud towards the target point cloud . A pre-defined alignment metric between the transformed source point cloud and the target point cloud can be defined as the objective loss function to update the parameters θ . For a given dataset D , a stochastic gradient-descent based algorithm can usually be utilized to optimize the parameters θ by minimizing the pre-defined loss function : θ∗ = argmin θ [ E ( Si , Gi ) ∼D [ L ( Si , Gi , gθ ( Si , Gi ) ) ] ( 1 ) where L represents the pre-defined loss function . 2.2 LEARNING SHAPE EMBEDDING . For the input point clouds , the learning shape embedding is a non-linear multi-layer perceptron ( MLP ) -based function neural network that can extract shape features and capture the geometric information . Formally , let Pi denotes the input point clouds and fx ⊂ Rm denotes the feature of x , ∀x ∈ Pi , where m is the dimension of output layer . Our Learning Shape Descriptor includes two key components : encoding network and feature information . We define the encoding network g1 : R3 → Rm which uses multi-layer perceptrons ( MLP ) with ReLu activation function for feature extraction : fx = g1 ( x ) x∈Pi ( 2 ) The feature information is combined by extracted feature and point coordinates . Specifically , ∀x ∈ Pi , we concatenate the learned feature fx with the coordinates x as the combined feature [ fx , x ] ∈ R ( m+3 ) . Thus , the shape descriptor of input point cloud Pi is : { [ fx , x ] } x∈Pi .

Summary: This paper studies point cloud registration with deep neural networks. The key insight is learning a region-based transformation is more robust than learning a per-shape transformation even though shapes are rigid. The proposed method follows a DCP pipeline except that it performs point cloud reconstruction as well as self-supervised region segmentation. First, it uses a PointNet to lift points into a high-dimensional space and tag each point with a pseudo label. Then, points are clustered based on the region labels. Next, a per-region transformation is predicted based on the features of points in each region. Finally, the global transformation is an ensemble of region transformations.