Original: Correspondence-Free Point Cloud Registration with SO(3)-Equivariant Implicit Shape Representations
Revised: Correspondence-Free Point Cloud Registration with SO(3)-Equivariant Implicit Shape Representations

Original: Anonymous Author s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress
Revised: AfﬁliationAddress

Original: Abstract: This paper proposes a correspondence-free method for point cloud rotational registration.
Revised: Abstract: This paper proposes a correspondence-free method for point cloud rotational registration.

Original: We learn an embedding for each point cloud in a feature spacethat preserves the SO(3)-equivariance property, enabled by recent developmentsin equivariant neural networks.
Revised: We learn an embedding for each point cloud in a featurespace that preserves the SO(3)-equivariance property, enabled by recent developments in equivariant neural networks.

Original: The proposed shape registration method achievesthree major advantages through combining equivariant feature learning with implicit shape models.
Revised: The proposed shape registration methodachieves three major advantages through combining equivariant feature learningwith implicit shape models.

Original: First, the necessity of data association is removed because ofthe permutation-invariant property in network architectures similar to PointNet.
Revised: First, the necessity of data association is removedbecause of the permutation-invariant property in network architectures similar to PointNet.

Original: Second, the registration in feature space can be solved in closed-form using Horn’smethod due to the SO(3)-equivariance property.
Revised: Second, the registration in feature space can be solved in closed-formusing Horn’s method due to the SO(3)-equivariance property.

Original: Third, the registration is robustto noise in the point cloud because of  implicit shape learning.
Revised: Third, the registration is robust to noise in the point cloud because of the joint training of registrationand implicit shape reconstruction.

Original: The experimentalresults show superior performance compared with existing correspondence-freedeep registration methods.
Revised: The experimental results show superior performance compared with existing correspondence-free deep registration methods.

Original:  Introduction
Revised:  Introduction

Original: Point clouds are a major form of 3D information representation in perception with numerous applications today, including intelligent robots and self-driving cars.
Revised: Point clouds are a major form of 3D information representation in perception with numerous applications today, including intelligent robots and self-driving cars.

Original: The registration of two differentpoint clouds capturing the same underlying object or scene refers to estimating the relative transformation that aligns them.
Revised: The registration of two differentpoint clouds capturing the same underlying object or scene refers to estimating the relative transformation that aligns them.

Original: Point cloud registration enables information aggregation across severalobservations, which is essential for many tasks such as mapping, localization, shape reconstruction,and object tracking.
Revised: Point cloud registration enables information aggregation across severalobservations, which is essential for many tasks such as mapping, localization, shape reconstruction,and object tracking.

Original: Given perfect point-to-point pairings between the point-cloud-pair, the relativetransformation can be solved in closed-form using Horn’s method [1].
Revised: Given perfect point-to-point pairings between the point-cloud-pair, the relativetransformation can be solved in closed-form using Horn’s method [1].

Original: However, such a pairing is not available or corrupted in real-world problems, posing a signiﬁcantchallenge for the point cloud registration task, namely data association.
Revised: However, such a pairing is not available or corrupted in real-world problems, posing a signiﬁcantchallenge for the point cloud registration task, namely data association.

Original: In the deep learning era,the strong feature learning capacity of neural networks enables better description and discriminationof points, improving their matching.
Revised: In the deep learning era,the strong feature learning capacity of neural networks enables better description and discriminationof points, improving their matching.

Original: Overall, these methods can be categorized as correspondencebased registration.
Revised: Overall, these methods can be categorized as correspondencebased registration.

Original: An alternative approach, called correspondence-free registration, is to completely circumvent thematching of points by learning a global representation for the entire point cloud and estimating the transformation by aligning their representations instead of the original points.
Revised: An alternative approach, called correspondence-free registration, is to completely circumvent thematching of points by learning a global representation for the entire point cloud and estimatingthe transformation by aligning their representations.

Original: Similar tocorrespondence-based registration, correspondence-free registration can also be with or without theinvolvement of deep learning.
Revised: Similar to correspondence-based registration,correspondence-free registration can also be with or without the involvement of deep learning.

Original: The main obstacle in this type of approach is that the mapping between the input Euclidean space and the output feature space realized by neural networks is nonlinearand obscure.
Revised: Themain obstacle in this type of approach is that the mapping between the input Euclidean space andthe output feature space realized by neural networks is nonlinear and obscure.

Original: Thus it is common to rely on iterative optimization with local linearization like GaussNewton [2].
Revised: Thus it is commonto rely on iterative optimization with local linearization like Gauss-Newton [2].

Original: The performance of these approaches deteriorates when the initial pose difference getslarger.
Revised: The performance ofthese approaches deteriorates when the initial pose difference gets larger.

Original: In this paper, we innovatively employ equivariant neural networks and implicit shape learning intocorrespondence-free registration by constructing a feature space with two major advantages.
Revised: In this paper, we innovatively employ equivariant neural networks and implicit shape learning intocorrespondence-free registration by constructing a feature space with two major advantages.

Original: First,thanks to the equivariance property, the feature space preserves the same rotation operation as the
Revised: First,thanks to the equivariance property, the feature space preserves the same rotation operation as the

Original: Euclidean input space, allowing us to solve the feature-space registration in closed form using Horn’s method.
Revised: Euclidean input space, allowing us to solve the feature-space registration in closed form using Horn’smethod.

Original: As a result, the accuracy is independent of the initial pose difference.
Revised: As a result, the accuracy is independent of the initial pose difference.

Revised: Second, we improve the registration robustness against noise in point clouds by training the features for both the implicitshape reconstruction task and the registration task.

Original: Second, the implicitshape learning encourages the network to learn features describing the underlying geometry ratherthan speciﬁc points sampled on the geometry, encouraging features to be robust to noise.
Revised: The implicit shape learning encourages thenetwork to learn features describing the underlying geometry rather than speciﬁc points sampled onthe geometry, and the features are better prepared for registration by including the registration in thetraining loop.

Original: In thiswork, we focus on the rotational registration task.
Revised: In this work, we focus on the rotational registration task.

Original: In particular, this work has the following contributions.
Revised: In particular, this work has the following contributions.

Original: We propose a correspondence-free point cloud registration method via SO(3) -equivariancerepresentation learning and closed-form pose estimation.
Revised: We propose a correspondence-free point cloud registration method via SO(3) -equivariance representation learning and closed-form pose estimation.

Original: The proposed method achieves accurate registration results independent of initial pose differences with robustness against noisy points.
Revised: The proposed method achieves accurate registration results independent of initial pose dif- ferences with robustness against noisy points.

Original: Open source software will be available (after the ﬁnal decision).
Revised: Open source software will be available (after the ﬁnal decision).

Original:  Preliminaries on Equivariance and Vector Neurons
Revised:  Preliminaries on Equivariance and Vector Neurons

Original: Symmetry (or equivariance) in a neural network is an important ingredient for efﬁcient learning andgeneralization.
Revised: Symmetry (or equivariance) in a neural network is an important ingredient for efﬁcient learning andgeneralization.

Original: It also allows us to build a clear connection between the changes in the input spaceand the output space.
Revised: It also allows us to build a clear connection between the changes in the input spaceand the output space.

Original: For example, convolutional neural networks are translation-equivariant.
Revised: For example, convolutional neural networks are translation-equivariant.

Original: Itmakes a network generalize to image shifting and enables huge parameter saving.
Revised: Itmakes a network generalize to image shifting and enables huge parameter saving.

Original: However, othersymmetries are less explored.
Revised: However, othersymmetries are less explored.

Original: For example, most point cloud processing networks (e.g., PointNet [3], DGCNN [4]) are not equivariant to rotations.
Revised: For example, most point cloud processing networks (e.g., PointNet [3], DGCNN [4]) are not equivariant to rotations.

Original: We leverage the recently proposed Vector Neuron layers [5] to enable rotation-equivariant featurelearning for point clouds.
Revised: We leverage the recently proposed Vector Neuron layers [5] to enable rotation-equivariant featurelearning for point clouds.

Original: For the sake of completeness, we brieﬂy introduce the preliminary onequivariance and the design of layers.
Revised: For the sake of completeness, we brieﬂy introduce the preliminary onequivariance and the design of layers.

Original: A function f : X → X is equivariant to a set of transformations G , if for any g ∈ G , f and gcommutes, i.e., g · f ( x ) =
Revised: A function f : X → X is equivariant to a set of transformations G , if for any g ∈ G , f and gcommutes, i.e., g · f ( x ) =

Original: f ( g · x ) , ∀ x ∈ X .
Revised: f ( g · x ) , ∀ x ∈ X .

Original: For example, applying a translation on a 2D image andthen going through a convolution layer is identical to processing the original image with a convolution layer and then shifting the output feature map.
Revised: For example, applying a translation on a 2D image andthen going through a convolution layer is identical to processing the original image with a convolution layer and then shifting the output feature map.

Original: Therefore convolution layers are translationequivariant.
Revised: Therefore convolution layers are translationequivariant.

Original: Vector Neurons extend the equvariance property to SO(3) rotations for point cloud neural networks.
Revised: Vector Neurons extend the equvariance property to SO(3) rotations for point cloud neural networks.

Original: The key idea is to augment the scalar feature in each feature dimension to a vector in R 3 and redesignthe linear, nonlinear, pooling, and normalization layers accordingly.
Revised: The key idea is to augment the scalar feature in each feature dimension to a vector in R 3 and redesignthe linear, nonlinear, pooling, and normalization layers accordingly.

Original: In a regular network, the featurematrix with feature dimension C corresponding to a set of N points is V ∈ R N × C .
Revised: In a regular network, the featurematrix with feature dimension C corresponding to a set of N points is V ∈ R N × C .

Original: In Vector Neuronnetworks, it is  V ∈ R N × C × 3 .
Revised: In Vector Neuronnetworks, the feature matrix is of form V ∈ R N × C × 3 .

Original: The mapping between layers can be written as f : R N × C l × 3 → R N × C l +1 × 3 , where l is the layer index.
Revised: The mapping between layers can be writtenas f : R N × C l × 3 → R N × C l +1 × 3 , where l is the layer index.

Original: Following this design, the representation of SO(3) rotationsin feature space is straight-forward: g ( R ) · V :=
Revised: Following this design, the representationof SO(3) rotations in feature space is straight-forward: g ( R ) · V :=

Original: V R , where g ( R ) denotes the rotation operationin the feature space, parameterized by the 3-by-3 rotation matrix R ∈ SO(3) .
Revised: V R , where g ( R ) denotes the rotation operation in the feature space, parameterized by the 3-by-3 rotation matrix R ∈ SO(3) .

Original: In the following, weignore the ﬁrst dimension N of V for simplicity.
Revised: Inthe following, we ignore the ﬁrst dimension N of V for simplicity.

Original: The linear layer for Vector Neurons is similar to a traditional MLP:
Revised: The linear layer for Vector Neurons is similar to a traditional MLP:

Original: f lin ( V ) =
Revised: f lin ( V ) =

Original: WV , where
Revised: WV , where

Original: W ∈ R C l +1 × C l .
Revised: W ∈ R C l +1 × C l .

Original: It is easy to see that such a mapping is SO(3) -equivariant:
Revised: It is easy to see that such a mapping is SO(3) -equivariant:

Original: [Equation]
Revised: [Equation]

Original: Designing nonlinearities for Vector Neuron networks is less trivial.
Revised: Designing nonlinearities for Vector Neuron networks is less trivial.

Original: A naive ReLu on all elements inthe feature matrix will break the equivariance.
Revised: A naive ReLu on all elements inthe feature matrix will break the equivariance.

Original: Instead, a vectorized version of ReLu is designed asthe truncation of vectors along a hyperplane.
Revised: Instead, a vectorized version of ReLu is designed asthe truncation of vectors along a hyperplane.

Original: Speciﬁcally, given an input feature matrix V , predict acanonical direction k = UV ∈ R 1 × 3 where U ∈ R 1 × C .
Revised: Speciﬁcally, given an input feature matrix V , predict acanonical direction k = UV ∈ R 1 × 3 where U ∈ R 1 × C .

Original: Then the ReLU is deﬁned as:
Revised: Then the ReLU is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: otherwise
Revised: otherwise

Original: [Equation]
Revised: [Equation]

Original: for each row v ∈ R 1 × 3 of V .
Revised: for each row v ∈ R 1 × 3 of V .

Original: This design preserves SO(3) equivariance because k and V will rotatetogether, and (cid:104) V R, kR (cid:105) =
Revised: This design preserves SO(3) equivariance because k and V willrotate together, and (cid:104) V R, kR (cid:105) =

Original: (cid:104) V, k (cid:105) ,
Revised: (cid:104) V, k (cid:105) ,

Original: meaning that rotations do not change the activation.
Revised: meaning that rotations do not change the activation.

Original: For the design of pooling layers and normalization layers of Vector Neurons, please refer to theoriginal paper for details [5].
Revised: Forthe design of pooling layers and normalization layers of Vector Neurons, please refer to the originalpaper for details [5].

Original:  Methodology
Revised:  Methodology

Original: The overall idea of correspondence-free registration is to transform the alignment of points in inputspace to the alignment of global features where point-wise matching is not needed.

Original: In this work, weemploy a SO(3) -equivariant neural network to construct a feature space preserving the same rotationrepresentations as to the input space, thus simplifying solving the rotation for feature alignment.
Revised: We employ a SO(3) -equivariant neural network to construct a feature space preserving the samerotation representations as to the input space, thus simplifying solving the rotation for feature alignment.

Original: Furthermore, deep implicit representation learning improves the robustness of the features againstnoise and allows registration between two different scans of the same geometry.
Revised: Furthermore, deep implicit representation learning and end-to-end training of registrationimproves the robustness of the features against noise and allows registration between two differentscans of the same geometry.

Original: 3.1 SO(3) -equivariant point cloud global feature extractor
Revised: 3.1 SO(3) -equivariant point cloud global feature extractor

Original: We choose PointNet as the feature extractor backbone.
Revised: We choose PointNet as the feature extractor backbone.

Original: The permutation invariance property impliesthat two point clouds correspond to the same feature embeddings if they only differ in the permutation of points.
Revised: The permutation invariance property impliesthat two point clouds correspond to the same feature embeddings if they only differ in the permutation of points.

Original: We replace the layers in PointNet with the corresponding Vector Neuron version,similar to the work of Deng et al.
Revised: We replace the layers in PointNet with the corresponding Vector Neuron version [5].

Original: Denote the feature extraction network as f , a point cloud P ∈ R N × 3 , and its rotated and permuted copy P (cid:48) =
Revised: Denote the feature extraction network as f , a point cloud P ∈ R N × 3 , and its rotated and permutedcopy P (cid:48) =

Original: MPR , where M ∈ R N × N is a permutationmatrix and R ∈ SO(3) is a 3-by-3 rotation matrix.
Revised: MPR , where M ∈ R N × N is a permutation matrix and R ∈ SO(3) is a 3-by-3 rotationmatrix.

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: where Q := f ( P )
Revised: where Q := f ( P )

Original: ∈ R C × 3 , guaranteed by the  SO(3) -equivariance and permutation-invarianceproperties, which is essential for feature-space registration.
Revised: ∈ R C × 3 is the feature of P extracted by the encoder. Equation (3) is guaranteedby the SO(3) -equivariance and permutation-invariance properties, which is essential for our featurespace registration.

Revised: A Vector Neuron network is designed to take inputs of R N × C 0 × 3 , while the point cloud input is

Revised: P ∈ R N × 3 .

Revised: Simply reshaping P to R N × 1 × 3 with C 0 = 1 does not work, because a linear layerwith weight W ∈ R C 1 × 1 will make the resulted feature

Revised: WP ∈

Revised: R N × C 1 × 3 linearly dependent in thethird dimension for each point.

Revised: Following [5], we use one DGCNN edge-convolution [4] layer toinitialize the R N × C 0 × 3 feature from raw R N × 3 point cloud using the neighboring points.

Revised: See [5]for more details.

Revised: Furthermore, we explore replacing the k-nearest neighbor operation with randomsampling of k points in a ball with certain radius in the edge convolution layer.

Revised: The motivation isto alleviate the sensitivity of the feature against point cloud density (see Sec. 4.2.3).

Revised: However, theevaluations in this paper are using k-NN by default unless speciﬁed.

Original: 3.2 Deep implicit representation learning
Revised: 3.2 Deep implicit representation learning

Original: Being able to register two rotated copies of the same point cloud is not enough for practical use.
Revised: Being able to register two rotated copies of the same point cloud is not enough for practical use.

Original: In real application, the measurements have noise, and the individual points captured by differentscans generally do not correspond to the same physical points.
Revised: Inreal application, the measurements have noise, and the individual points captured by different scansgenerally do not correspond to the same physical points.

Original: To our knowledge, we are amongthe ﬁrst to address the need for registration across noisy and different scans of point clouds in deepcorrespondence-free registration methods.

Original: Our solution is to repurpose the global feature (for registration ultimately) for implicit shape representation.
Revised: Our solution is to repurpose the global feature (for registration ultimately) for implicit shape representation.

Original: Following  Occupancy Network, we build an encoder-decoder network.
Revised: Following the Occupancy Network [6], we build an encoder-decoder network.

Original: The encoder is theaforementioned SO(3) -equivariant feature extractor, and the decoder takes the  feature  and a queriedposition as input, predicting the occupancy value of that position.
Revised: The encoder is the aforementioned SO(3) -equivariant feature extractor f , taking a point cloud P ∈ R N × 3 as input. The decoder takes the encoded shape feature Q = f ( P ) ∈ R C × 3 and a queried positionp ∈ R 3 as input, predicting the occupancy value of that position v ( p ; Q ) ∈

Original: In this way, the encoder is encouraged to predict identical features if the inputpoint clouds correspond to the same geometry evenif they scan different points.
Revised: Notice that p neednot to be coming from P . In practice, we sample n querying points for each input to calculate theimplicit shape reconstruction (occupancy value prediction) loss:

Original: Gaussian noise isalso added to the input during training to improvethe robustness.
Revised: [Equation] where v gt ( p ) ∈ { 0 , 1 } is the ground truth occupancy value of a query point p . The sampling strategyof the querying points is consistent with the Occupancy Network [6].

Original: In order to output an identical implicit function ﬁeld, we adopted the same invariantlayer designed by Deng et al. [5] to convert the SO(3) -equivariant features to be SO(3) -invariantbefore feeding them into the decoder.
Revised: In this way, we implicitlyencourage the feature f ( P 1 ) , f ( P 2 ) , ... to be close as long as P 1 , P 2 , ... are point clouds of the samegeometry.

Original: 3.3 Point cloud registration through feature alignment
Revised: 3.3 Point cloud registration through feature alignment

Original: With the above network design, we can further relax the connection between the two point clouds
Revised: With the above network design, we can further relax the connection between the two point clouds

Original: P (cid:48) and P in (3).
Revised: P (cid:48) and P in (3).

Original: They do nothave to be permutations of each other, as long as they are sampled onthe same geometry.
Revised: The features Q and Q (cid:48) are approximately rotation-equivariant even when P and P (cid:48)are not permutations of each other, as long as they are sampled on the same geometry.

Original: However, the equation only holds approximately in this case since the implicitshape model is not trained perfectly.
Revised: We say it isapproximate because a trained implicit shape reconstruction network can only represent the implicitshape approximately. Therefore, we also include the registration step in the training loop, directlyencouraging the accurate registration of point clouds with noise. As shown below, the registration isof closed form and differentiable, allowing end-to-end training of better features.

Original: One can see that (3) is exactly in the form of orthogonal Procrustes problem.
Revised: One can see that (3) is exactly in the form of orthogonal Procrustes problem.

Original: The learned features
Revised: The learned features

Original: Q and Q (cid:48) can be regarded as two pseudo point clouds and each row of them are automaticallymatched .
Revised: Q and Q (cid:48) can be regarded as two pseudo point clouds and each row of them are automaticallymatched .

Original: This problem has a standard closed-form solution.
Revised: This problem has a standard closed-form solution.

Original: First calculate the cross covariancematrix
Revised: First calculate the cross covariancematrix

Original: H =
Revised: H =

Original: Q T Q (cid:48) .
Revised: Q T Q (cid:48) .

Original: Then conduct Singular Value Decomposition (SVD) on H : H = USV T .
Revised: Then conduct Singular Value Decomposition (SVD) on H : H = USV T .

Original: Theoptimal rotation matrix ˆ R is given as:
Revised: Theoptimal rotation matrix ˆ R is given as:

Original: [Equation]
Revised: [Equation]

Original: where Λ = diag (1 , 1 , det ( V U T ))
Revised: where Λ = diag (1 , 1 , det ( V U T ))

Original: ∈ R 3 × 3 .
Revised: ∈ R 3 × 3 .

Original: Not only can each row q ∈ R 3 of the feature matrix Q be regarded as a point in Euclidean space,q can also be interpreted as an element in so (3) , and visualized as a vector ﬁeld applied on the Euclidean space where the input point clouds live.

Original: Each feature generates such a vector ﬁeld. Becauseof the rotation-equivariance property, the vector ﬁeld generated from the feature of the rotated pointcloud is identical to the rotated vector ﬁeld generated from the feature of the original point cloud.
Revised: We design the registration loss as the mean square error between the residual rotation matrix and theidentity matrix:

Original: See Fig.
Revised: [Equation]

Original: 2 for an example.
Revised: where R est is calculated from (5), and (cid:107)(cid:107) F denotes the Frobenius norm.

Original:  Experiments
Revised:  Experiments

Original: Consistent with existing literature, we conduct experiments on the ModelNet40 dataset [6].
Revised: Our network is trained on the ModelNet40 dataset [7].

Original: It iscomposed of 12311 CAD models from 40 categories of objects.
Revised: It is composed of 12311 CAD models from40 categories of objects.

Original: Following the ofﬁcial split, 9843models are used for training, and 2468 models are used for testing.
Revised: Following the ofﬁcial split, 9843 models are used for training, and 2468models are used for testing.

Original: The models are preprocessed 
Revised: The models are preprocessed using the tool provided by Stutz and

Original: Tested using ModelNet40 ofﬁcial test set.
Revised: Geiger [8] to obtain watertight and simpliﬁed models that are centered and scaled to a unit cube. Given watertight models, the occupancy value of any 3D coordinate can be determined dependingon whether it is inside or outside the model. We also evaluate our method on a real-world dataset: 7Scenes [9], which is a subset of 3DMatch[10]. We choose it in order to be consistent and comparable with one of our baselines FMR [11].7Scenes consists of 7 sequences of RGBD scans of indoor environments.

Original: The max initial rotation angle refers to the maximum rotation angle allowed in the generation of initial random rotations during testing.
Revised: RGB information is notused in the experiments. The training of the network is composed of two stages. The network is ﬁrst trained for occupancyvalue prediction only, after which the network is trained for the occupancy prediction and registrationtasks jointly. In the ﬁrst stage, 1024 points are randomly sampled on each mesh model as the inputpoint cloud.

Original:  The rotation axis is fully randomized. The last column (180 degrees) corresponds to unconstrained
Revised: with batch size 24. In the second stage, a pair of point clouds are sampled on the sameshape model with different pose and number of points. The registration loss between the pair iscalculated, together

Original: [Table] [Table]
Revised: [Table] [Table]

Original: using the tool provided by Stutz and Geiger [7] to obtain watertight and simpliﬁed models that arecentered and scaled to a unit cube. During training, 1000 points are randomly sampled on each mesh model as the input point cloud. The occupancy value is queried at 2048 points in the unit cube to compose the loss function. Thesampling strategy of the querying points is consistent with the Occupancy Network [8].
Revised: [Table]

Original: The dimension of the learned global feature is set as C = 342 , corresponding to 342 × 3 = 1026 dimensionsfor a traditional feature vector, close to the dimension chosen by previous literature including PointNetLK [9], PCRNet [10], and Feature-Metric Registration (FMR) [11].
Revised: The dimension of the learned global feature is set as C = 342 , corresponding to 342 × 3 = 1026dimensions for a traditional feature vector, close to the dimension chosen by previous literature including PointNetLK [12], PCRNet [13] [Table] and Feature-Metric Registration (FMR) [11].

Original: The network is trained on asingle Tesla V100 GPU with batch size 24. 
Revised: The networkis trained on a single Tesla V100 GPU. 4.2 Evaluation on synthetic data: ModelNet

Original: 4.1 Registration with rotated copies of point clouds
Revised: 4.2.1 Registration with rotated copies of point clouds

Original: We ﬁrst tested the registration of two points clouds that are only different by rotation and permutation.
Revised: We ﬁrst tested the registration of two points clouds that are only different by rotation and permutation.

Original: Both point clouds have 1000 points sampled from the mesh model.
Revised: Both point clouds have 1000 points sampled from the mesh model.

Original: The initial rotations aregenerated randomly using axis-angle representation to specify the maximum magnitude of the initial rotations.
Revised: The initial rotationsare generated randomly using axis-angle representation to specify the maximum magnitude. Theerror is also measured as the angle in axis-angle representations.

Original: We compare with two representative correspondence-free deep registration methodsas baselines: PCR-Net [10] and FMR [11].
Revised: We compare with two representative correspondence-free deep registration methods as baselines: PCR-Net [13] and FMR [11].

Original: The experiments on the baseline methods are based ontheir ofﬁcial open-source implementation and pretrained weights.
Revised: The experiments on the baseline methods are based on their ofﬁcial open-source implementationand pretrained weights.

Original: The results are shown in Table 1.
Revised: The results are shown in Table 1.

Original: We can see that both PCR-Net and FMR are sensitive to the initial rotation angle and only workproperly when the initial rotation is small because they rely on iterative reﬁnements.
Revised: We can see that both PCR-Net and FMRare sensitive to the initial rotation angle and only work properly when the initial rotation is smallbecause they rely on iterative reﬁnements.

Original: In comparison,the registration accuracy of our method is independent of the initial rotation angle, and it providesalmost perfect rotational registration.
Revised: In comparison, the registration accuracy of our methodis independent of the initial rotation angle, and it provides almost perfect rotational registration.

Original: 4.2 Registration with rotated copies of point clouds corrupted with Gaussian noise
Revised: 4.2.2 Registration with rotated copies of point clouds corrupted with Gaussian noise

Original: In practice, the pair of point clouds for registration will not be identical to each other.
Revised: In practice, the pair of point clouds for registration will not be identical to each other.

Original: Thereforewe test the case where the source and target point clouds are both corrupted by noise.
Revised: Thereforewe test the case where the source and target point clouds are both corrupted by noise.

Original: We add a
Revised: We add a

Original: Gaussian noise with σ = 0 .
Revised: Gaussian noise with σ = 0 .

Original: 01 to all points before putting them through the networks.
Revised: 01 to all points before putting them through the networks.

Original: The restof the setup is the same as in Sec.4.1.
Revised: The rest ofthe setup is the same as in Sec.

Original: As shown in Table 2, PCR-Net and FMR show a similartrend of increasing error when the initial angle gets larger.
Revised: As shown in Table 2, PCR-Net and FMR show a similartrend of increasing error when the initial angle gets larger.

Original: Our method, though showing a slightlylarger registration error, still outperforms the baseline in most columns.
Revised: Our method, though showing a slightlylarger registration error, still outperforms the baseline in most columns.

Original: Most importantly, the errorremains consistent across different initialization.
Revised: Most importantly, the errorremains consistent across different initialization.

Original: The result shows that our method not only works inan ideal situation where the point clouds are noise-free but also delivers accurate rotation estimationunder noise corruption.
Revised: The result shows that our method not only works inan ideal situation where the point clouds are noise-free but also delivers accurate rotation estimationunder noise corruption.

Original: [Table]
Revised: 4.2.3 Registration with point clouds with different densities

Original: To further test the robustness of the proposed method against variations in the point clouds, wetested the registration performance when given two point clouds of different densities.
Revised: To further test the robustness of the proposed method against variations in the point clouds, wetested the registration performance when given two point clouds of different densities.

Original: Here wesample 1000 and 500 points for the pair of point clouds, respectively.
Revised: Here wesample 1000 and 500 points for the pair of point clouds, respectively.

Original: The registration accuracy isshown in Table 3.
Revised: The registration accuracyis shown in Table 3.

Original: The trend of the three methods is similar to what is shown in Sec.
Revised: The trend of the three methods is similar to what is shown in Sec.

Original: 4.1 and Sec.4.2.
Revised: 4.2.1 and Sec.

Original: We do observe an increase of error in our method in this experiment.
Revised: We do observe an increase of error in our method in this experiment.

Original: A possible reason is that  at thebeginning of our PointNet encoder is an edge-convolution layer to initialize the feature dimensionsof each point, which is dependent on the neighborhood.
Revised: A possible reasonis that the DGCNN layer for feature initialization at the beginning of the encoder is sensitive to thepoint cloud density, since it takes the relative coordinate of neighboring points as inputs. Therefore, we experimented replacing the k-nearest neighbor with random sampling in a ball centered at each point when collecting neighboring points for the edge-convolution of that point. Inpractice, we sample 20 neighboring points for both strategy, and the radius for ball sampling is 0.2. [Table]

Original: Change of density may change the featureat the ﬁrst layer, affecting the feature space registration.
Revised: Since our method is invariant to [Table] , in this only [Table]

Original: This issue isto be analyzed in more detailin future work.
Revised: [Table] of Table [Table] The result is in Table 4.

Revised: One can see that the ball sampling strategy is able to improve the registration performance for point clouds of different density.

Revised: However, the error is slightly highercompared with k-NN when the point clouds are rotated copies or with Gaussian noise with the samenumber of points.

Revised: It is understandable since the ball sampling introduces randomness in the featureinitialization, and could slightly worsen the registration when the inputs are originally ideal.

Revised: With either of the strategies, under the condition of density difference, the registration error isslightly larger than that of the baselines when the initial rotation angle is small, but is much smallerwhen the initial rotation is arbitrary.

Revised: A possible way for further improvement is to use our methodas initialization, and then run an iterative-optimization-based method such as FMR or PointNetLKfor reﬁnement.

Revised: It is not closely related to our key innovation, therefore not further discussed in thispaper.

Revised: 4.3 Evaluation on real-world data: 7Scenes

Revised: The model evaluated on 7Scenes is ﬁrst trained on ModelNet40 with the two stages mentionedin Sec. 4.1, and then ﬁnetuned on 7Scenes with the registration loss only.

Revised: The train/test split isconsistent with [11].

Revised: 4.3.1 Registration with point clouds that are sampled differently

Original: However, our result still outperforms the baselines when the initial rotation angle islarger than 90 degrees.
Revised: The experiments on 7Scenes consists of two parts. In the ﬁrst part, the pair of point clouds are fromthe same frame, but sampled differently. The original dense RGBD scan has more than 100,000points. 1000 points are sampled randomly to form an input point cloud.

Revised: The pair of point cloudsthen go through random rotation to form the ﬁnal input.

Revised: In this case, the points in one point cloudis neither a noisy version nor a subset of the other point cloud.

Original: Overall, our proposed orientation registration  method can estimate consistent results independent ofinitial rotations.
Revised: To our knowledge, we are one of theﬁrst to report registration results under this setting in correspondence-free methods [12, 11, 13, 14]. Table 5 shows that our method is working properly on real world scan of point clouds that aredifferent samplings of the underlying geometry. 4.3.2 Registration with point clouds that are partially overlapping In the second part, we report registration results of point clouds coming from adjacent frames in thesequence.

Revised: Because of camera movement, the pair of point clouds only overlaps partially.

Revised: Since ourmethod only estimates rotations, we ﬁrst align the adjacent frames using ground truth pose, thenrandomly rotate two point clouds separately in a common frame, so that the input point clouds arerelated by a rotation.

Revised: To our knowledge, we are also one of the ﬁrst to show quantitative results onpartially overlapping point clouds in correspondence-free methods [12, 11, 13, 14].

Revised: The results are in Table 6.

Original: It outperforms the baselines even under  the presence of noise, showing the practicality of our proposed method.
Revised: Both our method and the baseline deteriorate in this setting. For FMR,the system completely fails when taking partially-overlapping inputs. Our method consistently givesbetter rotation estimation under all initial conditions. Still, the accuracy has a lot of room for improvement, indicating that there is still a long way to go for correspondence-free registration methods.

Revised: For example, recent work by Huang et al.

Original: Under density variance, the accuracy of  our  method deteriorates,showing an improvement direction of our method.
Revised: [15] predicting the overlapping part of point cloudpairs may inspire correspondence-free methods to better handle partial overlapping inputs by predicting features focused on the overlapping parts. [Table] [Table] Overall, our proposed rotational registration method can estimate consistent results independent ofinitial rotations.

Original: Nevertheless, it still outperforms the baselinemethods under large initial rotations.
Revised: It outperforms the correspondence-free baselines in all evaluation settings wheninitial rotational angle is arbitrary.

Original: In this work, we connect ideas in several different ﬁelds, including sensor registration, implicit shapelearning, and equivariant neural networks.
Revised: Experiments on both synthetic data and real-world data areconducted, covering conditions with noisy inputs, varying density, different sampling, and partialoverlapping.

Original: It may beneﬁt the understanding of our method provideda broader context of each of the ﬁelds.
Revised: Although outperforming the baseline, the partial overlapping case is not perfectlyhandled by our method yet, and we brieﬂy discussed possible future directions.

Original:  Here we ﬁrst review point cloud registration methods withand without data association with a focus on deep learning-based approaches. Then we reviewliterature about equivariant neural networks and deep implicit models brieﬂy.
Revised:  Related Work and Discussion

Original: 5.1 Correspondence-based point cloud registration
Revised: 5.1 Correspondence-based point cloud registration

Original: A major challenge here is to recover the corresponding pairs of points from a pair of point clouds.
Revised: A major challenge here is to recover the corresponding pairs of points from a pair of point clouds.

Original: ICP simply matches the closest points together, solves the transformation, and iteratively rematchesthe closest points after aligning the two point clouds using the estimated transformation [12].
Revised: ICP simply matches the closest points together, solves the transformation, and iteratively rematchesthe closest points after aligning the two point clouds using the estimated transformation [16].

Original: Pointto-line [13], point-to-plane [14], plane-to-plane [15], and Generalized-ICP [16] build local geometricstructures to the loss formulation.
Revised: Pointto-line [17], point-to-plane [18], plane-to-plane [19], and Generalized-ICP [20] build local geometricstructures to the loss formulation.

Original: A challenge in the matching is to distinguish points corresponding to different underlying locationsand to recognize points corresponding to the same location, requiring strong feature descriptor forpoints, on which deep learning approaches show expertise.
Revised: Finding correspondences require strong feature descriptor for points, on which deep learning approaches show expertise.

Original: Through metric learning, a feature descriptor can be learned such that matching points are close to each other in the feature space, whilenon-matching points are far away.
Revised: Through metric learning, a feature descriptor can be learned such thatmatching points are close to each other in the feature space, while non-matching points are far away.

Original: Approaches following this idea include PPFNet [17], 3DSmoothNet [18], SpinNet [19], and FCGF [20].
Revised: Approaches following this idea include PPFNet [21], 3DSmoothNet [22], SpinNet [23], and FCGF[24].

Original: Good point matching leads to accurate pose estimationwhen solving the orthogonal Procrustes problem.
Revised: Good point matching leads to accurate pose estimation when solving the orthogonal Procrustesproblem.

Original: Therefore, the error in pose estimation can beused to supervise point matching and feature learning.
Revised: Therefore, the error in pose estimation can be used to supervise point matching and feature learning.

Original: DCP [21], RPM-Net [22], DGR [23], and3DRegNet [24] are some of the methods leveraging ground truth pose to supervise point featurelearning.
Revised: DCP [25], RPM-Net [26], DGR [27], and 3DRegNet [28] are some of the methodsleveraging ground truth pose to supervise point feature learning.

Original: In a point cloud, it is likely that a subset of them (e.g. corner points) have more salientfeatures and are easier to be identiﬁed than others (e.g. points on a ﬂat surface).

Original: They are calledkeypoints.

Original: Keypoints can be matched more reliably, resulting in better pose estimation.

Original: In thisregard, neural networks are designed to better pick the keypoints (e.g., USIP [25] and SampleNet[26]).
Revised: Neural networks are designed tobetter pick the keypoints (e.g., USIP [29] and SampleNet [30]).

Original: Keypoint selection and feature learning may also be considered jointly
Revised: Keypoint selection and featurelearning may also be considered jointly.

Original: Representative works 
Revised: Representative works include 3DFeat-Net [31], D3Feat[32], DeepICP [33], and PRNet [34].

Original: The main remaining challenge is that such matching pairs may not exist in the input point cloudsin the ﬁrst place, because point clouds are sparse and a point may not be captured repeatedly bydifferent scans.
Revised: The main remaining challenge is that such matching pairs may not exist in the input point cloudsin the ﬁrst place, because point clouds are sparse and a point may not be captured repeatedly bydifferent scans.

Original: Soft matching (or many to many) [31, 32] is proposed to address this problem,but it is at best an approximation of the underlying true matching using sparse samples, which candeteriorate the performance when the sparsity increases.
Revised: Soft matching (or many to many) [35, 36] is proposed to address this problem,but it is at best an approximation of the underlying true matching using sparse samples, which candeteriorate the performance when the sparsity increases.

Original: 5.2 Correspondence-free point cloud registration
Revised: 5.2 Correspondence-free point cloud registration

Original: Correspondence-free registration treats a point cloud as a whole rather than a collection of points,requiring a global representation for an entire point cloud.
Revised: Correspondence-free registration treats a point cloud as a whole rather than a collection of points,requiring a global representation for an entire point cloud.

Original: In this way, the limitation brought bythe sparsity as mentioned in Sec.
Revised: In this way, the limitation brought by thesparsity as mentioned in Sec.

Original: 5.1 is circumvented.
Revised: 5.1 is circumvented.

Original: CVO [33, 34] represents a point cloud as afunction in a reproducing kernel Hilbert space, transforming the registration problem to maximizingthe inner product of two functions.
Revised: CVO [37, 38] represents a point cloud as a function in a reproducing kernel Hilbert space, transforming the registration problem to maximizing theinner product of two functions.

Original: A series of deep-learning-based methods attempt to extract aglobal feature embedding for a point cloud and solve the registration by aligning the global features.
Revised: A series of deep-learning-based methods attempt to extract a globalfeature embedding for a point cloud and solve the registration by aligning the global features.

Original: Examples include PointNetLK [9], PCRNet [10], and Feature-Metric Registration [11].
Revised: Examples include PointNetLK [12], PCRNet [13], and Feature-Metric Registration [11].

Original: A limitationof global-feature-based registration is that the nonlinearity of the feature extraction networks leavesus few clues about the structure and properties of the feature space.
Revised: A limitationof global-feature-based registration is that the nonlinearity of the feature extraction networks leavesus few clues about the structure and properties of the feature space.

Original: Therefore these methods relyon iterative local optimization such as the Gauss-Newton algorithm.
Revised: Therefore these methods relyon iterative local optimization such as the Gauss-Newton algorithm.

Original: Consequently, these methodsrequire good initialization to achieve decent results.
Revised: Consequently, these methodsrequire good initialization to achieve decent results.

Original: Our method differentiates from previous workin that the feature extraction network is SO(3) equivariant, enabling well-behaved optimization inthe feature space (detailed in Sec. 3).
Revised: Our method differentiates from previous work in that the feature extraction network is SO(3) equivariant, enabling well-behaved optimization inthe feature space (detailed in Sec. 3).

Original: 5.3 Group equivariant neural networks
Revised: 5.3 Group equivariant neural networks

Original: Neural networks today mainly preserve symmetry against translation, limiting the capacity of networks to deal with broader transformations presented in data.
Revised: Neural networks today mainly preserve symmetry against translation, limiting the capacity of networks to deal with broader transformations presented in data.

Original: Researchers have been working ongroup-equivariant neural networks to address this issue.

Original: For example, the SO(3) group is a classical

Original: Lie group and frequently appears in practice [35, 36].

Original: One line of work forms kernels as some steerable functions so that the rotations in the input space can be transformed into rotations in the outputspace [37, 38, 39].
Revised: One line of work forms kernels assome steerable functions so that the rotations in the input space can be transformed into rotationsin the output space [39, 40, 41].

Original: However, this strategy can limit the expressiveness of the feature extractor sincethe form of the kernels is constrained.
Revised: However, this strategy can limit the expressiveness of the featureextractor since the form of the kernels is constrained.

Original: Another strategy is to lift the input space to a higher-dimensional space where the group is containedso that equivariance is obtained naturally [40, 41].
Revised: Another strategy is to lift the input space to a higher-dimensional space where the group is contained so that equivariance is obtained naturally [42, 43].

Original: The simplest example is that the group of 2Dtranslations is contained in R 2 ; therefore, 2D convolution embeds equivariance to 2D translations.

Original: However, for each lifted input to K group elements, we need to integrate over the entire groupfor computing its convolution.
Revised: However, for each lifted input to K groupelements, we need to integrate over the entire group for computing its convolution.

Original: While being mathematically sound, this approach involves the use ofgroup exp and log maps (for mapping from and to the Lie algebra) and a Monte Carlo approximationto compute the convolution integral over a ﬁxed set of group elements [41, Sections 4.2-4.4].
Revised: While beingmathematically sound, this approach involves the use of group exp and log maps (for mapping fromand to the Lie algebra) and a Monte Carlo approximation to compute the convolution integral overa ﬁxed set of group elements [43, Sections 4.2-4.4].

Original: Cohen and Welling [42] studied the problem of learning the irreducible representations of commutative Lie groups.
Revised: Cohen and Welling [44] studied the problem of learning the irreducible representations of commutative Lie groups.

Original: Kondor and Trivedi [40] studied the generalization of equivariance and convolutionin neural networks to the action of compact groups.
Revised: Kondor and Trivedi [42] studied the generalization of equivariance and convolutionin neural networks to the action of compact groups.

Original: The latter also discussed group convolutions andtheir connection with Fourier analysis [35, 36].
Revised: The latter also discussed group convolutions andtheir connection with Fourier analysis [45, 46].

Original: In this context, the convolution theorem naturallyextends to the group-valued functions and the representation theory [43].
Revised: In this context, the convolution theorem naturallyextends to the group-valued functions and the representation theory [47].

Original: A new design of SO(3) equivariant neural networks is proposed by Deng et al.
Revised: A new design of SO(3) equivariant neural networks is proposed by Deng et al.

Original: This approachallows incorporating the SO(3) equivariance property into existing network architectures such as
Revised: This approachallows incorporating the SO(3) equivariance property into existing network architectures such as

Original: PointNet and DGCNN by replacing the linear, nonlinear, pooling, and normalization layers withtheir vector neuron version.
Revised: PointNet and DGCNN by replacing the linear, nonlinear, pooling, and normalization layers withtheir vector neuron version.

Original: Our equivariant feature learner builds on this simple and useful idea.
Revised: Our equivariant feature learner builds on this simple and useful idea.

Original: 5.4 Deep implicit shapemodeling
Revised: 5.4 Deep implicit shape

Revised: [Figure]

Original: Implicit shape modeling represents a shape as a ﬁeld on which a function such as an occupancy orsigned distance function is deﬁned.
Revised: Implicit shape modeling represents a shape as a ﬁeld on which a function such as an occupancy orsigned distance function is deﬁned.

Original: The surface is typically deﬁned by the equation f ( x, y, z )
Revised: The surface is typically deﬁned by the equation f ( x, y, z )

Original: = cfor some constant c .
Revised: = cfor some constant c .

Original: In recent years, deep models are developed to model such implicit functionﬁelds, where a network is trained to predict the function value given a query input point coordinate.
Revised: In recent years, deep models are developed to model such implicit functionﬁelds, where a network is trained to predict the function value given a query input point coordinate.

Original: Some representative works include DeepSDF [44], OccNet [8], NeRF [45].
Revised: Some representative works include DeepSDF [48], OccNet [6], NeRF [49].

Original: Implicit functions model a shape continuously, up to the inﬁnite resolution, which offers opportunities in registration to circumvent the challenge in data association due to sparsity in point clouds.
Revised: Implicit functions model a shape continuously, up to the inﬁnite resolution, which offers opportunities in registration to circumvent the challenge in data association due to sparsity in point clouds.

Original: DI-Fusion [46] leverages deep implicit models to do surface registration and mapping by optimizingthe position of the query points to minimize the absolute SDF value.
Revised: DI-Fusion [50] leverages deep implicit models to do surface registration and mapping by optimizingthe position of the query points to minimize the absolute SDF value.

Original: iNeRF [47] performs RGBonly registration through deep implicit models by minimizing the pixel residual between observedimages and neural-rendered images.
Revised: iNeRF [51] performs RGBonly registration through deep implicit models by minimizing the pixel residual between observedimages and neural-rendered images.

Original: In this work, we also exploit the continuous nature of the deepimplicit model but by using the learned latent feature instead of the decoded function ﬁeld.
Revised: In this work, we also exploit the continuous nature of the deepimplicit model but by using the learned latent feature instead of the decoded function ﬁeld.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we presented a correspondence-free rotational registration method for point clouds.
Revised: In this paper, we presented a correspondence-free rotational registration method for point clouds.

Original: The method is built upon the developments in equivariant neural networks and implicit shape representation learning.
Revised: The method is built upon the developments in equivariant neural networks and implicit shape representation learning.

Original: We construct a feature space where the rotational relations in Euclidean spaceare preserved, in which registration can be done efﬁciently.
Revised: We construct a feature space where the rotational relations in Euclidean space arepreserved, in which registration can be done efﬁciently.

Original: We circumvent the need for data association and solve the rotation in closed form, achieving low errors independent of initial rotationangles.
Revised: We circumvent the need for data associationand solve the rotation in closed form, achieving low errors independent of initial rotation angles.

Original: Furthermore, we leverage implicit representation learning  to enhance the network robustnessagainst non-ideal cases where the points are subject to noise.
Revised: Furthermore, we leverage implicit representation learning and end-to-end registration to enhancethe network robustness against non-ideal cases where the points are not exactly corresponding.

Revised: We conducted experiments on synthetic and real-world data, showing the network’s ability to handlevarious input imperfections.

Original: Nevertheless, the robustness of our method against density variations still has room for improvement.
Revised: Nevertheless, the robustness of our method against partial overlappinginputs still has room for improvement.

Original: Furthermore, there are two open issues unsolved.
Revised: Furthermore, there are two open issues unsolved.

Original: First, generalizing the SO(3) -equivariant neuralnetwork to deal with translation so that the registration of rotation and translation can be solvedtogether.
Revised: First,generalizing the SO(3) -equivariant neural network to SE(3) so that the registration of rotation andtranslation can be solved together.

Original: Secondly, generalizing the registration tolarger-scale problems to handle outdoor LiDARor stereo camera scans.
Revised: Secondly, generalizing the registration to handle outdoor LiDARscans.

Original: These problems are left for future studies.
Revised: These problems are left for future studies.