SE(3)-Equivariant Point Cloud-Based Place Recognition
Authors: Chien Erh Lin
Jingwei Song
Ray Zhang
Minghan Zhu
Maani Ghaffari
Venue: CoRL 2022 Poster (2022)
Keywords: Place Recognition
SE(3)-Invariant
Equivariant Representation Learning
3D Point Clouds

Abstract: This paper reports on a new 3D point cloud-based place recognition framework that uses SE(3)-equivariant networks to learn SE(3)-invariant global descriptors. We discover that, unlike existing methods, learned SE(3)-invariant global descriptors are more robust to matching inaccuracy and failure in severe rotation and translation configurations. Mobile robots undergo arbitrary rotational and translational movements. The SE(3)-invariant property ensures that the learned descriptors are robust to the rotation and translation changes in the robot pose and can represent the intrinsic geometric information of the scene. Furthermore, we have discovered that the attention module aids in the enhancement of performance while allowing significant downsampling. We evaluate the performance of the proposed framework on real-world data sets. The experimental results show that the proposed framework outperforms state-of-the-art baselines in various metrics, leading to a reliable point cloud-based place recognition network. We have open-sourced our code at: https://github.com/UMich-CURLY/se3_equivariant_place_recognition.

SE(3)-Equivariam Point Cloud-Based Place Recognition
Chien Erh Lin, Jingwei Song, Ray Zhang, Minghan Zhu, and Maani Ghaffari University Michigan, Ann Arbor, MI, USA (chienerh, jingweso, rzh, minghanz, maanigi) @umi ch. edu
Abstract: This paper reports on new 3D point cloud-based place recognition framework that uses SEu3ieeqivvarian networks to learn SE(3)-invariant global descriptors. We discover that, unlike existing methods, learned SE33a-invrriant global descriptors are more robust to matching inaccuracy and failure in severe rotation and translation configurations. Mobile robots undergo arbitrary rota- tional and translational movements. The SE(3)-invariant property ensures that the learned descriptors pose and can represent the intrinsic geometric information of the scene. Further- more, have discovered that the attention module aids in the enhancement of performance while allowing significant downsampling. We evaluate the perfor- mance of the proposed framework real-world data sets. The experimental re- sults show that the proposed framework outperforms state-of-the-ar baselines in various metrics, leading reliable point cloud-based place recognition network. We have open-sourced our code at https: /ttp:/ggttttt gighub....... com/ ch CURLY/ se
robust the rotation and translation changes in the robot
Keywords: Place Recognition, SE(3)-Invariant, SE(3)-Equivariant Representa- tion Learning, 3D Point Clouds
1 Introduction
Place recognition, also known as loop closure detection, can be defined as linking the sensor's in-situ observations and the prebuilt reference map. is critical component in Simultaneous I Localization and Mapping (SLAM) and enables a robot to determine if has seen a place before and provides loop closure candidates [1]. Among numerous robot perception sensors, 3D (stereo, LiDAR, and RGB- D) sensors are gaining popularity. They are widely equipped in modern service robots, autonomous cars [2]. and drones [3] due to their better environment perception ability and decreasing prices. Thus, place recognition techniques with 3D data can be used in estimating the agent's location in scenarios such self self-driving vehicles, autonomous indoor navigation, scientific exploration.
Extracting consistent features from 3D data an important research topic but remains underexplored and unsolved [4]. One key issue in present place recognition methods is that they do not consider transformation changes in data or expect robustness via simple data augmentation [5]. Thus, the extracted global descriptors and their place recognition performance are sensitive to transformation variations in the training and testing point cloud samples. Our research aims at designing a rotation and translation-invaria global descriptor for point clouds, called 3E(3)-invariant feature, to solve the transformatioon-scniiiivvyy problem.
The attention mechanism of transformers [6] enables networks to learn the correlation between input features and obtain the importance of each feature. Some place recognition frameworks like PCAN [7] use attention mechanism local features to re-weight each feature. However, they usually apply an attention mechanism to feature space to learn the importance of each feature. In our work, we apply the attention mechanism on the 3D points to learn which point to reserve during the downsampling process.
In this paper, we propose place recognition framework that exploits SE(3)-equivariant representa- tion learning to perform place recognition in challenging rotation and translation scenes (Figure l).
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

Attentive
SE(3)- ((L3)--quvvariant Local Feature Encoder
SE(3)-Invarian
(nvaa-Invariant
SE(3)-Invariant Global Descriptor
Downsampling
Local Feature
Glohal Descriptor Extraction
Point Cloud
(Optional)
Pooling
c.g.
Scl--atersion
EPN
Max-paeling
NetVLAD
Linex
or
or
PPN
GeM
Figure 1: Overview of the proposed SFG3) -equivvrraa point cloud-baaed place recognition pipeline. Optionally, 3D point clouds preproeessed with attentive downsampling. Next, -qqiiarqui local features are learned using -Equivequi networks. In this work, leverage EPN and ENP Then, SRC3)-invariant local features are extracted by max-pooling Lastly, SE(3)-invariant global descriptors are computed by global pooling methods. this work, we use NetVLAD and Generalized Mean (GeM). The global descriptors are SE(3) SEvaaainaa and can perform place recognition tasks.
We learn -Eu3)-equvvarian local features via group-equivariant encoder networks. And our pipeline is agnostic to the particular approach for learning equivariant features. Then, SE(3)-invariant global descriptors are learned by aggregating local features. Moreover, we apply a self-attention mecha- nism for downsampling point clouds to decrease memory usage and increase efficiency training. We train and test our network on real-world data sets. also validate our proposed framework for rotation and translation scenes. Experimental results show that approach consistently outper- forms existing state-of-the-ard approaches. The experiment of the trained network on unseen data sets verifies the generalizability and scalability of our proposed framework.
The main contributions of this work can be summarized follows:
We propose a new pipeline for place recognition using SE(3)-equivarian encoders to learn SE(3)-invariant descriptors with only geometric information from 3D point clouds. The proposed method is robust against arbitrary rotation and translation of robot poses. It is generalizable and scalable to unseen data sets, thereby removing the need for data augmen- tation.
2. We apply a self-attention mechanism downsample point clouds which maintains place recognition in high performance up to 50% downsampling rate.
3. The code open-sourced at https: /ttp::///itttain /gggttttbbbb.o///g equlvariant_placerreonniii
2
Related Work
We first review place recognition works that use geometric information from 3D point clouds. Then, present existing works that point cloud descriptors with rotation-invarian or translation- invariant properties. Later, we discuss existing group-equivariant networks for point clouds. At last, we provide a brief introduction to designing attention mechanisms to improve place recognition performance.
Geometry-Basee Place Recognition Previously, place recognition using 3D point clouds relied on histograms or hand-engineered features such Fast Histogram [8], M2DP [9] and Scan Con- text [10]. PointNetVLAD [11] is pioneering work to apply a learning- based feature extractor to place recognition tasks. combines local feature encoder PointNet [12] and global descriptor ag- gregator NetVLAD [13] to allow end-to-end representation training from a given 3D point cloud. Later, MinkLoc3D [14] presents sparse voxelized point cloud representation and sparse convolu- tions. Other al algorithms such as LPD-Net [15], OverlapNet [16], and LCDNet [17] provide different learning-based ways of constructing global descriptors for place recognition tasks. However, these algorithms are not robust to rotational and translational pose changes.

---

Exploiting Symmetry in Place Recognition. Considering that the observer may be in different orientations or locations, researchers propose some rotation-invarian hand-crafted descriptors such as histograms of range in LiDAR scan [18], Scan Context [10, 19], frequency-doma Scan Con- text [20], LiDAR-Iris [21], and height coding descriptor [22] to perform place recognition. While these hand-crafted features are rotation-invare
some structural information is ignored when com- posing them. Later, deep learning features are widely used since their performances surpass hand-
crafted features [23].
Only a few works try to encode traaton--ivaarian or translation-invariant features into learning- based place recognition algorithms. PointNetVLAD [11] and LCDNet [17] try to increase robust- ness by randomly rotating imput point clouds during training. To achieve yaw-angle-invariang
Over- lapNet [16] and EverlapTransformg [24] use range images and Lu et al. [25] propose RING descriptor. To achieve rotation-invariane RINet [5] exploits additional semantic information with ro- tation equivariant convolution, SeqSphere [26] uses spherical convolution in spherical view, and RPR-Net [27] exploits rotation-invariant convolution. Nevertheless, these strategies do not con- sider both 3D rotation and translation differences in the pose, thus might not be sufficient in more challenging scenarios.
Networks for 3D Point Clouds. While a few works take rotation-equivariant and mnnnslation-qqivaarinn properties into consideration in place recognition tasks, a series of works design network architectures with the equivariance property for general feature learning. For ex- ample, Spherical CNNs [28] and Vector Neuron [29] enable -quiivaqiiaaiian representation learn- ing. Equivariant Point Network (EPN) [30] performs SE(3) separable convolution, which separates 6D convolution into convolutions in the 3D Euclidean space and in SO(3). EPPN [31] proposes a lightweight variant of SE(3) SEu3)-equivariant network for point clouds. These networks generally ad- dress the group-equivariant feature learning problem in classification and segmentation tasks. They are only tested with point clouds in single object shapes but have not been tested extensively on 3D point clouds in real-world outdoor scenes. We propose a new pipeline for place recognition that exploits symmetry via group-equivariant networks. Since EPN and E can encode SE(3)- equivariant information of point clouds, we adopt these two networks. The proposed framework is the first attempt to develop SE(3)-qqivarrint place recognition framework to bridge the gap between the group-equee eouu-equuvariant and place recognition literature.
Attention Mechanism in Place Recognition. An attention mechanism has been applied to some place recognition tasks learn the contextual features. PCAN [7], SOE-Net [32], Retriever [33], and everlapTransforee [24] include this technique to learn the relations of different features. Among these applications, attention mechanisms are used to learn the importance local features. this work, we explore applying the attention mechanism to the input 3D points to learn the points' significance for downsampling.
Methodology
This section details our proposed framework for 3E(3)-invarian) place recognition using 3D point clouds. Figure presents an overview of the proposed approach. The framework consists of three parts: attentive downsampling, SE(3)-invariant local feature extraction, and SF(3)-invariant global descriptor generation. We will fully discuss each component the following subsections.
3.1 Attentive Downsampling
3D point clouds measured from LiDAR or RGB-D sensors may contain hundreds of thousands of points. To perform place recognition efficiently in neural networks, we exploit the attention mech- anism downsample point cloud measurements while preserving meaningful information. For a point cloud with N points P E R Nx3, we apply the multi- attention module [6] using the Py- Torch library [34] to learn the attention weights Watten E N33 from the input point cloud. Multi- head attention is defined as MultiHead X) = Concat(heady, headz, heady) W point does the features concatenation. The number of parallel attention heads is set as 3. Each attention head defined as head, = Attention X X XWWK, XW Here, we set query Q = XW? key K = XWK and value V = XWV, where X the input point cloud P to perform self-attention and learn the correlation between the input 3D points.
input
where (oncat(-)
3

---

With the attention weights, we summarize over the feature space to obtain point-wise atten- tion weights W pw--atten R., which represent the significance of each point. W pwr-atten Ei=12 Wii atten- where W atten RN is the attention weight in dimension We select top-k attention weights and keep the corresponding points p p== R2x3
3.2 Local Equii-Equivariant Features
Learning equivariant representation from point clouds can provide efficiency and generalizability in challenging robot perception tasks. Equivariance is form of symmetry for functions that preserve the transformation applied on the input to the output.
Equivariance generalizes the concept of invariance, which means that the output of functions is independent of the transformations applied to the input. Mathematically, a function fine X - Y invariant to a set of transformations T, if for any E fmm
fimo x), Va E
The general linear group of degree n, denoted by GL,,, (R), is the set of all n x n nonsingular real ma- trices, where the group binary operation is the ordinary matrix multiplication. The dimensiimn (3D) special orthogonal group, denoted by (3) = {R E GLy(RR RR = I3. det(R) = +1} is the rotation group RR3 where I3 denotes the 3 x 3 identity matrix. The 3D special Fuclidean group, denoted by SF(3) = {H = (R, t) R E SO(3), t E RR3) is the group of rigid transforma- tions, i.e., direct isometries IR3 [35].
In this work, we leverage Equivariant Point Network (EPN) [30] and EPPN [31] to learn the SE(3)- equivariant feature and capture the inherent symmetry of 3D point cloud data, In the original EPN [30], given a 3D point x, a rotation g, a feature representation function F I SO(3) + R.P and kernel h R x SO(3) + R.,, the discretized SE())-equivarian convolutional opera- tor is defined as the dot product between the translated and rotated kernel and the function F: (F* ()) = Ex.E Egjec F(a,, (,t:))))) ( (x - 4z).95 9), where and G are the discretized sets corresponding to IE' and SO(3), respectively. To reduce the computation cost in 6D convolu- tion, the authors separate the kernel into two smaller kernels representing SE(3) point convolution and SE(3) group convolution, respectively. This design preserves SE(3)vqquivarian features from the input point cloud while maintaining affordable computation.
We also experimented with E [31], lightweight and more efficient variant of EPN [30]. EPPN leverages quotient representations to embed equivarauve in a spherical feature space, result- ing in much fewer feature dimensions than EPN. Therefore, it drastically reduces memory consump- tion and runtime while preserving the rotational equivariance. Such property is highly relevant to our task since we work with large-scale point clouds in outdoor environments.
3.3 Local -E(3)-Invariant Feature Pooling
After learning from an SE(3)-equivariant encoder, we obtain SB(3)-equivariant features (c( NxCXR where P is the input point cloud, f.).) is the mapping from point cloud to SE(3)- equivariant features, N is the number of points, C is the number of local features, and R is the num- ber of rotation group discretization. We then apply the pooling method to extract SE(3)-invariant features. To avoid the group attentive pooling failing if the point cloud is circularly symmetric as discussed in [30], we propose to apply max-pooling on the rotational dimension for each spa- tial point to generate SE(3)-invariant features and increase the robustness for different shapes of point clouds. In the max-pooling step, we only keep the maximum feature from one of the dis- cretized rotation group elements. After max-pooling, the SE(3)-invariant feature is then represent as Jine () E R NxC The last part of the local feature extractor is a linear layer to map the SE(3)- invariant features the desired dimension. See Figure for an illustration.
3.4 Global Invariat Place Representation
Global descriptors are computed by aggregating local features using NetVLAD or Gener- alized Mean (GeM) [36]. NetVLAD learns cluster centers of VLAD (Vector of Locally Aggregated Deseriptors) in CNN framework. The output descriptors V are defined as clIorthe
V(j, k) =N eiettr (a+(i) (i)).. This equation shows j-th dimensions of the i-th

---

descriptor, where z: is the local feature. wk, bg, and Cj are trainable parameters to learn the center of cluster k.
GeM is a trainable pooling layer that generalizes max-pooling and mean-pooling. With local feature input the GeM pooling output is defined as SGeM [ Ex,ez x;) where is pooling
parameter that can be learned set manually.
To learn discriminative global descriptors, we use lazy quadruplet loss [[I] to train the network to perform place recognition tasks. For each iteration of training, there is an anchor point cloud Pa. "positive" point cloud P. that is similar to Pa. and a "negative" point cloud {Pr that are dissimilar to Pa. and random point cloud in the training set Pn. The lazy quadruplet loss is defined as Loss(Pa Pp, Pn, Pa-)) = max; ([a + 6p6n,1+) + max ([3 + 6pōng 1ng14) where a and / are constant values provide margin. The lazy quadruplet loss can minimize the L2 distance between the anchor and the positive representation 6p = d(. (PPp)) while maximizing the distance between the anchor and the negative representation ōng d(.(Pa). ((Pn;) Paş E (Px)-
4 Experimental Results and Discussion
We construct SE(3)-invariant place recognition descriptors using the described method. In this sec- tion, we examine the performance of place recognition, the SE(3)-invariant properties, and the de- sign of attentive downsampling.
Model Training. We train our networks on Oxford RobotCar [37] benchmark created by Uy and Lee [11]. Oxford benchmark contains 45 sequences of a vehicle taking measurements using SICK LMS- 151 2D LiDAR in similar routes for different times, days, and seasons. Each point cloud a submap of a prebuilt map. The ground points are removed, and the point clouds are normalized to be zero mean and rescaled to within the range of [-1, 1]. Training and testing sets are geographically split with ratio of 70 % and 30 %. For creating training tuples, ground truth location within 10 meters considered a positive sample, while location more than meters away is considered a negative sample. We train with 21,711 sub-maps. In EPN-NerVLAD, point clouds are first downsampled to 2048 points using the attention mechanism. For Squii-equivarian encoders (EPN and PN), we construct them with two layers of equivariant layers, one with 32 local features and one with 64 local features. The number of dimensions for the discrete rotation group is 60 in EPN and 12 in E?PN. The encoder is followed with max-pooling and linear layer map local features to 1024 dimensions. Then, we use NetVLAD or GeM to learn global descriptors with dimensions For GeM, we follow MinkLoc3D's structure and set the pooling parameter p 3. The network is trained for 30 epochs with a learning rate of 5 x 10-5 The set as a= 0.5, 3 0.2. The network parameters are optimized by ADAM [38].
256.
in lazy quadruplet loss
Note that we do not need random rotation during the training process since the network is designed to generate the same descriptor we rotate translate the point cloud. The decreased need for data augmentation is an advantage of the proposed framework.
Place Recognition Evaluation. In place recognition tasks, precision and recall are the two well- established evaluation metrics [39]. Precision = TP/(TP + FP) is the percentage of true loop closures among all the places we recognize. Recall = TP/ + FN) is the percentage of places we recognize among all true loop closures. TP the number of true- prue-positive cases, FP represents the number of false-positive cases, and FN stands for the number of false-positive cases. The F1 to obtain a balancing metric between
score is defined as Fl = eeppecisionnreeaat (precision precision and recall.
Place Recognition on Oxford benchmark. We first evaluate the performance of the proposed method on the Oxford benchmark. The Oxford RobotCar data set consists of data collected by similar route different times and seasons. Hence, every sequence revisits the path traveled by other sequences. When performing the evaluation, we generate the SE(3)- invariant global descriptor for each input point cloud. Then, we find the top top 25, and top 1% of candidates' matches similar to the query point cloud in each sequence. We calculate the precision and recall rate, then average among different query point clouds in different sequences. The average recall curve represents the model performance for the top 25 matches. With these evaluation metrics and seikit-learn library [40], we report precision-recall curves, F1-recall curves, and average recall curves of the proposed method and other state-of-the-an methods are shown
vehicles driving
5

---

Precision recall Curve
Fecrecal Curve
Average recall gN Curve
1.-
1.
3.--
Ivversen recall G0N
Contect
11Seure
Scan Cornect
Contect
M2/20
Mrrlac13
Recall
Recall
top
(a)
(b)
(c)
Figure 2: Place recognition evaluation on Oxford benchmark of the proposed methods and state-of-the-art
approaches MinkLoc3D [14), PoiD [11], M2DP [9], and Scan Context [10].
Table I: Experimental result showing the average recall (%) at top 1% and at top on Oxford and in- in-house benchmark. Scan Context and M2DP are non-learning methods. Three methods in the middle rows use NetVLAD global pooling method. The last two methods in the bottom rows use GeM a global pooling methad. is with attentive downsampling.
Oxfond
U.S.
R.A.
B.D.
@ 1%
AR
AR@
AR@1
AR@ 1%
AR@
AR 1%
AR@1
hand-crafted
Scan Context [10]
32.91
75.96
53.69
44.57
M2DP [9]
34.69
23.14
45.03
32.41
44.62
34.34
39,34
32.95
Poin
84.94
71.39
80.79
65.33
73.86
61.83
69.29
61.78
NetVLAD-based
EPN NeVLAA+
89.17
77.69
87.82
74.03
81.98
70.09
76.91
69. 69.14
NeIVIAD
91.83
82.46
88.04
84.73
80.58
80.93
79.84
79.31
GeM-based
Min [14]
97.91
93.76
95.04
86.01
91.19
81.17
88.48
82.66
E'PN-GeM
93.24
84.79
95.29
88.08
90.46
83.67
87.68
83.29
in Figure 2. All learning- methods are trained on the same Oxford benchmark training set. However, MinkLoc3D is trained with more efficient training strategy. Scan Context [10] and M2DP [9] construct band-engineered features to perform place recognition. The result from Scan Context and M2DP can be further improved with hyper-parameter tuning. The figure shows that the proposed networks E'PN-NetVLAD and EPN-NetVLAD outperform PointNetV all of which share the same global feature extraction method. MinkLoc3D and E?PN-Ged both use GeM pooling for global feature extraction. Though MinkLoc3D performs the best among all methods, E*PN-GeM and EePN-NetVLAD still perform consistently within 5% difference.
Place Recognition on in-house benchmark. To show the generalizability of the proposed method, we also evaluate all methods on in-house data sets [11], which are constructed from Velodyne-64 LiDAR scans in three kinds of regions that are unseen to the network, including the university sector (U.S.), residential area (R.A.), and business district (B.D.). Table 1 shows the average recall at top 1% and at top for each method on Oxford and in-house benchmark. Our method performs better than others for networks with NetVLAD global feature extraction regardless of selecting several or only one loop closure candidates. Our method achieves the best performance among all the data sets we did not train on. For methods that use GeM pooling, MinkLoc3D performs better on Oxford but similarly on U.S., R.A., and B.D. compared to the proposed P2PN-GeM method, which shows the generalizability among data sets of the proposed method.
KITTI benchmark, We also evaluate the proposed methods on KITTI odometry data set [41]. Different from data in Oxford, 3D point clouds in the KITTI data set are collected by Velodyne HDL-64E, randomly downsampled to 4096 points, and the points of ground are not removed. The point clouds are scaled to [-1, 1] with zero mean. Sequence 00 is chosen for evaluation because of its large set of pairs for loop closure in the same orientation. While sequence 08 is chosen for evaluation because it contains 100% reverse loop closure. Reverse loop closures provide scenes with 180-degree viewing angle differences and advance a more challenging scenario. For sequence 00, the first 70 seconds construct the reference database, and the remaining part of the sequence is used as test queries. For sequence 08, the first 85 seconds and segment from 259 to 264 seconds construct the reference database, and the rest of the sequence is used as test queries. We ignore two nearby frames to avoid matching consecutive scans falsely. Table 2 reports the average recall at the top

---

Table 2: Average recall (%) top 1% on KITTI se- quence 0D and 08. * with attentive downsampling.
Table 3: Experimental result showing the average recall
(%) at top 1% of EPN-NetVLAD when the input point cloud is downsampled with different percentages and different methods on Oxford henchmark.
KITTI Sequence 00 AR
KITTI Sequeace 08 AR 1%
AR@I
EPN-NaVVLAD* (Ours)
Puin [1I]
78.21
73.18
61.90
70.68
63.84
32.47
37.69
17.61
Numher of Points
4096
3000
2048
1600
1024
E*PN-NetVLAD (Ours)
84.96
71.67
71.38
57.23
Downsampling Rate
0%
27 %
50%
61%
75 %
Random Downsampling
71.66
63.34
57.29
53.19
43.17
MinLLo33D [14]
28.07
4.01
17.30
3.50
Attentive Downsampling
71.66
71.65
71.05
66.22
E-PN-GeM (Ours)
86.22
77.94
71.70
59.75
Table 4:
Data augmentation experiment on Oxford benchmark with different training data size and whether random transformation is applied during training. is with attentive downsampling.
Random Transformation
Training Size: Sequences
Training Size:
Sequences
AR 1%
AR@
AR 1%
AR
PoiD [II]
69.38
54,00
86.88
73.12
PointNetVLAD [II]
V
80.85
65,55
84.94
71.39
EPN-NetVLAD (Ours)
75.15
89.17
77.69
E E'PN-NaVLAD (Ours)
85.16
70.61
93.78
85.04
MinkLoc3D [14]
97.91
93.76
E'PN-GeM (Ours)
88.49
76.73
94.76
87:45
1% and top 1 for place recognition in sequence 00 and 08. All methods are trained using the same Oxford training data set. The table shows that the SE(3)-invariant property in the proposed methods helps them perform better in these challenging scenarios, supporting the better generalization claim.
Attentive Downsampling. We design experiment to test the performance of point cloud down- sampling using an attention mechanism. We study EENNN's performance with random and attentive downsampling methods. In this experiment, the network is constructed with only one layer of EPN with 64 local features and trained on three sequences of the Oxford data set to simplify the task. The test set includes point clouds in 23 sequences of the Oxford benchmark. The results of dif- ferent downsamplings are presented in Table 3. It shows that an attention mechanism to downsample point clouds can maintain high place recognition performance up 50% downsampling rate.
Data Augmentation Experiment. In Table 4, we experiment with different amounts of training data in the Oxford benchmark. PointNetVLLDD relies on both random transformation and increasing training data size to achieve high performance, whereas E'PN-NetVL.n can achieve similar per- formance with only training on three sequences. MinkLoc3D performs the best among all methods. However, still requires random transformations in the training data.
Descriptors with SE(3)-Invarian Property. In addition to the place recognition experiments on Oxford and in-house benchmark, we take one point cloud in the Oxford benchmark, and construct simulated data by manually rotating and translating the point cloud to test the model performance with severe rotation and translation. We visualize the local features and global descriptors when the input point cloud is transformed under rotation, translation, rotated, then translated, and translated, then rotated. Figure shows the results and the cosine similarity score between each transformed feature/descriptor and the original feature/descriptor. We can see that even if the point cloud is rotated or translated, the output features and descriptors remain the same and has 100 % similarity to the original one.
Oxford Experiment with Random SE(3) Transformations. We augment point clouds from the Oxford data set with random SE(3) transformations to create a more challenging place recognition experiment. The point clouds are transformed under purely SO(3))rotation, purely 3D translation, and with both rotation and translation. With original point clouds in range between [-1, 1]. 3D rotations are applied randomly (uniformly sampled between -π and along all three axes) using SciPy library [42], and 3D translations are applied with standard deviation of 1.0. We use the model trained on 45 sequences of the Oxford benchmark to perform place recognition on this augmented data set which includes a total of 1320 point clouds that has 440 query point clouds, each of which has two positive pairs. Table shows the average recall rate. We observe that with the attention downsampling module, the performance of EPN-NerVILD decreases. The attentive downsampling module selects different points to feed into the network, whereas other methods compute over the same points in this case.
7

---

Pard CoNat
Features
Global Descriptors
wiated
tranputed
ptsted then transiated
trerakied than reteted
orginal
vctated then trandateed
IEGR0 OVOR DONG 1003
rotaned
tranalited the retested
transie rocabed
gondbated
index
Index
(a) point clouds under transformations
(b) SE(3 local features (first 100)
(c) SE(3)-invariant global features
Figure 3: Visualization of one input point cloud. the same point cloud under transformations, their correspond- ing local features, and the output global descriptors. There are 1024 local features each point in the point cloud. We only show the first 100 features of the first point for this visualization.
Table 5: Average recall (%) at top 1 % for performing place recognition task on simulated data where the point clouds are transformed under rotation or/and transla- tion. with attentive downsampling.
Table 6: Run time performance per point cloud which contains 4096 points. The run times are computed with- any network optimization. attentive downsam- pling performed to reduce point cloud size to 2048 points.
Rotation and Trunslation
Method
Rotation
Trunslation
Parameters
Run Time
LLDD
6.60
3.21
2,96
MinkLoc3D
13.71
PointNerVT.AD [11]
19.779,145
0.006
EPN-Net
30.13
40.69
27.55
MinkLoc3D [14]
1,055,713
0.005
EPNANeVVLAD
49.62
100.00
$1.19
EPN-NetVLAD (Ours)
17.135.376
2,052
E'PN.NeVVLAD
71.38
E'PN-Ne (Ours)
17,167.488
E2PN-GeM
75.97
100.00
73.91
EP GeM (Ours)
192,513
0.082
Run Time Performance. We tested our method on a system equipped with an Intel 19-10900K CPU and an Nvidia GeForce RTX 3090. For obtaining one global descriptor from one 3D point cloud with 4096 points, we record the number of parameters in the network and report the run time performance per point cloud in Table 6. E'PN-GeM has the lowest number of parameters. MinkLoc3D has the shortest inference time. Changing the global descriptor extraction method from NetVLAD to GeM drastically decrease the number of parameters but does not affect the run time substantially. We conjecture that the higher run times of SE(3) -quiivraqva networks are caused by the lack of network optimization. EPN and E:PN are coded with eustom functions to perform group convolutions, while other networks have network structures optimized on GPU. Thus, it is possible that SEuii-equivarian networks can be further optimized to improve run time.
Limitation
The major limitation of the proposed framework is the relatively slow time and the need for optimized libraries to perform real-time place recognition. However, with the development of more powerful computing hardware, we expect this limitation be largely resolved in the near future. In addition, the study of equivariant encoders under other Lie groups to enable invariance to, e.g., scale and deformation is an interesting future direction that we did not discuss in this paper.
6 Conclusion
We have designed a place recognition framework that exploits SE(3)- representation learning. In particular, SE(3)-invariant features learned from 3D point clouds improve robustness to large transformations and generalizability in place recognition tasks. In addition, we propose using an attention mechanism in place recognition to downsample the input point cloud while maintaining high performance. Our experimental results on real-world data sets show the proposed method per- forms well in various metrics. Future work includes a lightweight design of the equivariant encoder for real-time onboard applications and the extension of this framework to stereo cameras where image data can also be incorporated into the learned representation.
8

---

Acknowledgments
Toyota Research Institute provided funds to support this work. Funding for M. Ghaffari was in part provided by NSF Award No. 2118818.
References
[1] H. Johannsson, M. Kaess, M. Fallon, and J. Leonard. Temporally scalable visual SLAM using a reduced pose graph. In 2013 IEEE International Conference on Robotics and Automation, pages 54-61, IEEE, 2013.
[2] C. Häne, L. Heng, G. H. Lee, F. Fraundorfer, P. Furgale, T. Sattler, and M. Pollefeys. 3D visual percep- tion for self-driving cars using multi-camera system: Calibration, mapping, localization, and obstacle detection. Image and Vision Computing. 68:14 2017.
[3] Sanfourche, V. Vittori, G. Le Besnerais. eVO: A realtime embedded stereo odometry for MAV applications. In Proceedings of the JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 110072211 14, IEEE, 2013.
[4] X. Zhang, L. Wang, and Y. Su. Visual place recognition: A survey from deep learning perspective. Pattern Recognition, 1 113:107760 2021.
[5] L. Li, X. Kong, X. Zhao, T. Huang, W. Li, F. Wen, H. Zhang, and Y. Liu. RINet: Efficient 3D lidar-based place recognition using rotation invariant neural network. IEEE Robotics and Automation Letters, 7(2): 4321 +321-4328, 2022.
[6] A. Vaswani, Shazeer, N. Parmar, J. Uszkoreit, L. Jones, N. Gomez, Kaiser, and I. Polosukhin. Attention you need. Advances in neural information processing systems, 30, 2017.
[7] Zhang and Xiao. PCAN: 3D attention map learning using contextual information for point cloud based retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 12436-12445, 2019.
[8] T. Röhling. J. Mack. and D. Schulz. A fast histogram- based similarity measure for detecting loop closures in 3-D lidar data. In 2015 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 736-741. IEEE, 2015.
[9] L. He, X. Wang, and H. Zhang. M2dp: A novel 3D point cloud descriptor and its application in loop closure detection. 2016 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages
IEEE, 2016.
[10] G. Kim and A. Kim. Scan context: Egocentric spatial descriptor for place recognition within 3D point cloud map. In 2018 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4802 4809. IEEE, 2018
[11] M. A. Uy and G. H. Lee. PointNetVI.AD: Deep point cloud based retrieval for large-scale place recog- nition. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pages 4470 4479, 2018.
[12] C. Qi. Su, Mo, and Guibas. Pointnet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.
[13] R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic. NetVLAD: CNN architecture for weakly supervised place recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 5297-5307, 2016.
[14] J. Komorowski. Minkloc3d: JEEE/CVF Winter Conference on Applications of Computer Vision, pages 1790 1799, 2021.
Point cloud based large-scale place recognition. In Proceedings of the
[15] Z. Liu, S. Zhou, C. Suo, P. Yin, W. Chen, H. Wang. H. Li, and Y.H.. Liu. Lpd-net: 3D point cloud learning for large-scale place recognition and environment analysis. Proceedings of the IEEE/CVE International Conference on Computer Vision, pages 2831-2840, 2019.
[16] X. Chen, T. Läbe, A. Milioto, T. Röhling, O. Vysotska, A. Haag J. Behley, C. Stachniss, and F. Fraun- hofer. OverlapNet: Loop closing for LiDAR based SLAM. In Proceedings of the Robotics: Science and Systems Conference, 2020.

---

[17] D. Cattaneo, M. Vaghi, and A. Valada. LCDNet: Deep loop closure detection for LiDAR SLAM based unbalanced optimal transport. arXiv preprint arXiv:2133.00066. 2021.
[18] H. Yin, Y. Wang. X. Ding, L. Tang. S. Huang, and R. Xiong. 3D based global localization using
siamese neural network. IEEE Transactions Intelligent Transportation Systems, 21(4):1380-1322 2019.
[19] G. Kim, S. Choi, and A. Kim. Scan context++: Structural place recognition robust rotation and lateral variations in urban environments. IEEE Transactions on Robotics, 2021.
[20] Y. Fan, X. Du, and J. Shen. Fresco: Frequency-domain scan context for lidar-based place recognition with translation and rotation invariance, arXiv preprint arXiv:2206. 12628, 2022.
[21] Y. Wang. 7. Sun, C.-7. Xu, S. Sarma, J. Yang, and H. Kong. Lidar iris for loop-closure detection. arXiv preprint 1xXiv:1912.03882 2019.
[22] D. Xu, J. Liu, J. Hyyppä, Y. Liang, and W. Tao. A heterogeneous 3D map-based place recognition solution using virtual lidar and polar grid height coding image descriptor. ISPRS Journal of Photogrammetry and Remote Sensing. 183:1 18, 2022.
[23] C. Masone and B. Caputo. A survey deep visual place recognition. IEEE Access, 9:19516-19547, 2021.
[24] J. Ma, J. Zhang, J. Xu, R. Ai, W. Gu, and X. Chen. OvertapTransffommr::: An efficient and yaw-angle- invariant transformer network for LiDAR-hased place recognition. IEEE Robotics and Automation Let- ters, 2022.
[25] S. Lu, X. Xu, H. Yin, Xiong, and Y. Wang. One ring rule them all: Radon sinogram for place recognition, orientation and translation estimation. arXiv preprint arXiv:2204. 07992, 2022.
[26] P. Yin, F. Wang, A. Egorov, J. Hou, J. Zhang, and H. Choset. Segspherevlad: Sequence matching enhanced invariant place recognition. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5024-5029 5029. IEEE, 2020.
[27] 7.. Fan, 7. Song, W. Zhang, H. Liu, He, and X. Du. Rpr-net: point cloud-based rotation-aware large scale place recognition network. arxiv preprint 2008. 12790, 2022.
[28] C. Esteves, C. Allen-Blanccettee A. Makadia, and K. Daniilidis. Learning SO(3) equivariant representa- tions with spherical cnns. Proceedings of the European Conference on Computer Vision, pages 52-68, 2018.
[29] C. Deng, O. Litany, Y. Duan, Poulenard, A. Tagliasacchi, and L. Guibas. Vector neurons: A gen- eral framework for -qqivvraua networks. Proceedings of the IEEE International Conference on Computer Vision, 2021.
[30] H. Chen, S. Liu, W. Chen, H. Li, and R. Hill. Equivariant point network for 3D point cloud analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattenn Recognition, pages 14514-14523, 2021.
[31] M. Zhu, M. Ghaffari, W. A. Clark, and H. Peng. EPPN: Efficient SE(3)-equivariant point network. arXiv preprint 2XX::2206.05398, 2022.
[32] Y. Xia, Y. Xu, S. Li, R. Wang. J. Du, D. Cremers, and U. Stilla. SOE-Net: A self-attention and orientation encoding network for point cloud based place recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 11348 -1357, 2021.
[33] L. Wiesmann, R. Marcuzzi, C. Stachniss, and J. Behley. Retriever: Point cloud retrieval in compressed 3D maps. In Proceedings of the IEEE International Conference on Robotics and Automation, 2022.
[34] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan. T. Killeen, Z. Lin. N. Gimelshein, I. Antiga, A. Desmaison, A. Kopf, E. Yang, 7. DeVito, M. Raison, A. Tejani, S. Chil- amkurthy, B. Steiner, I.. Fang, J. Bai, and S. Chintala. Pytorch: An imperative style, high- performance deep learning library. H. Wallach, H. Larcchelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc., 2019. URL http://papers neurips. wttppppee/cpppp/////pppppppp.pet 9015-pytorch-and anntorch-an-imperr wtweerraiiee-stlle--i---------ee pdf.
[35] T. D. Barfoot. State Estimation for Robotics. Cambridge University Press, 2017.
10

---

[36] F. Radenović. G. Tolias, and O. Chum. Fine-tuning can image retrieval with no human annotation. IEEE transactions on pattern analysis and machine intelligence, 41(7):1655-6668 2018.
[37] W. Maddern, G. Pascne, C. Linegar, and P. Newman. 1 year, 1000 km: The Oxford robotcar dataset. The International Journal of Robotics Research, 36(1):3-15 2017.
[38] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[39] S. Lowry, N. Sünderhauff P. Newman, J. J. Leonard, D. Cox, P. Corke, and M. J. Milford. Visual place recognition: A survey. IEEE Trunsactions Robotics, 32(1):1-19, 2015.
[40] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer,
R. Weiss, V. Dubourg, Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning Python. Journal of Machine Learning Research, 12:2825 22222252800, 2011.
[41] A. Geiger, P. Lenz, and R. Urtasun. Are ready for autonomous driving? the kitti vision benchmark suite. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2012.
[42] P. Virtanen, R. Gommers, T. Oliphant, M. Haherland, T. Reddy, D. Cournapeau, F. Burovski, P. Peter- son, W. Weckesser, Bright, Nelson, E. Jones, R. Kern, E. Larson, C. Carey, 1. Polat, Feng. E. W. Moore, J. VanderPlas, D. Lax- alde, Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing Python. Nature Methods, 17:261-272. 2020. doi:10. 1038//41592--09 -19-0666-----..
van der Walt, M. Brett, Wilson, K. J. Millman, Mayorov, A.


Reviews:
[{"review_id": "QnFdeDyQYV4O", "reviewer": "Reviewer_GJ8U", "summary": "This paper addresses the problem of location matching with LiDAR point clouds, and proposes EPN-NetVALD using SE(3)-invariant global features. It consists of three major components: 1. Self-attention based point cloud downsampler; 2. Equivariant Point Network for point cloud equivariant feature extraction; 3. NetVLAD, a global feature extractor.  EPN extractor uses two separate kernels to perform SE(3) point convolution and SE(3) group convolution and generates SE(3)-equivariant local features which are aggregated into SE(3)-invariant local features by pooling. NetVLAD then clusters the local features into a single global descriptor by finding the center of local features.  The paper demonstrates that the proposed method outperforms the current STOA method on Oxford RobotCar dataset. ", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1.  The idea of introducing SE(3) equivariant learning into point cloud and scene recognization is novel and interesting. Given an unchanged scene, the changes introduced by different viewing angle can be considered as a rotation and translation transformation of the scene.  Which can be captured by a SE(3) invariant descriptor, and the invariance considerably reduces the task difficulty for the model as rotation and translation mapping don't need to be learned during training. \n2. The ablation studies for self-attention downsampler vs random downsampler and rotation and translation augmented dataset for testing SE(3)-invariance are good, as they clearly demonstrate the contribution of downsampler and SE(3)-invariant descriptors towards overall model performance.\n\nWeakness:\n1. The way to extract SE(3) invariant features are questionable, since only the invariant features are used for global feature learning as well as place matching, the necessity of introducing SE(3) equivariant network is unclear, especially when EPN is the bottleneck of the overall network runtime. Instead, for example in SE(3) transformer, only the 0 type spherical harmonic needs to be pooled to extract the invariant features. \n2. For place recognization for an unchanged scene,  changing the viewing angle not only introduces rotation and translation but also changes the observed areas. Which is not discussed in this paper. \n3. In the method section, the extraction of invariant features is unclear, what is the rotational dimension and what is the neighborhood the max pooling is acted on? ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "This paper addresses the problem of location matching with LiDAR point clouds, and proposes EPN-NetVALD using SE(3)-invariant global features. The idea of introducing SE(3) equvariant learning into point cloud and scene recognization is novel and interesting, as the viewing angle changes can be considered as observing a rotated and translated scene. This paper also demonstrates EPN-NetVALD STOA performance and has a clear ablation study of contribution of model's separate components. \n\nHowever, although SE(3) equivariant and invariant learning is the main idea for this paper, it doesn't go into the details of SE(3) theory and how each such Lie group can be used for equivariant and invariant feature extraction. Additionally, since EPN-NetVALD's local feature extractor is built on EPN with minor modifications (only the pooling layer), the technical details of this part is unclear. Given NetVALD is also well established by previous methods, combining two networks into one pipeline makes me concerned bout this paper's novelty.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "1. Since this paper is all about SE(3) equivariant and invariant, the related work and method section needs to go into more detail about the SE(3) Lie group.\n2. For place recognization for an unchanged scene,  changing the viewing angle not only introduces rotation and translation but also changes the observed areas. Which is not discussed in this paper. \n3. In the method section, the extraction of invariant features is unclear, what is the rotational dimension and what is the neighborhood the max pooling is acted on? "}}, {"review_id": "PepqHOCGAH2", "reviewer": "Reviewer_kN1U", "summary": "The paper presents an equivariant SE3 transformations based architecture (building on existing EPN model) and NetVLAD global pooling (existing work) with quadruplet margin based contrastive loss (existing work). Standard benchmark datasets are used to show improved results against existing methods.\nThe main contribution of the paper is the combination of the above mentioned components to achieve invariant global features. Secondly, the proposed attentive downsampling is shown to be more robust than random downsampling of the input data.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- The main strength of the paper is the demonstration that combining the existing components, that is, equivariant network + group pool, global pool and quadruplet loss, can lead to SE3 transformation robust global features. \n- The attentive downsampling is shown to be superior to random downsampling.\n- The paper is well written and easy to understand.\n\nWeaknesses and areas of improvement are listed below:\n\n**Claims**\n\nThe contribution claim 1 and 2 overlap in discussing SE3 invariance. Also, claim 2 needs to be justified for scalability (what does it mean here?). Moreover, isn’t generalizability a must for machine learning based methods, should this really be a contribution?\n\n**Method**\n\nSection 3.1 is somewhat misleading. If Q,K,V,P are all the same, then there remains no learnable weight because by definition attention A = Q.K, and P_out = A.V, where typically Q,K,V are all learnable linear transformations of P. The authors seem to have made a direct use of Pytorch library functions without reflecting their understanding of what is being done under the name of ‘attention’, which in this case could have been heavily simplified if there are no learnable parameters and if Q,K,V,P are all the same.\n\nSection 3.2 explains an existing method with modifications listed in L172-174. The authors mention that they replace group pooling with “max-pooling in rotation dimension for each spatial point”. So, what exactly is different here as the original group pooling would also have been applied in the rotational dimension. Is switching to max-pool operation better than what was originally used? \n\n\n**Results and Analyses**\n\n*Benchmarking*:\nL88 critiques existing works arguing about their lack of robustness against translation and rotation but this has not been extensively demonstrated through benchmarking.\n\nThe authors did not compare against MinkLoc3D and other recent methods (see https://github.com/jac99/MinkLoc3D). PointNetVLAD’s results have been outperformed by a significant margin now. Also, MLP in PointNetVLAD and 3D sparse convolution in MinkLoc3D carry different inductive biases in training, where the former is understood to be more likely to fail under 3D transformations. \n\n*Rotation vs Translation*:\nAs mentioned in the previous comment, convolution based learning such as that in MinkLoc3D should be able to handle 3D translation. The authors should consider presenting results separately for translation and rotation on top of the combined results.\n\n*Global Pooling*:\nVLAD vs GeM is another interesting question as VLAD can be more resource consuming and is more complicated than GeM.\n\n*Simple baseline*:\nSince the ultimate goal of the work is to achieve invariance to SE3 transformations, the authors should compare against a naive baseline where an equivalent standard CNN (MinkLoc3D?) is used but with significant training data augmentation (similar to the test set referred to in Section 4.3). This will highlight whether or not achieving invariance via equivariance is necessary or not.\n\n*Discretized Rotation Angles*:\nEquivariant networks operate on a discrete domain, which means that performance may be low for the rotation angles that were not considered during training. The authors should show performance against rotation angles and mention such limitations if existing.\n\n**Minor**\n\n- L29, many 3d place recognition methods already exist, as mentioned in Section 2\n- L35, what “transformation changes”\n- L46, local features are defined on and represent 3D points and their neighborhoods, L46-47 need to provide a more clear argument\n- SeqSphereVLAD [a] is a closely related work which the authors can consider reviewing.\n\n[a] Yin, Peng, et al. \"Seqspherevlad: Sequence matching enhanced orientation-invariant place recognition.\" 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Benchmarking is not strong in this paper and it seems that some of the recent methods will likely perform much better than the baselines considered in this work. Furthermore, an important simple baseline with trainind data augmentation might render computationally-expensive equivariant networks useless. Attentive downsampling, being one of the novel contributions, is not described well. Combination of existing components to come up with the proposed method is good engineering advance but the paper lacks any insights into aspects like inductive biases of different architectures (MLP, 3D convolutions, equivariant networks), rotation vs translation, and importantly, comparison against different training strategy such as training data augmentation.\n\nPost-rebuttal:\nThe authors have done an amazing job at addressing all the concerns so raised. This includes clarifications on claims and methodology and more importantly, a range of new experiments which has significantly improved the paper.\n\nThe overall results on different benchmark datasets seem to be mixed, some in favor of the proposed method and others against (compared to MinkLoc3D). However, Table 4 has perhaps the strongest results in the authors' favor, which puts the inherent (and desired) characteristics of the proposed method at the forefront, that is, robustness to rotations.\n\nThe authors could perhaps have also included rotation transformations on datasets other than the simulated one and KITTI to show an expected performance drop for MinkLoc3D, e.g. by simulating an opposite direction travel.\n\nEven though the paper builds on existing works, it currently boasts a well executed set of experiments that highlight the limitations of existing works and datasets while proposing an alternative approach tuned to viewpoint/orientation equivariance/invariance characteristics, thus, an improved rating.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "As already summarized above, additional experimental analyses and improved description of the method (as pointed above) will be critical to improve the rating of this paper.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {}}, {"review_id": "8Ey04yNA8GS", "reviewer": "Reviewer_xfgJ", "summary": "This paper presents EPN-NetVLAD, a method for 3D point cloud based place recognition using SE(3)-Invariant features.\n\nThe key design of EPN-NetVLAD is that it leverages the Equivariant Point Network (EPN) proposed in [13] to learn local SE(3)-equivariant features, and then uses a global pooling to obtain SE(3)-invariant features.\n\nExperiments show that EPN-NetVLAD outperforms existing methods that do not consider SE(3)-invariance, especially when the point clouds undergo large rotation and translation.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Leveraging SE(3)-equivariant features for more robust place recognition\n- Paper is easy to read, with clear motivation and intuition\n\nWeakness\n- I feel there is not much technical contribution in this paper. It is basically plugging the SE(3)-equivariant network into place recognition (even though this paper is the first to do so, I dont think this is a surprising idea and result). ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I personally think more technical meat is needed to meet my calibration of an accepted paper in CoRL.\nHowever, I would not strongly object if this paper is accepted. Overall a good paper making some contribution to the field.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "N/A", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}, {"review_id": "4e7RdygxQN_", "reviewer": "Reviewer_cMpG", "summary": "This work proposed a place recognition method with robust pose invariance. It uses Equivariant Point Network (EPN) to learn SE(3)-equivariant features, which are further processed by NetVLAD to describe the place. Additionally, attentive downsampling is used to select representative points for improved computational efficiency. The training is based on the RobotCar dataset, and the evaluation uses standard metrics (PR and F1) for place recognition.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The paper is well written and easy to follow, the literature review is thorough, the methodology is clear and sound, and the experimental results are trustable.\n\n**Weaknesses**\n- The novelty of the method is slightly weak in the reviewer's mind. It seems like the system is a combination of three existing methods with a couple of minor modifications. Nevertheless, the combined system achieves outstanding results.\n- The highlight of the method is high pose invariance for place recognition; however, the evaluation is mainly based on the RobotCar dataset, which is a driving dataset without significant viewing angle variation. It will be more persuasive if the authors include more challenging datasets to verify the pose invariance of the proposed method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "It is a well-written paper. The proposed method is clear and sound, although it slightly lacks of novelty. The evaluation is trustable, but it will be more persuasive if they include datasets with significant viewing-angle change.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "- In 3.2, what is the motivation for replacing the group attentive pooling in EPN with max-pooling? Is it because of the attentive downsampling used in the first step?\n- During the training process, do the authors also randomly rotate the input point cloud during training as PointNetVLAD and LCDNet do? Otherwise, how is the viewing angle invariance achieved?\n- For the results in Table 1, what does the evaluation setup look like? How challenging (e.g., summer-winter place recognition) is it? \n- In 4.5, the authors mentioned that the run-time is reduced from 2.05 seconds to 0.53 seconds when reducing discrete rotation groups from 60 to 20, how does the accuracy drop in this case?"}}]

Author rebuttal:

Decision: Accept (poster)