Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents EPN-NetVLAD, a method for 3D point cloud based place recognition using SE(3)-Invariant features.

The key design of EPN-NetVLAD is that it leverages the Equivariant Point Network (EPN) proposed in [13] to learn local SE(3)-equivariant features, and then uses a global pooling to obtain SE(3)-invariant features.

Experiments show that EPN-NetVLAD outperforms existing methods that do not consider SE(3)-invariance, especially when the point clouds undergo large rotation and translation.
strengths_and_weaknesses: Strengths:
- Leveraging SE(3)-equivariant features for more robust place recognition
- Paper is easy to read, with clear motivation and intuition

Weakness
- I feel there is not much technical contribution in this paper. It is basically plugging the SE(3)-equivariant network into place recognition (even though this paper is the first to do so, I dont think this is a surprising idea and result). 
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I personally think more technical meat is needed to meet my calibration of an accepted paper in CoRL.
However, I would not strongly object if this paper is accepted. Overall a good paper making some contribution to the field.
issues: N/A
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper addresses the problem of location matching with LiDAR point clouds, and proposes EPN-NetVALD using SE(3)-invariant global features. It consists of three major components: 1. Self-attention based point cloud downsampler; 2. Equivariant Point Network for point cloud equivariant feature extraction; 3. NetVLAD, a global feature extractor.  EPN extractor uses two separate kernels to perform SE(3) point convolution and SE(3) group convolution and generates SE(3)-equivariant local features which are aggregated into SE(3)-invariant local features by pooling. NetVLAD then clusters the local features into a single global descriptor by finding the center of local features.  The paper demonstrates that the proposed method outperforms the current STOA method on Oxford RobotCar dataset. 
strengths_and_weaknesses: Strengths:
1.  The idea of introducing SE(3) equivariant learning into point cloud and scene recognization is novel and interesting. Given an unchanged scene, the changes introduced by different viewing angle can be considered as a rotation and translation transformation of the scene.  Which can be captured by a SE(3) invariant descriptor, and the invariance considerably reduces the task difficulty for the model as rotation and translation mapping don't need to be learned during training. 
2. The ablation studies for self-attention downsampler vs random downsampler and rotation and translation augmented dataset for testing SE(3)-invariance are good, as they clearly demonstrate the contribution of downsampler and SE(3)-invariant descriptors towards overall model performance.

Weakness:
1. The way to extract SE(3) invariant features are questionable, since only the invariant features are used for global feature learning as well as place matching, the necessity of introducing SE(3) equivariant network is unclear, especially when EPN is the bottleneck of the overall network runtime. Instead, for example in SE(3) transformer, only the 0 type spherical harmonic needs to be pooled to extract the invariant features. 
2. For place recognization for an unchanged scene,  changing the viewing angle not only introduces rotation and translation but also changes the observed areas. Which is not discussed in this paper. 
3. In the method section, the extraction of invariant features is unclear, what is the rotational dimension and what is the neighborhood the max pooling is acted on? 
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: This paper addresses the problem of location matching with LiDAR point clouds, and proposes EPN-NetVALD using SE(3)-invariant global features. The idea of introducing SE(3) equvariant learning into point cloud and scene recognization is novel and interesting, as the viewing angle changes can be considered as observing a rotated and translated scene. This paper also demonstrates EPN-NetVALD STOA performance and has a clear ablation study of contribution of model's separate components. 

However, although SE(3) equivariant and invariant learning is the main idea for this paper, it doesn't go into the details of SE(3) theory and how each such Lie group can be used for equivariant and invariant feature extraction. Additionally, since EPN-NetVALD's local feature extractor is built on EPN with minor modifications (only the pooling layer), the technical details of this part is unclear. Given NetVALD is also well established by previous methods, combining two networks into one pipeline makes me concerned bout this paper's novelty.
issues: 1. Since this paper is all about SE(3) equivariant and invariant, the related work and method section needs to go into more detail about the SE(3) Lie group.
2. For place recognization for an unchanged scene,  changing the viewing angle not only introduces rotation and translation but also changes the observed areas. Which is not discussed in this paper. 
3. In the method section, the extraction of invariant features is unclear, what is the rotational dimension and what is the neighborhood the max pooling is acted on?
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper presents an equivariant SE3 transformations based architecture (building on existing EPN model) and NetVLAD global pooling (existing work) with quadruplet margin based contrastive loss (existing work). Standard benchmark datasets are used to show improved results against existing methods.
The main contribution of the paper is the combination of the above mentioned components to achieve invariant global features. Secondly, the proposed attentive downsampling is shown to be more robust than random downsampling of the input data.
strengths_and_weaknesses: - The main strength of the paper is the demonstration that combining the existing components, that is, equivariant network + group pool, global pool and quadruplet loss, can lead to SE3 transformation robust global features. 
- The attentive downsampling is shown to be superior to random downsampling.
- The paper is well written and easy to understand.

Weaknesses and areas of improvement are listed below:

**Claims**

The contribution claim 1 and 2 overlap in discussing SE3 invariance. Also, claim 2 needs to be justified for scalability (what does it mean here?). Moreover, isn’t generalizability a must for machine learning based methods, should this really be a contribution?

**Method**

Section 3.1 is somewhat misleading. If Q,K,V,P are all the same, then there remains no learnable weight because by definition attention A = Q.K, and P_out = A.V, where typically Q,K,V are all learnable linear transformations of P. The authors seem to have made a direct use of Pytorch library functions without reflecting their understanding of what is being done under the name of ‘attention’, which in this case could have been heavily simplified if there are no learnable parameters and if Q,K,V,P are all the same.

Section 3.2 explains an existing method with modifications listed in L172-174. The authors mention that they replace group pooling with “max-pooling in rotation dimension for each spatial point”. So, what exactly is different here as the original group pooling would also have been applied in the rotational dimension. Is switching to max-pool operation better than what was originally used? 


**Results and Analyses**

*Benchmarking*:
L88 critiques existing works arguing about their lack of robustness against translation and rotation but this has not been extensively demonstrated through benchmarking.

The authors did not compare against MinkLoc3D and other recent methods (see https://github.com/jac99/MinkLoc3D). PointNetVLAD’s results have been outperformed by a significant margin now. Also, MLP in PointNetVLAD and 3D sparse convolution in MinkLoc3D carry different inductive biases in training, where the former is understood to be more likely to fail under 3D transformations. 

*Rotation vs Translation*:
As mentioned in the previous comment, convolution based learning such as that in MinkLoc3D should be able to handle 3D translation. The authors should consider presenting results separately for translation and rotation on top of the combined results.

*Global Pooling*:
VLAD vs GeM is another interesting question as VLAD can be more resource consuming and is more complicated than GeM.

*Simple baseline*:
Since the ultimate goal of the work is to achieve invariance to SE3 transformations, the authors should compare against a naive baseline where an equivalent standard CNN (MinkLoc3D?) is used but with significant training data augmentation (similar to the test set referred to in Section 4.3). This will highlight whether or not achieving invariance via equivariance is necessary or not.

*Discretized Rotation Angles*:
Equivariant networks operate on a discrete domain, which means that performance may be low for the rotation angles that were not considered during training. The authors should show performance against rotation angles and mention such limitations if existing.

**Minor**

- L29, many 3d place recognition methods already exist, as mentioned in Section 2
- L35, what “transformation changes”
- L46, local features are defined on and represent 3D points and their neighborhoods, L46-47 need to provide a more clear argument
- SeqSphereVLAD [a] is a closely related work which the authors can consider reviewing.

[a] Yin, Peng, et al. "Seqspherevlad: Sequence matching enhanced orientation-invariant place recognition." 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020.

robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Benchmarking is not strong in this paper and it seems that some of the recent methods will likely perform much better than the baselines considered in this work. Furthermore, an important simple baseline with trainind data augmentation might render computationally-expensive equivariant networks useless. Attentive downsampling, being one of the novel contributions, is not described well. Combination of existing components to come up with the proposed method is good engineering advance but the paper lacks any insights into aspects like inductive biases of different architectures (MLP, 3D convolutions, equivariant networks), rotation vs translation, and importantly, comparison against different training strategy such as training data augmentation.

Post-rebuttal:
The authors have done an amazing job at addressing all the concerns so raised. This includes clarifications on claims and methodology and more importantly, a range of new experiments which has significantly improved the paper.

The overall results on different benchmark datasets seem to be mixed, some in favor of the proposed method and others against (compared to MinkLoc3D). However, Table 4 has perhaps the strongest results in the authors' favor, which puts the inherent (and desired) characteristics of the proposed method at the forefront, that is, robustness to rotations.

The authors could perhaps have also included rotation transformations on datasets other than the simulated one and KITTI to show an expected performance drop for MinkLoc3D, e.g. by simulating an opposite direction travel.

Even though the paper builds on existing works, it currently boasts a well executed set of experiments that highlight the limitations of existing works and datasets while proposing an alternative approach tuned to viewpoint/orientation equivariance/invariance characteristics, thus, an improved rating.
issues: As already summarized above, additional experimental analyses and improved description of the method (as pointed above) will be critical to improve the rating of this paper.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work proposed a place recognition method with robust pose invariance. It uses Equivariant Point Network (EPN) to learn SE(3)-equivariant features, which are further processed by NetVLAD to describe the place. Additionally, attentive downsampling is used to select representative points for improved computational efficiency. The training is based on the RobotCar dataset, and the evaluation uses standard metrics (PR and F1) for place recognition.
strengths_and_weaknesses: **Strengths**
- The paper is well written and easy to follow, the literature review is thorough, the methodology is clear and sound, and the experimental results are trustable.

**Weaknesses**
- The novelty of the method is slightly weak in the reviewer's mind. It seems like the system is a combination of three existing methods with a couple of minor modifications. Nevertheless, the combined system achieves outstanding results.
- The highlight of the method is high pose invariance for place recognition; however, the evaluation is mainly based on the RobotCar dataset, which is a driving dataset without significant viewing angle variation. It will be more persuasive if the authors include more challenging datasets to verify the pose invariance of the proposed method.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: It is a well-written paper. The proposed method is clear and sound, although it slightly lacks of novelty. The evaluation is trustable, but it will be more persuasive if they include datasets with significant viewing-angle change.
issues: - In 3.2, what is the motivation for replacing the group attentive pooling in EPN with max-pooling? Is it because of the attentive downsampling used in the first step?
- During the training process, do the authors also randomly rotate the input point cloud during training as PointNetVLAD and LCDNet do? Otherwise, how is the viewing angle invariance achieved?
- For the results in Table 1, what does the evaluation setup look like? How challenging (e.g., summer-winter place recognition) is it? 
- In 4.5, the authors mentioned that the run-time is reduced from 2.05 seconds to 0.53 seconds when reducing discrete rotation groups from 60 to 20, how does the accuracy drop in this case?
review_title: null

Meta-review: metareview: Main strengths:
1. Introducing SE(3) equivariance into 3D descriptor learning is a nice contribution.

Main weaknesses:
1. Novelty needs to be further justified.
2. Core premise of the method needs further explanations, particularly on whether the method is practical in real-life applications that will experience different observable areas/occlusions when the viewing poses change.
3. Several technical details need to be further clarified.
4. Experiments/benchmarking are not fully convincing.

The clarifications on methodology and new results provided in the rebuttal have addressed the major concerns raised by the reviewers, thus an accepted outcome is justified.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)