Paper: Rotation-Equivariance and Position Encodings for Enhancing Local Descriptors (ICLR-2024)

Reviewer comment (Novelty):
In the first paragraph of Section 1, you give examples of image matching, 3D reconstruction, and object tracking, but the actual paper you cite is a Roxf/par paper on image retrieval. Additional citations of related papers are needed.

Author response:
Thank you for your valuable feedback.  
We addressed issues related to image retrieval literature.  
We have supplemented ablation experiments for the proposed feature fusion method, demonstrating that to achieve the best results, it is crucial to retain local rotational equivariance of feature maps while obtaining global position information. A smooth integration process in feature fusion is essential for optimal performance.  
We are still actively conducting additional experiments and making further revisions to the paper. Thank you very much for your valuable feedback.

Reviewer comment (Experiments):
The whole framework is a litte complicated. The authors do not explain the necessity of each module clearly, neither perform abation study to show each module's effect.

Author response:
Thank you for your valuable feedback. We have conducted additional ablation experiments and comparative experiments on runtime.

We performed ablation experiments on the proposed feature fusion method, indicating that for optimal results, it is crucial to retain the local rotational equivariance of feature maps while obtaining global position information and ensuring a smooth integration process in feature fusion.

Reviewer comment (Evaluation):
The efficiency of descriptior extraction is also important for some application, but the authors do not show the running time of the proposed method in the experiments.

Author response:
Thank you for your valuable feedback. We have conducted additional ablation experiments and comparative experiments on runtime.

We calculated the average runtime for feature point extraction and matching of a pair of images on the rotated-hpatches dataset: AWDesc: 0.3106 seconds, ours: 0.4785 seconds. The experiments demonstrate that our method exhibits high real-time performance.

Reviewer comment (Writing):
In the description of figure2, all "are fed" is written as "are feed".

Author response:
We have corrected a spelling error in Figure 2.

Reviewer comment (Evaluation):
Could the authors compare the running time of the proposed method and other descriptors?

Author response:
Thank you for your valuable feedback. We have conducted additional ablation experiments and comparative experiments on runtime.

We calculated the average runtime for feature point extraction and matching of a pair of images on the rotated-hpatches dataset: AWDesc: 0.3106 seconds, ours: 0.4785 seconds. The experiments demonstrate that our method exhibits high real-time performance.

Reviewer comment (Evaluation):
Despite I understand the idea behind the rotation equivariance descriptor design, the paper hardly justifies how the performance improves the state-of-the-art.

Author response:
Thank you for your valuable feedback. We have conducted additional ablation experiments on the proposed feature fusion method, indicating that for optimal results, it is crucial to retain the local rotational equivariance of feature maps while obtaining global position information and ensuring a smooth integration process in feature fusion.

Reviewer comment (Experiments):
The authors claim "If we were to combine local rotation equivariant descriptors with positional information, it wouldn’t necessarily achieve complete rotational equivariance in theory" while referring AWDesc Wang et al. (2023), SuperGlue Sarlin et al. (2020), LoFTR Sun et al. (2021). In the experimental section, only AWDesc is truly compared while SuperGlue and LoFTR are the widely accepted baselines nowadays.

Author response:
Our method falls into the category of feature point descriptors, similar to SuperPoint and AWDesc, while SuperGlue and LoFTR are considered matchers. When evaluating descriptor performance, we often use the default RANSAC method in OpenCV for camera relative pose estimation on datasets. Therefore, methods like MAGSAC, MAGSAC++, AdaLAM, and PROSAC are not within the scope of feature point descriptor research.

Reviewer comment (Experiments):
There is no comparison regarding whether the proposed descriptor improves any classical matcher such as RANSAC variants (e.g. MAGSAC, MAGSAC++, AdaLAM, PROSAC...) or learnable matchers such as SuperGlue, ACNe, or NG-RANSAC.

Author response:
Our method falls into the category of feature point descriptors, similar to SuperPoint and AWDesc, while SuperGlue and LoFTR are considered matchers. When evaluating descriptor performance, we often use the default RANSAC method in OpenCV for camera relative pose estimation on datasets. Therefore, methods like MAGSAC, MAGSAC++, AdaLAM, and PROSAC are not within the scope of feature point descriptor research.

Reviewer comment (Experiments):
There is also no clear discussion nor proof of how this improves against dense matching methods such as LoFTR.

Author response:
Methods like LoFTR, which focus on dense matching, follow a detector-free end-to-end matching architecture where feature point descriptors are not explicitly extracted, making them less flexible for downstream applications. In our track, which emphasizes feature point descriptors, the extracted feature points are explicit and flexible.

Reviewer comment (Experiments):
Many SOTA descriptors are not being directly compared such as Superpoint, D2-Net, R2D2, etc.

Author response:
Regarding the rotation-equivariant capability of methods like Superpoint, D2-Net, and R2D2, which is mentioned in DISK and RELF papers, we compare our method with AWDesc under normal conditions without significant rotation and with RELF under substantial rotation. We can supplement experiments for Superpoint, D2-Net, and R2D2 in the appendix.

Reviewer comment (Evaluation):
The proposed method is losing to 2011 methods such as BRISK in Table 1, which is surprising.

Author response:
Traditional methods like BRISK strike a balance between speed and performance, while KAZE is a robust method for rotation-equivariant, similar to SIFT. We chose KAZE for experimental convenience (due to SIFT copyright).

Reviewer comment (Evaluation):
Why is the proposed method weaker than most 2011–2012 methods in Table 1 if it truly solves the rotation equivariance problem? The reviewer finds the discussion of the planar dataset not acceptable.

Author response:
Traditional methods like BRISK strike a balance between speed and performance, while KAZE is a robust method for rotation-equivariant, similar to SIFT. We chose KAZE for experimental convenience (due to SIFT copyright).

Reviewer comment (Reproducibility):
How exactly is this descriptor learning used with modern matchers (classical or deep learning)?

Author response:
We appreciate your suggestion to conduct experiments using the combination of 'ours' and 'SuperGlue,' but in the track of feature point descriptors, this approach may not be the most direct and effective experiment.

Reviewer comment (Experiments):
Lack of ablation and analysis. The paper introduces two major improvements to an existing local descriptor extraction algorithm; however, it is surprising to not see an ablation on the delta introduced by each change. Especially since there are already baseline methods using group equivariance convolution as well, it could easily lead readers to think the extra improvement is coming from global positional embedding, which arguably could be injected not as part of the local descriptor, but during the matching process. While a thorough ablation requires more work, for this type of paper where multiple features are introduced, it would be extremely valuable to cover that.

Author response:
Your valuable feedback has been immensely helpful to us. We have conducted additional ablation experiments on the proposed feature fusion method, demonstrating that preserving the local rotational equivariance of feature maps while obtaining global position information and ensuring a smooth integration process in feature fusion is essential for achieving optimal results.

Reviewer comment (Presentation):
Presentation. The authors should focus more on the main contribution of the paper (with more text and figures). The current version contains a lot of introductory-level content, e.g., on what is a feature pyramid network or what is multi-head attention, which is acceptable when publishing to a general audience not working in the Computer Vision field, but could be redundant (and diminish the authors' contribution) for an ICLR submission. In addition, there are some typos and grammar issues worth fixing.

Author response:
We have carefully revised the presentation of the article, removing some fundamental concepts that occupied significant space.

Reviewer comment (Experiments):
As mentioned above, an ablation on the two major features introduced in this paper on some example dataset should be included, and the content should be kept more focused to allow room for this ablation.

Author response:
Your valuable feedback has been immensely helpful to us. We have conducted additional ablation experiments on the proposed feature fusion method, demonstrating that preserving the local rotational equivariance of feature maps while obtaining global position information and ensuring a smooth integration process in feature fusion is essential for achieving optimal results.