Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the problem of image retrieval and proposes a method that tries to learn better representations through the use of local feature attention and reduces memory and latency overheads by not requiring re-ranking with local features and only using global one.

For this, the paper proposes a module called Local Attention Learning Module (LALM) that performs both spatial- and channel-wise attention on local features and that it's plugged between two consecutive blocks of the backbone architecture. The output of this module is then concatenated to the input of the next block. The rationale behind this is that the following blocks will act as an homography transformation of the local features and will produce a better global representation. The method also uses intermediate supervision by computing a cross-entropy loss on top of the output features produced by the LAL Module.
main_review: Strengths:
- The paper proposes an interesting approach that obtains very good results using only global features, outperforming two-stage methods that use local features for re-ranking.
- Experimental evaluation is also complete and thorough, with a very extensive ablative study.

-----
Weaknesses:
- The main weakness of the paper it has to do with its presentation: it has numerous syntax and grammatical errors and typos, which makes for a challenging read sometimes. Some paragraphs are difficult to follow and in my opinion should be rewritten. The main problem with this is that it affects the motivation of the paper: I feel I couldn't fully grasp the rationale behind some design choices or the conclusions drawn from some of the experiments. It is difficult to provide a complete list of all the things that should be addressed by the authors but I'll discuss in section "Presentation" below what I think are the most importante ones. Nevertheless, I highly encourage the authors to do another pass on the writing.
- Slightly related to the point above, I'm missing a deeper and more detailed explanation about the claim that the last convolutional blocks act as an homography transformation because there's no experimental or theoretical study that proves that in the text. Overall, the contributions of the paper are not clear enough.
- Not too important: I understand that the focus of the paper is removing the re-ranking step, but it would have been interesting to see if the local features produced by the LALM module could have been used for this purpose to further improve the results in applications that can afford the extra memory and latency costs.
- I wondered why the authors decided to use average-pooling and cross-entropy loss for the intermediate supervision instead of GeM or ArcFace as for the global features. Any insight about why this shouldn't work better?
- Apologies if I missed this in the text, but how are R50-UGALR* and R50-UGALR in Table 2 are different? I couldn't find a reference in the text to the dimensionality reduction mentioned here.

Presentation:
- The caption of Figure 1 only half describes the diagram there so I would suggest to extend it to give a better overview of the method. It is also missing the legend/explanation of what is "A", "X", "M", and "C".
- "At present, there is such a problem in image retrieval: although the results of global feature-based retrieval are correct in the category, they are not sorted in detail" -> technically, the problem in instance-retrieval datasets is that they're not correct in the category so that's why re-ranking is performed.
- "But if we don’t differentiate the importance of attention and features, it’s hard to make sure that the learned attention is not part of features, which is not an ideal way for attention learning." -> this sentence should be re-written
- "However, these methods have not been well modified yet to solve image retrieval problems." -> this sentence should be re-written
- "in the next small section." -> in the next section
- The nomenclature used in Figure 2 does not match with the text nor with the formulas in Eq 1-4
- In page 4 the text mentions "D", which I'm assuming it's the output of "X" in Figure 1, but it has never been defined before.
- The previous to last paragraph in page 4 is difficult to follow and it should be re-written more clearly. This is a crucial paragraph to properly understand the rationale behind the LALM module and the intermediate supervision and in its current shapes it loses impact.
- "We assume that there is an identical average target available for every category of objects" -> I couldn't understand this assumption
- Eq 6 makes reference of La but it was never defined in the text before (only in figure 1)
- "Obviously, UGALR completely outperforms previous global features based methods" -> "As we can see, UGALR..."
- "The most amazing two improvements we made are" -> "The largest two improvements..."
- "Amazingly, our approach using only global features outperforms" -> "Surprisingly, our approach..."
- Spaces are usually missing just before parenthesis
summary_of_the_review: Overall, even though it doesn't introduce any impactful contribution, I think that the paper proposes an interesting method that is able to obtain very good results using only global features, outperforming methods that also combine re-ranking with local features. However, I find it difficult to recommend this paper for its acceptance in its current shape since its presentation is not adequate. Introduction and Method sections are sometimes difficult to follow and I couldn't fully grasp some of the ideas and rationale behind the design choices. Another pass in the text is required, and figures and formulas should also be updated and improved.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: In this work, the authors propose a Unifying Global and Attention-based Local Features Retrieval method (referred to as UGALR). UGALR accelerates extraction speed and reduces memory consumption by removing the re-ranking process and learning local feature matching with convolutional neural networks instead of the RANSAC algorithm. In addition, UGALR learns more accurate and semantic local information by combining spatial and channel attention with intermediate supervision.
main_review: The proposed method is feasible. However, its novelty is pretty limited. Also, many points are confusing for readers.
summary_of_the_review: Due to the limited contributions, I do not recommend this paper to publish on ICRL.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new pipeline architecture for image retrieval. The main purpose of the proposed architecture is to overcome the time and space consumption due to the images' re-ranking that exploits local features (after the search using global features). 
In order to do that, UGALR is proposed as a single-stage pipeline able to combine global and local features to speed up the retrieval process, removing the re-ranking step. This is achieved using a CNN able to learn the homography transformation in local feature matching.
Moreover, the architecture combines spatial and channel attention with the aid of intermediate supervision, obtaining better performances than the SOTA models presented in the analysis of the results. This is achieved with the introduction of a Location Attention Learning Module (called LALM).

main_review: Strengths:
* the proposed architecture is deeply described, justifying each of its parts;
* the experiments are performed using two datasets and the proposed model is compared with a lot of other models, providing a serious and robust discussion;
* the ablation analysis is complete and justifies the use of each component of the architecture, describing numerically why each macro-layer (e.g. LALM) helps to improve general performances;
* all the results against the SOTA models are complete and well described;
* one of the main (well-described) goals of the architecture is to speed up the retrieval process reducing the used space. A comparison of these aspects has been clearly done, demonstrating the great advantages of UGALR with respect to the other models.
* the benefits in terms of time and space do not preclude an improvement in accuracy performance, which is higher than in the baseline models.

Weaknesses:
* even if complete, the description of the architecture seems a little bit confusing. For example, the use of capital letters in figure 1 (and then in the text)  is confusing;
* the comparison in terms of time and memory seems ok but is not very clear if all the models were tested on the same GPU (using the same environment). If not, this is a real problem and the results cannot be taken into account;
* there is no error analysis. The accuracy is higher with respect to baseline models, but not that much; in order to study why, an error analysis would be helpful, maybe providing a comparison with the common errors of the other SOTA models.

Minor grammar errors:
* in figure 2 you use $\varphi$ (varphi) but in the text $\phi$ (phi) is always used;
* lack some spaces between words and brackets (but this could be a template prerogative)

summary_of_the_review: The paper describes the proposed architecture proving a complete explanation of all of its parts (with the help of a complete ablation analysis). The comparison with a lot SOTA models on two datasets confirms the robustness of the proposed model. Even if the improvements in terms of accuracy are not so high (not a negative point, there is however a clear improvement) there is a huge improvement in terms of speed and memory used. An improvement for the paper would be the addition of a complete error analysis that now is not present and a better description of the entire architecture that, at present, is not very clear. Moreover, is not clearly specified if all the models have been tested on the same GPU (by the authors) to test speed and memory usage, this is an important point to specify because is one of the main goals of the paper.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: The paper proposes a new method to combine global and local image features, targeted at image retrieval applications. It designs a new local feature model branch where both spatial and channel attention are used. The local feature branch undergoes supervision directly (in the paper called “intermediate supervision”), and this branch’s output is also concatenated to the global feature branch’s output in order to produce a final image embedding at the end. 
main_review: Strengths:
S1) Combining local and global image features is a promising research direction.
S2) The paper provides several experiments, with many ablations.
S3) Some performance improvements seem to be observed compared to previous work.

Weaknesses:
W1) The authors naively claim several times in the paper that their method is learning homography transformations (Introduction, Methodology, Experiments, Conclusions). However, one can clearly see that this is not the case. The authors are simply concatenating local feature maps with the ones from the global feature branch and supervising by ArcFace classification loss. The authors seem to be assuming that homography learning is implicitly happening, which is naive. Similarly, the abstract claims that the proposed method is “learning local feature matching with convolutional neural networks instead of RANSAC algorithm” – this is incorrect.
W2) The paper is not well written, and several parts are hard to understand. For example, in Fig 1 the paper has blocks named A, X, M, C – what do they refer to? Similarly, in page 4 the text start referring to “D” (“D is then followed (...)”), but D had never been introduced before. D (and O, which is also later introduced) should be added to Fig 1, and A/X/M/C should be explained in the caption/text. It is also very difficult to understand the way that local and global feature maps are combined – the paragraph “Backbone Network” and the one before it are quite confusing.
W3) The authors seem to have suspiciously excluded results from previous work. For example, Tab 1 does not show numbers for “R50-DELG (global+local)” in the “+1M” dataset setups, which are available in the exact same paper where the other numbers from the table were taken from. I suspect that they were not included because in half of the cases the proposed method underperforms compared to “R50-DELG (global+local)” – specifically, the ROxf+1M cases.
W4) In Tab 2, the authors simply take latency numbers reported in previous work and compare to their own for the new method, using a different GPU. The comparison does not seem appropriate.

summary_of_the_review: The paper addresses an important technical direction of fusing local and global image features. However, it has several major flaws, such as: claiming learnable matching/homography when none of that is happening, suspiciously excluding previous work’s performance numbers in some cases; poor writing.
review_title: null

Meta-review: The paper proposes a new method to combine global and local image features, targeted at image retrieval applications. The main idea is a model branch where both spatial and channel attention are used. The local feature branch undergoes supervision directly and this branch’s output is concatenated to the global feature branch’s output in order to eventually produce the final image embedding. 

The reviewers appreciated the care in the evaluation (ablative analysis) and the promise of the approach compared to existing baselines. The reviewers also expressed concerns about several claims, for instance that the proposed approach is able to learn homography transformations, the quality of the exposition, and missing baselines. The reviewers also pointed out that several parts of the paper were hard to follow and important details were missing. 

The authors submitted responses to the reviewers' comments. After reading the response, updating the reviews, and discussion, the reviewers considered that ‘the concatenation of local features with global ones works does not mean at all that some geometric transformation is learned’ and the justification provided for omitting baselines (suggested by the reviewers) were unconvincing. The feedback provided was already fruitful, yet major issues still remain.

We encourage the paper to pursue their approach further taking into account the reviewers' comments, encouragements, and suggestions. The detailed feedback lays out a clear path to generate a stronger submission to a future venue.

Reject.
Decision: Reject