Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Towards the issue that existing methods try to excavate the multiple correlations in different aspects, this paper provides a unified perspective to learn the multiple correlations, which enables the information propagation along features of different samples within each mini-batch. The proposed MetricFormer introduces a correlation-aware transformer into the internal structure of deep metric networks. It can be ﬂexibly integrated with the existing metric learning framework. Experiments on three widely-used datasets demonstrate its superiority.
strengths_and_weaknesses: Pros:

(1) The idea of transformer-based multiple correlations exploration is interesting.

(2) The method is simple and effective. Experimental results are good.

Cons:

(1) The writing could be improved. There are several grammar mistakes and typos. For example, "a elaborate" should be "an elaborate" in line 6. The order of "Samples" and "Classes" are reversed in Table 1. The abbreviation of "PA" and "M" should be explained in Table 2.

(2) The novelty might be insufficient. The contribution of existing correlations combination  is not significant.

(3) Please analyze the computational complexity of the proposed transformer based correlation computation and compare it with existing methods.

(4) The reference is inadaquate. [1] also investigates multiple correlations for different tasks. Please also discuss the difference.

(5) Will the codes be released?

[1] Deep Comprehensive Correlation Mining for Image Clustering, ICCV 2019
questions: Please refer to my detailed comments above.
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors improved DML by exploring multiple informative relationships among different samples and features from a unified perspective. Current DML methods can only excavate the multiple correlations in different aspects. Considering this problem, they proposed an elaborate metric transformer named MetricFormer to effectively capture and model the multiple correlations. The proposed is generic, effective, and can be easily incorporated into many existing DML methods.
strengths_and_weaknesses: [Strengths]
- This is a well-written paper. The motivation is clear and the method is easy to follow. By introducing the well-designed transformer blocks, the proposed method can effectively learn more representative features.

- The authors studied DML from a new and unified perspective of correlation exploration. Previous works excavate the multiple correlations in different aspects, while the authors consider the multiple correlations from a unified perspective by the proposed MetricFormer.  The proposed method is generic. It can be widely applied to strengthen existing DML approaches since it can excavate multiple correlations in a unified perspective.

- Comprehensive experiments are provided to understand and evaluate the proposed methods.

[Weaknesses]
- Although the authors mentioned in their limitations, I think they still should report and analyze their model size and inference cost in their experimental results.

- The effectiveness of the Local Embedding Graph should be discussed in the ablation study.


questions: - More descriptions should be added to the figures in this paper. For example, The number N of the used Batch-wise Correlation Block should be highlighted in Figure 2.
limitations: Yes, limitations have been discussed. I do not find the potential negative societal impact of this work.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a Transformer-based method named MetricFormer for metric learning. The proposed MetricFormer includes three blocks: feature decoupling block, feature-wise correlation block, and batch-wise correlation block. Given an image, feature decoupling block uses several trainable queries and cross-attention to extract different sub-features to focus on different spatial regions. Feature-wise correlation block uses self-attention to model the relations among different sub-features of a sample, while batch-wise correlation block uses self-attention to model the relations among different samples in a mini-batch.
strengths_and_weaknesses: Strengths:
- The authors conduct sufficient experiments to validate each component of their method and the results on three benchmarks achieve competitive results compared with state-of-the-art methods.

Weaknesses:
- The biggest weakness lies in the novelty. Specifically, feature decoupling block and diversity loss are consistent with [1]; feature-wise correlation block is similar to [1,2,3,4]; batch-wise correlation block is similar to [5];  Consistency Preserving Strategy is similar to [6]. All these methods have been verified in person re-id. The authors just combine them in a framework, so the novelty is limited.
- The presentation should be improved greatly. For example, it is preferred to use consistent format to list the references.

[1] Diverse Part Discovery: Occluded Person Re-Identification With Part-Aware Transformer. In CVPR, 2021.

[2] Spatial-temporal graph convolutional network for video-based person re-identification. In CVPR, 2020.

[3] Learning multi-granular hypergraphs for video-based person re-identification. In CVPR, 2020.

[4] Multi-granularity reference-aided attentive feature aggregation for video-based person reidentification. In CVPR, 2020.

[5] Spectral Feature Transformation for Person Re-Identification. In ICCV, 2019.

[6] Masked Graph Attention Network for Person Re-identification. In CVPR workshop, 2019.

questions: As pointed out in the "Weaknesses" part, what is the most novel point in the proposed method?
limitations: The authors have pointed out the limitation and negative impact at the end of the paper, but not provided strategies to address them. I have no suggestions for improvement to the authors either.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a method called MetricFormer, which can extract an ensemble of distinct features, model the relationship among samples in a minibatch, and discover the structural pattern in the ensemble of features. The experimental results on three public datasets are better compared with other approaches. 
strengths_and_weaknesses: Strengths:
1. The Metric transformer includes three modules: (1) feature decoupling module:  extracting an ensemble of district features; (2) batch-wise correlation block: modeling the sample relationship in each minibatch; (3) feature-wise correlation block: discovering the structural relations among the ensemble of features.
2. The proposed method results are promising in three public datasets, compared with other approaches. An ablation study is performed to show the effectiveness of each module.

Weaknesses:

The presentation of this paper needs to be improved. In equation (3), there is no relation between the two equations. Y^{-}_ should be included in the second equation.   Is the dimension or variable D at line 128 the same as the variable C at line 122? What do \alpha and \beta mean in equation (9)? What do MetricFormer-M and MetricFormer-PA mean in Table 2?  
questions: Please see my questions in the weakness section. 
limitations: The equations and variables need to be well defined and add more explanations.
review_title: null

Meta-review: metareview: This paper proposes a unified framework based on the transformer model for capturing multiple relationships among data and features in similarity or metric learning. It is an interesting direction that is worth exploring, and the special way to combine multiple transformer blocks may have more general relevance. The proposed framework can be used with many existing metric learning methods. The research contribution of this paper is significant and reports good experimental results. Nevertheless, as pointed out by several reviewers, the presentation of the paper has room for improvement. The authors are recommended to put more efforts to improve its presentation. Besides, revision should also be made to address the other comments and suggestions of the reviewers, which includes explaining the novelty of this work more clearly.

confidence: Certain
Decision: Accept