Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper applies Transformer to the matching problem and metric learning. The direct applications of ViT and the vanilla Transformer, and two solutions adapted specifically for matching images through attention. A new simplified decoder is proposed for efficient image matching, focusing on similarity computation and mapping. 
main_review: 
Originality: Middle 

(+) This paper is to compare and extend the prevalent ViT[6] and DeTR[2], which contain both self-attention and cross (gallery-probe)-attention, in the person re-id. 

(-) There are many works that use self-attention for person re-id. Such existing works are not mentioned at all. For example, 

[A] Z.Zhang et al., Relation-Aware Global Attention for Person Re-identification, CVPR2020.
[B] R.Hou et al., Interaction-and-Aggregation Network for Person Re-identification, CVP2019. 

(-) There are several cross-attention works on person re-id; approach which maximizes the similarity of local features of gallery and probe pair. Such existing works are not explained. For example, 

[C] S.Zhao et al., Do Not Disturb Me: Person Re-identification Under the Interference of Other Pedestrians, ECCV2020 


Quality: Middle

(+) Direct application of Transformer and naïve application of Transformer to image matching problem are compared. 

(+) The accuracies are better, and the training time is shorter than the compared naïve application of Transformer. 

(+) The proposed method shows better rank-1 rates and mAP than the results reported on state-of-the-art domain generalized person re-id works. 

(-)  Details of baseline architecture are not clear.  
- ViT is an image classification model. Is the same binary cross-entropy loss in GS sampler applied for all methods? Is the same MLP head applied as the proposed model?   
- For Transformer-Cross, what is the value $V$ in the decoder in Eq.(1) ? Why are the Resnet query features directly used (line155) without encoders, unlike the proposed model?

(-) The performance of Transformer-Cat in Table 2 is lower than vanilla Transformer. How about appending the results of the Transformer in the lower block?

(-) The proposed method applies many improvements to the Transformer. For example, 
- The proposed method uses the shared FC parameter to query and key (line-174-175). 
- Multi-scale similarity scores are summed in the proposed model as in Eq.(8). 

However, Table 2 compares only with the final model. Thus, we can not see how much each component improved the naive application of the Transformer. We can see the ablation study, but improvement to the naïve application of Transformer is more understandable by adding the new component to Transformer-Cross. 

(+) This paper shows that replacing the first encoder directly with the ResNet feature map improves the results while being more efficient (line 241-242 and A.2). 

(-) Why is only the first encoder removed? Does the replaced place  and the number of replaced encoders affect performance? 


Clarity: Middle

(+)  The paper is easy to understand. 

(-)  What loss function is used should be made clear. GS sampler seems to be the negative sampling method and not loss function. 

(-)  It would be better to clarify MLPHead_1 and MLPHead_2 in Figure1. 

Line 289-290, “We start with Dot Product + GMP + MLPHead_1”. MLPHead_1 seems to be MLPHead_2.  

When MLPHead_2 is not used, the output dimension seems to be D. How can we output the score?  
When MLPHead_1 is not used, the input dimension to FC3 seems to be changed to hw.  


(-) For the ablation study, the author used their own measure mACC, which is averages of rank-1 rates and mAP on different datasets and runs. It is not clear if the average of different measures is possible. Also, we cannot compare the results of Table2 and Table3 because it is not used in Table2. 

--
Significance: Middle

(+) The Transformer is a timely topic. ViT[6] and DeTR[2] are prevalent in computer vision. 

(+) This paper clearly outperforms the state-of-the-art performance on domain generalized person re-id. 

(-) The theoretical contribution is limited. This paper has made only engineering to improve the rank-1 rates and mAP and computational time.
  

limitations_and_societal_impact: The authors discussed the social concerns of person re-id.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper studies person re-identification using visual transformers with query and gallery interactions. The paper presents first simple query and gallery image concatenation and cross attention layer for interaction query and gallery interactions; and later presents the proposed TransMatcher architecture, which replaces the decoder self-attention by only keeping query-key matching followed by global max pooling and multi-layer perceptron head maps composed of fully connected and batch norm layers.

main_review: The analysis on the standard transformer for detection is interesting. I further appreciated that it presents the naive extensions that do not lead to improvement along with the proposed model that leads to improvement.

The visual transformers is an active area of research and most of the very recent (a few months) papers are also cited in the related work. 
The experimental setup contains comparisons between the proposed approach and several earlier studies on the re-identification task. However, it would be further interesting to see how the proposed model performs compared to DeTR which is known for object detection on the re-identification task.

It is not very clear how the global max pooling is symmetrized, could you please provide the equations of this part and the concatenation afterwards, what is the dimension of the output? 

How are the prior score embeddings obtained? It is not clear why it is used, what is the rationale behind it?  

Sharing the code will increase the reproducibility of results. Could you please share your implementation?


---
Line 170, 171, 173, 177 $\mathbb{R}$ instead of $R$ for the dimensions

limitations_and_societal_impact: It mentions the limitations and possible societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper investigates how to apply the transformer architecture to solve the image match problem of generalised person Re-ID. The features of query/gallery images are first extracted with a ResNet model, then a transformer encoder without positional embedding is applied to query/gallery features, respectively. A customised decoder is designed to obtain the final match score for the following pairwise metric learning. The experiments are conducted on several common Re-ID datasets and the proposed method have shown better performance than the baseline.
main_review: STRENGTH
+ The performance on CUHK03/Market-1501/MSMT17 is superior to the baseline methods.
+ The idea of applying the transformer model to person Re-ID seems to be new.
+ The paper writing is easy to follow.

WEAKNESS
- There are other attention modules, e.g. no-local or channel-wise attention, why particularly use the transformer if no positional embedding is required?
- The proposed pipeline is still a naive solution. TThe transformer encoder is simply used to refine the ResNet features, while the query /gallery features are combined together through a simple dot product in decoder. I don't think the cross attention between query/gallery can be learned in this way - it is more likely the performance improvement is gained by the transformer encoder instead of the proposed decoder.
- The comparison with ViT and transformer in Table 2 is not quite fair, since the authors use a smaller transformer model (line 263-267) due to memory overflow problem. The original settings may actually work better
- Since both person-ID and self-attention have the same concept "query" but having quite different meanings, the authors should make clear which query is referred to in the paper to avoid confusion.
- Intuitive visualizations can be helpful to reveal the reason why the transformer can work better for person-ReID problems.

Overall, although this paper has reported higher performance than the baseline, the motivation of using transformer is not very clear, the pipeline design is naive and lack novelties, and some experimental results are still questionable. Considering the superior performance, I vote for a marginally below.

------After rebuttal------
The authors solved parts of my concerns and therefore I increase the rating to marginally above (6).
limitations_and_societal_impact: Yes, I think so.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a Transformer-Based Deep Image Matching approach for generalizable person re-identification. It adopts two simple solutions based on the transformer to achieve pair image matching. The idea is simple and easy to follow. 
main_review: This paper proposes a Transformer-based image matching to achieve generalizable person re-identification. 

This idea is based on the assumption that local feature matching helps to improve the generalization ability of models. I agree with this conclusion. The cross image attention proposed in this paper is indeed making sense for image-to-image matching. The ablation studies and experiments are convincing and demonstrate the improvement of the proposed TansMatcher. 


However, the authors only use the transformer to achieve the local matching, therefore, the contribution is limited. 

limitations_and_societal_impact: There are four limitations:
1.	In this experiment, single dataset training and single dataset testing cannot verify the generalizable ability of models, it should conduct experiments on large-scale datasets.
2.	The efficiency of such pairwise matching is very low, making it difficult to be used in practical application systems. 
3.	I hope to see that you can compare your model with ResNet-IBN / ResNet of FastReID, which is practical work in the person Reid task.
4.	I think the authors only use the transformer to achieve the local matching, therefore, the contribution is limited.
review_title: null

Meta-review: This paper copes with the interesting problem of using transformer-based models for image matching. This work shows that such a mechanism can successfully be used for person re-identification. The reviewers have recognised the simplicity of the proposed approach and good simple baselines described along with the approach. The experimental results show improvement over recent work on domain generalized person re-id. Despite the lack of important theoretical novelty, the tackled topic is timely and the proposed model simple. I suggest the acceptance of this work.
Decision: Accept (Poster)