Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes an unsupervised domain adaptation method for person re-identification. The proposed method handles noises on pseudo labels created by unsupervised clustering. Two networks are used for training, and the estimated confidences of other models are used for the next training iterations. The temporally average model is used for each network to avoid error amplification. Also, soft softmax-triplet loss is proposed to handle soft labels for triplet loss. 

The handling label noises in unsupervised domain adaptation on person re-identification are new. The proposed model produces very high performance and the contribution for person re-identification community is good. 

However, I would like to see more insights into the proposed model for the contribution of the general deep learning conference. 

First, this paper lacks a survey of works on handling label noises. For example, 
B.Han, Q.Yao, X.Yu, G.Niu, M.Xu, W.Hu, I.Tsang, M.Sugiyama, Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels, NeurIPS2018. 

I could not fully understand why the temporal averaging of model parameters prevents the two models from being the same. I would like to see a theoretical explanation or experimental evidence for this claim. 

The proposed method also uses noisy hard pseudo labels for training, as shown in Eq.(9). 
Why are the noisy hard labels used? What is the performance when only soft labels are used for model updates?

In the experiment, \lambda^t_{id} = 0.5, \lambda^t_{tri} = 0.8 are used. Why these parameters are different between softmax and triplet losses?

p.1 (Zhang et al., 2018b) and p.5 (Zhang et al., 2019a) are missing in references. 



review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: After reading the reviews and the comments, I confirm my rating.

=================

The paper proposes an unsupervised framework to address the problem of noisy pseudo labels in clustering-based unsupervised domain adaptation (UDA) for person re-identification. The noise derives from the limited transferability of source-domain features, the unknown number of target-domain identities, and the imperfect results of the clustering algorithm. 

The proposed framework, Mutual Mean-Teaching (MMT), performs pseudo label reﬁnery by optimizing the neural networks under the joint supervisions of off-line reﬁned hard pseudo labels and on-line reﬁned soft pseudo labels. Inspired by the teacher-student approaches (Reference: Tarvainen & Valpola, 2017; Reference: Zhang et al., 2018b), the proposed MMT framework provides robust soft pseudo labels in an on-line peer-teaching manner to simultaneously train two same networks. The networks gradually capture target-domain data distributions and thus reﬁne pseudo labels for better feature learning.

The main contribution is proposing an unsupervised framework (MMT) capable of tackling the noise problem in state-of-art UDA methods for person re-identification, via producing reliable soft labels in order to achieve better performance. Since the conventional triplet loss cannot properly work with soft labels, a softmax-triplet loss is proposed to enable training with soft triplet labels for mitigating the pseudo label noise.

The proposed MMT is evaluated on Market1501, DukeMTMC-reID, and MSMT17 datasets with four adaptation tasks: Market-to-Duke, Duke-to-Market, Market-to-MSMT, and Duke-to-MSMT. It outperforms the state-of-the-art methods with significant improvements in terms of Mean average precision (mAP) and Cumulative matching characteristic (CMC). In addition to that, ablation studies conducted to evaluate each component in the proposed MMT framework.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The authors' response addressed my concerns. After reading the reviews and the comments, I choose to stand with the other reviewers.

===================

This paper uses mean-teacher to ease the noisy pseudo label of clustering methods for domain adaptive Person re-identification task. The authors also propose a variant of triplet loss for soft labels. Experiments show they achieve considerable improvement over state-of-the-art methods.

Questions:
1. What's the difference between net 1 and net 2 in Fig. (2)? It seems they are redundant.

2. The results in Table (1) seems to indicate that, in MSMT, if M_t is set to be near the actual identity numbers (1041), the performance will be much better. This makes me suspect that the proposed method benefits from ground truth information of the target domain, which makes the comparison unfair.

review_title: Official Blind Review #2

Meta-review: The paper proposes an unsupervised framework for domain adaptation in the context of person re-identification to reduce the effect of noisy labels. They use refined soft labels and propose a soft softmax-triplet loss to support learning with these soft labels. 

All reviewers have unanimously agreed to accept the paper and appreciated the comprehensive experiments on four datasets and ablation studies which give some insights about the proposed method. I agree with the assessment of the reviewers and recommend that this paper be accepted.
Decision: Accept (Poster)