Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Summary:
- key problem: address "class mismatch" in adversarial learning methods for unsupervised domain adaptation (UDA);
- contributions: 1) extension of the domain adversarial learning objective to leverage class prototypes (exponential moving average of features weighted by predicted class probabilities) in addition to pseudo-labels and intermediate representations (cf. eqs.5-11), 2) state-of-the-art results on several UDA tasks (Office-Home, ImageCLEF-DA, sim2real on Cityscapes).

Recommendation: weak accept (with some reservations below).

Key reason: interesting and effective use of prototypes for UDA.
- The formulation of the prototypes and additional learning objectives for UDA are clear and seem novel, although I would like to see a discussion of additional related works:
-- "Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results", Tarvainen and Valpola, NeurIPS'17;
-- "Unsupervised Domain Adaptation with Similarity Learning", Pinheiro, CVPR'18;
-- "Transferable Prototypical Networks for Unsupervised Domain Adaptation", Pan et al, CVPR'19.
- The effectiveness of the contributions is validated on multiple UDA tasks, and the ablative analysis supports the claims (that prototype-level alignment and within-class compactness helps).

Main reservation: the specific problem is not clearly formalized.
- What is the often mentioned but not clearly described "class mismatch" problem in UDA? To the best of my knowledge, this not a standard problem (could not find any mention in the previous literature, no citations or definitions in the submission). Is it that the target label space is different than the source label space (e.g., different ontologies)? In this case, what is the information on the target label space that enables unsupervised adaptation from the source one? What is the inductive bias / prior / assumptions? 
- Alternatively, is the tackled problem only the noise in the pseudo-labels?
- In any case, the submission would greatly benefit from a clearer mathematical formalism and experimental characterization of the specific problem tackled here, especially in light of claims like "conditioning the alignment on pseudo labels can not well address the mismatch problem. Compared with the pseudo labels, the class prototypes are more robust and reliable in terms of representing the distribution of different semantic classes."

Additional Feedback:
- missing references on sim2real UDA: "DADA: Depth-aware Domain Adaptation in Semantic Segmentation" (Vu et al, ICCV'19), "SPIGAN: Privileged Adversarial Learning from Simulation" (Lee et al, ICLR'19)

## Post rebuttal update

I would like to thank the authors for replying to our questions. The clarifications with respect to related works and missing references is helpful, although a bit high-level (i.e. not necessarily describing the relative advantages of the proposed method). Nonetheless, the expected benefits of prototypes is still not entirely clear enough here, for instance regarding the main statistical assumptions that the method needs to make to get robust prototypes (e.g., in the presence of outliers or specific forms of "inaccuracies" in the pseudo-labels or "domain misalignment"). Therefore, due to the overall lack of mathematical clarity in the text and rebuttal, my main reservation remains, and I will change my "weak accept / borderline" score to weak reject. I encourage the authors to formalize the problem in a clearer, non-ambiguous way, discussing more explicitly the limitations of the proposed method.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper proposes to leverage prototypes to solve the mismatch problem in unsupervised domain adaptation. It further imposes intra-class compactness to help ambiguous classes. Experiments show it achieves new state-of-the-art results in several datasets.

pros:
+ intra-class compactness to help ambiguous classes

concerns:
-- Prototypes does not come from nowhere. They come from predictions. If you worry about the quality of target predictions (pseudo labels), then Eq. 8 and Eq. 9 are questionable. The intra-class compactness relies on p_t, too. The authors should explain why prototypes are superior than pseudo labels in [1].
-- How does the authors select hyper-parameters? There are lots of magic numbers in Section 4.1 about hyper-parameters but no clues about how to tune them. Recently there is a paper [2] about model selection for UDA, maybe the authors should try it.

details:
- terminology: "intra-class" is better than "within class"
- separate citations: e.g. entropy minimization, mean-teacher, and virtual adversarial training, have been successfully applied to UDA (Vu et al., 2019; French et al., 2018; Shu et al., 2018) -> entropy minimization (Vu et al., 2019), mean-teacher (French et al., 2018), and virtual adversarial training (Shu et al., 2018), have been successfully applied to UDA
- confusion: At the last of Section 3.2, it says \hat{f}=M^{T}p. But in Eq. 9, \hat{f} and M^{T}p are concatenated, which is confusing: why do you concatenate two identical vectors?
- Implementation Details: Section 4.1, paragraph 4: \lambda^{f}_{adv} =5e-3, \lambda^{f}_{adv} and \lambda^{p}_{adv} increase from 0 to 1. It is confusing that \lambda^{f}_{adv} both is a constant and changes continuously. 

[1] Conditional adversarial domain adaptation, Long et.al, in NeurIPS 2018
[2] Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation, You et.al, in ICML 2019
review_title: Official Blind Review #1

Meta-review: The paper focuses on adversarial domain adaptation, and proposes an approach inspired from the DANN. The contribution lies in additional terms in the loss, aimed to i) align the source and target prototypes  in each class (using pseudo labels for target examples); ii) minimize the variance of the latent representations for each class in the target domain. 

Reviews point out that the expected benefits of target prototypes might be ruined if the pseudo-labels are too noisy; they note that the specific problem needs be more clearly formalized and they regret the lack of clarity of the text. The sensitivity w.r.t. the hyper-parameter values needs be assessed more thoroughly. 

One also notes that SAFN is one of the baseline methods; but its best variant (with entropic regularization) is not considered, while the performance thereof is on par or greater than that of PACFA for ImageCLEF-Da; idem for AdapSeg (consider its multi-level variant) or AdvEnt with MinEnt. 

For these reasons, the paper seems premature for publication at ICLR 2020. 
Decision: Reject