Venue: UAI 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept, e.g., novelty, outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: Q1 Summary and contributions: This paper presents an unsupervised class aware domain adaptation approach. The approach is based on distributional optimal transport and seeks to align unlabeled target examples distribution with labeled source class-conditioned distributions. The model is trained in a cyclic manner involving pseudo-labeling, knowledge distillation, and minimizing the optimal distribution transport cost. Experimental results suggest performance improvement of proposed approach on three benchmark datasets.
Q3 Main strengths: - Paper is well-written and the presentation is good

- Combines pseudo-labeling with student-teacher and optimal transport of distributions

Q4 Main weakness: - Theoretical understanding of the proposed approach would further improve the quality of the paper. It is not clear if entropy regularized approach has any particular limitations

- Some implementation details of how the losses are optimized could be useful. 

Q5 Detailed comments to the authors: - It would be useful to comment on semi-supervised domain adaptation and class-aware unsupervised domain adaptation. As such, the class aware unsupervised domain adaptation is highly sensitive to the pseudo-labeling strategy. In challenging problems, it is likely that some target samples are required to achieve effective class alignment. (see [1], [2] )

- Distribution vs fixed optimal transport differences are not well explained, which seems to be the main contribution of the paper.

[1] A Dirt-t Approach to Unsupervised Domain Adaptation, 2018 

[2] DIRL: Domain-Invariant Representation Learning for Sim-to-Real Transfer, 2020

Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q7 Justification for your score: See above. Experienced in publishing and reviewing papers in domain adaptation
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., lack of novelty, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: Q1 Summary and contributions: The paper tackles unsupervised domain adaptation (UDA) via a cycle class-consistent model combining optimal transport (OT) and knowledge distillation (KD). To achieve computation efficiency, they regularize the OT problem with negative entropy to obtain a pseudo labels, i.e. solving the regularized/"soft" problem instead of the original one, to learn the teacher. Knowledge distillation is then utilized to distill and transfer the teacher's knowledge to the student. 
Q3 Main strengths: The idea of OT for domain adaptation is not new. This work differs a bit perhaps in the way the cost function is defined, taking into account target and source class-conditional distributions. However, the combination of OT and KD, which is again a known method, is a good step to verify if such combination could work or improve over known methods. The work has methodological contribution rather than theories, so it's important to illustrate its performance in practice. In this aspects, the experiments are extensive and show marginal outperformance of the proposed method over SOTAs. 
Q4 Main weakness: The experiments show mostly marginal improvement over SOTAs, which could have been an effect of fine tuning rather than the superiority of the proposed method. 
Q5 Detailed comments to the authors: Regularizing OT with negative entropy would result in dense and strictly positive solution, i.e. pseudo label, which could affect the performance. It may be worth solving the original problem without regularization to see its performance (note: the original one is simple linear optimization, so there should be empirical solvers). Another point is that using smaller $\epsilon$ than 0.1, the regularizing coefficients of entropy, may result in better performance perhaps. The negative entropy is of order $-H(X) =O( \log(n))$, so usually $\epsilon$ is set to be ~ $\epsilon = O(desiredError / \log(n))$ which can be small. My point is that a bit more tuning of $\epsilon$ may help.
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q7 Justification for your score: It is a nice method, yet the empirical results are not strong enough. Given that the nature of this work is not theoretical, I believe more favorable performance is desired.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area of AI or moderate to high impact on more than one area of AI, with good to excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations. 
review_content: Q1 Summary and contributions: The paper proposes a new domain adaptation method, based on optimal transport between target samples and source classes, combined with a distillation between source and target classifiers. Experiments are performed that show an improvement upon the previous state-of-the-art.
Q3 Main strengths: * Very good results
* An ablation study is performed to show that all parts of the model are important, and sensitivity to hyperparameters.
Q4 Main weakness: * The final method is very complicated, with five different terms in the loss function, many hyperparameters, and multiple classifier models.

* The intuition behind the method is not explained well.

* It is not clear how the ablation studies were carried out. What datasets were used? How do we know that the reported results are not simply the best out of many runs with different parameter values? Was a validation set used?

* Some details are not clear from the paper, such as "we employ Virtual Adversarial Training (VAT)", which involves selecting adversarial perturbations. This is completely glossed over.

Q5 Detailed comments to the authors: * In the model distillation, why is h^t seen as the teacher? It works nicely with T standing for both teacher and target, but intuitively, I would expect that you are able to train a model on the source domain from which another model can learn. How can you learn a student model from a bad teacher trained only with pseudo labels.

* What is the optimal transport problem used for exactly? The labels are inferred with pseudo-labeling, what remains learnable in (3) or (8)?

* Standard deviation of the results over multiple runs should be included in the tables. This is common in other DA papers.

* The transport cost c(x, P^S_m) is defined as the class conditional likelihood p(x | y=m), but to compute this you would need a density estimator, not just a classifier. A classifier will give the converse, p(y=m | x)

* The OT problem in (8) enforces a certain class balance, as specified by π. How is π chosen? Are these determined empirically from the source domain?

* Using h^T for teacher and h^S for student is confusing when S and T superscripts are also used for source and target domain.

* The name Cycle Consistent makes me think of cycle consistency loss in CycleGANs, but that is completely unrelated to this work.

Q2(4) Quality of experiments (Optional): 2: Fair: The experimental evaluation is weak: important baselines are missing, or the results do not adequately support the main claims.
Q7 Justification for your score: The results are good, the method is explained in sufficient detail.
However, the method is complicated and could be motivated better.
I also have some concerns about the hyperparameter selection.
review_title: null

Meta-review: Meta Review: This paper proposes an unsupervised class-aware domain adaptation approach, which is based on distributional optimal transport and seeks to align unlabeled target examples distribution with labeled source class-conditioned distributions. The model is trained in a cyclic manner involving pseudo-labeling, knowledge distillation, and minimizing the optimal distribution transport cost. The authors conduct experiments to demonstrate the performance improvement of proposed approach on three benchmark datasets. All the reviewers agree that the paper is well written and easy to understand. Review XPWv and 1mtQ agree that the idea of OT for domain adaptation itself is not new, but the authors explore the combination of OT and KD and improve the existing methods. R2 is the only reviewer who is against accepting this submission, and R2's only concern is about whether the small differences to baseline are significant. The authors responded by showing the significant performance on the Office-31 dataset. Due to the reasons above, I make the decision to accept the paper.
Decision: Accept (Poster)