Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work is progressing towards solving the Universal Domain Adaptation (UniDA) problem, where we don’t assume any prior knowledge about the target classes. The proposed algorithm, named UniOT, utilizes Optimal Transport (OT) to handle two main issues: i) common class detection and ii) private class discovery.

The common class detection is run by establishing partial alignment between the target features and the source prototypes, while the private class discovery is conducted through aligning the target features with the learnable target prototypes. Both are achieved by solving the OT optimization problem.

From the empirical evaluations on the Office, Office-Home, VisDa, and DomainNet, the proposed algorithm UniOT performs significantly better than the existing UniDA methods indicated by the H-scores. Under varying ratios of common classes, UniOT shows a certain level of robustness compared to DANCE and UAN.
strengths_and_weaknesses: Strength:
- The OT optimization approach to addressing the UniDA problem by encouraging common class detection and private class discovery is novel and technically sound.
- The performance gain over the existing methods is substantial along the line of UniDA.
- The manuscript is well-written.

Weakness:
- Would be great if information of both training and inference elapsed time is provided, especially at the inference stage as it requires to run an additional OT optimization.

questions: Still related to the previous point, I'd like to see in more detail about the time complexity of the training and inference.
limitations: It is implicitly stated in the conclusion that the current approach requires relatively high memory usage and will find a more memory-efficient way.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes an optimal transport framework to jointly consider common class detection and private class discovery for UniDA. The key point is automatically discovering the intrinsic difference between common and private classes based on the statistical information of the assignment matrix.
strengths_and_weaknesses: [Strengths]
This paper addresses Universal Domain Adaptation. The partial alignment for common class detection and representation learning for private class discovery based on UOT and OT models have achieved relatively high classification accuracy. The method of experimental comparison is comprehensive.

[Weaknesses]
The proposition of this method, to automatically discover private specific classes, lacks meaning in practice. Since we do not have the labels of these classes, we cannot use these data, but it brings more complex computing resource consumption. The fundamental goal of UniDA is to improve the precision and recall rate of common classes. Identifying private classes is only a process, so it can be regarded as a class.
The new metric H^3-score is the same problem.
questions: How do you tune the hyperparameters in the method, such as Equation 15? Only sensitivity to one hyper-parameter is stated in the supplementary material.
limitations: Yes
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this work, the authors proposed a unified domain adaptation framework supported by optimal transport in order to transfer among those domains having different categories. Specifically, the authors detect those common classes by predicting the statistics of an unbalanced OT objective. Next, the private class discovery is achieved by doing local alignments in the target domain.  
strengths_and_weaknesses: Pros:
- Overall, this paper is well written. The authors provided detailed information about how they designed the training process step by step.
- The authors have provided enough quantitative results in order to demonstrate the effectiveness of their method. 

Cons:
 - The proposed method is relatively ad-hoc. The proposed method serves as discriminative losses for supervised learning problems and improves performance in the target domain. 
 - Missing discussion on Batch optimal transport. The authors mainly implemented their framework using batch samples. However, It is known that minibatch optimal transport leads to nonoptimal coupling so there should be some discussions on how batch optimal transport affects this method.
 - Qualitative illustrations will be very helpful in illustrating this method's performance. For example, the authors did a great job in creating figure 2 to give a high-level illustration of the framework. It would be great if the authors could give an example of a batch of data, not only the OT couplings but also the corresponding source and target samples/polytopes. 

questions: Please see my comments in the last box. I am willing to improve my score if there are more interpretations and qualitative illustrations. 
limitations: No negative societal impact.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a novel framework, named UniOT, which jointly considers common class detection and private class discovery in Universal Domain Adaptation (UniDA). UniOT leverages Unbalanced Optimal Transport (UOT) to detect common classes in the target domain without any predefined threshold values. In addition, UniOT learns an OT-based representation learning for private class discovery by considering both global discrimination of clusters and local consistency of samples. Furthermore, the authors introduce a new evaluation metric for the UniDA problem. Finally, experiments show that UniOT outperforms several baselines on UniDA.

strengths_and_weaknesses: In this paper, the authors raise two interesting problems of UniDA that previous works do not consider carefully. Their method to tackle the aforementioned issues is novel to the best of my knowledge. Experimental results back the efficiency of the proposal in UniDA. The authors also conducted adequate ablation studies to verify the importance of different components in their framework. Overall, this paper is well-structured and easy to follow. The main concerns include the clarity of the paper in both methodology and experiment parts.
questions: I have the following questions:

1) **Related work.** For the application of OT in (P)DA, partial OT instead of UOT can also be used to allow partial displacement [r1].  
[r1] Khai Nguyen, Dang Nguyen, Tung Pham, and Nhat Ho. Improving mini-batch optimal transport via partial transportation. In Proceedings of the 39th International Conference on Machine Learning, 2022.
2) **Figure 2.** The authors should add more description in Figure 2 or its description because it is quite difficult to understand the whole framework at the beginning of the paper. Even after reading the Methodology, it is still time-consuming to figure out different components and their functionalities.
3) **Formulation of OT.** What is the definition of **M** in Equations 2 and 3? Why do the authors convert a minimization problem into a maximization one?
4) **Common class detection.** The authors claimed that this method does not use some manually specified or hand-tuned threshold values to detect common and target-private samples. Though the threshold in Equation (7) is not completely random, users have to tune $\epsilon, \kappa$ for UOT instead. Therefore, it seems like this method only parametrizes the threshold value into other hyper-parameters.
5) **The number of target prototypes.** How is K chosen? Why is it much larger than the actual number of classes?
6) **Equation 13.** Should different notations be used for the prediction of $z_i^t$ and $\tilde{z}_i^t$? Speaking of which, does the prediction of the nearest neighbor of $z_i^t$ lie in the $(B+i)$-th row of $\tilde{Q}^{tt}$ based on the description in L232-233?
7) **Target-private samples classification.**  When is the private class in the target domain classified? L259-260 state that they are all classified as unknown. As far as I understood, UniOT still treats all target-private samples as a single class as in prior works. 
8) **Experimental settings.** The authors use a lot of formulas and settings from other papers. It is better to describe them again in the Appendix to improve the readability and make the paper more self-contained. 
9) **Baselines' results.** Why are the performance of DANCE in Table 2 and Table 3 worse than weak baselines such as ResNet or DANN? In addition, the authors should add a symbol to indicate which results are taken directly from the original papers and which are the reproduced results.

**Minor:**
* There is a typo in L270. It is either A2D or Amazon to Webcam.
* In column D2A of Table 4, the best H-score is not in the last row so it should not be in bold. In addition, it seems to be unnecessary to additionally report two random tasks for each dataset. It is alright to just report the average performance over all the tasks.

I am happy to increase my score if the above questions are adequately addressed.

limitations: The authors specified one limitation of their method in Section 5.
review_title: null

Meta-review: metareview: This work tackles universal domain adaptation, a challenging problem that is usually encountered in real practice. The proposed UniOT algorithm utilizes Optimal Transport to enable common class detection and private class discovery. It is interesting to see that OT can be extended to simultaneously solve these two problems. Reviewers were at the borderline in their preliminary opinions, but after rebuttal and reconsideration, most reviewers acknowledged that their concerns were addressed, and improved their final rating substantially. AC considered the paper itself as well as all reviewing threads, and concluded that the paper has put forward a nice Unified OT framework for the challenging universal domain adaptation problem, yielding promising empirical performance while introducing some nice technical benefits such as the removal of the tedious weight thresholding for outlier class discovery. Thus the paper is recommended for acceptance.
confidence: Certain
Decision: Accept