Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: While doing domain adaptation, we usually have a source classification loss and a domain alignment loss. However, these losses might compete with each other and we might reach a situation where one loss can’t be reduced without degrading the other one. We call that the Pareto front. For DA, we don’t know which loss is better than the other one. In this paper, they come up with an optimization strategy that helps us navigate to an ideal portion on the Pareto front. Essentially, they stop the optimization if one loss can’t be improved without degrading the other.
main_review: - Originality: 1. Seems to be a modified version of EPO where the only difference is that this one is specifically designed for DA. This one also uses a custom surrogate loss for DA while EPO uses KL divergence. 2. They came up with their own equations for 3. They claim that Pareto optimization hadn’t really been used before for DA, but I found a paper (Locality Preserving Projection for Domain Adaptation with Multi-Objective Learning Le Shu, Tianyang Ma, Longin Jan Latecki) which did. Needs a comparison against that.

- Quality: Well thought out and precise experiments. Technically sound with well-supported claims. However, I am worried we might get stuck in local minima in some situations (especially in the beginning stages of training) with their optimization strategy (losses need to increase sometimes to get out of local minima, but their conditions don’t allow for it). Would appreciate it if they could tell us about their experience wrt that.

- Clarity: The theoretical section 4 is not very well written and could be polished and made simpler, and easier to understand. One thing, I still don’t understand how you determine a solution to be an ideal solution on the Pareto front  (I get that we can’t choose anything on the extreme ends, but where did you get the Expected Solution on Fig. 1 and the ParetoDA (ideal) in Fig. 3). Is that from the intersection of the preference vector from the surrogate loss with the Pareto boundary?

- Significance: Overall, it is a plug-and-play optimization technique for DA which can be easily used with already existing SOTA model architectures to get performance boosts. Could be a valuable tool for DA. Maybe they should talk from the perspective of autoML for DA.

AFTER THE DISCUSSION PERIOD: I would like to thank the authors and reviewers and AC. I think that enough information and clarification have been exchanged so that the major issues have been adequately addressed. One of them was also the comparison with previous work, which the authors have addressed very effectively. I would have given a higher score if fewer things had to be revised for the camera ready, but anyways, I am in favor of accepting the paper.
limitations_and_societal_impact: Yes, they have.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper addresses the issue of previous domain adaptation optimization scheme, which learning objectives conflict with each other. The main contribution is to dynamically control the overall optimization direction so that no objective is harmed resulting out to boost the performance. To this end, it designs desired pareto optimal solution with surrogate objective function that mimics the target classification task called Pareto Domain Adaptation.
main_review: Originality:

The main originality of this paper seems to come from two methodologies, CIM loss and Pareto optimal solution.  For CIM loss, utilizing domain label to compute class-wise domain prediction to modify the target predictions is thought to be novel.  And then, it shows well comparison with other gradient-based scheme and proves the effectiveness and originality of dynamic pareto optimal solutions.

Quality:

Designing process of CIM loss and Pareto Optimal solution is well supported by theoretical analysis and final results.
However, the major curiosity that I have is relationship between CIM loss and pareto optimal solution. Ablation study in Table 5. doesn't guarantee that CIM loss is better solution for ParetoDA compared with basic IM loss. It would be more convincing if the author can provide IM + ParetoDA results to prove CIM as a better guidance for the proposed adaptation method.  
I also wonder whether this framework can be applied to self-training based domain adaptation[1], specifically where the explicit domain alignment module doesn't exist.  
-  [1]  Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training,  Yang et al.


Clarity:

This paper is well written and easy to follow. Specifically, figure in this paper well describes the intention of the author and proves the effectiveness of suggested model.


Significance:

This paper seems to show quite significant result with boosting improvement  and robustness  to the different tasks and backbone model.
limitations_and_societal_impact: Author well express the limitation and potential negative societal impact in supplemental material.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper represents a domain adaptation (DA) problem in a multi-objective optimization scheme, which can be solved by a gradient-based perspective. The main contributions of the paper include:

1. Proposes a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to optimize all training objectives. That ParetoDA scheme can be plugged into various DA methods and enhance their performance. 
2. To find a desired Pareto optimal solution, the authors design a surrogate objective function that mimics the target classification task. 
3. Provides experimental results that show the effectiveness of ParetoDA compared to some DA methods.
main_review: Quality: The idea of representing the domain adaptation setting to a multi-objective optimization scheme is interesting. However,  I'm a bit concern about the theoretical contributions of the paper since the method is quite straightforward. Moreover, there is no theoretical guarantee for the existence of a Pareto optimal of the corresponding multi-objective optimization problem by using the proposed ParetoDA. Also, it would be much more interesting if the authors can develop an upper bound for the generalization error of the DA.

Clarify: Here are several critical points:
- It's unclear for me about the formulation of the loss $\mathcal L_T$ in (5). Why minimizing that loss can handle the domain-shift in the domain adaptation problem? That's important since it can verify the main contribution of the paper.
- (Line 218) It's unclear how to select the validation set $X_t^{Val}$. 
- The linear programming (6) misses the condition $\sum w=1$? Does this affect the solution of (6), and consequently the performance of the DA framework?
Originality: To the best of my knowledge, this paper is the first work that introduces the use of multi-objective optimization in a domain adaptation problem.

Significance: Based on my above comments and evaluations, I personally think that it's a bit below the acceptance rate of a NeurIPS publication.

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper considers the common class of domain adaptation algorithms consisting in minimizing the sum of a source risk term and a divergence between the domains. Since both terms are required to be small, the authors examine the tradeoff between them via their associated Pareto front. This latter having non-convex regions non-reachable by the classic weighted sum minimization procedures, a multi-objective optimization procedure is proposed in order to reach all of the regions of the Pareto front and to access tradeoffs that are not accessible by the classic approach. Specifically, the authors use a modified version of a multi-objective optimization procedure, where they define a guidance direction that adjusts the directions of both gradients of the two cost function terms. The guidance direction is the gradient of a surrogate loss assessing performance on the target domain. To show the empirical performance of their idea, the authors present experiments in which their method improves previously proposed DA algorithms.
main_review: The paper is clearly written and easy to follow. The idea is an interesting enhancement for DA methods, and the explanations about multi-objective optimization are succinct.

## Strengths

* Bringing multi-objective optimization to domain adaptation, which is an extension to several proposed algorithms.
* Utilizing the domain discriminator to construct a surrogate for the target domain loss, an original idea that goes beyond the classic role of aligning the two domains.
* The adaptation of the idea from EPO [6] to the specific case of domain adaptation by modifying the guidance direction ${\bf d}_{bal}$.

## Weaknesses:

* Although soft-labels are used for the target domain, no comparison to methods using pseudo-labels is used.
* Not mentioning of the link to methods performing class-wise domain alignment after the optimization problem of Equation (4) (e.g. [5])
* Related Works section:
  * It is necessary to mention theoretical DA contributions, at least the seminal ones (e.g. [1,2]). Indeed, the idea of the domain discriminator introduced in [1] has inspired [3], in addition to the more recent GAN paper [4].
* I believe that there is a problem in the proof of Theorem 1, specifically for the case $\gamma^{\*}\leq 0$ (lines 54 to 57 in the supplementary material). Indeed, we have $({\bf G}{\bf w}^*)^T{\bf g_v} \leq 0 \Leftrightarrow \sum_{i=1}^3w_i{{\bf g}_i^T{\bf g_v}}\leq 0$. This implies that there is some $j \in \{1,2,3\}$ such that   $w_j{{\bf g}_j^T{\bf g_v}}\leq 0$, which further implies that  ${{\bf g}_j^T{\bf g_v}}\leq 0$ if $w_j>0$. This, however, does not imply that $\forall i \in {1,2,3}, {\bf g}_i^T{\bf g}_v < 0$ (contradicting the statement made in line 55 " there is no gradient ${\bf g}_j$, $j \in  \{1, 2, 3\}$, for which ${\bf g}_j^T {\bf g}_v \geq 0$").
* Lacking notation definitions:
  * $\mathcal L$ (line 87) is not defined.
  * ${\mathit{ v}}_{\mathbf c}$ (Equation set (3)), and $\lambda_0, \lambda_1$ (line 338) are not defined  
  * In Equation (5), $X_t$ is not defined. Is it the finite set of observed target instances ?
  * The definition of gradients ${\bf g}_j$ is given in line 222, after their first mention in line 208.
  * In Equation (4), either a $\max$ should be instead of $\min$, or a minus sign has to be added, as the expression is a log-likelihood to be maximized.
  * In the objective functions in (6), $\bf 1$ should be defined.

## Comments

* Line 173 "The probability of the i-th ..." --> "The class-conditional probability of the i-th ..."

### Minor comments:

* Line 151 ". While" --> Either ", while", or ". Meanwhile/On the other hand...".
* Line 215: "Chapter" --> "Section"
* Lines 181-182: 
  * "containing" --> "contain" (otherwise the sentence seems incomplete).
  * "belongs" --> "belong"
* Supplementary, line 53: "then $\gamma^*$" --> "then $\gamma^*>0$".
* Line 252: "$n\gg m$" --> "$d\gg m$" ? 

## Suggestions

* The "soft labels" term for the target domain, mentioned in the supplementary (line 69), offers more intuition about the objective in Equation (4). It should be mentioned in the main paper.
* I recommend performing the experiments from Figure 3 to other datasets (toy 2d data) .
* The last paragraph title should contain "multi-objective", as "Gradient-based optimization" refers to mono-objective first order methods, as multi-objective optimization is not yet commonly adopted by the ML community.

* Although its meaning is given in line 219, the notation ${\bf g}_v$ can be confused for the column vector of $\bf G$ that has index $v$. I suggest using another letter for denoting the guidance direction.

## Questions: 

* Is the extension to regression immediate ?
* Can the surrogate target loss be used to select hyper-parameters in an unsupervised way for other DA methods ?
* Isn't the condition  $\mathcal L_{val}>0$ too strict in the definition of optimization problem (6) ? Is it beneficial to relax it ? (e.g. $\mathcal L_{val}>\varepsilon$ and $\mathcal L_{val}<\varepsilon$) for a given small $\varepsilon$ ?
* Have you tried to visualize the Pareto front for different domain gaps ? It is interesting to see how it evolves as the gap between the two domains becomes larger.

## References

[1] Ben-David, S., Blitzer, J., Crammer, K. et al. A theory of learning from different domains. Mach Learn 79, 151–175 (2010).

[2] Mansour, Y., Mohri, M., Ro, J., Theertha Suresh, A. &amp; Wu, K.. (2021).  A Theory of Multiple-Source Adaptation with Limited Target Labeled Data . <i>Proceedings of The 24th International Conference on Artificial Intelligence and Statistics</i>, in <i>Proceedings of Machine Learning Research</i> 130:2332-2340

[3] Ganin, Y., & Lempitsky, V. (2015, June). Unsupervised domain adaptation by backpropagation. In International conference on machine learning (pp. 1180-1189). PMLR.

[4] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.

[5] Long, M., Cao, Z., Wang, J., & Jordan, M. I. (2018). Conditional Adversarial Domain Adaptation. *Neural Information Processing Systems*.

[6] Mahapatra, D., & Rajan, V. (2020, November). Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. In *International Conference on Machine Learning* (pp. 6597-6607). PMLR.

---------
## Update after authors rebuttal
I thank the authors for their rebuttal that clarified the different concerns I raised.
limitations_and_societal_impact: One limitation is stated in the conclusion, which is the restriction to the traditional domain adaptation setting. It would be clearer to specify what is meant by traditional (single-sourced, closed set, etc). The societal impacts are discussed in the supplementary material.
review_title: null

Meta-review: The paper proposes a novel way to solve the Domain adaptation problem by formulating it as a multi-objective optimization problem instead of a sum of objective values. This allows for solving the DA problem with less hyper parameters that are hard to select in practice and can have an important impact fr applications of DA. Despite this original idea the paper had originally  borderline reviews due to some missing explanations and experiments. The authors did a very good job at answering the reviewers concerns with new experiments and even theoretical results. The reviewers agreed after the replies  that the paper is interesting and deserve to be published. 

Please take into account all reviewers comments (missing references and clarifications) and insert the novel results (both numerical and theoretical) in the main paper and supplementary for the final version because the consensus among reviewers is that it will make the paper much better and interesting for the community.
Decision: Accept (Poster)