Pareto Domain Adaptation
Authors: Fangrui Lv
Jian Liang
Kaixiong Gong
Shuang Li
Chi Harold Liu
Han Li
Di Liu
Guoren Wang
Venue: NeurIPS 2021 Poster (2021)
Keywords: transfer learning
domain adaptation
pareto optimization

Abstract: Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to an unlabeled target domain that follows different distribution from the source. To achieve this, DA methods include a source classification objective to extract the source knowledge and a domain alignment objective to diminish the domain shift, ensuring knowledge transfer. Typically, former DA methods adopt some weight hyper-parameters to linearly combine the training objectives to form an overall objective. However, the gradient directions of these objectives may conflict with each other due to domain shift. Under such circumstances, the linear optimization scheme might decrease the overall objective value at the expense of damaging one of the training objectives, leading to restricted solutions. In this paper, we rethink the optimization scheme for DA from a gradient-based perspective. We propose a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives. Specifically, to reach a desirable solution on the target domain, we design a surrogate loss mimicking target classification. To improve target-prediction accuracy to support the mimicking, we propose a target-prediction refining mechanism which exploits domain labels via Bayes’ theorem. On the other hand, since prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution on the target domain, we propose a dynamic preference mechanism to dynamically guide our cooperative optimization by the gradient of the surrogate loss on a held-out unlabeled target dataset. Our theoretical analyses show that the held-out data can guide but will not be over-fitted by the optimization. Extensive experiments on image classification and semantic segmentation benchmarks demonstrate the effectiveness of ParetoDA

Pareto Domain Adaptation
Fangrui Ly,''*
Jian Liang, 2+ Kaixiong Gong,'
Shuang Li,
Chi Harold Liu,"
Han Li,
Di Liu,
Guoren Wang'
Beijing Institute of Technology, China Alibaba Group, China
[fangruilv, kxgong, shuangli, wanggzbit) @bit .dd Eluchi020gmall 2 (xuelang 1jv hhan. 1h, wendi 1d) Ba baababaabbaaa nn.
Abstract
Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to unlabeled target domain that follows different distribution from the source. To achieve this, DA methods include a source classification objective Lg to extract the source knowledge and domain alignment objective Ln diminish the domain shift, ensuring knowledge transfer. Typically, former DA methods adopt weight hyper-parameters
linearly combine the training objectives form an overall objective C. However, the gradient directions of these objectives may conflict with each other due domain shift. Under such circumstances, the linear optimization scheme might decrease the overall objective value at the expense of damaging one of the training objectives, leading restricted solutions. In this paper, we rethink the optimization scheme for DA from a gradient-based perspec- tive. We propose Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives. Specifically, to reach a desirable solution on target domain, we de- sign surrogate loss mimicking target classification. To improve target-predd prgeetprediction accuracy to support the mimicking, we propose a target-prediction refining mecha- nism which exploits domain labels via Bayes' theorem. On the other hand, since prior knowledge of weighting schemes for objectives is often unavailable to guide optimization approach the optimal solution the target domain, propose dynamic preference mechanism to dynamically guide our cooperative optimization by the gradient of the surrogate loss held-out unlabeled target dataset. Our theoretical analyses show that the held-out data can guide but will not over-fitted by the optimization. Extensive experiments image classification and semantic segmentation benchmarks demonstrate the effectiveness of ParetoDA. Our code is available
Introduction
Domain adaptation (DA) is well-established paradigm for learning model unlabeled target domain with the assistance a labeled related source domain, which has attracted surge of interests in the machine learning community [34] 33 [3] By bridging the domain gap the premise of ensuring the performance source classification task, DA can adapt the model learned from the source domain to the target in the presence of data bias, which solves the dilemma of label scarcity in 57. Extensive DA approaches have been proposed in recent years, achieving great performances in many areas such as image classification [1] 27 56).
many real-world applications [55 47
semantic segmentation
49 52 [6,, and object detection 33
43 54
Equal contributions from both authors. Corresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Most DA methods are devoted to aligning the distributions across domains by explicitly minimizing
some discrepancy metries 226 28 [6] or adopting adversarial learning 22 However, since the domain alignment objective also depends on the target domain that deviates from the source, the gradient direction of it may conflict with that of classification objective on the source domain. Thus, when the optimization proceeds to certain stage, one objective will deteriorate if further improving the other one. Under this circumstance, no solution can reach the optimal of each objective at the same time. We can only obtain set of so-called Pareto optimal solutions instead, where we cannot further decrease all objectives simultancously 110 compose Pareto front as shown in Fig.
All Pareto optimal solutions in loss space
2: 2465 + 22LD
Initial Point
*** * Solution
Expected Solution
P Preference Vector
Co
LD
Lp
CD
F ixed
La 1.
D Dynamic
bn 1.6s V-
Cp
Cs
G Guidacce Direction
Ls
Lg
Ls
Ls
(a)
(b)
(c)
(d)
Figure 1: Illustration of different optimization schemes. In each panel, the blue curve is the Pareto front where the region underneath is unaccessible. (a)-(b): Linear scheme that adopts weight hyper-parameters to unify the objectives. The green and purple dash lines represent different hyper- parameters. (c): Previous gradient-based scheme, which reaches Pareto optimal solutions with or without the guidance of preference vector. (d): Our ParetoDA that dynamically guides the optimization by the gradient of the aagecccllssiiiccatoommmiiiiiiiii loss on a held-out unlabeled target dataset, approaching the expected solution that minimizes the target classification loss.
This dilemma obviously has been overlooked by most former DA methods, which generally adopt some empirical weight hyper- parameters linearly combine the objectives, constructing an overall objective. This linear weighting scheme has two major concerns. First, as pointed out in 4 it can only obtain solutions on the convex part of the Pareto front of the objectives, while cannot reach the non-convex part, as shown in Fig. (a)- That is because it only considers reducing the overall objective value, which might damage some objectives during optimization (see the red up- up-arrow). Unfortunately, the Pareto fronts of DA methods are often non-convex, due to loss conflict caused by domain shift, See examples shown in Fig. 2 Second, one could hardly reach the desired solution that performs best on target domain by tuning the since the obtained solution will deviate severely when the trade-offs slightly change (see the slopes of the dash lines in Fig. T (a)-(b) for reference).
Domain alignment locs
CDAN
DANN
0.4
6.. 0.5
25
25 32
Source classification loss
Figure 2: Pareto fronts of DANN M
The non-convex
and CDAN (27) on task W-A (Office-31).
weight
To remedy these issues, we rethink the optimization scheme for DA from a gradient-based perspective. By weighting objectives to control the overall optimization direction so that no objective deteriorates, one can reach a Pareto optimal solution on the Pareto front 23 30 B2 Moreover, 30 precisely sets objective weights according to a prefixed preference vector (see Fig. (c)). However, neither of them suits DA, because 1) in DA, the goal is to minimize the target classification loss Lt. but neither L. or LD directly corresponds to this goal; 2) the target domain has no label to construct a classification loss for the goal: 3) prior knowledge of weighting schemes for objectives is often unavailable to guide optimization approach the optimal solution that minimizes Li.
In this paper, we propose a Pareto Domain Adaptation (ParetoDA) approach, which cooperatively optimizes training objectives and is specifically designed for DA. Specifically, to minimize Cr. we design
(TCM) loss Lr by the mutual information 17 leveraging the target predictions. In this loss, target predictions act as (soft) labels for themselves. Therefore, the accuracy of the predictions is important to support the mimicking. To refine the predictions by maximally exploiting the information at hand, we propose target prediction refining mechunism which models the domain label as conditional information into the prediction probability by Bayes' theorem. Intuitively, we ensembles class-wise domain discriminator (trained with the domain labels)

---

as another classifier- only when the class condition is correct, the discrininator can correctly predict the domain. On the other hand, to weight Lr, and En to guide the cooperative optimization towards minimizing LT we propose dynamic preference mechanism to model the gradient of Cr on a held-out unlabeled target dataset as the guidance. We evaluate the performance on the held-out data because 1) it suggests generalization performance, and 2) it is more stable and convincible than training data. One may be concerned that involving the gradient the held-out data in training may eventually cause over-fitting of the data. Fortunately, our theoretical analyses show that the held-out data can guide but will not be over-fitted by the optimization our method. Our contributions are:
We rethink the optimization scheme in existing DA methods, which can only reach restricted solutions if the optimization objectives conflict with each other. And existing gradient-base strategies do not suit DA.
We propose a Pareto Domain Adaptation method to obtain the desired Pareto optimal solution learned and dynamically guided (using the held-out unlabeled target data) by the designed surrogate objective function that mimics target classification. To better support the mimicking, we also propose target prediction refining mechanism. As a general technique, ParetoDA can be plugged into various DA methods and enhance their performance.
We evaluate ParetoDA on two DA applications: image classification and semantic segmenta- tion. Experimental results demonstrate the effectiveness of ParetoDA.
2 Background
Consider m objectives, each with non-negative loss function (,) Rd R+, E [m] (1.2....,m) m). where 0 is the shared parameters of the objectives to be optimized. There may exist no single solution that reach the optimal solution of each objective at the same time due to they may conflict with each other. What we can obtain instead is a set of so-called Pareto optimal solutions, the follows (58):
related definitions
Pareto dominance. Suppose two solutions 0,, O2 in Rd, we define 01 < O2 if (0, S (02), Vi E [m]] and (,,)) < (02),, E [m]. We say that 02 dominates O₂ iff 01 O2. Note that 02 A 02 if O2 is not dominated by 0j.
Pareto optimality. If a solution 0₁ dominates 02, then 02 is clearly preferable as it perform better or equally on each objective. solution 0* Pareto optimal if it not dominated by any other solutions. That is, 0 0" YO E I
{
Pareto front. The set of all Pareto optimal solutions in loss space is Pareto front, where each point represents a unique solution.
Preference vector. A Pareto optimal solution can be viewed as intersection of the Pareto front with a specific direction in loss space (a ray). We refer to this direction as the preference vector.
In gradient- gradient-based optimization, we can reach a Pareto optimal solution by starting from an arbitrary initialization and iteratively finding the next solution 0 that dominates the previous one 6* ie.. g++ 0' ndddes. where 17 is the step size and ddes is the optimization direction. Let 9: C; the gradient of the i-th objective function at 0, the descent direction dats given by dh 2 0, Vi E [m]. In other words, the descent direction daes should optimize or not damage each objective. Thus, moving against dltss, starting from amounts When d' 0, the optimization process ends, implying that direction can further optimize the all the objectives simultancously. At this moment, if we still want to improve the performance of a specific objective, other objectives will deteriorate.
decrease in the objective values.
This paper focuses on dynamically searching for desired Pareto optimal solution with the aid of the proposed TCM loss in DA problems, aiming to enhance the performance on the target domain.
3 Related Work
Domain Adaptation aims to transfer knowledge across domains. The theory proposed by Ben-David et al. 2 reveals that the key ideology of DA is to diminish the domain gap between two domains, enabling successful transfer. Existing DA methods mainly mitigate domain shift via minimizing
3

---

some statistic metrics 26 56 28, 44 or adversarial training II| 27 20 22 To name few,
DAN 226 mitigates the layer-wist wise discrepancy via Maximum Mean Discrepancy (MMD) metric.
MDD 56 introduces Margin Disparity Discrepancy with rigorous generalization bounds to measure the diserepancy between domains. On the other hand, inspired by GANs [4, DANN [II] introduces a domain discriminator to enforce the features from two domains to be indistinguishable. CDAN (27
further extends this idea by integrate classification information into domain discriminator, leading to precise domain alignment. By contrast, we adopt domain discriminators to refine the target predictions with domain labels. In addition, these methods directly optimize the linear combination of all objectives, while overlooking on the potential conflict between them. In such case, the linear scheme is hard to tune and may only access to restricted solutions on the convex part of pareto front 4.. We propose gradient- baseed optimization scheme to search the desirable Pareto optimal solution on the entire pareto front without weight hyper-parameters, which is orthogonal to prior DA methods.
WeighttBassel Optimization is an intuitive strategy to optimize multiple objectives at the same time, which unifies all the training objectives using weight hyper-parameters to form an overall objective. Since it is hard to manually assign proper weights in prior, many DA methods conduct Deep Embedded Validation [53] to select weight hyper-parameters. Recently, some methods propose to dynamically weight the objectives leveraging gradient magnitude [6]. the rate of change in losses (25,, task uncertainty [1]], or learning non-linear loss combinations by implicit differentiation [3]]. However, these weight- based methods focus seeking balanced solution, which is not bound to be optimal for target classification in DA. By contrast, we explore the gradient- baaeed optimization, aiming to find an optimal solution for DA problems.
Multi- Kujti-Ohjecttve Gradient-Based Optimization leverages the gradients of objectives to reach the final Pareto optimal solution. MGDA (1 proposes to simply update the shared network parameters along a descent direction which leads to solutions that dominate the previous one. Nonetheless, MGDA may reach any Pareto optimal solution since it has extra control for the optimization direction. To ameliorate this problem, PMTL 23 splits the loss space into separated sub-regions based on several preference vectors, and achieve Pareto optimal solutions in each sub-region of the Pareto front. Further, EPO [30] can reach a exact Pareto optimal solution by enforcing the decent direction correspond a specific preference vector. However, prior knowledge of which Pareto solution is the optimal one for target classification is often unavailable in DA. In this paper, we propose PA algorithm to dynamically searching the desirable Pareto optimal solution for target domain guided by a introduced TCM loss on a held-out target data.
4 Pareto Domain Adaptation
4.1 Decompose DA
Domain adaptation (DA) aims to train model that performs well on an unlabeled target domain with the assistance of labeled source domain. Formally, in DA, we can access source domain N, D== = {ar Mt li=1 with N, labeled instances and a target domain D; = {œt J i=1 with N, unlabeled samples, where y E {1,2,..., ..., K} is the corresponding label of the source data æt. Both source and target domains share one identical label space Vd that contains K classes
Existing DA methods are mainly devoted to establishing domain aligmment by explicitly minimizing the domain discrepancy or adversarial training, which have obtained promising results. Discrepancy- based methods attempt to find metric that can precisely depict the domain discrepancy, then adopt it minimize the domain gap. While adversarial-baa a feature extractor and a discriminator, where the feature extractor tries to fool the discriminator, leading to comain-invarian features. All in all, these approaches can be decomposed as follows:
methods play a mini-max game between
Discrepancy-basede min Ls + LD
0.
Adversarial- based: min L. + LD, max LD. 0.0
(2)
&a
where Ls is the source classification loss, Ln is the domain alignment loss, and 0, o.., &a are the
parameters of shared feature extractor, classifier and domain discriminator, respectively.
However, the optimization of Ln may conflict with that of Ls due to the domain shift, under which circumstances the de facto optimization scheme can only access restricted Pareto optimal solutions.

---

Moreover, our goal is to minimize the target classification loss LT, but neither Ls or Lp directly corresponds to this goal. Therefore, to search for a desirable Pareto optimal solution for this goal, we propose a gradient-based optimization method for DA. Concretely, we first design a novel surrogate loss that mimics target classification loss, and then leverage the gradient of it on held-out target dataset to guide the cooperative optimization of objectives.
4.2
Gradient-Based Optimization for DA
4.2.1
Target-Prediction Refining Mechanism
The goal of ParetoDA is to cooperatively optimize the multiple objectives in DA, aiming minimize the target classification loss CT. which is inaccessible due to the absence of target labels. To tackle this issue, design TCM loss Cr by the mutual information [I7] leveraging the target predictions. In this loss, target predictions act as (soft) labels for themselves. Therefore, the accuracy of the predictions is important to support the mimicking.
Former methods generally adopt the outputs of classifier as target predictions, without considering that the domain labels might be beneficial for classification. To refine the predictions by maximally exploiting the information at hand, we propose to model the domain label as conditional information into the prediction probability by Bayes' theorem. Formally, the class-conditional probability d, æ, 0, &e) instead of p(y = k x, x,0, Φc), where the class
of sample æ is p(y k f1,..., }} and the domain label d E {0, ). K is the number of classes, 0 and 1 denote the source and target domain respectively. Based on Bayes' theorem, we have:
p( = d y = k. x, 0, 6.) y = k x. O (.)
ply = k d, 0,
Σx p(= = d
K. 0, 6.) ppyppy k/ x., 0,0 pl = d x. v p(y - x. 0 (.) upy )p
(3)
ΣN p(z = d x, U.
x, 0, Φc)
Pkld-
Here, we adopt multi-layer perception (MLP) network to model p(2 d x, Ux). called class-wise discriminator, where is denotes the parameters of the k-th domain discriminator corresponding to class k, which aims to distinguish the domain label of the samples from class k. Pksjd can be p(s=d
regarded as a multiplicative ensemble of two predictions: Ev p(z=d[a,u..) and p(.==klz.0.. (.. We interpret how the former prediction works for target classification: for one specific class-wise discriminator, it can distinguish the target domain from the source well only when the input samples contain the corresponding class information, i.e., belong to the corresponding category. Otherwise, the probability of predicting samples as the target will be low.
We train all the class-wise discriminators with (derivations are in the supplementary material):
K
min UK
- 1= d=n
skjal(= = d) logp(= = d xx vx))
(4)
Note that the Syjd here only acts as a weighting scalar, and for source domain skid = I(y = k) is hard class label (ground truth label) while for target domain, Skjdd = Pkid is soft class label (refined prediction). This objective from an Exspetation-Maxiiiziinn
derivation.
Now we acquire refined predictions Pkja for target samples which can substitute for the original predictions. As discussed in [12] 42), the ideal target predictions should be individually certain and globally diverse, which can be obtained by maximizing the mutual information between input samples and the predictions. Thus, adopt the information maximization (IM) loss [17) our improved version prediction Pkjd: the formulation is as follows:
K
K
CT
Σ
Pap logpen - EZED. Σ Pki logpaji
(5)
where Pki denotes p(y = k 2= 1, x, 0), i.e., the probability that the target sample x is classified into class k under the prior condition that it is divided into target domain, and وx denotes D.Pe. represents the mean of the k-th elements in the outputs of the whole target domain.
Hence, we take Lr as the surrogate to mimic the target classification. This loss can be directly added to original DA methods to assist the model adaption to the target domain. More importantly, we leverage it to guide the search for the desirable Pareto optimal solution.
5

---

4.2.2 Dynamic Preference Mechanism
This section introduces how we establish gradient-basee baaeed optimization for DA handle the inconsis- tency among objectives and how we dynamically guide the optimization direction towards the desired Pareto optimal solution.
Given losses Cs, LD and Lr. in each optimization step, we first model the update direction d as convex combination of gradients of the three losses, i.e., d G'u,, where S {wr Em Wy = 1), m = 3, and G = (VLs, VoLD; VoLr]. where 0 are the shared parameters, i.e., the parameters of the shared feature extractor. A fundamental goal of gradient-based optimization to find a d to simultaneously minimize all the losses. That is, the network can be optimized along consistent direction and learn the shared knowledge between source and target domains efficiently and stably. Therefore, the direction d first need to satisfy the constraint d' 9) 2 0, Vj Jj is the j-th column of G.
RTl 2==1
[ma], where
Recently, a state-of-art method EPO 30 further controls the direction to reach a Pareto optimal solu- tion with designated trade-off among objectives, using a preference vector fixed apriori. Concretely, EPO guides the optimization direction via minimizing the KL divergence between normalized loss vector weighted by the preference vector and a uniform probability vector. Since the prior knowledge for the trade-off' among objectives in DA is often unavailable and may vary with methods and training procedures, to weight Lr, Ls and LD to guide the cooperative optimization towards minimizing Cr,, propose model the gradient of Lr on held-out unlabeled target dataset as the optimization guidance. We evaluate the performance on the held-out data because 1) it suggests generalization performance, and 2) it is more stable and convineible than training data. In practice, we randomly split 10% data from the original target set as the validation set and the rest 90% are taken as the training set. Note that do not use any ground truth labels on the validation set, instead calculate the proposed TCM loss on it, and then leverage this validation loss to guide the optimization direction.
Formally, we denote by Lv the TCM loss on the held-out data, then we can directly obtain the gradient descent direction: 9. = V VaL. Thus we can replace the guidance direction dhai [ in EPO by 9e as dynamical guidance of the optimization direction. For clarity, the optimization problem can be formulated as a linear programming (LP) problem:
wt = arg max (Gw)² (I > 0)g. + I(Cval = O)G1/m), wES"
(Gw) T 9j 2 I(J + 0)(TT))) g, J
(6)
s.t.
(Gw) T gj 2 0, Vj
Σ 1= w = 1). 1(.) is an indicator function, E R is a vector whose elements are all J. (اlي] 93 > = g; < 0}. and J = (j o 9, maxy 9 g One may be concerned that involving the gradient of the held-out data in training may eventually cause over-fitting of the data. Fortunately, we use the following result to show that the held-out data can guide but will not be over-fitted by the optimization of our method.
where St" = (w E RT
Theorem 1. Let w be the solution of the problem in Eq. (6), and d' = Gw" be the resulted update direction. If Lv 0, then the dominating direction d' becomes descent direction, i.e.,
(d') gj 20, Vj E (1,2,3)
On the other hand, if LV alid 0, Let y (d g. be the objective value of the problem in Eq. (6) Then,
if 7" > 0, then (d') 'd'TTT. g. > 0;
(8)
if 7" S 0, then (d')T g, 2 0, Vj E (1,
The proof of Theorem is deferred to the supplementary material. Note that we add small c le-3 to relax the condition Ly > 0 and L = 0 in Eq. (6) as L > c and L S in practice, since the guiding significance of L becomes trivial when it is less than a small value.
See 30 for details.

---

Now we interpret the learning mechanism in Eq. (6. When Lv 0, the learning is in the pure descent mode, in which the
Algorithm Update Equations for ParetoDA
1: Input: Shared parameters O R step size 1)
update direction d' = Gw" approximates the mean gradient G1 /m, and d' can de- crease all the training losses simultancously. Whereas when LV > O, the learning is in the guidance descent mode, in which approximates P.. Since d' = Gw" is convex combination of columns in G, then Y > 0 means that 9. is consistent with some columns in G, and d' is forced to decrease the loss whose gradient the most consistent with g means the highest contribu- tion to the model performance. On the other hand, >" I O means none of the columns in G is consistent with g.. In this case, d' is not forced to consistent with 9.. but only required to decrease the training losses, while approximating 9u to the maximum feasible extent.
2: Calculate objective values: Ls, Ln. LT
3: Calculate guidance loss on target validation set: Leat (using Eq. (5))
4: Calculate the gradients of training objectives:
G-[91. 92, Sa g Ls, Ver CrrCrl
Calculate the gradient of the guidance loss:
9. Vo Leal
6: Obtain w² by solving LP (Eq. 6
7: Optimization direction: d'
Gw"
Output: Updated parameters l 0 nd
In summary, ge can dynamically guide the optimization direction towards the desired Pareto solution, but will not over-fit fit the held-out data, and no longer need prior knowledge set a preference vector. The theoretical guarantee for the existence of a Pareto optimal is deferred to the supplementary material. And the updating algorithm of every training iteration is presented in Alg, Note that this optimize scheme is only applied to the shared parameters.
Time Complexity. Let m Z be the number of objective, and d E Z, the dimension of model parameters. The computation of (Gw)² gu of (Gw)² o G both has run time O(m2d). With the best LP solver 77). our LP (Eq. 6). that has variables and at most 2m + 1 constraints, has runtime of O' (m² 2.38
). As in DA, we usually have >> m, our total run time is O(m²d + +m..35 = (d).
Theoretical Insight. In Domain adaptation literature, the theory proposed by Ben-David et al. 2 provides an upper bound for the generalization error on target data, which composed of three terms: 1) the expected error on source domain es (h): 2) the H HAH divergence between two domains dHAH(Ds, Da); 3) the combined error À of the ideal joint hypothesis h". The formulation is:
er (h) S es() 2 ZHHSH (D,, D:) + X,
where A = <c(())- + er(h") and h-" = arg minne (h(h) + (h))
The goal of DA methods is to lower the upper bound of the expected target error. (f) is expected to be small thanks to the supervision in source domain. And À is also generally considered sufficiently small 41 27 Thus, most DA methods mainly focus on reducing dи (D,, D:) by domain alignment. However, one of the three terms goes down and another term goes up, the model still not effectively reduce the upper bound of ey (h). Excessive either source classification or domain alignment will cause damage to the other.
In our ParetoDA, by adopting gradient-based optimization, we can simultaneously optimize the source classification loss and the domain alignment loss, so as make (s(h) and dHAH (D,, D.) decrease synchronously. Moreover, both the training of our proposed TCM loss target data and the optimization guidance of this loss on held-out data contribute to the classification ability of the model to the target domain, and then reduce the combined error X of the ideal joint hypothesis h Consequently, the upper bound ey (h) can be effectively reduced our work, which validates the effectiveness of ParetoDA theoretically.
Experiment
5.1 Dataset and Setup
We evaluate ParetoDA on three object classification datasets and two semantic segmentation datasets. Office-31 [4 is typical benchmark for cross-domain object classification involving three distinct
20 is used hide o()) and (1) (1/5) factors, 8 being the relative accuracy. See 7 for details.
7

---

Table Accuracy($e (%) on Office-Home for unsupervised DA (ResNet-50)
Method
Ar:CI
Ar.Pr
Rr Cl:Ar
Cl:Pr
CERw
Pr:Ar
Pr:C1
Pr.ww
Rw:Ar Rw:CI
Rw:Pr
Avg.
ResNet
34.9
50.0
58.0
31.4
41.9
46.2
38.5
31.2
60.4
53.9
41.2
59.9
46.1
DAN 26
43.6
57.0
67.9
45.8
60.4
44.0
43.6
67.7
63.1
51.5
74.3
56.3
TAT 24
51.6
69.5
75.4
59.4
69.5
68.6
59.5
50.5
76.8
70.9
56.6
81.6
65.8
TPN 35
51.2
71.2
76.0
65.1
72.9
72.8
55.4
48.9
76.5
70.9
53,4
80.4
66.2
BNM 9
52.3
73.9
80.0
63.3
72.9
74.9
61.7
49.5
79.7
70.5
53.6
82.2
67.9
MDD [56]
54.9
73.7
77.8
60.0
71.4
71.8
53.6
78.1
72.5
60.2
82.3
GSP [51
56.8
75.5
78,9
61.3
69.4
74.9
61.3
52.6
79.9
73.3
54.2
83.2
68.4
DANN 11
45.6
59.3
70.1
47.0
58.5
60.9
46.1
43.7
68.5
63.2
51.8
76.8
57.6
+BSP S
51.4
68.3
75.9
56.0
67.8
68.8
57.0
49.6
75.8
70.4
57.1
80.6
64.9
+Meetaliggn 50
48.6
69.5
76.0
58.1
65.7
68.3
54.9
44.4
75.3
68.5
50,8
80.1
63.3
+ParetoDA
55.2
74.4
79.0
61.9
72.4
72.9
55.8
81.1
74.4
61.1
82.4
CDAN 27
50.7
70.6
76.0
57.6
70.0
70.0
57.4
50.9
77.3
70.9
56.7
81.6
65.8
+BSP S
52.0
68.6
76.1
58.0
70.3
70.2
58.6
50.2
77.6
72.2
59.3
81.9
66.3
+MeeaAlign 550
55.2
70.5
77.6
61.5
70.0
70.0
58.7
55.7
78.5
73.3
61.0
81.7
67.8
+ParetoDA
56.8
75.9
80.5
64.4
73.7
65.6
55.2
81.3
75.2
61.1
83.9
70.6
domain: Amazon (A), DSLR (D) and Webcam (W); Office-Home [48] is a more difficult benchmark composed by four image domains: Art (Ar), Clip Art (CI), Product (Pr) and Real-World (Rw). The images of different domains are more visually dissimilar with each other; VisDA-2017 [37] is large-scale synthetie-to-rea dataset, containing over 280K images across 12 categories; Cityscapes 8 is a real-world semantic segmentation dataset with 5,000 urban scenes which are divided into training, validation and test splits: GTA5 38 includes 24,966 game screenshots from the game engine of GTA5. Following [46] 20) establish transfer semantic segmentation scenario: GTAS + Cityscapes. Specifically, we use the shared 19 categories and report the test results on the validation set of Cityscapes. To evaluate our approach,
apply the proposed ParetoDA former DA methods, i.e. DANN, CDAN, and MDD, and split 10% data randomly from the original target set as validation set. All experiments stochastic gradient descent (SGD) optimizer with momentum 0.9 and weight decay lc-4 for all experiments. Following the standard protocols in [27 29 we take the average classification accuracy and mean loU (mloU) for image classification and semantic segmentation as evaluation metrics, respectively. More details presented in the supplementary material due to limited space.
implemented via Py PyTorch [36]. and adopt the common-used
5.2 Experimental Results
Office-31. From the results in Table
Table 1: Classification accuracy (%) on Office-31 (ResNet- 50).
1 we can observe that DA methods, such as DANN and CDAN exceed the ResNet model. This validates that mit- igating the domain gap between two domains crucial for enhancing the generalization performance of models. We evaluate ParetoDA on three basic DA methods. The results show that ParetoDA can consistently improve the performance of the basic methods significantly. particular, ParetoDA improves DANN by 8.0%, revealing that ParctoDA is an extra booster to former DA methods. On the other hand, the encouraging results validate the importance of searching for de- sirable Pareto optimal solution.
Method
A:W
D.W
W:D
A:D
D.W
W:A
Avg
ResNet 11
68.4
96.7
99.3
68.9
62.5
60.7
76.1
DAN 26
80.5
97.1
99.6
78.6
63.6
62.8
80.4
JADA
22
90.5
97.5
100.0
88.2
70.9
70.6
86.1
BNM
91.5
98.5
100.0
90.3
70.9
71.6
87.1
TAT 24
92.5
99.3
100.0
93.2
73.1
72.1
88.4
GSP 51
92.9
98.7
99.8
94.5
75.9
74.9
89.5
DANN 11
82,0
96.9
99.1
79.7
68.2
67.4
82.2
+BSP 5
93.0
98.0
100.0
90.0
71.9
73.0
87.7
+MCC 18
98.6
99.3
74.0
75.0
89.4
+ParetoDA
95.5
98.7
100.0
93.8
76.7
76.3
90.2
CDAN 27
94.1
98.6
100.0
92.9
71.0
69.3
87.7
+BSP S
93.3
98.2
100.0
93.0
73.6
72.6
88.5
+MCC 18
94.7
98.6
100.0
95.0
73.0
73.6
89.2
DPAretoDA
95.0
98.9
100.0
95.4
77.6
75.7
90.4
MDD
56
93.5
98.4
100.0
94.5
74.6
72.2
88.9
+ParetoDA
95.4
98.9
100.0
94.4
76.2
75.8
90.1
Office-Home Compared with Office-31, Office Homice-Homeae is more difficult dataset, whose results are shown in T In this difficult scenario, ParetoDA can still enhance the performance former DA methods, with 11.8% and 4.8% improvement on DANN and CDAN respectively. Based these promising results, we can infer that ParetoDA can reach desirable Pareto optimal solution which beneficial for the performance of models.

---

Table 3:
Accuracy(%) on VisDA-2017 for unsupervised DA (ResNet-101).
Method
plane
bcycl.
bus
car
horse
knile
meycl.
person
plant
sktbrd
train
truck
Avg.
ResNet
15
55.1
53.3
61.9
59.1
80.6
17.9
79.7
31.2
81.0
26.5
73.5
8.5
52.4
DAN
26
87.1
63.0
76.5
42.0
90.3
42.9
85.9
53.1
49.7
36.3
85.8
20.7
61.1
MCD
41
87.0
60.9
83.7
64.0
88.9
79.6
84.7
76.9
88.6
40.3
83.0
25.8
71.9
JADA
22
91.9
78.0
81.5
68.7
90.2
84,1
84.0
73.6
88.2
67.2
79.0
38.0
77.0
TPN 35
93.7
85.1
69.2
81.6
93.5
61.9
89.3
81,4
93.5
81.6
84.5
49.9
80.4
DTA 21
93.7
82.2
85.6
83.8
93.0
81.0
90.7
82.1
95.1
78.1
86.4
32.1
81.5
DANN 11
81.9
77.7
82.8
44.3
81.2
29.5
65.1
28.6
51.9
54.6
82.8
7.8
57.4
+BSP 5
92.2
72.5
83.8
47.5
87.0
54.0
86.8
72.4
80.6
66.9
84.5
37.1
72.1
DA
95.3
82.4
82.9
56.2
92.6
95.0
87.1
81.1
90.2
90.8
87.8
46.7
82.4
CDAN 27
83.2
66.9
83.0
50.8
84.2
74,9
88.1
74.5
83.4
76.0
81.9
38.0
73,9
+BSP
92.4
61.0
81.0
57.5
89.0
80.6
90.1
77.0
84.2
77.9
82.1
38.4
75.9
DA
95.9
82.8
81.3
58.7
93.9
93.7
85.9
83.0
91.9
92.0
87.1
51.8
83.2
Table 4: Semantic segmentation performance mloU (%) on Cityscapes validation set.
road
side.
buil
Wall
Fence
pole
Light
9igo
track
bus
Urain
mbike
bike
Method
mioU
ResNet-
16.8
77.2
21.0
25.5
81.3
53.8
26.4
17.2
25.9
25.3
36.0
AdaSeg/Net
46
86.5 36.0
79.9
23.4
23.3
23,9
35.2
14.8
8
33.3
75.6
58.5
27.6
73.7
32.5
35.4
3.9
30.1
28.1
42.4
Cycada [16
86.7 35.6
80.1
19.8
17.5
38.0
39.9
41.5
82.7
27.9
73,6
64,9
19
65.0
12.0
28.6
4.5
31.1
42.0
42.7
CLAN 22
87.0 27.1 79.6
27.3
23.3
28.3
35.5
24.2 83.6
27.4
74.2
58.6
28.0
76.2
33.1
36.7
6.7
31.9
31.4
43.2
AdvEnt 49
89.9 36.5 81.6
29.2
25.2
28.5
32.3 22.4 83.9
34.0
77.1 51.4
27.9 83.7 29.4
39.1
28.4
23.3
43.8
+ParctoDA
79.6 29.1 83.4 30.4 22.4 32.6 39.2 26.7 82.5 26.5 82.2 63.0 29.8 85,9 36.1
43.5
9.2
29,9
43.7
46.1
FDA [22
90.0 40.5 79.4 25.3 26.7 30.6 31.9 29.3 79.4 28.8 76.5 56.4 27.5 81.7 27.7 45.1
17.0
23.8
29.6
44.6
+ParetoDA
$5.4.37.6 84.0 32.0 23.8 33.1 39.6 25.9 84.0 30.6 81.6 613 30.9 85.4 31.9 45.0
7.6
32.4
45.4
47.2
CBST [3
86.8 46.7 76.9 26.3 24.8 42.0 46.0 38.6 80.7 15.7 48.0 57.3 27.9 24.5 49.6
25.5
45.1
45.2
+ParctoDA
82.4 50.7 80.2 34,9 36.7 45,9 47.2 45.1 72.9 27.8 46.0 62.3 33.0
81.4
29.5
44.3
14.0
29.8
49.3
48.1
VisDA-2017. To validate the model performance on a large-scale dataset, we carry out experiments on VisDA-2017. The results presented in Table 3 As a plug-in module, BSP improves the basic models by enhancing the discriminability of feature representations. ParetoDA aims to improve the optimization for DA methods, which is also orthogonal to former DA methods. In addition, ParetoDA achieves greater performance gains, further indicating the effectiveness and versatility our method.
GTAS + Cityscapes. Compared with classification tasks, segmentation demands more cost for the pixel-level annotation. Hence, it's vital importance to conduct domain adaptation for semantic seg- mentation. We evaluate our method on the adaptive semantic segmentation task, GTAS -> Cityscapes, aiming to transfer the knowledge from synthesized frames to real scenes. The segmentation results exhibited in Table 4 We apply ParetoDA on two methods, i.e., AdvEnt and FDA, which boosts them by large margins on average. This manifests that ParetoDA can still reach an appropriate optimization solution for enhancing model performance
this pixel-level transfer scenario. Besides, ParetoDA is not limited to handle the conflict between the classical source classification loss and domain alignment loss, framework can also be applied to where the explicit domain alignment module doesn't exist or there exist multiple losses. For instance, we adopt ParetoDA on a self self-trainnng based domain adaptation method CBST (59] and it is clear that ParetoDA is still effective.
5.3 Analytical Experiments
Ablation Study. To verify each component of Pare- toDA, we establish ablation study on Office-31. The average results are shown in Table 5 TCM performs better than IM, demonstrating that integrating domain labels is beneficial for target predictions. Applying the conventional Pareto optimization method (30 with fixed preference vector on DANN achieves con- siderable performance gain. This reveals that the gradient- -asedd optimization method superior to the linear weighting scheme. Further, with our proposed ParetoDA, DANN obtains the best results, validating the importance of dynamical preference learning and that TCM loss serves a better guidance for optimization.
Table 5: Ablation Study of ParetoDA
Office- 31 (ResNet-50).
IM TCM EPO ParetoDA Accuracy
DANN
82.2
DANN
d
87.9
DANN
88.4
DANN
87.7
DANN
89.7
DANN
89.9
DANN
90.2

---

Unser Schame
0.06
E20
Pare:D (idea)
0.04
0.03
0.01
70
0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 Source clessification lass
(a) Visualization of optimization path
(b) Ohjective scale sensitivity analysis
Figure Analytical experiments conducted on W-+ (Office-31) based on DANN. (a): Visualization of optimization paths of different optimization schemes. (b): Objective scale sensitivity analysis of ParetoDA by varying the scale factors of objectives.
Optimization Path Visualization. To intuitively verify that ParetoDA can dynamically search the desired Pareto optimal solution under the guidance of our proposed loss, visualize the optimization paths of different optimization schemes (see Fig. ((a)). Note that all these schemes start from the same initial point, but only plot several solutions of the final stage of optimization for better illustration. It is clear that the linear weighting scheme not only deteriorates one of the objectives at some steps, but also reaches solution the convex part of the Pareto front, which deviates severely from the ideal one. The conventional gradient-based optimization method EPO (with fixed preference vector [1, I, 1)) controls the optimization direction to avoid damaging any of the objectives. However, its solution heavily relies on the pre-defined preference vector for training objectives, which ambiguous in DA. By contrast, our proposed ParetoDA further dynamically adjusts the optimization direction with the gradient direction of the proposed TCM loss. For comparisons, we illustrate the ideal solution guided by the supervised classification loss on target domain. One can observe that the optimization direction of ParetoDA is consistent with the ideal one in general, and reach a final solution closest to the ideal one.
The scale of the objectives may vary with methods or datasets. To test the adaptivity of ParetoDA to different objective scales, conduct experiments on a random task W-A (Office-31) based on DANN by varying Ao. λ E {0.1, 0.5, 1.0, 1.5), which represent the scale factor of object classification objective and domain alignment objective respectively. Fig. 3 shows that ParetoDA is not sensitive to the scale variety of objectives, which indicating that Pa is robust across different methods and datasets.
Sensitivity to Different Objective Scales.
]
Adaptivity to Deeper Backbone Networks. In this work, we reach a desirable Pareto optimal solution for the target domain by controlling the optimization direction. Whereas deeper networks, might be more difficult to control the optimization direction due to the increased complexity. For fair comparison, on classification tasks. However, ParetoDA can be easily adapted deeper backbone networks. Table 6 presents the results of ParetoDA based on DANN with different backbones, demonstrating that ParetoDA can still consistently enhance the perfor- mance of the basic method.
ResNet-50
Table 6: Test top-1 accuracy (%) on task W-A (Office-
31) of methods with different backbone networks.
Networks
Source-only
DANN [T]]
+ParetoDA
ResNet-50
60.7
67.4
76.3
ResNet- 101
65.8
74.5
77.5
ResNet- 152
68.2
76.1
79.0
Conclusion
In this paper, we rethink the optimization scheme for DA from a gradient-based optimization perspective. We present Pareto Domain Adaptation (ParetoDA) to dynamically search a desirable Pareto optimal solution for boosting model performance, which is orthogonal to former DA methods. Extensive experiments on both classification and segmentation tasks validate the effectiveness of PA worth noting that we mainly concentrate on the traditional domain adaptation setting for clarity, while the generalization to other variants of DA needs to be further explored.
10

---

Acknowledgments
This work was supported by the National Natural Science Foundation of China (61902028).
Funding Transparency Statement
Funding in direct support of this work: National Natural Science Foundation of China (61902028).
References
[1] Multiple-gradient descent algorithm (mgda) for multiobjective optimization. Comptes Rendus Mathematique, 350(5))31-338, 2012.
[2] S. Bendavid, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan. A theory of learning from different domains. MIJ, 79(1-2):151-175, 2010.
[3] K. Bousmalis, G. Trigeorgis, N. Silberman, D. Krishnan, and D. Erhan. Domain separation networks. In NeurlPS, pages 343-351, 2016.
[4] P. Boyd and Vandenberghe. Corvex Optimization. Cambridge University Press, 2014.
[5] X. Chen, S. Wang, M. Long, and J. Wang. Transferability vs. diseriminability: Batch spectral penalization for adversarial domain adaptation. In ICML, pages 1081-1090, 2019.
[6] Z. Chen, Badrinarayanan, Lee, and A. Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In J. G. Dy and A. Krause, editors, ICML, pages 793-802, 2018.
[7] M. B. Cohen, Y. T. Lee, and Z. Song. Solving linear programs in the current matrix multiplication time. In SOTC, pages 938-942, 2019.
[8] M. Cordts, M. Omran, Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, pages 3213-3223, 2016.
[9] S. Cui, S. Wang, J. Zhuo, L. Li, Q. Huang, and T. Qi. Towards discriminability and diversity: Batch nuclcar-norm maximization under label insufficient situations. CoRR, abs/2003. 122337 2020.
[10] M. Ehrgott. Multicriteria Optimization (2. ed.). Springer, 2005.
[11] Y. Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation, In ICML, pages 1180-1189 1100-111922000100 2015.
[12] R. Gomes, A. Krause, and P. Perona. Discriminative clustering by regularized information maximization. NeurlPS, pages 775-783, 2010.
[13] B. Gong, Y. Shi, F. Sha, and K. Grauman. Geodesic flow kernel for unsupervised domain adaptation. In CVPR, pages 2066 1066-2073, 2012.
[14] I. Goodfellow, J. Pouget-Abadie M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NeurIPS, pages 2672-2680. 2014.
[15] K. He, X. Zhang, S. Ren, and Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.
[16] J. Hoffman, E. Tzeng, T. Park, J.-Y. Zhu, Isola, K. Saenko, A. Efros, and T. Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In JCMI., pages 1994-2003, 2018.
[17] W. Hu, T. Miyato, S. Tokui, E. Matsumoto, and M. Sugiyama. Learning discrete representations via information maximizing self-augmented training. ICMI,, volume 70, pages 1558-1567. PMLR, 2017.
11

---

[18] Y. Jin, X. Wang, M. Long, and J. Wang. Minimum class confusion for versatile domain adaptation. In ECCV, volume 12366, pages (644480, 2020.
[19] A. Kendall, Y. Gal, and R. Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In CVPR, pages 7482-7491, 2018.
[20] C.-Y. Lee, T. Batra, M. H. Baig, and D. Ulbricht. Sliced wasserstein discrepancy for unsuper- vised domain adaptation. In CVPR, pages 10285-10295, 2019.
[21] S. Lee, D. Kim, N. Kim, and S. Jeong. Drop adapt: Learning discriminative features for unsupervised domain adaptation. In ICCV, pages 91-100, 2019.
[22] S. Li, C. H. Liu, B. Xie, L. Su, Z. Ding, and G. Huang. Joint adversarial domain adaptation. In ACM MM, pages 729-737, 2019.
[23] X. Lin, H. Zhen, Li, Q. Zhang, and S. Kwong. Pareto multi-task learning. In Newr/PS, pages 12037-12047, 2019.
[24] H. Liu, M. Long, J. Wang, and M. I. Jordan. Transferable adversarial training: A general approach to adapting deep classifiers. In ICML, pages 4013-4022, 2019.
[25] S. Liu, E. Johns, and A. J. Davison. End-to-end multi-task learning with attention. In CVPR, pages 1871-1880, 2019.
[26] M. Long, Y. Cao, Wang, and M. Jordan. Learning transferable features with deep adaptation networks. In ICML, pages 97-105, 2015.
[27] M. Long, Z. Cao, J. Wang, and M. I. Jordan. Conditional adversarial domain adaptation. In NeurlPS, pages 1647-1657, 2018.
[28] M. Long, H. Zhu, J. Wang, and M. Jordan. Deep transfer learning with joint adaptation networks. In ICML, pages 2208-2217, 2017.
[29] Y. Luo, L.. Zheng. T. Guan, J. Yu, and Y. Yang. Taking closer look domain shift: Category- level adversaries for semantics consistent domain adaptation. In CVPR, pages 2502-2511, 2019.
[30] D. Mahapatra and V. Rajan. Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. In /CMI., pages 6597-6607, 2020.
[31] A. Navon, I. Achituve, H. Maron, G. Chechik, and E. Fetaya. Auxiliary learning by implicit differentiation. CoRR, abs/2007. abs/2007.02663, .02993,,660 2020.
[32] A. Navon, A. Shamsian, G. Chechik, and E. Fetaya. Learning the pareto front with hypernet- works. In ICLR, 2021.
[33] S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang. Domain adaptation via transfer component analysis. TNNLS, 22(2):199-211 2200 2011.
[34] S. Pan and Q Yang. A survey transfer learning. TKDE, 22(10)) 2010.
[35] Y. Pan, T. Yao. Y. Li, Y. Wang, C.-W. Ngo, and T. Mci. Transferrable prototypical networks for unsupervised domain adaptation. In CVPR, June 2019.
[36] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In NeurlPS, pages 8024-8035, 2019.
[37] X. Peng, B. Usman, N. Kaushik, J. Hoffman, Wang, and K. Saenko. Visda: The visual domain adaptation challenge. CoRR, abs/1710.06924, 2017.
[38] S. R. Richter, V. Vineet, S. Roth, and V. Koltun. Playing for data: Ground truth from computer games. In ECCV, pages 102-118, 2016.
[39] A. I.. Rodriguez and K. Mikolajezyk. Domain adaptation for object detection via style consis- tency. In BMVC, page 232. BMVA Press, 2019.
12

---

[40] K. Saenko, B. Kulis, M. Fritz, and T. Darrell. Adapting visual category models to new domains.
In ECCV, pages 213-226, 2010.
[41] K. Saito, K. Watanabe, Y. Ushiku, and T. Harada. Maximum classifier discrepancy for unsuper- vised domain adaptation. In CVPR, pages 3723-3732, 2018.
[42] Y. Shi and F. Sha. information-thhoreiice learning of discriminative clusters for unsupervised domain adaptation. In JCML, 2012.
[43] S. Song, H. Yu, Z. Miao, J. Fang. K. Zheng, C. Ma, and S. Wang. Multi-spectral salient object detection by adversarial domain adaptation. In AAAL, pages 12023-12030, 2020.
[44] B. Sun, Feng, and K. Saenko. Return of frustratingly easy domain adaptation. In AAAI, pages 2058-200 2065, 2016.
[45] B. Sun and K. Saenko. Deep coral: Correlation alignment for deep domain adaptation. In ECCV, pages 443-450. Springer, 2016.
[46] Y. Tsai, W. Hung. S. Schulter, K. Sohn, M. Yang, and M. Chandraker. Learning to adapt structured output space for semantic segmentation. In CVPR, pages 7472-7481 7481, 2018.
[47] R. Tzeng, Hoffman, N. Zhang, K. Saenko, and T. Darrell. Deep domain confusion: Maximiz- ing for domain invariance. CoRR, abs/1412.3474, 2014.
[48] H. Venkateswara, J. Eusebio. S. Chakraborty, and S. Panchanathan. Deep hashing network for unsupervised domain adaptation. In CVPR, pages 5018-5027, 2017.
[49] T. Vu, H. Jain, M. Bucher, M. Cord, and P. Perez. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In CVPR, pages 2517-2526, 2019.
[50] G. Wei, C. Lan, W. Zeng, and Z. Chen. Metaalign: Coordinating domain alignment and classification for unsupervised domain adaptation. CoRR, 0lx/2103.13575. 2021.
[51] H. Xia and Z. Ding. Structure preserving generative cross-domain learning. In CVPR, pages 4363-4372, 2020.
[52] Y. Yang and S. Soatto. FDA: fourier domain adaptation for semantic segmentation. In CVPR. pages 4084-4094, 2020.
[53] K. You, X. Wang. M. Long, and M. Jordan. Towards accurate model selection in deep unsuper- vised domain adaptation. In ICML, pages 7124-7133, 2019.
[54] W. Yun, B. Han, J. Lee, J. Kim, and J. Kim. Target-style-aware unsupervised domain adaptation for object detection. IEEE Robotics Autom. Lett., ((2):3825-3832, 2021.
[55] K. Zhang, B. Schölkopf, K. Muandet, and Z. Wang. Domain adaptation under target and conditional shift. In ICML, pages 819-827, 2013.
[56] Y. Zhang, T. Liu, M. Long, and M. Jordan. Bridging theory and algorithm for domain adaptation. In ICML, pages 7404-7413, 2019.
[57] H. Zhao, S. Zhang, G. Wu, M. F. Moura, Costeira, and G. J. Gordon. Adversarial multiple source domain adaptation. In NeurIPS 2018, pages 8568-8579, 2018.
[58] E. Zitzler and L. Thiele. Multiobjective evolutionary algorithms: comparative study and the strength pareto approach. IEEE Trans. Evol. Comput., 344):257-271, 1999.
[59] Y. Zou, Z. Yu, B. V. K. V. Kumar, and J. Wang. Unsupervised domain adaptation for semantic segmentation via Blss-balanced self-training. In ECCV, volume 1 11207, pages 297-313, 2018.
13


Reviews:
[{"review_id": "T7Pkhyq-nCz", "reviewer": "Reviewer_MkjU", "summary": "This paper represents a domain adaptation (DA) problem in a multi-objective optimization scheme, which can be solved by a gradient-based perspective. The main contributions of the paper include:\n\n1. Proposes a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to optimize all training objectives. That ParetoDA scheme can be plugged into various DA methods and enhance their performance. \n2. To find a desired Pareto optimal solution, the authors design a surrogate objective function that mimics the target classification task. \n3. Provides experimental results that show the effectiveness of ParetoDA compared to some DA methods.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Quality: The idea of representing the domain adaptation setting to a multi-objective optimization scheme is interesting. However,  I'm a bit concern about the theoretical contributions of the paper since the method is quite straightforward. Moreover, there is no theoretical guarantee for the existence of a Pareto optimal of the corresponding multi-objective optimization problem by using the proposed ParetoDA. Also, it would be much more interesting if the authors can develop an upper bound for the generalization error of the DA.\n\nClarify: Here are several critical points:\n- It's unclear for me about the formulation of the loss $\\mathcal L_T$ in (5). Why minimizing that loss can handle the domain-shift in the domain adaptation problem? That's important since it can verify the main contribution of the paper.\n- (Line 218) It's unclear how to select the validation set $X_t^{Val}$. \n- The linear programming (6) misses the condition $\\sum w=1$? Does this affect the solution of (6), and consequently the performance of the DA framework?\nOriginality: To the best of my knowledge, this paper is the first work that introduces the use of multi-objective optimization in a domain adaptation problem.\n\nSignificance: Based on my above comments and evaluations, I personally think that it's a bit below the acceptance rate of a NeurIPS publication.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "R3pa0WWp6hO", "reviewer": "Reviewer_DGKA", "summary": "While doing domain adaptation, we usually have a source classification loss and a domain alignment loss. However, these losses might compete with each other and we might reach a situation where one loss can’t be reduced without degrading the other one. We call that the Pareto front. For DA, we don’t know which loss is better than the other one. In this paper, they come up with an optimization strategy that helps us navigate to an ideal portion on the Pareto front. Essentially, they stop the optimization if one loss can’t be improved without degrading the other.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- Originality: 1. Seems to be a modified version of EPO where the only difference is that this one is specifically designed for DA. This one also uses a custom surrogate loss for DA while EPO uses KL divergence. 2. They came up with their own equations for 3. They claim that Pareto optimization hadn’t really been used before for DA, but I found a paper (Locality Preserving Projection for Domain Adaptation with Multi-Objective Learning Le Shu, Tianyang Ma, Longin Jan Latecki) which did. Needs a comparison against that.\n\n- Quality: Well thought out and precise experiments. Technically sound with well-supported claims. However, I am worried we might get stuck in local minima in some situations (especially in the beginning stages of training) with their optimization strategy (losses need to increase sometimes to get out of local minima, but their conditions don’t allow for it). Would appreciate it if they could tell us about their experience wrt that.\n\n- Clarity: The theoretical section 4 is not very well written and could be polished and made simpler, and easier to understand. One thing, I still don’t understand how you determine a solution to be an ideal solution on the Pareto front  (I get that we can’t choose anything on the extreme ends, but where did you get the Expected Solution on Fig. 1 and the ParetoDA (ideal) in Fig. 3). Is that from the intersection of the preference vector from the surrogate loss with the Pareto boundary?\n\n- Significance: Overall, it is a plug-and-play optimization technique for DA which can be easily used with already existing SOTA model architectures to get performance boosts. Could be a valuable tool for DA. Maybe they should talk from the perspective of autoML for DA.\n\nAFTER THE DISCUSSION PERIOD: I would like to thank the authors and reviewers and AC. I think that enough information and clarification have been exchanged so that the major issues have been adequately addressed. One of them was also the comparison with previous work, which the authors have addressed very effectively. I would have given a higher score if fewer things had to be revised for the camera ready, but anyways, I am in favor of accepting the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, they have.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "15"}, "extra_text": {}}, {"review_id": "HdyX2vT6AkF", "reviewer": "Reviewer_rFU1", "summary": "This paper addresses the issue of previous domain adaptation optimization scheme, which learning objectives conflict with each other. The main contribution is to dynamically control the overall optimization direction so that no objective is harmed resulting out to boost the performance. To this end, it designs desired pareto optimal solution with surrogate objective function that mimics the target classification task called Pareto Domain Adaptation.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\n\nThe main originality of this paper seems to come from two methodologies, CIM loss and Pareto optimal solution.  For CIM loss, utilizing domain label to compute class-wise domain prediction to modify the target predictions is thought to be novel.  And then, it shows well comparison with other gradient-based scheme and proves the effectiveness and originality of dynamic pareto optimal solutions.\n\nQuality:\n\nDesigning process of CIM loss and Pareto Optimal solution is well supported by theoretical analysis and final results.\nHowever, the major curiosity that I have is relationship between CIM loss and pareto optimal solution. Ablation study in Table 5. doesn't guarantee that CIM loss is better solution for ParetoDA compared with basic IM loss. It would be more convincing if the author can provide IM + ParetoDA results to prove CIM as a better guidance for the proposed adaptation method.  \nI also wonder whether this framework can be applied to self-training based domain adaptation[1], specifically where the explicit domain alignment module doesn't exist.  \n-  [1]  Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training,  Yang et al.\n\n\nClarity:\n\nThis paper is well written and easy to follow. Specifically, figure in this paper well describes the intention of the author and proves the effectiveness of suggested model.\n\n\nSignificance:\n\nThis paper seems to show quite significant result with boosting improvement  and robustness  to the different tasks and backbone model.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Author well express the limitation and potential negative societal impact in supplemental material.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "HBbD-pQVCT4", "reviewer": "Reviewer_M9tJ", "summary": "The paper considers the common class of domain adaptation algorithms consisting in minimizing the sum of a source risk term and a divergence between the domains. Since both terms are required to be small, the authors examine the tradeoff between them via their associated Pareto front. This latter having non-convex regions non-reachable by the classic weighted sum minimization procedures, a multi-objective optimization procedure is proposed in order to reach all of the regions of the Pareto front and to access tradeoffs that are not accessible by the classic approach. Specifically, the authors use a modified version of a multi-objective optimization procedure, where they define a guidance direction that adjusts the directions of both gradients of the two cost function terms. The guidance direction is the gradient of a surrogate loss assessing performance on the target domain. To show the empirical performance of their idea, the authors present experiments in which their method improves previously proposed DA algorithms.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is clearly written and easy to follow. The idea is an interesting enhancement for DA methods, and the explanations about multi-objective optimization are succinct.\n\n## Strengths\n\n* Bringing multi-objective optimization to domain adaptation, which is an extension to several proposed algorithms.\n* Utilizing the domain discriminator to construct a surrogate for the target domain loss, an original idea that goes beyond the classic role of aligning the two domains.\n* The adaptation of the idea from EPO [6] to the specific case of domain adaptation by modifying the guidance direction ${\\bf d}_{bal}$.\n\n## Weaknesses:\n\n* Although soft-labels are used for the target domain, no comparison to methods using pseudo-labels is used.\n* Not mentioning of the link to methods performing class-wise domain alignment after the optimization problem of Equation (4) (e.g. [5])\n* Related Works section:\n  * It is necessary to mention theoretical DA contributions, at least the seminal ones (e.g. [1,2]). Indeed, the idea of the domain discriminator introduced in [1] has inspired [3], in addition to the more recent GAN paper [4].\n* I believe that there is a problem in the proof of Theorem 1, specifically for the case $\\gamma^{\\*}\\leq 0$ (lines 54 to 57 in the supplementary material). Indeed, we have $({\\bf G}{\\bf w}^*)^T{\\bf g_v} \\leq 0 \\Leftrightarrow \\sum_{i=1}^3w_i{{\\bf g}_i^T{\\bf g_v}}\\leq 0$. This implies that there is some $j \\in \\{1,2,3\\}$ such that   $w_j{{\\bf g}_j^T{\\bf g_v}}\\leq 0$, which further implies that  ${{\\bf g}_j^T{\\bf g_v}}\\leq 0$ if $w_j>0$. This, however, does not imply that $\\forall i \\in {1,2,3}, {\\bf g}_i^T{\\bf g}_v < 0$ (contradicting the statement made in line 55 \" there is no gradient ${\\bf g}_j$, $j \\in  \\{1, 2, 3\\}$, for which ${\\bf g}_j^T {\\bf g}_v \\geq 0$\").\n* Lacking notation definitions:\n  * $\\mathcal L$ (line 87) is not defined.\n  * ${\\mathit{ v}}_{\\mathbf c}$ (Equation set (3)), and $\\lambda_0, \\lambda_1$ (line 338) are not defined  \n  * In Equation (5), $X_t$ is not defined. Is it the finite set of observed target instances ?\n  * The definition of gradients ${\\bf g}_j$ is given in line 222, after their first mention in line 208.\n  * In Equation (4), either a $\\max$ should be instead of $\\min$, or a minus sign has to be added, as the expression is a log-likelihood to be maximized.\n  * In the objective functions in (6), $\\bf 1$ should be defined.\n\n## Comments\n\n* Line 173 \"The probability of the i-th ...\" --> \"The class-conditional probability of the i-th ...\"\n\n### Minor comments:\n\n* Line 151 \". While\" --> Either \", while\", or \". Meanwhile/On the other hand...\".\n* Line 215: \"Chapter\" --> \"Section\"\n* Lines 181-182: \n  * \"containing\" --> \"contain\" (otherwise the sentence seems incomplete).\n  * \"belongs\" --> \"belong\"\n* Supplementary, line 53: \"then $\\gamma^*$\" --> \"then $\\gamma^*>0$\".\n* Line 252: \"$n\\gg m$\" --> \"$d\\gg m$\" ? \n\n## Suggestions\n\n* The \"soft labels\" term for the target domain, mentioned in the supplementary (line 69), offers more intuition about the objective in Equation (4). It should be mentioned in the main paper.\n* I recommend performing the experiments from Figure 3 to other datasets (toy 2d data) .\n* The last paragraph title should contain \"multi-objective\", as \"Gradient-based optimization\" refers to mono-objective first order methods, as multi-objective optimization is not yet commonly adopted by the ML community.\n\n* Although its meaning is given in line 219, the notation ${\\bf g}_v$ can be confused for the column vector of $\\bf G$ that has index $v$. I suggest using another letter for denoting the guidance direction.\n\n## Questions: \n\n* Is the extension to regression immediate ?\n* Can the surrogate target loss be used to select hyper-parameters in an unsupervised way for other DA methods ?\n* Isn't the condition  $\\mathcal L_{val}>0$ too strict in the definition of optimization problem (6) ? Is it beneficial to relax it ? (e.g. $\\mathcal L_{val}>\\varepsilon$ and $\\mathcal L_{val}<\\varepsilon$) for a given small $\\varepsilon$ ?\n* Have you tried to visualize the Pareto front for different domain gaps ? It is interesting to see how it evolves as the gap between the two domains becomes larger.\n\n## References\n\n[1] Ben-David, S., Blitzer, J., Crammer, K. et al. A theory of learning from different domains. Mach Learn 79, 151–175 (2010).\n\n[2] Mansour, Y., Mohri, M., Ro, J., Theertha Suresh, A. &amp; Wu, K.. (2021).  A Theory of Multiple-Source Adaptation with Limited Target Labeled Data . <i>Proceedings of The 24th International Conference on Artificial Intelligence and Statistics</i>, in <i>Proceedings of Machine Learning Research</i> 130:2332-2340\n\n[3] Ganin, Y., & Lempitsky, V. (2015, June). Unsupervised domain adaptation by backpropagation. In International conference on machine learning (pp. 1180-1189). PMLR.\n\n[4] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.\n\n[5] Long, M., Cao, Z., Wang, J., & Jordan, M. I. (2018). Conditional Adversarial Domain Adaptation. *Neural Information Processing Systems*.\n\n[6] Mahapatra, D., & Rajan, V. (2020, November). Multi-task learning with user preferences: Gradient descent with controlled ascent in pareto optimization. In *International Conference on Machine Learning* (pp. 6597-6607). PMLR.\n\n---------\n## Update after authors rebuttal\nI thank the authors for their rebuttal that clarified the different concerns I raised.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "One limitation is stated in the conclusion, which is the restriction to the traditional domain adaptation setting. It would be clearer to specify what is meant by traditional (single-sourced, closed set, etc). The societal impacts are discussed in the supplementary material.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a novel way to solve the Domain adaptation problem by formulating it as a multi-objective optimization problem instead of a sum of objective values. This allows for solving the DA problem with less hyper parameters that are hard to select in practice and can have an important impact fr applications of DA. Despite this original idea the paper had originally  borderline reviews due to some missing explanations and experiments. The authors did a very good job at answering the reviewers concerns with new experiments and even theoretical results. The reviewers agreed after the replies  that the paper is interesting and deserve to be published. 

Please take into account all reviewers comments (missing references and clarifications) and insert the novel results (both numerical and theoretical) in the main paper and supplementary for the final version because the consensus among reviewers is that it will make the paper much better and interesting for the community.
Decision: Accept (Poster)