Pareto Domain Adaptation
Year: 2022

Abstract:
Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to an unlabeled target domain that follows different distribution from the source. To achieve this, DA methods include a source classification objective LS to extract the source knowledge and a domain alignment objective LD to diminish the domain shift, ensuring knowledge transfer. Typically, former DA methods adopt some weight hyper-parameters to linearly combine the training objectives to form an overall objective L. However, the gradient directions of these objectives may conflict with each other due to domain shift. Under such circumstances, the linear optimization scheme might decrease the overall objective value at the expense of damaging one of the training objectives, leading to restricted solutions. In this paper, we rethink the optimization scheme for DA from a gradient-based perspective. We propose a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives. Specifically, to reach a desirable solution on the target domain, we design a surrogate loss mimicking target classification. To improve target-prediction accuracy to support the mimicking, we propose a target-prediction refining mechanism which exploits domain labels via Bayes’ theorem. On the other hand, since prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution on the target domain, we propose a dynamic preference mechanism to dynamically guide our cooperative optimization by the gradient of the surrogate loss on a held-out unlabeled target dataset. Our theoretical analyses show that the held-out data can guide but will not be over-fitted by the optimization. Extensive experiments on image classification and semantic segmentation benchmarks demonstrate the effectiveness of ParetoDA. Our code is available at https://github.com/BIT-DA/ParetoDA.

heading: 1 Introduction
text: Domain adaptation (DA) is a well-established paradigm for learning a model on an unlabeled target domain with the assistance of a labeled related source domain, which has attracted a surge of interests in the machine learning community [34, 33, 13]. By bridging the domain gap on the premise of ensuring the performance of source classification task, DA can adapt the model learned from the source domain to the target in the presence of data bias, which solves the dilemma of label scarcity in many real-world applications [55, 47, 11, 45, 3, 57]. Extensive DA approaches have been proposed in recent years, achieving great performances in many areas such as image classification [11, 27, 56], semantic segmentation [49, 52, 16], and object detection [39, 43, 54].
∗ Equal contributions from both authors. † Corresponding author.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
Most DA methods are devoted to aligning the distributions across domains by explicitly minimizing some discrepancy metrics [26, 28, 56] or adopting adversarial learning [11, 27, 41]. However, since the domain alignment objective also depends on the target domain that deviates from the source, the gradient direction of it may conflict with that of classification objective on the source domain. Thus, when the optimization proceeds to a certain stage, one objective will deteriorate if further improving the other one. Under this circumstance, no solution can reach the optimal of each objective at the same time. We can only obtain a set of so-called Pareto optimal solutions instead, where we cannot further decrease all objectives simultaneously [10]. All Pareto optimal solutions in loss space compose Pareto front as shown in Fig. 1.
𝐿𝐴 ↓, 𝐿𝑆 ↓
Dynamic
𝐿𝐴 Fixed
Dynamic
Preference Vector
(d)
𝐿𝑆
𝐿𝐴
𝑳𝑨 ↓, 𝑳𝑺 ↓
(b) 𝐿𝑆
𝐿𝐴
𝑳𝑨 ↓, 𝑳𝑺 ↓
𝑳𝑨 ↑, 𝑳𝑺 ↓
(a)
min 𝐿 = 𝜆1𝐿𝑆 + 𝜆2𝐿𝐷 min 𝐿 = 𝜆3𝐿𝑆 + 𝜆4𝐿𝐷
𝐿𝐷
This dilemma obviously has been overlooked by most former DA methods, which generally adopt some empirical weight hyperparameters to linearly combine the objectives, constructing an overall objective. This linear weighting scheme has two major concerns. First, as pointed out in [4], it can only obtain solutions on the convex part of the Pareto front of the objectives, while cannot reach the non-convex part, as shown in Fig. 1 (a)-(b). That is because it only considers reducing the overall objective value, which might damage some objectives during optimization (see the red up-arrow). Unfortunately, the Pareto fronts of DA methods are often non-convex, due to loss conflict caused by domain shift. See examples shown in Fig. 2. Second, one could hardly reach the desired solution that performs best on target domain by tuning the weight hyper-parameters, since the obtained solution will deviate severely when the trade-offs slightly change (see the slopes of the dash lines in Fig. 1 (a)-(b) for reference).
To remedy these issues, we rethink the optimization scheme for DA from a gradient-based perspective. By weighting objectives to control the overall optimization direction so that no objective deteriorates, one can reach a Pareto optimal solution on the Pareto front [1, 23, 30, 32]. Moreover, [30] precisely sets objective weights according to a prefixed preference vector (see Fig. 1 (c)). However, neither of them suits DA, because 1) in DA, the goal is to minimize the target classification loss L∗T , but neither LS or LD directly corresponds to this goal; 2) the target domain has no label to construct a classification loss for the goal; 3) prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution that minimizes L∗T . In this paper, we propose a Pareto Domain Adaptation (ParetoDA) approach, which cooperatively optimizes training objectives and is specifically designed for DA. Specifically, to minimize L∗T , we design a target-classification-mimicking (TCM) loss LT by the mutual information [17] leveraging the target predictions. In this loss, target predictions act as (soft) labels for themselves. Therefore, the accuracy of the predictions is important to support the mimicking. To refine the predictions by maximally exploiting the information at hand, we propose a target prediction refining mechanism which models the domain label as conditional information into the prediction probability by Bayes’ theorem. Intuitively, we ensembles a class-wise domain discriminator (trained with the domain labels)
as another classifier—only when the class condition is correct, the discriminator can correctly predict the domain. On the other hand, to weight LT ,LS and LD to guide the cooperative optimization towards minimizing L∗T , we propose a dynamic preference mechanism to model the gradient of LT on a held-out unlabeled target dataset as the guidance. We evaluate the performance on the held-out data because 1) it suggests generalization performance, and 2) it is more stable and convincible than training data. One may be concerned that involving the gradient of the held-out data in training may eventually cause over-fitting of the data. Fortunately, our theoretical analyses show that the held-out data can guide but will not be over-fitted by the optimization of our method. Our contributions are:
• We rethink the optimization scheme in existing DA methods, which can only reach restricted solutions if the optimization objectives conflict with each other. And existing gradient-based strategies do not suit DA.
• We propose a Pareto Domain Adaptation method to obtain the desired Pareto optimal solution learned and dynamically guided (using the held-out unlabeled target data) by the designed surrogate objective function that mimics target classification. To better support the mimicking, we also propose a target prediction refining mechanism. As a general technique, ParetoDA can be plugged into various DA methods and enhance their performance.
• We evaluate ParetoDA on two DA applications: image classification and semantic segmentation. Experimental results demonstrate the effectiveness of ParetoDA.

heading: 2 Background
text: Consider m objectives, each with a non-negative loss function Li(θ) : Rd → R+, i ∈ [m] = {1, 2, ...,m}, where θ is the shared parameters of the objectives to be optimized. There may exist no single solution that can reach the optimal solution of each objective at the same time due to they may conflict with each other. What we can obtain instead is a set of so-called Pareto optimal solutions, the related definitions are as follows [58]:
Pareto dominance. Suppose two solutions θ1,θ2 in Rd, we define θ1 ≺ θ2 if Li(θ1) ≤ Li(θ2),∀i ∈ [m] and Li(θ1) < Li(θ2),∃i ∈ [m]. We say that θ1 dominates θ2 iff θ1 ≺ θ2. Note that θ1 ⊀ θ2 if θ2 is not dominated by θ1. Pareto optimality. If a solution θ1 dominates θ2, then θ1 is clearly preferable as it perform better or equally on each objective. A solution θ∗ is Pareto optimal if it is not dominated by any other solutions. That is, θ ⊀ θ∗,∀θ ∈ Rd − {θ∗}. Pareto front. The set of all Pareto optimal solutions in loss space is Pareto front, where each point represents a unique solution.
Preference vector. A Pareto optimal solution can be viewed as an intersection of the Pareto front with a specific direction in loss space (a ray). We refer to this direction as the preference vector.
In gradient-based optimization, we can reach a Pareto optimal solution by starting from an arbitrary initialization and iteratively finding the next solution θt+1 that dominates the previous one θt, i.e., θt+1 = θt − ηddes, where η is the step size and ddes is the optimization direction. Let gi = ∇θLi be the gradient of the i-th objective function at θ, the descent direction ddes is given by dTdesgi ≥ 0,∀i ∈ [m]. In other words, the descent direction ddes should optimize or not damage each objective. Thus, moving against ddes, starting from θ, amounts to a decrease in the objective values. When dTdesgi = 0, the optimization process ends, implying that no direction can further optimize the all the objectives simultaneously. At this moment, if we still want to improve the performance of a specific objective, other objectives will deteriorate.
This paper focuses on dynamically searching for a desired Pareto optimal solution with the aid of the proposed TCM loss in DA problems, aiming to enhance the performance on the target domain.

heading: 3 Related Work
text: Domain Adaptation aims to transfer knowledge across domains. The theory proposed by Ben-David et al. [2] reveals that the key ideology of DA is to diminish the domain gap between two domains, enabling successful transfer. Existing DA methods mainly mitigate domain shift via minimizing
some statistic metrics [26, 56, 28, 44] or adversarial training [11, 27, 41, 20, 22]. To name a few, DAN [26] mitigates the layer-wise discrepancy via Maximum Mean Discrepancy (MMD) metric. MDD [56] introduces Margin Disparity Discrepancy with rigorous generalization bounds to measure the discrepancy between domains. On the other hand, inspired by GANs [14], DANN [11] introduces a domain discriminator to enforce the features from two domains to be indistinguishable. CDAN [27] further extends this idea by integrate classification information into domain discriminator, leading to precise domain alignment. By contrast, we adopt domain discriminators to refine the target predictions with domain labels. In addition, these methods directly optimize the linear combination of all objectives, while overlooking on the potential conflict between them. In such case, the linear scheme is hard to tune and may only access to restricted solutions on the convex part of pareto front [4]. We propose a gradient-based optimization scheme to search the desirable Pareto optimal solution on the entire pareto front without weight hyper-parameters, which is orthogonal to prior DA methods.
Weight-Based Optimization is an intuitive strategy to optimize multiple objectives at the same time, which unifies all the training objectives using weight hyper-parameters to form an overall objective. Since it is hard to manually assign proper weights in prior, many DA methods conduct Deep Embedded Validation [53] to select weight hyper-parameters. Recently, some methods propose to dynamically weight the objectives leveraging gradient magnitude [6], the rate of change in losses [25], task uncertainty [19], or learning non-linear loss combinations by implicit differentiation [31]. However, these weight-based methods focus on seeking a balanced solution, which is not bound to be optimal for target classification in DA. By contrast, we explore the gradient-based optimization, aiming to find an optimal solution for DA problems.
Multi-Objective Gradient-Based Optimization leverages the gradients of objectives to reach the final Pareto optimal solution. MGDA [1] proposes to simply update the shared network parameters along a descent direction which leads to solutions that dominate the previous one. Nonetheless, MGDA may reach any Pareto optimal solution since it has no extra control for the optimization direction. To ameliorate this problem, PMTL [23] splits the loss space into separated sub-regions based on several preference vectors, and achieve Pareto optimal solutions in each sub-region of the Pareto front. Further, EPO [30] can reach a exact Pareto optimal solution by enforcing the decent direction correspond a specific preference vector. However, prior knowledge of which Pareto solution is the optimal one for target classification is often unavailable in DA. In this paper, we propose ParetoDA algorithm to dynamically searching the desirable Pareto optimal solution for target domain guided by a introduced TCM loss on a held-out target data .

heading: 4 Pareto Domain Adaptation

heading: 4.1 Decompose DA
text: Domain adaptation (DA) aims to train a model that performs well on an unlabeled target domain with the assistance of a labeled source domain. Formally, in DA, we can access a source domain Ds = {xsi , ysi } Ns i=1 with Ns labeled instances and a target domain Dt = {xti} Nt i=1 with Nt unlabeled samples, where ysi ∈ {1, 2, ...,K} is the corresponding label of the source data xsi . Both source and target domains share one identical label space Ys that contains K classes. Existing DA methods are mainly devoted to establishing domain alignment by explicitly minimizing the domain discrepancy or adversarial training, which have obtained promising results. Discrepancybased methods attempt to find a metric that can precisely depict the domain discrepancy, then adopt it to minimize the domain gap. While adversarial-based methods play a mini-max game between a feature extractor and a discriminator, where the feature extractor tries to fool the discriminator, leading to domain-invariant features. All in all, these approaches can be decomposed as follows:
Discrepancy-based: min θ,φc LS + LD (1)
Adversarial-based: min θ,φc LS + LD, max φd LD, (2)
where LS is the source classification loss, LD is the domain alignment loss, and θ,φc,φd are the parameters of shared feature extractor, classifier and domain discriminator, respectively.
However, the optimization of LD may conflict with that of LS due to the domain shift, under which circumstances the de facto optimization scheme can only access restricted Pareto optimal solutions.
Moreover, our goal is to minimize the target classification loss L∗T , but neither LS or LD directly corresponds to this goal. Therefore, to search for a desirable Pareto optimal solution for this goal, we propose a gradient-based optimization method for DA. Concretely, we first design a novel surrogate loss that mimics target classification loss, and then leverage the gradient of it on a held-out target dataset to guide the cooperative optimization of objectives.

heading: 4.2 Gradient-Based Optimization for DA

heading: 4.2.1 Target-Prediction Refining Mechanism
text: The goal of ParetoDA is to cooperatively optimize the multiple objectives in DA, aiming to minimize the target classification loss L∗T , which is inaccessible due to the absence of target labels. To tackle this issue, we design a TCM loss LT by the mutual information [17] leveraging the target predictions. In this loss, target predictions act as (soft) labels for themselves. Therefore, the accuracy of the predictions is important to support the mimicking.
Former methods generally adopt the outputs of classifier as target predictions, without considering that the domain labels might be beneficial for classification. To refine the predictions by maximally exploiting the information at hand, we propose to model the domain label as conditional information into the prediction probability by Bayes’ theorem. Formally, the class-conditional probability of sample x is p(y = k | z = d,x,θ,φc) instead of p(y = k | x,θ,φc), where the class k ∈ {1, . . . ,K} and the domain label d ∈ {0, 1}, K is the number of classes, 0 and 1 denote the source and target domain respectively. Based on Bayes’ theorem, we have:
p(y = k | z = d,x,θ,φc) = p(z = d | y = k,x,θ,φc)p(y = k | x,θ,φc)∑ k′ p(z = d | y = k′,x,θ,φc)p(y = k′ | x,θ,φc)
= p(z = d | x,vk)p(y = k | x,θ,φc)∑ k′ p(z = d | x,vk′)p(y = k′ | x,θ,φc) = ρk|d.
(3)
Here, we adopt a multi-layer perception (MLP) network to model p(z = d | x,vk), called class-wise discriminator, where vk denotes the parameters of the k-th domain discriminator corresponding to class k, which aims to distinguish the domain label of the samples from class k. ρk|d can be regarded as a multiplicative ensemble of two predictions: p(z=d|x,vk)∑
k′ p(z=d|x,vk′ ) and p(y=k|x,θ,φc)∑ k′ p(y=k ′|x,θ,φc) .
We interpret how the former prediction works for target classification: for one specific class-wise discriminator, it can distinguish the target domain from the source well only when the input samples contain the corresponding class information, i.e., belong to the corresponding category. Otherwise, the probability of predicting samples as the target will be low.
We train all the class-wise discriminators with (derivations are in the supplementary material):
min v1,...,vK
− K∑
k=1 1∑ d=0 sk|dI(z = d) log p(z = d | x,vk)|. (4)
Note that the sk|d here only acts as a weighting scalar, and for source domain sk|d = I(y = k) is hard class label (ground truth label) while for target domain, sk|d = ρk|d is soft class label (refined prediction). This objective is from an Expectation-Maximization derivation.
Now we acquire refined predictions ρk|d for target samples which can substitute for the original predictions. As discussed in [12, 42], the ideal target predictions should be individually certain and globally diverse, which can be obtained by maximizing the mutual information between input samples and the predictions. Thus, we adopt the information maximization (IM) loss [17] on our improved version prediction ρk|d, the formulation is as follows:
LT = K∑
k=1
ρ̂k|1logρ̂k|1 − Ex∈Dt K∑
k=1
ρk|1logρk|1, (5)
where ρk|1 denotes p(y = k | z = 1,x,θ), i.e., the probability that the target sample x is classified into class k under the prior condition that it is divided into target domain, and ρ̂k|1 denotes Ex∈Dtρk|1, represents the mean of the k-th elements in the outputs of the whole target domain.
Hence, we take LT as the surrogate to mimic the target classification. This loss can be directly added to original DA methods to assist the model adaption to the target domain. More importantly, we leverage it to guide the search for the desirable Pareto optimal solution.

heading: 4.2.2 Dynamic Preference Mechanism
text: This section introduces how we establish gradient-based optimization for DA to handle the inconsistency among objectives and how we dynamically guide the optimization direction towards the desired Pareto optimal solution.
Given losses LS , LD and LT , in each optimization step, we first model the update direction d as a convex combination of gradients of the three losses, i.e., d = Gw, where w ∈ Sm = {w ∈ Rm+ | ∑m j=1 wj = 1}, m = 3, andG = [∇θLS ,∇θLD,∇θLT ], where θ are the shared parameters, i.e., the parameters of the shared feature extractor. A fundamental goal of gradient-based optimization is to find a d to simultaneously minimize all the losses. That is, the network can be optimized along a consistent direction and learn the shared knowledge between source and target domains efficiently and stably. Therefore, the direction d first need to satisfy the constraint dTgj ≥ 0,∀j ∈ [m], where gj is the j-th column of G.
Recently, a state-of-art method EPO [30] further controls the direction to reach a Pareto optimal solution with designated trade-off among objectives, using a preference vector fixed apriori. Concretely, EPO guides the optimization direction via minimizing the KL divergence between a normalized loss vector weighted by the preference vector and a uniform probability vector. Since the prior knowledge for the trade-off among objectives in DA is often unavailable and may vary with methods and training procedures, to weight LT ,LS and LD to guide the cooperative optimization towards minimizing L∗T , we propose to model the gradient of LT on a held-out unlabeled target dataset as the optimization guidance. We evaluate the performance on the held-out data because 1) it suggests generalization performance, and 2) it is more stable and convincible than training data. In practice, we randomly split 10% data from the original target set as the validation set and the rest 90% are taken as the training set. Note that we do not use any ground truth labels on the validation set, instead we calculate the proposed TCM loss on it, and then leverage this validation loss to guide the optimization direction.
Formally, we denote by LV al the TCM loss on the held-out data, then we can directly obtain the gradient descent direction: ĝv = ∇θLV al. Thus we can replace the guidance direction dbal 1 in EPO by ĝv as a dynamical guidance of the optimization direction. For clarity, the optimization problem can be formulated as a linear programming (LP) problem:
w∗ = arg max w∈Sm (Gw)T (I(LV al > 0)ĝv + I(LV al = 0)G1/m),
s.t. (Gw)Tgj ≥ I(J 6= ∅)(ĝTv gj), ∀j ∈ J̄ − J∗, (Gw)Tgj ≥ 0, ∀j ∈ J∗,
(6)
where Sm = {w ∈ Rm+ | ∑m
j=1 wj = 1}, I(·) is an indicator function, 1 ∈ Rm is a vector whose elements are all 1, J = {j|ĝTv gj > 0}, J̄ = {j|ĝTv gj ≤ 0}, and J∗ = {j|ĝTv gj = maxj′ ĝTv gj′}. One may be concerned that involving the gradient of the held-out data in training may eventually cause over-fitting of the data. Fortunately, we use the following result to show that the held-out data can guide but will not be over-fitted by the optimization of our method.
Theorem 1. Letw∗ be the solution of the problem in Eq. (6), and d∗ = Gw∗ be the resulted update direction. If LV al = 0, then the dominating direction d∗ becomes a descent direction, i.e.,
(d∗)Tgj ≥ 0, ∀j ∈ {1, 2, 3}. (7)
On the other hand, if LV alid > 0, Let γ∗ = (d∗)T ĝv be the objective value of the problem in Eq. (6). Then,
if γ∗ > 0, then (d∗)T ĝv > 0; if γ∗ ≤ 0, then (d∗)Tgj ≥ 0, ∀j ∈ {1, 2, 3}. (8)
The proof of Theorem 1 is deferred to the supplementary material. Note that we add a small = 1e−3 to relax the condition LV al > 0 and LV al = 0 in Eq. (6) as LV al > and LV al ≤ in practice, since the guiding significance of LV al becomes trivial when it is less than a small value.
1See [30] for details.
Algorithm 1 Update Equations for ParetoDA 1: Input: Shared parameters θt ∈ Rd, step size η 2: Calculate objective values: LS ,LD, LT 3: Calculate guidance loss on target validation set: Lval (using Eq. (5))
4: Calculate the gradients of training objectives: G=[g1, g2, g3] =[∇θtLS ,∇θtLD,∇θtLT ] 5: Calculate the gradient of the guidance loss: ĝv = ∇θtLval 6: Obtain w∗ by solving LP (Eq. (6)) 7: Optimization direction: d∗ = Gw∗ 8: Output: Updated parameters θt+1 = θt − ηd∗ Now we interpret the learning mechanism in Eq. (6). When LV al = 0, the learning is in the pure descent mode, in which the update direction d∗ = Gw∗ approximates the mean gradient G1/m, and d∗ can decrease all the training losses simultaneously. Whereas when LV al > 0, the learning is in the guidance descent mode, in which d∗ approximates ĝv. Since d∗ = Gw∗ is a convex combination of columns in G, then γ∗ > 0 means that ĝv is consistent with some columns in G, and d∗ is forced to decrease the loss whose gradient is the most consistent with ĝv—which means the highest contribution to the model performance. On the other hand, γ∗ ≤ 0 means none of the columns in G is consistent with ĝv . In this case, d∗ is not forced to consistent with ĝv , but only required to decrease the training losses, while approximating ĝv to the maximum feasible extent.
In summary, ĝv can dynamically guide the optimization direction towards the desired Pareto solution, but will not over-fit the held-out data, and we no longer need prior knowledge to set a preference vector. The theoretical guarantee for the existence of a Pareto optimal is deferred to the supplementary material. And the updating algorithm of every training iteration is presented in Alg. 1. Note that this optimize scheme is only applied to the shared parameters.
Time Complexity. Let m ∈ Z+ be the number of objective, and d ∈ Z+ the dimension of model parameters. The computation of (Gw)T ĝv of (Gw)TG both has run time O(m2d). With the best LP solver [7], our LP (Eq. 6), that has m variables and at most 2m+ 1 constraints, has a runtime 2 of O∗(m2.38). As in DA, we usually have d m, our total run time is O(m2d+m2.38) = O(d). Theoretical Insight. In Domain adaptation literature, the theory proposed by Ben-David et al. [2] provides an upper bound for the generalization error on target data, which is composed of three terms: 1) the expected error on source domain S(h); 2) the H∆H divergence between two domains dH∆H(Ds, Dt); 3) the combined error λ of the ideal joint hypothesis h∗. The formulation is:
T (h) ≤ S(h) + 1
2 dH∆H(Ds, Dt) + λ, (9)
where λ = S(h∗) + T (h∗) and h∗ = arg minh∈H S(h) + T (h).
The goal of DA methods is to lower the upper bound of the expected target error. S(h) is expected to be small thanks to the supervision in source domain. And λ is also generally considered sufficiently small [41, 27]. Thus, most DA methods mainly focus on reducing dH∆H(Ds, Dt) by domain alignment. However, if one of the three terms goes down and another term goes up, the model still can not effectively reduce the upper bound of T (h). Excessive either source classification or domain alignment will cause damage to the other.
In our ParetoDA, by adopting gradient-based optimization, we can simultaneously optimize the source classification loss and the domain alignment loss, so as to make S(h) and dH∆H(Ds, Dt) decrease synchronously. Moreover, both the training of our proposed TCM loss on target data and the optimization guidance of this loss on held-out data contribute to the classification ability of the model to the target domain, and then reduce the combined error λ of the ideal joint hypothesis h∗. Consequently, the upper bound of T (h) can be effectively reduced in our work, which validates the effectiveness of ParetoDA theoretically.

heading: 5 Experiment

heading: 5.1 Dataset and Setup
text: We evaluate ParetoDA on three object classification datasets and two semantic segmentation datasets. Office-31 [40] is a typical benchmark for cross-domain object classification involving three distinct
2O∗ is used to hide mo(1) and logO(1)(1/δ) factors, δ being the relative accuracy. See [7] for details.
domain: Amazon (A), DSLR (D) and Webcam (W); Office-Home [48] is a more difficult benchmark composed by four image domains: Art (Ar), Clip Art (Cl), Product (Pr) and Real-World (Rw). The images of different domains are more visually dissimilar with each other; VisDA-2017 [37] is a large-scale synthetic-to-real dataset, containing over 280K images across 12 categories; Cityscapes [8] is a real-world semantic segmentation dataset with 5,000 urban scenes which are divided into training, validation and test splits; GTA5 [38] includes 24,966 game screenshots from the game engine of GTA5. Following [46, 20], we establish a transfer semantic segmentation scenario: GTA5 → Cityscapes. Specifically, we use the shared 19 categories and report the test results on the validation set of Cityscapes. To evaluate our approach, we apply the proposed ParetoDA on former DA methods, i.e., DANN, CDAN, and MDD, and split 10% data randomly from the original target set as the validation set. All experiments are implemented via PyTorch [36], and we adopt the common-used stochastic gradient descent (SGD) optimizer with momentum 0.9 and weight decay 1e-4 for all experiments. Following the standard protocols in [27, 11, 29], we take the average classification accuracy and mean IoU (mIoU) for image classification and semantic segmentation as evaluation metrics, respectively. More details are presented in the supplementary material due to limited space.

heading: 5.2 Experimental Results
text: Office-31. From the results in Table 1, we can observe that DA methods, such as DANN and CDAN exceed the ResNet model. This validates that mitigating the domain gap between two domains is crucial for enhancing the generalization performance of models. We evaluate ParetoDA on three basic DA methods. The results show that ParetoDA can consistently improve the performance of the basic methods significantly. In particular, ParetoDA improves DANN by 8.0%, revealing that ParetoDA is an extra booster to former DA methods. On the other hand, the encouraging results validate the importance of searching for a desirable Pareto optimal solution.
Office-Home. Compared with Office-31, Office-Home is a more difficult dataset, whose results are shown in Table 2. In this difficult scenario, ParetoDA can still enhance the performance of former DA methods, with 11.8% and 4.8% improvement on DANN and CDAN respectively. Based on these promising results, we can infer that ParetoDA can reach a desirable Pareto optimal solution which is beneficial for the performance of models.
VisDA-2017. To validate the model performance on a large-scale dataset, we carry out experiments on VisDA-2017. The results are presented in Table 3. As a plug-in module, BSP improves the basic models by enhancing the discriminability of feature representations. ParetoDA aims to improve the optimization for DA methods, which is also orthogonal to former DA methods. In addition, ParetoDA achieves greater performance gains, further indicating the effectiveness and versatility of our method.
GTA5→ Cityscapes. Compared with classification tasks, segmentation demands more cost for the pixel-level annotation. Hence, it’s of vital importance to conduct domain adaptation for semantic segmentation. We evaluate our method on the adaptive semantic segmentation task, GTA5→ Cityscapes, aiming to transfer the knowledge from synthesized frames to real scenes. The segmentation results are exhibited in Table 4. We apply ParetoDA on two methods, i.e., AdvEnt and FDA, which boosts them by large margins on average. This manifests that ParetoDA can still reach an appropriate optimization solution for enhancing model performance on this pixel-level transfer scenario. Besides, ParetoDA is not limited to handle the conflict between the classical source classification loss and domain alignment loss, our framework can also be applied to where the explicit domain alignment module doesn’t exist or there exist multiple losses. For instance, we adopt ParetoDA on a self-training based domain adaptation method CBST [59] and it is clear that ParetoDA is still effective.
5.3 Analytical Experiments
Ablation Study. To verify each component of ParetoDA, we establish ablation study on Office-31. The average results are shown in Table 5. TCM performs better than IM, demonstrating that integrating domain labels is beneficial for target predictions. Applying the conventional Pareto optimization method [30] with fixed preference vector on DANN achieves considerable performance gain. This reveals that the gradient-based optimization method is superior to the linear weighting scheme. Further, with our proposed
ParetoDA, DANN obtains the best results, validating the importance of dynamical preference learning and that TCM loss serves as a better guidance for optimization.
Optimization Path Visualization. To intuitively verify that ParetoDA can dynamically search the desired Pareto optimal solution under the guidance of our proposed loss, we visualize the optimization paths of different optimization schemes (see Fig. 3(a)). Note that all these schemes start from the same initial point, but we only plot several solutions of the final stage of optimization for better illustration. It is clear that the linear weighting scheme not only deteriorates one of the objectives at some steps, but also reaches a solution on the convex part of the Pareto front, which deviates severely from the ideal one. The conventional gradient-based optimization method EPO (with fixed preference vector r = [1, 1, 1]) controls the optimization direction to avoid damaging any of the objectives. However, its solution heavily relies on the pre-defined preference vector for training objectives, which is ambiguous in DA. By contrast, our proposed ParetoDA further dynamically adjusts the optimization direction with the gradient direction of the proposed TCM loss. For comparisons, we illustrate the ideal solution guided by the supervised classification loss on target domain. One can observe that the optimization direction of ParetoDA is consistent with the ideal one in general, and reach a final solution closest to the ideal one.
Sensitivity to Different Objective Scales. The scale of the objectives may vary with methods or datasets. To test the adaptivity of ParetoDA to different objective scales, we conduct experiments on a random task W→A (Office-31) based on DANN by varying λ0, λ1 ∈ {0.1, 0.5, 1.0, 1.5}, which represent the scale factor of object classification objective and domain alignment objective respectively. Fig. 3(b) shows that ParetoDA is not sensitive to the scale variety of objectives, which indicating that ParetoDA is robust across different methods and datasets.
Adaptivity to Deeper Backbone Networks. In this work, we reach a desirable Pareto optimal solution for the target domain by controlling the optimization direction. Whereas on deeper networks, it might be more difficult to control the optimization direction due to the increased complexity.
For a fair comparison, we use ResNet-50 on classification tasks. However, ParetoDA can be easily adapted to deeper backbone networks. Table 6 presents the results of ParetoDA based on DANN with different backbones, demonstrating that ParetoDA can still consistently enhance the performance of the basic method.

heading: 6 Conclusion
text: In this paper, we rethink the optimization scheme for DA from a gradient-based optimization perspective. We present a Pareto Domain Adaptation (ParetoDA) to dynamically search a desirable Pareto optimal solution for boosting model performance, which is orthogonal to former DA methods. Extensive experiments on both classification and segmentation tasks validate the effectiveness of ParetoDA. It is worth noting that we mainly concentrate on the traditional domain adaptation setting for clarity, while the generalization to other variants of DA needs to be further explored.

heading: Acknowledgments
text: This work was supported by the National Natural Science Foundation of China (61902028).
Funding Transparency Statement
Funding in direct support of this work: National Natural Science Foundation of China (61902028).

Review summaries:
While doing domain adaptation, we usually have a source classification loss and a domain alignment loss. However, these losses might compete with each other and we might reach a situation where one loss can’t be reduced without degrading the other one. We call that the Pareto front. For DA, we don’t know which loss is better than the other one. In this paper, they come up with an optimization strategy that helps us navigate to an ideal portion on the Pareto front. Essentially, they stop the optimization if one loss can’t be improved without degrading the other.

The paper considers the common class of domain adaptation algorithms consisting in minimizing the sum of a source risk term and a divergence between the domains. Since both terms are required to be small, the authors examine the tradeoff between them via their associated Pareto front. This latter having non-convex regions non-reachable by the classic weighted sum minimization procedures, a multi-objective optimization procedure is proposed in order to reach all of the regions of the Pareto front and to access tradeoffs that are not accessible by the classic approach. Specifically, the authors use a modified version of a multi-objective optimization procedure, where they define a guidance direction that adjusts the directions of both gradients of the two cost function terms. The guidance direction is the gradient of a surrogate loss assessing performance on the target domain. To show the empirical performance of their idea, the authors present experiments in which their method improves previously proposed DA algorithms.

This paper represents a domain adaptation (DA) problem in a multi-objective optimization scheme, which can be solved by a gradient-based perspective. The main contributions of the paper include:  1. Proposes a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to optimize all training objectives. That ParetoDA scheme can be plugged into various DA methods and enhance their performance.  2. To find a desired Pareto optimal solution, the authors design a surrogate objective function that mimics the target classification task.  3. Provides experimental results that show the effectiveness of ParetoDA compared to some DA methods.

This paper addresses the issue of previous domain adaptation optimization scheme, which learning objectives conflict with each other. The main contribution is to dynamically control the overall optimization direction so that no objective is harmed resulting out to boost the performance. To this end, it designs desired pareto optimal solution with surrogate objective function that mimics the target classification task called Pareto Domain Adaptation.