Pareto Domain Adaptation

1 Introduction . Domain adaptation ( DA ) is a well-established paradigm for learning a model on an unlabeled target domain with the assistance of a labeled related source domain , which has attracted a surge of interests in the machine learning community [ 34 , 33 , 13 ] . By bridging the domain gap on the premise of ensuring the performance of source classification task , DA can adapt the model learned from the source domain to the target in the presence of data bias , which solves the dilemma of label scarcity in many real-world applications [ 55 , 47 , 11 , 45 , 3 , 57 ] . Extensive DA approaches have been proposed in recent years , achieving great performances in many areas such as image classification [ 11 , 27 , 56 ] , semantic segmentation [ 49 , 52 , 16 ] , and object detection [ 39 , 43 , 54 ] . ∗ Equal contributions from both authors . † Corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Most DA methods are devoted to aligning the distributions across domains by explicitly minimizing some discrepancy metrics [ 26 , 28 , 56 ] or adopting adversarial learning [ 11 , 27 , 41 ] . However , since the domain alignment objective also depends on the target domain that deviates from the source , the gradient direction of it may conflict with that of classification objective on the source domain . Thus , when the optimization proceeds to a certain stage , one objective will deteriorate if further improving the other one . Under this circumstance , no solution can reach the optimal of each objective at the same time . We can only obtain a set of so-called Pareto optimal solutions instead , where we can not further decrease all objectives simultaneously [ 10 ] . All Pareto optimal solutions in loss space compose Pareto front as shown in Fig . 1 . 𝐿𝐴 ↓ , 𝐿𝑆 ↓ Dynamic 𝐿𝐴 Fixed Dynamic Preference Vector ( d ) 𝐿𝑆 𝐿𝐴 𝑳𝑨 ↓ , 𝑳𝑺 ↓ ( b ) 𝐿𝑆 𝐿𝐴 𝑳𝑨 ↓ , 𝑳𝑺 ↓ 𝑳𝑨 ↑ , 𝑳𝑺 ↓ ( a ) min 𝐿 = 𝜆1𝐿𝑆 + 𝜆2𝐿𝐷 min 𝐿 = 𝜆3𝐿𝑆 + 𝜆4𝐿𝐷 𝐿𝐷 This dilemma obviously has been overlooked by most former DA methods , which generally adopt some empirical weight hyperparameters to linearly combine the objectives , constructing an overall objective . This linear weighting scheme has two major concerns . First , as pointed out in [ 4 ] , it can only obtain solutions on the convex part of the Pareto front of the objectives , while can not reach the non-convex part , as shown in Fig . 1 ( a ) - ( b ) . That is because it only considers reducing the overall objective value , which might damage some objectives during optimization ( see the red up-arrow ) . Unfortunately , the Pareto fronts of DA methods are often non-convex , due to loss conflict caused by domain shift . See examples shown in Fig . 2 . Second , one could hardly reach the desired solution that performs best on target domain by tuning the weight hyper-parameters , since the obtained solution will deviate severely when the trade-offs slightly change ( see the slopes of the dash lines in Fig . 1 ( a ) - ( b ) for reference ) . To remedy these issues , we rethink the optimization scheme for DA from a gradient-based perspective . By weighting objectives to control the overall optimization direction so that no objective deteriorates , one can reach a Pareto optimal solution on the Pareto front [ 1 , 23 , 30 , 32 ] . Moreover , [ 30 ] precisely sets objective weights according to a prefixed preference vector ( see Fig . 1 ( c ) ) . However , neither of them suits DA , because 1 ) in DA , the goal is to minimize the target classification loss L∗T , but neither LS or LD directly corresponds to this goal ; 2 ) the target domain has no label to construct a classification loss for the goal ; 3 ) prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution that minimizes L∗T . In this paper , we propose a Pareto Domain Adaptation ( ParetoDA ) approach , which cooperatively optimizes training objectives and is specifically designed for DA . Specifically , to minimize L∗T , we design a target-classification-mimicking ( TCM ) loss LT by the mutual information [ 17 ] leveraging the target predictions . In this loss , target predictions act as ( soft ) labels for themselves . Therefore , the accuracy of the predictions is important to support the mimicking . To refine the predictions by maximally exploiting the information at hand , we propose a target prediction refining mechanism which models the domain label as conditional information into the prediction probability by Bayes ’ theorem . Intuitively , we ensembles a class-wise domain discriminator ( trained with the domain labels ) as another classifier—only when the class condition is correct , the discriminator can correctly predict the domain . On the other hand , to weight LT , LS and LD to guide the cooperative optimization towards minimizing L∗T , we propose a dynamic preference mechanism to model the gradient of LT on a held-out unlabeled target dataset as the guidance . We evaluate the performance on the held-out data because 1 ) it suggests generalization performance , and 2 ) it is more stable and convincible than training data . One may be concerned that involving the gradient of the held-out data in training may eventually cause over-fitting of the data . Fortunately , our theoretical analyses show that the held-out data can guide but will not be over-fitted by the optimization of our method . Our contributions are : • We rethink the optimization scheme in existing DA methods , which can only reach restricted solutions if the optimization objectives conflict with each other . And existing gradient-based strategies do not suit DA . • We propose a Pareto Domain Adaptation method to obtain the desired Pareto optimal solution learned and dynamically guided ( using the held-out unlabeled target data ) by the designed surrogate objective function that mimics target classification . To better support the mimicking , we also propose a target prediction refining mechanism . As a general technique , ParetoDA can be plugged into various DA methods and enhance their performance . • We evaluate ParetoDA on two DA applications : image classification and semantic segmentation . Experimental results demonstrate the effectiveness of ParetoDA . 2 Background . Consider m objectives , each with a non-negative loss function Li ( θ ) : Rd → R+ , i ∈ [ m ] = { 1 , 2 , ... , m } , where θ is the shared parameters of the objectives to be optimized . There may exist no single solution that can reach the optimal solution of each objective at the same time due to they may conflict with each other . What we can obtain instead is a set of so-called Pareto optimal solutions , the related definitions are as follows [ 58 ] : Pareto dominance . Suppose two solutions θ1 , θ2 in Rd , we define θ1 ≺ θ2 if Li ( θ1 ) ≤ Li ( θ2 ) , ∀i ∈ [ m ] and Li ( θ1 ) < Li ( θ2 ) , ∃i ∈ [ m ] . We say that θ1 dominates θ2 iff θ1 ≺ θ2 . Note that θ1 ⊀ θ2 if θ2 is not dominated by θ1 . Pareto optimality . If a solution θ1 dominates θ2 , then θ1 is clearly preferable as it perform better or equally on each objective . A solution θ∗ is Pareto optimal if it is not dominated by any other solutions . That is , θ ⊀ θ∗ , ∀θ ∈ Rd − { θ∗ } . Pareto front . The set of all Pareto optimal solutions in loss space is Pareto front , where each point represents a unique solution . Preference vector . A Pareto optimal solution can be viewed as an intersection of the Pareto front with a specific direction in loss space ( a ray ) . We refer to this direction as the preference vector . In gradient-based optimization , we can reach a Pareto optimal solution by starting from an arbitrary initialization and iteratively finding the next solution θt+1 that dominates the previous one θt , i.e. , θt+1 = θt − ηddes , where η is the step size and ddes is the optimization direction . Let gi = ∇θLi be the gradient of the i-th objective function at θ , the descent direction ddes is given by dTdesgi ≥ 0 , ∀i ∈ [ m ] . In other words , the descent direction ddes should optimize or not damage each objective . Thus , moving against ddes , starting from θ , amounts to a decrease in the objective values . When dTdesgi = 0 , the optimization process ends , implying that no direction can further optimize the all the objectives simultaneously . At this moment , if we still want to improve the performance of a specific objective , other objectives will deteriorate . This paper focuses on dynamically searching for a desired Pareto optimal solution with the aid of the proposed TCM loss in DA problems , aiming to enhance the performance on the target domain . 3 Related Work . Domain Adaptation aims to transfer knowledge across domains . The theory proposed by Ben-David et al . [ 2 ] reveals that the key ideology of DA is to diminish the domain gap between two domains , enabling successful transfer . Existing DA methods mainly mitigate domain shift via minimizing some statistic metrics [ 26 , 56 , 28 , 44 ] or adversarial training [ 11 , 27 , 41 , 20 , 22 ] . To name a few , DAN [ 26 ] mitigates the layer-wise discrepancy via Maximum Mean Discrepancy ( MMD ) metric . MDD [ 56 ] introduces Margin Disparity Discrepancy with rigorous generalization bounds to measure the discrepancy between domains . On the other hand , inspired by GANs [ 14 ] , DANN [ 11 ] introduces a domain discriminator to enforce the features from two domains to be indistinguishable . CDAN [ 27 ] further extends this idea by integrate classification information into domain discriminator , leading to precise domain alignment . By contrast , we adopt domain discriminators to refine the target predictions with domain labels . In addition , these methods directly optimize the linear combination of all objectives , while overlooking on the potential conflict between them . In such case , the linear scheme is hard to tune and may only access to restricted solutions on the convex part of pareto front [ 4 ] . We propose a gradient-based optimization scheme to search the desirable Pareto optimal solution on the entire pareto front without weight hyper-parameters , which is orthogonal to prior DA methods . Weight-Based Optimization is an intuitive strategy to optimize multiple objectives at the same time , which unifies all the training objectives using weight hyper-parameters to form an overall objective . Since it is hard to manually assign proper weights in prior , many DA methods conduct Deep Embedded Validation [ 53 ] to select weight hyper-parameters . Recently , some methods propose to dynamically weight the objectives leveraging gradient magnitude [ 6 ] , the rate of change in losses [ 25 ] , task uncertainty [ 19 ] , or learning non-linear loss combinations by implicit differentiation [ 31 ] . However , these weight-based methods focus on seeking a balanced solution , which is not bound to be optimal for target classification in DA . By contrast , we explore the gradient-based optimization , aiming to find an optimal solution for DA problems . Multi-Objective Gradient-Based Optimization leverages the gradients of objectives to reach the final Pareto optimal solution . MGDA [ 1 ] proposes to simply update the shared network parameters along a descent direction which leads to solutions that dominate the previous one . Nonetheless , MGDA may reach any Pareto optimal solution since it has no extra control for the optimization direction . To ameliorate this problem , PMTL [ 23 ] splits the loss space into separated sub-regions based on several preference vectors , and achieve Pareto optimal solutions in each sub-region of the Pareto front . Further , EPO [ 30 ] can reach a exact Pareto optimal solution by enforcing the decent direction correspond a specific preference vector . However , prior knowledge of which Pareto solution is the optimal one for target classification is often unavailable in DA . In this paper , we propose ParetoDA algorithm to dynamically searching the desirable Pareto optimal solution for target domain guided by a introduced TCM loss on a held-out target data .

Summary: While doing domain adaptation, we usually have a source classification loss and a domain alignment loss. However, these losses might compete with each other and we might reach a situation where one loss can’t be reduced without degrading the other one. We call that the Pareto front. For DA, we don’t know which loss is better than the other one. In this paper, they come up with an optimization strategy that helps us navigate to an ideal portion on the Pareto front. Essentially, they stop the optimization if one loss can’t be improved without degrading the other.