Adversarial Reweighting for Partial Domain Adaptation

1 Introduction . Deep learning has achieved impressive success in image recognition [ 15 , 20 , 39 ] . However , deep learning models often rely on massive labeled training data , requiring an expensive and time-consuming labeling process . To alleviate the dependency of deep learning models on a large number of labeled data , domain adaptation ( DA ) [ 27 ] transfers the knowledge from a relevant source domain with rich labeled data to the target domain . The core of DA is to build a predictive model for the target domain using the training data of the source domain , and the model is expected to be robust to the distribution discrepancy ( a.k.a. , domain shift ) between source and target domains . Domain adaptation methods often train the robust model by aligning distributions of different domains by moment matching [ 17 , 19 , 24 , 28 , 40 , 48 ] or adversarial training [ 8 , 25 , 38 , 40 , 41 , 42 , 50 ] . The conventional closed-set DA methods generally assume that the source and target domains share the same label space . However , this assumption is often not realistic in practice . It is usually difficult to find a relevant source domain with identical label space as the target domain . This motivates the research on the learning problem of partial domain adaptation [ 3 , 49 ] . Partial domain adaptation ( PDA ) is an important subcategory of domain adaptation . PDA [ 3 , 4 , 49 ] tackles the scenario that the label space of the target domain is a subset of that of the source domain . Partial domain adaptation is more challenging than vanilla closed-set domain adaptation . Because , besides the challenge of the domain shift , the existence of source-only classes that do not exist in the target domain can cause class-wise feature mismatch when aligning distributions . This potentially leads to negative transfer [ 27 ] , i.e. , the DA approaches hurt the performance of learning in the target ∗The corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . domain . To mitigate the negative transfer , current PDA methods [ 3 , 4 , 5 , 21 , 23 , 34 , 46 , 49 ] commonly reweight the source domain data to decrease the importance of data belonging to the source-only classes . The target and reweighted source domain data are used to train the feature extractor by adversarial training [ 3 , 4 , 5 , 23 , 46 , 49 ] or kernel mean matching [ 21 , 34 ] to align distributions . In this paper , we first propose to measure the hardness of a dataset for PDA using the probability of target domain data being misclassified into source-only classes . We then observe that some of the state-of-the-art reweighted distribution alignment losses , e.g. , PADA [ 4 ] , BAA [ 23 ] , and reweighted MMD [ 34 ] , cause negative transfer on VisDA-2017 [ 32 ] and DomainNet [ 31 ] datasets that are more challenging than Office-31 [ 36 ] , Office-Home [ 43 ] and ImageNet-Caltech [ 12 , 35 ] , according to our hardness measure . Specifically , learning/adapting the feature extractor by aligning the feature distributions of the reweighted source and target domain data can even worsen the performance of the baseline model without feature distribution alignment . We find that this negative domain transfer effect is mainly because that these reweighted feature distribution alignment methods are not robust to the “ noises ” of source data weights , i.e. , some source-only-class data are mistakenly assigned with non-zero weights in the alignment losses . To tackle the negative domain transfer in PDA , we propose a novel adversarial reweighting ( AR ) approach , which adversarially learns to reweight the source domain data for aligning the distributions of the source and target domains . Specifically , our approach relies on adversarial training to learn the weights of source domain data to minimize the Wasserstein distance between the reweighted source domain and target domain distributions . The weight learning process is conducted in an adversarial reweighting model , using the dual form of the Wasserstein distance . We then define a reweighted cross-entropy loss on the reweighted source domain data and use the conditional entropy loss on the target data to train the transferable recognition network for the target domain . The network training and weight learning are performed alternately in an iterative training algorithm . The current PDA methods [ 3 , 4 , 5 , 21 , 23 , 34 , 46 , 49 ] design/learn source data weights based on the classifier [ 3 , 4 , 21 , 23 , 34 , 46 ] or discriminator [ 5 , 49 ] . They then train the feature extractor using a reweighted distribution alignment loss defined on the target and reweighted source data . Different from them , firstly , we learn the weights of source data in our proposed adversarial reweighting model to decrease the weight of source-only-class data . Secondly , we reduce the domain gap by reweighting the source domain data , instead of directly optimizing the feature extractor to match feature distributions This strategy may mitigate the negative domain transfer when the source domain data contain “ noisy ” weights . Note that Balaji et al . [ 2 ] and Huang et al . [ 16 ] also align data distributions by data reweighing . Huang et al . [ 16 ] proposes a shallow method that reweights the source data using the density ratio estimated by kernel mean matching for closed-set DA . Differently , we reweight the source data using an adversarial reweighting model for PDA in feature space in deep learning framework . Balaji et al . [ 2 ] proposes a robust optimal transport model to decrease the importance of the outliers of target data in distribution alignment for closed-set DA . Different from Balaji et al . [ 2 ] , we reweight the source domain data to decrease the importance of source-only-class data in the source classification loss to mitigate the negative transfer for PDA . We conduct extensive experiments on five benchmark datasets i.e. , Office-31 , ImageNet-Caltech , Office-Home , VisDA-2017 , and DomainNet . The results show that our method achieves state-of-theart results for PDA on ImageNet-Caltech , Office-Home , VisDA-2017 , and DomainNet datasets , and achieves competitive results on Office-31 . Ablation studies indicate that our method significantly outperforms its baseline on all datasets . Our code is available at https : //github.com/XJTU-XGU/ Adversarial-Reweighting-for-Partial-Domain-Adaptation . In the following sections , we discuss the limitations of feature adaptation by domain distribution alignment in Sect . 2 . Section 3 presents the details of our approach . Section 4 reports the experimental results and Sect . 5 concludes this work . 2 Limitations of Feature Adaptation by Domain Distribution Alignment . In this section , we first introduce the setting of PDA and summarize the popular reweighted distribution alignment methods for PDA [ 3 , 4 , 5 , 21 , 23 , 34 , 49 ] . We then propose a hardness measure for PDA benchmarks and show the limitations of the reweighted distribution alignment . Problem setting . In PDA , we are given a labeled dataset S = { xsi , ysi } ns i=1 from source domain and an unlabeled dataset T = { xtj } nt j=1 from target domain , where x s and xt ∈ Rd respectively denote the source and target data , ysi ∈ Ys is the label of xsi , and Ys = { 0 , 1 , · · · , K } is the source label space . The goal is to train a recognition network to predict the label of xtj . Notably , in PDA , the target label space Yt is a subset of Ys , i.e. , Yt ⊂ Ys , which is different from the setting ( Yt = Ys ) in the closed-set DA . The PDA methods often use a feature extractor F to extract the features and a classifier C to predict labels , and optionally use a discriminator D to discriminate domains . Summary of reweighted distribution alignment methods in PDA . The challenges of PDA come from two aspects : the domain shift between source and target domains , and the negative effect of source-only-class data in adaptation . To tackle these challenges , most popular PDA methods ( e.g. , SAN [ 3 ] , PADA [ 4 ] , ETN [ 5 ] , DRCN [ 21 ] , BA3US [ 23 ] , TSCDA [ 34 ] , and IWAN [ 49 ] ) commonly adapt the feature extractor by minimizing the feature distribution distance between the reweighted source domain and the target domain , measured by the reweighted distribution alignment losses . The weights in the reweighted distribution alignment losses are usually designed based on the output of the classifier or the discriminator . The widely adopted distribution distance metrics include the Maximization Mean Discrepancy ( MMD ) and the Jensen–Shannon ( JS ) divergence . Minimizing the MMD matches the kernel mean embedding of distributions in the Reproducing Kernel Hilbert Space . Minimizing the JS divergence is equivalent to the adversarial training as in the Generative Adversarial Network [ 10 ] . More comparisons of the PDA methods are given in Supp . A . Measuring the hardness of a dataset for PDA . We propose to measure the hardness of a dataset for PDA using the probability of the target domain data being misclassified into the source-only classes . Specifically , we first train a model by minimizing the source classification loss and the entropy loss on target domain , which is the baseline for PDA . Then the hardness of a PDA task is defined as the average predicted probability of the target domain samples being misclassified into the source-only classes , using the trained model . The average hardness of all tasks in the dataset is taken as the hardness of the dataset . We report the hardness of five PDA benchmark datasets in Fig . 1 ( a ) . Figure 1 ( a ) indicates that VisDA-2017 and DomainNet are more challenging than the other datasets . Adapting feature extractor by reweighted distribution alignment may lead to negative domain transfer . We show the limitations of the reweighted distribution alignment in this paragraph . We report the average classification accuracies for different reweighted distribution alignment losses ( including the reweighted adversarial training losses of PADA [ 4 ] and BAA [ 23 ] , and the reweighted MMD loss in [ 34 ] ) and the baseline ( w/o Align ) on five benchmark datasets in Fig . 1 ( b ) . We also give the detailed results in Supp . B . We can observe in Fig . 1 ( b ) that additionally minimizing these reweighted distribution alignment losses by the feature extractor worsens the performance of the baseline on some challenging datasets , e.g. , VisDA-2017 and DomainNet . This indicates that adapting the feature extractor by the reweighted distribution alignment losses leads to negative transfer on these datasets . We next investigate the reasons for this finding . Adapting feature extractor by reweighted distribution alignment is not robust to noisy weights . In the methods of [ 3 , 4 , 21 , 23 , 34 ] , the weight of source domain data ( xsi , y s i ) is defined as the average predicted probability for category of ysi on the target domain data , i.e. , the y s i -th element of 1nt ∑nt j=1 C ( F ( x t j ) ) . When the domain gap between source and target domains is large , the prediction by the source classifier on the target domain may be uncertain . Then , the predicted probability of classifying the target data as source-only classes may be non-zero or even possibly significantly larger than zero . Hence , reweighting source classes based on the outputs of the classifier may assign non-zero weights to the source-only classes , i.e. , the designed weights may contain “ noises ” . We analyze that the ineffectiveness of adapting the feature extractor by the reweighted distribution alignment could be because it is not robust to the “ noisy ” weights . To illustrate this , we conduct experiments with simulating the source domain data weights with different noise levels . We assign the noise to each source-only class in proportion to its weight predicted by the classifier aforementioned . If the weight predicted by the classifier is ci , i ∈ Ys , for the noise level p ∈ [ 0 , 1 ] , the simulated weight for each source-only class is pci/ ∑ k∈Ys\Yt c k , for i ∈ Ys\Yt . Similarly , the simulated weight for each source-shared class ( the class exists in both source and target label spaces ) is ( 1 − p ) ci/ ∑ k∈Yt c k , for i ∈ Yt . If p = 0 , the data of source-only classes are ideally assigned with weights of 0 , and as the p increases , these data are assigned with larger weights , which are taken as the “ noisy ” weights . We also use the simulated weights to reweight the importance of source data in the classification loss , denoted as “ RC ” . Figures 1 ( c ) and 1 ( d ) show the results for different reweighted distribution alignment losses with the simulated weights under varying noise levels . We can see that when the noise level is near zero , all the alignment losses outperform “ w/o Align ” , indicating that the alignment losses lead to positive domain transfer in this case . However , as the noise level increases , the performance of the alignment losses decreases rapidly and even becomes significantly worse than that of “ w/o Align ” . Specifically , in task S→R ( Fig . 1 ( c ) ) , when the noise level is 0.1 , the performance of the alignment losses begin to be inferior to that of “ w/o Align ” . Similarly , in task C→P ( Fig . 1 ( d ) ) , the performance of the alignment losses begin to be inferior to that of “ w/o Align ” at noise level 0.3 . While the “ real ” noise level ( the sum of weights based on the classifier over source-only classes ) is larger than 0.3 ( resp . 0.4 ) in the two tasks . Therefore , the negative domain transfer of the reweighted distribution alignment losses may be because of the “ noise ” in weights . It is also seen that when the noise level ranges from 0 to 0.4 , the approach “ w/o Align & w/ RC ” that reweights data importance in source classification loss consistently outperforms the baseline “ w/o Align ” in both two tasks . This indicates that reweighting data importance in the source classification loss is more robust to weight noise , compared with the reweighted distribution alignment losses , in the two tasks . The above observations indicate that adapting the feature extractor by the reweighted distribution alignment is not robust to the noise in source data weights , and can cause negative domain transfer on some challenging datasets . Surprisingly , reweighting data importance in the source classification loss may be more robust to weight noise than the reweighted distribution alignment losses .

Summary: The paper introduces the adversarial reweighting (AR) technique for estimating weights in the source domain. Compared to the previous methods on partial domain adaptation, the proposed method seems robust on the effect of negative domain transfer during the adaptation. Authors demonstrate the effectiveness of AR in several partial domain adaptation benchmarks.