Adversarial Reweighting for Partial Domain Adaptation

1 Introduction . Deep learning has achieved impressive success in image recognition [ 15 , 20 , 39 ] . However , deep learning models often rely on massive labeled training data , requiring an expensive and time-consuming labeling process . To alleviate the dependency of deep learning models on a large number of labeled data , domain adaptation ( DA ) [ 27 ] transfers the knowledge from a relevant source domain with rich labeled data to the target domain . The core of DA is to build a predictive model for the target domain using the training data of the source domain , and the model is expected to be robust to the distribution discrepancy ( a.k.a. , domain shift ) between source and target domains . Domain adaptation methods often train the robust model by aligning distributions of different domains by moment matching [ 17 , 19 , 24 , 28 , 40 , 48 ] or adversarial training [ 8 , 25 , 38 , 40 , 41 , 42 , 50 ] . The conventional closed-set DA methods generally assume that the source and target domains share the same label space . However , this assumption is often not realistic in practice . It is usually difficult to find a relevant source domain with identical label space as the target domain . This motivates the research on the learning problem of partial domain adaptation [ 3 , 49 ] . Partial domain adaptation ( PDA ) is an important subcategory of domain adaptation . PDA [ 3 , 4 , 49 ] tackles the scenario that the label space of the target domain is a subset of that of the source domain . Partial domain adaptation is more challenging than vanilla closed-set domain adaptation . Because , besides the challenge of the domain shift , the existence of source-only classes that do not exist in the target domain can cause class-wise feature mismatch when aligning distributions . This potentially leads to negative transfer [ 27 ] , i.e. , the DA approaches hurt the performance of learning in the target ∗The corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . domain . To mitigate the negative transfer , current PDA methods [ 3 , 4 , 5 , 21 , 23 , 34 , 46 , 49 ] commonly reweight the source domain data to decrease the importance of data belonging to the source-only classes . The target and reweighted source domain data are used to train the feature extractor by adversarial training [ 3 , 4 , 5 , 23 , 46 , 49 ] or kernel mean matching [ 21 , 34 ] to align distributions . In this paper , we first propose to measure the hardness of a dataset for PDA using the probability of target domain data being misclassified into source-only classes . We then observe that some of the state-of-the-art reweighted distribution alignment losses , e.g. , PADA [ 4 ] , BAA [ 23 ] , and reweighted MMD [ 34 ] , cause negative transfer on VisDA-2017 [ 32 ] and DomainNet [ 31 ] datasets that are more challenging than Office-31 [ 36 ] , Office-Home [ 43 ] and ImageNet-Caltech [ 12 , 35 ] , according to our hardness measure . Specifically , learning/adapting the feature extractor by aligning the feature distributions of the reweighted source and target domain data can even worsen the performance of the baseline model without feature distribution alignment . We find that this negative domain transfer effect is mainly because that these reweighted feature distribution alignment methods are not robust to the “ noises ” of source data weights , i.e. , some source-only-class data are mistakenly assigned with non-zero weights in the alignment losses . To tackle the negative domain transfer in PDA , we propose a novel adversarial reweighting ( AR ) approach , which adversarially learns to reweight the source domain data for aligning the distributions of the source and target domains . Specifically , our approach relies on adversarial training to learn the weights of source domain data to minimize the Wasserstein distance between the reweighted source domain and target domain distributions . The weight learning process is conducted in an adversarial reweighting model , using the dual form of the Wasserstein distance . We then define a reweighted cross-entropy loss on the reweighted source domain data and use the conditional entropy loss on the target data to train the transferable recognition network for the target domain . The network training and weight learning are performed alternately in an iterative training algorithm . The current PDA methods [ 3 , 4 , 5 , 21 , 23 , 34 , 46 , 49 ] design/learn source data weights based on the classifier [ 3 , 4 , 21 , 23 , 34 , 46 ] or discriminator [ 5 , 49 ] . They then train the feature extractor using a reweighted distribution alignment loss defined on the target and reweighted source data . Different from them , firstly , we learn the weights of source data in our proposed adversarial reweighting model to decrease the weight of source-only-class data . Secondly , we reduce the domain gap by reweighting the source domain data , instead of directly optimizing the feature extractor to match feature distributions This strategy may mitigate the negative domain transfer when the source domain data contain “ noisy ” weights . Note that Balaji et al . [ 2 ] and Huang et al . [ 16 ] also align data distributions by data reweighing . Huang et al . [ 16 ] proposes a shallow method that reweights the source data using the density ratio estimated by kernel mean matching for closed-set DA . Differently , we reweight the source data using an adversarial reweighting model for PDA in feature space in deep learning framework . Balaji et al . [ 2 ] proposes a robust optimal transport model to decrease the importance of the outliers of target data in distribution alignment for closed-set DA . Different from Balaji et al . [ 2 ] , we reweight the source domain data to decrease the importance of source-only-class data in the source classification loss to mitigate the negative transfer for PDA . We conduct extensive experiments on five benchmark datasets i.e. , Office-31 , ImageNet-Caltech , Office-Home , VisDA-2017 , and DomainNet . The results show that our method achieves state-of-theart results for PDA on ImageNet-Caltech , Office-Home , VisDA-2017 , and DomainNet datasets , and achieves competitive results on Office-31 . Ablation studies indicate that our method significantly outperforms its baseline on all datasets . Our code is available at https : //github.com/XJTU-XGU/ Adversarial-Reweighting-for-Partial-Domain-Adaptation . In the following sections , we discuss the limitations of feature adaptation by domain distribution alignment in Sect . 2 . Section 3 presents the details of our approach . Section 4 reports the experimental results and Sect . 5 concludes this work . 2 Limitations of Feature Adaptation by Domain Distribution Alignment . In this section , we first introduce the setting of PDA and summarize the popular reweighted distribution alignment methods for PDA [ 3 , 4 , 5 , 21 , 23 , 34 , 49 ] . We then propose a hardness measure for PDA benchmarks and show the limitations of the reweighted distribution alignment . Problem setting . In PDA , we are given a labeled dataset S = { xsi , ysi } ns i=1 from source domain and an unlabeled dataset T = { xtj } nt j=1 from target domain , where x s and xt ∈ Rd respectively denote the source and target data , ysi ∈ Ys is the label of xsi , and Ys = { 0 , 1 , · · · , K } is the source label space . The goal is to train a recognition network to predict the label of xtj . Notably , in PDA , the target label space Yt is a subset of Ys , i.e. , Yt ⊂ Ys , which is different from the setting ( Yt = Ys ) in the closed-set DA . The PDA methods often use a feature extractor F to extract the features and a classifier C to predict labels , and optionally use a discriminator D to discriminate domains . Summary of reweighted distribution alignment methods in PDA . The challenges of PDA come from two aspects : the domain shift between source and target domains , and the negative effect of source-only-class data in adaptation . To tackle these challenges , most popular PDA methods ( e.g. , SAN [ 3 ] , PADA [ 4 ] , ETN [ 5 ] , DRCN [ 21 ] , BA3US [ 23 ] , TSCDA [ 34 ] , and IWAN [ 49 ] ) commonly adapt the feature extractor by minimizing the feature distribution distance between the reweighted source domain and the target domain , measured by the reweighted distribution alignment losses . The weights in the reweighted distribution alignment losses are usually designed based on the output of the classifier or the discriminator . The widely adopted distribution distance metrics include the Maximization Mean Discrepancy ( MMD ) and the Jensen–Shannon ( JS ) divergence . Minimizing the MMD matches the kernel mean embedding of distributions in the Reproducing Kernel Hilbert Space . Minimizing the JS divergence is equivalent to the adversarial training as in the Generative Adversarial Network [ 10 ] . More comparisons of the PDA methods are given in Supp . A . Measuring the hardness of a dataset for PDA . We propose to measure the hardness of a dataset for PDA using the probability of the target domain data being misclassified into the source-only classes . Specifically , we first train a model by minimizing the source classification loss and the entropy loss on target domain , which is the baseline for PDA . Then the hardness of a PDA task is defined as the average predicted probability of the target domain samples being misclassified into the source-only classes , using the trained model . The average hardness of all tasks in the dataset is taken as the hardness of the dataset . We report the hardness of five PDA benchmark datasets in Fig . 1 ( a ) . Figure 1 ( a ) indicates that VisDA-2017 and DomainNet are more challenging than the other datasets . Adapting feature extractor by reweighted distribution alignment may lead to negative domain transfer . We show the limitations of the reweighted distribution alignment in this paragraph . We report the average classification accuracies for different reweighted distribution alignment losses ( including the reweighted adversarial training losses of PADA [ 4 ] and BAA [ 23 ] , and the reweighted MMD loss in [ 34 ] ) and the baseline ( w/o Align ) on five benchmark datasets in Fig . 1 ( b ) . We also give the detailed results in Supp . B . We can observe in Fig . 1 ( b ) that additionally minimizing these reweighted distribution alignment losses by the feature extractor worsens the performance of the baseline on some challenging datasets , e.g. , VisDA-2017 and DomainNet . This indicates that adapting the feature extractor by the reweighted distribution alignment losses leads to negative transfer on these datasets . We next investigate the reasons for this finding . Adapting feature extractor by reweighted distribution alignment is not robust to noisy weights . In the methods of [ 3 , 4 , 21 , 23 , 34 ] , the weight of source domain data ( xsi , y s i ) is defined as the average predicted probability for category of ysi on the target domain data , i.e. , the y s i -th element of 1nt ∑nt j=1 C ( F ( x t j ) ) . When the domain gap between source and target domains is large , the prediction by the source classifier on the target domain may be uncertain . Then , the predicted probability of classifying the target data as source-only classes may be non-zero or even possibly significantly larger than zero . Hence , reweighting source classes based on the outputs of the classifier may assign non-zero weights to the source-only classes , i.e. , the designed weights may contain “ noises ” . We analyze that the ineffectiveness of adapting the feature extractor by the reweighted distribution alignment could be because it is not robust to the “ noisy ” weights . To illustrate this , we conduct experiments with simulating the source domain data weights with different noise levels . We assign the noise to each source-only class in proportion to its weight predicted by the classifier aforementioned . If the weight predicted by the classifier is ci , i ∈ Ys , for the noise level p ∈ [ 0 , 1 ] , the simulated weight for each source-only class is pci/ ∑ k∈Ys\Yt c k , for i ∈ Ys\Yt . Similarly , the simulated weight for each source-shared class ( the class exists in both source and target label spaces ) is ( 1 − p ) ci/ ∑ k∈Yt c k , for i ∈ Yt . If p = 0 , the data of source-only classes are ideally assigned with weights of 0 , and as the p increases , these data are assigned with larger weights , which are taken as the “ noisy ” weights . We also use the simulated weights to reweight the importance of source data in the classification loss , denoted as “ RC ” . Figures 1 ( c ) and 1 ( d ) show the results for different reweighted distribution alignment losses with the simulated weights under varying noise levels . We can see that when the noise level is near zero , all the alignment losses outperform “ w/o Align ” , indicating that the alignment losses lead to positive domain transfer in this case . However , as the noise level increases , the performance of the alignment losses decreases rapidly and even becomes significantly worse than that of “ w/o Align ” . Specifically , in task S→R ( Fig . 1 ( c ) ) , when the noise level is 0.1 , the performance of the alignment losses begin to be inferior to that of “ w/o Align ” . Similarly , in task C→P ( Fig . 1 ( d ) ) , the performance of the alignment losses begin to be inferior to that of “ w/o Align ” at noise level 0.3 . While the “ real ” noise level ( the sum of weights based on the classifier over source-only classes ) is larger than 0.3 ( resp . 0.4 ) in the two tasks . Therefore , the negative domain transfer of the reweighted distribution alignment losses may be because of the “ noise ” in weights . It is also seen that when the noise level ranges from 0 to 0.4 , the approach “ w/o Align & w/ RC ” that reweights data importance in source classification loss consistently outperforms the baseline “ w/o Align ” in both two tasks . This indicates that reweighting data importance in the source classification loss is more robust to weight noise , compared with the reweighted distribution alignment losses , in the two tasks . The above observations indicate that adapting the feature extractor by the reweighted distribution alignment is not robust to the noise in source data weights , and can cause negative domain transfer on some challenging datasets . Surprisingly , reweighting data importance in the source classification loss may be more robust to weight noise than the reweighted distribution alignment losses .

Summary: This paper proposes a new method called Adversarial Reweighting (AR) for partial domain adaptation (PDA) where the target label space is a subset of the source label space. AR aims to estimate the weight of source instances via an adversarial training manner and Wasserstein distance and then exploits the weight to train a simple objective (re-weighted cross-entropy on source + entropy minimization on target). Results on several PDA benchmarks verify the effectiveness of AR.