Wildly Unsupervised Domain Adaptation and Its Powerful and Efficient Solution

1 INTRODUCTION . Domain adaptation ( DA ) aims to learn a discriminative classifier in the presence of a shift between training data in source domain and test data in target domain ( Ben-David et al. , 2010 ; Ganin and Lempitsky , 2015 ; Xiao and Guo , 2015 ; Zhang et al. , 2015 ; 2013 ) . Currently , DA can be divided into three categories : supervised DA ( Tzeng et al. , 2015 ) , semi-supervised DA ( Guo and Xiao , 2012 ) and unsupervised DA ( UDA ) ( Saito et al. , 2017 ) . When the number of labeled data is few in target domain , supervised DA is also known as few-shot DA ( Motiian et al. , 2017 ) . Since unlabeled data in target domain can be easily obtained , UDA exhibits the greatest potential in the real world ( Ganin and Lempitsky , 2015 ; Ganin et al. , 2016 ; Gong et al. , 2012 ; 2016 ; Long et al. , 2015 ; Saito et al. , 2017 ; 2018 ) . UDA methods train with clean labeled data in source domain ( i.e. , clean source data ) and unlabeled data in target domain ( i.e. , unlabeled target data ) to obtain classifiers for the target domain ( TD ) , which mainly consist of three orthogonal techniques : integral probability metrics ( IPM ) ( Ghifary et al. , 2017 ; Gong et al. , 2016 ; Gretton et al. , 2012 ; Lee and Raginsky , 2018 ; Long et al. , 2015 ) , adversarial training ( Ganin et al. , 2016 ; Gong et al. , 2018 ; Hoffman et al. , 2018 ; Li et al. , 2018 ; Saito et al. , 2018 ; Tzeng et al. , 2017 ) and pseudo labeling ( Saito et al. , 2017 ) . Compared to IPM- and adversarial-training-based methods , the pseudolabeling-based method ( i.e. , asymmetric tri-training domain adaptation ( ATDA ) ( Saito et al. , 2017 ) ) can construct a high-quality target-specific representation , providing a better classification performance . However , in the wild , the data volume of source domain tends to be large . To avoid the expensive labeling cost , labeled data in source domain normally come from amateur annotators or the Internet ( Lee et al. , 2018 ; Schroff et al. , 2011 ; Tommasi and Tuytelaars , 2014 ) . This brings us a new , more realistic and more challenging problem , wildy unsupervised domain adaptation ( abbreviated as WUDA , Figure 1 ) . This adaptation aims to transfer knowledge from noisy labeled data in source domain ( P̃s , i.e. , noisy source data ) to unlabeled target data ( Pxt ) . Unfortunately , existing UDA methods share an implicit assumption that there are no noisy source data . Namely , these methods focus on transferring knowledge from clean source data ( Ps ) to unlabeled target data ( Pxt ) . Therefore , these methods can not well handle the WUDA . To validate this fact , we empirically reveal the deficiency of existing UDA methods ( Figure 2 ) . To improve these methods , a straightforward solution is a two-step approach . In Figure 1 , we can first use label-noise algorithms to train a model on noisy source data , then leverage this trained model to assign pseudo labels for noisy source data . Via UDA methods , we can transfer knowledge from pseudo-labeled source data ( P̂s ) to unlabeled target data ( Pxt ) . Nonetheless , pseudo-labeled source data are still noisy , and such two-step approach may not eliminate noise effects . To circumvent the issue of two-step approach , we present a robust one-step approach called Butterfly . In high level , Butterfly directly transfers knowledge from P̃s to Pxt , and uses the transferred knowledge to construct target-specific representations . In low level , Butterfly maintains four networks dividing two branches ( Figure 3 ) : Two networks in Branch-I are jointly trained on noisy source data and pseudo-labeled target data ( data in mixture domain ( MD ) ) ; while two networks in Branch-II are trained on pseudo-labeled target data . The reason why Butterfly can be robust takes root in the dual-checking principle ( DCP ) : Butterfly checks high-correctness data out , from not only the data in MD but also the pseudo-labeled target data . After crosspropagating these high-correctness data , Butterfly can obtain high-quality domain-invariant representations ( DIR ) and target-specific representations ( TSR ) simultaneously in an iterative manner . If we only check data in MD ( i.e. , single checking ) , the error existed in pseudo-labeled target data will accumulate , leading to low-quality DIR and TSR . We conduct experiments on simulated WUDA tasks , including 4 MNIST-to-SYND tasks , 4 SYND-to-MNIST tasks and 24 human-sentiment tasks . Besides , we conduct experiments on 3 real-world WUDA tasks . Empirical results demonstrate that Butterfly can robustly transfer knowledge from noisy source data to unlabeled target data . Meanwhile , Butterfly performs much better than existing UDA methods when source domain ( SD ) suffers the extreme ( e.g. , 45 % ) noise . 2 WILDLY UNSUPERVISED DOMAIN ADAPTATION . In this section , we first define a new , more realistic and more challenging setting called wildly unsupervised domain adaptation ( WUDA ) , and explain the nature of WUDA . Then , we empirically show that representative UDA methods can not handle WUDA well , which motivates us to propose Butterfly ( see Section 3 ) . Definition 1 ( Wildly Unsupervised Domain Adaptation ) . Let Xt be a multivariate random variable defined on the space X with respective a probability density pxt , ( Xs , Ys ) be a multivariate random variable defined on the space X ×C with respective a probability density p̃s , where C = { 1 , . . . , K } . Let pxs be the marginal probability density of p̃s . Given i.i.d . data D̃s = { ( xsi , ỹsi ) } nsi=1 and Dt = { xti } nt i=1 drawn from P̃s and Pxt , in wildly unsupervised domain adaptation , we aim to train with D̃s and Dt to accurately annotate data drawn from Pxt , where pxs 6= pxt . Remark 1 . In Definition 1 , D̃s is noisy source data , Dt is unlabeled target data , and P̃s and Pxt are two probability measures corresponding to densities p̃s ( xs , ỹs ) and pxt ( xt ) . Nature of WUDA . Specifically , there are five distributions involved in WUDA setting : 1 ) a marginal distribution on source data , i.e. , pxs in Definition 1 ; 2 ) a marginal distribution on target data , i.e. , pxt in Definition 1 ; 3 ) an incorrect conditional distribution of label given xs , q ( ys|xs ) ; 4 ) a correct conditional distribution of label given xs , p ( ys|xs ) and 5 ) a correct conditional distribution of label given xt , p ( yt|xt ) . Based on Definition 1 and Appendix A.2 , noisy source data D̃s are drawn from p̃s ( xs , ys ) = pxs ( xs ) ( ( 1− ρ ) p ( ys|xs ) + ρq ( ys|xs ) ) , where ρ is the noise rate in source data . Namely , source data D̃s are mixture of correct source data from pxs ( xs ) p ( ys|xs ) and incorrect data from pxs ( xs ) q ( ys|xs ) . Target data Dt are drawn from pxt . In WUDA setting , we aim to train a classifier with D̃s and Dt . This classifier is expected to accurately annotate data from pxt , i.e. , to accurately simulate distribution 5 ) . WUDA ruins representative UDA methods . We take a simple example to illustrate why WUDA ruins representative UDA methods . We corrupt source data using symmetry flipping ( Patrini et al. , 2017 ) and pair flipping ( Han et al. , 2018 ) ( Appendix B ) . Namely , the corrupted source data ( D̃s in Definition 1 ) are drawn from P̃s whose density is p̃s ( xs , ys ) = pxs ( xs ) ( ( 1− ρ ) p ( ys|xs ) + ρq ( ys|xs ) ) . We draw the target data Dt from Pxt whose density is pxt . To instantiate source and target data , we leverage MNIST and SYND ( Figure 6 in Appendix C ) , respectively . Thus , we first construct two WUDA tasks with symmetry-flip noise : corrupted SYND→MNIST ( S→M ) and corrupted MNIST→SYND ( M→S ) . In Figure 2 ( a ) , we report target-domain accuracy of representative UDA methods , when the noise rate ρ of SD changes from 5 % to 70 % . It is clear that target-domain accuracy of these representative UDA methods drops quickly when ρ increases . This means that WUDA ruins representative UDA methods . Then , we construct another two WUDA tasks with pair-flip noise . In Figure 2 ( b ) , we report target-domain accuracy , when the noise rate ρ of SD changes from 5 % to 45 % . Again , WUDA still ruins representative UDA methods . Note that pair-flip noise is much harder than symmetry-flip noise , and its noise rate can not be over 50 % in practice ( Han et al. , 2018 ) . However , the proposed Butterfly network ( abbreviated as B-Net , Figure 3 ) performs robustly when ρ increases ( blue lines in Figure 2 ) . In following sections , we will introduce Butterfly framework , and explain why Butterfly achieves better target-domain accuracy consistently . 3 BUTTERFLY : TOWARDS ROBUST ONE-STEP APPROACH . To realize a robust WUDA approach , we propose a Butterfly framework ( a one-step approach , Algorithm 1 ) , which trains four networks dividing into two branches ( Figure 3 ) . By using DCP , Branch-I checks which data is correct in MD ; while Branch-II checks which pseudo-labeled target data is correct . To ensure these checked data highly-correct , we apply the small-loss trick based on memorization effects of deep learning ( Arpit et al. , 2017 ) . After cross-propagating these checked data ( Bengio , 2014 ) , Butterfly can obtain high-quality DIR and TSR simultaneously in an iterative manner . Loss function in Butterfly . Four networks trained by Butterfly share the same loss function but with different inputs . L ( θ , u ; F , D ) = 1∑n i=1 ui n∑ i=1 ui ` ( F ( xi ) , y̌i ) , ( 1 ) where n is the batch size , and F represents a network ( e.g. , F1 , F2 , Ft1 and Ft2 ) . D = { ( xi , y̌i ) } ni=1 is a mini-batch for training a network , where { xi , y̌i } ni=1 could be data in MD or TD ( Figure 3 ) , and θ represents parameters of F and u = [ u1 , ... , un ] T is an n-by-1 vector whose elements equal 0 or 1 . ` ( · , · ) is the cross-entropy loss . For two networks in Branch-I , following Saito et al . ( 2017 ) , we also add a regularizer |θTf11θf21| in their loss functions , where θf11 and θf21 are weights of the first fully-connect layer of F1 and F2 . With this regularizer , F1 and F2 will learn from different features . Nature of the loss L. In loss function L , we have n instances : ( xi , y̌i ) , where i = 1 , 2 , . . . , n. For the ith instance , we will compute its cross-entropy loss ( i.e. , ` ( F ( xi ) , y̌i ) ) , and we will denote this instance as “ selected ” if ui = 1 . Thus , the nature of L is actually the average value of cross-entropy loss of these “ selected ” instances . Note that , we need to set a constrain to prevent ∑n i=1 ui = 0 in L , which means that we need to select at least one instance to compute L. Training procedure of Butterfly . For two networks in each branch , they will first check high-correctness data out and then cross update their parameters using these data . Based on loss function defined in Eq . ( 1 ) , entire training procedures of Butterfly are shown in Algorithm 1 . First , we initialize training data for two branches ( D̃ for Branch-I and D̃lt for Branch-II ) , four networks ( F1 , F2 , Ft1 and Ft2 ) and the number of pseudo labels ( line 2 ) . In the first epoch ( T = 1 ) , following Saito et al . ( 2017 ) , D̃lt is the same with D̃s ( i.e. , we use noisy source data as pseudo-labeled target data ) since we can not annotate pseudo labels for target data when T = 1 . After mini-batch Ď is fetched from D̃ ( line 4 ) , F1 and F2 check high-correctness data out and update their parameters ( lines 5 ) using Algorithm 2 . Using similar procedures , Ft1 and Ft2 also update their parameters using Algorithm 2 ( lines 6-7 ) . In each epoch , after Nmax mini-batch updating , we randomly select nlt unlabeled target data and assign them pseudo labels using F1 and F2 ( lines 8 ) . Following Saito et al . ( 2017 ) , the Labeling function in Algorithm 1 ( line 8 ) assigns pseudo labels for unlabeled target data , when predictions of F1 and F2 agree and at least one of them is confident about their predictions ( probability above 0.9 or 0.95 ) . Using this function , we can obtain the pseudo-labeled target data D̃lt for training Branch-II in the next epoch . Then , we merge D̃ l t and D̃s to be D̃ for training Branch-I in the next epoch ( line 9 ) . Finally , we update nlt , R ( T ) and Rt ( T ) in lines 10-11 according to Saito et al . ( 2017 ) and Han et al . ( 2018 ) . Note that R ( T ) is a piecewise-defined linear function . Namely , when T ≥ Tk , R ( T ) = 1− τ ; when T ≤ Tk , R ( T ) = 1− T/Tk × τ . In Algorithm 1 , we use τ to represent the noise rate ( i.e. , the ratio of data with incorrect labels ) in MD and use τt to represent the noise rate in TD . However , in WUDA , we can not obtain the ground-truth τ and τt . Thus , we regard τ and τt as hyper-parameters . Note that τ and τt are robustly set to 0.4 and 0.05 in experiments . Checking process in Butterfly . In Algorithm 2 , we first obtain four inputs : 1 ) networks F1 and F2 , and 2 ) a mini-batch D , and 3 ) learning rate η and 4 ) remember rate α ( line 1 ) . Then , we will obtain the best u1 by solving a minimization problem ( line 2 ) . L represents the loss function defined in Eq . ( 1 ) . θ1 represents the parameters of the network F1 . Similarly we will obtain the best u2 ( line 3 ) . θ2 represents the parameters of the network F2 . Next , θ1 and θ2 are updated using gradient descent , where the gradients are computed using a given optimizer ( lines 4-5 ) . Finally , we substitute the updated θ1 into F1 and the updated θ2 into F2 and output F1 and F2 ( line 6 ) . Solution to minimization problems in Algorithm 2 . In line 2 or 3 in Algorithm 2 , we need to solve a minimization problem : minu′:1u′ > α|D| L ( θ , u′ ; F , D ) and return the best u′ as u ( u1 in line 2 and u2 in line 3 ) . In this paragraph , we will show how to quickly solve this problem using a sorting algorithm . Recall the nature of the loss L , we know L is the average value of cross-entropy losses of “ selected ” instances , and 1u′ is the number of these “ selected ” instances . Thus this minimization problem is equivalent to “ given a fixed F ( F1 or F2 ) and n instances in D , how to select at least k instances such that L is minimized ” , where k = dα|D|e . To solve this problem , we first use a sorting algorithm ( top_k function in TensorFlow ) to sort these n instances according to their cross-entropy losses ` ( F1 ( xi ) , y̌i ) . Then , we select k instance with the smallest cross-entropy losses . Finally , let ui of these k instances be 1 and ui of the other instances be 0 , and we can get the best u = [ u1 , . . . , un ] . The average value of cross-entropy losses of these k instances is the minimized value of L ( θ , u′ ; F , D ) under the constrain 1u′ > α|D| .

Summary: This paper introduces the idea of wildly unsupervised domain adaptation, where the source labels are noisy and the target data is unsupervised. To tackle this, the authors propose an architecture based one two branches: one acting on the mixed source-target data and the other on the target data only. During training, each branch is updated using the idea of co-teaching, by finding the samples with the lowest loss values. Pseudo-labeling is then applied to the target data, and the process iterates.