Unified Principles For Multi-Source Transfer Learning Under Label Shifts

1 INTRODUCTION . Transfer learning ( Pan & Yang , 2009 ) is based on the motivation that learning a new task is easier after having learned several similar tasks . By learning the inductive bias from a set of related source domains ( S1 , . . . , ST ) and then leveraging the shared knowledge upon learning the target domain T , the prediction performance can be significantly improved . Based on this , transfer learning arises in deep learning applications such as computer vision ( Zhang et al. , 2019 ; Tan et al. , 2018 ; Hoffman et al. , 2018b ) , natural language processing ( Ruder et al. , 2019 ; Houlsby et al. , 2019 ) and biomedical engineering ( Raghu et al. , 2019 ; Lundervold & Lundervold , 2019 ; Zhang & An , 2017 ) . To ensure a reliable transfer , it is critical to understand the theoretical assumptions between the domains . One implicit assumption in most transfer learning algorithms is that the label proportions remain unchanged across different domains ( Du Plessis & Sugiyama , 2014 ) ( i.e. , S ( y ) = T ( y ) ) . However , in many real-world applications , the label distributions can vary markedly ( i.e . label shift ) ( Wen et al. , 2014 ; Lipton et al. , 2018 ; Li et al. , 2019b ) , in which existing approaches can not guarantee a small target generalization error , which is recently proved by Combes et al . ( 2020 ) . Moreover , transfer learning becomes more challenging when transferring knowledge from multiple sources to build a model for the target domain , as this requires an effective selection and leveraging the most useful source domains when label shift occurs . This is not only theoretically interesting but also commonly encountered in real-world applications . For example , in medical diagnostics , the disease distribution changes over countries ( Liu et al. , 2004 ; Geiss et al. , 2014 ) . Considering the task of diagnosing a disease in a country without sufficient data , how can we leverage the information from different countries with abundant data to help the diagnosing ? Obviously , naı̈vely combining all the sources and applying one-to-one single source transfer learning algorithm can lead to undesired results , as it can include low quality or even untrusted data from certain sources , which can severely influence the performance . In this paper , we study the label shift problem in multi-source transfer learning where St ( y ) 6= T ( y ) . We propose unified principles that are applicable for three common transfer scenarios : unsupervised Domain Adaptation ( DA ) ( Ben-David et al. , 2010 ) , limited target labels ( Mansour et al. , 2020 ) and partial unsupervised DA with supp ( T ( y ) ) ⊆ supp ( St ( y ) ) ( Cao et al. , 2018 ) , where prior works generally treated them as separate scenario . It should be noted that this work deals with target shift without assuming that semantic conditional distributions are identical ( i.e. , St ( x|y ) 6= T ( x|y ) ) , which is more realistic for real-world problems . Our contributions in this paper are two-folds : ( I ) We propose to use Wasserstein distance ( Arjovsky et al. , 2017 ) to develop a new target generalization risk upper bound ( Theorem 1 ) , which reveals the importance of label distribution ratio estimation and provides a principled guideline to learn the domain relation coefficients . Moreover , we provide a theoretical analysis in the context of representation learning ( Theorem 2 ) , which guides to learn a feature function that minimizes the conditional Wasserstein distance as well as controls the weighted source risk . We further reveal the relations in the aforementioned three scenarios lie in the different assumptions for estimating label distribution ratio . ( II ) Inspired by the theoretical results , we propose Wasserstein Aggregation Domain Network ( WADN ) for handling label-shift in multi-source transfer learning . We evaluate our algorithm on three benchmark datasets , and the results show that our algorithm can significantly outperform stateof-the-art principled approaches . 2 RELATED WORK . Multi-Source Transfer Learning Theories have been investigated in the previous literature with different principles to aggregate source domains . In the popular unsupervised DA , ( Zhao et al. , 2018 ; Peng et al. , 2019 ; Wen et al. , 2020 ; Li et al. , 2018b ) adoptedH-divergence ( Ben-David et al. , 2007 ) , discrepancy ( Mansour et al. , 2009 ) and Wasserstein distance ( Arjovsky et al. , 2017 ) of marginal distribution d ( St ( x ) , T ( x ) ) to estimate domain relations and dynamically leveraged different domains . These algorithms generally consists source risk , domain discrepancy and an un-observable term η , the optimal risk on all the domains , which are ignored in these approaches . However , as Combes et al . ( 2020 ) pointed out , ignoring the influence of η will be problematic when label distributions between source and target domains are significantly different . Therefore it is necessary to take η into consideration by using a small amount of labelled data is available for the target domain ( Wen et al. , 2020 ) . Following this line , very recent works ( Konstantinov & Lampert , 2019 ; Wang et al. , 2019a ; Mansour et al. , 2020 ) started to consider measure the divergence between two domains given label information for the target domain by using Y-discrepancy ( Mohri & Medina , 2012 ) . However , we empirically showed these methods are still unable to handle label shift . Label-Shift Label-Shift ( Zhang et al. , 2013 ; Gong et al. , 2016 ) is a common phenomena in the transfer learning with S ( y ) 6= T ( y ) and generally ignored by the previous multi-source transfer learning practice . Several theoretical principled approaches have been proposed such as ( Azizzadenesheli et al. , 2019 ; Garg et al. , 2020 ) . In addition , ( Combes et al. , 2020 ; Wu et al. , 2019 ) analyzed the generalized label shift problem in the one-to-one single unsupervised DA problem but did not provide guidelines of levering different sources to ensure a reliable transfer , which is more challenging . ( Redko et al. , 2019 ) proposed optimal transport strategy for the multiple unsupervised DA under label shift by assuming identical semantic conditional distribution . However they did not consider representation learning in conjunction with their framework and did not design neural network based approaches . Different from these , we analyzed our problem in the context of representation learning and propose an efficient and principled strategies . Moreover , our theoretical results highlights the importance of label shift problem in a variety of multi-source transfer problem . While the aforementioned work generally focus on the unsupervised DA problem , without considering unified rules for different scenarios ( e.g . partial multi-source DA ) . 3 THEORETICAL INSIGHTS : TRANSFER RISK UPPER BOUND . We assume a scoring hypothesis defined on the input space X and output space Y with h : X ×Y → R , is K-Lipschitz w.r.t . the feature x ( given the same label ) , i.e . for ∀y , ‖h ( x1 , y ) − h ( x2 , y ) ‖2 ≤ K‖x1 − x2‖2 , and the loss function ` : R × R → R+ is positive , L-Lipschitz and upper bounded by Lmax . We denote the expected risk w.r.t distribution D : RD ( h ) = E ( x , y ) ∼D ` ( h ( x , y ) ) and its empirical counterpart ( w.r.t . D̂ ) R̂D ( h ) = ∑ ( x , y ) ∈D̂ ` ( h ( x , y ) ) . We adopted Wasserstein-1 distance ( Arjovsky et al. , 2017 ) as a metric to measure the similarity of the domains . Compared with other divergences , Wasserstein distance has been theoretically proved tighter than TV distance ( Gong et al. , 2016 ) or Jensen-Shnannon divergence ( Combes et al. , 2020 ) . Based on previous work , the label shift is generally handled by label-distribution ratio weighted loss : RαS ( h ) = E ( x , y ) ∼Sα ( y ) ` ( h ( x , y ) ) with α ( y ) = T ( y ) /S ( y ) . We also denote α̂t as its empirical counterpart , estimated from samples . Besides , to measure the task relations , we define a simplex λ with λ [ t ] ≥ 0 , ∑T t=1 λ [ t ] = 1 as the task relation coefficient vector by assigning high weight to the similar task . Then we first present Theorem 1 , which proposed theoretical insights about how to combine source domains through properly estimating λ. Theorem 1 . Let { Ŝt = { ( xi , yi ) } NSt i=1 } Tt=1 and T̂ = { ( xi , yi ) } NT i=1 , respectively be T source and target i.i.d . samples . For ∀h ∈ H with H the hypothesis family and ∀λ , with high probability ≥ 1− 4δ , the target risk can be upper bounded by : RT ( h ) ≤ ∑ t λ [ t ] R̂α̂tSt ( h ) + LK ∑ t λ [ t ] Ey∼T̂ ( y ) W1 ( T̂ ( x|Y = y ) ‖Ŝt ( x|Y = y ) ) + Lmaxd sup ∞ √√√√ T∑ t=1 λ [ t ] 2 βt √ log ( 1/δ ) 2N + Lmax sup t ‖αt − α̂t‖2 + Comp ( NS1 , . . . , NST , NT , δ ) , where N = ∑T t=1NSt and βt = NSt/N and d sup ∞ = maxt∈ [ 1 , T ] , y∈ [ 1 , Y ] αt ( y ) the maximum true label distribution ratio value . W1 ( ·‖· ) is the Wasserstein-1 distance with L2-distance as cost function . Comp ( NS1 , . . . , NST , NT , δ ) is a function that decreases with larger NS1 , . . . , NT , given a fixed δ and hypothesis familyH . ( See Appendix E for details ) Remarks ( 1 ) In the first two terms , the relation coefficient λ is controlled by αt-weighted loss R̂α̂tSt ( h ) and conditional Wasserstein distance Ey∼T̂ ( y ) W1 ( T̂ ( x|Y = y ) ‖Ŝt ( x|Y = y ) ) . To minimize the upper bound , we need to assign a higher λ [ t ] to the source t with a smaller weighted prediction loss and a smaller weighted semantic conditional Wasserstein distance . Intuitively , we tend to leverage the source task which is semantic similar to the target and easier to learn . ( 2 ) If each source have equal observations with βt = 1 , then the third term will become ‖λ‖2 , a L2 norm regularization , which can be viewed as an encouragement of uniformly leveraging all the sources . Combing these three terms , we need to consider the trade-off between assigning a higher λ [ t ] to the source t that has a smaller weighted prediction loss and conditional Wasserstein distance , and assigning balanced λ [ t ] for avoiding concentrating on only one source . ( 3 ) ‖α̂t − αt‖2 indicates the gap between ground-truth and empirical label ratio . Therefore if we can estimate a good α̂t , these terms can be small . In the practice , If target labels are available , α̂t can be computed from the observed data and α̂t → αt . If target labels are absent ( unsupervised DA ) , we need to design methods and to properly estimate α̂t ( Sec . 4 ) . ( 4 ) Comp ( NS1 , . . . , NST , NT , δ ) is a function that reflects the convergence behavior , which decreases with larger observation numbers . If we fixH , δ , N and NT , this term can be viewed as a constant . Insights in Representation Learning Apart from Theorem 1 , we propose a novel theoretical analysis in the context of representation learning , which motivates practical guidelines in the deep learning regime . We define a stochastic feature function g and we denote its conditional distribution w.r.t . latent variable Z ( induced by g ) as S ( z|Y = y ) = ∫ x g ( z|x ) S ( x|Y = y ) dx . Then we have : Theorem 2 . We assume the settings of loss , the hypothesis are the same with Theorem 1 . We further denote the stochastic feature learning function g : X → Z , and the hypothesis h : Z × Y → R. Then ∀λ , the target risk is upper bounded by : RT ( h , g ) ≤ ∑ t λ [ t ] RαtSt ( h , g ) + LK ∑ t λ [ t ] Ey∼T ( y ) W1 ( St ( z|Y = y ) ‖T ( z|Y = y ) ) , where RT ( h , g ) = E ( x , y ) ∼T ( x , y ) Ez∼g ( z|x ) ` ( h ( z , y ) ) . Theorem 2 reveal that to control the upper bound , we need to learn g that minimizes the weighted conditional Wasserstein distance and learn ( g , h ) that minimizes the weighted source risk . Comparison with previous Theorems . Our theory proposed an alternative prospective to understand transfer learning . The first term is α-weighted loss . And it will recover the typical source loss minimization if there is no label shift with αt ( y ) ≡ 1 ( Li et al. , 2019a ; Peng et al. , 2019 ; Zhao et al. , 2018 ; Wen et al. , 2020 ) . Beside , minimizing the conditional Wasserstein distances has been shown to be advantageous , compared with W1 ( St ( z ) ‖T ( z ) ) ( Long et al. , 2018 ) . Moreover , Theorem 2 explicitly proposed the theoretical insights about the representation learning function g , which remains elusive for previous multi-source transfer theories such as ( Wang et al. , 2019a ; Mansour et al. , 2020 ; Konstantinov & Lampert , 2019 ; Li et al. , 2019a ; Peng et al. , 2019 ) .

Summary: This paper aims to provide a unified principle for multi-source transfer learning under label shifts. Based on this principle, this paper claims that a unified algorithm is proposed for various multi-source label shift transfer scenarios: learning with limited target data, unsupervised domain adaptation and label partial unsupervised domain adaptation. The proposed algorithm is validated on three benchmark datasets. The proof seems correct via combining existing single-domain DA theory and the theory regarding Wasserstein distance. The main theorem (Theorem 1) assumes that we can get the label information in the target domain, which is not realistic in many DA problem settings (e.g., UDA or multi-source UDA in this paper). In many DA problem settings, we have to use pseudo labels to replace with true labels in the target domain, which should be analysed in the proposed theorem. However, this paper does not make any efforts to theoretically analyse the effect of pseudo labels, which results in that this paper has very limited impacts on the DA field. Besides, there are some misleading conclusions in this paper.