Continuous Transfer Learning

1 INTRODUCTION Source domain Target domain 𝓓𝓓𝑺𝑺 𝓓𝓓𝑻𝑻𝟏𝟏 𝓓𝓓𝑻𝑻𝟐𝟐 𝓓𝓓𝑻𝑻𝒕𝒕 𝓓𝓓𝑻𝑻𝒏𝒏⋯⋯𝓓𝓓𝑻𝑻𝟑𝟑 Negative transfer 𝓓𝓓𝑺𝑺 Figure 1 : Illustration of continuous transfer learning . It learns a predictive function in DTt using knowledge from both source domain DS and historical target domain DTi ( i = 1 , · · · , t 1 ) . Directly transferring from the source domain DS to the target domain DTt might lead to negative transfer with undesirable predictive performance . Transfer learning has achieved significant success across multiple high-impact application domains ( Pan & Yang , 2009 ) . Compared to conventional machine learning methods assuming both training and test data have the same data distribution , transfer learning allows us to learn the target domain with limited label information by leveraging a related source domain with abundant label information ( Ying et al. , 2018 ) . However , in many real applications , the target domain is constantly evolving over time . For example , the online movie reviews are changing over the years : some famous movies were not well received by the mainstream audience when they were first released , but became famous only years later ( e.g. , Citizen Cane , Fight Club , and The Shawshank Redemption ) ; whereas the online book reviews typically do not have this type of dynamics . It is challenging to transfer knowledge from the static source domain ( e.g. , the book reviews ) to the time evolving target domain ( e.g. , the movie reviews ) . Therefore , in this paper , we study the transfer learning setting with a static source domain and a continuously time evolving target domain ( see Figure 1 ) , which has not attracted much attention from the research community and yet is commonly seen across many real applications . The unique challenge for continuous transfer learning lies in the time evolving nature of the task relatedness between the static source domain and the time evolving target domain . Although the change in the target data distribution in consecutive time stamps might be small , over time , the cumulative change in the target domain might even lead to negative transfer ( Rosenstein et al. , 2005 ) . Existing theoretical analysis on transfer learning ( Ben-David et al. , 2010 ; Mansour et al. , 2009 ) showed that the target error is typically bounded by the source error , the domain discrepancy of marginal data distributions and the difference of labeling functions . However , it has been observed ( Zhao et al. , 2019 ; Wu et al. , 2019 ) that marginal feature distribution alignment might not guarantee the minimization of the target error in real world scenarios . This indicates that in the context of continuous transfer learning , marginal feature distribution alignment would lead to the sub-optimal solution ( or even negative transfer ) with undesirable predictive performance when directly transferring from DS to the target domain DTt at the tth time stamp . This paper aims to bridge the gap in terms of both the theoretical analysis and the empirical solutions for the target domain with a time evolving distribution , which lead to a novel continuous transfer learning algorithm as well as the characterization of negative transfer . The main contributions of this paper are summarized as follows : ( 1 ) We derive a generic error bound for continuous transfer learning setting with flexible domain divergence measures ; ( 2 ) We propose a label-informed domain discrepancy measure ( C-divergence ) with its empirical estimate , which instantiates a tighter error bound for continuous transfer learning setting ; ( 3 ) Based on the proposed C-divergence , we design a novel adversarial Variational Auto-encoder algorithm ( CONTE ) for continuous transfer learning ; ( 4 ) Extensive experimental results on various data sets verify the effectiveness of the proposed CONTE algorithm . The rest of the paper is organized as follows . Section 2 introduces the notation and our problem definition . We derive a generic error bound for continuous transfer learning setting in Section 3 . Then we propose a novel C-divergence in Section 4 , followed by a instantiated error bound and a novel continuous transfer learning algorithm in Section 5 . The experimental results are provided in Section 6 . We summarize the related work in Section 7 , and conclude the paper in Section 8 . 2 PRELIMINARIES . In this section , we introduce the notation and problem definition of continuous transfer learning . 2.1 NOTATION . We use X and Y to denote the input space and label space . Let DS and DT denote the source and target domains with data distribution pS ( x , y ) and pT ( x , y ) over X ⇥ Y , respectively . Let H be a hypothesis class on X , where a hypothesis is a function h : X ! Y . The notation is summarized in Table 3 in the appendices . 2.2 PROBLEM DEFINITION . Transfer learning ( Pan & Yang , 2009 ) refers to the knowledge transfer from source domain to target domain such that the prediction performance on the target domain could be significantly improved as compared to learning from the target domain alone . However , in some applications , the target domain is changing over time , hence the time evolving relatedness between the source and target domains . This motivates us to consider the transfer learning setting with the time evolving target domain , which is much less studied as compared to the static transfer learning setting . We formally define the continuous transfer learning problem as follows . Definition 2.1 . ( Continuous Transfer Learning ) Given a source domain DS ( available at time stamp j = 1 ) and a time evolving target domain { DTj } nj=1 with time stamp j , continuous transfer learning aims to improve the prediction function for target domain DTt+1 using the knowledge from source domain DS and the historical target domain DTj ( j = 1 , · · · , t ) . Notice that the source domain DS can be considered a special initial domain for the time-evolving target domain . Therefore , for notation simplicity , we will use DT0 to represent the source domain in this paper . It assumes that there are mT0 labeled source examples drawn independently from a source domain DT0 and mTj labeled target examples drawn independently from a target domain DTj at time stamp j . 3 A GENERIC ERROR BOUND . Given a static source domain and a time evolving target domain , continuous transfer learning aims to improve the target predictive function over DTt+1 using the source domain and historical target domain . We begin by considering the binary classification setting , i.e. , Y = { 0 , 1 } . The source error of a hypothesis h can be defined as follows : ✏T0 ( h ) = E ( x , y ) ⇠pT0 ( x , y ) ⇥ L ( h ( x ) , y ) ⇤ where L ( · , · ) is the loss function . Its empirical estimate using source labeled examples is denoted as ✏̂T0 ( h ) . Similarly , we define the target error ✏Tj ( h ) and the empirical estimate of the target error ✏̂Tj ( h ) over the target distribution pTj ( x , y ) at time stamp j . A natural domain discrepancy measure over joint distributions on X ⇥ Y between features and class labels can be defined as follows : d1 ( DT0 , DT ) = sup Q2Q PrDT0 [ Q ] PrDT [ Q ] ( 1 ) where Q is the set of measurable subsets under pT0 ( x , y ) and pT ( x , y ) 1 . Then , the error bound of continuous transfer learning is given by the following theorem . Theorem 3.1 . Assume the loss function L is bounded with 0 L M . Given a source domain DT0 and historical target domain { DTi } ti=1 , for h 2 H , the target domain error ✏Tt+1 on Dt+1 is 1Note that it is slightly different from L1 or variation divergence in ( Ben-David et al. , 2010 ) with only marginal distribution of features involved . bounded as follows . ✏Tt+1 ( h ) 1 µ̄ 0 @ tX j=0 µ t j ✏Tj ( h ) +M tX j=0 µ t j d1 ( DTj , DTt+1 ) 1 A where µ 0 is the domain decay rate2 indicating the importance of source or historical target domain over DTt+1 , and µ̄ = Pt j=0 µ t j . Remark . In particular , we have the following arguments . ( 1 ) It is not tractable to accurately estimate d1 from finite examples in real scenarios ( Ben-David et al. , 2010 ) ; ( 2 ) This error bound could be much tighter when considering other advanced domain discrepancy measures , e.g. , Adistance ( Ben-David et al. , 2007 ) , discrepancy distance ( Mansour et al. , 2009 ) , etc . ( 3 ) There are two special cases : when µ = 0 , the error bound of DTt+1 would be simply determined by the latest historical target data DTt , and if µ goes to infinity , DTt+1 is just determined by the source data DT0 because intuitively the coefficient µt j/µ̄ of historical target domain data DTj ( j = 1 , · · · , t ) converges to zero . Corollary 3.2 . With the assumption in Theorem 3.1 and assume that the loss function L is symmetric ( i.e. , L ( y1 , y2 ) = L ( y2 , y1 ) for y1 , y2 2 Y ) and obeys the triangle inequality , Then ( 1 ) if A-distance ( Ben-David et al. , 2007 ) is adopted to measure the distribution shift , i.e. , dH H = suph , h02H PrDT0 [ h ( x ) 6= h 0 ( x ) ] PrDT [ h ( x ) 6= h 0 ( x ) ] , we have : ✏Tt+1 ( h ) 1 µ̄ 0 @ tX j=0 µ t j ✏Tj ( h ) +M tX j=0 µ t j ✓ dH H ( DTj , DTt+1 ) + ⇤ j M ◆1 A where ⇤j = minh2H ✏Tj ( h ) + ✏Tt+1 ( h ) . ( 2 ) if discrepancy distance ( Mansour et al. , 2009 ) is adopted to measure the distribution shift , i.e. , ddisc ( DT0 , DT ) = maxh , h02H EDT0 [ L ( h ( x ) , h 0 ( x ) ) ] EDT [ L ( h ( x ) , h0 ( x ) ) ] , we have : ✏Tt+1 ( h ) 1 µ̄ 0 @ tX j=0 µ t j ✏Tj ( h ) + tX j=0 µ t j ddisc ( DTj , DTt+1 ) + ⌦j 1 A where ⌦j = EDTj [ L ( h ⇤ j ( x ) , y ) ] + EDTt+1 [ L ( h ⇤ j ( x ) , h ⇤ t+1 ( x ) ) ] + EDTt+1 [ L ( h ⇤ t+1 ( x ) , y ) ] , and h ⇤ j = argminh2H ✏Tj ( h ) for j = 0 , · · · , t , t+ 1 . The aforementioned domain discrepancy measures mainly focus on the marginal distribution over input features and have inspired a line of practical transfer learning algorithms ( Ganin et al. , 2016 ; Chen et al. , 2019 ) . However , recent work ( Wu et al. , 2019 ; Zhao et al. , 2019 ) observed that the minimization of marginal distributions can not guarantee the success of transfer learning in real scenarios . We propose to address this problem by incorporating the label information in the domain discrepancy measure ( see next section ) .

Summary: The paper proposed a transfer learning setting where the target domain varies/evolves over time and the source domain is considered static. The paper uses C-divergence to measure label-dependent domain discrepancy between source/previous target domain and the current target domain and provided a theoretical bound. The paper also used supervised VAE for CONTE algorithm and included C-divergence as a part of the objective function.