Gradual Domain Adaptation without Indexed Intermediate Domains

1 Introduction . The distributions of real-world data change dynamically due to many factors like time , locations , environments , etc . Such a fact poses a great challenge to machine-learned models , which implicitly assume that the test data distribution is covered by the training data distribution . To resolve this generalization problem , unsupervised domain adaption ( UDA ) , which aims to adapt a learned model to the test domain given its unlabeled data [ 13 , 15 ] , has been an active sub-field in machine learning . Typically , UDA assumes that the “ source ” domain , in which the model is trained , and the “ target ” domain , in which the model is deployed , are discrepant but sufficiently related . Concretely , Ben-David et al . [ 1 ] , Zhao et al . [ 75 ] show that the generalization error of UDA is bounded by the discrepancy of the marginal or conditional distributions between domains . Namely , the effectiveness of UDA may degrade along with the increase in domain discrepancy . Take one popular algorithm , self-training [ 38 , 48 , 49 ] , for example . Self-training adapts the source model by progressively labeling the unlabeled target data ( i.e. , pseudo-labels ) and using them to fine-tune the model [ 2 , 28 , 30 , 31 , 41 , 44 , 77 , 78 ] . Self-training works if the pseudo-labels are accurate ( as it essentially becomes supervised learning ) , but is vulnerable if they are not , which occurs when there exists a large domain gap [ 35 ] . To address this issue , several recent works investigate gradual domain adaption ( GDA ) [ 12 , 25 , 35 , 71 ] , in which beyond the source and target domains , the model can access additional unlabeled data from the intermediate domains that shift gradually from the source to the target . By adapting 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . the model along the sequence of intermediate domains — i.e. , from the ones close to the source to the ones close to the target — the large domain gap between source and target is chipped away by multiple sub-adaptation problems ( between consecutive intermediate domains ) whose domain gaps are smaller . Namely , every time the model moves a step closer to the target , instead of taking a huge jump that can significantly decrease the performance ( see Figure 1 for an illustration ) . GDA makes sense , as real-world data change gradually more often than abruptly [ 11 , 60 , 66 ] . The recent work by Kumar et al . [ 35 ] further demonstrates the strength of GDA both empirically and theoretically . One potential drawback of GDA is the need for a well-defined sequence of intermediate domains . That is , prior to adaptation , the additional unlabeled data must be grouped into multiple domains and indexed with intermediate domain labels that reflect the underlying data distribution shift from the source to the target . Such information , however , may not be available directly . Existing methods usually leverage side information like time tags [ 25 , 35 , 71 ] to define the sequence , which may be sub-optimal . In some applications , even the side information may not be accessible ( e.g. , due to privacy concerns ) , greatly limiting the applicability of GDA . In this paper , we therefore study GDA in the extreme case — the additional unlabeled data are neither grouped nor indexed . Specifically , we investigate how to discover the “ domain sequence ” from data , such that it can be used to drive GDA . We propose a two-stage coarse-to-fine framework named Intermediate DOmain Labeler ( IDOL ) . In the first stage , IDOL labels each intermediate data instance with a coarse score that reflects how close it is to the source or target . We study several methods that estimate the distance from a data instance to a domain . We find that a progressively trained domain discriminator — which starts with source vs. target data but gradually adds data close to either of them into training — performs the best , as it better captures the underlying data manifold . In the second stage , IDOL then builds upon the coarse scores to group data into domains by further considering the discriminative ( e.g. , classification ) knowledge the model aims to preserve from the source to the target . Since the additional unlabeled data are fully unlabeled , we take a greedy approach to identify the next intermediate domain ( i.e. , a group of data ) that can best preserve the discriminative knowledge in the current intermediate domain . Concretely , we employ self-training [ 38 ] along the domain sequence discovered so far to provide pseudo-labels for the current domain , and propose a novel cycle-consistency loss to discover the next domain , such that the model adapted to the next domain can be “ adapted back ” to the current domain and predict the same pseudo-labels . The output of IDOL is a sequence of intermediate domains that can be used by any GDA algorithms [ 25 , 35 , 71 ] . We validated IDOL on two data sets studied in [ 35 ] , including Rotated MNIST [ 36 ] and Portraits over years [ 14 ] . IDOL can successfully discover the domain sequence that leads to comparable GDA performance to using the pre-defined sequence ( i.e. , by side information ) . More importantly , IDOL is compatible with pre-defined sequences — by treating them as the coarse sequences — to further improve upon them . We also investigate IDOL in scenarios where the additional unlabeled data may contain outliers and demonstrate IDOL ’ s effectiveness even in such challenging cases . To our knowledge , our work is the first to tackle GDA without grouped and indexed intermediate domains . The success of IDOL opens up broader application scenarios that GDA can contribute to . 2 Background : gradual domain adaptation ( GDA ) . 2.1 Setup of UDA and GDA . We consider an unsupervised domain adaptation ( UDA ) problem , in which a classifier is provided with labeled data from the source domain S = { ( xSi , ySi ) } |S| i=1 and unlabeled data from the target domain T = { xTi } |T | i=1 . Both S and T are sampled IID from the underlying joint distributions of the source PS and target PT , respectively . A source model θS is typically produced by training on S directly . The goal of UDA is to learn a target model θT such that it can perform well on the target , measured by a loss L ( θT , PT ) . We focus on the standard UDA setup that assumes no label shifts [ 6 ] . Gradual domain adaptation ( GDA ) , on top of UDA , assumes the existence of a sequence of domains P0 , P1 , . . . PM−1 , PM , where P0 and PM are the source domain and target domain , respectively , along with M − 1 intermediate domains P1 , . . . , PM−1 . The data distributions of domains are assumed to change gradually from the source to the target along the sequence . One can access unlabeled data Um = { xUmi } |Um| i=1 from each of the intermediate domain , forming a sequence of unlabeled data U1 , . . . , UM−1 . We denote the union of them by U = { xUi } |U| i=1 . For brevity , we assume that each Um is disjoint from the others but the size |Um| is the same ∀m ∈ { 1 , · · · , M − 1 } . 2.2 GDA with pre-defined domain sequences using self-training [ 35 ] . We summarize the theoretical analysis of GDA using self-training by Kumar et al . [ 35 ] as follows . Self-training for UDA . Self-training [ 38 ] takes a pre-trained ( source ) model θ and the target unlabeled data T as input . Denote by z = f ( x ; θ ) the predicted logits z , self-training first applies f ( · ; θ ) to target data points x ∈ T , sharpens the predictions by argmax into pseudo-labels , and then updates the current model θ by minimizing the loss ` ( e.g. , cross entropy ) w.r.t the pseudo-labels , θT = ST ( θ , T ) = arg min θ′∈Θ 1 |T | |T |∑ i=1 ` ( f ( xi ; θ ′ ) , sharpen ( f ( xi ; θ ) ) ) , ( 1 ) where θT denotes the resulting target model and ST ( θ , T ) denotes the self-training process . A baseline use of self-training for UDA is to set θ as the source model θS . However , when the domain gap between S and T is large , θS can not produce accurate pseudo-labels for effective self-training . Gradual self-training for GDA . In the GDA setup , self-training is performed along the domain sequence , from the current model θm ( updated using Um already ) to the next one θm+1 using Um+1 θm+1 = ST ( θm , Um+1 ) , ( 2 ) starting from the source model θ0 = θS . By doing so , the model gradually adapts from S to T through the sequence U1 , . . . , UM to produce the final target model θT . θT = θM = ST ( θS , ( U1 , . . . , UM ) ) . ( 3 ) As the domain gap between consecutive domains is smaller than between S and T , the pseudo-labels will be more accurate in each step . As a result , self-training in GDA can lead to a lower target error than in UDA . We note that the domain sequences in [ 35 ] are pre-defined using side information . Theoretical analysis . Kumar et al . [ 35 ] make three assumptions for theoretical analysis : • Separation : for every domain Pm , the data are separable such that there exists an R-bounded ( linear ) classifier θm ∈ Θ with ‖θm‖2 ≤ R that achieves a low loss of L ( θm , Pm ) . • Gradual shift : for the domain sequence P0 , . . . , Pm with no label shifts , the maximum per-class Wasserstein-infinity distance ρ ( Pm , Pm+1 ) between consecutive domains should be ≤ ρ < 1R . • Bounded data : finite samples X from each domain Pm are bounded , i.e. , EX∼Pm [ ‖X‖22 ] ≤ B2 . Under these assumptions of GDA , if the source model θ0 has a low loss L ( θ0 , P0 ) on P0 , then with a probability δ , the resulting target model θM through gradual self-training has L ( θM , PM ) ≤ βM+1 ( L ( θ0 , P0 ) + 4BR+ √ 2 log 2M/δ√ |U| ) , where β = 2 1− ρR . ( 4 ) That is , the target error is controlled by the intermediate domain shift ρ . If we can sample infinite data ( i.e. , |U| → ∞ ) and the source classifier is perfect ( i.e. , L ( θ0 , P0 ) = 0 ) , with a small distance ρ ( Pm , Pm+1 ) ≤ ρ between consecutive domains , gradual self-training achieves a zero target error .

Summary: This paper is addressing the gradual domain adaptation (GDA) problem without the access of intermediate domain indices. This paper proposes a coarse-to-fine method to predict the indices of the intermediate domain sequence. The results are shown to be comparative to pre-defined methods.