Gradual Domain Adaptation without Indexed Intermediate Domains

1 Introduction . The distributions of real-world data change dynamically due to many factors like time , locations , environments , etc . Such a fact poses a great challenge to machine-learned models , which implicitly assume that the test data distribution is covered by the training data distribution . To resolve this generalization problem , unsupervised domain adaption ( UDA ) , which aims to adapt a learned model to the test domain given its unlabeled data [ 13 , 15 ] , has been an active sub-field in machine learning . Typically , UDA assumes that the “ source ” domain , in which the model is trained , and the “ target ” domain , in which the model is deployed , are discrepant but sufficiently related . Concretely , Ben-David et al . [ 1 ] , Zhao et al . [ 75 ] show that the generalization error of UDA is bounded by the discrepancy of the marginal or conditional distributions between domains . Namely , the effectiveness of UDA may degrade along with the increase in domain discrepancy . Take one popular algorithm , self-training [ 38 , 48 , 49 ] , for example . Self-training adapts the source model by progressively labeling the unlabeled target data ( i.e. , pseudo-labels ) and using them to fine-tune the model [ 2 , 28 , 30 , 31 , 41 , 44 , 77 , 78 ] . Self-training works if the pseudo-labels are accurate ( as it essentially becomes supervised learning ) , but is vulnerable if they are not , which occurs when there exists a large domain gap [ 35 ] . To address this issue , several recent works investigate gradual domain adaption ( GDA ) [ 12 , 25 , 35 , 71 ] , in which beyond the source and target domains , the model can access additional unlabeled data from the intermediate domains that shift gradually from the source to the target . By adapting 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . the model along the sequence of intermediate domains — i.e. , from the ones close to the source to the ones close to the target — the large domain gap between source and target is chipped away by multiple sub-adaptation problems ( between consecutive intermediate domains ) whose domain gaps are smaller . Namely , every time the model moves a step closer to the target , instead of taking a huge jump that can significantly decrease the performance ( see Figure 1 for an illustration ) . GDA makes sense , as real-world data change gradually more often than abruptly [ 11 , 60 , 66 ] . The recent work by Kumar et al . [ 35 ] further demonstrates the strength of GDA both empirically and theoretically . One potential drawback of GDA is the need for a well-defined sequence of intermediate domains . That is , prior to adaptation , the additional unlabeled data must be grouped into multiple domains and indexed with intermediate domain labels that reflect the underlying data distribution shift from the source to the target . Such information , however , may not be available directly . Existing methods usually leverage side information like time tags [ 25 , 35 , 71 ] to define the sequence , which may be sub-optimal . In some applications , even the side information may not be accessible ( e.g. , due to privacy concerns ) , greatly limiting the applicability of GDA . In this paper , we therefore study GDA in the extreme case — the additional unlabeled data are neither grouped nor indexed . Specifically , we investigate how to discover the “ domain sequence ” from data , such that it can be used to drive GDA . We propose a two-stage coarse-to-fine framework named Intermediate DOmain Labeler ( IDOL ) . In the first stage , IDOL labels each intermediate data instance with a coarse score that reflects how close it is to the source or target . We study several methods that estimate the distance from a data instance to a domain . We find that a progressively trained domain discriminator — which starts with source vs. target data but gradually adds data close to either of them into training — performs the best , as it better captures the underlying data manifold . In the second stage , IDOL then builds upon the coarse scores to group data into domains by further considering the discriminative ( e.g. , classification ) knowledge the model aims to preserve from the source to the target . Since the additional unlabeled data are fully unlabeled , we take a greedy approach to identify the next intermediate domain ( i.e. , a group of data ) that can best preserve the discriminative knowledge in the current intermediate domain . Concretely , we employ self-training [ 38 ] along the domain sequence discovered so far to provide pseudo-labels for the current domain , and propose a novel cycle-consistency loss to discover the next domain , such that the model adapted to the next domain can be “ adapted back ” to the current domain and predict the same pseudo-labels . The output of IDOL is a sequence of intermediate domains that can be used by any GDA algorithms [ 25 , 35 , 71 ] . We validated IDOL on two data sets studied in [ 35 ] , including Rotated MNIST [ 36 ] and Portraits over years [ 14 ] . IDOL can successfully discover the domain sequence that leads to comparable GDA performance to using the pre-defined sequence ( i.e. , by side information ) . More importantly , IDOL is compatible with pre-defined sequences — by treating them as the coarse sequences — to further improve upon them . We also investigate IDOL in scenarios where the additional unlabeled data may contain outliers and demonstrate IDOL ’ s effectiveness even in such challenging cases . To our knowledge , our work is the first to tackle GDA without grouped and indexed intermediate domains . The success of IDOL opens up broader application scenarios that GDA can contribute to . 2 Background : gradual domain adaptation ( GDA ) . 2.1 Setup of UDA and GDA . We consider an unsupervised domain adaptation ( UDA ) problem , in which a classifier is provided with labeled data from the source domain S = { ( xSi , ySi ) } |S| i=1 and unlabeled data from the target domain T = { xTi } |T | i=1 . Both S and T are sampled IID from the underlying joint distributions of the source PS and target PT , respectively . A source model θS is typically produced by training on S directly . The goal of UDA is to learn a target model θT such that it can perform well on the target , measured by a loss L ( θT , PT ) . We focus on the standard UDA setup that assumes no label shifts [ 6 ] . Gradual domain adaptation ( GDA ) , on top of UDA , assumes the existence of a sequence of domains P0 , P1 , . . . PM−1 , PM , where P0 and PM are the source domain and target domain , respectively , along with M − 1 intermediate domains P1 , . . . , PM−1 . The data distributions of domains are assumed to change gradually from the source to the target along the sequence . One can access unlabeled data Um = { xUmi } |Um| i=1 from each of the intermediate domain , forming a sequence of unlabeled data U1 , . . . , UM−1 . We denote the union of them by U = { xUi } |U| i=1 . For brevity , we assume that each Um is disjoint from the others but the size |Um| is the same ∀m ∈ { 1 , · · · , M − 1 } . 2.2 GDA with pre-defined domain sequences using self-training [ 35 ] . We summarize the theoretical analysis of GDA using self-training by Kumar et al . [ 35 ] as follows . Self-training for UDA . Self-training [ 38 ] takes a pre-trained ( source ) model θ and the target unlabeled data T as input . Denote by z = f ( x ; θ ) the predicted logits z , self-training first applies f ( · ; θ ) to target data points x ∈ T , sharpens the predictions by argmax into pseudo-labels , and then updates the current model θ by minimizing the loss ` ( e.g. , cross entropy ) w.r.t the pseudo-labels , θT = ST ( θ , T ) = arg min θ′∈Θ 1 |T | |T |∑ i=1 ` ( f ( xi ; θ ′ ) , sharpen ( f ( xi ; θ ) ) ) , ( 1 ) where θT denotes the resulting target model and ST ( θ , T ) denotes the self-training process . A baseline use of self-training for UDA is to set θ as the source model θS . However , when the domain gap between S and T is large , θS can not produce accurate pseudo-labels for effective self-training . Gradual self-training for GDA . In the GDA setup , self-training is performed along the domain sequence , from the current model θm ( updated using Um already ) to the next one θm+1 using Um+1 θm+1 = ST ( θm , Um+1 ) , ( 2 ) starting from the source model θ0 = θS . By doing so , the model gradually adapts from S to T through the sequence U1 , . . . , UM to produce the final target model θT . θT = θM = ST ( θS , ( U1 , . . . , UM ) ) . ( 3 ) As the domain gap between consecutive domains is smaller than between S and T , the pseudo-labels will be more accurate in each step . As a result , self-training in GDA can lead to a lower target error than in UDA . We note that the domain sequences in [ 35 ] are pre-defined using side information . Theoretical analysis . Kumar et al . [ 35 ] make three assumptions for theoretical analysis : • Separation : for every domain Pm , the data are separable such that there exists an R-bounded ( linear ) classifier θm ∈ Θ with ‖θm‖2 ≤ R that achieves a low loss of L ( θm , Pm ) . • Gradual shift : for the domain sequence P0 , . . . , Pm with no label shifts , the maximum per-class Wasserstein-infinity distance ρ ( Pm , Pm+1 ) between consecutive domains should be ≤ ρ < 1R . • Bounded data : finite samples X from each domain Pm are bounded , i.e. , EX∼Pm [ ‖X‖22 ] ≤ B2 . Under these assumptions of GDA , if the source model θ0 has a low loss L ( θ0 , P0 ) on P0 , then with a probability δ , the resulting target model θM through gradual self-training has L ( θM , PM ) ≤ βM+1 ( L ( θ0 , P0 ) + 4BR+ √ 2 log 2M/δ√ |U| ) , where β = 2 1− ρR . ( 4 ) That is , the target error is controlled by the intermediate domain shift ρ . If we can sample infinite data ( i.e. , |U| → ∞ ) and the source classifier is perfect ( i.e. , L ( θ0 , P0 ) = 0 ) , with a small distance ρ ( Pm , Pm+1 ) ≤ ρ between consecutive domains , gradual self-training achieves a zero target error .

Summary: This paper is concerned with gradual domain adaptation, when we have access to data from intermediate distributions between source and target and use them to gradually adapt the model through self-training. To address the challenge of lack of intermediate data annotated and grouped based on their distance to the source domain, the paper proposes IDOL (Intermediate Domain Labeler) to index unlabeled available data that presumably covers the gap between source and target  domains based on their distance to source.  IDOL consists of two steps: 1. A progressively trained domain discriminator is used to assign a score to each example. (a higher/lower score indicates that the example is closer to the source/target domain). 2. A model based on a cycle-consistency loss is applied to refine the coarse scores progressively. The main idea is to group examples into intermediate domains such that enough discriminative information is preserved that we could approximately regain the accuracy on the source domain by consuming the intermediate domains in the reverse direction..   **Empirical results**: Experiments on  Rotated MNIST and Portraits indicate that with IDOL we can achieve comparable performance on gradual domain adaptation as when we have the ground truth (pre-defined) domain sequences. Furthermore the experiments indicate that applying IDOL on pre-defined domain sequences, treating them as the outputs from step 1, and refining them through step 2 can lead to domain sequences that are better suited for gradual domain adaptation. In general, it seems that applying IDOL in many cases with partial or noisy annotation of the intermediated domain sequences can be beneficial.  Finally, IDOL is also compared with UDA on CIFAR10 to STL tasks, indicating superiority of both (GDA + confidence) and (GDA + IDOL) to UDA in this setting.