Gradual Domain Adaptation in the Wild: When Intermediate Distributions are Absent

1 INTRODUCTION . Learning algorithms are notorious for not being robust to changes in the environment , and their performance often drops when there is a shift in the data distribution ( Taori et al. , 2020 ; Hendrycks et al. , 2021 ; Koh et al. , 2021 ) . To be robust to the changes in the distribution , learning algorithms either need to learn representations that are invariant to the shift , or they should update their parameters to be more aligned with the new distribution . In either setting , we are still far from a satisfactory solution . While unsupervised domain adaptation techniques commonly rely on learning domain invariant representations , our focus is on shifting the model towards the target distribution . We consider the unsupervised setting where we do not have access to labels on the target domain . The setting we consider is similar to the setting suggested by Wang et al . ( 2021 ) . We look at this problem through the lens of curriculum learning . Curriculum learning ( Elman , 1993 ; Sanger , 1994 ; Bengio et al. , 2009 ) suggests presenting easier samples early on in the training process and gradually increasing the difficulty . In unsupervised domain adaptation , this is equivalent to gradually changing the distribution from the source domain to the target domain , i.e. , getting the model to adapt to intermediate distributions before being exposed to the target domain . Kumar et al . ( 2020 ) show that if learning algorithms are exposed to gradual changes in the data distribution under a self-training regime , the generalization gap ( from source to target distribution ) will be much lower . However , the problem remains unresolved when we do not have access to the intermediate steps of the distribution shift . Findings from Kumar et al . ( 2020 ) suggest that to be able to adapt the model by self-training , two conditions should be satisfied : ( i ) Access to samples from intermediate distributions between source and target ; ( ii ) Access to information about the amount of shift for each sample . While ( ii ) is hard to achieve in practice , ( i ) may be the case in many real-world scenarios . We hypothesize that if ( i ) holds , i.e. , the target distribution is diverse enough to include samples from intermediate distributions ; iterative self-training ( applying self-training iteratively ) and filtering examples based on the confidence of the model , incorporates an implicit curriculum and this curriculum helps the model to gradually adapt to the target domain . Furthermore , for cases where ( i ) does not hold , we propose GIFT ( Gradual Interpolation of Features toward Target ) . GIFT creates virtual examples from intermediate distributions by linearly interpolating between source and target data in the input and feature space of a neural network . Note that such interpolations are shown to be meaningful in works that established the idea of mixing up representations ( Verma et al. , 2019a ; Mroueh , 2020a ) . To gradually increase the difficulty of the virtual samples , we change the linear interpolation coefficient such that the samples start at the source distribution and gradually move towards the target distribution during training . Figure 1 , demonstrates how GIFT can improve iterative self-training , in an example with a Two-moon dataset ( two interleaving half circles ) . Our focus in this paper is not to provide a new state of the art on a specific benchmark , as we are limited by the availability of benchmarks that are specifically designed for gradual domain adaptation . Instead , we intend to provide a proof of concept for ( 1 ) how , why and when iteratively applying self-training is helpful for gradual domain adaptation ( Section 4.1 and 4.2 ) ; ( 2 ) There are cases that not only iterative self-training does not help the model but also it hurts model performance on both source and target domains . In those cases , GIFT can mitigate the issue by using interpolations of features as intermediate examples ( Section 4.1 ) . We evaluate iterative self-training and GIFT for unsupervised domain adaptation on datasets with natural distribution shifts and show that they improve the performance of models on the target dataset compared to non-iterative self-training . On a synthetic benchmark , we show that in the absence of ( i ) , GIFT performs better than iterative self-training . By tracking the accuracy and confidence of a model on different subsets of the target distribution , we show that with both GIFT and iterative self-training the order in which the models learn to correctly classify examples is aligned with their degree of perturbation . GIFT has two advantages over iterative self-training : ( 1 ) It works better when the number of training iterations is limited , ( 2 ) it works better when the target distribution is not diverse enough to include a mixture of easy and hard examples . 2 SELF-TRAINING FOR UNSUPERVISED DOMAIN ADAPTATION . Being able to properly handle distribution shift is one of the primary concerns of machine learning algorithms . A common setting for this problem is unsupervised domain adaptation , where we have access to labeled data from one or multiple source domains , and unlabeled data from the target domain . Let ( Xs , Y s ) denote the labeled data in the source domain , where Xs ∈ Rns×d is the sample input matrix and Y s ∈ Rns×k is the corresponding label matrix . Let Xt ∈ Rnt×d denote the unlabeled target domain data . Here ns and nt determine number of examples from source and target domains respectively , and k determines the number of classes . While , it is not necessary for simplicity we assume number of classes and generally the task is the same in source and target . We assume some underlying or common features exist between source and target , while there is a substantial distribution shift between the two domains . The goal is to bridge the domain difference and learn a classifier minimizing the empirical risk on the target domain . The first assumption is needed for domain adaptation to be successful ( Ben-David et al. , 2008 ) . The latter emphasizes that a model trained on the source domain presents a noticeable performance gap on the target domain and hence needs to adapt to the target distribution . Self-training uses a teacher model trained on source domain , Ps , to produce pseudo labels , along with confidence scores , on unlabeled data from the target domain , Pt , and uses these predictions to train a student model . In iterative self-training , the teacher model is periodically updated and replaced by the student model . We examine how , in iterative self-training , the confidence scores of the teacher model guide the student model to adapt to the target domain , and whether we can devise supplementary schemes that help the student adapt better . 3 GIFT : SELF-TRAINING WITH GRADUAL INTERPOLATIONS . If in addition to labeled data from the source domain and unlabeled data from the target domain , we also have access to the intermediate distributions , i.e. , unlabeled examples from data distributions between source and target , we can use them to boost the performance of the model on the target domain , by applying self-training in a gradual manner . Kumar et al . ( 2020 ) show that this gradual adaptation leads to a lower error bound on the target domain . In order for this approach to be applicable , we need samples from intermediate steps that are annotated with the amount of shift from the source distribution . However , in many cases in practice , we either do not have access to samples of intermediate distributions or such examples do not exist . Even if they exist , it is less likely that they are annotated with the degree of shift . To circumvent the issue of lack of samples from intermediate distributions , we create the virtual samples from intermediate distributions by interpolating the input and hidden representations of the data from the source domain Ps and target domain Pt . Namely , let Mφ be a neural network model trained on source data and let zsi correspond to the representation of input x s i ∈ Xs from the source domain . We choose xtj as a sample from the target domain ( we explain the procedure to pick x t j ∈ Xt below ) . Let zsj correspond to the representation of input x t j ∈ Xt that can be sampled from any of the layers of a neural network ( including the input layer ) , and λ ∈ [ 0 , 1 ] . We generate ẑij = ( 1− λ ) zsi + λztj , λ ∈ [ 0 , 1 ] ( 1 ) as a sample representation of a virtual intermediate distribution . Note that we do not seek to find a corresponding input to the interpolated representation . Gradually increasing λ from 0 to 1 corresponds to feature representations that change from corresponding source distribution representations ( λ = 0 ) to the target distribution representation ( λ = 1 ) . lambda_scheduler in algorithm 1 , represent the scheduling function for gradually increasing λ , which can be anything , in our experiments we use a linear function with fixed step size , δ , where the value of lambda increases by a constant value after every δ training steps . To choose which example from the source is interpolated with which example from the target , we either randomly align the samples or apply a cost-based alignment method based on the L2 distance of representations , and the similarity/equality of labels on the source and predicted target labels . To apply a cost-based alignment method , we use the Sinkhorn matching algorithm ( Sinkhorn , 1966 ; Peyré & Cuturi , 2019 ) to approximate the alignment with the lowest cost . Although iterative cost-based alignment methods such as Sinkhorn come at a computational cost , compared to random alignment , we have observed that random alignment without taking ( pseudo ) labels into account leads to worse performance . To find a better trade-off between alignment cost and performance , in our experiments , we also tried a non-iterative heuristic ( pseudo ) label-based random alignment , where we randomly align examples that have the same ( pseudo ) labels . This is shown in Algorithm 2 . In our experiments , we report the result from cost-based alignment . We observed that pseudo-random alignment and cost-based alignment lead to more or less similar results and in practice , we could simply use the Algorithm 1 GIFT : Gradual Interpolation Ps , : source domain ; Pt : target domain M : Neural net ( maps input to predictions ) . M : L : partial neural net ( maps input to features of layer L ) ML : : partial neural net ( maps features of layer L to predictions ) φ : student neural net parameters ; θ : teacher neural net parameters δ : step size for interpolation coefficient λ. N : number of training iterations for each teacher update . m : number of examples in a batch . α : confidence threshold Notation : [ m ] = { 1 , ... m } . 1 : Mφ ←Mθ , λ← 0 2 : while λ < = 1 do 3 : for step ∈ [ N ] do 4 : ( xs [ m ] , y s [ m ] ) ∼ Ps 5 : xt [ m ] ∼ Pt 6 : zs [ m ] ←M : L φ ( x s [ m ] ) 7 : zt [ m ] ←M : L θ ( x t [ m ] ) 8 : yt [ m ] ←Mθ ( x t [ m ] ) 9 : indexs , indext←align ( ys [ m ] , y t [ m ] ) ( Algorithm 2 ) 10 : ẑ [ m ] ← ( 1− λ ) × zsindexs + λ× z t indext

Summary: This work builds on the previous work[1] and extends it for a more general scenario, where per sample intermediate distribution shift is hard to quantify. They propose gradual feature interpolation for the case where samples from intermediate distribution are missing. Iterative self-training fails in such cases. The work presents results on synthetic and natural distributions to evaluate their claim.  [1] Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self-training for gradual domain adaptation. arXiv preprint arXiv:2002.11361, 2020.