Gradual Domain Adaptation in the Wild: When Intermediate Distributions are Absent

1 INTRODUCTION . Learning algorithms are notorious for not being robust to changes in the environment , and their performance often drops when there is a shift in the data distribution ( Taori et al. , 2020 ; Hendrycks et al. , 2021 ; Koh et al. , 2021 ) . To be robust to the changes in the distribution , learning algorithms either need to learn representations that are invariant to the shift , or they should update their parameters to be more aligned with the new distribution . In either setting , we are still far from a satisfactory solution . While unsupervised domain adaptation techniques commonly rely on learning domain invariant representations , our focus is on shifting the model towards the target distribution . We consider the unsupervised setting where we do not have access to labels on the target domain . The setting we consider is similar to the setting suggested by Wang et al . ( 2021 ) . We look at this problem through the lens of curriculum learning . Curriculum learning ( Elman , 1993 ; Sanger , 1994 ; Bengio et al. , 2009 ) suggests presenting easier samples early on in the training process and gradually increasing the difficulty . In unsupervised domain adaptation , this is equivalent to gradually changing the distribution from the source domain to the target domain , i.e. , getting the model to adapt to intermediate distributions before being exposed to the target domain . Kumar et al . ( 2020 ) show that if learning algorithms are exposed to gradual changes in the data distribution under a self-training regime , the generalization gap ( from source to target distribution ) will be much lower . However , the problem remains unresolved when we do not have access to the intermediate steps of the distribution shift . Findings from Kumar et al . ( 2020 ) suggest that to be able to adapt the model by self-training , two conditions should be satisfied : ( i ) Access to samples from intermediate distributions between source and target ; ( ii ) Access to information about the amount of shift for each sample . While ( ii ) is hard to achieve in practice , ( i ) may be the case in many real-world scenarios . We hypothesize that if ( i ) holds , i.e. , the target distribution is diverse enough to include samples from intermediate distributions ; iterative self-training ( applying self-training iteratively ) and filtering examples based on the confidence of the model , incorporates an implicit curriculum and this curriculum helps the model to gradually adapt to the target domain . Furthermore , for cases where ( i ) does not hold , we propose GIFT ( Gradual Interpolation of Features toward Target ) . GIFT creates virtual examples from intermediate distributions by linearly interpolating between source and target data in the input and feature space of a neural network . Note that such interpolations are shown to be meaningful in works that established the idea of mixing up representations ( Verma et al. , 2019a ; Mroueh , 2020a ) . To gradually increase the difficulty of the virtual samples , we change the linear interpolation coefficient such that the samples start at the source distribution and gradually move towards the target distribution during training . Figure 1 , demonstrates how GIFT can improve iterative self-training , in an example with a Two-moon dataset ( two interleaving half circles ) . Our focus in this paper is not to provide a new state of the art on a specific benchmark , as we are limited by the availability of benchmarks that are specifically designed for gradual domain adaptation . Instead , we intend to provide a proof of concept for ( 1 ) how , why and when iteratively applying self-training is helpful for gradual domain adaptation ( Section 4.1 and 4.2 ) ; ( 2 ) There are cases that not only iterative self-training does not help the model but also it hurts model performance on both source and target domains . In those cases , GIFT can mitigate the issue by using interpolations of features as intermediate examples ( Section 4.1 ) . We evaluate iterative self-training and GIFT for unsupervised domain adaptation on datasets with natural distribution shifts and show that they improve the performance of models on the target dataset compared to non-iterative self-training . On a synthetic benchmark , we show that in the absence of ( i ) , GIFT performs better than iterative self-training . By tracking the accuracy and confidence of a model on different subsets of the target distribution , we show that with both GIFT and iterative self-training the order in which the models learn to correctly classify examples is aligned with their degree of perturbation . GIFT has two advantages over iterative self-training : ( 1 ) It works better when the number of training iterations is limited , ( 2 ) it works better when the target distribution is not diverse enough to include a mixture of easy and hard examples . 2 SELF-TRAINING FOR UNSUPERVISED DOMAIN ADAPTATION . Being able to properly handle distribution shift is one of the primary concerns of machine learning algorithms . A common setting for this problem is unsupervised domain adaptation , where we have access to labeled data from one or multiple source domains , and unlabeled data from the target domain . Let ( Xs , Y s ) denote the labeled data in the source domain , where Xs ∈ Rns×d is the sample input matrix and Y s ∈ Rns×k is the corresponding label matrix . Let Xt ∈ Rnt×d denote the unlabeled target domain data . Here ns and nt determine number of examples from source and target domains respectively , and k determines the number of classes . While , it is not necessary for simplicity we assume number of classes and generally the task is the same in source and target . We assume some underlying or common features exist between source and target , while there is a substantial distribution shift between the two domains . The goal is to bridge the domain difference and learn a classifier minimizing the empirical risk on the target domain . The first assumption is needed for domain adaptation to be successful ( Ben-David et al. , 2008 ) . The latter emphasizes that a model trained on the source domain presents a noticeable performance gap on the target domain and hence needs to adapt to the target distribution . Self-training uses a teacher model trained on source domain , Ps , to produce pseudo labels , along with confidence scores , on unlabeled data from the target domain , Pt , and uses these predictions to train a student model . In iterative self-training , the teacher model is periodically updated and replaced by the student model . We examine how , in iterative self-training , the confidence scores of the teacher model guide the student model to adapt to the target domain , and whether we can devise supplementary schemes that help the student adapt better . 3 GIFT : SELF-TRAINING WITH GRADUAL INTERPOLATIONS . If in addition to labeled data from the source domain and unlabeled data from the target domain , we also have access to the intermediate distributions , i.e. , unlabeled examples from data distributions between source and target , we can use them to boost the performance of the model on the target domain , by applying self-training in a gradual manner . Kumar et al . ( 2020 ) show that this gradual adaptation leads to a lower error bound on the target domain . In order for this approach to be applicable , we need samples from intermediate steps that are annotated with the amount of shift from the source distribution . However , in many cases in practice , we either do not have access to samples of intermediate distributions or such examples do not exist . Even if they exist , it is less likely that they are annotated with the degree of shift . To circumvent the issue of lack of samples from intermediate distributions , we create the virtual samples from intermediate distributions by interpolating the input and hidden representations of the data from the source domain Ps and target domain Pt . Namely , let Mφ be a neural network model trained on source data and let zsi correspond to the representation of input x s i ∈ Xs from the source domain . We choose xtj as a sample from the target domain ( we explain the procedure to pick x t j ∈ Xt below ) . Let zsj correspond to the representation of input x t j ∈ Xt that can be sampled from any of the layers of a neural network ( including the input layer ) , and λ ∈ [ 0 , 1 ] . We generate ẑij = ( 1− λ ) zsi + λztj , λ ∈ [ 0 , 1 ] ( 1 ) as a sample representation of a virtual intermediate distribution . Note that we do not seek to find a corresponding input to the interpolated representation . Gradually increasing λ from 0 to 1 corresponds to feature representations that change from corresponding source distribution representations ( λ = 0 ) to the target distribution representation ( λ = 1 ) . lambda_scheduler in algorithm 1 , represent the scheduling function for gradually increasing λ , which can be anything , in our experiments we use a linear function with fixed step size , δ , where the value of lambda increases by a constant value after every δ training steps . To choose which example from the source is interpolated with which example from the target , we either randomly align the samples or apply a cost-based alignment method based on the L2 distance of representations , and the similarity/equality of labels on the source and predicted target labels . To apply a cost-based alignment method , we use the Sinkhorn matching algorithm ( Sinkhorn , 1966 ; Peyré & Cuturi , 2019 ) to approximate the alignment with the lowest cost . Although iterative cost-based alignment methods such as Sinkhorn come at a computational cost , compared to random alignment , we have observed that random alignment without taking ( pseudo ) labels into account leads to worse performance . To find a better trade-off between alignment cost and performance , in our experiments , we also tried a non-iterative heuristic ( pseudo ) label-based random alignment , where we randomly align examples that have the same ( pseudo ) labels . This is shown in Algorithm 2 . In our experiments , we report the result from cost-based alignment . We observed that pseudo-random alignment and cost-based alignment lead to more or less similar results and in practice , we could simply use the Algorithm 1 GIFT : Gradual Interpolation Ps , : source domain ; Pt : target domain M : Neural net ( maps input to predictions ) . M : L : partial neural net ( maps input to features of layer L ) ML : : partial neural net ( maps features of layer L to predictions ) φ : student neural net parameters ; θ : teacher neural net parameters δ : step size for interpolation coefficient λ. N : number of training iterations for each teacher update . m : number of examples in a batch . α : confidence threshold Notation : [ m ] = { 1 , ... m } . 1 : Mφ ←Mθ , λ← 0 2 : while λ < = 1 do 3 : for step ∈ [ N ] do 4 : ( xs [ m ] , y s [ m ] ) ∼ Ps 5 : xt [ m ] ∼ Pt 6 : zs [ m ] ←M : L φ ( x s [ m ] ) 7 : zt [ m ] ←M : L θ ( x t [ m ] ) 8 : yt [ m ] ←Mθ ( x t [ m ] ) 9 : indexs , indext←align ( ys [ m ] , y t [ m ] ) ( Algorithm 2 ) 10 : ẑ [ m ] ← ( 1− λ ) × zsindexs + λ× z t indext

Summary: This work proposes an iterative self-training approach for unsupervised domain adaptation. In particular, the authors aim at gradually adapting a model trained on a source domain to the target domain. Based on the claim that previous work on this setting assumed that samples from distributions that represent gradual changes from the source to the target domain are available when adapting the model, the authors proposed a strategy to generate such intermediate samples for cases where they are not available. The introduced approach, named GIFT, consists in performing manifold mix-up between representations of examples from the source and target domains considering an increasing value of the hyperparameter that accounts for the weight of the representation of the target domain example. By doing so, the authors claim an automatic curriculum is introduced in the training. Moreover, since labels from the target domain examples are not available, the authors also introduced a heuristic to pair the examples that are mixed. GIFT was empirically validated on experiments with synthetic domain shifts on the CIFAR-10 dataset and was shown to outperform iterative self-training in terms of target accuracy. Experiments on two datasets presenting natural domain shifts were also performed.