Original: G RADUAL D OMAIN A DAPTATION IN THE W ILD : W HEN I NTERMEDIATE D ISTRIBUTIONS ARE A BSENT
Revised: G RADUAL D OMAIN A DAPTATION IN THE W ILD : W HEN I NTERMEDIATE D ISTRIBUTIONS ARE A BSENT

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We focus on the problem of domain adaptation when the goal is shifting the model towards the target distribution rather than learning domain invariant representations.
Revised: We focus on the problem of domain adaptation when the goal is shifting the model towards the target distribution rather than learning domain invariant representations.

Original: It is shown that under the following two assumptions: (i) access to samples from intermediate distributions, and (ii) samples being annotated with the amount of change from the source distribution, self-training can be successfully applied on gradually shifted samples to adapt the model toward the target distribution.
Revised: It is shown that under the following two assumptions: (i) access to samples from intermediate distributions, and (ii) samples being annotated with the amount of change from the source distribution, self-training can be successfully applied on gradually shifted samples to adapt the model toward the target distribution.

Original: We hypothesize having (i) is enough to enable iterative self-training to slowly adapt the model to the target distribution by making use of an implicit curriculum.
Revised: We hypothesize having (i) is enough to enable iterative self-training to slowly adapt the model to the target distribution by making use of an implicit curriculum.

Original: In the case where (i) does not hold, we observe that iterative self-training falls short.
Revised: In the case where (i) does not hold, we observe that iterative self-training falls short.

Original: We propose GIFT (Gradual Interpolation of Features toward Target), a method that creates virtual samples from intermediate distributions by interpolating representations of examples from source and target domains.
Revised: We propose GIFT (Gradual Interpolation of Features toward Target), a method that creates virtual samples from intermediate distributions by interpolating representations of examples from source and target domains.

Original: Our analysis of various synthetic distribution shifts shows that in the presence of (i) iterative selftraining naturally forms a curriculum of samples which helps the model to adapt better to the target domain.
Revised: Our analysis of various synthetic distribution shifts shows that in the presence of (i) iterative selftraining naturally forms a curriculum of samples which helps the model to adapt better to the target domain.

Original: Furthermore, we show that when (i) does not hold, more iterations hurt the performance of self-training, and in these settings, GIFT is advantageous.
Revised: Furthermore, we show that when (i) does not hold, more iterations hurt the performance of self-training, and in these settings, GIFT is advantageous.

Original: Additionally, we evaluate self-training, iterative self-training and
Revised: Additionally, we evaluate self-training, iterative self-training and

Original: GIFT on two benchmarks with different types of natural distribution shifts and show that when applied on top of other domain adaptation methods, GIFT improves the performance of the model on the target dataset.
Revised: GIFT on two benchmarks with different types of natural distribution shifts and show that when applied on top of other domain adaptation methods, GIFT improves the performance of the model on the target dataset.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Learning algorithms are notorious for not being robust to changes in the environment, and their performance often drops when there is a shift in the data distribution (Taori et al., 2020;
Revised: Learning algorithms are notorious for not being robust to changes in the environment, and their performance often drops when there is a shift in the data distribution (Taori et al., 2020;

Original: Hendrycks et al., 2021; Koh et al., 2021).
Revised: Hendrycks et al., 2021; Koh et al., 2021).

Original: To be robust to the changes in the distribution, learning algorithms either need to learn representations that are invariant to the shift, or they should update their parameters to be more aligned with the new distribution.
Revised: To be robust to the changes in the distribution, learning algorithms either need to learn representations that are invariant to the shift, or they should update their parameters to be more aligned with the new distribution.

Original: In either setting, we are still far from a satisfactory solution.
Revised: In either setting, we are still far from a satisfactory solution.

Original: While unsupervised domain adaptation techniques commonly rely on learning domain invariant representations, our focus is on shifting the model towards the target distribution.
Revised: While unsupervised domain adaptation techniques commonly rely on learning domain invariant representations, our focus is on shifting the model towards the target distribution.

Original: We consider the unsupervised setting where we do not have access to labels on the target domain.
Revised: We consider the unsupervised setting where we do not have access to labels on the target domain.

Original: The setting we consider is similar to the setting suggested by Wang et al.
Revised: The setting we consider is similar to the setting suggested by Wang et al.

Original: We look at this problem through the lens of curriculum learning.
Revised: We look at this problem through the lens of curriculum learning.

Original: Curriculum learning (Elman, 1993;
Revised: Curriculum learning (Elman, 1993;

Original: Sanger, 1994; Bengio et al., 2009) suggests presenting easier samples early on in the training process and gradually increasing the difﬁculty.
Revised: Sanger, 1994; Bengio et al., 2009) suggests presenting easier samples early on in the training process and gradually increasing the difﬁculty.

Original: In unsupervised domain adaptation, this is equivalent to gradually changing the distribution from the source domain to the target domain, i.e., getting the model to adapt to intermediate distributions before being exposed to the target domain.
Revised: In unsupervised domain adaptation, this is equivalent to gradually changing the distribution from the source domain to the target domain, i.e., getting the model to adapt to intermediate distributions before being exposed to the target domain.

Original: Kumar et al.
Revised: Kumar et al.

Original: (2020) show that if learning algorithms are exposed to gradual changes in the data distribution under a self-training regime, the generalization gap (from source to target distribution) will be much lower.
Revised: (2020) show that if learning algorithms are exposed to gradual changes in the data distribution under a self-training regime, the generalization gap (from source to target distribution) will be much lower.

Original: However, the problem remains unresolved when we do not have access to the intermediate steps of the distribution shift.
Revised: However, the problem remains unresolved when we do not have access to the intermediate steps of the distribution shift.

Original: Findings from Kumar et al.
Revised: Findings from Kumar et al.

Original: (2020) suggest that to be able to adapt the model by self-training, two conditions should be satisﬁed: (i)
Revised: (2020) suggest that to be able to adapt the model by self-training, two conditions should be satisﬁed: (i)

Original: Access to samples from intermediate distributions between source and target; (ii) Access to information about the amount of shift for each sample.
Revised: Access to samples from intermediate distributions between source and target; (ii) Access to information about the amount of shift for each sample.

Original: While (ii) is hard to achieve in practice, (i) may be the case in many real-world scenarios.
Revised: While (ii) is hard to achieve in practice, (i) may be the case in many real-world scenarios.

Original: We hypothesize that if (i) holds, i.e., the target distribution is diverse enough to include samples from intermediate distributions; iterative self-training (applying self-training iteratively) and ﬁltering examples based on the conﬁdence of the model, incorporates an implicit curriculum and this curriculum helps the model to gradually adapt to the target domain.
Revised: We hypothesize that if (i) holds, i.e., the target distribution is diverse enough to include samples from intermediate distributions; iterative self-training (applying self-training iteratively) and ﬁltering examples based on the conﬁdence of the model, incorporates an implicit curriculum and this curriculum helps the model to gradually adapt to the target domain.

Original: Furthermore, for cases where (i) does not hold, we propose GIFT (Gradual Interpolation of Features toward Target).
Revised: Furthermore, for cases where (i) does not hold, we propose GIFT (Gradual Interpolation of Features toward Target).

Original: GIFT creates virtual examples from intermediate distributions by linearly interpolating between source and target data in the input and feature space of a neural network.
Revised: GIFT creates virtual examples from intermediate distributions by linearly interpolating between source and target data in the input and feature space of a neural network.

Original: Note that such interpolations are shown to be meaningful in works that established the idea of mixing up representations (Verma et al., 2019a; Mroueh, 2020a).
Revised: Note that such interpolations are shown to be meaningful in works that established the idea of mixing up representations (Verma et al., 2019a; Mroueh, 2020a).

Original: To gradually increase the difﬁculty of the virtual samples, we change the linear interpolation coefﬁcient such that the samples start at the source distribution and gradually move towards the target distribution during training.
Revised: To gradually increase the difﬁculty of the virtual samples, we change the linear interpolation coefﬁcient such that the samples start at the source distribution and gradually move towards the target distribution during training.

Original: Figure 1, demonstrates how GIFT can improve iterative self-training, in an example with a Two-moon dataset (two interleaving half circles).
Revised: Figure 1, demonstrates how GIFT can improve iterative self-training, in an example with a Two-moon dataset (two interleaving half circles).

Original: Our focus in this paper is not to provide a new state of the art on a speciﬁc benchmark, as we are limited by the availability of benchmarks that are speciﬁcally designed for gradual domain adaptation.
Revised: Our focus in this paper is not to provide a new state of the art on a speciﬁc benchmark, as we are limited by the availability of benchmarks that are speciﬁcally designed for gradual domain adaptation.

Original: Instead, we intend to provide a proof of concept for (1) how, why and when iteratively applying self-training is helpful for gradual domain adaptation (Section 4.1 and 4.2); (2) There are cases that not only iterative self-training does not help the model but also it hurts model performance on both source and target domains.
Revised: Instead, we intend to provide a proof of concept for (1) how, why and when iteratively applying self-training is helpful for gradual domain adaptation (Section 4.1 and 4.2); (2) There are cases that not only iterative self-training does not help the model but also it hurts model performance on both source and target domains.

Original: In those cases, GIFT can mitigate the issue by using interpolations of features as intermediate examples (Section 4.1).
Revised: In those cases, GIFT can mitigate the issue by using interpolations of features as intermediate examples (Section 4.1).

Original: We evaluate iterative self-training and GIFT for unsupervised domain adaptation on datasets with natural distribution shifts and show that they improve the performance of models on the target dataset compared to non-iterative self-training.
Revised: We evaluate iterative self-training and GIFT for unsupervised domain adaptation on datasets with natural distribution shifts and show that they improve the performance of models on the target dataset compared to non-iterative self-training.

Original: On a synthetic benchmark, we show that in the absence of (i), GIFT performs better than iterative self-training.
Revised: On a synthetic benchmark, we show that in the absence of (i), GIFT performs better than iterative self-training.

Original: By tracking the accuracy and conﬁdence of a model on different subsets of the target distribution, we show that with both GIFT and iterative self-training the order in which the models learn to correctly classify examples is aligned with their degree of perturbation.
Revised: By tracking the accuracy and conﬁdence of a model on different subsets of the target distribution, we show that with both GIFT and iterative self-training the order in which the models learn to correctly classify examples is aligned with their degree of perturbation.

Original: GIFT has two advantages over iterative self-training: (1) It works better when the number of training iterations is limited, (2) it works better when the target distribution is not diverse enough to include a mixture of easy and hard examples.
Revised: GIFT has two advantages over iterative self-training: (1) It works better when the number of training iterations is limited, (2) it works better when the target distribution is not diverse enough to include a mixture of easy and hard examples.

Original: On the left side, different colors determine the two ground truth classes.
Revised: On the left side, different colors determine the two ground truth classes.

Original: On the right, different colors depict predicted classes.
Revised: On the right, different colors depict predicted classes.

Original: Here the bottom left) is the (
Revised: Here the bottom left) is the (

Original: ) rotated version of the source data (top right).
Revised: ) rotated version of the source data (top right).

Original: In this example, it is not possible to achieve a good performance on both source and target at the same time, since they have conﬂicting labels for similar inputs.
Revised: In this example, it is not possible to achieve a good performance on both source and target at the same time, since they have conﬂicting labels for similar inputs.

Original: While the performance of the model trained only on source is around ∼ 50% on the target data, if we have access to ground truth intermediate steps we can improve this number by ∼ 40% by gradual self-training (Kumar et al., 2010).
Revised: While the performance of the model trained only on source is around ∼ 50% on the target data, if we have access to ground truth intermediate steps we can improve this number by ∼ 40% by gradual self-training (Kumar et al., 2010).

Original: In the absence of samples from intermediate distributions, our proposed method(GIFT) can increase the performance by ∼ 20% compared to no adaptation, while iterative self-training without access to intermediate examples leads to even worse accuracy.
Revised: In the absence of samples from intermediate distributions, our proposed method(GIFT) can increase the performance by ∼ 20% compared to no adaptation, while iterative self-training without access to intermediate examples leads to even worse accuracy.

Original: For this example, we use an MLP with one hidden layer and a Relu activation function.
Revised: For this example, we use an MLP with one hidden layer and a Relu activation function.

Original: The code is provided at https://github.com/anonymouse-github/GIFT .
Revised: The code is provided at https://github.com/anonymouse-github/GIFT .

Original:  S ELF - TRAINING FOR UNSUPERVISED DOMAIN ADAPTATION
Revised:  S ELF - TRAINING FOR UNSUPERVISED DOMAIN ADAPTATION

Original: Being able to properly handle distribution shift is one of the primary concerns of machine learning algorithms.
Revised: Being able to properly handle distribution shift is one of the primary concerns of machine learning algorithms.

Original: A common setting for this problem is unsupervised domain adaptation, where we have access to labeled data from one or multiple source domains, and unlabeled data from the target domain.
Revised: A common setting for this problem is unsupervised domain adaptation, where we have access to labeled data from one or multiple source domains, and unlabeled data from the target domain.

Original: Let ( X s , Y s ) denote the labeled data in the source domain, where X s ∈ R n s × d is the sample input matrix and Y s ∈ R n s × k is the corresponding label matrix.
Revised: Let ( X s , Y s ) denote the labeled data in the source domain, where X s ∈ R n s × d is the sample input matrix and Y s ∈ R n s × k is the corresponding label matrix.

Original: Let X t ∈ R n t ×
Revised: Let X t ∈ R n t ×

Original: d denote the unlabeled target domain data.
Revised: d denote the unlabeled target domain data.

Original: Here n s and n t determine number of examples from source and target domains respectively, and k determines the number of classes.
Revised: Here n s and n t determine number of examples from source and target domains respectively, and k determines the number of classes.

Original: While, it is not necessary for simplicity we assume number of classes and generally the task is the same in source and target.
Revised: While, it is not necessary for simplicity we assume number of classes and generally the task is the same in source and target.

Original: We assume some underlying or common features exist between source and target, while there is a substantial distribution shift between the two domains.
Revised: We assume some underlying or common features exist between source and target, while there is a substantial distribution shift between the two domains.

Original: The goal is to bridge the domain difference and learn agood classiﬁer for the target domain.
Revised: The goal is to bridge the domain difference and learn a classiﬁer minimizing the empirical risk on the target domain.

Original: The ﬁrst assumption is needed for domain adaptation to be successful (Ben-David et al., 2008).
Revised: The ﬁrst assumption is needed for domain adaptation to be successful (Ben-David et al., 2008).

Original: The latter emphasizes that a model trained on the source domain presents a noticeable performance gap on the target domain and hence needs to adapt to the target distribution.
Revised: The latter emphasizes that a model trained on the source domain presents a noticeable performance gap on the target domain and hence needs to adapt to the target distribution.

Original: Self-training uses a teacher model trained on source domain, P s , to produce pseudo labels, along with conﬁdence scores, on unlabeled data from the target domain, P t , and uses these predictions to train a student model.
Revised: Self-training uses a teacher model trained on source domain, P s , to produce pseudo labels, along with conﬁdence scores, on unlabeled data from the target domain, P t , and uses these predictions to train a student model.

Original: In iterative self-training, the teacher model is periodically updated and replaced by the student model.
Revised: In iterative self-training, the teacher model is periodically updated and replaced by the student model.

Original: We examine how, in iterative self-training, the conﬁdence scores of the teacher model guide the student model to adapt to the target domain, and whether we can devise supplementary schemes that help the student adapt better.
Revised: We examine how, in iterative self-training, the conﬁdence scores of the teacher model guide the student model to adapt to the target domain, and whether we can devise supplementary schemes that help the student adapt better.

Original:  GIFT: S ELF - TRAINING WITH G RADUAL I NTERPOLATIONS
Revised:  GIFT: S ELF - TRAINING WITH G RADUAL I NTERPOLATIONS

Original: If in addition to labeled data from the source domain and unlabeled data from the target domain, we also have access to the intermediate distributions, i.e., unlabeled examples from data distributions between source and target, we can use them to boost the performance of the model on the target domain, by applying self-training in a gradual manner.
Revised: If in addition to labeled data from the source domain and unlabeled data from the target domain, we also have access to the intermediate distributions, i.e., unlabeled examples from data distributions between source and target, we can use them to boost the performance of the model on the target domain, by applying self-training in a gradual manner.

Original: Kumar et al.
Revised: Kumar et al.

Original: (2020) show that this gradual adaptation leads to a lower error bound on the target domain.
Revised: (2020) show that this gradual adaptation leads to a lower error bound on the target domain.

Original: In order for this approach to be applicable, we need samples from intermediate steps that are annotated with the amount of shift from the source distribution.
Revised: In order for this approach to be applicable, we need samples from intermediate steps that are annotated with the amount of shift from the source distribution.

Original: However, in many cases in practice, we either do not have access to samples of intermediate distributions or such examples do not exist.
Revised: However, in many cases in practice, we either do not have access to samples of intermediate distributions or such examples do not exist.

Original: Even if they exist, it is less likely that they are annotated with the degree of shift.
Revised: Even if they exist, it is less likely that they are annotated with the degree of shift.

Original: To circumvent the issue of lack of samples from intermediate distributions, we create the virtual samples from intermediate distributions by interpolating the input and hidden representations of the data from the source domain P s and target domain P t .
Revised: To circumvent the issue of lack of samples from intermediate distributions, we create the virtual samples from intermediate distributions by interpolating the input and hidden representations of the data from the source domain P s and target domain P t .

Original: Namely, let M φ be a neural network model trained on source data and let z si correspond to the representation of input x si ∈ X s from the source domain.
Revised: Namely, let M φ be a neural network model trained on source data and let z si correspond to the representation of input x si ∈ X s from the source domain.

Original: We choose x tj as a sample from the target domain (we explain the procedure to pick x tj ∈ X t below).
Revised: We choose x tj as a sample from the target domain (we explain the procedure to pick x tj ∈ X t below).

Original: Let z sj correspond to the representation of input x tj ∈ X t that can be sampled from any of the layers of a neural network (including the input layer), and λ ∈ [0 , 1] .
Revised: Let z sj correspond to the representation of input x tj ∈ X t that can be sampled from any of the layers of a neural network (including the input layer), and λ ∈ [0 , 1] .

Original: We generate
Revised: We generate

Original: [Equation]
Revised: [Equation]

Original: as a sample representation of a virtual intermediate distribution.
Revised: as a sample representation of a virtual intermediate distribution.

Original: Note that we do not seek to ﬁnd a corresponding input to the interpolated representation.
Revised: Note that we do not seek to ﬁnd a corresponding input to the interpolated representation.

Original: Gradually increasing λ from 0 to 1 corresponds to feature representations that change from corresponding source distribution representations ( λ = 0 ) to the target distribution representation ( λ = 1 ).
Revised: Gradually increasing λ from 0 to 1 corresponds to feature representations that change from corresponding source distribution representations ( λ = 0 ) to the target distribution representation ( λ = 1 ).

Original: lambda_scheduler in algorithm 1, represent the scheduling function for gradually increasing λ , which can be anything, in our experiments we use a linear function with ﬁxed step size, δ , where the value of lambda increases by a constant value after every δ training steps.
Revised: lambda_scheduler in algorithm 1, represent the scheduling function for gradually increasing λ , which can be anything, in our experiments we use a linear function with ﬁxed step size, δ , where the value of lambda increases by a constant value after every δ training steps.

Original: To choose which example from the source is interpolated with which example from the target, we either randomly align the samples or apply a cost-based alignment method based on the L 2 distance of representations, and the similarity/equality of labels on the source and predicted target labels.
Revised: To choose which example from the source is interpolated with which example from the target, we either randomly align the samples or apply a cost-based alignment method based on the L 2 distance of representations, and the similarity/equality of labels on the source and predicted target labels.

Original: To apply a cost-based alignment method, we use the Sinkhorn matching algorithm (Sinkhorn, 1966; Peyré & Cuturi, 2019) to approximate the alignment with the lowest cost.
Revised: To apply a cost-based alignment method, we use the Sinkhorn matching algorithm (Sinkhorn, 1966; Peyré & Cuturi, 2019) to approximate the alignment with the lowest cost.

Original: Although iterative cost-based alignment methods such as Sinkhorn come at a computational cost, compared to random alignment, we have observed that random alignment without taking (pseudo) labels into account leads to worse performance.
Revised: Although iterative cost-based alignment methods such as Sinkhorn come at a computational cost, compared to random alignment, we have observed that random alignment without taking (pseudo) labels into account leads to worse performance.

Original: To ﬁnd a better trade-off between alignment cost and performance, in our experiments, we also tried a non-iterative heuristic (pseudo) label-based random alignment, where we randomly align examples that have the same (pseudo) labels.
Revised: To ﬁnd a better trade-off between alignment cost and performance, in our experiments, we also tried a non-iterative heuristic (pseudo) label-based random alignment, where we randomly align examples that have the same (pseudo) labels.

Original: This is shown in Algorithm 2.
Revised: This is shown in Algorithm 2.

Original: In our experiments, we report the result from cost-based alignment.
Revised: In our experiments, we report the result from cost-based alignment.

Original: We observed that pseudo-random alignment and cost-based alignment lead to more or less similar results and in practice, we could simply use the threshold
Revised: We observed that pseudo-random alignment and cost-based alignment lead to more or less similar results and in practice, we could simply use the threshold

Original: Notation: [ m ] = { 1 , ...m } .
Revised: Notation: [ m ] = { 1 , ...m } .

Original: 1: M φ ← M θ , λ ←2: while λ < = 1 do 3: for step ∈ [ N ] do 4: ( x s [ m ] , y s [ m ] ) ∼ P s 5: x t [ m ] ∼ P t 6: z s [ m ] ← M : L φ ( x s [ m ] ) 7: z t [ m ] ← M : Lθ ( x t [ m ] ) 8: y t [ m ] ← M θ ( x t [ m ] ) 9: index s , index t ← align ( y s [ m ] , y t [ m ] )
Revised: 1: M φ ← M θ , λ ←2: while λ < = 1 do 3: for step ∈ [ N ] do 4: ( x s [ m ] , y s [ m ] ) ∼ P s 5: x t [ m ] ∼ P t 6: z s [ m ] ← M : L φ ( x s [ m ] ) 7: z t [ m ] ← M : Lθ ( x t [ m ] ) 8: y t [ m ] ← M θ ( x t [ m ] ) 9: index s , index t ← align ( y s [ m ] , y t [ m ] )

Original: (Algorithm 2) 10: (cid:98)
Revised: (Algorithm 2) 10: (cid:98)

Original: z [ m ] ← (1 − λ ) ×
Revised: z [ m ] ← (1 − λ ) ×

Original: z s index s + λ × z t index t 11: (cid:98) y
Revised: z s index s + λ × z t index t 11: (cid:98) y

Original: [ m ] ← M L : θ ( (cid:98) z [ m ] ) 12: conf _ ranks ← rank ( max ( (cid:98) y
Revised: [ m ] ← M L : θ ( (cid:98) z [ m ] ) 12: conf _ ranks ← rank ( max ( (cid:98) y

Original: [ m ] ) − min ( (cid:98) y { ... } )) 13: conf _ indices ← conf _ ranks[: α ] 14:
Revised: [ m ] ) − min ( (cid:98) y { ... } )) 13: conf _ indices ← conf _ ranks[: α ] 14:

Original: (cid:98) z [ m ] , (cid:98) y
Revised: (cid:98) z [ m ] , (cid:98) y

Original: [ m ] ← (cid:98) z [conf _ indices] , (cid:98) y [conf _ indices] 15:
Revised: [ m ] ← (cid:98) z [conf _ indices] , (cid:98) y [conf _ indices] 15:

Original: Fit M φ to ( (cid:98) z
Revised: Fit M φ to ( (cid:98) z

Original: [ m ] , (cid:98) y
Revised: [ m ] , (cid:98) y

Original: [ m ] ) 16: end for 17: λ ← lambda_scheduler ( λ, δ ) 18:
Revised: [ m ] ) 16: end for 17: λ ← lambda_scheduler ( λ, δ ) 18:

Original: M θ ← M φ 19: end while
Revised: M θ ← M φ 19: end while

Original: 1: Input: y s [ m ] , ˆ y t [ m ] 2: Output: index s , index t 3: index s ← [1 , 2 , ..., len ( y s [ m ] )] 4: index t ← [] 5: for i ∈ index s do 6: indices ← [1 , 2 , ..., len (ˆ y t [ m ] )] 7: shuﬄed _ indices ← permute (indices) 8: matching _ scores[ m ] ← (cid:26) 1 for ˆ y t [ m ] =
Revised: 1: Input: y s [ m ] , ˆ y t [ m ] 2: Output: index s , index t 3: index s ← [1 , 2 , ..., len ( y s [ m ] )] 4: index t ← [] 5: for i ∈ index s do 6: indices ← [1 , 2 , ..., len (ˆ y t [ m ] )] 7: shuﬄed _ indices ← permute (indices) 8: matching _ scores[ m ] ← (cid:26) 1 for ˆ y t [ m ] =

Original: y s [ i ] 0 for ˆ y t [ m ] (cid:54) =
Revised: y s [ i ] 0 for ˆ y t [ m ] (cid:54) =

Original: y s [ i ] 9: index ← argmax ( matching _ scores [shuﬄed _ indices]) 10: index t .append (shuﬄed _ indices[index]) 11: end for pseudo-random alignments.
Revised: y s [ i ] 9: index ← argmax ( matching _ scores [shuﬄed _ indices]) 10: index t .append (shuﬄed _ indices[index]) 11: end for pseudo-random alignments.

Original: It is important to note that both for cost-based and pseudo-random alignment we use the pseudo labels predicted by the teacher model since we do not have access to ground truth labels for data points from the target domain.
Revised: It is important to note that both for cost-based and pseudo-random alignment we use the pseudo labels predicted by the teacher model since we do not have access to ground truth labels for data points from the target domain.

Original: In standard iterative self-training (Habrard et al., 2013), the data used in each self-training iteration is a subset of the target distribution, whereas in GIFT we apply the iterative self-training procedure on virtual intermediate distribution representations to gradually adapt the model to the target domain.
Revised: In standard iterative self-training (Habrard et al., 2013), the data used in each self-training iteration is a subset of the target distribution, whereas in GIFT we apply the iterative self-training procedure on virtual intermediate distribution representations to gradually adapt the model to the target domain.

Original: We start by using a teacher model trained on the source distribution and move to train the student model in the representation space.
Revised: We start by using a teacher model trained on the source distribution and move to train the student model in the representation space.

Original: The self-training procedure at each step proceeds by assigning pseudo labels to virtual intermediate representations that are generated by equation 1.
Revised: The self-training procedure at each step proceeds by assigning pseudo labels to virtual intermediate representations that are generated by equation 1.

Original: Next, the student model is updated using pseudo-labeled representations.
Revised: Next, the student model is updated using pseudo-labeled representations.

Original: Then the student becomes the teacher for the next iteration and the procedure continues.
Revised: Then the student becomes the teacher for the next iteration and the procedure continues.

Original: We start with λ = 0 , which generates representations from the source domain.
Revised: We start with λ = 0 , which generates representations from the source domain.

Original: At each iteration of self-training, we increase the value of λ to generate virtual representations that are closer to the target distribution and hence gradually move the model toward the target distribution ( λ = 1 ).
Revised: At each iteration of self-training, we increase the value of λ to generate virtual representations that are closer to the target distribution and hence gradually move the model toward the target distribution ( λ = 1 ).

Original: The details are shown in Algorithm 1.
Revised: The details are shown in Algorithm 1.

Original: Similar to standard iterative self-training, we assign a conﬁdence score to each pseudo-labeled data point and only update the student model with pseudo-labeled data which their conﬁdence score is in the top q -percentile.
Revised: Similar to standard iterative self-training, we assign a conﬁdence score to each pseudo-labeled data point and only update the student model with pseudo-labeled data which their conﬁdence score is in the top q -percentile.

Original: In
Revised: In

Original: Alorithm 1, q is speciﬁed by Conﬁdence Thershold .
Revised: Alorithm 1, q is speciﬁed by Conﬁdence Thershold .

Original: As a conﬁdence score, we use the gap between the highest and lowest logit for each sample, as proposed in Kumar et al.
Revised: As a conﬁdence score, we use the gap between the highest and lowest logit for each sample, as proposed in Kumar et al.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In our experiments, ﬁrst, we examine the power of iterative self-training and GIFT for unsupervised domain adaptation on a dataset with synthetic shifts, where we can track the amount of shift and control the gap between the source and targets distribution.
Revised: In our experiments, ﬁrst, we examine the power of iterative self-training and GIFT for unsupervised domain adaptation on a dataset with synthetic shifts, where we can track the amount of shift and control the gap between the source and targets distribution.

Original: We demonstrate the implicit curriculum followed by iterative self-training and GIFT and show that when condition (i) does not hold, GIFT performs better than iterative self-training.
Revised: We demonstrate the implicit curriculum followed by iterative self-training and GIFT and show that when condition (i) does not hold, GIFT performs better than iterative self-training.

Original: Next, we showcase the performance of these methods in more realistic settings, on datasets with natural distribution shifts.
Revised: Next, we showcase the performance of these methods in more realistic settings, on datasets with natural distribution shifts.

Original: The training setup that we propose has three main phases: (i)
Revised: The training setup that we propose has three main phases: (i)

Original: Pre-training: Pre-training the model on a large scale dataset (ImageNet-1k in our experiments) (ii) Fine-tuning: Training the pre-trained model on a labeled source domain with or without leveraging samples from an unlabeled target domain.
Revised: Pre-training: Pre-training the model on a large scale dataset (ImageNet-1k in our experiments) (ii) Fine-tuning: Training the pre-trained model on a labeled source domain with or without leveraging samples from an unlabeled target domain.

Original: (iii)
Revised: (iii)

Original: Adaptation: Applying a self-training based approach to shift the model towards the unlabeled target domain.
Revised: Adaptation: Applying a self-training based approach to shift the model towards the unlabeled target domain.

Original: In the ﬁne-tuning stage, step (ii), we compare (A) simple ﬁne-tuning on the source domain with standard augmentation techniques, i.e., random ﬂip and random crop, (B) ﬁne-tuning on the source domain with variants of Mixup (Zhang et al., 2018a) as an augmentation technique, and (c)
Revised: In the ﬁne-tuning stage, step (ii), we compare (A) simple ﬁne-tuning on the source domain with standard augmentation techniques, i.e., random ﬂip and random crop, (B) ﬁne-tuning on the source domain with variants of Mixup (Zhang et al., 2018a) as an augmentation technique, and (c)

Original: Domain Adversarial training that uses unlabeled examples from the target domain to learn domain invariant representations (Ganin et al., 2016).
Revised: Domain Adversarial training that uses unlabeled examples from the target domain to learn domain invariant representations (Ganin et al., 2016).

Original: In step (iii), the adaptation phase, we compare three self-training based strategies: (1) one-step self-training, (2) iterative self-training and (3) GIFT, which is similar to iterative self-training except that during the intermediate iterations the model is trained with samples from virtual intermediate states instead of actual samples from the target distribution.
Revised: In step (iii), the adaptation phase, we compare three self-training based strategies: (1) one-step self-training, (2) iterative self-training and (3) GIFT, which is similar to iterative self-training except that during the intermediate iterations the model is trained with samples from virtual intermediate states instead of actual samples from the target distribution.

Original: 4. B ENCHMARKS WITH S YNTHETIC P ERTURBATIONS
Revised: 4. B ENCHMARKS WITH S YNTHETIC P ERTURBATIONS

Original: To investigate the effect of the type and degree of shift, on the success of iterative self-training and
Revised: To investigate the effect of the type and degree of shift, on the success of iterative self-training and

Original: GIFT we compare their performance, on a synthetic benchmark.
Revised: GIFT we compare their performance, on a synthetic benchmark.

Original: In this benchmark, the target domain is created by applying synthetic perturbations on examples from the source domain, such as applying noise, rotating, scaling or translating images.
Revised: In this benchmark, the target domain is created by applying synthetic perturbations on examples from the source domain, such as applying noise, rotating, scaling or translating images.

Original: To create these benchmarks, we split the training set of CIFAR10 (Krizhevsky et al., 2009) into equal-sized splits, where each split contains examples with different degrees of perturbation.
Revised: To create these benchmarks, we split the training set of CIFAR10 (Krizhevsky et al., 2009) into equal-sized splits, where each split contains examples with different degrees of perturbation.

Original: We use four types of perturbation: rotation, scale, translate and blur.
Revised: We use four types of perturbation: rotation, scale, translate and blur.

Original: For rotation, we split the training data into three parts.
Revised: For rotation, we split the training data into three parts.

Original: In the ﬁrst split, images have a rotation angle of 0 to 5 degrees.
Revised: In the ﬁrst split, images have a rotation angle of 0 to 5 degrees.

Original: In the second split, images have a rotation angle of 5 to 55 degrees, and in the last split, images have a rotation angle of 55 to 60.
Revised: In the second split, images have a rotation angle of 5 to 55 degrees, and in the last split, images have a rotation angle of 55 to 60.

Original: We use the ﬁrst split, 0 _ 5 , as the source domain and the third split 55 _ 60 as the target domain.
Revised: We use the ﬁrst split, 0 _ 5 , as the source domain and the third split 55 _ 60 as the target domain.

Original: For scale, translate and blur, we split the training data into two splits, where we use the ﬁrst part with no perturbation as the source domain, and we apply the perturbation on the second part to get the out-of-distribution target domain.
Revised: For scale, translate and blur, we split the training data into two splits, where we use the ﬁrst part with no perturbation as the source domain, and we apply the perturbation on the second part to get the out-of-distribution target domain.

Original: For both scale and translate, we have no variation in the source and some variation in the target.
Revised: For both scale and translate, we have no variation in the source and some variation in the target.

Original: For the blur, we only have one degree of blurring perturbation in the target and none in the source domain.
Revised: For the blur, we only have one degree of blurring perturbation in the target and none in the source domain.

Original: The results on CIFAR10 with different perturbations are shown in Tables 1 and 2.
Revised: The results on CIFAR10 with different perturbations are shown in Tables 1 and 2.

Original: They indicate the superiority of GIFT to iterative self-training.
Revised: They indicate the superiority of GIFT to iterative self-training.

Original: The advantage of GIFT over iterative self-training is more apparent when the target distribution does not include a diverse set of samples (Translated vs
Revised: The advantage of GIFT over iterative self-training is more apparent when the target distribution does not include a diverse set of samples (Translated vs

Original: Blur) or when the two distributions are not overlapping (Translated(0%-100%) vs Translated(50%- 100%)).
Revised: Blur) or when the two distributions are not overlapping (Translated(0%-100%) vs Translated(50%- 100%)).

Original: Additionally, compared to GIFT, the performance of iterative self-training drops more
Revised: Additionally, compared to GIFT, the performance of iterative self-training drops more

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: signiﬁcantly when decreasing the total number of training steps.
Revised: signiﬁcantly when decreasing the total number of training steps.

Original: We can see this by contrasting the results in Table 1 and 2.
Revised: We can see this by contrasting the results in Table 1 and 2.

Original: Investigating the Curricula: To empirically conﬁrm the hypothesis that both iterative self-training and GIFT gradually guide the model to ﬁt the out-of-distribution target distribution, we track the accuracy and conﬁdence of the models on different subsets of the target data throughout the training.
Revised: Investigating the Curricula: To empirically conﬁrm the hypothesis that both iterative self-training and GIFT gradually guide the model to ﬁt the out-of-distribution target distribution, we track the accuracy and conﬁdence of the models on different subsets of the target data throughout the training.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: Number of teacher updates
Revised: Number of teacher updates

Original: [Figure]
Revised: [Figure]

Original: Scaled
Revised: Scaled

Original: Translated(0%-100%)
Revised: Translated(0%-100%)

Original: Blurred
Revised: Blurred

Original: Translated(50%-100%)
Revised: Translated(50%-100%)

Original: Mixed
Revised: Mixed

Original: [Figure]
Revised: [Figure]

Original: Number of teacher updates
Revised: Number of teacher updates

Original: [Figure]
Revised: [Figure]

Original: cc u
Revised: cc u

Original: [Figure]
Revised: [Figure]

Original: ac y (
Revised: ac y (

Original: [Figure]
Revised: [Figure]

Original: GIFT (
Revised: GIFT (

Original: [Figure]
Revised: [Figure]

Original: steps
Revised: steps

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Number of
Revised: Number of

Original: [Figure]
Revised: [Figure]

Original: This is shown in in Figures 2.
Revised: This is shown in in Figures 2.

Original: We see that, for both methods, the accuracy and conﬁdence measures are increasing incrementally from easier examples, i.e., examples with smaller amounts of perturbation, to harder examples, i.e., examples with larger amounts of perturbation.
Revised: We see that, for both methods, the accuracy and conﬁdence measures are increasing incrementally from easier examples, i.e., examples with smaller amounts of perturbation, to harder examples, i.e., examples with larger amounts of perturbation.

Original: For GIFT, the number of iterations is tied to the steps in which we increment the interpolation coefﬁcient, λ , and it is interesting to see that there is a correlation between λ and the accuracy and conﬁdence of the model on actual intermediate steps, as shown in Figure 2.
Revised: For GIFT, the number of iterations is tied to the steps in which we increment the interpolation coefﬁcient, λ , and it is interesting to see that there is a correlation between λ and the accuracy and conﬁdence of the model on actual intermediate steps, as shown in Figure 2.

Original: For iterative self-training, the model selects training examples for which to compute the loss based on its conﬁdence.
Revised: For iterative self-training, the model selects training examples for which to compute the loss based on its conﬁdence.

Original: Hence, tracking the conﬁdence as a function of translation percentage during training allows us to see whether the model is indeed selecting less perturbed examples earlier and more perturbed examples later, thus creates its own curriculum.
Revised: Hence, tracking the conﬁdence as a function of translation percentage during training allows us to see whether the model is indeed selecting less perturbed examples earlier and more perturbed examples later, thus creates its own curriculum.

Original: As we see in Figure 2, for both iterative self-training and GIFT the conﬁdence of the models decreases gradually for data with an increased level of perturbation.
Revised: As we see in Figure 2, for both iterative self-training and GIFT the conﬁdence of the models decreases gradually for data with an increased level of perturbation.

Original: Note also that the conﬁdence scores show very similar patterns to the accuracies.
Revised: Note also that the conﬁdence scores show very similar patterns to the accuracies.

Original: This conﬁrms that the reason behind the success of iterative self-training is the implicit curriculum strategy, and that the gradual interpolation strategy that we employ in GIFT can be a good proxy for gradual self-training when we do not have access to the gradually shifted data.
Revised: This conﬁrms that the reason behind the success of iterative self-training is the implicit curriculum strategy, and that the gradual interpolation strategy that we employ in GIFT can be a good proxy for gradual self-training when we do not have access to the gradually shifted data.

Original: Effect of number of intermediate steps: We examine the effect of the number of intermediate steps when the total number of training steps is ﬁxed.
Revised: Effect of number of intermediate steps: We examine the effect of the number of intermediate steps when the total number of training steps is ﬁxed.

Original: As illustrated in Figure 3, increasing the number of intermediate steps (teacher updates), which means taking smaller steps in the gradual adaptation procedure, leads to a better performance up to a threshold for GIFT and iterative self-training.
Revised: As illustrated in Figure 3, increasing the number of intermediate steps (teacher updates), which means taking smaller steps in the gradual adaptation procedure, leads to a better performance up to a threshold for GIFT and iterative self-training.

Original: If we keep increasing the number of intermediate steps, the performance of the models decreases rapidly for iterative self-training.
Revised: If we keep increasing the number of intermediate steps, the performance of the models decreases rapidly for iterative self-training.

Original: Whereas GIFT is more robust to the number of intermediate steps.
Revised: Whereas GIFT is more robust to the number of intermediate steps.

Original: This decrease in the performance is potentially due to accumulations of the errors of the self-training process or because with a ﬁxed number of training steps, increasing the number of intermediate steps leads to a decrease in the number of iterations in each self-training step, which could mean the model can not adapt well to each intermediate step.
Revised: This decrease in the performance is potentially due to accumulations of the errors of the self-training process or because with a ﬁxed number of training steps, increasing the number of intermediate steps leads to a decrease in the number of iterations in each self-training step, which could mean the model can not adapt well to each intermediate step.

Original: The robustness of the GIFT, in this case, could mean that this method needs fewer iterations in each intermediate step.
Revised: The robustness of the GIFT, in this case, could mean that this method needs fewer iterations in each intermediate step.

Original: Comparing Figure 3, we see that for both iterative self-training and GIFT, decreasing the number of total training steps (500 vs 1000), reduces the effective number of teacher updates.
Revised: Comparing Figure 3, we see that for both iterative self-training and GIFT, decreasing the number of total training steps (500 vs 1000), reduces the effective number of teacher updates.

Original: This conﬁrms the hypothesis that the model can
Revised: This conﬁrms the hypothesis that the model can

Original: [Table]
Revised: [Table]

Original: beneﬁt from more teacher updates if it has enough time to properly adapt to each intermediate step.
Revised: beneﬁt from more teacher updates if it has enough time to properly adapt to each intermediate step.

Original: We provide more analysis for this in Appendix C.
Revised: We provide more analysis for this in Appendix C.

Original: 4. B ENCHMARKS WITH N ATURAL D ISTRIBUTION S HIFT
Revised: 4. B ENCHMARKS WITH N ATURAL D ISTRIBUTION S HIFT

Original: We report results on two datasets with natural distribution shifts: FMoW (Christie et al., 2018) and
Revised: We report results on two datasets with natural distribution shifts: FMoW (Christie et al., 2018) and

Original: Camelyon17 (Bandi et al., 2018) from the WILDS benchmark (Koh et al., 2021).
Revised: Camelyon17 (Bandi et al., 2018) from the WILDS benchmark (Koh et al., 2021).

Original: Here we brieﬂy introduce each of these datasets:
Revised: Here we brieﬂy introduce each of these datasets:

Original: FMoW is a variant of the Functional Map of the World dataset that contains satellite imagery of the earth.
Revised: FMoW is a variant of the Functional Map of the World dataset that contains satellite imagery of the earth.

Original: The images belong to 62 building or land use categories, and the domain represents both the year the image was taken as well as its geographical region.
Revised: The images belong to 62 building or land use categories, and the domain represents both the year the image was taken as well as its geographical region.

Original: Here we only address the domain shift problem over time.
Revised: Here we only address the domain shift problem over time.

Original: For the adaptation phase, we use unlabeled samples from the out-of-distribution test split of the dataset.
Revised: For the adaptation phase, we use unlabeled samples from the out-of-distribution test split of the dataset.

Original: Camelyon17 is the patch-based variant of Camelyon17 (Bandi et al., 2018) with hospitals ( 0 , 1 , 2 , 3 , 4 ) as domains.
Revised: Camelyon17 is the patch-based variant of Camelyon17 (Bandi et al., 2018) with hospitals ( 0 , 1 , 2 , 3 , 4 ) as domains.

Original: The task is to predict if a given region of tissue contains a tumor.
Revised: The task is to predict if a given region of tissue contains a tumor.

Original: We use 0 , 1 , 2 as source during ﬁne-tuning, but for adaptation we only use labeled data from domain 0 .
Revised: We use 0 , 1 , 2 as source during ﬁne-tuning, but for adaptation we only use labeled data from domain 0 .

Original: We use 3 as the target domain.
Revised: We use 3 as the target domain.

Original: This is an example of a dataset, for which annotated intermediate distributions between different domains do not exist naturally.
Revised: This is an example of a dataset, for which annotated intermediate distributions between different domains do not exist naturally.

Original: As shown in Table 3 on datasets with natural shift, all three self-training based approaches, selftraining, iterative self-training and GIFT improve the accuracy over the best ﬁne-tuned models.
Revised: As shown in Table 3 on datasets with natural shift, all three self-training based approaches, selftraining, iterative self-training and GIFT improve the accuracy over the best ﬁne-tuned models.

Original: While on FMoW performance of GIFT is on par with iterative self-training, it achieves a higher accuracy on Camelyon17.
Revised: While on FMoW performance of GIFT is on par with iterative self-training, it achieves a higher accuracy on Camelyon17.

Original: An advantage for GIFT over iterative self-training in these scenarios is that, the performance of GIFT does not rely on the assumption (i) ,that the target distribution should include samples from intermediate steps since in the intermediate steps of training it merely learns from the interpolated samples.
Revised: An advantage for GIFT over iterative self-training in these scenarios is that, the performance of GIFT does not rely on the assumption (i) ,that the target distribution should include samples from intermediate steps since in the intermediate steps of training it merely learns from the interpolated samples.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Unsupervised domain adaptation: In unsupervised domain adaptation, we have access to labeled examples from the source domain(s) and unlabeled examples from the target domain(s) and the goal is to get a good performance on the target domain.
Revised: Unsupervised domain adaptation: In unsupervised domain adaptation, we have access to labeled examples from the source domain(s) and unlabeled examples from the target domain(s) and the goal is to get a good performance on the target domain.

Original: Unsupervised domain adaptation techniques fall within three main categories (Sun et al., 2020): (1) Methods based on matching the feature distributions of source and target domains.
Revised: Unsupervised domain adaptation techniques fall within three main categories (Sun et al., 2020): (1) Methods based on matching the feature distributions of source and target domains.

Original: Algorithms in this group rely on the assumption that the models can learn domain invariant representations, and they employ different self-supervised based losses to enforce this invariance by exposing the model to the unlabeled data from the target distribution (Ganin et al., 2016; Ben-David et al., 2006).
Revised: Algorithms in this group rely on the assumption that the models can learn domain invariant representations, and they employ different self-supervised based losses to enforce this invariance by exposing the model to the unlabeled data from the target distribution (Ganin et al., 2016; Ben-David et al., 2006).

Original: (2) Methods based on transforming source and target distributions.
Revised: (2) Methods based on transforming source and target distributions.

Original: They analyze the input space and project source and target data to a lower dimensional manifold and try to ﬁnd a transformation between the two (Fernando et al., 2013;
Revised: They analyze the input space and project source and target data to a lower dimensional manifold and try to ﬁnd a transformation between the two (Fernando et al., 2013;

Original: Gopalan et al., 2011; Harel & Mannor, 2010).
Revised: Gopalan et al., 2011; Harel & Mannor, 2010).

Original: Another approach in this line of work is to transform the source data to be as close to the target data as possible.
Revised: Another approach in this line of work is to transform the source data to be as close to the target data as possible.

Original: For example, Sun et al.
Revised: For example, Sun et al.

Original: (2017) matches the second order statistics of the input spaces.
Revised: (2017) matches the second order statistics of the input spaces.

Original: (3) Self-training based methods.
Revised: (3) Self-training based methods.

Original: Self-training:
Revised: Self-training:

Original: Recent works have shown signiﬁcant progress using self-training in computer vision (Xie et al., 2020; Yalniz et al., 2019; Zoph et al., 2020).
Revised: Recent works have shown signiﬁcant progress using self-training in computer vision (Xie et al., 2020; Yalniz et al., 2019; Zoph et al., 2020).

Original: Self-training has also been used for domain adaptation by generating pseudo labels in the target domain and directly training a model for the target domain (Xie et al., 2018;
Revised: Self-training has also been used for domain adaptation by generating pseudo labels in the target domain and directly training a model for the target domain (Xie et al., 2018;

Original: Saito et al., 2017; Chang et al., 2019; Manders et al., 2018; Zou et al.,
Revised: Saito et al., 2017; Chang et al., 2019; Manders et al., 2018; Zou et al.,

Original: (Chang et al., 2019) uses different normalization parameters for source examples and pseudo labeled examples in the target domain.
Revised: (Chang et al., 2019) uses different normalization parameters for source examples and pseudo labeled examples in the target domain.

Original: Zou et al.
Revised: Zou et al.

Original: (2019) introduced label-regularized self-training which generates soft pseudo-labels for self-training.
Revised: (2019) introduced label-regularized self-training which generates soft pseudo-labels for self-training.

Original: Different from these works, we use a curriculum learning approach where we generate pseudo labels for intermediate virtual examples and gradually adapt the model to the target domain.
Revised: Different from these works, we use a curriculum learning approach where we generate pseudo labels for intermediate virtual examples and gradually adapt the model to the target domain.

Original: Curriculum learning: Curriculum learning (Elman, 1993;
Revised: Curriculum learning: Curriculum learning (Elman, 1993;

Original: Sanger, 1994; Bengio et al., 2009) has led to better performance in terms of generalization and/or convergence speed in many domains such as computer vision (Pentina et al., 2015; Saraﬁanos et al., 2017; Guo et al., 2018; Wang et al., 2019), natural language processing (Cirik et al., 2016;
Revised: Sanger, 1994; Bengio et al., 2009) has led to better performance in terms of generalization and/or convergence speed in many domains such as computer vision (Pentina et al., 2015; Saraﬁanos et al., 2017; Guo et al., 2018; Wang et al., 2019), natural language processing (Cirik et al., 2016;

Original: Platanios et al., 2019) and neural evolutionary computing (Zaremba & Sutskever, 2014).
Revised: Platanios et al., 2019) and neural evolutionary computing (Zaremba & Sutskever, 2014).

Original: On the other hand, there have been some negative results in neural machine translation (Kocmi & Bojar, 2017; Zhang et al., 2018b; 2019).
Revised: On the other hand, there have been some negative results in neural machine translation (Kocmi & Bojar, 2017; Zhang et al., 2018b; 2019).

Original: In this work, we investigate it in the unsupervised domain adaptation scenario.
Revised: In this work, we investigate it in the unsupervised domain adaptation scenario.

Original: Different notions of “difﬁculty” of examples are used in the literature, such as using the loss value of a pre-trained model (Bengio et al., 2009), or the ﬁrst iteration in which an example is learned and remains learned after that (Toneva et al., 2019).
Revised: Different notions of “difﬁculty” of examples are used in the literature, such as using the loss value of a pre-trained model (Bengio et al., 2009), or the ﬁrst iteration in which an example is learned and remains learned after that (Toneva et al., 2019).

Original: Jiang et al.
Revised: Jiang et al.

Original: (2020) have proposed using a consistency score calculated based on the consistency of a model in correctly predicting a particular example’s label trained on i.i.d.
Revised: (2020) have proposed using a consistency score calculated based on the consistency of a model in correctly predicting a particular example’s label trained on i.i.d.

Original: draws from the training set.
Revised: draws from the training set.

Original: Wu et al.
Revised: Wu et al.

Original: (2021) show that all these difﬁculty scores are consistent.
Revised: (2021) show that all these difﬁculty scores are consistent.

Original: Here, we use the coefﬁcient of linear interpolation between source and target representation as a measure of the difﬁculty of a (virtual) sample.
Revised: Here, we use the coefﬁcient of linear interpolation between source and target representation as a measure of the difﬁculty of a (virtual) sample.

Original: Leveraging synthetic data for domain adaptation: We build on top of the existing work that incorporate synthetic data to bridge between source and target domain Gopalan et al.
Revised: Leveraging synthetic data for domain adaptation: We build on top of the existing work that incorporate synthetic data to bridge between source and target domain Gopalan et al.

Original: Gong et al.
Revised: Gong et al.

Original: ; Cui et al.
Revised: ; Cui et al.

Original: Na et al.
Revised: Na et al.

Original: Similar to Na et al.
Revised: Similar to Na et al.

Original: we use mixup to create intermediate samples, but instead of using ﬁxed mixup rates, we incorporate an annealing strategy so the created instances, gradually move away from the source domain and get closer to the samples from the target domain.
Revised: we use mixup to create intermediate samples, but instead of using ﬁxed mixup rates, we incorporate an annealing strategy so the created instances, gradually move away from the source domain and get closer to the samples from the target domain.

Original: Additionally, we apply mixup both in the input layer and representations obtained from various layers of the models.
Revised: Additionally, we apply mixup both in the input layer and representations obtained from various layers of the models.

Original: Gong et al.
Revised: Gong et al.

Original: (2019) proposes to create intermediate domains using CycleGAN (Hoffman et al., 2018), wheras in GIFT, we create the intermediate domains in the representational space of the model and abstract away from actual images corresponding to those representations.
Revised: (2019) proposes to create intermediate domains using CycleGAN (Hoffman et al., 2018), wheras in GIFT, we create the intermediate domains in the representational space of the model and abstract away from actual images corresponding to those representations.

Original:  D ISCUSSION AND C ONCLUSION
Revised:  D ISCUSSION AND C ONCLUSION

Original: We establish the importance of having a proper curriculum for the success of self-training algorithms for domain adaptation.
Revised: We establish the importance of having a proper curriculum for the success of self-training algorithms for domain adaptation.

Original: We demonstrate that iterative self-training can successfully adapt the model to the new target distribution if the target distribution contains the intermediate examples.
Revised: We demonstrate that iterative self-training can successfully adapt the model to the new target distribution if the target distribution contains the intermediate examples.

Original: We show that iterative self-training is less effective if the source and target distributions are not overlapping.
Revised: We show that iterative self-training is less effective if the source and target distributions are not overlapping.

Original: Our proposed method, GIFT, is speciﬁcally designed to deal with cases where there is a gap between source and target distributions.
Revised: Our proposed method, GIFT, is speciﬁcally designed to deal with cases where there is a gap between source and target distributions.

Original: In such scenarios, applying self-training iteratively, and ﬁltering examples based on the conﬁdence of the model would not result in a proper curriculum since the intermediate examples are missing.
Revised: In such scenarios, applying self-training iteratively, and ﬁltering examples based on the conﬁdence of the model would not result in a proper curriculum since the intermediate examples are missing.

Original: We propose and perform controlled experiments on distribution shift on a synthetic benchmark created from CIFAR10.
Revised: We propose and perform controlled experiments on distribution shift on a synthetic benchmark created from CIFAR10.

Original: More speciﬁcally, we control the amount of shift, the type of shift, and whether there is a gap between the data distributions in the source and target domain.
Revised: More speciﬁcally, we control the amount of shift, the type of shift, and whether there is a gap between the data distributions in the source and target domain.

Original: Our experiments conﬁrm the importance of the presence of samples from intermediate distributions for the success of iterative self-training and show that in the absence of intermediate samples, GIFT improves the results over iterative self-training.
Revised: Our experiments conﬁrm the importance of the presence of samples from intermediate distributions for the success of iterative self-training and show that in the absence of intermediate samples, GIFT improves the results over iterative self-training.

Original: We observe similar results on datasets with natural distribution shifts.
Revised: We observe similar results on datasets with natural distribution shifts.

Original: GIFT is inspired by existing works that rely on interpolating representations in the input or feature space
Revised: GIFT is inspired by existing works that rely on interpolating representations in the input or feature space

Original: (Gong et al., 2012; Verma et al., 2019a;b; Neyshabur et al., 2020).
Revised: (Gong et al., 2012; Verma et al., 2019a;b; Neyshabur et al., 2020).

Original: An interesting direction for future work is to investigate other ways of interpolating in the feature space and explore the use of more sophisticated interpolation schemes, such as schemes based on optimal transport.
Revised: An interesting direction for future work is to investigate other ways of interpolating in the feature space and explore the use of more sophisticated interpolation schemes, such as schemes based on optimal transport.

Original: Moreover, the experiments in this paper are limited to image classiﬁcation.
Revised: Moreover, the experiments in this paper are limited to image classiﬁcation.

Original: Extending our approach beyond classiﬁcation and to other data modalities such as textual data is an interesting next step.
Revised: Extending our approach beyond classiﬁcation and to other data modalities such as textual data is an interesting next step.

Original: Lastly, we emphasise the importance of designing benchmarks speciﬁcally for the purpose of evaluating gradual domain adaptation techniques under different conditions, as one of the most important steps in this line of research.
Revised: Lastly, we emphasise the importance of designing benchmarks speciﬁcally for the purpose of evaluating gradual domain adaptation techniques under different conditions, as one of the most important steps in this line of research.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: We have shared the details of the proposed algorithms and experimental setup in sections 3 and
Revised: We have shared the details of the proposed algorithms and experimental setup in sections 3 and

Original: A. Additionally, we provide a simple implementation of the main algorithms proposed at https: //github.com/anonymouse-github/GIFT .
Revised: A. Additionally, we provide a simple implementation of the main algorithms proposed at https: //github.com/anonymouse-github/GIFT .

Original: We ran experiments on a dataset with synthetic shifts, for which we thoroughly explain how it is created in section 4.1, as well as two datasets with natural distribution shifts which are already available as part of the WILDS benchmark (Koh et al., 2021)
Revised: We ran experiments on a dataset with synthetic shifts, for which we thoroughly explain how it is created in section 4.1, as well as two datasets with natural distribution shifts which are already available as part of the WILDS benchmark (Koh et al., 2021)

Original: A E XPERIMENTAL S ETUP
Revised: A E XPERIMENTAL S ETUP

Original: Optimizer and learning rate schedule.
Revised: Optimizer and learning rate schedule.

Original: In all our experiments on datasets with natural distribution shift, we use Adam optimizer.
Revised: In all our experiments on datasets with natural distribution shift, we use Adam optimizer.

Original: When training on the labeled source domain we use a learning rate schedule with cosine decay and initial learning rate of 1e-4 and when adapting to the target domain we use a learning rate of 1e-5 with an exponential decay rate of 0.9.
Revised: When training on the labeled source domain we use a learning rate schedule with cosine decay and initial learning rate of 1e-4 and when adapting to the target domain we use a learning rate of 1e-5 with an exponential decay rate of 0.9.

Original: For pretraining the models on
Revised: For pretraining the models on

Original: ImageNet-1k, we use a batch size of 1024 and the learning rate schedule is linear warmup (forepochs) + cosine decay with the base learning rate of 0.1.
Revised: ImageNet-1k, we use a batch size of 1024 and the learning rate schedule is linear warmup (forepochs) + cosine decay with the base learning rate of 0.1.

Original: For Perturbed Cifar10 experiments, we use SGD with momentum.
Revised: For Perturbed Cifar10 experiments, we use SGD with momentum.

Original: During the pretraining stage on the source domain, the learning rate schedule is cosine decay with an initial learning rate of 0.1.
Revised: During the pretraining stage on the source domain, the learning rate schedule is cosine decay with an initial learning rate of 0.1.

Original: In all experiments, we use L2 loss with the weight of 1e-5.
Revised: In all experiments, we use L2 loss with the weight of 1e-5.

Original: During the adaptation phase, we use a batch size of 256 and the learning rate is constant and set to 1e-3.
Revised: During the adaptation phase, we use a batch size of 256 and the learning rate is constant and set to 1e-3.

Original: For experiments on Perturbed Cifar with a higher number of training steps (20000 steps), we use a lower learning rate of 1e-4 in the adaptation phase.
Revised: For experiments on Perturbed Cifar with a higher number of training steps (20000 steps), we use a lower learning rate of 1e-4 in the adaptation phase.

Original: Neural network architectures.
Revised: Neural network architectures.

Original: For experiments on datasets with natural shifts, we use a ResNet101.
Revised: For experiments on datasets with natural shifts, we use a ResNet101.

Original: For the experiments on Perturbed Cifar10, we use a WideResNet28-10 with a dropout rate of 0.3.
Revised: For the experiments on Perturbed Cifar10, we use a WideResNet28-10 with a dropout rate of 0.3.

Original: For the virtual interpolations in GIFT: we use the ﬁrst three layers (input, initial convolution layer, and the layer above it).
Revised: For the virtual interpolations in GIFT: we use the ﬁrst three layers (input, initial convolution layer, and the layer above it).

Original: Training on source during the adaptation phase.
Revised: Training on source during the adaptation phase.

Original: We do not train the model on the source data during the adaptation phase.
Revised: We do not train the model on the source data during the adaptation phase.

Original: While in some cases this could result in a better performance on both the source and target domain, our assumption here is that there is no reason for the source and target data to be compatible, i.e. it is possible for the model to not be able to ﬁt both distributions simultaneously.
Revised: While in some cases this could result in a better performance on both the source and target domain, our assumption here is that there is no reason for the source and target data to be compatible, i.e. it is possible for the model to not be able to ﬁt both distributions simultaneously.

Original: Regularization During adaptation, we use the weight decay of 0.01.
Revised: Regularization During adaptation, we use the weight decay of 0.01.

Original: In addition to weight decay, we use another regularization term that encourages the model to stay close to its initial state.
Revised: In addition to weight decay, we use another regularization term that encourages the model to stay close to its initial state.

Original: This regularization term is simply computed as the L2 distance of the parameters of the model and their value at its initial state.
Revised: This regularization term is simply computed as the L2 distance of the parameters of the model and their value at its initial state.

Original: We set the weight for this factor to 0.001 in all adaptation experiments.
Revised: We set the weight for this factor to 0.001 in all adaptation experiments.

Original: Backpropagating gradients through interpolations.
Revised: Backpropagating gradients through interpolations.

Original: We train the model with the representations of virtual examples that we create by interpolating the representations from real examples.
Revised: We train the model with the representations of virtual examples that we create by interpolating the representations from real examples.

Original: During pretraining on the source domain, we use manifold mixup regularization, where we interpolate between representations of labeled source examples.
Revised: During pretraining on the source domain, we use manifold mixup regularization, where we interpolate between representations of labeled source examples.

Original: During the adaptation stage that is part of GIFT, we interpolate between labeled source representations and unlabeled target representations.
Revised: During the adaptation stage that is part of GIFT, we interpolate between labeled source representations and unlabeled target representations.

Original: One important hyper-parameter related to this is whether in the backward pass we back-propagate the gradients all the way down to the input layer or stop the gradients at the layer in which the interpolated representations are computed.
Revised: One important hyper-parameter related to this is whether in the backward pass we back-propagate the gradients all the way down to the input layer or stop the gradients at the layer in which the interpolated representations are computed.

Original: In our experiments, similar to Verma et al. (2019a)
Revised: In our experiments, similar to Verma et al. (2019a)

Original: we allow the gradients to pass through the interpolated activations.
Revised: we allow the gradients to pass through the interpolated activations.

Original: B T RAINING ON L ABELED S OURCE D OMAIN
Revised: B T RAINING ON L ABELED S OURCE D OMAIN

Original: We compare four different approaches for training the model on labeled source data.
Revised: We compare four different approaches for training the model on labeled source data.

Original: Standard ﬁne-tuning:
Revised: Standard ﬁne-tuning:

Original: Given labeled examples from a source domain, we employ a model that is pretrained on some large scale dataset, Imagenet-1k in our case, replace its head (projection layer) with a new head for the task at hand, and update all its parameters to ﬁt the source domain data.
Revised: Given labeled examples from a source domain, we employ a model that is pretrained on some large scale dataset, Imagenet-1k in our case, replace its head (projection layer) with a new head for the task at hand, and update all its parameters to ﬁt the source domain data.

Original: Mixup with convex combination interpolations: During ﬁne-tuning on labeled source data, we apply mixup/manifold (Zhang et al., 2018a; Verma et al., 2019a) on the input and activation from the ﬁrst layer of the model, and to compute the interpolations we simply compute the convex combination of features for two randomly aligned examples.
Revised: Mixup with convex combination interpolations: During ﬁne-tuning on labeled source data, we apply mixup/manifold (Zhang et al., 2018a; Verma et al., 2019a) on the input and activation from the ﬁrst layer of the model, and to compute the interpolations we simply compute the convex combination of features for two randomly aligned examples.

Original: Mixup with wasserstein interpolations: During ﬁnetuning on labeled source data, we apply a variant of mixup/manifold mixup where interpolations are computed using the closed-form Monge Map for Gaussian Wasserstein distances.
Revised: Mixup with wasserstein interpolations: During ﬁnetuning on labeled source data, we apply a variant of mixup/manifold mixup where interpolations are computed using the closed-form Monge Map for Gaussian Wasserstein distances.

Original: In our experiments, we observed that in some cases, interpolating examples in this alternative way leads to better results compared to the convex interpolations used in Verma et al. (2019a).
Revised: In our experiments, we observed that in some cases, interpolating examples in this alternative way leads to better results compared to the convex interpolations used in Verma et al. (2019a).

Original: Domain adversarial neural networks (DANN): Given labeled samples from a source domain and unlabeled samples from the target domain, DANN (Ganin et al., 2016) learns domain invariant representations, while minimizing its error on labels source data.
Revised: Domain adversarial neural networks (DANN): Given labeled samples from a source domain and unlabeled samples from the target domain, DANN (Ganin et al., 2016) learns domain invariant representations, while minimizing its error on labels source data.

Original: In our experiments, the output of the prelogits layer is fed to the domain classiﬁer, and the scheduling of the weight of the domain classiﬁcation loss is the same as what is suggested in Ganin et al.
Revised: In our experiments, the output of the prelogits layer is fed to the domain classiﬁer, and the scheduling of the weight of the domain classiﬁcation loss is the same as what is suggested in Ganin et al.

Original: B. M ANIFOLD M IXUP WITH W ASSERSTEIN I NTERPOLATION
Revised: B. M ANIFOLD M IXUP WITH W ASSERSTEIN I NTERPOLATION

Original: During training on the labeled source dataset, we use a variant of manifold mixup (Verma et al., 2019a) with an adapted strategy for interpolation.
Revised: During training on the labeled source dataset, we use a variant of manifold mixup (Verma et al., 2019a) with an adapted strategy for interpolation.

Original: In manifold mixup (Verma et al., 2019a), representations of virtual examples are created by linearly interpolating representations of two randomly aligned examples ( x i , x j ) in a randomly selected layer L of the neural network M θ .
Revised: In manifold mixup (Verma et al., 2019a), representations of virtual examples are created by linearly interpolating representations of two randomly aligned examples ( x i , x j ) in a randomly selected layer L of the neural network M θ .

Original: The labels for the interpolated examples, ˆ y ij , are computed by interpolating the labels of the aligned examples, ( y i , y j ) , using the same interpolation coefﬁcient, λ .
Revised: The labels for the interpolated examples, ˆ y ij , are computed by interpolating the labels of the aligned examples, ( y i , y j ) , using the same interpolation coefﬁcient, λ .

Original: This is summarized in equation 2.
Revised: This is summarized in equation 2.

Original: [Equation]
Revised: [Equation]

Original: Here M Lθ denotes the part of the neural network that outputs the activations of layer L .
Revised: Here M Lθ denotes the part of the neural network that outputs the activations of layer L .

Original: The interpolated representations are then fed into the rest of the neural network at layer, L , and together with the interpolated labels, they serve as additional ‘data’ to which the model is ﬁt.
Revised: The interpolated representations are then fed into the rest of the neural network at layer, L , and together with the interpolated labels, they serve as additional ‘data’ to which the model is ﬁt.

Original: In our experiments, we take a different approach to interpolation.
Revised: In our experiments, we take a different approach to interpolation.

Original: Inspired by the style transfer method discussed in Mroueh (2020b), we use interpolations based on the Wasserstein distance between two Gaussian distributions that are ﬁt to representations of two input images.
Revised: Inspired by the style transfer method discussed in Mroueh (2020b), we use interpolations based on the Wasserstein distance between two Gaussian distributions that are ﬁt to representations of two input images.

Original: i.e., the spatial features in the representations of two images are the data points for two datasets.
Revised: i.e., the spatial features in the representations of two images are the data points for two datasets.

Original: We estimate the empirical mean and diagonal covariance matrices for these data points and use the closed-form optimal transport map between two Gaussian distributions to interpolate the spatial features between two representations.
Revised: We estimate the empirical mean and diagonal covariance matrices for these data points and use the closed-form optimal transport map between two Gaussian distributions to interpolate the spatial features between two representations.

Original: More precisely, given two images x i and x j , we compute representations z i = M Lθ ( x i ) and z j = M Lθ ( x j ) , where z i and z j are three-dimensional tensors with a width W L , height H L and channel size C L .
Revised: More precisely, given two images x i and x j , we compute representations z i = M Lθ ( x i ) and z j = M Lθ ( x j ) , where z i and z j are three-dimensional tensors with a width W L , height H L and channel size C L .

Original: Each spatial feature vector of size C L within z i and z j is considered one data point.
Revised: Each spatial feature vector of size C L within z i and z j is considered one data point.

Original: We compute the average feature vectors within z i and z j , denoted by µ i and µ j respectively, as well as the variances σ 2 i and σ 2 j .
Revised: We compute the average feature vectors within z i and z j , denoted by µ i and µ j respectively, as well as the variances σ 2 i and σ 2 j .

Original: Note that we are approximating the empirical covariance matrices with diagonal matrices with the variances σ 2 i and σ 2 j on the diagonals.
Revised: Note that we are approximating the empirical covariance matrices with diagonal matrices with the variances σ 2 i and σ 2 j on the diagonals.

Original: Given these quantities, we can compute the closed-form Monge map between two Gaussian distributions with diagonal covariance matrices as
Revised: Given these quantities, we can compute the closed-form Monge map between two Gaussian distributions with diagonal covariance matrices as

Original: [Equation]
Revised: [Equation]

Original: Here z is understood to be a feature map of the same size as z i and z j .
Revised: Here z is understood to be a feature map of the same size as z i and z j .

Original: Interpolated representations are then computed with
Revised: Interpolated representations are then computed with

Original: [Equation]
Revised: [Equation]

Original: Similar to its use in style transfer (Mroueh, 2020b), we assume this transformation does not change the content of the representation, and we therefore do not apply an interpolation scheme to the labels y i and y j of datapoints x i and x j .
Revised: Similar to its use in style transfer (Mroueh, 2020b), we assume this transformation does not change the content of the representation, and we therefore do not apply an interpolation scheme to the labels y i and y j of datapoints x i and x j .

Original: Instead, we use the label ˆ y ij =
Revised: Instead, we use the label ˆ y ij =

Original: y i for the virtual representation ˆ z ij .
Revised: y i for the virtual representation ˆ z ij .

Original: In Verma et al. (2019a) the interpolation coefﬁcient λ is sampled from a Beta distribution Beta( α, β ) , where α and β are hyperparameters.
Revised: In Verma et al. (2019a) the interpolation coefﬁcient λ is sampled from a Beta distribution Beta( α, β ) , where α and β are hyperparameters.

Original: In our experiments we set both α and β to 1, so that we are effectively sampling λ uniformly from the interval [0 , 1) .
Revised: In our experiments we set both α and β to 1, so that we are effectively sampling λ uniformly from the interval [0 , 1) .

Original: C E FFECT OF N UMBER OF T EACHER U PDATES
Revised: C E FFECT OF N UMBER OF T EACHER U PDATES

Original: To better distinguish the effect of the number of teacher updates from the number of training steps between each two consecutive teacher updates, in Figure 4, we plot the accuracy as a function of the number of teacher updates with a ﬁxed number of training steps.
Revised: To better distinguish the effect of the number of teacher updates from the number of training steps between each two consecutive teacher updates, in Figure 4, we plot the accuracy as a function of the number of teacher updates with a ﬁxed number of training steps.

Original: That means, in this setup, the total number of training steps increases as we increase the number of teacher updates.
Revised: That means, in this setup, the total number of training steps increases as we increase the number of teacher updates.

Original: For GIFT, we observe an increasing trend in the accuracy as the number of teacher update increases on all the benchmarks.
Revised: For GIFT, we observe an increasing trend in the accuracy as the number of teacher update increases on all the benchmarks.

Original: However, for iterative self-training, we only see a beneﬁt in increasing the number of teacher updates for datasets with a range of perturbations in the target domain, such as the Translated
Revised: However, for iterative self-training, we only see a beneﬁt in increasing the number of teacher updates for datasets with a range of perturbations in the target domain, such as the Translated

Original: CIFAR10 datasets, as opposed to Blurred CIFAR10.
Revised: CIFAR10 datasets, as opposed to Blurred CIFAR10.

Original: Additionally, Figure 5 shows the effect of the number of teacher updates when the total number of training steps is 1000 for a non-gradual version of GIFT, where all the interpolations are presented to the model simultaneously.
Revised: Additionally, Figure 5 shows the effect of the number of teacher updates when the total number of training steps is 1000 for a non-gradual version of GIFT, where all the interpolations are presented to the model simultaneously.

Original: We observe that, in this case, compared to GIFT, where the value of the interpolation coefﬁcient λ gradually increases, having more teacher updates is less beneﬁcial.
Revised: We observe that, in this case, compared to GIFT, where the value of the interpolation coefﬁcient λ gradually increases, having more teacher updates is less beneﬁcial.

Original: Number of
Revised: Number of

Original: [Figure]
Revised: [Figure]

Original: Translated (0%-100%)
Revised: Translated (0%-100%)

Original: Translated(50%-100%)
Revised: Translated(50%-100%)

Original: Blurred
Revised: Blurred

Original: [Figure]
Revised: [Figure]

Original: Number of teacher updates
Revised: Number of teacher updates

Original: [Figure] [Figure]
Revised: [Figure] [Figure]