Gradual Domain Adaptation without Indexed Intermediate Domains
Authors: Hong-You Chen
Wei-Lun Chao
Venue: NeurIPS 2021 Poster (2021)
Keywords: Gradual Domain Adaptation
classification

Abstract: The effectiveness of unsupervised domain adaptation degrades when there is a large discrepancy between the source and target domains. Gradual domain adaption (GDA) is one promising way to mitigate such an issue, by leveraging additional unlabeled data that gradually shift from the source to the target. Through sequentially adapting the model along the "indexed" intermediate domains, GDA substantially improves the overall adaptation performance. In practice, however, the extra unlabeled data may not be separated into intermediate domains and indexed properly, limiting the applicability of GDA. In this paper, we investigate how to discover the sequence of intermediate domains when it is not already available. Concretely, we propose a coarse-to-fine framework, which starts with a coarse domain discovery step via progressive domain discriminator training. This coarse domain sequence then undergoes a fine indexing step via a novel cycle-consistency loss, which encourages the next intermediate domain to preserve sufficient discriminative knowledge of the current intermediate domain. The resulting domain sequence can then be used by a GDA algorithm. On benchmark data sets of GDA, we show that our approach, which we name Intermediate DOmain Labeler (IDOL), can lead to comparable or even better adaptation performance compared to the pre-defined domain sequence, making GDA more applicable and robust to the quality of domain sequences. Codes are available at https://github.com/hongyouc/IDOL.

Gradual Domain Adaptation without Indexed Intermediate Domains
Hong-You Chen
Wei-Lun Chao The Ohio State University, USA chao. 2098osu. edu
The Ohio State University, USA chen. 9300.931100 10suu edu
Abstract
The effectiveness of unsupervised domain adaptation degrades when there is a large discrepancy between the source and target domains. Gradual domain adaption (GDA) one promising way mitigate such an issue, by leveraging additional un- labeled data that gradually shift from the source to the target. Through sequentially adapting the model along the "indexed" intermediate domains, GDA substantially improves the overall adaptation performance. In practice, however, the extra unla- beled data may not be separated into intermediate domains and indexed properly, limiting the applicability of GDA. In this paper, we investigate how to discover the sequence intermediate domains when it not already available. Concretely, we propose coarse-to to-fee framework, which starts with coarse domain dis- covery step via progressive domain discriminator training. This coarse domain sequence then undergoes a fine indexing step via novel cycle-consistenge loss, which encourages the next intermediate domain to preserve sufficient discriminative knowledge of the current intermediate domain. The resulting domain sequence then be used by a GDA algorithm. On benchmark data sets of GDA, we show that approach, which comparable or even better adaptation performance compared to the pre-defined do- main sequence, making GDA more applicable and robust the quality of domain sequences. Codes are available at
name Intermediate DOmain Labeler (IDOL),
lead
h //t/ttgttb//////////////// com/ httm/hongyocccIDee
Introduction
The distributions of real-world data change dynamically due many factors like time, locations, environments, etc. Such a fact poses great challenge to machine-learnet models, which implicitly assume that the data distribution is covered by the training data distribution. To resolve this generalization problem, unsupervised domain adaption (UDA), which aims to adapt learned model to the test domain given its unlaheled data [13, 15], has been an active sub-field in machine learning.
Typically, UDA assumes that the "source" domain, which the model trained, and the "target" domain, in which the model is deployed, discrepant but sufficiently related. Concretely, Ben- David et al. [1]. Zhao et al. [75] show that the generalization error of UDA is bounded by the discrepancy of the marginal or conditional distributions between domains. Namely, the effectiveness of UDA may degrade along with the increase in domain discrepancy. Take one popular algorithm, self-training [38, 48, 49], for example. Self-tranh trainiiinngg adapts the source model by progressively labeling the unlabeled target data (i.e., pseudo-labels) and using them to fine-tune the model [2, 28, 30, 31, 41, 44, 77, 78]. Self-training works if the pseudo-labels are accurate (as it essentially becomes supervised learning). but is vulnerable if they are not, which occurs when there exists a large domain gap [35].
To address this issue, several recent works investigate gradual domain adaption (GDA) [12, 25, 35, 71], in which beyond the source and target domains, the model can access additional unlabeled data from the intermediate domains that shift gradually from the source the target. By adapting
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Direct UDA
Source
Target
Gradual shift
Indexed Domain Sequence
IDOL
Unindexed Intermediate Data
P On An
1905
1900s-2000s
2013
Figure 1: Gradual domain adaption (GDA) without indexed intermediate domains. In this setting, one is provided with labeled source, unlabeled target, and additional unlabeled intermediate data that have not been grouped and indexed into a domain sequence. Our approach intermediate domain labeler (IDOL) can successfully discover the domain sequence, which can then be leveraged by a GDA algorithm to achieve a higher target accuracy than direct unsupervised domain adaptation (UDA). Images are from the Portraits dataset [14].
the model along the sequence of intermediate domains - i.e., from the ones close to the source to the ones close to the target - the large domain gap between source and target chipped away by multiple sub-adaptation problems (between consecutive intermediate domains) whose domain gaps are smaller. Namely, every time the model moves a step closer to the target, instead of taking a huge jump that can significantly decrease the performance (see Figure for an illustration). GDA makes sense, as real-world data change gradually more often than abruptly [11, 60, 66). The recent work by Kumar al. [35] further demonstrates the strength of GDA both empirically and theoretically.
One potential drawback of GDA is the need for well-defined sequence of intermediate domains. That is, prior to adaptation, the additional unlabeled data must be grouped into multiple domains and indexed with intermediate domain labels that reflect the underlying data distribution shift from the source to the target. Such information, however, may not be available directly. Existing methods usually leverage side information like time tags [25, 35, 71] to define the sequence, which may be sub-optimal. In some applications, even the side information may not be accessible (e.g., due to privacy concerns), greatly limiting the applicability GDA.
In this paper, we therefore study GDA in the extreme case the additional unlabeled data are neither grouped nor indexed. Specifically, we investigate how to discover the "domain sequence" from data, such that it can be used to drive GDA. We propose a two-stage coarse-to-fin fine framework named Intermediate DOmain Labeler (IDOL). In the first stage, IDOL labels each intermediate data instance with a course score that reflects how close it is to the source or target. We study several methods that estimate the distance from data instance to a domain. We find that a progressively trained domain discriminator which starts with source vs. target data but gradually adds data close to either of them into training- - performs the best, as better captures the underlying data manifold.
In the second stage, IDOL then builds upon the coarse scores to group data into domains by further considering the discriminative (e.g., classification) knowledge the model aims to preserve from the source to the target. Since the additional unlabeled data are fully unlabeled, we take greedy approach identify the next intermediate domain (i.e., group of data) that can best preserve the discriminative knowledge in the current intermediate domain. Concretely, we employ self-training [38] along the domain sequence discovered so far to provide pseudo-labels for the current domain, and propose a novel cycle-consistency loss to discover the next domain, such that the model adapted to the next domain can "adapted back" to the current domain and predict the same pseudo-labeia The output of IDOL is sequence of intermediate domains that can be used by any GDA algorithms [25, 35, 71].
We validated IDOL on two data sets studied in [35], including Rotated MNIST [36] and Portraits over years [14]. IDOL can successfully discover the domain sequence that leads to comparable GDA performance to using the pre-defined sequence (i.e., by side information). More importantly, IDOL is compatible with pre-defined sequences by treating them as the coarse sequences to further improve upon them. We also investigate IDOL in scenarios where the additional unlabeled data may contain outliers and demonstrate IDOL's 's effectiveness even in such challenging cases. To our knowledge, our work is the first tackle GDA without grouped and indexed intermediate domains. The success of IDOL opens up broader application scenarios that GDA can contribute to.

---

2 Background: gradual domain adaptation (GDA)
2.1 Setup of UDA and GDA
We consider an unsupervised domain adaptation (UDA) problem, in which a classifier is provided with labeled data from the source domain S = =[22f ) and unlabeled data from the target domain T { | Both and T are sampled IID from the underlying joint distributions of the source Ps and target Py, respectively. A source model 0s is typically produced by training S directly. The goal of UDA is to learn a target model 0y such that it can perform well on the target, measured by a loss [ Pr). We focus the standard UDA setup that assumes label shifts [6].
Gradual domain adaptation (GDA), on top of UDA, assumes the existence of sequence of domains Po, Pi,... PM-1- PM. where Po and PM are the source domain and target domain, respec- tively, along with M - intermediate domains ,,,..., PM--- The data distributions of domains assumed change gradually from the source the target along the sequence. One can access
unlabeled data Um = {a. T from each of the intermediate domain, forming a sequence of unlabeled data U, MM-1- We denote the union of them by U (a [t4| Ji=1- For brevity,
assume that each Um is disjoint from the others but the size [m is the same Vm E (1, -
2.2 GDA with pre-defined domain sequences using self-training [35]
We summarize the theoretical analysis of GDA using self-training by Kumar et al. [35] as follows.
Self-training for UDA. Self-training [38] takes a pre-trained (source) model 0 and the target unla- beled data T as input. Denote by = f(æ; 0) the predicted logits self-training first applies f(:; 0) to target data points æ E T. sharpens the predictions by argmax into pseudo-labels, and then updates the current model 0 by minimizing the loss l (e.g., cross entropy) w.r.t the pseudo-labels,
ITI Σ (I(x,;;;) sharpen([(a,; 0)),
1
0, - ST = arg min
(I)
O' IT|
where 0r denotes the resulting target model and ST(O, 7) denotes the self-training process. A baseline use of self-training for UDA is to set 0 as the source model 0s. However, when the domain gap between and T is large, Os cannot produce accurate pseudo-labes for effective self teafnntrainnn.
Gradual self-training for GDA. In the GDA setup, self-training is performed along the domain sequence, from the current model Om (updated using Um already) to the next one 0m+1 using Um
0m+1 = (0,
starting from the source model 0g Os. By doing so, the model gradually adapts from S to T through the sequence Ui .,.....,. Um to produce the final target model 0T.
0 0 ST(Os, (u,,-..,uu)).
As the domain gap between consecutive domains is smaller than between S and T, the pseudo-labels will be more accurate in each step. As a result, self-training in GDA can lead lower target error than in UDA. We note that the domain sequences in [35] are pre-defined using side information.
Theoretical analysis. Kumar et al. [35] make three assumptions for theoretical analysis:
Separation: for every domain Pm. the data are separable such that there exists an R-bounded (linear) classifier 0n E O with |0 2 S. R that achieves low loss of C(0 m, Pm). Gradual shift: for the domain sequence Po...., P., with no label shifts, the maximum per-class Wasserstein-infiit distance p( (m, Pm+1) between consecutive domains should be < p R
Bounded data: finite samples X from each domain P., are bounded, i.e., Ex-P [|] < B².
Under these assumptions of GDA, if the source model Gg has a low loss L(0,, Po) on Po. then with a probability 8, the resulting target model On through gradual self-training has
L( PM)) & MM (C(Bu, Po) + 4BR + 22 2M/5
2
where 1 1-pR
(4)
Viu
That is, the target error is controlled by the intermediate domain shift, If we can sample infinite data (i.e., [4]] + oc) and the source classifier is perfect (i.e., L(0,, Po) = 0), with a small distance p(P. m, Pm+1) between consecutive domains, gradual self-trainin achieves zero target error.
3

---

3 GDA without indexed intermediate domains
3.1 Setup and motivation
In this paper, we study the case of GDA where the additional unlabeled data U are not readily divided into domain sequences -1. -.a...- In other words, we are provided with a single, gigantic unlabled set that has a wide range of support from the source to the target domain. One naive way to leverage it is to adapt from S to U and then to TT which however leads to much larger target error than applying self-training along the properly indexed intermediate domains [35]. We attribute this to the large p(S, or p(U, T) according to Equation 4. To take advantage of UA we must separate it into intermediate domains such that p(Pm, Pm+1) is small enough for every m E ,O,..., - 1}.
3.2 Overview of our approach: intermediate domain labeler (IDOL)
Given the source data S, the target data Un = =T, and the unlabeled intermediate data Uf = { [44] we propose the intermediate domain labeler (IDOL). whose goal is to sort intermediate data instances in ll and chunk them into M domains 141 _ (with equal sizes) for GDA to succeed.
Ji=1'
Taking gradual self- self-training in Equation 3 as an example, IDOL aims to solve the following problem
min [(OT,PT). S.t. 07 = SSTOs (UI,...,UMi) and (M....,MM-i)) (D,
1); (5)
where the target model O is produced by applying gradual self-training to the domain sequence discovered by IDOL. IDOL accepts the number of intermediate domains M - 1 as a hyper-parameter
Solving Equation 5 is hard, as evaluating any output sequence needs running through the entire gradual self-training process, not to mention that we have no labeled target data to estimate L Pr). In this paper, we propose solve Equation 5 approximately via a coarse-to- to-fiee procedure.
The coarse stage. We aim to give each a E l score qi, such that it tells æ's position in between the source and target. Specifically, a higher/lower qi indicates that x is closer to the source/target domain. With these scores, we can already obtain a coarse domain sequence by sorting them in the descending order and dividing them into M - 1 chunks.
The fine stage. Instead of creating the domain sequence U1, _MM-1 by looking at the score of each individual o. E M, we further consider how data grouped into the same domain can collectively preserve the discriminative knowledge from the previous domains - after all, the goal of UDA or GDA is to pass the discriminative knowledge from the labeled source domain to the target domain. To this end, we propose a novel cycle-consistency loss to refine the coarse scores progressively.
3.3 The coarse stage: assigning domain scores
In this stage, we assign each o.d E l a score qi, such that higher/lower qi indicates that a!! is closer to the source/target domain. We discuss some design choices. For brevity, we omit the superscript " Confidence scores by the classifier (x::)) We first investigate scoring each intermediate data instance by the confidence score of the source model 0s. i.e., q. = max f fxx; 0s) across classes. This is inspired by outlier detection [42] and the common practice of self-training, which selects only data with sufficiently large confidence scores to fine-tune upon [35, 38). Here we employ an advanced version, which is use the latest 0 (i.e., the model adapted to the current intermediate domain) to select the next intermediate domain. That is, every time we select the highest confidence instances from the remainnig ones in U to M-1-m
O. Overall, we can give data selected in the m-th round M- E [0, 1]. One drawback of this method is the blindness to
(m E {1,---,,M, .M- 1}) score the target domain T. Specifically, we find that using confidence scores tends to select easily classified examples, which do not necessarily follow the gradual shift from the source to the target.
Manifold distance with the source features. To model the flow from the source to the target, we argue that it is crucial consider both sides as references. We thus extract features of S, T. and 2 using the source model Os and apply manifold learning algorithm (e.g., UMAP [50]) to discover the
In [35], the authors investigated sampling U1 -1M-1 from U4 uniformly random with replacement.
4

---

underlying manifold. Denote by ((x) the dimension-reduced features of æ after UMAP, we compute
Tin,Ter ((.)) Ol the ratio of its distance to the nearest point in the T and S as the score q m (=((
Domain discriminator. We investigate another idea to emphasize the roles of the source and target, which is to train domain discriminator [13]. Concretely, we construct binary classifier g x) using deep neural networks, which is trained to separate the source data S (class: 1) and the target data T (class: 0). We use binary-cross entropy loss optimize the learnable parameter &
(()) =
Σ x ES
log(o(g(2>; ())))
1 IT|
Σ x #TET
log(1 - :$)))
(6)
S
where a is the sigmoid function. We can then assign score to H, E by q: g({;; Φ).
Progressive training for the domain discriminator. The domain discriminator g(: ()) compared to the manifold distance, better contrasts the source and target domains but does not leverage any of the data æ E U. That is, it might give a faithful score to an example æ: that is very close to S or TT, but for other examples that far away and hence out-of-domain from both ends, the score by g(-; Φ) becomes less reliable, not to mention that neural networks are usually poorly calibrated for these examples [21, 42]. We therefore propose progressively augment the source and target data with æ E U that has either a fairly high or low g(æ; Φ), and fine-tune the domain discriminator [24]
g(æ; Φ). We perform this process for K rounds, and every time include 2K new examples into the source and target. By doing so, the domain discriminator g( Φ) is updated with data from 2 and thus can provide more accurate scores to distinguish the remaining examples into the source or target side, More specifically, let N denote [24], we repeat the following steps for k from 1 to K:
1. Train g(,, Φ) using S and T, based on the loss in Equation 6.
2. Predict 9; = g(x;, Vz; E U.
3. Rank all 9: in the descending order.
4. The data with the .N
largest 9; scores form a new intermediate domain for the source side (their =). We remove them from 2/ and add them into S.
2K
qi is set to 2K 2K-k)
5. The data with the NX smallest è, scores form new intermediate domain for the target side (their
qi is set to 2K ). We remove them from 44 and add them into T.
2K if Overall, we give a data instance x E l selected in the k-th round (k E {1, K}) a score it is added into the source side, or 2K
2K
& if it is added into the target side.
3.4 The fine stage: cycle-consistency for refinement
The domain scores developed in subsection 3.3 can already give each individual example æ U rough position of which intermediate domain it belongs to. However, for GDA to succeed, we must consider intermediate data in a collective manner. That is, each intermediate domain should preserve sufficient discriminative (e.g., classification) knowledge from the previous one, such that after M rounds of adaptation through GDA, the resulting o, in Equation 5 will be able perform well on T. This is reminiscent of some recent claims in UDA [46, 75]: matching only the marginal distributions between domains (i.e.. blind to the class labels) may lead to sub-optimal adaptation results.
But how can we measure the umount of discriminative knowledge preserved by an intermediate domain (or in the target domain) if we do not have its labeled data? To seek for the answer, we revisit Equation and consider the case M 1. If the two domains Po and P1 are sufficiently close and the initial model Oo is well-trained, the resulting model 02 after self-training will perform well. Reversely, we can treat such a well-performing 0 as the well-trained initial model, and apply self- self-training again this time from P1 back to Po. The resulting model 0/ according to the same rationale, should perform well on Pa- In other words, 8 and 6 should predict similarly on data sampled from Po- The similarity between 0 and O, in terms of their predictions therefore can be used proxy to measure how much discriminative knowledge 0, after adapted to PT, preserves. It is worth noting that measuring the similarity between 0, and Og requires no labeled data from Po P..
Now let us consider a more general case that M is not limited to 1. Let us set 0g = Os. where 0s has been trained with the labeled source data S till convergence. Based on the concept mentioned
5

---

Forward Adaption
pm
Intermediate Data
Umm
Cycle-consistency on Predictions
Current Domain
Next Domain
Um
U
0'
m
Backward Adaption
Figure 2: cyclsscon (cf. Equation 8).
above, propose novel learning framework discover the domain sequences, 8b), ((x; s.t., 0 - ST(O ((0))),)., MM ST(O,, (4,)).,((,))
arg min in. MM- E [f(f(a:
,40)),
where 2o is the source data S with labels removed, and Um = T is the target data. Og is a gradually self-trained model from Oo that undergoes cycle U1 -,,.. UMa -,... Ug. In other words, Equation aims to find the intermediate domain sequences such that performing gradual self-training along in a forward and then backward fashion leads sycle-consistency [76].
Optimization. Solving Equation 7 is still not trivial due to its combinatorial nature. We therefore propose solve greedily, starting with discovering U1, and then U2, and then so Concretely, we decompose Equation 7 into a series of sub-probleme as illustrated Figure 2, each is defined as
Σ (f((;; O')) sharpen (f; Om))),
arg minl um Cle
mell
s.t.,
0 ( ml Umm);
0m+1 sT(mm.:M++1)
where Om is the model already adapted the current domain Um, and Un U \ remaining data that have not been selected by the m intermediate domains so far. That is, each sub- aub-problem discovers only the next intermediate domain Um+11 via cycle-consistency.
- M is the
Let ml and Umm { selecting Um++ um is equivalent to selecting a binary indicator vector q E {0, 10 in which q; = 1 means that æ; E U4,m is included into Um This representation turns the self-training step 0m+1 = ST(Om. Um++) in Equation into
ST q) arg min
Σ a (ff(x;;) sfarpeh(faaaa;aaammmmm 0m)))
(9)
BEA N
We choose to solve Equation approximately by relaxing the binary vector E (0, 1 into a real vector q E R-N, which fits perfectly into the meta-reweighting framework [29, 55]. Concretely, meta- reweighting treats q E R as learnable differentiable parameters associated to training examples (in our case, the data in (.) Meta-rewwiihting for Equation 8 can be implemented via the following six steps for multiple iterations.
1. Detach: +- Om.
a EiEu, 9s ((f(=; 0), sharpen(f(a;; 0m)))
2. Forward: (g))
12
00
3. Detach: 0 O(g),
oEjeum {( {(f(z;; 0())) shwrpen (zzz ('))
n1
Backward: 0(q) +- 0(g)
Umm
(
i
0(g)), sharpen sfar 0m)))
5. Update: <-
x
M Update: q, f- max (0, ).).
dq
'Here we use the same notation as the coarse scores defined in subsection 3.3, since later we will initialize these values indeed by the coarse scores.

---

The aforementioned updating rule gives 2; higher value of qi if it helps preserve the discriminative knowledge. After obtaining the updated q E RR,, we then sort it in the descending order and select the top M examples to be Um+1 (i.e., every intermediate domain has an equal size).
Discussion. The way we relax the binary-valued vector q E {0, 1 by real values is related to the linear programming relaxation of an integer programming problem. It has been widely applied, for example, in discovering sub- domains within a dataset [16]. Theoretically, we should constrain each element of q to be within [0. 1]. Empirically, we found that even without the clipping operation to upper-bound qu (i.e., just performing max {0, qi J), the values in qi do not explode and the algorithm is quite stable: we see a negligible difference of using upper-bounded qi or not in the resulting accuracy of GDA. This is also consistent with the common practice of using meta-reweighring [29, 55].
Initialization. We initialize qi by the coarse scores defined in subsection 3.3. This is for one important reason: Equation 8 searches for the next intermediate domain Un++1 only based on the current intermediate domain Un. Thus, it possible that Um-1 will include some data points that help preserve the knowledge in U.... but are distributionally deviated from the gradual transition between Un, and the target domain T. The scores defined in subsection 3.3 thus serve as the guidance; we can also view Equation 8 as a refinmenn step of the coarse scores. As will be seen in section 4, the qualities of the initialization and the refinement steps both play important roles in IDOL's success.
4 Experiment
4.1 Setup
We mainly study two benchmark datasets used in [35]. Rotated MNIST [36] is a popular syn- thetic dataset to study distributional shifts. It gradually rotates the original 50, 000 images in the MNIST [37] dataset with [0, 60] degrees uniformly, where [o, 5)/55, 55)/ 60) degrees are for the source/intermeeiitccfaageee domains, respectively. However, since original images have no ground truths of the rotation, the rotation annotations are not expected to be perfect. Portraits dataset [14] is real-world gender classification dataset of a collection of American high school seniors from the year 1905 to 2013. Naturally, the portrait styles change along years (e.g., lip curvature [14], fashion styles). The images are first sorted by the years and split. For both datasets, each domain contains 2000 images, and 1, 000 images are reserved for both the source and target domains for validation.
We follow the setup in [35]: each model is convolutional neural network trained for 20 epochs for each domain consequently (including training on the source data), using Adam optimizer [32] with learning rate 0.001, batch size 32, and weight decay 0.02. We use this optimizer the default if not specified.
of IDOL include K- = 2M rounds for progressive training and 30 epochs of refinement per step (with mini-batch 128), where M 19 for the Rotated MNIST and M = 7 for the Portraits. More details are in the supplementary material.
To study how the intermediate domain sequences affect gradual self-training, we consider baselines including "Source only" and "UDA" that directly self-trains on the target data (T) and all the intermediate data (Lf). We compare different domain sequences including Pre-defined indexes, e.g., rotations and years. We further consider the challenging setup that all intermediate data are unindexed. Random sequences are by dividing the unlabeled data into M -1 domains randomly, with self-training on the target in the end. For our IDOL, different domain scores introduced in subsection 3.3 are adopted for the coarse domain sequences, and we optionally apply cycle-consistency refinement to make them fine-grained.
4.2
Main study: comparison with pre-defined domain sequences
The main results on Rotated MNIST and Portraits are provided Table We first verify that without any domain sequences, UDA (on target and intermediate data) and GDA (on random sequences) do not perform well, though they slightly improve over the source model. On the contrary, training with pre-defined indexes that arguably guide the gradual adaptation significantly performs better.
IDOL is competitive to pre-defined sequenees. IDOL can produce meaningful domain sequences (see Figure 3), given only the intermediate unlubeled data without any order. The domain discrimi- nator with progressive training performs the best and comparable to pre-defined sequences. The confidence scores by the classifier are not enough to capture domain shifts.
7

---

Pre-defined
Source
Target
Target Source
IDOL
(a) Rotated MNIST (class "1")
(b) Portraits (class "female")
Figure 3: Random samples (the most representative class) from pre-defined indexes or IDOL. sequences.
Table Gradual self- self- training on Rotated MNIST and Portraits. Bottom section: IDOL with domain scores.
Coarse scores
Indexed?
Adaptation
Refined?
Rotated MNIST
Portraits
Source only UDA (T) UDA (T + 4)
31.9111.7
1..3.1 76.9+2.11
None
X
33.0=2.2
38.0++1.6
78.9++3.0
x
8..9+1.2
83.8+0.88
Pre-defined [35]
/
GDA
93.3:12.3
85.81:0.4
Random
X
GDA
x /
57.5.12.7
39.5++2.0
82,5112.2
81.1+-1.8
Classifier confidence
45.5
79.3
Manifold distance
72.413.1
81.910.8
Domain discriminator
x
GDA
82.1
82.3+009
85.7+2.7
83,410.88
Progressive domain discriminator
87.512.0
85
Refinement helps and coarse initialization matters. We ob-
serve that is crucial to have high- high-quality coarse sequences as
Target Acc. (96)
the initialization; the fine indexes by cycle-consistency refine-
ment could further improve the coarse sequences, validating its
effectiveness. We note that the Rotated MNIST dataset treats the original MNIST data as O-degree rotation and artificially rotates the data given rotation index. However, for data in the original MNIST dataset, there already exist variations in terms of rotations. This can be seen in Figure 3: in the first row of O-degree rotation based on the pre-defined indexes, the examples of digit 1 do have slight rotations. Interestingly, in this situation, IDOL could potentially capture the true rotation of each example further improve GDA.
Years
Step
(a) Target Acc. of gradual ST.
1955
1915
1950
Analysis. To understand why refinement helps, we take closer look the Portraits dataset that is naturally sorted by years. Are years the best factor to construct the domain sequence? We compare with the refined sequence by IDOL. In Figure we monitor the target accuracy of each step of adaptation and find that the accuracy fluctuates by using years. Interestingly, learning with IDOL stably and gradually improves the target accuracy and ends up with higher accuracy, validating that refinement with cycle-consistency indeed produces better sequences. Specifically, the IDOL sequence has a reasonable 0.727 correlation with years but they are not perfectly aligned. Several factors such as fashions, hairstyles, eyeglasses, and lip curvatures may not perfectly align with years [14]. Even within the same year, individuals could have variations these factors well. We thus hypothesize that IDOL discover the domain sequence that reflects smoother transition of these factors.
1940
DDL
Pre-defined
Sequence
(b) Years vs. IDOL sequence.
Figure 4: Portraits with year indexes IDOL domain sequence.

---

We note that the intermediate domains constructed by IDOL (domain scores with or without refine- ment) may not match the target "class" distributions. We monitor the number of examples of class e in each domain Um. i.e., [Mm.el. and compute /ー >m min- [| 1.0, i.e., class-balanced. We observe fine-grained indexes are indeed more class-balanced (coarse vs fine-grained); 1.33 vs 1.23 on Rotated MNIST and 1.27 vs 1.25 on Portraits.
max. 1immel which ideally should be
4.3 Case study: learning with partial or outlier information
If partial or outlier information of the intermediate domains is given, is IDOL still helpful? We examine the question by three experiments. First, we consider unsorted domains: the data grouped into M - domains according to the pre-defined indexes but the domains are unordered. We find that if we sort the domains with the mean domain scores |
Umm Eu, 9: then can perfectly recover the pre-defined order. Second, we investigate if we have fewer, coarsely separated pre-defined domains: we double the size of /- We find that on Rotated MNIST (w/ pre-defined indexes), the accuracy degrades by 11% IDOL can recover the accuracy since IDOL, outputs a sorted sequence of data points, from which we can construct more, fine-grained intermediate domains. Third, we investigate the addition of outlier domains, in which the starting/end intermediate domains do not match the source/target domains exactly. For example, on Rotated MNIST, the intermediate domains are expanded to cover [-30, 90] degrees. GDA on such domain sequence with outlier domains degrades to 77.0 + 2.0 accuracy. Refinement can still improve it to 81.3 + 1.4.
4.4 Case study: learning with low-quality intermediate data or indexes
In practice, the intermediate unlabeled data or the pre-defined do- main indexes for them might not be perfect. We consider several realistic cases to show that method can make GDA more robust to the quality of the intermediate data or the domain sequences. First, we investigate fewer intermediate data: Equation 4 implies that the target error increases if the intermediate data are too sparse. We repeat the experiments in subsection 4.2 but with only 30% interme- diate data. Second, we consider noisy indexes: building upon 30% clean indexes, we further add more data with noisy, random indexes.
12k clean noisy
m
IDOL, nolsy
20
25
#Noisy-index data (k)
(a) Rotated MNIST.
Figure 5 summarizes the results. We first notice that with only 30% intermediate data, even if the indexes are clean, the accuracy decreases drastically by 14% for MNIST and by 5% for Portraits. For MNIST, adding more noisily-indexed data actually decreases the accuracy. Portraits gains some improvements with the noisily- indexed data, suggesting that the task could benefit from learning with intermediate data (even without accurate indexes). One ad- vantage of IDOL is that we can annotate domain indexes for any unindexed data. On both datasets, IDOL stably improves with more data available, significantly outperforming that without IDOL.
Target m
4k clean noisy IDOL, noisy
4
#Nolsy-index data
(b) Portraits.
Figure 5: Different numbers of noisily-indexed intermediate data.
4.5
Case study: CIFARIO-STL UDA with additional, open-domain unlabeled data
We examine IDOL on a CIFARIO-1O. 10-to-STL. [5, 34] UDA task which is known to be challenging due to the small data size [4]. The STL dataset has an additional unlabeled set, whose data sub-sampled from ImageNet [9]. We use this set as the intermediate data, which contains unknown or outlier domains and classes. Here, we do not aim compete with the state of the arts on the CIFARIO- 10-ttoSTTL UDA task, but study a more general application scenario for IDOL.
Table CIFARIO-STL
Method
Target Acc.
Source only
76.6.10.44
UDA (T) (Ir 10 10-4 4)
69.4-00.4
UDA (T) (Ir = 10-")
7..
UDA (T +26)
61.11008
GDA w/ conf.
0..
GDA w/ IDOL
78.1+0.44
To leverage these data, we use M 3 and filter out 80% of them using the classifier confidence. We first train ResNet-20 [23] source model. In Table 2, we observe that direct UDA using self-training on STL. or unlabeled data are inferior to the source model, even if the learning rate is carefully tuned. GDA with IDOL achieves the highest accuracy, demonstrating its robustness and wide applicability.

---

5 Related Work
Domain adaption. UDA has been studied extensively [1. 53, 63], and many different approaches have been proposed such as minimizing domain divergence [13, 20, 45, 61. 62, 65], cross-domain [39, 57, 58, 61] and domain-invariant features [18, 26, 54, 67-69, 73]. Self-training recently emerges as a simple yet effective approach [2, 28, 30, 31, 41, 64, 77]. It uses the source model to provide pseudo-labels for unlabeled target data, and approaches UDA via supervised learning in the target domain [38, 48, 49]. Theoretical analyses for applying self-training in UDA are provided in [3, 70].
Gradual domain adaption. Many UDA works show the benefits of gradually bridging the domain gap. With features as input, several work [8, 15, 19] construct Grassmannian manifolds between the source and the target. For deep models, progressive DA [27] proposes create the synthetic intermediate domain with an image-to-imant
translation network. Other works [7, 17] instead generate intermediate data with jointly-trained trained generators. Na et al. [51] augments the intermediate data with Mixup [74]. Unlike all the above work that focus on building synthetic intermediate domains given only the source and target data, gradual domain adaption proposed in [12, 25, 35, 71] studies how to leverage extra real intermediate data with pre-defined domain indexes.
Learning with cycle consistency. The concept of cycle consistency has been successfully applied in many machine learning tasks. On a high level, cycle supervision enforces that the translation to the target should be able be translated back to match the source. For instance, back translation [22, 59] is popular technique to perform unsupervised neural translation in natural language processing. Many computer vision tasks such as image generation [56], image segmentation[40)) style trans- fer [76], etc., can also be improved by matching the structural outputs with cyclic signals. We propose a novel approach that uses cycle consistency to discover intermediate domains for gradual adaption.
Discovering feature subspaces. We further discuss the connections of our work to existing efforts about subspace learning. One powerful technique is subspace clustering [10, 33, 52] that partitions data into many subspaces (c.g.. classes) unsupervisedly. However, it may not be applied to GDA di- rectly since it groups with discriminative features which might not capture domain shifts. Specifically for UDA, some works aim to discover sub-spaces in the source/target domains [16, 24, 43, 47, 72]. The assumption is that the datasets might not be collected from one environment but generated from many different distributions, rerfouuugg it as multi-domain adaption problem. We note that, the sub-domains discovered in these methods are fundamentally different from the domain sequence discover. The sub-domains are assumed have dedicated adaption mapping the target domain. The domain sequences in GDA are for constructing a path from the source to the target via intermediate data. How to link these learning techniques to GDA will be interesting future work.
6 Conclusion
Gradual domain adaptation leverages intermediate data to bridge the domain gap between the source and target domains. However, it relies on a strong premise prior knowledge of domain indexes to sort the intermediate data into a domain sequence. We propose the IDOL algorithm that can produce comparable even better domain sequences without pre-defined indexes. With simple progressivell-traine trained domain discriminator, it matches the performance of using pre-defined indexes, and further improves with the proposed cycle-consistency refinement. Essentially, IDOL is a useful tool to augment any GDA algorithms with high-quality domain sequences given unindexed data.
Acknowledgments and funding transparency statement
This research is partially supported by NSF IIS 2107077 and the OSU GI Development funds. We are thankful for the generous support of the computational resources by the Ohio Supercomputer Center.
References
[1] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79(1-2):51 175, 2010. 1, 10
[2] Minmin Chen, Kilian Q Weinberger, and John Blitzer. Co- fo-training for domain adaptation. In NeurIPS, 2.11. 1, 10
10

---

[3] Yining Chen, Colin Wei, Ananya Kumar, and Tengyu Ma. Self-training avoids using spurious features under domain shift. In NewrlPS, 2020. 10
[4] Safa Cicek and Stefano Soatto. Unsupervised domain adaptation via regularized conditional alignment. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 1416-1425, 2019. 9
[5] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215 223. JMLR Workshop and Conference Proceedings, 2011."
[6] Remi Tachet des Combes, Han Zhao, Yu-Xiang Wang, and Geoff Gordon, Domain adaptation with conditional distribution matching and generalized lahel shift. In NeurlPS, 2020. 3
[7] Shuhao Cui, Shuhui Wang, Junbao Zhuo, Chi Su, Qingming Huang, and Qi Tian. Gradually vanishing bridge for adversarial domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12455-12464, 2020. 10
[8] Zhen Cui, Wen Li, Dong Xu, Shiguang Shan, Xilin Chen, and Xuelong Li. Flowing riemannian manifold: Domain adaptation by shifting covariance. IEEE transactions cybernetics, 44(12):2264-2773 2014. 10
[9] Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Fei-Fei. Imagenet: large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248- 255.. Teee, 2009. 9
[10] Ehsan Elhamifar and René Vidal. Sparse subspace clustering: Algorithm, theory, and applications. JEEE transactions pattern analysis and machine intelligence, 25(11)22765-7711 2013. 10
[11] Ali Farshchian, Juan A Gallego, Joseph P Cohen, Yoshua Bengio. Lee E Miller, and Sara A Solla. Adversarial domain adaptation for stable brain-machine interfaces. arXiv preprint arXiv: /rxiv/8800.00455 2018.
[12] Michael Gadermayr, Dennis Eschweiler, Barbara Mara Klinkhammer, Peter Boor, and Dorit Merhof. Gradual domain adaptation for segmenting whole slide images showing pathological variability. In International Conference on Image and Signal Processing, pages 461-464466 1661449...0 Springer, 2018. 1, 10
[13] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François I Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMLR, 17(1): 2096-2030, 2016. 5, 10
[14] Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexei A Efros. A century of portraits: A visual historical record of american high school yearbooks. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1-7, 2015. 7,8
[15] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kemnel for unsupervised domain adaptation, 2012 IEEE conference on computer vision and pattern recognition, pages 2066-2073. IEEE,
2012. 1, 10
[16] Boqing Gong, Kristen Grauman, and Fei Sha. Reshaping visual datasets for domain adaptation. Advances in Neural Information Processing Systems, 2013. 7, 10
[17] Rui Gong, Wen Li, Yuhua Chen, and Luc Van Gool. Dlow: Domain flow for adaptation and generalization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2477- 2486, 2019. 10
[18] Ian Goodfellow, Jcan Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672-2680. 2014. 10
[19] Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. Domain adaptation for object recognition: An unsupervised approach. In 2011 international conference on computer vision, pages 9999-1006 IEEE,
2011. 10
[20] Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. Covariate shift by kernel mean matching. Datuset shift in machine learning, 3(4):5, 2009. 10
[21] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference Machine Learning, pages 1321-1333 PMLR, 2017. 5

---

[22] Di He. Yingce Xia. Tao Qin. Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma. Dual learning for machine translation. Advances in neural information processing systems, 29:820-828, 2016. 10
[23] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016. 9, 16
[24] Judy Hoffman, Brian Kulis, Trevor Darrell, and Kate Saenko. Discovering latent domains for multisource domain adaptation. In European Conference on Computer Vision, pages 702-715. Springer, 2012. 10
[25] Judy Hoffmann Trevor Darrell, and Kate Saenko. Continuous manifold based adaptation for evolving visual domains. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 867-874. 2014. 1. 2, 10
[26] Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. ICML, 2018. 10
[27] Han-Kai Hsu, Chun-Han Yao, Yi-Hsuan Tsai, Wei-Chih Hung, Hung- Yu Tseng, Maneesh Singh, and Ming-Hsuann Yang. Progressive domain adaptation for object detection. JEEE Winter Conference on Applications of Computer Vision (WACV), 2020. 10
[28] Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. Cross-domain weakly- supervised object detection through progressive domain adaptation. In CVPR, 2018. 1, 10
[29] Muhammad Abdullah Jamal, Matthew Brown, Ming- Yang. Liqiang Wang, and Boqing Gong. Rethinking class-balanced methods for long-tailed visual recognition from a domain adaptation perspective. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7610-
7619, 2020. 6, 15
[30] Mehran Khodabandeh, Arash Vahdat, Mani Ranjbar, and William G Macready. A robust learning approach to domain adaptive object detection. In ICCV, 2019. 1, 10
[31] Seunghyeon Kim, Jachoon Choi, Tackyung Kim, and Changick Kim. Self-training and adversarial background regularization for unsupervised domain adaptive one-stage object detection. In ICCV, 2019. 1.
10
[32] Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In ICLR, 2015. 7, 16
[33] Hans-Peter Kriegel, Peer Kröger, and Arthur Zimek. Clustering high-dimensional data: A survey on subspace clustering, baaeer clustering, and correlation clustering. Acm transactions on knowledge discovery from data (tkdd). 3(1):1558, 2009. 10
[34] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 9
[35] Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self- self-training for gradual domain adaptation. In International Conference on Machine Learning, pages 5468-544 5479. PMLR, 2020. 1, 3, 4, 7, 8, 10, 16,
17,
[36] Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra, and Yoshua Bengio. An empirical evaluation of deep architectures on problems with many factors of variation. In Proceedings of the 24th international conference on Machine learning, pages 473-480, 2007. 2,7
[37] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-baset learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278-2324, 1998. 7
[38] Dong- Lec. Pscudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, 2013. 1, 2, 3, 4, 10
[39] Seungmin Lee, Dongwan Kim, Namil Kim, and Seong- -yynn Jeong. Drop to adapt: Learning discriminative features for unsupervised domain adaptation. In ICCV, 2019. 10
[40] Yunsheng Li, Lu Yuan, and Nuno Vasconcelos. Bidirectional learning for domain adaptation of semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6936-6945, 2019. 10
[41] Jian Liang, Ran Hc, Zhenan Sun, and Tieniu Tan. Distant supervised centroid shift: A simple and efficient approach to visual domain adaptation. In CVPR, 2019. 1, 10
[42] Shiyu Liang, Yixuan Li, and Rayadurgam Stikant, Enhancing the reliability of out-of distribbuui image detection in neural networks. arXiv preprint arXiv: 02690, 2017. 4,
12

---

[43] Ziwei Liu,, Zhongqi Miao, Xingang Pan, Xiaohang Zhan, Dahua Lin, Stella X Yo, and Boqing Gong, Open compound domain adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 12406-12415, 2020. 10
[44] Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip S Yu. Transfer feature learning with joint distribution adaptation. In Proceedings of the IEEE international conference on computer vision, pages 2200-2200 2013.
[45] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. Learning transferable features with deep adaptation networks. arXiv preprint arXiv: 027911 2015, 10
[46] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. arXiv preprint (i.:::/005..060000 2017. 5
[47] Massimiliano Mancini, Lorenzo Porzi, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci. Boosting domain adaptation by discovering latent domains. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition. pages 377 3771-3780, 2018. 10
[48] David McClosky, Eugene Charniak, and Mark Johnson. Effective self self-training for parsing. ACL, 2006.
I, 10
[49] David McClosky, Eugene Charniak, and Mark Johnson. Reranking and self-training for parser adaptation. In ACL, 2006. 1, 10
[50] Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv: 1802.0336, 2018. 4, 17
[51] Jaemin Na, Heechul Jung, HyungJin Chang, and Wonjun Hwang. Fixbi: Bridging domain spaces for unsupervised domain adaptation. arXiv preprint 20XXv:2011.0922000 2020. 10
[52] Andrew Ng. Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 14:849 856, 2001. 10
[53] Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 22(2):19-2100 2010. 10
[54] Zhangyi Pei, Zhangjie Can, Mingsheng Long, and Jianmin Wang. Multi-adversarial domain adaptation. In Thirty-Second AAAI Conference on Artificial Intelligence 2018. 10
[55] Mengye Ren, Wenyuan Zeng, Bin Yang. and Raquel Urtasun. Leamnngg reweight examples for robust deep learning. In International Conference on Machine Learning, pages 4334 4343. PMLR, 2018. 6, 7, 15, 17
[56] Paolo Russo, Fabio Carlucci, Tatiana Tommasi, and Barbara Caputo. From source to target and back: symmetric bi-directional adaptive gan. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 8099-8108, 2018. 10
[57] Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Sacnko. Adversarial dropout regularization. arXiv preprint arXiv: 171.01555, 2017. 10
[58] Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR, 2018. 10
[59] Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06700 2015. 10
[60] Tegiyot Singh Sethi and Mehmed Kantardzic. On the reliable detection of concept drift from streaming unlabeled data. Expert Systems with Applications, 82:77-99, 2017. 2
[61] Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. A dirt-t approach to unsupervised domain adaptation. In ICLR, 2018 10
[62] Masashi Sugiyama, Matthias Krauledat, and Robert MÅäller. Covariate shift adaptation by impor- tance weighted cross validation. JMLR, 8(May):985- 1005, 2007. 10
[63] Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In AAAI. 2016. 10
[64] Qingyi Tao, Hao Yang, and Jianfei Cai. Zero annotannottionn object detection with web knowledge transfer. In ECCV, 2018. 10
13

---

[65] Eric Tzeng, Judy Hoffmann Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint urXi::1412.347, 2014. 10
[66] Alexander Vergara, Shankar Vembu, Tuba Ayhan, Margaret A Ryan, Margie I. Homer, and Ramón Huerta. Chemical gas sensor drift compensation using classifier ensembles. Sensors and Actuators B: Chemical, 166:320-329, 2012.
[67] Riccardo Volpi, Pietro Morerio, Silvio Savarese, and Vittorio Murino. Adversarial feature augmentation unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5495 5504, 2018. 10
[68] Ximei Wang, Liang Weirui Ye, Mingsheng Long, and Jianmin Wang. Transferable attention domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 5345-5352, 2019.
[69] Yan Wang, Chen Xiangyu, You Yurong. Erran Li Li, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger, and Chao Wei-Lun. Train in germany, test in the usa: Making 3d object detectors generalize. In CVPR, 2020. 10
[70] Colin Wei, Kendrick Shen, Yining Chen, and Tengyu Ma. Theoretical analysis of self-training with deep networks unlabeled data. In ICLR, 2021. 10
[71] Markus Wulfmeeirr Alex Bewley, and Ingmar Posner. Incremental adversarial domain adaptation for continually changing environments. In 2018 IEEE International conference robotics and automation (ICRA), pages 4489 4495. IEEE, 2018.
10
[72] Caiming Xiong. Scott McCloskey, Shao Shao-Hang Hsieh, and Jason Corso. Latent domains modeling for visual domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014. 10
[73] Minghao Xu, Jian Zhang, Bingbing Ni, Teng Li, Chengjic Wang, Qi Tian, and Wenjun Zhang. Adversarial domain adaptation with domain mixup. In AAAI, 2020.
[74] Hongyi Zhang. Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. Paz. mixup: Beyond empirical risk minimization. ICLR, 2018. 10
[75] Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representa- tions for domain adaptation. In International Conference on Machine Learning, pages 7523-75557577 -5322, PMLR, 2019. 5,5
[76] Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages
2017. 6, 10
[77] Yang Zou, Zhiding Yu. BVK Vijaya Kumar, and Jinsong Wang. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In ECCV, 2018. 1, 10
[78] Yang Zou, Zhiding Yu, Xiaofeng Liu, BVK Kumar, and Jinsong Wang. Confidence regularized self training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2019.
14


Reviews:
[{"review_id": "iKGlWZfCRdq", "reviewer": "Reviewer_9fF3", "summary": "The paper propose, IDOL, a method for gradual domain adaptation. The novelty of IDOL is that it can learn to discover the sequence of intermediate domains.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: \n---\nThis paper touches a novel problem, that is doing gradual domain adaptation with unlabeled data without pre-defined intermediate domains. The solution is technically novel. Specifically, the author leverages the idea of cycle-consistency and meta-learning to learn the domain index. I find this methodology is original and may benefit to other tasks. \n\nQuality & Clarity: \n---\nFor the method section, in general, the authors do a good job. However I feel there is some details about the specific design missing. I understand, at high level, the author is using a meta-learning style approach. From my understanding, meta-learning has a large computation overhead. I am curious about the algorithm complexity of the proposed method as well as experimental comparison of the learning efficiency between the fine stage algorithm against other coarse stage baselines. In appendix algorithm 3, the author uses mini-batch to update the \\theta(q) for T times. I am concern the efficiency of the meta-learning when T goes large.\nAnother place I feel not well explained is why set q = max{q,0} instead of q = clip(q,min=0,max=1). The author says during the optimization they relax the binary vector q to be real value. I am not sure whether making a bounded value q to be unbounded will causes instability during the optimization. Can author give any theoretical verification about such relaxation?\n\nFor the experiment section, the paper delivers nice results. Table 1 shows the proposed coarse-steps are better than random indexing, while with refinement, IDOL can reach similar performance as the pre-defined index. I like the analysis in section 4.2. The visualization of per-step accuracy, relationships between learned and pre-defined indices, and class imbalance analysis provide a nice understanding of the IDOL. That being said, I still have a few question:\n1. In figure 4, the author shows the target accuracy stably improves after each adaptation using IDOL sequences. Figure 4(b) shows IDOL sequences do not have much correlation with years. I am curious about what information does IDOL sequences captures. \n2. The IDOL seems have worse performance than pre-defined (rotation) index in the Rotated MNIST dataset. I am assuming in this task, rotation is the best domain index. Can authors analysis how close the learned IDOL align with the rotation? If IDOL is not perfectly align with rotations, is there any potential reasons that hinders it learn the best domain index?\n\nSignificance:\n---\nThe paper propose a new problem of doing GDA with undefined intermediate domains. The problem is important. The result presented is significant and could be a nice baseline for the future research. \n\nIn summary, I am voting for the acceptance due to its novelty and nice experiments. My score is not final and will depend on how the author address my questions. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The author gives a brief discussion of the limitations of this work in supplementary. Maybe the author can talk about something in more detail like when graduate domain adaptation is not applicable or when learning domain indexing is really hard. For example, in health care applications, some domain changes are caused by binary (categorical) attributes such as races, genders. Can we still apply gradual DA?", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "L0B2-ggMyV", "reviewer": "Reviewer_B4p7", "summary": "This paper is addressing the gradual domain adaptation (GDA) problem without the access of intermediate domain indices. This paper proposes a coarse-to-fine method to predict the indices of the intermediate domain sequence. The results are shown to be comparative to pre-defined methods. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. claim concerns:\na) One of the main concerns is the definition of “intermediate domain” in this paper. It highly depends on how the intermediate domains are generated. This paper only briefly mentions that the assumption is that the intermediate domains are under the data distribution between source and target domains. It would be great to have more analysis about: 1) What if the distributions of intermediate domains are not gradually/smoothly underlying between source and target domains, but closer to on side instead? 2) What if the intermediate domains are under very different distributions? With the above analyses, it will be easier to clarify in what situation the proposed method works.\n\n2. technical detail concerns:\na) The descriptions of “progress training for the domain discriminator” are not clear (lines 169-180). It would be clearer if there are pseudo codes or algorithms like the one in Sec. 3.4 (line 220-225).\n\n3. experiment concerns:\na) In Table 1, does “Refined?” mean the fine stage as described in Sec. 3.4? If so, why does this refinement also work for the “pre-defined” case? It would be great to analyze the pre-defined case with and without refinement (i.e., visualized Table 1 using a similar way as Figure 4).\nb) Sec. 4.5 partially answers the question about “What if the intermediate domains are under very different distributions?”. From Table 2, it shows that the proposed method does not work well for this case. It would be better to have more analysis and explanation here. \nMoreover, the result of “UDA (T)” is weird. The numbers in Table 2 are not consistent with those in RCA [4], which shows a 81.6 accuracy as “UDA (T)” and also outperforms “Source only”.\n\n4. other concerns:\na) The proposed method is like an additional step to generate predicted domain indices, so it would be better to have time analysis to show the computation cost. The paper only briefly mentions that using short descriptions. It would be better to have more details (e.g., a Table with numbers).\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, but it would be better to discuss more limitations as mentioned above.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "DO8BEQU_RH4", "reviewer": "Reviewer_New7", "summary": "This paper is concerned with gradual domain adaptation, when we have access to data from intermediate distributions between source and target and use them to gradually adapt the model through self-training. To address the challenge of lack of intermediate data annotated and grouped based on their distance to the source domain, the paper proposes IDOL (Intermediate Domain Labeler) to index unlabeled available data that presumably covers the gap between source and target  domains based on their distance to source.\n\nIDOL consists of two steps:\n1. A progressively trained domain discriminator is used to assign a score to each example. (a higher/lower score indicates that the example is closer to the source/target domain).\n2. A model based on a cycle-consistency loss is applied to refine the coarse scores progressively. The main idea is to group examples into intermediate domains such that enough discriminative information is preserved that we could approximately regain the accuracy on the source domain by consuming the intermediate domains in the reverse direction.. \n\n**Empirical results**:\nExperiments on  Rotated MNIST and Portraits indicate that with IDOL we can achieve comparable performance on gradual domain adaptation as when we have the ground truth (pre-defined) domain sequences. Furthermore the experiments indicate that applying IDOL on pre-defined domain sequences, treating them as the outputs from step 1, and refining them through step 2 can lead to domain sequences that are better suited for gradual domain adaptation. In general, it seems that applying IDOL in many cases with partial or noisy annotation of the intermediated domain sequences can be beneficial.  Finally, IDOL is also compared with UDA on CIFAR10 to STL tasks, indicating superiority of both (GDA + confidence) and (GDA + IDOL) to UDA in this setting.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**: The proposed idea builds up on top of  existing works on gradual domain adaptation that study how leveraging samples from intermediate distributions can help in the unsupervised domain adaptation process. The idea is novel and theoretically motivated. It is a two step process of forming a sequence of domains given a set of unlabeled data points.  \n\n**Quality and Clarity**: I find the paper well written and very readable. The experiments and analysis provided in the paper validate the success of the IDOL in labeling intermediate sequence both qualitatively and quantitatively, but in a limited scope.\n\n**Significance**: Based on the empirical results provided in the paper, the method seems to lead to significant improvements over the baselines it is compared with. However, both the baselines and the benchmarks are not selected generously. Even though the experiments are limited to a few small datasets, I can understand if this is mainly due to lack of benchmarks that would be applicable for the purpose of this study. There might be some other applications for the proposed approach to make sequences of domains  from a set of unlabeled examples. E.g., for analytical studies.\n\n**Question for the authors:**\n- Have you looked into the variations in the domain sequences formed in different trials? and maybe different factors that would affect them? e.g. number of intermediate samples? nature of the shift in distribution? (e.g., comparing rotated MNIST with the Face dataset).\n\n**Main weakness(es)**:\n- Limited scope of the experiments (both in terms of baselines and datasets).\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I don't think the authors have adequately addressed the limitations of the proposed method (or about negative societal impact ).\nI think there might be some room for discussion there. E.g., could this gradual process results in some unintentional biases in the models? \n\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers gradual domain adaptation without access to intermediate sequence of distributions between source and target. This is a relatively new problem and a very important one. Author discussions helped address the reviewer concerns and therefore I suggest the paper to be accepted. I ask the authors to please include the additional information discussed in the rebuttal period in the camera ready.  I also strongly recommend the authors add more explanations about the limitations of the current experiment setting (GDA) and discuss the possible ways to actively address these issues. Although the paper is interesting and the results are a good first step to tackle the problem of gradual domain adaptation, the experiment are limited, limitations of the proposed method are not investigated and the baselines are simple. Therefore, the paper does not qualify for a spotlight.
Decision: Accept (Poster)