Lifelong Domain Adaptation via Consolidated Internal Distribution
Year: 2021

Abstract:
We develop an algorithm to address unsupervised domain adaptation (UDA) in continual learning (CL) settings. The goal is to update a model continually to learn distributional shifts across sequentially arriving tasks with unlabeled data while retaining the knowledge about the past learned tasks. Existing UDA algorithms address the challenge of domain shift, but they require simultaneous access to the datasets of the source and the target domains. On the other hand, existing works on CL can handle tasks with labeled data. Our solution is based on consolidating the learned internal distribution for improved model generalization on new domains and benefiting from experience replay to overcome catastrophic forgetting.

heading: 1 Introduction
text: Deep neural networks relax the need to for manual feature engineering by learning to generate discriminative features in an end-to-end blind training procedure [1, 2]. Despite significant advances in deep learning, however, robust generalization of deep neural network on unseen data is still a primary challenge when domain shift exists between the training and the testing data [3, 4]. Domain shift is a natural challenge in continual learning (CL) [5, 6] where the goal is to learn adaptively and autonomously when the underlying input distribution drifts over extended time periods. Distributional shifts over time usually lead to performance degradation of trained models which in turn necessitates model retraining to acquire knowledge about new distributions. Current CL algorithms mainly consider tasks, i.e., domains, with fully labeled datasets. Hence, these algorithms require annotating massive training datasets for new observed domains. Persistent manual data annotation, however, is practically prohibitive because of being an economically costly and time-consuming process [7].To relax this constraint, our goal is to develop an algorithm for continual adaptation of a model for tackling the challenge of domain shift in a CL setting using solely unannotated datasets.
Unsupervised Domain adaptation (UDA) is a highly relevant learning setting to our problem of interest. The goal in UDA is to train a model for a target domain with unannotated data by transferring knowledge from a related source domain in which annotated data is accessible [3]. A primary group of UDA algorithms map the training data points for both domains into a shared latent embedding space and align the distributions of the source and the target domains in that space [8]. Hence, a source-trained classifier that receives its input from the shared embedding space would generalize on the target domain as well. The domain alignment procedure has been implemented either using generative adversarial learning [9, 10, 11, 12, 13] or by directly minimizing the distance between the two distributions [14, 15, 16, 17, 18]. Existing UDA algorithms are not suitable for continual learning because the underlying model can be trained if datasets from both domains are accessible. Moreover, these methods usually consider only a single target domain and a single source domain. Finally, simply updating the underlying model to generalize in the current encountered domain is not sufficient. Because upon updating the model, the network likely would forget the past learned domains as the result of retroactive interference, referred as the phenomenon of catastrophic forgetting [19, 20] in the
35th Conference on Neural Information Processing Systems (NeurIPS 2021)
CL literature. Consequently, we also need to tackle catastrophic forgetting in our unexplored learning setting. Our method can be considered as an improvement over existing UDA and CL methods.
Contributions: we develop an algorithm for lifelong unsupervised adaptation of a model on new domains using solely unannotated data. Our idea is based on consolidating the internally learned distribution that encodes the learned knowledge by the model when the initial source domain is learned. We use this multimodal distribution to update the model such that the learned distributions for all subsequent unannotated domains are coupled. To overcome catastrophic forgetting, we store important representative samples for all tasks and replay them back when the model is updated. We provide a theoretical analysis to demonstrate that our method mitigates catastrophic forgetting and also leads to improved generalization. We validate our method using standard UDA benchmarks.

heading: 2 Background and Related Work
text: Lifelong UDA setting lies on the intersection of both UDA and CL learning settings.
Unsupervised domain adaptation: an effective approach for domain alignment in UDA is to use a probability distance to measure distributional discrepancy and then train an encoder to minimize the cross-domain distance at its output as a latent shared embedding space [21, 22, 23, 24, 25, 26]. The Wasserstein distance (WD) [27] is a suitable metric for this purpose due to possessing nonvanishing gradients. This property is beneficial because deep learning optimization problems are primarily solved using gradient-based optimization techniques [23]. In this work, we rely on the sliced Wasserstein distance (SWD) [24] variant of the Wasserstein distance because it can be computed efficiently using a closed form solution via empirical samples of the data distribution.
Continual learning: Existing CL methods primarily use either model regularization or experience replay to tackle catastrophic forgetting. The idea of model regularization [20, 5] is to identify the network weights that contribute significantly in retaining knowledge about a past learned task and then consolidate them when the model is updated to learn the subsequent tasks. Alternatively, the model can be expanded progressively to learn the new tasks using added weights [28]. We rely on experience replay which is implemented using pseudo-rehearsal [19]. The idea is to identify important training data points that contribute significantly to learning a task and store them in a memory buffer [29]. These samples then would be replayed along with the current task’s data to represent the past learned distributions [30, 31] to retain the acquire knowledge about the past tasks. The major challenge is to identify the important data points. We rely on the consolidated internal distribution to identify the important data points. To mitigate the sample selection challenge, some CL algorithms rely on generative experience replay, where the memory buffer is replaced with the ability of generating synthetic pseudo-samples that are similar to the samples of the past learned tasks [31, 32, 33, 34].
Domain Adaptation in Continual Learning Settings: Existing works in this direction are highly limited. Wulfmeier et al. [35] study addressing gradual shifts in changing environments. Bobu et al. [36] and Wu et al. [37] explore addressing the problem of domain shift in continual learning settings but both works assume that datasets for all learned tasks are observable at each time-step. This assumption limits practicality of these works. Porav et al. [38] explore a setting similar to our setting but the proposed method relies on image translation which makes the work highly domain specific. In our work, we relax these limitations and develop a more general algorithm.

heading: 3 Problem Statement
text: Consider a classification problem in a source domain S with a labeled training dataset DS = (X0,Y0), where X0 = [x01, . . . ,x 0 N ] ∈ X ⊂ Rd×N , Y0 = [y01 , ...,y0N ] ∈ Y ⊂ Rk×N , and the training data points are drawn independently from an unknown source distribution, i.e., x0i ∼ p0(x). Given a deep neural network fθ with learnable weights θ, we can solve for an optimal model using the standard empirical risk minimization (ERM): θ̂0 = argminθ ∑ i L(fθ(x0i ),y0i ), where L(·) is a suitable loss function. If the training dataset is large enough and the network structure is complex enough, the ERM optimal model will generalize well on unseen data points, drawn from p0(x) [39]. The challenge in CL is that the input distribution may be non-stationary. Hence, the testing samples may be drawn from drifted distributions. The resulting distributional gap will lead to poor generalization during the testing time. Our goal is to update the source-trained model fθ̂0 continually using solely unlabeled data to avoid poor generalization without forgetting past experiences. To model this process,
we consider a set of sequentially arriving target domains T t, t = 1 . . . T, with unlabeled datasets DtT = (Xt), where Xt ∈ Rd×Mt , xti ∼ pt(x), and ∀t1, t2 : pt1 ̸= pt2 (see Figure 1). Since these domains are unlabeled using ERM is implausible. As stated, common UDA methods cannot address CL settings.
To address challenges of “domain shift” and “catastrophic forgetting”, we consider that the base network fθ(·) can be decomposed into a deep encoder ϕv(·) : X → Z ⊂ Rp and a classifier subnetwork hw(·) : Z → Y , i.e., fθ = hw ◦ ϕv, where θ = (w,v). Our method is based on consolidating the internal distribution that is formed in the embedding space Z . We assume that as the result of initial training on the source domain, the source classes become separable in Z . If at each time-steps, we update the model such that the internal distribution remains stable, i.e., the distance between the distributions ϕ(p0(x0)) and ϕ(pt(xt)) is minimized, then the model continues to generalize well on the target domains, despite initially being trained with the source domain labeled dataset. This strategy has been used extensively by the existing UDA algorithms but we are constrained with the accessibility of DS in CL, i.e., the term ϕ(p0(x0)) cannot be computed.
4 Proposed Solution
Our solution is based on consolidating the intermediate distribution that is learned in the discriminative embedding space to retain model generalizability. Upon learning the initial source domain, the encoder is trained to map the input source distribution into a multi-modal distribution pJ(z) in the embedding space. Each mode in this distribution corresponds to one of the input classes and the training data points that belong to a particular class, are mapped to the same cluster (see Figure 1, middle). The internally learned multimodal distribution is represented empirically by the source domain data representations {(ϕv(x0i ),y0i )}Ni=1. We use a GMM p0J(z) with k component as the parametric model to estimate the internal distribution:
p0J(z) = k∑ j=1 α0jN (z|µ0j ,Σ0j ), (1)
where α0j denotes the mixture weights, and µ 0 j
and Σ0j denote the mean and co-variance matrices for the components. Computing these parameters is generally a challenging iterative procedure. But since the labels are accessible, we can compute the parameters for each mode independently. Let S0j denote the support set for the mode j, i.e., S0j = {(x0i ,y0i ) ∈ DS | argmaxy0i = j}. The MAP estimates for the GMM parameters would be:
α̂0j = |S0j | N
, µ̂j = ∑
(x0i ,y 0 i )∈S 0 j
1
|S0j | ϕv(x
0 i ), Σ̂ 0 j = ∑ (x0i ,y 0 i )∈S 0 j 1 |S0j | ( ϕv(x 0 i )− µ̂0j )⊤( ϕv(x 0 i )− µ̂0j ) . (2)
We can update the estimates in Eq. 2 for t > 1, to be described later. Let p̂tJ(z) denote the estimated GMM at the time-step t. Our idea is to consolidate this distribution to retain model generalization. In the absence of source data, we adapt the model such that the encoder always aligns the target distributions with this distribution in the embedding space. We implement this process by drawing random samples from the GMM and build a labeled pseudo-dataset: D̂tP = (Z t P , Ŷ t
P), where ZtP = [z t 1, . . . ,z t Np ] ∈ Rp×Np , ŶP = [ŷp,t1 , ..., ŷ p,t Np
] ∈ Rk×Np , and zti ∼ p̂tJ(z). We determine the labels according to the model predictions. We include only those samples for which the model prediction confidence-level is more than a preset threshold τ to exclude outlier samples. To retain the model generalization power, we solve the following problem at time t:
min v,w N∑ i=1 L ( hw(z p i ), ŷ p,t i ) + λD ( ϕv(pt(Xt)), p̂ t J(Z t P) ) , (3)
where D(·, ·) denotes a probability discrepancy measure and λ is a trade-off parameter. The first term enforces the classifier to preserve its generalizability on the internal distribution. The second term enforces alignment of the target domain distribution with the internally learned distribution in the embedding space. We use the SWD metric [27] for the term D(·, ·). Upon updating the model, we can also update the internal distribution p̂tJ(z) using the pseudo-dataset samples.
Solving Eq. (3) would help the model to generalize well on the new domain T t. However, catastrophic forgetting will remain untackled because the encoder subnetwork is updated unconditionally to minimize the domain discrepancy term in Eq. 3.To tackle catastrophic forgetting, we rely on experience replay. When a task is learned, we select a small subset of the training data points as representative samples, to be stored in a memory buffer. These samples are replayed during model adaptation to mitigate catastrophic forgetting. Possible selection strategies includes mean of features (MoF) [40], ring buffer [41], and reservoir sampling [42]. Existence of the internal distribution makes MoF a natural choice. Upon model update at time-step t and then updating the GMM, we can compute the distance of representations for all the data points of T t from their corresponding cluster mean, i.e., dtj,l = ∥µtj − ϕ(xtl)∥22,∀xtl s.t. ŷtl = argmax fθ̂t(x t l) = j. Given a memory budget of Nb samples, we pick the Mb = Nb/k samples per class that have the closest distance to the mean to form the buffer-stored dataset Dtb = D t−1 b ∪ (Xtb, Ŷ tb ). These samples are the most informative samples to estimate the internal distributions. We update Eq. (3) to tackle catastrophic forgetting as follows:
min v,w N∑ i=1 L ( hw(z p i ), ŷ p,t i ) + Nb∑ i=1 L ( hw(ϕv(x b i )), ŷ b i ) + λD ( ϕv(pt(Xt)), p̂ t J(Z t P) ) + λD ( ϕv(pt(X t b)), p̂ t J(Z t P) ) .
(4)
Adding the second supervised term in Eq. (4) helps to mitigate catastrophic forgetting. The fourth term helps to consolidate the internal distribution across all the previous tasks. Our proposed solution for lifelong domain adaptation, named Lifelong Domain Adaptation Using Consolidated Internal Distribution (LDAuCID), is presented and visualized in Algorithm 1 and Figure 1, respectively.
5 Theoretical Analysis Algorithm 1 LDAuCID (λ, τ,Nb) 1: Source Training: 2: Input: source labeled dataset DS = (X0,Y0) 3: θ̂0 = (ŵ0, v̂0) = argminθ ∑ i L(fθ(x 0 i ),y 0 i )
4: Internal Distribution Estimation: 5: Use Eq. (2) and estimate α0j ,µ 0 j , and Σ 0 j 6: Memory Buffer Initialization 7: D0b = (X0b , Ŷ 0b )
Pick the Nb/k samples with the least dtj,l = ∥µtj − ϕ(xtl)∥22, ŷ0,ib =
argmax f θ̂t (x 0,Nb b )
8: Continual Unsupervised Domain Adaptation: 9: for t = 1, . . . , T do
10: Input: target unlabeled dataset DtT = (Xt) 11: Pseudo-Dataset Generation: 12: D̂tP = (ZtP , Ŷ t
P) = 13: ([zp,t1 , . . . , z p,t N ], [ŷ p,t 1 , . . . , ŷ p,t N ]), where:
zp,ti ∼ p̂ t−1 J (z), 1 ≤ i ≤ Np and ŷp,ti = argmaxj{hŵt(z p,t i )} if with
confidence τ : maxj{hŵt(z p,t i )} > τ
14: for itr = 1, . . . , ITR do 15: draw data batches from DtT and D̂P 16: Update the model by solving Eq. (4) 17: end for 18: Internal Distribution Estimate Update: 19: Use Eq. (2) similar to step 5 above. 20: Memory Buffer Update 21: Dtb = Dt−1b ∪ (X t b , Ŷ t b ), where (X t b , Ŷ t b ) is computed similar to step 7 above. 22: end for
We provide an analysis within a standard PAClearning framework [39]. Consider the set of classifier sub-networks H = {hw(·)|hw(·) : Z → Rk,v ∈ RV } as the hypothesis class within PAC-learning. Let e0 denote the expected error on the source domain, et denote the expected error on the target domains, and eJt denote the expected error on the pseudo-dataset for a given h ∈ H. Also, let p̂0(x) = 1N ∑N n=1 δ(ϕv(x s n)) and p̂t(x) =
1 Mt ∑Mt m=1 δ(ϕv(x t m)) denote the empirical source and target distributions in the embedding space. We provide the following theorem.
Theorem 1: Consider LDAuCID algorithm at learning time-step t = T . Then for all the previously learned tasks t < T , the following holds:
et ≤ eJT−1 +W (ϕ(p̂t), p̂tJ) + T−2∑ s=t W (p̂sJ , p̂ s+1 J )
+ e(w∗) + √( 2 log( 1
ξ )/ζ )(√ 1 Mt + √ 1 Np
+
√ 1
Nb
) ,
(5)
where e(w∗) denotes expected error for the best joint-trained optimal model in the hypothesis
space, i.e., the model trained as w∗ = argminw ec(w) = argminw{et(h) + etJ(h)}, W (·, ·) denotes the WD distance, and ζ and ξ are two constants.
Proof: Included in the Appendix.
Theorem 1 explains why LDAuCID algorithm is effective. Major terms in the right-hand side of Eq. 5, as an upperbound for the expected error for each task (domain), are continually minimized by LDAuCID. The first term is minimized because the internal distribution random samples are used to minimize the empirical error term as the first term of Eq. (4). The second term is minimized as the third terms of Eq. (4) when the task distribution is aligned with the empirical internal distribution in the embedding space at time t. The third term which is a summation which models the effect of continual learning. The sth term in this sum is minimized when the task T s is learned and we deliberately update the internal distribution. This additive term grows as more tasks are learned after learning a particular task which can potentially make the upperbound looser, leading to forgetting effects. The fourth term is a constant small term if the tasks are related, i.e., share the same classes. LDAuCID does not minimize this terms. It just suggests that the model should work well for a joint-training scenario in order to perform well in a sequential learning setting. The last term is a constant term which is negligible if we have sufficient numbers of source and target training data points. It also states that storing more samples in the memory buffer leads to better performance, as expected. We conclude if the upperbound in Eq. (5) is sufficiently tight, the domains are relevant, and the hypothesis space is a suitable space to learn the tasks, then adapting the model using LDAuCID can both tackle catastrophic forgetting and also improve model generalization on the target domains.

heading: 6 Empirical Validation
text: Our implemented code is accessible as an Appendix.

heading: 6.1 Data Sets and Tasks
text: We are not aware of any prior work that addresses UDA in a lifelong learning setting that we can directly compare against. For this reason, we use four existing UDA benchmark datasets, adopt them to build sequential UDA tasks, and validate our method on these classic datasets. For comparison purpose, we run our algorithm in the learning setting of the existing UDA methods with one source and one target domains. For fair comparison against these works, we have followed the evaluation protocols that are used by most of the recent classic UDA papers.
Digit recognition tasks: the common MNIST (M), the USPS (U), and the SVHN (S) datasets are used as three domains. Majority of the existing UDA methods report their results on three tasks defined between these domains: M → U , U → M, and S → M tasks. We perform experiments on two S → M → U and S → U → M sequential UDA tasks to cover the three classic UDA tasks. ImageCLEF-DA Dataset: this dataset consists of the 12 shared image classes between the Caltech256 (C), the ILSVRC 2012 (I), and the Pascal VOC 2012 (P) visual recognition datasets. The dataset is fully balanced with 50 images per class, i.e., 600 images for each domain. There are six possible binary UDA tasks. We perform experiments on C → I → P and C → I → P ternary domain tasks. Office-Home Dataset: this a more challenging object recognition dataset which consists of 15, 500images in office and home settings. The images are grouped into 65 classes. There are 4 domains with relatively large gaps: Artistic images (A), Clip Art (C), Product images (P ), and Real-World images (R) which leads to possibility of defining 12 pair-wise binary UDA tasks. We preform experiments on the sequential UDA tasks A → C → P → R and R → P → C → A tasks. Office-Caltech Dataset: this object recognition dataset is built using the 10 shared classes between the Office-31 and Caltech-256 datasets. There are four visual domains: Amazon (A), Caltech (C), DSLR (D), and Webcam (W ) with 2533 images in total. There are 12 definable binary UDA tasks. We perform experiments on the sequential UDA tasks A → C → D → W and W → D → C → A.

heading: 6.2 Network Structure and Evaluation Protocol:
text: We use the VGG16 network as the base model for the digit recognition tasks. Following the literature, we use the Decaf6 features for the Office-Caltech tasks. For the other two datasets, we use the
ResNet-50 network which is pre-trained on the ImageNet dataset as the backbone of the network. To study the model learning dynamics over time, we generate learning curves by reporting the model performance on the testing split of the learned tasks versus the training epochs. Our purpose is simulate continual training during execution time. Providing the learning curves allows to study dynamics of learning. For comparison purpose and after learning a target domain task, we report the average classification rate and the standard deviation on the target domain for five runs for each task. We train the base model using the source labeled data. We report the performance of the model before adaptation as a simple ablation study to demonstrate the effect of domain shift. Then we adapt the model using the target unlabeled data using LDAuCID algorithm and report the performance of LDAuCID on the target domain. For more details on implementation, please refer to the Appendix.

heading: 6.3 Results
text: Learning curves for the above eight sequential UDA tasks are visualized in Figure 2. In these learning curves, the model has been trained for 100 epochs, i.e., a notion for time, for each task and then we have moved forward to learn the subsequent task. We have stored 10 samples per class per domain in the memory buffer for experience replay. We first observe that domain shift leads to an initial performance drop on the target domain in all the eight tasks, however there is a jumpstart in performance. That is, the learning curves for the subsequent target tasks initialize significantly higher than random label assignment. For example, we observe in Figure 2a the initial testing accuracies for the domain M and U are in the +60% range. This initial jumpstart occurs due to similarities between the domains, leading to knowledge transfer from past experiences. The rising behavior of the learning curves after this initial performance, when the model is being trained on T t, demonstrates that LDAuCID effectively improves performance of the source-trained model. This improvement occurs for all the target domain tasks. We note that this the final performance improvements on the Office-Home dataset is less because domain gap between the domains are larger for this dataset.
The second important observation is that catastrophic forgetting has been mitigated quite well. We observe that after moving forward to learn the subsequent domains for all the eight tasks, the model performance on the previously learned tasks is relatively stable and forgetting effect is not dramatic. We note that the forgetting effects for the SVHN dataset in S → U → M and S → M → U tasks are more severe. This is likely because this dataset is more challenging and hence a larger number of samples in the memory buffer may be required. We conclude that LDAuCID algorithm has successfully improved the model generalization while mitigating catastrophic forgetting.
To measure ability of our algorithm, we also compare LDAuCID against existing UDA. We are not aware of any prior work that addresses UDA in a lifelong learning setting. For this reason, we
employ our algorithm in the special case of having only two domains and compare its performance with the classic UDA algorithms. We consider that only one target domain exists and report our performance on this domain. We still address UDA in a sequential learning setting, but is the best we can do for comparison. For comparison, we included the classic UDA methods: GtA [10], DANN [43], ADDA [3], MADA [11], DAN [14], DRCN [21], RevGrad [15], JAN [16], JDDA [44], and UDAwSD [45]. We include results of these works in case the original paper have reported performance on the corresponding dataset. In the Tables, bold font denotes the best performance.
Comparative results for six bi-domain digit recognition tasks are summarized in Table 1. In each table, we have included the Source Only performance as a baseline. It reports the performance of the source-trained model on the target domain without adapting the model. Despite the fact that the classic UDA methods use the full source dataset for joint-training, we observe that LDAuCID outperforms the UDA methods in two of the tasks and its performance on the remaining task is competitive. We observe that ETD is outperforming LDAuCID and on average possesses the highest performance. This is not unexpected because this method is quite close to our method. ETD relies on a version of WD distance for improvement for classic UDA joint-training scheme.
Table 2 presents the results for the ImageCLEF-DA dataset tasks. We note LDAuCID has lead to a significant improvement on this dataset. As mentioned, this dataset is fully balanced both in terms of number of data points per domain and also per class. Note that in our method we rely on the empirical versions of the distributions for domain alignment (check the Appendix for details). A balanced dataset makes the empirical distribution a less biased approximation of the real distribution. We conclude that balanced datasets across the domains can boost performance of our method.
Table 3 summarizes the comparative results for the Office-Home dataset. We observe that CDAN has the best average performance but LDAuCID is still competitive and on four of the tasks outperforms CDAN. As we discussed, domains of this dataset has larger gap compared to the rest of the datasets we used. This could be verified in Figure 2 from the lower jumpstart performance value. As a result, minimizing the second term of the upperbound in Eq. 5 is more challenging for the Office-Home dataset. Performance of CDAN is higher likely because CDAN aligns the two distributions classconditionally. We conclude that a possible direction to improve our method further in the future is to benefit from class-conditional alignment techniques, e.g, pseudo-labeling the target domain data.
From inspecting Tables 1–4, we conclude that LDAuCID performs quite competitively on all the UDA tasks, despite addressing lifelong UDA constrains in which most of the source domain data points are not accessible. Although LDAuCID likely needs be upperbounded by classic UDA algorithms in terms of performance, but we observe it outperforms many of these UDA methods on some of the standard UDA tasks. Although our motivation was to address UDA in a CL setting, our observations conclude LDAuCID can be used for addressing classic UDA. We hope that development of subsequent methods for lifelong UDA learning regime would make a more thorough comparison possible.

heading: 6.4 Analytic and Ablative Studies
text: To study the effect of the algorithm empirically, we have analyzed the geometry of data points in the embedding space. Data geometry approximates the distributions that are learned in the output of the encoder, i.e., empirical version of the internally learned distribution. For this purpose, we take advantage of the UMAP [54] visualization tool to reduce the dimension of the data representations in the embedding space for 2D visualization purpose. In Figure 3, we have visualized the testing splits of the source domain and the two target domains along with a number of randomly drawn samples of the internally learned GMM distribution for the S → M → U digit recognition task. In this figure,
each point represents one data point and each color represents one of the ten digit classes. Each row of Figure 3 represents the data geometry at the embedding space at the end of time-step t. For example, the second row shows data geometry after learning the SVHN and MNIST datasets. This means that if we check the sub-figures vertically for each of the domains, i.e., a column of the figure, we can inspect the effect of learning more tasks versus time. Checking all the columns, we see that upon learning a task, the learned knowledge is retained when the model is adapted in the future, i.e., the next bottom rows, because classes stay relatively separable. This stability suggests that catastrophic forgetting has been continually mitigated when the model is updated. We can also see at the last row, all the domains share the same distribution similar to the internally learned GMM distribution. This observation suggests that our method successfully aligns the distributions to share the same internal distribution. Finally and as an example, if we compare the distribution of the MNIST dataset in the first row versus the second row, we see that as the result of domain adaptation, its distribution in the embedding space becomes more similar to the SVHN dataset (source domain). We conclude that Figure 3 empirically confirms the result that we could deduce analytically from Theorem 1.
Similar to most learning algorithms, setting up the proper values for the hyperparamters is important for optimal performance. We have studied the effect of the values for the hyperparameters λ and τ on the model performance for the binary UDA task S → M in Figure 3e and Figure 3f to suggest how to tune these parameters for practical usage. We have visualized the average performance with a dark curve. The lighter shaded region around the curve denotes the standard deviation. We observe in Figure 3e that the value for the parameter λ does not have a significant effect on performance. This observation is according to expectation because the ERM loss term L(·) in Eq. (3) is relatively small prior to beginning of the alignment process as a result of pre-training on the source domain. In other words, since the dominant term for optimization is the domain-alignment term in Eq. (3), careful finetuning of the trade-off parameter λ is not necessary. We also observe in Figure 3f that when the confidence parameter is τ ≈ 1, the model performance on the target domain improves. This accords with our intuition because potential outlier GMM distribution samples cause label pollution which is a challenge for domain alignment in UDA. From the results presented in this section, we can deduce that our method is effective and the empirical observations support our theoretical analysis.
We have provided additional controlled experiments on the S → M → U task in Table 5 to draw better intuitions about our proposed algorithm. Since we observed that our algorithm is more effective for balanced datasets, we first study the robustness of our algorithm with respect to data imbalance. To this end, we introduced data imbalance for all domains in the S → M → U task by considering that for digit i, only i+110 portion of the data points are available. We have reported the observed performance in Table 5, where each row shows performance for each task at the end of learning period for task t. We observe that despite reduced performance, our method still addresses catastrophic forgetting and leads to improved performance in the target domains for this imbalanced dataset.
Finally, we note another unexplored aspect in our algorithm is the effect of Nb on performance as we set its value arbitrarily. We set Nb = 10 in previous experiments because compared to the sizes of the datasets, Nb would be very small. We studied effect of increasing Nb in Table 5 for Nb = 50 and Nb = 100. As expected, we observe that using a larger buffer size leads to improvements. In practice, the buffer size should be selected to be as large as possible given the hardware storage limitations.

heading: 7 Conclusions
text: We develop an algorithm for domain adaptation in a continual learning setting. The core assumption is that when we train a neural classifier, the input distribution is mapped to an internal distribution, in an embedding space modeled by a neural network hidden layer. Our method is based on consolidating this internally learned distributional such that all learned tasks share a similar distribution in the embedding space. As a result, the model generalizability is retained when new tasks are learned. Additionally, catastrophic forgetting is mitigated using experience replay by storing and then replaying the input samples that are more informative for estimating the internally learned distribution. We used a simple approach to estimate the internal distribution but we foresee that better estimations of the internal distribution can boost the continual learning performance. Future research includes studying the effect the tasks’ order on continual learning performance and extensions to incremental learning setting, where new classes can be learned beyond the initial training phase.

heading: Acknowledgement
text: We thank the anonymous reviewers whose constructive and continual feedback helped to improve the presentation, clarity, and analysis of the proposed work.

Review summaries:
Unsupervised Domain Adaptation (UDA) is studied in a task sequential continual learning setting. The source domain is assumed to have supervised data and several target domains without labels are being sequentially adapted to. An algorithms is engineered for the specific case of supervised classification source datasets which can be neatly clustered in some high-level representation; then, learning on related (target) datasets which can be classified in the same (source) classes can be done sequentially and without the use of labels. Catastrophic forgetting is mitigated using a standard experience replay approach, but also through the assumptions of the conceptual framework, i.e. strong overlap between source and target classes, which is known to be the easier setting in CL works. Empirical validation is done primarily in the using classic domain adaptation benchmarks, with the difference of using a single source domain for multiple target domains in a sequential fashion. The theoretical analysis and ablation study round up the work, including a discussion of algorithm hyper-parameters.

This paper presents a novel method for unsupervised domain adaptation in continual learning setting using only unannotated data. The proposed method models the internally learned distribution of initial annotated source domain as multimodal distribution and updates it using the learned distributions of subsequent unannotated target datasets to couple all the domains together i.e., consolidated internal distribution.   To minimize catastrophic forgetting, the proposed method also performs experience replay during domain adaptation on target dataset using important representative samples of all historical tasks.

This paper proposes a problem formulation at the intersection between continual learning (CL) and unsupervised domain adaptation (UDA). It introduces a method to perform UDA throughout the lifespan of a model, mitigating catastrophic forgetting of previous information. The proposed method is validated on benchmarks proposed ad-hoc for this problem, as well as on standard UDA benchmarks - showing that is can compete with state-of-the-art approaches even in standard settings.