Deja Vu: Continual Model Generalization for Unseen Domains
Authors: Chenxi Liu
Lixu Wang
Lingjuan Lyu
Chen Sun
Xiao Wang
Qi Zhu
Venue: ICLR 2023 poster (2023)
Keywords: Domain Generalization
Domain Adaptation

Abstract: In real-world applications, deep learning models often run in non-stationary environments where the target data distribution continually shifts over time. There have been numerous domain adaptation (DA) methods in both online and offline modes to improve cross-domain adaptation ability. However, these DA methods typically only provide good performance after a long period of adaptation, and perform poorly on new domains before and during adaptation – in what we call the “Unfamiliar Period”, especially when domain shifts happen suddenly and significantly. On the other hand, domain generalization (DG) methods have been proposed to improve the model generalization ability on unadapted domains. However, existing DG works are ineffective for continually changing domains due to severe catastrophic forgetting of learned knowledge. To overcome these limitations of DA and DG in handling the Unfamiliar Period during continual domain shift, we propose RaTP, a framework that focuses on improving models’ target domain generalization (TDG) capability, while also achieving effective target domain adaptation (TDA) capability right after training on certain domains and forgetting alleviation (FA) capability on past domains. RaTP includes a training-free data augmentation module to prepare data for TDG, a novel pseudo-labeling mechanism to provide reliable supervision for TDA, and a prototype contrastive alignment algorithm to align different domains for achieving TDG, TDA and FA. Extensive experiments on Digits, PACS, and DomainNet demonstrate that RaTP significantly outperforms state-of-the-art works from Continual DA, Source-Free DA, Test-Time/Online DA, Single DG, Multiple DG and Unified DA&DG in TDG, and achieves comparable TDA and FA capabilities.

Published as a conference paper at ICLR 2023
DEJA VU: CONTINUAL MODEL GENERALIZATION FOR UNSEEN DOMAINS
Chenxi Liu'; Lixu Wang Lingjuan Lyu"t Chen ,un?, Xiao Wang', Qi Zhu' Worthwestern University, 2Sony AI (chenxiliu2020 Lixuwang2025) nottwwessee..................... edu, [Lingjuan.In) Lv, chen. esony com, (wangxiao, gehu/enorthwesteem.cc
ABSTRACT
In real-world applications, deep learning models often run in non-stationary vironments where the target data distribution continually shifts over time. There have been numerous domain adaptation (DA) methods in both online and offline modes to improve cross-domain adaptation ability. However, these DA meth- ods typically only provide good performance after long period of adaptation, and perform poorly on new domains before and during adaptation in what we call the "Unfamiliar Period", especially when domain shifts happen suddenly and significantly. On the other hand, domain generalization (DG) methods have been proposed improve the model generalization ability on unadapted domains. However, existing DG works are ineffective for continually changing domains due to severe catastrophic forgetting of learned knowledge. To overcome these limitations of DA and DG in handling the Unfamiliar Period during continual do- main shift, we propose RaTP, a framework that focuses on improving models' target domain generalization (TDG) capability, while also achieving effective tar- get domain adaptation (TDA) capability right after training on certain domains and forgetting alleviation (FA) capability past domains. RaTP includes training- free data augmentation module to prepare data for TDG, a novel pseudo-labeling mechanism to provide reliable supervision for TDA, and a prototype contrastive alignment algorithm to align different domains for achieving TDG, TDA and FA. Extensive experiments on Digits, PACS, and DomainNet demonstrate that RaTP significantly outperforms state-of-the-an works from Continual DA, Source-Free DA, Test- Time/Online DA, Single DG, Multiple DG and Unified DA&DG TDG, and achieves comparable TDA and FA capabilities.
INTRODUCTION
A major concern in applying deep learning models to real-world applications is whether they able to deal with environmental changes over time, which present significant challenges with data distribution shifts. When the shift is small, deep learning models may be able to handle because their robustness is often evaluated and improved before deployment. However, when the data dis- tribution shifts significantly for a period of time, in what we call the "Unfamiliar Period", model performance on new scenarios could deteriorate to much lower level. For example, surveillance cameras used for environmental monitoring can work normally with excellent performance on clear days, but have inferior performance or even become "blind" when the weather turns bad or the lighting conditions become poor (Bak al., 2018). As another example, consider conducting lung imaging analysis for corona-viruses, deep learning models may present excellent performance after being trained on a large number of samples for certain variant (c.g., the Alpha variant of COVID- 19), but are difficult to provide accurate and timely analysis for later variants (e.g., the Delta or Omicron variant) and future types of corona-viruses (Singh et al. 2020) when they just appear. In the following, will first diseuss related works, highlight their limitations in addressing the poor model performance during the Unfamiliar Period, and then introduce our approach.
"Equal contributions (ordered alphabetically); 'Corresponding authors. Part of the work done during an internship Sony AI.

---

weather or Published as a conference paper at ICLR 2023
Domain adaptation (DA) methods have been proposed to tackle continual data drifts in dynamic environments in either online or offline mode. For example, Continual DA (Liu et al. 2020 Ros- tami 2 starts from labeled source domain and continually adapts the model to various tar- get domains, while keeping the model performance from degrading significantly on seen domains. However, existing Continual DA works often assume that the source domain can be accessed all the time, which may be difficult to guarantee in practical scenarios, especially considering the possi- ble limitation on memory storage and regulations on privacy or intellectual property. Source-Free DA (Yang et al. 202T; Qu et al., 2022) can overcome this issue and achieve target adaptation with- out the source domain data. In addition, Test-Time or Online DA (Wang et al., [022] Iwasawa & Matsuo, 2021 Panagiotakopoulos et al.) 2022) can improve the target model performance with small training cost; however, the target domain data is only learned once by the model and the per- formance improvement is limited (higher improvement would require large amount of data). With these DA methods, although the model may perform better on the new target domain after sufficient adaptation, its performance the target domain before and during the adaptation process, i.e., in the Unfamiliar Period, is often poor. In cases where the domain shift is sudden and the duration of seeing a new target domain is short, this problem becomes even more severe. In this work, believe that for many applications, it is very important to ensure that the model can also perform reasonably well in the Unfamiliar Period, i.e., before seeing lot of target domain data. For instance in environmental surveillance, having poor performance under crrommon/unnfaillin lighting conditions may cause significant security and safety risks. In the example of lung imag- ing analysis for corona-viruses, being able to quickly provide good performance for detecting new variants critical for the early containment and treatment of the disease.
Domain generalization (DG) methods also solve the learning problem on multiple data domains, especially for cases where the target domain is unavailable or unknown during training. However, existing DG works are typically based on accurate supervision knowledge of the source domain data, whether is drawn from a single domain (Wang et al. 2021c; Li et al. (20T)) or multiple domains (Yao et al., 2022 Zhang et al. (0022,, which may not be achievable in continually changing scenarios. Morcover, when DG is applied in scenarios with continual domain shifts, as it focuses more the target domain, there could be severe catastrophic forgetting of domains that have been learned. There are also some works unifying DA and DG (Ghifary et al. 2016; Motiaan et al.) 2017 Jin al., 2)2T); however they can only be used in standard DA or DG individually, thus still suffering their limitations. Bai et al. (2022) and Nasery et al. 2222)) study the smooth temporal shifts of data distribution, but they cannot handle large domain shifts over time.
Our Approach and Contribution. In this work, we focus on the study of Continual Domain Shift Learning (CDSL) problem. in which the learning model is first trained on labeled source domain and then faces a series of unlabeled target domains that appear continually. Our goal, in particular, is to improve model performance before and during the training stage of each previously- unneen target domain (i.e., in the Unfamiliar Period), while also maintaining good performance in the time peri- ods after the training. Thus, we propose a framework called RaTP that optimizes three objectives: (1) to improve the model generalization performance training, namely the target domain generalization (TDG) performance, (2) to provide good model performance on target domain right after its training, namely the target domain adaptation (TDA) performance, and (3) to maintain good performance on a trained domain after the model is trained with other domains, namely the forgetting alleviation (FA) performance. For improving TDG, RaTP includes a training-free free data augmentation module that is based on Random Mixup, and this module can generate data outside of the current training domain. For TDA, RaTP includes a Top² Pseudo Labeling mechanism that lays more emphasis on samples with a higher possibility of correct classi- fication, which can produce more accurate pseudo labels. Finally, for optimizing the model towards TDG, TDA, and FA at the same time, RaTP includes a Prototype Contrastive Alignment algorithm. Comprehensive experiments and ablation studies on Digits, PACS, and DomainNet demonstrate that RaTP can significantly outperform tate-of-theeart works in TDG, including Continual DA, Source- Free DA, Test-Time/Online DA, Single DG, Multiple DG, and Unified DA&DG. RaTP can also produce comparable performance in TDA and FA these basclines. In summary:
new target domain before and during its
We tackle an important problem in practical scenarios with continual domain shifts, i.e., to im- prove model performance before and during training on new target domain, in what we call the Unfamiliar Period. And we also try to achieve good model performance after training, providing the model with capabilities of target domain adaptation and forgetting alleviation.
2

---

Published as a conference paper at ICLR 2023
Performance
LOIR CrcA
Edliing Mathod
Domain
Source
Labelee Samples
Randhix
Acoptation
Petrector
PCA
Loe LDs L LPCx
Proposed Re
Target
Unlobeled
T2F1
BandMixx
After Spliting
Classiler
PCA
Domain
Samples
Samples
Figure I: Overview of applying our framework RaTP for Continual Domain Shift Learning (CDSL), RaTP first starts with labeled source domain, applies RancMix x on the full set of source data to generate aug- mentation data, and uses simplified version of CA model optimization. Then, for continually arriving target domains, RaTP uses T2PL. to generate pseudo labels all unlaheled samples, applies RandMi x on top subset of these samples based on their softmax confidence, and optimizes the model by PCA.
We propose a novel framework RaTP to achieve our goal. The framework includes training- free data augmentation module that generates more data for improving model's generalization ability, new pseudo-labeling mechanism to provide more accurate labels for domain adaptation, and prototype contrastive alignment algorithm that effectively aligns domains to simultaneously improve the generalization ability and achieve target domain adaptation and forgetting alleviation.
We conducted extensive experiments and comprehensive ablation studies that demonstrate the advantages of our RaTP framework over a number of state-of-the oftth-of- baseline methods.
METHODOLOGY
We first formulate the problem of Continual Domain Shift Learning (CDSL) in Section 2.1 and then introduce the three major modules of our framework TP... Specifically, Section 2.2 presents a Random Mixup data augmentation module RandMix that generates data for improving the tar- get domain generalization (TDG) unadapted domains, which is crucial technique for improv- ing model generalization in the Unfamiliar Period. Section 2.3 presents a Top" Pseudo Labeling approach T2PI. that provides accurate labels achieve target domain adaptation (TDA) for con- tinually arriving target domains. Finally, Section 2.4 presents a Prototype Contrastive Alignment algorithm PCA that optimizes the model to achieve TDG, TDA, and forgetting alleviation (FA) on seen domains. Figure shows I the overall pipeline of RaTP.
2.1 PROBLEM FORMULATION OF CONTINUAL DOMAIN SHIFT LEARNING
We assume that there is a labeled source domain dataset S = { (x., y;)llæ, P JiN, the beginning of CDSL. Here Px and Py are the input and label distributions, respectively, while Ns is the sample quantity. This paper chooses visual recognition as the learning task, in which the number of data classes is K. Then, we consider a sequence of continually arriving target domains T T= (T where T denotes the total number of domains. The 1-th domain contains Ny. data samples æ but no labels y, i.e., T = { x; PT Nyt and all the target domains share the same label space with the source domain. Note that the domain order is randomly determined (including the source domain), which means that the superscript / does not always indicate the same domain. The generalization performance for the 1-th domain T depends on both the previously seen f-1 target domains and the original source domain, i.c., $ = (S. - TT). Considering the possible limitation on memory storage and regulations property (Wang et al. 2021b), like regular continual learning (Dong et al. (022a), we also set an exemplar memory M to store [M] exemplars that are closest to the class centroids for each domain
No at
privacy (Guo et al. 1021) or intellectual
in where [M] < Ny, [M] < Ns. Moreover, the memory size set to be fixed during CDSL. We assume that the model is deep neural network, and without loss of generality, the neural network consists of a feature extractor fo at the bottom and a classifier 9m at the top. For each target domain dataset Tr, in addition the current model f3 gf. its inherited version l6_ go from the last
3

---

Published as a conference paper at ICLR 2023
domain is also stored for later usage. We aim to improve the generalization performance new target domain (TDG) before and during its training. i.c., in its Unfamiliar Period of CDSL, while also achieving effective target domain adaptation (TDA) and forgetting alleviation (FA).
2.2 RANDOM MIXUP AUGMENTATION
At the beginning of CDSL, only labeled source domain is available. Thus we face a pure singe- domain generalization problem where the model is trained on single domain S and tested on the other unseen domains T (each domain in T is possible as the domain order random). We apply data augmentation and design Random Mixup (RandMix x) method to solve this problem. RandMix x is not only helpful improve cross-domain transferability from ficial for remaining model's order agnostic generalization when it encounters low-quality domains. )
T but also bene-
RandMix x relies on Naug simple autoencoders R (Rt Naux where each autoencoder consists of an encoder eg and a decoder de, We want RandMix to be as simple as possible, preferably to work without training. Inspired by (Wang al. 2021c), the encoder eg and the decoder de are imple- mented as convolutional layer and transposed convolutional layer. With such implementation, even if the parameters & and Ç are randomly initialized from a normalized distribution, the autoen- coder can still generate reasonable augmentation data. In order to introduce more randomness, apply AdaIN (Karras et al. 2019) to inject noise into the autoencoder. Specifically, the used AdaIN contains two linear layers (01 1oz and when 14z and løz are fed with certain noisy input drawn from a normalized distribution (n ~ PN(o,,)). they can produce two corresponding noisy outputs with smaller variances. As observed in our experiments (Section 3.I). blindly pushing augmentation data away from the original (Li et al., 2021 Wang et al.) 2021c) possibly hurts the model general- ization ability, and injecting randomness with smaller variance is a better solution. The two noisy outputs are injected to the representations of the autoencoder as multiplicative and additive noises:
R(x)= = de (1es (n) x (n((e(())) + (n.),
(1)
where (-) represents an element-wise normalization layer. RandMix works feeding training data to all autoencoders and mixing the outputs with random weights drawn from a normalized distribution (w [wallwa ~ ) Finally, the mixture is scaled by sigmoid function
(xx) = 1/(1+
Naug
1)
R(x)= =
Woa +
(w, (()))
Ei=0 W;
In mini-batch training, every time there is a new data batch, parameters of autoencoders (E. O. AdaIN-s (Φ1, Φ2), AdaIN noisy inputs (n) and mixup weights (w) are all initialized again. With RandMix, x, we can generate augmentation data with the same labels corresponding to all labeled samples from the source domain. Then, these labeled augmentation samples will work together with the original source data and be fed into the model for training. However, for the following con- tinually arriving target domains, conducting RandMix x on all target data is unreasonable. In CDSL, all target domains T = {T*)T= are unlabeled. While we can apply approaches to produce pseudo labels, the supervision is likely unreliable and inaccurate. Therefore, to avoid error propagation and accumulation, we augment a subset of the target data rather than the full set. Specifically, we determine whether a target sample supposed to be augmented based on its prediction confidence:
R(x),, if max [[(()) 2 Teon x ã == 0,
PX PX
(3)
otherwise
where max[:] k denotes the maximum of a vector with K dimensions, and Feon confidence threshold that is set to 0.8 (sensitivity analysis of Teon is provided in Appendix).
2.3 Top² PSEUDO LABELING
In CDSL, all target domains arrive with only the data but no label. Thus we need pseudo label- ing mechanism to provide reasonably accurate supervision for subsequent optimizations. However, existing pseudo labeling approaches have various limitations. For example, softmax-based pseudo labeling (Lee et al., 2013) produces hard labels for unlabeled samples, but training with such hard labels sacrifices inter claer-class distinguishability within the softmax predictions. To preserve inter-class distinguishability. SHOT (Liang et al., 2020) proposes a clustering- based approach to align unla- beled samples with cluster centroids. However, when the model is applied in new domain, it
4

---

Published as a conference paper at ICLR 2023
is difficult to distinguish different classes since they are nearly overlapped together and the class boundary among them is extremely vague. Besides, such clustering- based methods usually use all training samples to construct centroids for each class. During clustering, for a particular cluster, samples that exactly belong to the same class are often not used well and those misclassified pose negative impact centroid construction. In this case, such negative impact hurts the pseudo la- beling accuracy by resulting in low-quality centroids. To address these issues, we propose a novel mechanism called Top² Pseudo Labeling (T2PL). Specifically, we use the softmax confidence to measure the possibility of correct classification for data samples, and select the top 50% set to con- struct class centroids:
I&
"top"
arg max l 19w, (fo,((ff()))]] ,F -K (UIEZL (2:1)
(4)
ØET
where 9w.k (-) denotes the k-th element of the classifier outputs. rtop controls the size of the selected top set, and if the data is class balanced, top 50% corresponds to Toop =2 Then the top set F is used to construct class centroids by prediction- weighted aggregation representations:
ps: ExSE Yuik (Jo(a:)) fo(m))
Em,er 9u.k (fo(
Subsequently, we can compute the cosine similarity between these centroids and representations of all current unlabeled data, and also select the top half set of samples that have much higher similarity to the centroids than the rest for each class:
Nyr U Poop arg max TET!
fo(z) PR (1) IIp }} FF= }
( {UIEZL (1x k(),)))
I/e
(6)
Then we use F to fit a k-Nearest Neighbor (kNN) classifier and assign the pseudo label as below. Note that KNN can decentralize the risk of misclassification in assigning pseudo labels by a single comparison between class centroids and data samples, and is more suitable for clustering samples that are highly overlapped than center-based approaches:
ÿ KNN (z., Euclidean
K
Flo and subscript Euclidean mean that the kNN works to find Plop closest neighbors from F with the measurement of Euclidean distance. We select the top 5% for each class O'oop = 20) in our implementation (sensitivity analyses of rtop and r'op are provided in the Appendix). With this KNN, T22 L makes better use of unlabeled samples with a relatively high possibility of correct classification than SHOT, and provides more accurate pseudo labels.
where the superscript Nyt
Nyt
2.4 PROTOTYPE CONTRASTIVE ALIGNMENT
Prototype learning (PL) (Pan et al. 2019 Kang et al. 2019 Tanwissth et al. 202 Dubey et al, 2021) has been demonstrated to be effective for solving cross cros--dominn sample variability and sample insufficiency (Wang et al.) 2021a). These two issues are reflected in our problem as the random uncertainty of augmentation data and limited data quantity of seen domains in the exemplar memory M. As a result, in this work, we adopt the idea of PI. and propose a new Prototype Contrastive Alignment (PCA) algorithm align domains together for improving model generalization ability.
First, we need to point out that regular PL is unsuitable to our problem. In regular PL, all samples fed into the model to extract representations, and these representations are aggregated class-by- class for constructing prototypes p' = {pi k= of certain domain T'. To achieve cross-domain alignment, prototypes of different domains for the same classes are aggregated and averaged. Then data samples from different domains are optimized to approach these However, such domain alignment has a problem of adaptivity gap (Dubey et al., 2021). As shown in Figure we assume that there is an optimal prototype location P\ of class h for all domains (SUT), and the objective is to move the source prototype Pÿ to the optimal by training on target domains (T) one-by-one. Meanwhile, we also regard that there are optimal prototypes P¹, for each domain itself, and suppose that the regular PL is applied to align prototypes of all these domains. To illustrate the problem of adaptivity gap, take the alignment between the source S and the first target domain T' as an example, and assume that the distance between Pj and P is larger than the distance between p and p In this case, after the domain alignment, the location of the current prototype
aveerag prototypes.
5

---

Published as a conference paper at ICLR 2023
Stage
P
Aduathits Gap
Alatikity Gop
P
P
city
P
PR
Gap
PR
PL
(a)
(b)
Figure 2: (a): When the model encounters domain whose prototypes (pt) are from the optimal ones (pt), domain alignment with regular prototype learning will enlarge the adaptivity gap. (b): RandMi x may be helpful to reduce the gap in many cases of (p2. pa). (c): However, in some cases RancMMx x may produce low-quality data augmentation (pi". p that negatively affects the regular protolype learning and enlarges the gap. Here, we have Adaptivity Gap > Adaptivity Gap > Adaptivity Gap
Figure 3: Accuracy Ma-
trix. For each domain, TDG:
of green el- ements; TDA: red ments; FA: mean of blue elements.
worse than the source prototype, which means that the adaptivity gap is enlarged. Although we can use RandMix to generate more data (p), p that might be helpful for reducing the adaptivity gap, there is randomness for each use of RandMi and we cannot guarantee such reduction every time (e.g., when the augmentation data is as PK and P in Figure(2]. Therefore, a better prototype construction strategy is needed to provide more representative prototypes.
Inspired by semi-supervised learning work (aito et al. 2019), we view neuron weights of the single linear layer and is placed exactly
classifier i as the prototypes. The classifier I. consists behind the feature extractor fe- Thus the dimension of its neuron weights is the same as the hidden dimension of extracted representations. Prototypes p -s are obtained with a CrossEntropy Loss:
K Σ
log = exp (pk fo( bb)
LCE
IE 2INPX
Σ exp (p' fo(w;) + b..)
where be, be are biases of the linear layer, and we set them as zeros during training, and [()) is an indicator function that if the subscript condition is true, Mrrue = 1, otherwise, IFale = 0.. Compared with regular prototypes, such linear layer prototypes are built on lot more data because RandMix initialized and used for every mini-batch during model training. Moreover, RandMi x has general effect on improving generalization ability, as shown our experiments later in Section 3.2 Thus believe that linear layer prototypes have smaller adaptivity gaps than regular ones.
With linear layer prototypes, we can align domains in a better way. Unlike the alignment on regular prototypes, we do not sum up and average prototypes of different domains. Instead, we simulta- neously maximize the similarities of data samples to prototypes of different domains, although in practice we may have prototypes of just two domains since there is only one old model being stored (Section 2.I.. We also introduce contrastive comparison into the domain alignment process that can enhance the distinguishability of different classes and be beneficial for our pseudo labeling. The contrastive comparison includes negative pairs of particular sample and samples from other classes. this case, our prototype contrastive alignment is formulated as follows:
K
Tg.= exxp (pf fo(x,)) ) + cxp (pk (I--T TTTTTT fo(x,)))
LPCA = "INPI
log
where
A
(9)
A exp (p: fo(x:))
Σ exp (TT fo(x:))
Σ 2FET"
Ig, exp (fo(z)) fo
Furthermore, we also adopt knowledge distillation from the stored old model to the current model for forgetting compensation. In this case, our final optimization objective is shown as follows:
L - - LPCA LDIS, where LDIS DKL. [ (x)) Il at (1o(z)))
(10)
Note that this optimization objective is used in all target domains. As for the souree domain, there no distillation LDSS and the nominator of LPGA only contains the first term, because there is no stored old model in the stage of source training. We denote this simplified optimization loss L'

---

Published as a conference paper at ICLR 2023
3 EXPERIMENTS
Our code is available at htt.:/ggttu...o/Sonnyiiiii comparison baselines introduced below, and more details are in the appendix.
The datasets, experiment settings, and
Datasets. Digits consists of 5 different domains with 10 classes, including MNIST (MT), SVHN (SN), MNIST-M (MM), SYN-D (SD) and USPS (US). PACS contains 4 domains with classes, including Photo (P), Art painting (A), Cartoon (C), and Sketch (S). DomainNet is the most chal- lenging cross-domain dataset, including Quickdraw (Qu), Clipart (CI), Painting (Pa), Infograph (In), Sketch (Sk) and Real (Re). Considering label noise and class imbalance, we follow Xie et al. (2022) split a subset of DomainNet for our experiments.
Experiment Settings. We apply ResNet-50 as the feature extractor for both PACS and DomainNet, and apply DTN as the feature extractor for Digits (Liang al. 2020). In the source domain, we randomly split 80% data as the training set and the rest 20% as the testing set. In target domains, all data are used for training and testing. The SGD optimizer with an initial learning rate 0.01 used for Digits, and 0.005 for PACS and DomainNet. The exemplar memory size is set 200 for all datasets, and the batch size is 64. For all experiments, we conduct multiple runs with three seeds (2022, 2023, 2024), and report the average performance.
Comparison Baselines. RaTP compared with a comprehensive set state-of-the-art works from Continual DA [CoTTA (Wang et al., (022). AuCID (Rostami [SHOT (Liang et al.) (2020), GSFDA Yang et 2021,, BMD (Qu et al.
2021 Source-Free DA
2022 J. Test- Test-Time/Online
DA [Tent (Wang et al.] (2020). T3A (Iwasawa
Matsuo 2021b], Single DG [L2D (Wang et al.,
2021c), PDEN (Li et al.) 2021)], Unified DA&DG [SNR (Jin et al. 2021) ]. and Multiple DG
[PCL Yao al. 2022). EFDM (Zhang et 2022
Performance comparisons between ours and other methods on Digits in TDG, TDA, and FA under two domain orders (shown with 1). We blue and bold the best, and bold the second best.
Table 1:
CoTTA
AuCID
SHOT
GSEDA
BMD
Tont
T3A
L2D
PDEN
SNR
PCL
EFDM
Ours
Order
Metric
MM
53.3
42.8
55.6
54.8
55.3
53.1
51.3
73.6
68.4
68.9
51.7
48.0
86.8
TDG
SN
23.6
16.1
25.9
28.2
24.9
29.7
22.9
35.4
35.6
35.4
22.8
12.6
43.7
SD
33.8
31.4
44.1
29.3
42.6
38.0
33.9
52.6
54.1
55.8
35.0
15.1
67.5
US
88.9
81.6
69.0
51.6
73.9
87.8
77.4
87.7
83.5
80.5
71.9
35.5
90.1
MT
Avg.
48.7
41.0
49.2
52.2
46.4
60.4
60.2
27.8
72.0
MT
99.0
99.1
99.4
99.4
99.4
99.0
99.0
99.3
99.1
99.2
99.2
99.2
99.4
MM
MM
40.7
42.3
85.3
54.5
80.7
52.8
50.2
86.1
84.7
84.1
53.0
12.9
90.2
+
TDA
SN
18.6
18.2
33.4
13.1
34.7
25.5
23.2
53.8
50.8
53.3
22.6
8.0
69.1
SN
SD
25.4
30.0
41.6
45.4
37.2
31.2
51.6
43.7
10.5
81.8
I
US
89.5
79.6
75.7
14.7
81.2
85.1
83.5
90.3
90.2
90.2
58.0
15.4
93.4
SD
Avg.
54.6
53.8
67.1
39.0
68.3
59.9
57.4
77.6
75.3
74.1
52.2
29.2
86.8
MT
93.6
93.6
73.6
33.9
78.8
93.2
88.4
92.2
91.2
91.9
83.0
31.6
95.4
US
MM
32.9
49.2
39.0
10.0
44.2
50.1
46.4
70.8
69.7
70.9
36.3
15.4
82.1
FA
16.5
22.5
16.0
8.5
17.3
24.7
16.0
50.4
54.2
20.6
10.3
70.5
SD
28.4
30.0
41.1
10.8
45.7
35.3
26.1
70.1
68.8
71.1
27.2
15.7
81.3
Avg.
42.9
48.8
42.4
15.8
46.5
50.8
44.2
72.3
70.0
72.0
41.8
18.3
82.3
SD
36.6
35.8
37.7
37.2
37.4
37.5
36.4
61.3
60.2
60.7
35.3
37.0
68.6
TDG
SN
17.6
19.2
18.4
22.0
18.3
27.2
20.6
52.7
51.5
52.2
21.4
13.4
62.0
29.7
36.2
25.3
33.4
29.4
42.1
34.0
60.8
60.2
17.7
68.7
MT
52.8
63.4
37.4
55.6
56.5
76.7
61.0
83.4
82.7
80.7
62.0
29.0
87.3
US
Avg.
34.2
38.7
29.7
37.1
35.4
45.9
38.0
65.5
63.8
63.5
38.1
24.3
71.7
US
98.5
98.6
98.6
98.3
98.6
98.5
98.5
99.0
98.8
98.9
98.6
98.3
98,99
SD
SD
23.2
24.3
32.9
40.2
30.9
39.8
25.7
74.8
71.8
71.8
37.1
9.6
82.6
I
TDA
13.2
17.6
21.6
17.9
23.4
29.5
22.0
72.1
67.4
64.7
19.5
10.1
78.7
SN
MM
25.7
30.4
19.4
20.0
29.6
39.7
36.1
78.9
76.7
74.4
29.4
10.8
81.8
MT
42.8
53.4
30.3
19.9
53.8
78.8
64.3
89.6
86.8
83.2
54.8
19.8
93.3
MM
Avg.
40.7
44.9
40.6
39.3
47.3
57.3
49.3
82.9
80.3
78.6
47.9
29.7
87.1
90.3
44.2
49.9
37.2
98.5
80.4
93.4
90.3
91.3
10.7
96.2
MT
SD
20.1
25.7
22.3
22.4
26.1
37.7
25.9
71.9
70.7
68.8
26.6
11.0
82.9
SN
11.8
19.1
13.5
8.6
14.1
28.6
13.8
56.8
55.6
55.0
17.7
14.3
68.4
MM
26.1
29.1
14.5
10.1
22.7
41.2
14.9
74.4
71.3
70.9
29.1
27.2
83.5
Avg.
38.3
41.1
23.6
22.8
30.0
51.5
33.8
74.1
72.0
71.5
38.8
15.8
82.8
7

---

Published as a conference paper at ICLR 2023
3.1 EFFECTIVENESS OF THE RATP FRAMEWORK
We assume that each domain corresponds single training stage. After the training of every stage, we test the model performance on all domains. In this case, we can obtain an accuracy matrix, as shown Figure Target domain generalization (TDG) measured by the mean model performance on domain before its training stage (mean of the green elements in Figure D. target domain adaptation (TDA) by the performance on a domain right after finishing its training stage (the red element), and forgetting alleviation (FA) by the mean performance has been updated with other domains (mean the blue elements). All these metrics are the higher, the better. Due to space limitation, (Table in two domain orders, and for DomainNet (Table 3 in one domain order. Results for more domain orders be found in the Appendix. We can observe that RaTP achieves significantly higher performance in TDG than all baselines in all cases, and comparable (and in many cases higher) performance in TDA and FA. Specifically, RaTP significantly outperforms the second- best baseline on TDG in all cases and by 3. 3.1~~00.399 in average, and the performance improvement particularly large over DA methods. For TDA and FA, RaTP is also the best most cases (and clearly outperforms the second best in many cases); and if not, very close to the best. These results demonstrate that our method can effectively improve the model performance during the Unfamiliar Period, adapt the model to different domains, and alleviate catastrophic forgetting on seen domains.
domain after the model
report experiment results for Digits (Tables I and PACS
Table 2: Performance comparisons between and other methods PACS in TDG, TDA, and FA under two domain orders (shown with }). We blue and bold the best, and bold the second best.
Order
Metric
CoTTA
AuCID
SHOT
GSFDA
BMD
Tent
T3A
L2D
PDEN
SNR
PCL
EFDM
Ours
A
71.1
67.0
68.2
75.0
68.4
71.6
71.1
67.5
67.0
67.4
66.4
66.5
73.9
TDG
47.0
29.9
32.0
44.9
34.9
48.5
35.9
49.2
47.8
42.7
42.5
37.7
53.1
46.3
40.8
29.0
46.4
31.3
48.6
38.1
52.7
52.5
45.3
41.4
44.0
60.7
P
Avg.
54.8
45.9
43.1
55.4
44.9
56.2
48.4
56.5
55.8
51.8
50.1
49.4
62.6
LLATKRE
P
99.4
99.4
97.0
98.8
97.0
99.4
99.4
99.1
99.2
99.0
98.8
99.1
99.5
TDA
A
77.0
64.6
85.0
82.1
86.9
76.6
74.2
72.8
70.2
71.8
77.4
72.4
87.5
58.0
70.4
81.6
75.0
68.2
66.7
61.4
60.9
61.2
37.6
75.1
54.4
54.3
62.9
81.5
61.3
58.7
S7.1
68.6
65.8
65.4
65.6
30.0
67.1
Avg.
73.8
69.1
78.8
86.0
80.1
75.7
74.4
75.5
74.0
74.4
75.8
59.8
82.3
P
99.4
96.6
94.9
71.0
94.3
99.4
95.9
91.1
93.2
96.9
77.1
79.3
97.9
FA
74.7
79.1
42.0
79.8
76.2
61.9
64.1
64.6
56.8
87.8
64.5
64.6
66.3
23.1
60.9
68.7
55.7
57.6
55.5
60.2
40.1
46.1
73.0
Avg.
80.3
78.6
80.1
45.4
78.3
81.4
71.2
71.0
70.9
73.9
59.1
60.7
86.2
C
58.4
50.5
46.4
70.1
46.5
58.3
58.4
63.7
63.1
63.1
50.1
64.2
69.6
TDG
A
49.8
40.8
42.0
46.7
43.7
50.4
47.8
52.0
50.8
51.5
36.4
33.6
68.7
54.7
50.8
71.3
60.0
70.9
54.2
63.7
70.2
68.9
70.0
63.3
34.9
83.0
Avg.
54.3
47.4
53.2
58.9
53.7
54.3
56.6
62.0
60.9
61.5
49.9
44.2
73.8
S
95.4
95.5
94.7
96.2
94.7
95.4
95.4
96.3
96.0
96.1
96.1
95.9
96.5
TDA
C
68.3
65.3
86.6
79.4
87.9
67.1
75.3
79.5
79.7
80.0
73.9
70.1
77.1
58.0
67.9
88.1
83.4
88.9
$7.3
72.9
69.7
68.3
69.8
55.7
14.5
87.7
75.1
95.0
69.0
58.5
77.9
85.9
87.0
9.8
97.3
Avg.
69.2
76.0
91.1
82.0
91.8
69.6
80.4
83.0
82.5
83.2
78.8
47.6
89.7
S
96.1
72.4
88.5
89.6
87.6
96.1
86.2
87.1
88.6
88.1
90.7
83.4
94.0
C
68.5
69.9
84.7
67.7
86.6
67.8
64.9
76.7
75.3
76.6
72.0
53.5
80.8
58.1
51.7
84.6
76.4
84.7
58.3
61.6
70.3
70.9
69.1
61.2
20.6
87.7
Avg.
74.2
64.7
85.9
77.9
86.3
74.1
70.9
78.0
78.3
77.9
74.7
52.5
87.5
3.2 ABLATION STUDIES
To demonstrate the effectiveness of each module RaTP, we conduct comprehensive ablation studies for RandMix, T2PL, and PCP. on Digits with the domain order
Effectiveness RandMix. We first remove RandMix from RaTP and see how the model perfor- mance will change. We also to apply RandMix
few other baselines, including SHOT (Liang) et al. 2020). GSFDA Yang et al., 2021). and PCL (Yoo et al. 2022). The experiment results in Fig- (a) demonstrate that, removing RandMix from Ta hurts the performance significantly, and attaching RandMix to other baseline methods clearly improves the model performance. Such ob- servations prove the effectiveness of Ra RandMix.
8

---

Published as a conference paper at ICLR 2023
Table Performance comparisons between ours and other methods on DomainNet TDG, TDA,
and FA (domain order is shown left with 4). We blue and bold the best, and bold the second best.
CoTTA
AuCID
SHOT
GSFDA
BMD
Tent
T3A
L2D
PDEN
SNR
PCL
EFDM
Ours
Order
Metric
Sk
29.4
27.9
22.7
29.1
23.1
29.4
29.4
31.6
29.6
30.6
27.1
31.4
33,4
42.4
22.2
52.0
25.0
50.3
45.4
52.3
49.4
51.4
41.1
27.3
54.6
TDG
In
16.0
16.7
13.0
14.0
11.8
16.6
14.8
13.2
12.2
9.7
12.7
12.6
16.2
Pa
28.4
23.9
15.0
26.4
16.5
27.3
23.0
24.2
22.6
26.2
13.3
14.0
41.0
Qu
Re
52.8
41.2
30.0
41.1
33.9
53.2
47.7
39.8
37.3
35.8
35.9
20.4
62.2
Avg.
34.8
30.4
20.6
32.5
22.1
35.4
32.1
32.2
30.2
30.7
26.0
21.1
41.5
Qu
92.7
92.0
94.8
92.0
92.0
92.1
91.8
92.0
91.8
91.5
90.9
92.0
I
Sk
37.9
31.1
34.9
46.5
37.2
38.2
36.9
37.1
34.2
35.0
34.1
12.2
49.8
Cl
TDA
CI
56.7
54.0
34.9
59.0
35.7
57.2
56.2
55.2
52.3
56.2
48.0
8.5
64.8
In
17.5
19.3
17.2
22.5
12.9
18.9
18.2
13.2
12.1
12.6
17.5
11.5
19.0
19.6
26.9
23.4
33.6
33.2
12.4
20.7
13.5
+
Re
58.2
46.5
34.5
42.9
37.3
60.2
57.2
42.3
39.6
41.8
33.9
4.5
66.5
Pa
Avg.
49.6
45.7
38.9
48.8
39.8
50.0
49.0
42.7
40.4
43.0
39.2
23.5
56.8
Qu
91.9
85.7
42.1
69.7
33.5
91.3
73.6
79.8
76.5
81.9
80.6
64.0
86.1
Re
Sk
38.6
34.8
29.0
33.8
29.2
39.3
33.1
32.7
31.4
33.0
29.0
13.8
51.3
53.0
29.2
36.8
32.2
58.3
48.6
44.5
43.2
42.9
21.7
63.8
In
17.9
17.5
15.4
12.3
12.8
19.3
17.6
13.1
11.8
10.9
14.0
12.5
19.6
Pa
35.1
28.9
21.5
33.3
22.3
34.4
33.7
15.8
16.9
23.7
5.6
5.0
47.8
Avg.
48.0
44.0
27.4
37.2
26.0
48.5
41.3
37.2
36.0
38.5
34.2
23.4
53.7
Pettomrane
60
60
1A 30
30 TDG
10C
TSA
T3G
TDA 1X
T3A
1A
(a) RandMi x (R.M.)
(b) T2PL,
(c) PCA
Figure 4: Ablation studies of RandMix, x, T2PL, and PCA on Digits. The average performance of all domains for three metries (TDG, TDA, FA) is shown. 'Ours' here denotes the full framework of RaTP, while 'Ours-' represents to remove a corresponding module (a, or c) from RaTP, 'Ours-
w/' means replacing the corresponding module with new
Effectiveness of T2PL. We replace T2PL in RaTP with other pseudo labeling approaches inves- tigate the gain from T2PL. including Softmax (Lee et al. 2013). SHOT (Liang et al. 2020). and BMD (Qu al. 2022). Figure 4(b) shows that RaTP performs the best when has T2PL.
Effectiveness of PAA. We first remove LPCA from the optimization objective, leaving only CrossEn- tropy Loss and Logits Distillation. We then try replace PCA with regular forwarding prototype learning (PL), supervised contrastive learning (SCL), and PCL Yao et al. 2022). Figure shows that the performance of RaTP degrades significantly without LPCA, and when we use other methods instead, the performance is still worse. This validates the effectiveness of PCA.
CONCLUSION
This paper focuses on improving the model performance before and during the training continu- ally arriving domains, in what we call the Unfamiliar Period. To solve this practical and important problem, we propose a novel framework RaTP that includes a training-free data augmentation mod- ule RandMix, x, a pseudo labeling mechanism T2PL, and prototype contrastive alignment training algorithm PCA. Extensive experiments demonstrate that RaTP can significantly improve the model performance in the Unfamiliar Period, while also achieving good performance in target domain adaptation and forgetting alleviation, in comparison with state-of-th of-tth--rt methods from Continual DA, Source-Free DA, Rest-ieim/OOnine DA, Single DG, Multiple DG, and Unified DA&DG.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENT
We gratefully acknowledge the support by Sony AI, National Science Foundation awards #2016240, # #1834701, #1724341, #2038853, Office of Naval Research grant N00014-19-1-240 and research awards from Meta and Google.
ETHICS STATEMENT
In this paper, our studies are not related to human subjects. practices data set releases, discrim- imation/biasffairness concerns, and also do not have legal compliance or research integrity issues. Our work is proposed address continual domain shifts when applying deep learning models in real-world applications. In this case, if the trained models are used responsibly for good purposes, believe that our proposed methods will cause ethical issues or pose negative societal impacts.
REPRODUCIBILITY STATEMENT
The source code is provided at http://ituuu.coo/ooyynaiaale All datasets we use are public, In addition, also provide detailed experiment parameters and random seeds in the Appendix.
REFERENCES
Guangji Bai, Ling Chen, and Liang Zhao. Temporal domain generalization with drift-aware dynamic neural network. arXiv preprint arXiv:2205. 10664, 2022.
Slawomir Bak, Peter Car,, and Jean- Francois Lalonde. Domain adaptation through synthesis for unsupervised person re-identification. In Proceedings of the European conference on computer vision (ECCV), pp. 189-205, 2018.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in neural informa- fion processing systems, 32, 2019.
Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 295-305, 2022.
Jiahua Dong, Yang Cong, Gan Sun, Zhen Fang, and Zhengming Ding. Where and how to transfer: knowledge eegregattioninnuuee transferability perception for unsupervised domain adaptation. IEEE Transactions Pattern Analysis and Machine Intelligence, 2021.
Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. Federated class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10164-10173, 2022a.
Xin Dong, Junfeng Guo, Ang Li, Wei-Te Ting, Cong Liu, and HT Kung. Neural mean discrep- ancy for efficient muttof-distrrbuttion detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 19217-19227, 2022b.
Abhimanyu Dubey, Vignesh Ramanathan, Alex Pentland, and Dhruv Mahajan. Adaptive methods for real-world domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14340-1434p 2021.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference machine learning, pp. 1180-1189, PMLR, 2015.
Muhammad Ghifary, David Balduzzi, W Bastiaan Kleijn, and Mengjie Zhang. Scatter component analysis: A unified framework for domain adaptation and domain generalization. IEEE transac- tions on pattern analysis and machine intelligence, 39(7):1414-1430. 2016.
Junfeng Guo, Ang Li, and Cong Liu. Aeva: Black-box backdoor detection using adversarial extreme value analysis. In International Conference on Learning Representations, 2021.
Junyuan Hong, Zhuangdi Zhu, Shuyang Yu, Zhangyang Wang, Hiroko H Dodge, and Jiayu Zhou. Federated adversarial debiasing for fair and transferable representations. Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pp. 617-627, 2021.
10

---

Published as a conference paper at ICLR 2023
Junyuan Hong, Lingjuan Lyu, Jiayu Zhou, and Michael Spranger. MECTA: Memory- continual test-time model adaptation. In International Conference on Learning Representations, 2023. URt/ttps::
Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic do- main generalization. Advances in Neural Information Processing Systems, 34:2427-2440, 2021.
Xin Jin, Cuiling Lan, Wenjun Zeng, and Zhibo Chen. Style normalization and restitution for domain generalization and adaptation. IEEE Transactions on Multimedia, 2021.
Guoliang Kang, Lu Jiang, Yi Yang, and Alexander Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4893-4902 2019.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019.
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, pp. 896, 2013.
Lei Li, Ke Gao, Juan Cao, Ziyao Huang, Yepeng Weng, Xiaoyue Mi, Zhengze Yu, Xiaoya Li, and Boyang Xia. Progressive domain expansion network for single domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 224- 233, 2021.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning. pp. 6028 6039. PMLR, 2020.
Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, and Jiashi Feng. Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2021.
Hong Liu, Mingsheng Long, Jianmin Wang, and Yu Wang. Learning to adapt to evolving domains. Advances in Neural Information Processing Systems, 33:22338-223 -23348, 2020.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Saeid Motian, Marco Piccirilli, Donald A Adjeroh, and Gianfranco Doretto. Unified deep super- vised domain adaptation and generalization. In Proceedings of the IEEE international conference on computer vision, pp. 5715-5725, 2017.
Anshul Nasery, Soumyadeep Thakur, Vihari Piratla, Abir De, and Sunita Sarawagi. Training for the future: A simple gradient interpolation loss generalize along time. Advances in Neural Information Processing Systems, 14:19198-19209, 2021.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. arXiv preprint arXiv:2204 02610, 2022.
Yingwei Pan, Ting Yao, Yehao Li, Yu Wang, Chong-Wor Wah Ngo, and Tao Mei. Transferrable prototyp- ical networks for unsupervised domain adaptation. In Proceedings of the IEEF/CVE conference computer vision and pattern recognition, pp. 2239-2247, 2019.
Theodoros Panagiotakopoulos, Pier Luigi Dovesi, Linus Siveenstam-Neelsee and Matteo Poggi. Online domain adaptation for semantic segmentation in ever-changing conditions. arXiv preprint arXiv:2207.10667. 2022.
Sanqing Qu, Guang Chen, Jing Zhang, Zhijun Li, Wei He, and Dacheng Tao. Bmd: A general class-balen balanced multicentric dynamic prototype strategy for source- free domain adaptation. arXiv preprint ar XiXx:22044008/11 2022.
11

---

Published as a conference paper at ICLR 2023
Sylvestre-Alvin Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampeer. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Mohammad Rostami. Lifelong domain adaptation via consolidated internal distribution. Advances in Neural Information Processing Systems, 34:1 1 11183, 2021.
Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi-supervised do- main adaptation via minimax entropy. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8050-8058, 2019.
Dilbag Singh, Vijay Kumar, Manjit Kaur, al. Classification of covid- 19 patients from chest et im- ages using multi oujei-obeective differential evolution-bast convolutional neural networks. European 2020.
Journal of Clinical Microbiology & Infectious Diseases,
Nima Tajbakhsh, Jac Y Shin, Suryakanth R Gurudu, Todd Hurst, Christopher B Kendall, Michael B Gotway, and Jianming Liang. Convolutional neural networks for medical image anal- ysis: Full training or fine tuning? IEEE transactions on medical imaging, (965):199-1311 2016.
Shixiang Tang, Peng Su, Dapeng Chen, and Wanli Ouyang. Gradient regularized contrastive learning for continual domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 2665-2677 2021.
Korawat Tanwisuth, Xinjie Fan, Huangjie Zheng, Shujian Zhang, Hao Zhang, Bo Chen, and Mingyuan Zhou. A prototype-oriented framework for unsupervised domain adaptation. Advances in Neural Information Processing Systems, 34:17194- 2021.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference Learning Repre- sentations, 2020.
Jindong Wang and Wang Lu. Deepdg: Deep domain generalization toolkit. https://github. com/
Lixu Wang, Shichao Xu, Xiao Wang, and Qi Zhu. Addressing class imbalance in federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 10165-10173,
2021a.
Lixu Wang, Shichao Xu, Ruiqi Xu, Xiao Wang, and Qi Zhu, Non-transfcrable learning: A new approach for model ownership verification and applicability authorization. In International Con- ference on Learning Representations, 2021b.
Qin Wang, Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 7201-7211, 2022.
Zijian Wang, Yadan Luo, Ruihong Qiu, Zi Huang, and Mahsa Baktashmotlagh. Learning to diversify for single domain generalization. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 834-843, 2021c.
Jiangwei Xie, Shipeng Yan, and Xuming He. General incremental learning with domain-aware categorical representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14351-14360, 2022.
Shichao Xu, Lixu Wang, Yixuan Wang, and Qi Zhu. Weak adaptation learning: Addressing cross- domain data insufficiency with weak annotator. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8917-8926, 2021.
Shiqi Yang, Yaxing Wang, Joost de Weijer, Luis Herranz, and Shangling Jui. Generalized source-free domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision pp. 8978-8987, 2021.
12

---

Published as a conference paper at ICLR 2023
Xufeng Yao, Yang Bai, Xinyun Zhang, Yuechen Zhang, Qi Sun, Ran Chen, Ruiyu Li, and Bei Yu. Pel: Proxy-based contrastive learning for domain generalization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp- 7097-7107, 2022.
Yabin Zhang, Minghan Li, Ruihuang Li, Kui Jia, and Lei Zhang. Exact feature distribution matching for arbitrary style transfer and domain generalization. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp.
2022.
Ronghang Zhu and Sheng Li. Crossmatch: Cross-classifie consistency regularization for open-set single domain generalization. In International Conference on Learning Representations, 2022.
URL http:tts//pppeeeeeee net. forum httpee/iiweeeeeetoooommmie...... 4RBsJwGGK
Ronghang Zhu, Xiaodong Jiang, Jiasen Lu, and Sheng Li. Cross-domain graph convolutions for ad- versarial unsupervised domain adaptation. IEEE Transactions Neural Networks and Learning Systems, 2021.
13

---

Published as a conference paper at ICLR 2023
SUMMARY OF THE APPENDIX
This Appendix includes additional details for the ICLR 2023 paper"Dejo Vu: Continual Model Generalization for Unseen Domains", including related work comparison, more implementation details of main experiments, additional experiment results, and sensitivity analysis on parameters used in the framework. The Appendix is organized as follows:
Section A discusses the difference between RaTP and related works.
Section provides more implementation details, including detailed experiment settings, data split- ting strategy, implementation details of the baseline methods, and network structures.
Section presents experiment results of additional domain orders on three datasets.
Section provides detailed sensitivity analysis of three parameters used in RaTP.
Section shows additional evaluations of RaTP in stationary domain adaptation.
A
RELATED WORK COMPARISON
Our work is relevant to number of different topics as we focus addressing important and practi- cal problems. We show detailed comparison among these relevant topics in Table Specifically, Fine-Tuning (Tajbakhsh et al. 2016 is usually conducted on a trained model with labeled target domain data. It can achieve certain domain adaptation capabilities after sufficient training, but the forgetting of previously learned knowledge is often severe. Continual Learning (Lopez-Paz & Ran- zato 2017 Rebuffi et al.) 2017). which shares the same settings as Fine-Tuning can effectively achieve alleviation of catastrophic forgetting. To relax the demand for target labels, series of do- main adaptation (DA) methods are proposed. Unsupervised DA (Ganin & Lempitsky 2015 Dong 2021: Xu et al. (021)) can work effectively when the target data is unlabeled, but suffers from poor forgetting alleviation capability in continually changing cases. Continual
iet al., 2021 Zhu et al.
DA (Wang et al. 2022 Rostami 202T Taagg et al., (22TT) is able to compensate for the forgetting of
learned knowledge, while achieving good domain adaptation, but it requires access to source data.
Source-Free DA (Yang al. 2021 Liang et al. 2020 202 Qu al. 2022) is proposed to adapt the model to unlabeled target domains without the need for source data, but the model forgets lot of learned knowledge during adaptation. Test-Time DA (Wang et al., 2020 Iwasawa & Matsuo) 2021 Chen et al., 2022, Niu et al.) 2022, [Hon et al.) (023) does not need source data and target labels, and can achieve effective domain adaptation without heavy catastrophic forgetting on old knowledge. However, all these DA methods cannot achieve effective target domain generalization on new domains before and during the training on them, which is the main focus of our work here.
Different from DA. Domain Generalization (DG), another cross-domain learning paradigm, can improve model out-of-distribution generalization (Dong et al.] 2022b] Hong et al., 202T) perfor- mance even without seeing target domains. According to the composition of the given source do- main, DG can be divided into two types. Single Source (Wang et al.) 2021c; Li et al.) 2021 Zhu
Li 2022) and Multi-Source (Yao al.) 2022 Zhang al. 2022). Both these two DG-s require labeled source domain data, and can address target domain generalization before any adaptation on target domains. However, they cannot achieve effective target domain adaptation and forgetting alle- viation, which are important in continually changing scenarios. Recently, there are some works (Jin) al., 202T) that try unify DA and DG, but they require significant resources such as source- labeled data and target-labeled data. Moreover, such Unified DA & DG also suffers from forgetting when being applied in continual learning settings. Compared with these studies, our work focuses on improving model performance on new target domains before and during the training them, in what we call the Unfamiliar Period in the main text, while also achieving effective target domain adaptation and forgetting alleviation. To ensure the application practicability, we assume that there is only unlabeled data from target domains. As shown in the main text, the proposed Ra TP framework can effectively solve the problem we target under such assumptions.
B IMPLEMENTATION DETAILS
Experiment Settings. To classify the datasets, we apply ResNet-50 as the feature extractor for both PACS and DomainNet. For Digits, we follow Liang et al. (2200) use DTN as the feature extractor. In source domain, we follow Wang & Lu to split 80% data as the training set and the rest 20% as the testing set. In all target domains, all data used for training and testing. For each dataset,
14

---

Published as a conference paper at ICLR 2023
Table 4: Characteristics of various problem settings that adapt model potentially shifted target domains, especially assuming the domain shift is continually changing.
Target Domain Generalization
Target Domain Adaptation
Forgetting Alleviation
Topics
Source Data Target Data
Target Label
Fine-Tuning Continual Learning
Unsupervised DA
X
X
AAKAK
X
Continual DA
X
Source-Free DA
X
Test-Time DA
X
Single Source DG
x
Multi- Sturce DG
X
X
Unified DA & DG
RaTP
keep the training steps per epoch the same for all domains in all different domain orders. We use 800 steps for Digits, 50 steps for PACS and 70 steps for DomainNet. Training epoch is 30 for all datasets and domains. The SGD optimizer with initial learning rate 0.01 is used for Digits and 0.005 for PACS and DomainNet. Moreover, we use momentum of 0.9 and weight decay of 0.0005 to schedule the SGD. The exemplar memory is set as 200 for all datasets, and we set the batch size of mini-batch training as 64. Only ResNet-50 is initialized as the pre-trained version of ImageNet.
Subset of NomainNet. domains containing very few images (~10). This makes it hard to assign pscudo labels for these classes. Thus, we refer to Xie et al. (2022) to select the top 10 classes with most images in all domains, which contain 26,013 samples in total. This dataset is still class imbalanced. with the smallest sample number for class is 32, while the largest 901, Therefore, this subset still quite challenging.
The original DomainNet is class imbalanced, with certain classes in some
Baseline Implementations. For fair comparison, we try our best to extend all baseline methods in our problem settings. AuCID) (Rostami) 2021) shares the most similar setting with But their exemplar memory size is not limited and can be enlarged as more domains appear. And they transform Digits to gray scale images while we treat them as RGB images. For Test- Test-Timm/OOnine DA [CoTTA (Wang et al., 2022), Tent (Wang et al.) 2020). T3A Iwasawa & Matsuo (2021), Ada- Con (Chen et al., 2022), EATA (Niu et al. [. we keep the setting that adapts the model on each target domain only once. For Single DG [L2D (Wang et al. (021c). PDEN (Li et al. 202T) and Multiple DG [PCL (Yao et al., 2022). EFDM (Zhang et al. 2022p]. we use SHOT (Liang al.) 2020) to assign pseudo labels for the optimization on target domains. In EFDM, we use samples from current domain as the content images and randomly select images the replay memory as the style images. SNR (Jin et al. 2021) directly modifies the model structures to match statistics of different domains, thus it can be viewed as a unified approach for both DA and DG. In our imple- mentation, we use SNR to modify the convolution filters of the feature extractor and optimize the model with SHOT. We also consider a more advanced version of SHOT for further comparison, i.e., SHOT++ (Liang et al. 2021). which equips an additional semi-supervised training step using Mix- Match (Berthelot et al. 2019). This may not be fair for other methods because all methods can in principle equip with this additional training step. Thus, also run SHOT++ without the MixMatch and denote it as SHOT+ All baseline methods are equipped with the same exemplar memory and the same feature extractor.
Network Structures. Our RandMix augmentation network includes four autoencoders that consist of a convolutional layer and transposed convolutional layer. All these layers have 3 channels and kernel sizes of 5, 9, 13, 17, respectively. The classification model contains feature extractor, a bottleneck module and classifier. Specifically, pre-trained ResNet-50 is used as the feature extractor for both PACS and DomainNet. BallowinggLang al. (2020), DTN, variant of LeNet is used as the feature extractor for Digits. We also introduce a bottleneck module between the feature extractor and the classifier. The bottleneck consists of fully-connected layer (256 units for Digits and 512 units for both PACS and DomainNet), a Batch Normalization layer, a ReLU layer and another fully-connected layer (128 units for Digits, 256 units for PACS, and 512 units
15

---

Published as a conference paper at ICLR 2023
for DomainNet). The classifier is cully-connected layer without the parameters of bias. Both contrastive loss and distillation loss are applied to the representations after the bottleneck.
C ADDITIONAL EXPERIMENT RESULTS
Following existing studies of continual domain shifts (Rostami, 2021 Wang et al. 2022 Tang al. 202T], we have presented experiment results under the most popular domain orders in the main text. To further explore the impact of domain order, which is rarely studied by existing works, conduct more experiments with additional domain orders for the three datasets. Specifically, for each dataset, carry out experiments under 10 different domain orders that randomly shuffled, shown in Table 5 We present the average performance for all orders in Table 6 (Digits), Table 7 (PACS), Table (DomainNet). Note that for the average performance in Tables 7 and 8 we select subset of the baseline methods that perform relatively better than the rest, including a few
additional basclines (i.c., SHOT+ (Liang al. 2021), SHOT+ (Liang al. 202T). AdaCon CChen
et al. 2022). EATA (Niu et al. 2022). L2D Wang et al. 2021c). and PDEN {Li et al., 2021 to carry out experiments for saving time and space. Under these additional domain orders, RaTP still achieves the best average TDG all three datasets, and comparable (and in many cases better) performance in TDA and FA. Combining these results with the ones shown in the main text, we can clearly that RaTP provides significant improvement TDG over previous state-of-the ahttthe-rt methods, while achieving comparable performance in TDA and FA.
Table 5: Different domain orders used assess their impact various approaches' performance. The results are reported in Tables 6 and 8 for the average performance of all orders.
Order Datasets
Digits
PACS
DomainNet
Order I
SN +M -MIIIMMM--SD----
A-C--P-S
Order 2
-N-
A
Ql-ln-Pa-QU--R--
Order 3
MM-UUS-MT-SD--SM
A-P-C-S
CI-Re-In-sQ--SS--PP
Order 4
-T-MM--N-SDD-US
C-A-+S-P
1n-=Ou-Cl-Pa--Re--50
Order 5
MT-MM--US-SN-+
C-S-PPA
Pa-sk-QQu-In-R----
Order 6
SD-MMM-SN-MT
P-A--C-S
Qu-Re-JCI-Pa-inn
Order
P-S--A-C
Qu -a------C--in-
Order
P-S--C-A
-t-
Order 9
US-MMTSSN-MM-SS
S-C-AAP
St-Re--Pa-cl-Quu--n
Order 10
US-SSD-SN-MM--NT
S-P-C-A
Sk-RR:
D SENSITIVITY ANALYSIS
We conduct sensitive analysis of the confidence threshold Toon in splitting RandMi x, and rtop and r in T2PI. All sensitive analysis experiments are conducted on Digits with the domain order of S-SD-SNN-MM MM-MM.. For splitting RandMix x, Feon controls the proportion of unlabeled data that needs to be augmented. As shown in Figure 5(a) different - correspond to different model performance, and can obtain the best performance when Teon 0.8, which is adopted for Digits and DomainNet. And rcon is set as 0.5 for PACS. As for the sensitivity analysis of the other two parameters, according to the results shown Figures(5(6))andd55c we can observe that larger Ttop that corresponds to smaller amount of fitting samples is detrimental to the model performance, while larger rtop that corresponds to fewer nearest samples for the kNN classification leads to slightly better performance. We choose Ttop and r'op 20 for Digits and PACS, Ttop = 1 and Teop 10 for DomainNet.
E EVALUATIONS ON STATIONARY DOMAIN ADAPTATION
While our work primarily focuses on continual domain shift, here we provide results of stationary domain adaptation for further comparison. In this setting, we consider the combination of one single source domain and one single target domain, and consider the generalization performance on the target domain accuracy after adaptation. distillation loss term (Eq. 10) and the second term in the nominator of LPCA (Eq. 9), as they force the model to achieve forgetting alleviation for previous domains and may impede the adaptation to the target domain. We conduct adaptation are shown in Tables Compared with other baselines, RaTP performs the best in most Paalesspee cases and achieves 4.4% higher average accuracy than the second best.
such situation, we remove the replay memory, the
stationary domain adaptation in PACS, and the results after
16

---

Published as a conference paper at ICLR 2023
Table 6: Performance comparisons between and other baseline methods Digits in average TDG, TDA, and FA under 10 different domain orders. We blue and bold the best performance, and bold the second best.
Metric & Orders
SHOT+
SHOT++
Tent
AdaCon
EATA
L2D
PDEN
Ours
Order I
66.2
68.3
71.1
72.6
71.3
72.3
69.4
77.0
Order 2
78.0
78.2
72.9
75.8
71.5
78.1
78.4
79.5
Order 3
68.3
65.8
70.7
67.0
69.6
71.7
70.5
77.0
Order 4
49.1
52.0
52.2
53.2
53.7
62.3
60.4
72.0
Order 5
54.0
54.1
53.1
51.1
53.6
62.7
61.4
72.9
TDG
Order 6
72.3
75.2
76.9
75.8
77.8
78.2
76.8
81.0
Order 7
74.8
76.0
76.9
73.0
76.1
78.1
76.8
81.9
Order 8
73.9
72.6
79.3
76.9
77.9
78.0
77.3
81.3
Order 9
35.1
39.0
41.3
41.3
44.1
61.7
61.7
73.2
Order 10
38.6
41.7
45.9
46.3
44.2
65.5
63.8
71.7
Avg.
61.0
62.3
64.0
63.3
64.0
70.9
69.7
76.8
Order 1
84.0
87.5
71.5
77.4
76.8
85.9
81.9
89.7
Order 2
91.6
94.8
77.5
76.0
76.9
91.3
89.5
90.7
Order 3
81.2
79.9
70.7
75.8
76.4
85.9
86.2
87.8
Order 4
73.8
79.6
59.9
64.9
65.0
77.6
75.3
86.8
Order 5
79.7
84.9
59.5
65.3
65.8
79.3
78.3
87.5
TDA
Order 6
87.0
92.1
80.2
80.5
81.1
89.7
89.0
90.0
Order 7
89.9
91.2
80.9
82.1
83.2
87.6
85.2
91.6
Order 8
89.0
91.5
80.5
80.2
82.2
88.6
85.9
89.7
Order 9
48.4
48.8
48.7
55.7
55.4
74.2
70.9
85.9
Order 10
61.2
62.9
57.3
58.3
57.1
82.9
80.3
87.1
Avg.
78.6
81.3
68.7
71.6
72.0
84.3
82.3
88.7
Order 1
60.0
67.1
67.8
75.2
76.2
75.2
71.4
83.8
Order 2
73.9
75.5
82.2
82.7
83.6
81.1
79.6
87.4
Order 3
70.7
71.2
72.9
80.4
85.5
85.1
81.9
90.1
Order 4
56.5
65.3
50.8
59.0
58.8
72.3
70.0
82.3
Order 5
77.0
79.1
61.4
71.7
71.2
74.9
73.9
85.2
FA
Order 6
59.3
67.4
81.2
80.4
79.7
76.8
74.1
84.9
Order 7
62.2
71.0
79.8
82.1
80.9
77.6
76.1
84.7
Order 8
57.2
66.0
80.0
81.9
79.4
75.0
72.6
83.3
Order 9
25.1
30.0
33.1
56.8
61.8
72.5
68.5
85.1
Order 10
39.7
52.5
51.5
52.0
52.4
74.1
72.0
82.8
Avg.
58.2
64.5
66.1
72.2
73.0
76.5
74.0
85.0
95
95
95
TDG
90
IDA
TDC
TDA
85
80
85
80
80
70
60
70
70
65
0.0
0.5
0.8
1.0
50
10
20
65
20
(a)
(b) Ttop
(c) Tlop
Figure 5: Sensitivity analysis of confidence threshold reon in RandMix; Ttop and roo in T2PL.
17

---

Published as a conference paper at ICLR 2023
Table 7: Performance comparisons between ours and other baseline methods on PACS in average TDG, TDA, and FA under 10 different domain orders. We blue and bold the best performance, and bold the second best.
Metric Orders
SHOT+
SHOT++
Tent
AdaCon
EATA
L2D
PDEN
Ours
Order 1
69.4
70.4
75.5
75.2
75.1
74.0
73.7
76.6
Order 2
67.0
68.7
73.1
74.6
72.5
76.0
71.6
75.8
Order 3
67.8
63.3
75.6
75.9
76.1
72.8
73.5
76.6
Order 4
69.5
66.1
78.5
77.1
77.4
78.1
77.2
79.8
Order 5
61.1
62.2
81.6
74.6
78.3
74.6
73.5
78.
TDG
Order 6
48.5
50.0
56.2
57.2
57.4
56.5
55.8
62.6
Order 7
36.6
43.2
52.5
55.4
54.3
54.9
52.0
56.8
Order 8
37.2
39.0
50.6
52.1
51.8
52.8
51.5
54.8
Order 9
53.1
52.7
54.3
57.3
48.2
62.0
60.9
73.8
Order 10
39.1
44.6
60.2
52.3
50.0
56.7
54.6
70.7
Avg.
54.9
56.0
65.8
65.2
64.1
65.8
64.4
70.5
Order
86.7
89.4
84.0
85,8
86.7
84.5
83.7
83.5
Order 2
87.8
89,4
82.0
81.6
85.3
83.6
83.3
84.5
Order 3
88.7
88.8
82.5
82.8
85.6
82.9
79.9
84.2
Order 4
89.2
91.2
88.2
88.7
89.2
84.6
83.0
86.8
Order 5
85.2
85.4
88.6
86.4
88.2
80.1
78.2
85.5
TDA
Order 6
83.1
85.3
75.7
78.6
79.2
75.5
74.0
82.3
Order 7
66.9
69.9
74.4
74.0
73.0
71.3
71.6
76.6
Order 8
64.0
68.8
72.5
73.9
72.3
68.5
69.8
74.9
Order 9
91.5
92.2
69.6
77.8
73.0
83.0
82.5
89.7
Order 10
75.9
83.2
69.8
69.7
70.4
74.4
72.1
90.1
Avg.
81.9
84.4
78.7
79.9
80.3
78.8
77.8
83.8
Order
73.0
78.6
89.5
90.7
91.4
85.6
85,2
87.8
Order 2
72.4
82.3
79.5
77.7
83.7
80.6
77.4
77.9
Order 3
81.8
78.9
88.5
89.5
90.5
84.8
78.7
87.5
Order 4
76.9
77.6
83.3
84.5
87.7
77.5
77.1
82.5
Order 5
82.9
86.1
89.0
88.0
90.8
76.7
76.5
84.2
FA
Order 6
79.6
84.3
81.4
80.7
83.5
71.0
70.9
86.2
Order 7
65.3
80.5
78.0
78.0
74.4
75.7
75.4
79.7
Order 8
58.3
83.5
73.3
74.0
73.3
72.6
72.1
73.4
Order 9
86.5
88.8
74.1
79.0
76.6
78.0
78.3
87.5
Order 10
72.0
89.5
73.1
73.7
74.3
73.4
71.4
89.6
Avg.
74.9
83.0
81.0
81.6
82.6
77.6
76.3
83.6
18

---

Published as a conference paper at ICLR 2023
Table 8: Performance comparisons between and other baseline methods DomainNet in aver- age TDG, TDA, and FA under 10 different domain orders. We blue and bold the best performance, and bold the second best.
Metric & Orders
SHOT+
SHOT+
Tent
AdaCon
EATA
L2D
PDEN
Ours
Order I
46.9
45.5
52.1
51.3
51.2
49.6
48.0
51.9
Order 2
52.2
50.0
31.0
52.7
55.2
55.6
51.2
57.4
Order 3
53.6
53.3
58.4
53.7
58.7
52.8
51.1
59.4
Order 4
40.8
41.9
50.6
51.3
51.1
48.2
47.0
54.7
Order 5
48.4
49.6
52.8
53.0
52.8
53.1
51.0
56.0
TDG
Order 6
34.0
35.3
33.1
32.9
33.6
36.5
37.2
46.7
Order 7
23.2
25.7
35.4
32.9
34.4
32.2
30.2
41.5
Order 8
59.2
59.9
61.0
62.0
62.0
62.1
61.4
61.8
Order 9
58.7
59.3
61.3
61.6
63.3
59.4
59.9
63.7
Order 10
56.2
60.1
41.2
61.3
59.0
57.4
56.4
62.3
Avg.
47.3
48.1
47.7
51.3
52.1
50.7
49.3
55.5
Order 1
68.4
70.5
59.0
60.4
60.0
59.9
60.8
68.1
Order 2
69.7
66.2
28.9
66.2
65.8
56.5
54.2
69.6
Order 3
72.6
73.4
65.6
68.6
69.4
54.8
52.7
70.3
Order 4
51.3
53.5
54.6
52.2
52.9
57.3
55.2
60.3
Order 5
68.5
70.9
60.3
60.3
58.7
56.9
55.7
68.2
TDA
Order 6
63.1
65.3
51.8
52.4
55.0
49.3
50.2
65.1
Order 7
47.7
48.1
50.0
50.8
51.7
41.7
40.4
56.8
Order 8
72.8
73.2
67.6
71.6
70.7
64.0
63.4
75.5
Order 9
74.2
75.9
67.6
71.3
69.3
61.9
62.2
74.1
Order 10
71.9
72.1
31.0
67.9
71.2
59.9
61.0
67.8
Avg.
66.0
66.9
53.6
62.2
62.5
56.2
55.6
67.6
Order
61.4
66.5
67.4
67.0
64.3
63.7
61.1
67.6
Order 2
64.5
68.9
34.1
62.6
65.8
48.9
46.3
67.7
Order 3
62.9
67.7
65.6
66.3
69.2
45.2
43.1
68.1
Order 4
42.1
65.4
56.4
53.3
52.7
41.5
39.5
56.5
Order 5
60.9
68.5
58.0
56.6
57.4
51.2
48.6
65.4
FA
Order 6
61.1
66.3
52.4
49.4
54.8
48.0
46.0
62.8
Order 7
42.8
51.7
48.5
48.5
47.7
37.2
36.0
53.7
Order 8
61.6
67.5
71.6
72.8
73.5
58.8
55.1
72.0
Order 9
67.4
77.3
76.8
76.1
76.0
66.5
65.6
79.6
Order 10
60.4
69.6
30.4
65.5
66.3
61.4
60.9
62.9
Avg.
58.5
66.9
56.1
61.8
62.8
52.2
50.2
65.6
Table Comparison of domain adaptation for stationary source-target pairs on PACS, We blue and bold the best average performance, and bold the second best.
Method
P-
P-
P-S
A
A-
A
A-$
C-P
C-A
C-S
S-P
S-A
S-CC
Avg.
SHOT
87.2
85.4
33.8
99.3
87.8
44.4
99.0
91.9
64.9
53.5
84.9
89.8
76,8
SHOT+
90.4
87.9
38.8
99.5
88.8
41.9
98.9
92.6
67.2
52.2
84.3
89.5
77.7
SHOT++
93.1
89.1
99.6
41.3
99.1
93.6
68.2
87.2
91.7
78.4
Tent
77.0
64.9
97.4
76.7
63.7
95.4
82.8
73.5
59.1
68.0
72.4
AdaCon
88.2
85.3
56.6
99.5
86.8
67.7
97.7
91.5
75.0
61.3
82.7
75.3
80.6
EATA
79.2
72.7
59.0
97.2
80.0
72.7
95.7
84.8
79.7
55.3
60.0
68.7
75.4
Ours
92.5
83.2
62.4
99.5
88.0
98.9
92.6
64.8
92.8
86.7
85.0
19


Reviews:
[{"review_id": "sh-KVl8RUXS", "reviewer": "Reviewer_sJyy", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript proposes to tackle the continual domain shift learning issue, in which the model is trained on a source domain and serveral unlabeled target domains. The authors highlight the pain points of the issue are concluded with three: 1. model generalization on 'before and during' training doamins(TDG); 2. better usage of unlabeled target domains(TDA); 3. a commonplace talk of an old scholar which is Catastrophic Forgetting. To address the issues, a Twofer method is proposed, which is composed by three components: 1. Random Mixup Augmentation; 2. Top^2 pseudo labeling; 3. prototype contrative alignment. Finally, extensive experiments are conducted.\n\n", "summary_of_the_review": "In summary, the biggest issue to me is the writing. I cannot get the motivation of the proposed modules. To this end, I think the manuscript needs further polish.", "strength_and_weaknesses": "Strength:\n1. The motivation is new and seems to be practical proposed by the authors;\n2. The experimental results are superior.\n\nWeaknesses:\n1. The writing of the manuscript is hard to follow, some definitions are unclear, i.e., what is distinguishability of data samples in 4th line of P5? Moreover, some formulas have better expression but not the adopted ones, which takes me a lot of time to grab the meaning. The choice of notation is unprofessional. \n2. The motivation of random mixup augmentation is weak. As far as I am concerned, deploying variant noise in training does harm to stability of training. Then, how it improves the generalization is hard to explain. Moreover, there are an ocean of noise generation method, why Eq. 2 is better?\n3. There lacks necessary explaination to Sec2.3 and the caption and content of Fig2 are unclear.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "cd8B7erTpC", "reviewer": "Reviewer_CHgo", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work is very interesting. It tries to simultaneously address domain generalization, domain adaptation and catastrophic forgetting problem when the learning model needs to tackle continual domain shifts over time, which is called Continual Domain Shift Learning (CDSL). To solve this, this work proposes a framework – Twofer, which consists of three major modules. Specifically, there is a data augmentation module that can generate more additional data to help enhance domain generalization ability. Then a novel KNN-based pseudo labeling technique is proposed for giving more accurate supervision that can be used in later adaptation. Finally, a modified prototype contrastive loss is used to align features among different domains. The proposed twofer is tested on various dataset and domain orders, with the comparison with extensive state-of-the-art baselines from continual domain adaptation, test-time adaptation, source-free domain adaptation, single domain generalization, multiple domain generalization, and even unified domain adaptation and generalization. According to experiment results, twofer can outperform these baselines on three metrics that are related to domain generalization, adaptation, and forgetting.", "summary_of_the_review": "In summary, the setting is practical and the proposed methods are novel and effective. I thus recommend this paper to appear at ICLR.", "strength_and_weaknesses": "Strength:\n\n1.\tThe idea of considering performance before adaptation in domain continual learning setting is novel and practical in the real world.\n\n2.\tThe training-free RandMix method can keep the cost low and achieve high generalization ability, which is very important in this continual domain generalization setting. In many cases, scenarios that require the use of this setting, such as surveillance cameras, have limited computing power.   \n\n3.\tOnly using a subset of features and introducing kNN for final selection in T2PL can filter out features that can cause side effects. I think the effect of this method will be more obvious when the data noisy is relatively high.  \n\n4.\tThe proposed Prototype Contrastive Alignment is novel. Instead of modeling the distribution of previous tasks and then aligning the features in existing works, PCA provides a new and more efficient way to align previous and current features by saving a small part of network parameters and reducing the domain adaptivity gap. Furthermore, this loss function is easy to be adapted to other frameworks.\n\n5.\tThe experiments show high effectiveness compared to other SOTA methods, especially when target domain is much more complicated than source domain, e.g., SVHN in Digital dataset.\n\nWeaknesses:\n\n1. The idea of RandMix looks like a simple combination of DSU and L2D. Not sure my understanding is correct or not.\n\n2. The paper introduces the kNN in T2PL without detailed explanations of why it can work. I think more analysis about it needs to be provided.\n\n3. It seems that the domain order is an important factor that influences the overall performance. In main paper, most of the average performance of three metrics is 5-10% better than the second best. But in appendix, it’s only 1-2% better or even worse than baseline. Can the author(s) explain the reason why?\n\n4. In table 1, the drop in accuracy on SVHN is too large for all source free adaptation methods and makes me doubt the accuracy of the experiments. Can the author(s) explain the reason and I’m also checking the experiments results in related works.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "C3wN-YYp6u", "reviewer": "Reviewer_jYsN", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work addresses adaptation to a sequence of discrete domains where accuracy before, during, and after adaptation is desired.\nAccuracy before processing a domain measures domain generalization, accuracy during or immediately following a domain measures adaptation, and accuracy after a domain measures forgetting (or remembering).\nThis setting is named continual domain shift learning (CDSL) in this work, and presented as a new problem statement because of (1) the changing of the domains and (2) the metrics before/during/after the domain is adapted on.\nThe proposed method, TwoFer, incorporates elements of domain adaptation and domain generalization, and its most important components are agressive data augmentation (RandMix), a pseudo-labeling scheme that clusters and filters predictions to keep only a certain percentile, and a prototype representation of classes in the target domains (prototype contrastive alignment or PCA).\nEach component is partly-tuned to its use here, but all of them have a strong relation to prior work that introduced it.\nExperiments cover three benchmarks that are common to domain adaptation and generalization: digits, PACS, and DomainNet.\nHowever, the evaluation protocol differs from the standard protocol, both in the desired measurement of sequences of domains, but also in a different sampling of splits.\nSince the results are sensitive to the order of the domains adapted on, two orders are evaluated in the main paper for each dataset (with one more each in the appendix).\nBaselines cover domain adaptation, in particular source-free and test-time adaptation, as well as domain generalization.\nAblations check the separate contribution of the data augmentation, pseudo-labeling, and prototypical loss components of the method, and analyze alternative choices or applications of each component.\n\n", "summary_of_the_review": "The dismissal of prior work and the divergent evaluation w.r.t. the established protocol used by the compared methods raises the possibility of serious experimental error.\nIt may be that everything is sound, but the rebuttal needs to clarify exactly how the evaluation was done to be sure. Ideally, the rebuttal would report results in the standard experimental setup for a fair comparison to accompany the main results of the paper in Tables 1 & 2.\nWithout this information, it is hard to gauge the correctness and empirical significance of the submission.\nI am open to counterevidence however, and will reconsider this submission in light of response and discussion.\n\n*For Rebuttal*\n\nMajor\n\n- Please report results in the standard evaluation protocols for PACS and DomainNet. In particular, please use the standard splits, and include results with stationary/non-continual/episodic domains. By comparing in the established setting, the experiments would establish that Twofer is at least as good there, and then the experiments could should improvement in the proposed protocol.\n- Please analyze the sensitivity of TwoFer and the baselines to domain order. If the variance due to order is larger than the reported gains, then the experiments may have inadvertently chased noise.\n- Please relate TwoFer to EATA and AdaContrast and report any comparisons, if any comparable comparisons are possible given the set of experiments in the submission.\n- Please clarify the novelty or lack thereof in the setting, compared to the missing related work raised, and re-articulate the contributions of CDSL vs. other studies of continual shift in existing work like CoTTA and EATA.\n\nMinor\n\n- Was DomainNet filtered according to the evaluation protocol in prior work like Saito et al. 2019 and Chen et al. 2022 (AdaContrast)?\n\n**Update after Response**\n\nThe substantial responses and experiments address concerns about experimental validity concerning data splits, domain ordering, and missing benchmarks. As such I am raising my score to 5 (borderline reject) to acknowledge these improvements while still urging the authors to consider the framing and organization of this work so that it is more comprehensible to an audience that spans adaptation, continual learning, and robustness. I have accordingly raised the marks for correctness and empirical novelty/significance as well.", "strength_and_weaknesses": "*Strengths*:\n\n- The setting emphasizes adaptation across multiple domains, in sequence, and the need for more metrics to do so. This setting measures not only accuracy _during adaptation_ to a given domain, but _before_ to measure domain generalization as well as _after_ to measure resilience to forgetting.\n- The baselines are drawn broadly from source-free adaptation and domain generalization (but they are not comprehensive, see weaknesses), and for some baselines such L2D, PDE, and PCL there is an attempt to equip them for the proposed setting by providing the same exemplar memory as the proposed method.\n- The augmentation model, building on domain diversification (Wang et al. 2021) and AdaIN (Karras et al. 2019), proves effective for the proposed method as well as baslines, and significantly impoves accuracy for adaptation across digits datasets (Figure 4a).\n- The ablation study (Section 3.2 and Figure 4) justifies augmentation by randmix, the pseudo-labeling by T2PL, and most of all the PCA loss. However, the ablation is only on Digits, which is the simplest dataset, and it would be more rigorous to repeat these experiments for PACS and DomainNet too in order to check for consistent effects.\n\n*Weaknesses*:\n\n- The proposed setting depends on the order of domains, but only two orders are shown in the main paper, and just one additional order in the appendix. The order in the Appendix for PACS and DomainNet shows _no or little improvement_ for Twofer, while Digits does still improve, which suggests the possibility that order can be quite important. This small sampling of orders is unlikely to give a non-noisy estimate of performance.\n- The framing is neither accurate nor cordial and collegial. This paper does not begin the study of continual shift, counter to the claims of the abstract and introduction. Rather it focuses its metrics on particular phases of adaptation, and emphasizes (1) rapid learning at the beginning of a domain and (2) less forgetting on past domains.\n- The evaluation protocol differs from the compared work, in defining its own splits of the data (see \"Experiment Settings\"), but how exactly the splits are defined is not clear. Alternative evaluations can be necessary to make a point, but such oddball evaluations should be paired with the standard evaluations for fair comparison and sanity checking of the results.\n- There is missing related work on test-time adaptation with more efficient updating and slower forgetting: Efficient Test-Time Model Adaptation Without Forgetting (EATA) at ICML'22. As this prior work addresses key claims of TwoFer, it needs to be compared with in the experimental evaluation. It should likewise be discussed should its contributions and technical details intersect with the claims in the proposed method.\n- There is missing related work on source-free adaptation: AdaContrast (Chen et al. CVPR'22) and SHOT++ (Liang et al. PAMI'21). This requires discussion and comparison, as results are reported for DomainNet and other standard benchmarks.\n- The proposed method heavily relies on data augmentation, which makes it more domain specific than other methods, which perform none or little, like Tent or EATA for example.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "AhkCUeLJNj", "reviewer": "Reviewer_Nus3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work focuses on a practical challenge that deep learning models cannot remain stable performance when being deployed in real-world environments. For this problem, the author(s) defines three objectives, i.e., target domain generalization, target domain adaptation and forgetting alleviation. To achieve these objectives, a framework called Twofer is proposed to better align domains over time. According to extensive evaluation experiments and comprehensive baseline comparison, the effectiveness of Twofer can be demonstrated.", "summary_of_the_review": "This paper is clear and technically solid. The experiments are thorough with impressive results, and the analysis is extensive. ", "strength_and_weaknesses": "Strength:\n\n1. The objective of target domain generalization is appealing and significant, and I particularly like the definition of ‘unfamiliar period’ in the paper. Although a number of domain adaptation studies have been proposed, including continual and source-free, most of them neglect the generalization before adaptation. This paper bridges this gap by allowing the deployed models to evolve themselves and remain stable performance all the time. \n\n2. The proposed Twofer is composed of three major modules, which work together for achieving better domain alignment. These three modules are well-motivated.\n\n3. The author(s) compare twofer with a number of SOTA baselines. The experiments are extensive, and the results can demonstrate the effectiveness of proposed methods.\n\nWeakness:\n\n1. If I understand correctly, twofer needs to store a small number of samples at each stage for later usage, is it possible to relax this requirement? More should be discussed. \n\n2. I notice that there is no section of related work, which is important to this work. Therefore, I believe the author(s) needs a section or a table to present the difference among different DA and DG topics.\n\nQuestions:\n\n1.\tThe author(s) mentions that blindly pushing augmentation data away from the original possibly hurts the generalization performance. Could the author(s) give more explanation here? \n\n2.\tBesides, is it possible to extract useful information from seen domains for guiding the data augmentation in the future? Because the used augmentation module is simple in terms of network structure, the training cost is acceptable if learnable augmentation can bring noticeable performance gain.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster