SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This paper considers a more challenging problem of continual domain generalization over temporal drift (CDGTD) than conventional DG, where the model is incrementally trained with sequential domains and is required to generalize to unseen domains that are not too far into the future. To this end, we propose an Evolving Standardization (EvoS) method to learn the evolving pattern of sequential domains over the temporal drift and hope to achieve the generalization by transforming the domain distribution into a common normal distribution via feature standardization. Experiments on real-world datasets including images and texts verify the efficacy of EvoS. Since existing DG works focus on conventional setting, we hope this work can encourage more research works on CDGTD.
[1] R. Aljundi, M. Lin, B. Goujaud, and Y. Bengio. Gradient based sample selection for online continual learning. In NeurIPS, pages 11816-11825, 2019.
[2] M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant risk minimization. arXiv:1907.02893, 2019.
[3] G. Bai, C. Ling, and L. Zhao. Temporal domain generalization with drift-aware dynamic neural networks. In ICLR, 2023.
[4] S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan. A theory of learning from different domains. Mach. Learn., 79(1-2):151-175, 2010.
[5] S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. Analysis of representations for domain adaptation. In NeurIPS, pages 137-144, 2006.
[6] G. Blanchard, G. Lee, and C. Scott. Generalizing from several related classification tasks to a new unlabeled sample. In NeurIPS, pages 2178-2186, 2011.
[7] G. Blanchard, G. Lee, and C. Scott. Generalizing from several related classification tasks to a new unlabeled sample, 2011.
[8] M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In NeurIPS, 2020.
[9] F. M. Castro, M. J. Marín-Jiménez, N. Guil, C. Schmid, and K. Alahari. End-to-end incremental learning. In ECCV, pages 233-248, 2018.
[10] A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny. Efficient lifelong learning with A-Geom. In ICLR, 2019.
[11] T. Chen, S. Kornblith, M. Norouzi, and G. E. Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597-1607, 2020.
[12] X. Chen, W. Chen, V. Dinavahi, Y. Liu, and J. Feng. Short-term load forecasting and associated weather variables prediction using resnet-lstm based deep learning. IEEE Access, 11:5393-5405, 2023.
[13] L. Deng. The MNIST database of handwritten digit images for machine learning research [best of the web]. SPM, 29(6):141-142, 2012.
[14] S. Ebrahimi, F. Meier, R. Calandra, T. Darrell, and M. Rohrbach. Adversarial continual learning. In ECCV, volume 12356, pages 386-402, 2020.
[15] Y. Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML, pages 1180-1189, 2015.
[16] S. Ginosar, K. Rakelly, S. Sachs, B. Yin, and A. A. Efros. A century of portraits: A visual historical record of american high school yearbooks. In ICCV Workshops, pages 652-658, 2015.
[17] R. Gong, W. Li, Y. Chen, and L. Van Gool. Dlow: Domain flow for adaptation and generalization. In CVPR, 2020.
[18] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. In CVPR, pages 2261-2269, 2017.
[19] Y. Iwasawa and Y. Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In NeurIPS, pages 2427–2440, 2021.
[42] A. Nguyen, T. Tran, Y. Gal, and A. Baydin. Domain invariant representation learning with domain density transformations, 2021.
[43] G. Ortiz-Jiménez, M. E. Gheche, E. Simou, H. P. Maretic, and P. Frossard. CDOT: continuous domain adaptation using optimal transport. arXiv:1909.11448, 2019.
[44] T. Qin, S. Wang, and H. Li. Generalizing to evolving domains with latent structure-aware sequential autoencoder. In ICML, pages 18062-18082, 2022.
[45] J. Rajasegaran, M. Hayat, S. H. Khan, F. S. Khan, and L. Shao. Random path selection for incremental learning. arXiv:906.01120, 2019.
[46] S. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert. ictarl: Incremental classifier and representation learning. In CVPR, pages 5533-5542, 2017.
[47] G. Saha and K. Roy. Continual learning with scaled gradient projection. In AAAI, pages 9677-9685, 2023.
[48] V. Sanh, L. Debut, J. Chaumont, and T. Wolf. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv:1910.01108, 2019.
[49] S. Schneider, E. Rusak, L. Eck, O. Bringmann, W. Brendel, and M. Bethge. Improving robustness against common corruptions by covariate shift adaptation. In NeurIPS, 2020.
[50] B. Sun and K. Saenko. Deep coral: Correlation alignment for deep domain adaptation. In ECCV Workshops, pages 443-450, 2016.
[51] R. Taori, A. Dave, V. Shankar, N. Carlini, B. Recht, and L. Schmidt. Measuring robustness to natural distribution shifts in image classification. In NeurIPS, 2020.
[52] R. Tiwari, K. Killamsetty, R. K. Iyer, and P. Shenoy. GCR: gradient coreset based replay buffer selection for continual learning. In CVPR, pages 99–108, 2022.
[53] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In IROS, 2017.
[54] D. Wang, E. Shelhamer, S. Liu, B. A. Olshausen, and T. Darrell. Tent: Fully test-time adaptation by entropy minimization. In ICLR, 2021.
[55] H. Wang, H. He, and D. Katabi. Continuously indexed domain adaptation. In ICML, volume 119, pages 9898-9907, 2020.
[56] H. Yao, C. Choi, B. Cao, Y. Lee, P. W. Koh, and C. Finn. Wild-time: A benchmark of in-the-wild distribution shift over time. In NeurIPS, 2022.
[57] H. Yao, Y. Wang, S. Li, L. Zhang, W. Liang, J. Zou, and C. Finn. Improving out-of-distribution robustness via selective augmentation. In ICML, pages 25407-25437, 2022.
[58] L. Yuan, B. Xie, and S. Li. Robust test-time adaptation in dynamic scenarios. In CVPR, pages 15922-15932, 2023.
[59] Q. Zeng, W. Wang, F. Zhou, C. Ling, and B. Wang. Foresee what you will learn: Data augmentation for domain generalization in non-stationary environments. arXiv:2301.07845, 2023.
[60] F. Zenke, B. Poole, and S. Ganguli. Continual learning through synaptic intelligence. In ICML, pages 3987-3995, 2017.
[61] H. Zhang, M. Cissé, Y. N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk minimization. In ICLR, 2018.
[62] K. Zhou, Z. Liu, Y. Qiao, T. Xiang, and C. C. Loy. Domain generalization: A survey. TPAMI, 45(4):4396-4415, 2023.
[63] K. Zhou, Y. Yang, Y. Qiao, and T. Xiang. Domain generalization with mixstyle. In ICLR, 2021.
A. Broader Impacts & Limitations
B. Ethics Statement & Licenses
C. Algorithm of EvoS
D. Experimental Setup Details
E. Experimental Results with Error Bars
F. Complexity Analysis
G. Additional Results
A Broader Impacts & Limitations
Our work focuses on the problem of continual domain generalization over temporal drift (CDGTD), which aims to generalize the model to unseen future domains by leveraging underlying evolutionary patterns. The effectiveness of our method on several real-world datasets means that it may potentially benefit relevant applications and communities that deal with temporal drifts, e.g., advertisement recommendation, autonomous driving, popularity forecast of media content, etc. Nevertheless, we should also be cautious about possible failures of our method when encountering sudden distribution shifts. In the future, we may explore the automatic identification of domains with severe distribution shifts, allowing us to proactively reject them and mitigate the risk of severe accidents.
B Ethics Statement & Licenses
All the datasets used in the paper are publicly available and are only intended to compare the performances of different algorithms on classification tasks, adhering to the following licenses of these datasets:
Yearbook: MIT licensed
- MNIST: CC BY-SA 3.0
- fMoW: The Functional Map of the World Challenge Public License
Huffpost: CC0: Public Domain
arXiv: CC0: Public Domain
C Algorithm of EvoS
The training and inference procedure for EvoS is provided in Algorithm 1 and 2, respectively.
D Experimental Setup Details
D.1 Dataset Description
Yearbook used in our paper is from [56]. It is based on the Portraits [16] dataset (MIT license), containing the  $32 \times 32$  grayscale images of yearbook portraits from 128 American high schools in 27 states. The data spanning from 1930 to 2013 reflects the evolving fashion styles and shifting social norms throughout the decades. The task is the binary classification of genders. Note that it is only used for comparing the generalization performances of different algorithms on classification tasks. For this dataset, we treat every four years as a domain, resulting in 16 domains. Table 11 provides the number of samples in each domain. The first 16 domains are used for training ( $T = 16$ ), the last 5 domains for testing ( $K = 5$ ).
RMNIST is a variant of the MNIST [13] dataset, which comprises 9 domains generated by applying the rotations with degree of  $0^{\circ}$ ,  $10^{\circ}$ ,  $\dots$ ,  $80^{\circ}$  in order to MNIST to simulate the temporal drift. The task is to classify a  $28 \times 28$  grayscale digit image from 0 to 9. We use the first 6 domains for training  $(T = 6)$  and the last 3 for testing  $(K = 3)$ .
Algorithm 1: Training procedure for EvoS
Input: sequential labeled training domains  $\{\mathcal{D}^1,\mathcal{D}^2,\dots ,\mathcal{D}^T\}$  , hyper-parameters  $\alpha ,\lambda ,W$  feature encoder  $\mathcal{E}$  classifier  $\mathcal{C}$  , learnable statistic vectors  $\hat{\mu}^{1},\hat{\boldsymbol{a}}^{1},\hat{\boldsymbol{\mu}}^{2},\hat{\boldsymbol{a}}^{2}$  , multi-head self attention modules  $\{\mathcal{A}_w\}_{w = 1}^W$  , memory pool  $\mathcal{M}$  , batch size  $B$  , training steps  $E$    
for  $t = 1$  to  $T$  do   
/ train on t-th domain   
for  $e = 1$  to  $E$  do   
Randomly sample a batch of data  $\{\pmb {x}_i^t,y_i^t\}_{i = 1}^B$  from domain  $\mathcal{D}^t$  .   
Obtain features  $\{\pmb {f}_i^t\}_{i = 1}^B$  by feeding  $\{\pmb {x}_i^t\}_{i = 1}^B$  into the feature encoder  $\mathcal{E}$  -   
 $\mathcal{L}_{total}^{t} = 0.$    
/ multi-scale attention module   
for  $w = 1$  to  $W$  do if  $t\geq w + 2$  then Prepare the input  $\ddot{\pmb{S}}_w^{t - 1} = [\dot{\pmb{s}}_w^1;\ddot{\pmb{s}}_w^2;\dots ;\ddot{\pmb{s}}_w^{t - w}]$  for  $\mathcal{A}_w$  by using a sliding time window with length  $w$  on the memory pool  $\mathcal{M}$  , as described in Eq. 4. Obtain the output  $\ddot{\pmb{S}}_w^t = \mathcal{A}_w(\ddot{\pmb{S}}_w^{t - 1}) = [\ddot{\pmb{s}}_w^1;\ddot{\pmb{s}}_w^2;\dots ;\ddot{\pmb{s}}_w^{t - w}]$  Compute the average of output tokens:  $\ddot{s}_w^{t - w + 1} = avg(\ddot{S}_w^t) = \frac{1}{t - w}\sum_{i = 1}^{t - w}\ddot{s}_w^i$  Split  $\ddot{s}_w^{t - w + 1}$  into  $w$  parts:  $[\hat{s}_w^{t - w + 1},\dots ,\hat{s}_w^{t - 1},\hat{s}_w^t ] = \bar{s}_w^{t - w + 1}$  Calculate loss  $\mathcal{L}_{con}^{t,w}$  by Eq. 11 and  $\mathcal{L}_{total}^{t} + = \mathcal{L}_{con}^{t,w}$    
/\* calculate losses   
if  $t > = 3$  then Generate the statistics for current domain as  $[\hat{\mu}^t,\hat{\pmb{a}}^t] = \hat{\pmb{s}}^t = \frac{1}{W}\sum_{w = 1}^W\hat{\pmb{s}}_w^t$  Calculate loss  $\mathcal{L}_{ce}^{t}$  by Eq. 8 and  $\mathcal{L}_{stan}^{t}$  by Eq. 10 and  $\mathcal{L}_{total}^{t} + = (\mathcal{L}_{ce}^{t} + \mathcal{L}_{stan}^{t})$  if  $t\geq 2$  then Sample features  $\{\pmb {f}'_1^m,\dots ,\pmb {f}'_B^m\}_{m = 1}^{t - 1}$  by Eq. 12 as the proxy of historical domains. Calculate loss  $\mathcal{L}_{adv}^{t}$  by Eq. 13 and  $\mathcal{L}_{total}^{t} + = \lambda \mathcal{L}_{adv}^{t}$  Optimize corresponding modules by minimizing loss  $\mathcal{L}_{total}^{t}$    
/\* store statistics into the memory pool M   
if  $t > = 3$  then Generate statistics  $\hat{\mu}^t,\hat{\pmb{a}}^t$  by repeating step 8 to 17. Store the learned/generated statistics into the memory pool  $\mathcal{M}$  by  $\mu^t,\pmb {a}^t\gets \hat{\mu}^t,\hat{\pmb{a}}^t$    
return Final E, C,  $\{\mathcal{A}_w\}_{w = 1}^W$  , M.
fMoW used in our paper is from [56]. It collects the  $224 \times 224$  RGB satellite images from 2002 to 2017 over 200 countries, where the visual features present in satellite data undergo changes over time due to both human and environmental activities. The task to classify the functional purpose of the buildings or the land in the images into one of 62 categories. For this dataset, we treat every year as a domain and Table 12 provides the number of samples in each domain. The first 13 domains are used for training ( $T = 13$ ) and the last 3 domains are used for testing ( $K = 3$ ).
Huffman in [56] comprises news headlines from the Huffington Post from 2012 to 2018, the task of which is to classify the news headline into one of 11 news categories ("Black Voices", "Business", "Comedy", "Crime", "Entertainment", "Impact", "Queer Voices", "Science", "Sports", "Tech", "Travel"). The data spanning for 2012 to 2018 presents changes in the content or style of news along the time dimension. For this dataset, we use the first 4 years for training  $(T = 4)$  and the last 3 years for testing  $(K = 3)$ . The number of samples in each domain is provided in Table 13.
Arxiv in [56] contains paper titles and their corresponding primary categories spanning from 2007 to 2022. The content of Arxiv preprints evolves over time, reflecting the dynamic nature of research fields. And the task is to classify a research paper into one of 172 categories based on its title. We use the data from the first 9 domains for training  $(T = 9)$  and the data from the last 7 years for testing  $(K = 7)$ . The number of samples in each domain is presented in Table 14.
Algorithm 2: Inference procedure for EvoS
Input: sequential target domains  $\{\mathcal{D}^{T + 1},\mathcal{D}^{T + 2},\dots ,\mathcal{D}^{T + K}\}$  , feature encoder  $\mathcal{E}$  , classifier  $\mathcal{C}$  multi-head self attention modules  $\{\mathcal{A}_w\}_{w = 1}^W$  , memory pool  $\mathcal{M}$    
for  $t = T + 1$  to  $T + K$  do   
/\* generate statistics via multi-scale attention module \*/   
for  $w = 1$  to  $W$  do   
Prepare the input  $\ddot{S}_w^{t - 1} = [\ddot{s}_w^1;\ddot{s}_w^2;\dots ;\ddot{s}_w^{t - w}]$  for  $\mathcal{A}_w$  by using a sliding time window with length  $w$  on the memory pool  $\mathcal{M}$  , as described in Eq. 4.   
Obtain the output  $\check{S}_w^t = \mathcal{A}_w(\check{S}_w^{t - 1}) = [\check{s}_w^1;\check{s}_w^2;\dots ;\check{s}_w^{t - w}]$  .   
Compute the average of output tokens:  $\check{s}_w^{t - w + 1} = avg(\check{S}_w^t) = \frac{1}{t - w}\sum_{i = 1}^{t - w}\check{s}_w^i.$  Split  $\check{s}_w^{t - w + 1}$  into  $w$  parts:  $[\hat{s}_w^{t - w + 1},\dots ,\hat{s}_w^{t - 1},\hat{s}_w^t ] = \check{s}_w^{t - w + 1}.$    
Generate the statistics for current domain as  $[\hat{\mu}^t,\hat{\boldsymbol{a}}^t] = \hat{\boldsymbol{s}}^t = \frac{1}{W}\sum_{w = 1}^{W}\hat{s}_w^t.$    
/* inference on t-th domain \*/   
for  $x^{t}\in \mathcal{D}^{t}$  do   
Obtain features  $f^{t}$  by feeding  $x^{t}$  into the feature encoder  $\mathcal{E}$  .   
Conduct feature standardization by  $z^{t} = \frac{f^{t} - \hat{\mu}^{t}}{\hat{\sigma}^{t}} = \frac{f^{t} - \hat{\mu}^{t}}{\exp(\hat{a}^{t})}.$    
Generate the prediction  $\hat{y}^t$  by feeding  $z^t$  into the classifier  $\mathcal{C}$    
/* store statistics into the memory pool M \*/   
Store the generated statistics into the memory pool  $\mathcal{M}$  by  $\mu^t,\pmb {a}^t\gets \hat{\mu}^t,\hat{\pmb{a}}^t$
For each training domain of all datasets, we randomly select  $90\%$  of data as training split and  $10\%$  of data as validation split. The evaluation of generalization performance is based on the whole data of each test domain.
All experiments are implemented via PyTorch and the backbones we use mainly adhere to [56]. Yearbook uses a 4-layer convolutional network from [56] as the backbone. For multi-scale attention module (MSAM), we set the dimension of head as  $d_h = 8$  for each attention module  $\mathcal{A}_w$  and the number of heads for  $\mathcal{A}_w$  as  $w \cdot n_h$ , with  $n_h = 16, w = 1, \dots, W$ . And the Adam optimizer with  $lr = 1e - 3$  is used to optimize the model, where the batch size  $B$  is set to 64 and the training epochs of each domain are set to 50. As for the hyper-parameters, we use  $\alpha = 1.0, \lambda = 1.0, W = 3$ .
RMNIST adopts the ConvNet in [44] as the backbone, and we set the dimension of head as  $d_h = 8$  for each attention module  $\mathcal{A}_w$  and the number of heads for  $\mathcal{A}_w$  as  $w \cdot n_h$ , with  $n_h = 32, w = 1, \dots, W$ . The Adam optimizer with  $lr = 1e - 3$  is adopted for model optimization. The batch size and training epochs of each domain are set to 64 and 50, respectively. And  $\alpha = 2.0, \lambda = 1.0, W = 3$  is used.
fMoW employ the DenseNet-121 [18] pretrained on ImageNet as the backbone. Besides, we use a bottleneck layer [28] to reduce the feature dimensions into 256, and set the dimension of head as  $d_h = 8$  for each attention module  $\mathcal{A}_w$  and the number of heads for  $\mathcal{A}_w$  as  $w \cdot n_h$ , with  $n_h = 64, w = 1, \dots, W$ . Similarly, the Adam optimizer is used, where the learning rate is set to  $lr = 2e - 4$ . The batch size is set to  $B = 64$  and each training domain is trained for 25 epochs. As for the hyper-parameters, we set  $\alpha = 1.0, \lambda = 1.0, W = 3$  for fMoW dataset.
Huffpost and Arxiv use the pretrained DistilBERT base model [48] as the backbone. For MSAM, we set the dimension of head as  $d_h = 8$  for each attention module  $\mathcal{A}_w$  and the number of heads for  $\mathcal{A}_w$  as  $w \cdot n_h$ , with  $n_h = 128$ ,  $w = 1, \dots, W$ . Also the Adam optimizer is used with the learning rate  $lr = 2e - 5$  and batch size  $B = 64$ . And  $\alpha = 1.0$ ,  $\lambda = 1.0$ ,  $W = 3$  is used for the two dataset. For the Huffpost dataset, each training is trained for 50 epochs, while the training epochs of each domain are set to 5 for the Arxiv dataset.
We run each task on a single NVIDIA GeForce RTX 3090 GPU for three random trials. For baselines, we also select their hyper-parameters via the grid search using the validation splits of training domains.
Table 5: Accuracy (%) on Yearbook, RMNIST and fMoW. The best and second-best results in CDGTD setup are bolded and underlined. (Yearbook:  $K = 5$ , RMNIST:  $K = 3$ , fMoW:  $K = 3$ )
Method	Incremental training	Access multiple domains	Yearbook Accuracy (%) ↑	RMNIST Accuracy (%) ↑	fMoW Accuracy (%) ↑
\( \mathcal{D}^{T+1} \)	OOD avg.	OOD worst	\( \mathcal{D}^{T+1} \)	OOD avg.	OOD worst	\( \mathcal{D}^{T+1} \)	OOD avg.	OOD worst
Offline	X	✓	89.30 (3.15)	88.46 (2.72)	86.81 (3.06)	98.15 (0.41)	92.14 (0.91)	83.89 (1.38)	72.43 (1.31)	59.76 (1.91)	49.85 (3.04)
IRM [2]	X	✓	97.09 (0.38)	94.52 (0.41)	92.58 (0.52)	95.10 (1.98)	85.05 (3.17)	72.52 (3.81)	64.77 (1.57)	54.92 (2.16)	46.51 (3.47)
CORAL [50]	X	✓	95.94 (1.43)	91.79 (3.04)	88.84 (6.71)	93.04 (0.35)	79.10 (1.09)	62.96 (1.70)	62.14 (1.32)	51.42 (2.13)	42.19 (3.07)
Mixup [61]	X	✓	94.98 (1.84)	91.12 (1.92)	88.35 (3.45)	97.11 (0.41)	89.66 (0.99)	79.63 (1.47)	70.27 (0.91)	57.73 (1.15)	48.04 (1.43)
LISA [57]	X	✓	95.51 (0.78)	92.97 (0.57)	91.29 (0.42)	96.21 (0.51)	87.04 (1.94)	75.15 (2.43)	70.05 (0.64)	55.52 (0.96)	44.61 (1.02)
CDOT [43]	X	✓	95.17 (1.98)	92.90 (1.81)	91.46 (2.04)	97.96 (1.00)	90.19 (0.93)	79.67 (1.27)	-	-	-
CIDA [55]	X	✓	92.36 (1.22)	90.67 (1.45)	88.45 (1.69)	97.43 (1.08)	89.19 (0.89)	78.32 (1.46)	-	-	-
GI [41]	X	✓	97.42 (0.32)	96.37 (0.33)	95.73 (0.52)	97.78 (0.04)	91.00 (0.70)	82.46 (2.01)	61.62 (1.27)	50.83 (2.19)	42.78 (2.57)
LSSAE [44]	X	✓	93.93 (1.91)	92.12 (2.06)	88.75 (4.11)	96.73 (0.31)	90.36 (0.46)	82.13 (0.71)	59.15 (1.25)	48.66 (1.90)	41.38 (2.49)
IncFinetune	✓	X	96.61 (0.17)	94.72 (0.11)	93.48 (0.54)	98.62 (0.22)	92.80 (0.65)	84.61 (1.15)	65.52 (0.34)	53.99 (0.46)	45.23 (0.83)
Mixup [61]	✓	X	90.21 (1.92)	89.83 (1.58)	88.43 (1.67)	98.43 (0.12)	92.38 (0.29)	83.45 (0.34)	64.84 (0.23)	52.00 (0.37)	42.54 (0.47)
SimCLR [11]	✓	X	95.94 (0.55)	93.07 (0.67)	89.65 (1.70)	98.23 (0.24)	90.98 (0.61)	81.05 (1.12)	64.97 (0.36)	53.20 (0.59)	44.71 (0.63)
SwAV [8]	✓	X	97.37 (0.12)	94.27 (0.07)	91.44 (2.31)	98.08 (0.09)	90.85 (0.38)	80.96 (0.85)	66.47 (0.10)	54.51 (0.18)	45.29 (0.21)
EWC [22]	✓	X	97.18 (0.12)	95.12 (0.07)	93.64 (0.37)	98.56 (0.08)	92.02 (0.31)	82.80 (0.67)	66.23 (0.12)	54.55 (0.16)	45.80 (0.18)
SI [60]	✓	X	97.09 (0.09)	94.67 (0.11)	93.48 (0.21)	98.61 (0.04)	93.27 (0.15)	85.65 (1.21)	66.61 (0.05)	54.89 (0.11)	46.46 (0.15)
A-GEM [10]	✓	X	94.36 (0.69)	90.96 (0.51)	88.88 (0.29)	95.99 (0.66)	86.95 (1.63)	75.45 (2.59)	54.54 (1.98)	47.61 (2.46)	41.13 (3.57)
SGP [47]	✓	X	95.65 (0.58)	92.92 (0.45)	91.39 (0.22)	97.12 (0.37)	88.97 (0.50)	78.05 (0.95)	-	-	-
DRAIN [3]	✓	X	96.23 (0.33)	94.71 (0.45)	93.73 (0.64)	98.52 (0.07)	93.09 (0.14)	85.75 (0.24)	67.22 (0.04)	55.05 (0.09)	46.24 (0.12)
EvoS	✓	X	97.37 (0.03)	95.53 (0.36)	94.78 (0.46)	98.64 (0.02)	93.84 (0.16)	87.04 (0.36)	67.18 (0.05)	54.64 (0.11)	45.86 (0.21)
For fMoW, backbone DenseNet-121 is too big to apply full GI and DRAIN. So we apply DRAIN only to the classifier and apply GI without the fine-tuning stage.
Table 6: Accuracy (%) on Huffpost and Arxiv. The best and second-best results in CDGTD setup are bolded and underlined. (Huffpost:  $K = 3$ , Axriv:  $K = 7$ )
Method	Incremental training	Access multiple domains	Huffpost Accuracy (%) ↑	Arxiv Accuracy (%) ↑
DT+1	OOD avg.	OOD worst	DT+1	OOD avg.	OOD worst
Offline	X	✓	72.74 (0.14)	71.50 (0.56)	69.63 (0.81)	57.49 (0.15)	52.38 (0.43)	49.28 (0.67)
IRM [2]	X	✓	71.04 (0.45)	70.31 (0.67)	68.97 (0.87)	51.11 (1.04)	45.89 (2.77)	42.86 (3.98)
CORAL [50]	X	✓	71.34 (0.51)	70.08 (0.69)	68.68 (0.94)	50.98 (1.34)	45.77 (2.92)	42.71 (4.15)
Mixup [61]	X	✓	73.34 (0.02)	71.16 (0.07)	69.29 (0.12)	57.58 (0.03)	52.77 (0.19)	49.62 (0.24)
LISA [57]	X	✓	72.19 (0.06)	70.24 (0.56)	68.60 (0.91)	56.53 (0.02)	52.41 (0.11)	49.67 (0.23)
GI [41]	X	✓	68.06 (1.51)	66.32 (2.78)	64.64 (3.67)	53.43 (1.65)	49.19 (2.49)	46.13 (3.06)
IncFinetune	✓	X	73.57 (0.02)	71.98 (0.06)	69.80 (0.11)	56.22 (0.02)	52.43 (0.14)	49.37 (0.21)
Mixup [61]	✓	X	73.07 (0.08)	71.52 (0.21)	69.44 (0.29)	56.64 (0.02)	52.95 (0.06)	49.97 (0.11)
EWC [22]	✓	X	73.64 (0.02)	71.53 (0.29)	68.99 (0.46)	56.60 (0.04)	52.78 (0.07)	49.73 (0.12)
SI [60]	✓	X	72.58 (0.07)	71.50 (0.15)	69.61 (0.34)	49.98 (1.09)	47.27 (2.46)	44.77 (3.11)
A-GEM [10]	✓	X	72.23 (0.06)	71.16 (0.14)	69.10 (0.28)	52.02 (0.99)	48.91 (2.67)	46.03 (2.09)
DRAIN [3]	✓	X	73.42 (0.02)	71.75 (0.16)	69.69 (0.20)	56.04 (0.06)	52.07 (0.18)	48.97 (0.34)
EvoS	✓	X	73.42 (0.02)	72.36 (0.05)	70.19 (0.08)	56.60 (0.02)	53.15 (0.04)	50.19 (0.10)
For Huffpost and Arxiv, backbone DistilBERT-base is too big to apply the full GI and DRAIN. So we apply DRAIN only to the classifier and apply GI without the fine-tuning stage.
E Experimental Results with Error Bars
In this section, we report the mean and standard derivation (denoted as mean (std)) for each task, when running with 3 random trials. The results with error bars on Yearbook, RMNIST and fMoW are provided in Table 5, and the error bars on Huffpost and Arxiv are given in Table 6.
F Complexity Analysis
Time Complexity of MSAM. Taking one of the multi-head self-attention module  $\mathcal{A}_w$  in MSAM as an example, we denote  $d_f$  as the feature dimension of its input tokens, and  $d_h$ ,  $n_h$  and  $n_i$  denote the feature dimension of its heads, the number of its heads and the number of its input tokens, respectively. Then the time complexity of  $\mathcal{A}_w$  is  $\mathcal{O}((n_i^2 + n_i \cdot d_f) \cdot (d_h \cdot n_h))$ . Since  $n_i$  will be no larger than the number of training domains  $T$  and  $d_h \cdot n_h$  is usually set to  $d_f$  in transformers, the time complexity of MSAM can be roughly approximated as  $\mathcal{O}(W \cdot (T^2 d_f + T \cdot d_f^2))$ , where  $W$  is the number of multi-head self-attention modules in MSAM. It is roughly equivalent to the time complexity of a single multi-head self-attention layer in conventional transformers multiplied by  $W$ .