Dealing with Synthetic Data Contamination in Online Continual Learning
Authors: Maorong Wang
Nicolas Michel
Jiafeng Mao
Toshihiko Yamasaki
Venue: NeurIPS 2024 poster (2024)
Keywords: Online Continual Learning
Image Generation
Replay-based method
Entropy Selection
Primary area: online_learning
TL;DR: Investigating the dataset contamination caused by synthetic data in Online Continual Learning, and proposing a method to alleviate the performance degradation with entropy selection and real-synthetic similarity maximization.

Abstract: Image generation has shown remarkable results in generating high-fidelity realistic images, in particular with the advancement of diffusion-based models. However, the prevalence of AI-generated images may have side effects for the machine learning community that are not clearly identified. Meanwhile, the success of deep learning in computer vision is driven by the massive dataset collected on the Internet. The extensive quantity of synthetic data being added to the Internet would become an obstacle for future researchers to collect "clean" datasets without AI-generated content. Prior research has shown that using datasets contaminated by synthetic images may result in performance degradation when used for training. In this paper, we investigate the potential impact of contaminated datasets on Online Continual Learning (CL) research. We experimentally show that contaminated datasets might hinder the training of existing online CL methods. Also, we propose Entropy Selection with Real-synthetic similarity Maximization (ESRM), a method to alleviate the performance deterioration caused by synthetic images when training online CL models. Experiments show that our method can significantly alleviate performance deterioration, especially when the contamination is severe. For reproducibility, the source code of our work is available at https://github.com/maorong-wang/ESRM.

Dealing with Synthetic Data Contamination in Online Continual Learning
Maorong Wang'
Nicolas Michel Jiafeng Mao
Toshihiko Yamasaki
The University of Tokyo Univ Gustave Eiffel, CNRS, LIGM
(ma_wang, yamasaki u.t.:attiieccmm....ooooooooooo u-tokyo..c..Jjj jc.
nicolas mi www.chel@eeee www.eeeveeeeeeeelllle...eeee mao@hal. t. u-tokyo..c..o... jp.
Abstract
Image generation has shown remarkable results in generating images, in particular with the advancement of diffusion-based models. However, the prevalence of Al-generated images may have side effects for the machine learning community that are not clearly identified. Meanwhile, the success of deep learning in computer vision is driven by the massive dataset collected on the Internet. The extensive quantity of synthetic data being added to the Inter- net would become an obstacle for future researchers to collect "clean" datasets without Al-generated content. Prior research has shown that using datasets con- taminated by synthetic images may result in performance degradation when used for training. In this paper, we investigate the potential impact of contaminated datasets on Online Continual Learning (CL) research. We experimentally show that contaminated datasets might hinder the training of existing online CL. methods. Also, we propose Entropy Selection with Real-synthetic similarity Maximization (ESRM), a method alleviate the performance deterioration caused by synthetic images when training online CL models. Experiments show that our method can significantly alleviate performance deterioration, especially when the contamina- tion is severe. For reproducibility, the source code of our work is available at https: h/pp:://git http//ggtttttt...////// hatp://cc/mooooogggaa////////// warngggwaan//..M...
realistic
Introduction
Continual Learning (CL) [10, 31, 41, 12] solves the problem learning from a sequence of ever- emerging machine-learning tasks without forgetting previously learned knowledge. Defined by learning manners, CL can be classified into two categories [4]: offine CL and online CL. In offline CL (i.e conventional CL), the learners can access the training dataset on current task multiple times before proceeding to the next task. In online CL., the training data also comes in a continual data stream, and the continual learner only sees the training data once. Besides learning manners, there are also three typical CL settings [38]: lass-Incremental Learning (TIL), Domain- Incremenaa Learning (DIL), and Class-Increment Learning (CIL). In this paper, investigate the more challenging CIL setting in the online CL manner.
Image generation with deep generative models has shown remarkable success. Thanks to denoising diffusion models [19, 34], Internet users are capable of generating high-fidelity and realistic images within several seconds. Despite the astonishing quality of those images to human eyes, research has shown that Al-generated content may be harmful when used to train machine learning models, leading to potential performance deterioration [18, 28), bias amplification [8], loss of diversity [28]. etc.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Recently, it has become a trend for researchers to collect datasets from the Internet, and synthetic data contamination would become a potential threat to the CL community. Moreover, the online CL is particularly affected as assessing the soundness of data in the online scenario is impractical. In this work, we first aim to investigate how this new form of dataset contamination might affect the existing Online CL methods. Then, we empirically observe the characteristics of synthetic data when used to train online CL models and form four observations of synthetic data properties in online CL, which might be of interest to the community. Moreover, we investigate synthetic data properties and exhibit specific differences in terms of entropy and representations when compared to real data. Guided by these properties, (ESRM), a method to alleviate the performance degradation caused by the synthetic contamination. As replay-basee method, ESRM consists of two key components: Entropy Selection (ES) and Real-synthetic similarity Maximization (RM). ES selects more realistic samples in the memory buffer to alleviate catastrophic forgetting. RM is contrastive learning based optimization strategy, that aims to alleviate the performance artifact caused by synthetic data contamination.
propose Entropy Selection with Real-synthetic similarity Maximization
The major contribution of this paper can be summarized as follows:
We investigate the potential impact of synthetic data contamination on existing online CL methods and outline four observations regarding the properties of synthetic data in continual scenarios.
We propose ESRM, method to alleviate the performance deterioration caused by synthetic data contamination.
Comprehensive experiments show that ESRM can successfully mitigate the performance deterioration caused by synthetic contamination, especially when the contamination is severe.
2 Related Work
Synthetic data contamination. Recently, diffusion models [19, 34] have achieved high-fidelity image generation and surpassed GANs in terms of image quality and diversity. Likewise, text-to- image generation based on diffusion models can generate astonishing images that faithfully follow the users' text instructions. Furthermore, these generative models demonstrate excellent extrapolation capabilities (i.e., meaningfully combining concepts that would be nearly impossible to combine in reality), such as "a photo of an astronaut riding a house". Various generative models are open-sourced to the public, and users can use these models to generate realistic images in seconds. However, while people are appreciating the new format of art and flooding the Internet with fabulous images, such synthetic images are difficult to differentiate from the real ones. Therefore, these generated images are becoming a potential source of contamination for the future datasets collected from the Internet. Research has proven that synthetic data contamination may lead to a significant performance drop when supervising machine learning models in non-continun con-contiuall scenarios [18, 28]. Also, training deep models with such contaminated dataset may give rise to bias amplification [8] and loss of diversity [28]. To tackle the issue caused by the synthetic contamination, researchers have proposed different strategies to detect the synthetic data with deep learning based detectors [30, 27J. However, the challenge brought by synthetic data contamination to the CL. community is exclusive, and is even more problematic in the online scenario since it is almost impossible to assess the quality of the training data, due to the unique challenge brought by the online setting.
Continual Learning. The mainstream CL strategies be classified into four categories: wrrameter-isolation--aae
prompt-based, and replay-based. Regularization- based methods [7, 26, 1, 23, 46] design and apply extra regularization terms to balance the learn- ing and forgetting of CL learners. Parameter-ssolation--e
regularization-basee
methods [14, 36, 37, 35, 3] tackle the CL problem by allocating task-specific parameters. Prompt-based methods [44, 43] take the idea of prompt learning and use prompt pools against catastrophic forgetting. Replay-base meth- ods [33, 5, 6, 16, 17, 45] store a small portion of historical data with memory buffer. Among all of the strategies, aplay-baser methods have prevailed in Online CL with better performance and simplicity. Early work [33] proposed Experience Replay (ERR) suggesting using a random replay buffer to alleviate catastrophic forgetting. Dark Experience Replay (DER++) [5] proposes to store the logits the memory buffer and leverage the stored logits as dark knowledge to extend ER. ER-ACE [6] is a variant of ER with asymmetric cross-entropy loss. OCM [16] alleviates catastrophic

---

Time step
Projector
Entropy Selection (ES)
Feature Extractor
Real synthetic similarity maximization (RM)
Entropy labeling
Classifier
Entropy Feedback
Labels
Cross-entropy Loss
Probability Distribution
High Entropy samples (Likely be real)
Push away
Pull together
Ignored
Low Entropy samples (Likely to be generated)
Figure 1: Overview of proposed ESRM framework for online CL. The proposed ESRM framework
has two main components: Entropy Selection (ES) and Real-synthetic similarity Maximization (RM). Motivated by Obs. and Obs. 3, ES is buffer management strategy designed to use entropy as criterion to select real samples in the memory buffer, thereby alleviating catastrophic forgetting and performance degradation caused by the contamination. RM aims to bridge the embedding gap between synthetic and real data, as noted in Obs. 4, using a contrastive learning technique.
forgetting by maximizing the mutual information between current and past representations. GSA [17] addresses the cross-task class discrimination problem with gradient self-adaption. OnPro [45] solves the shortcut learning problem with online prototype learning. In this paper, these methods used as baselines to assess the impact of synthetic data contamination.
Preliminary
3.1 Synthetic dataset generation
To simulate a dataset contaminated with synthetic data, we employed five diffusion-based models to generate synthetic counterparts of the original datasets. These twin datasets contain the same number of images and the same classes, while all images are synthetic. The models used in generation include Stable Diffusion XL [34], Stable Diffusion vl.4, Stable Diffusion v2.1, VQDM [15], and GLIDE [29]. The synthetic data contamination was simulated across four benchmark datasets used in online CL, including CIFAR-10 [24], CIFAR-100 [24], TinylmageNet [25], and ImageNet-100 100 [13, 20].
The generation of synthetic twin datasets is guided by the category names of the original dataset. For each class, we devised a simple yet effective prompt. For instance, for the class "helicopter", we employed the prompt "an image of a helicopter". After the generation by the diffusion model, adjusted the image size to match that of the original dataset.
In the experiments, design two different settings: a) all the synthetic twin datasets generated from Stable Diffusion XL, one of the state-of-the-at generative models; and b) Synthetic images are generated with the aforementioned five diffusion models, with each model contributing 20% the synthetic dataset.
For setting a), denote the generated dataset SDXL-CI0, SDXL-C100, SDXL-Tiny. and SDXL- In100, respectively, while for setting b), we denote the generated dataset as Mix-C10, Mix-MMii-0 and Mix-Tiny. Some examples of generated images and more detailed information about the synthetic twin dataset generation can be found in Appendix D.2. Notably, the generation results in Fig. 10 reveal the lack of diversity for synthetic data compared with the real data.
3.2 Simulation of synthetic data contamination
To simulate the contamination by the synthetic data, we substitute a portion of the original dataset with its synthetic twin, where P is the contamination ratio. We designate these contaminated datasets using specifie notations. For example, we denote the CIFAR-100 contaminated by SDXL-C100 as C100/SDXL. More details about the simulation of contamination included in Appendix D.3. In

---

Real data
Synthetic data
1030
Red Syntheelc
20000
Symthentc
27000
3000
Count
Count
6:00
5000
-00
2000
1000
Enfropy
Entropy
(a) ER
(b) OnPro
(a) ER
(b) OnPro
Figure 3: T-SNE visualization of the mem- ory data at the end of training on In- 100/SDXL (P 50%). For clarity, only the first 10 classes are visualized.
Figure 2: The entropy distribution of the training
dataset produced by ER and OnPro on In-100/SDXI, (P = 50%) at the end of the training.
the following sections, we will investigate the effect of synthetic contamination with these simulated datasets.
Synthetic Data Contamination in Online CL
In this section, we explore the potential impact of synthetic data contamination on the existing online CL methods and exhibit specific properties of synthetic data.
4.1 Contamination cause of performance degradation
When dealing with synthetic data contamination, is crucial to measure the impact of such con- tamination on existing methods. In that sense, we train ResNet-18 models in the online continual setting on the contaminated dataset C10/SDXL, C100/SDXL, Tiny/SDXL, and In- 100/SDXL with representative online CL methods. We observed that as the contamination ratio P increases, the performance of all existing methods drops significantly, as shown in Table 2. Detailed information about the experiment settings can be found in Sec. 6. With such experiments, following observation:
form the
Observation 1 As a source of potential contamination, synthetic data is harmful to the performance of existing online CL methods. Performance degradation increases as contamination becomes more severe.
4.2 Detecting synthetic data matters
Replay-based methods are characterized by the existence of replay buffer, which helps allevi- forgetting and implicitly improves network plasticity [42]. The quality of data in the re- play buffer intuitively influences network per- formance. Motivated by Obs. 1, we hypothesize that the presence of synthetic data in the replay buffer might degrade performance. We trained ER on the C100/SDXL dataset, with extra in- formation on the samples' synthetic status (i.e., whether the image is real or synthetic). We em- ployed two memory strategies: storing only real data in the replay buffer and storing only synthetic data. The results showed Table indicate that knowing the synthetic status and storing only real data in the replay buffer can achieve performance at high contamination ratios. For instance, at contamination ratio of P = 80%, ER achieves an accuracy of 38.13% on C100/SDXL when only real images stored in the replay buffer. This result comparable to the accuracy achieved when training on the clean CIFAR- 100 dataset (38.700 70%), which contains four times more real data. These findings also demonstrate the potential of synthetic models to enhance performance through data generation and augmentation in an online continual learning scenario.
P 70% Acc. t
P = 80% Acc.
Method
Memory strategy
Real Ouly
38.442090
38. 13.1.4
ER
Random
32.82+1.2
31.33a133
Synthetic Only
22.45:1.88
22.01
Table 1: The performance of ER with different memory strategies on C100/SDXL dataset, with different contamination ratio P.
par with the no-contamination scenario, even

---

The memory busfer plays key role in splay-based methods, and storing real samples in the memory buffer is effective against performance degrudation caused by contamination.
Observation 2
4.3 Synthetic data properties
Lower entropy distribution. One noteworthy characteristic of synthetic data is its lower entropy distribution compared real data, as observed from the perspective of continual learners. Fig. shows the entropy distribution produced by a representative method ER and a state-of-the-ar method OnPro when trained on the contaminated dataset. The values in the histogram are calculated the end of the training on the whole training dataset (In- 1hn-100/SDXL, P = 50%). From the figure, we can spot salient distribution difference between synthetic data and real data. Moreover, the synthetic samples have a peak in entropy distribution close to O. Extra entropy histogram with other baselines can be found in Appendix C.I. Thus, we conclude with another finding:
Observation 3 Compared with real data, synthetic data entropy tends to be lower.
Feature gap in the embedding space. We find another intriguing property of synthetic data in the feature embeddings. Fig. 3 shows the I-SNE [39] visualization of the memory data produced by ER and OnPro on the In- In-100/SDXL dataset. For clarity, we only visualize the embeddings of the first 10 classes. With the synthetic data contamination, the features of the synthetic samples are better clustered than the real data. The pattern of the clustering of synthetic data indicates the ease of classification, which is on par with the limited diversity of synthetic data (cf. Fig. 10), and the low entropy distribution (cf. Obs. 3). Moreover, the embeddings of real data are inferior and fail to align with the superior embeddings of synthetic samples, which explains the performance degradation of inference real test datasets. Extra visualization produced by other basclines is illustrated in Appendix C.2.
Observation 4 With the limited diversity of synthetic data, the synthetic data are better clustered than the real data, leading to a misalignment the embedding space between synthetic samples and real samples. Such misalignment likely contributes to performance deterioration.
Proposed Method
Fig. 1 presents the main framework of our proposed FSRM to alleviate the performance degradation caused by synthetic data contamination. In this section, we introduce the two components of ESRM: Entropy Selection (ES) and Real-synthetic similarity Maximization (RM). Then, we explain the whole ESRM framework.
5.1 Entropy selection
The introduced ES aims to select more representative sam- ples in the memory buffer. For replay-based methods, having high-quality samples in the memory buffer helps alleviate forgetting and achieve better overall performance, As per Obs. 2, selecting real images into the memory buffer can provide more representative and reliable features and therefore lead to better performance. Motivated by this, we propose ES, a memory management strategy. Guided by Obs 3 (synthetic data has lower entropy distributions), the core idea of ES is to select more real samples in the memory buffer based on the entropy distribution of the current batch.
sampling
A Same class with lowest entropy
04
Time stop
A
Replace
Drop low entropy samples
Figure 4: Overview of the proposed En- tropy Selection strategy. The color of the samples indicates the class, and the num- ber in the samples represents the entropy predicted by the learner.
At the beginning of the training, ES initializes an empty buffer. When a new batch comes, ES drops 50% of the batch samples with lower entropy, and stores the remaining samples, along with their entropy of the prediction. Once the buffer is full, as shown in Fig. 4, ES takes four steps to replace the elements in the buffer. Firstly, ES drops 50% of low entropy samples the incoming batch. Then, ES uses Reservoir Sampling [21, to decide whether to keep or discard the remaining incoming samples. If Reservoir Sampling decides to keep the incoming sample, it will nominate buffer sample. After that, ES checks the class of the nominated sample and chooses sample of the same class with the lowest
5

---

entropy in the memory buffer. And finally, ES replaces the chosen memory sample with the incoming sample, along with its entropy. Moreover, all entropy values the memory buffer are updated by the current model's prediction at the end of each task. The pseudo-code of ES is given in Appendix B.
5.2 Real-synthetic similarity maximization
Prior to introducing the loss function of RM, we present the network structures of our ESRM. As shown in Fig. 1, the continual learner consists of three components: a feature extractor f. a projection head 9, and classifier σ. The output dimension of the projection head g is set 128. For each sample a from incoming data stream X the projected embedding can be formulated as:
==(((x)).
(1)
As per Obs. 4, the gap in the embedding space might be cause for the performance degradation in the contamination setting. To tackle this issue, we propose RMM The main idea of RM is to maximize the cosine similarity between the features of real and synthetic data. Motivated by the seminal supervised contrastive loss [22], we propose our variant RM loss.
RM aims to maximize the similarity between two groups of data X1 and X2. We define the loss to match the similarity of samples in group X1 to group X2 as:
exples - 2p/T)
LM((X, = Σ
Σ PEP())
log
1EL. P(1)
Eaet eap(2z Z2; 2d/ 2d/T)²
X1), where 4, - E
Ia E X23 are the set of the indices of X1, X2, respectively. And P(i) = {p X : = 3i is the set of the indices of positive samples in group X2, which share the same class with Tú. is the temperature hyperparameter which is set to 0.07. Since (X,, X2) only optimize samples in the group X1, in the optimization, it is used together with
(X2, X1). Due to RM aims to maximize the inter- group similarity between the X1 group and the X2 group, we do not need to perform augmentations as shown in similar work, such as SimCLR [9] and SupCon [22]. Intuitively, the way to handle the similarity matrix is illustrated in Fig.
For an incoming batch Xnew, we use entropy criteria to split into two groups X and X of the same size. Group X includes 50% of the samples with the highest entropy, and as per Obs. 3, tend to contain more real images. On the contrary, group X=e contains low entropy samples which tend to be synthetic. To alleviate the gap in feature embeddings mentioned in Obs. 4, we maximize the inter-group similarity between X and Xnew with Cas (X X and L (X X Moreover, to align the holistic feature embeddings between the stream data X and memory data Xmem, we also applied CM X Xmmmm and LM (X : X in the loss function.
Thus, the proposed RM to alleviate the feature gap in Obs. 4 can be achieved by employing Lym:
LRM LMs(X (XTew X ) L X ) (x X ) LxrXXeem X (3)
5.3 Overall framework of ESRM
The overall framework of ESRM is shown in Fig. 1. Besides ES and RM, following [42], ESRM employs a self delf-distillation technique to alleviate the overconfidence problem of the replay-bast based methods. For the combined batch X (X X we apply the seff-distillation as:
LSDC DKL DKI ((C (aag(())))))
(4)
where (- is the Bullback-Leibler divergence, t is another temperature hyperparameter which is set to 4, of (aug(X)) (X)) is the fixed copy of &f (aug(X))) without gradient propagation, and aug(-) is the data augmentation used in the training process, with detailed information in Appendix D.5.
Thus, the total loss of ESRM can be formulated as:
(5) where = CcE=CE(d(fX)), y) (E(o((((ug(()))))loc((o y) is the cross-entropy loss, and A1 A2 are the balancing hyperparamteres. We set X1 and X = 0.5 after small hyperparameter search as illustrated in the Appendix D.6.
LESRM LCE + A, + AzCRMs

---

Dutaset
CIFARIO
CI0/SDXL
Ratio
0%
50%
70%
80%
90%
95%
ER
63,931240
60.40.1.44-3.53
57.07:s.u(-66.8
14.18xa.n(-99755
50.69:3.s0-1335 (33.244)
47.39 (1655)
DER++
64.3
60.24.2.2(-4.07)
56.11s2mx(8822)
51.62s3.3(-12.699
44,4323.0(-19.88
40.46G2m00-23.885
ERACE
19:251
16.177220044
50.70s2m(-9.49)
46 (6.864401(-13333)
41.86:2.0011..33
37.56x2
OCM
72.66a161
69.70s..22-2..900
66. 66.68t1o(-55988)
63.7921.57--8877
60.4141.30-122225
57.07
GSA
66.91s157
63.47:1x00-3:44
59.44:2.31-74477
56.60442.x(-10333
49.9342.n(-16.999
45.7742mt-21..88
OnPro
74.87a1.58
72.46:136 2.441)
68.79
66.07=1.21(-8.. (-..0)0)()88))))
62.37a1.m0-122 (22.50
56.41)25((118.66
ESRM
67.35
67. (7.95)).ee(0.6)
(7.4721.330.12)
66.81215%x-05444
64.59sL_M-22.76
60.0422.0(-7.331
Dataset
CIFAR100
C100/SDXL
Ratio
0%%
50%
70%
80%
90%
95%
ER
38.70s1.4
36.37aL.v(-2333)
32.82=1.2(-55 (2)8888
31.33atax(-7.37)
27.09s1.m(-11.0
25.56s1.20-13 (1.3..22(113114) 13.14)
DER+
37.6242.90
32.35emm2(-5.27)
28.32=2
25.57
19.5641
16.00s1o0(-21.66
ERACE
39.82.1.77
34.15:1.33(-5661
28.16:1.37-111.4 (1.66)
25.13:1.30-14.69
19.37.1.00-20.46
16.14:12x-223.688
OCM
42.01a1.77
40.21a.15(-1.80)
37.542136(-4.47)
34.78s1
31.40a..1((10061]
28.84:120(133117
GSA
42.27
39.21at.
35,21stmnt-7.066
32.64a1 84{ 84(-9.63)
27.62a1.18(114.65
23.88s18((-18.39)
OnPro
41.47a1.09
39.26et.72--222
(5.6440o0--8833)
33.20wn70(-8.222
30,20aasu(-11.22
26.77a1.Ps-14.70)
ESRM
47.72
46.57mm.2l-1.155
45.92
14148an.nn-3.244
40.99m.z0(-6773)
37.45Husu(100.27)
Dataset
Tiny
Tiny/SDXXI
Ratio P
0%
50%
70%
80%
90%
93%
ER
28.06+181
18.03-1 (.00033
13.59.t..o(-11 (o14.4..o(-11.477
11 11.294:.00-13.777
6.38-a.30(18..cm
3.88 (88-0As(-21118)
DER++
19.40s371
12.55a2.26 6.85)
9.71x141(-9.69)
45(6114
4.49:aa3(-14.99
2.81=041(-16.59)
ERACE
26.38s1.0
17.
17.04e0.90-9)
11.23aa.s0-155155
7.83a1.16-18.55
(092i1s(-222.29
2.54g015(-23.84)
OCM
31.94.1.44
25.21
10.144t.no(-11880
16.16snos(-15778
10.35east(-21.59
5.37a0Ast
GSA
25.34.1.0
15.59.2.00 9.75)
(..55..ss-122.799
9.31 20( 20116000
5.95.nas(-1) 19.39)
3.87a0.35(
21.47)
OnPro
26.38
16.92-1 (..46)6)
13,2344.m6(13.155
8.82-1 22(2166((17.66)
4.80-1.12-21.558
2.68:0.50t-23.000
ESRM
32.15s1.20
29.36:9.50-2.79)
27.81:102(-.344
25.7a1.6(-6.455)
19.09:157(-13.0)
13.02sa2v(-19113)
Dataset
In-100
In-100/SDXL
Ratio P
0%
50%
70%
80%
90%
95%
ER
33.35
30.49shs1(-2.88)
26.47:s00(-6683)
23.61auns(-9.o6
21 21.09saor(-12.22
18.544aun(-14.766
DER+
34.89,227
29. (9.988tss(-4.777)
26. (6--711
23.72a..v-111.03
20.57s160(-14.18)
17.52aam(-17.22)
BRACK
38.434145
32.96 (5(55.11)
28.99.a7(-9.38)
25.28
20.5141.:s(11.886
16.914asi(2214.
OCM
26.70s236
26.43=0.s20-0227
23.70aL.(33.00)
23.61
22.21::1o(-4.49)
20.56e1.20-6.144)
GSA
40.85;104
37.35 37.35=1.2(-3.68)
32.34:1.30-8.69)
29.47
25. 88)
21 21.894ncs(-19) 14)
OnPro
38.4721.1
70.117727002.5888
34.04
33.01etmm-55.777
29.34 -09-441)
25.1421.20-13. (43.66))))))
ESRM
39.72a1.5
39.642970-0008)
39.62sm0e(-0110)
39.53snxx(-O..
36.58:nn0(-3114)
32.86an300-66.88
Table 2: Average Accuracy (%;; higher is better) four benchmark datasets with different contami- nation ratios P. Numbers in parentheses indicate the performance degradation due to contamination compared to the clean setting. The average and deviation over five runs are reported for ImageNet-100 100 and 10 runs for other datasets.
6 Experiments
6.1 Experiment setup
Datasets. In the experiments, we used four benchmark datasets in evaluation, including CIFAR- 10/100, TinylmageNet, and ImageNet-1000 100. All of the datasets are split into tasks containing non- overlapping classes. The details about the task split are available in Appendix D.1.
Baselines. We evaluate the effectiveness of ESRM against six representative and state-of-the-art baselines, including ER [33], DER++ [5), ERACE [6], OCM [16), GSA [17], and OnPro [45].
Implementation details. We use full-width ResNet-18 18 (not pre-trained) as the backbone for all experiments. For a fair comparison, we conduct hyperparameter search CIFAR-100 (Memory Size = 5K) and apply the same hyperparameter to all settings. Stream batch size is set to 10 and memory batch size is set to 64. We do not use multiple updates trick for incoming batches as detailed in [2]. Detailed information about task allocation, hyperparameter search protocol, and data augmentation can be found in Appendix D.
Buffer size. For CIFAR- 10 and CIFAR- 100 experiments, the buffer size is set to 1,000 and 5,000, respectively. For the harder Tiny Imtt experiments, the buffer size is set to 10,000. The buffer size of ImageNet-100 100 set to 5,000. Appendix C.5 demonstrates more experiments with different memory buffer sizes.
7

---

Dataset
In-100
In- 100/DXL
Ratio P
0%
50%
70%
80%
90%
95%
ER
53.87
52.12
48. 18.38
45.2 15.21
39.24.1.58
35.27a0.9
DER++
61.1412.31
56.9941.5
53.5642.8
50.2622.06
44.08s1.89
37.9242.19
ERACE
49.25s1.94
43.83a076
37.92x1.49
33.5
26.78s1.14
22.70
OCM
22.78s1.62
19.37s0.s
a9.8
19.43sam
20.44.025
18.98:1.4
GSA
61.83at.72
58. 58.15
53.6340.46
49.55a1.25
42.35
36.1620.90
OnPro
39.28aa.84
38.98:3.16
40.07a1.72
38.45a2.00
36,4521.51
32.9441.33
ESRM
74.85:a.i
71.25:6.74
66.51
62.16x0.39
55.30:a.50
50.41s097
Table Learning Accuracy (%; higher is better) on In-100/SDXL with various contamination ratio P.
Dataset
In-100
1n-100/DDXL
Ratio P
0%
50%
70%
80%
90%
95%
39.38x3.51
42.20s1.70
45.95x1.52
48.38x2.10
47.12s1.4
47.83x2.26
DFR++
42.26.5.7
48.35.k.20
51.00
53.73
54.93 54.28
ERACE
24.21+2.7
25.22:3.29
24.2342.29
16.10.
26.26:4425
28 80
OCM
4.67=1.66
4.38-2.16
5,96:3.12
4.86=142
6,45=1.78
6.52+131
GSA
35.51.2.1
37.50+1.
40.79.3.08
41.72-094
41.83
40.55.0.56
OnPro
15.9642.21
23.12
22.87a3.1
23.8941.00
24.72s1.80
28.1742.91
ESRM
49.79x1.61
46.02.1.19
40.83x1.29
36.90.1.74
34.61
35.40.2.58
Table 4: Relative Forgetting (9;; lower is better) on In- 10-100/SDXL with various contamination ratio P.
P 70%
P 80%
70%
P 80%
Memory strategy
Method
Acc.
Acc.
Acc.
Acc.
Real Only (Idealized)
47. 47.43
46.85.076
Baseline
42.61
41.2210.65
ES (Ours)
45.92+n.22
44.48=041
w/o LSDe
44.30
42.59+0.09
Random
44.84+0.0
42.62-087
w/o LRM
44.03-0.3
42.32+0.22
Synthetic Only (Worst)
25.91
24.74-091
ESRM
45.92-0.4.
44.48.0.1
Table 5: Comparison of different memory strate- gies on C100/SDXL dataset, with different con- lamination ratio P.
Table 6: Ablation of loss functions on C100/SDXL dataset, with different contamina- tion ratio P. "Baseline" indicates ES+LCE.
6.2 Results and analysis
Final Average Accuracy. Table 2 shows the final average accuracy of learners trained with four datasets, including C10/SDXL, C100/SDXL, Tiny/SDXL, and In I0-10L/ with different contami- nation ratios P. More results on C10/Mix, C100/Mix, and Tiny/Mix are given in Appendix C.3. For can notice a significant performance drop when synthetic data contamination
the six baselines, appears. Notably, when the contamination is severe (contamination ratio P 2 70%), the performance degradation is significant. Also, it can be observed that ESRM is less impacted by synthetic data contamination. For most datasets and contamination ratio P. the ESRM performance drop is the lowest of the compared methods. This remarkably true for large values of P.
Apart from the robustness against synthetic data contamination, the absolute performance ESRM is also attractive. In most cases, ESRM outperforms the baseline methods by large margin. More interestingly, for some datasets, such as CIFAR-100 and ImageNet-100, even with an extreme contamination ratio, ESRM can still achieve substantial performance, while the performance of most baseline methods is unsatisfactory.
Plasticity and Stability Metrics. We measure the model's plasticity and stability with Learning Accuracy (LA) [32] and Relative Forgetting (RF) [42], respectively. As shown in Table 3 and 4, for baseline methods, both plasticity and stability performance are hindered by synthetic data contamination. From the model plasticity perspective, ESRM alleviates the problem with a larger plasticity. For the model stability, ESRM solves the problem of stability degradation with the presence of contamination. is notable that with RM loss and self- ditt-iisiilationn loss, ESRM implicitly trades off some model stability in favor of plasticity. Plasticity and stability performance other datasets are available in Appendix C.4.
Iamainn-ncremental Learning Results. While standard evaluation protocol in online CL., we also evaluated the performance ESRM Domain- Incremental Learning (DIL) scenarios. We conducted the experiment with the 20 coarse labels of the
I Learning (CIL) setting is the
8

---

Dataset
DIL-CIFAR20
DIL-C20/DXXL
Ratio
50%
70%
80%
ER
53,56+1442
51.91+1as(-1.65)
49.61
47.25+073(-6.11)
DER+
56,43a0.65
52.4621.19(33.97)
48.73
44.6011.M(-11.33)
OCM
56,02+1.17
53.87+0.5(-2.115)
52.694mvn(-3.33)
50.57+os0(-5.45)
GSA
53.67+2.60
76.54e1.76--2113)
47.1611.76(-6.51)
45.55e1.17(-8.12)
OnPro
31.81+1.21
30.43=0mz(-1.33)
29. 29.1920.22-2..66
27.79=0sc(-4.02)
Ours
64.27a0.46
63.35a0.00-0.92)
61.8610.54-22.411
59.94
Table 7: Final Average Accuracy (%; higher is better) on DIL-C20/SDXL dataset. Numbers in parentheses indicate the performance degradation due to synthetic contamination compared the clean setting. The average and deviation over 10 runs are reported.
Real data
Synthetic data
Synthelic Image Buller
0.8
0.7
R-0.05
P-0.8
0.4
0.3
0.0
6000
9000
Iteration
Figure 6: The percentage of synthetic data in the memory buffer throughout the training of ESRM the In- 100//O/D dataset with different contam- ination ratios (P). The average value of 5 runs plotted.
Figure 5: T-SNE visualization of memory data produced by ESRM at the end of training on the In- 100//DDD (P = 50%) dataset. For clarity, only the first 10 classes are visualized,
CIFAR-100 dataset. Since the 100 classes in CIFAR- are grouped into 20 superclasses with fine- grained classes for each superclass, we split the CIFAR- 100 dataset with 5 domain increment steps. For each step, we feed the model with the training data of a fine-grained class from each superclass. Because the model only classifies 20 coarse labels, refer to this dataset DIL HIL-CIFAR20. Similar to the simulated CIFAR 100/SDXL dataset, we replace the images in the DIL-CIFAR20 dataset with its Stable Diffusion XL. generated counterpart with contamination ratio P. per the protocol in Sec. 3.2.
Table 7 shows the final average accuracy with different contamination ratios. Notably, we adapted the CIL-specifically designed components in OnPro and GSA to the DIL scenario, and the performance suffered a decent loss. We did not report ACE results because its Asymmetric Cross Entropy (ACE) loss converges to standard cross-entropy loss in the DIL scenario, making it equivalent to vanilla ER. The experimental results show that ESRM can yield robust performance against domain shift in the DIL setting, under different synthetic contamination situations, which validates the efficiency of ESRM under DIL settings.
6.3 Ablation studies
Effect of ES. To evaluate the effect of ES, we substitute ES with three different memory strategies: random sampling, storing real data only, and storing synthetic data only. Note that storing real or synthetic data requires knowing the ground truth of image's 's synthetic status, which is not practical realistic settings. Additionally, storing only real data is an idealized case for memory management, while storing only synthetic data represents the worst-case scenario. As shown in Table 5, both ES and random reservoir sampling [21, 40] outperform the worst-case scenario by a large margin. Moreover, ES outperforms the random sampling significantly.
Effects of loss terms. We also conduct experiments to verify the effects of loss terms in Eq. As shown in Table 6, Both LSnC and LRM can benefit the final average accuracy of the classification. Furthermore, the combination the two loss terms can further improve the final accuracy, validating that both terms complement each other.

---

7 Discussions
The alleviation of feature misalignment. As mentioned in Obs. 4, baseline methods suffer performance degradation due to the mis- alignment between the inferior feature embedding of real images and the superior feature embedding of synthetic samples. Fig. 5 presents the t-SNE visualization of memory data at the end of training of ESRM on the In- In-100/SDXL dataset. Similar to Fig. 3, only the first 10 classes are visualized for clarity. Compared to ER and OnPro, the embeddings of synthetic and real samples in ESRM are better aligned, facilitated by the RM.
1.0
0.8
0.6
0.4
0.2
0.0
0.2
1.0
Training dynamics of memory buffer. To intuitively demonstrate the effect of ES, we visualized the percentage of synthetic data in the memory buffer throughout the whole training process. Fig. 6 displays the curve of the percentage of synthetic data in the memory buffer when the model is trained with ESRM on In-100/SDXL with different contamination ratios P. To generate this curve, we checked the memory buffer every 10 iterations. As shown in the figure, the percentage of synthetic data is close the contamination ratio P in the early stages of training. As training progresses, the amount of synthetic data decreases. This trend intuitively illustrates the effect of ES in selecting real samples. Furthermore, we take model trained with ESRM the end of training and use the model's entropy as the criterion to categorize the synthetic status of samples in the training dataset and generate an ROC curve, as shown Fig. We regard real data as positive and use models trained with ESRM on In- 10-1OO/SDXL (P- 50%) to categorize the samples in the training set. The AUC of the ROC curve O. 7,9998, showing the effect of the entropy criterion in discriminating real and synthetic samples.
FPR
Figure 7: The ROC curve of the model trained with ESRM on the In- 100//O//DD dataset (P 50%) in predicting the synthetic status of samples in the training dataset. Real sam- ples are regarded as positives and synthetic samples as nega- tives.
8 Conclusion
With the widespread availability of advanced generative models, the prevalence of Al- Al-generated images appears inevitable, posing a potential challenge for researchers attempting to collect datasets devoid of Al-generated content from the Internet. In this paper, examine the potential side effects of Al Al-powered image generation on the continual learning community. First, we experimentally demonstrate that synthetic data has become a potential source of data pollution. We spot a catastrophic performance loss when the contaminated datasets are used to train continual learning models. Based on experiments, we identify and summarize four typical characteristics of synthetic data when involved in the training of continual learners. Additionally, we propose ESRM, a method designed to alleviate performance deterioration, maintaining satisfactory performance even with highly contami- nated datasets. Lastly, we hope our work highlights the need for improved regulation and systematic control over generated data, such as watermarking Al-generated content before publication. Internet data is valuable resource accumulated over decades. We believe ensuring the integrity of Internet data crucial for the future soundness of AI development.
References
[1] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In ECCV, 2018.
[2] Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In NewrlPS, 2019.
[3] Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: with a network of experts. In CVPR, 2017.
Lifelong learning
[4] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In NeurIPS, 2019.
[5] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple baseline. In NeurIPS, 2020.
10

---

[6] Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change online continual learning. arXiv preprini arXiv:2104.000
2021.
[7] Arslan Chaudhry, Puneet Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In ECCV, 2018.
[8] Tianwei Chen, Yusuke Hirota, Mayu Otani, Noa Garcia, and Yuta Nakashima. Would deep generative models amplify bias in future models? arXiv preprint arXiv:2404.032-2. 2024.
[9] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597-1607, 2020.
[10] Zhiyuan Chen and Bing Liu. Lifelong machine learning. Synthesis Lectures Artificial Intelligence and Machine Learning, 12(3):1-200) 2018.
[11] Yubei Chen Chun-Hsiao Yeh. IN100pytorch: Pytorch implementation: Training resnets on imagenet-100 attpp://gii thub. http:///////eccccccccecceceaaeee www.eehhhhhhemaeeeetteeeeee -00-Pytocch, 2022.
[12] Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. continual learning survey: Defying forgetting in classification tasks. TPAMI, 44(7):3366-3385, 2021.
[13] Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255 2009.
[14] Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint 1XXv:1701.087::: 2017.
[15] Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. In CVPR, pages 10696-10706, 2022.
[16] Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. In ICML, pages 8 8109-8122 126,, 2022.
[17] Yiduo Guo, Bing Liu, and Dongyan Zhao. Dealing with cross-task task class discrimination in online continual learning. In CVPR, pages 11878-1188,,022.
[18] Ryuichiro Hataya, Han Bao, and Hiromi Arai. Will large-scale generative models corrupt future datasets? ICCV, pages 20555-20565, 2023.
[19] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurlPS, volume 33, pages 6840-6851, 2020.
[20] Saihui Hou, Xinyu Pan, Chen Change Loy. Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In CVPR, pages 831-839, 2019.
[21] David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In AAAI, volume 32, 2018
[22] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In NeurIPS, 2020.
[23] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska--arwwinkka. et al. Overcoming catastrophic forgetting in neural networks. PNAS, 114 14(13):3521-3522 2017.
[24] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
[25] Ya Le and Xuan Yang, Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.

---

[26] Sang-Wor Woo Lee, Jin- -wa Kim, Jaehyun Jun, Jung- Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In NeurlPS, 2017.
[27] Huan Liu, Zichang Tan, Chuangchuang Tan, Yunchao Wei, Jingdong Wang, and Yao Zhao Forgery-awar adaptive transformer for generalizable synthetic image detection. In CVPR, pages 10770-10780, 2024.
[28] Gonzalo Martínez, Lauren Watson, Pedro Reviriego, José Alberto Hernández, Marc Juarez, and Rik Sarkar. Towards understanding the interplay of generative artificial intelligence and the internet. In International Workshop on Epistemic Uncertainty in Artificial Intelligence, pages 59-73. 2023.
[29] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with rext-guided diffusion models. In ICML, volume 162, pages 11784-16804, 2022.
[30] Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. In CVPR, pages 24480-24489, 2023.
[31] German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
[32] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Ger- ald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv: 1910, 2018.
[33] David Rolnick, Arun Ahuja, Jonathan Schwarz. Timothy Lillicrap, and Gregory Wayne. Experi- ence replay for continual learning. In NeurlPS, 2019.
[34] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer: High- resolution image synthesis with latent diffusion models. In CVPR, pages 10684-10695, 2022.
[35] Amir Rosenfeld and John K Tsotsos. Incremental learning through deep adaptation. TPAMI, 42(3):651-663, 2018
[36] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: (666666066771 2016.
[37] Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In /CML, 2018.
[38] Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning, arXiv preprint arXiv:1904.077 077334, 2019.
[39] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 9(1), 2008
[40] Jeffrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1)337-57, 1985.
[41] Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00443 2023.
[42] Maorong Wang, Nicolas Michel, Ling Xiao, and Toshihiko Yamasaki. Improving plasticity in online continual learning via collaborative learning. arXiv preprint arXiv:2312.00600, 2023.
[43] Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen- Yu Lee, Xiaoqi Ren, Guolong Su. Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal- continual learning. In ECCV, pages 631-648, 2022.
[44] Zifeng Wang. Zizhao Zhang, Chen-Yu Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In CVPR, pages 149,149,11 2022.
12

---

[45] Yujie Wei, Jiaxin Ye, Zhizhong Huang, Junping Zhang, and Hongming Shan. Online prototype learning for online continual learning. In ICCV, pages 18764-18774, 2023.
[46] Friedemann Zenke, Ben Poole, and Surya Ganguli.. Continual learning through synaptic intelligence. In ICML, 2017.
13

---

Algorithm PyTorch-like pseudo-code of ES.
# model: continual learning model.
# criteria(): loss function in Eq. 5.
a_seen_so__farss count of images
by the buffer.
random(): function returns random values (0,1).
buffer_lad labbls: 1D Tensor of size [buffer_size] storing the labels.
# buffer_ents 1D Tensor of size [buffer_size] storing the entropy values. update_ent()> Update all entropy values in the buffer.
__ssen_so_far" for t in tasks:
for img, label in dataloader:
train the network with stream data model.train())
mem_img, nem_label mem nem_sample() c_img, c_label = concat((ingg mem, img), (label, mem_label)) combined batch loss = criteria (model (aug(c_img)) c c_label) # Eq. 5. loss loss. backward optimizer.step()
ES updates calculate the entropy criteria of stream data model.eval()
logits model (img)
prob = tmax (logits)
entropy -torch. surrcob * (occh.oogpprob)
update buffer
threshold torch puattileeentropyy 0.5) # Step in ES stream_ing = imglentropy > threshold]
Seream_label = label (entropy threshold]
stream_entropp entropy [entropy threshold]
for x, ent in zip(stream_jng img, saream_labell (arram_ent)) nominate = int (aaaddmm=((_sseen_sso..... _aa + 1)) # Step in ES if n_seen_so_fs far muffer_sizer # if the buffer is not full nominate n_seen_so_far
data replace_dataa (nominaate,,, x, y, ent)
seen_sso_fat +-
elif if nominate < buffer_sizz
neminate @aninate_class class buffer buffer_aabela labels [nominate] # Step in ES nominate clase]_ccaass____ argmin applace_data(ie x, y, ent) Step 4 in ES m_seen_so_faa +=
idx buffer_en ent [buffer_
update_ent
A Limitations
The paper investigates the potential impact of synthetic data contamination on online CL research and proposes method to alleviate side effects caused by the contamination. Nevertheless, our research has some limitations. Firstly, in the evaluation, we only use five generative models, including Stable Diffusion v1.4, Stable Diffusion v2.1, Stable Diffusion XL, VQDM, and GLIDE. Other excellent commercial generative works, such as Midjourney and DALL-E, are not included in the data generation. With limited computation//rsoorres. we could exhaust all generative methods.
Secondly, the way we produce the generated dataset is simple: we use prompts like "an image of a <class "amm>" in the generation. In reality, the users' prompts are usually more diverse. Some works use LLMs to simulate more realistic prompts, where we leave a further in-depth analysis for future research.
B Pseudo code for ES
The Pirrch-like pseudo code showing how the ES updates the memory buffer is shown in Alg. 1.
14

---

24000
Real
20300
Real
20030
Symtherti
2000
5000
Symthwte
27020
20200
Count
5050
Cwunt
6000
Count
$300
Count
4003
6020
3003
6000
9200
2000
1003
1200
Endrapy
Entrapy
Endropy
Entrapy
(a) ER
(b) DER+
(c) ERACE
(d) OCM
Roal
1000
Rest
1500
0000
Count
Count
1000
400
200
Entropy
Entropy
Entropy
GSA
(I) OnPro
(g) ESRM
Figure 8; The entropy distribution of the training set produced by all methods on In-100/SDXL P 50% at the end of the training.
Real data
Synthetic data
(a) ER
(b) DER+
(c) ERACE
(d) OCM
GSA
(I) OnPro
(g) ESRM
Figure 9: T-SNE visualization of the memory data at the end of training on In- 100/SDXL (P = 50%).
For clarity, only the first 10 classes are visualized.
C Extra Experiments
C.1 Entropy distribution of other baselines
As mentioned in Sec. we show the entropy distribution produced by other baselines, when trained on In-100/SDXLL (P 50%). Fig. 8 illustrates the entropy distribution histogram, which is calculated the end of the training on the whole contaminated dataset. Similar to the result Fig. 2, the entropy distribution of the synthetic data is saliently lower than the entropy distribution of the real data.
15

---

Dutaset
CIFARIO
C10/Mix
Ratio
0%
50%i
70%
80%
90%
95%
ER
63,931240
61.74.2.36 2.19)
1.54:204-66.39)
57.18x2.i(-6ii (
53.71:2.30--0..22 (-1.222)
22.2112xx-111.77
DER++
64.3
61.6222.77-2.66
((-6667)
57.31<a..6-7.000
53.28x4.0(-11.03)
51.68:316612.633
ERACE
19.291
58.21
55.01=2.5<-5118
51.15
49.16s3.ut-11000
07.74gaaa--22.445
OCM
72.66s161
70.13
70.
66. (0.62211m(-6.04)
65 65.08st.x(-77588
62.862135<-9.88
61.6041.<-111000
GSA
66.91s157
33.055
62(444)
60.94s1 x0(-5.977
57.51
55.844..0t-11.077
OnPro
74.87a1.58
72.45=1.90 (2.45)
69.79s1.26-5.08
67.13a1.41
64.23 10.64)
63.63 (1.24)))11124)
ESRM
67.35
08.0421.3x0066)
67.00s1.1o(0.25)
66.73stmx-0066)
64.74ans6(22661
64.38...0-22.977)
Dataset
CIFAR100
C100/Mix
Ratio
0%%
50%
70%
80%
90%
95%
ER
38.70s144
36.5421.10-2.1106
33.82=1.1t-4.880
32.33sm.7(-6.377
30. 30.24ana0(-8.444
28.36x1.20-10.33 (0)).4)
DER+
37.6242.90
35.38=t4s0-22222
31.58=19((-6.04)
29.7112
26.7541.1(-10.87)
24.78+
ERACE
39.82.1.37
35.655r.00-44.)
31.44:222(-8.38)
29.21:1.0(-10.66)
26.24.1.n0-133 133.588
(3.42:144(-16.44
OCM
42.01a1.7
39.78an.se(-2.22
37.19=119(-4.82)
35.51e..50-6.500
33.37210u(-8864)
32.01 (0000000
GSA
42.27x1.53
39. 388aL.5(-22.39)
36.72:nms(55.55)
34.27a1 50{ 5x(-8000
31.83s142(-10.kk
30.17stt8i-12.11 (-1.10
OnPro
41.47a1.09
38.7841.27-2.666
36.53ano0(44.44)
34.48an
32.08=1.0((9.399
31.061.71(100.40
ESRM
47.72
45.58
44.02mnas(3370)
43.81
41.10mn7((66.62)
38.13um,00(99559
Dataset
Tiny
Tiny/Mix
Ratio P
0%
50%
70%
80%
90%
93%
ER
25.06+188
17.28-1 7.78).78)
13.52x1..v(111 1012,1..90-11.55. 11.54)
10.26.:..0-14...
1.73-0..(-18.333
3.46:0Ast-211 21.60)
DER++
19.40s371
13.86G2444-5.55
10.56=a.7(-8.844
7.35:1.66-12.055
3.66 (-51,444)
1.59a0.a -17.8811
ERACE
26.38s1.0
18.70st23(-7.66)
13.99sas1(-12.39)
10.35st.o0-16.03
5.62ansr(-20.766
3.01
OCM
31.94:1.44
24.14et.n(-7.80)
19.1841.no-12.788
15.06soxs(-16.88
8.82-nss(-23.11
4.28smoo(-277
GSA
25.34.1.0
17.11st.4008.221
13.49:..s(-11.888
10.02 (53.3)
6.00=na6(-193 199.44)
3.26,0.8(( 22.08)
26.38
17.03-1
11.84s1a0(114554
8.82+1
3.79
1.72xexst
ESRM
32.15s1.29
27.37at20(-4.78)
25.65xL.D(-6.50)
23.52a124(-8.63)
17.29:1.6((14.86)
11.50sa744-20.. 20.65)
Table 8: Average Accuracy (%; higher is better) four benchmark datasets with different contami- nation ratios P. Numbers in parentheses indicate the performance degradation due to contamination compared to the clean setting. The average and deviation over 10 runs reported.
C.2 T-SNE visualization of other baselines
We show additional experiments of t-SNE visualization of memory data produced by other baseline methods. As shown in Fig. 9, we visualize the memory embeddings of different baselines on the In- In-100/SDXL (P 50%) dataset. For clarity, we only visualize the first 10 classes, Similar the results in Fig. 3, the synthetic data are better clustered compared with the real data. This proves the Obs. 4.
C.3 Performance on C10/Mix, C100/Mix and Tiny/Mix Dataset
As mentioned in Sec. 6, we would like to include the experiment results on C10/Mix, C100/Mix. and Tiny/Mix datasets. The final average accuracy of different methods on the dataset is included in Table 8. Similar to the results in Table 2, ESRM has less performance degradation and better performance in most cases.
C.4 Plasticity and stability performance other datasets
As mentioned in Sec. 6.2, the plasticity and stability metrics of methods on other datasets are demonstrated Table 9 and 10. Similar to the results in Table and 4, in most settings, the plasticity metric (LA) and stability metric (RF) of baseline methods drop with an increased contamination ratio P. For the model plasticity, ESRM alleviates the issue with a larger plasticity. From stability perspective, ESRM addresses the issue of stability degradation with the presence of contamination.
C.5 The impact of buffer size.
In Sec. 6.2, we evaluate the effectiveness of ESRM in limited buffer size setting. To evaluate our methods" scalability against different buffer sizes M and contamination ratio P, we compare the accuracy of different methods on C100/SDXL with different buffer sizes M, as shown in Table 11. Similar to the results in Table 2, ESRM can obtain better performance when the dataset contaminated with synthetic data.
16

---

Dutuset
CIFARIO
CI0SSDL
Ratio P
0%
50%
70%
80%
90%
95%
ER
20,05a3
79.56a2.51
74.76s3.04
74.15
72.694302
69.60s433
DER+
78
74.51
71.33ss116
67. 68.4.8
62.96622
60.45
ERACE
60.64:324
57.16x3.8
49.61.513
46.44s5.77
43.70:344
10.94:411
OCM
79.84-301
79.25x2.28
76.43+326
76.0349.1
74.41
71.11-194
GSA
77.28a3.1
74.7243.39
69. 10.165
66.5
60.17s4.51
56.57:726
OnPro
84,87-2.5
83.12.3.00
82,98.1.75
79.81-2.52
77.99.2.8
73.39.8.2
ESRM
91.36.1
88.19
85.57a1.59
83.32at.70
79.48.2.
76.09
Datuset
CIFARI00
C100/SDXL
Ratio P
0%
50%
70%
80%
90%
95%
ER
30.47z131
48.60a1.77
46.0821.69
43.19at.28
39.6621.47
37.2
+
55,91
50.80s2.2
46.42
43.1 13.177
36. 14
31,44:220
ERACE
41.29b1.72
35.83e0.47
30.06s1
26. 16.1.6
21.70a1.64
18.10
OCM
43.56-1 s6
43.30+1
40.59+1:2
38.6241.9
35.73+1.57
33.93-181
GSA
50.76s1
48.30x2.28
43.70s2.17
40.49a1.75
34.29:1.48
30,92s1.56
OnPro
41.98-144
41.58
39.36-1.36
36. 4..87
3...95
30.97
ESRM
90.85an
67.07a0.7
62.83
59.45at.00
54.3320.2
49,89
Dataset
Tiny
Tiny/SDXI.
Ratio P
0%
50%
70%
80%
90%
95%
64.44z1
56.7621.23
51.90a1.5
47 97.24
36.6321.39
24.89
+
70.28a243
64.84a1.4
61 1.2552779
58 8241.2
54.52+1.8
19.5921
ERACE
4.60ao.58
3.502045
3.00g0.50
2.80anos
2.24
1.96s0.34
OCM
+4.91
9. 9.34
7.49
6.65am9
5.43e1
4,56x
GSA
14.95.052
8.67a1.38
6.68
5.06in6i
3.19an.82
2.37:0.36
OnPro
15.82-104
11.18+1.40
9.45+1.2
8 2.
8.30~1.30
6.75
ESRM
81.47a035
73.76a0.44
66.65an.44
60.88a0.66
50.58.1.15
38.43a091
Dataset
CIFAR10
C10/Mix
Ratio P
0%
50%
70%
80%
90%
95%
ER
20.05
77.03x3.53
75.0623.60
74.57a3.1
72.41a3.1
71.6823.9
DER++
78.35:1
75.53
74.16.3.44
70.95s2.88
69.57.2.2
67.31s227
ERACE
60.642324
57.57
52.87a3.40
49.89a471
49.55
46.53
OCM
79.84
78.7543.58
76.91
75.78st.aa
73.60e2.2
72.36-2.4
GSA
77.28=511
72.38s2.9
72.11
69 66.2.8
64. 43.4.3
60.80
OnPro
84.87-2.5
82,9642.9
80.94.2.79
79.00
76.9442.5
76.17-331
ESRM
91.36x1
88.8
85.98
84.60a1.6
82.62.ase
81.1
Dataset
CIFART00
C100/Mx
Ratio P
0%
50%
70%
80%
90%
95%
ER
50.47a131
48.642199
45.45a1.37
43.30a1.20
41.36:1.29
39.28s1.44
DER++
55.91.380
$4.30x3.50
50.02a1.4
47.44:2.88
43.39.161
41.68:221
ERACE
41.29a172
37.41a1.00
33.962131
31.89a1.16
28.65e1.1
26.62ana9
OCM
43.56s1
42.41sa.
39.26s145
16.16s1.0
37.45e1.48
36. 16
GSA
50.762164
48.07+1.1
45.37a1.4
42.83a1.59
39.65e1.21
37.85=144
OnPro
41,98-144
41.6241.46
39.24.1.3
36.95+1.0
36.02+1.0
34.94 14.94.11
ESRM
70.85an95
67.53+1.08
64.042091
61.34.1.01
57.574073
54.76
Tiny
Tiny/Mix
Ratio P
0%
50%
70%
80%
90%
95%
64.44.1
55.1641.00
50.39s2.9
44 44.2441.88
32.22+1.9
22.06+1.4 74
DER++
70.28:242
64.64x1.19
62.77-216
60.53,2.12
53.28:280
46.38:273
ERACE
4.6020.58
3.0420.7
2.63a0.55
2.29
1.87a037
1.8620.5
OCM
14.91.223
8.58at.30
6.18
4.72s1.3
4.01=0.7
3.42:0.53
GSA
14.95a052
8. 2.662t.18
6.5321.00
4.90snos
2.84
2.0620.49
OnPro
15.82=1 04
9.834
7.92+1.22
6.8941.13
5.62a0a5
4.32+1.28
ESRM
81.47
74.50
66.60s1.00
60.56e9:75
48.22+1.16
34.33a112
Table 9: Learning Accuracy (%; higher is better) on contaminated datasets with various contamination
ratio P.
17

---

Datuset
CIFARIO
CIOSSDL
Ratio P
0%
50%
70%
80%
90%
95%
ER
20.30g450
23.47a3.00
65
25.71a5.14
29.74
30.08a6
DER+
17.36s202
18.644357
19.65s264
21 46x358
25.74
28. 44
ERACE
9.59
10.69.1.30
13.97
15.83:4.33
15.70.3.2
20.51:200
OCM
10.43-251
12.44.1.71
13.43+3.0
16.3949.01
18.21
18.68-3.9
GSA
16.25.262
17.41x2.9
16.50s3.44
17.10a4.2
21.34a5.0
21.12:70:
OnPro
12.07.319
13.05
15.65/206
16.88x2.16
19.12
21.94:5.57
ESRM
26.12.205
23.03
21.27
20.24.242
20.52+270
23.99s3
Dutuset
CIFARI00
C100/SDXL
Ratio P
0%
50%
70%
80%
90%
95%
ER
23.86,273
26.06a4.53
28.51
27.34x3.41
31.48a295
30.91
+
33. 53
35.7345.0
38.5669277
39.97-5.72
46.20:4.12
47.89
ERACE
9.81
12.26s199
13.86s2775
14.15a364
18,5425.33
19.61
OCM
8.35+1.0
10.46x3.0
10.93
12.61+1.00
15.41.2.4
17.60-s00
GSA
19.62.277
21.56x3.38
22.69:2.05
23.05s4.13
23.45:3.0
25.38:4.5:
OnPro
11.91-187
13.28
15.99
16.58-2.13
17.01
18.78-3.5
ESRM
32.6721
30.75a1.45
26.93
25.24
24.53a1.04
24.8521.77
Dataset
Tiny
Tiny/SDXI.
Ratio P
0%
50%
70%
80%
90%
95%
61.842265
69.0843.09
74.91a3.1
76.9422.9
83.45a2.34
86.34.227
+
52.51
80.834351
84.35
87.53.2.58
91.88 91.88:2m
94,46a112
ERACE
36.40s274
44.92a2.49
54.44z307
62.57=5.27
76. 1923.9
78.71
OCM
32.25e1
37.15
43.82a28:
49.6992.54
58.53.2w
73.77
GSA
44.78:276
55.7245.88
58.39s3.46
64.32:..04
71.39:3.4
16.41:4415
OnPro
42.81
52.63.4.1
56.83+3.2
65.1849.77
75.11
80.98-154
ESRM
61.53a136
61.35an.6
59.79s1.2
59.5522.17
65.442272
71.04.2.0
Dataset
CIFAR10
C10/Mix
Ratio P
0%
50%
70%
80%
90%
95%
ER
20.30
20.20
22.77a462
22.0525.6
25. 25.60a3.91
26.89
DER++
17.36:262
17.7622m
19.10sa8:
18.49:3.56
21.71,7.2
20.94.421
ERACE
9.59
10.6742.66
11.61a211
13.58a4.21
14.09s4.0
12,06.261
OCM
10.43a251
11.04a282
12.89
14.24s5.7
15.01
15.15
GSA
16.25x2
15.33
16.34g3.1
16. 16.2823.5
15. 15.9422.53
16.46s297
OnPro
12.07
12.7244.00
12.93.30s
13.39.
14.36x2.2
14.49-
ESRM
26.12:205
23.75x239
21.81a2.33
21.83.2.24
22.77a1.15
23.48:277
Dataset
CIFART00
C100/Mx
Ratio P
0%
50%
70%
80%
90%
95%
ER
23,862273
25.47a3.32
25.92a3.13
252.8
26.70.2.9
27.95=3.55
DER++
13.11
34.53
36.41.3.53
36. 16.96s4555
37.83.2.
39.97.23s
ERACE
9.81
11.32+1.06
14.62a375
15.3343.12
16.162343
18.34:336
OCM
8.35szon
9.87=2m1
9.9841.21
11.1243.01
13.8745/2
14.54:3
GSA
19.62s273
20.00a188
20.71
22.26x297
22.06a3.0
22.01s444
OnPro
11.91
14.28.2.00
13.79e1332
14.50.2.16
15.714511
16.36.4
ESRM
32.67e1
32.72+1.
31.342122
30.1942.0
29.94s1.44
30.192170
Tiny
Tiny/Mix
Ratio P
0%
50%
70%
90%
95%
61.84:205
39.864333
74.38
77 77.56.3.9
31.40smm
87.49
DER+
72.51a55:
78.6414.6
83.17
87.87:3.32
93.01
96.49:126
ERACE
36.40s274
43.29a2.1
48.53
55.6623.50
66.8143m
74.77
OCM
42.25=1
38.31
45.04a3m0
49.95s2.56
61.10:1.4
74.3924.0
GSA
44.78.276
51.892409
57.09k529
63.6424.54
71.41a371
81.18
OnPro
42.81446
52,024x.00
58.822.5.9
63.49 63.4994.0
76.6845m
82.61
ESRM
61,53e136
64.2141.54
62.78s1.6
62.82a2.15
67,374225
71.86
Table 10: Relative Forgetting (%;; lower better) on contaminated datasets with various contamination
ratio P.
18

---

Dataset
C100M==1k
C100/SDX (M=lk)
Ratio
50%
70%
80%
90%
95%
ER
24.92.1.1
23.25.awx(11.69
22.18-156-227777
20.63.1.( (2(-9.99))
18.96:..4(55.66)
17.79.09t(-7113)
DER++
25.86s243
21.92s1.x(-3.944
19,47at14466399
16.6921.65-99..1
13.60:10<6-122266
11.15s.3((14471
ERACE
28.22a1.0
23 (6,
19.38e9775-88884)
17.7L=a96-100.. 10.55)
13.85a(7((11..37
11.94gotr(-16.28)
OCM
28.02.074
26.54-1mm(-1448)
24.94-1m(-3.00)
24.1741.00-3.85
22.12xe6s(-5.90)
20.65-1mx(-7.37)
GSA
28.15
(0920))477-1955)
23.52at14(-4.63)
22.90za01(-5.25
19.20st.aX-8.995
16.89u1.00-11.26)
OnPro
26.92-1.1
26.31-1.90-0661
24.85-0994
23.10-1:44-3.888
20.98 (0388--0000--94
19.77-t.a0-7.15)
ESRM
27.14,079
26.57al.00-0.57)
26.17=031(-0.77)
25.05:0.30
24.43
23.35aLu7(33.79)
Dataset
C100M-22)
C100/SDXL (M=2K)
Ratio P
0%
50%
70%
80%
90%
95%
ER
32.07a1.51
30.24:115(-1.83)
27.38=141(-4.69)
25.9521.22(-.112
23.68a1.14488399
21.79aass(-10.:
+ER++
33.37a2.1
27.95=21z(-5.422)
24.60e1ss(-8.77)
21.07 12.300
17.39su06-15.90
14.15et.2(-19.222
34.30+1.48
(2.69)17(--5.61)
24.01et.Hff 10.29)
21.23
16.47-09x(-17.87
13.25xt.m(221.00
OCM
35.69s1.36
32.3991um(-3.30)
31.15=03s(4454)
28.38 28.3311.2(-7.31)
26,76gc.m(-88933
24.75eustt-10.44
GSA
35.31.1.47
32.72-u.51-2.99
28.97-121(-6.34)
26.87.1 8.44)
23-42-1 (1.8)
19.80 (555511
OnPro
33.52an.00
31.33H075(-2.199
30.02=100(33.50)
27.90aoss(-5.662
24.38a0.0f
22.58aLus(( 10.94)
ESRM
36.25sa.79
34.55s133(11700)
34.13=u0x(-2122)
33.77
31.70s0ad(-4.55)
29.21=n31(-7.04)
Table 11: Final Average Accuracy (%; higher is better) C100/SDXL, dataset with different memory size M and contamination ratio P. Numbers in parentheses indicate the performance degradation due to contamination compared to the clean setting. The average and deviation over 10 runs are reported.
SDXL-In100
Original In-100
Figure 10: Random sampled images from class "n01558993"" (Robin) in SDXL-In and original ImageNet-100 dataset. For clarity, we have cropped some backgrounds and resized the vanilla ImageNet- 100 samples.
D Implementation Details
D.1 Dataset.
As discussed Sec. 3, used four benchmark datasets evaluation, including 10/100, TinylmageNet, and ImageNet-1000 100. In the experiments, all of the datasets are split into tasks containing non-overlapping classes. The details about the task split are as follows:
CIFAR-10 [24] has ten classes with 50,000 training images and 10,000 test images. The image is 32 32 in size. The dataset is split into five disjoint tasks with classes per task.
CIFAR-100 [24] has 100 classes with 50,000 training samples and 10,000 test samples. Image size 32 32. It is split into 10 disjoint tasks with 10 classes per task.
Tiny ImageNet [25] has 200 classes, 100,000 training samples, and 10,000 samples. Image size is 64 x 64. The dataset is split into 100 non-overlapping tasks with two classes per task.
ImageNet-100 [20] is subset of ImageNet- 1k [13] dataset. It consists of 100 classes. We follow [11] for the class selection. We do not perform image resizing in generating ImageNet- 100 from the original ImageNet- 1k dataset. The dataset split into 10 disjoint classes with 10 classes per task.
D.2 Details about synthetic dataset generation.
Image size of SDXL-INI00. In the synthetic dataset generation, we manually adjust the size of generated images match that of the original dataset. For the ImageNet- 100 dataset, since the image size is not fixed, we resize the generated images to 224 x 224, to align with the training protocols.
19

---

Generative Model
Diffusion Steps
Upsumple((eeinne)) Sieps
Guidance Scale
SDL.4
50
N/A
7.5
SD2.1
N/A
7.5
SDXL
5.0
VQDM
100
N/A
7.5
GLIDE
100
27
3.0
Table 12: Hyperparameters used in image generation.
Class-wis distribution of sources for Mix-C10/CI0//iny dataset. In the main manuscript, we mentioned that the dataset of setting b) Sec. 3.1 is generated from five synthetic models: Stable Diffusion v1.4, Stable Diffusion v2.1, Stable Diffusion XL, VQDM, and GLIDE. Each method contributes 20% of the dataset in setting b). In our implementation, we ensure that this distribution is consistent across all classes in the dataset, so that each class has an equal number of images from each generation model.
Hyperparameters used in image generation. For Stable Diffusion and VQDM, we use source code and model snapshots from huggingface, as mentioned in Table 14. For Glide experiments, use the official implementation and the released model snapshots. Following the recommendation, we use the refiner Stable Diffusion XL and the upsampler in GLIDE. The diffusion steps and guidance scale hyperparameters we used are shown in Table 12. For other hyperparameters, we follow the recommendations from Huggingface and GLIDE's official implementation. We use the prompt "An image of a class_name. name." as the text guidance to generate the image and interpolate the generated image to the size of the target dataset (32 for CIFAR, 64 for TinylmageNet, and 224 for ImageNet-100),
Samples from the generated datasets. Fig. 10 shows some samples from class "n01558993 (Robin) in the SDXL--In100 dataset along with the samples from the original dataset. We can notice a significant loss of diversity in the samples from SDXI.-In100.
D.3 Details about synthetic contamination simulation.
As mentioned Sec. 3.2, we generate synthetic twins of benchmark datasets and substitute fixed portion P' of the original datasets with their synthetic counterparts. Similar to the class-wise distribution Mix-C10/C100/Ting we also conduct the mixture class-wise. For datasets setting a), we assure that the contamination ratio in each class is also P. For datasets in setting b), while maintaining a consistent class-wise contamination ratio, we also ensure that each individual synthetic model contributes 20% of the contamination in each class.
D.4 Task sequence.
In some work, the authors use fixed task sequence for fair comparison. However, the final performance largely affected by the task order. For fair comparison, we randomly assign the class to tasks and shuffle the sequence of tasks with 10 fixed random seeds. This ensures the evaluation is not biased to task difficulty.
D.5 Data augmentation.
Data augmentation is effective boosting the training of online continual learners. Methods may benefit differently from different augmentation intensities, and some methods may favor simpler augmentations instead of complicated ones. For fair comparison, it is vital to ensure all methods their optimal performance. Thus, we introduce two different augmentation strategies:
1. Partial strategy. The partial augmentation is a weaker version of augmentation, consisting of random cropping with p 0.5, followed by random horizontal flip with p 0.5.
Full strategy. The full augmentation strategy is a stronger version of augmentation. The full augmentation strategy is superset of its partial counterpart, which consists of random cropping, random horizontal flip, color jitter, and random grayscale. The parameters for color jitter are set to (0.4, 0.4, 0.4, 0.1) with 0.8. The probability of random grayscale is set to p 0.2.
20

---

We define the data augmentation strategy of each method with hyperparameter search, as detailed in Appendix D.6.
D.6 Hyperparameter search protocol.
For hyperparameters in all of the methods (except DER++ on TinylmageNer we conduct a hyperpa- rameter search on the clean CIFAR-100 dataset with a memory size of 5,000, and apply the same hyperparameter to all of the other settings. The exhaustive list of the hyperparameter search is shown in Table 13.
Special treatment for DER++. DER++ encounters a catastrophic performance defect (close to 0) when trained the TinylmageNet dataset using optimizer with momentum. Thus, applied another hyperparameter search for DER+ on Tiny and found the SGD optimizer (without Momentum) gives reasonable performance. All the experiments of DER++ TinyImageNet conducted using these new hyperparameters.
D.7 Hardware and computation.
All of the experiments conducted NVIDIA A100 GPUs. The average training time of each method on CIFAR-I00 (Memory size 5k), ImageNet- 100 (Memory size = 5k), and Tiny ImageNet (Memory size 10k) is shown in Fig. 11. The training efficiency is much faster than OCM and OnPro, while on par with the most efficient method.
D.8 Useful source code links.
For continual learning baselines, we use the codebase listed in Table 14 to reimplement baseline methods. For image generation methods, we use the Diffuser library from Hugging Face for Stable Diffusion and VQDM experiments, and we use the codebase in Table 14 for GLIDE.
CIFAR-100
Training Time (Minutes)
IN-100
Tiny-ImageNet
10a
ER
DER++
ER-ACE
OCM
GSA
OnPro
ESRM
Figure 11: The average training time of each method trained on CIFAR-100 (M=5k), IN-100 (M=5k), and Tiny (M=10k) dataset. For better readability, the values are plotted on the logarithm scale. The numbers averaged from 10 runs.
21

---

Method
HP
Values
optimizer
[SGD, AdamW]
10.1, 0.05, 0.01, 0.005, 0.001, 0,0005, 0.0001]
ER
weight decay
[0. lc-4]
momentum (for SGD)
0.9] ppartial, full]
aug.
optimizer
(SGD. AdamW]
10.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001|
weight decay
[0, le-4]
DER+
momentum (for SGD) aug. strat.
[0, 0.9]
Ipartial, full] [0.1] 1, 0.2, 0.5, 1.0] [0.5, 1.0]
alpha
beta
optimizer Ir weight decay momentum (for SGD) aug. strat.
ISGD, AdamW]
[0.1. 0.050.0, 0.01. 0.005, 0,001. 0.0005, 0.0001]
ER-ACE
[0. lc-4]
[0, 0.9]
ppartial, full]
optimizer
[AdamW]
[0.001]
OCM
weight decay aug. strat.
[lc-4]
ppartial, full]
optimizer
(SGD, AdamW]
10.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001]
GSA
weight decay momentum (for SGD) aug. strat.
[0, le-4]
[0, 0.9] [partial, full]
optimizer
(SGD, AdamW]
[0.1. 0.05, 0.01. 0.005, 0.001, 0.0005, 0.0001]
OnPro
weight decay momentum (for SGD) aug. strat.
[0, le-4]
[0. 0.9] [partial, full]
optimizer
ISGD, AdamWI
[0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001]
weight decay momentum (for SGD) aug. strat. A
[0, Ic-4]
ESRM
0.9] [partial, full]
[0.1, 0.2, 0.5, 1, 2.
[0.1, 0.2, 0.5, 1, 2]
Table 13: Exhaustive list of hyperparameters searched on CIFAR-100.
Buseline
Links
ER & ER-ACE DER++
https: //github.b con/
htupe: :/ //t.hh/bbbbbbb. con/ voon/sinngellll slb/ namoo
https: //ghtbub con/ gydpku/OCS
GSA
htaps: //pp:://ee //tttub...bb../////////.......tt con/ G
OnPro
httpa: /ttttttbttbbtteee/e// wtw.huuww.oo/weeeeeeeeeeeee http.////////1111//////eee/
Stable Diffusion & VQDM
https //g thub. www.hhaggingfacc///111
GLIDE
https: pttpa://gthe //gt:://ttttt thub. con/ con/ppenaa////p//am/aaa/aa glide text2im.
Table 14: Baselines and their source code URLs.
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: All the main claims made in both the abstract and introduction can accurately reflect the paper's contribution.
22

---

Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitation section is included in the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-tex. system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
23

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The source code of our work, along with the result of the hyperparameter search are included in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code is included in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
24

---

Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/ /ttg://dde///euuuiiiseeeee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: CooeeSumiiissee / /nips. wttpccccpppccceeeeccccccccccccct Sttpa/////iieeeeeeeuuuubiiiiii ssionnolicyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The main experimental setting is included in the paper, and all of the details are included in the source code in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Error bars are reported in the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
25

---

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Detailed information is included in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /atea///neurraeee cc http//ulllceee heoolicc/hhiiie csQuidel d ines?
Answer: [Yes]
Justification: The Code of Ethics is fully respected and obeyed in our research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
26

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have cited the used assets in the appendix. The licenses of existing assets are properly respected.
Guidelines:
existing assets. The answer NA means that the paper does
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- e-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
27

---

Justification:
We have a Readme file along with our souree code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "n7dlsw0Chs", "reviewer": "Reviewer_ttuB", "summary": "This paper investigates the negative impact of synthetic data contamination on existing online continual learning methods. An entropy selection with the real-synthetic similarity maximization method is proposed to alleviate the performance deterioration.", "questions": "Please refer to the weaknesses.", "limitations": "The limitations are discussed while the potential negative societal impact is not discussed. However, for this work, I think it is not necessary to discuss this.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Detailed analysis of synthetic data contamination and its influence on continual learning.\n2. This paper is technically clear and easy to follow.", "weaknesses": "1. The creation of simulated data is the cornerstone of this research, but there is a lack of detailed explanation on how to generate these data in the main paper.\n2. For Observation 4 \"With the limited diversity of synthetic data\", how about mixing the synthetic data from different generation models to increase the diversity, since the synthetic images on the internet also form different models? While the observation remains unchanged in this case?\n3. In order to simulate real data collection more realistically, in addition to synthetic data, new open-domain real data should also be incorporated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and valuable comments. \n\n## Weaknesses\n\n1. We would like to include more detailed information on the synthetic dataset generation process. For Stable Diffusion and VQDM, we use source code and model snapshots from huggingface, as mentioned in Table 12 in the appendix. For Glide experiments, we use the official implementation and the released model snapshots. Following the recommendation, we use the refiner in Stable Diffusion XL and the upsampler in GLIDE. The diffusion steps and guidance scale hyperparameters we use are as follows:\n\n    | Generative Model | Diffusion Steps | Upsample(Refiner) Steps | Guidance Scale |\n    |:-----------------:|:-----------------:|:----------------:|:----------------:|\n    | SD1.4            | 50              | N/A            | 7.5            |\n    | SD2.1            | 50              | N/A            | 7.5            |\n    | SDXL             | 40              | 40             | 5.0            |\n    | VQDM             | 100             | N/A            | 7.5            |\n    | GLIDE            | 100             | 27             | 3.0            |\n\n    For other hyperparameters, we follow the recommendations from Huggingface and GLIDE's official implementation. We use the prompt \"An image of a {class_name}.\" as the text guidance to generate the image and interpolate the generated image to the size of the target dataset (32 for CIFAR, 64 for TinyImageNet, and 224 for ImageNet-100). We will include such detailed information in the revised manuscript to help the reproducibility of our paper. Also, for reproducibility, we will include the source code for synthetic dataset generation in our project codebase and make it publicly available. \n\n2. As suggested in Sec. 3.1, we have two different strategies for simulating synthetic data contamination: **(a)** using data generated from SDXL only and **(b)** mixing the data generated from different generative models. In Table 3 of our paper, We have included the t-SNE visualization result on **(a)** setting to ground the observation 4. For **(b)** setting, besides the final average accuracy shown in Table 7 in the appendix, we include extra t-SNE visualization results in the rebuttal material, as introduced in the general rebuttal (3). The t-SNE visualization of feature representations given in Fig. I(a) of the rebuttal material shows that even when generating synthetic data with a mixture of generation models, a gap still exists in feature space between generated and non-generated data when training with ER. Such experiments additionally support the claims of Observation 4.\n\n\n3. Following the suggestion, we extended our experiments from Class-Incremental Learning (CIL) only to include Domain-Incremental Learning (DIL) experiments on the DIL-CIFAR20 dataset. As mentioned in the general rebuttal (2), the DIL results of ESRM outperform other baselines, showing the robustness of ESRM against synthetic contamination.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UBeElbuAr5", "reviewer": "Reviewer_pFAa", "summary": "Image generation has been showing promising achievements in the last few years, but the generative models may not be able to keep up with the distribution of the real samples. Due to low diversity, synthetic images perform poorly in downstream learning tasks. This paper tackles this problem by diversifying the sample selection using the proposed ESRM framework, which prioritizes the high-entropy samples in the experience replay memory with an RM loss function. The authors first performed experimental analysis to build their motivation in section 4 and proposed their solution to the problem in section 5. The experiments are diversified with several datasets, benchmarks, and extensive ablation studies.", "questions": "Authors can refer to the weaknesses section for major questions. Here, I listed minor points for authors reference:\n\n1. It seems there is a typo in the line 231. \n2. It could be good to highlight the best results in Tables 3 and 4, the same as Table 2.", "limitations": "I agree with the limitation the authors brought up.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is nicely written, with clear language and sufficient details.\n\n3. Section 4's experimental analysis of existing CL methods clearly motivates the proposed model. \n\n4. The authors propose a novel loss function to address the issue of low-entropy samples in CL. This approach builds upon existing supervised constructive loss techniques. Their method aims to maximize the cosine similarity between embeddings of real and synthetic samples while simultaneously minimizing the impact of low-entropy samples on the learning process.\n\n4. In the ablation study, the authors studied different components of the proposed framework that show the superiority of the overall ESRM.", "weaknesses": "1. The paper only deals with diffusion generative models. I understand that diffusion models are the most powerful image generators but it would be good to experiment other generative models.\n\n2. The accuracy scores are overall low in CL. Does the proposed method \n\n3. Why is a fixed 50% dropping rate selected for ES? \n\n4. Downstream ML tasks have been explored with synthetic data to validate synthetic datasets, and related works clearly miss that angle. The authors should also discuss them and maybe add some of them to benchmarks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and valuable comments. \n\n## Weaknesses\n\n1. In our experiments, we prioritize the importance of diffusion-based contamination. There are two reasons: Firstly, training with synthetic images generated with GAN-based or autoencoder-based methods yields catastrophic performance, especially when the contamination is severe. Another reason is that we find GAN-based or autoencoder-based synthetic images are relatively easy to distinguish from their real counterparts, which hinders them from constituting a de facto “contamination”. For example, in our preliminary experiments, we tried to use GALIP [1], a state-of-the-art text-to-image GAN-based generative method to generate images with our text prompt, as shown in the general rebuttal (6). Fig. III in the rebuttal material shows the GALIP is subject to an extremely limited diversity problem. While we prioritize diffusion-based contamination, we regard using LLM-enhanced prompts to diversify GALIP-generated images as an important future direction.\n\n2. In this work, we focus on the online Continual Learning (CL) setting specifically, where the synthetic contamination is a more factual and severe issue because the online setting restricts us from assessing the quality of the training data beforehand. Unfortunately, due to the online restrictions, there is still a salient performance gap between the performance of conventional CL and the state-of-the-art online CL. Also, it is noteworthy that ESRM could achieve a clean performance that is on par with the current state-of-the-art methods (OCM, GSA, and OnPro). We hope our research sheds light on the research field and helps narrow down the gap. \n\n3. Intuitively and factually, the dropping rate in ES is a hyperparameter that is affected by the contamination ratio of the training dataset. There are two reasons to choose a fixed (50%) dropping rate. **(a)** We assume the continual learner does not have prior information about the contamination ratio of the dataset, which is more realistic in the real world. This is also the reason why we do not and should not perform a hyperparameter search for the dropping rate for each contamination ratio. **(b)** A drop rate of 50% gives competitive performance even under extreme conditions (Contamination ratio = 95%). We believe that the obtained performances demonstrate that a fixed ES dropping rate is resilient to different conditions (dataset, contamination ratio).\n\n4. There is indeed excellent work on validating synthetic data, for example, UniFD [2], FatFormer [3], etc. We will include such discussions in the related work section in the revised manuscript. Experimentally, we replaced the ESRM's entropy selection strategy with the pretrained UniFD synthetic data detector, as illustrated in the general rebuttal (5). With our test, the accuracy of the UniFD detector is 66.19% on the C100/SDXL dataset. Table II in the rebuttal material shows that UniFD is more effective than random selection, but its performance is still limited. This is due to a distribution mismatch between our dataset and the dataset used to train the detectors. Moreover, due to the online constraint, we had to set the threshold of UniFD at 0.5, because we can not search the optimal parameter without the knowledge of the dataset distribution.\n\n## Questions\n\n&emsp; Thank you for your suggestions, we will carefully revise the manuscript and update the tables for a better readability of our paper.\n\nReference: \n\n[1] \"GALIP: Generative adversarial clips for text-to-image synthesis.\" CVPR 2023\n\n[2] \"Towards universal fake image detectors that generalize across generative models.\" CVPR 2023\n\n[3] \"Forgery-aware adaptive transformer for generalizable synthetic image detection.\" CVPR 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kRSlgzx7OE", "reviewer": "Reviewer_h2ny", "summary": "This paper  investigates the impact of AI-generated images on the performance of online continual learning (CL) models. It introduces a novel method called Entropy Selection with Real-synthetic similarity Maximization (ESRM) to mitigate the negative effects of synthetic data contamination. ESRM leverages entropy-based sample selection and a contrastive learning approach to align the feature embeddings of real and synthetic data, thereby enhancing the robustness of CL models against the degradation caused by synthetic data.", "questions": "Please refer to the weaknesses above.", "limitations": "The authors have discussed the limitations in the paper.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors clearly articulate the problem, methodology, and results, enhancing the paper's accessibility and understanding for a broad readership.\n\n- The paper uniquely identifies the issue of synthetic data contamination in online continual learning, a significant challenge for the future of this field. The work has substantial implications for the ML community, offering a pioneering approach to maintaining the integrity of continual learning models in the presence of synthetic data.\n\n- This paper proposes ESRM, an innovative method that combines entropy selection and contrastive learning to mitigate the negative effects of synthetic data, demonstrating creativity in addressing this new problem.\n\n- The paper is underpinned by robust technical approaches, ensuring the quality and reliability of the proposed solution through comprehensive experimental validation.", "weaknesses": "- The paper evaluates the impact of synthetic data using a limited set of generative models. And the experiments are primarily conducted on image classification datasets.  Expanding this to include a broader range of tasks, particularly truly text-to-images, text-to-text, could strengthen the findings.\n\n- The method for generating synthetic datasets is straightforward, using simple prompts. Incorporating more complex and diverse prompts, potentially using large language models to simulate user queries, could better reflect real-world synthetic data contamination.\n\n- The reliance on entropy as a key metric for distinguishing real from synthetic data might not be universally applicable. Further exploration of this metric's effectiveness across different domains and its theoretical underpinnings could bolster the method's credibility.\n\n- While the paper provides insights into the method's effectiveness, there is limited discussion on its computational efficiency and scalability, especially when dealing with large-scale datasets or high contamination ratios. Addressing these aspects could be crucial for practical applications where computational resources and time are critical constraints.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and valuable comments. \n\n## Weaknesses\n1. We indeed tested our method with a limited set of generative models, due to the computation constraint. Also, it is true that almost all of the existing work in online Continual Learning (CL) is limited to Class-Incremental Learning (CIL) scenarios, where the major task is image classification. We fully agree the scope of the tasks can be beneficial, which we take as an important direction for future work.\n\n2. Following the suggestion, we conducted extra rebuttal experiments as shown in the general rebuttal (1). We use LLaMA to generate more diverse and complex prompts to simulate a more realistic contamination scenario. The experimental result in Table I in the rebuttal material validates the effectiveness of our method when the synthetic images are generated with LLM-enhanced prompts.\n\n3. The entropy criteria is indeed important in our work, and we would like to prove the universality of the entropy metric both empirically and theoretically.\n   Empirically, we expand the experiments with broader settings. The Domain-Incremental Learning setting and LLM enhancement setting in Table I of the rebuttal material further proved the universality of ESRM.\n   Theoretically, classifiers trained on a limited diversity dataset often overfit, resulting in confident predictions on the training data but poor generalization to new data. Such a pattern is universal and not limited to synthetic contamination. Methods like Label Smoothing [4] and Confidence Penalty [5] are proposed to alleviate such problems. In our paper, this limited diversity issue is magnified by synthetic contamination, where the generative model's output diversity remains imperfect for current AI research. With limited diversity, synthetic data are easier for the model's feature extractor to cluster (cf. Fig. 3 in the paper) and for the classifier to classify, leading to more confident (lower entropy) predictions (cf. Fig. 2 in our paper). We will include such experiments and theoretical discussions to further improve our manuscript.\n\n4. We regard computation efficiency and scalability as crucial criteria for our method, especially in online continual learning scenarios. Thus, we included the training time on the CIFAR-100 dataset in Sec. D.7 and Fig. 11 in the appendix. Following the suggestion, to show how the computation of different baselines is scaled with larger datasets, in general rebuttal (4), we include the training time of all baselines on TinyImageNet and ImageNet-100 datasets. As shown in Fig. II of the rebuttal material, the training time of our method is significantly faster than some state-of-the-art methods like OCM and OnPro, while it is on par with the most efficient method. Also, we would like to clarify that the training computation is agnostic with contamination ratio, because in the generation of the contaminated dataset, we *replace* the real images with synthetic images while not changing the size of the dataset.\n\nReference:\n\n[4] \"Rethinking the inception architecture for computer vision.\" CVPR 2016\n\n[5] \"Regularizing neural networks by penalizing confident output distributions.\" arXiv:1701.06548 (2017)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank the reviewers for their comments and suggestions, which enabled us to improve the manuscript significantly. We respond to each reviewer individually. Here, we introduce the rebuttal experiments in the attached PDF file. 

## 1. Results with LLM Enhanced Prompts
As suggested by reviewer h2ny, we expand the synthetic dataset generation from the baseline prompt ("An image of {class_name}") to the LLM-enhanced prompt. We leverage the open-source instruction-fine-tuned LLaMA-3 model with 8 billion parameters (`meta-llama/Meta-Llama-3-8B-Instruct`) to generate the enhanced prompt for CIFAR-100 dataset. For each class, 50 prompts are generated, and we use Stable Diffusion XL to generate 10 images per prompt to formulate the LLM-enhanced CIFAR-100 dataset (denoted as C100/SDXL-LLaMA). Some examples of LLM-generated prompts are as follows:
1. A juicy red oval apple with a tiny worm hole is sitting on the wooden kitchen counter.
2. A vibrant orange tropical aquarium fish with iridescent scales swims lazily in a large glass tank on the sunlit windowsill.

As shown in Table I in the rebuttal material, for all baseline methods, when the synthetic dataset is generated with LLM-enhanced prompts, the performance deterioration of synthetic contamination is still significant. Even with such advanced prompts, ESRM can significantly alleviate the performance deterioration while achieving satisfactory results. Notably, LLM-enhanced results (C100/SDXL-LLaMA in Table I of the rebuttal material) sometimes bring performance drops compared with non-LLM-enhanced results (C100/SDXL in Table 2 of our paper). This is because the LLM might introduce undesired items into the language prompt.

## 2. Domain-Incremental Learning (DIL) results
While our current experiment mainly focuses on the Class-Incremental Learning (CIL) setting of online Continual Learning, we include extra experiments in Domain-Incremental Learning (DIL) scenarios, following the suggestion from reviewers ttuB and h2ny. 
We conducted the experiment with the 20 coarse labels of the CIFAR-100 dataset. Since the 100 classes in CIFAR-100 are grouped into 20 superclasses with 5 fine-grained classes for each superclass, we split the CIFAR-100 dataset with 5 domain increment steps. For each step, we feed the model with the training data of a fine-grained class for each superclass. We refer to this dataset as DIL-CIFAR20, as the model only classifies coarse labels.

Similar to the simulated CIFAR100/SDXL dataset, we replace the images in the DIL-CIFAR20 dataset with its Stable Diffusion XL generated counterpart with a contamination ratio P, as per the protocol in Sec. 3.2.

Table I of the rebuttal material shows the final average accuracy with different contamination ratios. Notably, we adapted the CIL-specifically designed components in OnPro and GSA to the DIL scenario, and the performance suffered a decent loss. We did not report ERACE results because its Asymmetric Cross Entropy (ACE) loss converges to standard cross-entropy loss in the DIL scenario, making it equivalent to vanilla ER.

The experimental results show that ESRM can yield robust performance against domain shift in the DIL setting, under different synthetic contamination situations, which validates the efficiency of ESRM under DIL settings.

## 3. Visualization on C100/Mix dataset
As suggested by reviewer ttuB, to better ground the claim in observation 4, we included extra visualization experiments on the C100/Mix dataset, where the synthetic data are generated from the mixture of different generative models. From the t-SNE visualization result in Fig. I of the rebuttal material, we can see that similar to Fig. 3 in the paper, the feature gap of the baseline method (ER) in the embedding space still holds even when the synthetic data is generated with different generative models. Also, with ESRM, the feature misalignment problem is alleviated.

## 4. Training time on other datasets
We have shown the training time on the CIFAR-100 dataset in Fig. 11 in the appendix. As suggested by reviewer h2ny, to show how different methods scale larger datasets, we include the training time of all baselines on TinyImageNet and ImageNet-100 datasets. As shown in Fig. II in the attached material, we plot the training time on the logarithm scale for better readability. The computation efficiency of ESRM is always on par with the most efficient method (ER), while significantly outperforming OCM and OnPro.

## 5. Benchmarking Pre-trained synthetic image detectors
As suggested by reviewer pFAa, we replace ES in our memory strategy with UniFD [2], a pre-trained synthetic image detector. We use the prediction of UniFD to validate the synthetic status (i.e., whether the image is real or synthetic) of the training images, and store the real images in the memory buffer. As shown in Table II of the rebuttal material, UniFD outperforms random selection but still has limited effectiveness despite leveraging extra-information from pre-training. One explanation is that it's not designed for online learning scenarios. Due to the online constraint, we set the threshold of UniFD to 0.5, instead of searching for the best threshold parameter (because we do not know the dataset distribution). Also, the domain gap between the UNiFD training set and our dataset further limits the performance.

## 6. Image Generation Results with GANs
We include the image generation result with GALIP [1], a state-of-the-art text-to-image GAN model. Following our image generation protocol, we use the prompt "An image of {class_name}" to generate synthetic images. We use the GALIP model pretrained on CC12M dataset. As shown in Fig. III in the attached material, the generation result of GALIP suffers from a huge lack of diversity problem. 

Reference: 

[1] "GALIP: Generative adversarial clips for text-to-image synthesis." CVPR 2023

[2] "Towards universal fake image detectors that generalize across generative models." CVPR 2023

Meta-review: All reviewers found this paper very novel and interesting.  The authors presented a strong response to the comments raised by reviewers.
Decision: Accept (poster)