Improving Plasticity in Online Continual Learning via Collaborative Learning
Supplementary Material
8. Extra Experiments
Impact of the number of augmentation stages in DC.
As mentioned in Sec. 4, DC comprises three augmentation
stages, including one geometric distortion stage and two
RandAugment stages. In this ablation study, we aim to investigate how more or less augmentation stages will impact
the final performance. In this experiment, we generalize thenumber of augmentation stages in CCL-DC from 0 (whichis equivalent to CCL without DC) to N . For N
1, we
apply one geometric distortion stage and N 1 RandAugment stages. As shown in Fig. 6, CCL-DC performs better
when the number of augmentation stages increases. However, the training time and memory footprint also increase
with more augmentation stages. Thus, for a trade-off, we
set the number of stages to 3 in the main paper.

Figure 6. The performance of ER incorporating CCL-DC withvarying numbers of augmentation stages on CIFAR-100 (M =2k).
All numbers are averaged over 10 runs.

T-SNE visualization. Another advantage of CCL-DC is
its ability to enhance the feature discrimination of continual
learners. Fig. 10 illustrates the t-SNE visualization of thememory data’s embedding space at the end of the training.
We can see that the feature representation of the methodwith CCL-DC is more discriminative compared with thebaseline.
Classification loss curve on other baselines. In Sec. 6,
we present the classification loss curve of the model during
training and illustrate how CCL-DC can assist the model indescending deeper into the loss landscape. We present theclassification curve for the remaining baselines in Fig. 11.
With improved plasticity, for every baseline method, CCLDC can improve the training by descending deeper at theend of each task.
Independent network performance. Although the ensemble method gives extra performance at inference time,by averaging the logit output of two networks in CCL-DC,
it also doubles the computation. In some cases, computational efficiency becomes more crucial during inference.
Continual learners trained with CCL-DC are also able to doinference independently, albeit with a slight performancedrop compared with ensemble inference. Table 9 shows theaccuracy achieved through independent inference. It is evident that the performance loss in independent inference,
when compared to ensemble inference, is minimal (approximately 1%).
Performance with NCM classifier. Besides t-SNE, we
can evaluate the feature discrimination using the clustering

Figure 7. GradCAM++ visualization on the training set of
ImageNet-100 (M =5k). Shortcut learning exists in the baseline
methods despite making correct predictions.

methods. We remove the final FC classifier and use NearestClass-Mean (NCM) classifier with intermediate representations. Table 11 demonstrates that CCL-DC can greatly enhance the NCM accuracy, which evidences the capability of
CCL-DC in improving feature discrimination.

GradCAM++ visualization. Shortcut learning is anothercommonly observed issue that hinders the generalization
capability of continual learners [44]. In Fig. 7, we use GradCAM++ on the training set of ImageNet-100 (M =5k) at
the end of the training of ER and GSA. Although both ER
and GSA make correct predictions, we observed that they
focus on irrelevant objects, which indicates a tendency toward shortcut learning. Also, we can see that by integrating
CCL-DC, the shortcut learning can be greatly alleviated.

Dataset

CIFAR10

Memory Size M

CIFAR100Tiny-ImageNet

ImageNet-100

500

1000

1000

2000

5000

2000

5000

10000

5000

ER + Ours (Ind.)ER + Ours (Ens.)

65.66±2.35
66.43±2.48

73.37±1.7074.10±1.71

32.97±1.06
33.43±1.06

43.58±1.05
44.45±1.04

52.96±1.1653.81±1.1616.32±1.58
16.56±1.63

28.68±1.20
29.39±1.2337.14±0.93
37.73±0.85

41.82±1.5443.11±1.49

DER++ + Ours (Ind.)
DER++ + Ours (Ens.)

68.15±1.4068.79±1.4273.56±1.1274.25±1.1033.81±0.90
34.36±0.89

42.79±1.3843.52±1.35

52.04±0.81
52.95±0.86

11.11±1.53
10.99±1.3921.47±1.93
21.68±1.94

27.37±2.6428.01±2.4644.22±1.2545.70±1.32

ER-ACE + Ours (Ind.)
ER-ACE + Ours (Ens.)

69.35±1.24
70.08±1.3874.86±1.06
75.56±1.1436.34±1.08
37.20±1.15

44.15±1.05
45.14±1.00

52.94±0.4453.92±0.48

17.99±1.5618.32±1.4925.69±2.00
26.22±2.01

31.69±1.69
32.23±1.70

43.92±1.7145.15±1.94

OCM + Ours (Ind.)OCM + Ours (Ens.)

73.00±0.88
74.14±0.85

76.66±1.3877.66±1.46

34.02±1.2235.00±1.1542.39±1.3643.34±1.5150.19±1.3651.43±1.37

22.53±1.2823.36±1.1832.16±0.96
33.17±0.97

38.02±0.94
39.25±0.88

41.71±1.07
43.19±0.98

GSA + Ours (Ind.)
GSA + Ours (Ens.)

68.10±1.58
68.91±1.6874.78±1.2775.78±1.1635.14±1.40
35.56±1.3943.84±1.3444.74±1.32

54.29±1.1055.39±1.09

16.53±1.62
16.70±1.66

27.57±1.6128.11±1.7036.12±1.5937.13±1.75

43.27±1.05
44.28±1.16

OnPro + Ours (Ind.)
OnPro + Ours (Ens.)

73.65±2.1674.49±2.1477.84±1.33
78.64±1.4234.20±1.1234.76±1.1241.18±0.83
41.89±0.82

49.18±0.81
50.01±0.85

21.22±1.05
21.81±1.0231.13±0.71
32.00±0.72

37.30±0.9338.18±1.02

46.84±1.3347.93±1.26

Table 9. Comparison of the final average accuracy achieved through independent inference and the use of the ensemble method on fourbenchmark datasets with difference memory buffer size M . All values are averages of 10 runs.
DatasetCIFAR10

Memory Size M

CIFAR100Tiny-ImageNet

ImageNet-100

500

1000

1000

2000

5000

2000

5000

10000

5000

ER
ER + Ours

33.16±3.50
30.22±3.75

20.94±6.79
19.85±2.55

32.65±1.7843.28±1.67

22.20±2.2629.35±1.50

13.29±1.98
16.88±1.9958.38±1.69
69.56±1.5446.87±1.6053.13±0.85

40.77±2.4542.63±0.80

23.38±2.1028.48±1.50

DER++
DER++ + Ours

24.21±2.75
25.08±2.8818.42±1.8418.47±3.1234.49±4.3942.76±1.31

25.55±3.2631.13±2.41

20.01±2.8818.45±2.89

62.03±2.83
72.59±1.29

51.57±4.60
57.71±1.8049.51±3.04
50.31±2.3428.77±4.10
27.22±2.17

ER-ACE
ER-ACE + Ours

12.72±3.56
22.86±2.23

10.66±2.48
16.07±2.38

12.67±1.6235.85±1.129.11±0.78
25.84±1.96

5.92±1.09
14.21±0.85

19.12±0.63
24.10±2.00

17.14±0.67
19.14±1.91

15.59±1.24
15.14±1.60

14.11±1.19
26.02±2.33

OCM
OCM + Ours

13.68±4.25
11.59±2.24

11.63±2.62
9.18±2.03

14.99±1.55
16.69±2.36

9.16±1.75
10.07±1.37

3.76±1.16
3.99±0.78

26.12±1.6326.16±1.90

19.74±1.30
19.99±1.96

15.92±1.4715.56±1.06

3.25±0.90
8.91±1.18

GSA
GSA + Ours

25.45±2.8628.47±3.08

16.42±3.59
19.00±2.08

33.97±2.5542.41±1.4422.74±1.83
31.09±1.8612.31±2.35
16.77±1.87

27.23±2.0164.86±1.1923.61±2.26
48.23±1.28

20.58±2.09
35.79±1.46

24.53±1.5932.77±2.07

OnPro
OnPro + Ours

17.94±3.69
19.89±4.01

14.20±2.60
14.62±2.75

16.76±2.4728.93±2.1912.42±1.39
20.23±1.03

6.72±0.94
10.55±1.89

28.01±1.5928.21±1.58

23.52±1.75
20.86±1.1320.32±1.70
16.17±0.63

7.59±1.179.90±1.93

Table 10. Forgetting Measure (%, lower is better) on four benchmark datasets with difference memory buffer size M , with and without our
proposed CCL-DC scheme. The result of our method is given by the ensemble of two peer models. All values are averages of 10 runs.
MethodNCM Acc. ↑Logit Acc. ↑ER
ER + Ours

36.56±0.60
44.76±0.55

31.89±1.4544.45±1.04

ER-ACE
ER-ACE + Ours

34.91±1.0245.62±1.04

34.21±1.5345.14±1.00

OnPro
OnPro + Ours

34.32±0.95
42.82±0.67

33.33±0.93
41.89±0.82

Table 11. Final average accuracy on CIFAR-100 (M =2k), withand without NCM classifier.

9. Counterintuitive performance of ER-ACE
As shown in Table 4, ER-ACE suffers from counterintuitive
performance on plasticity, especially when the task number
is large (e.g., TinyImageNet experiments). This is because
ER-ACE employs Asymmetric Cross-Entropy loss (ACE)during the training of batch images. ACE manually masks

out the old classes for batch image training, which reduces
the feature drift of old classes and enhances ER-ACE’s stability, as stated in the original paper. However, ACE cuts
the gradient for old classes in classification loss, which limits the optimizer’s maneuverability in the final classification layer. This loss of maneuverability is significant when
there are many tasks involved, and thus we may observe
the LA close to 0 in the later stages of training. Despite
the low plasticity, ER-ACE has a good overall performance
because: (1) Memory replay partially compensates for thisloss in terms of plasticity (Learner can still learn from samples in the memory buffer), and (2) ER-ACE has higher stability. Additionally, we witness a major stability drop in
ER-ACE when incorporating CCL-DC. As indicated in the
Algorithm 1, although we also use ACE in the classification
loss of DC, we do not perform masking in the distillation
loss. The distillation between probability distributions ofpeer models retrieves some plasticity, but it also leads toextra feature drift, which hurts the stability to some extent.

10. Experiment Details
Dataset As mentioned in Sec. 5, we use four datasetsto evaluate the effectiveness of our method. The original
datasets are split into several tasks of disjoint classes. The
detailed information about dataset split and task allocation
is as follows:
CIFAR-10 [25] has 10 classes with 50,000 training samples
and 10,000 test samples. Images are sized at 32 ⇥ 32. In our
experiments, it is split into five non-overlapping tasks withtwo classes per task.
CIFAR-100 [25] has 100 classes with 50,000 training samples and 10,000 test samples. The images are 32 ⇥ 32 in
size. It is split into 10 disjoint tasks with 10 classes per
task.
TinyImageNet [26] has 200 classes with 100,000 training samples and 10,000 test samples. Images are sized at
64 ⇥ 64. It is split into 100 non-overlapping tasks with twoclasses per task.
ImageNet-100 [22] is the subset of ImageNet-1k [15] containing 100 classes. We follow [11] for the class selection.
The images are 224 ⇥ 224 in size. It is split into 10 disjoint
tasks with 10 classes per task.
Task Sequence. In online CL, some work uses a fixedtask sequence throughout all runs to evaluate the performance, for the sake of fair comparison. However, we foundthat the evaluation heavily depends on the task order. For
fair comparison, we randomize the allocation of classes totasks and the sequence of tasks using 10 fixed random seeds(for 10 runs in our experiments). This ensures our evaluation result is not biased to task difficulty.
Data augmentation for baseline methods. Data augmentation has been demonstrated to be successful in improving the performance of online CL [6, 19, 35]. However, methods benefit differently from different augmentation intensities, and some methods may gain more performance with simple augmentations instead of complicatedones. Thus, to achieve optimal performance for comparison, we involve two different augmentation strategies forbaseline methods:
1. Partial augmentation strategy. The partial augmentation is a strategy with weak augmentation. It comprises random cropping with p = 0.5, followed by random horizontal
flip with p = 0.5.
2. Full augmentation strategy. The full strategy is a superset of the partial strategy. It consists of random cropping,horizontal flipping, color jitter, and random grayscale. The(a) Performance when M = 15(b) Performance when N = 3

Figure 8. The impact of N and M in RandAugment on the performance for ER + Ours on CIFAR-100 (M =2k). As shown in the
figure, the best performance is achieved with N = 3 and M = 15.
All numbers are averaged over 10 runs.

(a) Performance when

2 =2(b) Performance when

1 = 0.5

Figure 9. The impact of 1 and 2 on the performance for ER +Ours on CIFAR-100 (M =2k). As shown in the figure, the best
performance is achieved with 1 = 0.5 and 2 = 2. All numbers
are averaged over 10 runs.

parameter of color jitter is set to (0.4, 0.4, 0.4, 0.1) with
p = 0.8, while the probability of random grayscale is 0.2.
For fair comparison, the models trained with CCL-DC
also employ the same data augmentation strategy in thebaseline loss part, as illustrated in Algorithm 1.
Hyperparameter search for baselines. For hyperparameters in the baseline methods, as indicated in Sec. 5, we
perform a hyperparameter search on CIFAR-100 (M =2k)
for the baseline methods. Table 12 shows the exhaustive listof the grid search. Note that we used the hyperparametersfrom the original OCM paper to reduce the hyperparameter search space due to computational constraints. For faircomparison, after finding the optimal hyperparameters forthe baseline methods, we apply the same hyperparameterswhen incorporating CCL-DC.
Hyperparameter search for CCL-DC. CCL-DC also
has four unique hyperparameters, including N , M in RandAugment of DC, and 1 , 2 in Eq. 9 and Eq. 10.
In CCL-DC, we use RandAugment to generate samples with different difficulty. Thus, two additional hyperparameters in RandAugment (N and M ) are involved in
CCL-DC. Since the transformation intensity (N and M ) is
highly related to the dataset instead of the baseline method,We conduct a hyperparameter search for every datasetwith ER + CCL-DC and apply the same hyperparameter
across all baseline methods when incorporating CCL-DC.
We searched in 4 settings of N and 7 settings of M (i.e.,
N = {1, 2, 3, 4} and M = {5, 7, 9, 11, 13, 15, 17}). With
our grid search, we find that (N = 3, M = 15) is the best
for CIFAR-10 and CIFAR-100. (N = 1, M = 11) achieves
the best results on Tiny-ImageNet and (N = 3, M = 11) isthe best for ImageNet-100. We also visualize some of theexperimental results on CIFAR-100, as shown in Fig. 8.
CCL-DC also comprises two hyperparameters 1 and 2
in Eq. 9 and Eq. 10. Similar to the hyperparameter searchstrategy we do for baseline hyperparameters, for each baseline method with CCL-DC, we initiate another hyperparameter search for 1 and 2 on CIFAR-100 (M =2k) and apply the hyperparameter to all of the settings. We searchedfrom 1 , 2 = {0.25, 0.5, 1, 2, 4, 8}. We visualize some experimental results in Fig 9.
Hardware and Computation. All the experiments in ourwork are conducted on NVIDIA A100 GPUs. Fig. 12 shows
the training time for each method with and without CCLDC on CIFAR-100 (M =2k).

(a) ER(b) ER + Ours(c) DER++(d) DER++ + Ours(e) ER-ACE(f) ER-ACE + Ours(g) OCM(h) OCM + Ours(i) GSA(j) GSA + Ours(k) OnPro(l) OnPro + Ours

Figure 10. T-SNE visualization of memory data at the end of training on CIFAR-100 (M =2k).

Figure 11. Classification loss curve on CIFAR-100 (M =2k). The curve is calculated on all training samples of the current task. Since
there are 10 tasks in total, the curve has 10 peaks.

Method

HP

ValuesER

optimizer
lrweight decay
momentum (for SGD)
aug. strat.

[SGD, AdamW][0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001][0, 1e-4][0, 0.9][partial, full]DER++

optimizer
lrweight decay
momentum (for SGD)
aug. strat.
alpha
beta

[SGD, AdamW][0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001][0, 1e-4][0, 0.9][partial, full][0.1, 0.2, 0.5, 1.0]
[0.5, 1.0]

ER-ACE

optimizer
lrweight decay
momentum (for SGD)
aug. strat.

[SGD, AdamW][0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001][0, 1e-4][0, 0.9][partial, full]OCM

optimizer
lrweight decayaug. strat.

[AdamW][0.001][1e-4][partial, full]

GSA

optimizer
lrweight decay
momentum (for SGD)
aug. strat.

[SGD, AdamW][0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001][0, 1e-4][0, 0.9][partial, full]OnPro

optimizer
lrweight decay
momentum (for SGD)
aug. strat.

[SGD, AdamW][0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001][0, 1e-4][0, 0.9][partial, full]

Table 12. Exhaustive list of hyperparameters searched on CIFAR-100 (M =2k).

Figure 12. Training time of each method on CIFAR-100 (M =2k).