Learning Invariant Features for Online Continual Learning
Authors: Yiduo Guo
Bing Liu
Dongyan Zhao
Venue: Submitted to ICLR 2023 (2023)
Keywords: continual learning
online continual learning

Abstract: It has been shown recently that learning only discriminative features that are sufficient to separate the classes in a task using a traditional learning method has a major shortcoming for continual learning (CL). This is because many features that are not learned may be necessary for distinguishing classes of some future tasks. When such a future task arrives, these features have to be learned by updating the network, which causes catastrophic forgetting (CF). A recent work on online CL showed that if the learning method can learn as many features as possible from each class, called holistic representations, CF can be significantly reduced to achieve a large performance gain. This paper argues that learning only holistic representations is still insufficient. The learned representations should also be invariant and those features that are present in the data but are irrelevant to the class (e.g., the background information) should be ignored for better generalization across tasks. This new condition further boosts the performance significantly. This paper proposes several strategies and a loss to learn holistic and invariant representations and evaluates their effectiveness in online CL.

Under review as a conference paper at ICLR 2023
LEARNING INVARIANT FEATURES FOR ONLINE CONTINUAL LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
It has been shown recently that learning only discriminative features sufficient to separate the classes has a major shortcoming for continual learning (CL). This is because many features that are not learned may be necessary for distinguishing classes of some future tasks. When such a future task arrives, these features have to be learned by updating the network, which causes catastrophic forgetting (CF). A recent online CL work showed that if the learning method can learn as many features as possible from each class, called holistic representations, CF can be significantly reduced to achieve a large performance gain. This paper argues that learning only holistic representations is still insufficient. The learned representa- tions should also be invariant and those features that are present in the data but are irrelevant the class (e.g., background information) should be ignored for better generalization across tasks. This new condition further boosts the performance significantly. This paper proposes several strategies and a loss learn holistic and invariant representations and evaluates their effectiveness in online CL.
INTRODUCTION
A major challenge of continual learning (CL) is forgeerin (CF) (MeCloskey & Cohen 1989). which is caused by updating network parameters learned from previous tasks in learning a new task. Although many empirical approaches have been proposed to deal with CF, limited the- oretical work has been done to study the necessary conditions for CL. to overcome CF. Recently, Guo et al. (2022) argued that it is necessary to learn holistic representations of the data. This work proposes another condition, invariance, and argues that the learning of each task itself needs to be improved so that future tasks would not need to make major changes to old parameters.
is well-known that supervised learning losses (c.g., cross-entropy) learn only discriminative fea- tures that are sufficient to separate the classes in a task. This is problematic for CL due to two main reasons. (1) many features that are not learned may be necessary to distinguish classes of some future tasks. When such future task comes, these features have be learned, which may make significant changes to the existing parameters and cause CF. (2) even if the previous parameters are completely protected, the classes in the new task still make the classification challenging because each task only learns discriminative features for its own classes, which causes confusion in clas- sification when we need classify all classes learned so far. We call this biased representation learning. For example, task-1 I learns to classify black pig and dove. The learner may only learn the color features (e.g., black and white) as they are sufficient to classify the two classes. However, task-2 learns rabbit and cow, which can be black or white. The color features learned from task-1 are no longer sufficient. Shape-based features need to be learned, which can make major changes to the existing parameters and cause CF. If the shape-based features had been learned from task- 1, learning of task 2 will not need to update the parameters linked to the representation of pigs and doves as much, which gives less CF (mitigating (1) above). Since there are now 4 classes to clas- sify, it can be confusing. Some cows or rabbits may be classified as pigs or doves due to the same color. Some pigs may be classified cows or rabbits due shape-based features learned in task-2. However, if all features have been learned in task-1 and task-2, such wrong classifications will be reduced (dealing with (2)). Recently, Guo et al. (2022) proposed learn holistic representations from the input data to cover as many characteristics of the input as possible. Their system OCM
'The code has been submitted in the supplemental material.

---

Under review as a conference paper at ICLR 2023
learns holistic representations in online CL by maximizing the mutual information (MI) between the input data and the learned feature representations to ensure that as much information in the input is reflected in the learned features. This results in major performance gain in online CL.
This paper argues that learning holistic representations is still sub-optimal. It is also necessary for CL to learn features that are invariant for each class. Those features that are present in the input but are irrelevant to the class should be ignored for better generalization to future tasks. For example, to classify images of apple and fish, some green background and red color of apple may be learned, but these features are not invariant to apple. When new classes cows and ladybird need to be learned, the feature green (or red) is shared (see Figure aT)) and may cause high logit outputs to apple and cow (or ladybird). Then the learner has to modify the representations of apple to reduce its logit values, which causes CF. That is, variant features are unsuitable for establishing decision boundaries. If the learner has learned shape and other invariant features for apple, the input of cow (or ladybird) will not activate the parameters linked to the apple representation. Then, in learning cow and ladybird, changes to the parameters that are important for apple will be limited, resulting in less CF. This paper aims to learn invariant and holistic representations for each class. This additional invariance condition gives another boost to online CL performance, Note that invariance is not critical for traditional supervised learning due to the i.idd assumption 2 but for CL, it is very important because each new task introduces new distributions and want each class to be distinguishable against any past and future classes, which may have similar variant features that can confuse the model.
This paper works in the online class-incrementee learning (CIL) setting method called IFO (Invariant Feature learning for Online CL), which adds the invariance condition to holistic representation learning to also learn invariant features. We propose two new methods and one new optimization objective to achieve invariance. The first method is to construct a diverse set of environments and force the model to learn invariant features across the environments. The second method is a novel use of the replay data to learn invariant features and to deal with a local sampling bias issue. Finally, we combine the methods and propose a new optimization goal to learn invariant features. Theoretical justifications are also given. We verify the effectiveness of IFO in three online CL scenarios: traditional disjoint task scenario, blurry task boundary scenario and data shift scenario. The results show the proposed IFO outperforms strong basclines by large margin.
P It proposes replay- Maalay
2 RELATED WORK
Although many CL approaches have been proposed, little work has been done to study the necessary
conditions for CL. The replay approach saves small amount of past data and uses it to protect/adjust
the previous knowledge in learning new task (Rebuffi et al., [2017] Wu et al. [2019]
Hou et al.) 2019
Chaudhry et al. 2020 Zhao ct al. 2021 Korycki & Krawezyk| 22tvwwckk_222]]3jppp Sokar et al.
2021
Yan et al,
2021 Wang et al. 2022a). Pseudo-reple generates replay samples (Shin et al.
2017
Hu ct al.
2019 Sokar et al.
2021). Using regularizations to penalize changes to important parameters of
previous tasks is another approach (Kirkpatrick et al.) 2017 Ritter et al. 2018 Ahn et al. 2019 Yu
et al. 2020 Zhang et al. 2020). Parameter-isolation approaches protect models of old tasks using
masks and/or network expansion (Ostapenko et al. 2019 von Oswald et al., 2020 Li et al.] 2019
Hung et al., 20199 Rajasegaran et al.) 2020; Abati et al. 2020; Wortsman et al. 2020 (aha et al.,
2021 Zhu et al. (2021) found that using data augmentations can learn more transferable features.
Online CL methods are mainly based on replay. ER randomly samples the replay data (Chaudhry ct al. 2020), MIR chooses replay samples whose losses increase most (Aljundi et al.. 2019a). ASER uses the Shapley value theory (Shim et al. 2021). and GDumb produces class balanced replay data (Prabhu et al., (202)). GSS diversifies the gradients of the replay data (Aljundi et al.) 2019b). DER++ uses knowledge distillation (Buzzega et al. 2020), SCR uses contrastive loss (Mai et al. (02T), and NCCL calibrates the network (Yin et a 2021) )). Applications of online CL are also reported (Yan et al., 2021 Wang et al., 2021). Bang et al. (02T)) and [Bang et al. (2022) proposed two blurry online CL settings. IFO is also a replay method but focuses on learning invariant features.
Domain generalization (DG) is also related. DG learns model with inputs from multiple given source domains with the same class labels and test with inputs from unseen target domains. Ex-
A When oft-of- data or data shift is involved, invariance is also important (Arjovsky et al. 2019 In the CIL setting, task related information (e.g., task-id) is provided in testing. The other popular CI setting is task-incremental learning (TII.), which needs the task-id to he provided for each test instance.
2

---

Under review as a conference paper at ICLR 2023
isting DG methods typically leverage data augmentations to expand the diversity of the source do- mains (Wang et al. 2020 Wu et al. 20200 Arjovsky et al. [2019] Rame et al., 2022). There are also works a single domain. Yang et al. (2021) used pre-traind domain augmenter to create novel stylized images. Yue et al. (2019) used auxiliary datasets to help create images. Our training data
have no identified domains and we use additional datasets. DG does not do CL..
3 PROBLEM FORMULATION
This paper uses three online CL settings. (1) traditional disjoint tasks setting, where the system learns a set of tasks incrementally one after another. Each task consists of several classes. The data for each task comes in stream and the learner sees the data only once. In replay method, when a small batch of data of the current task / arrives from the data stream Die (Xyer Y!ee") (where Xye is set of new samples and Yeee is its set of corresponding labels), a small batch of replay data Duuf -X y/trf) is sampled from the memory buffer M and used to jointly train in one iteration.
(2) blurry task setting (Koh al. 2021), where the classes from a previous task may appear again later (more details in Sec. 5.2
data environment shift setting (see details in Sec. 5.3)
Our model F consists of two parts: the feature extractor Se and the classifier ag. Se extracts features from the input x to form a high-level representation and ds and the softmax operation map the representation to prediction probability for each class.
Learning holistic representations. As discussed in Sec. fa should learn holistic feature represen- tations (Guo et al. 2022) but cross-entropy loss Lee alone is unable to do that. Geirhos et al. (2020) also found that Lee encourages models to stop learning once simple features suffice to minimize the
loss. This means fo may learn simple discriminative features for the current distribution and ignore
other semantic features present in the input data. OCM (Guo al. (022)) tackles this problem by
learning as many features as possible from the imput X via maximizing the mutual information be-
tween the input X and the hidden representation fx(X) (I(X; fe (x (X)) and the mutual information
(T(.)) between the representation fo(X) and the label Y (x()) to learn many character-
isties of the input as possible. OCM uses the InfoNCE loss (Poole et al. 2019) (LinfaNCE) to optimize
for the replay-based method. Note that it maximizes I(Y; f( (X))) by using only X.!a/ to calculate the cross- enosopy loss as fo is maximized when Y follows the uniform distribution. X!!" is sampled from the class uniformly distributed data of all seen classes stored in the memory buffer. The loss for OCM is
I(X; (X()) (i.e,, I(X!""; (o(T"") + I(x;""; f fo((""")) ))
D, Dt = Limoac(( (XT." (x("))))) + LimONce (xt", (x(""")) + Lugarce
f0ad (X!")) (1)
where the last term tries to prevent F from forgetting features of previous classes by maximizing the mutual information between the hidden representations of x given by the current feature extractor Sa and the feature extractor f before the current task, i.e., I [X (xb")) J (x!"))
Invariant features. As discussed in Sec. [ in addition to learning holistic feature representations of the input data, this paper argues that the learned feature should also be invariant to each class. We give more rigorous definition of the invariant class-related feature here.
Definition feature s is invariant for a class label y E Y if the following holds,
Ex [I(s E 2) |Corr(æ, v)]l p e
where X₃ the input variable under the environment 2, Z is the set of all possible environments and not limited to the dataset D, Corr is a function computing the correlation between the input rr and its label y. I. is the absolute value function and p is a constant. Tī(s E x)) is the indicator function that identifies if feature s exists in input æ. Note that by environment we not only mean the background of an object (c.g., forest as the background) but also the object in the input (e.g.. a building showing different colors in rainy days and sunny days).
Our goal. Combining the two concepts, we aims to learn holistic invariant representations for each class, i.e., learning invariant features many possible, which help reduce CF and improve the generalization power. Below, we focus on proposing techniques to learn invariant features. We rely on the method in OCM to learn as many invariant features possible to achieve holistieness.
3

---

Under review as a conference paper at ICLR 2023
4 OPTIMIZATION OBJECTIVE AND PROPOSED METHOD
The proposed optimization objective for holistic invariant representation learning is Cau Chottsni + Limarionz where Cholistie aims learn holistic features and Cinvoriant emphasizes invariance and sup- presses non-invariant features. We focus only on designing Limariant and directly use the holistic representation loss of OCM, LOCM D Dee"),, as Lholistic. To learn invariant features based on Definition 1, we learn features that have a positive correlation with each class regardless of the envi- ronments to form the class representation. To formalize our objective, we borrow an idea from causal inference (Jung et al. (020). which replaces the observational distribution P(Y [X] (Xpew) with the interventional distribution P(Y,ew) "do [do(X"e))) in Empirical Risk Minimization, where do(X) moves the environmental effects from the prediction of Y. Then the Interventional Empirical Risk written (Wang et al. 2022b):
R(D = "u=-P( foa
ΣΣΣ V
where z is sample from the ideal environment variable Z that includes all possible environments. For replay-based method, the interventional empirical risk is R(DDe))) + R(D) Optimizing this loss is equivalent to making the model ignore variant features and learn invariant features defined by Definition because features that only have a positive correlation with the label in few envi- romments are filtered out by the operation (x) and the model only uses features that have positive correlation with the label across all environments (not influenced by specific environments) to form the representation. However, directly optimizing the interventional empirical risk is hard as 2 is hard to observe or annotate. In the following, we use Eq. t guide the design of our method.
4.1 CREATING MORE ENVIRONMENTS 2 FOR LEARNING INVARIANT FEATURES
We propose method here to make the model learn invariant features by aligning the inputs with the same semantic meaning (class label) from different environments. The method has two parts. The first part creates augmented inputs with different environments from the original input. The augmented inputs have the same class label as the original input but diverse variant features or environments. The second part is new optimization objective for the model to better learn invariant features across the augmented inputs and filter out variant ones based on the definition in Eq. 2 Unlike most existing augmentations that are designed for learning rich features or other purposes, our augmentations are specifically designed for learning invariant features.
Choice of Data Augmentation. Different from domain generalization, CL does not provide fixed number visually distinct domains//nvironments any prior knowledge about the domains (c.g., the domains/environments where the objects to be classified reside in). To achieve a similar effect of different domains or environments, we propose two data augmentations for the input image æ to create augmented images with different z's.
(1). Color change. For an input image x, we first randomly reorder the RGB channels of the image to create an image x' and then sample a from Beta distribution to create an augmented image,
$ A x (1 A) -x'
We use this augmentation create augmented images with different color environments (Fig. [b)). The augmented image x' has the same label as x. We denote this augmentation as Augentor (T. 8). where s means that we repeat the steps times create different augmented images.
(2). Adding more environments. For input x of size d, we first resize x to r d where r is the resize rate. We name this resized image as resize(x, r1 d). Then we randomly sample anther image Ty from the batch x- Xxew and replace the center part 22 with resize (Ir Y2 -d). We denote the augmented image as (zg, whose original object is not changed dramatically with reasonable choice of r1. But the background change introduces features from image 22 (see (c)).
New Optimization Objective. To make the model focus on invariant features, we force the repre- sentations of the augmented images to be near to each other, which penalizes the model if it uses simple color features (Augeotor) features related fixed backgrounds ( (Augptu) to form the class
4

---

Under review as a conference paper at ICLR 2023
Tusk
Task c +
R1 d)
Original image
Original image
Lodybird
Applee in the tree
Cow in the grass
x d'reorder the channels)
(randdoyyy sampled)
(a)
(b)
(c)
Figure 1: (a) is the illustration for the example in the Introduction and (b) is the illustration for the data augmentation Augeotor and (c) is the illustration for the data augmentation Augptus-
representation (Definition 1). As we consider classification tasks, we use the two types of aug- mentations separately to create augmented images of z Aug(x) Augptus (x) U Augeotor (x. 8), and calculate the cross-entropy loss of the union of the augmented images + images) and the original image x (1 image). The overall risk is turned into:
Σ
(oe v)
R(x, y) UxEAug(O)((z)
E
dist dis((T((Ts) (+42)
(5)
where I₁, 2j are images in Aug(x) U{z} (x, # xz) and dist(.:.) computes the cosine distance between the two representations. With the regularization of the second term, F learns invariant fea- tures across different environments and ignore those that only have strong positive correlations with the class under one/few environments 2). We further compare the proposed data augmentations with other related augmentations and analyze the new optimization objective in Sec. 5.4
Theoretical justification: We construct an environment set Z and want the model to learn invariant features across different environments. However, the number of constructed environments is limited, the model may memorize each environment and use environment-relate. features to reduce the loss. By optimizing the second term in Eq. [3] we force the representations of different augmented images of the original image to be very similar. Then the learner will learn little information about Z from the augmented images, which means P(Z(z) P(Z), where E Aug(z). Then we have
M(..) = ΣΣ> Coclo)(Ja(s)) Plyle, P(z)))(
Σ ΣΣ Lee (o ALgg))( Ufr)
Len(2,
(6)
8 8
So minimizing the risk in Eq. is approximated by minimizing the ideal interventional empirical risk under the environments variable Note, the constructed Z is a subset of the ideal environment set Z. Increasing the diversity of samples in Z makes it closer to the ideal set Z.
4.2 STORING MORE DATA AND DEALING WITH LOCAL SAMPLING BIAS
Data augmentation can reduce the influence of environments only partially. For example, Samoyed dog and Golden Retriever are dogs in different environments and it's hard to use the data augmen- tation to generate images of Samoyed dog when only have the images of Golden Retriever. But training J' with features. So our method also focuses on better utilizing the original data learn invariant features across different environments. First of all, OCM (Guo et al. (022) discards the new data batch Drew
diverse images of objects of class can make the model learn more invariant
and uses only the buffer batch D/d (which also contains some saved new task data) to calculate L.s loss. which makes the model use less data leamn We add D into empirical risk loss,
R(D! UD
E TEX
H(x, H) Nlb
Σ XEX
R(x, v) Now
where Neww and Aww are the batch sizes of DRew and D/d respectively. Following (Ahn al. 202 to avoid the interference from the new data to the replayed data. for the second term in Eq. we clip the logits of classes that do not appear in ynew when calculating the cross-entropy loss.
5

---

Under review as a conference paper at ICLR 2023
Resizing and storing more data: Due to the limited buffer size, only small number of samples stored. However, storing more samples helps learn more invariant features because more images also mean more diversity of the environments. Without increasing the memory size, we propose to store fixed rate (storage rate) of resized images Specifically, we use a quarter of the original memory space to store resized images. The size of a resized image is set as 72 where r2 is the resized rate and d is the size of the original image. When we sample data from the buffer, we resize the sampled resized images to their original size and mix them with other sampled images to construct
x N resize( d), where r; is an image sampled from the memory buffer. Note that Wang et aL. (022a)) used data compression in offline CL to store more data by solving an optimization objective for 72- But it needs the whole training set of the task and the converged model parameters after the training of the task. Those not available in online CL. In Appendix 1, we justify the use of more samples to enable F learn better invariant feature representations and to make the risk in Eq. 7 better approximate the ideal objective in Eq. Now, our Lamarian loss is
Lanvriinn ( D D
Σ
R(E, v) Niw
R(E, v) Naew
(8)
=EU Lill reste((s.darc)
The optimization bat loss for learning task & then is
" Dr = LDcu(D D( Limarimm (D .
(9)
where the first term learns holistic feature representations and the second term learns invariant class representations to establish the class decision boundaries. We call this method IFO.
Local sampling bias (LSB). In online CL., the model only gets new data batch of a task each time. easy to learn variant features if similar environments appear in a few recent data batches. We call this local sampling bias (LSB). To mitigate this problem, we propose to use the buffer data that has the same label as the new data to augment the new data. Specifically, for each sample x in Diew, first calculate the cosine distance between fo(r) and the hidden representations of all stored data of the same label y. Then we choose the stored data sample that has the maximum representation distance with and denote it as i. We don't 't randomly choose stored data sample be very different from x. We use à as T2 in the process of creating Augptu( Then the augmented image Augplu (x) has the same object of but new environment that is totally different from x. Optimizing Eq. S helps the model to get rid of the false high correlation between variant features that appear in recent new data batches and the class label.
want á to
Additionally, we calculate the cosine distance between fa(z) and the hidden representations of all buffer data that do not belong to class y (of z). Then we choose the buffer sample that has the mini- mum representation distance with x and denote it as F. # has similar features/environmeny to but a totally different class. We collect i for each x in batch Deew to construct a batch De boew that has the same size as Drew Adding the loss
Rir.y) into the original invariant loss (Eq makes New
xE bny"
the model learn to distinguish new data and data from other classes under similar environments and also to avoid the LSB problem. We denote the new invariant loss as Livarioue (Dee Di",, Dumndoof ). Considering the LSB problem, we obtain a new model called IFO++, which optimizes
Cail Dr D. = Locu (D.",, ) + Lomarauu ( De Diul Deendut
(10)
5 EXPERIMENT RESULTS
We evaluate the proposed IFO in three online CL. scenarios: standard disjoint task scenario, blurry task boundary scenario, and data environment shift scenario.
5.1 DISIOINT ONLINE CONTINUAL LEARNING SCENARIO
Datasets. In this scenario, use five datasets. For MNIST (LeCun et al.) 1998). we split its 10 classes into 5 different tasks, 2 classes per task. For CIFARIÓ (Krizhevsky & Hinton 2009), we also split the 10 classes into 5 different tasks with two classes per task. For CIFARTO0 (Kriheeskky
Note that this is image compression (Wang et al. 2022a), which should also work hut more complex.

---

Under review as a conference paper at ICLR 2023
& Hinton 2009), we split the 100 classes into 10 different tasks with 10 classes per task. For Tinylmagenet (Le & Yang. 2015), we split the 200 classes into 100 different tasks with classes per task for stress testing. For ImageNet (Deng et al., 2009), we split the 1000 classes int into 10 different tasks with 100 classes per task. Each task runs with only one epoch for online CL.
Baselines. See the 12 baselines in column of Table 1 We run their official codes (Appendix 2).
Backbone, Optimizer, Buffer and Data Augmentation. We follow (Guo et al. 2022) and use ResNet- 18 (not pra-trdi the backbone for our method and baselines in the CIFAR1O, CI- FARI00, TinyImageNer, and ImageNet settings. A fully-connected network with two hidden layers (400 ReL.U units) is used the backbone for MNIST. We use the Adam optimizer and set the learn- ing rate as 1e-3 for all methods and set Nueww and Nu as 10 and 64 respectively for all methods and use the reservoir sampling for our method. For the buffer size B, we use A to store resized images 3R and 4
to store the original-sized images. As the numbers of resized images and stored original- sized images are similar, if the training iteration is an even number, we store resized images and otherwise we store original-sized images. For other baselines, only the original-sizen images are stored. The whole memory size is the same for all methods. Following (Guo et al., 2022), the data augmentation methods horizontal-flip, all methods to improve the performance (no drops). Note that these augmentations have also been used in other papers, e.g., Fini et al. (2020). We set in Eq. 5 as 5 and Ti in Augplus as 0.75 and r2 as 0.5. More details are in Appendix
wundom-resized-coet and random-gray-
are applied to
Table I: Accuracy on the MNIST (5 tasks), CIFARIO (5 tasks), CIFARIOO (10 tasks) and TinyIn: Im- ageNet (100 tasks) datasets with different memory buffer sizes B. All values are the averages of 15 runs. See the results on ImageNet in Figure
aa..
Method
MNIST
CIPARIO
CIFAR100
TingImageNet R-4k
B
R-0.18
B-0.5k
8-1k
8-0.2
B-0.5k
8-1k
8-1A
B-24
R-Sk
B-22
B-102
Chondlyy al
2018
56.9255
57.7 61.6
22.7.18
22.7=19
22.6407
5.8003
5.8833
6.5922
0.9001
211a1
3.9a2
GSS Aljundh
20196
70.4115
80,7+5a
87.5
26.9
30,7=13
40.1+14
11.180
13.3605
17,480.
3,3405
10.0002
10.5=02
ER[Chaudaay et al.
2120
78.7804
88.0342 90.3a01
29.7a1
35.2a00
44,3a0.
11.700
15.0zn9
14,4309
5.600
10.1407
11.7am
MIR
2019
79,010
W8.3161
91.3119
$7.3103
400108
41.0110
15.7.03
191101
24.1103
6,1
11.119
133
ASEX Shim al
2021
61 12:21
71.0a56
82,
27.810
362211
44.7
16.4203
12:211
27.1
5,3403
8.2453
10.3
GDunk (rabba di
$1.5
91.0e00
910.
35.9
50,7=07
63.5e00
14.1003
20.1=01
36.040.
12.6.0.1
12.7403
15.7
SCR al
2021
86.2
92.8
94.6600
47,2
58.2000
64.1.11
26.5202
31,6205
36.5802
10.6811
17.2601
20.411
DER
et
M020
74.4.11
91.5100
12.1.00
44.2.11
47.911a
54.7172
13.5.02
19.7115
27.0107
4,5.0s
10.1
17...05
X021
90.2ga1
V2.7.61
93 93.9
54.7205
56.0.4
58.2g1
182218
19.7205
22.4300
5,5007
116221
Co'CClarcaa
2020
83.1
91.7=0 91.5e6.1
42.1=11
51.0=077
58.8
17.1=04
24.2
32.2+0.5
10.1=02
158404
22.5=11
8811
X121
93.014
49.5:003
59.22.4
64.010
260001
33.1,001
39.5104
9.6307
15.2114
21.1
OCM al.
2022
90.7+00
45.7
96.7a0
(9.4203
70.0a11
77.2
28.1203
35.0e044
42,480.
15.7a02
21.2
27.0a03
IFO
92.5+04
96,1 97,0e0
65,0en3
73.5m02
78.040
38.5=05
45.5+03
49.4
20.5e05
27.2+05
34.604
IFO++
93,0=a2 96.4502 97.0
69.5.01
76.4-05
78.5
40.5:004
47.9000
51.3104 21.7003
28.9
38.8-01
Accuracy results. We report the average accuracy of all tasks after learning of the final task in
Table T
We observe that our IFO outperforms all baselines by large margin. Our main baseline is OCM as we added the condition of invariance to its holistic representation learning. IFO boosts the performance of OCM significantly. IFF's performance is especially strong when the buffer size is small (e.g., 10 samples per class) The reason is that the baselines tend to overfit the buffer data and learn variant features when the buffer size is small, which IFO able to avoid by learning invariant features. Further, the improvement of IFO does not decrease when the dataset gets more complex (e.g.. TinylmageNet). IFO++ further boosts the performance by considering local sampling bias.
For the hardest ImageNet dataset, due to the poor overall performance, we compare IFO with top-3 baselines (OCM, SSIL, and SCR). Figure 2a) () shows the average accuracy of all tasks seen so far after learning each task. IFO outperforms the three baselines in the whole learning process. Also, the average accuracy first arises and then drops. This is because the random-initialized model doesn't 't have enough features to solve the first task until the second task arrives. The later drop is due to CF.
Forgetting rate. We report the average forgetting rate (Chaudhry et al., 2020) in Table A.3 in Appendix 3. We see that IFO gives less forgetting than OCM, especially CIFARIO and Tinylma- geNet. Also, IFO forgets the least except for GDumb and SCR on Tiny of the two baselines are much lower than IFO. For ImageNet, IFO also fares well (see Appendix 3).
But the accuracy
Learning invariant representations. We use two types of experiments for this evaluation: (1) model robustness unseen environments and (2) eigenvalue distribution of the learned representa- tions. Due to space limits, we give the details in Appendix 4. All experiments verify the effectiveness of our method in learning invariant features.
7

---

Under review as a conference paper at ICLR 2023
Imageilet setting Adam
Liburr CIARLOO witting and Adom retiriier
15
20
50 20
300
(
(b)
Figure 2: (a) results in the disjoint setting using ImageNet. The buffer size is 10k. For (b) and (c), 'sample'' of the x-axis means the number of new data samples that the model has seen and the unit is 1k. (b) results of the online i-Blurry setting using CIFARI00. The buffer size is 2k. (c) results of the online i-Blurry setting using TinylmageNer. The buffer size is 4k.
5.2 BLURRY ONLINE CONTINUAL LEARNING SCENARIO
In this scenario (Koh et al.) 2021 D. the classes of the training data for dataset is split into two parts, N% of the classes as the disjoint part and the rest of 100 - N% of the classes as the blurry part. Each task consists of some disjoint classes (with all their data) and some blurry class samples. The disjoint classes of a task do not appear in any other task. The blurry classes data are sampled from the blurry part. Specifically, for task, 100 M% of the samples are from some randomly selected dominant blurry classes (these classes will not be dominant classes in other tasks) and M% of the samples are from the other minor classes. This strategy of forming tasks is called i-Blurry.
Following (Koh et al., 2021), in our experiments on the CIFAR 100 (5 tasks) and TinylmageNe (5 tasks) datasets. We use ResNet- 34 and Adam optimizer with an initial learning rate of 0.0003 for all systems. We compare their original system CLIB with our CLIB+INV (replacing CLIB's cross-entropy loss with our Limvariant loss in Eq. ( and three best-performing baselines (OCM, SSIL and SCR). Here we do not use LOCM as our main contribution is the Limarian: loss. More details is in Appendix 5.
the i-Blurry setup with N 50 and M 10 (i-Blurry-50-)
Accuracy results Following (Koh al. 202 , measure any time inference and plot the accuracy-to-samples curve in Figure 2 2())& The setting is as follows. After the model sees every 100 new samples, we test the model using the original test set of all classes in the CIFARI00 or TinyImageNet dataset and record the accuracy. The figures show that CLIB+INV clearly out- performs CLIB, especially after the model has seen some tasks, which indicates that effectively learning invariant class representations improves the overall generalization ability in incremental learning. We also observe that the three best baselines' performances are weaker than that of CLIB because the three baselines are not designed for this setting.
5.3 DATA ENVIRONMENT SHIFT IN ONLINE CONTINUAL LEARNING
In this scenario, the model needs to learn the same classes from different environments sequentially, and is then tested in an unseen environment. We use the PACS dataset (Li et al.] (017) simulate this scenario. This dataset has four different environments: art painting, cartoon, photo, and sketch. Each environment has data points of the same seven classes. We choose each set of three environ- ments the training environments and the remaining environment as the test environment. Thus four experiments are conducted with different training and test environments. We report the average test performance over the four experiments as the empirical estimation of the ability of our method in learning invariant features. The core of this setting is to learn invariant features under the shift of environments. So focus on the representation-lcare strategy to overcome forgetting. The hyperparameters are given in Appendix 6.
loss and do not consider/use the replay
Table 5 in Appendix 6 shows that the performance of our Limvarians loss (Eq. 8) is beyond that of the traditional cross-entropy loss E.e over the four environments. This means our Limarian loss learns in- variant features better. Another observation is that optimizing the classification loss (Linvariant or Cee) with the Chotisti loss improves the performance further because learning more features of one class enables the model to learn more knowledge deal with test samples from unseen environments.
8

---

Under review as a conference paper at ICLR 2023
Table 2: Ablation accuracy average of 5 runs. B is the memory buffer size.
Datasel
no alig
OCM4Miup OCMFCuiMi MCMMM Aug OCM+ClasAug
OCMMA
insue
CIEARIOO (d=21)
41.8404
48,0e0
44.6442
33.2+35
36,3e53
33.7.m
39.5
57.4445
42.0eu2
44,060s
Trnglmagcher (8-26)
18.010
19.5
200000
144102
16,002
17.5
17.7
25.105
185500
19.9134
5.4 ABLATION STUDY AND ANALYSIS
Ablation study of the components in IFO. Table 6 shows the results without using Augeolrr (no Augevior), Augpius (no Augpiu) or the second term of Eq. S (no align). The performances of these experiments drop, which show the contribution of the proposed augmentations and loss. As IFO is basically OCM plus invariant feature learning, we also tried some other data augmentations and losses designed for feature learning, Mixup (Zhang et al. 2017) (OCM+Mixup)), Cutmix (Yun et a 2019) (OCM+CutMix), Aug (Fini et al. 2020) (OCM+MemoryAun and ClassAug (Zhu et al. 2021) (OCM+ClassAug). CutMix, MemoryAug and ClassAug improve the performance of OCM but Mixup does not. However, the results of these augmentations and their losses poorer than that of IFO. For "OCM+XX"ev, we add Lee loss of the new data batch into OCM. The performance improves slightly as the model learns more invariant features from more data. For "no new data", only D in Eq. used. For "no resized image", we do not store resized images. The performances of both experiments drop, which means that considering more samples to learn invariant features useful. Ablation experiments on IFO++ are discussed Appendix 7.
Influence of hyperparameters in IFO. For in Eq. 5 from Figure (a) in Appendix 8, we observe that the performance has a positive correlation with the number s as the model gradually focuses on invariant features rather than strongly depending on simple colors. We set to 5 as it achieves the best performance with less compute. For the storage rate for resized images, we need to find a balance between storing more samples and avoiding huge information loss caused by the resizing operation. From Figure 4b), we found that using a quarter of the memory space to store resized images achieves good balance. For rate 11 in the augmentation Augpiue. we need to avoid intro- ducing trivial features (r1 is too high) and causing a huge information loss in the original image (r1 is too low). For rate r2 in Sec. 4.2 we have the same balancing issue with the storage rate (more saved samples vs information loss). Based on Figure 4c), we set r as 0.75 and r2 as 0.5.
Influence of learning invariant features for continual learning (CL). To further investigate how learn- ing invariant features helps the performance of CL methods, we measure three abilities of OCM and IFO: (1) the ability establish decision boundaries between the classes within the new task by recording the accuracy performance of the new task. From Figure (]a) in Appendix 9, we see that IFO's accuracy of the new task outperforms that of OCM as learning invariant features makes the model improve its generalization power. (2) the ability to maintain the learned decision boundaries within a task by calculating the average task incremental accuracy of the previous tasks. Our method IFO again outperforms OCM (Figure (()) as IFO mitigates the overfitting problem of the limited buffer data. (3) the ability to establish class boundaries across tasks. We measure this by considering only the logit of the true label of each test instance in task and the logits of the classes from the other tasks when the model predicts the label of the test instance. IFO's performance is still better than that of OCM (Figure (])) as IFO reduces the variant features, which enables the model to project those class representations to different locations in the space, making the decision bound- aries easier to establish. Also from Figure/6) we can clearly see that improving the last ability is the biggest challenge for online CL. More details are given in Appendix 9.
6 CONCLUSION
Despite the fact that numerous empirical techniques have been proposed to solve the class- incremental learning (CIL) problem, limited work has been done to study the necessary conditions for good CIL performance. Recent work in (Guo et al. 2022) proposes that it is neccesary to learn holistic feature representations, which improved the online CIL performance significantly. This pa- per further argued that it is also necessary to learn invariant features for each class, and proposed several methods to help learn invariant features in addition to learning holistic representations. Ex- perimental results showed that the new condition gave another boost to the online CII. performance.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Davide Abati, Jakub Tomezak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehteshami Bejnordi. Conditional channel gated networks for awsr- continual learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3931-3940, 2020.
Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon. Uncertainty-based continual learn- ing with adaptive regularization. In NeurIPS, 2019.
Hongjoon Ahn, Jihwan Kwak, Subin Lim, Hyeonsu Bang. Hyojun Kim, and Taesup Moon. Ss- il: Separated softmax for incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 844-853, 2021.
Rahaf Aljundi, Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Min Lin, Laurent Charlin, and Tinne Tayrel Online continual learning with maximally interfered retrieval. arXiv preprint arXiv:/908.0 04742, 2019a.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019b.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: /907.028990 2019.
Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Ha, and Jonghyun Choi. Rainbow mem- ory: Continual learning with a memory of diverse samples. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 8218-8227, 2021.
Jihwan Bang. Hyunsco Koh, Seulki Park, Hwanjun Song, Jung- Woo Ha, and Jonghyun Choi. Online continual learning contaminated data stream with blurry task boundaries. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9275-9284, 2022.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark expe- rience for general continual learning: a strong, simple baseline. arXiv preprint arXiv:2004.07211, 2020.
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co21: Contrastive continual learning. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 9516-9525, 2021. 2021.
Arslan Chaudhry, Mare' Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420 2018.
Arslan Chaudhry, Naeemullah Khan, Puneet K Dokania, and Philip HS Torr. Continual learning in low-rank orthogonal subspaces. arXiv preprint arXiv:2010.1 1635, 2020.
Ekin D Cubuk, Barret Zoph. Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 113-123, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Enrico Fini, Stéphane Lathuiliere, Enver Sangineto, Moin Nabi, and Elisa Ricci. Online continual learning under extreme memory constraints. In European Conference on Computer Vision. pp. 720-735. Springer, 2020.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 (1116)6666777 2020.
Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. In International Conference on Machine Learning, pp. 8109-8126. PMLR, 2022.
10

---

Under review as a conference paper at ICLR 2023
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common ruptions and perturbations. Proceedings of the Interaatoonal Conference on Learning Represen- tations, 2019.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 831-839, 2019.
Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma, Dongyan Zhao, and Rui Yan. Overcoming catastrophic forgetting for continual learning via model adaptation. In ICLR, 2019.
Steven C. Y. Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu- Song Chen. Compacting, picking and growing for unforgetting continual learning. In NeurIPS, 2019.
Yonghan Jung, Jin Tian, and Elias Bareinboim. Learning causal effects via weighted empirical risk minimization. Advances in neural information processing systems, 33:12697-127099 2020.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, and Others. Overcoming catastrophic forgetting in neural networks. volume 114, pp- 3521-3526. National Acad Sciences, 2017.
Hyunseo Koh, Dahyun Kim, Jung-Woo Ha, and Jonghyun Choi. Online continual learning on class incremental blurry task configuration with anytime inference. arXiv preprint arXiv:2110.10031, 2021.
Lukasz Korycki and Bartosz Krawczyk. Class-incremental experience replay for continual learning under concept drift. arXiv preprint arXiv:2104.1 /867,, 2021.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical Report TR- 2009, University of Toronto, Toronto., 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7:7, 2015.
Yann LeCun, Corinna Cortes, and Christopher JC Burges. The mnist database of handwritten digits. http://yann.l lecun. wtt.n..om//xxb/mniss////o 1998.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. Proceedings of the IEEE international conference on computer vision, pp- 5542-5550, 2017.
Xilai Li. Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: continual structure learning framework for overcoming catastrophic forgetting. In ICML, 2019.
Zheda Mai, Ruiwen Li, Hyunwoo Kim, and Scott Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier online class-incremental continual learning. In Proceedings of the IEEE/CV Conference Computer Vision and Pattern Recognition (CVPR) Workshops, pp- 3589-3599, 2021.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning & motiv., volume 24. 1989.
Oleksiy Ostapenko. Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In CVPR, pp. 11321- 11329, 2019.
Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. In International Conference on Machine Learning, pp. 5171- 5180. PMLR. 2019.
Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. In EECV, pp. 524-540, 2020.

---

Under review as a conference paper at ICLR 2023
Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. itaml: An incremental aask-agnostic meta-learning approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588-13597) 2020.
Alexandre Rame, Corentin Dancette, and Matthieu Cord. Fishr: Invariant gradient variances for out-of-distribution generalization. In International Conference on Machine Learning, pp. 18347- 18377. PMLR, 2022.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, pp. 2001-2010, 2017.
Hippolyt Ritter, Aleksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. In NIPS, pp. 3738-3748, 2018.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. arXiv preprint arXiv: arXiv:2103.09700 103.09762, 2021.
Dongsub Shim, Zheda Mai, Jihwan Jeong, Scott Sanner, Hyunwoo Kim, and Jongseong Jang. On- line class-incrent incremental continual learning with adversarial shapley value. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 9630-9638, 2021.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. In NIPS, pp. 2994-3003, 2017.
Ghada Sokar, Decebal Constantin Mocanu, and Mykola Pechenizkiy. Learning invariant represen- tation for continual learning. arXiv preprint 101,22000.0616,, 2021.
Johannes von Oswald, Christian Henning, João Sacramento, and Benjamin F Grewe. Continual learning with hypernetworks. ICLR, 2020.
Jianren Wang, Xin Wang, Yue Shang-Guse and Abhinav Gupta. Wanderlust: Online continual object detection in the real world. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10829-10838. 2021.
Liyun Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing Hong, Shifeng Zhang, Zhenguo Li, Yi Zhong, and Jun Zhu. Memory replay with data compression for continual learning. arXiv preprint arXiv:2202.0653 065692 2022a.
Tan Wang, Qianru Sun, Sugiri Pranata, Karlekar Jayashree, and Hanwang Zhang. Equivariance and invariance inductive bias for learning from insufficient data. arXiv preprint arXiv:2207.12258. 2022b.
Yufei Wang, Haoliang Li, and Alex C Kot. Heterogeneous domain generalization via domain mixup. In ICASSP 2020-2020 IEEE International Conference on Acousties, Speech and Signal Process- ing (ICASSP), pp. 3622-3626. IEEE, 2020.
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), NeurlPS, 2020.
Yuan WI, Diana Inkpen, and Ahmed El-Roby. Dual mixup regularized learning for adversarial domain adaptation. In European Conference on Computer Vision, pp. 540-555. Springer, 2020.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye. Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374-382, 2019.
Shipeng Yan, Jiale Zhou, Jiangwei Xie, Songyang Zhang, and Xuming He. An em framework for online incremental learning of semantic segmentation. Proceedings of the 29th ACM Interna- tional Conference on Multimedia, pp. 3052-3060, 2021.
Fu-En Yang, Yuan-Chir Cheng, Yu-Yun Shiau, and Yu-Chiang Frank Wang. Adversarial teacher- student representation learning for domain generalization. Advances Neural Information Pro- cessing Systems, 34:1 19448-194600 2021.
12

---

Under review as a conference paper at ICLR 2023
Haiyan Yin, Ping Li, et al. Mitigating forgetting in online continual learning with neuron calibration. Advances in Neural Information Processing Systems, 34, 2021.
Lu Yu, Bartlomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6982-6991, 2020.
Xiangyu Yue, Yang Zhang, Sicheng Zhao, Alberto Sangiovanni-Vincene Kurt Keutzer, and Bo- qing Gong. Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data. Proceedings of the IEEE/CVE International Conference Computer Viston, pp. 2100-21111111 2110, 2019.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceed- ings of the IEEE/CVF international conference on computer vision, pp. 6023-6032, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:
2017.
Junting Zhang, Jie Zhang, Shalin Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, and C.-C. Jay Kuo. Class-incremental learning via deep model consolidation. In CVPR, 2020.
Hanbin Zhao, Hui Wang, Yongjian Fu, Fei Wu, and Xi Li. Memory efficient class-incremental learning for image classification. IEEE Transactions on Neural Networks and Learning Systems, 2021.
Fei Zhu, Zhen Cheng, Xu- Yao Zhang, and Cheng-lin Liu. Class-incremental learning via dual augmentation. Advances Neural Information Processing Systems, 34:1 14306-143188, 2021.
13

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 Theoretical justification FOR SECTION 4.2
In the ideal case (data is sufficient and highly diverse), X can be observed in any environment z. That leads to X(Z = P(Z) (the independence of Z and X) and we have
R(D
E E () DEYpew aEXpew
(1I)
Leclo((f((t),)))GGG DEY, reXpco
where Ey Sealol U)), the traditional cross-entropy loss. We also have R(X:" Y/ = Eyer,' Exex, E Lor u))
So adding X and resizing and storing more buffer data construct X makes the model have more samples from different en- vironments to train. And that makes the risk in Eq. /etter approximated to the ideal objective in Eq. B
A.2 THE DETAILS OF DISJOINT ONLINE CONTINUAL LEARNING SCENARIO
Due to the limitation of computational resources, we download the downsampled version of Ima- geNet (3 x 32 x 32) from the official website and conduct experiments on this dataset. We don't use the Augeolor in the MNIST setting. We set the alpha and the beta of the beta distribution in Augcolor as 1.
For AGEM, following the original paper, we use the SGD optimizer and set the learning rate 0.1. We use the random method to update the buffer and to sample data.
For GSS, based the original paper, number of buffer batches randomly sampled from the memory to estimate the maximal gradients co- sine similarity score is set 10 and the randomly sampled buffer batch (Xbudler size for calculating the score is 64.
the same optimizer and learning rate above. The
For ASER, we use the number of samples per class for random sampling as 1.5. We allow 3 nearest neighbors for KNN-SV computation. We the same SV-based methods for both Memory-Update and Memory-Retriewe as given in the original paper.
value of Adversarial SV and Cooperative SV, and the maximum
For MIR/ER, decay as 0.0001. We set the sub-sample size as 128.
the Adam optimizer and set the learning
0.001 and fix the weight
For DER++ we use the Adam optimizer, set the learning rate 0.001, fix the weight decay as 0.0001 and the value of alpha (a) as 0.1, and fix the beta (3) as 0.5.
For GDumb, we the Adam optimizer, set the learning rate 0.001 and fix the weight decay as 0.0001. We use the CutMix as the regularization to overcome over-fitting we follow the official code and set the number of epochs for training the whole buffer data 256 for MINIST, CIFARIO, and CIFARIO6 100 datasets, and 32 for the Tinylmagenet dataset. We set the gradient clip as 10.
For SCR, we use the Adam optimizer, set the learning rate as 0.001 and fix the weight decay as 0.0001. We set the temperature for contrastive loss as 0.07. We employ linear layer with the size [dim,,, 128] as the contrastive head. We follow the official code and use the horizontal-fiip. random-resized crop, gamddommgray-scallcccoorrriiiiio
as its data augmentations.
The official code of these systems can be found from the following locations.
The code of ER and MIR:
com wttp:/o../o........ http:/hhww...coooooiooo/////aaaa
Retrieval
The code
of ASER and SCR:
https://giihbb..omm//atttttt
oni online-coo cont inual aatee--oottmmall-eaannnnn.
The code of GDumb: htt
The code DER++ https: D.Z-:a/twss//atba...ooo/nnnnngga
The code for AGEM: Act.hh/wh........o.ooooooaorrerr
14

---

Under review as a conference paper at ICLR 2023
The code for
GSS:
htDD:///gthaab..co//aaaatll
aradient-basede Satpsee--SSappe---eeettto. The code for Co"L: hatpss//ggaaub..oo////btt//////1
The code for II.2A:
The code for SSIL:
https:
The code for OCM:
A.3 THE FORGETTING RATE TABLE
Table 3: Average forgetting rate. All numbers are the averages of 15 runs. See the forgetting rates for the ImageNet dataset in the text below.
Method
MNIST B-R.SE
CIFARIO B-03K
CIFAR100
TinyImageNet B=4k
B-IE
0-00
B-
B-T
B-22
B-5h
B-2k
B=T0k
Chandhey al. 2013
$2.5
30.1.47
32.0.24
1238
43.2441
43.3nn
45.7
43.9
73.9
78.9
GSS Aijundi al
2019b
26.1122
17.8
10.5pa7
75.311
65.911
54.9.2m
308.5a
30.7.08
26.4.03
128111
72.6144
115:62
HR
HR Channtryy
22.7105
4200a3
26.7a03
20.7.07
34.2.0
31.7:59
35.3200
68
2742
MIk
(Arandi 23
All1%
22.310+
9.0.05
5.710s
40.0116
25.910
24.5.04
24.5
21.4.03
21.010:
61.113
60...
39.5.65
ASER Shim
2121
33.8211
24.8205
138854
71.1218
59.1215
50A515
25,0
12.28
13.250
65.7291
64.2302
62.22611
GDumb Przteu
10.3,01
6.2.01
48103
26.505
24.5.00
189..
16.7.0
17.6.03
168
15.9
14.610
11.7.62
SCK Man el al
10,7291
4.7401
4.0503
41.3691
31.5302
24.7
17.560
11,6905
3.6301
19.4203
15,4303
14.9267
DF
(2120
25.0103
7.3g03
30.1
$1,8135
18.7
43,
44.0.
25.8.33
67.2.11
63.6109
55.2.67
112A
A01
8.740.
7.240
4.1201
36.0
32.1
29.1
24.6
12.5
2000
63.5
60.1203
57.611
Co
2041
14.7s0:
7.1481
3.1401
32.0+a
21.0a03
16.9
169
16,602
9,90m7
60,5eas
52,5+00
42.5
XShI
11.3101
2.7
28:01
36.0
29.0.04
13.5
33.911
44.4
36.6107
29.0.77
OCM et 2122
4.700:
1,8461
1,3001
23.0e2
14.0em7
12.0
12.2
8,5093
4,5
23.5a19
21,0a03
18.6:55
42
1.1
16.3
2.0.07
109
8:3101
4.3107
22.5103
18.6.00
13.5.08
IF0++
3.0ma:
0.8302
0.9101
12.700
6.1auz
2.0.03
9.8.05
7.3044
3.3aus
21.3203
13.9
129:52
From this forgetting table (Table A.3), we observe an obvious drop in forgetting rate from OCM to IFO and IFO++ (our methods). That means our methods forget less. In the ImageNet setting, the forgetting rates for the four top methods are 11.47 (SCR), 12.1 (SSIL), 11.7 (OCM) and 10.9 (IFO).
A.4 LEARNING INVARIANT REPRESENTATIONS.
blurry (Sec 5.2 and data environ- ment shift (Sec. 5.3 experiments have already indicated that our proposal is able to learn invariant features better. Here we use two additional and more specific types of experiments further eval- uate the effectiveness of learning invariant features: (1) model robustness on unseen environments and (2) eigenvalue distribution of the learned representations.
Although better results of our method in the disjoint (Sec 5.1
(1) Model robustness on unseen environments. To verify that our method has learned invariant features to form the class representation, we conduct the following experiment (this is not a con- tinual learning setting). After training on the full CIFAR100 data, we test the trained model on the CIFARIO0-C dataset (Fendrycks
Dietterich 2019). which a model robustness benchmark consisting of 19 corupption types with five levels of severities applied to the original test set of CI- FAR100. The corruptions come from four main categories: noise, blur, weather, and digital. Each corruption has five-level severities and "5" indicates the most corrupted one. Those corruptions not used in the training, so the test can be viewed as a test of the trained model in unseen envi- ronments. A model that has learned invariant features should achieve a higher performance. From Table we observe that IFO indeed outperforms the OCM method on 19 unseen environments. The gap between IFF and OCM gets larger with the level of severity increases. We conduct similar ex- periment on the Inn-lmmaeNNe-CCCC dataset (another robustness dataset in (Hendrycks & Dietterich] 2.19), and the conclusion consistent (Table 4 Those experiments empirically verify that our method learns more invariant features than OCM.
(2) eigenvalue distribution of the learned representations. Zhu et al. (2021) observed that repre- sentations fo(x)) with larger eigenvalues transfer better and suffer less forgetting, Guo et al. (2022) showed the number of eigenvectors with significant variances is good indicator holistic degree of the learned representations. We follow their experiment setting and plot the eigenvalues of the eigen- vectors of the representations learned with ER, OCM and IFO from the first 50 classes of CIFAR- 100 that IFO and OCM
as single supervised learning task (see Appendix 4). From Figure 3 have much more significant directions (eigenvectors) than the basic ER method (Chaudhry et al, 2020), which shows that both IFO and OCM have learned more diverse/holistic representations.
15

---

Under review as a conference paper at ICLR 2023
Table 4: Test accuracy in the robustness benchmark. All numbers the averages of 15
CIFARTIICC
Taay-lmagsNer44:
Mithhod
Sevesity
Severity 2
Severiy 3
Severity 4 Sevenity
Sevenity
Severny
Severity
Severity 4 Severity 5
0CM
36,5102
349.00
32,0.44
28.9.47
24,1,00
137702
116.18
901113
6.2103
5,0003
39,400
30.
5,6001
Our IFO method has a similar number of significant directions to that of OCM, but the eigenvalues of the cigenvectors of IFO and OCM are very different (see Appendix 4), which gives us indirect evidence that IFO is able to emphasize more or less of some directions compared to OCM. Coupled with IFO's better result, we can conclude that IFO finds better features.
disjoint CIFAR100 setting and Adam optimizer
40
35
ER
25
Eigenvalue
20
10
5
10
14
Eigenvector Index
Figure 3: We plot the Eigenvalue distribution of the learned representations by ER, OCM, and IFO using the CIFAR100 dataset.
we see that the main difference of the eigenvalue distributions of OCM and IFO is the different cigenvalues they learned in the first 10 eigenvectors. That means that the representation quality not only depends on the diversity of cigenvectors (holistic), but also the different emphases (different eigenvalues) on the features for class representations to achieve invariance.
From Figure
A.5 MORE DETAILS OF I-BLURRY ONLINE CONTINUAL LEARNING SCENARIO
Following (Koh et al. 2021 ), for all method, we use the batch size 16 and 3 updates per streamed
sample for CIFAR 100 and the batch size of 32 and 3 updates per streamed sample for ImnylmageNet, and employ ResNet-34 for CIFAR 100 and Tiny ImageNet. In the CIFAR 100 setting, each task has 10 unique disjoint classes and 10 dominant blurry classes exclusively. The selection process of classes for each task is random. In the Imagemaeeeet setting, each task has 20 unique disjoint classes and 20 dominant blurry classes exclusively, AutoAugment (Cubuk et al. 2019) and CutMix (Yun al. 2019) are also used as data augmentations. For CLIB and CLIB+IN learning rate schedule (Koh al.) 2021) with 7 = 0.95 and m 10 for the two datasets. We use their official code https::////........ / gi hhub. com naver-aveaaaa
we use the same adaptive
B Blurry to run the experiments.
A.6 DETAILS OF DATA ENVIRONMENT SHIFT ONLINE CONTINUAL LEARNING SCENARIO
We use ResNet- 18 (not pre-trained) as the backbone for our method and baselines and use the Adam optimizer and set the learning rate 1e-3 for all methods. The batch size for the data batch is 10 and there is no need to store or sample buffer data [bing: why?]. Data of each environment is run by one epoch. From Table 5 we observe that the model optimized with our invariance loss achieves higher test performance than that of the traditional cross-entropy loss in the unseen environment.
A.7 ABLATION STUDY ON IFO++
16

---

Under review as a conference paper at ICLR 2023
Test accuracy in the unseen environment. All numbers are the averages of 15 runs. 'Art-painting' means that the model first learns the other three environments sequentially (order: Cartoon - Photo + Sketch), then it is tested on the data points of the environment 'Art-painting' 'Cartoon' means that the model first learns the other three environments sequentially (order: Art-Puinting -> Photo -> Sketch), then it is tested on the data points of the enviromment 'Cartoon'. 'Photo' means that the model first learns the other three environments sequentially (order: Art-Puimting -> Cartoon - Sketch), then it is tested on the data points of the environment PPotto'. 'Sketch' means that the model first learns the other three environments sequentially (order: Art- Pai-nann + Cartoon + Photo), then it is tested on the data points of the environment 'Sketch',
Table 5:
Method Art-painting
Cartoon
Photo
Sketch
Live Linvarians
12.7
15.8 12.5. 12.1
13.5=002
17.4+0.3|
13.3+00 13.2+06
Cholisti + Leee
14.7=005
17.8+
13.2+t0. 13. 13.1
Cholistic + Linveriant
15.0=0.4
18,9+0.2
14.6:10.1 17.2+004
Table 6: Ablation accuracy average of 5 runs, B is the memory buffer size.
Daiaset
IT IFO+random 17,"
[FOemux
CIFARIOO (8=24)
45.6
46 46.0am4
48,0ta11
26.8313
47.0e03
Tayynnggee (8-20)
20,600
IFO+ goes further than IFO to mitigate the local sampling bias (LSB) problem in online CL. We report its ablation results in Table 6 In experiments 'IFO+random' and 'IFO+min', we randomly sample a buffer data as & and choose the data sample with the minimum distance from x as I, respectively. Experimental results show that their performances are similar or even poorer than that of IFO. This is because their à may not provide a new environment for x to create its augmented image Augptar (2). The performance of randomly choosing i to construct Deow,buf buf (IFO+random Darw:bud is poorer than that of IFO++ because IFO++ requires the model to distinguish D and D Doew.buf sampled from a similar environment to Dee", which penalizes the local sampling bias. In experiments 'IFO+max' and "IFO+D Dew.buus we only consider E to augment and T to construct De and calculate the Limariiau (Deen, D loss, respectively. Both of them improve the performance of IFO further, but their combination (which is IFO++) achieves the best performance.
A.8 ABLATION ANALYSIS OF
IN IFO
Based on the results in In the experiments of Figure 4 and Figure we set s to 5 as it achieves the best performance with less compute. Also, we set the storage rate as 0.25 and 1'1 as 0.75 and r2 as 0.5.
A.9 ANALYSIS OF THE INFLUENCE OF LEARNING INVARIANT FEATURES
We conduct experiments on the CIFAR 100 dataset (10 tasks) and the buffer size of 2k. From Fig- ure 6 we observe that the performance of the first two abilities of OCM and IFO are non-decreasing and bigger than 70 at the end. But the performance of the last ability decreasing and is smaller than
digcir CIAA100 Adarn
sannt settingg Adamm optrmicr
Performance with diferent
Performance with different
Performence with diferrnet membe rites 13
12
42
40
0.2
8.0
020
3.5
1.50
3323
3300
0.70
0300
0403
0.00
5.773
1300
(a)
(b)
(c)
Figure 4: (a) results of CIFAR100 with different s. (b) results of CIFAR100 with different storage
rate. (c) results of CIFAR100 with different resize rates r and r2. We set B as 2k for all experiments.
17

---

Under review as a conference paper at ICLR 2023
Ingimagelie watiing and Alam
dippett Tmyinagelst wettnng and Adam opt mowr
dapart Tinghnagellei setting and Abam optimuu
Performance with difereet
Feffomerce with dfforrt resie rats Fofformrco with offorrr resie rotes
22
30
0.75
Storae
Fas.m 188
(a)
(b)
(c)
Figure 5: (a) results of TinylmageNet with different (b) results of TinylmageNet with different
storage rates, (c) results of Tiny with different resize rates and r2. We set B as 2k for all experiments.
CITA1100 wetting and Adiam
CIAR300
dispeet FAR120 watting and Adaan optimuzr
DCN
OCV
DCN
F0
F0
03
10
a
(b)
(c)
Figure 6: (a) the performance of learning new tasks. (b) the performance of maintaining learned decision boundaries. (c) the performance of establishing decision boundary between new classes and previous classes.
50 the end (OCM (35.8) and IFO (46.36)). Also, find that the average accuracy of OCM and IFO are 35.0 and 45. 5 respectively in the Table That means the main limitation of the performance of the current model in establishing boundaries between classes from different tasks.
18


Reviews:
[{"review_id": "rdahmRtlQSX", "reviewer": "Reviewer_SrHu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThis paper proposes a fix for the catastrophic forgetting problem (CF) in continual learning (CL) with neural networks. The authors propose an extension of OCM (Guo, 2022), a method that seeks to learn features which preserve mutual information with the original inputs, the outputs, and the previously learned representations over the course of continual learning.\n\nThe extension proposed in this paper is called \"IFO\" which stands for Invariant Feature learning for Online CL. IFO is presented as an add-on to OCM that aims to learn \"invariant\" features which are defined as being both discriminative and present across different many different environments. To do this, IFO incorporates additional image augmentations while training under the OCM loss. Specifically, more color augmentations are used, and an augmentation which superimposes backgrounds from other images is used to create a more diverse set of contexts for objects being classified.\n\nThe paper evaluates IFO within 2 continual learning paradigms: 1) task disjoint, and 2) incremental-blurry task continual learning.\nIn the task disjoint setting, IFO has increased accuracy over other the CL methods on 4 datasets image datasets -- MNIST, CIFAR10/100, TinyImageNet. IFO outperforms all other CL baselines in terms of accuracy, including OCM which it shares a loss function with. IFO also has a lower forgetting rate.  In the incremental-blurry setting, IFO modestly improves over the baseline method, CLIB.\n\nFinally, the paper presents some ablation experiments to determine which aspects of IFO improve performance most. In these experiments, other image augmentation strategies like CutMix and MixUp are used in place of the color and background augmentations used by IFO.\n", "summary_of_the_review": "The empirical results for IFO look promising, but the methodological novelty and theoretical motivations are not big enough to advocate for strong acceptance to ICLR.  However, I am weakly in favor of acceptance to ICLR.", "strength_and_weaknesses": "The strength of this work is that IFO's empirical results show improved accuracy and decreased forgetting on the 4 image datasets tested on. And stable/time-invariant representation learning is an important problem in both continual learning and in information retrieval where representational shifts necessitate reindexing large datasets.\n\nThe weakness of this work is that makes several smaller contributions -- combining the loss function of OCM with a correlation-based loss to induce embedding stability, and adding a novel image augmentation scheme to create synthetic backgrounds -- rather than making one large contribution.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "eehsTUVTxy", "reviewer": "Reviewer_jHRu", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new approach for online continual learning extending the OCM method introduced by Guo et al. (2022). The proposed approach consists in an optimisation problem amd loss function that not only enforces to learn holistic representations of the data (as in OCM) but also class-invariant features. The idea is to simulate images of objects in different environments (via data augmentation), which alter the background of the images but also the colour of the objects themselves.\nExperimental results show the effectiveness of the approach for different image classification benchmarks and two class-incremental learning protocols (disjoint and i-Blurry). The proposed method outperforms the state of the art in accuracy on these datasets.\n", "summary_of_the_review": "Overall, the paper proposes an interesting and original contribution to continual online learning with rather thourough experimental validation on several difficult benchmarks outperforming the state of the art.\n\n\n", "strength_and_weaknesses": "Strenghts:\n- Original approach with some theoretical justification and intuition\n- Experimental results are convincing\n\nWeaknesses:\n- The two data augmentations are somewhat \"handcrafted\" \n- Some parts are hard to follow (Section 3) without knowing OCM\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "4ROsXBzswW", "reviewer": "Reviewer_K6JM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work, the authors argue that learning only holistic representations is still insufficient. The learned representations should also be invariant and those features that are present in the data but are irrelevant to the class (e.g., the background information) should be ignored for better generalization across tasks. This new condition further boosts the performance significantly. This paper proposes several strategies and a loss to learn holistic and invariant representations and evaluates their effectiveness in online CL.", "summary_of_the_review": "The paper is well written and the authors analyze the problems of online continual learning from a novel perspective. However, the proposed method is not novel enough.\n\n", "strength_and_weaknesses": "Strengths :\n\n1. This paper analyze the problems of online continual learning from a novel perspective.\n2. The method is easy to follow, and the performance is improved to a certain extent.\n\nWeaknesses :\n\n1. The novelty of this paper is incremental. Data augmentation methods have been proposed a lot in continual learning, such as [1][2]. The authors should compare the proposed data augmentation methods with these methods.\n2. The idea of saving more samples proposed in the paper has already been proposed in continual learning, such as image compression [3], and the authors did not compare with the method.\n[1] Class-Incremental Learning via Dual Augmentation, NeurIPS 2021\n[2] Online Continual Learning Under Extreme Memory Constraints,ECCV2020\n[3] Memory Replay with Data Compression for Continual Learning, ICLR 2022\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "2anekoRCCCV", "reviewer": "Reviewer_YuYN", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper argues for invariant representations to enable continual learning. It proposes representations that generalize across environments to achieve invariance and shows that this helps reduce catastrophic forgetting.\n", "summary_of_the_review": "The paper clearly shows that adding the invariance term helps generalization to new classes while showing lower forgetting. This is an important contribution, so I am leaning towards acceptance. The main weakness would be not studying sub-population/domain shifts, however, I do not think this is critical for acceptance. Also, the paper needs to be carefully edited for clear acceptance.", "strength_and_weaknesses": "**Strengths**\n\n[S1] The proposed method of learning invariant features is important for continual learning. Improving invariance by constraining augmented views (environments) to have similar representations is sensible and empirically shown to improve generalization.\n\n[S2] The experiments clearly show better generalization to new tasks as well as lower forgetting of older tasks, all stemming from the proposed invariant feature learning. With clear benefits on both disjoint and blurry tasks, this technique should be the baseline to beat for newer works. \n\n[S3] The method shows large accuracy gains on multiple buffer sizes, showing that the invariance objective has better sample efficiency too.\n\n**Weaknesses**\n\n[W1] Since the choice of the augmentations (Aug color, Aug plus) are not clearly explained at the beginning, they seem arbitrary. Only in Sec 5.3, other schemes are discussed. I think the rationality for augmentation choices should be explained in Sec 4.1. \n\n[W2] The claim: ‘invariance is not important for single/multitask learning due to i.i.d. assumption, is not entirely true. Single/multitask learning can have ood assumptions too, and there are multiple works already addressing invariant representation learning albeit in a non-continual learning setting e.g., IRM [1]\n\n[W3] Invariance is probably better tested when there is a distributional shift. The experiments do not tackle sub-population or domain shifts, it only tackles shift in terms of new classes. But realistically, continual data streams even from same class may change over time (e.g., due to seasonal changes, inclusion of more domains e.g., race, noise characteristics etc), but this is not studied in this work.\n\n[1] Arjovsky, Martin, et al. \"Invariant risk minimization.\" arXiv preprint arXiv:1907.02893 (2019).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject