IMEX-Reg: Implicit-Explicit Regularization in the Function Space for Continual Learning
Authors: Prashant Shivaram Bhat
Bharath Chennamkulam Renjith
Elahe Arani
Bahram Zonooz
Venue: Accepted by TMLR ()

Abstract: Continual learning (CL) remains one of the long-standing challenges for deep neural networks due to catastrophic forgetting of previously acquired knowledge. Although rehearsal-based approaches have been fairly successful in mitigating catastrophic forgetting, they suffer from overfitting on buffered samples and prior information loss, hindering generalization under low-buffer regimes. Inspired by how humans learn using strong inductive biases, we propose \textbf{IMEX-Reg} to improve the generalization performance of experience rehearsal in CL under low buffer regimes. Specifically, we employ a two-pronged implicit-explicit regularization approach using contrastive representation learning (CRL) and consistency regularization. To further leverage the global relationship between representations learned using CRL, we propose a regularization strategy to guide the classifier toward the activation correlations in the unit hypersphere of the CRL. Our results show that IMEX-Reg significantly improves generalization performance and outperforms rehearsal-based approaches in several CL scenarios. It is also robust to natural and adversarial corruptions with less task-recency bias. Additionally, we provide theoretical insights to support our design decisions further.

Published in Transactions on Machine Learning Research (04/2024)
IMeX-Reg:
Regularization in the Function Space for Continual Learning
Prashant ; Bharath Renjith',, Elahe Arani Bahram Zonooz!.
'Einhhoven University of Technology (TU/e) "Wavv
{p. s. sbaat, e arariii, b. b.zonooz)@tue. nn, blarathcreejjiti@gaaa...cm
Reviewed OpenReview: http:://ppaaaeevveerrrerrrrrrrrr
Abstract
Continual learning (CL) remains one of the long-standing challenges for deep neural networks due to catastrophic forgetting of previously acquired knowledge. Although rehearsal-based approaches have been fairly successful in mitigating catastrophic forgetting, they suffer from overfitting on buffered samples and prior information loss, hindering generalization under low-buffer regimes. Inspired by how humans learn using strong inductive biases, we propose IMEX-Reg to improve the generalization performance of experience rehearsal employ a two-pronged implicit-explice
in CL under low buffer regimes. Specifically, regularization approach using contrastive representation learning (CRL) and consistency regularization. To further leverage the global relationship between representations learned using CRL, we propose a regularization strategy to guide the classifier toward the activation correlations the unit. hypersphere of the CRL. Our results show that TMEX- significantly improves generalization performance and outperforms rehearsal-baae based approaches in several CL scenarios. It is also robust to natural and adversarial corruptions with less task-recency bias. Additionally, we provide theoretical insights to support our design decisions further.
1 Introduction
Deep neural networks (DNNs) deployed in the real world frequently encounter dynamic data streams and must learn soquentially as the data bocomes increasingly accossible over time (Parisi et al., 2019). However, continual learning (CL) over sequence of tasks causes catastrophic forgetting (Mellskkey & Cohen 1989
phenomenon in which acquiring new information disrupts consolidated knowledge knowledge, and in the worst case, the previously acquired information is completely forgotten. Rehearsal-basee approaches (Aljundi et al. 2019
Buzzega et al.) 2020 Arani et al. 2022) that maintain a bounded memory buffer to store and replay samples from previous tasks have been fairly effective in mitigating catastrophic forgetting. In practice, however, the buffer size is often limited due memory constraints (such as on edge devices) and in longer task sequences due restricted sample-to-tast tastkk ratios. In such low buffer regimes, repeated learning on bounded memory drastically reduces the ability of CL models approximate past behavior, resulting in overfitting on buffered samples Bhht et al. 2022a), exacerbated representation drift at the task boundary (Jeeveswaran et al. 2023 and prior information loss [Zhang et al. 2020), impoding generalization across tasks.
Humans, on the other hand, exhibit a remarkable ability to consolidate and transfer knowledge between distinct contexts in ever-changing environments (Barnett & Ceci 2002), rarely interfering with consolidated knowlodge (French) 1999). In the brain, CL is mediated by plethora of neurophysiological processes that harbor strong inductive biases to encourage learning generalizable features, which require minimal adaptation when encountered with novel tasks. On the contrary, due to the lack of good inductive biases, DNNs often latch onto patterns that are only represcntative of the statistics of the training data (inz et al. 2019
* "Equal contribution. EEqual advisory role. "odde available https://gitthb www.///eerA--ee. Lab/IMEX

---

Published in Transactions on Machine Learning Research (04/2024)
Projection
C
head h
D:
Backbone f
DEU
6
Lop
Lecr
D m
Ler
O
12
Classifier 9im Projection gmip
Figure 1: Implicit Explicit Regularization in CL: TMEX-Reg employs CRL (Lrep) and consistency reg- ularization (C2, and (.r)) to bias the learning towards generalization. To further leverage desirable traits of learning on unit-hypersphere using CRL, IMEX-Reg aligns the geometric structures within the classifier projection's 's hypersphere with that of the projection head's hypersphere (Ceer) thereby compensating for the weak supervision under low-buller regimes.
Consequently, DNNs are typically suscept ible to changes in the input distribution KKoh et al. 2021 Hendrycks et al. 2021). Therefore, leveraging inductive biases to incorporate prior knowledge in DNN can bias the learning process toward generalization.
Regularization has traditionally been used introduce inductive bias in DNNs prefer some hypotheses others and promote generalization (Ruder] 2017). Multitask lcarning (MTL), form of inductive transfer that involves learning auxiliary tasks, acts implicit regularizer by introducing inductive bias without imposing explicit constraints on the learning objective (Ruder) 2017p. Sharing representations between related tasks in MTL helps DNNs to generalize better on the original task CCaruaaa 1997). Moreover, assuming that the tasks in MTL share a common hepothesis class, sharing representations across tasks primarily benefits tasks with limited training samples (Liu et al. 2019). Contrastive representation learning (CRI.) (Chen et al., 2020b; Khosla al.] 2020) as an auxiliary task MTL promotes generalization the shared parameters by maximizing the similarity between positive pairs and minimizing the similarity between negative pairs. A vast number of unsupervised CRL methods learn representations with unit-norm constraint, effectively restricting the put space to the unit hypersphere (e.g. He et al. (2020)). Intuitively, having the features live on the unit hypersphere leads to several desirable traits: Fixed-norm vectors are known to improve training stability in modern machine learning where dot products are ubiquitous (Xu & Durrett 2018). Moreover, if features of class are sufficiently well clustered, they are linearly separable with the rest of the feature space (Wang & Isola, 2020). In addition to implicit regularization using CRL as an auxiliary task, the alorementioned desirable characteristics can be further leveraged for explicit classifier regularization, thereby compensating for weak supervision under low buffer regimes.
We propose IMEX-Reg Reg, two-pronged CL approach aimed at implicit regularization using hard parameter sharing and multi-task learning, and an explicil regularization in the function space to guide the optimization miniatiiion of the CL model towards generalization. As CRL captures the global relationship between samples using instance discrimination tasks, we sock align the gcometric structures within the classifier hypersphere with those of the projection head hypersphere to compensate for the weak supervision under low buffer regimes. Our contributions are as follows:
2

---

Published in Transactions on Machine Learning Research (04/2024)
Inspired by how humans leverage inductive biases, propose IMEX-Reg, two-pronged implicit (Khosla et al. 2020) explicit AArani et al. 2022 regularization approach to mitigate catastrophic forgetting in image classification in CL.
As having the features lic on the unit-hypersphere leads to several desirable traits, we propose a regularization stralegy lo guide the classifier toward the activation correlations the unit-hyyersphere of the CRL.
We show that IMEX-Reg significantly improves generalization performance and outperforms rehearsal- based approaches in mitigating catastrophic forgetting in CL. IMEX-Reg is robust to natural and adversarial corruptions and well-calibrated with less task-recency bias.
We also provide theoretical insights on feature similarity between CRL and cross-entropy loss, and the Johnson-Linddenstrane (JL) lemma (Daguupaa & Gupta 2003) connecting our explicit regularization loss to support our design decisions better.
2
Related Works
baaser approaches: Continual learning on a sequence of tasks with non-stationary data distributions results in catastrophic forgetting of older tasks, as training the CL. model with new information interferes with previously consolidated knowledge (McClelland
al.) 1995 Parisi et al. 2019 Experience- Rehearsal (ER) (Ratelif)) 1990 Robins] 1995) is one of the first works to address calastrophic forgetting by explicitly maintaining memory buffer and interleaving previous task samples from the memory with the current task samples. Several works are built on top of ER to reduce catastrophic forgetting further in CL under low buffer regimes. The low-bulfer regime is characterized by a small sample-to-be to-takk ratio, signifying limited data availability per task, while the high-buffer regime entails a large sample-to-task ratio, indicating ample data per task, influencing the model's ability to handle CL challenges. The low-buffer regime poses significant challenges for CL systems, such as higher susceptibility to catastrophic forgetting and the need for effective memory consolidation and utilization. Deep Retrieval and Imagination (DRT) (Wang et al. 2022a) uses a generative model to produce additional (imaginary) data based on limited memory. ER-ACE (Caccia et al. 2022) focuses on preserving learned representations from drastic adaptations by combating representation drift under low buffer regimes. Gradient Coreset Replay (GCR) (Twaari et al., 2022) proposes maintaining a coreset to select and update the memory buffer to leverage learning across tasks in a resource-efficient manner. Although rehearsal- scenarios, they suffer from overfitting on buffered samples (Bhat et al.] 2022a),, exacerbated representation drift at the task boundary (Jeeveswaran et al.) 2023), and prior information loss (Zhang et al. 2020 in low buffer regimes, thus hurting the generalizability of the model.
methods are fairly effective in challenging CL
Regularization in Experionce-Rehearsee in reducing the generalization error in DNNs. Although the parameter norm penalty is one way to regularize the CL model, parameter sharing using multitask learning (Caruana 1997) can lead to better generalization
Regularization, implicit or explicit, is an important component
and lower generalization error bounds if there is valid statistical relationship between tasks (Baxter 1995).
Contrastive representation learning (Chen et al., 2020b He et al. 2020 Henaff 2020) that solves pretext prediction tasks to learn generalizable representations across a multitude of downstream tasks is an ideal candidate as an auxiliary task for implicit regularization. In CL, TARC (Bhat et al., 2022b) proposes a two-stage learning paradigm in which the model learns generalizable representations first using Supervised Contrastive (SupCon) (Khosla et al. 2020) loss, followed by a modified supervised learning stage. Similarly, C CCha et al., 2021) first lcarns representations using modified SupCon loss and then trains a classifier only on the last task samples and buffer data. However, these approaches require training in two phases and knowledge of the task boundary. OCDNet {Li et al. 2022) employs student model and distills relational and adaptive knowledge using a modified SupCon objective. However, OCDNet does not leverage the generic information captured within the projection head to further reduce the overfitting of the classifier.
Explicit regularization in the function space imposes soft constraints on the parameters and optimizes the learning goal to converge upon a function that maps inputs to outputs. Therefore, several methods opt to directly limit how much the input/output function changes between tasks to promote generalization. Dark Experience Replay (DER++) (Buzzega et. al.] 2020) saves the model responses in the buffer and applies
3

---

Published in Transactions on Machine Learning Research (04/2024)
consistency regularization while replaying data from the memory buffer. Instead of storing the responses in the bulfer, the Complementary Learning System-based ER (CLS-ER) (Arani et al. 2022) maintains dual semantic memories to enforce consistency regularization. However, in addition to consistency regularization, these approaches might further benefit from multitasking and explicit classifier regularization to enable further generalization CL
Inter-task class separation: The problem of inter-task class separation in Class-II, remains a significant challenge due to the difficulty in establishing clear boundaries between classes of current and previous tasks (Lesort et al., 2019). When a limited number of samples from previous tasks are available in the buffer, the CL model lends to overfit on the buffered samples and incorrectly approximates the class boundaries between classes from current and previous tasks. Kim & Choi (021 splits the Class-IL problem into intra-old, intra-new and cross-task knowledge. The cross-task knowledge specifically addresses the inter-task class separation through knowledge distillation. Similarly, Kim et al. (2022) decomposes the Class-IL into two sub-problems: within-task prediction (WP) and task-id prediction (TP). Essentially, a good WP and good TP or out-of-distribution detection are necessary and sufficient for good Class-IL performance (Kim et al., 2022). On the other hand, some approaches propose to address inter-task forgetting without decomposition. Attractive and repulsive training (ART) (Choi & Choi 2022), which effectively captures the previous feature space into set of class-wisc flags, and thereby makes old and new similar classes less correlated in the new feature space. LUCIR (Hou et al., 2019) incorporates cosine normalization, less-forget constraint, and inter-class separation, to mitigate the adverse effects of the imbalance in Class-IL. Although these approaches address fine grained problem of inter-task class separation in Class-IT, catastrophic forgetting is greatly reduced as a direct consequence.
Memory Overfitting: Rehearsal-based approaches that store and replay previous task samples have been quite successful in mitigating forgetting across CL scenarios. These strategies, however, suffer from a common pitfall: as the memory buffer only stores a tiny portion of historical data, there is a significant chance that they may overfit, hurting generalization (Verwimp et al.
2021 Therefore, several methods resort to augmenta aion techniques either by combining multiple data points into one (Boschini et al., 2022) or by producing multiple versions of the same buffer data point (Bang et al.) 2021). Gradient-based Memory EDiting (GMED) (Jin 2021 proposes to create more "challenging" examples for replay by providing a framework for editing
ct al. stored examples in continuous input space via gradient updates. Instead of individually editing memory data points without considering distribution uncertainty, Distributionally Robust Optimization (DRO) (Wang et al. 2022b) framework focuses on population-level and distribution level evolution. On the other hand, Lipschitz Driven Rehearsal (LiDER) (Bonicelli et al.) 2022) proposes a surrogate objective that induces smoothness in the backbone network by constraining its layer-wise Lipschitz constants w.r.t. replay examples. As many of these approaches are intended to be orthogonal to the existing rehearsal-based approaches, they allow for seamless integration and effective reduction in memory overfitting in CL.
Inductive bias: : Fundamentally, learning benefits from prior knowledge about the types of problems to be solved. The incorporation of such prior knowledge into an optimization system is facilitated through inductive biases. In the context of the brain, continual learning (CL) is underpinned by a variety of neurophysiological processes that embody robust inductive biases, fostering the acquisition of generalizable features that demand minimal adaptation when confronted with novel tasks (Kudithipudi et al. 2022 Significantly, these inductive biases have evolved to enhance an organism's adaptability and survival within the broader ocological landscape (Richards al.] 2019). Notably, (i) many species, particularly humans, undergo protracted developmental periods characterized by extensive experiential learning, and (ii) deep neural networks have demonstrated proficiency in low-data settings owing to their effective inductive biases (Snell et al. 2017
Traditionally, regularization techniques have served to imbibe DNNs with inductive biases, favoring certain hypotheses over others and promoting generalization. To this end, we propose intertwining implicit and explicit regularization to promote gencralization in CL under low buffer regimes. This leverages generic representations learned within the projection head to compensate for weak supervision under low buffer regimes.
4

---

Published in Transactions on Machine Learning Research (04/2024)
3 Method
Continual learning typically consists of t .T, sequence of tasks with the model learning one task at a time. Each task is specified by a lask-specific data distribution Dj with {(r,, bi)l pairs. Our CL model $ = [{. glin 9mlp; h consists of four randomly initialized, learnable components: a shared backbone f, a linear classifier Rönr an MLP classifier projection 9mip: and a projection head h. The classifier gii represents all the classes that belong to all the tasks, and the projection head captures the emboddings of the no-omalleee representation. Classifier embeddings are further projected onto unit hypersphere using another projection network Imlp- CL is especially challenging when data pertaining to previous tasks vanish as the CL model progresses to the next task. Therefore, to approximate the task-specific data distributions seen previously, we seek to maintain a memory buffer Dm using Reservoir sampling (Vitter) 1985) (Algorithm 21. To restrict the empirical risk on all tasks seen so far, ER minimizes the following objective:
Cer
1 b
E (a.))b, bED,UD,
(oi (f())))
(1)
where b is a training batch, Lee is cross-entropy loss, is the index of the current task, and af.) is the softmax function. When the buffer size is limited, the CL. model learns features specific to buffered samples rather than representative features that are class- or task-wide, resulting in poor performance on previously seen tasks, Therefore, we propose IMEX-Reg, aimed at implicit regularization using parameter sharing and multitask learning, and explicit regularization in the function space to guide the optimization of the CL model towards gencralization. We describe in detail the different components of our approach in the following sections.
3.1 Implicit regularization
We seek to learn an auxiliary task that complements continual supervised learning. We consider CRL using SupCon (Khosla et al. 2020) loss as an auxiliary lask to accumulate generalizable represent ations shared parameters. Ideally, CRL involves highly correlated multiple augmented views of the same sample which are then propagated forward through the encoder and the projection head h. It is a common practice within CRL literature to employ non-linear projection head h after CNN backbone f as it improves representat ion quality (Chen et al.) 2020c). To learn visual representations, the CL model should learn to maximize cosine similarity (fy-normalized dot product) between the positive pairs of multiple views while simultaneously pushing away the negative embeddings from the rest of the batch. The loss takes the following form:
Crep Σ NEI
Σ (z) / log PEP())
Σ exp ((2).222))1) ]
P(J)
(2)
(EN()
where = == =(f(.) is any arbitrary 128 28-dimensional fz-normalized projection, T is temperature parameter, I is a set of indicos, N(1)= = Mi) is a set of negative indices, P(i) = {p E N(1) : Up = y: is a set of projection indices that belong to the same class as the anchor 2 and [P()) is its cardinality. In the following conjecture, we provide intuition behind choosing CRL as an auxiliary task CL.
Conjecture 1. (Feature similarity) Features learned by f through CRL are similar to those learned via cross-entropy as long as: (i) The augmentation in CRL do not corrupt semantic information, and (ti) The labels in cross-entropy rely mainly on this semanlic information (Wen & Li 2021).
Let at and x be two augmented positive samples such that y = yp Furthermore, we assume that our raw data samples are generated in the following form: Fpp = Sp + & where Go represents semantic information in the image, while &p ~ D3- -N(0, o)) represents spurious noise. Given semantic preserving augmentations, (Wen & Li 2021] state that similar discriminative features are learned by contrastive learning and cross-entropy. Similarly, since the CRL in Equation 2 employs both semantic preserving augmentations and labels to create positive pairs, we assume that the inner product of semantic information (Eç) z(,) will overwhelm that of the noisy signal ( "gt-)) As we expect labels in cross-entropy to focus semantic features to learn classification, we hypothesize that both CRL and cross-entropy share a common hypothesis class and sharing representations across these tasks especially benefits CL under low buffer regimes.
5

---

Published in Transactions on Machine Learning Research (04/2024)
3.2 Explicit regularization
A CL model equipped with multitask learning implicitly encourages the shared encoder I' to learn generalizable features. However, the classifier Rün that decides the final predictions is still prone to overfitting on buffered samples under low buffer regimes. Therefore, we seek to explicitly regularize the CL model in the function space defined by the classifier Jii.- To this end, we denote the output activation of the encoder f as F E R that of the projection head h as Z E R and that of the classifier projection gmdpp as E RbxD,, where Dj, Dg Dr denote the dimensions of output Euclidean spaces. Let Fg : IR + IR and Fn R2 + Pt be the function spaces represented by the classifier Jiin and the projection head h. Let 0 and 0eNa be parameters of the CL. model and its corresponding exponential moving average (EMA). Following CLS-ER (Arani el al.
2022. we stochastically update the EMA model as follows:
0 0au I1 OSMa + (1-1)0, if u( (eNAA
otherwise
where 1) is a decay parameter and y is an update rate. The EMA of model can be considered to form a self-ensemble of intermediate model states that leads to better internal representation (Arani et al. 2022)
Therefore, we leverage the soft targets (predictions) of the EMA model to regularize the learning trajectory in the function spaces Fg and Fn of the CL model:
E (0 D
11 Del 1/
cr
(4)
Ch
E (D).M))~D...
|| - zel1?
where ||r respectively, and & and de are those of the EMA model. As soft targets carry more information per training sample than ground truth labels (Hinton et al. 2015), knowledge of previous tasks can be better preserved by ensuring consistency in predictions, leading to drastic reductions in overfitting.
||r is the Frobenius norm, z and 9 are the projection head and classifier responses of the CL model,
It is pertinent to note that restricting the output space to a unit hypersphere in representation learning not only enhances training stability but also creates well-clustered projections in the hypersphere that are linearly separable from the rest of the samples (Wang & Isola| 2020). We hypothesize that by regularizing the classifier's unit hypersphere using the projection hcad's unit hypersphere could potentially guide the classifier's decision space to discern more effective boundaries. As semantically similar inputs tend to elicit similar responses, we seek to align geometric structures within the classifier's 's hypersphere with that of the projection head's hypersphere to further leverage the global relationship between samples established using instance discrimination task. We assume that there exist a mapping function M: Rh - R.,, and its inverse M-1 DR + DR. that establish a connection between the geometric relationship between the points both hyperspheres. When learning in the hypersphere, angular information rather than magnitude forms the key semantics in the CL model (Chen et al., 2020a). Therefore, to guide the classifier loward the activation correlations in the unit hypersphere of the projection head, we regularize the differences in the outer products of Z and C, i.c.,
GA. = = E R
(5)
Gg CCT R.b
1
Leer
|toopgadd(Gn) Gglli
(6)
where stopgrad(.) ensures that the backpropagation of gradients occurs only through the classifier. Equation regularizes both the classifier and the projection head using the EMA of the CL model, while Ceer in
Equation 6 captures the mean element-wise squared difference between G2, and G2 matrices of the CL model.
Theorem 2. JJohnson-Lindenstree
Lemma): Let (0,1) and Dg O be such that for any integer Dg 2 4 (e2/2 c²/3) Inn. Then, for any set of points Z E Ph, there exists mapping function
M: : R Da + D2 such that for all pairs of samples p.q
(1 l)pp -112 (lM(p) M(g)ll (1 +e]lp q
6

---

Published in Transactions on Machine Learning Research (04/2024)
Algorithm 1 Proposed Method: IMEX-Reg
Input: Data streams Dr, Model to = 9mlpp hi,, Hyperparameters a, and X, Memory buffer
Dm f
for all tasks & (1, 7) do
for all Iterations E {1, 21,2,.. .., }} do
4:
==
5:
Sample a minibatch (X,, Y)) E DD
6:
Ft f(xe) Ye, Ze, -.= (FF), h(F)), (mF))
7:
if Dm + then
Sample minibatch (Xm; Ym)) E Dm Fm f(Xm) Y, Z, C= =g((mm)) h(Fm), 9mlp (0(p(((Fm))
10:
11:
12:
Fe =f((mm)
13:
Ye, Ze, = =g
(m)) he (e(Fm), Smlp. (ge(Fm))
14:
C C A [L + L.crl
Equation
15:
C + fer a Lrep + 3 foor
b Equations and 6
16:
Update 48 and Dm Update Pema return model t
17:
Equation
3
18:
Johnson-Ländenstrauss (JI.) lemma (Dasgupta & Gupta 2003 states that any p points a high dimensional Euclidean space can be mapped onto & dimensions where 2 O (logp/e) without distorting the Euclidean distance between any two points more than factor of + Under JL. lemma, possible to map geometric relationship between points in higher dimensional projection head hypersphere to lower dimensional classifier hypersphere without the loss of generality. To this end, we propose a mapping function Leer that preserves the geometric structures when mapping from projection head to classifier. Application of JL lemma in this context implies that the distortion in geometric relationship between points when mapping from projection head to classifier hypersphere can be limited to e.
hypothesize that it is
3.3 Putting it all together
During training, the batches of the current task are propagated forward through by to obtain classification and projection embeddings. Specifically, & learns generalizable features through Equation P and task-specific features through Equation [ To better consolidate the information pertaining previous tasks, we maintain memory buffer D., and an EMA of the CL model, which also serves We enforce consistency in predictions on rehearsal data using Equation ! To further reduce overfitting and discourage label bias in the classifier, we seek to emulate geometric structures using Equation 0 During each training iteration, the memory buffer is updated using Reservoir sampling (Vitter 1985) and the EMA is stochastically updated using Equation The overall learning objective as follows:
inference model for evaluation.
L
Ler + Crep Leer
E (z.8)~Dm.
A [C2. ch
where 3 and À are hyperparameters. Except for stopgrad(.) in Equation 5 the entire network receives weight updates through different loss functions in Equation IMEX-Reg is illustrated in Figure and is detailed in Algorithm [
4 Experiments
4.1
Experimental setup
We build on top of the Mammoth (Buzzega et al. 2020 CL repository in PyTorch. We evaluate CL models under Class- Tlass-nncrementll Learning (Class-IT.),
Learning (Task-IT.), and Generalized Class-IT.

---

Published in Transactions on Machine Learning Research (04/2024)
Table Top-1 accuracy (%) of different CL models Class-II. and Task-IL, scenarios with varying complexities and memory buffer sizes. The best results are marked in bold.
Seq-C CIFARIO
BeF-CIFARI00
Seq- ImyylmaeeNet
Buffer
Methods
Venue
Class-IL
Task-IL
Class-IL IL
Task-IL
Class-IL
Task-IL IL
SGD Joint
19.62+0.05
61.02+3.33
27.49=0.28
40.46=2.99
07.92+0.26
18.31
02.20+0.15
98.31
70.56=0.28
86. 16.1940.43
59.99+0.19
82.0440.10
ER
1.87
91.19
21.40=0.22
61.36:20.55
8.57=0.01
38.17:2.00
ER-ACE
ICLR'22
62.08
92.20
35.17=1.17
63. 0..09
11.25=20.51
44.17+1102
GCR
CVPR'22
1..
90.8
13.69
61.22
13.05::0.91
42.11:1.01
DRI
2AAI'2
1..16
92.87
17.58
44.28
200
DER ++
14.8
91.92+0.60
29.60 1.14
62.49::1.00 1.02
10.96
40.87+1.16
CLS-ER
ICLR'22
66.19
93.90
43.80=1.89
73.49
23.47:10.80
49.60:10.2
Co
ICCV'21
65.57
93.43+1077
31 3.000
55.02220.6
13.88+00.40
42.37:10.74
OCDNet
IJCAT'22
73.38
40.43
44.29=00-4
73.53:20.24
17.60:10.97
56. 1911911.31
MMEx-Reg
71.56:10.18
94.77=0.81
48.5420.23
75.61=0.78
24.15
62.91
ER
57.74+0.27
93.61#0.25
20.01
28.23:20.17
9.99
48. 644:0.46
ER-ACE
ICLR'22
48.45
93.47#1.00
40.67=0.06
66.45=2011
17.73+t0.50
49 19,.9911.1
GCR
CVPR'22
74.69:20.85
94.44+00.22
45.91=1.30
71.64:2.10
52.99:10.89
DRI
AAAT222
72.78
93.85+00466
22.63+:0.81
52.89
500
DER++
2e
1.
93.58
11.41000.96
70.61 #:008
19.38
51.91
CLS-ER
ICLR'22
75.22
94.944+0.53
51.40=1100
78. 18.10.2..2
31.03::0.56
60.41
Co
ICCV'21
74.26
95.90
39.21 =0.39
62.98
10.1220..4
53 53.04+0.69
OCDNet
IJCAT'22
80.6440.77
96.57+t007
54.13=0.30
78.51
26.09
74.76:2
IMEX-Reg
77.61
95.96+0.33
56.53=0.80
80.51 100.0
31.41 +:021
67.44
(GCIL) (Van de Ven & Tolias 2019 Arani et al. 2022). More information on the datasets, task partition, and the corresponding network architecture used in these scenarios can be found in Appendix A To provide a comprehensive analysis, we compare IMEX-Reg with several approaches that aim to improve generalization under low buffer regimes in CL. We consider ER-ACE, GCR, DRI (aimed at improving generalization) DER++ CLS-ER (use explicit consistency regularization by leveraging soft targets), Co²L and OCDNet (representation and/or auxiliary CRL) as our baselines. Furthermore, we provide lower bound 'SGD', without using any mechanism to minimize catastrophic forgetting, and an upper bound 'Joint', where the training is carried out using the entire dataset. We report the average accuracy along with the standard deviation on all tasks after CL training with three random seeds. We also provide the results of the forgetting analysis.
4.2 Experimental results
Table presents the comparison of our method with the baselines for the Class-IL and Task-IL settings. Several observations can be made from these results: (i) Although CL. methods aimed at generalization improve over ER, they lack strong inductive biases proposed in this work, thus failing to make significant improvements low-buffer regimes. (ii) Explicit consistency regularization shows great promise in reducing overfitting over ER. As can be seen, DER++, DRT, and CLS-ER show significant reductions catastrophic forgetting compared to ER. However, IMEX-Reg incorporates implicit and explicit regularization to promote generalization and outperforms these methods by a large margin in most scenarios. In Seq- TinyImageNer IMEX-Reg outperforms CLS-ER by relative margin of 2.9% and 1.2% in buffer sizes 200 and 500 respectively. Note that CLS-ER employs two semantic memories, while IMEX-Reg uses only one.
C and OCDNet generalize well across tasks in CL, showing the efficacy of CRL in CL. Specifically, OCDNet combines consistency regularization and CRL in addition to self-super rotation prediction. OCDNet outperforms IMEX-Reg Scq-CIFAR10 in both buffer sizes, greatly benefiting from rotation prediction. However, OCDNet lags behind IMEX-Res -eg by large margin in more challenging datasets. Essentially, IMEX-Reg compensates the classifier's weak supervision with the generic geometric structures learned by the
8

---

Published in Transactions on Machine Learning Research (04/2024)
Table Top-1 accuracy (%) of different CL. models for Uniform and Longtail GCII-CIFARI00 settings with different memory buffer sizes. The best results are marked in bold.
Method
Uniform
Longtail
SGD
10.38:00.20
9.61=0.19
Joint
18.59.1195
58.4211.32
Buffer
100
200
500
100
200
500
ER
14.43.1..66
16.52.0.10
23.62.10.66
13.22.10.6
16.2010.30
22.36
ER-ACE
23. 1.661
27.644:0.7
30. 1.11
23. 28.05=0.118
25.10=2.64
31.88=0.73
DER++
21.17=+166
27.73++0.9
35.83+0062
10.29=1108
26.48=2.07
14.1
CLS-ER
33.42:+0.30
35.88:+0.41
38.9440.38
33.92=0079
35.67=0.72
38.79 38.79
OCDNet
37.21+0.69
39.94+0.05
43.58:00677
35.61+1.18
30.97=0.70
43.57=0.23
IMEX-Reg
39.48.10.2
43.1910.47
49.07.10.9
38.39 00.46
42.66 2..82
46.81 1104
Table 3: Forgetting analysis for various CL models across datasets in Class- IT, setting.
Buffer
Methods
Seq-CIFAR10
Seq- CIFAR100
Seq- TinyImageeet
ER
61.24+2.02
75.5440.44
76.37=0.53
+
32.5942.32
1..7721.72
72.74=0.50
200
OCDNet
22.6342.06
33.41
20.37:1.00
IMEX-Reg Reg
24.694+0.41
32.19:0051
27.93=2299
45.35-10.07
67.7411.29
75.27
DER+ 22.38+4.41
50.99
64.58=2.01
500
OCDNet
14.93+1.42
21.6+0016
17.88+0.56
IMEX-Reg
17.00:10.40
18.83
30.47:1.35
projection head through an explicit regularization, achieving a relative 9.6% and 37.22% improvement over OCDNet in CIFAR1000 and Bigg-innImmar ImaeeNt,,, respectively, for a low buffer size of 200. This reinforces our earlier hypothesis Conjecture that leveraging desirable traits from CRL can indeed guide the classifier towards gencralization under low buffer regimes.
Generalized Class-IL (GCIL) exposes the CL model to more challenging and realistic learning scenario by using probabilistic distributions to sample data from the CIFAR100 dataset in each task (Mi et al. 2020 Table 2 shows the comparison of different CL methods in GCIL setting under two variations, Uniform and Longtail (class imbalance). As can be seen, IMEX-Reg outperfomms all baselines by a large margin across all buffer sizes. IMEX-Reg performs significantly better than ER, ER-ACE, DER++ and CLS-ER, emphasizing the importance of implicit regularization through auxiliary CRL in improving generalization, Although OCDNet combines auxiliary CRL and explicit consistency regularization, OCDNet falls behind IMEX-Reg by considerable margin across all GCIL scenarios. Fiven under class imbalance (Longtail) and low buffer size of 100, IMEX-Reg achieves relative improvement of 7.8% over OCDNet. These results indicate that IMEX-Reg learns generalizable features through auxiliary CRL while enriching the classifier through explicit regularization the function space, making it suitable for challenging CL. scenarios.
4.3 Forgetting Analysis
Continually learning sequence of novel tasks often interferes with previously learned information resulting in catastrophic forgetting. Therefore, in addition to the average accuracy of the previous tasks, it is crucial to measure how much of the learned information is preserved. The forgetting measure (ff²) (Chaudhry et al. 2018) for a task after learning & tasks is defined as the difference between the maximum knowledge gained for the task in the past and the knowledge currently in the model. Let Aij be the test accuracy of the model for task j after learning lask then,
max Atj Auj. Vj < 1E(istL..,k-1)
(9)
9

---

Published in Transactions on Machine Learning Research (04/2024)
Seq CIFAR10
Seq CIFAR100
Seq TinyTmagegeeet
Stability
80
Plasticity
Accuracy (%6
Trade-off off
60
40
20
ER
DER++ IMENetIMEX-Res
ER
+ OCDNetIMEX Reg
ER
DER+ + OCDNetIMEX Reg
Figure 2: Comparison of Stability-Plassiiityy Trade-off for different CL models across different datasets.
Then the average forgetting mcasure for the model after learning tasks is given by
Fr
f T-1 f=0
(10)
The lower Fr signifies less forgetting of previous tasks. Typically, Ai is computed at the task boundary after learning the task i. However, since the EMA is updated stochastically, the maximum accuracy for a previous task is not necessarily achieved the task boundaries. Therefore, we evaluate the EMA on previous tasks after every epoch and keep track of the maximum accuracy observed for the previous tasks. Forgetting analysis of IMEX-Reg is then computed with respect to the maximum accuracy observed for the previous lasks and the final evaluation accuracy for the tasks at the end of training.
Table 3 shows the average forgetting measures for different CL methods across different datasets and buffer sizes in a Class-IL sctting. IMEX-Reg and OCDNet achicve significantly lower forgetting than other basclines, owing to the ability the EMA to preserve the consolidated knowledge. In Seq- Togg-Tinnmmageeet. OCDNet suffers from far less forgetting than IMEX-Reg. However, this can be attributed to how often the EMA is updated. By restricting the update frequency EMA, we can considerably reduce forgetting but at the cost of adapting to new information. An ideal CL model should not be too restrictive and instead should find optimal balance between forgetting and learning novel tasks.
4.4 Stability-Plasticity Trade-off
Figure 2 comparcs the stability, plasticity, and tradc-off (Appendix A.10 of different CL methods across different datasets for a buffer size of 500. ER and DER++ have high plasticity and quickly adapt to novel information; yet, they do not retain previously learned information. On the other hand, IMEX-Reg and OCDNet employ an EMA that stochastically updated and better preserves consolidated knowledge. Hence, both IMEX-Reg and OCDNet achieve much higher stability a relatively lower cost of plasticity, resulting in higher stability-plastiiityy trade-off. Tn Table 3 we see that OCDNet suffered the least forgetting in Seq- TinyImageNee However, Figure shows that OCDNet achieved the least plasticity in Seq- Tog-TinylmmaeNee IMEX-Reg, the other hand, achieved much better (rade-off, even though it forgets more than OCDNet.
5 Model Characteristics
Task Recency Bias. Learning continuously on sequence of tasks biases the model predictions toward recently learned lasks in the Class-IL scenario (Hou et al.) 2019). An ideal CL model is expected to have the least bias, with predictions evenly distributed across all tasks. To gauge the task-recency bias in IMEX-Reg we compute the average task probabilities by averaging the softmax outputs of all samples associated with each task on the test at the end of the training. Figure Frghtt)r righht shows the task-recency bias of different CL. models trained CTFAR100 with buffer size 200. Evidently, TMEX-Reg predictions more evenly
10

---

Published in Transactions on Machine Learning Research (04/2024)
40
ER
Task
0.8
DER++
Task
Task Probability
Accuracy (%)
OCDNet IMEX-Reg
Task
0.6
Task
20
Task
0.4
10
0.2
0.0
0.25
1.0 Attack Intensity
0.5
2.0
ER
DER++
OCDNet
IMEX-Reg
Figure 3: (Left) Robustness to PGD adversarial attack at varying strengths and (Right) Average probability of predicting each task for different CT methods trained on Seq-CTFAR100 with 5 tasks. IMEX-Reg shows the highest robustness and the least recency bias with probabilities evenly distributed across tasks.
distributed with least recency bias. IMEX-Reg induces robust inductive biases that skew the learning process towards generic representations instead of aligning more with the current task, thereby reducing recency bias.
Robustness to Adversarial Attacks. Adversarial attacks generate specially crafted images with imperceptible perturba- tions to fool the network into making false predictions (Szegedy eet al.] 2013). We analyze adversarial robustness by perform- ing a PGD-10 attack [Madry et al., 2017) with varying attack intensitios on different models trained on Seq-CIFAR100 with a buffer size 200. Figure leeft) shows that IMEX-Reg is more resistant to adversarial altacks compared to other base- lines. OCDNet and IMEX-Reg are significantly more robust than + and ER even at higher attack intensitics, owing to the auxiliary CRL that encourages the model to learn gener- alizable features. However, IMEX-Ree Reg further biases the model towards generalization through explicit classifier regularization and outperforms OCDNet across all attack intensities. Thus, inducing right-inductive biases in the model can improve robustness in addition improved performance.
OCDNet
Reg
Frost
Fog
Elastic transform
Gaussian
Defocus blur
Gaussian
Glass blur
Brightness
Impulse noise
mCA
0.20440...8
JPEG compress
Motion blur
Speckle noise
Pixelate
Spatter
Saturate shot noise Snow
Figure 4: Relative top-1 accuracy (%) (av- eraged over severity levels) for 19 different natural corruptions for different CL mod- els trained on Seq-CIFAR CIFAR100 with 5 tasks. The average accuracy across all corruptions is shown as mCA.
Robustness to Natural Corruptions: Data in the wild is often corrupted by changes in illumination, digital imaging artifacts, or weather conditions. Therefore, autonomous agents deployed in the real world should be robust these natural corruptions, especially in safety-critical applications. To eval- uate the robustness to natural corruptions, we test several CL methods trained
Seq- CIFARO0 with buffer size 500 on CIFAR100-C (Hendrycks & Dietterich 2019
Figure 4 shows the accuracy of different CL models compared to IMEX-Reg for 19 different corruptions averaged at five severity levels. Although OCDNet combines knowledge distillation and CRL, is unable to improve its robustness over DER++ However, IMEX-Ree further enriches the classifier with the generic geometric structures learned the projection head, proving to be more robust to natural corruptions. Thus, the intertwining of implicit and explicit regularization in IMEX-Reg enables generalization not just across tasks, but also across domains that drastically different. from the training set.
11

---

Published in Transactions on Machine Learning Research (04/2024)
Perfect Calibration
Output
Gap
100
Top-1 Accuracy (%)
80
ER
DER++ ECE: 22.49
OCDNet ECE: 6.23
IMEX-Reg ECE: 4.50
ECE: 59.30
60
40
20
8.00
0.25 0.50 0.75 1.000.00 0.25 0.50 0.75 1.000.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 Confidence
Figure 5: Reliability diagrams with Expected Calibration Error (ECE) for CL methods trained Seq- CIFAR100 with 5 tasks. The lower ECE value signifies better calibrated model. Compared to baselines, IMEX-Reg is well-calibrated with the lowest ECE value.
Model Calibration CL systems deployed in the real world are expected be reliable by exhibiting a sufficient level of prediction uncertainty. Expected Calibration Error (ECE) provides a good estimate of reliability by gauging the difference in expectation between confidence and accuracy (Guo et al.] 2017 Figure 5 shows the comparison of our method with other baselines using a calibration framework (Küppers et al. 2020). Compared to other baselines, IMEX-Reg achieves the lowest ECE value and is considerably well-calibrated. In addition to improving generalization, regularizing the classifier implicitly through the auxiliary CRL and explicitly in the function space by aligning with geometric structures learned in the projection head prevents the model from being overconfident.
5.1 Ablation Study
We seek to provide a better understanding of the contribu- tions of each of the components in our proposed method. Table 4 shows the ablation study of IMEX- trained on CTFACI0 for a buffer size of 200 with 5 tasks. In line with our earlier hypothesis, intertwining implicit regulariza- tion using parameter sharing and multitask learning, and explicit regularization in the function space guide the op- timization of the IMEX-Reg towards better generalization representations. As can be seen, each of these components has significant impact on the performance of IMEX-Res Reg under a low buffer regime. Additionally, leveraging generic geometric structures learned within CRL through explicit classifier regularization complements weak supervision under low buffer regimes and results in lower generalization error.
Table 4: Comparison of the contributions of each of the components in IMEX-Reg. The absence of EMA implics consistency regular- ization by storing past logits.
Lor EMA
Lrep
Leer
Accuracy
V
48.5440.23
X
06.85+1048
/
X
X
43.38++106
X
29.60+1.14 X
X
X
x
21.40
6 Conclusion
We propose (MEX-Reg, two-pronged CL approach aimed at implicit regularization using hard parameter sharing and multitask learning, and an explicit regularization in the function space to guide the optimization of the CL model towards generalization. The novelty of our method mainly lies in emulating rich geometric structures learned within the projection head hypersphere to compensate for weak supervision under low buffer regimes. Through extensive experimental evaluation, we show that IMEX-Reg significantly benefits from each of these strong inductive biases and exhibits strong performance across several CL sccnarios. Furthermore, we show that IMEX-Reg is robust and mitigates task recency bias. Leveraging unlabeled data through CRL in IMEX-Reg could further improve generalization performance across tasks in CT..
12

---

Published in Transactions on Machine Learning Research (04/2024)
Acknowledgments
The research was conducted when all the authors were also affiliated with Advanced Research Lab, NavInfo Europe, The Netherlands.
References
Rahaf Aljundi, Min Lin, Baptiste Goujand, and Yoshua Bengio. Gradient based sample selection for online continual learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bucc E. Fox, and R. Garnett (cds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based on complementary learning system. In International Conference on Learning Representations, 2022.
Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung- Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with a memory of diverse samples. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 2218-227,, 2021.
Susan M Barnett and Stephen J Ceci. When and where do apply what learn?: A laxonomy for far transfer. Psychological bulletin, 128(4):612, 2002.
Jonathan Baxter. Learning internal representations. In Proceedings of the eighth annual conference on Computational learning theory, pp. 311-320, 1995.
Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. Consistency the key further mitigating catastrophic forgetting in continual learning. In Conference Lifelong Learning Agents, pp. 1195-1212. PMLR, 2022a.
Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. Task agnostic representation consolidation: a self-supervised based continual learning approach. In Conference on Lifelong Learning Agents, pp. 390 405. PMLR, 2022b.
Lorenzo Bonicelli, Matteo Boschini, Angelo Porrello, Concetto Spampinato, and Simone Calderara. On the effectiveness of lipschitz-driven rehearsal in continual learning. Advances in Neural Information Processing Systems, 35:31886 1at:188663191,, 2022.
Matteo Boschini, Pietro Buzzega, Lorenzo Bonicelli, Angelo Porrello, and Simone Calderara. Continual semi-supervised learning through contrastive interpolation consistency. Pottern Recognition Letters, 162: 9-14, 2022.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 15920-15930. Curran Associates, Inc., 2020.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In International Conference on Learning Representations, 2022.
Rich Caruana. Multitask learning. Machine learning, 28:41-78 75, 1997.
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co21: Contrastive continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9516 9525, 2021.
Arslan Chaudhry, Puncct K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the Europeon Conference Compuler Vision (ECCV), pp. 532 547, 2018.
13

---

Published in Transactions on Machine Learning Research (04/2024)
Beidi Chen, Weiyang Liu, Zhiding Yu, Jan Kautz, Anshumali Shrivastava, Animesh Garg, and Animashree Anandkumar. Angular visual hardness. In International Conference on Machine Learning, pp- 1647-1648.
PMLR, 2020a.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597 1607. PMLR, 2020b.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and GeolTrey E Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33: 22243 22255, 2020c.
Hong-Jun Choi and Dong-Wan Choi. Attractive and repulsive training to address inter-task forgetting issues in continual learning. Neurocomputing. 500:486-498, 2022.
Sanjoy Dasgupta and Anupam Gupta. An elementary proof of a theorem of Johnson and Lindenstrauss. Random Structures & Algorithmss, 22(1):60-66) 65, 2003.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognilive sciences, 3(4): 128-135, 1999.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321 1330. PMLR, 2017.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729-97229 9738, 2020.
Olivier Henaff. Data-efficient image recognition with contrastive predictive coding. In International conference machine learning, pp. 4182-4192. PMLR, 2020.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8340 8349, 2021.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.00 02531, 2(7), 2015.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVE conference on computer vision and puldern recognition, pp. 831-839, 2019.
Kishaan Jeeveswaran, Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. Birt: Bio-inspired replay in vision transformers for continual learning, In International Conference Machine Learning, pp. 14835. PMLR, 2023.
Xisen Jin, Arka Sadhu, Junyi Du, and Xiang Ren. Gradient-based editing of memory examples for online task-free continual learning. Advances in Neural Information Processing Systems, 34:29193 22229,9-2225,, 2021.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Tsola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Newral Information Processing Systems, 33:18661 18118661-1663,, 2020.
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu. A theoretical study on solving continual learning. Advances in Neural Information Processing Systems, 35:5065-555 50799 2022.
14

---

Published in Transactions on Machine Learning Research (04/2024)
Yoog-Yong uoo Kim and Dong- Wan Choi. Split-and-bridge Adaptable class incremental learning within single neural network. In Proceedings of the A AAT Conference on Artificial Intelligence, volume 35, pp. 1137-8145, 2021.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani,
Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664 PMLR, 2021.
Dhircesha Kudithipudi, Mario Aguilar-Simon, Jonathan Babb, Maxim Bazhenov, Donglas Blackiston, Josh Bongard, Andrew P Brna, Suraj Chakravarthi Raja, Nick Cheney, Jeff Clune, et al. Biological underpinnings for lifelong learning machines. Nature Machine Intelligence, 4(3):196-22) 210, 2022.
Fabian Küppers, Jan Kronenberger, Amirhossein Shantia, and Anselm Haselhoff. Multivariate confidence calibration for object detection. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020.
Timothée Lesort, Andrei Stoian, and David Filliat. Regularization shortcomings for continual learning. arv preprint Xr:::11912.00 03049, 2019.
Jin Li, Zhong Ji, Gang Wang, Qiang Wang, and Feng Gao. Learning from students: Online contrastive distillation network for general continual learning. In Lud De Raedt (ed.), Proceedings of the Thirty- First International Joint Conference on Artificial Intelligence, IJCAI-22, pp. Conferences Artificial Intelligence Organization, 7 2022.
International Joint
Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. Multi-task deep neural networks for natural language understanding. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4487 4496, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXie preprint 1rXin:1706 700600008, 2017.
James I, McClelland, Bruce I. McNaughton, and Randall C O'Reilly. Why there complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 132(3): 1995.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Fei Mi, Lingjin Kong, Tao Lin, Kaicheng Yu, and Boi Faltings. Generalized class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 240-241, 2020.
German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: review. Neural Networks, 113:54-71, 2019.
Roger Rateliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2))285, 1990.
Blake A Richards, Timothy P Lillicrap. Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al. A deep learning framework for neuroscience. Nature neuroscience, 12(11):771111770, 2019.
Anthony Robins. Catastrophic forgetting, rehearsal and pscudorchearsal. Connection Science, 7(2):123 1146 1995.
Sebastian Ruder, An overview of multi-tash learning in deep neural networks. arXiv preprint arxiv: 1el.1006.0099 2017.
15

---

Published in Transactions on Machine Learning Research (04/2024)
Fahad Sarfraz, Elahe Arani, and Bahram Zonooz. Synergy between synaptic consolidation and experience replay for general continual learning. In Sarath Chandar, Razvan Pascanu, and Doina Precup (eds.), Proceedings of The 1st Conference on Lifelong Learning Agents, volume 199 of Proceedings of Machine Learning Research, pp. 936. PMLR, 22-24 Aug 2022.
Fabian H. Sinz, Xaq Pitkow, Jacob Reimer, Matthias Bethge, and Andreas S Tolias. Engineering a less artificial intelligence. Neuron, 103(6):967 979, 2019.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advonces in neural information processing systems, 30, 2017.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 6199, 2013.
Rishabh Tiwari, Krishnateja Killansetty, Rishabh Tyer, and Pradeep Shenoy. Ger: Gradient coreset based replay buffer selection for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pollern Recognition, pp. 99 108, 2022.
Gido M Van de Ven and Andreas S Tolias. Threc scenarios for continual learning. arXiv preprint arXiv:1904.0778, 2019.
Eli Verwimp, Matthias De Lange, and Tinne Tuytelaars. Rehearsal revealed: The limits and merits of revisiting samples in continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9385-9394, 2021.
Jeffrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1):37 57, 1985.
Tongzhou Wang and Phillip Tsola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning, pp. 9929 9939. PMLR, 2020.
Zhen Wang, Liu Liu, Yiqun Duan, and Dacheng Tao. Continual learning through retrieval and imagination. In AAAT Conference on Artificial Intelligence, volume 8, 2022a.
Zhenyi Wang, Li Shen, Le Fang, Qiuling Suo, Tichang Duan, and Mingchen Gao. Improving task-free continual learning by distributionally robust memory evolution. In International Conference Machine Learning, pp. 22985-22998. PMLR, 2022b.
Zixin Wen and Yuanzhi Li. Toward understanding the feature learning process self-supervised contrastive learning. In International Conference on Machine Learning, pp. 11112-11122. PMLR, 2021.
Jiacheng Xu and Greg Durrett. Spherical latent spaces for stable variational autocncoders. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4503-4400 0-114,11, 2018.
Song Zhang, Gehui Shen, Jinsong Huang, and Zhi-Hong Deng. Self-supervised learning aided dass-incrrmennal lifelong learning. arXio preprint ar 200: 05882, 2020.
16

---

Published in Transactions on Machine Learning Research (04/2024)
A Appendix
A.1 Limitations and Future Work
IMEX-Reg is two-pronged CL approach aimed at implicit regularization using hard parameter sharing and multitask learning, and an explicit regularization in the function space to guide the optimization of the CL model toward generalization. In addition to CRL and consistency regularization, IMEX-Reg entails an explicit classifier rogularization to emulate rich geometric structures Icarned within the projection head hypersphere to compensate for weak supervision under low-buffer regimes. As IMEX-Reg involves many inductive biases to help improve the generalization performance in CL, it necessitates hyperparameter tuning to find the best combination of these biases. Furthermore, finding the right set of inductive biascs for novel domains may not be trivial. In addition, the selected inductive biases should be such that they complement each other and especially benefit. CL.
As IMEX-Reg involves an auxiliary CRL, representation learning in the projection head could be further improved by leveraging the vast unlabeled data. Learning generic representations through unlabeled data could relax the need to store a large number of imagos in the buffer. We leave the extension of IMEX-Reg to other domains, and different CL scenarios, and enhancing with unlabeled data for future work.
A.2 Continual Learning Settings
We evaluate the performance of our method in three different CL settings, namely, Task Incremental Learning (Task-IL), Class Incremental Learning (Class-IL), and General Contimual Learning (GCL).
In Task-IL and Class-IL, cach task consists of a fixed number of novel classes that the model must learn. A CL model learns several lasks sequentially while distinguishing all classes it has seen far. Task-IL is very similar to Class-IL, with the exception that task labels are also available during inference, making the easiest scenario. Although Class-IL is a widely studied and relatively harder CL setting, it makes several assumptions that are realistic, such as a fixed number of classes in each task and no reappearance of classes. (Mi et al. 2020 GCL relaxes such assumptions and presents more challenging real-world-like scenarios where the task boundaries are blurry and classes reappear with different distributions.
A.3 Datasets
For Task-LL and Class-IL scenarios, we obtain CIFACO,, Seq- CIFACIOO, and Sen--inylnnageet by splitting CIFAR10, CIFAR100, and TinylmageNet into 5, 5, and 10 tasks of 2, 20, and 20 classes, respectively. The three datasets present progressively challenging scenarios (increasing the number of tasks or number of classes per task) for a comprchensive analysis of diffcrent CL methods. Generalized Class-IL (GCIL) (Mi et al. 2020) exposes the model to more challenging scenario by utilizing probabilistic distributions to sample data from the CIFAR100 dataset in each task. The CIFAR100 dataset split into 20 tasks, with each task containing 1000 samples with maximum of 50 classes. GCIL provides two variations for sample distribution, Uniform and Longtail (class imbalance). GCIL is the most realistic scenario with varying numbers of classes per task and classes reappearing with different sample sizes.
A.4 Model and Training
We use the same backbone as recent approaches CL (Buzzega et al. 2020 Arani et al. 2022) i.c., a ResNot- 18 backbone without pretraining for all experiments. We use linear layer, 3-layer MLP with BatchNorm and ReLu, and 2-Layer MLP for the classifier, projection head, and classifier projection, respectively.
To cnsure uniform experimental settings, we extended the Mammoth framework (Buzzoga et al. 2020 and followed the same training scheme such as the SGD optimizer, batch size, number of training epochs, and learning rate for all experiments, unless otherwise specified. We employ random horizontal flip and random crop augmentations for supervised learning in Soq-CIFAR10, Soq-CIFAR100, TonyImIitm,,,,,,,, and experiments. For contrastive representation learning in the projection head, we transform the input batch using a stochastic augmentation module consisting of random resized crop, random horizontal
17

---

Published in Transactions on Machine Learning Research (04/2024)
Table Relative training time comparison of several CL methods trained on Seq- CLFAR1000 with buffer size 200. IMEX-Reg and OCDNet takes computational time owing to CRL
Method
+ CLS-ER
OCD-Net
IMEX-Reg
Relative time taken
1x
1.09x
1.4xx
1.57x
flip followed by random color distortions. We trained all our models NVIDIA's GeForee RTX 2080 Ti (IIGB). On average, it took around hours to train IMEX-Reg on Seq-CIFARIO and Seq-CIFAR100, and approximately 8 hours to train on eeg-TinyLmageNet
A.5 Computational Cost
Table 5 provides a relative comparison of training times for different CL approaches trained on Seq- CIFAR100 with a buffer size of 200. A large part of IMEX-Reg's s as well OCDNet's 's training time can be attributed to computations involving CRL. Our contribution Leer adds up only a minimal computational overhead. Although IMEX-Reg takes longer time to train, the performance improvement from these components is significant enough sidestep computational overhead. Moreover, during inference we discard the projection head and hence IMEX-Rog's processing time is the same as the other approaches
A.6 Buffer Efficiency
In our experimental evaluation, we assessed various CL models using established benchmarks, encompassing a range of scenarios where the number of samples per class varies from 50 to 1. We observed that as the number of samples per class decreases, the generalization of methods tends to deteriorate, implying less effective utilization of buller samples. However, IMEX-Reg consistently demonstrates superior performance even with reduced number of buffer samples. For instance, with a buffer size of 500 to IMEX-Reg with buffer size of 200. As can be seen from Table 1 and Table 2 IMEX-Reg notably oulperforms DER++ in nearly all scenarios, despite having half the buffer size. This enhancement in performance can be attributed to IMEX-Reg Reg's ability to learn generalizable representations, thus effectively leveraging its buffer samples.
compare the performance of DER+
A.7 Reservoir Sampling
provides the steps for the reservoir sampling strategy (Vitter 1985 for maintaining fixed-size memory buffer from a data stream. Each sample in the data stream is assigned an equal probability of being represented the memory buffer. When the buffer is full, sampling and replacement are performed random without assigning any priority the samples that are added or replaced.
Algorithm 2
Algorithm Reservoir sampling (Vitter 1985)
Input: Data streams D., Memory Buffer Dm, Maximum buffer size M, Number seen samples N, Current sample (a, E D. if M >N then Dm (M] (x.v)
else
= randommnnegerr = 0, max M)
if i < M then
D + {z, y}
return Dm
18

---

Published in Transactions on Machine Learning Research (04/2024)
A.8 Hyperparameters
Table 0 provides the best hyperparameters used to report the results in Table D In addition to these hyperparameters, we use standard batch size of 32 and a minibatch size of 32 for all our experiments.
Table 6: The best hyperparameters for IMEX-Rey Reg to reproduce the results reported in Table
Dataset
Buffer
LR
Epochs
200
0.03
50
0.4
0.999
0.1
0.1
0.3
CeI-CI
500
0.03
0.4
0.999
0.2
0.3
200
0.03
50
0.08
0.999
0.1
0.3
0.15
Seq-CIFAR100
500
0.03
50
0.08
0.999
0.1
0.2
0.15
200
0.03
20
0.1
0.999
0.1
0.1
0.2
ImageNet
500
0.03
20
0.15
0.999
0.1
0.1
0.3
100
0.03
100
0.1
0.999
0.2
0.2
0.15
GCIL100 Uniform
200
0.03
100
0.1
0.999
0.2
0.2
0.15
500
0.03
100
0.1
0.999
0.2
0.2
0.15
100
0.03
100
0.1
0.999
0.2
0.2
0.15
GCIL100 Longtail
200
0.03
100
0.1
0.999
0.2
0.2
0.15
500
0.03
100
0.1
0.999
0.2
0.2
0.15
A.9 Hyperparamter tuning
Table provides the results on hyperparameter tuning. As is evident from the results, our method is not very sensitive to the choice of hyperparameters.
Table 7: Hyperparamter tuning for IMEX-Reg on Seq-C1FA with buffer size 200. As can be seen, OMEX-Reg is quite robust to the choice of hyperparameters.
Varying a, for 0.3, X 0.15
Varying 8, for 0.1, X 0.15
Varying X, for 0.1, B 0.3
Top-1 1 Ace %
8
Top-1 Acc %
X
Top-1 Acc %
0.05
47.72
0.1
48
0.1
48.07
0.1
48.54
0.2
47.95
0.15
48.54
0.2
47.90
0.3
48.54
0.2
47.4
0.3
47.26
0.4
48.06
A.10
Stability-Platticity Trade-off
While lorgetting Fr quantifies how much knowledge preserved in the model, we need to evaluate how well it is adapting novel tasks. The extent to which CL systems need be plastic in order to acquire novel information and stable in order to retain existing knowledge is known as the stability-plastica There is inherent trade-off between the plasticity and stability of the model, and estimating this trade-off can shed some light on this dilemma. Sarfraz et al. (2022) proposes Trade-off measure that approximates the balance between the stability and plasticity of the model. After learning the final task T,, the stability tasks; S = Σn- Ari-
dilemma.
(S) the model is estimated as the average performance of all previous
The plasticity (P) of the model is measured as the average performance of each task after learning for the first time; P P= P= Ag-
To find an optimal balance between the stability and plasticity of the model, Sarfraz et al. the harmonic mean of 8 and P as trade-off measure, that is,
2022
computes
2SP
Trade-off + P
(11)
19


Reviews:
[{"review_id": "TVuiWoUppC", "reviewer": "Reviewer_HQzg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths : \n1. The empirical results are strong and convincing. The experimental section is well crafted and highlights well the proposed approach. \n2. The authors do a good job in additional analyses. The forgetting analysis, the stability / plasticity section are well-designed. Specifically, the analysis on model calibration and task recency bias, issues known to plague prior approaches in CL, are convincing and provide good support to the proposed method. \n3. The ablation section is concise and easy to follow. \n\n### Weaknesses : \n1. **Motivation**. The proposed method is introduced as a solution to limit overfitting in small buffer regimes. While this has been shown to be an issue in prior works, the authors provide no empirical evidence that this problem is addressed by their method. Moreover, the argument made that having the features lie on the unit-hypersphere is somewhat weak. I agree that for SSL one needs to normalize the features to stabilize training, but I disagree that unit-norm makes representations easier to linearly separate for a classification head. \n2. **Memory Usage**. The paper presents results in settings where the buffer memory is quite limited. While the setting is relevant, I am not convinced by the author's use of this memory. A Resnet-18 has approx 11 million parameters, so the memory to store both EMA and standard model is about 2 models x 11 miliion params x 4 bytes / param = 88 Megabytes. Storing 200 cifar-100 samples takes 200 x 32 x 32 x 3 x 1 byte / pixel = 0.6 Megabytes. Given the impact of the buffer size on performance, it's unclear whether the proposed approach makes optimal use of this memory (and this is without counting the memory needed to store intermediate activations required for backprop)\n3. **Compute**. Lastly, the authors have not discussed the computation cost of their method. For instance, how would other methods fare if instead of forwarding data through the EMA model, that compute budget was used differently ?\n\n\n\n### Small comments and questions : \n1. Notation : The notation of $g$ and $g'$ is a bit confusing, maybe try using $g_{lin}$ and $g_{mlp}$ ? \n2. I am not sure to understand what the role of $g'$ is, as it seems to be only used $\\mathcal{L}_{cr}$. Have you considered removing this network, and instead computing $\\mathcal{G}_h$ with either the logits ? Also, could you provide intuition on why the stopgrad operator is used ? Why not backprop through both matrices ?\n3. For equation 3, why not simply use a smaller decay factor instead of of sporadically performing the update ? Is it for computational efficiency ?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a new approach to offline Class-IL and Task-IL, for replay based approaches where the buffer size is limited. The solution consists of a carefully crafted combination of losses designed to regularize the model and limit overfitting on the buffer. Concretely, the approach augments standard replay with the following losses : \n1. $\\mathcal{L}_{rep}$, a supervised contrastive objective on both new and buffered datapoints\n2. $\\mathcal{L}_{cr}$ a loss regularizing the classifier and projection head outputs to stay close to outputs of an EMA model\n3. $\\mathcal{L}_{ecr}$ ensuring that the similarities across points in the batch computed for both classifier head and projection heads align. \n\nThe authors evaluate their method on several well-established CL benchmarks and compare against strong baselines, showing that their method matches or surpasses previous state-of-the-art methods. \nThe authors also provide extensive analysis and ablations of their method, shedding light on how the proposed method's properties.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### Critical changes\n1. An small analysis of the impact of their method on buffer overfitting\n2. A convincing argument / experiment showing that their use of the limited buffered memory is efficient\n3. A discussion about the computational cost of their method\n\n### Other changes\nAdressing the questions I raised above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LxsxZOy8K5", "reviewer": "Reviewer_vQkA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Strengths:\n\t- The paper presents thorough quantitative analysis of their approach compared to a comprehensive set of baselines and settings using various analyses pertinent to continual learning. IMEX-Reg outperforms most baselines in the presented analyses.\n\t- The authors present an ablation study that validates the value of each component of IMEX-Reg.\n- Weaknesses\n\t- While the results are strong, the benefit of the approach overall is confusing.\n\t- It is unclear how the projection layer $h$ , despite being randomly initialized and not updated through training, provides meaningful signal for effective regularization. Despite being randomly initialized and not updated, is the output from $h$ still able to capture the representation from Backbone $f$ ? Why can't we use the output directly from $f$ for $L_{cr}^{h}$ , $L_{rep}$ and apply L2 normalization for $L_{ecr}$ ? The output from $h$ contributes to all of the regularization terms except $L_{cr}^{g}$ , so it is important that this part is clarified.\n\t- Based on the above, it's unclear how the main contribution, guiding the classifier's projections toward the activation correlations in the unit-hypershere of the contrastive representation loss, leads to improvements.\n\t- It is unclear how IMEX-Reg is \"inspired by how humans leverage inductive biases\". Is it that humans needing minimal adaptations to learn new tasks mapped to these regularization terms? I don't think enough is known about the human learning process or at least it's not elaborated enough with related work in this paper to make this claim.\n\t- Without clearly defining low-buffer regime vs high-buffer regime and providing strong practical motivations for why studying the low-buffer regime is important, readers would be interested in how IMEX-Reg performs in a high-buffer regime compared to baselines and whether it still provides comparative gains.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- This paper presents IMEX-Reg, a regularization method for continual learning that leverages both implicit regularization through an auxiliary loss using contrastive representation loss and explicit regularization using exponential moving average to enforce consistency. It also includes another regularization term that penalizes the differences between the unit normalized representations from the projection layer and the unit normalized representations from the classifier's projection layer.\n- The approach shows strong performance across all Task-IL, Class-IL, and GCIL settings, outperforming baselines from recent work except for OCDNet on the Seq-CIFAR10 dataset. IMEX-Reg gets a low forgetting rate, is more robust to adversarial attacks or noise, and is better calibrated.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- There is an overall need for improvement in clarity of the main narrative.\n- Clarify how the projection layer $h$ is initialized and updated. Section 3.1 seems to indicate that $h$ is randomly initialized as an arbitrary 128-dimensional L2 normalized projection, and then the stopgrad term in Section 3.2 seems to indicate that $h$ is not updated. Also, what is the dimension size for the classifier projection $g$ ? Figure 1 seems to indicate that it's smaller than that of $h$.\n- Why is $L_{cr}^{h}$ and $L_{cr}^{g}$ only applied to $D_{m}$ but not $D_{t}$ in Algorithm 1? Wouldn't applying the same terms when training with samples from $D_{t}$ also help with making sure that the model trained from $D_{1,\\ldots,t-1}$ remain consistent?\n- The introduction can contain more specifics with regards to the contributions to make it clearer earlier that the approach was tested on image classification datasets and at a high-level what the theoretical insights are.\n- In the introduction, the motivation for studying CL with small buffer sizes mention privacy concerns, but this seems misplaced. Unless we use a federated learning setting, isn't privacy concern applicable to general ML, not confined to low-buffer regime CL? Also, it is unlikely that CL is performed on edge devices. A better motivation for low buffer CL is possibly when the number of tasks is large and each task has a lot of data such that training with new data even with a memory buffer becomes impractical, but I'm not sure whether the tasks explored in this paper correctly evaluate these setups.\n- What is a low-buffer regime as opposed to a high-buffer regime? Is it a certain percentage of each task data or an absolute number or an absolute memory constraint?", "broader_impact_concerns": "There are no specific concerns for broader impact for this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "iPLzU0dYO5", "reviewer": "Reviewer_Ptwz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "An appealing aspect of this paper is that the motivation for including each of the different components of the proposed continual learning is quite extensively described. Generally, the paper is quite well written, although one critical note in this regard is that the paper is rather verbose. The paper has the tendency to describe things in an overly complex way, which can make it hard for the reader to understand the basics or underlying concepts, which often are quite straight-forward.\n\nMy main criticism of this paper is that, unfortunately, I do not think that it provides any new insights. After reading this paper, I’m afraid I don’t think I learned anything.\nThe main contribution of the paper is combining various established approaches to continual learning together (experience replay, contrastive representation learning, functional regularization and model averaging), and demonstrating that the resulting combined method performs better than current replay methods that use only some of these approaches. Doing this has probably required a decent amount of engineering effort to get everything to work, but the final result is not surprising or insightful. Moreover, given that the benchmark problems that are used in this paper are all highly synthetical and far away from any real-world applications, there is also virtually no practical value to this engineering effort.\n\nI’m therefore afraid that I don’t think that this paper makes a contribution to the continual learning literature.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method for continual learning that combines aspects of experience replay, contrastive representation learning, functional regularization and model averaging. It is shown that the resulting method is able to obtain higher performance than current replay-based methods on a number of relatively standard continual learning benchmarks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "I would encourage the authors to focus on generating new insights and/or on demonstrating practical value of their engineered method.\n\nCurrently, it seems that the main contribution of this paper is getting a new “high score” on popular, but highly synthetical continual learning benchmarks (e.g. Split CIFAR). However, given the many degrees of freedom in these benchmarks (e.g. no control for computational complexity, no control for number of hyperparameters, no control for how many times the benchmark is tried), the value of such a new “high score” is very limited. (Moreover, it is very hard if not impossible for reviewers to really judge whether it indeed is a new high score to start with.) Rather, the value of these standard continual learning benchmarks is that they can be used for generating new insights by e.g. performing carefully controlled comparisons, but I’m afraid this is not what the current paper does.\n\nAnother avenue this paper could take is demonstrating practical value of the engineered method on actual real-world problems. The paper appears to attempt to claim that “generalized class-IL” provides such a real-world problem, but that is certainly not the case as this is simply a specific instance/variation of incremental CIFAR.", "broader_impact_concerns": "No concerns in this regard.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to combine several existing techniques for continual learning to improve the performance, and this is shown to work on several benchmarks. The paper was reviewed by three reviewers. All reviewers acknowledged the empirical performance of the proposal. All reviewers, however, raised concerns around novelty of the proposal, which I agree with. Given that novelty is not an explicit decision-making factor in TMLR, and given that the paper meets the bar in terms of substantiation of the claims, it is recommended to be accepted subject to minor revisions. Congratulations!

- Please remove mentions of novelty claims in the paper and focus on factually describing the proposal.

- At the end of page 2, in contributions, please give references to where the each of the proposed components appears.

- The equations are generally missing punctuation, e.g., the unnumbered eq. in Theorem 2. Please check all equations for proper punctuation.

- Please make the code publicly available as promised.

- Please add clarity to the claim of "leveraging desirable traits of learning in the unit hypersphere through explicit classifier regularization in the function space."
Decision: Accept (poster)