Incremental Learning of Structured Memory via Closed-Loop Transcription
Authors: Shengbang Tong
Xili Dai
Ziyang Wu
Mingyang Li
Brent Yi
Yi Ma
Venue: ICLR 2023 poster (2023)
Keywords: Generative Replay Incremental Learning
Closed Loop Transcription

Abstract: This work proposes a minimal computational model for learning structured memories of multiple object classes in an incremental setting. Our approach is based on establishing a {\em closed-loop transcription} between the  classes and a corresponding set of subspaces, known as a linear discriminative representation, in a low-dimensional feature space. Our method is simpler than existing approaches for incremental learning, and more efficient in terms of model size, storage, and computation: it requires only a single, fixed-capacity autoencoding network with a feature space that is used for both discriminative and generative purposes. Network parameters are optimized simultaneously without architectural manipulations, by solving a constrained minimax game between the encoding and decoding maps over a single rate reduction-based objective. Experimental results show that our method can effectively alleviate catastrophic forgetting, achieving significantly better performance than prior work of generative replay on MNIST, CIFAR-10, and ImageNet-50, despite requiring fewer resources.

Published as a conference paper at ICLR 2023
INCREMENTAL LEARNING OF STRUCTURED MEMORY VIA BLOSED-LOOW TRANSCRIPTION
Shengbang Tong', Xili Dai Ziyang Wu', Mingyang Li", Brent Yi', Yi Ma!
University of California. Berkeley, The Hong Kong University of Science and Technology(Guangzho) 3 Tsinghua-Berkeley Shenzhen Institute (TBSI), Tsinghua University
ABSTRACT
This work proposes minimal computational model for learning structured memo- ries of multiple object classes in an incremental setting. Our approach is based on establishing closed-loop transcription between the classes and corresponding set of subspaces, known as a linear discriminative representation, in a low-dimensional feature space. Our method is simpler than existing approaches for incremental learning, and more efficient in terms of model size, storage, and computation: it requires only a single, fixed-caaccity autoencoding network with a feature space that is used for both discriminative and generative purposes. Network parameters are optimized simultaneously without architectural manipulations, by solving a con- strained minimax game between the encoding and decoding maps over single rate reduction- baaed objective. Experimental results show that our method can effec- tively alleviate catastrophic forgetting, achieving significantly better performance than prior work of generative replay on MNIST, CIFAR-IO, and ImageNet-50, despite requiring fewer resources.
INTRODUCTION
Artificial neural networks have demonstrated a great ability to learn representations for hundreds or even thousands of classes of objects, in both discriminative and generative contexts. However, net- works typically must be trained offline, with uniformly sampled data from all classes simultaneously. When the same network is updated to learn new classes without data from the old ones, previously learned knowledge will fall victim to the problem of catastrophic forgetting 1989). This is known in neuroscience as the -aoblitty-llasticity dilemma: the challenge of ensuring that a neural system can learn from a new environment while retaining essential knowledge from previous ones (Grossberg 1987)
forgeetnnnn (McCloskey Cohen
In contrast, natural neural systems (e.g. animal brains) do not seem to suffer from such catastrophic forgetting at all. They capable of developing new memory of new objects while retaining memory of previously learned objects. This ability, for either natural or artificial neural systems, is often referred to incremental learning, continual learning, sequential learning, or life-long learning (Allred & Roy 2020).
While many recent works have highlighted how incremental learning might enable artificial neural systems that are trained in more flexible ways, the strongest existing efforts toward answering the ptability-llastiiity dilemma for artificial neural networks typically require raw exemplars (Rebuffi
et al. 2017 Chaudhry et al. 2019b) or require external task information (Kirkpatrick et al. 2017)) Raw exemplars, particularly the case of high-dimensional inputs like images, are costly and difficult to scale, while external mechanisms
which, as surveyed in Section [ include secondary networks and representation spaces for generative replay, incremental allocation network resources, network duplication, or explicit isolation of used and unused parts of the network require heuristics and incur hidden costs.
In this work, are interested in an incremental learning setting that counters these trends with two key qualities. (1) The first is that it is memory-based. When learning new classes, no raw exemplars of old classes available to train the network together with new data. This implies that one has rely on a compact and thus structured "memory" of old classes, such as incrementally learned generative representations of the old classes, well as the associated encoding and decoding mappings (Kemker

---

Published as a conference paper at ICLR 2023
High- dim data space X
Low-dim feature space Z
cucodor
R
f(x.8)
Ziver
Z
2
Dee
Xxxx
closed-loop transcription
Buse
Xus
Zots
Ziioo
a(2.1)
X
decoder
Zost Zes
now input data Xaus
2
feature of old and new memory. 2 2', replay loop f(X)
replayed memory = s(z):
Figure 1: Overall framework of our closed-loop transcription based incremental learning for structured LDR memory. Only single, entirely self-contained, encoding-decodiin network is needed: for new data class Xnew, new LDR memory Znew is incrementally learned as minimax game between the encoder and decoder subject to the constraint that old memory of past classes Zold intact through the closed-loop transcription (or replay): Zold Zold = [( Zaa4).
& Kanan 2018).
(2) The second is that it is self-contained. Incremental learning takes place in a single neural system with a fixed capacity, and in common representation space. The ability to minimize forgetting is implied by optimizing an overall learning objective, without external networks, architectural modifications, or resource allocation mechanisms.
Concretely, the contributions of our work are as follows:
2023 can (1) We demonstrate how the closed-loop transcription (CTRL) framework (Dai al. 2022
be adapted for memory-based, self-contained mitigation of catastrophic forgetting (Figure Dj. To the best of our knowledge, these qualities have not yet been demonstrated by existing methods. Closed- loop transcription aims to learn linear discriminative representations (LDRs) via a rate reduction- based (Yu al. 2020; Ma al. 2007 Ding al. 2023) minimax game: our method, which we call incremental closed-low transcription (i-CTRL), shows how these principled representations and objectives can uniquely facilitate incremental learning of stable and structured class memories. This requires only fixed-sized neural system and common learning objective, which transforms the standard CTRL minimax game into constrained one, where the goal is to optimize a rate reduction objective for each new class while keeping the memory of old classes intact.
(2) We quantitatively evaluate i-CTRL on class incremental learning for a range of datasets: MNIST (LeCun et al. 1998), CIFAR-10 (Krizhevsky al. 2009), and ImageNet-50 (Deng et al.) 2009). Despite requiring fewer resources (smaller network and nearly no extra memory butfer), I-CTRL outperform comparable alternatives: it achieves 5.8% improvement in average classifica- tion accuracy over the previous state of the art on CIFAR- 10, and 10.6% improvement in average accuracy on ImageNet-50.
(3) We qualitatively verify the structure and generative abilities of learned representations. Notably, the self-contained i-CTRL system's common representation used for both classification and generation, which climinates the redundancy of external generative replay representations used by prior work.
(4) We demonstrate a "'lasss "class-unsupervised" incremental reviewing process for i-CTRL. As an in- cremental neural system learns more classes, the memory of previously learned classes inevitably degrades: by seeing class only once, we can only expect to form a temporary memory. Facilitated by the structure of our linear discriminative representations, the incremental reviewing process shows that the standard i-CTRL objective function can reverse forgetting in trained i-CTRL system using samples from previously seen classes even if they are unlabeled. The resulting semi-supervised process improves generative quality and raises the accuracy of i-CTRL from 59.9% to 65.8% on CIFAR-10, achieving jointly-optimal performance despite only incrementally provided class labels.
2 RELATED WORK
A significant body of work has studied methods for addressing forms of the incremental learning prob- lem. In this section, we discuss a selection of representative approaches, and highlight relationships to i-CTRL.
2

---

Published as a conference paper at ICLR 2023
In terms of how new data classes are provided and tested, incremental learning methods in the literature can be roughly divided into two groups. The first group addresses task incremental learning (task-IL), where model is sequentially trained on multiple tasks where each task may contain multiple classes to learn. At test time, the system is asked to classify data seen so far, provided with task identifier indicating which task the test data is drawn from. The second group, which many recent methods fall under, tackles class-incremental learning (class-IL). Class-IL is similar to task-IL but does not require a task identitifier at inference. Class-IL is therefore more challenging, and is the setting considered by this work.
In terms of what information incremental learning relies on, existing methods mainly fall into the following categories.
Regularization-aasee methods introduce penalty terms designed mitigate forgetting of previously trained tasks. For instance, Elastic Weight Consolidation (EWC) (Kirkpatrick et al. 2017) and Synaptic Intelligence (SI) (Zenke et al., 2017) limit changes of model parameters deemed to be important for previous tasks by imposing a surrogate loss. Alternativlly, Learning without Forgetting (LwFF) (Li & Hoiem, 2017) utilizes a knowledge distillation loss to prevent large drifts of the model weights during training on the current task. Although these methods, which all apply regularization on network parameters, have demonstrated competitive performance on task-IL scenarios, our evaluations (Table D show that their performance does not transfer to the more challenging class-IL settings.
rrchitecture-based methods explicitly alter the network architecture incorporate new classes of data. Methods such as Dynamically Expandable Networks (DEN) ( (Yoon et al. 2017). Progressive Neural Networks (PNN) (Rusu et al. 2016). Dynamically Expandable Representation (DER) (Yan) it al. 2 and ReduNet (Wu et al. 2021) add new neural modules to the existing network when required learn a new task. Since these methods are not dealing with self-contained network with a fixed capacity, one disadvantage of these methods is therefore their memory footprint: their model size often grows linearly with the number of tasks or classes. Most mechteecture-bssed methods target the less challenging task-ll. problems and are not suited for class-Il. settings. In contrast, our work addresses the class II setting with only a simple, off-the-shelf network (see Appendix [B] for details). Note that the method Redunet also uses rate-reduction inspired objective function to conduct class incremental learning. Our method is different from the method that (i) our method does not require dynamic expansion of the network (ii) Our method aims to learn continuous encoder and (icoder. Empirically, our method has shown better performance and scalability.
Exemplar-based methods combat forgetting by explicitly retaining data from previously learned tasks. Most early memory-based methods, such as iCaRL. (Rebuffi et al., (017)) and ER (Chaudhry al. 2019a), store a subset of raw data samples from each learned class, which is used along with the new classes to jointly update the model. A-GEM (Chaudhry et al.) 2018) also relies on storing such an exemplar set: rather than directly training with new data, A-GEM calculates reference gradient from the stored data and projects the gradient from the new task onto these reference directions in hope of maintaining performance on old tasks. While these methods have demonstrated promising results, storing raw data of learned classes is unnatural from neuroscientific perspective(Robins) (995) and resource-intensive, particularly for higher-dimensional inputs. A fair comparison is thus nol possible: the structured memory used by i-CTRL is highly compact in comparison, but as demonstrated in Section 4 still outperforms several exemplar-ban methods.
Generative memory- -base methods use generative models such as GANs or autoencoders for replaying data for old tasks or classes, rather than storing raw samples and exemplars. Methods such as Deep Generative Replay (DGR) (Shin et al. 2017). Memory Replay Gans (MeRGAN) (Wu et al.) 2018), and Dynamic Generative Memory (DGM) (Ostapenko et al. 2019) propose to train a GAN on previously seen classes and use synthesized data alleviate forgetting when training on new tasks. Methods like DAE (Zhou et al.] (0012) learn with add and merge feature strategy. To further improve memory efficiency, methods such as FearNet (Kemker & Kanan 2018) and EEC (Ayub & Wagner 2020) store intermediate features of old classes and use these more compact representations for generative replay. Existing generative memory-based approaches have performed competitively on class-IL IL without storing raw data samples, but require separate networks and feature representations for generative and discriminative purposes. Our comparisons are primarily focused on this line of work, as our approach also uses a generative memory for incremental learning. Uniquely, however, we do so with only a single closed-loop encoding-decodiie network and store a minimum amount of
3

---

Published as a conference paper at ICLR 2023
information mean and covariance for each class. This closed-loop generative model is more stable to train (Dai et [002]] Tong et al. 20222, and improves resource efficiency by obviating the need to train separate generative and discriminative representations.
METHOD
3.1 LINEAR DISCRIMINATIVE REPRESENTATION AS MEMORY
Consider the task of learning memorize k classes of objects from images. Without loss of generality, we may assume that images of each class belong to a low-dimensional submanifold in the space of images RD, denoted as M,, for j Px* that are partitioned into &t subsets X X with each subset X;
.,, k. Typically, we are given n samples
X x= ,....."'! sampled from Mj.j = l...... k. The goal here is to learn a compact representation, or a "memory" of these k classes from these samples, which can be used for both discriminative (e.g. classification) and generative purposes (e.g. sampling and replay).
Autoencoding. We model such a memory with an autoencoding tuple {f.g, 2 that consists of an encoder f(-, f) parameterized by 0, that maps the data æ E TR continuously to compact feature much lower-dimensional space IRd, and a decoder g(:, 71 parameterized by 7), that maps a feature
back to the original data space IR
f(: 0) + = R g(. 1) : 2 D.
((x", 0) E TRd be the set of corresponding features. Similarly let X = g(Z.n)) n)) be the decoded data from the features. The autoencoding tuple can be illustrated by the following diagram:
For the set of samples X, we let = (X [z] ." C R with = f f(
g
f(,,0)
Z a(x.n)
X
(2)
We refer to such a learned tuple: (f(, ((c,8),9(,3m n). 7} as a compact "memory" for the given dataset X.
Structured LDR autoencoding. For such a memory to be convenient to use for subsequent tasks, including incremental learning, we would like representation Z that has well-understood structures and properties. Recently, Chan et al. (Chan et al. (021) proposed that for both discriminative and generative purposes, Z should be linear discriminative representation (LDR). More precisely, let Z, f j, j I,... k be the set of features associated with each of the k classes. Then each Z, should lie on a low-dimensse di-eemiiniinnll linear subspace Sj in Rd which is highly incoherent (ideally orthogonal) to others S; for i * j. Notice that the linear subspace structure enables both interpolation and extrapolation, and incoherence between subspaces makes the features discriminative for different classes. As we will see, these structures are also easy to preserve when incrementally learning new classes.
f(x,,
3.2 LEARNING LDR VIA CLOSED-LOOP TRANSCRIPTION
As shown Yu et al. 2020 the incoherence of learned LDR features Z = f(X, 0) can be promoted by maximizing coding rate reduction objective, known as the MCR² principle:
max AR(Z) = AR.Z.... Zz) =
2 log det (I + aZZ")
Y 2
log det (I a Z,
R(Z)
R(Z,)
where, for a prescribed quantization eтror
ne? 02j
d
2
As noted in YYu et al. 2020), maximizing the rate reduction promotes learned features that span the entire feature space. is therefore not suitable to naively apply for the case incremental learning, as the number of classes increases within fixed feature space] The The closed-loop transcription (CTRL) framework introduced by (Dai et al.) 2022) suggests resolving this challenge by learning the encoder f(; 8) and decoder g(;,)) m) together as minimax game: while the encoder tries maximize the rate reduction objective, the decoder should minimize it instead. That is, the decoder g minimizes
As the number of classes is initially small in the incremental setting, if the dimension of the feature space d high, maximizing the rate reduction may over-estimate the dimension of each class.
4

---

Published as a conference paper at ICLR 2023
resources (measured by the coding rate) needed for the replayed data for each class X, g J 17),
decoded from the learned features Z, = ff 0). to emulate the original data X, well enough.
As it is typically difficult to directly measure the similarity between X, and X, (Dai et al. 2022)
proposes measuring this similarity with the rate reduction of their corresponding features 2, and
2, = - (0, 7). () here represents concatenation):
AR AR(Z, 2,) = R(Z J 2,) - (R(Z)) + R( R(2)).
(3)
The resulting AR gives a principled "distance" between subspace-like Gaussian ensembles, with the property that AR(Z, 2,) = 0 iff (ov(Zz) = Cov( (2,) (Ma et al. 2007
min max AR Z) + AR( 2) +
AR AR(Z, 2z),
(4)
j=1
one can learn a good LDR Z when optimized jointly for all classes. The learned representation Z has clear incoherent linear subspace structures in the feature space which makes them very convenient to use for subsequent tasks (both discriminative and generative).
3.3 INCREMENTAL LEARNING WITH AN LDR MEMORY
The incoherent linear structures for features of different classes closely resemble how objects are encoded in different areas of the inferotemporal cortex of animal brains (Chang & Tsao 2017
Bao et al. 2020). The closed-loop loop transcription X Z + X + Z also resembles popularly hypothesized mechanisms for memory formation (Ven et al.] 2020 Josselyn & TTonegawa (020).
This leads to a question: since memory in the brains is formed in an incremental fashion, can the above closed-loop transcription framework also support incremental learning?
LDR memory sampling and replay. The simple linear structures of LDR make it uniquely suited for incremental learning: the distribution of features Z, of each previously learned class can be explicitly and concisely represented by principal subspace S, in the feature space. To preserve the memory of an old class j. we only need to preserve the subspace while learning new classes. To this end, simply sample representative prototype features on the subspace along its top principal components, and denote these features as Zj,old- Because of the simple linear structures of LDR, we can sample from Zjoodd by calculating the mean and covariance of Zjoold after learning class j. The storage required is extremely small, since we only need to store means and covariances, which are sampled from as needed. Suppose a total of old classes have been learned so far. If prototype features, denoted Zoid = [Z Zotal. for all of these classes can be preserved when learning new classes, the subspaces { representing past memory will be preserved as well. Details about sampling and calculating mean and convariance can be found in the Appendix and Appendix -
Incremental learning LDR with an old-memory constraint. Notice that, with the learned auto- encoding 2 one can replay and use the images, say X.td g( ZoZd ()) associated with the memory features to avoid forgetting while learning new classes. This is typically how generative models have been used for prior incremental learning methods. However, with the closed-loop framework, explicitly replaying images from the features is not necessary. Past memory can be effectively preserved through optimization exclusively the features themselves.
Consider the task of incrementally learning a new class of objects e We denote a corresponding new sample set as X Xnew- The features of X new denoted Znew (0) = s[( new: 0). We concatenate them together with the prototype features of the old classes Zatd and form Z = [Znew (0)) (0), Zota).
(0)=
We denote the replayed images from all features as X = [eeewl 17), X xota(n)] although we do not actually need to compute or use them explicitly. We only need features of replayed images, denoted
2 f X 0) ne (0, 1),, (0, m))
Mirroring the motivation for the multi-class CTRL objective 4 we would like the features of the new class Znew to be incoherent to all of the old ones Zota. As Znew is the only new class whose features needs to be learned, the objective reduces to the case where k = 1:
II Appendix A we consider the more general setting where the task contains a small batch of new classes, and present algorthmic details in that general setting.
5

---

Published as a conference paper at ICLR 2023
min max (R(Z) + AR(Z) ARR aR((neww Znew).
(5)
However, when we update the network parameters (0, 11) to optimize the features for the new class, the updated mappings f and will change features of the old classes too. Hence, to minimize the distortion of the old class representations, we can try to enforce Cov (jtt = Cov Zj.odd). . In other words, while learning new classes, we enforce the memory of old classes remain "self-consistent" through the transcription loop: Zold
gis.1)
Xold
f(a,0)
Zold-
Mathematically, this is equivalent to setting A Zaia; Zota) = AR(Z jold; 2 = 0. Hence, the above minimax program (5) is revised as a constrained minimax game, which we refer to as incremental closed-loop transcription (i-CTRL). The objective of this game is identical to the standard multi-class CTRL objective (4). but includes just one additional constraint:
min max
( + A AR( 2) + AR
Znew: 2now)
subject to
A R Roodd Zota) =0
In practice, the constrained minimax program can be solved by alternating minimization and maxi- mization between the encoder f(.. 0) and decoder g(-,n) as follows:
max
A - A R 2) + A AR( Znews Znew) ) 7 AR( Zoid. Zota),
(8)
min
( - A 2 + A- AR(Znew, Bnow) + AR (R( Zota);
(9)
where the constraint AR( Z And) = 0 in 7 has been converted (and relaxed) to a Lagrangian term with corresponding coefficient 7 and sign. We additionally introduce another coefficient À for weighting the rate reduction term associated with the new data. More algorithmic details are given in Appendix A
Jointly optimal memory via incremental reviewing, As we will see, the above constrained minimax program can already achieve state of the art performance for incremental learning. Nevertheless, developing an optimal memory for all classes cannot rely on graceful forgetting alone. Even for humans, if an object class learned only once, we should expect the learned memory to fade as we continue to learn new others, unless the memory can be consolidated by reviewing old object classes.
To emulate this phase of memory forming, after incrementally learning whole dataset, we may go back to review all classes again, one class at a time. We refer to going through all classes once as one reviewing "cycle" If needed, multiple reviewing cycles can be conducted. It is quite expected that reviewing can improve the learned (LDR) memory. But somewhat surprisingly. the closed-loop framework allows us to review even a "class-nnspeervise"" manner: when reviewing data of an old class say X,, the system does not need the class label and can simply treat X; as new class X That is, the system optimizes the same constrained mini-max program 7 without any modification; after the system is optimized, one can identify the newly learned subspace spanned by Znew: and it to replace or merge with the old subspace Sy. As our experiments show, such an class-unsupervised incremental review process can gradually improve both discriminative and generative performance of the LDR memory, eventually converging to that of a jointly-learned memory.
4 EXPERIMENTAL VERIFICATION
We now evaluate the performance of our method and compare with several representative incremental learning methods. Since different methods have very different requirements in data, networks, and computation, it is impossible to compare all in the same experimental conditions. For a fair comparison, we do not compare with methods that deviate significantly from the IL setting that method is designed for: as examples, this excludes methods that rely on feature extracting networks pre-trained on additional datasets such as FearNet (Kemker & Kanan 2018) or methods that expand the feature space such as DER (Yan et al. 2021). Instead, we demonstrate the effectiveness of our method by choosing baselines that can be trained using similar fixed network architectures without any pretraining. Nevertheless, most existing incremental learning methods that we can compare against still rely on buffer that acts as memory of past tasks. They require significantly more storage than i-CTRL, which only needs to track first and second moments of each seen class (sec Appendix [ for algorithm implementation details).
to distinguish from the term "epach" used in the conventional joint learning setting.

---

Published as a conference paper at ICLR 2023
MNIST
CIFAR-10 10
Category
Method
10-splits
5-splits
10-splits
5-splits
Last
Avg
Last
Avg
Last
Avg
Last
Avg
LWF (Li & Hoiem 2017
0.196
0.455
0.19
0.440
Regularization
(Zenke et
2017)
0.193
0.461
0.196
0.441
Architecture
ReduNet Wu et al
2021
0.961
0.982
0.539
0.645
iCaRL (Rebufh et al.
2017
0.322
0.588
0.725
0.803
0.212
0.431
0.487
0.632
A-GEM Chaudhry
2018 al.
0.382
0.574
0.597
0.764
0.115
0.293
0.204
0.473
Exemplar
CLR-ER Arani
2022
0.895
0.662
DGMw (Ostapenko ct al. 2019)
0,965
0.562
EВC (Ayub & Wagner 2020)
0.978
0.669
Generative
EECS
Ayub & Wagner
2020
0.963
0.619
i-CTRL (ours)
0.975
0.989
0.978
0.990
0.599
0.727
0.627
0.723
Table 1: Comparison MNIST and CIFAR-IO.
4.1 DATASETS, NETWORKS, AND SETTINGS
We conduct experiments on the following datasets: MNIST (LeCun et al. 1998), CIFAR-10
Krizhevsky et al. 2014), and ImageNet-50 (Deng et al. 2009). All experiments are conducted for the more challenging class-IL setting. For both MNIST and CIFAR-10. the 10 classes are split into 5 tasks with 2 classes each or 10 tasks with class each; for ImageNet-50, the 50 classes are split into 5 tasks of 10 classes each. For MNIST and CIFAR-10 10 experiments, for the encoder f and decoder g. we adopt very simple network architecture modified from DCGAN (Radford et al. (016). which is merely a four-layer convolutional network. For ImageNet-50, use a deeper version of DCGAN which contains only 40% of the standard ResNet-18 18 structure.
4.2 COMPARISON OF CLASSIFICATION PERFORMANCE
We first evaluate the memory learned (without review) for classification. Similar to (Yu et al. 2020). we adopt a simple nearest subspace algorithm for classification, with details given in Appendix B Unlike other generative memory- based incremental learning approaches, note that we do not need to train a separate network for classification.
MNIST and CIFAR-10. Table [ compares i-CTRI against representative SOTA generative-replay incremental learning methods different categories on the MNIST and CIFAR-10 datasets. We report results for both 10-splits and 5-splits. in terms of both last accuracy and average accuracy (following definition in iCaRL (Rebuffi et al. (017)). Results on regularization based and exemplar- based methods are obtained by adopting the same benchmark and training protocol as in (Buzzega et al.) 2020). All other results are based on publicly available code released by the original authors. We reproduce all exemplar- based methods with buffer size no larger than 2000 raw images or features for MNIST and CIFAR-10, which is a conventional buffer size used in other methods. Compared to these methods, i-CRRL uses single smaller network and only needs to store means and covariances.
For a simple dataset like MNIST, we observe that i-CTRL outperforms all current SOTA on both settings. In the 10-task scenario, it 1% higher on average accuracy, despite the SOTA is already as high as 97.8%. In general incremental learning methods achieve better performance for smaller number of steps. Here, the 10-step version even outperforms all other methods in the 5-step setting.
For CIFAR-10, 10, we observe more significant improvement. For incremental learning with more tasks (i.e splits = 10), to our best knowledge, EEC/EECS (Ayub Wagner] (2020) represents the current SOTA. Despite the fact that EEC uses multiple autoencoders and requires significantly larger amount of memory (see T the appendix), we see that i-CTRL. outperforms EEC by more than 3%. For a more fair comparison, we have also included results of EECS from the same paper, which aggregate all autoencoders into one. i-CTRL outperforms EBCS by nearly 10%. We also observe that i-CTRL with 10 steps is again better than all current methods that learn with 5 steps, in terms of both last and average accuracy.
iCaRL-S
EEIL-S
DGMw
ЕЕC
EECS
i-CTRL
0.290
0.118
0.178
0.352
0.309
0.458
Table 2: Comparison on ImageNet-50. The results of other methods are as reported in the EEC paper.
7

---

Published as a conference paper at ICLR 2023
1750
4300
50000
12b0x
20000
37500
50000
Figure 2: Block diagonal structure of Z Z in the feature space for MNIST (left) and CIFAR-10 (right).
ImageNet-50. We also evaluate and compare our method on ImageNet-50, which has a larger number of classes and higher resolution inputs. Training details can be found in Appendix [ We adopt results from (Ayub & Wagner, 2020) and report average accuracy across five splits. From the table, we observe the same trend, very significant improvement from the previous methods by almost 10%! Since ImageNet-50 is a more complicated dataset, we can even further improve the performance using augmentation. More discussion can be found in Appendix I1
4.3 GENERATIVE PROPERTIES OF THE LEARNED LDR MEMORY
Unlike some of the incremental methods above, which learn models only for classification purposes (as those in Table TT. the i-CTRL model is both discriminative and generative. In this section, we show the generative abilities of our model and visualize the structure of the learned memory. We also include standard metries for analysis in Appendix H
Visualizing auto-encoding properties. We begin by qualitatively visualizing some representative images X and the corresponding replayed X on MNIST and CIFAR-10. The model is learned incremenallly with the datasets split into 5 tasks. Results are shown in Figure where we observe that the reconstructed X preserves the main visual characteristics of X including shapes and textures. For simpler dataset like MNIST, the replayed X are almost identical to the input X! This is rather remarkable given: (1) our method does not explicitly enforce & z x: for individual samples as most autoencoding methods do, and (2) after having incrementally learned all classes, the generator has not forgotten how to generate digits learned earlier, such as 0, 1, 2. For a more complex dataset like CIFAR-10, we also demonstrates good visual quality, faithfully capturing the essence of each image.
Principal subspaces of the learned features. Most generative memory-based methods utilize autoencoders, VAEs, GANs for replay purposes. The structure distribution of the learned features Z, for each class is unclear in the feature space. The features Z, of the i-CTRL memory, on the other hand, have clear linear structure. Figure 2 visualizes correlations among all learned features Z Z), in which we observe clear block-diagonal patterns for both datasets 4 This indicates
the features for different classes Zj indeed lie subspaces that are incoherent from one another. Hence, features of each class can be well modeled as a principal subspace in the feature space. A more precise measure of affinity among those subspaces can be found Appendix [
Replay images of samples from principal components. Since features of each class can be modeled as principal subspace, we further visualize the individual principal components within each of those subspaces. Figure 4 shows the images replayed from sampled features along the top-4 principal components for different classes, on MNIST and CIFAR-10 respectively. Each row represents samples along one principal component and they clearly show similar visual characteristics but distinctively different from those in other rows. We see that the model remembers different poses of '4' after having learned all remaining classes. For CIFAR-10, 10, the incrementally learned memory remembers representative poses and shapes of horses and ships.
4.4 EFFECTIVENESS OF INCREMENTAL REVIEWING
We verify how the incrementally learned LDR memory can be further consolidated with unsuper- vised incremental reviewing phase described at the end of Section 3.3 Experiments are conducted on CIFAR-10, 10, with 10 steps.
'Notiee that these patterns closely resemble the similarity matrix of response profiles of object categories from different of the inferotemporal cortex, as shown Extended DataFig.3 of (Bao al. 2020).
8

---

Published as a conference paper at ICLR 2023
7
1 4
9
4 / /
6
4
8 2 7
5
6
9
9
5
0
6
9
5
5 7
0
/ 1 5
9
5
3
0
0
6
3
8
3
9
0
6
3
8
7
0
6
0
5
5
0
6
5
5 8 4
8
7
6
0
0
7
3
7
8
O
C 6
(a) MNIST
(h) MNIST X
(c) CIFAR-10
(d) CIFAR-IO X
Figure 3: Visualizing the auto-cncoding property of the Iearned i-CTRL (X = g f(X)).
4 4
4
4 4 4 4
4
7 7
7
7
777
4 4
4 4
4
7 7
7
7
7
4
4
7 7
7
7
7
4
4
4
4
4
7
7
7
7
7
(a) sampled dotd of '4' (b) sampled Botd of '7' (c) sampled Hold of (d) sampled hold of 'ship'
'horse'
Figure 4: Visualization of 5 reconstructed å g(z) from z's with the closest distance to (top-4) principal components of learned features for MNIST (class '4' and class '7') and CIFAR-IO (class 'horse" and "ship").
Improving discriminativeeness of the memory. In the reviewing process, all the parameters in the training are the same as incremental learning Table 3 shows how the overall accuracy improves steadily after each cycle of incrementally reviewing the entire dataset. After few (here 8) cycles, the accuracy approaches the same as that from learning all classes together via Closed-Loop Transcription in joint fashion (last column). This shows that the reviewing process indeed has the potential to learn a better representation for all classes of data, despite the review process still trained incrementally.
Rev. cycles
2
JL
Accuracy
0.599
0.626
0.642
0.650
0.658
0.655
Table 3: The overall test accuracies after different numbers of review cycles on CTFAR-10.
Improving generative quality the memory. Figure5]jeft shows replayed images of the first class 'airplane' at the end of incremental learn- ing of all ten classes, sampled along the top-3 principal components every two rows (16 im- ages) are along one principal direction. Their visual quality remains very decent observed almost no forgetting. The right figure shows replayed images after reviewing the first class once. We notice a significant improvement in visual quality after the reviewing, and principal components of the features the subspace start correspond to distinctively different visual attributes within the same class.
Figure 5: Visualization of replayed images fold of class I-'airplane' in CIFAR-10, before (left) and after (right) one reviewing cycle.
5 CONCLUSION
This work provides simple and unifying framework that can incrementally learn a both discriminative and generative memory for multiple classes of objects. By combining the advantages of a closed- loop transcription system and the simple linear structures of a learned LDR memory, our method outperforms prior work and proves, arguably for the first time, that both stability and plasticity can be achieved with only fixed-sized neural system and a single unifying learning objective. The simplicity of this new framework suggests that its performance, efficiency and scalability can be significantly improved in future extensions. In particular, believe that this framework be extended the fully unsupervised or self-supervised settings, and both its discriminative and generative properties can be further improved.

---

Published as a conference paper at ICLR 2023
ETHICS STATEMENT
All authors agree and will adhere to the conference's Code of Ethics. We do not anticipate any potential ethics issues regarding the research conducted in this work.
REPRODUCIBILITY STATEMENT
Settings and implementation details of network architectures, optimization methods, and some common hyper-pane peremeaaeestess are described in the Appendix B We will also make source code available upon request by the reviewers or the area chairs.
ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
Yi Ma acknowledges support from ONR grants N00014-204 1-2002 and N00014-22-1-2102. the joint Simons Foundation-NS': DMS grant #2031 199, well as partial support from Berkeley FHL Vive Center for Enhanced Reality and Berkeley Center for Augmented Cognition, Tsinghua-Berkeley Shenzhen Institute (TBSI) Research Fund, and Berkeley Research (BAIR).
REFERENCES
Jason M. Allred and Kaushik Roy. Controlled forgetting: Targeted stimulation and dopaminergic plasticity modulation for unsupervised lifelong learning in spiking neural networks. Frontiers in Neuroscience, 14, 2020. ISSN 1662-453X. doi: 10.3389/fnnns.20000000
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based on complementary learning system. arXiv preprint arXiv:2201. 12604, 2022.
Ali Ayub and Alan Wagner. EEC:
Learning to encode and regenerate images for continual learning. In International Conference on Learning Representations, 2020.
Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung- Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with memory of diverse samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8218-8227, 2021.
Pinglei Bao, Liang She, Mason McGill, and Doris Y. Tsao. A map of object space in primate inferotemporal cortex. Nature, 583:103-108, 2020.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2020.
Kwan Ho Ryan Chan, Yaodong Yu, Chong You. Haozhi Qi, John Wright, and Yi Ma. ReduNet: white-box deep network from the principle of maximizing rate reduction. CoRR, abs/2105. 10446. 2021. URL http:ttps///rrri..rrg///sas///
Le Chang and Doris Tsao. The code for facial identity in the primate brain. Cell, 169:1013-1028.c14 06 2017. doi: 00.1016fj.cel1.0017.00
Arslan Chaudhry, Marc Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00440, 2018
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Dokania, Philip HS Torr, and Mare' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv: /902. 10486, 2019a.
Arslan Chaudhry, Mareus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv: 1902. /0486, 2019b.
10

---

Published as a conference paper at ICLR 2023
Ting Chen, Simon Korblithh, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Xili Dai, Shengbang Tong, Mingyang Li, Ziyang Wu, Michael Psenka, Kwan Ho Ryan Chan, Pengyuan Zhai, Yaodong Yu, Xiaojun Yuan, Heung-Yeung Shum, et al. Ctrl: Closed-loop transcription to ldr via minimaxing rate reduction. Entropy, 24(4))456, 2022.
Xili Dai, Ke Chen, Shengbang Tong, Jingyuan Zhang, Xingjian Gao, Mingyang Li, Druv Pai, Yuexi- ang Zhai, Xlaojun Yuan, Heung- Yeung Shum, et al. Closed-loop transcription via convolutional sparse coding. arXiv preprint 2332. 09347, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Tianjiao Ding, Shengbang Tong, Kwan Ho Ryan Chan, Xili Dai, Yi Ma, and Benjamin D Haeffele. Unsupervised manifold linearizing and clustering. arXiv preprint 23Xiv:2301.01800 2023.
Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance. Cogn. Sci., 11:23-63, 1987.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of Wasserstein GANs. arXiv preprint (7xXiv:70440000 2017.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Sheena A. Josselyn and Susumu Tonegawa. Memory engrams: Recalling the past and imagining the future. Science, 367, 2020.
Ronald Kemker and Christopher Kanan. FearNet: Brain-inspired model for incremental learning. In International Conference on Learning Representations, 2018.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6988) 2014.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinske. et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 (13):3521-3526, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Citeseer, 2009.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The CIFAR-10 dataset. online: http://www. cs. toronto. edu/kriz/cifar html, 55, 2014.
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1) (66(1))222888 1):2778-2324 1998.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions pattern analysis and machine intelligence, 40(12):2935-22 2947, 2017.
Yi Ma, Harm Derksen, Wei Hong, and John Wright. Segmentation of multivariate mixed data via lossy data coding and compression. PAMI, 2007.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 11321-11329, 2019.

---

Published as a conference paper at ICLR 2023
Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. In European conference on computer vision, pp. 524-540) Springer, 2020.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv preprint arXiv:1511.0645 2016.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. iCaRL: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Anthony V. Robins. Catastrophie forgetting, rehearsal and pseudorehearsal. Connect. Sci., 7:123-144 146, 1995.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1001::1606.04671, 2016.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances neural information processing systems, 29, 2016.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. arXiv preprint arXiv: 1705,086590 2017.
Mahdi Soltanolkotabi, Ehsan Elhamifar, and Emmanuel J Candes, Robust subspace clustering. The Annals of Statistics, 42(2):669-666 6999 2014.
Shengbang Tong, Xili Dai, Yubei Chen, Mingyang Li, Zengyi Li, Brent Yi, Yann LeCun, and Ma. Unsupervised learning of structured representations via closed-loop transcription. arXiv preprint arXiv:2210. 16782, 2022.
Gido M Ven, Hava Siegelmann, Andreas S Tolias, et al. Brain-inspired replay for continual learning with artificial neural networks. Nature Communications, 11(1):1-14, 2020.
Chenshen Wu, Luis Herranz, Xialei Liu, Joost van de Weijer, Bogdan Raducanu, Memory replay GANs: Learning to generate new categories without forgetting, Advances in Neural Information Processing Systems, 31:5962-5972. 2018.
Ziyang Wu, Christina Baek, Chong You, and Yi Ma. Incremental learning via rate reduction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1125-1133, 2021.
Shipeng Yan, Jiangwei Xie, and Xuming He. DER: Dynamically expandable representation for class incremental learning. arXiv preprint arXiv:2103. 16788, 2021.
Jaehong Yoon, Eunho Yang. Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01544 2017.
Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Yi Ma. Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Advances in neural information processing systems, 2020.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995, PMLR, 2017.
Guanyu Zhou, Kihyuk Sohn, and Honglak Lee, Online incremental feature learning with denoising autoencoders. In Artificial intelligence and statistics, pp. 1453-1461. PMLR, 2012.
12

---

Published as a conference paper at ICLR 2023
A ALGORITHM OUTLINE
For simplicity of presentation, the main body of this paper has described incremental learning with each incremental task containing one new class of data. general, however, each incremental task may contain finite C new classes. In this section, we detail the algorithms associated with i-CTRL this more general setting.
Suppose we divide the overall task of learning multiple classes of data D into a stream of smaller tasks DJ_ D2 D' D'...... D", where each task consists of labeled data D' = {x', YO from classes, i.. X (Xi { The overall i-CTRL process is summarized in Algorithm 3
We begin by training the model on the first task D optimized via the original objective function 4 We then use FORMING MEMORY MEAN AND COVARIANCE I to find M the means and covariances of the representations of classes in the first task. When learning new task D', we first sample Zold using MEMORY SAMPLING 2 We then take (X', Y*) from D', and calculate X > Z X Zt using f(, ((, m) to obtain Z' and Zt We next compute Zold + Xold - Zoid using [(-, 9( 1)). So far, we get Z = [Z', Zaral and 2 = [Z', Zata]. The encoder updates by optimizing the objective (8)
max A Z)) AR 2) + XAR Z' 2)) 7 A Ri Zold; Boid).
The decoder updates yy via optimizing the objective
min AR Z) + AR(Z) AAR(Z' Z¹) + 7AR( Zole).
We optimize these objectives until the parameters converge. After the training session ends, we calculate M of this learn task using FORMING MEMORY MEAN AND COVARIANCE The process of training task is repeated until all tasks are learned.
Algorithm 1 FORMING MEMORY MEAN AND COVARIANCEL Z k,
Require: classes of features Z = Zj Zy..... Zel the entire 1-th task, t E [L...., T.. The parameters k and which correspond to top-k features each of top-r cigenvectors which we will sample on each class;
1: for j j= 1, 2,.....; C do
2: Calculate the top-r eigenvectors V of Zj. Vi = [ U,] where v, means the n-th =
eigenvector;
3:
for 1, ,, do
4:
Calculate projection distance d v,. Z):
5:
Choose the top-k features from Z based on the distance d to form the set S;;
6: Calculate mean μ4 and covariance Σ, based on the set S;;
7: end for
8: Obtain memory mean and covariance of j-th class Bj = [(11, Σ1), .... (Mr, E,));
9: end for
10: Memory of mean and covariance set for 1-th task M - [B,..., Bc).
Ensure: M
B IMPLEMENTATION DETAILS
A simple network architecture. Tables and give details of the network architecture for the decoder and the encoder networks used for experiments reported in Section 4 All a values in Leaky-ReLU (i.e. IReL.U) of the encoder are set to 0.2. We set (næ = 128 and nc = 1) for MNIST, (nz 128 and ne 3) for CIFAR-I0 and CIFAR- 100, (nz = 256 and nc = 3) for ImageNet-50. For ImagetNet-50, we added down sample and up sample layer in f and g respectively to match the resolution of ImageNet-50. The details architecture are given in Appendix B The dimension d of the feature space set accordingly for different datasets, d 128 for MNIST and CIFAR-10. d =256 for ImageNet-50. More details about the algorithmic settings and ablation studies are given in the Appendix.
13

---

4, stride=2, pad=1 deconv. Tanh Published as a conference paper at ICLR 2023
Algorithm 2 MEMORY SAMPLING(M'. ..... K,
Require: A set of Memory M M ...,.,,, where M = [B,,... Bcl, k, r and C, which is the
number of classes in each task; Initialize an empty Zold:
2: for . do
for j ....... C do
4:
get B, from M', B, = [(11, Ei)...., (Mr, E.)];
For each direction E r, sample k number of samples from distribution N(n, Ei), add
them Zoldi
6: end for end for
Ensure: Zold
Algorithm 3 i-CTRL
Require: A stream of tasks D D2 DD....,D" DT, where D' (X', Y A pre-trained encoder f(,
and decoder g(: 17) on D k and r;
Calculate Z via [(x+, 0);
2: Find M by FORMING MEMORY MEAN AND COVARIANCE(Z kk) for 2, do
4:
Sample Zole = MEMORY SAMPLING(M). M kr C);
while not converged do
6:
Draw samples in (X', Y") from the t-th task D::
Compute expressions: X + Z + X Z:
8:
Replay the old memory Zold Xold
Zoldi
Z Iz' Zota]: 2 [ Zotal;
10:
Update via the optimization objective
8
Update 7) via the optimization objective end while
12:
Calculate Z' via f(x',(XX 8);
14:
Find M by FORMING MEMORY MEAN AND COVARIANCE(Z'S k,rr
end for
Ensure: ((, and s(:1)
Optimization settings. For all experiments, we use Adam (Kingma & Ba 2014 as our optimizer, with hyperparameters 81 0.5, 82 0.999. Learning rate is set be 0.0001. We choose e² 1.0, = 1, and X 10 for both equation (8) and (9) in all experiments. For MNIST, CIFAR-10 and CIFAR- each task is trained for 120 epochs; For ImageNet-50, the first task D is trained for 500 epochs with constraint on augmentation used in (Chen et al.] (2020) and 150 epochs for rest incremental 4 tasks using the normal i-CTRL objective All experiments are conducted with
RTX 3090 GPUs.
Prototype settings we use prototype sampling in this method, so the storage becomes almost
trivial. For MNIST, we choose 6, k 10. For 11,AR-10, we choose r 12, k 20. For ImageNet-50, we us r = 10, = 15. For CIFAR-100, we us r = 10, 20.
E R Xx
4, stride=1, pad=0 deconv. BN 256 ReL.U
4 x 4, stride=2, pad=1 deconv. BN 128 ReLU
4 x 4, stride=2, pad=1 deconv. BN 64 ReLU
Table 4: Network architecture of the decoder g(:, 7).
14

---

Published as a conference paper at ICLR 2023
Image R-2232226
4 X 4, stride=2, pad=1 conv 64 IReLU 4, stride=2, pad=1 conv. BN 128 IReLU stride=2, pad=1 conv, BN 256 IReLU
4 x 4, stride=l, pad=0 conv 122
Table 5: Network architecture of the encoder f(,
052
Figure Aflinity'betweenn between inemòry subspaces within CIFAR-10.
A simple nearest subspace classifier. Similar to (Dai et al. 2022) and Yu et al., 2020), adopt a very simple nearest subspace algorithm to evaluate how discriminative our learned features are for classification. Suppose Zj are the learned features of the j-th class. Let μıj IR be its mean and Uj E Rd be the first ry principal components for Zj. where ry is the estimated dimension of class j. For a test data x', its feature z' is given by f(x', 8). Then, its class label can be predicted by j' arg minje ( (I 10--), U,U, (2' (4) B- is especially noteworthy that our method does not need to train a separate deep neural network for classification whereas most other methods do.
C RESOURCE COMPARISON DETAILS
In Table of the appendix, both i-CTRL and EEC methods are tested on CIFAR-10. Under joint learning, CTRL follows the setting in Appendix A.4 of (Dai et al. 2022), but we adopt the architec- tures of the encoder and decoder detailed in Table and Table reespetivvl The training batch size is 1600 over 1400 epochs because the generative CTRL model is more challenging train than the simple classifier network that EEC uses in the joint learning setting; EEC uses the ResNet architecture from (Gulrajani al. 2017) for the classifier, with training batch size and training epochs of 128 and 100 respectively.
D AFFINITY BETWEEN LEARNED SUBSPACES
As we see in Figure 2 the Iearned features of different classes are highly incoherent and their correlations form block-diagonal pattern. We here conduct more quantitative analysis of the affinity among subspaces learned for different classes. The analysis is done on features learned for CIFAR-10 10 using 10 splits with 2000 features. For two subspaces S and S' of dimension d and d' we follow the definition of normalized affinity in (Soltanolkotabi et al., 2014):
Σ ded' cos? 0i
aff(S = =1
(10)
d'
We calculate the Sf S') through |IU U'I where UIU" is the normalized column space of features Z/Z' that can be obtained by SVD.
The affinity measures the angle between two subspaces. The larger the value, the smaller the angle. As shown in Figure we see that similar classes have higher affinities. For example, 8-ship and 9-trucks have higher affinity in the figure, whereas 6- 6-frogs has much lower affinity than these two classes. This suggests that the affinity score of these subspaces captures similarity in visual attributes between different classes.
15

---

Published as a conference paper at ICLR 2023
Method
Resource
JL
IL
Diff
i-CTRL
Model Size
2 M
same
(ours)
Train Time
15 hours
1.5 hours
10x faster
Model Size M
10 M
10x larger
EEC
Train Time
0.6 hours
24 hours
7x slower
Table 6: The resource comparison the joint learning (JL.) and incremental learning (IL) of different methods. Both methods are tested CIFAR-10 10 and the details of the comparison setting can be found in Appendix C
E INCREMENTAL LEARNING VERSUS JOINT LEARNING.
One main benefit of incremental learning is to learn one class (or one small task) at time. So should result in less storage and computation than jointly learning. T shows this is indeed the case for our method: IL on CIFAR-10 is times faster than JI. However, this is often not the case for many existing incremental methods such as EEC (Ayub & Wagner 2020), the current SOTA in generative memory-basee methods, Not only does its incremental mode require a much larger model size (than its joint mode and o. it also takes significantly (7 times) longer train.
F ABLATION STUDIES
We conduct all ablation studies under the setting of CIFAR-10 split into 5 tasks with feature size of 2000, and default values of lt = 20, r = 12, = 10, and ς = 1. We use the average incremental accuracy as measure for these studies.
F.1 IMPACT OF CHOICE OF OPTIMIZATION PARAMETERS
Parameter m and r for memory sampling. Here, we verify the impact of the memory size of Algorithm on the performance of method. The feature size is determined by two hyperparameters which is the number of the PCA directions and m, which is the number of sampled features around each principal direction. The value of varies from 10 14, and the value of varies from 20 to 40. Table reeports the results of the average incremental accuracy. From the table, we observe that as long as the selection of m and are reasonable range, the overall performance is stable.
m=20
m=30
m=40
10
0.713
0.720
0.728
1=
0.719
0.727
0.725
14
0.718
0.721
0.725
Table 7: Ablation study on varying m and r in PROTOTYPESAMPLING, in terms of the average incremental accuracy.
Hyperparameter À and y in the learning objective. À and y are two important hyperparameters in the objective functions for both (8) and (9). Here, want to justify selection of À and y and demonstrate the stability of our method to their choices. We analyze the sensitivity of the performance to the À and respectively. In Table we set 7 1 and change the value of À from 0.1 to 50. The results indicate the accuracy becomes low only when À are chosen to be extreme (c.g 0.1, 1, 50). We then change the value of y in a large range from 0.01 to 100 with fixed at 10. Results in Table 9 indicate that the accuracy starts to drop when 7 is larger than 10. Hence, in all our experiments reported in Section we set À 10 and ? for simplicity.
"Note in our method, both JL and IL optimize on the same network. The JL mode is trained on all ten classes together. hence normally takes more epochs converge and longer time train. But the IL mode converges much faster, should have.
'For EEC, since its classifier and generators are separated. under the JL setting, it only needs 8-layers convolutional network to train a classifier for all classes. In the incremental mode, it requires multiple generative models. Note that our JI. model is also generative model hence requires more time to train as well.
16

---

Published as a conference paper at ICLR 2023
0.1
10
20
50
Accuracy
0.592
0.620
0.712
0.701
0.691
Table 8: Ablation study on varying À in terms of the average incremental accuracy.
0.01
0.1
100
Accuracy
0.713
0.716
0.712
0.700
0.655
Table 9: Ablation study on varying in terms of the average incremental accuracy.
F.2 SENSITIVITY TO CHOICE OF RANDOM SEED
is known that some incremental learning methods such as (Kirkpatrick et al. 2017) can be sensitive to random seeds. We report in Table TO the average incremental accuracy of i-CTRL with different random seeds (conducted on CIFAR-TO split into 10 tasks, with feature size 2000). As we can see, the choice of random seed has very little effect on the performance of i-CTRL..
Random Seed
5
10
15
100
Average Accuracy
0.720
0.720
0.720
0.720
0.721
Last Accuracy
0.594
0.592
0.593
0.594
0.594
Table 10: Ablation study on varying random seeds.
F.3 THE SIGNIFICANCE AUGMENTATION IN TRAINING IMAGENET-50
In this section, we study the the impact of using augmentation from Chen et al. (2200) to train the first task has on our method. From Tab IT we conclude that augmentation did help the model the learn better representation. Even without it, it has shown that our method still outperform the current generative-replay based method by more than 10%. Through this experiment, we think that add augmentation may be the solution for generative-replay based methods scale up even larger datasets. We leave that to future study.
F.4 THE SIGNIFICANCE OF CONSTRAINT IN MINMAX OPTIMIZATION
Here, we want to justify the significance of this constraint in the context of incremental learning. We report in table 12 the performance of i i-CTRL with and without constraint. Without the constraint, i-CTRL fall into the victim catastrophic forgetting. We can conclude that constraint has played a significant role in the success of our method.
G COMPARISON WITH MORE BASELINES
Due the limitation of space in main paragraph, we present here table with more comparison with other methods.
From the table, we see that comparing to the previous methods especially exemplar-based methods, our method still leads them numerically. We have also conducted on experiments on 1IFAR-100 On more complex data such as that i-CTRL has led the current exemplar- camplaa-bbssdd methods. It is noteworthy that there is generative- replayed based methods in the table. Since it hard for many of the current generative-replay based methods scale up to complex setting.
et al. 2014), is also observed in Tab [4]
H QUANTITATIVE EVALUATION OF LEARNED GENERATOR
In this section, we use FID score (Heusel et al. 2017) and Inception Scores (IS) Salimans et al.. 2016) to quantitatively measure the performance of our incrementally learned generator. As there
17

---

Published as a conference paper at ICLR 2023
iCaRL-S
BEIL-S
DGMw
EEC
EECS
i-CTRL aug)
i-CTRL(with aug)
0.290
0.118
0.178
0.352
0.309
0.458
0.523
Table 11: Comparison on ImageNet-S0. The results of other methods are as reported the EEC paper.
With Constraint
Without Constraint
Average Accuracy
0.723
0.380
Last Accuracy
0.627
0.223
Table 12: Ablation study on the important of constraint in
MNIST
CIFAR-10 10
Method
10-splits
5-splits
10-splits
5-splits
Last
Avg
Last
Avg
Last
Avg
Last
Avg
Regularization
LwF (Li & Hoiem 2017
0.196
0.455
0.196
0,440
SI Zenke al. 2017)
0.193
0.461
0.196
0.441
Architecture
ReduNet (Wu et al.
2021
0.961
0.982
0.539 0.645
Exemplar
iCaRL (Rebuffi al.
2017
0.322
0.588
0.725
0.803
0.212
0.431
0.487
0.632
A-GEM (Chaudhry et al. 2018
0.382
0.574
0.597
0.764
0,115
0.293
0.204
0.473
CLR-ER (Arani
2022)
0.895
0.662
ER Re-Reeervoir (Chaudhry hry et al.
2019b
0,685
GDumb (Prabhu
2020)
0.919
0.618
Rainbow Memory [ang et al
2021
0.927
+ Buzzega et al. 2020
0.648
Generative Memory
DGMw Ostapenko et al. 2019
0.965
0.562
EBC Ayub & Wagner
2020
0.978
0.669
FECS (Ayub & Wagner
2020
0.963
0.619
i-CTRI (ours)
0.975
0.989
0.978
0.990
0.599
0.727
0.627 0.723
Table 13: Comparison on MNIST and CIFAR-10.
Last Accuracy
Last Accuracy
Last Accuracy
Method
5-split
10-split
20-split
LwFiLi & Hoiem 2017
0.252
0.141
(Reuui al. 2017
0.346
GDUMB(Prabhu et al.
2020
0.241
Rainbow Memory Bang et al.
2021
0.414
i-CTRL
0.435
0.392
0.378
Table Comparison on CIFAR-100.
ISt
FID
DCGAN (Radforr 2016)
37.4
i-CTRL (ours)
36.7
Table 15: Comparison IS and FID CIFAR-10
exist very few generative-based replay-based incremental methods offer quantitative result for us to compare. We here compare with DCGAN (Radford et al.) 2016), which is the backbone of our method, trained jointly for all classes. Based on 15blc is seen that method has competitive FID and IS score comparing to DCGAN. Hence, despite trained incrementally, our method still generates high-quality images.
18

---

Published as a conference paper at ICLR 2023
I I-CTRL IN EXTREME SETTING
In this section, we conduct some ablation study of i-CTRL implemented extreme settings.
I.1 IMBALANCED DATASETS
Often in real life, the data encounter not perfectly balanced. To testify our model's performance in this situation, we conduct experiment on
In this subsection, CIFAR-10 is split into 5 tasks, with task and task 4 having only half of the original data. We call this setting Meallanc-CIFAR-II i-CTRL is trained with parameters same as section B From table 16 we observe that imbalance CIFAR-10 has very little impact the performance of method.
CIFAR- 10 (balance)
CIFAR-10 10 (imbalance)
Last Accuracy
Average Accuracy
Last Accuracy
Average Accuracy
i-CTRL
0.627
0.727
0.623
0.723
Table 16: Comparison of i-CTRL performance on CIFAR-10 and imbalance CIFFR-------
I.2 SMALL SUBSET OF DATA
Another interesting extreme scenario to examine would be small subset of dataset. Again, we may not get large number of dataset for to train every time. To testify i-CTRL's performance in this scenario, we design
10. For example, We denote CIFAR-10(50%), meaning we have deleted 50% of data from every class in CIFAR-10. We run i-CTRL CIFAR-10660%), ) (0080).. CIFAR-10( 100%) without tuning any parameter. From Table observe that smaller daatset will have impact the performance of method. If the portion is larger than 20%, the impact is relatively small. When the size of data reduces to only 20%, the impact becomes larger. Nonetheless, since CIFAR-10(20%) is nearly new dataset, we can reduce the impact by tuning parameters. In Table 18
(IFAAR-10(20%) CIFAR-10(40%)
we present results of i-CTRLon CIFAR (002-10 after
CIFAR-10 (balance)
Last Accuracy
Average Accuracy
CIFAR- CIFAR-10(20%)
0.476
0.625
CIFAR- (0(40)
0.575
0.691
CIFAR- 10(60))
0.589
0.709
KIFAR-10(80%)
0.615
0.721
CIFAR-IO (0(100)))))
0.627
0.727
Table 17: Comparison of i-CTRL performance on diffeeent scales of CIFAR-10
tuning the hyperparamter À and epochs. Since CIFAR-10(20%) is very small dataset, we reduce the number of X and epochs to avoid forgetting previous learned classes. We observe that smaller and epochs can greatly improve the performance of i-CTRL on very small subset of data like CIFAR- 10020)..
CIFAR-10 (balance)
Last Accuracy
Average Accuracy
CIFAR-10/20%), À 10, epochs=120 120
0.476
0.625
CIFAR-10/20%), 66
0.541
0.671
Table 18: Comparison of i-CTRL performance CIFAR-10(20%) with different hyperparameters
19

---

Published as a conference paper at ICLR 2023
J USING AFFINITY TO MEASURE THE PERFORMANCE
In this section, we discuss affinity between the memory subspaced learned can be used to evaluate the performance of our method. Following the setting of subset CIFAR-I0, visualize the affinity in Fig 7 From the figure, we see that as the subset of CIFAR-I0 becomes smaller, the affinity learned by i-CTRL becomes more distant. can be used as sign for unsatisdying performance because ideally. we would want the affinity between similar classes (truck and car) to be close. If the affinity graph shows that the model does not capture this kind of relationship, it sign that the overall performance could be worse.
135
175
350
125
140
030
630
(a) Affinity between memory subspace Affinity between memory sub- within CIFAR(100%)
space within CIFAR(80%)
2000
125
075
025
(c) Affinity between memory subspace (d) Affinity between memory within (6FAR(60%)
space within CIFAR(40%)
200
135
150
125
075
025
Affinity between memory subspace
within CIFAR(20%)
Figure 7: Visualization of the affinity between memory subspace under different subset of CIFAR-10
20


Reviews:
[{"review_id": "gDTNQkahK0", "reviewer": "Reviewer_sS4d", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper provides a framework for incrementally learning a generative and discriminative model for multi-class problems using a Linear Discriminative Representation (LDR) and a coding rate reduction objective. The closed-loop transcription framework (CTRL) is formulated as a minimax game wherein the encoder tries to maximize the rate reduction objective and the decoder minimizes it instead. The \"distance\" between Gaussian ensemble subspaces is estimated.\n", "summary_of_the_review": "The paper provides a framework for incrementally learning a generative and discriminative model for multi-class problems using a Linear Discriminative Representation (LDR) and a coding rate reduction objective. \n\nThe closed-loop transcription framework (CTRL) is formulated as a minimax game wherein the encoder tries to maximize the rate reduction objective and the decoder minimizes it instead. The \"distance\" between Gaussian ensemble subspaces is estimated.\n\nThe authors state \"To this end, we simply sample m representative prototype features on the subspace along its top r principal components, and denote these features as Zj,old\" -> This step clearly depends on how well the principal components capture the variance of the data in the first place and affects the samples collected using the mean and variance of Zj,old.\na. How robust is the scheme with imbalanced classes?\nb. If incremental learning process learns from a small subset of data, this could affect the principal components (and hence their variances). How is this dealt with?\nc. Are there empirical results to justify how a change in percentage of variance of principal components affects the overall performance of the algorithm -- in terms of performance metrics, imbalanced multiple classes and relevant details? \nd. Can affinity based subspaces be used to measure the effect of percentage of variance captured by principal components?\n\nAppendix A contains a lot of crucial algorithmic details and the authors should consider incorporating those in the main paper. ", "strength_and_weaknesses": "Strength\n- Paper has extensive theoretical and empirical analysis\n- It is written coherently and studies related work carefully", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "Uy9X8kAV7j", "reviewer": "Reviewer_72tt", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work, the authors present a method to mitigate forgetting in Continual Learning based on closed-loop transcription (CTRL). The authors adapt CTRL to condense the classes' information in a memory composed of an encoder and decoder capable of generating representations of previous tasks. Using linear discriminative representations, CTRL encourages the models to learn compressed representations corresponding to different classes so as to maximize discriminative and generative properties. Taking advantage of these features, the authors propose to store the mean and covariance of the Z representations for each class, and then use these to mitigate forgetting, aiming for the covariance to remain stable while training a new task. The authors show promising results in various benchmarks.", "summary_of_the_review": "I found the paper well written, with its motivations and results correctly showing the contributions. My only concern is a possible limitation not mentioned by the authors: the capacity to store many tasks/classes of the subspace (what happens when we add more than 50 tasks).", "strength_and_weaknesses": "S:\n- The idea presented is very well-motivated not only from CL's point of view but also from the perspective of how we accumulate knowledge as humans.\n- The fact that it is a generative model with fixed capacity makes it an exciting proposal since it does not require adding new weights to a model or re-training models with the problems of overfitting a small group of data.\n- Applying CTRL to an incremental learning environment is a smart idea. Accumulating \"classes\" in separate subspaces and saving this space's mean and covariance can help to mitigate forgetting.\n- In general, the ideas are well explained, and the structure of the paper is adequate.\n- The experiments carried out in section 4.3 are interesting. It might have been interesting to see how it behaves with larger images like ImageNet.\n\nW:\n- One problem is that the method is incremental from CTRL. The authors propose only minor (but critical) modifications to CTRL to apply it to CL.\n- A possible limitation of this model is the subspaces that the model finds. I imagine that as the number of classes increases, finding incoherent subspaces becomes more challenging. To the authors: Did you test with benchmarks with more classes? Like CIFAR100 or TinyImageNet, which has 200 classes.\n\nQ:\n- I wonder if the first subspaces that the model learns can affect the model's behavior in the future. I imagine that learning easy or difficult subspaces in the first tasks can affect in one way or another the subspaces that the model finds later. What are your opinions about it?\n- The constraint added in Eq 8 and 9 can be seen as a regularization similar to distillation on the covariance. The gamma value regulates the importance of maintaining the subspace of the previous classes. Did you carry out experiments on how this value affects training?\n- What does the Union in Eq3 represent? The concatenation?\n- How many times was each experiment run? Could you get the std from the experiments?\n- I like the idea of ​​using the nearest neighbor as a classifier. The other methods used a typical classifier or something like iCARL? There is evidence that most forgetting occurs in the classifier, so it might be interesting to compare it equally.\n- In the section \"Improving discriminativeness of memory\". The factor that regulates the difference between covariances, is used? Or is using gamma equal to zero?\n\nI recommend changing Fig5. It is not easy to see that the photos are grouped in rows of two.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "6QqEn5cens", "reviewer": "Reviewer_HArM", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method, I-CTRL, for incremental class learning that learns a ‘closed loop transcription’ between each of the classes and a corresponding subspace of a low-dimensional feature space, know as a linear discriminative representation. The closed loop transcription is achieved by optimising an autoencoder (that maps the inputs to the feature space and back) with a rate reduction-based minimax objective. The objective ensures that the subspaces are highly incoherent to each other, such that updating one does not interfere with the others, preventing catastrophic forgetting. Furthermore, since the subspaces are linear, each class can be summarised with an estimated mean and covariance matrix in the feature space. These summaries are stored in memory as training progresses and are then used to optimise the minimax objective in feature space by essentially ensuring that the features are consistent with their sampled reconstructions through the closed loop transcription. Experiments are run on a number of incremental class learning benchmarks (MNIST, CIFAR10 and Imagenet-50), where the method is shown to outperform all the baselines, which include the SOTA among methods that (a) do not expand their architecture throughout training or (b) maintain a large number of raw exemplars. It is also shown that the method can benefit from unsupervised replay of old examples. The method can be used as both a discriminative and a generative model, and some visualisations are provided that demonstrate the quality of generated samples, as well as the block-diagonal structure of the learnt feature space.", "summary_of_the_review": "This paper presents a novel approach to continual learning along with impressive results on incremental class learning benchmarks with a minimal memory footprint. While the claims of SOTA are potentially misleading, when a large class of methods (dynamic architectures and replay-based methods) are questionably discarded as unfair comparisons, the results are undoubtedly strong and will be of great interest to the CL community.", "strength_and_weaknesses": "Strengths\n- The method demonstrates impressively strong results on a number of tasks in the challenging incremental learning setting, improving greatly over relevant baselines.\n- The idea is compelling, and although it uses techniques from previous works (Wu et al 2021, Dai et al 2022), it is adapted uniquely for the incremental class learning setting restricted to a fixed network capacity. The novelty of the method, its minimal memory footprint and the impressive results will certainly be of interest to the continual learning community.\n- The paper is easy to read and the method clearly described.\n\nAreas for improvement / questions\n- Though it is clear that I-CTRL has the advantage over Redunet that (i) the results are better and (ii) it does not require dynamic expansion of the network, since they seem to use the same objective function, it would be good to have a clearer description of how the two methods differ in how they use it.\n- Minimax objectives can often be unstable and sensitive to hyperparameters - was there evidence of instability during training and how sensitive is the performance to the values of lambda and gamma that weight the rate reduction losses for the new features and old features respectively?\n- Though the results without augmentation are reported in the appendix are still better than the baselines ( though 6-7% worse accuracy than with augmentations), would it not be fairer to use these as comparison to other methods, which presumably do not use augmentation (from a glance at the EEC paper, it does not).\n- Many would disagree with the decision to discard methods that keep raw exemplars as unfair comparisons and on this basis call the results of I-CTRL SOTA (which they would not be otherwise, e.g. Dark Experience Replay outperforms I-CTRL on CIFAR-10 with a relatively modest buffer size of 500 exemplars). Indeed ICTRL still makes use of a memory buffer and one version of iCTRL that is advertised involves a form of replay of previous exemplars. While I certainly believe that the minimal memory capacity required by I-CTRL is a great feature and is rightly highlighted in the paper, the SOTA claim is perhaps a little strong given that one can easily argue that replay-based methods are very practical continual learning methods given that memory storage is generally cheap. \n- It is mentioned in the introduction that EWC requires application-specific external mechanisms - what aspect of EWC is application-specific?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster