EEC: Learning to Encode and Regenerate Images for Continual Learning
Authors: Ali Ayub
Alan Wagner
Venue: ICLR 2021 Poster (2021)
Keywords: Continual Learning
Catastrophic Forgetting
Cognitively-inspired Learning

Abstract: The two main impediments to continual learning are catastrophic forgetting and memory limitations on the storage of data. To cope with these challenges, we propose a novel, cognitively-inspired approach which trains autoencoders with Neural Style Transfer to encode and store images. Reconstructed images from encoded episodes are replayed when training the classifier model on a new task to avoid catastrophic forgetting. The loss function for the reconstructed images is weighted to reduce its effect during classifier training to cope with image degradation. When the system runs out of memory the encoded episodes are converted into centroids and covariance matrices, which are used to generate pseudo-images during classifier training, keeping classifier performance stable with less memory. Our approach increases classification accuracy by 13-17% over state-of-the-art methods on benchmark datasets, while requiring 78% less storage space.

Published as a conference paper at ICLR 2021
EEC: LEARNING TO ENCODE AND REGENERATE IM- AGES FOR CONTINUAL LEARNING
Ali Ayub, Alan R. Wagner The Pennsylvania State University State College, PA, USA, 16803 ja alan. r. wagner [agner)epss.. @psu. edu. edw..edu.uueeeeeeeee..... edu
ABSTRACT
The two main impediments to continual learning are catastrophic forgetting and memory limitations on the storage of data. To cope with these challenges, we propose novel, cognitively-inspire, approach which trains autoencoders with Neural Style Transfer to encode and store images. During training on a new task, reconstructed images from encoded episodes are replayed in order avoid catas- trophic forgetting. The loss function for the reconstructed images is weighted to reduce its effect during classifier training cope with image degradation. When the system runs out of memory the encoded episodes are converted into centroids and covariance matrices, which are used to generate pseudo-images during classi- fier training, keeping classifier performance stable while using less memory. Our approach increases classification accuracy by 13-17% over state-of-the-arr meth- ods on benchmark datasets, while requiring 78% less storage space
INTRODUCTION
Humans continue to learn new concepts over their lifetime without the need to relearn most previous concepts. Modern machine learning systems, however, require the complete training data to be available at one time (batch learning) (Girshick] (2015). this paper, we consider the problem of continual learning from the class-incremental perspective. Class-incremental systems are required to learn from a stream of data belonging to different classes and are evaluated in a single-headed evaluation (Chaudhry et al. 2018). In manglehheaded evaluation, the model is evaluated on all classes observed so far without any information indicating which class is being observed.
Creating highly accurate class incremental learning systems is a challenging problem. One simple way to create a class-incremental learner is by training the model on the data of the new classes, without revisiting the old classes. However, this causes the model to forget the previously learned classes and the overall classification accuracy decreases, a phenomenon known as catastrophic for- getting (Kirkpatrick et al. 2017). Most existing class-incremental learning methods avoid this prob- lem by storing a portion of the training samples from the carlier learned classes and retraining the model (often a neural network) on a mixture of the stored data and new data containing new classes (Rebuffi et al.) [017 Hou et al.) (019)). Storing real samples of the previous classes, how- ever, leads to several issues. First, as pointed out by Wu al. (201 8b), storing real samples exhausts memory capacity and limits performance for real-world applications. Second, storing real samples introduces privacy and security issues (Wu et al. 2018b). Third, storing real samples is not biologi- cally inspired, i.c. humans do not need to relearn previously known classes.
This paper explores the "strict" class-incremental learning problem in which the model is not al- lowed to store any real samples of the previously learned classes. The strict class-incremental learn- ing problem is more akin to realistic learning scenarios such as a home service robot that must learn continually with limited on- on-board memory. This problem has been previously addressed us- ing generative models such as autoencoders (Kemker & Kanan 2018) or Generative Adversarial Networks (GANs) (Ostapenko al. 2019). Most approaches for striet class-incremental learning
A preliminary version of this work was presented at 1C20 Workshop on Lifelong Machine Learning Ayub& Wagner 2020c).

---

Published as a conference paper at ICLR 2021
use GANs to generate samples reflecting old class data, because GANs generate sharp, fine- grained images (Ostapenko et al., 2019). The downside of GANs, however, is that they tend to generate images which do not belong any of the learned classes, hurting classification performance. Au- toencoders, on the other hand, always generate images that relate to the learned classes, but tend to produce blurry images that are also not good for classification.
To cope with these issues, we propose a novel, Dognitively-inspieee approach termed Encoding Episodes as Concepts (EEC) for continual learning, which utilizes convolutional autoencoders to generate previously learned class data. Inspired by models of the hippocampus (Renoult et al.) 2013). we use autoencoders create compressed embeddings (encoded episodes) of real images and store them in memory. To avoid the generation of blurry images, we borrow ideas from the Neural Style Transfer (NST) algorithm proposed by Gatys et al. (2016) to train the autoencoders. For efficient memory management, we use the notion of memory infegration, from hippocampal and neocortical concept learning (Mack et al.. 2018), to combine similar episodes into centroids and covariance matrices eliminating the need to store real data.
This paper contributes: 1) an autoencoder based approach to strict caas-incrrmental learning which uses Neural Style Transfer to produce quality samples reflecting old class data (Sec. [.1];; 2) a cognitively-inssiree memory management technique that combines similar samples into a cen- troid/covariance representation, drastically reducing the memory required (Sec. (.2); 3) data fil- tering and a loss weighting technique to manage image degradation of old classes during classifier training (Sec. B.3). We further show that EEC outperforms ofate-of-the-art (SOTA) approaches on benchmark datasets by significant margins while also using far less memory.
2 RELATED WORK
Most recent approaches to class-incremental learning store portion of the real images belonging to the old classes to avoid catastrophic forgetting. Rebuffi et al. (2017) (iCaRL) store old class images and utilize knowledge distillation (Hinton al.J (203)) for representation learning and the nearest class mean (NCM) classifier for classification of the old and new classes. Knowledge distillation uses a loss term force the labels of the images of previous classes to remain the same when learning new classes. Castro et al. (2018) (EEIL) improves iCaRL with an end-to-end learning approach. /Wu al. (2019) also stores real images and uses a bias correction layer to avoid any bias toward the new classes.
To avoid storing old class images, some approaches store features from the last fully- layer of the neural networks (Xiang et al. 2019, Hayes & Kanan 2020 Ayub & Wagner) 2020b d) These approaches, however, use network pretrained on ImageNet to extract features, which gives them an unfair advantage over other approaches. Because of their reliance pretrained network, these approaches cannot applied in situations when new data differs drastically from ImageNet (Russakovsky et al. 2015
These difficulties have forced researchers to consider using generative networks. Methods employ- ing generative networks tend to model previous class statistics and regenerate images belonging to the old classes while attempting to learn new classes. Both (Shin et al. (2017) and Wu et al. (201 Sa) use generative replay where the generator is trained and real images from the new classes. This approach, however, causes images belonging to classes learned in earlier increments to start semantically drift, i.e. the quality of images degrades be- cause of the repeated training on synthesized images. OStaaenko et al. (2019) avoids semantic drift by training the GAN only once on the data of each class. Catastrophic forgetting is avoided by ap- plying clastic weight consolidation (Kirkpatrick et 2017), in which changes in important weights needed for old classes are avoided when learning new classes. They also grow their network when it runs out of memory while learning new classes, which can be difficult to apply in situations with restricted memory. One major issue with GAN based approaches is that GANS tend to generate images that do not belong to any of the learned classes which decreases classification accuracy. For these reasons, most approaches only perform well on simpler datasets such as MNIST (LeChun) 1998) but perform poorly on complex datasets such as ImageNet. Conditional GAN can be used to mitigate the problem of images belonging to none of the classes as done by Ostapenko et al. (2019), however the performance is still poor complex datasets such ImageNet-50 (see Table and
mixture of generated old class images
2

---

Published as a conference paper at ICLR 2021
Table We avoid the problem of generating images that do not belong to any learned class by training autoencoders instead of GANs.
Comparatively little work has focused on using autoencoders generate samples because the images generated by autoencoders are blurry, limiting their usefulness for classifier training. Hatton (2014) uses autoencoders binary pixel images and Kemker & Kanan (2018) (FearNet) uses network pre-trained on ImageNet to extract feature embeddings for images, applying the autoencoder to the feature embeddings. Neither of these approaches are scalable RGB images. Moreover, the use of a pre-trained network to extract features gives FearNet an unfair advantage over other approaches.
3 ENCODING EPISODES AS CONCEPTS (EEC)
Following the notation of Chaudhry al. (2018), we consider S. {(x),, yf) to be the set of samples z; E X' and their ground truth labels y, belonging to task t. a class- incremennae setup, S, contain one or multiple classes and data for different tasks is available to the model in different increments. In each increment, the model is evaluated on all the classes seen so far.
Our formal presentation of continual learning follows Ostapenko et al. (019), where task solver model (classifier for class-incrent incremental learning) D has to update its parameters An on the data of task L in increment such that it performs equally well all the 1 previous tasks seen so far. Data for the tasks is not available when the model is learning task l. The subsections below present our approach.
3.1 AUTOENCODER TRAINING WITH NEURAL STYLE TRANSFER
An autoencoder is a neural network that is trained to compress and then reconstruct the input (Goodfellow et al. 2016), formally fr X + X. The network consists of an encoder that com- presses the input into a lower dimensional feature space (termed as the encoded episode in this paper), Jenc X' -> F and a decoder that reconstructs the input from the feature embedding, Idec F - X. Formally, for given input E X. the reconstruction pipeline fr is defined as: fr( = (gdeo (x)) The parameters 0, of the network are usually optimized using an l2 loss (C,) between the inputs and the reconstructions:
C. = -z=
(I)
Although autoencoders are suitable for dimensionality reduction for complex, high-dimensional data like RGB images. the reconstructed images lose the high frequency components necessary for correct classification. To tackle this problem, we train autoencoders using some of the ideas that underline Neural Style Transfer (NST). NST uses a pre-trained CNN to transfer the style of one image to another. The process takes three images, an input image, a content image and style image and alters the input image such that it has the content image's content and the artistic style of the style image. The three images are passed through the pre-trained CNN generating convolutional feature maps (usually from the last convolutional layer) and l2 distances between the feature maps of the input image and content image (content loss) and style image (style loss) are calculated. These losses are then used to update the input image.
Intuitively, our intent here is to create reconstructed images that are similar to the real images (in the pixel and convolutional space), thereby improving classification accuracy. Hence, we only utilize the idea of content transfer from the NST algorithm, where the input image the image reconstructed by the autoencoder and content image is the real image corresponding the reconstructed image. The classifier model, D, is used to generate convolutional feature maps for the NST, since it is already trained on real data for the classes in the increment algorithm, we use the content loss (Lcont) to train the autoencoder, rather than updating the input image directly. Formally, let fo X' F, be the classifier pipeline that converts input images into convolutional features. For an input image, of of task I, the content loss is:
contrast the traditional NST
Lcont (lf( (z(f(
The autoencoder parameters optimized using combination of reconstruction and content losses:
3

---

Published as a conference paper at ICLR 2021
Excode
Decader
Recoontcced
Caribits and Douatiecces
CRVTES CONMFARSN
(+]
Figure I: Complete architecture of EEC. (a) For each task, convolutional autocncoder is trained real images a combination of reconstruction loss and content Ceant. The encoded episodes are stored memory and converted into centroids and covariance matrices when the system runs out of memory. (h) The classifier trained on a combination of real, reconstructed and pscudo-images in each increment.
(a)
(b)
Figure 2: Reconstructed images generated by autocncoders trained (a) using NST and (b) not using NST.
C - + XL
where, À is a hyperparamter that controls the contribution each loss term towards the complete loss. During autoencoder training, classifier D acts as fixed feature extractor and its parameters are not updated. This portion of the complete procedure is depicted in Figure (a).
To provide illustration of our approach, we perform experiment with ImageNet-50 dataset. We trained one autoencoder on 10 classes from ImageNet-50 with NST and one without NST. Figure 2 depicts the reconstructed images by the two autoencoders. Note that the images generated by the autoencoder trained without using NST are blurry. In contrast, the autoencoder trained using NST creates images with fine-grained details which improves the classification accuracy.
3.2 MEMORY INTEGRATION
As mentioned in the introduction, continual learning also presents issues associated with the storage of data in memory. For EEC, for each new task f. the data is encoded and stored memory. Even though the encoded episodes require less memory than the real images, the system run out of memory when managing a continuous stream of incoming tasks. To cope with this issue, propose process inspired by memory integration in the hippocampus and the neocortex (Mack et al. (018). Memory integration combines a new episode with previously learned episode summarizing the information in both episodes in single representation. The original episodes themselves are forgotten.
still
Consider a system that store a total of K encoded episodes based its available memory. Assume that at increment t -- 1, the system has total of K,-1 encoded episodes stored. It is now required store K, more episodes in increment t. The system runs out of memory because K, + K,-1 K. Therefore, it must reduce the number of episodes to K, K,- + K; K. Because each task composed of a of classes at each increment, we reduce the total encoded episodes belonging to different classes based on their previous number of encoded episodes. Formally, the reduction the number encoded episodes N,, for a class y is calculated as (whole number):
K K
Ny(new) = N2
(4)
To reduce the encoded episodes to N2, (new) for class y, inspired by the memory integration process, use incremental clustering process that combines the closest encoded episodes to produce
4

---

Published as a conference paper at ICLR 2021
(6] No Centrolds
[3) 4000 Centrolca
[c] 2300 Centroida
(4) 1000 Cantrida
(e] 230 Centroide
Figure 3: Visualization of the encoded episodes (a) and pseudo-encoded episodes generated using different number centroids and covariance matrices (b-e). Note that the pseudo-eneoded episodes generated from a different number of centroids are all similar to the original encoded episodes. However, storing more centroids generates a feature space closer to the original feature space.
troids and covariance matrices. This clustering technique is similar to the Agg-Vat Var clustering pro- posed in our earlier work (Ayub & Wagner (2200 a). The distance between encoded episodes is cal- culated using the Huclidean distance, and the centroids are calculated using the weighted mean of the encoded episodes. The process is repeated until the sum of the total number of centroidsscoovarincce matrices and the encoded episodes for class equal N, (new). The original encoded episodes are removed and only the centroid and the covariances kept (see Appendix A for more details).
3.3 REHEARSAL., PSEUDOREHEARSAT AND CLASSIFIER TRAINING
Figure D (b) depicts the procedure for training the classifier D when new data belonging to task / becomes available. The classifier is trained on data from three sources: 1) the real data for task 4, 2) reconstructed images generated from the autoencoder's decoder, and 3) pseudo-imagee generated from centroids and covariance matrices for previous tasks. Source (2) uses the encodings from the previous tasks to generate set of reconstructed images by passing them through the autoencoder's decoder. This process is referred to as rehearsal (Robins) 1995).
Pseudorehearsal: If the system also has old class data stored centroids/covarianee matrix pairs, pseudorehearsal is employed. For each centroid/covariance matrix pair of a class we sample a multi- variate Gaussian distribution with mean the centroid and the covariance matrix generate large set of pseudo-encoded episodes. The episodes are then passed through the autoencoder's decoder generate escudo-imager for the previous classes. Many of the pscudo-images are noisy. To filter the noisy pscudo-images, we pass them through classifier D, which has already been trained on the prior classes, to get predicted labels for each pseudo-image. We only keep those pseudo-images that have the same predicted label as the label of the centroid they originated from. Among the fil- tered aseudo-images, we select the same number of geudo-images as the total number of encoded episodes represented by the centroid and discard the rest (see Appendix A for the algorithm).
To illustrate the effect of memory integration and pseudorehearsal, we performed an experiment on MNIST dataset. We trained an autoencoder on 2 classes (11379 images) from MNIST dataset with the embedding dimension of size 2. After training. we passed all the training images for the two classes through the encoder to get 11379 encoded episodes (see Figure (a)). Next, we applied our memory integration technique on the encoded episodes to reduce the encoded episodes to 4000, 2000, 1000 and 200 centroids (and covariances). No original encoded episodes were kept. Pseudore- hearsal was then applied on the centroids to generate pseudo-encoded episodes. The pseudo-encoded episodes for different numbers of centroids are shown in Figure (b-e). Note that the feature space for the pseudo-encoded episodes generated by different number of centroids is very similar to the original encoded episodes. For a larger number of centroids, the feature space looks almost exactly the same as the original feature space. For the smallest number of centroids (Figure the feature space becomes less uniform and more divided into small dense regions. This because smaller number of centroids are being used to represent the overall concept of the feature space across dif- ferent regions resulting in large gaps between the centroids. Hence, the pseudo-encoded episodes generated using the centroids are more dense around the centroids reducing uniformity. Still, the overall shape of the feature space is preserved after using the centroids and pseudorehearsal. Hence, approach conserves information about previous classes, with less memory, contributing to classifier performance while also avoiding catastrophic forgetting. The results presented Section 4.3 on IeageNet-50 confirm the effectiveness of memory integration and pseudorehearsal.
5

---

Published as a conference paper at ICLR 2021
Sample Decay Weight: The reconstructed images and pseudo-images still be quite different from the original images, hurting classifier performance. We therefore weigh the loss term for reconstructed and pseudo-images while training D. For this, estimate the degradation in the reconstructed and pseudo-images. To estimate degradation in the reconstructed images, we find the ratio of the classification accuracy of the reconstructed images (c") to the accuracy of the original images (c") on network D trained on previous tasks, This ratio is used to control the weight of the loss term for the reconstructed images. For a previous task 1, the weight 1 (the sample decay weight) for the loss term L of the reconstructed images is defined as:
T t-1 e 26
(sample decay coefficient) denotes the degradation in reconstructed images of task t- 1 and 0--1 represents the number of times an autoencoder has been trained on the pseudo- images or reconstructed images of task 1 - 1. The value of sample decay coefficient ranges from 0 to 1, depending on the classification accuracy of the reconstructed images c.-1- If ct-1 G-1 Tf- 1 = 0 (no degradation) and - = 1. Similarly, the sample decay weight r for loss term C for the pseudo-ing images is based on the classification accuracy e² of pseudo-image for task 1. Thus, for new increment, the total loss LD for training D on the reconstructed and pseudo-
where 7f-1 2
images of the old tasks and real images of the new task / is defined as:
CD= = C.+ Σ (T
(6)
1m1
EXPERIMENTS
We tested and compared EEC several SOTA approaches on four benchmark datasets: MNIST, SVHN, CIFAR-10 10 and ImageNet-500 We also report the memory used by our approach and its performance in restricted memory conditions. Finally, we present an ablation study to evaluate the contribution of different components of EEC. Experiments on CIFAR-10 100 and comparison with additional methods are presented in Appendix Other generative memory approaches by previous authors did not test on the CIFAR- 100 dataset.
4.1 DATASETS
The MNIST dataset consists of grey-scale images of handwritten digits between 0 to 9, with 50,000 training images, 10.000 validation images and 10,000 test images. SVHN consists of colored cropped images of street house numbers with different illuminations and viewpoints. It contains 73,257 training and 26,032 test images belonging to 10 classes. CIFAR-10 consists of 50,000 RGB training images and 10,000 test images belonging to 10 object classes. Each class contains 5000 training and 1000 test images. ImageNet-50 taining 50 classes with 1300 training images and 50 validation images per class. All of the dataset images were resized to 32 x2, in concordance to (Ostapenko et al. 2019).
smaller subset of the iLSVRC-2012 dataset con-
4.2 IMPLEMENTAION DETAILS
We used Pytorch (Paszke et al. 2019) and an Nvidia Titan RTX GPU for implementation and train- ing of all neural network models. A 3-layer shallow convolutional autoencoder was used for all datasets (see Appendix B). which requires approximately 0.2MB of disk space for storage. For classification, the MNIST and SVHN datasets the same classifier the DCGAN discriminator (Radford et al. 2015) was used, for CIFAR- 10 the ResNet architecture proposed by Gulrajani et al. 2017) used and for ImageNet-50, ResNet-18 (He et al. 2016) was used.
In concordance with Ostapenko et al. (2019) (DGMw), we report the average incremental accuracy on and 10 classes (Ag and Aςo) for the MNIST, SVHN and CIFAR- 10 datasets trained contin- ually with one class per increment. For ImageNet-50. the average incremental accuracy on 3 and 5 increments (Ago and Asa) is reported with 10 classes in each increment. For fair comparison,

---

Published as a conference paper at ICLR 2021
MNIST
(%)
SVHN (%)
CIFAR-10 (%)
ImageNet-5h (%)
Methods
As
A10
As
A1o
As
Aιg
A3g
Aso
JT
99.87
99.24
95.56
94.14
85.50
77.82
57.35
49.88
iCaRL-S (Rebufli
2017)
84.61
55,80
57.30
43.69
29.38
28.98
RWalk-S (C Chaudhry et a.
2018)
82.50
FRIL-S Castro et al.
2018)
27.87
11.80
EWC-M (Seff'
2017)
70.62
77.03
39.84
33.02
DGR Shin et al. 2017)
90.39
85.40
61.29
47.28
MeRGAN [W et al.
2018a)
99.15
96.83
80.90
66.78
DGMw (Ostapenko ct al.
2019)
98.75
96.46
83.93
74.38
64.94
56,21
32.14
17.82
EEC (Ours)
99.20
97.83
95.29
89.59
85.12
66.91
45.39
35.24
Difference
+0.05
+1.00
+11.3
+15.2
+20.2
+10.7
+13.2
+6.26
EECS (Ours)
98.00
96.26
94.30
81.12
82.20
61.91
41.13
30.89
Table I: Comparison of EEC and EECS to approaches that store and use real samples (denoted with an S) and those that generate samples, for class-incremental learning on MNIST, SVHN, CIFAR- 10 and ImageNet-50 datasets. The difference between EEC and the SOTA approach is shown in bold.
we typically compare against approaches with generative memory replay component that do not use pre-trained feature extractor and are evaluated in a single-headed fashion. Among such ap- proaches, to the best of our knowledge, DGMw represents the sfate-of-the-art benchmark on these datasets which is followed by MeRGAN (Wu et al., 2018a), DGR (Shin et al., 2017) and EWC-M (Seff et al.) 2017). Joint training (JT) is used to produce compare these methods against two variants of our approach: EEC and EECS. EEC uses a separate autoencoder for classes in new increment, while EECS uses single autoencoder that is retrained on the reconstructed images of the old classes when learning new classes. For both EEC and EECS, results are reported when all of the encoded episodes for the previous classes are stored. The results for EEC under restricted memory conditions are also presented.
upperbound for all four datasets. We
We performed each experi- ment 10 times with different random seeds and report average accuracy over all runs.
Hyperparameter values and training details reported in Appendix C
4.3 COMPARISON WITH SOTA METHODS
Table I compares EEC and EECS against SOTA approaches on the MNIST, SVHN, CIFAR- 10 and ImageNet-50 datasets. We compare against two different types of approaches, those that use real images (episodic memory) of the old classes and those that generate previous class images when learning new classes. Both EEC and EECS outperform EWC-M and DGR by significant margins on the MNIST on As and Anu- MeRGAN and DGMw perform similarly to our methods on the As and Auo experiments. Note that MeRGAN and EEC approach the JT upperbound on As. Further, accuracy for MeRGAN, DGMw and EEC changes only slightly between As and A1o. suggesting that MNIST is too simple of dataset for testing continual learning using generative replay.
Considering SVHN, a more complex dataset consisting of colored images of street house numbers, the performance of our method remains reasonably close to the JT upperbound, even though the performance of other approaches decreases significantly. For As. EEC achieves only 0.27 % lower accuracy than JT and 11.36% % higher than DGMw (current best method). For A EEC is 4.55% lower than JT but still 15.21% % higher than DGMw and achieves 5.66% higher accuracy on A10 than DGMw did on As. EECS performs slightly lower than EEC on As but the gap widens on A10- However, EECS also beats the SOTA approaches on both As and A1o.
Considering the more complex CIFAR-I0 and ImageNet-50 datasets, only DGMw reported results for these datasets. On CIFAR- 10, EEC beats DGMw on As by a margin of 20.18 % and on Aie by a margin of 10.7%. Similar to the SVHN results, the accuracy achieved by EEC on A even higher than DGMw's accuracy on As. In comparison with the JT, EEC performs similarly on As (0.38% lower), however on A1 it performs significantly lower. For ImageNet-50, again we see similar results as both EEC and EECS outperform DGMw on Ago and Aso by significant margins (13.29% % and 17.42% % respectively). Similar to SVHN and CIFAR-10 10 results, accuracy for EEC on Aso is even higher than DGMw's accuracy on Ago. Further, EEC also beats iCaRL. (episodic
7

---

Published as a conference paper at ICLR 2021
Figure 4: Reconstructed Images for CIFAR-10 (top) and ImageNet-50 (bottom) after all tasks
memory SOTA method) with margins of 16.01 % and 6.26% A3o and Aso, respectively, even though iCaRL has an unfair advantage of using stored real images.
Discussion: Our method performed consistently across all four datasets, especially As for MNIST, SVHN and CIFAR-I0. In contrast, DGMw (the best current method) shows signifi- cantly different results across the four datasets. The results suggest that the current generative memory-based SOTA approaches are unable to mitigate catastrophic forgetting on more complex RGB datasets. This could be because GANs tend generate images that do not belong to any of learned classes, which can drastically reduce classifier performance. Our approach copes with these issues by training autoencoders borrowing ideas from the NST algorithm and retraining of the clas- sifier with sample decay weights. Images reconstructed by EEC for CIFAR- 10 after 10 tasks and for ImageNet-50 after 5 tasks are shown in Figure (more examples in supplementary file).
Memory Usage Analysis: Similar to DGMw, we analyze the disc space required by our model for the ImageNet-50 dataset. For EEC, the autoencoders use a total disc space of MB, ResNet- 18 uses about 44 MB, while the encoded episodes use total of about 66.56 MB. Hence, the total disc space required by EEC is about 1 111.56 MB. DGMw's 's generator (with corresponding weight masks), however, uses 228MB of disc space and storing pre-processed real images of ImageNet-Sii requires disc space of 315MB. Hence, our model requires 51.07 % ((228-1 (228-111,56)/22 = 0.5107) less space than DGMw and 64.58% less space than the real images for ImageNet-50 dataset.
EEC was tested with different memory budgets on the ImageNet-50 dataset to evaluate the impact of memory management technique. The memory budgets (K) defined the sum of the total number of encoded episodes, centroids and covariance matrices (diagonal entries) stored by the system. Figure shows the results in terms of A30 and A50 for wide range of memory budgets. The accuracy of EEC changes only slightly over different memory budgets. Even with a low budget of K=5000, the A3a and A5o accuracies are only 3.1 and 3.73% lower than the accuracy of EEC with unlimited memory. Further, even for K=5000, EEC beats DGMw (current SOTA on ImageNet- 50) by margins of 10.17% % and 13.73% on A30 and Aso. respectively. The total disc space required for K=5000 is only 5.12 MB and the total disc space for the complete system is 50.12 MB (44 MB for ResNet- 18 and 1 MB for autoencoders), which is 78.01 % less than DGMw's 's required disc space (228 MB). These results clearly depict that approach produces the best results even with extremely limited memory, a trait that is not shared by other SOTA approaches. Moreover, the results also show that our approach is capable of dealing with the two main challenges of continual learning mentioned earlier: catastrophic forgetting and memory management.
4.4 ABLATION STUDY
We performed an ablation study to examine the contribution of using: 1) the content loss while training the autoencoders, 2) pseudo-rehearsal and 3) the sample weight decay. These experiments were performed on the ImageNet-50 dataset. Complete encoded episodes of previous tasks were used for ablations and 3, and a memory budget of K=5000 was used for ablation 2. We created hybrid versions of EEC to test the contribution of each component. Hybrid 1: termed as EEC-noNST
does not use the content loss while training the autoencoders. Hybrid 2: termed as EEC-CGAN uses conditional GAN instead of NST based autoencoder. Hybrid 3: termed as BEC-VAE uses variational autoencoder and Hybrid 4: termed as EEC-DNA uses a denoising autoencoder instead of proposed NST based autonecoder. Hybrid 5: termed as EEC-noPseudo simply removes the extra encoded episodes when the system runs out of memory and does not use -eeuurseeeaaral... Hybrid 6: termed as EEC-noDecay does sample weight decay during classifier training on new and
8

---

Published as a conference paper at ICLR 2021
Methods
A30
A
BEC-noNST
39.51
28.73
EEC-CGAN
39.94
29.12
GEC-VAE
39.14
27.63
BEC-DNA
39.42
28.36
noCany
41.62
30.47
EEC
45.39
35,24
EEC-noPseudo
37.31
25.24
10:30
20030
8:003
40:02
3000
EEC (K=5000)
42.29
31.51
iderooy BLoges
Figure 5: The accuracy of EEC Aso and Aso ImageNet-50 dataset with different memory budgets.
Table 2: Ablation study results in terms of Aso and Ase for ImageNet-50 dataset.
old tasks. Except for the changed component, all the other components in the hybrid approaches were the same as used in EEC.
All of the hybrids show inferior performance compared to the complete EEC approach (Table 2). Evaluated on A30 and Aso. EEC-noNST, EEC-CGAN, EEC-VAE and EEC-DNA all result
~5.8% and ~6.5% lower accuracy, respectively than EEC. EEC-noDecay results in 3. 3.77% and 4.77% lower accuracy on Agu and Aso respectively, than EEC, As a fair comparison with EEC using
K=5000, EEC-noPseudo results in 4.98% and 6.27% lower accuracy for A30 and Aso- respectively. The results show that all of these components contribute significantly to the overall performance of be more
EEC, however training the autoencoders with content loss and pseudo-rehearsal important. Also, note that the conditional GAN with the other components of our approach (such as sample weight decay) produces 7.80% and 11.3% higher accuracy on A30 and A50. respectively, than DGMw. Further note that all the hybrids that do not use the NST based autoencoder (Hybrids 1-4) produce similar accuracy, since they use the sample weight decay to help mange the degradation of reconstructed images. These results show the effectiveness of sample weight decay to cope with image degradation.
4.5 EXPERIMENTS ON HIGHER RESOLUTION IMAGES
As mentioned in Subsection 4.2 all the experiments the four datasets were done using resized images of size 32x32. To show further insight into our approach, we performed experiments on ImageNet-50 dataset with higher resolution images of size x56 and randomly cropped to 224x224. Evaluated on A3o and Asa- EEC achieved 70.75% % and 56.89 accuracy, respectively using images of size 224x224. These results show that using higher resolution images improve the performance of EEC by significant margins (25.36% and 21.65% increase on A30 and A50). Note that storing real high resolution images requires much larger disk space than storing images of size 32x32. For ImageNet-50. storing original images of size 224 requires 19.13GB, while images of size 32x32 require only 315MB. In contrast, EEC requires only 13.04 GB (66% less than real images of size 224x224) when using the same autoencoder architecture that was used for images of size 32x32. Using different architecture, we can bring the size of encoded episodes to be the same for images of size 32x32 but the accuracy achieved is lower. These results further show effectiveness our approach to mitigate catastrophic forgetting while using significantly less memory compared to real images.
5 CONCLUSION
This paper has presented novel and potentially powerful approach (EEC) to strict class-incrementaa learning. Our paper demonstrates that the generation of high quality reconstructed data can serve as the basis for improved classification during continual learning. We further demonstrate techniques for dealing with image degradation during classifier training on new tasks as well as a cognitively- inspired clustering approach that can be used to manage the memory. Our experimental results demonstrate that these techniques mitigate the effects of catastrophic forgetting, especially on com- plex RGB datasets, while also using less memory than SOTA approaches. Future continual learning approaches can incorporate different components of our approach such as the NST-based autoen- coder, pseudo-rehearse and sample decay weights for improved performance.

---

Published as a conference paper at ICLR 2021
REFERENCES
Ali Ayub and Alan R. Wagner. Centroid based concept learning for rgb-d indoor scene classification.
In British Machine Vision Conference (BMVC), 2020a.
Ali Ayub and Alan R. Wagner. Cognitively-inspired model for incremental learning using a few examples. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020b.
Ali Ayub and Alan R. Wagner. Storing encoded episodes as concepts for continual learning. arXiv:2007.06637 2020c.
Ali Ayub and Alan R. Wagner. Tell me what this is: Few-shot incremental object learning by robot. arXiv:2008. 00819, 2020d.
Francisco M. Castro. Manuel J. Marin-Jimenez, Nicolas Guil, Cordelia Schmid, and Karteek Ala- hari. End-to-end incremental learning. In The European Conference on Computer Vision (ECCV), pp. 248, September 2018
Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In The European Conference on Computer Vision (ECCV), pp. 532-547, September 2018.
Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 2414-2423, June 2016.
Ross Girshick. Fast R-CNN. In The IEEE International Conference Computer Vision (ICCV), pp. 1440-1448, December 2015.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. In Advances in Neural Information Processing Systems 30, pp. 5767-5777, 2017.
Motonobu Hattori. biologically inspired dual-network memory model for reduction of catas- trophic forgetting. Neurocomput., 134:262-268, June 2014. ISSN 0925-2312.
Tyler L. Hayes and Christopher Kanan. Lifelong machine learning with deep streaming linear dis- criminant analysis. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770- 778, June 2016.
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop, 2015.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified clas- sifier incrementally via rebalancing. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning. In International Conference Learning Representations, 2018. URL UtRlhtttps://opereviieee
Rb
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Baasskaa-Brwissa,,, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic for- getting in neural networks. Proceedings of the National Academy of Sciences of the United States of America, 11((13)3521-355 2017.
10

---

Published as a conference paper at ICLR 2021
Y. LeChum. The mnist database of handwritten digits, 1998. URL http::/yannnn.... http://yannn.c.ccm com/ exdb/mnist/
Michael L. Mack, Bradley C. Love, and Alison R. Preston. Building concepts one episode at a time:
The hippocampus and concept formation. Neuroscience Letters, 680:31-38, 2018.
Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. Learning to re- member: A synaptic plasticity driven framework for continual learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11321-11322 June 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vito,o Martin Raison, Alykhan Tejani, Sasank Chilamkurthy. Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR, 111.06444,, 2015.
Aiivestre-Alisse Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. iCaRL: Incremental classifier and representation learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2001-2010, July 2017.
Louis Renoult, Patrick S. R. Davidson, Erika Schmitz, Lillian Park, Kenneth Camphell, Morris Moscovitch, and Brian Levine. Autobiographically significant concepts: More episodic than se- mantic nature? an electrophyysological investigation of overlapping types of memory. Journal of Cognitive Neuroscience, 27(1):57-72, 2015.
Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):
123-146, 1995.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge. Int. J. Comput. Vision, 11(3))211-252, December 2015. ISSN 0920-5691.
Ari Seff, Alex Beatson, Daniel Suo, and Han Liu. Continual learning in generative adversarial nets. ArXiv, abss/1705.0835, 2017.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems 30, pp. 2990-2999, 2017.
Chenshen Wu, Luis Herranz, Xialei Liu, yaxing wang, Joost van de Weijer, and Bogdan Raducanu. Memory replay gans: Learning generate new categories without forgetting. In Advances in Neural Information Processing Systems 31, pp. 5962-5972, 2018a
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, Zhengyou Zhang, and Yun Fu. Incremental classifier learning with generative adversarial networks. CORR, abs/ hss/1802.00853 2018b.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Y. Xiang, Y. Fu, Ji, and H. Huang. Incremental learning using conditional adversarial networks. In 2019 JEEE/CVF International Conference on Computer Vision (ICCV), pp. 661 6618-6627, 2019.
11

---

Published as a conference paper at ICLR 2021
A EEC ALGORITHMS
The algorithms below describe portions the complete EEC algorithm. Algorithm 1 is for au- toencoder training (Section 3.1 in paper), Algorithm 2 is for memory integration (Section 3.2 in paper), Algorithm is for rehearsal, pseudo-rchearsal and classifier training (Section 3.3 in paper) and Algorithm 4 for filtering aseudo-images (Section 3.3 in paper).
Algorithm 1: EEC: Train Autoencoder
Input: S. ((r). y/
data points with ground truths for task
require: A
require: fr = (gdec o Jenc)
D autoencoder pipeline convolutional feature extractor from classifier D D. Number of epochs D encoded episodes with ground truths for task /
require: fc : X' 7 Fe
require: Napoch
Output: F. = {(S, )
1: for 1:j N Napoch do
2:
X = (x()))
D get reconstructed images for task / Reconstruction loss
3:
L, ||x X2
4:
Lcont = (1fe(X)) fo(X))l2
5:
Content loss complete autoencoder loss update autoencoder parameters such that minimizes L get encoded episodes for images of task t
6:
(1 + Xooont + minimize(L)
7:
8:
Fi (Xi)
Algorithm 2: EEC: Memory Integration
Input: [F:)-]] where F; = {(S) 1
encoded episodes set for previous t tasks maximum number of episodes number of encoded episodes for task t reduced encoded episodes for previous tasks centroids for previous tasks D covariances for previous tasks & number of encoded episodes for previous tasks
require: K
require: Kg
Output: F(new)
Output: C [C
Output:
Σ (Ei]/=)
Initialize: K,-1 = Di=1 n
1: K, = K; + K, K
2: for = 1; <
do
3: for j 1;j & y do
4: repeat
D for each class task i find centroids and covariances Oj + combine x Voi y jD Combine closest points for class j in task
5:
(yew)) = (cent) + nn -i= (c(nnt)): number centroids for class
6:
7:
until Ny(new) N2 K.
12

---

Published as a conference paper at ICLR 2021
Algorithm 3: EEC: Train Classifier
Input: S, {(1] v D
data points with ground truths for task t encoded episodes for previous tasks centroids for previous tasks D> covariances for previous tasks no. of episodes clustered in each centroid
Input: F {Fi) where F, {[f]. Vj )}
Input: {C:) where C, = {(c), }})]}=
N (cene)
Input: Σ = [2 where M; = =((). vブ) 1 (cent)
Input: M (M, | where M₁ {m]) aceuracy of original training images of previous t tasks > Classifier model > decoder part of autoencoders for each of the previous tasks b Number of epochs
1==
N (ceen))
require: cf-1
require: D
require: (geec 11-1
require: Napoch
1: for = 1; < do
2: for 1:j No (cenl) do appeend wwwp:eemmmmlllltrrrrrtoooooaaeeo o)))
3: s
>> generate pseudo-samples for
each concept
4: X, (large)= = gaee (F, (lurge))> generate large no. pseudo-images from pseudo-samples > Algorithm get reconstructed images for encoded episodes
5:
X, = FILTTER(X (large), M.)
6:
X; (1.c
7:
for = 1;
8:
Y = =(X;))
predictions for reconstructed images for task predictions for pseudo-images for task
9:
Y D(X))
T D 100 (-==0)) n
accuracy for reconstructed images of task
Σn' -x=0)==X0
11:
accuracy for pseudo images of task i
12:
N 1
A sample decay coefficient for reconstructed images of task i > sample decay coefficient for pseudo images of task i
13:
Y = 1
14:
N = e 7 o
15:
I Ya:
for
1;j N Nepoch do
17:
for = 1 11- do
Y ((;;)
predictions for reconstructed images for task
19:
Y D(X))
D predictions for pseudo-images for task i
20:
C Crosspnt Y,) Y))
loss for reconstructed images of task i
21:
C CrossFnnropp
loss for psuedo-images of task i
22:
Y = D(X.)
predictions for data points for task t > loss for task /
23:
L. Cross CrossEntropyl Entropy YY)
LD L. + E C +
25:
OD + minimize(LD)
update classifier parameters such that it minimizes Lp
Algorithm EFC: Filter Pseudo-images
Input: S,arge)) = {a 1
(large) where Xę(large) { 1n
Yllagge) = {v) n (large)
Input: Mr = {m
(cent)
require: D)
Output: =. ((x), 32
b filtered pseudo- for task
(large) D(X,
2: X; = {2. xlarge)) v = vt 12 (large)
-aaudo-imaaes with correct predicted label
X, {af M
keep M, pseudo-images
13

---

Published as a conference paper at ICLR 2021
B AUTOENCODER ARCHITECTURES
Name
Layer Type
Filter Size
Stride 2
Output Shape
conv I
Conv2D
3x3
(None.64,16..))
batch
BatchNorm2d
(None, 62,,6,,,666 16,166
relul
ReLU
NNone,64,16,166
conv2
Conv2D
3x3
(None,32,)
batch morc_ relu2
BatchNorm2d
INone.32,8,8
Encoder
ReLU
(None.32,8,8)
conv3
Conv2D
3x3
INone,16,444
batch norm3
BatchNorm22
(None,16,,,4)
relu3 conv_trans3
ReLU
(None, 16,4,4)
ConvTransposee]
3x3
2
(Nome.32,8,8)
batch norm3 relu3 conv trans2 batch
BatchNorm2d
(None,32,8,8)
ReLU
(None,32,5)
TrnyTraaspose2D
3x3
(6,16))
BatchNorm22
(None,64, 16.16.
Decoder
relu2_ conv_transl batch_norm1. relul_
ReLU
16.1164,16.16)
ConyTranspose2D
3x3
(None,1,32,32)
BatchNorm2d
(N3nc,1,32,32) 1,32,32)
ReLU
1,3,,,))2,322
Table 3: Convolutional Autoencoder Architecture for MNIST, SVHN, CIFAR-10 and ImageNet-50
datasets
C HYPERPARAMETERS
Hyperparameters
Values
No. of epochs
100
Starting Learning Rate (LR)
0.001
LR Decay Milestones
{50)
LR Decay Factor
1/10
miniBatch Size
Optimizer
Adam
Weight Decay 0.0005
0.7
Table 4: Hyper-parameters for EEC autoencoder training
Hyperparameters
Values
No. epochs (Ist increment)
200
No. of epochs (next increments) Starting Learning Rate (LR)
45
0.1
LR Decay Milestones
{60,120,160)
LR Decay Factor
1/5
miniBatch Size
128
Optimizer
SGD
Momentum
0.9
Weight Decay
0,0005
Table 5:
for EEC classifier training
14

---

Published as a conference paper at ICLR 2021
Methods
Accuracy
iCaRL (Rebuffi et al.
2017)
59.5
EEIL
Castro al. 2018)
64.0
BiC (Wu et al. 2019)
64.3
LUCIR (Hou et al. 2019
63.4
EEC (Ours)
65.6
FearNet (Kemker Kanan 2018
66.2
CAN CANTXXing et al.]
67.1
EEC-P (Ours)
76.7
Table 6: Comparison of EEC against episodic memory approaches and approaches that use a pre- trained CNN in terms of average incremental accuracy (%) with 10 classes per increment. EEC-P uses pre-trained CNN for feature extraction.
D EXPERIMENTS ON CIFAR-100
For a fair comparison, the main set of approaches that we compared to are generative memory approaches. These approaches were only tested the four datasets (Section 4 However, many episodic memory approaches and approaches that use pre-trained CNN features did not present results the four datasets in Section 4 Hence, test approach (EEC) CIFAR-100 to compare against more episodic memory approaches and approaches that use a pre-trained CNN as feature extractor.
CIFAR-100 consists of 60,000 32 x 32 images belonging to 100 object classes There are 500 training images and 100 test images for each class. We divided the dataset into 10 batches with 10 classes per batch. During training, in each increment we provided the algorithm a new batch of 10 classes. We report the average incremental accuracy over the 10 increments as the evaluation metric. For evaluation against approaches that use pre-trained network, we first extracted features from a pre-trained ResNet-34 on ImageNet and then applied EEC on the feature vectors instead of raw RGB images. This version of EEC is denoted as EEC-P. The hyperparmeter values and training details for this experiment are presented in Appendix C We performed this experiment 10 times with different random seeds and report the average accuracy over all the runs.
Among the episodic memory approaches, we compare against iCaRL (Rebuffi et al. 2017). EEIL (Castro et al. 2018), BiC (Wu et al., 2019) and LUCIR (Hou et al. 2019). Among the approaches that use a trained CNN, we compare against FearNet (Kemker & Kanan 2018) and the method proposed by Xiang et al. (2019) (CAN). These approaches have been introduced in Section 2
Table 6 shows the comparison of EEC against all the above mentioned approaches. EEC beats all the late-of-ttheare (SOTA) episodic memory approaches by a significant margin, even though these approaches have an unfair advantage of using stored real images the old classes. Although the difference between EEC and other approaches (1.3%) is lower than the difference on the other four datasets. EEC-P also beats the SOTA approaches that use a pre-trained CNN by much greater mar- gin (9.6%) %). These results further show the efffectiveness of our approach for mitigating catastrophic forgetting.
E COMPARISON WITH FEARNET
FearNet (emnker & Kanan 2018 is one of the few approaches that uses some ideas similar to EEC. In particular, FearNet uses autoencoders reconstruct old data and stores single centroid and covariance matrix for each of the old classes. FearNet also uses pseudorehearsal on the centroids and covariances of old classes to regenerate pseudo-encoded episodes of the old classes, which is the same approach. However, unlike EEC, FearNet uses pre- treitrained feature extractor which is limitation. Further, for pscudorchearsal. FearNet only stores a single centroid and covariance matrix for each of the old classes, which is not enough capture the entire feature space of the classes. In contrast, EEC uses memory integration based clustering technique to store multiple centroids and covariance matrices per class to better capture the overall feature space of the old classes.
To illustrate the difference between the two approaches, we performed the same experiment on MNIST dataset in Section 3.3 For FearNet, we allowed the model store single centroid (and
15

---

Published as a conference paper at ICLR 2021
(a) No Centroids
(b) 200 Centroids
(c) Centroids
Figure 6: Visualization encoded episodes (a), pseudo-encoded episodes generated using 200 centroids
and covariance matrices (b) and pseudo-encoded episodes generated using 2 centroids and covariance matrices (c).
Methods
A.3
As0
Memory (MB)
JT
57.38
49.88
315
EEC (r 0.25)
54.85
45.92
161.48
EEC (r = 0.2)
54.04
45.14
151,496
EEC (r = -0.15)
53.38
44.26
141.412
ΣЕC (r 0.11
51.67
42.94
131.53
EEC (r
45.39
35.24
111.56
Table 7: Performance of EEC for different values
covariance matrix) for each of the two classes and then used pseudorehearsal to generate pseudo- encoded episodes. Sigure[Jshows the comparison between the original feature space (Figure 0 (a)), pseudo-encoded episodes generated with 200 centroids using memory integration (Figure (b)) and pseudo-encoded episodes generated using single centroid per class (2 centroids) as in FearNet (c) The feature space generated by FearNet is almost circular and does not resemble original feature space. Also, there is overlap between the feature spaces of the two classes which will lead to similar images for the two classes which will eventually hurt classifier performance. In contrast, the pseudo-encoded episodes generated by memory integration capture the overall concept of the original feature space without any overlap between the feature spaces of the two classes. Results in Table CIFAR-101 dataset also confirm that EEC is significantly superior to FearNet in terms of the average incremental accuracy (10.5% % improvement in accuracy).
F EXPERIMENT ON IMAGENET-50 WITH ORIGINAL AND RECONSTRUCTED IMAGES
In this experiment, we allow EEC partially store original images and the as encoded episodes. We define a ratio of original images and encoded episodes as r no/N, where no is the number of original images stored per class and N is the total number of images (original and reconstructed) per class. This experiment was performed with images of size x2x 32. Table 7 shows the accuracy for EEC in terms of Ago and Aso for different values of r 0 shows EEC with no original images and only the encoded episodes. By increasing to 0.1, we see dramatic increase in accuracy, however the increase in memory minimal (131.53 MB for = 0.1 as compared to 11 1..56 MB for 0). As the value of continues to increase. EEC's accuracy gets closer and closer to the JT upperbound.
16


Reviews:
[{"review_id": "stv-HRT9-s", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\n\n \nThe paper proposes an approach which trains autoencoders with Neural Style Transfer to encode and store images. The method is applied to the problem of continual learning to overcome the catastrophic forgetting and memory limitation on the storage data. The authors report that the presented approach increases the classification accuracy by 13-17% over SOTA methods \n\n##########################################################################\n\nReasons for score: \n\n \nOverall, I vote for accepting. The paper presents nice ideas, outperforms SOTA methods and is clearly written. The contribution could have been stronger with a more detailed evaluation and better presentation.\n \n##########################################################################\n\nPros: \n\n \n1. I really like the idea of using NST to train the autoencoder. \n\n2. Clarity of the paper.\n \n \n##########################################################################\n\nCons: \n\n1. The system uses a pretraied Style Transfer Network with Imagenet. Does it offer an unfair advantage over other approaches?\n2. It would be great to compare the NST Autoencoder with Variational or Denoising Autoencoder. \n3. The validation has been performed using low-resolution images (32x32)\n4. The improvement on Imagnet-50 A5 against iCaRL-S is 6.28. So, the difference should be 6.28 and not 17.4. \n5. It would be great to see the results for A30 and A50 with imagenet as it was done by (Ostapenko et.al. 2019)\n6. Figure 4 shows the accuracy on ImageNet-50 with different budgets. It seems that the accuracy is still increasing, did you tried with larger values?\n\nMinor points: \n- Table 1 - it would be great to include the reference paper of each method\n\n\n \n##########################################################################\n\nQuestions during rebuttal period: \n\n \nPlease address and clarify the cons above \n\n \n#########################################################################\n\nSome typos: \n\n(1) Page 5: corariance -> covariance\n(2) Page 7: the the accuracy --> the accuracy\n\n\nUpdate after rebuttal:\nMy initial concerns were clarified by the authors and the paper substantially improved. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "pzU1PTuPOvX", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: \n\nThe paper tackles catastrophic forgetting for continual learning. It proposes to train autoencoders with Neural Style Transfer to generate previous images. The encoded episodics can be converted into centroids and covariances matrices in order to save memory usage. It shows significant improvements in the experimental parts.\n\nStrengths:\n\n- The results on tiny datasets (MNIST, SVHN and CIFAR-10) are promising. On Tiny ImageNet-50, the performance compared to other methods looks good, but it is relatively bad compared to some other datasets.\n- It is well written and easy to follow.\n\n\nConcerns:\n\n- The overall idea is very close to what FearNet proposed by using autoencoders. The main difference is to use autoencoders on images instead of features.  Neural Style Transfer is helpful to generate more realistic images, therefore it makes sense to help during incremental learning to replay previous knowledge. However, it seems obvious improving image quality will result in better performance for methods based on replay.\n\n- From the experimental results compared to other methods, it is still limited to relative small resolution images and a few tasks. It would be interesting to see the performance on more challenging datasets with large resolution and more tasks.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "LdmjNlmeCPm", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In continual learning settings, one of the important technique for avoiding catastrophe forgetting is to replay data points from the past. For memory efficiency purposes, representative samples can be generated from a generative model, such as GANs, rather than replaying the original samples which can be large in number. It is argued that GANs generate new samples which may not belong exactly to one of the classes, so a new generative model is proposed. Experimental results are appealing.\n\nI have some basic concerns. \n(1) First of all, the idea of generating new samples for replay is not motivated well enough; in the experiments, even the model replaying on original images takes memory in few hundred MBs. \n(2) Second, why is there a need for generating new samples? Why not select a representative subset of the original set of samples. There are tons of methods to select samples informatively, within the paradigm of active learning and beyond.\n(3) Why not use conditional-GANs for generating data points specific to a class?\n\nSince autoencoders have a problem of generating blurry images, the proposed autoencoding model borrows ideas from neural style transfer algorithm. Specifically, besides the reconstruction loss, content loss is introduced utilizing the idea of content transfer from the neural style transfer algorithm. It seem to make sense in reference to Fig. 2, though it needs better explanation.   \n\nFor memory efficiency of the autoencoder itself, it is stores centroids and covariances of the episodes (representations of images), from which pseudo-encoded episodes are generated. This doesn't seem very principled, and may or may not work in different empirical settings, depending upon the intrinsic dimensionality of images, and the difficulty of the task. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper uses an autoencoder with neural style transfer to generate images from previously seen classes to avoid catastrophic forgetting in continual learning.

While reviewers had some concerns about the paper (experiments on high-resolution images, comparison with FearNet), authors have addressed all the concerns. R1's concern about the motivation for generation instead of replaying actual images is not necessary since this is not the first work to use generative replay.
Decision: Accept (Poster)