EEC: Learning to Encode and Regenerate Images for Continual Learning

1 INTRODUCTION . Humans continue to learn new concepts over their lifetime without the need to relearn most previous concepts . Modern machine learning systems , however , require the complete training data to be available at one time ( batch learning ) ( Girshick , 2015 ) . In this paper , we consider the problem of continual learning from the class-incremental perspective . Class-incremental systems are required to learn from a stream of data belonging to different classes and are evaluated in a single-headed evaluation ( Chaudhry et al. , 2018 ) . In single-headed evaluation , the model is evaluated on all classes observed so far without any information indicating which class is being observed . Creating highly accurate class-incremental learning systems is a challenging problem . One simple way to create a class-incremental learner is by training the model on the data of the new classes , without revisiting the old classes . However , this causes the model to forget the previously learned classes and the overall classification accuracy decreases , a phenomenon known as catastrophic forgetting ( Kirkpatrick et al. , 2017 ) . Most existing class-incremental learning methods avoid this problem by storing a portion of the training samples from the earlier learned classes and retraining the model ( often a neural network ) on a mixture of the stored data and new data containing new classes ( Rebuffi et al. , 2017 ; Hou et al. , 2019 ) . Storing real samples of the previous classes , however , leads to several issues . First , as pointed out by Wu et al . ( 2018b ) , storing real samples exhausts memory capacity and limits performance for real-world applications . Second , storing real samples introduces privacy and security issues ( Wu et al. , 2018b ) . Third , storing real samples is not biologically inspired , i.e . humans do not need to relearn previously known classes . This paper explores the ” strict ” class-incremental learning problem in which the model is not allowed to store any real samples of the previously learned classes . The strict class-incremental learning problem is more akin to realistic learning scenarios such as a home service robot that must learn continually with limited on-board memory . This problem has been previously addressed using generative models such as autoencoders ( Kemker & Kanan , 2018 ) or Generative Adversarial Networks ( GANs ) ( Ostapenko et al. , 2019 ) . Most approaches for strict class-incremental learning 1A preliminary version of this work was presented at ICML 2020 Workshop on Lifelong Machine Learning ( Ayub & Wagner , 2020c ) . use GANs to generate samples reflecting old class data , because GANs generate sharp , fine-grained images ( Ostapenko et al. , 2019 ) . The downside of GANs , however , is that they tend to generate images which do not belong to any of the learned classes , hurting classification performance . Autoencoders , on the other hand , always generate images that relate to the learned classes , but tend to produce blurry images that are also not good for classification . To cope with these issues , we propose a novel , cognitively-inspired approach termed Encoding Episodes as Concepts ( EEC ) for continual learning , which utilizes convolutional autoencoders to generate previously learned class data . Inspired by models of the hippocampus ( Renoult et al. , 2015 ) , we use autoencoders to create compressed embeddings ( encoded episodes ) of real images and store them in memory . To avoid the generation of blurry images , we borrow ideas from the Neural Style Transfer ( NST ) algorithm proposed by Gatys et al . ( 2016 ) to train the autoencoders . For efficient memory management , we use the notion of memory integration , from hippocampal and neocortical concept learning ( Mack et al. , 2018 ) , to combine similar episodes into centroids and covariance matrices eliminating the need to store real data . This paper contributes : 1 ) an autoencoder based approach to strict class-incremental learning which uses Neural Style Transfer to produce quality samples reflecting old class data ( Sec . 3.1 ) ; 2 ) a cognitively-inspired memory management technique that combines similar samples into a centroid/covariance representation , drastically reducing the memory required ( Sec . 3.2 ) ; 3 ) a data filtering and a loss weighting technique to manage image degradation of old classes during classifier training ( Sec . 3.3 ) . We further show that EEC outperforms state-of-the-art ( SOTA ) approaches on benchmark datasets by significant margins while also using far less memory . 2 RELATED WORK . Most recent approaches to class-incremental learning store a portion of the real images belonging to the old classes to avoid catastrophic forgetting . Rebuffi et al . ( 2017 ) ( iCaRL ) store old class images and utilize knowledge distillation ( Hinton et al. , 2015 ) for representation learning and the nearest class mean ( NCM ) classifier for classification of the old and new classes . Knowledge distillation uses a loss term to force the labels of the images of previous classes to remain the same when learning new classes . Castro et al . ( 2018 ) ( EEIL ) improves iCaRL with an end-to-end learning approach . Wu et al . ( 2019 ) also stores real images and uses a bias correction layer to avoid any bias toward the new classes . To avoid storing old class images , some approaches store features from the last fully-connected layer of the neural networks ( Xiang et al. , 2019 ; Hayes & Kanan , 2020 ; Ayub & Wagner , 2020b ; d ) . These approaches , however , use a network pretrained on ImageNet to extract features , which gives them an unfair advantage over other approaches . Because of their reliance on a pretrained network , these approaches can not be applied in situations when new data differs drastically from ImageNet ( Russakovsky et al. , 2015 ) . These difficulties have forced researchers to consider using generative networks . Methods employing generative networks tend to model previous class statistics and regenerate images belonging to the old classes while attempting to learn new classes . Both Shin et al . ( 2017 ) and Wu et al . ( 2018a ) use generative replay where the generator is trained on a mixture of generated old class images and real images from the new classes . This approach , however , causes images belonging to classes learned in earlier increments to start to semantically drift , i.e . the quality of images degrades because of the repeated training on synthesized images . Ostapenko et al . ( 2019 ) avoids semantic drift by training the GAN only once on the data of each class . Catastrophic forgetting is avoided by applying elastic weight consolidation ( Kirkpatrick et al. , 2017 ) , in which changes in important weights needed for old classes are avoided when learning new classes . They also grow their network when it runs out of memory while learning new classes , which can be difficult to apply in situations with restricted memory . One major issue with GAN based approaches is that GANs tend to generate images that do not belong to any of the learned classes which decreases classification accuracy . For these reasons , most approaches only perform well on simpler datasets such as MNIST ( LeChun , 1998 ) but perform poorly on complex datasets such as ImageNet . Conditional GAN can be used to mitigate the problem of images belonging to none of the classes as done by Ostapenko et al . ( 2019 ) , however the performance is still poor on complex datasets such as ImageNet-50 ( see Table 1 and Table 2 ) . We avoid the problem of generating images that do not belong to any learned class by training autoencoders instead of GANs . Comparatively little work has focused on using autoencoders to generate samples because the images generated by autoencoders are blurry , limiting their usefulness for classifier training . Hattori ( 2014 ) uses autoencoders on binary pixel images and Kemker & Kanan ( 2018 ) ( FearNet ) uses a network pre-trained on ImageNet to extract feature embeddings for images , applying the autoencoder to the feature embeddings . Neither of these approaches are scalable to RGB images . Moreover , the use of a pre-trained network to extract features gives FearNet an unfair advantage over other approaches . 3 ENCODING EPISODES AS CONCEPTS ( EEC ) . Following the notation of Chaudhry et al . ( 2018 ) , we consider St = { ( xti , yti ) } n t i=1 to be the set of samples xi ∈ X and their ground truth labels yti belonging to task t. In a class-incremental setup , St can contain one or multiple classes and data for different tasks is available to the model in different increments . In each increment , the model is evaluated on all the classes seen so far . Our formal presentation of continual learning follows Ostapenko et al . ( 2019 ) , where a task solver model ( classifier for class-incremental learning ) D has to update its parameters θD on the data of task t in an increment such that it performs equally well on all the t − 1 previous tasks seen so far . Data for the t − 1 tasks is not available when the model is learning task t. The subsections below present our approach . 3.1 AUTOENCODER TRAINING WITH NEURAL STYLE TRANSFER . An autoencoder is a neural network that is trained to compress and then reconstruct the input ( Goodfellow et al. , 2016 ) , formally fr : X → X . The network consists of an encoder that compresses the input into a lower dimensional feature space ( termed as the encoded episode in this paper ) , genc : X → F and a decoder that reconstructs the input from the feature embedding , gdec : F → X . Formally , for a given input x ∈ X , the reconstruction pipeline fr is defined as : fr ( x ) = ( gdec ◦ genc ) ( x ) . The parameters θr of the network are usually optimized using an l2 loss ( Lr ) between the inputs and the reconstructions : Lr = ||x− fr ( x ) ||2 ( 1 ) Although autoencoders are suitable for dimensionality reduction for complex , high-dimensional data like RGB images , the reconstructed images lose the high frequency components necessary for correct classification . To tackle this problem , we train autoencoders using some of the ideas that underline Neural Style Transfer ( NST ) . NST uses a pre-trained CNN to transfer the style of one image to another . The process takes three images , an input image , a content image and a style image and alters the input image such that it has the content image ’ s content and the artistic style of the style image . The three images are passed through the pre-trained CNN generating convolutional feature maps ( usually from the last convolutional layer ) and l2 distances between the feature maps of the input image and content image ( content loss ) and style image ( style loss ) are calculated . These losses are then used to update the input image . Intuitively , our intent here is to create reconstructed images that are similar to the real images ( in the pixel and convolutional space ) , thereby improving classification accuracy . Hence , we only utilize the idea of content transfer from the NST algorithm , where the input image is the image reconstructed by the autoencoder and content image is the real image corresponding to the reconstructed image . The classifier model , D , is used to generate convolutional feature maps for the NST , since it is already trained on real data for the classes in the increment t. In contrast to the traditional NST algorithm , we use the content loss ( Lcont ) to train the autoencoder , rather than updating the input image directly . Formally , let fc : X → Fc be the classifier pipeline that converts input images into convolutional features . For an input image , xti of task t , the content loss is : Lcont = ||fc ( xti ) − fc ( fr ( xti ) ) ||2 ( 2 ) The autoencoder parameters are optimized using a combination of reconstruction and content losses : L = ( 1− λ ) Lr + λLcont ( 3 ) where , λ is a hyperparamter that controls the contribution of each loss term towards the complete loss . During autoencoder training , classifier D acts as a fixed feature extractor and its parameters are not updated . This portion of the complete procedure is depicted in Figure 1 ( a ) . To provide an illustration of our approach , we perform an experiment with ImageNet-50 dataset . We trained one autoencoder on 10 classes from ImageNet-50 with NST and one without NST . Figure 2 depicts the reconstructed images by the two autoencoders . Note that the images generated by the autoencoder trained without using NST are blurry . In contrast , the autoencoder trained using NST creates images with fine-grained details which improves the classification accuracy .

Summary: In continual learning settings, one of the important technique for avoiding catastrophe forgetting is to replay data points from the past. For memory efficiency purposes, representative samples can be generated from a generative model, such as GANs, rather than replaying the original samples which can be large in number. It is argued that GANs generate new samples which may not belong exactly to one of the classes, so a new generative model is proposed. Experimental results are appealing.