Memory Replay with Data Compression for Continual Learning

1 INTRODUCTION . The ability to continually learn numerous tasks and infer them together is critical for deep neural networks ( DNNs ) , which needs to mitigate catastrophic forgetting ( McCloskey & Cohen , 1989 ) of the past . Memory replay of representative old training samples ( referred to as memory replay ) has been shown as an effective solution , and achieves the state-of-the-art ( SOTA ) performance ( Hou et al. , 2019 ) . Existing memory replay approaches are mainly built on a small memory buffer containing a few original data , and try to construct and exploit it more effectively . However , due to the low storage efficiency of saving original data , this strategy of building memory buffer will lose a lot of information about the old data distribution . On the other hand , this usually requires huge extra computation to further mitigate catastrophic forgetting , such as by learning additional parameters ( Liu et al. , 2021a ) or distilling old features ( Hu et al. , 2021 ) . Different from “ artificial ” memory replay in DNNs , a significant feature of biological memory is to encode the old experiences in a highly compressed form and replay them to overcome catastrophic forgetting ( McClelland , 2013 ; Davidson et al. , 2009 ; Carr et al. , 2011 ) . Thus the learned information can be maintained in a small storage space as comprehensively as possible , and flexibly retrieved . Inspired by the compression feature of biological memory replay , we propose memory replay with data compression , which can largely increase the amount of old training samples that can be stored in the memory buffer by reducing their storage cost in a computationally efficient way . ∗Equal Contribution †Corresponding Authors Given a limited storage space , data compression introduces an additional degree of freedom to explicitly balance the quality and quantity for memory replay . With a properly selected quality , using a naive JPEG compression algorithm ( Wallace , 1992 ) can achieve comparable or better performance than recent strong approaches with less extra computation ( Fig . 1 , purple arrow ) , and can further improve their performance ( Fig . 1 , gray arrow ) . However , to empirically determine the compression quality is usually inefficient and impractical , since it requires learning a task sequence or sub-sequence repeatedly1 . We propose a novel method based on determinantal point processes ( DPPs ) to efficiently determine it without repetitive training . Further , we demonstrate the advantages of our proposal in realistic applications such as continual learning of object detection for autonomous driving , where the incremental data are extremely large-scale . Our contributions include : ( 1 ) We propose memory replay with data compression , which is both an important baseline and a promising direction for continual learning ; ( 2 ) We empirically validate that the trade-off between quality and quantity of compressed data is highly nontrivial for memory replay , and provide a novel method to efficiently determine it without repetitive training ; ( 3 ) Extensive experiments show that using a naive data compression algorithm with a properly selected quality can largely improve memory replay by saving more compressed data in a limited storage space . 2 RELATED WORK . Continual learning needs to overcome catastrophic forgetting of the past when learning a new task . Regularization-based methods ( Kirkpatrick et al. , 2017 ; Wang et al. , 2021b ) approximated the importance of each parameter to the old tasks and selectively penalized its changes . Architecturebased methods ( Rusu et al. , 2016 ) allocated a dedicated parameter subspace for each task to prevent mutual interference . Replay-based methods ( Rebuffi et al. , 2017 ; Shin et al. , 2017 ) approximated and recovered the old data distribution . In particular , memory replay of representative old training samples ( referred to as memory replay ) can generally achieve the best performance in class-incremental learning ( Liu et al. , 2021a ; Hu et al. , 2021 ) and in numerous other continual learning scenarios , such as audio tasks ( Ehret et al. , 2020 ) , few-shot ( Tao et al. , 2020b ) , semi-supervised ( Wang et al. , 2021a ) , and unsupervised continual learning ( Khare et al. , 2021 ) . Most of the work in memory replay attempted to more effectively construct and exploit a small memory buffer containing a few original data . As the pioneer work , iCaRL ( Rebuffi et al. , 2017 ) proposed a general protocol of memory replay for continual learning . To better construct the memory buffer , Mnemonics ( Liu et al. , 2020b ) parameterized the original data and made them optimizable , while TPCIL ( Tao et al. , 2020a ) constructed an elastic Hebbian graph by competitive Hebbian learning . On the other hand , BiC ( Wu et al. , 2019 ) , LUCIR ( Hou et al. , 2019 ) , PODNet ( Douillard et al. , 2020 ) , DDE ( Hu et al. , 2021 ) and AANets ( Liu et al. , 2021a ) attempted to better exploit the memory buffer , such as by mitigating the data imbalance between old and new classes ( Hou et al. , 2019 ; Wu et al. , 2019 ; Hu et al. , 2021 ) . In contrast to saving original data , some work aimed at improving the efficiency of remembering the old data distribution . One solution is to continually learn a generative model to replay generated data ( Shin et al. , 2017 ; Wu et al. , 2018 ) or compress old training data ( Caccia et al. , 2020 ) . However , 1A naive grid search approach is to train continual learning processes with different qualities and choose the best one , resulting in huge computational cost . Also , this strategy will be less applicable if the old data can not be revisited , or the future data can not be accessed immediately . continual learning of such a generative model is extremely challenging , which limits its applications to relatively simple domains , and usually requires a lot of extra computation . Another solution is feature replay : GFR ( Liu et al. , 2020a ) learned a feature generator from a feature extractor to replay generated features , but the feature extractor suffered from catastrophic forgetting since it was incrementally updated . REMIND ( Hayes et al. , 2020 ) saved the old features and reconstructed the synthesized features for replay , but it froze the majority of feature extractor after learning the initial phase , limiting the learning of representations for incremental tasks . Data compression aims to improve the storage efficiency of a file , including lossless compression and lossy compression . Lossless compression needs to perfectly reconstruct the original data from the compressed data , which limits its compression rate ( Shannon , 1948 ) . In contrast , lossy compression can achieve a much higher compression rate by degrading the original data , so it has been broadly used in realistic applications . Representative hand-crafted approaches include JPEG ( or JPG ) ( Wallace , 1992 ) , which is the most commonly-used algorithm of lossy compression ( Mentzer et al. , 2020 ) , WebP ( Lian & Shilei , 2012 ) and JPEG2000 ( Rabbani , 2002 ) . On the other hand , neural compression approaches generally rely on optimizing Shannon ’ s rate-distortion trade-off , through RNNs ( Toderici et al. , 2015 ; 2017 ) , auto-encoders ( Agustsson et al. , 2017 ) and GANs ( Mentzer et al. , 2020 ) . 3 CONTINUAL LEARNING PRELIMINARIES . We consider a general setting of continual learning , which needs to incrementally learn numerous tasks t = 1 , 2 , 3 , ... from their task-specific training dataset Dt = { ( xt , i , yt , i ) } Nti=1 , where Dt is only available when learning task t , ( xt , i , yt , i ) is a data-label pair and Nt is the number of such training samples . For classification tasks , the training samples of each task might be from one or several new classes . All the classes ever seen are evaluated at test time , and the classes from different tasks need to be distinguished . This setting is also called class-incremental learning ( van de Ven & Tolias , 2019 ) . Suppose a DNN with parameter θ has learned T tasks and attempts to learn a new task . Since the old training datasets ⋃T t=1Dt are unavailable , the DNN will adapt the learned parameters to fit DT+1 , and tend to catastrophically forget the old tasks ( McCloskey & Cohen , 1989 ) . An effective solution of overcoming catastrophic forgetting is to select and store representative old training samples Dmbt = { ( xt , i , yt , i ) } Nmbt i=1 in a small memory buffer ( mb ) , and replay them when learning the new task . For classification tasks , mean-of-feature is a widely-used strategy to select Dmbt ( Rebuffi et al. , 2017 ; Hou et al. , 2019 ) . After learning each task , features of the training data can be obtained by the learned embedding function F eθ ( · ) . In each class , several data points nearest to the mean-of-feature are selected into the memory buffer . Then , the training dataset of task T + 1 becomes D′T+1 = DT+1 ⋃ Dmb1 : T , including both new training samples DT+1 and some old training samples Dmb1 : T = ⋃T t=1D mb t , so as to prevent forgetting of the old tasks . However , due to the limited storage space , only a few original data can be saved in the memory buffer , namely , Nmbt Nt . Although numerous efforts in memory replay attempted to more effectively exploit the memory buffer , such as by alleviating the data imbalance between the old and new classes , this strategy of building memory buffer is less effective for remembering the old data distribution . 4 METHOD . In this section , we first present memory replay with data compression for continual learning . Then , we empirically validate that there is a trade-off between the quality and quantity of compressed data , which is highly nontrivial for memory replay , and propose a novel method to determine it efficiently . 4.1 MEMORY REPLAY WITH DATA COMPRESSION . Inspired by the biological memory replay that is in a highly compressed form ( Carr et al. , 2011 ) , we propose an important baseline for memory replay , that is , using data compression to increase the amount of old training samples that can be stored in the memory buffer , so as to more effectively recover the old data distribution . Data compression can be generally defined as a function F cq ( · ) of compressing the original data xt , i to xq , t , i = F cq ( xt , i ) with a controllable quality q . Due to the smaller storage cost of each xq , t , i than xt , i , the memory buffer can maintain more old training samples for replay , namely , Nmbq , t > N mb t for D mb q , t = { ( xq , t , i , yt , i ) } Nmbq , t i=1 in D mb q,1 : T = ⋃T t=1D mb q , t . For notation clarity , we will use Dmbq to denote D mb q , t without explicitly writing out its task label t , likewise for xi , yi , Nmbq , N mb and D. The compression rate can be defined as rq = Nmbq /N mb ∝ Nmbq . In analogy to the learning theory for supervised learning , we argue that continual learning will also benefit from replaying more compressed data , assuming that they approximately follow the original data distribution . However , the assumption is likely to be violated if the compression rate is too high . Intuitively , this leads to a trade-off between quality and quantity : if the storage space is limited , reducing the quality q of data compression will increase the quantity Nmbq of compressed data that can be stored in the memory buffer , and vice versa . Here we evaluate the proposed idea by compressing images with JPEG ( Wallace , 1992 ) , a naive but commonly-used lossy compression algorithm . JPEG can save images with a quality in the range of [ 1 , 100 ] , where reducing the quality results in a smaller file size . Using a memory buffer equivalent to 20 original images per class ( Hou et al. , 2019 ) , we make a grid search of the JPEG quality with representative memory replay approaches , such as LUCIR ( Hou et al. , 2019 ) and PODNet ( Douillard et al. , 2020 ) . As shown in Fig . 2 , memory replay of compressed data with an appropriate quality can substantially outperform that of original data . However , whether the quality is too large or too small , it will affect the performance . In particular , the quality that achieves the best performance varies with the memory replay approaches , but is consistent for different numbers of splits of incremental phases .

Summary: In this paper, the authors propose a data compression method for memory-replay based continual learning algorithms. With the data compression method,  more old training samples can be stored in the memory to better capture old data distribution. However, there is a trade-off between the quality and quantity of compressed data, the authors propose to use determinantal point processes to determine the quality of the data compression.  Extensive experiments show that with the proposed method, a naive compression method can achieve the SOTA on several continual learning benchmarks.