Memory Replay with Data Compression for Continual Learning

1 INTRODUCTION . The ability to continually learn numerous tasks and infer them together is critical for deep neural networks ( DNNs ) , which needs to mitigate catastrophic forgetting ( McCloskey & Cohen , 1989 ) of the past . Memory replay of representative old training samples ( referred to as memory replay ) has been shown as an effective solution , and achieves the state-of-the-art ( SOTA ) performance ( Hou et al. , 2019 ) . Existing memory replay approaches are mainly built on a small memory buffer containing a few original data , and try to construct and exploit it more effectively . However , due to the low storage efficiency of saving original data , this strategy of building memory buffer will lose a lot of information about the old data distribution . On the other hand , this usually requires huge extra computation to further mitigate catastrophic forgetting , such as by learning additional parameters ( Liu et al. , 2021a ) or distilling old features ( Hu et al. , 2021 ) . Different from “ artificial ” memory replay in DNNs , a significant feature of biological memory is to encode the old experiences in a highly compressed form and replay them to overcome catastrophic forgetting ( McClelland , 2013 ; Davidson et al. , 2009 ; Carr et al. , 2011 ) . Thus the learned information can be maintained in a small storage space as comprehensively as possible , and flexibly retrieved . Inspired by the compression feature of biological memory replay , we propose memory replay with data compression , which can largely increase the amount of old training samples that can be stored in the memory buffer by reducing their storage cost in a computationally efficient way . ∗Equal Contribution †Corresponding Authors Given a limited storage space , data compression introduces an additional degree of freedom to explicitly balance the quality and quantity for memory replay . With a properly selected quality , using a naive JPEG compression algorithm ( Wallace , 1992 ) can achieve comparable or better performance than recent strong approaches with less extra computation ( Fig . 1 , purple arrow ) , and can further improve their performance ( Fig . 1 , gray arrow ) . However , to empirically determine the compression quality is usually inefficient and impractical , since it requires learning a task sequence or sub-sequence repeatedly1 . We propose a novel method based on determinantal point processes ( DPPs ) to efficiently determine it without repetitive training . Further , we demonstrate the advantages of our proposal in realistic applications such as continual learning of object detection for autonomous driving , where the incremental data are extremely large-scale . Our contributions include : ( 1 ) We propose memory replay with data compression , which is both an important baseline and a promising direction for continual learning ; ( 2 ) We empirically validate that the trade-off between quality and quantity of compressed data is highly nontrivial for memory replay , and provide a novel method to efficiently determine it without repetitive training ; ( 3 ) Extensive experiments show that using a naive data compression algorithm with a properly selected quality can largely improve memory replay by saving more compressed data in a limited storage space . 2 RELATED WORK . Continual learning needs to overcome catastrophic forgetting of the past when learning a new task . Regularization-based methods ( Kirkpatrick et al. , 2017 ; Wang et al. , 2021b ) approximated the importance of each parameter to the old tasks and selectively penalized its changes . Architecturebased methods ( Rusu et al. , 2016 ) allocated a dedicated parameter subspace for each task to prevent mutual interference . Replay-based methods ( Rebuffi et al. , 2017 ; Shin et al. , 2017 ) approximated and recovered the old data distribution . In particular , memory replay of representative old training samples ( referred to as memory replay ) can generally achieve the best performance in class-incremental learning ( Liu et al. , 2021a ; Hu et al. , 2021 ) and in numerous other continual learning scenarios , such as audio tasks ( Ehret et al. , 2020 ) , few-shot ( Tao et al. , 2020b ) , semi-supervised ( Wang et al. , 2021a ) , and unsupervised continual learning ( Khare et al. , 2021 ) . Most of the work in memory replay attempted to more effectively construct and exploit a small memory buffer containing a few original data . As the pioneer work , iCaRL ( Rebuffi et al. , 2017 ) proposed a general protocol of memory replay for continual learning . To better construct the memory buffer , Mnemonics ( Liu et al. , 2020b ) parameterized the original data and made them optimizable , while TPCIL ( Tao et al. , 2020a ) constructed an elastic Hebbian graph by competitive Hebbian learning . On the other hand , BiC ( Wu et al. , 2019 ) , LUCIR ( Hou et al. , 2019 ) , PODNet ( Douillard et al. , 2020 ) , DDE ( Hu et al. , 2021 ) and AANets ( Liu et al. , 2021a ) attempted to better exploit the memory buffer , such as by mitigating the data imbalance between old and new classes ( Hou et al. , 2019 ; Wu et al. , 2019 ; Hu et al. , 2021 ) . In contrast to saving original data , some work aimed at improving the efficiency of remembering the old data distribution . One solution is to continually learn a generative model to replay generated data ( Shin et al. , 2017 ; Wu et al. , 2018 ) or compress old training data ( Caccia et al. , 2020 ) . However , 1A naive grid search approach is to train continual learning processes with different qualities and choose the best one , resulting in huge computational cost . Also , this strategy will be less applicable if the old data can not be revisited , or the future data can not be accessed immediately . continual learning of such a generative model is extremely challenging , which limits its applications to relatively simple domains , and usually requires a lot of extra computation . Another solution is feature replay : GFR ( Liu et al. , 2020a ) learned a feature generator from a feature extractor to replay generated features , but the feature extractor suffered from catastrophic forgetting since it was incrementally updated . REMIND ( Hayes et al. , 2020 ) saved the old features and reconstructed the synthesized features for replay , but it froze the majority of feature extractor after learning the initial phase , limiting the learning of representations for incremental tasks . Data compression aims to improve the storage efficiency of a file , including lossless compression and lossy compression . Lossless compression needs to perfectly reconstruct the original data from the compressed data , which limits its compression rate ( Shannon , 1948 ) . In contrast , lossy compression can achieve a much higher compression rate by degrading the original data , so it has been broadly used in realistic applications . Representative hand-crafted approaches include JPEG ( or JPG ) ( Wallace , 1992 ) , which is the most commonly-used algorithm of lossy compression ( Mentzer et al. , 2020 ) , WebP ( Lian & Shilei , 2012 ) and JPEG2000 ( Rabbani , 2002 ) . On the other hand , neural compression approaches generally rely on optimizing Shannon ’ s rate-distortion trade-off , through RNNs ( Toderici et al. , 2015 ; 2017 ) , auto-encoders ( Agustsson et al. , 2017 ) and GANs ( Mentzer et al. , 2020 ) . 3 CONTINUAL LEARNING PRELIMINARIES . We consider a general setting of continual learning , which needs to incrementally learn numerous tasks t = 1 , 2 , 3 , ... from their task-specific training dataset Dt = { ( xt , i , yt , i ) } Nti=1 , where Dt is only available when learning task t , ( xt , i , yt , i ) is a data-label pair and Nt is the number of such training samples . For classification tasks , the training samples of each task might be from one or several new classes . All the classes ever seen are evaluated at test time , and the classes from different tasks need to be distinguished . This setting is also called class-incremental learning ( van de Ven & Tolias , 2019 ) . Suppose a DNN with parameter θ has learned T tasks and attempts to learn a new task . Since the old training datasets ⋃T t=1Dt are unavailable , the DNN will adapt the learned parameters to fit DT+1 , and tend to catastrophically forget the old tasks ( McCloskey & Cohen , 1989 ) . An effective solution of overcoming catastrophic forgetting is to select and store representative old training samples Dmbt = { ( xt , i , yt , i ) } Nmbt i=1 in a small memory buffer ( mb ) , and replay them when learning the new task . For classification tasks , mean-of-feature is a widely-used strategy to select Dmbt ( Rebuffi et al. , 2017 ; Hou et al. , 2019 ) . After learning each task , features of the training data can be obtained by the learned embedding function F eθ ( · ) . In each class , several data points nearest to the mean-of-feature are selected into the memory buffer . Then , the training dataset of task T + 1 becomes D′T+1 = DT+1 ⋃ Dmb1 : T , including both new training samples DT+1 and some old training samples Dmb1 : T = ⋃T t=1D mb t , so as to prevent forgetting of the old tasks . However , due to the limited storage space , only a few original data can be saved in the memory buffer , namely , Nmbt Nt . Although numerous efforts in memory replay attempted to more effectively exploit the memory buffer , such as by alleviating the data imbalance between the old and new classes , this strategy of building memory buffer is less effective for remembering the old data distribution . 4 METHOD . In this section , we first present memory replay with data compression for continual learning . Then , we empirically validate that there is a trade-off between the quality and quantity of compressed data , which is highly nontrivial for memory replay , and propose a novel method to determine it efficiently . 4.1 MEMORY REPLAY WITH DATA COMPRESSION . Inspired by the biological memory replay that is in a highly compressed form ( Carr et al. , 2011 ) , we propose an important baseline for memory replay , that is , using data compression to increase the amount of old training samples that can be stored in the memory buffer , so as to more effectively recover the old data distribution . Data compression can be generally defined as a function F cq ( · ) of compressing the original data xt , i to xq , t , i = F cq ( xt , i ) with a controllable quality q . Due to the smaller storage cost of each xq , t , i than xt , i , the memory buffer can maintain more old training samples for replay , namely , Nmbq , t > N mb t for D mb q , t = { ( xq , t , i , yt , i ) } Nmbq , t i=1 in D mb q,1 : T = ⋃T t=1D mb q , t . For notation clarity , we will use Dmbq to denote D mb q , t without explicitly writing out its task label t , likewise for xi , yi , Nmbq , N mb and D. The compression rate can be defined as rq = Nmbq /N mb ∝ Nmbq . In analogy to the learning theory for supervised learning , we argue that continual learning will also benefit from replaying more compressed data , assuming that they approximately follow the original data distribution . However , the assumption is likely to be violated if the compression rate is too high . Intuitively , this leads to a trade-off between quality and quantity : if the storage space is limited , reducing the quality q of data compression will increase the quantity Nmbq of compressed data that can be stored in the memory buffer , and vice versa . Here we evaluate the proposed idea by compressing images with JPEG ( Wallace , 1992 ) , a naive but commonly-used lossy compression algorithm . JPEG can save images with a quality in the range of [ 1 , 100 ] , where reducing the quality results in a smaller file size . Using a memory buffer equivalent to 20 original images per class ( Hou et al. , 2019 ) , we make a grid search of the JPEG quality with representative memory replay approaches , such as LUCIR ( Hou et al. , 2019 ) and PODNet ( Douillard et al. , 2020 ) . As shown in Fig . 2 , memory replay of compressed data with an appropriate quality can substantially outperform that of original data . However , whether the quality is too large or too small , it will affect the performance . In particular , the quality that achieves the best performance varies with the memory replay approaches , but is consistent for different numbers of splits of incremental phases .

Summary: In this work, the authors propose memory replay with data compression, which is both an important yet neglected baseline and a promising direction for continual learning. Using a naive technique of data compression with a properly selected quality, the proposed method can achieve the SOTA performance in a time-efficient and plug-and-play way. Since the compression quality is highly nontrivial for the efficacy of memory replay, the authors provide a novel method based on determinantal point processes (DPPs) to determine it efficiently, and validate their method in both class-incremental learning and semi-supervised continual learning of object detection.