Year: 2022

Abstract:
Continual learning needs to overcome catastrophic forgetting of the past. Memory replay of representative old training samples has been shown as an effective solution, and achieves the state-of-the-art (SOTA) performance. However, existing work is mainly built on a small memory buffer containing a few original data, which cannot fully characterize the old data distribution. In this work, we propose memory replay with data compression to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer. Observing that the trade-off between the quality and quantity of compressed data is highly nontrivial for the efficacy of memory replay, we propose a novel method based on determinantal point processes (DPPs) to efficiently determine an appropriate compression quality for currently-arrived training samples. In this way, using a naive data compression algorithm with a properly selected quality can largely boost recent strong baselines by saving more compressed data in a limited storage space. We extensively validate this across several benchmarks of class-incremental learning and in a realistic scenario of object detection for autonomous driving.

heading: 1 INTRODUCTION
text: The ability to continually learn numerous tasks and infer them together is critical for deep neural networks (DNNs), which needs to mitigate catastrophic forgetting (McCloskey & Cohen, 1989) of the past. Memory replay of representative old training samples (referred to as memory replay) has been shown as an effective solution, and achieves the state-of-the-art (SOTA) performance (Hou et al., 2019). Existing memory replay approaches are mainly built on a small memory buffer containing a few original data, and try to construct and exploit it more effectively. However, due to the low storage efficiency of saving original data, this strategy of building memory buffer will lose a lot of information about the old data distribution. On the other hand, this usually requires huge extra computation to further mitigate catastrophic forgetting, such as by learning additional parameters (Liu et al., 2021a) or distilling old features (Hu et al., 2021).
Different from “artificial” memory replay in DNNs, a significant feature of biological memory is to encode the old experiences in a highly compressed form and replay them to overcome catastrophic forgetting (McClelland, 2013; Davidson et al., 2009; Carr et al., 2011). Thus the learned information can be maintained in a small storage space as comprehensively as possible, and flexibly retrieved. Inspired by the compression feature of biological memory replay, we propose memory replay with data compression, which can largely increase the amount of old training samples that can be stored in the memory buffer by reducing their storage cost in a computationally efficient way.
∗Equal Contribution †Corresponding Authors
Given a limited storage space, data compression introduces an additional degree of freedom to explicitly balance the quality and quantity for memory replay. With a properly selected quality, using a naive JPEG compression algorithm (Wallace, 1992) can achieve comparable or better performance than recent strong approaches with less extra computation (Fig. 1, purple arrow), and can further improve their performance (Fig. 1, gray arrow). However, to empirically determine the compression quality is usually inefficient and impractical, since it requires learning a task sequence or sub-sequence repeatedly1. We propose a novel method based on determinantal point processes (DPPs) to efficiently determine it without repetitive training. Further, we demonstrate the advantages of our proposal in realistic applications such as continual learning of object detection for autonomous driving, where the incremental data are extremely large-scale.
Our contributions include: (1) We propose memory replay with data compression, which is both an important baseline and a promising direction for continual learning; (2) We empirically validate that the trade-off between quality and quantity of compressed data is highly nontrivial for memory replay, and provide a novel method to efficiently determine it without repetitive training; (3) Extensive experiments show that using a naive data compression algorithm with a properly selected quality can largely improve memory replay by saving more compressed data in a limited storage space.

heading: 2 RELATED WORK
text: Continual learning needs to overcome catastrophic forgetting of the past when learning a new task. Regularization-based methods (Kirkpatrick et al., 2017; Wang et al., 2021b) approximated the importance of each parameter to the old tasks and selectively penalized its changes. Architecturebased methods (Rusu et al., 2016) allocated a dedicated parameter subspace for each task to prevent mutual interference. Replay-based methods (Rebuffi et al., 2017; Shin et al., 2017) approximated and recovered the old data distribution. In particular, memory replay of representative old training samples (referred to as memory replay) can generally achieve the best performance in class-incremental learning (Liu et al., 2021a; Hu et al., 2021) and in numerous other continual learning scenarios, such as audio tasks (Ehret et al., 2020), few-shot (Tao et al., 2020b), semi-supervised (Wang et al., 2021a), and unsupervised continual learning (Khare et al., 2021).
Most of the work in memory replay attempted to more effectively construct and exploit a small memory buffer containing a few original data. As the pioneer work, iCaRL (Rebuffi et al., 2017) proposed a general protocol of memory replay for continual learning. To better construct the memory buffer, Mnemonics (Liu et al., 2020b) parameterized the original data and made them optimizable, while TPCIL (Tao et al., 2020a) constructed an elastic Hebbian graph by competitive Hebbian learning. On the other hand, BiC (Wu et al., 2019), LUCIR (Hou et al., 2019), PODNet (Douillard et al., 2020), DDE (Hu et al., 2021) and AANets (Liu et al., 2021a) attempted to better exploit the memory buffer, such as by mitigating the data imbalance between old and new classes (Hou et al., 2019; Wu et al., 2019; Hu et al., 2021).
In contrast to saving original data, some work aimed at improving the efficiency of remembering the old data distribution. One solution is to continually learn a generative model to replay generated data (Shin et al., 2017; Wu et al., 2018) or compress old training data (Caccia et al., 2020). However,
1A naive grid search approach is to train continual learning processes with different qualities and choose the best one, resulting in huge computational cost. Also, this strategy will be less applicable if the old data cannot be revisited, or the future data cannot be accessed immediately.
continual learning of such a generative model is extremely challenging, which limits its applications to relatively simple domains, and usually requires a lot of extra computation. Another solution is feature replay: GFR (Liu et al., 2020a) learned a feature generator from a feature extractor to replay generated features, but the feature extractor suffered from catastrophic forgetting since it was incrementally updated. REMIND (Hayes et al., 2020) saved the old features and reconstructed the synthesized features for replay, but it froze the majority of feature extractor after learning the initial phase, limiting the learning of representations for incremental tasks.
Data compression aims to improve the storage efficiency of a file, including lossless compression and lossy compression. Lossless compression needs to perfectly reconstruct the original data from the compressed data, which limits its compression rate (Shannon, 1948). In contrast, lossy compression can achieve a much higher compression rate by degrading the original data, so it has been broadly used in realistic applications. Representative hand-crafted approaches include JPEG (or JPG) (Wallace, 1992), which is the most commonly-used algorithm of lossy compression (Mentzer et al., 2020), WebP (Lian & Shilei, 2012) and JPEG2000 (Rabbani, 2002). On the other hand, neural compression approaches generally rely on optimizing Shannon’s rate-distortion trade-off, through RNNs (Toderici et al., 2015; 2017), auto-encoders (Agustsson et al., 2017) and GANs (Mentzer et al., 2020).

heading: 3 CONTINUAL LEARNING PRELIMINARIES
text: We consider a general setting of continual learning, which needs to incrementally learn numerous tasks t = 1, 2, 3, ... from their task-specific training dataset Dt = {(xt,i, yt,i)}Nti=1, where Dt is only available when learning task t, (xt,i, yt,i) is a data-label pair and Nt is the number of such training samples. For classification tasks, the training samples of each task might be from one or several new classes. All the classes ever seen are evaluated at test time, and the classes from different tasks need to be distinguished. This setting is also called class-incremental learning (van de Ven & Tolias, 2019). Suppose a DNN with parameter θ has learned T tasks and attempts to learn a new task. Since the old training datasets ⋃T t=1Dt are unavailable, the DNN will adapt the learned parameters to fit DT+1, and tend to catastrophically forget the old tasks (McCloskey & Cohen, 1989).
An effective solution of overcoming catastrophic forgetting is to select and store representative old training samples Dmbt = {(xt,i, yt,i)} Nmbt i=1 in a small memory buffer (mb), and replay them when learning the new task. For classification tasks, mean-of-feature is a widely-used strategy to select Dmbt (Rebuffi et al., 2017; Hou et al., 2019). After learning each task, features of the training data can be obtained by the learned embedding function F eθ (·). In each class, several data points nearest to the mean-of-feature are selected into the memory buffer. Then, the training dataset of task T + 1 becomes D′T+1 = DT+1 ⋃ Dmb1:T , including both new training samples DT+1 and some old training
samples Dmb1:T = ⋃T t=1D mb t , so as to prevent forgetting of the old tasks.
However, due to the limited storage space, only a few original data can be saved in the memory buffer, namely, Nmbt Nt. Although numerous efforts in memory replay attempted to more effectively exploit the memory buffer, such as by alleviating the data imbalance between the old and new classes, this strategy of building memory buffer is less effective for remembering the old data distribution.

heading: 4 METHOD
text: In this section, we first present memory replay with data compression for continual learning. Then, we empirically validate that there is a trade-off between the quality and quantity of compressed data, which is highly nontrivial for memory replay, and propose a novel method to determine it efficiently.

heading: 4.1 MEMORY REPLAY WITH DATA COMPRESSION
text: Inspired by the biological memory replay that is in a highly compressed form (Carr et al., 2011), we propose an important baseline for memory replay, that is, using data compression to increase the amount of old training samples that can be stored in the memory buffer, so as to more effectively recover the old data distribution. Data compression can be generally defined as a function F cq (·) of compressing the original data xt,i to xq,t,i = F cq (xt,i) with a controllable quality q. Due to the smaller storage cost of each xq,t,i than xt,i, the memory buffer can maintain more old training samples for
replay, namely, Nmbq,t > N mb t for D mb q,t = {(xq,t,i, yt,i)} Nmbq,t i=1 in D mb q,1:T =
⋃T
t=1D mb q,t . For notation clarity, we will use Dmbq to denote D mb q,t without explicitly writing out its task label t, likewise for xi, yi, Nmbq , N mb and D. The compression rate can be defined as rq = Nmbq /N
mb ∝ Nmbq . In analogy to the learning theory for supervised learning, we argue that continual learning will also benefit from replaying more compressed data, assuming that they approximately follow the original data distribution. However, the assumption is likely to be violated if the compression rate is too high. Intuitively, this leads to a trade-off between quality and quantity: if the storage space is limited, reducing the quality q of data compression will increase the quantity Nmbq of compressed data that can be stored in the memory buffer, and vice versa.
Here we evaluate the proposed idea by compressing images with JPEG (Wallace, 1992), a naive but commonly-used lossy compression algorithm. JPEG can save images with a quality in the range of [1, 100], where reducing the quality results in a smaller file size. Using a memory buffer equivalent to 20 original images per class (Hou et al., 2019), we make a grid search of the JPEG quality with representative memory replay approaches, such as LUCIR (Hou et al., 2019) and PODNet (Douillard et al., 2020). As shown in Fig. 2, memory replay of compressed data with an appropriate quality can substantially outperform that of original data. However, whether the quality is too large or too small, it will affect the performance. In particular, the quality that achieves the best performance varies with the memory replay approaches, but is consistent for different numbers of splits of incremental phases.

heading: 4.2 QUALITY-QUANTITY TRADE-OFF
text: Since the trade-off between quality q and quantityNmbq is highly nontrivial for memory replay, it needs to be carefully determined. Formally, after learning each task from its training datasetD, let’s consider several compressed subsets Dmbq = {(xq,i, yi)} Nmbq i=1 , where q is from a set of finite candidates Q = {q1, q2, q3, ...}. Each Dmbq is constructed by selecting a subset Dmb∗q = {(xi, yi)} Nmbq i=1 of N mb q original training samples from D (following mean-of-feature or other principles). The size Nmbq is determined as the maximum number such that the compressed version of Dmb∗q to a quality q can be stored in the memory buffer (see details in Appendix A). Thereby, a smaller q enables to save a larger Nmbq , and vice versa. The objective is to select a compressed subset that can best represent D, namely, to determine an appropriate q for memory replay.
To understand the effects of data compression, which depends on the compression function F cq (·) and the continually-learned embedding function F eθ (·), we focus on analyzing the features of compressed data fq,i = F eθ (F c q (xi)). We first calculate the feature matrix M c q = [f̄q,1, f̄q,2, ..., f̄q,Nmbq ] of each compressed subset Dmbq , where each column vector f̄q,i is obtained by normalizing fq,i under L2norm to keep ||f̄q,i||2 = 1. Similarly, we obtain the feature matrix M∗q = [f̄1, f̄2, ..., f̄Nmbq ] of each original subset Dmb∗q . Then, we can analyze the quality-quantity trade-off from two aspects:
On the empirical side, in Fig. 3 we use t-SNE (Van der Maaten & Hinton, 2008) to visualize features of the original subset (light dot), which includes different amounts of original data, and its compressed subset (dark dot), which is obtained by compressing the original subset to just fit in the memory buffer. With the increase of quantity and the decrease of quality, the area of compressed subset is initially similar to that of original subset and expands synchronously. However, as a large number of low-quality compressed data occur out-of-distribution, the area of compressed subset becomes much larger than that of its original subset, where the performance also severely declines (see Fig. 2, 3).
On the theoretical side, given a training dataset D, we aim to find the compressed subset Dmbq that can best represent D by choosing an appropriate compression quality q from its range. To achieve this goal, we introduce Pq(Dmbq |D) to characterize the conditional likelihood of selecting Dmbq given input D under parameter q. The goal of learning is to choose appropriate q based on the training tasks for making accurate predictions on unseen inputs. While there are a variety of objective functions for learning, here we focus on the widely-used maximum likelihood estimation (MLE), where the goal is to choose q to maximize the conditional likelihood of the observed data:
max q Pq(Dmbq |D). (1)
The construction of Dmbq can be essentially viewed as a sampling problem with the cardinality Nmbq . Here, we apply Determinantal Point Processes (DPPs) to formulate the conditional likelihood Pq(Dmbq |D), since DPPs are not only elegant probabilistic sampling models (Kulesza & Taskar, 2012), which can characterize the probabilities for every possible subset by determinants, but also provide a geometric interpretation of the probability by the volume spanned by all elements in the subset (detailed in Appendix C.2). In particular, a conditional DPP is a conditional probabilistic model which assigns a probability Pq(Dmbq |D) to each possible subset Dmbq . Since the network parameter θ is fixed during compression, and the feature matrix M cq = F e θ (D mb q ), we rewrite Pq(Dmbq |D) as Pq(M cq |D) equivalently. Formally, such a DPP formulates the probability Pq(Dmbq |D) as
Pq(M cq |D) = det(LMcq (D; q, θ))∑
|M |=Nmbq det(LM (D; q, θ))
, (2)
where |M cq | = Nmbq and L(D; q, θ) is a conditional DPP |D| × |D| kernel matrix that depends on the input D, the parameters θ and q. LM (D; q, θ) (resp., LMcq (D; q, θ)) is the submatrix sampled from L(D; q, θ) using indices from M (resp., M cq ). The numerator defines the marginal probability of inclusion for the subset M cq , and the denominator serves as a normalizer to enforce the sum of Pq(M cq |D) for every possible M cq to 1. Generally, there are many ways to obtain a positive semi-definite kernel L. In this work, we employ the most widely-used dot product kernel function, where LMcq = M c> q M c q and LM = M >M .
However, due to the extremely high complexity of calculating the denominator in Eq. (2) (analyzed in Appendix C.1), it is difficult to optimize Pq(M cq |D). Alternatively, by introducing Pq(M∗q |D) to characterize the conditional likelihood of selecting M∗q given input D under parameter q, we propose a relaxed optimization program of Eq. (1), in which we (1) maximize Pq(M∗q |D) since Pq(M cq |D) ≤ Pq(M∗q |D) is always satisfied under lossy compression; and meanwhile, (2) constrain that Pq(M cq |D) is consistent with Pq(M∗q |D). The relaxed program is solved as follows.
First, by formulating Pq(M∗q |D) similarly as Pq(M cq |D) in Eq. (2) (detailed in Appendix C.2), we need to maximize
L1(q) = Pq(M∗q |D) = det(LM∗q (D; θ))∑
|M∗|=Nmbq det(LM∗(D; θ))
, (3)
where the conditional DPP kernel matrix L(D; θ) only depends on D and θ. For our task, Pq(M∗q |D) monotonically increases with Nmbq . Thus, optimizing L1 is converted into maxq Nmbq equivalently, with significantly reduced complexity (detailed in Proposition 1 of Appendix C.3).
Second, to constrain that Pq(M cq |D) is consistent with Pq(M∗q |D), we propose to minimize
L2(q) = ∣∣∣∣Pq(M cq |D)Pq(M∗q |D) − 1 ∣∣∣∣ = ∣∣∣∣∣ det(M c>q M cq )det(M∗>q M∗q )Zq − 1 ∣∣∣∣∣ = ∣∣∣∣∣ ( Volcq Vol∗q )2 Zq − 1 ∣∣∣∣∣ = ∣∣R2qZq − 1∣∣ , (4) where Zq = ∑ |M∗|=Nmbq
det(LM∗ (D;θ))∑ |M|=Nmbq det(LM (D;q,θ)) . In particular, det(M∗>q M ∗ q ) has a geometric interpretation
that it is equal to the squared volume spanned by M∗q (Kulesza & Taskar, 2012), denoted as Vol ∗ q, likewise for det(M c>q M c q ) with respect to Vol c q. Then we define Rq = Volcq Vol∗q
as the ratio of the two feature volumes. To avoid computing Zq, we can convert optimizing L2 into minimizing |Rq − 1| equivalently, since both of them mean maximizing q (detailed in Proposition 2 of Appendix C.4).
Putting L1 and L2 together, our method is finally reformulated as max q g(q)
s.t., q ∈ Q, |Rq − 1| < , (5)
where is a small positive number to serve as the threshold of Rq. g(·) : R → N represents the function that outputs the maximum number (i.e., Nmbq ) such that the compressed version of D mb∗ q to a quality q can be stored in the memory buffer. Note that we relax the original restriction of minimizing |Rq − 1| by enforcing |Rq − 1| < , since maximizing Nmbq for L1 and maximizing q for L2 cannot be achieved simultaneously. Of note, Rq is calculated by normalizing the feature volume Volcq with Vol ∗ q, both of which depend on q (i.e., N mb q ). Therefore, can largely mitigate the sensitivity of q to various domains and can be empirically set as a constant value (see Sec. 4.3).
Since the function g(·) in Eq. (5) is highly non-smooth, gradient-based methods are not applicable. Indeed, we solve it by selecting the best candidate in a finite-size set Q. Generally, the candidate values in Q can be equidistantly selected from the range of q, such as [1, 100] for JPEG. More candidate values can determine a proper q more accurately, but the complexity will grow linearly. We found that selecting 5 candidate values is a good choice in our experiments. Once we solve Eq. (5), a good trade-off is achieved by reducing q as much as possible to obtain a larger Nmbq , while keeping the feature volume Volcq similar to Vol ∗ q. This is consistent with our empirical analysis in Fig. 3.

heading: 4.3 VALIDATE OUR METHOD WITH GRID SEARCH RESULTS
text: In essence, the grid search described in Sec. 4.1 can be seen as a naive approach to determine the compression quality, which is similar to selecting other hyperparameters for continual learning (Fig. 4, a). This strategy is to learn a task sequence or sub-sequence using different qualities and choose the best one (Fig. 4, b), which leads to huge extra computation and will be less applicable if the old data cannot be revisited, or the future data cannot be accessed immediately. In contrast, our method described in Sec. 4.2 only needs to calculate the feature volumes of each compressed subset Dmbq and original subset Dmb∗q (Fig. 4, c), without repetitive training.
Now we validate the quality determined by our method with the grid search results, where LUCIR and PODNet achieve the best performance at the JPEG quality of 50 and 75 on ImageNet-sub, respectively (see Fig. 2). We present Rq in each incremental phase and the averaged Rq of all incremental phases for 5-phase split in Fig. 5 and for 10- and 25-phase splits in Appendix Fig.13. Based on the principle in Eq. (5) with = 0.5, it can be clearly seen that 50 and 75 are the qualities chosen for LUCIR and
PODNet, respectively, since they are the smallest qualities that satisfy |Rq − 1| < . Therefore, the quality determined by our method is consistent with the grid search results, but the computational cost is saved by more than 100 times. Interestingly, for each quality q, whether |Rq − 1| < is generally consistent in each incremental phase and the average of all incremental phases. We further explore the scenarios where Rq might be more dynamic in Appendix D.4.

heading: 5 EXPERIMENT
text: In this section, we first evaluate memory replay with data compression in class-incremental learning of large-scale images. Then, we demonstrate the advantages of our proposal in realistic semi-supervised continual learning of large-scale object detection for autonomous driving.2

heading: 5.1 CLASS-INCREMENTAL LEARNING
text: Benchmark: We consider three benchmark datasets of large-scale images for continual learning. CUB-200-2011 (Wah et al., 2011) is a large-scale dataset including 200-class 11,788 colored images of birds with default format of JPG, split as 30 images per class for training while the rest for testing. ImageNet-full (Russakovsky et al., 2015) includes 1000-class large-scale natural images with default format of JPEG. ImageNet-sub (Hou et al., 2019) is a subset derived from ImageNet-full, consisting of randomly selected 100 classes of images. Following Hou et al. (2019), we randomly resize, crop and normalize the images to the size of 224×224, and randomly split a half of the classes as the initial phase while split the rest into 5, 10 and 25 incremental phases. We report the averaged incremental accuracy with single-head evaluation (Chaudhry et al., 2018) in the main text, and further present the averaged forgetting in Appendix E.3.
Implementation: We follow the implementation of representative memory replay approaches (Hou et al., 2019; Douillard et al., 2020) (detailed in Appendix B.1), where we focus on constraining a certain storage space of the memory buffer rather than a certain number of images. The storage space is limited to the equivalent of 20 original images per class, if not specified. We further discuss the effects of different storage space in Fig. 7, a fixed memory budget in Appendix E.4 and less compressed samples in Appendix E.5. For data compression, we apply a naive but commonly-used JPEG (Wallace, 1992) algorithm to compress images to a controllable quality in the range of [1, 100].
Baseline: We evaluate representative memory replay approaches such as LwF (Li & Hoiem, 2017), iCaRL (Rebuffi et al., 2017), BiC (Wu et al., 2019), LUCIR (Hou et al., 2019), Mnemonics (Liu et al., 2020b), TPCIL (Tao et al., 2020a), PODNet (Douillard et al., 2020), DDE (Hu et al., 2021) and AANets (Liu et al., 2021a). In particular, AANets and DDE are the recent strong approaches implemented on the backbones of LUCIR and PODNet, so we also implement ours on the two backbones. Since both AANets and DDE only release their official implementation on LUCIR, we further reproduce LUCIR w/ AANets and LUCIR w/ DDE for fair comparison.
Accuracy: We summarize the performance of the above baselines and memory replay with data compression (ours) in Table 1. Using the same extra storage space, ours achieves comparable or better performance than AANets and DDE on the same backbone approaches, and can further boost their performance by a large margin. The improvement from ours is due to mitigating the averaged forgetting, detailed in Appendix E.3. For CUB-200-2011 with only a few training samples, all old data can be saved in the memory buffer with a high JPEG quality of 94. In contrast, for ImageNet-sub/-full, only a part of old training samples can be selected, compressed and saved in the memory buffer,
2All experiments are averaged by more than three runs with different random seeds.
where our method can efficiently determine the compression quality for LUCIR and PODNet (see Sec. 4.3 and Appendix D.2), and for AANets and DDE (see Appendix D.3).
Computational Cost: Limiting the size of memory buffer is not only to save its storage, but also to save the extra computation of learning all old training samples again. Here we evaluate the computational cost of AANets, DDE and memory replay with data compression (ours) on LUCIR (see Fig. 6 for ImageNet-sub and Appendix Table 3 for CUB-200-2011). Both AANets and DDE require a huge amount of extra computation to improve the performance of continual learning, which is generally several
times that of the backbone approach. In contrast, ours achieves competing or more performance improvement but only slightly increases the computational cost.
Storage Space: The impact of storage space is evaluated in Fig. 7. Limiting the storage space to equivalent of 10, 20, 40 and 80 original images per class, ours can improve the performance of LUCIR and PODNet by a similar margin, where the improvement is generally more significant for
more splits of incremental phases. Further, ours can achieve consistent performance improvement when using a fixed memory budget (detailed in Appendix E.4), and can largely save the storage cost without performance decrease when storing less compressed samples (detailed in Appendix E.5).

heading: 5.2 LARGE-SCALE OBJECT DETECTION
text: The advantages of memory replay with data compression are more significant in realistic scenarios such as autonomous driving, where the incremental data are extremely large-scale with huge storage cost. Here we evaluate continual learning on SODA10M (Han et al., 2021), a large-scale object detection benchmark for autonomous driving. SODA10M contains 10M unlabeled images and 20K labeled images of size 1920× 1080 with default format of JPEG, which is much larger than the scale of ImageNet. The labeled images are split into 5K, 5K and
10K for training, validation and testing, respectively, annotating 6 classes of road objects for detection.
Since the autonomous driving data are typically partially-labeled, we follow the semi-supervised continual learning (SSCL) proposed by Wang et al. (2021a) for object detection. Specifically, we randomly split 5 incremental phases containing 1K labeled data and 20K unlabeled data per phase, and use a memory buffer to replay labeled data. The storage space is limited to the equivalent of 100 original images per phase. Following Han et al. (2021), we consider Pseudo Labeling
and Unbiased Teacher (Liu et al., 2021b) for semi-supervised object detection (the implementation is detailed in Appendix B.2). Using the method described in Sec. 4.2 with the same threshold (i.e., = 0.5), we can efficiently determine the compression quality for SSCL of object detection, and validate that the determined quality indeed achieves the best performance in grid search (see Fig. 8). Then, compared with memory replay of original data, our proposal can generally achieve several times of the performance improvement on finetuning, as shown in Table 2.

heading: 6 CONCLUSION
text: In this work, we propose that using data compression with a properly selected compression quality can largely improve the efficacy of memory replay by saving more compressed data in a limited storage space. To efficiently determine the compression quality, we provide a novel method based on determinantal point processes (DPPs) to avoid repetitive training, and validate our method in both class-incremental learning and semi-supervised continual learning of object detection. Our work not only provides an important yet under-explored baseline, but also opens up a promising new avenue for continual learning. Further work could develop adaptive compression algorithms for incremental data to improve the compression rate, or propose new regularization methods to constrain the distribution changes caused by data compression. Meanwhile, the theoretical analysis based on DPPs can be used as a general framework to integrate optimizable variables in memory replay, such as the strategy of selecting prototypes. In addition, our work suggests how to save a batch of training data in a limited storage space to best describe its distribution, which will motivate broader applications in the fields of data compression and data selection.

heading: ACKNOWLEDGEMENTS
text: This work was supported by NSF of China Projects (Nos. 62061136001, 61620106010, 62076145, U19B2034, U181146), Beijing NSF Project (No. JQ19016), Beijing Outstanding Young Scientist Program NO. BJJWZYJH012019100020098, Tsinghua-Peking Center for Life Sciences, Beijing Academy of Artificial Intelligence (BAAI), Tsinghua-Huawei Joint Research Program, a grant from Tsinghua Institute for Guo Qiang, and the NVIDIA NVAIL Program with GPU/DGX Acceleration, Major Innovation & Planning Interdisciplinary Platform for the “Double-First Class” Initiative, Renmin University of China.

heading: ETHIC STATEMENT
text: This work presents memory replay with data compression for continual learning. It can be used to reduce the storage requirement in memory replay and thus may facilitate large-scale applications of such methods to real-world problems (e.g., autonomous driving). As a fundamental research in machine learning, the negative consequences in the current stage are not obvious.

heading: REPRODUCIBILITY STATEMENT
text: We ensure the reproducibility of our paper from three aspects. (1) Experiment: The implementation of our experiment described in Sec. 4.1, Sec. 4.3 and Sec. 5 is further detailed in Appendix B. (2) Code: Our code is included in supplementary materials. (3) Theory and Method: A complete proof of the theoretical results described in Sec. 4.2 is included in Appendix C.

heading: A TRADE-OFF BETWEEN QUALITY AND QUANTITY
text: Given a limited storage space, there is an intuitive trade-off between the quality q and quantity Nmbq of compressed data. Here we plot the relation of q and Nmbq for ImageNet and SODA10M used in our paper. As shown in Fig. 9, reducing the quality enables to save more compressed data in the memory buffer, and vice versa. Then, we present compressed images with various qualities in Fig. 10. Although reducing the quality tends to distort the compressed data and thus hurts the performance of continual learning, it can be clearly seen that the semantic information of such compressed data is hardly affected. There-
fore, a potential follow-up work is to regularize the differences between compressed data and original data, so as to further improve the performance of memory replay.
B IMPLEMENTATION DETAIL

heading: B.1 CLASS-INCREMENTAL LEARNING
text: Following the implementation of Hou et al. (2019); Douillard et al. (2020); Tao et al. (2020a); Liu et al. (2021a); Hu et al. (2021), we train a ResNet-18 architecture for 90 epochs, with minibatch size of 128 and weight decay of 1× 10−4. We use a SGD optimizer with initial learning rate of 0.1, momentum of 0.9 and cosine annealing scheduling. We run each baseline with one Tesla V100, and measure the computational cost. We evaluate the performance of class-incremental learning with the averaged incremental accuracy (AIC) (Hou et al., 2019). After learning each phase, we can calculate the test accuracy on all classes learned so far in the previous t phases as At. Then we can calculate AICt = 1 t ∑t i=1 Ai.
To evaluate the effects of class similarity on the determined quality of data compression, we select similar or dissimilar superclasses from ImageNet, which includes 1000 classes ranked by their semantic similarity. Specifically, we select 10 adjacent classes as a superclass, and construct 10 adjacent (similar) or remote (dissimilar) superclasses. The index of classes selected in 10 similar superclasses is [1, 2, 3, ..., 100], while in 10 dissimilar superclasses is [1, 2, ..., 10, 101, 102, ..., 110, ..., 910].

heading: B.2 LARGE-SCALE OBJECT DETECTION
text: For memory replay, we follow Wang et al. (2021a) to randomly select the old labeled data and save them in the memory buffer, which indeed achieves competitive performance as analyzed by Chaudhry et al. (2018). We follow the implementation of Han et al. (2021) for semi-supervised object detection. Specifically, we use Faster-RCNN (Ren et al., 2015) with FPN (Lin et al., 2017) and ResNet-50 backbone as the object detection network for Pseudo Labeling (Han et al., 2021) and Unbiased Teacher (Liu et al., 2021b). For each incremental phase, we train the network for 10K iterations using the SGD optimizer with initial learning rate of 0.01, momentum of 0.9, and constant learning rate scheduler. The batch size of supervised and unsupervised data are both 16 images. For Pseudo Labeling, we first train a supervised model on the labeled set in the first 2K iterations. Then we evaluate the supervised model on the unlabeled set. A bounding box with a predicted score larger than 0.7 in the evaluation results is selected as a pseudo label, which would be used to train the supervised model in the later 8K iterations in each phase. For Unbiased Teacher, we follow the default setting and change the input size to comply with SODA10M. We run each baseline with 8 Tesla V100. The performance is evaluated by the metrics (AP, AP50, AP75) used in the prior works (Liu et al., 2021b; Han et al., 2021).

heading: C THEORETICAL ANALYSIS

heading: C.1 DETERMINANTAL POINT PROCESSES (DPPS) PRELIMINARIES
text: Arising in quantum physics and random matrix theory, determinantal point processes (DPPs) are elegant probabilistic models of global, negative correlations, and offer efficient algorithms for sampling, marginalization, conditioning, and other inference tasks.
A point process P on a ground set V is a probability measure on the power set 2N , where N = |V| is the size of the ground set. Let B be a D ×N matrix with D ≥ N . In practice, the columns of B are vectors representing items in the set V . Denote the columns of B by Bi for i = 1, 2, · · · , N . A sample from P might be the empty set, the entirety of B, or anything in between. P is called a determinantal point process if, given a random subset Y drawn according to P (2N possible instantiations for Y ), we have for every S ⊆ B,
P(Y = S) ∝ det(LS), (6)
where S = {Bi1 , Bi2 , · · · , Bi|S|}, P(Y = S) characterizes the likelihood of selecting the subset S from B. For some symmetric similarity kernel L ∈ RN×N (e.g., L = BTB), where LS is the similarity kernel of subset S (e.g., L = STS). That is, LS is the submatrix sampled from L using indices from S. Since P is a probability measure, all principal minors det(LS) of L must
be non-negative, and thus L itself must be positive semidefinite. These requirements turn out to be sufficient: any L, 0 L I , defines a DPP. L is often referred to as the marginal kernel since it contains all the information needed to compute the probability of any subset S being included in Y . Hence, the marginal probability of including one element Bi is P(Bi ∈ Y ) = Lii, and two elements Bi and Bj is P(Bi, Bj ∈ Y ) = LiiLjj −L2ij = P(Bi ∈ Y )P(Bj ∈ Y ) − L2ij . A large value of Lij reduces the likelihood of both elements to appear together in a diverse subset. This demonstrates why DPPs are “diversifying”. Below is several important conclusions about DPPs related to our work (Kulesza & Taskar, 2012).
Geometric Interpretation. By focusing on L-ensembles in DPPs, determinants have an intuitive geometric interpretation. If L = B>B, then
P(Y = S) ∝ det(LS) = (Vol(S))2, (7) where the first two terms specify the marginal probabilities for every possible subset S, and the last term is the squared |S|-dimensional volume of the parallelepiped spanned by the items in S.
Quality vs. Diversity. While the entries of the DPP kernel L are not totally opaque in that they can be seen as measures of similarity— reflecting our primary qualitative characterization of DPPs as diversifying processes. In most practical situations, we want diversity to be balanced against some underlying preferences for different items in B. Kulesza & Taskar (2012) proposed a decomposition of the DPP that more directly illustrates the tension between diversity and a per-item measure of quality. Specifically, L = B>B can be taken one step further, by writing each column Bi as the product of a quality term qi ≥ 0 and vector of normalized diversity features fi ∈ RD, ‖fi‖ = 1. Then, Lij = qifTi fjqj , where f > i fj ∈ [−1, 1] is a signed measure of similarity between Bi and Bj . Let φij = f>i fj = Lij√ LiiLjj , and φ = {φij}Ni,j=1. Then, depending on Eq. (6), we have
P(Y = S) ∝  ∏ Bij∈S q2ij  det(φS), (8) where qij can be seen as a quality score of an item Bij in B, and φS is the submatrix sampled from φ using indices from S.
Alternative Likelihood Formulas. In an L-ensemble DPP, the likelihood of a particular set S ⊆ B is given by
PL(S) = det(LS)∑ S′ det (LS′) = det(LS) det(L+ I) , (9)
where S′ is one of all 2N possible subsets in B, and LS′ is the submatrix sampled from L using indices from S′.
This expression has some nice intuitive properties in terms of volumes, and, ignoring the normalization in the denominator, takes a simple and concise form. However, as a ratio of determinants on matrices of differing dimension, it may not always be analytically convenient. Minors can be difficult to reason about directly, and ratios complicate calculations like derivatives. Such a high computational cost is just one key challenge in DPPs.
Conditional DPPs. A conditional DPP P(Y = S|B) is a conditional probabilistic model which assigns a probability to every possible subset S ⊆ B. The model takes the form of an L-ensemble:
P(Y = S|B) ∝ det(LS(B)), (10) where L(B) is a positive semidefinite conditional N ×N kernel matrix that depends on the input B.
k-DPPs. A k-DPP is obtained by conditioning a standard DPP on the event that the set Y has cardinality k, which is concerned only with the content of a random k-set (i.e., without the size k) (Kulesza & Taskar, 2012). Formally, the k-DPP PkL gives the probability of a particular set S as
PkL(Y = S) = det(LS)∑
|S′|=k det(LS′) , (11)
where |S| = k and L is a positive semidefinite kernel. Although numerous optimization algorithms have been proposed to solve DPPs problems, the high computational complexity about denominator cannot be avoided generally. On the one hand, computing denominator is a sum over ( N k ) terms. On the other hand, computing the determinant of each term through matrix decomposition is with O(k3) time. Just as Kulesza & Taskar (2012) claimed, sparse storage of larger matrices is possible for DPPs, but computing determinants remains prohibitively expensive unless the level of sparsity is extreme.
If the dot product kernel function is adopted to compute the kernel L in Eq. (11), similar to Geometric Interpretation of standard DPPs above, the probability PkL(Y = S) is further defined with volume sampling (Deshpande & Rademacher, 2010), i.e.,
PkL(Y = S) = det(LS)∑
|S′|=k det(LS′) ∝ det(LS) = (k! · (Vol(Conv(0̄ ∪ S)))2), (12)
where L = B>B, Conv(·) denotes the convex hull, and Vol(·) is the k-dimensional volume of such a convex hull.
C.2 MODELING OUR CASE AS A CONDITIONAL Nmbq -DPP
In this work, we aim to find the compressed subset Dmbq that can best represent the training dataset D by choosing an appropriate compression quality q from its range. To achieve this goal, we introduce Pq(Dmbq |D) to characterize the conditional likelihood of selecting Dmbq given input D under parameter q. Since the network parameter θ is fixed during compression, and the feature matrix M cq = F e θ (D mb q ), we rewrite Pq(Dmbq |D) as Pq(M cq |D) equivalently (see Fig. 11 for the details of constructing Dmbq and M c q ). Depending on the nice properties of DPPs in Sec. C.1, we formulate our goal as a conditional DPP, where D is the associated ground set, and M cq is a desired subset in feature space. Then Pq(M cq |D) is a distribution over all subsets of D with cardinality Nmbq , which is formally called a conditional Nmbq -DPP, since |M cq | = Nmbq for any possible subset. Thus, such a DPP formulates the conditional probability Pq(M cq |D) as
Pq(M cq |D) = det(LMcq (D; q, θ))∑
|M |=Nmbq det(LM (D; q, θ))
, (13)
where L(D; q, θ) is a conditional DPP |D| × |D| kernel matrix that depends on the input D parameterized in terms of parameters θ and q. LM (D; q, θ) (resp., LMcq (D; q, θ))) is the submatrix sampled from L(D; q, θ) using indices from M (resp., M cq ).
Similarly, Pq(M∗q |D) can be formulated by a conditional Nmbq -DPP as
Pq(M∗q |D) = det(LM∗q (D; θ))∑
|M∗|=Nmbq det(LM∗(D; θ))
, (14)
where |M∗q | = Nmbq . Unlike L(D; q, θ) above, the conditional DPP kernel matrix L(D; θ) only depends on D and θ without q, since q is fixed at its maximum, i.e., without compression. LM∗(D; θ) (resp., LM∗q (D; θ)) is the submatrix sampled from L(D; θ) using indices from M ∗ (resp., M∗q ).
Differences: Of note, two important differences between standard k-DPPs and our Nmbq -DPP lie in the optimization variable and objective. For this work, we need to find an optimal set cardinality (i.e., Nmbq ), while it is fixed for standard k-DPPs. Although both of them can finally determine a desired subset with the maximum volume, our Nmbq -DPP can uniquely determine it by N mb q , while standard k-DPPs obtain it by maximizing Eq. (11).

heading: C.3 THE FIRST GOAL
text: First, we need to maximize
L1(q) = Pq(M∗q |D) = det(LM∗q (D; θ))∑
|M∗|=Nmbq det(LM∗(D; θ))
∝ det(M∗>q M∗q ) = (Vol ∗ q) 2 = (Nmbq ! · (Vol(Conv(0̄ ∪M∗q)))2),
(15)
where |M∗q | = Nmbq . L(D; θ) is a conditional DPP |D| × |D| kernel matrix that depends on the input D parameterized in terms of θ. LM∗(D; θ) (resp., LM∗q (D; θ)) is the submatrix sampled from L(D; θ) using indices from M∗ (resp., M∗q ). The numerator defines the marginal probability of inclusion for the subset M∗q , and the denominator serves as a normalizer to enforce the sum of Pq(M∗q |D) for every possible M∗q to 1. Here we employ the commonly-used dot product kernel function, where LM∗q = M ∗> q M ∗ q and LM∗ = M
∗>M∗. Conv(·) denotes the convex hull, and Vol(·) is the Nmbq -dimensional volume of such a convex hull.
Eq. (15) provides a geometric interpretation that the conditional probabilityPq(M∗q |D) is proportional to the squared volume of M∗q with respect to N mb q , denoted as Vol ∗ q. For our task, Pq(M∗q |D) monotonically increases with Nmbq . Thus, optimizing L1 is converted into maxq Nmbq equivalently (detailed in Proposition 1 as below).
Proposition 1 For any matrix X = (x1,x2, · · · ,xn), where xi ∈ Rd is the i-th column of X , d > n and X>X 6= I , defineM = {i1, i2, · · · , im} as a subset of [n] containing m elements, and XM as the submatrix sampled from X using indices fromM. Then for
P (M) := det ( X>MXM )∑ |M′|=m det ( X>M′XM′
) , we can ensure that the following two optimization programs
max m
P (M)
and
max m |M|
are equivalent.
Below is one critical lemma for the proof of Proposition 1.
Lemma 1 For any matrix X = (x1,x2, · · · ,xn), where xi ∈ Rd is the i-th column of X and d > n, define M = {i1, i2, · · · , im} as a subset of [n] containing m elements, and XM as the submatrix sampled from X using indices fromM. Then for
P (M) := det ( X>MXM )∑ |M′|=m det ( X>M′XM′
) , we can ensure that
[n] = argmaxMP (M). This means when |M| = n, the probability P (M) takes the maximum value.
Proof. From the definition, we have 0 ≤ P (M) = det ( X>MXM )∑ |M′|=m det ( X>M′XM′
) = det ( X>MXM
) det ( X>MXM ) + ∑ {|M′|=m}∩{M′ 6=M} det ( X>M′XM′ ) ≤ 1.
If |M′| = m < n, we have ( n m ) > 1 possible options for subset XM′ , where det ( X>M′XM′ ) ≥ 0.
Only if |M| = n, then ( n m ) = 1 and the probability P (M) will definitely take the maximum value, i.e. P (M) = 1. In addition, X>X has full rank due to d > n, and then det(X>X) > 0 always holds. That is, P (M) = 1 can be guaranteed only when selecting all n items together. This means [n] = argmaxMP (M). It completes the proof. Proof of Proposition 1 From the definition in Deshpande & Rademacher (2010), we have
P (M) = det ( X>MXM )∑ |M′|=m det ( X>M′XM′
) ∝ det ( X>MXM
) = (m! · (Vol(Conv(0̄ ∪XM)))2),
(16)
where Conv(·) denotes the convex hull, and Vol(·) is the m-dimensional volume of such a convex hull. Here, from the monotonicity side, it can be easily verified that for XM ∪ xim+1 , where xim+1 ∈ {XXM, we have
(m! · (Vol(Conv(0̄ ∪XM∗)))2) < ((m + 1)! · (Vol(Conv(0̄ ∪XM ∪ xim+1)))2),
which means P (M) is monotonically increasing with the increase of |M|. From the extremum side, by using Lemma 1, we can ensure that the following two optimization programs
max m
P (M)
and
max m |M|
have the same optimal value. Thus, maximizing P (M) with respect to |M| can be converted into maxm |M| equivalently.

heading: C.4 THE SECOND GOAL
text: Second, we need to minimize L2(q) = ∣∣∣∣Pq(M cq |D)Pq(M∗q |D) − 1 ∣∣∣∣ = ∣∣∣∣∣det(LMcq (D; q, θ))det(LM∗q (D; θ)) Zq − 1 ∣∣∣∣∣ =
∣∣∣∣∣ det(M c>q M cq )det(M∗>q M∗q )Zq − 1 ∣∣∣∣∣ = ∣∣∣∣∣ ( Volcq Vol∗q )2 Zq − 1 ∣∣∣∣∣ = ∣∣R2qZq − 1∣∣ , (17)
where Zq = ∑ |M∗|=Nmbq
det(LM∗ (D;θ))∑ |M|=Nmbq det(LM (D;q,θ)) . In particular, det(M∗>q M ∗ q ) has a geometric interpretation
that it is equal to the squared volume spanned by M∗q (Kulesza & Taskar, 2012), denoted as Vol ∗ q, likewise for det(M c>q M c q ) with respect to Vol c q. Then we define Rq = Volcq Vol∗q
as the ratio of the two feature volumes. To avoid computing Zq, we can convert optimizing L2 into minimizing |Rq − 1| equivalently, since both of them mean maximizing q (detailed in Proposition 2 as below).
Below we are going to introduce Proposition 2. We first introduce some necessary assumptions, then we present the theoretical guarantee of Proposition 2.
Assumption 1 We make the following assumptions:
1. Image dataset: For any image dataset D = (x1, x2, · · · , xn), where xi is the i-th image in D, define Dq = (x1, x2, · · · , xn) as a compressed image dataset with the compression quality q for D. The compression quality is bounded as q ∈ Q (Q can be a finite/infinite set), in which the higher of q, the better of image quality.
2. Feature matrix: Denote X = (f1,f2, · · · ,fn) ∈ Rd×n and Xq = (f q1 ,f q 2 , · · · ,f qn) ∈
Rd×n as the feature matrices ofD andDq with the same fixed feature extractor, respectively. Here, d > n, X>X 6= I and X>q Xq 6= I .
3. Selection principle: DefineM = {i1, i2, · · · , im} as a subset of [n] containingm elements, and XM as the submatrix sampled from X using indices fromM. Likewise, defineMq = {j1, j2, · · · , jm} as a subset of [n] containing m elements, and XMq as the submatrix sampled from Xq using indices fromMq . Both of them use the same selection principle.
4. Buffer storage space: g(·) : R → N represents the function that outputs the maximum number (i.e. m ) such that the compressed version of Dq to a quality q can be stored in the memory buffer. That is, m is uniquely determined by q.
Proposition 2 Under Assumption 1, for two probabilities P (M) := det ( X>MXM )∑ |M′|=m det ( X>M′XM′
) and
P (Mq) := det ( X>MqXMq ) ∑ |M̃|=m det ( X> M̃ XM̃
) , we can ensure that the following two optimization programs
min q ∣∣∣∣∣∣ det ( X>MqXMq ) det ( X>MXM ) − 1 ∣∣∣∣∣∣
and
min q ∣∣∣∣P (Mq)P (M) − 1 ∣∣∣∣
are equivalent.
Below are two critical lemmas for the proof of Proposition 2.
Lemma 2 Under Assumption 1, for a ratio Rq := det ( X>MqXMq ) det ( X>MXM
) , we can ensure that
supQ = argminq |Rq − 1|.
This means when q = supQ (i.e, without image compression), |Rq − 1| takes the minimum value.
Proof. From the definition, we know |Rq − 1| ≥ 0. To minimize |Rq − 1|, we need the ratio Rq = 1, i.e.,
det ( X>MqXMq ) = det ( X>MXM ) . (18)
Of note, both XMq and XM are selected from the source dataset D with the same feature extractor, selection principle and set cardinality (i.e., m). That is, the only difference between them is image compression with respect to q.
By recurring to the Geometric Interpretation in k-DPPs (i.e., Eq. (12)), we have det ( X>MqXMq ) = (m! · (Vol(Conv(0̄ ∪XMq)))2)
and
det ( X>MXM ) = (m! · (Vol(Conv(0̄ ∪XM)))2).
Then, Eq. (18) holds only when XMq and XM have the same convex hull. Due to great difficulty of analytically defining compression function, the two convex hull cannot be mathematically given. However, by conducting extensive experiments on this task (as shown in Fig. 3, Fig. 5, Fig. 8, Fig. 13 and Fig. 14), we find that the volume of XMq is larger than that of XM for a specific m (i.e., q). Additionally, with q decreasing (i.e., m increasing), the volume of XMq is increasing more quickly. Thus, we have an empirical conclusion that the two volumes are the same only when q = supQ (without image compression). It is reasonable since without image compression, the two selection problems about XMq and XM are identical. This means when q = supQ, |Rq − 1| takes the minimum value, which completes the proof.
Lemma 3 Under Assumption 1, for P (M) := det ( X>MXM )∑ |M′|=m det ( X>M′XM′
) and
P (Mq) := det ( X>MqXMq ) ∑ |M̃|=m det ( X> M̃ XM̃
) , we can ensure that
supQ = argminq ∣∣∣∣P (Mq)P (M) − 1 ∣∣∣∣ .
This means when q = supQ (i.e, without image compression), ∣∣∣P (Mq)P (M) − 1∣∣∣ takes the minimum value.
Proof. From the definition, we know ∣∣∣P (Mq)P (M) − 1∣∣∣ ≥ 0. To minimize ∣∣∣P (Mq)P (M) − 1∣∣∣, we need the ratio P (Mq) P (M) = 1. In fact,
P (Mq) P (M)
= det ( X>MqXMq ) det ( X>MXM ) · ∑|M′|=m det (X>M′XM′)∑ |M̃|=m det ( X> M̃ XM̃
) = Rq · Zq, (19) where Rq = det ( X>MqXMq ) det(X>MXM) and Zq = ∑ |M′|=m det(X > M′XM′)∑ |M̃|=m det ( X> M̃ XM̃
) . By using Lemma 2, we know that when q = supQ, Rq = 1 always holds. More importantly, Zq = 1 also holds if q = supQ, since the numerator and denominator in Zq will be equal without
compression. That is, when q = supQ, the ratio P (Mq)P (M) = 1 and ∣∣∣P (Mq)P (M) − 1∣∣∣ takes the minimum value. This completes the proof.
Proof of Proposition 2 From Lemma 2 and Lemma 3, we can ensure that under Assumption 1, the following two optimization programs
min q ∣∣∣∣∣∣ det ( X>MqXMq ) det ( X>MXM ) − 1 ∣∣∣∣∣∣
and
min q ∣∣∣∣P (Mq)P (M) − 1 ∣∣∣∣
have the same optimal value. Thus, maximizing ∣∣∣P (Mq)P (M) − 1∣∣∣ with respect to q can be converted into
minq |Rq − 1| equivalently, where Rq = det ( X>MqXMq ) det(X>MXM) .

heading: D EMPIRICAL ANALYSIS

heading: D.1 T-SNE VISUALIZATION OF NORMALIZED FEATURES
text: To provide an empirical analysis of the quality-quantity trade-off and validate the theoretical interpretation, we use t-SNE (Van der Maaten & Hinton, 2008) to visualize features of all training data, the compressed subset Mq, and the original subset M∗q . First, with the increase of quantity N mb q , the area of original subset is expanded and can better cover the training data distribution, as shown in Fig. 12. This result is consistent with maximizing Nmbq for L1. Second, with the decrease of quality q and increase of quantity Nmbq , the compressed data tend to be distorted and thus become out-of-distribution, which has been discussed in the main text Fig. 3. This result is consistent with enforcing |Rq − 1| < for L2.
D.2 Rq FOR 5-, 10- AND 25-PHASE IMAGENET-SUB
We presentRq in each incremental phase and the averagedRq of all incremental phases for 5-, 10- and 25-phase ImageNet-sub in Fig.13. Based on the principle in Eq. (5) (we set = 0.5 as the threshold of Rq), it can be clearly seen that 50 and 75 is a good quality for LUCIR and PODNet, respectively. Also, the determined quality is the same for different numbers of splits, which is consistent with the results of grid search in Fig. 2.

heading: D.3 AANETS AND DDE
text: Similar to LUCIR and PODNet, we apply the method described in Sec. 4.2 to determine the compression quality for AANets and DDE. Since both AANets and DDE only release their official implementation on LUCIR, here we focus on LUCIR w/ AANets and LUCIR w/ DDE. We present Rq in each incremental phase and the averaged Rq of all incremental phases for 5-phase ImageNetsub in Fig. 14. The determined qualities for both LUCIR w/ AANets and LUCIR w/ DDE are consistent in different incremental phases, and are the same as that of LUCIR.

heading: D.4 STATIC VS DYNAMIC QUALITY
text: For ImageNet-sub with randomly-split classes, whether |Rq − 1| < of each quality q is consistent among incremental phases and their average (see Fig. 13 and Fig. 14). Thus, the determined quality by our method usually serves as a static hyperparameter in continual learning, which has been validated by the results of gird search (see Fig. 2). Now, we wonder in which case the determined quality is dynamic, and whether using a dynamic quality is better than using a static one. Intuitively, the determined quality might be affected by the similarity of incremental classes, because the model learned to predict similar classes might be more sensitive to the image distortion caused by data compression.
Since the ImageNet classes are ranked by their semantic similarity, we select 10 adjacent classes as a superclass, and construct 10 adjacent (similar) or remote (dissimilar) superclasses (detailed in Appendix B.1). Similar to the 5-phase ImageNet-sub, we first learn 5 superlcasses as the initial phase, and then incrementally learn one superclass per phase with LUCIR. The results are presented in Fig. 15. For dissimilar superclasses, whether |Rq − 1| < is consistent in each incremental phase and their average, so the determined quality is stable at 50, validated by the results of grid search. For similar superclasses, the quality determined by the averaged Rq is 75, where using a stable quality of 75 in-
deed achieves the best performance in grid search. On the other hand, it can be clearly seen that |R75 − 1| > before phase 3 while |R75 − 1| < after it. Then we use the dynamically-determined quantity and quantity for each incremental phase. However, the dynamic quality and quantity result in severe data imbalance, so the performance is far lower than using a static quality of either 50, 75 or 90. A promising further work is to alleviate the data imbalance for the scenarios where the determined quality is highly dynamic.

heading: E ADDITIONAL RESULTS

heading: E.1 COMPUTATIONAL COST
text: In Table 3, we present the detailed results of computational cost and averaged incremental accuracy of LUCIR, LUCIR w/ AANets, LUCIR w/ DDE and LUCIR w/ Ours for CUB-200-2011 and ImageNet-sub.

heading: E.2 STORAGE SPACE
text: In Table 4, we present the detailed results of different storage space.

heading: E.3 AVERAGED FORGETTING
text: In addition to averaged incremental accuracy, we evaluate the averaged forgetting, which is calculated by averaging the test accuracy of each task minus its highest test accuracy achieved in continual learning (Caccia et al., 2020). In Table 5, we present the averaged forgetting of classes in the initial phase, which suffers from the most severe forgetting. It can be clearly seen that the averaged forgetting is largely alleviated by ours on each backbone.

heading: E.4 FIXED MEMORY BUDGET
text: Here we evaluate memory replay with data compression (ours) under a fixed memory budget (i.e., storage space) on ImageNet-sub. Following a widely-used setting (Rebuffi et al., 2017; Wu et al., 2019), the memory budget (i.e., storage space) is limited to equivalent of 2000 original images (20 original images per class × a total of 100 classes in ImageNet-sub). We further evaluate a much smaller memory budget, fixed to 1000 original images. Under such a fixed memory budget, ours substantially boosts the performance of each backbone approach as shown in Table 6.

heading: E.5 LESS COMPRESSED SAMPLES
text: In Table 7, we present the results of LUCIR with different numbers of compressed samples on ImageNet-sub. Similar to the main text, we select the JPEG quality of 50 for LUCIR, where the storage space of 20 original images can save 85 such compressed images. Memory replay of 85 compressed images of quality 50 achieves a much better performance than that of 20 original images. When reducing the quantity from 85 to 70, 55 and 40, the accuracy will also decline. However, memory replay of 40 compressed images of quality 50 still outperforms that of 20 original images, where the average memory can be saved by 52.94%.

Review summaries:
This paper studies the problems of classification and object detection from natural image datasets in a continual learning setting, whereby the different classes/objects to be learned are not observed together but sequentially. It is assumed that a memory buffer of a certain size is available to store data in. The strategy used by the paper is to fill this memory buffer with compressed data samples (with JPEG) rather than the original images. Such compression introduces a quantity-quality trade-off, which this paper empirically analyses. Additionally, the paper proposes an automated way to select the amount of data compression based on determinantal point processes.

In this work, the authors propose memory replay with data compression, which is both an important yet neglected baseline and a promising direction for continual learning. Using a naive technique of data compression with a properly selected quality, the proposed method can achieve the SOTA performance in a time-efficient and plug-and-play way. Since the compression quality is highly nontrivial for the efficacy of memory replay, the authors provide a novel method based on determinantal point processes (DPPs) to determine it efficiently, and validate their method in both class-incremental learning and semi-supervised continual learning of object detection.

In this paper, the problem of catastrophic forgetting for continual learning is explored. The core idea is to benefit from data compression to reduce the space to store data samples and then replay the reconstructed data points that are built using the compressed versions. JPEC method has been explored empirically to determine an optimal compression rate through solving an optimization problem. Experiments on three benchmark datasets are performed to demonstrate that the method is effective.

In this paper, the authors propose a data compression method for memory-replay based continual learning algorithms. With the data compression method,  more old training samples can be stored in the memory to better capture old data distribution. However, there is a trade-off between the quality and quantity of compressed data, the authors propose to use determinantal point processes to determine the quality of the data compression.  Extensive experiments show that with the proposed method, a naive compression method can achieve the SOTA on several continual learning benchmarks.