Memory Replay with Data Compression for Continual Learning
Authors: Liyuan Wang
Xingxing Zhang
Kuo Yang
Longhui Yu
Chongxuan Li
Lanqing HONG
Shifeng Zhang
Zhenguo Li
Yi Zhong
Jun Zhu
Venue: ICLR 2022 Poster (2022)
Keywords: Continual Learning
Memory Replay
Data Compression

Abstract: Continual learning needs to overcome catastrophic forgetting of the past. Memory replay of representative old training samples has been shown as an effective solution, and achieves the state-of-the-art (SOTA) performance. However, existing work is mainly built on a small memory buffer containing a few original data, which cannot fully characterize the old data distribution. In this work, we propose memory replay with data compression to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer. Observing that the trade-off between the quality and quantity of compressed data is highly nontrivial for the efficacy of memory replay, we propose a novel method based on determinantal point processes (DPPs) to efficiently determine an appropriate compression quality for currently-arrived training samples. In this way, using a naive data compression algorithm with a properly selected quality can largely boost recent strong baselines by saving more compressed data in a limited storage space. We extensively validate this across several benchmarks of class-incremental learning and in a realistic scenario of object detection for autonomous driving.

Published as a conference paper at ICLR 2022
MEMORY REPLAY WITH DATA COMPRESSION FOR CONTINUAL LEARNING
Liyuan Wang!
Xingxing Zhang!
Kuo Yang
Longhui Yu
Chongxuan Li
Lanqing H
Shifeng Zhang
Zhenguo Li
Yi Zhong 2.t
Jun Zhu
1 Dept. of Comp. Sci. & Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua University Schooo of Life Sciences, IDG/McGovern Institute for Brain Research, Tsinghua University "Tsinghua-Pekinn Center for Life Sciences *Gaoling School of AI, Renmin University of China 'Beijing Key Laboratory of Big Data Management and Analysis Methods Huawei Noah's Ark Lab
ABSTRACT
Continual learning needs to overcome catastrophic forgetting of the past. Memory replay of representative old training samples has been shown as an effective solution, and achieves the sate-of-the-aad art (SOTA) performance. However, existing work is mainly built on small memory buffer containing a few original data, which cannot fully characterize the old data distribution. In this work, propose memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer. Observing that the trade-off between the quality and quantity of compressed data is highly nontrivial for the efficacy of memory replay, we propose a novel method based on determinantal point processes (DPPs) to efficiently determine appropriate compression quality for currently-arrived training samples. In this way, using naive data compression algorithm with properly selected quality can largely boost recent strong baselines by saving more compressed data a limited storage space. We extensively validate this across several benchmarks of class-incrent incremental learning and in a realistic scenario of object detection for autonomous driving.
INTRODUCTION
The ability to continually learn numerous tasks and infer them together critical for deep neural networks (DNNs), which needs to mitigate catastrophic forgetting (McCloskey & Cohen 1989) of the past. Memory replay of representative old training samples (referred to as memory replay) has been shown effective solution, and achieves the state-of-the-art (SOTA) performance (Hou et al. 2019). Existing memory replay approaches are mainly built on a small memory buffer containing few original data, and try to construct and exploit it more effectively. However, due the low storage efficiency of saving original data, this strategy of building memory buffer will lose a lot of information about the old data distribution. On the other hand, this usually requires huge extra computation to further mitigate catastrophic forgetting, such as by learning additional parameters (iu et al. 202 or distilling old features (Hu et al. 2021).
Different from "artificial" memory replay in DNNs, a significant feature of biological memory is to encode the old experiences highly compressed form and replay them to overcome catastrophic forgetting (McCleland) 2013 Davidson et al. 2009 Carr et al. 2011). Thus the learned information can be maintained in small storage space as comprehensively as possible, and flexibly retrieved. Inspired by the compression feature of biological memory replay, we propose memory replay with data compression (MRDC), which can largely increase the amount of old training samples that can be stored in the memory buffer by reducing their storage cost in a computationally efficient way.
"Equal Contribution
1 "oorrsponndig Authors

---

Published as a conference paper at ICLR 2022
5-Phase
10-Phase
25-Phase
IN DDE
Averageet Accurecy (%6
Averaged Aczurney (%)
Averagen Accuracy
75-
of Oure
13-
O/Oum
$ + 22
wt AAlite
-2-
71-
DOC:Our
w'00e
w noe
63-
100
70
LUCR
67-
63-
LUC
20
40
60
60
20
40
80
50
100
159
Training Time (hour)
Training Time (hour)
Training Time (hour)
Figure 1: Averaged incremental accuracy and training time on ImageNet-sub. Using JPEG for data compression can achieve comparable or better performance than recent strong approaches with less extra computation (purple arrow), and can further improve their performance (gray arrow).
Given a limited storage space, data compression introduces an additional degree of freedom to explicitly balance the quality and quantity for memory replay. With properly selected quality, using a naive JPEG compression algorithm [Wallace 1992) can achieve comparable or better performance than recent strong approaches with less extra computation (Fig. T purple arrow), and further improve their performance (Fig. [ gray artow). However, to empirically determine the compression quality is usually inefficient and impractical, since it requires learning a task sequence or sub- sebueee y [ We propose novel method based on determinantal point processes (DPPs) to efficiently determine without repetitive training. Further, we demonstrate the advantages of our proposal in realistic applications such as continual learning of object detection for autonomous driving, where the incremental data are extremely large-scale.
Our contributions include: (1) We propose memory replay with data compression, which is both an important baseline and a promising direction for continual learning; (2) We empirically validate that the trade-off between quality and quantity of compressed data is highly nontrivial for memory replay, and provide a novel method to efficiently determine it without repetitive training; (3) Extensive experiments show that using naive data compression algorithm with properly selected quality can largely improve memory replay by saving more compressed data in a limited storage space.
RELATED WORK
Continual learning needs to overcome catastrophic forgetting of the past when learning a new task. Regularization-base methods (Kirkpatrick et al. [2117] Wang et al. (222bb) approximated the importance of each parameter to the old tasks and selectively penalized its changes. Architecture- based methods (Rusu et al. 2016) allocated dedicated parameter subspace for each task to prevent mutual interference. Replay-based methods (Rebuffi et al., 2017] [Shin et al.] (017)) approximated and recovered the old data distribution. In particular, memory replay of representative old training samples (referred to as memory replay) can generally achieve the best performance in class-incremental learning (Liu et al. 2021a Hu et al. 202T) and in numerous other continual learning scenarios, such as audio tasks (Ehret et al. 2020), few-shot (Tao et al. 20200), semi-supervised (Wang et al. 2021a).
and unsupervised continual learning (Khare et 20.1).
Most of the work in memory replay attempted to more effectively construct and exploit small memory buffer containing few original data. As the pioneer work, iCaRL (Rebufñ et al.) 2017) proposed general protocol of memory replay for continual learning. To better construct the memory buffer, Mnemonics (Liu et al. 2020b) parameterized the original data and made them optimizable, while TPCIL (Tao et al. 2020a) constructed an elastic Hebbian graph by competitive Hebbian learning. On the other hand, BiC (Wu et al. 2019). LUCIR (Hou et al. (019)). PODNet (Douillard ct al. 2020), DDE (Hu et al. 2021) and A.Nets (Liu et al. 202 attempted to better exploit the memory buffer, such as by mitigating the data imbalance between old and new classes (Hou et al.
2019 Wu et al. 2019 Hu et al. 2021).
In contrast to saving original data, several work attempted to improve the efficiency of remembering the old data distribution. One solution is to continually learn generative model to replay generated data (Shin et al. 2017: Wu et al. 2018) or compress old training data (Caccia et al. 2020). However,
A naive grid search approach is to train continual learning processes with different qualities and choose the best one, resulting in huge computational cost. Also, this strategy will be less applicable if the old data cannot be revisited, or the future data cannot be accessed immediately.
2

---

Published as a conference paper at ICLR 2022
continual learning of such generative model is extremely challenging, which limits its applications to relatively simple domains, and usually requires a lot of extra computation. Another solution is feature replay: GFR (Liu et al.) 2020a) learned a feature generator from feature extractor to replay generated features, but the feature extractor suffered from catastrophic forgetting since it was incrementally updated. REMIND (Hayes al. 2020) saved the old features and reconstructed the synthesized features for replay, but it froze the majority of feature extractor after learning the initial phase, limiting the learning of representations for incremental tasks.
Data compression aims to improve the storage efficiency of file, including lossless compression and lossy compression. Lossless compression needs to perfectly reconstruct the original data from the compressed data, which limits its compression rate (Shannon 1948). In contrast, lossy compression can achieve much higher compression rate by degrading the original data, so has been broadly used in realistic applications. Representative hand-crafted approaches include JPEG (or JPG) (Wallace 1992). which is the most commonly-usee algorithm of lossy compression (Mentzer et al., 2020). WebP (Lian & Shilei 2012) and JPEG2000 (Rabbani 2002). On the other hand, neural compression approaches generally rely on optimizing Shannon's rate-distortion trade-off, through RNNs (Toderici
et al. 2015 2017). auto-encoders (Agustsson et al. 2017) and GANs (Mentzer et al.
2020
3
CONTINUAL LEARNING PRELIMINARIES
We consider a general setting of continual learning that a deep neural network (DNN) incrementally learns numerous tasks from their task-specific training dataset Dr {(ar,, // where D, is only available when learning task t, (Ira. vt.s) is data-label pair and N, is the number of such training samples. For classification tasks, the training samples of each task might be from one or several new classes. All the classes ever seen are evaluated at test time, and the classes from different tasks need to be distinguished. This setting is also called lnasemnacremttall learning (van de Ven & Tolias 2019). Suppose such a DNN with parameter has learned T tasks and attempts to learn a new task. Since the old training datasets U Dr are unavailable, the DNN will adapt the Icarned parameters to fit Dr+1, and tend to catastrophically forget the old tasks McClelland al. (1995).
No
An effective solution of overcoming catastrophic forgetting is to select and store representative old in a small memory buffer (mb), and replay them when learning the new task. For classification tasks, mean-of-feature is a widely-used strategy to select Dr (Rebuffi et al., 2017. Hou et al. 2019). After learning each task, features of the training data can be obtained by the learned embedding function (-). In each class, several data points nearest to the mean-of rran-ooffeeae of-fur are selected into the memory buffer. Then, the training dataset of task T + 1 becomes D/+11 DT+1 Di! I.T, including both new training samples D and some old training samples Dr.b- = U Dimb, so as to prevent forgetting of the old tasks.
training samples Dmb = {Z2,,5 ft,;)) ]=== Nmm
However, due to the limited storage space, only few original data can be saved in the memory buffer, namely, Nrmu < N₁. Although numerous efforts in memory replay attempted to more effectively exploit the memory buffer, such as by alleviating the data imbalance between the old and new classes, this strategy of building memory buffer is less effective for remembering the old data distribution.
4 METHOD
In this section, we first present memory replay with data compression for continual learning. Then, we empirically validate that there is trade-off between the quality and quantity of compressed data, which is highly nontrivial for memory replay, and propose a novel method to determine it efficiently.
4.1 MEMORY REPLAY WITH DATA COMPRESSION
Inspired by the biological memory replay that is in a highly compressed form (Carr et al. 2011 we propose an important baseline for memory replay, that is, using data compression to increase the amount of old training samples that can be stored in the memory buffer, to more effectively recover the old data distribution. Data compression can be generally defined as a function (.).) of compressing the original data ot,; to a. = (2.,)) with a controllable quality Due to the smaller storage cost of each fg than Ft,i i, the memory buffer maintain more old training samples for
3

---

Published as a conference paper at ICLR 2022
(8)
(b)
72-
Avwraged (%)
Accurnec (N)
Soneong Accurens (N)
Soemmpee Acsurncc (%)
73-
12-
Averqqed Accurccy
70-
Averged Accurecy
78-
75-
LUCIR
6o-
PODKet
22-
1o-
n-
73-
%
78-
69
Ax
25
50
100
25
50
75
100
25 50
75
100
50
75
100
23
75
100
25
100
Quality
Quality
Quality
Quality
Quality
Quality
Figure 2: Memory replay with data compression on ImageNet-sub. We make grid search of the JPEG quality {10, 25, 50, 75,90). 90). The quality of 100 refers to the original data without compression.
replay, namely, Ny > Nuwh for Di.t = {(Eg,,.. Vr,s)) No.t in Datt = - Dg... For notation clarity, we will use Dmu to denote Dme without explicitly writing out its task label t, likewise for Vi. yr. Nyrbb Nmm and D. The compression rate can be defined ri N2rb /Nmb Nop.
In analogy to the learning theory for supervised learning, we argue that continual learning will also benefit from replaying more compressed data, assuming that they approximately follow the original data distribution. However, the assumption is likely to be violated if the compression rate is too high. Intuitively, this leads to a trade-off between quality and quantity: if the storage space is limited, reducing the quality q of data compression will increase the quantity N of compressed data that can be stored in the memory buffer, and vice versa.
Here we evaluate the proposed idea by compressing images with JPEG (Wallace 1992). a naive but commonly-used lossy compression algorithm. JPEG save images with quality in the range of [1, 100), where reducing the quality results in smaller file size. Using a memory buffer equivalent to 20 original images per class (Hou et al., 2019), we make a grid search of the JPEG quality with representative memory replay approaches, such as LUCIR (Hou et al. 2019)) and PODNet (Douillard al., 2020). As shown in Fig. [ memory replay of compressed data with an appropriate quality can substantially outperform that of original data. However, whether the quality is too large or too small, will affect the performance. In particular, the quality that achieves the best performance varies with the memory replay approaches, but is consistent for different numbers of splits of incremental phases.
4.2 QUALITY-UAANIIE TRADE-OFF
Since the trade-off between quality and quantity Nyw is highly nontrivial for memory replay, it needs to be carefully determined. Formally, after learning each task from its training dataset D, let's consider several compressed subsets Dmu = {(xgg, v:)) Nm
where is from a set of finite candidates Nmb Q = {91, 92, Q3: }. Each D.mb is constructed by selecting a subset D (xx, vi) of N7mb original training samples from D (following mean-of-feature or other principles). The size Nmm is determined as the maximum number such that the compressed version of Dma to a quality q can be stored the memory buffer (see details in Appendix A). Thereby, a smaller g enables to save a larger Nmb, and vice versa. The objective is to select a compressed subset that can best represent D, namely, to determine an appropriate for memory replay.
To understand the effects of data compression, which depends on the compression function F+() and the continually- embedding function 175()). we focus on analyzing the features of compressed data J. F (F (z:)). We first calculate the feature matrix M;- = Jg.2 ...,, "q, N.. of each compressed subset q where each column vector fq.s is obtained by normalizing fa. under Lz- norm to keep |lf 2l = 1. Similarly, we obtain the feature matrix MG = [f1, F2, .... fxmol] ] of each
original subset Dymbe Then, we can analyze the quanityy trade-off from two aspects:
On the empirical side, in Fig. Feee use t-SNE (Van der Maaten & Hinton 2008) to visualize features of the original subset (light dot), which includes different amounts of original data, and its compressed subset (dark dot), which is obtained by compressing the original subset to just fit in the memory buffer. With the increase of quantity and the decrease of quality, the area of compressed subset is initially similar to that of original subset and expands synchronously. However, low-quality compressed data occur -iitof-disttiuution the area of compressed subset becomes much larger than that of its original subset, where the performance also severely declines (see Fig.
large number of
4

---

Published as a conference paper at ICLR 2022
Decrease Quality, Increase Quantity
Compressed Data
Na
Ng mb
Original Data
Figure 3: t-SNE visualization of features of the original subset (light dots) and its compressed subset (dark dots) after learning 5-phase ImageNet-sub with LUCIR. From left to right, the quantity is increased from 37, 85 to 200, while the JPEG quality is reduced from 90, 50 to 10. We plot five classes out of the latest task and label them in different colors. The crossed area is autrof-distribution.
On the theoretical side, given a training dataset D, we aim to find the compressed subset Dmb that can best represent D by choosing appropriate compression quality from its range. To achieve this goal, we introduce P, (Dmbb] D) to characterize the conditional likelihood of selecting D given input D under parameter q. The goal of learning is to choose appropriate q based the training tasks for making accurate predictions on unseen inputs. While there are variety of objective functions for learning, here focus on the widely-used maximum likelihood estimation (MLE), where the goal is to choose q to maximize the conditional likelihood of the observed data:
max Pg (D D).
(1)
9
The construction of Dmb can be essentially viewed as sampling problem with the cardinality Nyb Here, we apply Determinantal Point Processes (DPPs) to formulate the conditional likelihood (DDDrbDD) since DPPs are not only elegant probabilistic sampling models (Kulesza & Taskar 2012). which can characterize the probabilities for every possible subset by determinants, but also provide a geometric interpretation of the probability by the volume spanned by all elements in the subset (detailed in Appendix C.2). particular, conditional DPP a conditional probabilistic model which assigns probability (DDra D) to each possible subset Dmub Since the network parameter 0 is fixed during compression, and the feature matrix M₂ P (Dimb, we rewrite Pa (Drh D) as Py D)) equivalently. Formally, such a DPP formulates the probability Pa (y(Dmb)))) as
det(L (D:g. 0))
P,(M Σ EM det ( (D; g, 0))
where I.Vel Nqm and L(DD;q 0) is conditional DPP D x [D] kernel matrix that depends on the input D, the parameters and q. LM (D;q,0) (resp., Las: (D; q. f)) is the submatrix sampled from L(D:q, ()) using indices from M (r,sp.,,.) The numerator defines the marginal probability of inclusion for the subset Mi, and the denominator serves as a normalizer to enforce the sum of (M, D) for every possible Mt to 1. Generally, there are many ways to obtain positive semi- kernel L. this work, we employ the most widely-used dot product kemeel function, where LMę- MrT Mn and La M M.
However, due to the extremely high complexity of calculating the denominator in Eq. ( (analyzed in Appendix C. it is difficult to optimize Py (Mg)) D). Alternatively. by introducing (MG D) to characterize the conditional likelihood of selecting M, given input D under parameter q. we propose relaxed optimization program of Eq. {T). in which we (1) maximize P. (A since Py (M(MII D) < Pp, (M(()) is always satisfied under lossy compression; and meanwhile, (2) constrain that Py (Mg) D) is consistent with Py(MiD)). The relaxed program is solved as follows.
First, by formulating (MM similarly as (y(M)) in Eq. (2) (detailed in Appendix C.2), we need maximize
dL (D:
L (g) Py (M
(3)
EIM"=N det (L (D;0))*
where the conditional DPP kernel matrix L(D; 8) only depends on and 0. For our task, PM D) monotonically increases with Nmb Thus, optimizing C, converted into max, Nmb equivalently, with significantly reduced complexity (detailed in Proposition] of Appendix C.3).
5

---

Published as a conference paper at ICLR 2022
Second, to constrain that (M is consistent with P. (d( we propose to minimize
Pg(MgLD) (q) Po(My 1 1
d M9)
1 Volg
Zg
Zg
= Rzz - (4)
d M2)
Vol
where 2g M i= det(Lne- (D:0))
-xp det lesa (Da.00))))) In particular, d My
has a geometric interpretation
that it is equal to the squared volume spanned by M (Kulesza & Taskar 2012). denoted Vola
likewise for del (M; Mş) with respect to Volg. Then we define Rg
Vol.
as the ratio of the two
Vol
feature volumes. To avoid computing Zg we can convert optimizing Lg into minimizing RRg 1|
equivalently, since both of them mean maximizing q (detailed in Proposition Appendix C.4).
Putting L1 and C2 together, our method finally reformulated as
max g(9)
(S)
s.t., q Q, R. 1
where c is small positive number to serve as the threshold of Rg. g(.) : - N represents the function that outputs the maximum number (i.c., Nymhh such that the compressed version of D to quality q be stored in the memory buffer. Note that relax the original restriction of minimizing R. q for L2 cannot be achieved simultancously. Of note, Rg is calculated by normalizing the feature volume Volg with Vola, both of which depend on q (i.e., Nyma)). Therefore, e can largely mitigate the sensitivity of to various domains and can be empirically set as a constant value (see Sec. 4.3).
by enforcing Rg - < 6, since maximizing Nob for L1 and maximizing
Since the function g(-) in Eq. (5) is highly non-smooth, gradient-based methods are not applicable. Indeed, we solve it by selecting the best candidate in a finite-size set Q. Generally, the candidate values in Q can be equidistantly selected from the range of q, such as [1, 100] for JPEG. More candidate values can determine proper more accurately, but the complexity will grow linearly. We found that selecting 5 candidate values a good choice in our experiments. Once we solve Eq. (5). good trf is achieved by reducing much as possible to obtain larger Nmb, while keeping the feature volume Volg similar to Vol. This is consistent with our empirical analysis in Fig.
4.3 VALIDATE OUR METHOD WITH GRID SEARCH RESULTS
In essence, the grid search described in Sec. 4.1 be seen naive ap-
Training
Memory
Multiplc Epochs
proach to determine the compression quality, which is similar selecting other hyperparameters for continual learning (Fig. a). This strategy is learn a task sequence or sub-sece sebueeeeence using different qualities and choose the best one (Fig. b), which leads to huge extra computation and will be less applicable if the old data cannot be revisited, or the future data cannot be accessed immediately. In contrast, our method described in Sec. 4.2 only needs calculate the feature volumes of each compressed subset Drub and
F8
Hyperparameter
Multiple Epocha
9:
Quality q
4:
R
(6) Approach Determine
01
Condidate Subsote wio Compression
Candidate Suboets w/ Comprecsion
|R,,ttee IRu
original subset 1)mb+ (Fig. 4 c). with Figure 4: Properly select quality without repetitive training.
out repetitive training.
Now we validate the quality determined by our method with the grid search results, where LUCIR and PODNet achieve the best performance at the JPEG quality of 50 and 75 on ImageNet-sub,, respectively (see Fig, 2b. We present Rg in each incremental phase and the averaged R. of all incremental phases for 5-phase split in Fig. 5 and for 10- and 25 phase splits in Appendix Fig [3] Based on the principle in Eq. (5) with 0.5, it can be clearly seen that 50 and 75 are the qualities chosen for LUCIR and

---

Published as a conference paper at ICLR 2022
PODNet
75
50
100
Averoged FL
Averaged R.
02" 25
0
25
50 Quality
75
100
25
50 Quality
75
100
Phose
Phose
Figure 5: For 5-phase ImageNet-sub, we present Rg in each incremental phase with various compres- sion qualities g, and the averaged Rg of all incremental phases.
PODNet, respectively, since they are the smallest qualities that satisfy |R - < e. Therefore, the quality determined by our method is consistent with the grid search results, but the computational cost is saved by more than 100 times. Interestingly, for each quality q. whether RRg - is generally consistent in each incremental phase and the average of all incremental phases. We further explore the scenarios where Rg might be more dynamic in Appendix D.4.
EXPERIMENT
In this section, we first evaluate memory replay with data compression (MRDC) in class-incremental learning of large-scale images. Then, we demonstrate the advantages of our proposal in realistic semi- cemi-supervised continual learning of large-scale object detection for autonomous driving
5.1 CLASS-INCREMENTAL LEARNING
Benchmark: We consider three benehmark datasets of large-scale images for continual learning. (Wah et al. 2011 is a large-scale dataset including 200-class 11,788 colored images of birds with default format of JPG, split 30 images per class for training while the rest for testing. ImageNet-ful (Russakovsky et al., (011))1)) includes 1000-class large-scale natural images with default format of JPEG. ImageNet-sub (Hou al. 2019) subset derived from ImageNet-full, consisting of randomly selected 100 classes of images. Following Hou et al. (2019), we randomly resize, crop and normalize the images the size of 224 x 224, and randomly split a half of the classes the initial phase while split the rest into 5, 10 and 25 incremental phases. We report the averaged incremental accuracy with single-head evaluation (Chaudhry al., 2018) the main text, and further present the averaged forgetting in Appendix E.3.
Implementation: We follow the implementation of representative memory replay approaches (Hou et al. 2019; Douillard et al. 2020) (detailed in Appendix B.1), where we focus on constraining certain storage space the memory buffer rather than a certain number of images. The storage space is limited to the equivalent of 20 original images per class, if not specified. We further discuss the effects of different storage space in Fig. a fixed memory budget in Appendix E.4 and less compressed samples in Appendix E.5. For data compression, apply naive but commonly-used JPEG (Wallace
1992 algorithm to compress images to a controllable quality in the range of [1, 100).
Baseline: We evaluate representative memory replay approaches such as L LwF Li & Hoiem 2017).
iCaRL (Rebuffi et al.) 2017). BiC (Wu et al. 2019), LUCIR (Hou et al. 2019). Mnemonics (Liu
et al. 20206), TPCIL. [Tao al. 2020a), PODNet (Douillard et al.) 2020). DDE (Hu et al. 2021) and AANets (Liu et al. 2021a). In particular, AANets and DDE are the recent strong approaches implemented on the backbones of LUCIR and PODNet,
we also implement ours on the two backbones. Since both AANets and DDE only release their official implementation on LUCIR, we further reproduce LUCIR w/ AANets and LUCIR w/ DDE for fair comparison.
Accuracy:
We summarize the performance of the above baselines and memory replay with data compression (MRDC, ours) Table ! Using the same extra storage space, ours achieves comparable or better performance than AANets and DDE on the same backbone approaches, and can further boost their performance by a large margin. The improvement from ours due to mitigating the averaged forgetting, detailed in Appendix F.3. For CUB-200-2011 with only a few training samples, all old data can be saved in the memory buffer with a high JPEG quality of 94. In contrast, for ImageNet-sub/-full only part of old training samples can be selected, compressed and saved in the
All experiments averaged by more than three runs with different random seeds.
7

---

Published as a conference paper at ICLR 2022
Table 1: Averaged incremental accuracy (%) of classification tasks. The reproduced results are presented with + standard deviation, while others are reported results. The reproduced results might slightly vary from the reported results due to different random seeds. With olass-balance finetuning. PoODNet reproduced by Hu et al. (2021) underperforms that in Douillard et al. (2020).
100-210-0111
ImageNet-sabb
ImageNer-Full Full
Method
5-phase
10-phase
25-phase
5-phase
10-phase
25-phase
5-phase
10-phase
LwFF
Hoiem
2017
39.42 an.s
20.0
36.33 20is
53.62
47.64
38.90
sCaRL
Rebuffi et al.
2017
39.49 so.sa
39.31 80.0
38,77 20.2
65
59.88
52.97
51.50
46.89
BiC
Wa
2019
45.25 40.0
10.27
64.96
57.73
Mnemonncs
Liu ct al.
2020b
72.58
71.37
69.74
64.54
63.01
TPCIL
Tao al.
2020a
76.27
62.88
LUCIR
al.
45 20
20
61
61.57
w/ AANets (Láu et
1021a
72.55
69,22
67.60
64.94
62.39
2021
67.51'
65.77
w/ MRDC (Ours)
46.68 10.0
47.28 s0l1
48.01 40.2
73.56
20.20
72,70
70.53 m0.53
67,53 m
65.29 so..o
AANets (Reproduced)
46,87 20.03
47.34 2mm
47.35 2000
72.91 20.40
71.93
70.70
20.0
63.37
20
62,46 14
w/ AANcts (RDr(
49,02
49.84
51,33
73.79
73.73
73.47
64.99
63.04
DDE (Reprodusad)
45,86 20
46.48
20
46,46
20
73.04
70.84
66.61
20:
66.95 a0
65.21 os
DDE MRIX (Ours)
47,16
48.33
48,37
75.12
20:17
73.39
70.83
67, 90 ao
66.67 xs
PODNet Douillard et al
2020
44.92
44.49
43.79
75.54
74.33
68.31
66.95
64.13
AANets (Lu ct
2021a.
76.96
75.58
71.78
67.73
64.85
DDE Hu et al.
2021
76.71
75.41
66.42
64.71
MRDC (Ours)
46.00
20.24
46.09
27
45.84
20.03
78.08
20.06
76.02
54
72.77
20.7
68.91
10
66.31
20
memory buffer, where our method can efficiently determine the compression quality for LUCIR and PODNet (see Sec. 4.3 and Appendix D.2), and for AANets and DDE (see Appendix D.3).
Computational Cost: Limiting the size of memory buffer is not only to save its storage, but also to save the extra computation of learning all old training samples again. Here we evaluate the computational cost of AANets, DDE and memory replay with data compression (MRDC, ours) on LUCIR (see Fig. 6 for ImageNet- sub and Appendix Table 3 for CUB- 200-2011) 1). Both AANets and DDE require huge amount of extra com- putation to improve the performance of continual learning, which is gener- ally several times that of the backbone approach. In contrast, ours achieves competing or more performance improvement but only slightly increases the computational cost.
5-Phase
10-Phase
25- 25-Phase
Avaraged Ausuncy (N)
(ano)) surl Batnea
morrgge Accurncy (h)
Averagge Acsurny (%)
-2a
42-
Licow
AAtero
wing
Dure
Luck
Lucp A440n wone
chor
Figure 6: Averaged incremental accuracy (the column, left Y-axis) and computational cost (the triangle, right Y-axis) on ImageNet-sub. We run each baseline with one Tesla V100.
(a)
S-Phase
10-Phase
23-Phase
(b)
S-Phase
10-Phase
25-Phose
78
80-
Avenged Accurmay (S)
Averaged Accurssy (N))
Averaged Accurncy (8)
Moeeoooe Resumest (%)
Averaged Accuraty (50
Avaragent Acturacy (%6)
74-
74-
78-
26
LUCIR
LUCIR
LUCIR
PODNet
PODNet Oura
PODNet
w/ Oura
Oure
Ours
Ours
Ours
28 40
20
40
40
20
40
46
Storage Space
Storage
Space
Storage Space
Storage
Space
Storage Space
Figure 7: The effects of different storage space (equal to 10, 20, 40 and 80 original images per class) on ImageNet-sub. LUCIR (a) and PODNet (b) are reproduced from their officially-relcased codes.
Storage Space: The impact of storage space is evaluated in Fig. 7 Limiting the storage space to equivalent of 10, 20, 40 and 80 original images per class, ours can improve the performance of LUCIR and PODNet by similar margin, where the improvement is generally more significant for
8

---

Published as a conference paper at ICLR 2022
more splits of incremental phases. Further, ours can achieve consistent performance improvement when using a fixed memory budget (detailed in Appendix E.4), and can largely save the storage cost without performance decrease when storing less compressed samples (detailed in Appendix E.5).
5.2 LARGE-SCALE OBJECT DETECTION
The advantages of memory replay with data compression are more sig- nificant in realistic scenarios such as autonomous driving, where the incre- mental data are extremely large-scale with huge storage cost. Here we eval- uate continual learning on SODAIOM (Han et al. (02T). a large-scale object detection benchmark for autonomous driving. SODA 10M contains 10M un- labeled images and 20K labeled im- ages of size 1920 x 1080 with default format of JPEG, which is much larger than the scale of ImageNet. The la- beled images are split into 5K, 5K and 10K for training, validation and testing,
43.8-
20-
10-
8-
43.6-
15-
Avaraged
43.4-
of 200
100
90
80
75
49.2
50
25
100
Quality
Phase
Quality
Figure 8: We present the grid search results (left), Rg in each incremental phase (middle), and the averaged R. of all incre- mental phases (right) for SSCL on SODA 10M. The quality determined by our method is 50, which indeed achieves the best performance in the grid search.
respectively, annotating 6 classes of road objects for detection.
Since the autonomous driving data are typically partially-Iabelee we follow the semi-supervised continual learn- ing (SSCL) proposed by Wang al. (2021a) for object detection. Specifi- cally, we randomly split 5 incremen- tal phases containing K labeled data and 20K unlabeled data per phase, and use a memory buffer to replay labeled data. The storage space limited to the equivalent of 100 original im- ages per phase. Following Han et al. (202I). consider Pseudo Labeling
Table 2: Detection results with + standard deviation (% of semi-supervised continual learning on SODA 10M. FT: finetuning. MR: memory replay.
Method
AP
APso
AP
FT
40.36 20.34
63.83 10.0
43.82 =4.44
Pscudo
MR
40.75
s0.30
65.
/
43.53 s0 /-0.99
Labeling
Ours
41.50
soiss booa//1h /+1.14
65.36
44.95
s0:2
++.13
FT
42,88
66.70
45.99 gn:a
Unbiased
MR
43.10 souss /40.2
66.88
coss +0.18
46.62
s0:2 +0.63
Teacher
Ours
43.72 20.23 40.84
67.80
+1.10
47.36 2021/ +1.37
and Unbiased Teacher (Liu et al. 2021b) for semi-supervised object detection (the implementation is detailed in Appendix B.2). Using the method described in Sec. 4.2 with the same threshold (i.e.,
= 0.5), we can efficiently determine the compression quality for SSCL of object detection, and validate that the determined quality indeed achieves the best performance in grid search (see Fig. ().
Then, compared with memory replay of original data, our proposal can generally achieve several times of the performance improvement on finetuning, as shown in Table 2
CONCLUSION
In this work, we propose that using data compression with a properly selected compression quality can largely improve the efficacy of memory replay by saving more compressed data in limited storage space. To efficiently determine the compression quality, we provide a novel method based on determinantal point processes (DPPs) to avoid repetitive training, and validate our method both class- cans-inncrmmetal learning and semi-supervised continual learning of object detection. Our work not only provides an important yet under-explored baseline, but also opens up promising new avenue for continual learning. Further work could develop adaptive compression algorithms for incremental data to improve the compression rate, or propose new regularization methods to constrain the distribution changes caused by data compression. Meanwhile, the theoretical analysis based on DPPs can be used as a general framework to integrate optimizable variables in memory replay, such as the strategy of selecting prototypes. In addition, storage space to best describe its distribution, which will motivate broader applications in the fields of data compression and data selection.
work suggests how to save batch of training data in limited

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENTS
This work was supported by NSF of China Projects (Nos. 6206113600 61620106010, 62076145. U19B2034, U181146), Beijing NSF Project (No. JQ19016), Beijing Outstanding Young Scientist Program NO. BJJWZYJHO 20000000000000 10022009,, Tsinghua-Peking Center for Life Sciences, Beijing Academy of Artificial Intelligence (BAAI). Tsinghua Huawei Tsinghua Institute for Guo Qiang, and the NVIDIA NVAIL Program with GPU/DGX Acceleration, Major Innovation & Planning Interdisciplinary Platform for the "Double-Firs Class" Initiative, Renmin University of China.
ssinghua-Huawei Joint Research Program, grant from
ETHIC STATEMENT
This work presents memory replay with data compression for continual learning. It can be used to reduce the storage requirement in memory replay and thus may facilitate large-scale applications of such methods to real-world problems (e.g., autonomous driving). As fundamental research in machine learning, the negative consequences in the current stage are not obvious.
REPRODUCIBILITY STATEMENT
ensure the reproducibility our paper from three aspects. (1) Experiment: The implementation of our experiment described in Sec. 4.1, Sec. 4.3 and Sec. 5 is further detailed in Appendix B. (2) Code: Our code is included in supplementary materials. (3) Theory and Method: A complete proof of the theoretical results described in Sec. 4.2 is included in Appendix C.
10

---

Published as a conference paper at ICLR 2022
REFERENCES
Eirikur Agustsson, Fabian Mentzer, Michael Tschannen, Lukas Cavigelli, Radu Timofte, Luca Benini, and Luc Van Gool. Soft-to-hard vector quantization for end-to-end learning compressible representations. arXiv preprint (.Xiv:7704.0045: 2017.
Lucas Caccia, Eugene Belilovsky, Massimo Caccia, and Joelle Pineau. Online learned continual com- pression with adaptive quantization modules. International Conference on Machine Learning, pp. 1240-1250. PMLR, 2020.
Margaret F Carr, Shantanu P Jadhav, and Loren M Frank. Hippocampal replay in the awake state: potential substrate for memory consolidation and retrieval. Nature Neuroscience, 14(2):147-153, 2011.
Arslan Chaudhry, Puneet Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 532-547, 2018.
Thomas J Davidson, Fabian Kloosterman, and Matthew Wilson. Hippocampal replay of extended experience. Neuron, 63(4):497-507, 2009.
Amit Deshpande and Luis Rademacher. Efficient volume sampling for row/column subset selection. In 2010 IEEE 51 Annual Symposium on Foundations of Computer Science, pp. 329-338. IEEE, 2010.
Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In Proceedings of the European Conference on Computer Vision (ECCV), volume 12365, pp. 86-102, 2020.
Benjamin Ehret, Christian Henning, Maria R Cervera, Alexander Meulemans, Johannes Von Os- wald, and Benjamin F Grewe. Continual learning in recurrent neural networks. arXiv preprint arXiv:2006. 12 1090 2020.
Jianhua Han, Xiwen Liang, Hang Xu, Kai Chen, Langing Hong, Chaoqiang Ye, Wei Zhang, Zhenguo Li. Chunjing Xu, and Xiaodan Liang. Sodal0m: Towards large-scale object detection benchmark for autonomous driving. arXiv preprint arXiv:2106.11 11118, 2021.
Tyler I. Hayes, Kushal Kafle, Robik Shrestha, Manoj Acharya, and Christopher Kanan. Remind your neural network to prevent catastrophic forgetting. European Conference on Computer Vision, pp. 466-483. Springer, 2020.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 831-839, 2019.
Xinting Hu, Kaihua Tang, Chunyan Miao, Xian- Sheng Hua, and Hanwang Zhang. Distilling causal learning. arXiv preprint arXiv:2103.01737, 2021.
effect of data in
Shivam Khare, Kun Cao, and James Rehg. Unsupervised class-incremental learning through confu- sion. arXiv preprint arXiv:2104 04450, 2021.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska. Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114 (13):3521-3526, 2017.
Alex Kulesza and Ben Taskar. Determinantal point processes for machine learning. arXiv preprint arXiv: 1207.6083, 2012.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935-2944, 2017.
Li Lian and Wei Shilei. Webp: A new image compression format based on vp8 encoding. Microcon- trollers & Embedded Systems, 3, 2012.
11

---

Published as a conference paper at ICLR 2022
Tsung- Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117-2125, 2017.
Xialei Liu, Chenshen Wu, Mikel Menta, Luis Herranz, Bogdan Raducanu, Andrew D Bagdanov, Shangling Jui, and Joost van de Weijer. Generative feature replay for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshops, pp. 2020a.
Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun. Mnemonics training: Multi-class incremental learning without forgetting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2020b.
Yaoyao Liu, Bernt Schiele, and Qianru Sun. Adaptive aggregation networks for glass-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2544-2553, 2021a.
Yen-Cheng Liu, Chih- Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda. Unbiased teacher for semi-supervised object detection. arXiv preprint (000::2002.094000 2021b..
James L McClelland. Incorporating rapid neocortical learning of new schema-consisten information into complementary learning systems theory. Journal of Experimental Psychology: General, 142 (4):1190) 190, 2013.
James L McClelland, Bruce L McNaughton, and Randall C O'Reellly Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of Learning and Motivation, volume 24, pp. 109-165. Elsevier, 1989.
Fabian Mentzer, George Toderici, Michael Tschannen, and Eirikur Agustsson. High fidh-fiielity genera- tive image compression. arXiv preprint arXiv:2006.09965 2020.
Majid Rabbani. Jpeg2000: Image compression fundamentals, standards and practice. Journal of Electronic Imaging, 11(2):286, 2002.
Sylvestre-Alvin Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems, volume 28, pp. 91-99, 2015.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211-25) 2015.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1rXiv:1606.04677 2016.
Claude Elwood Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27(3)377--423, 1948.
Hanul Shin, Jung Kwon Lec, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2990-2999, 2017.
Xiaoyu Tao, Xinyuan Chang, Xiaopeng Hong, Xing Wei, and Yihong Gong. Topology-preserving class- caassinnceemntall learning. In European Conference on Computer Vision, pp. 254-270. Springer, 2020a.
12

---

Published as a conference paper at ICLR 2022
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong. Few-shot class-incremental learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition, pp. 22183122192 2020b.
George Toderici, Sean M O' Mallley Sung Jin Hwang, Damien Vincent, David Minnen, Shumeet Baluja, Michele Covell, and Rahul Sukthankar. Variable rate image compression with recurrent neural networks. arXiv preprint arXiv:1511.06085
2015.
George Toderici, Damien Vincent, Nick Johnston, Sung Jin Hwang, David Minnen, Joel Shor, and Michele Covell. Full resolution image compression with recurrent neural networks. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 5306-5314, 2017.
Gido M van de Ven and Andreas Tolias. Three scenarios for continual learning. arXiv preprint 1904.07744 2019.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(11), 2008.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
Gregory K Wallace. The jpeg still picture compression standard. IEEE Transactions on Consumer 1992.
Electronics,
Liyuan Wang, Kuo Yang, Chongxuan Li, Lanqing Hong, Zhenguo Li, and Jun Zhu. Ordisco: Effective and efficient usage of incremental unlabeled data for semi-supervised continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 5383-5392, 2021a
Liyun Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao, Kaisheng Ma. Jun Zhu, and Yi Zhong. Afec: Active forgetting of negative transfer continual learning. Advances in Neural Information Processing Systems, volume 34, 2021b.
Chenshen Wu, Luis Herranz, Xialei Liu, Joost van de Weijer, Bogdan Raducanu, et al. Memory replay gans: Learning to generate new categories without forgetting. In Advances In Neural Information Processing Systems, pp. 5962-5972, 2018.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374-382, 2019.
13

---

Published as a conference paper at ICLR 2022
A TRADE-OFF BETWEEN QUALITY AND QUANTITY
Given limited storage space, there an intuitive trade-off between the quality q and quantity Numb of com- pressed data. Here we plot the re- lation of q and Namb for ImageNet
(a) 200
ImageNet
(b) 300
SODA10M
250
Quantity
Quantity
200-
and SODA 10M used in our paper. As shown in Fig. D reducing the quality enables to save more compressed data in the memory buffer, and vice versa. Then, present compressed images with various qualities in Fig. [0] Al- though reducing the quality tends to distort the compressed data and thus hurts the performance of continual learning, it can be clearly seen that the semantic information of such pressed data is hardly affected, There- fore, a potential follow-up work regularize the differences between compressed data and original data, so as to further improve the performance of memory replay.
100
150
100-
50-
50-
0-
0-
25
50
75
100
25
50
75
100
Quality
Quality
Figure 9: Limiting the storage space the equivalent of (a) 20 original images per class for ImageNet and (b) 50 original images per phase for SODAI0M, we plot the relation between the quality and quantity of compressed data. The quality of 100 refers to the original data without compression.
(a) Original
90
75
50
25
10
(b)
Figure 10: Compressed images of ImageNet. (a) Each column contains compressed images of a specific JPEG quality. (b) Two exemplar images with different qualities.
14

---

Published as a conference paper at ICLR 2022
B IMPLEMENTATION DETAIL
B.1 CLASS-INCREMER LEARNING
Following the implementation of Hou et al. (2019): Douillard et al. (2020); Tao et al. (2020a) Liu
al. (202 lay: Hu et al. (2021), we train ResNet- architecture for 90 epochs, with minibatch size of 128 and weight decay of 1 x 10-4 We use a SGD optimizer with initial learning rate of 0.1, momentum of 0.9 and cosine annealing scheduling. We run each baseline with one Tesla V100, and measure the computational cost. We evaluate the performance of dlass-incrementes learning with the averaged incremental aceuracy (AIC) (Hou et al. 2019p. After learning each phase, we can calculate the test accuracy on all classes learned so far in the previous / phases as A, Then we can calculate - - Ai.
To evaluate the effects of class similarity on the determined quality of data compression, we select similar or dissimilar superclasses from ImageNet, which includes 1000 classes ranked by their semantic similarity. Specifically, we select 10 adjacent classes as a superclass, and construct 10 adjacent (similar) or remote (dissimilar) superclasses. The index of classes selected in 10 similar superclasses is [1, 3, 100], while in 10 dissimilar superclasses is [1, 2, 10, 101, 102, 110,
.... 910]
B.2 LARGE-SCAAL OBJECT DETECTION
For memory replay, we follow Wang et al. (2021a) randomly select the old labeled data and save them in the memory buffer, which indeed achieves competitive performance as analyzed by
Chaudhry al. (018). We follow the implementation of [Han et al. (202T) for semi-supervised object detection. Specifically, we use Faster-RCNN (Ren et al. 2015) with FPN (Lin et al. (017)) and ResNet-50 backbone as the object detection network for Pscudo Labeling (Han et al. (021) and Unbiased Teacher (Liu et al. 2021b). For each incremental phase, train the network for 10K iterations using the SGD optimizer with initial learning rate of 0.01, momentum of 0.9, and constant learning rate scheduler. The batch size of supervised and unsupervised data are both 16 images. For Pseudo Labeling, we first train a supervised model on the labeled set in the first 2K iterations. Then we evaluate the supervised model on the unlabeled larger than 0.7 in the evaluation results is selected as a pseudo label, which would be used to train the supervised model in the later 8K iterations in each phase. For Unbiased Teacher, we follow the default setting and change the input size to comply with SODAIOM, We run each baseline with 8 Tesla V100. The performance evaluated by the metries (AP, A AP AP7s) used in the prior works (Liu et al. 2021b Han et al. 2021)
A bounding box with predicted score
C THEORETICAL ANALYSIS
C.I DETERMINANTAL POINT PROCESSES (DPPS) PRELIMINARIES
Arising in quantum physics and random matrix theory, determinantal point processes (DPPs) are elegant probabilistic models of global, negative correlations, and offer efficient algorithms for sampling. marginalization, conditioning, and other inference tasks.
A point process P on ground set V is probability measure on the power set 2N, where N = | the size of the ground set. Let be x N matrix with D Z N. In practice, the columns of B are vectors representing items in the set V. Denote the columns of B by B, for i = 2, N. sample from might be the empty set, the entirety B, or anything in between. P called determinantal point process if, given random subset Y drawn according to P ( possible instantiations for Y), we have for every S C B,
P(Y = S) x dot(Ls);
(6)
where S = [Bi,, B.,,,..,,,, B P(Y = S) characterizes the likelihood of selecting the subset S from B. For some symmetric similarity kernel L E N (e.g.. L T B), where Lg is the similarity kemee of subset S (e.g.. L S). That is, Ls is the submatrix sampled from L using indices from S. Since P is probability measure, all principal minors (T.s) of L must
15

---

Published as a conference paper at ICLR 2022
be non-negative, and thus L itself must be positive semidefinite. These requirements turn out to be sufficient: any L, 0 3 3 I, defines a DPP.
L is often referred to the marginal kernel since it contains all the information needed to compute the probability of any subset S being included in Y. Hence, the marginal probability of including one element B, is ((B EY) Lii, and two elements B; and Bj is P P(B, B, E EY) L 22 Y) Li A large value of Lij reduces the likelihood of both elements to appear together in diverse subset. This demonstrates why DPPs are "diversifying". Below is several important conclusions about DPPs related our work (Kulesza Taskar 2012).
P(B, E (B,
Geometric Interpretation. By focusing L-ensembles in DPPs, determinants have an intuitive geometric interpretation.
B B, then
P(Y = S) x (el(Ls) = (Vol(5))²
where the first two terms specify the marginal probabilities for every possible subset S, and the last term is the squared (S1-dimensional volume of the parallelepiped spanned by the items in
Quality vs. Diversity. While the entries of the DPP kernel L are not totally opaque in that they can be seen as measures of similarity
our primary qualitative characterization of DPPs as diversifying processes. In most practical situations, we want diversity to be balanced against some underlying preferences for different items in B. Kulesza & Taskar (2012) proposed a decomposition of the DPP that more directly illustrates the tension between diversity and per-item measure of quality. Specifically, I. B B can be taken one step further, by writing each column B; as the product of quality term a: 0 and vector of normalized diversity features E R.P l|fell 1. Then, Lij- = f: (Eff"fs9, fgg, where f. fj E [-1,
1] is a signed measure of similarity between B; and B3.
Let dig J Jj V Viuly
Ly and {dbs) J. -==--- Then, depending on Eq. (6), we have
P(Y = 5) x
IIA di ES
det (os),
where q; can be seen as quality score of an item B, in B, and ds is the submatrix sampled from using indices from S.
Alternative Likelihood Formulas. In an L-ensemble DPP, the likelihood of a particular set S C B is given by
det(Ls)
det(Ls)
PL(S) Eg del (Lsr) (e 7)"
where S' is one of all 2 possible subsets B, and Ls is the submatrix sampled from L using indices from S
This expression has some nice intuitive properties in terms of volumes, and, ignoring the normalization in the denominator, takes simple and concise form. However, as a ratio of determinants on matrices of differing dimension, it may not always analytically convenient. Minors can be difficult to reason about directly, and ratios complicate calculations like derivatives. Such a high computational cost is just one key challenge in DPPs.
Conditional DPPs. A conditional DPP P(Y S)BB is conditional probabilistic model which assigns a probability to every possible subset S C B. The model takes the form of an L-ensemble:
P(Y B) x (B)),s
(10)
where L(B) is positive semidefinite conditional N x N kernel matrix that depends on the input B.
k-DPPs. A 4PDP is obtained by conditioning a standard DPP on the event that the set Y has
cardinality k. which is concerned only with the content of random k-set (i.c., without the size
k)) Kulesza Taskar 2012). Formally, the kDDPP P) gives the probability of a particular set S as
dct(Ls)
PI(Y =) Σ = det
(11)
16

---

Published as a conference paper at ICLR 2022
where [S] = k and L is a positive semidefinite kernel. Although numerous optimization algorithms have been proposed to solve DPPs problems, the high computational complexity about denominator cannot be avoided generally. On the one hand, computing denominator is a sum over (*) terms. On the other hand, computing the determinant of each term through matrix decomposition is with O(k2) time. Just as Kulesza & Taskar (2012) claimed, sparse storage of larger matrices is possible for DPPs, but computing determinants remains prohibitívely expensive unless the level of sparsity is extreme.
If the dot product kernel function is adopted to compute the kernel I. in Eq. (II), similar to Geometric Interpretation of standard DPPs above, the probability P = S) is further defined with volume sampling (Deshpande & Rademacher, 2010), i.e.
det(Ls) PL = =) Σ
det((Ts) (k!! (Vol(Conv(0 U U
(12)
| (L
where L =B B, Conv(-) denotes the convex hull, and Vol(.) is the k-dimensional volume of such a convex hull.
C.2 MODELING OUR CASE AS A CONDITIONAL Nmbb-DP
In this work, we aim to find the compressed subset Dmbb that can best represent the training dataset D by choos- ing an appropriate compression quality from its range. To achieve this goal, we introduce (, | to charac- terize the conditional likelihood of selecting Dmb given input D under parameter q. Since the network parame- ter is fixed during compression, and the feature matrix Mc = F { (Dmb), we rewrite (D (D) as P (M DD) equivalently (see Fig. [ for the details of constructing Dmb and Mt). Depending on the nice properties of DPPs in Sec. C.I we formulate our goal as a conditional DPP, where D is the associated ground set, and MG is desired subset in feature space. Then P. pa(M (Mg) D) is distribution over all subsets of D with cardinality Numh which is for- mally called a conditional Nmb-DDP since [M] = Nmb for any possible subset. Thus, such DPP formulates the conditional probability Po (M
D= = (a;
Mcan-of-Featurr or Other Principles
{ yi) Nmb
Dmbs
(.(()
Dmb
(.).) fix
Fg (.) fix
M q
M
Figure 11: Construction of Dmm Dmb
M, and Mq. @ is fixed in this stage.
as
det(L 0))
P₂ D)
(13)
Σ Ngg det((T. (D; q, 0))
where L(D;g, g, 0) is a conditional DPP DD x |D] kernel matrix that depends on the input D parame- terized in terms of parameters 0 and q. LM (D( q, (resp., Lang (D; q, 0))) is the submatrix sampled from L(D: q. 8) using indices from M (resp., MO).
Similarly, DM) can be formulated by conditional N -DP as
dct(LLs;(D:b)))
Pg(Mi1D) Σ Niarr d (D;0))*
(14)
where | NMmh. Unlike L(D: q. f) above, the conditional DPP kernel matrix L(D;8) only depends on 1 and 9 without q, since g is fixed its maximum, i.c., without compression. LM- LM-(D:)) (resp., Lxz; (D; 0)) is the submatrix sampled from L(D; 0) using indices from A1* (resp., M,))
Differences: Of note, two important differences between standard k-DPPs and our N lie in the optimization variable and objective. For this work, we need to find an optimal set cardinality (i.c., Nr while is fixed for standard k-DPPs. Although both of them can finally determine desired subset with the maximum volume, our _mmb_ can uniquely determine it by Nmm,, while standard kDDPPs obtain it by maximizing Eq. (11)
17

---

Published as a conference paper at ICLR 2022
C.3 THE FIRST GOAL
First, we need to maximize
dct(LM; (D;8)) (D;:))
Ci(9) 1100=ppp0 P,((M)) E|M- Nym det (L (D;0)
(15)
x de q M2) = (Volg)²
= (N (VolConv(0UMM))))
where |M Nqm.. L(D;0) is conditional DPP [D] [D] kernel matrix that depends the input D parameterized in terms of 0. LM- (D; 0) (resp., LM; (D;0)) is the submatrix sampled from L(D;)) using indices from M* (resp., Mÿ). The numerator defines the marginal probability of inclusion for the subset M,, and the denominator serves as a normalizer to enforce the sum of (MG D) for every possible M1 to 1. Here we employ the commonly-used dot product kernel function, where M:;;- MG Mn and L-- M M Conv() denotes the convex hull, and Vol(.) the Amme-dimenion volume of such convex hull.
Eq. (15) provides geometric interpretation that the conditional probability P. (M( is proportional to the squared volume of M; with respect to Nymb, denoted as Volç, For our task, (, D) monotonically increases with Nob Thus, optimizing L is converted into max, Nmd equivalently (detailed Proposition as below).
Proposition 1 For any matrix X (I1, T2.-.. (.n) where x; E Rd the i-th column of X. d > n and X X # I, define M = (i1, 12. immt subset of [n] containing m elements, and X the submatrix sampled from X using indices from M. Then
det (M)
P(M) EmMi=m det (X
we can ensure that the following two optimization programs
max P(M)
and
max [
equivalent.
Below is one critical lemma for the proof of Proposition 1
Lemma For any matrix = (x1, D2 Fn.) where H; E Rd is the i-th column of X and d n, define M i2, (im) } subset of [n]] containing m elements, and X M as the submatrix sampled from X using indices from M. Then
det (XMXM)
P(M) = Σ EMM{= det (X (XTr
we can ensure that
[n] argmaxx P(M).
This means when |M| = n, the probability P(M) takes the maximum value.
det (X Proof. From the definition, we have
0 P(M) EIMY det (XI)XMM)
det (x XMM)
det (XMXM) + +Σ ( E|MMM det (XJ (N 1.
18

---

Published as a conference paper at ICLR 2022
If | = n, we have (M) possible options for subset X where det (Me) 0. Only if M n, then (m)
and the probability P(M) will definitely take the maximum value, In addition, X X has full rank due d >> n, and then det(X X) always holds. That is, P(M) = can be guaranteed only when selecting all items together. This means [n] = argmax P(M). It completes the proof.
Le. P(M)
Proof of Proposition 1 From the definition in Deshpande & Rademacher 2010), we have
det (XMxMM
P(M) = EmMi== det (X (xx))
(16)
x det (XM
= (m!- (Vol(Conv(( U XM))))
where Conv(-) denotes the convex hull, and Vol(.) is the m-dimensional volume of such a convex hull. Here, from the monotonicity side, it can be easily verified that for X U ai where
3 Tim41
Xx we have
(m! (Vol(Conv/ll UX ")))²) < ((m + 1)!- (Vol Conv(( U UMM U (xmm4.)))
which means P(M) is monotonically increasing with the increase of [M].
From the extremum side, by using Lemma programs
|
ensure that the following optimization
max P(M)
and
max [
have the same optimal value. Thus, maximizing P(M) with respect to [M can be converted into maxm
[M equivalently.
C.4
THE SECOND GOAL
Second, we need to minimize
Py(MSID)
det(LM; (Di,,
C2(9)
Zg
D)
det(LM; (D;0))
(17)
de Mg)
Vol
Zg
Zg
12
det(M; M;)
Vol
E Me det(Ligs (D;0))
where Zg Eim-ngg der(T. ( ()) In particular, dc M₂) has a geometric interpretation
that it is equal to the squared volume spanned by M, Kulesza & Taskar 2012) denoted Vol. likewise for dM Mş) with respect to Vol. Then define R. Vol, the ratio of the two feature volumes. To avoid computing Zς. we convert optimizing Lg into minimizing RRg 1| cquivalently, since both of them mean maximizing q (detailed in Proposition as below).
vol,
Below are going to introduce Proposition 2 We first introduce necessary assumptions, then we present the theoretical guarantee of Proposition
Assumption We make the following assumptions:
1. Image dataset: For any image dataset (x1 ....... ..,)). where x; is the i-th image in D. define Dg (I1, .In) compressed image dataset with the compression quality g for D. The compression quality is bounded as q E Q (Q can be a fiite//nfimit set), which the higher of the better of image quality.
19

---

Published as a conference paper at ICLR 2022
2. Feature matrix: Denote X f2. .fn) Rax" as the feature matrices of D and Dy with the same fixed feature extractor, respectively, Here, d d> n. X X and X Xg + I.
Raxn and X2 f f2.... ..)
3. Selection principle: Define M {21,82..... im as a subset of [n] containing m elements, and X the submatrix sampled from X using indices from M. Likewise, define Mg J2. ... :mm } as a subset of [n] containing m elements, and XM, as the submatrix sampled from Xg using indices from Mg. Both of them use the same selection principle.
4. Buffer storage space: g(.) R + N represents the function that outputs the maximum number (i.e. m ) such that the compressed version of Da to quality can be stored in the memory buffer. That is, m uniquely determined by q.
Proposition? 2 Under Assumption 7 for two probabilities
det (X)
P(M) = EiM'E= det (X (x
and
P(M,) = det (XTs, XNso
MjMi=m det (XT XXX M X(xs)
we can ensure that the following two optimization programs
det [x (XM, XMa)
min
det (x XM)
9
and
P(Mg)
min
P(M)
are equivalent.
Below are two critical lemmas for the proof of Proposition
Lemma Under Assumption for ratio
del XJd, XM.)
Rg = det (X XM)
ensure that
sup Q = argmin, Rg - 1/-
This means when q = sup Q (i.e, without image compression), |R, 1| takes the minimum value.
Proof. From the definition, we know RRq - 11 2 0. To minimize RR - 11, we need the ratio Rg = 1. i...
det (x,, X xM.) det (XTXM)
(18)
Of note, both X,M, and XM are selected from the source dataset /) with the same feature extractor, selection principle and set cardinality (i.e., m). That is, the only difference between them is image compression with respect to
By recurring the Geometric Interpretation in k-DPPs (i.e., Bq. 12
have
det (MM,X,n XM,) (m! (Vol[Conv(OUX,,]] )
20

---

Published as a conference paper at ICLR 2022
and
det (XM)XM) (m! (Vol(Conv(0 X
Then, Eq. (18) holds only when XM, and XM have the same convex hull. Due to great difficulty of analytically defining compression function, the two convex hull cannot be mathematically given.
However, by conducting extensive experiments this task (as shown in Fig. [] Fig. Fig. g
Fig.
13
and Fig. 14 we find that the volume of XM, is larger than that of XM for a specific m (i.e.. f)
Additionally, with y decreasing (i.e., increasing), the volume of X X,, is increasing
quickly.
Thus, we have an empirical conclusion that the two volumes are the same only when q = sup Q (without image compression), is reasonable since without image compression, the selection problems about X M, and X are identical. This means when sup Q, Rgg 1 takes the minimum value, which completes the proof.
Lemma 3 Under Assumption 7 for
del (X (M)
P(M) = Σ [Myl-mm det (N
and
det (X Air,NMo
P(Mg) Σ Mi/E= det (XI.N)
M M
we can ensure that
P(Mg) P(M)
sup Q argmin,
This means when q sup Q (i.e, without image compression), value.
P(M.) P(M)
takes the minimum
Proof.
From the definition, we know P(M)
PLM,]
2 0. To minimize
P(M.) P(M)
1/.
we need the ratio
P(M)
P(M.) 1. fact,
P(Mg) det (xTr, My XMA)
E| det (XI R.g Zn
(19)
P(M) det XM) Em det (XTXM²)
dec Aec(XMa, XMa) and Zy E|miim det det(x XMr) dot Not(XXxxMx XM²
where Ry
ΣiM-um dot M Xx)
know that when sup Q. Rg always holds. More importantly. sup Q, since the numerator and denominator in Z, will be equal without compression. That is, when g = sup Q, the ratio P(Mg)
By using Lemma Z, = also holds
and
P(M.) takes the minimum P(M)
P(M)
value. This completes the proof.
Proof of Proposition] From Lemma 2 and Lemma we can ensure that under Assumption | the following two optimization programs
det X My XM,,)
min
det (X MMM
and
min P(Mg)
P(M)
have the same optimal value. Thus, maximizing
P(Mg) P(M)
with respect to can be converted into
dot dot(x (xT, MM, XMa)
ming R.g equivalently, where Rg det(XXxXM)
21

---

Published as a conference paper at ICLR 2022
D EMPIRICAL ANALYSIS
D.1 T-SNE VISUALIZATION OF NORMALIZED FEATURES
Increase Quantity
Original Data NO mb
Training Data
Figure 12: t-SNE visualization features of original subsets (dark dots) and all training data (light dots) for 5-phase ImageNet-sub with LUCIR. We randomly select five classes out of the latest task and label them different colors.
To provide an empirical analysis of the quality- trade-off and validate the theoretical inter- pretation,
t-SNE (Van der Maaten & Hinton 2008) to visualize features of all training data, compressed subset My. and the original subset M;. First, with the increase quantity Nmb the area of original subset is expanded and can better cover the training data distribution, shown in Fig. 12 This result is consistent with maximizing Nmb for L1. Second, with the decrease of quality q and increase of quantity Nmb the compressed data tend to be distorted and thus become of-distribution, which has been discussed the main text Fig. This result is consistent with enforcing Rg 1 < for L2.
y
D.2
R2 FOR 5-, 10- AND 25-PHASE IMAGENET-SUR
LUCIR
PODNet
10-
10-
B-
50
15-
100
5-Phase
Averaged
90
Averaged
d 10
4-
0-
25
50
100
25
50 Quality
75
100
Phase
Quality
Phase
10
10
10
10-
100
8-
100
90
10-Phase
8-
Averaged
6-
75
50
25
4
P.7..
4.
2-
Phase
Quality
Quality
10
V
100
90
8-
100
90
25-Phase
6-
Averaged
8-
6-
25
2-
1-
15
25
50 Quality
100
10
20
25
50
75
100
Phase
Phase
Quallity
Figure 13: We present R, in each incremental phase with various compression qualities (left), and the averaged R2 of all incremental phases (right). From top to bottom are 5-, 10- and 25-phase Stb.eeettSuub.rsectiii..... ImageNet-Sub, respectively.
22

---

Published as a conference paper at ICLR 2022
We present Rg in each incremental phase and the averaged Rg of all incremental phases for 5-, 10- and 25- 15ase ImageNet-sub in Fig 13 Based on the principle in Eq. (5) (we set = 0.5 as the threshold of R2), it can be clearly seen that 50 and 75 is good quality for LUCIR and PODNet, respectively. Also, the determined quality is the same for different numbers of splits, which consistent with the results of grid search in Fig.
D.3 AANETS AND DDE
LUCIR w/ AANets
LUCIR w/ DDE
100
90
8-
75
8-
8-
25
Averaged
25
Averaged
4
2-
2-
100
Phase
Quality
Quality
Figure 14: Determine the compression quality for AANets and DDE. We present Rg in each incremental phase (left), and the averaged Ry of all incremental phases (right) for 5-phase ImageNet- sub. The quality 100 refers to the original data without compression.
Similar to LUCIR and PODNet, we apply the method described in Sec. 4.2 to determine the com- pression quality for AANets and DDE. Since both AANets and DDE only release their official implementation on LUCIR, here we focus on LUCIR w/ AANets and LUCIR w/ DDE. We present R2 each incremental phase and the averaged R. of all incremental phases for 5-phase ImageNet- sub in Fig. 14 The determined qualities for both LUCIR w/ ANeet and LUCIR w/ DDE are consistent different incremental phases, and are the same as that of LUCIR.
D.4 STATIC VS DYNAMIC QUALITY
For ImageNet-sub with randomly-split classes, whether [R, - 1 < of each quality q consistent among incremental phases and their average (see Fig. and Fig. 14). Thus, the determined quality by our method usually serves as static hyperparameter
continual learning, which has been validated by the results of gird search (see Fig. 2). Now, we wonder in which case the determined quality dynamic, and whether using dynamic quality better than using a static one. Intuitively, the determined quality might be affected by the similarity of incremental classes, because the model learned predict similar classes might be more sensitive to the image distortion caused by data compression.
Since the ImageNet classes are ranked by their semantic similarity, we select 10 adjacent classes as a superclass, and construct 10 adjacent (similar) or remote (dissimilar) superclasses (detailed in Appendix B. Simi- lar to the 5-phase ImageNet-sub, we first learn 5 superlcasses as the initial phase, and then incrementally learn one superclass per phase with LUCIR. The results are presented in Fig. [5] For dissimilar superclasses, whether
Accuracy Quality
Averaged R2 Quality
Rg Phase Py-se
10-
Disimllar Superciass
74-
Averagen P.
10-
50
25
73-
Avernged
72-
Quality
Averaged Accurncy 1%)
60
10-
12-
100
90
Siniler Superclase
66-
Averaged R,
10-
75
50
RR - 1| < is consistent in each in- cremental phase and their average, so the determined quality is stable at 50, validated by the results of grid search.
Rep
1.5
Rig 1.55
25 30
25
50
Quality
Quality
Phose
For similar superclasses, the quality determined by the averaged R, is 75, where using stable quality of 75 in-
Figure 15: Continual learning of ten similar or dissimilar superclasses with LUCIR.
23

---

Published as a conference paper at ICLR 2022
deed achieves the best performance in grid search. On the other hand, it can be clearly that [R75 - > e before phase 3 while R75 1| < с after it. Then we use the dynamically-detemme quantity and quantity for cach incremental phase. However, the dynamic quality and quantity result in severe data imbalance, so the performance far lower than using a static quality of either 50, 75 or 90. A promising further work is to alleviate the data imbalance for the scenarios where the determined quality is highly dynamic.
E ADDITIONAL RESULTS
E.1 COMPUTATIONAL COST
In Table we present the detailed results of computational cost and averaged incremental accuracy of LUCIR, LUCIR w/ AANets, LUCIR w/ DDE and LUCIR w/ Ours for CUB-200-2011 and ImageNet-sub.
Table 3: Comparison of computational cost and averaged incremental accuracy. We run each baseline with one Tesla V100.
CUB-200-2011
ImageNet.sub
Methods
S-phase
10-phase
25-phase
5-phase
10-phase
25-phase
LUCIR
1.97 h 44.663%
3,60 h 45.5%
8.03 h 45.4%
8.65 b/ 70.84%
10.47 h/ 68.32%
15.63 h/ 61.44%
LUCIR w/ AANets
3.52 h 46.87%
5.56 h 47, 7,.3%%
14.34 / 47.35%
21.94 h 72.55%
40.58 h 69.22%
91.26 h 67 .00%
45.866
13.09 46.48%
466.56%
55.36 36
72.34%
h/ 70.20%
66.31%
LUCIR w/ Ours
2.70 46.68%
5,00 h 47.28%
11.00 h 48.00%
9.09 b 73.56%
11.27 7/7.70%
18.83 h/ 70.53%
E.2 STORAGE SPACE
In Table we present the detailed results of different storage space.
Table 4: Averaged incremental accuracy (%) ImageNet-sub. The storage space of the memory buffer is limited to the equivalent of 10, 20, 40 and 80 original images per class, respectively. The results of LUCIR and PODNet are reproduced from their afficially-releaseee codes.
Storage Space
10
20
40
80
LUCIR
68.83
70.90
72.64
73.56
Ours
71.39/+2.55
73.56
+2.66
74.37
+1.73
75.74/ +2.19
5-phase
PODNet
74.99
76.44
78.28
79.18
Ours
76.55/
78.08
+1.64
79.43 /1.115
00.17/40.99
LUCIR
64.64
68.31
70.84
72.56
Ours
68.13/+30 +3.49
72.70/
+4.39
74.11/+3.28
0..09/43.5 /3.55
10-phase
PODNet
70.73
73.91
76.63
78.26
Ours
72.78/+22 +2.05
76.02/42.11
77.82/+1.19
79.27 / +1.01
LUCIR
58.98
64.46
67.55
70.12
Ours
67.75/+5.77
/6.007
/6.07
/6.09/
25 25-phase
PODNet
59.41
67.17
72.57
76.21
w/ Ourx
64.99/+5.58
72.27
/
76.61
/+4.04
78.82/42.61
E.3 AVERAGED FORGETTING
In addition to averaged incremental accuracy, we evaluate the averaged forgetting, which is calculated by averaging the test accuracy of each task minus its highest test accuracy achieved in continual learning (Caccia et al.] 2020). In Table 5 present the averaged forgetting of classes in the initial phase, which suffers from the most severe forgetting. can be clearly seen that the averaged forgetting is largely alleviated by each backbone.
24

---

Published as a conference paper at ICLR 2022
Table 5: Averaged forgetting (%) of classes in the initial phase. The storage space is limited to the equivalent of 20 original images per class. DDE (Hu al. 2021 and AANets (Liu et al. 2021a) are reproduced from their officially-released code.
CUB-200-2011
ImageNet-sub
Mathod
S-phase
10-phase
25-phase
5-phase
10-phase
25-phuse
LUCIR (Hoe etal. 2019
-2.86
-4.6
-4.72
-15.08
-17.32
-22.40
w/ Oury
-0.84
-1.08
-0.75
-1.111
-14.64
-17.96
w/ AANets (Reproducod)
-10.37
-11.78
-9.54
-12.8
w/ AANets Ourz
-1.40
-3.77
-6.82
-7.46
-8.13
-12.32
w/ DDE (Repoduced)
-141
-5.24
-3.91
14.02
-17.68
-23.33
w/ DDE
-0.71
-11.24
-12.36
-17.80
E,4 FIXED MEMORY BUDGET
Here we evaluate memory replay with data compression (ours) under fixed memory budget (i.e.. storage space) on ImageNet-sub. Following a widely-used setting (Rebuffi et al. (2017] Wu et al. 2019), the memory budget (i.e., storage space) is limited to equivalent of 2000 original images (20 original images per class total of 100 classes in ImageNet-sub), We further evaluate much smaller memory budget, fixed to 1000 original images. Under such a fixed memory budget, ours substantially boosts the performance of each backbone approach shown in T
Table 6: Averaged incremental accuracy (%) on ImageNet-sub, under fixed memory budget of 1000 or 2000 original images.
1000 Original Images
2000 Original Images
Method
5-phase
10-phase
25.phase
5-phase
10-
25.phase
LUCIR Hou et 2019
70.16
66.32
62.07
71.87
w/ Ours
72.37
69.63
68.16
73.73
73.07
72.02
w/ DDE (Reproduced)
72.61
70.82
65.58
73.47
71.92
66.70
w/ DDE Ours
74.40
72.62
69.03
75.35
73.93
71.17
E.5 LESS COMPRESSED SAMPLES
In Table present the results of LUCIR with different numbers of compressed samples ImageNet-sub. Similar to the main text, we select the JPEG quality of 50 for LUCIR, where the storage space of 20 original images can save 85 such compressed images. Memory replay of 85 compressed images of quality 50 achieves a much better performance than that of 20 original images. When reducing the quantity from 85 70, 55 and 40, the accuracy will also decline. However, memory replay of 40 compressed images of quality 50 still outperforms that of 20 original images, where the average memory can be saved by 52.94%.
Table Averaged incremental accuracy (%) of LUCIR on ImageNet-sub with different numbers of compressed data. "5-phase", "10-phase" and "25-phase" refer to the accuracy of 5-, 10- and 25-phase ImageNet-sub, respectively.
Quality
50
50
50
50
Original
Quantity
85
70
55
40
20
Total Storage
100%
82.35%
64.71%
47.06%
100%
................................
5-phase
73.63
73.22
72.79
72.29
72.06
10-phase
72.65
72.01
71.35
70.16
68.59
25-phase
70.38
68.37
66.90
65.33
63.27
25


Reviews:
[{"review_id": "p-N-d65GKKt", "reviewer": "Reviewer_iLt4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n-\tThe empirical comparisons appear to be performed to good standards.\n-\tThe results are convincing, even if not very surprising.\n-\tPractical benefit of the proposed memory replay with compression approach is demonstrated on a large scale object detection task.\n\nWeaknesses / suggestions:\n-\tThe claim that memory replay with compression is neglected is somewhat too strong. In particular, I think there are two existing lines of work that would be good to discuss:\n* Firstly, there is highly related previous work addressing the problem of online continual compression: http://proceedings.mlr.press/v119/caccia20a.html\n* Secondly, one motivation for generative replay (rather than replaying stored samples) is that learning a generative model can result in a compressed representation of the original data (even if in practice this is not always the case).\n-\tSection 3 is not completely clear and somewhat confusing. Based on the description in this section, I would think that this paper deals with domain-incremental learning (https://arxiv.org/abs/1904.07734) or the new instances setting (http://proceedings.mlr.press/v78/lomonaco17a.html), but from the rest of the paper I understand it deals with class-incremental learning / the new classes setting.\n-\tThe paper generously uses the term state-of-the-art (SOTA). In almost all cases, I don’t think the use is fully justified. I also don’t think the use of this term is relevant or necessary for this paper, and I would recommend removing most if not all mentions of SOTA in the paper.\n", "summary_of_the_paper": "This paper studies the problems of classification and object detection from natural image datasets in a continual learning setting, whereby the different classes/objects to be learned are not observed together but sequentially. It is assumed that a memory buffer of a certain size is available to store data in. The strategy used by the paper is to fill this memory buffer with compressed data samples (with JPEG) rather than the original images. Such compression introduces a quantity-quality trade-off, which this paper empirically analyses. Additionally, the paper proposes an automated way to select the amount of data compression based on determinantal point processes.", "summary_of_the_review": "I think this is a solid paper. It is generally clear, well-written and insightful, even if its contributions are not surprising or technically challenging. I support acceptance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "oHerUklN1N2", "reviewer": "Reviewer_kRF2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths :\n\n1. This proposed method demonstrates its advantages in realistic continual learning scene.\n2. The experiment of this paper is sufficient.\n3. The method is easy to follow, and the performance is improved to a certain extent.\n4.The paper is well written, and the supplementary material is abundant.\nWeaknesses :\n1. The novelty of this paper is incremental. \n2. Only the average accuracy is reported. There is a lack of some other common used metrics like average forgetting.\n", "summary_of_the_paper": "In this work, the authors propose memory replay with data compression, which is both an important yet neglected baseline and a promising direction for continual learning. Using a naive technique of data compression with a properly selected quality, the proposed method can achieve the SOTA performance in a time-efficient and plug-and-play way. Since the compression quality is highly nontrivial for the efficacy of memory replay, the authors provide a novel method based on determinantal point processes (DPPs) to determine it efficiently, and validate their method in both class-incremental learning and semi-supervised continual learning of object detection.", "summary_of_the_review": " The novelty of this paper is incremental but the experiment of this paper is sufficient.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "S-jYMC8q7as", "reviewer": "Reviewer_gA5x", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1. The paper is well motivated. The idea is simple but makes sense in the context of memory based continual learning. \n\n2. The experiment to show the tradeoff between the quality and quantity of compressed data is also interesting. It might be expected but the authors conduct a detailed analysis.\n\n3. The experiments are also extensive which show the benefits of the proposed approach.\n\nWeaknesses:\n\n1. The motivation of the proposed method is not that clear. Why maximize the conditional likelihood is a good way to select q? \n\n2. In continual learning, we would like the samples in the memory to represent the old data distribution,  why the proposed objective based on DPP can achieve this?\n\n\nMore questions:\n\n\n1. Also, Figure 3 is a little confusing, which one is dark dot and which is light dot?\n\n2. Why  keeping the feature volume is a good criteria for selecting q?\n\n3. Is it possible to store examples with mixed quality?", "summary_of_the_paper": "In this paper, the authors propose a data compression method for memory-replay based continual learning algorithms. With the data compression method,  more old training samples can be stored in the memory to better capture old data distribution. However, there is a trade-off between the quality and quantity of compressed data, the authors propose to use determinantal point processes to determine the quality of the data compression.  Extensive experiments show that with the proposed method, a naive compression method can achieve the SOTA on several continual learning benchmarks. ", "summary_of_the_review": "In this paper, the authors propose to compress the data for memory-based continual learning. Since there is a tradeoff between quantity and quality of the compressed data, the authors propose a novel method to decide the quality of the data. The idea is well-motivated and novel for continual learning. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RnYUXnCJMy", "reviewer": "Reviewer_iR3u", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths \n1. The problem of catastrophic forgetting has gained considerable attention.\n\n2. The paper is written well and easy to follow.\n\n3. Experiments support the claims.\n\nWeaknesses:\n\n1. The idea is not novel.\n\n2. Unnecessary complexities in the implementation of the idea.\n\n3. Analytical and ablative experiments are limited.", "summary_of_the_paper": "In this paper, the problem of catastrophic forgetting for continual learning is explored. The core idea is to benefit from data compression to reduce the space to store data samples and then replay the reconstructed data points that are built using the compressed versions. JPEC method has been explored empirically to determine an optimal compression rate through solving an optimization problem. Experiments on three benchmark datasets are performed to demonstrate that the method is effective.", "summary_of_the_review": "1. The proposed idea is not novel. It is somewhat trivial that whenever we have storage limitations, data compression is going to be helpful. This on its own is not an idea at the level of a top-tier venue.\n\n\n2. I find subsection 4 to be an unnecessary section to make the idea look non-trivial but I think it only is complicating the method unnecessarily.  A naive grid search can do the job. The justification that the authors have provided is that grid search is of \"huge computational cost\". But first of all, it is not clear how large this computational cost is. Second, even if it is large, it is not going to be a huge burden for continual learning. Because you can determine q one time before starting model execution. This is what we do in most cases when we want to tune a hyperparameter. There is no theoretical contribution in that section either and along with the appendices, only known results are rewritten. As a result, the idea contribution of this manuscript is highly limited.\n\n\n3. In experiments, it is mentioned that 20 samples per class are stored. However, this is going to lead to a memory buffer that is going to grow as more classes are used. The appropriate way to do this is to consider a memory buffer with a fixed size and then discard samples as new tasks are learned to replace a portion of old samples with new samples.\n\n4. In the results, standard deviation should be added to make the comparison more informative.\n\n5. It is not very clear how much advantageous data compression would be for continual learning. An additional analytic experiment can be to show that how much memory can be saved by storing less compressed samples while getting no considerable performance degradation.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This works considers limitations of rehearsal-based methods in the context of continual learning (classification and object detection). Rehearsal-based methods provide a strong baseline, but a loss in predictive performance arises when the memory is limited in size. The authors propose to leverage compression (JPEG) to increase the number of data (images) stored in the memory. The approach is evaluated in the context of an autonomous driving application.

The additional experiments conducted by the authors were highly appreciated and helped clarify open questions (e.g., class-incremental learning set-up, DPP objective to determine size of the memory, quantity vs quality of compressed data, etc.). The authors addressed the issues raised by three out of four reviewers, who did not have further comments. The remaining reviewer found that the methodological contributions of this paper, namely of using compression in the context of CL, was pretty straightforward. However, the authors addressed the concerns raised by the reviewer regarding the selection of the compression quality q as far as I am concerned and conducted additional experiments to further demonstrate the usefulness of the approach. I would encourage the authors to include this discussion in the final version of the paper. I would also encourage them to include the additional experiments they conducted with fixed memory size and amount of memory that can be saved.
Decision: Accept (Poster)