M EMORY
R EPLAY
WITH
D ATA
C OMPRESSION
FOR
C ONTINUAL L EARNING
Anonymous authors
Paper under double-blind review
A BSTRACT
To continually learn a sequence of tasks, deep neural networks need to overcome catastrophic forgetting of the past.
Memory replay of representative old training samples has been shown as an effective solution, and achieves the state-of-the-art (SOTA) performance.
However, existing memory replay approaches are mainly built on a small memory buffer containing a few original data, which cannot fully characterize the old data distribution.
In this work, we propose memory replay with data compression, which is both an important baseline and a promising direction for continual learning.
Data compression can largely reduce the storage of old training samples and thus increase their amount that can be stored in the memory buffer.
We empirically validate that there is a trade-off between the quantity and quality of compressed data, which is highly nontrivial for the efﬁcacy of memory replay.
Then we propose a novel method based on determinantal point processes (DPPs) to efﬁciently determine an appropriate compression quality after learning the currently-arrived training samples.
Across several benchmarks of class-incremental learning, using a naive data compression algorithm with a properly selected quality can largely improve the performance of recent strong baselines in a plug-and-play way.
We further demonstrate the advantages of our proposal in realistic continual learning of object detection for autonomous driving, where the incremental data are extremely large-scale with high storage cost.
I NTRODUCTION
The ability to continually learn numerous tasks and infer them together is critical for deep neural networks (DNNs), which needs to mitigate catastrophic forgetting (McCloskey & Cohen, 1989) of the past.
Memory replay of representative old training samples (referred to as memory replay ) has been shown as an effective solution, and achieves the state-of-the-art (SOTA) performance (Hou et al., 2019).
Existing memory replay approaches are mainly built on a small memory buffer containing a few original data, and try to construct and exploit it more effectively.
However, due to the low storage efﬁciency of saving original data, this strategy of building memory buffer will lose a lot of information about the old data distribution.
On the other hand, this usually requires huge extra computation to make better use of the memory buffer to mitigate catastrophic forgetting, such as by learning additional parameters (Liu et al., 2020b) or distilling old features (Hu et al., 2021).
Different from “artiﬁcial” memory replay in DNNs, a signiﬁcant feature of biological memory is to encode the old experiences in a highly compressed form and replay them to overcome catastrophic forgetting (McClelland, 2013; Dragoi & Buzs´aki, 2006; Davidson et al., 2009; Carr et al., 2011).
Thus, a large amount of information about the past can be maintained in a small storage space as comprehensively as possible, and ﬂexibly retrieved.
Inspired by the compression feature of biological memory replay, we propose memory replay with data compression, which can largely increase the amount of old training samples that can be stored in the memory buffer by reducing their storage cost.
As the operation of data compression only needs a small amount of extra computation, this strategy can both more effectively and more efﬁciently recover the old data distribution.
Given a limited storage space, data compression introduces an additional degree of freedom to explicitly balance the trade-off between quality and quantity for memory replay.
With a properly selected quality, using a naive JPEG compression algorithm (Wallace, 1992) can achieve comparable or more performance improvements with the lowest extra computation than the recent strong approaches
(Fig. 1, purple arrow), and can further boost their performance (Fig. 1,
However, it is often computation-inefﬁcient to empirically determine the compression quality, which also requires repeated access to a task sequence (or a sub-sequence) 1 .
We propose a novel method based on determinantal point processes (DPPs) to efﬁciently determine it without extra training.
Further, we demonstrate the advantages of our proposal in realistic applications such as continual learning of object detection for autonomous driving, where the incremental data are extremely large-scale.
Our contributions include: (1) We present that memory replay with data compression is both an important baseline and a promising direction for continual learning; (2) We empirically validate that the trade-off between the quality and quantity of compressed data is highly nontrivial for memory replay, and provide a novel method to efﬁciently determine an appropriate compression quality without extra training; (3) Extensive experiments show that using a naive data compression algorithm with a properly selected quality can largely improve the efﬁcacy of memory replay and boost the performance of recent strong baselines in a plug-and-play way.
R ELATED W ORK
Continual learning needs to overcome catastrophic forgetting of the past when learning a new task.
Regularization-based methods (Kirkpatrick et al., 2017; Chaudhry et al., 2018) approximated the importance of each parameter to the old tasks and selectively penalized its changes.
Architecturebased methods (Rusu et al., 2016) allocated a dedicated parameter subspace for each task to prevent mutual interference.
Replay-based methods (Rebufﬁ et al., 2017; Shin et al., 2017) approximated and recovered the old data distribution.
In particular, memory replay of representative old training samples (referred to as memory replay ) can generally achieve the best performance in class-incremental learning (Liu et al., 2021a;
Hu et al., 2021) and in numerous other continual learning scenarios, such as few-shot (Tao et al., 2020b), semi-supervised (Wang et al., 2021), and unsupervised continual learning (Khare et al., 2021), and continual learning of audio tasks (Ehret et al., 2020).
Existing works in memory replay are mainly built on a small memory buffer containing a few original data, and attempt to construct and exploit it more effectively.
As the pioneer work, iCaRL (Rebufﬁ et al., 2017) proposed a general protocol of memory replay for continual learning.
To better construct the memory buffer, Mnemonics (Liu et al., 2020c) parameterized the original data and made them optimizable, while TPCIL (Tao et al., 2020a) constructed an elastic Hebbian graph (EHG) optimized by competitive Hebbian learning.
On the other hand, BiC (Wu et al., 2019), LUCIR (Hou et al., 2019),
PODNet (Douillard et al., 2020), DDE (Hu et al., 2021) and AANets (Liu et al., 2021a) attempted to more effectively exploit the memory buffer, such as by mitigating the data imbalance between old and new classes (Hou et al., 2019; Wu et al., 2019;
Hu et al., 2021).
In contrast to saving original data, numerous efforts attempted to remember the old data distribution more efﬁciently.
One solution is based on continual learning of a generative model, which can be used to replay generated data (Shin et al., 2017; Wu et al., 2018) or compress old training data (Caccia et al., 2020).
However, continual learning of such a generative model is extremely challenging, which limits its applications to relatively simple domains, and is generally computation-inefﬁcient.
Another solution is based on feature-replay: GFR (Liu et al., 2020a) replayed generated features by learning a feature generator from a feature extractor, but the feature extractor suffered from catastrophic forgetting since it was incrementally updated.
REMIND (Hayes et al., 2020) saved the old features and reconstructed the synthesized features for replay, but it froze the majority of feature extractor after learning the initial phase, which limits the learning of representations for incremental tasks.
Data compression aims to improve the storage efﬁciency of a ﬁle, including lossless compression and lossy compression.
Lossless compression needs to perfectly reconstruct the original data from the compressed data, which limits its compression rate (Shannon, 1948).
In contrast, lossy compression can achieve a much higher compression rate by degrading the original data, so it has been broadly used in realistic applications.
Representative hand-crafted approaches include JPEG (or JPG)
(Wallace, 1992), which is the most commonly-used algorithm of lossy compression (Mentzer et al., 2020),
WebP (Lian & Shilei, 2012) and JPEG2000 (Rabbani, 2002).
On the other hand, neural compression approaches generally rely on optimizing Shannon’s rate-distortion trade-off, through RNNs (Toderici et al., 2015; 2017), auto-encoders (Agustsson et al., 2017) and GANs (Mentzer et al., 2020).
C ONTINUAL L EARNING P RELIMINARIES
We consider a general setting of continual learning, which needs to incrementally learn numerous tasks t = 1 , 2 , 3 , ... from their task-speciﬁc training dataset D t = { ( x t,i , y t,i ) } N t i =1 , where D t is only available when learning task t , ( x t,i , y t,i ) is a data-label pair and N t is the number of such training samples.
For classiﬁcation tasks, the training samples of each task might be from one or several new classes.
At test time, all the classes ever seen are evaluated, and the classes from different tasks need to be distinguished. This setting is also called class-incremental learning (van de Ven & Tolias, 2019).
Suppose a DNN with parameter θ has incrementally learned T tasks.
When learning a new task, since the old training datasets (cid:83) Tt
=
1 D t are unavailable, the DNN will adapt the learned parameters to ﬁt D T +1 , and tend to catastrophically forget the old tasks (McCloskey & Cohen, 1989).
An effective solution of overcoming catastrophic forgetting is to select and store representative old training samples D mbt =
{ ( x t,i , y t,i ) } N mbt i =1 in a small memory buffer ( mb ), and replay them when learning the new task.
For classiﬁcation tasks, mean-of-feature is a widely-used strategy to select
D mbt (Rebufﬁ et al., 2017; Hou et al., 2019).
After learning each task, features of the training data can be obtained by the learned embedding function F eθ ( · ) , normalized under L 2 -norm.
In each class, several data points nearest to the mean-of-feature are selected into the memory buffer.
Then, the training dataset of task T + 1 becomes D (cid:48)
T +1 =
D T +1 (cid:83) D mb 1: T , including both new training samples
D T +1 and replay samples D mb 1: T = (cid:83) Tt =1 D mbt , so as to prevent forgetting of the old tasks.
However, due to the limited storage space, only a few original training samples can be stored in the memory buffer, namely, N mbt (cid:28) N t .
Although numerous memory replay approaches attempted to more effectively exploit the memory buffer, such as by alleviating the data imbalance between the old and the new training samples, this strategy of building memory buffer will lose a lot of information about the old data distribution.
Several recent approaches attempted to improve the efﬁcacy of memory replay through learning additional parameters (Liu et al., 2020b) or distilling old features (Hu et al., 2021), but resulting in huge computation cost.
M ETHOD
In this section, we ﬁrst present memory replay with data compression for continual learning.
Then, we empirically validate that there is a trade-off between the quality and quantity of compressed data, which is highly nontrivial for memory replay, and propose a novel method to determine it efﬁciently.
4. M EMORY R EPLAY WITH D ATA C OMPRESSION
Inspired by the biological memory replay that is in a highly compressed form (Carr et al., 2011), we propose an important baseline, that is, using data compression to increase the amount of old training samples that can be stored in the memory buffer, so as to more effectively recover the old data distribution.
Data compression can be generally deﬁned as a function F cq ( · ) of compressing the original data x t,i to x q,t,i = F cq ( x t,i ) with a controllable quality q .
Due to the smaller storage cost of each x q,t,i than x t,i , the memory buffer can maintain more old training samples for replay, namely,
In analogy to the learning theory for supervised learning, we argue that continual learning will also beneﬁt from replaying more compressed data, assuming that they approximately follow the original data distribution.
However, the assumption is likely to be violated if the compression rate is too high.
Intuitively, this leads to a trade-off between quality and quantity: if the storage space is limited, reducing the quality q of data compression will increase the quantity N mbq of compressed data that can be stored in the memory buffer, and vice versa.
Although lossless compression can perfectly reconstruct original data from compressed data, its compression rate is generally much lower than that of lossy compression (Shannon, 1948), so we mainly consider lossy compression for memory replay.
Here we evaluate the proposed idea by compressing images with JPEG (Wallace, 1992), a naive but commonly-used lossy compression algorithm.
JPEG can save images with a quality in the range of [1 , 100] , where reducing the quality results in a smaller ﬁle size.
Using a memory buffer equivalent to 20 original images per class (Hou et al., 2019), we make a grid search of the JPEG quality with representative memory replay approaches, such as LUCIR (Hou et al., 2019) and PODNet (Douillard et al., 2020).
As shown in Fig.
2, memory replay of compressed data with an appropriate quality can substantially outperform that of original data.
However, whether the quality is too large or too small, it will affect the performance.
In particular, the quality that achieves the best performance varies with the memory replay approaches, but is consistent for different numbers of splits of incremental phases.
4. Q UALITY -Q UANTITY T RADE - OFF
Since the trade-off between quality q and quantity N mbq is highly nontrivial for memory replay, it needs to be carefully determined.
Formally, after learning each task from its training dataset D , let’s consider several compressed subsets D mbq = { ( x q,i , y i ) } N mbq i =1 , where q is from a set of ﬁnite candidates
Q = { q 1 , q 2 , q 3 , ... } .
Each D mb q is constructed by selecting a subset D mb
[Equation]
original training samples from D (following mean-of-feature or other principles).
The size N mbq is determined as the maximum number such that the compressed version of D mb ∗ q to a quality q can be stored in the memory buffer (see details in Appendix A).
Thereby, a smaller q enables to save a larger N mbq , and vice versa.
The objective is to select a compressed subset that can best represent D , namely, to determine an appropriate q for memory replay.
The effects of data compression to memory replay depend on the data compression approaches F cq ( · ) , and the memory replay approaches for continual learning of the embedding function F eθ ( · ) .
Since both of them are integrated to the feature embedding f q,i =
F eθ ( F cq ( x i )) , we ﬁrst calculate the feature matrix M cq =
[ ¯ f q, 1 , ¯ f q, 2 , ..., ¯ f q,N mbq ] of each compressed subset D mbq , where each column vector ¯ f q,i is obtained by normalizing the feature embedding f q,i under L 2 -norm to keep || ¯ f q,i || 2 = 1 .
Similarly, we obtain the feature matrix M ∗ q =
[ ¯ f 1 , ¯ f 2 , ..., ¯ f N mbq ] of each original subset D mb ∗ q .
Then, we can analyze the quality-quantity trade-off in two aspects:
On the empirical side, in Fig.
3 we use t-SNE (Van der Maaten & Hinton, 2008) to visualize features of the original subset (light dot), which includes different amounts of original data, and its compressed subset (dark dot), which is obtained by compressing the original subset to just ﬁt in the memory buffer.
With the increase of quantity and the decrease of quality, the area of compressed subset is initially similar to that of original subset and expands synchronously.
However, as a large number of low-quality compressed data occur out-of-distribution, the area of compressed subset becomes much larger than that of its original subset, where the performance also severely declines (see Fig. 2).
On the theoretical side, given a training dataset D , we aim to ﬁnd the compressed subset D mb q that can best represent D by choosing an appropriate compression quality q from its range.
To achieve this goal, we introduce P q ( D mbq | D ) to characterize the conditional likelihood of selecting D mbq given input D under parameter q .
Since the network parameter θ is ﬁxed during compression, and the feature matrix M cq =
F eθ ( D mbq ) , we rewrite P
q
( D mbq | D ) as P q ( M cq | D ) equivalently.
By focusing on maximum likelihood estimation (MLE), we propose to solve
[Equation]
We draw inspirations from determinantal point processes (DPPs), which are elegant probabilistic sampling models in data selection ﬁelds (Kulesza & Taskar, 2012).
In particular, DPPs specify the probabilities for every possible subset with the volume spanned by all elements in this subset, using determinants.
Depending on the nice properties of DPPs (e.g., high-quality diverse selection), our goal can be modeled as a DPP, i.e., a distribution over all subsets of D with cardinality N mbq (detailed in Appendix C.2).
Formally, such a DPP formulates the probability P q
( M cq | D ) as
[Equation]
where | M cq | = N mbq , and L ( D ; q, θ ) is a conditional DPP | D | × | D | kernel matrix that depends on the input D , parameters θ and q .
L M ( D ; q, θ ) (resp.,
L M cq ( D ; q, θ ) ) is the submatrix sampled from L ( D ; q, θ ) using indices from M (resp.
, M cq ).
The numerator deﬁnes the marginal probability of inclusion for the subset M cq , and the denominator serves as a normalizer to enforce the sum of P q ( M cq | D ) for every possible M cq to 1.
Generally, there are many ways to obtain a positive semi-deﬁnite kernel L .
In this work, we employ the most widely-used dot product kernel function, where L M cq =
M c (cid:62) q M cq and L M = M (cid:62) M .
However, due to the extremely high complexity of calculating the denominator in Eq.
(2) (analyzed in Appendix C.1), it is difﬁcult to optimize P q ( M cq | D ) .
Alternatively, by introducing P q ( M ∗ q | D ) to characterize the conditional likelihood of selecting M ∗ q given input D under parameter q , we propose a relaxed optimization program of Eq.
(1), in which we (1) maximize P q ( M ∗ q | D ) since
P q ( M cq | D ) ≤ P q ( M ∗ q | D ) is always satisﬁed under lossy compression; and meanwhile, (2) constrain that P q ( M cq | D ) is consistent with P q ( M ∗ q | D ) .
The relaxed program is solved as follows.
First, by formulating P q ( M ∗ q | D ) similarly as P q ( M cq | D ) in Eq.
(2) (detailed in Appendix C.2), we need to maximize
[Equation]
where the conditional DPP kernel matrix L ( D ; θ ) only depends on D and θ .
For our task, P q ( M ∗ q | D ) monotonically increases with N mbq .
Thus, optimizing L 1 is converted into max q N mbq equivalently, with signiﬁcantly reduced complexity (detailed in Proposition 1 of Appendix C.3).
Second, to constrain that P q ( M cq | D ) is consistent with P q ( M ∗ q | D ) , we propose to minimize
L 2 ( q ) =
[Equation]
where Z q =
[Equation]
q,θ )) .
In particular, det( M ∗(cid:62) q M ∗ q ) has a geometric interpretation that it is equal to the squared volume spanned by M ∗ q (Kulesza & Taskar, 2012), denoted as Vol ∗ q , likewise for det( M c (cid:62) q M cq ) with respect to Vol cq .
Then we deﬁne R q =
Vol cq
Vol ∗ q as the ratio of the two feature volumes.
To avoid computing Z q , we can convert optimizing L 2 into minimizing | R q − 1 | equivalently, since both of them mean maximizing q (detailed in Proposition 2 of Appendix C.4).
Putting L 1 and L 2 together, our method is ﬁnally reformulated as
[Equation]
where (cid:15) is a small positive number to serve as the threshold of R q .
g ( · ) : R → N represents the function that outputs the maximum number (i.e. N mbq ) such that the compressed version of D mb ∗ q to a quality q can be stored in the memory buffer.
Note that we relax the original restriction of minimizing | R q − 1 | by enforcing | R q − 1 | < (cid:15) , since maximizing N mbq for L 1 and maximizing q for L 2 cannot be achieved simultaneously.
Of note, R q is calculated by normalizing the feature volume Vol cq with Vol ∗ q , both of which depend on q (i.e., N mbq ).
Therefore, (cid:15) can largely mitigate the sensitivity of q to various domains and can be empirically set as a constant value (see Sec. 4.3).
Since the function g ( · ) in Eq.
(5) is highly non-smooth, gradient-based methods are not applicable.
Indeed, we solve it by selecting the best candidate in a ﬁnite-size set Q .
Generally, the candidate values in Q can be equidistantly selected from the range of q , such as [1 , 100] for JPEG.
More candidate values can determine a proper q more accurately, but the complexity will grow linearly.
We found that selecting 5 candidate values is a good choice in our experiments.
Once we solve Eq.
(5), a good trade-off is achieved by reducing q as much as possible to obtain a larger N mbq , while keeping the feature volume Vol cq similar to Vol ∗ q .
This is consistent with our empirical analysis in Fig.
[Figure]
In essence, the grid search described in Sec.
4.1 can be seen as a naive approach to determine the compression quality, which is similar to selecting other hyperparameters for continual learning (Fig. 4, a).
This strategy is to learn a task sequence (or a subsequence) using different qualities and choose the best one (Fig. 4, b), which leads to huge extra computation and will be less applicable if the old data cannot be revisited, or the future data cannot be accessed immediately.
In contrast, our method described in Sec.
4.2 only needs to calculate the feature volumes of each compressed subset
D mbq and original subset D mb ∗ q (Fig. 4, c), without additional training.
Now we validate the compression quality determined by our method with the grid search results, where LUCIR and PODNet achieve the best performance on ImageNet-sub at the JPEG quality ofand 75, respectively (see Fig. 2).
We present R q in each incremental phase and the averaged R q of all incremental phases for 5-phase split in Fig.
5 and for 10- and 25-phase splits in Appendix Fig.13.
Based on the principle in Eq.
(5) (we empirically set (cid:15) = 0 . 5 ), it can be clearly seen that 50 and are the qualities chosen for LUCIR and PODNet, respectively, since they are the smallest qualities that satisfy | R q − 1 | < (cid:15) .
Therefore, the quality determined by our method is consistent with the grid search results, but much more computation-efﬁcient.
Interestingly, for each quality q , whether | R q − 1 | < (cid:15) is generally consistent in each incremental phase and the average of all incremental phases.
We further explore the scenarios where R q might be more dynamic in Appendix D.4.
E XPERIMENT
In this section, we ﬁrst evaluate memory replay with data compression in class-incremental learning of large-scale images.
Then, we demonstrate the advantages of our proposal in realistic semi-supervised continual learning of large-scale object detection for autonomous driving.
5. C LASS -I NCREMENTAL L EARNING
Benchmark : We consider three benchmark datasets of large-scale images for continual learning.
CUB-200-2011 (Wah et al., 2011) is a large-scale dataset including 200-class 11,788 colored images of birds with default format of JPG, split as 30 images per class for training while the rest for testing.
ImageNet-full (Russakovsky et al., 2015) includes 1000-class large-scale natural images with default format of JPEG.
ImageNet-sub (Hou et al., 2019) is a subset derived from ImageNet-full, consisting of randomly selected 100 classes of images.
Following Hou et al.
(2019), we randomly resize, crop and normalize the images to the size of 224 × 224 , and randomly split a half of the classes as the initial phase while split the rest into 5, 10 and 25 incremental phases.
We conduct single-head evaluation (Chaudhry et al., 2018) and report the averaged incremental accuracy (detailed in Appendix B.1).
We further present the metric of averaged forgetting in Appendix E.3.
Implementation :
The implementation is the same as representative memory replay approaches (Hou et al., 2019; Douillard et al., 2020), detailed in Appendix B.1.
We mainly consider a memory buffer where the storage space is equal to 20 original images per class.
We further discuss the effects of buffer size in Fig.
7, a ﬁxed memory budget in Appendix E.4 and less compressed samples in
Appendix E.5.
For data compression, we apply JPEG (Wallace, 1992), a naive but commonly-used lossy compression algorithm, to compress images to a controllable quality in the range of [1 , 100] .
Baseline : We evaluate representative memory replay approaches such as LwF (Li & Hoiem, 2017), iCaRL (Rebufﬁ et al., 2017), BiC (Wu et al., 2019), LUCIR (Hou et al., 2019), Mnemonics (Liu et al., 2020c), TPCIL (Tao et al., 2020a), PODNet (Douillard et al., 2020), DDE (Hu et al., 2021) and AANets (Liu et al., 2021a).
In particular, AANets and DDE are the recent strong approaches implemented on the backbones of LUCIR and PODNet, so we also implement ours on the two backbones.
Since both AANets and DDE only release their ofﬁcial implementation on LUCIR, we further reproduce LUCIR w/ AANets and LUCIR w/ DDE for fair comparison.
Accuracy: We summarize the performance of the above baselines and memory replay with data compression (ours) in Table 1.
For class-incremental learning, AANets and DDE are recent strong approaches, both of which are implemented on the backbones of LUCIR and PODNet.
With the same extra storage space, ours achieves comparable or greater performance improvements than AANets and DDE on the same backbone approaches, and can further greatly boost their performance.
For
CUB-200-2011 with only a few training samples, all old training samples can be saved in the memory buffer with a high JPEG quality of 94.
In contrast, for ImageNet-sub and ImageNet-full, only a part of old training samples can be selected, compressed and saved in the memory buffer, where our
[Table]
method can efﬁciently determine an appropriate quality of data compression for LUCIR and PODNet (see Sec.
4.3 and Appendix D.2), and for AANets and DDE (see Appendix D.3).
Computation Cost: Limiting the size of memory buffer is not only to save its storage cost, but also to save the computation cost of learning all old training samples again.
Here we evaluate the computation cost of
AANets, DDE and memory replay with data compression (ours) on LUCIR (see Fig.
6 for ImageNet-sub and
Appendix Table 3 for CUB-200-2011).
Both AANets and DDE require a huge amount of extra computation to improve the performance of continual learning, which is generally several times that of LUCIR.
In contrast, ours achieves competing or the largest performance improvement but only slightly increases the computation cost.
Buffer Size: We further evaluate the effects of memory buffer size for memory replay with data compression (ours).
As shown in Fig.
7, using the storage space equivalent to 10, 20, 40 andoriginal images per class, ours can consistently improve the performance of LUCIR and PODNet, where the improvement is generally more signiﬁcant for more splits of incremental phases.
5. L ARGE -S CALE O BJECT D ETECTION
The advantages of memory replay with data compression are more signiﬁcant in realistic scenarios such as autonomous driving, where the incremental data are extremely large-scale with huge storage cost.
Here we evaluate continual learning on SODA10M (Han et al., 2021), a large-scale object detection benchmark for autonomous driving.
SODA10M contains 10M unlabeled images and 20K labeled images of size 1920 × 1080 with default format of JPEG, which is much larger than the scale of ImageNet.
The labeled images are split into 5K, 5K and 10K for training, validation and testing, respectively, annotating 6 classes of road objects for detection.
[Table]
Since the autonomous driving data are typically partially-labeled, we follow the semi-supervised continual learning (SSCL) proposed by Wang et al.
(2021) for object detection.
Speciﬁcally, we randomly split 5 incremental phases containing 1K labeled data and 20K unlabeled data per phase, and use a memory buffer to replay labeled data.
The memory buffer size is equal to 100 original images per phase.
Following Han et al.
, we consider Pseudo Labeling and Unbiased Teacher (Liu et al., 2021b) for semi-supervised object detection (the implementation is detailed in Appendix B.2).
Using the method described in Sec.
4.2 with the same threshold (i.e., (cid:15) = 0 . 5 ), we can efﬁciently determine the compression quality for SSCL of object detection, and validate that the determined quality indeed achieves the best performance in grid search (see Fig.
Then, compared with memory replay of original data, memory replay with data compression (ours) can generally achieve several times of the performance improvement on ﬁnetuning, as shown in Table 2.
C ONCLUSION
In this work, we propose memory replay with data compression, which is both an important baseline and opens up a promising new avenue for continual learning.
Using a naive data compression algorithm with a properly selected compression quality, our proposal can greatly improve the performance of recent strong baselines in a plug-and-play way.
Since the compression quality is highly nontrivial for the efﬁcacy of memory replay, we provide a novel method based on determinantal point processes (DPPs) to determine it efﬁciently, and validate our method in both class-incremental learning and semi-supervised continual learning of object detection.
As data compression hardly affects the semantic information (detailed in Appendix A), a potential follow-up work is to regularize the differences between compressed data and original data, so as to further improve the performance of continual learning.
In addition, our work inspires a new application of DPPs, which can provide a theoretical basis for memory replay in continual learning.
E THIC S TATEMENT
This work presents memory replay with data compression, which is an effective yet neglected baseline for continual learning.
It can be used to reduce the storage requirement in memory replay and thus may facilitate large-scale applications of such methods to real-world problems (e.g., autonomous driving).
As a fundamental research in machine learning, the negative consequences in the current stage are not obvious.
R EPRODUCIBILITY S TATEMENT
We ensure the reproducibility of our paper from three aspects.
(1) Experiment: The implementation of our experiment described in Sec.
4.1, Sec.
4.3 and Sec. 5 is further detailed in Appendix B. (2)
Code: Our code is included in supplementary materials.
(3) Theory and Method: A complete proof of the theoretical results described in Sec.
4.2 is included in Appendix C.
A T RADE - OFF BETWEEN Q UALITY AND Q UANTITY
Given a limited memory buffer space, there is an intuitive trade-off between the quality q and quantity N mbq of compressed data.
Here we plot the relation of q and N mbq for ImageNet and SODA10M used in our paper.
As shown in Fig.
9, reducing the quality enables to save more compressed data in the memory buffer, and vice versa.
Then, we present compressed images with various qualities in Fig.
Although reducing the quality tends to distort the compressed data and thus hurts the performance of continual learning, it can be clearly seen that the semantic information of such compressed data is hardly affected.
Therefore, a potential follow-up work is to regularize the differences between compressed data and original data, so as to further improve the performance of memory replay.
B I MPLEMENTATION D ETAIL B. C LASS -I NCREMENTAL L EARNING
Following the same implementation as Hou et al.
; Douillard et al.
(2020); Tao et al.
(2020a);
Liu et al.
(2021a)
; Hu et al.
(2021), we train a ResNet-18 architecture for 90 epochs, with minibatch size of 128 and weight decay of 1 × 10 − 4 .
We use a SGD optimizer with initial learning rate of 0.1, momentum of 0.9 and cosine annealing scheduling.
We run each baseline with one Tesla V100, and measure the computation cost.
We evaluate the performance of class-incremental learning with the averaged incremental accuracy (AIC)
(Hou et al., 2019).
After learning each phase, we can calculate the test accuracy on all classes learned so far in the previous t phases as A t .
Then we can calculate
[Equation]
To evaluate the effects of class similarity on the determined quality of data compression, we select similar or dissimilar superclasses from ImageNet, which includes 1000 classes ranked by their semantic similarity.
Speciﬁcally, we select 10 adjacent classes as a superclass, and constructadjacent (similar) or remote (dissimilar) superclasses.
The index of classes selected in 10 similar superclasses is [1, 2, 3, ..., 100], while in 10 dissimilar superclasses is [1, 2, ..., 10, 101, 102, ..., 110, ..., 910].
B. L ARGE -S CALE O BJECT D ETECTION
For memory replay, we follow Wang et al.
(2021) to randomly select the old labeled data and save them in the memory buffer, which indeed achieves competitive performance as analyzed by Chaudhry et al.
We follow the implementation of Han et al.
(2021) for semi-supervised object detection.
Speciﬁcally, we use Faster-RCNN (Ren et al., 2015) with FPN (Lin et al., 2017) and ResNetbackbone as the object detection network for Pseudo Labeling (Han et al., 2021) and Unbiased
Teacher (Liu et al., 2021b).
For each incremental phase, we train the network for 10K iterations using the SGD optimizer with initial learning rate of 0.01, momentum of 0.9, and constant learning rate scheduler.
The batch size of supervised and unsupervised data are both 16 images.
For Pseudo
Labeling, we ﬁrst train a supervised model on the labeled set in the ﬁrst 2K iterations.
Then we evaluate the supervised model on the unlabeled set.
A bounding box with a predicted score larger than 0.7 in the evaluation results is selected as a pseudo label, which would be used to train the supervised model in the later 8K iterations in each phase.
For Unbiased Teacher, we follow the default setting and change the input size to comply with SODA10M. We run each baseline with 8 Tesla V100.
The performance is evaluated by the metrics ( AP , AP 50 , AP 75 ) used in the prior works (Liu et al., 2021b;
Han et al., 2021).
C T HEORETICAL A NALYSIS C. D ETERMINANTAL P OINT P ROCESSES (DPP S ) P RELIMINARIES
Arising in quantum physics and random matrix theory, determinantal point processes (DPPs) are elegant probabilistic models of global, negative correlations, and offer efﬁcient algorithms for sampling, marginalization, conditioning, and other inference tasks.
A point process P on a ground set V is a probability measure on the power set 2 N , where N = |V| is the size of the ground set.
Let B be a D × N matrix with D ≥ N .
In practice, the columns of
B are vectors representing items in the set V .
Denote the columns of B by B i for i = 1 , 2 , · · · , N .
A sample from P might be the empty set, the entirety of B , or anything in between.
P is called a determinantal point process if, given a random subset Y drawn according to P ( 2 N possible instantiations for Y ), we have for every S ⊆ B ,
[Equation]
where S =
{ B i 1 , B i 2 , · · · , B i | S | } , P ( Y = S ) characterizes the likelihood of selecting the subset
S from B .
For some symmetric similarity kernel L ∈ R N × N (e.g., L = B T B ), where L S is the similarity kernel of subset S (e.g., L = S T S ).
That is, L S is the submatrix sampled from L using indices from S .
Since P is a probability measure, all principal minors det( L S ) of L must be non-negative, and thus L itself must be positive semideﬁnite.
These requirements turn out to be sufﬁcient: any L , 0 (cid:22) L (cid:22) I , deﬁnes a DPP.
L is often referred to as the marginal kernel since it contains all the information needed to compute the probability of any subset S being included in Y .
Hence, the marginal probability of including one element B i is P ( B i ∈ Y )
=
L ii , and two elements B i and B j is P ( B i , B j ∈ Y ) =
L ii L jj − L 2 ij =
P ( B i ∈ Y ) P ( B j ∈ Y ) − L 2 ij .
A large value of L ij reduces the likelihood of both elements to appear together in a diverse subset.
This demonstrates why DPPs are “diversifying”.
Below is several important conclusions about DPPs related to our work (Kulesza & Taskar, 2012).
Geometric Interpretation.
By focusing on L-ensembles in DPPs, determinants have an intuitive geometric interpretation.
If L = B (cid:62) B , then
[Equation]
where the ﬁrst two terms specify the marginal probabilities for every possible subset S , and the last term is the squared | S | -dimensional volume of the parallelepiped spanned by the items in S .
Quality vs. Diversity.
While the entries of the DPP kernel L are not totally opaque in that they can be seen as measures of similarity— reﬂecting our primary qualitative characterization of DPPs as diversifying processes.
In most practical situations, we want diversity to be balanced against some underlying preferences for different items in B .
Kulesza & Taskar (2012) proposed a decomposition of the DPP that more directly illustrates the tension between diversity and a per-item measure of quality.
Speciﬁcally, L = B (cid:62) B can be taken one step further, by writing each column B i as the product of a quality term q i ≥ 0 and vector of normalized diversity features f i ∈ R D
[Equation]
(cid:107) f
[Equation]
(cid:107) = 1 .
Then, L ij = q i f T i f j q j , where f (cid:62) i f j ∈ [ − 1 , 1] is a signed measure of similarity between B i and B j .
Let φ ij =
f
(cid:62) i f j =
L ij √ L ii L jj , and φ = { φ ij } Ni,j =1 .
Then, depending on Eq.
(6), we have
[Equation]
where q i j can be seen as a quality score of an item B i j in B , and φ S is the submatrix sampled from φ using indices from S .
Alternative Likelihood Formulas.
In an L-ensemble DPP, the likelihood of a particular set S ⊆ B is given by
[Equation]
where S (cid:48) is one of all 2 N possible subsets in B , and L S (cid:48) is the submatrix sampled from L using indices from S (cid:48) .
This expression has some nice intuitive properties in terms of volumes, and, ignoring the normalization in the denominator, takes a simple and concise form.
However, as a ratio of determinants on matrices of differing dimension, it may not always be analytically convenient.
Minors can be difﬁcult to reason about directly, and ratios complicate calculations like derivatives.
Such a high computation cost is just one key challenge in DPPs.
Conditional DPPs.
A conditional DPP P ( Y = S | B ) is a conditional probabilistic model which assigns a probability to every possible subset S ⊆ B .
The model takes the form of an L-ensemble:
[Equation]
10) where L ( B ) is a positive semideﬁnite conditional N × N kernel matrix that depends on the input B .
k -DPPs.
A k -DPP is obtained by conditioning a standard DPP on the event that the set Y has cardinality k , which is concerned only with the content of a random k -set (i.e., without the size k ) (Kulesza & Taskar, 2012).
Formally, the k -DPP P kL gives the probability of a particular set S as
[Equation]
where | S | = k and L is a positive semideﬁnite kernel.
Although numerous optimization algorithms have been proposed to solve DPPs problems, the high computation complexity about denominator cannot be avoided generally.
On the one hand, computing denominator is a sum over (cid:0) Nk (cid:1) terms.
On the other hand, computing the determinant of each term through matrix decomposition is with O ( k 3 ) time.
Just as Kulesza & Taskar (2012) claimed, sparse storage of larger matrices is possible for DPPs, but computing determinants remains prohibitively expensive unless the level of sparsity is extreme.
If the dot product kernel function is adopted to compute the kernel L in Eq.
(11), similar to Geometric
Interpretation of standard DPPs above, the probability P kL ( Y = S ) is further deﬁned with volume sampling (Deshpande & Rademacher, 2010), i.e.,
[Equation]
where L = B (cid:62) B , Conv( · ) denotes the convex hull, and Vol( · ) is the k -dimensional volume of such a convex hull.
[Figure]
In this work, we aim to ﬁnd the compressed subset D mbq that can best represent the training dataset D by choosing an appropriate compression quality q from its range.
To achieve this goal, we introduce P q ( D mbq | D ) to characterize the conditional likelihood of selecting D mbq given input D under parameter q .
Since the network parameter θ is ﬁxed during compression, and the feature matrix
M cq =
F eθ ( D mbq ) , we rewrite P
q
( D mbq | D ) as P q ( M cq | D ) equivalently (the constructions of D mbq and M cq are detailed in Fig.
Depending on the nice properties of
DPPs in Sec.
C.1, we formulate our goal as a conditional
DPP, where D is the associated ground set, and M cq is a desired subset in feature space.
Then P q ( M cq | D ) is a distribution over all subsets of D with cardinality N mbq , which is formally called a conditional N mbq -DPP, since | M cq | =
N mbq for any possible subset.
Thus, such a DPP formulates the conditional probability P q ( M cq | D ) as
[Equation]
where L ( D ; q, θ ) is a conditional DPP | D | × | D | kernel matrix that depends on the input D parameterized in terms of parameters θ and q .
L M ( D ; q, θ ) (resp.,
L M cq ( D ; q, θ )) ) is the submatrix sampled from L ( D ; q, θ ) using indices from M (resp.
, M cq ).
Similarly, P q ( M ∗ q | D ) can be formulated by a conditional N mbq -DPP as
[Equation]
(14) where | M ∗ q | = N mbq (the constructions of M ∗ q are detailed in Fig. 11).
Unlike L ( D ; q, θ ) above, the conditional DPP kernel matrix L ( D ; θ ) only depends on D and θ without q , since q is ﬁxed at its maximum, i.e., without compression.
L M ∗ ( D ; θ ) (resp., L M ∗ q ( D ; θ ) ) is the submatrix sampled from
L ( D ; θ ) using indices from M ∗ (resp., M ∗ q ).
Differences: Of note, two important differences between standard k -DPPs and our N mbq -DPP lie in the optimization variable and objective.
For this work, we need to ﬁnd an optimal set cardinality (i.e.,
N mbq ), while it is ﬁxed for standard k -DPPs.
Although both of them can ﬁnally determine a desired subset with the maximum volume, our N mbq -DPP can uniquely determine it by N mbq , while standard k -DPPs obtain it by maximizing Eq.
C. T HE F IRST G OAL
First, we need to maximize
[Equation]
where | M ∗ q | = N mbq .
L ( D ; θ ) is a conditional DPP | D | × | D | kernel matrix that depends on the input D parameterized in terms of θ .
L M ∗ ( D ; θ ) (resp., L M ∗ q ( D ; θ ) ) is the submatrix sampled from
L ( D ; θ ) using indices from M ∗ (resp., M ∗ q ).
The numerator deﬁnes the marginal probability of inclusion for the subset M ∗ q , and the denominator serves as a normalizer to enforce the sum of
P q ( M ∗ q | D ) for every possible M ∗ q to 1.
Here we employ the commonly-used dot product kernel function, where L M ∗ q = M ∗(cid:62) q M ∗ q and L M ∗ =
M ∗(cid:62) M ∗ .
Conv( · ) denotes the convex hull, and Vol( · ) is the N mbq -dimensional volume of such a convex hull.
Eq.
(15) provides a geometric interpretation that the conditional probability P q ( M ∗ q | D ) is proportional to the squared volume of M ∗ q with respect to N mbq , denoted as Vol ∗ q .
For our task, P q ( M ∗ q | D ) monotonically increases with N mbq .
Thus, optimizing L 1 is converted into max q N mbq equivalently (detailed in Proposition 1 as below).
Proposition 1 For any matrix X =
( x 1 , x 2 , · · · , x n ) , where x i ∈ R
d is the i -th column of X , d > n and X (cid:62) X (cid:54) =
I , deﬁne M = { i 1 , i 2 , · · · , i m } as a subset of [ n ] containing m elements, and
X M as the submatrix sampled from X using indices from M .
Then for
[Equation]
we can ensure that the following two optimization programs
[Equation]
and
[Equation]
are equivalent.
Below is one critical lemma for the proof of Proposition 1.
Lemma 1 For any matrix X =
( x 1 , x 2 , · · · , x n ) , where x i ∈ R
d is the i -th column of X and d > n , deﬁne M =
{ i 1 , i 2 , · · · , i m } as a subset of [ n ] containing m elements, and X M as the submatrix sampled from X using indices from M .
Then for
[Equation]
we can ensure that
[Equation]
This means when |M| = n , the probability P ( M ) takes the maximum value.
Proof.
From the deﬁnition, we have
[Equation]
If |M (cid:48) | = m < n , we have (cid:0) nm (cid:1) >
1 possible options for subset X M (cid:48) , where det (cid:0) X (cid:62)M (cid:48)
X M (cid:48) (cid:1) ≥ 0 .
Only if |M| = n , then (cid:0) nm (cid:1) = 1 and the probability P ( M ) will deﬁnitely take the maximum value, i.e. P ( M ) = 1 .
In addition, X (cid:62) X has full rank due to d > n , and then det( X (cid:62) X ) > 0 always holds.
That is, P ( M ) = 1 can be guaranteed only when selecting all n items together.
This means [ n ] = argmax M P ( M ) .
It completes the proof.
Proof of Proposition 1 From the deﬁnition in Deshpande & Rademacher (2010), we have
[Equation]
where Conv( · ) denotes the convex hull, and Vol( · ) is the m -dimensional volume of such a convex hull.
Here, from the monotonicity side, it can be easily veriﬁed that for X M ∪ x
i m +1 , where x i m +1 ∈ (cid:123)
X X M , we have
[Equation]
which means P ( M ) is monotonically increasing with the increase of |M| .
From the extremum side, by using Lemma
[Equation]
we can ensure that the following two optimization programs
[Equation]
and
[Equation]
m |M| have the same optimal value.
Thus, maximizing P ( M ) with respect to |M| can be converted into max m |M| equivalently.
C. T HE S ECOND G OAL
Second, we need to minimize
[Equation] [Equation]
where Z q =
[Equation]
In particular, det( M ∗(cid:62) q M ∗ q ) has a geometric interpretation that it is equal to the squared volume spanned by M ∗ q (Kulesza & Taskar, 2012), denoted as Vol ∗ q , likewise for det( M c (cid:62) q M cq ) with respect to Vol cq .
Then we deﬁne R q =
Vol cq
Vol ∗ q as the ratio of the two feature volumes.
To avoid computing Z q , we can convert optimizing L 2 into minimizing | R q − 1 | equivalently, since both of them mean maximizing q (detailed in Proposition 2 as below).
Below we are going to introduce Proposition 2.
We ﬁrst introduce some necessary assumptions, then we present the theoretical guarantee of Proposition 2.
Assumption 1 We make the following assumptions:
Image dataset: For any image dataset D = ( x 1 , x 2 , · · · , x n ) , where x i is the i -th image in D , deﬁne D q = ( x 1 , x 2 , · · · , x n ) as a compressed image dataset with the compression quality q for D .
The compression quality is bounded as q ∈ Q (Q can be a ﬁnite/inﬁnite set), in which the higher of q , the better of image quality.
Feature matrix:
Denote X = ( f 1 , f 2 , · · · , f n ) ∈ R d × n and X q = ( f q 1 , f q 2 , · · · , f qn ) ∈ R d × n as the feature matrices of D and D q with the same ﬁxed feature extractor, respectively.
Here, d > n , X (cid:62) X (cid:54) =
I and X (cid:62) q X q (cid:54)
=
I .
Selection principle: Deﬁne M = { i 1 , i 2 , · · · , i m } as a subset of [ n ] containing m elements, and X M as the submatrix sampled from X using indices from M .
Likewise, deﬁne M q = { j 1 , j 2 , · · · , j m } as a subset of [ n ] containing m elements, and X M q as the submatrix sampled from X q using indices from M q .
Both of them use the same selection principle.
Buffer size: g ( · ) :
R → N represents the function that outputs the maximum number (i
[Equation]
e. m ) such that the compressed version of D q to a quality q can be stored in the memory buffer.
That is, m is uniquely determined by q .
Proposition 2 Under Assumption 1, for two probabilities
[Equation]
and
[Equation]
we can ensure that the following two optimization programs
[Equation]
det (cid:16)
[Equation]
(cid:62)M
[Equation]
(cid:17) det (cid:0) X (cid:62)M X M (cid:1) − 1 (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) and min q (cid:12)(cid:12)(cid:12)(cid:12) P ( M q )
P ( M ) − 1 (cid:12)(cid:12)(cid:12)(cid:12) are equivalent.
Below are two critical lemmas for the proof of Proposition 2.
Lemma 2 Under Assumption 1, for a ratio
[Equation]
we can ensure that
[Equation]
This means when q = sup Q (i.e, without image compression), | R q − 1 | takes the minimum value.
Proof.
From the deﬁnition, we know | R q − 1 | ≥ 0 .
To minimize | R q − 1 | , we need the ratio R q = 1 , i.e.,
[Equation]
Of note, both X M q and X M are selected from the source dataset D with the same feature extractor, selection principle and set cardinality (i.e., m ).
That is, the only difference between them is image compression with respect to q .
By recurring to the Geometric Interpretation in k -DPPs (i.e., Eq.
(12)), we have
[Equation]
and
[Equation]
Then, Eq.
(18) holds only when X M q and X M have the same convex hull.
Due to great difﬁculty of analytically deﬁning compression function, the two convex hull cannot be mathematically given.
However, by conducting extensive experiments on this task (as shown in Fig. 3, Fig. 5, Fig. 8, Fig.and Fig. 14), we ﬁnd that the volume of X M q is larger than that of X M for a speciﬁc m (i.e., q ).
Additionally, with q decreasing (i.e., m increasing), the volume of X M q is increasing more quickly.
Thus, we have an empirical conclusion that the two volumes are the same only when q = sup Q (without image compression).
It is reasonable since without image compression, the two selection problems about X M q and X M are identical.
This means when q = sup Q , | R q − 1 | takes the minimum value, which completes the proof.
Lemma
Under Assumption 1, for
[Equation]
and
[Equation]
we can ensure that
[Equation]
This means when q = sup Q (i.e, without image compression), (cid:12)(cid:12)(cid:12) P ( M q ) P ( M ) − 1 (cid:12)(cid:12)(cid:12) takes the minimum value.
Proof.
From the deﬁnition, we know (cid:12)(cid:12)(cid:12) P ( M q ) P ( M ) − 1 (cid:12)(cid:12)(cid:12) ≥ 0 .
To minimize (cid:12)(cid:12)(cid:12) P ( M q ) P ( M ) − 1 (cid:12)(cid:12)(cid:12) , we need the ratio
P ( M q ) P ( M ) = 1 .
In fact,
[Equation]
where R q = det
[Equation]
Z q =
[Equation]
By using Lemma 2, we know that when q = sup Q , R q = 1 always holds.
More importantly,
Z q = 1 also holds if q = sup Q , since the numerator and denominator in Z q will be equal without compression.
That is, when q = sup Q , the ratio P ( M q ) P ( M ) =
1 and (cid:12)(cid:12)(cid:12) P ( M q ) P ( M ) − 1 (cid:12)(cid:12)(cid:12) takes the minimum value.
This completes the proof.
Proof of Proposition 2 From Lemma 2 and Lemma 3, we can ensure that under Assumption 1, the following two optimization programs
[Equation]
and min q
[Equation]
have the same optimal value.
Thus, maximizing (cid:12)(cid:12)(cid:12) P ( M q ) P ( M ) − 1 (cid:12)(cid:12)(cid:12) with respect to q can be converted into min q | R q − 1 | equivalently, where R q = det
[Equation]
D E MPIRICAL A NALYSIS D. T -SNE V ISUALIZATION OF N ORMALIZED F EATURES
To provide an empirical analysis of the quality-quantity trade-off and validate the theoretical interpretation, we use t-SNE (Van der Maaten & Hinton, 2008) to visualize features of all training data, the compressed subset M q , and the original subset M ∗ q .
First, with the increase of quantity N mbq , the area of original data is expanded and can better cover the training data distribution, as shown in Fig.
This result is consistent with maximizing N mbq for L 1 .
Second, with the decrease of quality q and increase of quantity N mbq , the compressed data tend to be distorted and thus become out-of-distribution, which has been discussed in the main text Fig.
This result is consistent with enforcing | R q − 1 | < (cid:15) for L 2 .
D. R q FOR 5-, 10- AND 25- PHASE I MAGE N ET - SUB
We present R q in each incremental phase and the averaged R q of all incremental phases for 5-, 10- and 25-phase ImageNet-sub in Fig.13.
Based on the principle in Eq.
(5) (we set (cid:15) = 0 .
5 as the threshold of R q ), it can be clearly seen that 50 and 75 is a good quality for LUCIR and PODNet, respectively.
Also, the determined quality is the same for different numbers of splits, which is consistent with the results of grid search in Fig.
D. AAN ETS AND DDE
Similar to LUCIR and PODNet, we apply the method described in Sec.
4.2 to determine the compression quality for AANets and DDE.
Since both AANets and DDE only release their ofﬁcial implementation on LUCIR, here we focus on LUCIR w/ AANets and LUCIR w/ DDE.
We present
R q in each incremental phase and the averaged R q of all incremental phases for 5-phase ImageNetsub in Fig.
The determined qualities for both LUCIR w/ AANets and LUCIR w/ DDE are consistent in different incremental phases, and are the same as that of LUCIR.
D. S TATIC VS D YNAMIC Q UALITY
For ImageNet-sub with randomly-split classes, whether | R q − 1 | < (cid:15) of each quality q is consistent among incremental phases and their average (see Fig.
and Fig. 14).
Thus, the determined quality by our method usually serves as a static hyperparameter in continual learning, which has been validated by the results of gird search (see Fig. 2).
Now, we wonder in which case the determined quality is dynamic, and whether using a dynamic quality is better than using a static one.
Intuitively, the determined quality might be affected by the similarity of incremental classes, because the model learned to predict similar classes might be more sensitive to the image distortion caused by data compression.
Since the ImageNet classes are ranked by their semantic similarity, we select 10 adjacent classes as a superclass, and construct 10 adjacent (similar) or remote (dissimilar) superclasses (detailed in Appendix B.1).
Similar to the 5-phase ImageNet-sub, we ﬁrst learn 5 superlcasses as the initial phase, and then incrementally learn one superclass per phase with LUCIR.
The results are presented in Fig.
For dissimilar superclasses, whether | R q − 1 | < (cid:15) is consistent in each incremental phase and their average, so the determined quality is stable at 50, validated by the results of grid search.
For similar superclasses, the quality determined by the averaged R q is 75, where using a stable quality of 75 in- deed achieves the best performance in grid search.
On the other hand, it can be clearly seen that | R 75 − 1 | > (cid:15) before phase 3 while | R 75 − 1 | < (cid:15) after it.
Then we use the dynamically-determined quantity and quantity for each incremental phase.
However, the dynamic quality and quantity result in severe data imbalance, so the performance is far lower than using a static quality of either 50, 75 or 90.
A promising further work is to alleviate the data imbalance for the scenarios where the determined quality is highly dynamic
[Table]
E A DDITIONAL R ESULT E. C OMPUTATION C OST
In Table
[Table]
we present the detailed results of computation cost and averaged incremental accuracy of LUCIR, LUCIR w/ AANets, LUCIR w/ DDE and LUCIR w/ Ours for CUB-200-2011 and
ImageNet-sub.
[Table]
E. B UFFER S IZE
In Table
[Table]
we present the detailed results of the effects of memory buffer size.
[Table]
E. A VERAGED F ORGETTING
In addition to averaged incremental accuracy
[Table]
we evaluate the metric of averaged forgetting, which is calculated by averaging the test accuracy of each task minus its highest test accuracy achieved in continual learning (Caccia et al., 2020).
In Table 5, we present the averaged forgetting of classes in the initial phase, which suffers from the most severe forgetting.
It can be clearly seen that the averaged forgetting is largely alleviated by ours on each backbone.
[Table]
E. F IXED M EMORY B UDGET
Here we evaluate memory replay with data compression (ours) under a ﬁxed memory budget on
ImageNet-sub.
Following a widely-used setting (Rebufﬁ et al., 2017; Wu et al., 2019), the memory budget is equivalent to 2000 original images (20 original images per class × a total of 100 classes in ImageNet-sub).
We further evaluate a much smaller memory budget, ﬁxed to 1000 original images.
Under such a ﬁxed memory budget, ours substantially boosts the performance of each backbone approach as shown in Table 6.
accuracy (%) on ImageNet-sub, under a ﬁxed memory budget ofor 2000 original images.
[Table]
E. L ESS C OMPRESSED S AMPLES
In Table
[Table]
we present the results of LUCIR with different numbers of compressed samples on
ImageNet-sub.
Similar to the main text
[Table]
we select the JPEG quality of 50 for LUCIR, where the storage space of 20 original images can save 85 such compressed images.
Memory replay ofcompressed images of quality 50 achieves a much better performance than that of 20 original images.
When reducing the quantity from 85 to 70 , 55 and 40 , the accuracy will also decline.
However, memory replay of 40 compressed images of quality 50 still outperforms that of 20 original images, where the average memory can be saved by 52 .
[Table]