Coreset Selection via Reducible Loss in Continual Learning
Authors: Ruilin Tong
Yuhang Liu
Javen Qinfeng Shi
Dong Gong
Venue: ICLR 2025 Poster (2025)
Keywords: Continual learning
Coreset selection
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Rehearsal-based continual learning (CL) aims to mitigate catastrophic forgetting by maintaining a subset of samples from previous tasks and replaying them. The rehearsal memory can be naturally constructed as a coreset, designed to form a compact subset that enables training with performance comparable to using the full dataset. The coreset selection task can be formulated as bilevel optimization that solves for the subset to minimize the outer objective of the learning task. Existing methods primarily rely on inefficient probabilistic sampling or local gradient-based scoring to approximate sample importance through an iterative process that can be susceptible to ambiguity or noise. Specifically, non-representative samples like ambiguous or noisy samples are difficult to learn and incur high loss values even when training on the full dataset. However, existing methods relying on local gradient tend to highlight these samples in an attempt to minimize the outer loss, leading to a suboptimal coreset. To enhance coreset selection, especially in CL where high-quality samples are essential, we propose a coreset selection method that measures sample importance using reducible loss (ReL) that quantifies the impact of adding a sample to model performance. By leveraging ReL and a process derived from bilevel optimization, we identify and retain samples that yield the highest performance gain. They are shown to be informative and representative. Furthermore, ReL requires only forward computation, making it significantly more efficient than previous methods. To better apply coreset selection in CL, we extend our method to address key challenges such as task interference, streaming data, and knowledge distillation. Experiments on data summarization and continual learning demonstrate the effectiveness and efficiency of our approach.

Published as a conference paper at ICLR 2025
CORESET SELECTION VIA REDUCIBLE Loss IN CONTINUAL LEARNING
Ruilin Tong', Yuhang Liu², Javen Qinfeng Shi?, Dong Gongl
1 School of Computer Science and Engineering, University of New South Wales 2 Australian Institute for Machine Learning. The University of Adelaide (ruilin. tong, dong. Gunssw.uuww.....m edu..u [yuhang. javen. shi j addelaide edu.au
ABSTRACT
continual learning (CL) aims to mitigate catastrophic forgetting by maintaining subset of samples from previous tasks and replaying them. The rehearsal memory can naturally constructed as a coreset, designed to form a compact subset that enables training with performance comparable using the full dataset. The coreset selection task can be formulated as bilevel optimiza- tion that solves for the subset to minimize the outer objective of the learning task. Existing methods primarily rely on inefficient probabilistic sampling or local gradient-based scoring to approximate sample importance through an iter- ative process that can be susceptible to ambiguity or noise. Specifically, non- representative samples like ambiguous or noisy samples are difficult learn and incur high loss values even when training on the full dataset. However, exist- ing methods relying on local gradient tend to highlight these samples in an at- tempt to minimize the outer loss, leading to suboptimal coreset. To enhance coreset selection, especially in CL where high-quality samples are essential, we propose a coreset selection method that measures sample importance using re- ducible loss (ReL) that quantifies the impact of adding a sample to model per- formance. By leveraging ReL and process derived from bilevel optimization, we identify and retain samples that yield the highest performance gain. They are shown be informative and representative. Furthermore, ReL requires only forward computation, making it significantly more efficient than previous meth- ods. To better apply coreset selection in CI., extend method to address key challenges such as task interference, streaming data, and knowledge dis- tillation. Experiments on data summarization and continual learning demon- strate the effectiveness and efficiency of our approach. Our code is available at https::/gittub. wwt.:/tww.....ooooooeoeeeeaaaaae woomm/Rilling wwt.e//o//ooooog///nnoooonnnolee www.eceeeeeeeeeeee..e.ee--
INTRODUCTION
Continual learning (CL) aims to learn novel knowledge from non-stationary stream of data taining different tasks, while maintaining the learned knowledge (Ring) 1997 Rebuffi et al. 2017). Models tend to forget previously learned tasks when they trained new ones, a phenomenon known as catastrophic forgetting (McCloskey & Cohen 1989). A straightforward and effective way for countering forgetting is experience replay (ER) (Buzzega et al. 2020) Lopez- & Ranzato, 2017 Chaudhry et al. (219)). which maintains a small rehearsal memory to store a subset of pre- Vious data and replay it during training new tasks to mitigate forgetting. It is also convenient to incorporate ER with other types of CL methods, c.g. parameter isolation- based methods (Yan et al., 2022 Wang et al. 2022a) and regularization- based methods (Kirkpatrick et al.) (2017). Early meth- ods for ER often selected old data randomly (Vitter 1985) Lopez-Paz & Ranzato] 2017 Chaudhry et al. 2018b Buzzega et al., 2020) Riemer et 20188) However, this indiscriminate approach may not yield optimal results, selecting an informative subset of data for memory storage is crucial for the effectiveness of experience replay.
*D. Gong the corresponding author.

---

Published as a conference paper at ICLR 2025
A natural way enhance ER is to select coreset for memory, which is small subset of the dataset designed to allow a model trained on it to achieve performance comparable to one trained on the full dataset. The coreset selection can be formulated as bilevel optimization problem that solves for the subset over the full dataset to minimize the outer training objective. Greedy Coreset (Borsos et al.) 2020) iteratively adds samples into the coreset through an approximation of the implicit gradient as an indicator of the sample importance and a greedy solver of the bilevel problem. PBCS (Zhou et al.) 2022b) formulates the coreset with probabilistic masks on samples and solves the coreset selection problem from a global perspective, which relies on inefficient sampling process. BCSR (Hao et al., 2024 learns probabilities while preserving the nested structure of bilevel optimization.
Bilevel coreset selection methods usually treat all samples equally in the outer loss. However, non- representative samples, such as ambiguous or noisy ones that are common real-world data, are often not well learned, even when training on the full dataset. This suggests that these samples are not valuable to be represented by coreset. In the bilevel selection framework, these samples heavily impact the outer loss due to their high loss values. In methods that rely coreset expansion process with approximate implicit gradients (Borsos et al. 2020). these samples often assigned high scores for reducing the outer loss and tend to be selected into the coreset. Selecting ambiguous or noisy samples hinders the improvement of model performance after training on coreset. Additionally, previous bilevel coreset selection methods computationally expensive. The computation of inverse Hessian matrix is computationally expensive (Borsos et al.] [2220] Hao
incremental
al. 2024). PBCS (Zhou et al.] 2022b) relying on policy gradient requires a large number of sampling iterations, and reducing sampling numbers degrades performance significantly.
To enhance coreset selection, especially in CL where high-quality samples are essential, we propose a coreset selection method that measures sample importance using reducible loss (ReL) (Minder- mann et al. 2022), named CSReL. We study how the model's performance changes when a sample is added to the training dataset and quantify this based on ReL. The CSReL can effectively highlight an informative sample while excluding ambiguous or noisy ones according to the model's status. We assert that samples with high performance gains are representative and contribute relatively new knowledge compared to the already selected samples. Our method integrates the greedy incremental framework from bilevel optimization with ReL. In the incremental process, ReL is computed as the difference between the losses on the remaining samples computed by a holdout model trained on the full dataset, and a current model trained on the selected subset. Samples with the highest ReL are selected into the coreset for maximizing performance gain. ReL only requires forward computation Mindermann et al. (022), without the need for backward computation, making it significantly more efficient than other methods. To better adapt coreset selection to rehearsal- CL. we extend our method address the unique challenges of CL, including task interference, streaming data, and knowledge distillation. Our main contributions are:
We address the issue of selecting ambiguous or noisy samples in previous bilevel coreset selection methods by proposing coreset selection approach based on ReL reflecting the performance gain. We show that samples with high performance gain are both representative and informative, and our selection criterion effectively prevents the inclusion of ambiguous or noisy samples.
We propose an efficient coresct selection approach based on ReL, which is well-suited to be extended for addressing the unique challenges of CL: 1) Reducing task interference, 2) Selecting coresets from streaming data, and 3) Selecting coresets for knowledge distillation.
We demonstrate the effectiveness and efficiency of our method through extensive experiments on data summarization and CL tasks. Additionally, we show the effectiveness and compatibility of our method by enhancing the performance of existing CL methods.
2 RELATED WORK
Coreset selection. Coreset selection aims to select the most informative subset from full dataset. Previous coreset selection methods are designed for k-means (Feldman & Langberg 2011). 1). Gaus-
sian mixture model (Lucic et al.; 2018). logistic regression (Huggins et al., (016) and Bayesian
inference (Campbell & Broderick 2019). These methods are only suitable for traditional methods, while cannot be applied in deep neural networks. Greedy Coreset (Borsos et al. 2020) extended coreset selection to deep neural networks by formulating coreset selection problem as bilevel opti- mization problem. PBCS (Zhou et al. 2022b) selects globally with probabilistic masks. BCSR (Hao et al. 2024) considers nested nature based on PBCS (Zhou et al,, (0226). Previous bilevel selection methods may select ambiguous or noisy samples in an effort to reduce the outer loss. Additionally,
2

---

Published as a conference paper at ICLR 2025
these methods are computationally expensive. Our coreset selection approach effectively prevents the selection of ambiguous and noisy samples while being more efficient.
Continual learning. Continual learning aims to adapt learning agent sequence of tasks, and pre- vious tasks is not available once learnt, including regularization-bete based methods (Kirkpatrick et al.
2017
Zenke et al. 2017; Chaudhry al., 20188] Ritter et al. 2018 Li & Hoiem| 2017 Nguyen 2017; Ebrahimi et al.) [019]] Yan et al. [0022] Uha et al.
et al.
2023). parameter isolation based
methods (ann al. [2021] Wang al. 2022a Zhou et al. [202aa [iin et al. 2023 Ostapenko al.
2202
Wang et al. (024) and rehearsal based methods (Lopez-Paz & Ranzato 2017 Rebuffi et al.
2017
Chaudhry et al. 2018b] Riemer et al. 2018:; Aljundi et 2019c0 Chaudhry al., 2019
Borsos et al. [2200] Zhou et al.) [0226] Yoon et al.
2021; Aljundi et al.) 2019a Isele & Cosgun Buzzega al. 2021 Caccia et al. 2021). ). this work, we focus on rehearsal- methods
2018
and sample selection for maintaining high-quality memory.
Coreset for continual learning. Selecting samples for memory plays a crucial role in the perfor-
mance of rehearsal-based CL. Previous works, such as Aljundi et al. (2019c): [Sun et al. (2022b);
[ang et al. (202T); Wiewel & Yang (2021): Hurtado et al. (2023), are primarily based on heuristic insights. Wang al. (2022b) proposes compressing memory data to store more samples, while OCS
(Yonn et al. 2021 and GCR (Twari et al. 2022) select coresets through gradient matching. Greedy Coreset (Borsos et al. 2020), BCSR (Hao et al. 2024), and PBCS (Zhou et al., 2022b) apply core-
set selection directly to single-task datasets for memory. We investigate effective coreset selection method and the application of it for CL., while considering task interference, streaming scenarios, and knowledge distillation, in CL.
3
CORESET SELECTION WITH REDUCIBLE LOsS
3.1 BILEVEL OPTIMIZATION FOR CORESET SELECTION
Coreset selection aims to select subset from given dataset DD so that the model trained on coreset could achieve comparable performance as the model trained on D. The selection can be achieved via weighing the samples. Let loss function the i-th sample be ((z;, 0;;0) with O being model parameters. Coreset selection can be formulated a bilevel optimization problem (Borsos et al. 2020):
C(O'(w)) = Σ (D]
min
i=1
((I,; M/s 0" (w)),,
PII WER II [wloom
S.L. 0" (w) E arg L(0) -EE1 [P]
w. w;((x;, ye: 0)),
where wi is selection weight for sample (zi, w is corresponding selection vector, 0" (w) is solution for inner optimization problem, [D] denotes the number of samples in D. Coreset size is constrained by ||wllo S m.
Although the problem in Eq. order approaches (Borsos et al. 2020). The first-order optimality condition for the minimizer 0" (w) of ((0) implies that the gradient of ((0) with respect to (w.r.t) 0 evaluated at 0 (w) must vanish, ač(o"(ww
known be NP-hard, it can be heuristically solved using first-
0. This reflects that, at the optimal results, the gradient of the loss w.r.t. 0 vanishes. Relying on the implicit function theorem, we can obtain the gradient of 0* (w) w.r.t. w, i.c., the implicit gradient, enabling us to compute the gradient of outer loss function C(O'(w)) w.r.t the selection weights w. Focusing on the importance of each sample, the gradient w.r.t. each w; is derived as follows:
i.e, 30 (w)
V ((O = - "H Vol(s,, yi; 0 (w)),
(2)
where H denotes Hessian matrix of (()). The irrelevant items are dropped Eq. 20: and more details are in Appendix D
To solve for w with the coreset size constraint, Greedy Coreset (Borsos et al. 2020) integrates the first-order method into cone constrained generalized matching pursuit approach (Locatello et al.) 2017). This method iteratively selects samples into the coreset before reaching the size constraint. At each iteration step, given S. new sample with the largest negative implicit gradient among the remaining samples is selected. Considering that 0* (w) is trained on S in the greedy process, we use concise notation Os in place of 0* (w)). Through the connection between the bilevel optimization
3

---

Published as a conference paper at ICLR 2025
task and influence functions (Koh & Liang 2017 Eq. can be rewritten as
V ww, ((0s) = VeL(Os)" "H ((x, 0s)) = VeL(Os)
(3)
where V.,, wr Os represents the optimal parameter change when (Ira, yy) is added S and the model is retrained (Borsos et al., 2020). Eq. (3) shows that the gradient of wi, can be expressed as the inner product between the outer loss gradient VgL(Os) and the optimal parameter change V., O..
Despite the above progress, non-representative samples, such as ambiguous or noisy samples, may exhibit large loss values, contributing significandy L(0s). Including such samples in the coreset can reduce L(Os), leading to a negative inner product in Eq. (3) and significant parameters changes. According to the gradient-based basedd sample importance score as in Eq. (2) and (3). sample causing large changes in parameters and gradients can have a high likelihood of being selected into S, which may not be helpful for optimizing the real learning objective. The related effect has been observed in the coreset selection process of Greedy Coreset (Borsos et al. 2020), with detailed experiments and results provided in Appendix H
3.2 MAXIMIZE PERFORMANCE GAIN IN CORESET SELECTION WITH REL
Selecting ambiguous or noisy samples into coreset can lead to ineffectiveness in ensuring the per- formance of the model trained on it or even degrading it, which can be more crucial in CL with only restricted memory size for reply. To address these issues, we propose a coreset selection method by incorporating the reducible loss (Mindermann et al., (022) into the greedy incremental selection scheme derived from bilevel optimization. To alleviate the issue caused by gradient-based score, we try to directly capture the reduction of outer objective in Eq. ① as the selection objective. It aims to select samples that lead to the maximum reduction in the outer objective when added to the coreset, S, and can also been seen as rephrase the gradient-based score relying on Eq. (2) and (3) with the original objectives. Specifically, when adding (z,, 8/) into S, we consider the following gain:
(4) where (x,y denotes all samples in 'D and (r;, 3A) E D'S. Eq. (4) is not tractable to (Ii, y1) and selecting samples with maximum performance gain requires training models on every S (Ii,, Ui), which is impractical. Following Mindermann et al. (2022). we apply Bayes rule and conditional
G = log p [y|x;& U (I), v.) log (y)x;S),
independence make computation of G, tractable to (Ii V1)
p (v m;; D U S) p (y S) log p (y D PUS)P(L (y(x;
log (y|x: S (z,, 9:)) log
p ( Ø4, x; S)
p (1)
log (yl S S (zt, 9:)) log (y S) log p (ys 2 DUS) - log ())
(5)
The fist line in Eq. 5 is obtained by Bayes rule and conditional independence. The second line indicates G2 = log p 9 xy; D US log p (y) (T,
From the conclusion in Liu al. (2019). vanilla neural network is special case of Bayesian neural network with a uniform prior distribution and a Dirac-Delta posterior distribution. Predictive distribution in Eq. (5) could be approximated by vanilla neural network, and the performance gain of vanilla neural network is
G = log (y: [z;; Opus) - log (yi my; 0s) = E(v,, yo; 0s) - E(x;, yes Opus),
(6)
where 0 and 0, denote parameters of model trained on D and respectively. Based on the fact that loss in classification task is negative log-probability, G; can be approximated by loss difference, i.e., ReL. detailed derivation shown in Appendix A. Since C D and [S] << D. approximate p by OD and refer the model trained on D holdout model.
Using ReL as the selection criterion, we construct the coresct in a greedy, incremental manner. Initially, we train holdout model on D and start with an empty set S. In each step, we initialize and train the model on the current to obtain 0g. Samples with the maximum ReL are selected into S until the subset reaches the predefined size. We refer and detailed coreset selection procedure is shown in Alg.
coreset selection method as CSReL,
3.3 DISCUSSION ON REL AND CSREL
We demonstrate in Appendix B that ReL in Eq. 6 is an proportional approximation of the negative weight implicit gradient with mild assumptions. Both Rel. and implicit gradients have large
4

---

Published as a conference paper at ICLR 2025
values representative and informative samples. For ReL, sample (z,, is representative if f(z,, yr: 0p) is low and informative if f(z,, Ms: 0s) is high, indicating new information to S. From the implicit gradient perspective, adding such samples to S reduces loss multiple samples after training model on S U (Ii, U=), resulting in a high absolute implicit gradient value for this sample.
One advantage of CSReL method is its robustness against non-representative samples, such as ambiguous
Algorithm 1: The proposed CSReL
noisy ones. In ReL, these samples tend to have high loss values on both OD and 0g. leading to lower ReL values. Compared to implicit gradient, ReL excludes ambiguous or noisy samples from the indication of holdout model, even if these samples have large loss val- ues on gg. Since ReL. approximates performance gain, can conclude that samples with high per- formance gain are representative and contribute relatively new knowledge, rather than being am- ReL is
Input: Dataset D, coreset size m, steps bout Result: Coreset C
Train holdout model θD arg ming L(0): Initialize Sq = 0; select size of one step n = m/lout: for k = 1 to fout do Train model on current subset
Os = arg ming L(0);
Compute ReL
G; = ((2 Yr: 0s) { Vs: 0p),
biguous or noisy. Another advantage that ReL. only requires one forward pass without any backward computation, making it more ef- ficient. In comparison, the implicit gradient ne- cessitates computing the inverse Hessian matrix, which is computationally expensive.
(I,, 3c) E S)
Select top-n samples Ts by ReL and
update current coreset
Sk+1 Sk UT4
C
ReL was first introduced in Mindermann et al. 2022) for training data scheduling, In our work, we show that ReL well-suited for the coreset selection task and can effectively address the issue of selecting ambiguous or noisy samples in previous coreset selection methods. A detailed discussion comparing CSReL with other related works provided in Appendix
4 CORESET SELECTION FOR CONTINUAL LEARNING
4.1 APPLYING CORESET SELECTION TO CONTINUAL LEARNING
Continual learning seeks to adapt a CL model to a sequence of tasks with no shared classes between tasks, where data from previous tasks is unavailable during the training of the current task. In this work, we denote the dataset of the (-th task as Dr. We focus on the challenging bass-incrrmmntal setting, where the model has a single classification head, and task identity is not provided during inference. Rehearsal- based methods apply a memory buffer M to store part of the previous data and replay the stored data during training the CL model to prevent forgetting previous tasks.
We aim to summarize an informative subset as memory for rehearsal- based CL and natural ap- proach to select coresets from each task as memory, as proposed in Greedy Coreset (Borsos et al., (020) and PBCS (Zhou et al. (20226). We introduce our CSReL continual learning method (CSReL-CLL, which selects a coreset from each task using Alg. I and equally assign memory size to previous tasks. Detailed algorithm is shown in Appendix E We illustrate the overview of our CSReL-CL approach in Appendix Q to facilitate a clear understanding of our method.
4.2 CONSIDERING PREVIOUS TASKS WHILE UPDATING MEMORY
Only to summarize single task data D, for memory may result in interference between tasks, since there may be samples that represent D, but interfere other previous tasks. We aim select samples from D. to represent both the current task data and the previous data, so that the optimal subset can represent D, while minimizing interference with the previous data. This approach helps to avoid selecting these harmful samples. Since previous data is not available, we use memory data to represent the previous data. We modify performance gain for selecting i-th sample as
Prv = log p (yia|x1:t) St U (I4, v)) log (yi=lxx=e;;))
(7)
where (X1:1, V1:)) denotes all samples in D U M and (ar; y:) denotes the i-th sample in Dr. S, is current selected subset from D.. Following derivation in Section 3, the ReL is
GPT
(( ys: Os.) (xrs Mis OD.WM).
(8)
5

---

Published as a conference paper at ICLR 2025
The holdout model is trained on both current task data and memory. In practice, since MM < [Pel we resample memory data for training holdout model. We use the abbreviation CSReL-CL-Ptv to denote our CL method that considers previous tasks, detailed algorithms are shown in Appendix F
4.3 SELECTING CORESET FROM STREAMING DATA
Previous summarization methods require the availability of all current data, which may not be prac- tical for large datasets or privacy-related datasets. Reservoir sampling (Vitter) 1985) is an effective method for updating samples from a stream of data without needing the full dataset. Our key idea is to scale up selection probability in reservoir sample so that sample with higher ReL could have higher probability to be selected, while keeping the streaming nature of reservoir sampling.
Since Dr is not available and the CL model is trained on Dr U M, we approximate @D.UM by the CL model in Eq. (8) and modify ReL for reservoir sampling as
G ((s; Vr: 0s.) (zi, Us: Oent);
where Gent is parameters of the CL model, S is selected subset of current task and (I; ys) denotes i-th sample in current batch. ReL in Eq. (9) acts as a scaling factor on the update probability. The selected samples randomly replace existing samples in the memory. We refer our modified reservoir sampling method as CSReL-RS. The detailed algorithm is provided in Appendix
4.4 CORESET SELECTION FOR KNOWLEDGE DISTILLATION
Knowledge distillation (KD) Hinton et al. 2015) is an effective and widely used method for rehearsal-based CL. In these methods (Buzzega et al. 2020; Yan et al. 2022). the previous sam- ples' logits from the previous models are also save in memory, and a KD loss is applied to regularize the model to produce similar logits for the previous samples.
We claim that coreset selection should take into account both label prediction and KD. When select- ing samples in task t. we use (z,, ys. o;) to denote the i-th sample D,, where o, is the logit of this sample provided by the current CL model. Based on Section 3
we define the performance gain as
G.D
log p (O1:) Xx::; S U (Ira, 0:)) log (0:;
(10)
where (X1:t; 01:t) denote all sample-logit pairs in DUU. M. To make the probability of the logit tractable, we assume the probability of the predicted logit follows a Multivariate Gaussian Distribu- tion with an identity covariance matrix. Following Section 3
we can compute ReL. for KD as
CKD MSE (I;; Oz; 0s) - MSE (æ,, o OD,UM),
(11)
where MSE is mean square error (MSE). To apply objective in Eq. 11 to reservoir sampling, we assume the CL model is trained on D. U M. as the logits are provided by the CL model and the MSE of this model is O. Linearly combined ReL (ReL-emb) with factors as, and B, is
Crmb
G,G RS + B.G
(12)
EXPERIMENTS
5.1
DATA SUMMARIZATION
We evaluate our CSReL selection method on MNIST (Deng 2012). CIFAR-10 (Krizhevsky al. 2009) and more challenging CIFAR-100 (Krizhevsky et al. 2009). For MNIST and CIFAR-10, to make a fair comparison, we follow the settings of Borsos et (2020). For CIFAR-100, we use ResNet-18 (He et al. 2016) as backbone.
We compare our method with competitive baselines: Greedy Coreset (Borsos et al. 2020 PBCS (Zhou et al., 2022b), we also use uniform sampling as the worst-case. The evaluation metric is the test accuracy of the model trained from scratch on the selected data. Detailed model and training setting are shown in Appendix W
We plot test accuracy against selected subset size in Figure the results demonstrate that our method performs on par with Greedy Coreset on MNIST dataset and outperforms all baseline methods on CIFAR-10 and CIFAR-100 datasets. Notably, our method performs better with larger coreset sizes. Further comparisons and explanations are shown in Appendix V Analysis on difficulty of selected samples by our method and Greedy Coreset in Appendix [ demonstrate that our method could identify discriminative samples while avoiding selecting ambiguous or noisy samples.

---

Published as a conference paper at ICLR 2025
orealyownd
Germened
Test accurnyy
Uniten
Pars
Ghoaly omtad
Gready soad
der metted
Liniken
Uniiam
PICS
POES
(a) CNN on MNIST
(b) ResNet-18 on CIFAR-I0
(c) ResNet-18 on CIFAR-100
Figure 1: Performance comparisons in data summarization task show that method performs
on par with Greedy Coreset on the MNIST dataset, while outperforming all other baselines on the CIFAR-10 and CIFAR- 100 datasets.
Table 1:
Final average accuracy, with red and blue indicating the top and second-best values. Our memory construction method performs on par with Greedy Coreset on the MNIST dataset and con- sistently outperforms other baselines on more complex datasets.
Methods
Split MNIST
Split CIFAR-10
Split CIFAR-100
Perm MNIST
Uniform sampling
93.60+0.66
37.05#3.06
13.82+1.31
78.38+0.82
k-means of features
93.56=#1.24
35.78+0.56
14.3140.54
78.0840.55
k-center of embeddings
94.03+1.22
36.78+4.05
14.59+0.32
77.93+0.32
Hardest samples
87.26+2.50 2.0
27.80+1.20
12.19+0.05
77.0440.60
iCaRL's selection
94.32+0.20
35.3843.12
14.43+0.51
78.87+0.23
OCS
84.86+2.69
37.12+1.94
13.27+0.45
75.07=0.91
Greedy Coreset
95.73=0.19
37.68+2.63
15.04+0.48
79.23+0.37
GCR
93,22+1 .04
37.59+
13.54 1.06
78.73+0. 13
PBCS
94.22+0.61
38.37+1.01
16.20+0.27
76.30+0.81
BCSR
93.81
38.1443.64
15.11#1.24
78.30+0.81
CSReL-CC
95.6840.35
38.9742.61
17.48+0.21
79.5940.38
CSReL-CL-PrV
93.55+0.13
39.82+0.83
18.4740.17
80.0220.12
5.2 CONTINUAL LEARNING
We conduct experiments on different CL settings to evaluate our methods, and use the final average accuracy as evaluation metric, which reflects average accuracy across all tasks after training model on the last task. All of our experimental results are obtained from multiple runs with different random seeds. We selecte hyperparameters with a focus on both performance and robustness.
5.2.1 CORESET SELECTION FOR CONTINUAL LEARNING
We evaluate CSReL-CL and CSReL-CL-Pry on Split MNIST (Zenke et al. 2017). Perm MNIST
[Gooffellow et al. 2013). Split CIFAR-10 and CIFAR-100. To make fair comparison, follow
the setting of Borsos et al. (2020) and Zhou et al. (002bb). We set 100 memory size for Split MNIST for Split CIFAR-1O 10 and Split CIFAR-100 For Split-CIFAR-
and Perm MNIST and 200 memory
100 dataset, we split totally 100 classes into 10 disjoint tasks, and use ResNet-18 18 (He et al. 2016 as backbone. Detailed experiment settings and hyperparameters are shown Appendix X.I
Baselines include: uniform sampling, k- k-center clustering in last layer embedding (Sener & Savarese 2017) and feature space (Nguyen al.
2017), iCaRL's selection (Rebuiti al.
2017). hardest-to-
classify samples (Aljundi et al.) 2019b).
Greedy Coreset (Borsos et al., 2020). GCR (Tiwari et al.)
2022
OCS
(Yonn al., 2021). PBCS (hhou et al. 2022b) and BCSR Hao et al.
2024).
The average accuracies in Table T show that our method performs comparably to Greedy Coreset on the Split MNIST dataset and outperforms all other baselines on the remaining datasets, highlighting the superiority of our approach. Additionally, the experimental results indicate that considering previous tasks can further enhance CL performance. We further demonstrate the effectiveness of method compared most recent baselines by conducting additional experiments under the same settings as BCSR, as detailed in Appendix M The scalability and effectiveness of our method with complex backbone models further demonstrated in Appendix

---

Published as a conference paper at ICLR 2025
Table Final average accuracy with red and blue indicating the top and second-best values. Summa- rizing data with our CSReL-RS consistently improves the performance of existing continual learning methods, particularly in knowledge distillation scenarios.
Split CIFAR-I00
Split Tiny ImageNet
Methods
200
500
200
500
A-GEM
9,40+0.05
9,42++0.08
8.07=0.08
8.06+0.04
ER
14.18+0.45
21.08=0. 16
8.49=0.16
9.99+0.29
FDR
15.32+0.73
22.83+0.73
8.70=0.19
10.54:0.21
CSReL-ER
15.35+0.73
22.65+0.81
8.66+0.06
10.44+0.17
DER
21.58+1.72
35.20=0.84
11.87+0.78
17.75+1.14
DER+
26.2742.32
36.00=1.92
10.9620.17
19.38+
LODE-DER++
27.96+0.91
39. 1420.74
14.46+0.90
21.15+0.68
CSR+L-DER++
27.79+0.60
39.80=1.45
16.78+0.78
21.22::0.22
CSReL-LODE-DER+
28.51+0033
41.96+0.78
17.01#0.43
22.83+0.23
5.2.2 CORESET SELECTION FOR EXISTING CONTINUAL LEARNING METHODS
To further demonstrate the effectiveness and compatibility of our method with other CL approaches, we replace reservoir sampling in ER (Riemer et al., 2018), DER+ (Buzzega al. 2020), and LODE-DER+ (Liang & Li, 2024) with our CSReL-RS, resulting in CSReL-ER, CSReL-DER++ and CSReL-LODE- DER++ Since DER++ applies knowledge distillation, apply ReL-emb for sample selection for all methods involving DER++. Following settings of the Mammoth framework (Buzzega al. 2020), evaluate methods on Split CIFAR-10K and Split Tiny ImageNet (Wu et al. 2017) with memory sizes of 200 and 500. Both datasets are equally split into 10 tasks with ResNet-18 serving backbone model. Detailed settings are shown in Appendix X.2
We compare our methods with other commonly compared rehearsal-basee methods including ER, A-GEM (Chaudhry al. 20188), FDR (Benjamin al. 2018), DER (Buzzega et al. (2020), DER++ and DEDE..ER++. The results are presented in Table 2 with baseline method results on Split Tiny ImageNet taken from Bozzega et al. (2020) and Liang & (024)).
The results in Table 2 show that summarizing the core- set using our CS effectively enhances existing continual learning methods, particularly when apply- ing knowledge distillation. Combining CSReL-RS with LODE-DER++ consistently outperforms other methods across all datasets. These findings highlight both the ef- fectiveness and compatibility of our approach. Addition- ally, we evaluate the scalability our method on a 500- class subset of ImageNet- IK in Appendix N
on
66.
5.3 ABLATION STUDY
Grodd Comsetccaacy
Jan of Grudly Cortec Jun of scded
Analysis on representing full dataset. Both Greedy Coreset (Borsos et al., 2020) and our method minimize loss on the full dataset of model trained on coreset. A lower loss indicates that coreset better represents the full dataset. To evaluate this, we train models on the selected subsets and compute the average loss on the full dataset as a metric to assess how well the selected data represents the full dataset. We conduct experiments on CIFAR-10 by selecting 2,000 samples from pool of 10,000 with our method and Greedy Coreset. Models are trained on subsets with different sizes during the selection process, and we evaluate the trained models based on both the average loss on the full dataset and test set accuracy.
Average loss dataset and test accuracy of models trained subsets show that coreset lected by our method could better repre- sent full dataset and achieve higher curacy.
Figure 2:
the full
We plot test accuracy and average loss on full dataset against subset size in Figure The results indicate that models trained on subsets selected by our method achieve greater reduction in average full dataset loss, especially when the subset size is small. This suggests that our method selects samples with higher performance gains and thus better represent the full dataset. The corresponding test accuracy is consistent with the loss curve, demonstrating the effectiveness of our method.
8

---

Published as a conference paper at ICLR 2025
Text accurccy
Tesl
On nalled
Opricied
Grady Corsor
Guady Ceneit
Noise mioo
Noise raño
Nobic ratio
(a) CNN on MNIST
(b) ResNet-18 18 on CIFAR-10
(c) ResNett-18 18 on CIFAR-100
Figure 4: Test accuracy under different noise ratios shows that the performance of our method drops
only slightly as the noise ratio increases and outperforms other baselines in noisy condition.
Data summarization under label noise. To demonstrate the robustness of our method under noisy condi- tion, we test data summarization per- formance under label noise case on MNIST, CIFAR-10 and CIFAR-100 dataset, with the same experiment setting and evaluation method as periments in Section 5.I Specif- ically, randomly corrupt sam- ples at different ratios and then use this corrupted dataset as the selection pool to select the coreset of 200 sam- ples. We compare method with
Ted BONEEN
Sag ID
Step
(a) CIFAR-10
(b) CIFAR-100
Figure 3: Performance and time cost w.r.t step size. Select- ing more samples within step will degrade performance while reducing time cost, indicating that a trade-off should be made between performance and efficiency.
Greedy Coreset (Borsos et al. (020) and PBCS (Zhou et al. 20226). No clean holdout set is pro- vided, namely, holdout model our method trained on noisy dataset, outer loss Greedy Coreset and PBCS is computed on noisy dataset.
As shown in Figure the performance of our method drops slightly as the noise ratio increases across all datasets, while the performance of other methods drops significantly. We also count number of selected noisy samples in Appendix K showing that our method selects significantly fewer noisy samples. These results demonstrate the robustness of our method to data noise. Given that low noise ratios are common practical scenarios, such as web data, our approach effectively avoids selecting noisy data for the coreset.
Selection steps data summarization. In our multi-step selection algorithm, selecting more samples in step and using fewer steps can reduce computation costs. However, selecting more samples one step may result in redundant samples, as samples with similar ReL may contain similar knowledge. Therefore, trade-off should be made between efficiency and per- formance. We select coresets of 200 samples with different step size on CIFAR-10 and CIFAR- 100 and train models these coresets. Test accuracy of models and time cost corresponding to step size are shown in Figure
We observe that performance decreases as the step size increases, while the time cost decreases. These results verify our claim of trade-off between time cost and performance. For CIFAR- 10 dataset, performance drops slightly when the step size is smaller than 10. However, in the more complex CIFAR-100 dataset, performance drops significantly as the step size increases.
Corgeting
Effectiveness of considering knowledge distillation. To verify the effectiveness of considering knowledge distilla- tion proposed in Section 4.4 we set different MSE factor 3, while fixing all other hyper parameters, a₂ is set to 1.0 in all the experiments. We conduct experiments
Avtrage SAATTY
begeizg
Figure 5: Avg. accuracy and forgetting under different B.. Increasing B, de- creases forgetting and the best accuracy is achieved when Bg is set to 0.2.
Split CIFAR- 100 with CSReL-RS, and use ReL-cmb
Eq. (12 for selection. F shows the average accuracy and forgetting w.r.t. different B, values.

---

Published as a conference paper at ICLR 2025
As 3g increases, forgetting continually decreases, indicating that more emphasis is placed on select- ing samples that encourage the current model to mimic the output of the previous model, thereby increasing regularization strength. The best performance is achieved when B, 0.2, suggesting that excessive regularization can hinder the learning of new tasks, highlighting the need for balance. Therefore, using ReI.-cmb proves effective for coreset selection.
Holdout model training epochs. In our work, holdout model serves as an indicator for which sample worthy of learning and which sample is not learned yet, we test the influence of holdout model training epochs with respect to the quality of selected samples on CIFAR-I0. Specifically, we train holdout models with different epochs, then select coresets with our method and train models on selected coresets. We evaluate models with test accuracy. Coreset size is set to 200, backbone and other hyperparameters are the same as Section 5.1
Performance with respect to different holdout training epochs in the data summarization task: Performance increases as training epochs increase and then remains stable.
Table 3:
From result in Table 3 as holdout model training epochs increases, quality of selected coreset increases and re- mains stable after holdout training epochs reaches 80. In- dicating that quality of holdout model affects quality of selected data, and holdout model should be well trained for coreset selection. We also evaluate the impact of hold- out model training epochs on the continual learning task using the Split CIFAR- 100 dataset, as detailed in Ap- pendix [ Our results show that continual learning per- formance remains stable across a wide range of holdout model training epochs.
Train epochs
Test accuracy
20
32.98+0.75
40
35.13+0.75
60
35.12+0
80
37.57+0.78
100
38.74+1 .08
Time cost of coreset selection. To demonstrate the efficiency of our CSReL coreset selection method, we provide the time costs for coreset selection on different datasets in Scction[5.]] shown in Table The time cost of our method scales moderately across different datasets and backbones, and training the holdout model remains manageable with larger datasets and backbones. This demonstrates that our method is well-suited for larger datasets and more complex backbones.
120
37.89+0.27
Table 4: Time cost of selecting core- set with CSRel. The time cost scales mildly on different datasets.
Dataset
Holdout train Selection
MNIST
159.84s 349.74s
102.70s
CFIAR-10
737.68s
Both our method and Greedy Coreset (Borsos et al. (0200) use matching pursuit for coreset selection. However, Greedy Coreset computes the Neural Tangent Kernel (NTK) (Jacot et al. 2018). which takes over
CIFAR-100
-10
350.87s
735.85s
4000 seconds. In comparison, our CSReL coreset selection method is significantly faster, demon- strating its efficiency. All our experiments are conducted on NVIDIA RTX3090. Details of back- bone parameters and time cost shown in Appendix P
Further ablation studies. In greedy incremental selection, selected samples are not removed, mean- ing that samples chosen early may have a lower performance gain as more samples are added the coreset. As a result, the selection order can influence the performance of the coreset. We demon- strate the robustness of our method with respect to selection order in Appendix R Additionally, difficult and noisy samples cannot be distinguished solely using the ReL. metric. However, in greedy incremental selection, we show that our method can effectively differentiate between these sample types in Appendix [S] Furthermore, we present a feature map of the selected samples in Appendix U illustrating that our method consistently selects more representative samples.
CONCLUSION
In this work, we address the problem of selecting ambiguous or noisy samples in previous bilevel coreset selection methods by using reducible loss as an approximation for performance gain. This approach enables the identification of representative and informative samples while excluding noisy or ambiguous ones, leading improved performance. Additionally, we propose an efficient core- set selection method designed to address the unique challenges of continual learning, such as task interference, streaming scenarios, and knowledge distillation. Extensive experiments validate the effectiveness of our approach in both data summarization and continual learning tasks. In future work, we plan to explore coreset selection for fine-tuning large pretrained models, allowing them to acquire new knowledge while maintaining generalizability.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This paper was partially supported by Australian Research Council (ARC) Discovery Early Career Researcher Award (DECRA) project DE230101591 to D. Gong.
REFERENCES
Rahaf Aljundi, Eugene Belilovsky. Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered re- trieval. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Gameett (eds.), Advances in Neural Information Processing Systems 32, pp. 11849- 11860. Curran Associates, Inc., 2019a. URL http::/paperr: nips cc/paper/ 9357- wwwoonnnnnoooooooeeeee pdf
in cont
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11254-11266
2019b.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019c.
Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Ha, and Jonghyun Choi. Rainbow mem-
ory: Continual learning with memory of diverse samples. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 8218-8227, 2021.
Ari S Benjamin, David Rolnick, and Konrad Kording. Measuring and regularizing networks in function space. arXiv preprint arXiv:1805.08200 2018.
Zalán Borsos, Mojmir Mutny, and Andreas Krause. Coresets via bilevel optimization for continual learning and streaming. Advances in neural information processing systems, 18899-9944480 2020.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In Advances in neural infor- mation processing systems, pp. 15930- 2020.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, and Simone Calderara. Rethinking experience replay: bag of tricks for continual learning. In 2020 25th International Conference on Pattern Recognition (ICPR), pp. 2180-2187 IEEE, 2021.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning, arXiv preprint 1000022004.00025, 2021.
Trevor Campbell and Tamara Broderick. Automated scalable bayesian inference via hilbert coresets. Journal of Machine Learning Research, 20(15):1-38, 2019.
Arslan Chaudhry, Puncet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European conference on computer vision (ECCV), pp. 532-547, 2018a.
Arslan Chaudhry, Mare' Aurclio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint /rXiv: 2018b.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny. Thalaiyasingam Ajanthan, P Dokania, P Torr, and M Ranzato. Continual learning with tiny episodic memories. In Workshop on Multi- Task and Lifelong Reinforcement Learning, 2019.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web). IEEE signal processing magazine, (9( 141-144 2012.
11

---

Published as a conference paper at ICLR 2025
Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, and Mareus Rohrbach. Uncertainty-guiee continual learning with bayesian neural networks. arXiv preprint 190.::990002255, 2019.
Talfan
Evans, Shreya Pathak, Hamza Merzic, Jonathan Schwarz, Ryutaro Tanno, and Olivier Henaff. Bad students make great teachers: Active learning accelerates large-scale visual un- derstanding, arXiv preprinl arXiv:2312.05322 2023.
Dan Feldman and Michael Langberg. A unified framework for approximating and clustering data. In Proceedings of the forty-third annual ACM symposium on Theory of computing, pp. 569-578, 2011.
Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient-based hyperparameter optimization. In International conference on machine learning, pp. 1165-1173. PMLR, 2017.
Ian Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empiri- cal investigation of catastrophic forgetting in gradient-base neural networks. arXiv preprint arXiv:1312.6211, 2013.
Jie Hao, Kaiyi Ji. and Mingrui Liu. Bilevel coreset selection in continual learning: A new formula- tion and algorithm. Advances in Neural Information Processing Systems, 36, 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint 1.Xiv:1503.0253: 2015.
Jonathan Huggins, Trevor Campbell, and Tamara Broderick. Coresets for scalable bayesian logistic regression. Advances in neural information processing systems, 29, 2016.
Julio Hurtado, Alain Raymond-Sáne -Séz,, Vladimir Araujo, Vincenzo Lomonaco, Alvaro Soto, and Da- vide Bacciu. Memory population in continual learning via outlier elimination. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3481-3490, 2023.
David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 32, 2018.
Rishabh Tyer, Ninad Khargoankar, Jeff Bilmes, and Himanshu Asanani. Submodular combinatorial information measures with applications in machine learning. In Algorithmic Learning Theory, pp. 722-754. PMLR, 2021.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. Advances in neural information processing systems, 31, 2018.
Saurav Jha, Dong Gong, He Zhao, and Lina Yao. Npel: Neural processes for uncertainty-awarc continual learning, Advances in Neural Information Processing Systems, 36:34329-34353, 2023.
Hyundong Jin, Gyeong-hyeon Kim, Chanho Ahn, and Eunwoo Kim. Growing a brain with sparsity- inducing generation for continual learning. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pp. 18961-18970, 2023.
Chris Dongjoo Kim, Jinseo Jeong, Sangwoo Moon, and Gunhee Kim. Continual learning on noisy data streams via self- selfi-urified replay. In Proceedings of the IEEE/CVE international conference on computer vision, pp. 537-547, 2021.
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):551-3522 2017.
12

---

Published as a conference paper at ICLR 2025
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885-1894, PMLR, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions pattern analysis and machine intelligence, 40 40(12):2935-2947, 2017.
Yan-Shuo Liang and Wu-Jun Li. Loss decoupling for task-agnostic continual learning. Advances in Neural Information Processing Systems, 36, 2024.
Yuhang Liu, Wenyong Dong, Lei Zhang, Dong Gong, and Qinfeng Shi. Variational bayesian dropout with a hierarchical prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7124-7133, 2019.
Francesco Locatello, Michael Tschannen, Gunnar Rätsch, and Martin Jaggi. Greedy algorithms for cone constrained optimization with convergence guarantees. Advances in Neural Information Processing Systems, 30, 2017.
David Lopez-Paz and Marc Aurelio Ranzato. Gradient episodic memory for continual learning- Advances in neural information processing systems, 30:6467-6476, 2017.
Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit differentiation. In International conference artificial intelligence and statistics, pp- 1540-1552. PMLR, 2020.
Mario Lucic, Matthew Faulkner, Andreas Krause, and Dan Feldman. Training gaussian mixture models at scale via coresets. Journal of Machine Learning Research, 18(160):1-55, 2018.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Sören Mindermann, Jan M Brauner, Muhammed Razzak, Mrinank Sharma, Andreas Kirsch, Win- nie Xu, Benedikt Höltgen, Aidan N Gomez, Adrien Morisot, Sebastian Farquhar, et al. Prioritized training on points that are learnable, worth learning, and not yet learnt. In International Confer- ence on Machine Learning, pp. 15630-15649, PMLR, 2022.
Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. Variational continual learning. arXiv preprint arXiv: 2017.
Oleksiy Ostapenko, Pau Rodriguez, Massimo Caccia, and Laurent Charlin. Continual learning via local module composition. Advances in Neural Information Processing Systems, 34:30298- 30312, 2021.
Sylvestre-Alviss Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interfer- ence. arXiv preprint arXiv:1810.11910, 2018.
Mark B Ring. Child: first step towards continual learning. Machine Learning, 28:77-104, 1997.
Hippolyt Ritter, Aleksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. Advances in Neural Information Processing Systems, 31, 2018.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv: /00::7708000000 00489, 2017.
13

---

Published as a conference paper at ICLR 2025
Shivakanth Sujit, Somjit Nath, Pedro Braga, and Samira Ebrahimi Kahou. Prioritizing samples in reinforcement learning with reducible loss. Advances Neural Information Processing Systems, 36:23237-23258 2023.
Qing Sun, Fan Lyu, Fanhua Shang, Wei Feng, and Liang Wan. Exploring example influence in continual learning. Advances in Neural Information Processing Systems, 35:27075-27086, 2022a.
Shengyang Sun, Daniele Calandriello, Huiyi Hu, Ang Li, and Michalis Titsias. Information- theoretic online memory selection for continual learning. arXiv preprint arXiv:2204.04763, 2022b.
Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, and Pradeep Shenoy. Ger: Gradient coreset based replay buffer selection for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 908, 2022.
Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions Mathematical Software
(TOMS), 11(1):37-57, 1985.
Zhijing Wan, Zhixiang Wang. Yuran Wang. Zheng Wang, Hongyuan Zhu, and Shin ichi Satoh. Contributing dimension structure of deep feature for coreset selection. arXiv preprint 2rXiv:2401.16193, 2024.
Fu- Yun Wang. Da-Wet Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and com- pression for class-ineremental learning. In European conference on computer vision, pp. 398-414. Springer, 2022a.
Huiyi Wang. Haodong Lu, Lina Yao, and Dong Gong. Self-expansion of pre-trained models with mixture of adapters for continual learning. arXiv preprint arXiv:2403. 0224403..8800 18866, 2024.
Liyuan Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing Hong, Shifeng Zhang, Zhenguo Li, Yi Zhong, and Jun Zhu. Memory replay with data compression for continual learning, arXiv preprint 2rXiv:2202.06592, 2022b.
Felix Wiewel and Bin Yang. Entropy-based sample selection for online continual learning. In 2020
28th European Signal Processing Conference (EUSIPCO), pp. 1477-1481. IEEE, 2021.
Jiay Wu, Qixiang Zhang, and Guoxi Xu. Tiny imagenet challenge. Technical report, 2017.
Kan Wu, Jinnian Zhang, Houwen Peng. Mengchen Liu, Bin Xiao, Jianlong Fu, and Lu Yuan. Tinyvit: Fast pretraining distillation for small vision transformers. In European conference on computer vision (ECCV), 2022.
Xilie Xu, Jingfeng Zhang, Feng Liu, Masashi Sugiyama, and Mohan Kankanhalli. Efficient adver- sarial contrastive learning via robustness aware coreset selection. Thirty- Conference on Neural Information Processing Systems, 2023. URL Bttthttt////peeeveeeee net forum?
id= (d=fAAA8
Qingsen Yan, Dong Gong, Yuhang Liu, Anton van den Hengel, and Javen Qinfeng Shi. Learning bayesian sparse networks with full experience replay for continual learning. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 109-118, 2022.
Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3014-3023, 2021.
Jachong Yoon, Divyam Madaan, Eunho Yang, and Sung Ju Hwang. Online coreset selection for rehearsal-baad based continual learning. arXiv preprint 108.:0500000088. 2021.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995, PMLR, 2017.
Yihua Zhang. Prashant Khanduri. loannis Tsaknakis, Yuguang Yao, Mingyi Hong, and Sijia Liu. An introduction to bilevel optimization: Foundations and applications in signal processing and machine learning. IEEE Signal Processing Magazine, 41(1):38-50 2024.
14

---

Published as a conference paper at ICLR 2025
Da-Wei Wei Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. A model or 603 exemplars: Towards memory-efficient class incremental learning. arXiv preprint arXiv:2205.132218 2022a.
Xiao Zhou, Renjie Pi, Weizhong Zhang, Yong Lin, Zonghao Chen, and Tong Zhang. Probabilistic bilevel coreset selection. In International Conference Machine Learning, pp. 27287-27302. PMLR, 2022b.
15

---

Published as a conference paper at ICLR 2025
Appendix
Table of Contents
Approximating Predictive Distribution with Reducible Loss
B Relation between ReL and Implicit Gradient
17
Discussion with Other Reducible Loss Related Works
18
D Derivation of Outer Loss Gradient with Respect to Sample Weight
18
E Algorithms for CSReL Continual Learning
19
F Algorithms for CSReL-CL Considering Previous Tasks
20
G Algorithms for CSReL Reservoir Sampling
20
Loss of High Implicit Gradient Candidates
22
Difticulty of Selected Samples
23
Analysis on Representing Full Dataset
26
Data Summarization under Data Noise
26
L Forgetting in Continual Learning Experiments
27
Continual Learning on BCSR Setting
27
N Continual Learning Experiment on ImageNet 500 Class
27
0 Effeet of Holdout Model Training Epoch in Continual Learning
28
P Time Cost and Model Size in Data Summarization Experiment
28
Q Overview of Coreset Selection for Continual Learning
29
Impact Selection Order
29
S Distinguishing Difficult Samples from Noisy Samples
30
T Coreset Selection for Continual Learning with Large Backbone Models
30
U Feature Map of Selected Samples
31
Additional Data Summarization Results
32
W Data Summarization Experiment Details
33
X Continual Learning Experiment Details
34
X.I Data Summarization for Continual Leaming
34
16

---

Published as a conference paper at ICLR 2025
X.2 Continual Learning with Modified Reservoir Sampling and Knowledge Distillation
34
Visualization of Selected Data
35
Computation Resources and Experiment Time Cost
35
AA URL of Cited Assets
36
Broader impact
36
A
APPROXIMATING PREDICTIVE DISTRIBUTION WITH REDUCIBLE LOss
Predictive distribution in Eq.
b requires parameter distribution of model trained on D U U and S
p (w: m DUS) = / p (ye 2 0) (O
p (v / (yilz; (0]S) de.
From the conclusion in (Liu al. 2019). vanilla neural network special case of a Bayesian neural network with a uniform prior distribution and a Dirac-Delta posterior distribution. Therefore, performance gain of vanilla neural network is
G log p (y: I Puss) log p (y/ z 0s).
(13)
where pus and Os denote parameters model trained DUS and respectively. For classifica- tion task and cross coss-entropy loss, we use loss to replace the negative log- bog-probability. the performance gain is
Gi = E( ys: 0s) - E(xi, Us: Opss).
(14)
B RELATION BETWEEN REL AND IMPLICIT GRADIENT
We show that ReL in Eq. (14) is a proportional approximation to negative implicit gradient (Borsos ett al. 2020) under greedy selection framework with binary sample weight. Given model trained on S, parameter Opus is approximated by updating Os with one Newton step, namely
Op ~ Os Hfu V VL(s),
(15)
where H full denotes Hessian matrix of loss E ED (z Mei 0s) with respect to 0s. Since D and samples are i.i.d., we could assume that the Hessian matrix H provides statistical approximation of the full Hessian H full in the neighbor space of Os. the following relationship holds
1
D H
H full
H. [S]
Approximating G, in Eq. (4) with first-order Taylor expansion and substituting parameter differ- ence in Eq. (15) results in
S [D]
G Vi( 0s) (0s 0p)
Vi(r,, ye 0 VL(Os)
(16)
Since CSR is computed over all candidate samples and the ratio |S|/|D| does not affect the ranking of candidate samples, Eq. (66) indicates Rel. is proportional approximation to negative implicit
gradient in Greedy Coreset (Borsos et al. 2020).
For samples which are representative and not well represented by S, both implicit gradient and ReL. have high scores. Suppose (I,, 34) is such kind sample, for ReL, representativeness indicates (xx,, yi 0p) is low, and not represented by S means {(z,, y:; 0s) is high, thus ReL is high. From the aspect of implicit gradient, adding these samples into S will reduce loss on multiple samples after training model on S J (I; 31) since this sample contains knowledge similar to other samples that not yet present S. According to the definition of implicit gradient dC((s)) /dw,, a reduction in loss across multiple samples indicates high absolute value of the implicit gradient for that sample.
17

---

Published as a conference paper at ICLR 2025
C DISCUSSION WITH OTHER REDUCIBLE LOss RELATED WORKS
Reducible loss was firstly introduced in Mindermann et al. (2022) for selecting samples that are both learnable and worth learning from each batch of data. Sujit et al. (023) further applied ReL. for selecting samples to replay in reinforcement learning, where unselected samples may be chosen in later epochs or episodes. However, in the coreset selection task, unselected samples discarded and not reused after the selection process. As result, the methods proposed in previous works cannot be directly applied to coreset selection task.
Online training data scheduling can also select subset from each incoming batch for training (Evans al., 2023). with the goal of maximizing the performance of the model trained on the selected data. The target model is trained with only single step on this subset, which may prevent it from fully learning the knowledge contained in these samples. When the training dataset is large enough and the pruning rate is not too high, the model can gradually acquire sufficient knowledge, allowing this method to perform well. However, the coreset selection task, we aim select a small subset from the entire dataset, meaning the pruning rate is much higher, and the full dataset may not be as large. This can result in lower performance in the coresct selection task.
Our work adopts the same matching pursuit selection framework as Greedy Coreset (Borsos et al, 2020). To address the issue of unselected data being excluded after selection, we train the model on the selected subset until convergence, ensuring that the model fully captures the knowledge tained in the selected data. The incremental selection method ensures that each newly selected sample adds new knowledge the existing subset.
Our work proves that Rel. could act as an indicator for sample selecting in model convergence condition. Eq. (16) shows that ReL. is a proportional approximation of the negative implicit gra- dient, while Eq. (3) illustrates that the implicit gradient computes how changes in sample weights affect the outer loss via the chain rule. The optimal parameter change d0g/ /dw; estimates how the model parameters will change when sample weights are modified, assuming the model is trained to convergence. Thus, the case of a converged model, ReL can serve reliable indicator.
Building on the coreset selection method from Greedy Coreset (Borsos et al. 2020), we address the issue of selecting ambiguous or noisy samples using the implicit gradient and directly set the performance gain in Eq. 4 as our selection objective. Rel. serves as an approximation of this performance gain. Both Rel. and the implicit gradient can identify representative and informative samples. However, compared to the implicit gradient, ReL is more effective at avoiding the selection of ambiguous and noisy samples, resulting in improved performance over Greedy Coreset (Borsos) ett al. 2020). Therefore, method could effectively select coreset of representative and informative samples. related research area involves data compression (Wang et al.) (022b), which seeks to store more information with limited storage capacity by compressing data.
D DERIVATION OF OUTER Loss GRADIENT WITH RESPECT TO SAMPLE WEIGHT
In this section, we derive Eq. (2) from the bilevel optimization formulation presented in Eq (T. We follow logical sequence of steps to arrive at the gradient expression in the desired form. The gradient has been investigated in previous works (Lorraine et al., 2020 Franceschi et al., 2017 Zhang et al. 2024). For completeness, we provide simplified version here.
Step 1: First-Order Optimality Condition for (ww) We begin by noting that 0" (w() is the minimizer of the weighted loss function L(O),, which is defined as:
[D]
= ((
w;l(x; v:;
1=1
where O are the parameters of the model. At the optimal solution 0" (w), ). the gradient of the weighted loss with respect to 0 must vanish:
Oc(0)
0. 80 0 (=)
18

---

0. Published as a conference paper at ICLR 2025
This condition represents the first-order optimality condition for the minimizer of (()), implying that at 0 (w), the gradient of the weighted loss function with respect to is zero.
Step 2: Implicit Function Theorem Application Next, apply the implicit function theorem to relate the optimal model parameters 0* (w) to the weight vector w. Since (w) is implicitly defined by the equation
ac(o)
0,
d0
0=--(w)
can compute the derivative of 0* (w) with respect wy Differentiating both sides of the equation with respect to W; gives:
a
OC(O)
0.
Ow;
00
8- (w))
By the chain rule, this becomes:
8 (
00 (w)
8L(()
a02
(==-(w)
ow,
d0dw, 0==-(w)
Here, the Hessian matrix H of ((0) is the second derivative of 2(0) with respect to 0, evaluated at 0" (w). We can thus rewrite this as:
a0 (0"(w) Ow;
OL(0)
Ow. (0=0-(w)
Step 3: Gradient of (w)) with Respect to w, Now we seek the gradient of the outer loss function C(0"(w) (w)) with respect to the weight wy The outer loss function
[D]
[(O"(w))
Σ ((x,, Mis 0" (w)).
By the chain rule, the gradient of ((0"(w)) with respect wi is:
V w. [()((w)) = VoL(0((w)) 00 (w)
Ow;
Substitute the expression for (0"(w) (w) from the implicit function theorem:
the
V (()) (O (W)) 'Volle ye 0 (w)).
This gives the desired expression for the gradient of the loss function with respect to wx as provided in Eq. 2
E ALGORITHMS FOR CSREL CONTINUAL LEARNING
In this work, we apply our coreset selection method to rehearsal-based CL. Training objective at task t
1 Σ (Z, Uii 0 Oent) +
(en UD; 0cnt)
E(amm Umi Oent).
(17)
B
Bm
where (24, yt) and (I'- Dm) are samples from the current task dataset and memory M respectively. The memory M stores coresets from all previous tasks, defined as M Cf_ is hyper- parameter for balancing regularization force from memory, and B,, are batches from current task and memory respectively.
After training task f, we select coreset from D, with Alg I and store the holdout model loss each selected sample. To shrink memory of previous data, re-select memory data shrink memory of previous tasks, using memory data of previous tasks as selection pool. Since the holdout model loss for previous memory samples are already stored in the memory, there is need to retrain the holdout model during re-selection. We refer our CL method as CSReL Continual Learning (CSReL- CL.) and detailed algorithm is shown in Alg
19

---

Published as a conference paper at ICLR 2025
Algorithm 2: CSRel. Continual Learning
Input: Dataset sequence D1.7, memory size K Initialize memory MM = 0;
for range( 1) do
Train continual model with replay Oon arg ming Lent (D; // Update memory
Compute size for each task K = K/i.j E []: for j in range(i) do
L Reselect samples for previous tasks from C by Alg
coreset size is 1ng
Select samples for current task from D; by Alg Cel k; Form new memory M = Cg:
F ALGORITTMS FOR CSREL-CL CONSIDERING PREVIOUS TaskS
Algorithm 3: Coreset selection considering previous tasks
Input: Dataset D, select size m, selection steps tout, memory M
Result: Coreset C
Define holdout loss function: Lria( ( 1/([D) [M]) EiEDUM (10,; Ve;
Initialize Sy
Train holdout model OD = arg ming Lita O);
Select size of one step n = m louti
// Outer loop for in runge((out) do
0s = arg ming (());
Compute ReL G.tt f(ri, yı; 0s) l(r,, Mi; 0D), (wi,ws) E DV
Select top-n samples Ti by ReL:
Update current coreset S Sz UT;
Sk
Algorithm 4: CSReL-CL-Pry
Input:
Dataset sequence Di.r, memory size K Initialize memory M 0;
for i in range(1,7 + 1) do
// Train continual model with replay
0 ent = arg ming Cene U M;6); // Update memory
Compute size for each task k; = K/i,j E [1: for j in range(i) do
Reselect samples for task from C, by Alg coreset size is kü
Select samples for current task from D. with M by Alg Ccpl = kyp
Form new memory M = C3;
G ALGORITHMS FOR CSREL RESERVOIR SAMPLING
ReL for selecting sample in data stream is
RS l(x,, yis 0s.) E(x,, Mii Oont).
(18)
where Pent parameters of continual learning model, S, is selected subset of current task and (I,, yı) denotes i-th sample in B,
To apply our selection method to reservoir sampling, we maintain an memory model which is trained on St, with parameters Os Rel. computed in Eq. (18) is applied to modify update probability that
20

---

Published as a conference paper at ICLR 2025
samples with higher ReL will have higher probability to be selected memory. Specifically, for each batch, we firstly compute ReL for each sample with Eq. (18). then we normalize ReL. by Softmax function probability scaling factor. The final update probability is original update probability multiplied by scaling factor. Once one sample is selected, this sample randomly replace one existing sample in current memory. To update information contained in Os.. retrain additional model on selected data of current task when number of newly updated samples reaches threshold Nupd. and reinitialize Os. when new task comes. We refer our modified reservoir sampling method as CSReL-RS.
CSReL-RS update process is shown in Alg. where B current training batch, is number of seen samples in this stream, and K is memory size. CL process applying our CSReL-RS is shown in A
Algorithm 5: CSReL-RS
Input: Current batch B = {11, }.) continual model Pent, memory model Os., number of
seen samples n, memory K, retrain threshold Nupd: number of updated samples Rupd
Compute ReL: G; = ((r;; ye: 0s.) E(wi, yos 0 Omn)): expl G.S).
Compute probability scaling factor: 8; = E (xpp |B]
for range |B] do
if n < [K] then
// Add sample to memory
M = M (z),
n Rapd + = 1;
else
r 10, 1);
// Scale probability of update by 8
if r < (K s;) then
// Replace existing sample
pos randint(O, [M]);
Mipos] = (Iz, 3:);
Rapd 1;
if Ahppd 2 Nupd then
Initialize 0s,;
Train Os, on memory data of current task;
Tupd" = 0;
Algorithm 6: Continual learning with CSReL-RS
Input: Dataset sequence Di.r. Memory size K, loss function Cent Initialize memory M, = 0:
Initialize continual model Ootti
for t in range(1,T 1) do for D;
Sample memory batch Byy Update continual model 0ent = Cent Lene (x:
Here reuse loss B before update O.n
Update memory M by Alg
21

---

Published as a conference paper at ICLR 2025
Subset
50
Subset
Subset 160
100177
Implich prrdimm
Lose Implici gnaiiml
Implicl goulee
Lode
Candidate
Candidate
Candidate sample Subser
Candidate sample
125
Subect 150
Subset 300
Inpliic graliest
Implict graleen
Los aulle
Looe Implica graiieen
Leus
2
Candidate sample
Candadato
Candidate samplo
Candidate sample
inplicit gradient
Holdout model loss
Figure 6: Implicit Gradient and holdout model loss of top- candidates in Greedy Coreset selection on CIFAR-10 dataset.
Subset sizo 25
Subsetsize 50
Subset sizzo 75
Subset size 100
Implect grwiban
Impleer gund-an
Impikst anddon
Losk
impleik
Cundidate sample
Candidate sample Subsct size 1500
Candidate sample
Candidate sample
Sobaut size 125
Subset size: 175
Subset size 200
Implicit ara 5ke1
Inplicit git destt
Looa
2
Implici
Implicit
Candidate sample
Candidate sample
Candidate sample
Candidac sample
implicit gndien
Holdout medal
Figure 7: Implicit Gradient and holdout model loss of top- candidates in Greedy Coreset selection on CIFAR-100 100 dataset.
H Loss OF HIGH IMPLICIT GRADIENT CANDIDATES
To verify the claim that samples with high implicit gradients may be noisy ambiguous, plot the implicit gradient and holdout model loss of the top 10 implicit gradient candidates during the selection process of Greedy Coreset. We use the original implementation of Greedy Coreset for selection, with the holdout model trained on the full selection pool. Our experiments are conducted on the CIFAR- 10 and CIFAR- 100 datasets, and we plot the candidates after every 25 coreset samples selected. The results for CIFAR-10 10 and CIFAR- 100 shown in Figures 6 and respectively.
The x-axis represents the rank of candidate samples, the left y-axis denotes the implicit gradient value, and the right y-axis shows the loss value. From Figure 6 we observe that some samples have high holdout model loss values. After selecting 125 and 200 samples, the top-ranked candidate samples exhibit holdout model losses greater than 2.0, indicating that these samples almost misclassified by the holdout model. similar pattern is observed in the CIFAR- 100 experiment after selecting 150 and 200 samples shown in Figure 7
22

---

this loss Published as a conference paper at ICLR 2025
Sc viee 24
Spartii 93
S Seietior 15
Sabie des Ied
Scadto
Difficill lewd
IVfludly lidl cae
DiTaily and Can
DiTtudly level
DNTRu lend
Durteday lovd
Dufaady
DUTtuay
(a) Number of selected sample by our method from each difficulty level during selection on MNIST
Sibeciie
Sabid 169
Difiicall lond
Uktbealty Subee zet 150
Duftidlly Schvet mic 113
Dufieaaly lerd Nabvd alo 288
Sebvd dec 133
UNfiiraly lord
Lkfkelly Indl
Dufbudyy zmd
Diftuaal love
(b) Number of selected sample by Greedy Coreset from each difficulty level during selection MNIST
Figure 8: Difficulty analysis on MNIST during selection.
Since samples with high holdout model loss are often ambiguous or noisy, these results suggest that such samples may exhibit high implicit gradient values and could be selected for the coreset.
DIFFICULTY OF SELECTED SAMPLES
We analyze difficulty of selected samples. To define sample difficulty, we firstly train a model on full dataset convergence, then compute loss each sample in full dataset, as metric difficulty. This difficulty means how hard-to-learn of one sample. Based on difficulty, we equally split all samples into groups with the incremental of difficulty, indicating levels of difficulty. For each difficulty level, we count how many samples in this level are selected, aiming to analyze preference of one selection method. We analyze preference along the selection procedure for our method and Greedy Coreset (Borsos et al., 2020). and selection experiment is the same in Section 5. For both methods, we analyze preference with subset size 25. 50, 75, 100. 125, 150, 175, 200. Results on MNIST, CIFAR-10 10 and CIFAR- 100 are shown in Figure 8 FigurePandd Figure 10 respectively.
For MNIST dataset, from Figure both our method and Greedy Coreset tend to select harder samples. When subset size smaller than 100, our method selects more hard samples, this explains why our method under-perform Greedy Coreset when subset size is smaller in Figure (a), and as subset size increases, the preference of both methods tend to be the same, this is also consistent to test accuracy in Figure [ (a). Note that, compared CIFAR-10 and CIFAR-100. since MNIST much simpler, both methods tend to select hard-to-leam samples, indicating these samples are discriminative.
23

---

Published as a conference paper at ICLR 2025
Sabvd
Sdhoaa Sdacad
Unilinatt lord
Jkileall Sobec zec 100
Duibady Schee mee (1)
Udtlicaty lord Nabve aee 288
butuet see 123
DNfiuraity lceel
DuTaudy xnd
Diflueay level
(a) Number of selected sample by our method from each difficulty level during selection CIFAR-10
Schendi
Scbetrie 35
Difficaty Salue des 135
2edtkit Scbarcze 150
Diffadyy Schout dou 175
DUfTualty Salure daa
Solondd wrdde
DVficddy land
DuTaaly and
DUTC440 lend
(b) Number of selected sample by Greedy Coreset from each difficulty level during selection CIFAR-10
Figure 9; Difficulty analysis CIFAR-10 during selection.
24

---

Published as a conference paper at ICLR 2025
Scbeniby
Subeer
Difficiity livel
Dufiulyy
DVftadiy
(a) Number selected sample by our method from each difficulty level during selection on CIFAR-100
Sypethee 31
Nabvet sted Ite
lend)
curicuity lnvel
Duftadyy
ENtticatt
(b) Number of selected sample by Greedy Coreset from each difficulty level during selection on CIFAR-100
Figure 10: Difficulty analysis on CIFAR-100 during selection.
(a) CIFAR-10
(b) CIFAR-100
Figure 11: Visualization of selected samples by Greedy Coreset in the last two difficulty groups in CIFAR-IE and CIFAR-100.
25

---

Published as a conference paper at ICLR 2025
Ohrmattd Gredy Carect PBCS
Dermebad Grealy Conoe:
Goodl Curost PSC3
Selectel sample
Sclectel mone somples
FBCS
Selected
Noise miño
Noise raño
Nobii ratio
(a) CNN on MNIST
(b) ResNet-18 on CIFAR-I0
(c) ResNet-18 on CIFAR-I00
Figure 13: Number of selected noisy samples under different noise ratio. Our method selects much less noisy samples compared to other two methods, demonstrating that our method are more robust under noisy data case.
For CIFAR-10 dataset, from Figure our method tend to select more easy-to-learn samples com- pared to Greedy Coreset. Note that our method selects no samples from the last two difficulty levels which indicate ambiguous noisy samples, while Greedy Coreset selects much more samples in last three difficulty levels. We have visualized the samples selected by Greedy Coreset from the last two difficulty levels in Figure 11] and observed that, while these samples correctly labeled, some samples are non-typical and may contain misleading features that negatively impact the model's learning process. These results demonstrate that CSReL could effectively avoid selecting noisy or ambiguous samples. Experiments in Figure (b) indicate that these harder samples may harm model performance.
For CIFAR- 100 dataset, from Figure 10 the results are similar to results on CIFAR- 10. Our method selects much less samples in high difficulty levels. According test performance in Figure our methods selects less ambiguous samples or noisy samples, and perform better than Greedy Coreset.
In conclusion, our method could identify discriminative samples and effectively avoid selecting ambiguous or noisy samples.
J ANALYSIS ON REPRESENTING FULL DATASET
Same experiment in Section 5 also conduct experiments on CIFAR- 100 dataset. We train mod- els selected subset along the selection procedure, and evaluate these models by test accuracy and av- erage loss full dataset. loss and accuracy with respect to subset size shown in Figure 12]
Loss full distsett
Text accunacy
From results in Figure 12 model trained subset selected by our method could achieve higher test ac- curacy and lower average loss full dataset. These results demonstrate that, compared to Greedy core- (Borsos et al., (020)), our method selects a subset with higher performance gain. This finding aligns with the results of our ablation study on the CIFAR- 10 dataset, presented in Section 5.3
Subbect
Greedy Conuat acoeaay
Taos of Grundy Connet
Ozrmehhet
of mathed
Figure 12: Average loss the full dataset and test accuracy of models trained on sub- sets with different size of our method and Greedy Coreset on CIFAR-I00 dataset.
K DATA SUMMARIZATION UNDER DATA NOISE
We count the number of noisy samples selected by different methods in the experiment of Section 5.3 with results shown Figure Our method selects significantly fewer noisy samples, particularly the challenging CIFAR-10 and CIFAR-100
26

---

Published as a conference paper at ICLR 2025
Table 5: Forgetting continual learning experiments.
Methods
Split MNIST
Split CIFAR-10
Split CIFAR-100
Perm-MNIIST
Uniform sampling
6.26+1.20
26.9324.54
79.55+2.19
10.2940.94
k-means of features
6.29+1.42
60.98+1.53
79.00=1.61
10.55+0.63
k-center of embeddings
5.05+1.69
56.71+4.22
78.82+1.12
10.28+0.01
Hardest samples
15.35=3.16
72.55+1.94
73.97
11.7440.77
iCaRL's selection
5,45+0032
58.64+4.11
79.65+1.02
9.77+0.26
Greedy Coreset
3.20:0.49
58.41+3.68
79.01+0.86
9,45+0.23
PBCS
5.65+0.80
53.25#1.61
17.00+1.76
12.10+1.00 004
CSReL-CL
3.03+0.50
43.43=5.44
75.09#
8.8120.50
CSReI-CI-Pry
3.92+0.04
37.9448.21
72.3640.15
8.44+0.19
datasets compared MNIST. These findings align with the performance drop observed in Section demonstrating that our method is more robust in handling noisy data.
5.3
L FORGETTING IN CONTINUAL LEARNING EXPERIMENTS
In this section, present forgetting metric (Chaudhry al. 201 which measures performance
degradation in subsequent tasks for experiments in Section 5.2
Computation of forgetting is
5 1E(1....,-1) atj Ok.jr
Vj < <,
where 0k.j denotes accuracy task j after training k-th task. We evaluate forgetting of CL. experi- ments in Table
Among coreset selection methods, our method has the minimal forgetting on all datasets. Our method could also outperform other sample selection methods in most datasets. This demonstrates the effectiveness of our method for selecting a informative subset to prevent forgetting.
M CONTINUAL LEARNING ON BCSR SETTING
To further demonstrate the effectiveness of method, evaluate CSReL-CL under the same setting as BCSR (Hao et al. 2024) on the Split CIFAR-100 dataset, which is equally divided into 20 tasks. The memory size set 100, and task IDDs provided during inference. We replace the selection method in BCSR with our CSReL selection. The results are presented in Table 6 with BCSR and other baseline results referenced from the BCSR paper (Hao al. 2024
The results in Table 6 show that our method could outperform other coreset selection baselines. demonstrating the effectiveness of selection method.
N CONTINUAL LEARNING EXPERIMENT ON IMAGENET 500 CLASS
To evaluate the scalability of our CSReL-RS
large dataset, we select 500 classes from
ImageNet- 1K and split them into 10 tasks, with the memory size set to 1000. The baselines are ER (Riemer et al. 20188) and DER++ (Buzzega al. 2020), replace reservoir sampling in the bascline methods with CSReL-RS as CSReL-ER and CSReL-DER++. For DER+,, we use ReL- emb for selection. The final average accuracy is presented in Table
Results in Table 7 show that replacing reservoir sampling in ER (Riemer et al. 2018 with CSReL- RS leads to slight performance improvement, while combining method with DER++ (Buzzega ct al. 2020) results in a significant performance boost. These findings demonstrate that our method remains effective larger datasets, showcasing its scalability for larger datasets.
27

---

Published as a conference paper at ICLR 2025
Table 6: Average accuracy on Split CIFAR-I00 under BCSR setting, our CSReL selection could consistently outperform other methods.
Method
Average accuracy
Forgetting
k-means features (Nguyen et al. 2017)
57.82+0.69
0.070=0.003
k-means embedding (Sener & Savarese 2017
59.7740.24
0.061
Uniform
58.99#0.54
0.074=0.004
iCaRL (Rebuffi et al.
2017)
60.74+0.09
0.044#0.026
Grad Matching (Campbell & Broderick 2019
59.17#0.38
0.067=0.00
SPR (Kim et al.
2021)
59.56+0.73
0.143+0.064
MetaSP (Sun et al.] 2022a)
60. 10.1440.25
0.056*0.2233
Greedy Coreset (Borsos et al. 2020
59.3940. 16
0.066*0.017
GCR (iwari et
2022)
58.73+0.43
0.073
PBCS (Zhou et al.
20226
55,64+2.26
0,062
OCS Yoon et al.
202T
52.57+0.37
0.088+0.001
BCSR (Hao et al. CSReL.
2024
61.60#0.14
0.051
62.10+0.45
0.094+0.006
Table 7: Average accuracy on ImageNet 500 class dataset, our method could consistently outperform the random counterpart.
Method
Average accuracy
ER
9.85+0.01 10.30+0.26
CSReL-ER
DER+
15.94+0.99 94
CSReL-DER++
19.03=0.21
O EFFECT OF HOLDOUT MODEL TRAINING EPOCH IN CONTINUAL LEARNING
We also test the effect of holdout model training epoch in CL task. Specifically, we test different holdout model training epochs on Split CIFAR-100 100 dataset with the same setting as Section 5.2. The results are shown in Table g
In practice, we find training holdout model 10 epochs enough for good performance, compared to 100 training epochs, the computation overhead is small. We show that final results is relatively stable with respect to different holdout model training epochs.
P TIME COST AND MODEL SIZE IN DATA SUMMARIZATION EXPERIMENT
For data summarization experiment in Section 5.1 we list time cost and model parameters on MNIST, CIFAR-IO and CIFAR-100 in Table 9 All experiments are conducted on NVIDIA RTX 3090. Specifically, We use 5-layer CNN on MNIST and use ResNet- 18 for CIFAR-10 and CIFAR- 100, and 200 coreset samples are selected from 10000 candidate samples for each dataset.
Compared to MNIST, selecting corest in CIFAR-10 10 and CIFAR- 100 takes 7x time. However, com- pared to scaling of model size and data size, the time cost of training holdout model scales mildly.
Table 8: Effect of holdout model training epoch in continual learning.
Holdout training epochs
Average accuracy
10
17.48+0.21
15
17.21+0.07
20
17.43+0.55
28

---

Published as a conference paper at ICLR 2025
Table 9: Time and model parameters in data summarization experiment.
Dataset
Model parameters
Holdout training time
Selection time
MNIST
184.59K
159.84s 349.74 74s
102. 002.70s
CIFAR-10
11.16M
737.68s
CIFAR-100
11.21M
350.87s
735.85s
Dataset
Coreset
Coreset selection
CL model
Holdout model
Coresct
model
Memory
Candidatc batch
Task
Task
Task
Overview of coreset continual learning process. Coreset is selected from current task dataset after training on current tasks. The selected coresct are added into memory for replay. In coreset selection, holdout model is trained multiple steps. In each selection step, I train coreset model on currently selected coreset, (2 select candidate samples, 3 compute ReL for each candidate sample, 4 rank candidate samples by ReL., 3 select samples with top-n ReL into coreset.
Figure 14:
current task dataset initially, then coreset is selected in
Besides, time cost selection also scales mildly as model size increases and data size. Our base- line Greedy Coreset (Borsos et al. (2020) takes 4000 seconds for computing Neural Tangent Kernel, compared to Greedy Coreset (Borsos et al. 2020), our method much more efficient.
OVERVIEW OF CORESET SELECTION FOR CONTINUAL LEARNING
To provide a clearer understanding of our method, we further illustrate our CSReL-CL approach, proposed Section 4.1 Figure [4 For the continual learning task, we treat the training dataset of the current task as the entire dataset and select the coreset from it. After training the continual learning (CL) model on each task, the holdout model then trained on the dataset of the current task.
In CSReL-CI--rr proposed in Section 4.2 holdout model trained on current task dataset and memory. The holdout model is updated after training on each task, rather than being trained only at the initial step. In the CSReL-RS method, introduced Section [43] we use the CL model itself as the holdout model. Consequently, the holdout model incorporates updated information, enabling it to provide valuable insights for the selection process.
R IMPACT OF SELECTION ORDER
Since our method selects samples for the coresct using a greedy incremental framework. meaning that once sample is selected, it is no longer considered in subsequent selections. The problem
29

---

Published as a conference paper at ICLR 2025
of selection order is a common challenge in greedy methods. There possibility that the perfor- mance gain of selected samples might decrease or that some selected samples could be detrimental. To address this, we conducted experiments on MNIST, CIFAR- 10 and CIFAR- 100 by iteratively removing n samples from the 200-size selected coreset of with the lowest performance gain, then reselecting new samples from the remaining candidates. The results, presented in Table [Q] show that performance remains stable after removal and reselection. Additionally, the reselection process produces coresets different from the original ones.
One possible explanation is the existence of multiple subsets containing a similar amount of knowledge. Our method selects samples that are both representative and complementary to the current coreset. This implies that select- ing different initial samples results in similar fi- nal performance because our method inherently identifies subsets that complement the initial se- lection. Further evidence for this comes from
Table 10: Test accuracy after removing and rese- lecting samples for coreset.
n
MNIST
CIFAR-10
CIFAR- 100
0
96.67
39.46
9.26
10
96.53
39.58
10.01
20
96.51
39.60
8.93
our evaluation the coreset selection method
50
96.42
39.16
10.08
under different random seeds in Section [5.1 Despite the variation in selected subsets, the performance remains consistent. Therefore, the order of selection is not significant concern in real-world data scenarios.
100
96.09
39.31
10.19
S DISTINGUISHING DIFFICULT SAMPLES FROM NOISY SAMPLES
We note that difficult samples and noisy samples cannot be distinguished solely based on Eq. and (6), as both types of samples may exhibit high loss values on the holdout model. However, our method selects coreset iteratively. If noisy or ambiguous samples are included in the coreset, they may hinder the coreset model 0g from effectively learning other clean samples, leading to increased loss for the clean candidates on the coreset model. As result, the ReL of clean candidates may increase, making them more likely to be selected in subsequent iterations.
While difficult samples could be informative and not harm the learning of other samples, selecting difficult samples does not preclude the selection of other difficult samples, as demonstrated in Fig- ure ((a in Appendix [[] For the relatively simple MNIST dataset, difficult samples tend be more informative and are therefore selected. Additionally, we analyzed the difficulty levels of selected samples in the noisy MNIST experiment discussed in the Ablation Study (Section 5.3 in Figure
15 The results show that more easy samples are selected from the noisy MNIST dataset, due to the learning of these samples is disrupted by the presence selected noisy samples. Consequently, dif- ficult samples and noisy samples yield different outcomes. Additionally, we note that the noise rates in the last two difficulty groups shown in Figure 15 are 0.24% and 79.76% respectively. Our method group while selecting fewer samples from the
tends to select more samples from the second-to-las last group. This is because the holdout model can effectively learn difficult samples, whereas noisy samples cannot be well learned by either the holdout model the coreset model.
In summary, our method is capable of selecting more difficult samples when they are informative and consistent with other samples. For noisy and ambiguous samples, however, our approach prioritizes cleaner samples to mitigate the disruption caused by noisy data. In low noise-ratio scenarios, which common in real-world applications, noisy samples can be identified in the holdout model can effectively learn difficult samples but fails to learn noisy ones. As result, our method effectively treat difficult and non-representative samples, ensuring appropriate selection.
T CORESET SELECTION FOR CONTINUAL. LEARNING WITH LARGE BACKBONE MODELS
To further demonstrate the effectiveness and efficiency of method over other bilevel coreset selection methods in continual learning. We conducted additional experiments applying complex backbones ResNet-50 and VIT-Tiny (Wu al. 2022) CIFAR-100 and Tiny-ImageNet respec-
30

---

Published as a conference paper at ICLR 2025
Selvet das
S Siberiie V)
Schorr
Save de 191
Difficily levi Savin
Diffuriy lovel
Diffially lowel Salas
Deffedry kovl das
Sabou Giv
Dittiutt lovd.
Difiably lavil
Diffidiy
Difflddy lnv
Figure 15: Number of selected samples from each difficulty group in noisy MNIST experiment, noise-ratio=0.1.
Table 11: Average accuracy and time cost of Greedy Coreset, BCSR and CSReL-CL on CIFAR-100 and Imng-ImageNer dataset.
CIFAR-100
Tiny-ImageNet
Average accuracy
Time cost
Average accuracy
Time cost
Greedy Coreset BCSR Our method
28.1740.57
21h 58m
12.1740.07
24h 55m
27.32+0.10
11h 26m
12.33+0.08 17.51+0.11
28h 13m
32.51
10h 04m
15h 13m
tively. Both datasets were evenly split into 10 tasks, with memory sizes set 1000 and 2000 for CIFAR-100 and Tiny-ImageNet, respectively.
We compared method with related bilevel coreset selection methods for continual learning. including Greedy Coreset (Borsos et al. 2020) and BCSR (Hao et al.) (2024). The average accuracy and time cost are summarized in Table II Our method outperforms other bilevel coreset selection approaches by a large margin on both datasets while also being more efficient, particularly when using the more complex VIT-Tiny backbone.
U FEATURE MAP OF SELECTED SAMPLES
We have visualized the features of coreset samples selected by different methods from CIFAR-10 using t-SNE. The features extracted using ResNet-18 model trained on CIFAR-10, as shown in Figure Representative samples share common features with other samples and are effectively learned by the model, thus could be well classified. The features of the coreset selected by our method are concentrated near the high density part and better separated compared to those selected by the baseline methods, demonstrating the ability of our approach select representative samples.
Our work defines informative sample as samples contains new knowledge compared currently selected coreset. To demonstrate that our method selects both representative and informative sam- ples, we train model on the first 150 selected samples and visualize their features as dots. We then visualize the next 50 selected samples triangles. Figure [7] shows the features of coreset samples selected by our method and by Greedy Coreset. The later-selected samples by our method tend to lie near the margins of feature clusters, indicating they are less well-learned by the model. Fur- thermore, samples selected by our method are more closely aligned with the feature clusters each class, while those selected by Greedy Coreset are more scattered. This demonstrates that our method effectively selects samples that are both representative and informative. However, it important to note that the informativeness of sample cannot be determined solely based its features,
31

---

Published as a conference paper at ICLR 2025
(a) Greedy Coreset
(b) PBCS
CSReL
Features of selected samples by Greedy Coreset (Borsos et al. 2020), PBCS (Zhou et al., 20226) and CSReL. The shadow part is features of randomly selected samples from the full dataset. Our method selects more representative samples compared to other methods.
Figure 16:
(a) Greedy Coreset
(b) CSReL
Figure 17: Features of currently selected samples and later selected samples, illustrating that our method selects both representative and informative samples.
well learned samples, such as ambiguous noisy ones, may negatively impact the model's 's learning process.
We have also plotted the features of samples selected by iCaRL's selection (Rebuffi et al. 2017 and k-means features selection (Nguyen al., (017) in Figure [8 Compared Greedy Coreset Borsos al. 2020) and PBCS (Zhou et al. 2022b)), features are less scattered.
V ADDITIONAL DATA SUMMARIZATION RESULTS
We further conduct coreset selection experiments with RCS (Xu et al. 2023) and CDS (Wan al., 2024) on the CIFAR-10 and CIFAR-100 datasets under the same settings as Section 5.1, with the coreset size set 200. The results presented in Table 12
RCS (t al. 2023) defines a bilevel selection objective for adversarial contrastive learning, which differs from the supervised classification task addressed in work. As result, RCS underper-
ESME2
1-5NE
L-SNE1
(a) iCaRL's selection
(b) k-means features selection
Figure 18: Features of samples selected by iCaRL's selection and k-means features selection.
32

---

Published as a conference paper at ICLR 2025
Table 12:
Test accuracy of model trained on coreset selected by different methods, our method outperforms all other baseline methods.
Method
CIFAR-10
CIFAR-100
RCS
27.17=1.11
4.87+0.60
GC+CDS
33.03+1.74
8,06+0.23
Greedy Coreset
16.53+1.40
6,94+0.23
PBCS
07.41
6.77+0.18
CSReL (ours)
39.58+0.14
8.91=0.42
forms other coreset selection methods. GC+CDS (Wan et al. 20244 leverages the contributing di- mension structure within the Graph Cut framework (Iyer et. 2021). constructing coresets based on balance between representativeness and diversity. However, unlike bilevel optimization methods, this approach relies on hyperparameter tuning to achieve this balance, which may be suboptimal. Additionally, the improvement model performance when adding sample the coreset cannot be directly evaluated solely through representativeness and diversity metrics. In contrast, bilevel optimization methods select samples by evaluating performance in the outer loop, thereby providing stronger performance guarantees for the resulting coreset.
bilevel coreset selection methods outperform RCS (Xu al. 2023) and GC+CDS WWa on the CIFAR-16 10 dataset and perform slightly below GC+CDS (Wan et al.) 2024) on the CIFAR- 100 dataset. These results indicate that bilevel coreset selection methods remain effective and robust. Furthermore, our CSReL selection method surpasses all baseline methods, showcasing its superior effectiveness and timeliness.
2024)
W DATA SUMMARIZATION EXPERIMENT DETAILS
We conduct experiments on MNIST, CIFAR-10 and CIFAR-100, coreset size of all datasets is set to 200. For experiments on MNIST and CIFAR-IO, we follow experiment settings of Borsos et al. 2020) and Zhou et al. (2022b). Different from Greedy Coreset which uses Neural Tangent Kernel Jacot et al., 2018)) as inner model, we use same model structure for holdout model and current model.
We use CNN as backbone for MNIST, which contains two blocks of convolution, dropout, max- pooling and ReLU activation, two convolution layers have 32 and 64 filters with 5 x 5 kernel size. Two fully connected layers of size 128 and 10 with dropout follows convolution blocks. The dropout probability 0.5. We train CNN selected coreset using SGD optimizer with learning rate 2e -e training batch size is set to 32 and training epochs is set to 3000. This training protocol is applied in all compared methods. The is not stable under different random seeds, therefore, we use SGD optimizer instead.
why we don't use Adam optimizer is that found performance
We use same ResNet-18 as in Borsos 10 and CIFAR-100 dataset. Note that there is no experiments on CIFAR-10 and CIFAR-100, we train ResNet on selected coreset using Adam opti- mizer with learning rate 5 training batch size is set to 64 and training epochs is set to 1800. This training protocol is applied in all compared methods.
(2020) and Zhou et (2022b) as backbone for CIFAR-
layer in this backbone. For
In experiment on MNIST, holdout model is trained by SGD optimizer with learning rate 1.5e training batch size 32 and training epoch 125. In each selection step, current model trained by SGD optimizer with learning rate 2 batch size is set 32. Since selected coreset is very small, to avoid overfitting ,we train current model on selected subset with 16 epochs. Initial coreset size is set to 0 and selection step is 200.
In experiment on CIFAR-10 10 and CIFAR-100, holdout model is trained by SGD optimizer with learning rate 3e -3 batch size is set 32 and training epoch is 100. In each selection step, current model is trained by SGD optimizer with learning rate 2e -2 batch size is set to 32, current model training epoch is 16. Initial coreset size is set to 0 and selection step is 200.
33

---

Published as a conference paper at ICLR 2025
Table 13: Optimization hyperparameters in continual learning experiments in Table
Dataset
Batch size
Epochs
Optimizer
Learning rate
Loss factor
Split MNIST
256
400
Adam
5e-4
100.0
Split CIFAR-10
256
400
Adam
5e-4
20.0
Split CTFAR- 100
32
100
SGD
2e-2
4.0
Permuted MNIST
256
400
Adam
5e-4
0.1
Table 14: Optimization hyperparameters in continual learning experiments in Table
Dataset
Batch size
Optimizer
Learning rate
CE Loss factor
KD loss factor
Split CIFAR-100
32
SGD
2e-2
1.0
0.2
Split Tiny- NeageNet
32
SGD
3e-2
1.0
0.1
X CONTINUAL LEARNING EXPERIMENT DETAILS
X.1 DATA SUMMARIZATION FOR CONTINUAL LEARNING
Datasets:
We conduct experiments on Split MNIST, Split CIFAR-10, Split CIFAR-100 and Perm MNIST. Split MNIST, CIFAR-10 and MNIST consist of 10 classes. Following Borsss et al. (2020) split CIFAR-IO 10 and MNIST into 5 tasks with classes for each task. We split CIFAR-10K into I0 tasks with 10 classes for each class. For Perm MNIST contains 10 tasks and each task is randomly permuted version of MNIST. Also following Borsos al. (2020), in experiments Split MNIST, Split CIFAR-10 and Perm MNIST, we randomly select 1000 samples from each task for training, for Split 100, we use all data of each task.
Following Borsos et al. (020). we apply normalization to Split MNIST and Perm MNIST. For more complicated CIFAR- and CIFAR-100 dataset, we apply random crop, random horizontal flip and normalization.
Augmentations:
Backbones: We use same CNN structure in Section W for Split MNIST, and use the same ResNet- 18 in Section W for Split CIFAR-10. For Split CIFAR-100,, we add BatchNormalization layer in each convolution block of ResNet-18. For Perm MNIST, we use fully connected net with two hidden layers with 100 units, ReLU activations, and dropout with probability 0.2 on the hidden layers.
Optimization:
Optimization related hyperparameters shown in Table 13 Eq. (T7). Following Borsos et al. 2020) and Zhou et al. (2022b), we use Adam optimizer (Kingma 2014) for experiments on Split MNIST, Split CIFAR- and Permuted MNIST. For Split CIFAR-100 dataset, we use SGD optimizer for all compared methods.
the loss factor is in
X.2 CONTINUAL LEARNING WITH MODIFED RESERVOIR SAMPLING AND KNOWLEDGE DISTILLATION
Datasets: We conduct experiments on Split CIFAR-100 and Split Tiny-ImageNet dataset. We split CIFAR-100 100 into 10 tasks with 10 classes for each class. For Split Tiny-ImageNet, split totally 200 classes into 10 disjoint tasks. All samples of each task are used during training.
Augmentations: We keep the same augmentation used in mammoth (Buzzega al. 2020). Both Split CIFAR-100 and Split Ein-ImmgeNee dataset applics random crop, random horizontal flip and normalization.
Backbones: Both dataset use the same ResNet-18 in mammoth implementation (Buzzega et al, 2020).
Optimization: We list all the hyperparameters related to optimization Table consistent with corresponding baselines.
Training epoch
34

---

Published as a conference paper at ICLR 2025
Figure 19: Visualization of selected sample in each class from CIFAR-10.
Table 15: Time cost for offline continual learning tasks
Split MNIST
Split CIFAR-10
Perm MNIST
Split CIFAR-100
Time cost
4.8m
27.6m
17.5m
69.4m
Y
VISUALIZATION OF SELECTED DATA
To make the effectiveness of our method straight forward, visualize the first 10 selected samples of each class CIFAR-IO, pictures are shown in Figure each row collects images from same class.
From visualization, we can see that sample selected by our method is clear, unambiguous and verse.
Z COMPUTATION RESOURCES AND EXPERIMENT TIME COST
We conduct all experiments on NVIDIA GeForce RTX 3090 graphical cards with 24 GB memory for each graphical card. Our CPU type is Intel(R) Core(TM) 19-12900K, memory of our server is 32 GB. We list time cost for all main offline continual learning experiments in Table 15 and list time cost for CSReL-RS in Table T6
Table 16: Time cost for continual learning tasks using CSReL-RS
Split CIFAR-100
Split Tiny ImageNet
CSReL-ER CSReL-DER++
56m
7h 47m
lh 20m
8h 56m
35

---

Published as a conference paper at ICLR 2025
AA URL OF CITED ASSETS
cotpemmm.com/exd///mss.... kriz/cifar-10-pythoon We download CIFAR-100 dataset from https://www.cs.oorntto...d kriz/cifar- 10-1110-ppyttt 10-.y-oootttt.....zz
We download MNIST dataset from
We download CIFAR- 100 dataset from https:/wwww.ss.orrtto...uuu kriz/cifar- Ariz/cifar-10--pythonr 100-python.tur.gz
Tiny ImageNet dataset is downloaded by mammoth implementation, http://gthhl..ommmmggglaaammnnt
the URL is
Greedy Coreset inplemmttttnnnntnniionn is downloaded from http:///gghiiboommmaaaaaooosssss
AB
BROADER IMPACT
Coreset selection aims summarize informative subset from full dataset, which could signif- icantly reduce training cost and energy consumption. Storage required may also be reduced with coreset selection method. Our coreset selection method could be applied not ony continual learn- ing, but also Neural Architecture Search and Reinforcement learning. Continual learning our work could also improve intelligent agent for continually learning new knowledge, enabling more general applied models and personalized models.
36


Reviews:
[{"review_id": "Vhk2vOd0pP", "reviewer": "Reviewer_MUYB", "summary": "The work presents a concise way of preventing select noisy data into the coreset due to their high loss value. By using a holdout model as the baseline, it incrementally selects samples with high log prob loss. Which is mathematically shown by the authors that such samples are informative but not well represented yet in the coreset. The work shows improvements on various common continual learning datasets.", "questions": "1. How do you train the holdout model? I believe it is trained only on the initial dataset, without any information about the incoming task. What if the holdout model is not good enough to calculate a meaningful loss for later tasks? For example, if the incoming task is very different from the previous one (a huge domain shift), the loss of holdout model might be poor for every input. In such a scenario, CSRL might not work.\n2. What's the difference between eq.4 and eq.7? What's the difference between $\\mathcal{S}$ and $\\mathcal{M}$?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. A detailed, structured, and rigorous paper with extensive derivations and experiments.\n2. Conducts a variety of experiments, including those on standard continual learning datasets, more challenging ImageNet variants, training overhead, and extensive ablation studies.\n3. Presents a clear and straightforward idea for selecting good samples to add to the coreset, supported by sound equations.", "weaknesses": "1. Not easy to understand at the first glance. It might be better to add some teaser figure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JUlvdyWl2t", "reviewer": "Reviewer_kUqK", "summary": "This paper introduces a method for coreset selection using reducible loss to improve continual learning (CL) performance. Traditional experience replay-based CL methods face challenges when selecting representative samples from non-stationary data streams, often retaining ambiguous or noisy data that can degrade model performance. The proposed method uses reducible loss as a selection criterion, estimating each sample's performance gain to determine its representativeness and informativeness. This selection framework is computationally efficient, requiring only a forward computation without gradient backpropagation, thus avoiding the computational cost associated with implicit gradient methods. Additionally, the approach is extended to handle challenges in CL, including task interference and data streaming, demonstrating robustness against noisy data and effective scalability for large datasets like ImageNet. Experimental results show that this method consistently improves CL model accuracy and reduces forgetting when applied to various datasets.", "questions": "1.\tThis paper builds upon the limitations identified in Greedy Coreset (NIPS 2020). However, given that four years have passed since its publication, does this work lack timeliness? Can its performance surpass current state-of-the-art methods?\n2.\tThis paper uses a coreset-based replay method, so it should include more comparative experiments with other coreset selection approaches. The comparative experiments in this paper are insufficient.\n3.\tSince the paper mentions that reducible loss eliminates the need for backpropagation and reduces computational cost compared to traditional bi-level coreset selection strategies, an additional experiment should be included to validate this conclusion.\n4.\tDoes training a holdout model using all the data contradict the continual learning (CL) setting?\n5.\tThe paper mentions that “samples with high performance gain are both representative and informative.” Could the distribution of high-performance samples in feature space be provided as evidence for this claim?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper introduces a coreset selection method based on reducible loss for continual learning, aiming to improve model performance by selecting representative samples efficiently. By focusing on performance gain, this approach reduces computational demands and mitigates the impact of noisy data. It addresses key continual learning challenges, such as task interference and data streaming, making it particularly valuable for dynamic and large-scale applications.", "weaknesses": "The coreset selection method based on reducible loss presented in this paper shows promise for continual learning applications, but some of the design choices lack thorough validation, and certain aspects of the experimental comparisons and theoretical evidence could be more comprehensive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zl1wyqnOsZ", "reviewer": "Reviewer_Ungg", "summary": "The authors focus on the coreset selection method used in continual learning scenario. They believe that the not well learned samples, like ambiguous and noise samples should not be included in the coreset. Therefore, a new method CSRL is designed with reducible loss to filter the not-well-learned samples during coreset selection. The experiments are conducted on MNIST, CIFAR-10, and CIFAR-100 with several earlier competitor methods.", "questions": "Please see my questions in Weaknesses.", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. I find the authors make great efforts in polishing this work. I found multiple supplementary explanations for the main text in the supplementary materials, which I believe is commendable and should be encouraged. \n2. The paper is well-organized and writing is almost clear.", "weaknesses": "About Method:\n\n1. The proposed CSRL seems to be severely affected by the selection order and subset division way. If Algorithm 1 is correctly implemented, this means that the selected samples in the coreset will not be considered in the following selection process. Obviously, the gain score G of samples stored in the coreset will change as the model gradually learns from all subsets. The authors ignore this possibility, which may lead to the result that previously selected candidates are more likely to be stored in the coreset. In other words, the gain score may not be fair for all candidates.\n\n2. The gain score G seems only to assess which samples are easier to learn fully, rather than which are more core. From the expressions in Equations (4) to (6), it is evident that the authors do not differentiate between difficult samples and noise when applying this gain score, instead treating both as noise. Clearly, difficult samples should also be treated appropriately during coreset selection.\n\n3. I notice that the authors provide the results in Appendix H. However, I believe these results do not verify that CSRL is more reasonable and may even exacerbate the concerns I raised in W1. Since the overall difficulty level of the entire dataset is not provided, we cannot determine whether CSRL regulates the coreset selection more effectively. As I expressed in W1, the results can also be interpreted to mean that CSRL is severely affected by the selection order (which seems to be random) during application, leading to selection results that might also be random as the size of the coreset increases. This suggests that CSRL has less impact on regulating coreset selection. In contrast, the other competing method appears to perform better. Additionally, the authors claim that selecting fewer hard-to-learn samples will make a difference, which further confirms that hard samples are considered the same as noise samples.\n\n\nAbout Experiments:\n\n4. I noticed that the settings used by the authors are somewhat outdated. First, the largest network structure in this paper is only ResNet-18, and the authors further provide training and selecting time results based on this backbone to show the efficiency of their method. From both the performance and efficiency perspectives, I find this unconvincing. With such simple network structures, it is difficult to compare performance differences between new and old methods. Let alone that many recent state-of-the-art methods are not included in the main experimental results.\n\n5. In terms of the experimental results, I find them unconvincing because the authors only adopt competitor methods from 3 and 5 years ago in all the experimental results presented in the main text. It becomes even more puzzling when I notice that the authors are aware of the latest state-of-the-art methods (e.g., BCSR, GCR) but only compare CSRL with them on one of the datasets used in BCSR in the appendix. Why are the latest state-of-the-art methods not compared under the same settings in this paper's main text? I believe that both the absence of state-of-the-art methods in the main text and the partial comparison results of these methods shown only in the appendix do not support the authors' claim that the proposed CSRL is state-of-the-art.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)