Rethinking Memory in Continual Learning: Beyond a Monolithic Store of the Past
Authors: Yaqian Zhang
Bernhard Pfahringer
Eibe Frank
Albert Bifet
Venue: Accepted by TMLR ()

Abstract: Memory is a critical component in replay-based continual learning (CL). Prior research has largely treated CL memory as a monolithic store of past data, focusing on how to select and store representative past examples. However, this perspective overlooks the higher-level memory architecture that governs the interaction between old and new data. In this work, we identify and characterize a dual-memory system that is inherently present in both online and offline CL settings. This system comprises: a short-term memory, which temporarily buffers recent data for immediate model updates, and a long-term memory, which maintains a carefully curated subset of past experiences for future replay and consolidation. We propose  \textit{memory capacity ratio} (MCR), the ratio between short-term memory and long-term memory capacities, to characterize online and offline CL. Based on this framework, we systematically investigate how MCR influences generalization, stability, and plasticity. Across diverse CL settings—class-incremental, task-incremental, and domain-incremental—and multiple data modalities (e.g., image and text classification), we observe that a smaller MCR, characteristic of \textit{online CL}, can yield comparable or even superior performance relative to a larger one, characteristic of \textit{offline CL}, when both are evaluated under equivalent computational and data storage budgets. This advantage holds consistently across several state-of-the-art replay strategies, such as ER, DER, and SCR. Theoretical analysis further reveals that a reduced MCR yields a better trade-off between stability and plasticity by lowering a bound on generalization error when learning from non-stationary data streams with limited memory. These findings offer new insights into the role of memory allocation in continual learning and underscore the underexplored potential of online CL approaches.

Published in Transactions on Machine Learning Research (12/2025)
Rethinking Memory in Continual Learning: Beyond a Mono- lithic Store of the Past
Yaqian Zhang
yagiana@huuiiatoo.ccmm
AI Institute, University of Waikato
Bernhard Pfahringer
bernhard@@wnaikatoo.cc
AI Institute, University of Waskato
Eibe Frank
eibe@woikatoo.co oc.
Institute, University of Waskato
Albert Bifet
abifet@waikato.com ac.
Institute, University of Waskato
LTCI, Télécom Paris
Reviewed on OpenReview: https: openreview. net/ forum? id- wgj VUTY
Abstract
Memory is a critical component in replay-based continual learning (CL). Prior research has largely treated CL memory a monolithic store of past data, focusing on how to select and store representative past examples. However, this perspective overlooks the higher-level memory architocture that governs the interaction between old and new data. In this work, we identify and characterize a dual-memory system that is inherently present in both online and offline CL settings. This system comprises: a short-term memory, which temporarily buffers recent data for immediate model updates, and long-term memory, which maintains carefully curated subset of past experiences for future replay and consolidation. We propose memory capacily ratio (MCR), the ratio between short-term memory and long-term memory capacities, to characterize online and offline CL. Based on this framework, we systematically investigate how MCR. influences generalization, stability, and plasticity. Across diverse CL settings class-ineres intrmmnarr, inssiincremental, and memainninerrmmenall and multiple data modalitics (e.g., image and text classification), we observe that a smaller MCR, characteristic of online CL, can yield comparable even superior performance relative larger one, characteristic of offline CL, when both are evaluated under equivalent computational and data storage budgets. This advantage holds consistently across several state-of-the-art replay strategies, such as ER, DER, and SCR. Theoretical analysis further reveals that reduced MCR yields a better trade-off between stability and plasticily by lowering a bound on generalization error when learning from non-stationary data streams with limited memory. These findings offer new insights into the role of memory allocation in continual learning and underscore the underexplored potential of online CL approaches [T
1 Introduction
Deep neural networks are commonly trained by collecting large dataset upfront to create independent and identically distributed (IID) mini-batches that are used to optimize network parameters using stochastic gradient descent (SGD). When learning from non-IID data where new tasks appear over time, these networks often suffer from catastrophic forgetting, where new information overwrites what has been learned previously.
Code:: httpe//ssthaab.com/aaaaaaa//ii /lnng term-short-term

---

Published in Transactions on Machine Learning Research (12/2025)
Algorithms for continual learning (CL) aim to address this by enabling the acquisition of new skills and knowledge with minimum impact on what has been learned in the past. Two primary problem set settings are considered in this context: offline CL and online CL. Offline CL assumes that when a new task needs to be Iearned, a new training dataset becomes available from which IID mini-batcher can be drawn. In contrast, online CL operates on a continuously arriving stream of data. Instead of waiting for a complete dataset, the model begins updating as soon as mini-batches of new data arrive.
A major challenge in both online and offline CL is balancing the acquisition of new knowledge, referred to "plasticity', with the retention of previous knowledge, known "stability", especially when resources like memory and computation are limited. Replay-based methods, which maintain subset of past samples, often referred to as exemplars, to adjust gradients during optimization, have shown promise in mitigating forgetting. However, previous studies that applied these methods in online CL Mai et al. (202)); ;
ct al. (2023) often report lower performance compared to studies that applied the same methods in offline CL Masana et al. (2022); :: Buzzoga et al. (2020). This has led to the widespread impression that online continual learning is inherently more difficult. To better conceptualize this difficulty, recent work (Zhang) et al. 2022 Jung et al. 2022) has highlighted the problem of insufficient plasticity: without the ability to store the full data distribution of new tasks and train for multiple epochs, online methods are more prone to underfitting of the new task.
In this paper, we challenge this notion by showing that the performance limitations seen in online CL are primarily due to constrained memory and computing resources rather than the online setting itself. Perhaps more interestingly, we show that, depending on the replay mechanism used, online CL can achieve greater plasticity than offiine CL. For instance, we demonstrate that the online version of DER++ (Buzzega et al., 2020) exhibits greater plasticity than its offline counterpart. More specifically, we provide a controlled comparison between online and offline CL by accounting for disparities in compute and memory budgets and introduce the "memory capacity ratio" (MCR) to represent the relative sizes of short-and long-term memory. Here, short-term memory refers to the storage space available to store new data, and long-term memory represents the capacity available to store past data. We demonstrate that the fundamental distinction between online and offline CL lies in their MCRs: online CL operates with a smaller short-term memory equal to the size of incoming streaming batch size, while offline CL utilizes larger one equal to the size of the task data (see Algorithm 1).
Since the short-term memory contains new information that the model has not encountered, and long- term memory stores previously learned information that has been repeatedly trained, the capacity ratio between these two memory types is closely related to the stability and plasticity of contimual learning systems. We systematically investigate two fundamental open questions: " "How does the MCR affect stability, plasticity, and generalization in erperience replay?" and "How does the effect of the MCR interact with forgetting mitigation strategies (e.g., contrastive replay knowledge distillation regularization) and the chamcteristics of the learning problem (e.g., whether it is class incremental, domain incremental or task incremental)?? ?
We conduct thcorctical analysis based on bound on the generalization error in a simplified sctting with 0-1 classification loss showing that. 1) the effect of the MCR depends on lask similarity and problem structure; 2) a small MCR can lead to a lower bound under certain conditions (Section 4 Subsequently, we investigate whether this thcoretical result can be extended to practical CL algorithms with various complex forgetting mitigation strategies. We empirically study the effect of MCR by controlling the memory budget, training data, forgetting mitigation strategies, and training regimes. The generalization performance of continual learning is measured by the accuracy of a separate test set representing all the information to be learned. Across four state-of-tho the-art replay strategies, we observe that a smaller MCR leads to comparable or superior performance compared to a larger MCR value.
Contrary to conventional wisdom, we show that online CL, with a smaller MCR, does not necessarily compromise plasticity compared to offline approaches. Our analysis reveals that, depending on the replay mechanism and regularization design employed, reducing the MCR can lead to three types of changes in stability and plasticity: a) improved stability at the cost of reduced plasticity, b) enhanced plasticity with reduced stability, or c) sinultaneous improvements in both stability and plasticity (Section 6. Overall, our work provides new insights into memory allocation dynamics in continual learning and highlights the potential
2

---

Published in Transactions on Machine Learning Research (12/2025)
Algorithm 1: Unified continual replay framework instantiated with experience replay
// Non-stationary data stream: Dj U, where dt is the incoming batch
// Short-term memory Mshort: storing new data
// Loong-trmm memory M storing past data
// offline CL: MMshort is equal the task size U X4
// online CL: |Mshrrs is equal to the stream batch size [2]
// The memory management policy selects samples from Mahort for storage in Miong function Continuallearning(GH M.Mh,t,,MMtrt
Mshort MMssorrt U // Update short-term memory
3
if Mahart is full then
4
+ ModelTraining (,,aann Miong)
Mlong C+ Mshort U MMong // Update long-term memory
Mahort +
return 0, MsMort M
8
function ModelTraining (0, Mshort Mtong)
for epochs do
10
for short in Mahort do sample Blong from Miong
12
0 - 0 1/ LBshort U Blongi 0) // Using basic experience replay example
13 return
of online CL to achieve superior performance particularly in environments where memory and computational resources are constrained.
2 Related Work
Continual Learning. General continual learning (Delange et al., 2021 Buzzega et al. 2020) an idealized scheme for learning from infinite data stream, with desiderata like constant memory, online learning, task boundaries, no task labels, and graceful forgetting. Various relaxations exist with different assumptions. Early work focused on so-called inaeeminrremettal" settings (Mallya & Lazebnik, 2018 Serra et al. 2018) that assume access to task labels during training ttettingg//ettng.. Despite promising results, relying on a task oracle is impractical. Recent so-called "class-increni innreminccemmtta"" and alomain-ineremmntal"" learning approaches remove this assumption (Mirza al.) 2022 Masana al. 2022 Van de Ven & Tolias 2019). Nevertheless, these methods still require the knowledge of task boundaries allow multi-epoch training over tasks. In contrast, online continual learning (Chaudhry et al.] 2019 Aljundi et al.) 2019 Mai et al., 2022 does not collect specific task dataset but rather trains the model incrementally as mini batches of new data become available.
Memory and Compute. In the literature, online and offline continual learning are studied separately with different memory and computational setups (see Table in the Appendix B Studies and surveys of offline continual learning usually utilize 50-250 training epochs per task on standard benchmarks. Online continual learning, by design, can only use a single "epoch", but the number of gradient update steps can be increased through repeated iterations yielding multiple gradient-based updates for each incoming batch (Zhang et al., 2022
et al., 2023). However, it is common practice limit the number of iterations to fewer than 10, potentially putting online learning disadvantage. Considering storage costs, common practice in continual learning is to compare different online or offline methods based on fixed budget for storing exemplars, subset of past samples. This approach views memory in continual learning as a single system comprised solely of past exemplars. This perspective overlooks the higher-level memory architecture that governs the interaction between old and new data in dealing with a continuous data strcam. In contrast, we consider the actual amount of data that needs to be stored, comprising both short-term and long-term memory. By controlling the total storage required to hold data, referred to as memory, and the number of
3

---

Published in Transactions on Machine Learning Research (12/2025)
Infinite data stream
55
50
LTM STM
Accuracy (%)
LTM
STM
40
35-
ER
ICARL
30
DER++
Model
SCR
25
0.007
2.5 Amax
Amin
Continual learning systems (e.g. affective robots)
Memory Capacity Ratio
(a)
(b)
Figure 1: Long-term and short-term memory structure in continual replay: (a) The short-term memory (STM) greedily stores recent incoming data and the long-term memory (LTM) selectively stores a subset of data after model training. (b) The effect the memory capacity ratio (MCR=STM/LIM) reducing MCR significantly boosts the learning efficacy of continual learning algorithms including ER, iCARL, DER++, SCR on Split-Mini-mmageNet.
iterations yiclding gradients, we examine the effect of relative capacity of long-term and short-term memory on learning efficacy.
Theoretical Study in Continual Learning. There are several theoretical studies deriving bounds on
generalization performance for continual learning. (Peng al.) 2023 Doan al. 2021 provide theoretical
analysis with PAC-Bayes bounds. Meanwhile, Ye & Bors (022 derives generalization bound for the 0 loss using discrepancy distance and Rademacher complexity. Although closely related to our work, their analysis considers only the effect of replayed exemplars and does not account for the contribution of incoming data that is simultancously used for model updates. In contrast, we address this limitation by explicitly modeling the influence samples stored long-term memory as well as those held in short-term memory our generalization bound.
3 Rethinking CL Memory: Beyond a Monolithic Store of Past Data
3.1
Problem Setup
We consider the general contimual learning problem proposed in Delange et al. (2021) which deals with infinite non-stationary data streams, and approach this problem with unified continual replay framework.
Non-stationary Data Stream. At each time step 6, a continual learning algorithm A receives an incoming batch of data samples of size B, X; (x,, H drawn from the current data distribution P,, Each batch forms part of non-stationary (potentially infinite) stream of data D, = U, The distribution P, may change at any time. A basic approach to continual learning from such a stream of data would be to atlempt to minimize the empirical risk on all the data seen so far:
min R2( = min (),
(1)
with a. loss function L, a CL network function f > y, and its associated parameters
Task Boundaries. The period where the data distribution P. stays the same often called a task. Task boundaries are defined as the time steps where changes the data distribution occur, i.c., (T.} = [t,P # F.-1]. Given the task boundaries of data stremm, each task data can be denoted C, Un,ste,, 24 and U,C, Dz.
4

---

Published in Transactions on Machine Learning Research (12/2025)
3.2 Unified Replay Framework with Dual- Mual-Memory Structure
Prior work in continual learning typically treats memory as a monolithic buffer of past data and focuses primarily strategies for managing stored exemplars. In this paper, argue that there is higher-level memory structure governing not only how new and historical data interact, but also how buffered data systems interface with a continuous data stream. This memory structure involves storage policies for both recent and past samples. We show that such a memory structure arises naturally in existing online and offline continual learning sctups.
Concretely, we model this higher-level structure as a dual-memory system consisting of short-term memory and long-term memory, as illustrated in Figure [ We formalize the replay process under this dual-memory view through Unified Continual Replay (UCR) framework.
Unified Continual Replay Framework UCR(M,, M). Given total data storage budget M (M < (Pa]) and a stream of data D, = UpAr with batch size B, the storage space is divided into two parts: short-term memory of size Mahorrs (M, 2 B) and a long-term memory of size M - Mahort- The short-term memory M'soort greedily stores recent batches from the data stream until it is full: Ma = U. Maturi A model training session triggered when Mishort is full. After the model training session, M Mistort emptied, with some of the short-term memory samples moved into Miong based on some sample selection policy m
+++ Ca Mathrr UMinngg Mion
This framework agnostic to the specific replay strategy. Algorithm provides an example instantiation using experience replay with K training epochs. Algorithm 2 in the appendix gives an example instantiation using DER++
Crucially, online and offiine CL can be considered as two extreme cases of this framework. Offline CL stores the whole task data in short-term memory based on the knowledge of task boundaries: Mshort |UTSeTTT, & = C. Online CL only stores the current batch in the short-term memory: Mahort | = B.
Memory Capacity Ratio. We define the Memory Capacity Ratio as Mahort/ Mahort). Offline CL yields a large value of MCR, i.c., fliee >maz C/(M - C), where C. is the task size, while online CL yields a small MCR: Aonline Amin B/ B), where B the streaming batch size.
Semi-offline CL. Apart from the two extreme cases, representing offiine and online continual learning respectively, the unified framework enables investigation of novel scmi-offline setting, where the short-term memory stores more than one streaming batch, and B < Msmall < C. We investigate what the optimal value for X is in [Amin; Xmar]-
4 Theoretical Study on Generalization Bound with 0-1 Loss
To theoretically understand the effect of MCR on continual learning efficacy, consider bound on generalization error for 0-1 loss. To analyze generalization in a non-lID setting, we follow work on domain adaptation Mansour et al. 2009) with 0-1 loss and derive bound based on the concept of discrepancy distance.
Definition 1 (Discrepancy Distance). Mansour et al. (2009). Let be a set of functions mapping X to Y and let L Y x Y + R+ define loss function over Y. The discrepancy distance between two distributions Qi and Q2 over X is defined by
disc (Q1,Q2) = max (Qq, h)',h) LQz (h', (h', h,
where the expected loss of two functions over a distribution is denoted as Lo(s = Ex-e(((((())
We consider the stored data (i.e., data stored in the long-term and short-term memory) as the source domain, and the test data of the data stream as the target domain. Let ID, and M, denote the true probability distributions of the data stream and the stored samples at time step respectively. Let M, denote the empirical distribution of stored samples with finite sample size of M. The true labeling function of all
5

---

Published in Transactions on Machine Learning Research (12/2025)
the data seen far is defined h Given the optimal solutions hig = argmin LM, (h, and hb = er LD (h, h2y,, the generalization bound is presented in Theorem
Theorem 1. Let H be hypothesis set bounded by some Au > 0 for the loss function L L(h, h') S Ag. For all h. h' E H, assume that the loss function I. is symmetric and obeys the triangle inequality. Then, for any h E H and any 8 > O, with probability at least 1 - 8, the following generalization bound holds:
log 2 2M
LT, Cn,(h, (b)) S CM, (h, his, RM. + +Ao
(D, + Cr, (hin, kG. + Lr, (hi, (2)
where RM(H) is the empirical Rademacher complexity of the hypothesis set over the stored samples M, = -M UMMnngg'
The proof follows derivations of Theorem 8 in Mansour et al. (2009) and Theorem in Ye & Bors 2022 and is presented in Appendix A Given the high expressivity of deep networks, La, (h3,, hG)) approaches zero
The memory management policy governs the ionshio between the memory distribution (M,) and the data stream (ID)) While Theorem 1 holds regardless of their similarity, the alignment between the two affects the tightness of the bound through the term LT, : on the right hand side of Theorem 1. Since ID, and M, both cover the new and past task domains, we assume Lp, (h5g, AB,) to be reasonably small and thus the bound is primarily driven by discy(D), M,).
Remarks. Theorem highlights that a key factor influencing generalization performance is the discrepancy disc (Ds,, MI,, which measures the distribution difference between the stored data and the data stream. While much research focuses constructing representative exemplars (the long-term memory). the role of the
incoming buffer (short-term memory) is often overlooked (Peng et al. 2023 Ye & Bors 2022). Theorem reveals that it is the joint distribut ion of the long-term and short-term memories, rather than the long-term memory alone, that determines the generalization bound.
We analyze the discrepancy distance disc (D,, at task boundaries in Proposition 1.
Proposition 1. Assume Dj denotes the probability distribution of the most recent task C, and DD, denotes the probability distribution of all past tasks seen so far UI,..,i-IC. Given the number of samples seen in the data stream N₂ = E, |2g] and the number of samples seen in the previous tasks N, = -b=i k=i-1 [C&], we have:
No A(N, - M)
disc (ID,, ML, N, AN, - AM) disc (ID,,
(3)
The proof is based on the property of reservoir sampling Vitter (1885) and shown in Appendix A.2 Proposition 1 reveals the effect of X, problem structure, and task similarity, on the discrepancy distance.
Based on Eq 9 we quantitatively compute the effect of MCR as:
Corollary 1.
No (N₂ - M) (N, - XN, XM) discy (DD, DT)
V disci
(4)
Equation reveals several insights:
1. The effect of the MCR A: When N₂ > M and (P, ,P,) a 0, we have Vx disc > 0. This suggests a smaller A leads to a lower discrepancy distance and a lower generalization bound.
2. Bounded vs. unbounded memory: In the case of N; = M, i.e, an unbounded memory system for learning data strcams, we have VA disc In this case, the choice of MCR does not affect the gencralization bound.
"Throughout this work, we assume the true labeling function ht is deterministic Ben-David & Urner 2014 i.e. y= (z(x))
uniquely maps each imput to single label y. This excludes cases with label noise (Frenay & Verleysen 2013). inherent ambiguity (e.g, subjective annotatious), stochastic data-gencrating processss. Extensions to probabilistic labeling functions P(ylz) left for future work.
3 Althhugh this analysis focused classification tasks, similar derivation he performed regression tasks with MSE
loss.

---

Published in Transactions on Machine Learning Research (12/2025)
A related result is 0 disci, > 0 and aNaA
8M9A < 0. This suggests the effect of MCR. becomes more evident with longer data stream N₂ and more limited memory budget M.
8² disey.
3. Task similarity: The effect of MCR also depends on task similarity Pt and Pt, and the loss function L. With smaller ( PT), the effect of MCR diminishes.
5 Empirical Study on Forgetting Mitigation Strategies and Training Dynamics
While our previous analysis of MCR's effect on generalization bounds reveals important theoretical insights about the relationships between MCR, task similarity, and problem structure, several key limitations remain unaddressed.
Forgetting Mitigation Losses The generalization bound analysis assumes symmetric loss functions. However, modern continual learning methods typically employ complex asymmetric loss functions (c.g., cross-entropy and contrastive losses) with some auxiliary loss designs, like regularization loss terms based knowledge distillation. It is unclear how these forgetting mitigation strategies interplay with the long-term and short-term allocation mechanisms. Intuitively, the incorporation of forgetting mitigation loss may reduce the necessary size of long-term memory.
Training Dynamics. In addition, the theoretical framework of generalization bounds primarily examines relationships between model complexity, sample size, and generalization error, without explicitly considering training dynamics, such as the effects of SGD optimization and model initialization. This limitation is especially relevant to continual learning, where model initialization plays fundamental role: at the start of each new task, the model has already been optimized for previous tasks. This creates an initial state where the model performs well on data stored in long-ter term memory but poorly on new data in short-term memory. This performance disparity at initialization may also reduce the necessary size of long-term memory and thus influence the effect of the MCR.
To address these limitations, we investigate the effect of the MCR empirically, by considering practical CL training dynamics and forgetting mitigation strategies. We investigate whether the previous theoretical results hold with complex loss functions and training dynamics.
Control Variables. To isolate the effect of the MCR, we consider the following control factors in our empirical study:
Stored Data Size (M): We compare the effect of the MCR under fixed total memory budget M. This ensures the training data size used in each model update session is consistent and eliminates potential confounding factors related to the training data size.
Seen Data Size (N): We assume that all samples of the incoming batch will be fitted in the short-term memory, ie., Mshort 2 B. This guarantees that all samples in the data stream will be used by the model some stage, regardless of MCR value.
Compute Resources (K): We control the number of iterations to be the same when comparing the effect of the MCR. This ensures a fair comparison of the computational resources utilized across different experimental conditions.
Algorithms and Hyperparameters: We compare the effect of the MCR under the exact same CL algorithms, using the same data augmentation techniques and hyperparameter settings. This allows for an isolated evaluation of the impact of the MCR.
5.1 Experiment Setup
Image Classification. The main experiments involve three standard CL benchmarks based on image classificatio:: Split-CIFAR100 with 20 tasks, Split-Mini-mmagcNe Vinyals et al. (2016) with 10 tasks, and CORE50 Lomonaco & Maltoni (2017) with 9 tasks. Table D lists the image size, the number of classes, the number of tasks, and data size per task for the three CL benchmarks. The task size in these three benchmarks
7

---

Published in Transactions on Machine Learning Research (12/2025)
60
Amax Amio
Amaz
80
Amin
Accuracy (%)
Accuracy (%)
50
60
40
35
25
2.5
12.5
25.0
37.5
50.0
3.5 4.5
7.5
12.5
Data stream size (10?)
Memory Size (103)
(a) Data stream size N
(b) Exemplar budget M
Figure 2: Memory Capacily Ratio and problem structure: the effect of MCR (A) on continual learning efficacy becomes more evident with longer data stream sequence N, and smaller exemplar budget. M. This experiment employs ER in CIEAR100 with 20 tasks.
is 2500, 5000, and 12000 respectivcly. We consider the data stream batch size to be 50. We use ResNet-18 for all experiments. We apply standard data augmentation (random cropping and flipping) in most experiments. For SCR and CORE50, we additionally use color jittering and random grayscale conversion. Hyperparameter details can be found the Appendix D
Table 1: Dataset statistics for continual image classification.
IMAGE SIze
# #'TAS
# CLASS
TRAIN PER TASK
TEST PER TASK
3x32x32
20
100
2,500
250
SPLIT- MLLT-MINJ-L IMAGENET
3x84x84
10
100
5,000
1,000
SPuT-CORE50-NC
3x128x128
9
50
12,000
4,500
Text Classification. We also consider continual text classification with three text classification datasets, including AG News (news classification), Yelp (sentiment analysis), and Yahoo! Answer (Q&A classification).
A summary of the text. datasets is shown in Table 2
Following (de Masson D Auttum al. 2019 Huang et al. 2021 we utilize the pretrained BERT-base- baneu--as--uucaeed model from the HuggingFace Transformers Wolf et al. (2020) library the base fcature extractor. The experiments are conducted with batch size of 8. The learning rate is set to 3e-5 and the weight decay for all parameters is set to 0.01.
Evaluation Metric. The performance of CL is measured by the final accuracy after training on all lasks, defined as Ar = + Ej=r arj, where a, denotes the model's accuracy on the held-out test set of task j after training on task i. We conduct all experiments across three random sceds to account for potential variability factors, including task splitting in continual learning benchmarks, random sample selection in reservoir sampling, random model initialization and stochastic gradient descent. The plots presented with 95% confidence intervals to provide measure of statistical significance. Furthermore, the tables report the mean values along with the corresponding standard deviations.
Table 2: Dataset statistics for continual text classification.
Order
Dataset
Type
# #Caass
Train
Test
AGNews
News
8000
7600
2
Yelp
Sentiment
5
8000
7600
Yahoo
Q&A
10
20000
7600
8

---

Published in Transactions on Machine Learning Research (12/2025)
Table 3:
The effect of memory capacity ratio in different benchmarks and algorithms. Given memory budget M, and data stream with task size C' and batch size B, small MCR Amin B/(M B) values lead to significant performance advantage over large MCR values Amaz C/(M - C).
MCR
ER
ICARL
DER ++
SCR
S S-CIFAR100-20
Amaz
35.1 + 0.8
44.8 + 1.1
47.2 + 0.4
45.1 + 0.4
Amin
44.6 1 2.4
50.0 + 1.6
50.9 1 0.9
51.9 1 0.5
9.5
5.2 f
3.71
6.8 f
S-MIN-IMAGGNETTII
Amaax
31.2 + 1.1
42.5
41.0 2.7
46.3 0.4
Amiin
43.7 + 1.2
46.8 + 1.5
46.4 1.3
51.8 + 0.7
12.5
3.3
5.4
5.51
S-CORE-9
Amaax
40.1 + 2.4
45.8 + 1.6
38.2 + 1.5
62.1
1.3
Amnn
50.7 + 1.7
50.1 + 1.6
46.7 + 2.6
69.7 = 0.3
10.6
4.3 f
8.5 f
7.6
Replay Strategies. We consider three types of replay-basod approaches: 1) Direct rehearsal. Our main experiment focused experience replay Chaudhry al. (2019), which simple approach that achieves competitive performance especially in large-scale settings Prabhu et al. 22023) ER incorporates past exemplars directly in the training via cross-entropy loss. 2) Knowledge distillation. Many replay-based methods leverage knowledge distillation to construct a regularization loss, using past model as the teacher and the current model as the student (Li & Hoiem 2017 Rebuffi et al. 2017 Buzzega et al. 2020; Hinton et al. 2015). A classic method is iCaRL which maintains a past model and computes distillation loss based on the past network's outputs related to old classes. Instead of computing logits based on a past model, the distillation loss of DER++ uses the network's logits sampled throughout the optimization trajectory, and the distillation loss is computed over past exemplars. 3) Contrastive replay. Some recent works (Cha et al. 2021 Mai et al. 2021 Khosla et al.] 2020) investigate the use of self-supervised learning techniques to learn strong representation to reduce forgetting. A representative method is SCR Mai et al. (2021), which replaces cross-entropy with contrastive loss to capture more information about exemplars and achieves stato-of-the-art performance.
Memory and Compute. To make the experiment results comparable existing work, we follow the standard practice in the offline continual learning literature. We set the total memory budget be task size plus 2000. More specifically, the total memory budget is 4500, 7000 and 14000 for CIFAR100, Mini-Imagent and CORE50 respectively. With this setup, the results obtained with Amaz are aligned with the existing offline continual learning results that employ a 2000 exemplar budget. Regarding the computational resources, we employ 50 training epochs (K = 50) as the default setting.
5.2 Main Findings
Strategies. Different replay techniques make use of exemplars to preserve past knowledge in different, ways. Intuitively, replay method with very strong forgetting mitigation design may not need a large number of exemplars from previous tasks to perform well. Thus, one interesting question is how the effect of the MCR changes with different forgetting designs. Fig L presents the results of different CL strategies on Split-Wini-mmageNet. Interestingly, our results show that a small MCR seems to lead to better performance across different algorithms including ER, iCaRL, DER++, and SCR. The performance boost in direct rehearsal ER seems larger than in other replay methods. Table presents detailed results comparing the results of the smallest MCR Amin and the largest Amaz three standard CL benchmarks. Amin leads to consistent performance improvement over Xmin across three datasets (ER: 10 12%, iCaRL: 4 6%, + 3- - 8%, SCR: 5 5-7%% -%)..
Continual Learning Problem Structure. We further investigate whether the effect of MCR is influenced by different problem structures along two dimensions: 1) Data stream size Fig a) suggests that with longer task sequences the advantage of small MCR becomes more obvious. 2) Memory budget Fig 2 b) presents results of different sizes of memory budget, including 3500, 4500, 7500, and 12500. Given the limited memory
9

---

Published in Transactions on Machine Learning Research (12/2025)
Table 4: The effect of memory capacity ratio X in task-incremental, iomain-incremental and pretrained class- inssemncremettll scttings.
CL Settings
Task Incremental
Domain-Increnen Incremental
Pretrained ResNet
Pretrained Transformer
Dataset
CIFARI00
CLRS
Mini-ImageNer
Text classification
Amaz
83.4 1: 1.3
34.2 2.6
36.6 0.9
65.8 1 0.5
Amin
83.1 + 2.3
36.8 + 1.3
48.3 1.1
71.9 + 0.3
Table 5: The effect of MCR on different sample selection methods, including MIR and ASER.
Mini-ImageNet
CIFAR-100
Mothod
MCR
M short
Accuracy
Mahort
Accuracy
MIR
Amin
50
43.4 + 0.3
50
43.4 3.1
Amid
2500
40.6 0.5
1250
41.4 1.4
Amaz
5000
30.8 + 0.6
2500
33.4 2.8
ASER
Amin
50
40.6 - 0.8
50
1.7
Amid
2500
38.2 + 0.6
1250
41.0 2.2
Amaz
5000
31.1 1.0
2500
33.8 + 1.2
budget, the advantage of a small MCR is more evident. When the budget is larger than 12500 (around 5 tasks), the performance replay remains the same regardless of the choice of MCR.
Different Continual Learning Settings. So far, our experiments focused on the class-incremental setting and training the model from scratch. We now investigate this phenomenon in other settings. The effect of the MCR À in iask-incremental (TI), domain-inerements. (DI) and pretrained class-incremental scttings is shown in Table 4 When init initializing the model with a pre-trained ResNet18, we observe similar trend: ER with a small MCR (48.3 + 1.1%) significantly outperforms ER with large MCR values (36.6 + 0.9%). Additionally to what is observed with ResNet and image data, we also observe similar results in text classification experiments with pre-trained Transformer models (Amin 71.9 + 0.3% and Amaz 65.8 + 0.3%). However, in task-incremental learning, we observe the effect of the MCR ratio not significant (Amin 83.1 + 2.3% and Amaz 83.4 + 1.3%).
Memory Management Policies. Table 5 presents results using different memory update and retrieval strategies, including MIR (Aljundi et al. 2019) and ASER (Shim et al. 2021). Unlike random selection, MIR retrieves samples that are most interfered i.e., those whose predictions would be most negatively affected by upcoming parameter updates. ASER, on the other hand, selects samples based on the Adversarial Shapley value, which scores memory samples by their contribution to preserving latent decision boundaries of previously observed classes. Consistent with our main findings, Table 5 shows that reducing MCR also improves performance when using advanced sample selection methods such as MIR and ASER.
In summary, empirical study reveals that the relative capacity of long-term and short-term memory exhibits complex interplay with forgetting- miitigtnnni strategics and problem structure. For certain problems, such as lask-ineremental learning or scenarios with large memory budget, the choice of MCR does not significantly impact performance. However, in other settings like incremenatt learning with a limited memory budget, the performance of replay methods can be significantly improved by adopting smaller MCR values. To understand how MCR influences the learning efficacy of continual learning systems, we examine the stability and plasticity dynamics next.
10

---

Published in Transactions on Machine Learning Research (12/2025)
0.725
6,825
ER
0.100
ICARL
0.75
DER++
0.72
SCR
Amak
0.70
9.70
Amar Amar
-065
Plasticity
6.750
6.775
B.10
B.675
Amax
0.65
min
0.66
Amin
0.80
0.600
0.60
Ub4
Artin
0.40
0.700
min
Amin
0.59
0.62
0.60
0.35
0.30
6.656
0.58
-05
-0.4
-0.3
-0.25
-0.2
-0:15
-0.10
-0.30
-0.25
-0.20
-0.55
-0.10
-0.5
-0.25
-0.15
-0.10
Stability
Stability
Stability
Stability
Figure 3: Stability, plasticity, and generalization across two runs with different initializations and lask splits. Lower MCR yields diverse stability-plasticil patterns but consistently improves the overall trade-off.
6 Stability and Plasticity
Mcasuring Stability and Plasticity. Beyond the evaluation of the overall performance as discussed in Sec- tion 5.1 we also examine the stability and plasticity during the learning process. Plasticity refers to the ability to learn the new tasks and is usually measured as the accuracy on the new task. Stability refers to the ability to maintain previous knowledge. There are two measures proposed in the literature related to stability: one metric "forgetting" (Chaudhry et al. 2018 which is defined as F1 E (ar, max a1,))
T-1
and the related metric "backward transfer" (Lopez-Paz & Ranzato 2017 B BT= - E dT.; 04.z- Since the continual learning systems require tradeoff of stability and plasticity, Zhang al. (2022) decompose
the overall CL performance into a stability-based term and a plasticity- term, as follows:
T-1
1 TT
T- 1
Ar =
T 1 04,4
Br
a4s
Fr.
T Stability
T
T
Plasticity
Insights into Stability-lasttictyy Dynamics. Using the definitions of stability, plasticity, and general- ization from Zhang et al. (2022), we plot the effect of the MCR on these metrics in Fig Since a lower MCR allocates more space for storing past exemplars and less space for new samples from the current task, an intuitive expectation would be that reducing the MCR leads to better stability and worse plasticity. Surprisingly, this behavior is observed only in some algorithms (ER, iCaRL, and SCR). For other algorithms (e.g., DER++), reducing MCR leads to various types of stability-plasticit behaviors, including better plasticity and better stability at some stages, and better plasticity and worse stability at other stages, shown in Fig. 3 These behaviors in DER++ suggest that increasing the ratio of long-term memory can aid in the learning of new tasks at certain stages.
To understand why increasing long-term memory can contribute plasticity in some cases, we consider the distribution composition of long-term buffers. The long-term memory contains subset of past data, and its coverage is determined by where the data stream is split. As illustrated in Fig ! the data stream is split into two parts based on the size of the short-term memory (Mahort): one part contains the recent M samples, which are greedily stored in the short-term memory, and the other part contains all the samples before Mshort, which are selectively captured by the long-term memory. With reservoir sampling, the long-term memory stores a fraction Pnew of new task exemplars and 1- --neew of old task exemplars: Pnew = N Mater is the data stream size and C' is the task size. When Mahort < (i.e., in online and semi-ollline CL), we have Pnew > O, meaning the long-term memory contributes to learning the current task, promoting plasticity. We take DER as an example to illustrate how the MCR may lead to complex stability-llasticty changes. The analysis of other algorithms follows a similar manner and can be found in Appendix C We rewrite the gradient of DER follows:
C-Matort, where
11

---

Published in Transactions on Machine Learning Research (12/2025)
VLDER = E.yhhrt [oo] + Miting (V
msn]
(5)
= Mesors [Ce)] + Pnew x Expan
V pld
+ (1 Pnew) x E
phd maee
tong
As shown in Eq when the ratio of short-term memory is reduced, the first term of the right- hand side (RHS) suffers from greater overfitting of new task data, which harms plasticity. However, the second term of the RHS drives plasticity, and its contribution increases with smaller Mşhort because V MahorePnew < O. Therefore, the overall plasticity is determined by the interplay between the first and second terms. DER employs different loss functions for the first term (cross-entropy) and second term (logits- based knowledge distillation), thus creating the complex plasticity dynamics in Fig g
7 Discussion and Conclusion
A key challenge in continual learming is optimizing memory usage to enhance learning efficiency. While many prior works have investigated different strategies for selecting and storing representative past data, the higher-level memory architecture governing the interaction between old and new data remains understudied. In this work, we identify and formalize a unified dual-memory system that naturally arises in both online and offline CL settings. This system comprises: (1) short-term memory, which temporarily buffers recent data for immediate model updates, and (2) long-term memory, which maintains carefully curated subset of past experiences for future replay and consolidation. We investigate how the memory capacity ratio (MCR), defined as the relative allocation of resources between STM and LTM, affects learning efficacy in non-sta streams. Empirically, we find that reducing the MCR induces distinct stability-plastiiit trade-offs depending on the replay mechanism: some replay configurations improve both stability and plasticity, others enhance one at the expense of the other, yet a smaller MCR consistently matches or outperforms larger MCRs generalization. Theoretical analysis of generalization bounds in a simplified setting further supports this result.
We believe the impact of MCR has likely remained unexplored in prior work because 1) CL. memory is often treated as a monolithic store of exemplars, and 2) online/offline CL are typically studied in isolation with divergent setups. By re-examining continual replay through the lens of short- and long-term memory, our work offers a more comprehensive understanding of replay dynamics in non-stationary environments. Our findings suggest that designing a memory structure with a small MCR can be advantagcous building practical continual learning systems, such as those used in affective robotics (Churamani et al. 2020 smart home assistants, or wearable devices.
Limitations. This work examines rehearsal and knowledge distillation techniques for continual learning, Other approaches, such as correcting task recency bias (Wu et al. 2019 Hou et al. 2019) or expanding network capacity (Zhou et al. 2022 Yan et al. 2021), are not covered by our analysis. Additionally, we use a fixed MCR throughout the continual learning process. Since MCR directly influences the trade-off between stability and plasticity, dynamically adjusting MCR achieve a better balance between the two represents a promising direction for future research. Moreover, with time-varying MCR, an interesting avenue for exploration is designing adaptive memory management strategy that aligns with the changing memory size while maintaining representativeness across different tasks.
8 Acknowledgment
We thank the anonymous referees and the action editor for their constructive feedback. This research was supported by the New Zealand MBIE. TAIAO Data Science Programme.
References
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. Advances in Neural Information Processing Systems, 32:1 1411889-11 18499-18600, 11860, 2019.
12

---

Published in Transactions on Machine Learning Research (12/2025)
Shai Ben-David and Ruth Urner. The sample complexity of agnostic learning under deterministic labels. In Conference on Learning Theory, pp. 527-542. PMLRR 2014.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple bascline. Advances in neural information processing systems, 33:15920 15315920-15930, 2020.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on roducing abrupt representation change in online continual learning. In International Conference on Learning Representations, 2021.
Francisco M Castro, Manuel Marin-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End- to-end incremental learning. In Proceedings of the European conference on computer vision (ECCV), pp. 233 248, 2018.
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co2L: Contrastive continual learning. In Proceedings of the IREE/CVF International Conference Computer Vision, pp. 9516-9525, 2021.
Arslan Chaudhry, Marc' Aureloo Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. In International Conference Learning Represenlations, 2018.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K. Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. ar preprinl arXiv:1902.104 2019.
Aristotelis Chrysakis and Frarn-Franccne Moens. Online bias correction for task-free continual learning. ICLR 2023 at OpenReview, 2023.
Nikhil Churamani, Sinan Kalkan, and Hatice Gunes, Continual learning for affective robotics: Why, what and how? In 2020 29th IEEE international conference on robot and human interactive communication (RO-MAN), pp. 425 431. IEEE, 2020.
Cyprien de Masson D' D'Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. Episodic memory in lifelong language learning. Advances in Neural Information Processing Systems, 32, 2019.
Matthias Delange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions Pattern Analysis and Machine Intelligence, 2021.
Thang Doan, Mehdi Abbana Bennani, Bogdan Mazoure, Guillaume Rabusseau, and Pierre Alquier. A theoretical analysis of catastrophic forgetting through the ntk overlap matrix. In International Conference Artificial Intelligence and Statislics, pp. 1072-1080 PMLR, 2021.
Benoît Frénay and Michel Verleysen. Classification in the presence of label noise: survey. IEEE transactions on neural networks and learning systems, ((5))884--69, 2013.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.0955 02531, 2015.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 831-839, 2019.
Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, and Diyi Yang. Continual learning for text classification with information disentanglement based regularization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Longuage Technologies, pp. 2736 2746, 2021.
13

---

Published in Transactions on Machine Learning Research (12/2025)
Dahuin Jung, Dongjin Lee, Sunwon Hong, Hyemi Jang, Ho Bae, and Sungroh Yoon. New insights for the stability-plasticity dilemma in online continual learning. In The Fleventh International Conference Learning Representations, 2022.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 10118661-1667) 2020.
Z Li and D Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935 2947, 2017.
Vincenzo Lomonaco and Davide Maltoni. CORe50: a new dataset and benchmark for continuous object. recognition. In Conference on Robot Learning, pp. 17-26. PMLR, 2017.
David Lopez-Pas Paz and Marc' Aurclio Ranzato. Gradient episodic memory for continual lcarning. Advances in neural information processing systems, 30, 2017.
Zheda Mai, Ruiwen Li, Hyunwoo Kim, and Scott Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online inaseincremental continual learning. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pottern Recognition, pp. 3589 3599, 2021.
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 469:28 51, 2022.
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 7763-7773, 2018.
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and csw.cc..oo mogiml.
algorithms. In The 22nd Conference on Learning Theory, 2009. URL WTEeol120009/pppers//00 pdf
Mare Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost Van De Weijer. Class- incremental learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5513 45333-553,, 2022.
Jehanzeb Mirza, Marc Masana, Horst Possegger, and Horst Bischof. An efficient domsin--nncremental learning approach drive in all weather conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3001-3011 2022.
Liangzu Peng, Paris Giampouras, and René Vidal. The ideal continual learner: An agent that never forgets. In International Conference on Machine Learning, pp. 97585-27610. PMLR, 2023.
Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet K Dokania, Philip HS Torr, Ser-Nam Lim, Bernard Ghanem, and Adel Bibi. Computationally budgeted continual learning: What does matter? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3698-3707, 2023.
Sylvestre-Alvise Rebufii, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. iCaRL: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Joan Scrra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference on machine learning, pp. 4548 4557. PMLR, 2018.
Dongsub Shim, Zhoda Mai, Jihwan Jeong, Scott Sanner, Hyunwoo Kim, and Jongscong Jang. Online class-incremental continual learning with adversarial shapley value. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 9630-9638, 9638, 2021.
14

---

Published in Transactions on Machine Learning Research (12/2025)
Albin
Antonio Carta, Andrea Cossu, Julio Hurtado, Vincenzo Lomonaco, Joost Van de Weijer, and Hamed Hemati. A comprehensive empirical evaluation on online continual learning. In
Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3518 3528, 2023.
Gido M Van de Ven and Andreas Tolias. Three scenarios for continual learning. NewrIPS Continual Learning workshop, 2019.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, and Daan Wierstra. Matching networks for one shot learning. Advances in newral information processing systems, 29:3630-3633 2016.
JefTrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathemalical Sofbware (TOMS), 11(1):37-55) 57, 1985.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: Stale-ofthhe-ar natural language processing. In Qun Liu and David Schlangen (eds.), Proceedings of the 2020 Conference Empirical Methods in Natural Longuage Processing: System Demonstrations, pp. 38-45, Online, October 2020. Association for Computational Linguisties. doi: 10. 10.18653/v //2//00//2220..///// http:/20....mmmddmmeeeee oemop-demos.6. URL https://acllanhoologg) org/2020. munlp-demss..
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 382, 2019.
Shipong Yan, Jiangwei Xie, and Xuming Hc. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1014-30233 2021.
Fei Ye and Adrian Bors. Task-free continual learning via online discrepancy distance learning. Advances in Neural Information Processing Systems, 35:23675 23688, 2022.
Yaqian Zhang, Bernhard Pfahringer, Eibe Frank, Albert Bifet, Nick Jin Scan Lim, and Yunzhe Jia. A simple but strong baseline for online continual learning: Repeated augmented rehearsal. Advances in Newral Information Processing Systems, 35:14771-14783, 2022.
Da-Wei Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. model 603 exemplars: Towards memory- efficient dass-incrementar learning. In The Eleventh International Conference on Learning Representations, 2022.
A Proofs
A.1 Proof of Theorem 1
Proof. We follow the derivations of Theorem 8 in Mansour et al. (2009) and Theorem 1 in Ye & Bors 2022
Lo(h, hyy s LD hia) + Lo (hir h() + Co Co hy)
< Lm (h, 1(g)) + discy, (13, M) + Lr (hin, 15) + Co hwy
(6)
log 2
S Cox (h, hMd) + (H) + 3.A0
+ discy (D, + Lp hb) + Co Co hy),
2M
The first inequality is based on the triangle inequality of L. The second inequality is based on the definition of discrepancy distance diser, The third inequality based on the Rademacher Bound (Proposition 2 Mansour et al. (2009)).
In the proof, drop from the symbols conciseness.
15

---

Published in Transactions on Machine Learning Research (12/2025)
A.2 Proof of Proposition 1
Proof. Let = N= and a Marort Based on the definition of discrepancy distance, we have: M
N
disc,, (D,M)
1nCp (1/, + (1-))) ())
max N,V'H
(7)
((L a) Lxtions (h', h) + aCMisror (N,, ).
Since all the samples from the short-term memory come from the current task, Litenors (M' h) only depends the current task distribution and is not affected by the size of short-term and long-term memory, i.e. we have Lmanore (N' hi) = C++ (h', h). In contrast, the long-term memory is managed by the reservoir sampling method and thus Ledinng (N' h) is affected by the memory allocation factor X. Letting B = N-a37 we have
Liicnng (h', h) BCp (h', h) + (1 Be (N', h). Inserting these two results into Eq 7 gives:
disc (TB, M)
max 1(7 (1 a)) (Cp- (W',h) L (h', h)) A.EEH
= (Y (1-
max | (Cr- (h', h)) Lx+ (N'
(8)
h.N'H
N W-N(N- - M)
N(N ANN - XM) disc (TP (P_,P+).
This second equality is based on the fact that 7 - (1 -a3 (N- N(N-aM)
> when N> > M. The third
equality is based on the definitions of a, 7, B, X, and disci
A.3 Proof of Corollary 1
N, X(N, M)
V disci MI) = -
disc (ID, ,D;))
(N,. + AN, - AM)
= discr (D7,It) NT M)
A
V
N₂
N.
-AN; AM
No (N: - M)
1
= disscc(D,,113
Vx Ni + N₁ M
(9)
No
N₂ (N; - M)
N
disce .D,,DI))
No
( N₂ M)² X2
N, (N₂ - M)
discr, (D7, mt)
(Ne + XN, AMYZ
B A Review of Memory and Compute Setups in Continual Learning
Online and offline CL are typically studied separately with different memory and compute setups (see Table B For the compute cost, offline CL papers and surveys typically utilize 50-250 training epochs on standard benchmarks, while online CL by design steps in online CL be increased by using repeated iteration, conducting multiple gradient updates for each incoming batch (Zhang et al. 2022 Soutif-Cormerais et al. 2023), the number of itcrations is commonly chosen to be less than 10. For the storage cost, common practice in CL is comparing different online or offline CL methods based on a fixed budget of exemplars- a subset of past samples. Less discussed is the sample complexity and storage cost introduced by the new task data, which also occupies the constrained space and is used for the training. At each training session, offline CL needs to store the full task datasets for model training, whereas online CI. only stores the most recent incoming batches of the new task.
only use single epoch. Moreover, while the gradient update
16

---

Published in Transactions on Machine Learning Research (12/2025)
Table The compute and data storage cost of online and offline continual learning on the same CL benchmark Split-CFFRR100.
Data Storage (# samples)
Compute
Papers
Exemplar
New data
Iteration Keration/Epochi Epooch
F.R Chaudhry et al.
(2019)
ER-ACE Caccia et a.
CIR-Cia
[2021]
1
SCR Mai
2021
online CL
ER-OBC
Chryssaii & Moens
2023
2000,
5000
10
1
RAR Zhhan al.
2022
Survey
(Mai et al.
2022
1,5
Survey
(Soutif-Cormeee Cormerais al.
2023
3
ICARL Rebulfi et al. (0117
70
EEI..
Castro al.
2018
70
LUCIR
Hou et al. 2019
160
offline CL
DER++
Buzzega et 2020
2000,5000
2500,5000
50
MEMO
Zhou et al. 2022
170
BIC
Wu et
2019
250
Survey
Masana et al.
2022
100
online, semi-olline,, offline CL
ours
1+x
M
I+X
M
50
C Stability and plasticity analysis
As shown in Scction in DER, the new data and exemplar data use different loss functions. In ER, SCR, and iCaRL, the two employ the same type of loss function. The gradients can be rewritten as follows:
VL = M. [ + EMiing [Ve]
(10)
M V + Prew Eyues V (1 Prew) x E Eada long
[
M we have Prem < 1 in our main experiments. In particular, based on the task size M, and datastream sizes, we have: CIFAR100 Pnew E [0, 0.045], Mini-ImageNNet Prew E [0, 0.099], CORE-50 Puew E [0. 0.100)). Thus, the plasticity changes are dominated by the first term: as the size of the short-term memory decreases, the first. lerm faces greater overfitting on the new task data, harming plasticity. In contrast, DER++ employs different losses for the first and second terms, which leads to different plasticity dynamics shown in Figure 3
Based on Pnew
D Experiment Setup
Implementation Details. Our experiments were conducted on NVIDIA RIX 1080 Ti, 2080 Ti, and A6000 GPUs. In the main experiment, using batch size of 50, each for CIFAR100, Mini-ImageNet and CORE50 took roughly 2h, 9h, and 30h respectively. Following Masana et al. (022). all experiments utilized ResNet-18. We use standard dala sugmentation (random cropping and flipping), data stream batch size of 50 for CIFAR100 and Mini-Imagent Imagenet, 64 for CORE50. An equal number of exemplars is sampled at each gradient step. All models use vanilla SGD for optimization with a learning rate of 0.1. For iCaRL and SCR, a nearest-class-mean (NCM) classifier is applied as in the original publications. The default iteration and epoch number 50.
Hyperparamoters. The hyperparameter settings summarized in Table 7 The regularization strength
in DER++ and the temperature values in SCR follow the original papers.
E Unified Continual Replay Framework Instantiated with DER++
17

---

Published in Transactions on Machine Learning Research (12/2025)
Table Hyperparameter settings.
HYPERPARAMETER
ER
LR=0.1
ICARL
LR=0.L.CMM CLASSIFIER
SCR
TEMP =0.07, LR=0.1, NCM CLASSIFIER
DER
0.1, 0.5, 0.03 (CIFAR100) a a 3 0.8,Ir 0.1 (MINI-MAGGNETT) a= 0.1, 8 1.0,Ir 0.1 (CORE50)
Algorithm 2 integrates the DER++ method into the framework. Since DER++ the past logits output for knowledge distillation, the algorithm stores first the logit outputs from the learning trajectory in the short-term memory (line 17). These logits are then transferred to the long-term memory for use in future knowledge distillation steps (line 5).
Algorithm 2: Unified continual replay framework instantiated with DER++
// Non-stationary data stream: D. = U,dt where X₂ is the incoming batch // Short-term memory Mshort: storing new data
// LLng-termm memory Mingg: storing past data /fffline CL: |Mshorel is equal to the task size UngkeT &el
online CL: [Mshori is equal to the stream batch size [
// x is a memory management policy
// is logits output and scalars a are regularization hyperparameters of DER++ 1 function Continulllearningch. M,MM,,,,
Mahort Maoott UX // Update short-term memory
if Mahort full then 0.Mshort ModelTraining Mosort M Miong)
Miong Ca Mshort U // Update long-term memory Mshort + 0
return M.Mshort, MMong
function ModelTraining(O M Miong)
for K epochs do 10
for Bshort = =xx,u) y in Mshort do sample 13' B'oong- = (x', vf of =prev) from Miongs
11
sample 8" long {x" vn" ="reeu from Miong:
12
x, x²,',x" t aug(z'), aug(('), aug(x")
13
14
ho(x)
LDER++ Loo (2, 0) a a ho (2')11++ Blee (x" ,," - 0 V update Mahort with logits information Bohort = (1,
15
16
17
18
return Mshort
F Additional Experimental Results
The main paper shows the stability and plasticity analysis with Mini-ImageNett Here, Fig shows the results with CIFAR100 with different task numbers (20 tasks vs. tasks) and across architectures (ResNet-18 and 3es.et-34). These new results show behavior consistent with our previous findings. For ER, SCR, and ICARL, smaller memory consumption ratio (MCR) leads to increased stability but decreased plasticity.
18

---

Published in Transactions on Machine Learning Research (12/2025)
0.500
ER
0.595
ICARL
DER+
SCR
A Amer
min
0.84
Plasticity
1 max
mixx
0.000
0.00
0.82
0.875
min
0.475
0.73
0.78
0.82
min
Amin
0.70
0.429
0.63
0.74
0.000
0.775
0.00
-0.00
-4.55
-0.50
-0.45
-0.25
-0.15
-0.10
-0.6
-0.
-0.4
-0.
-0.45
-0.40
-0.35
-0.30
-0.25
Stability
Stability
Stability
Stability
(a) ResNet-18 with 50 tasks
ER
6.5
ICARL
DER +++
SCR
Plasticity
0.875
0.500
min
0.00
0.22
max
0.0
0.06
0.22
0.51
0.7
0.40
0.52
0.55
min
0.6
Amax
Amn
0.30
0.36
0.4
0:5
0.75
0.24
0.75
0.5
0.00-
0.750
-0.5
-050
-0.45
-8.00
-0.35
-0.30
-0.25
-0.0
-015
-0.8
-0.55
-0.4
0.3
0
-0.1
0.0
-0.40
-0.35
-0.30
-025 -020
Stability
Stability
Stability
Stability
(b) ResNet-18 with 20 tasks
0.
ER
ICARL
DER++ +
SCR
Plasticity
8.85
mmss
0.60
Amar
0.7
0.55
Amox
0.72
0.41
0.85
0.55
0.50
0.55
0.6
0.42
0.40
0.75
Amin
0.50
min
0.5
0.75
Amir
021
0:37
0.16
-0.
-0.
-0.4
-0.1
-2.25
-0.20
-0.15
-0.10
-0.0
-0.
-0.5
-0.2
-0.1
-24
-0.5
-0.2
Stability
Stability
Stability
Stability
(c) ResNet-34 with 20 tasks
Figure 4: Stability and plasticity analysis with different model architectures (ResNet18 and ResNet34) and different task numbers (20 and 50 tasks) in CIFAR100.
For DER++, reducing the MCR results in different stability behaviors, such as improved plasticity or improved stability at certain stages.
19


Reviews:
[{"review_id": "HdyvMc2W2c", "reviewer": "Reviewer_GRfr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a new framework under which several CL methods can be placed; this framework consists of a pool of long-term memory and a pool of short-term memory. In this framework, the authors define MCR as the ratio between short term and long term memory sizes. The authors then theoretically show that a smaller MCR yields a better generalization bound. Intuitively, this is because long-term memory is more representative of the true current data distribution than short-term memory. Empirically, the authors find that for common CL methods, MCR is indeed predictive of performance: lower MCR performs better.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The theoretical analysis is solid, adapting existing results in the literature to the proposed framework. The experiments are also sufficient: the paper considers a number of standard CL techniques over multiple datasets. The stability analysis is also interesting although perhaps would benefit from further study (in another paper).", "audience_yn": "Yes", "audience_explanation": "Yes: the paper presents a novel framework to analyse generalization in CL which is interesting in itself given the difficulty of theoretical analysis in this field. Moreover, the paper makes the claim that small MCR (as in online RL) can actually outperform large MCR algorithms (as in offline RL) which may be counter-intuitive. Regardless of whether these claims hold more generally beyond the settings which the authors consider, the paper would definitely be of interest to those aiming to understand CL from a theoretical perspective.", "requested_changes": "**Would strengthen**\n- Some may find it unclear how many different CL algorithms can fit into the structure of Algorithm 1; I recommend explaining how common algorithms like DER++ or ICARL are special cases of the algorithm\n- pi is not defined in line 5 of Algorithm 1\n- It's unclear how robust the results in section 6 are; I would recommend including results from multiple datasets & architectures to ensure the results are consistent across settings\n- Error bars/circles in Figure 3\n- Enlarge Figure 2 a little bit", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0xISBF09Qf", "reviewer": "Reviewer_wpCo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "A very clearly written paper, even for a non-theory person.\n1. Proposed a continual learning method which uses short and long term memory.\n2. Deeply explores the role of short divided by long term memory (called MCR)\n3. Touches upon plasticity, stability, and generalization aspects\n4. Comprehensive empirical justification with studying one outlier replay algorithm", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. There is clear algorithm description\n2. There is some theoretical analysis to show their proposal reduces generalization bound\n3. Ample empirical proof and individual replay algorithm study provided", "audience_yn": "Yes", "audience_explanation": "The paper may invigorate some people's thinking about continual learning online-offline hybrid systems. The theoretical work is not much in this paper and also makes a lot of assumptions. So, it may attract people to make more complex extensions to this.", "requested_changes": "1. Clarify. In class incremental setting, do model output neurons whose class labels have not \"arrived yet\", do they remain waiting? In simpler words, in CL, does model architecture remain unchanged?\n2. Clarify. Why did you choose a very simple STM and LTM storage scheme (greedy and reservoir sampling resp.)? I was expecting more. Atleast explain why this simple choice suffices.\n3. In theorem 1, pi i.e. sampling algorithm does not play a role? Clarify.\n4. At the end of theorem 1, you did not give final comments. So low MCR leads to low DISC_L - thereby reducing generalizartion error bound. But this was kind of expected result (not discrediting the work BTW). I feel there is some baseline missing so say you are theoretically outperforming it. Or maybe baseline is high MCR itself. Clarify\n5. Try to give some citation for true labeling function deterministic assumption you made.\n6. Try to improve Fig 3 placement. You start talkign about soon enough but figure is far away.\n7. Try to Comment and analyze why MCR can't be variable with time. Readers will find value in your paper if you leave some hope for extension towards the end. Your algorithm is a bit simple and you did not compare reservoir sampling with anything else so I am thinking on your behalf to make paper stronger. Think about it.\n8. Please bold numbers in your tables to show which numbers are best. It will improve readability. Tables looks bland right now and there are some inconsistent formattings.\n9. You did not define DI, TI, \"pretrained model\" settings in Table 4. I could not fully understand the comparisons to CI.", "broader_impact_concerns": "Definitely opens up conversation in continual learning which combines online and offline learning due to their unified framework.\nWhile the method proposed is quite simple, I can see much more sophisticated extensions to this. This paper, per my guess, will mostly serve as motivation for other people to improve upon (non-trivial achievement) but the paper itself is simple technical contribution.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mLNrxccZmj", "reviewer": "Reviewer_LdYh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work introduces a novel concept of dual memory for online and offline continual learning (CL) scenarios. Traditionally, memory in replay-based CL methodologies is considered as a single unit consisting of past representative examples. The authors propose looking at the interplay between past and current examples to strike a balance between plasticity and stability. They introduce a new metric called memory capacity ratio (MCR) to describe the relative number of past and new examples to store. They present findings on how MCR impacts the performance of online and offline CL methods.\n\nStrengths: \n1) Introduction of the concept of structured memory is conceptually sound and applicable for both online and offline settings.\n2) Presenting a quantifiable metric to study the effect of short term and long term memory on stability and plasticity enables important insights, like using a smaller MCR yields higher generalization. Such findings are particularly useful for practical applications.\n3) The experiments are well thought out by inclusion of different control variables for a fair comparison of results. \n\nWeaknesses:\n1) A smaller MCR implies fewer new examples for the model to learn new patterns. This can translate to slower adaptation to new data or data distributions. This can be tested, for example,  with fast switches between tasks or simulating domain shifts between tasks. The authors have not presented results under such conditions.\n2) Like the authors have mentioned under “Limitations”, there are no insights provided on sample selection strategies. With a lower MCR or short term memory, sample selection from current or recent tasks becomes critical for model updates. Incorporating a random reservoir sampling for exemplar selection might be the reason for differing stability/plasticity trends with different algorithms.\n3) While a controlled experimental setup enables better comparison between methods, fixed hyperparameters across datasets can lead to suboptimal performance and incorrect interpretation of results. It would be helpful to include more details on how the hyperparameters were selected and specify any assumptions made about them when interpreting the results. \n4) The paper explores and presents results on a range of MCR values. It may have been more valuable to provide an objective or method to find an optimal MCR.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Aside from the weaknesses mentioned which may complicate interpretation, the claims are supported by clear evidence.", "audience_yn": "Yes", "audience_explanation": "Applications with memory constraints like edge and IoT devices would benefit from advances in CL. Another important use case for this work is in domains like healthcare where privacy is critical and long term storage is not possible.", "requested_changes": "1) My main areas of concern is using fixed hyperparameters across datasets and algorithms and how it could be suboptimal for different MCR settings. \n2) One of the major findings is that a lower MCR can yield comparable or better performance compared to a higher MCR. This claim can be made more convincing by designing more extensive experiments like mentioned earlier.\n3) Minor: Figure 3 may be simplified. The caption can also be improved.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)