Gradient-Guided Epsilon Constraint Method for Online Continual Learning
Authors: Song Lai
Changyi Ma
Fei Zhu
Zhe Zhao
Xi Lin
Gaofeng Meng
Qingfu Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Online Continual Learning
Multi-Objective Optimization
Primary area: deep_learning

Abstract: Online Continual Learning (OCL) requires models to learn sequentially from data streams with limited memory. Rehearsal-based methods, particularly Experience Replay (ER), are commonly used in OCL scenarios. This paper revisits ER through the lens of $\epsilon$-constraint optimization, revealing that ER implicitly employs a soft constraint on past task performance, with its weighting parameter post-hoc defining a slack variable. While effective, ER's implicit and fixed slack strategy has limitations: it can inadvertently lead to updates that negatively impact generalization, and its fixed trade-off between plasticity and stability may not optimally balance current streaming with memory retention, potentially overfitting to the memory buffer. To address these shortcomings, we propose the \textbf{G}radient-Guided \textbf{E}psilon \textbf{C}onstraint (\textbf{GEC}) method for online continual learning. GEC explicitly formulates the OCL update as an $\epsilon$-constraint optimization problem, which minimize the loss on the current task data and transform the stability objective as constraints and propose a gradient-guided method to dynamically adjusts the update direction based on whether the performance on memory samples violates a predefined slack tolerance $\bar{\varepsilon}$: if forgetting exceeds this tolerance, GEC prioritizes constraint satisfaction; otherwise, it focuses on the current task while controlling the rate of increase in memory loss. Empirical evaluations on standard OCL benchmarks demonstrate GEC's ability to achieve a superior trade-off, leading to improved overall performance. Code is available at https://github.com/laisong-22004009/GEC_OCL.

Gradient-Guided Epsilon Constraint Method for Online Continual Learning
Song 1.2, Changyi Ma 2 Fei Zhu?, Zhe Zhao
Xi Lin', Gaofeng Mengt 2,3,4 Qingfu Zhang?
Department of Computer Science, City University of Hong Kong
Centre for Artificial Intelligence and Robotics, HKISI, CAS
Institute of Automation, Chinese Academy of Sciences
School of Artificial Intelligence, University of Chinese Academy of Sciences gfmengenlpr ia. en, zhaagocitan edu. hk
Abstract
Online Continual Learning (OCL) requires models to learn sequentially from data streams with limited memory. Rehearsal-based methods, particularly Ex- perience Replay (ER), are commonly used in OCL scenarios. This paper revisits ER through the lens of e-constraint optimization, revealing that ER implicitly em- ploys a soft constraint on past task performance, with its weighting parameter post-hoc defining slack variable. While effective, ER's implicit and fixed slack strategy has limitations: it can inadvertently lead to updates that negatively im- pact generalization, and its fixed trade-off between plasticity and stability may not optimally balance current streaming with memory retention, potentially over- fitting to the memory buffer. To address these shortcomings, we propose the Gradient-Guided Epsilon Constraint (GEC) method for online continual learn- ing. GEC explicitly formulates the OCL update as an e-constraint optimization problem, which minimize the loss the current task data and transform the sta- bility objective as constraints and propose a gradient-guided method to dynami- cally adjusts the update direction based on whether the performance samples violates a predefined slack tolerance E: forgetting exceeds this toler- ance, GEC prioritizes constraint satisfaction; otherwise, it focuses the current task while controlling the rate of increase in memory loss. Empirical evalua- tions standard OCL benchmarks demonstrate GEC's ability to achieve perior trade-off, leading to improved overall performance. Code is available at https: /tt::://ttttttt www.laiooonao...eaeaacoaoooooelt 22004009/GBB GECOO
memory
Introduction
Continual Learning (CL) aims to develop systems that can learn sequentially from a stream of tasks, critical capability for real-world applications where data evolves (De Lange al. 2021 On- line Continual Learning (OCL) presents a particularly challenging scenario where data arrives in small batches, is typically processed in single pass, and memory for past experiences is strictly limited [Mai et al. 2022 Lopez-Paz and Ranzato 2017] The central challenge in OCL is catas- trophic forgetting McCloskey and Cohen 1989 French 1999 where learning new tasks causes
severe degradation of performance on previously learned ones. This has been a focal point of recent
research De Lange al.
2021
Mai al. 2022
"Equal Contribution
Correspnnding Authors
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Among the various strategies proposed to mitigate forgetting, rehearsal- methods, which store and replay a small subset of past data,have proven to particularly effective in the OCL[Rebuftj et al. 2017, Rolnick et al., 2019]. Experience Replay (ER), in its basic form, combines the loss on current data with a weighted loss on replayed memory samples. The potential for overfitting to the limited memory buffer has motivated constraint-based replay methods like Gradient Episodic Memory (GEM) [Lopez-Paz and Ranzato 2017] and Averaged GEM (A-GEM) (Chaudhry et al.) 2018). These methods use memory samples primarily constrain the gradient direction of the current task, aiming to prevent an increase in loss on past tasks. However, there is substantial empirical evidence (Zhang al. 2022] suggesting that rehearsal- based methods like ER, which train directly on memory samples, consistently outperform methods that only use memory for gradient constraints.
In this paper, we investigate the underlying reasons for ER's strong empirical performance by revis- iting it from the perspective of 6-constraint optimization [Miettinen] 1999] (Section We posit that ER implicitly solves an e-constraint optimization problem, where the constraint on past task perfor- mance is soft and the weighting parameter À for memory samples implicitly defines a slackness. In contrast, methods like GEM and A-GEM enforce hard constraints (near-zero slack). While ER's implicit soft constraints contribute to its success, the fixed nature of À and the post-hoc, implicit definition of this slack present challenges. Considering the optimization generalization gap in OCL Ye and Bors 2022). it becomes crucial to understand how slack strategies affect the model's test performance.
opiimizationggenerllaaattooo
Our empirical analysis (Section 4 further explores this relationship. We find that ER's approach, which allows for some beneficial conflicts with memory task gradients, can lead better gener- alization than strictly adhering to memory constraints. Specifically, mild violations of memory constraints do not always lead to poorer test performance. However, ER is not without its flaws. Firstly, its implicit, post-hoc constraint mechanism means the optimization process can still venture into regions that yield negative generalization gains. Secondly, even when memory constraints are satisfied, fixed À can suboptimally manage the plasticity-stabilis trade-off, potentially leading to overfitting on the memory buffer by excessively minimizing memory loss.
Motivated by these insights, we propose the Gradient-Guided Epsilon Constraint (GEC) method (Section 5). GEC explicitly adopts a dynamic gradient- based e-constraint optimization approach. At each learning step, GEC formulates the update as quadratic program. The core idea is to: (1) When the constraint on past task forgetting (defined by a permissible slack E) is violated, GEC prioritizes satisfying this constraint. (2) When the constraint is satisfied, GEC focuses on optimizing the current task and controls the rate at which the memory constraint can be relaxed (i.e., memory loss can increase), preventing overfitting to the memory buffer and preserving generalization capacity. The adjustment mechanism allows GEC to effectively navigate the trade-offs inherent in OCL.
The main contributions of this paper are:
We revisit Experience Replay from an e-constraint optimization viewpoint, highlighting its implicit soft constraint nature and the role of its weighting parameter as a post-hoc slack definition. We also provide empirical evidence suggesting why ER often outperforms hard constraint-basee methods, pointing to the benefits of allowing constraint violations and leveraging positive transfer from memory.
By identifying key limitations of ER's fixed and implicit constraint strategy, we introduce the Gradient-Guide Guided Epsilon Constraint (GEC) method, novel OCL algorithm that ex- plicitly enforces and dynamically adjusts forgetting constraints using gradient information, addressing key limitations of ER's fixed and implicit strategy.
We provide theoretical justification for GEC, analyzing its update dynamics and conver- gence tendencies within the constrained optimization framework. Comprehensive experi- ments standard OCL benchmarks demonstrate that GEC achieves superior balance in online continual learning, outperforming existing state-of ofethh--aartt methods.
2 Preliminary
The e-constraint method MMettinen 1999 is a widely used technique for finding Pareto optimal solutions in multi-objective optimization (MOO) [Chankong and Haimes 2008). The core idea of

---

the e-constraint method is to convert a MOO problem into a series of single-objective problems. This is achieved by selecting one of the objective functions be minimized (or maximized) while transforming the other objective functions into inequality constraints. Specifically, for a problem with K objective functions S1 (xx for optimizing (xr) (the s-th objective) is:
be minimized, the c-constraint formulation
min x
[a(z)
s.t. 5 (L..,...,
where x is the vector of decision variables and ej are predefined upper bounds (tolerances) for the constrained objectives (z). By systematically varying the values of cj. one can generate different pareto optimal solutions [Gunantaraa 2018).
In CL, where a model must learn new information while preserving previously acquired knowledge, the objectives of minimizing current task loss and minimizing forgetting of past tasks often in conflict. The e-constraint framework provides a natural way to formulate and manage this trade-off, as will be explored in subsequent sections. In this paper, objective function of plasticity is selected to be optimized while stability is converted into constraints by setting an upper bound.
3 Revisiting ER from e-Constraint Perspective
3.1 Problem Setup
We consider an OCL setting where a model f(X;)) learns from a sequence of tasks {(X1, Yi)...,,. (XT, Yr)}. (e(](Xx; 0), Y:) is the loss for task I. At step & (processing a mini-batch from the current task 1), the model has access to a limited memory buffer M containing samples {(Xm: Ymm Imet from past tasks. The goal is to learn the current task while mitigating forgetting of past tasks. We denote the parameters before the current update as Q-- and after the update as 0g.
3.2 Continual Learning as c-Constraint Optimization
As suggested in (Aljundi 2019]. CL can be viewed as attempts to solve a e-constrained optimization problem. The primary objective is to minimize the loss on the current task data, (0). subject to constraints on the performance degradation of previous tasks, typically measured on samples from the memory buffer M. This can be formulated as:
min (x(: (), Y.) s.t. (m) S Em))), Vm E M, #
(2)
where Ms could represent individual past tasks an aggregate. (l) lm (f 0), (f f-- 1), Ymm is the change in loss for past task m (or memory sample/ / average memory loss) relative to its loss before the current update step k. (m) is the permissible slack for past task m at step k.
Methods like GEM (Lopez-Pazz and Ranzato 2017 and A-GEM (Chaudhry et al. 2018 explicitly enforce such constraints, typically aiming for Em (k) ss (non-increasing past task loss). For A- GEM, this is often single constraint on the average memory loss:
min (x(((X 0), Y-) s.t. (M( 0), YAs) (Mx Vas) $ 0
(3)
where Em is the average loss over samples in memory M. Compared with hard constraint of GEM and A-GEM. ER optimizes a composite loss:
min (fo(f(Xe:: 6), + ())
(4)
where X > Next we will show that ER also implicitly solves an c-constraint problem with soft, dependentn slack, where X plays a crucial role in defining this slack post-hoc.
Proposition 1 (ER as e-Constraint Optimization). Let 0* be a solution the Experience Replay optimization problem in Eq. (4) using parameters 0x-1 as the starting point for the optimization step. Then, 0" also a solution to the following e-constraint optimization problem:
min (x e: O),Y)) Yo) S.1. Exx (f(( as; 0), YMJ) - (x as; 0x-1), YMM S EM (k))))
(5)
where the slack can be chosen as ety (k) E (X m; 0*), YM) (ar(J(X 0x-1) YM). The magnitude of this effective slack E (k) is influenced by X.
3

---

foucs on (,)
(,)
(6)
(8)
(x(())
constroabi
fiord preference
GBC
construbet sarisfied
forscs on (,())
Be + Afte
Ve Se + Aute
4,083
(,00)
(,)
(8) ER e-constraint
(b) Fixed A
[c] Adaptive
(6) Trajectory
Figure 1: (a): Each choice of ER's linear weighting parameter for memory samples corresponds to an implicit slack tolerance e for the stability constraint. The dashed lines in (a) represent the iso-objective contours for ER's scalarized loss, where the optimal solution is found at the tangent point to the Pareto front; (b): Illustration
a trajectory taken by a fixed weighting strategy like ER. Such strategies potentially leads suboptimal paths in the objective space; (c): Depiction of dynamic update adjustment by the proposed GEC method. (d): Comparison of moving trajectories in the objective space between ER and GEC.
Proposition 2 (e-Constraint Optimization as ER under Convexity). Suppose the assumptions stated in Appendix hold, where (0) and (0) are convex and differentiable. Let 0* be solution to:
min (x(f 0), Y.) s.t. (f (X(m;;)) Yu) - ( 0x-1), YM) (k)
(6)
There exists weight X 2 O such that l also minimizes (o( (EM - (m Since (0x-1) is constant for the current optimization step, this is equivalent to minimizing (c(0) +
X- (0) plus constant.
Detailed proofs for Propositions and are provided in Appendix A.1 To help understanding,
Fig. a) an intuitive demonstration of the above propositions, which illustrates that ER's weight- ing parameter À implicitly defines a constraint slackness Th (k). This means ER operates with flexible, soft constraint. This insight motivates a deeper investigation into how different constraint strategies (different À values in ER, or the hard constraint of GEM) influence the model's general- ization performance.
4 Empirical Evidence for Beneficial Conflicts
To understand why ER often outperforms hard constraint methods like GEM, we analyze the re- lationship between gradient updates, memory buffer interactions, and generalization. In OCL, the update direction g is derived based on the current task and memory samples. The interaction with memory can be quantified.
Definition (Gradient Cosine Similarity (GCS)). Let g be the gradient vector used for the model update at step ie (i.e., 0g fk-1 1a-1-ng) ng). Let gMs = Vol (0x-1) be the gradient of the average loss on the memory buffer M, evaluated at 0g-1, 1. The Gradient Cosine Similarity defined as:
GCS = =cos=c (M4) [gglll
9M
smaller (more negative) GCS indicates that the update is more misaligned with the direction that would decrease memory loss. The change in memory loss is approximately -n(g gm) under locally linear assumption. Thus, a smaller GCS (more negative (g. 9M)) implies a larger increase in memory loss (constraint violation).
Definition 2 (Generalization Gain (GG)), Let 0 be the model parameters before an update, and be the gradient vector for the update. Let P(ő) be performance measure (e.g., accuracy) on a held-out test set comprising samples from all encountered tasks. 17 the learning rate. The Generalization Gain is:
GG P(0 - ( P(0).
(8)

---

2R -15)
-42
-04
-24
GCS
(6) Seq-CIFAR10
(6) Seq-CIFAR110
(c) SeggTiinyllnet
Figure 2: Relationship between GCS and GG for ER and GEM on Seq-CIFARIO, Seq- Beq-CIFARI00,
and Seq- Teq-TTiylmmaeeee
By analyzing the relationship between GCS (for different in ER, and for GEM) and GG in Fig 2 we observe the following:
Mild gradient conflicts can be tolerable for generalization.
When the model's update direction g moderately confticts with the memory gradient 9M (i.e., GCS is negative, indicating an increase in memory loss), this does not necessarily lead to a decrease in test performance. ER, by allowing such conflicts via its soft constraint, beneficial for generalization even if they slightly worsen the loss on the memory buffer. This sug- gests that strictly preventing any increase in memory loss (as in GEM) might be too restrictive and discard potentially useful updates for gencralization. This helps explain ER's empirical advantages over GEM-like methods. However, ER's s approach is not without drawbacks. We also observe that:
find updates that
Implicit post-hoc constraints risk negative generalization. Fixed X limity increusing rute of generalization gain when gradients align.
Firstly, ER's ss constraint is implicit and determined post-hoc by X. During optimization, the model can still move into regions that yield significant negative generalization gains (left areas). Secondly, when the memory constraint is already satisfied (i.e., memory loss is low or decreasing), fixed X in ER continues to allocate a portion the update to minimizing memory loss. This can lead to overfitting the memory buffer, the actual update direction might be driven to further reduce memory loss beyond what is necessary for preserving past knowledge. This over-correction can potentially harm overall generalization by not allowing the model to fully exploit plasticity. And inappropriate À leads to strong negative transfer (See Fig ())..
These limitations motivate method that (1) more explicitly manage the constraint past task forgetting to avoid severe negative generalization, and (2) dynamically adjust its focus between current task learning and memory constraint satisfaction, especially by preventing overfitting the memory buffer when constraints are met. This dynamic adjustment, illustrated in Fig. [(b) and (c), ensures that when the forgetting constraint is violated, the update prioritizes constraint satisfaction, and when satisfied, the update focuses on the current task while carefully controlling memory loss increase to maintain generalization.
5 Gradient-Guided Epsilon Constraint Method (GEC)
To address the identified limitations of ER and to explicitly manage the trade-off in OCL, propose the Gradient- Guided Epsilon Constraint (GEC) method. GEC treats OCL as an e-constraint opti- mization problem, where aim minimize the current task loss while ensuring that the increase in average memory loss does not exceed a predefined slack tolerance E.
Let (0())) be the loss on the current mini-batch at step k with parameters 0g, and (L,(0)) its gradient. Let Em (0x) be the average loss on samples from the memory buffer M, and Vl, (0x its gradient. We define constraint function h((p)) that measures the extent of forgetting relative to the
5

---

slack E:
h(Ox) (0x) mm (0k-1)
We desire h(0O)) < O. E. (0k_1) is the average memory loss before the current update (i.e., using parameters 0x-1). Similar to [Lopez-Paz Papez-Pazann and Ranzato 2017], at each update step k. GEC determines the update direction U) by solving the following quadratic program (QP):
Uk arg min vERd
{ (V((((0))
u1
s.t. (V.m) 20 (0s)
(10)
The objective of this QP is to find an update direction v that is close as possible to the current task's 's gradient (l.( The constraint (0x)((x), u) 2 Φh (0x) modulates this update based on the state of the memory constraint h(Ox). The term 6n (0x) is a dynamic barrier function defined as:
on (0+) h(O))) = a (em (0x) fm (0(0))1)-4 - 88)
(11)
where a > is hyperparameter controlling the strength of the barrier.
The behavior of this formulation is as follows:
Φn, (0x) > 0. The QP constraint forces the update Uy to have a sufficiently positive projection onto (0a)... Since the parameter update is 0s+1 = 0₂ 17 positive projection (Vem (8x), U2} aims to decrease En (or slow its increase significantll), thus working to satisfy h(Ox) S 0. The update prioritizes reducing constraint violation.
If h(0²)) > 0 (Constraint Violated):
2 (0%)
If h(0x)) S (Constraint Satisfied): (0x) < 0. The QP constraint (Ox), allows 2k to have a negative projection onto VL (0x) (up to the point dictated by dn (0x))) mean- ing L' allowed to increase. This permits more aggressive optimization of the current task loss Le(0x). The magnitude of on, (0x) controls how much f. is allowed to increase, preventing excessive forgetting or overfitting to the memory buffer by trying to decrease f, too much.
Φn (Ox)
The solution to the QP in Eq.
10 is:
2k Vlc(On) + d (Ox)
(12)
where the coefficient 0 is:
(0n( (O2), (V. [0x))]) +6
A max
(13)
A small constant 0 is added for numerical stability. The model parameters are then updated: 0x+1 < 02 77k, where 7) is the learning rate. While the formulation above considers single aggregate memory constraint for clarity, in practice, when task identities are available in the mem- ory buffer, the constraint can be naturally extended to handle per-task memory samples, leading to a multi-constraint optimization that better preserves task-specific knowledge. Since the slack toler- ance E is defined relative reference memory loss, we employ a dynamic threshold strategy that calibrates E based on the observed loss at task boundaries, along with a warm-up phase to estab- lish stable initial estimates, Detailed implementation choices and the full algorithm are provided in Appendix D and Appendix
5.1 Theoretical Properties
The GEC method exhibits desirable convergence characteristics within its constrained optimization framework. We analyze its behavior using a continuous analogue d9 dt -u(#), where v(0) is the GEC update direction. Let (+) be the current task loss and h(0) f,, (0) - E be the constraint violation measure. The dynamic barrier is on (0) = (0). The gradients are (f) and Vh(0) V. (0). The update v(0)= = (f. V (0). Then the properties of GEC are:
When the constraint is violated, i.e., l((0) > O, the GEC update actively works to reduce this violation. The measure of constraint violation [b())]] max(h(0), 0) is non- a increasing, Specifically, in continuous time, a [0.))] < -[ob (0:)) Since (0.) ah(0,) > 0 when h(,,) > 0, this implies [+(++)]++ decreases, guiding the model towards satisfying the constraint h(0,) 0.
Constraint Adherence:

---

Objective Minimization under Satisfied Constraints: When the constraint is satisfied, i.e., h(0) & 0, GEC prioritizes minimizing the current task loss (0). The current task loss (0) is non-increasing: ai (0+e I -I x(0:)l=
Σ This allows the model focus on learning the new task while the dynamic barrier ensures that h(0) does not grow uncontrollably.
These properties suggest that GEC offers principled way to dynamically balance learning new information and preserving past knowledge by adaptively managing the e-constraint based on the observed forgetting and the slack tolerance
Table 1: Performance comparison (AAA and Acc) with baseline methods on Seq-CIFARIO, Seq- CIFARI00, and Seq-Ting TenyTTiyImmaeeee using Reduced ResNet-18. Results averaged over 5 runs.
Beq-CIFARIO (N=5)
Seq-CIFAR 100 (N=20)
Seq- Tinylmageeett (N=20)
Method
[
0.6k
[M]
1k
IM
1k
[M|
5k
[M
2k
MM| = 5k
AAA
Acc
AAA
Acc
AAA
Acc
AAA
Acc
AAA
Acc
AAA
Acc
SGD
34.04
16.68
34.04
16.68
9.67
3.24
9.67
3.24
7.63
2.17
7.63
2.17
ER
54.68
39.43
54.91
42.04
17.86
11.89
20.67
14.87
16.27
10.52
16.10
11.89
Refresh CL
62.34
51.42
65.81
55.57
24.32
17.64
35.61
32.33
18.59
12.83
22.68
18.72
DER
49.06
25.80
48.25
23.90
10.96
3.71
10.47
3.68
8.04
2.46
7.65
2.04
DER+
57.17
47.03
61.01
50.31
17.30
8.72
17.08
8.98
12.42
5.57
11.93
5.26
CLSER
61.64
50.36
63.27
53.06
22.58
15.68
23.25
16.42
18.50
10.03
18.88
11.61
CBA
63.41
51.47
65.47
52.08
22.46
15.54
23.07
15.87
18.79
11.43
18.98
10.98
POCL
63.62
53,42
66.23
58.50
26.68 68
16.54
36.34
33.36
21.56
12.69
25.48
19.40
EWC
36.51
18.37
36.51
18.37
9.87
2.77
9.87
2.77
7.96
2.43
7.96
2.43
GEM
37.78
18.84
37.00
18.73
13.43
6.04
13.71
6.46
10.17
3.70
10.27
3.81
A-GEM
37.67
18.51
37.62
18.03
10.61
3.75
10.80
3.52
7.66
2.33
7.79
2.40
GEC(Ours)
65.21
54.08
69.42
59.12
29.63
17.78
38.11
34.22
22.45
12.77
27.66
20.25
Experiments
6.1 Experiment Setup
Following [Buzzega et al. 2020 Arani ct al. 2022 Lai et al. 2025 evaluate method on three commonly used benchmarks for online continual learning: Sequential CIFAR-10 (Seq-CIFAR10), Sequential CIFAR- (Seq-CIFAR10O) Krizhevsky et al. 2009 and Sequen- tial Imagentt (Seq-Tiny ImageNet) [Buzzega et al., 2020). For Seq. Seq-CIFARIO, the dataset is split into tasks, each containing 2 classes. We test with memory buffer sizes (lM) of 0.6k and Ik samples. For Seq-CIFAR 00, the dataset is split into 20 tasks, each with 5 classes, using memory buffer sizes of 1 and 5k samples. Seq-TinylmageNet is split into 20 tasks (10 classes per task), with memory buffer sizes of 2k and 5k.
Datasets
Evaluation Metrics. To comprehensively evaluate performance, let as denote accuracy the model on task i after training sequentially up to task j. Let T be the total number of tasks. We use the following standard metrics [Soutif- Coutif-Cormerai et al. 2023]: Average Anytime Accuracy (AAA)
measures the average performance over the learning process: AAA T > ( E/ ai.s). Av-
erage Accuracy (Acc) is average accuracy after training T tasks: Acc = T Σ a,, Forgetting
Measure (FM) quantifies how much the model forgets past tasks: FM _ I Ei= (a., a max) ), DT-1(a,,)
where max is the peak accuracy task i observed after training on task i. A less negative FM
indicates less forgetting.
To further dissect the learning dynamics, particularly for visualizing the trade-off managed by GEC, we decompose the performance into: Plasticity (Ace-P): The model's ability to learn the current task, measured as P = ΣT ag.k- Stability (Ace-S): The model's ability to retain knowledge of past tasks, measured as S -_ ET-I ( (aT
Implementation Details. Consistent with [Chrysakis and Moens 2023 we use the Reduced ResNet- 18 (He et al., 2016 the backbone network for all experiments. Models are trained using
7

---

Table 2: Forgetting Measure (FM) comparison with baseline methods on Seq- Seq-CIFARIO, Seq-
CIFAR100, and Seq-TinyImaa TinyImageNe using Reduced ResNet- 18. Results are averaged over 5 runs.
Seq-CIFARIO (N=5)
Seq- Seq-CIFARIO0 (N=20)
Seq TinylmageNet (N=20)
Method
M = 0.6k
[M] = 1k
I = 1k
[M = 5k
[M 2k
|M] 5k
SGD
-63.01
-61.01 1:300
-52.24_120
-5..24
-43.5820sk
43,58
ER
-35.68
-32.161810
-4,22_mee
-...5
-37.902028
-37.432281
Refresh CL
-25.11
-10663
-39,83-108
-20.34
-36.63
-25.13+1.27
DER
-53.60
-55.22+1.0
-60.58
-59.94
-47.1
+066
-4..27
DER++
-2..00
-28.01
-58.61 2000
-61.23..o19
-47.13 a.000
-47.6
CLSER
-97.03
-31.38
45.63__0.2
-50.71
-44.45 10.32 10:32
-4..2
CBA
-27.15
-2..31
-44.30=1.s
-50.1622766
17
-40.53
POCL
23.22
+12.8
-37.61 o.0
-1..55
+31.63
-20.22
EWC
-64.21 20:6
-64.21
-59.55_199
-56.5521.0
-44.30
-44.3021.0
GEM
-58.09
-57.64
-45.83ma
-50.61
-42.30
-42.71
A-GEM
-66. +161
-66.61 1.000
-60.51
-60.89
.45.61 +006
+004
-45.51
GEC(Ours)
-21.52+221
-15.69
-34.49 0.89
-15.12 1.0
-30.13+14
-19.54
Stochastic Gradient Descent (SGD) with batch size of 32. All experiments are conducted in the online glass-incremental learning setting, where each task is trained for a single epoch, and task identity is not available during inference. All reported results are averaged over 5 independent runs. Further details on the experimental setup and baseline methods can be found in Appendix
Baseline Methods. We compare GEC with a comprehensive set of baseline methods, including: Simple SGD, ER (Rolnick et al., 2019). regularization- based methods like Elastic Weight Consoli- dation (EWC) [Huszár] 2018), , gradient projection methods like Gradient Episodic Memory (GEM)
Lopez-Paz and Ranzato 2017] and Averaged GEM (A-GEM) [Chaudhry et al.] 2018), and more advanced rehearsal-based methods such as Dark Experience Replay (DER) [Buzzega et al.] 2020). DER with logits (DER++) [Buzzega et al., 2020], CLSER [Arani et al. 2022J. CBA [Wang et al.) 2023 Refresh-CL (Wang al.) 2024), and Pareto Optimized Continual Learning (POCL) [Wu ct al. 2024b) For some of the baseline experimental results, we followed the reported results from Wu et al. 2024b). Detailed descriptions of these baselines are provided in Appendix E
6.2
Experimental Results
To assess the effectiveness of GEC, we perform a comprehensive set of experiments and conduct ablation studies to address the following key questions:
How does GEC perform on OCL benchmarks? Table I presents the AAA and Acc on Seq-CIFAR10, 10, CIF-CIFAROOO, and Seq- Seq-TinyImageNee The methods in the bottom row are the methods that utilize hard and explicit constraints while the middle and top row are the meth- ods of representative OCL and CL methods, respectively. Our proposed GEC method consistently demonstrates strong performance across all datasets and memory buffer configurations. Notably, GEC surpasses other baseline methods, including strong rehearsal- based techniques CLSER and those designed for specific aspects of online continual learning. This suggests that GEC's dynamic constraint manA-GEMen effectively balances learning new tasks and preserving old knowledge. It noteworthy that on Seq-CIFAR 100 and Tegg-TiilmmageNe GEC achieves a significant improve- ment in AAA compared to most baselines, highlighting its efficacy in more challenging, longer task sequences.
Question 1:
Question 2: How effective is GEC in preventing catastrophic forgetting? Table 2 shows the Forgetting Measure (FM), A higher FM (less negative) indicates less forgetting. GEC demonstrates superior forgetting mitigation compared to many baselines. This underscores GEC's ability to pre- serve learned knowledge by explicitly constraining the increase in memory loss when forgetting exceeds the tolerance E. The superior performance of GEC across these diverse settings on FM val- idates our hypothesis that explicitly and dynamically managing the epsilon-constraint on memory loss leads to more robust and effective OCL strategy. The careful control over constraint relaxation when performance is within tolerance also prevents overfitting to the buffer, which can indirectly aid in better long-term retention.
8

---

GBC
13
(al Seq CIFAR1OO Plastioity
(b) Seq CIIACIIIAARRRR 00 Stabluty
(el Tingt-Tiitmageeee Plasticity
(d) Siy-aaammmeeeee Stability
Figure 3: Dynamics of Plasticity (Ace-P) and Stability (Acc-S) for GEC and ER on CIFARIO and Beq-Tinylmagee with |M| 1k.
Question 3: How does the adaptive weighting strategy affect stability-plastiiity dynamics? To further investigate how GEC balances learning new information (plasticity) and retaining past knowledge (stability), CIFAR100 and Seq-Tinylmagee ImngeNt,,, comparing GEC with ER (Figure GEC demonstrates a more favorable trajectory the stability-plastictyy space. In the training of the first few tasks when mem- ory constraints are violated (not shown directly, but implied by potential drops in stability for ER), GEC's mechanism prioritize constraint satisfaction helps maintain better stability (higher Acc-S) without unduly sacrificing plasticity (Acc-P on the current task). In the later stages of training, when the model stability remains stable, GEC's focus on plasticity, while controlling the increase rate of memory loss, allows it to achieve competitive (Seq-CIFARI00) or superior (Seq- Tiny ImageNet)) plasticity compared to ER, which might overfit the buffer. This dynamic adjustment contributes to GEC's overall better performance.
plot the dynamics of Acc-P and Acc-S during the training process Seq-
Question 4: How does the slack tolerance navigates stability-plasticiis pareto front? Our GEC method, by design, allows for exploring different trade-off's between current task learning and past task retention through the slack tolerance parameter Figure 4 qualitaaively illustrates how vary- ing E can allow GEC to trace out a pareto front in Seq-CIFAR TO with |M|=lk where each point represents different balance. A smaller would correspond potentially leading to higher stability at the cost of plasticity, while a larger allows for more flex- ibility. This controllability can successfully provide Additionally, GEC's solutions dominate all ER solutions.
stricter constraint on forgetting,
of well-representative Pareto solutions.
Question 5: How does GEC algorithm efficiency compare to other OCL algorithms? Balanc- ing computational cost and performance is a practical challenge in continual learning, particularly in online settings (Verwimp et al.) 2023). We analyze the time complexity of GEC by comparing its training time with other representative OCL methods on the Seq-CIFAR10 10 dataset, as shown in Ta- ble. While GEC introduces additional computational steps compared the simpler ER, it achieves this superior performance with significantly less overhead than other recent advanced methods like POCL and CBA, highlighting favorable trade-off between performance and efficiency.
Table 3: Training time comparison of different OCL Methods on Reg-CIFARIO.
Swebility (Acco-S 8
Method
Time (s)
AAA
Acc
ER
86,31
54.91
42.02
-32.16
POCL
474.31
66.23
58.50
-16.85
CBA
274.31
65.47
52.08
-27.31
Platicity (Ace-P)
GEC(Ours)
147.62
69.42
59.12
-15.69
Figure 4: Pareto front achievable
by GEC (Red) and ER (Blue).
7 Related Work
Our work is situated within the extensive body of research on Continual Learning, particularly fo- cusing on rehearsal-based strategies for OCL setting. CL. methods are broadly categorized into regularization-bat aechtecture-basel and behsarsal-based approaches (De Lange et al., 2021

---

Regularization methods like EWC KKirkpatrick et al. [017] and SI [Zenke et al. 2017 penalize changes to important parameters but can be overly conservative.
Rehearsal-basee methods, which are highly relevant to our work, maintain a small memory buffer of past examples. The foundational method, ER (Rolnick et al. 2019), co-trains current and past data, and our work provides a novel e-constraint interpretation of its mechanism. Constraint- based methods like GEM [Lopez-Paz and Ranzato 2017] and A-GEM [Chaudhry et al. 2018 use the buffer to project gradients, enforcing a hard constraint against increasing past task loss. In contrast, GEC proposes dynamic, soft-constraint approach, offering more flexible balance. More recent OCL methods like POCL [Wu et al., 2024b] and CBA (Wang et al. [023)] also tackle the cubbility-plastiiiiss trade-off, with POCL employing multi-objective optimization perspectives. GEC contributes to this line of work by offering a principled and efficient gradient- -uided mechanism rooted in e-constraint optimization. more comprehensive review related literature is available in Appendix B
8 Conclusion
This paper revisited Experience Replay (ER) in Online Continual Learning (OCL) from an e- constraint optimization perspective, highlighting limitations of its implicit and fixed slack strategy- We introduced the Gradient-Guided Epsilon Constraint (GEC) method, which explicitly formulates OCL updates as dynamically adjusted e-constraint problem solved via quadratic program. GEC modulates gradient updates based on predefined slack tolerance E, prioritizing constraint satisfac- tion when forgetting is high and focusing current task learning while controlling memory loss otherwise, thereby mitigating catastrophic forgetting and preventing memory overfitting, Compre- hensive experiments on standard OCL benchmarks substantiated GEC's superiority in achieving better stability- Saablittyylasticityy trade-off and outperforming state-of-the- methods.
Acknowledgements
This work was supported in part by the National Natural Science Foundations of China (Grant No. 62376267), the Beijing Natural Science Foundation (Grant No. L253019), the Research Grants Council the Hong Kong Special Administrative Region, China (GRF Project No. CityU 11215723), and the innoHK project. We would also like to thank Chengyu Lu of City University of Hong Kong for his valuable discussions and helpful insights.
References
R. Aljundi. Continual learning in neural networks. arXiv preprint arXiv:1910.0277 2019.
R. Aljundi, M. Lin, B. Goujaud, and Y. Bengio. Gradient based sample selection for online continual learning. Advances neural information processing systems, 32, 2019.
E. Arani, F. Sarfraz, and B. Zonooz. Learning fast, learning slow: A general continual learning method based on complementary learning system. arXiv preprint arXiv: 12604, 2022.
S. P. Boyd and L. Vandenberghe. Convex optimization. Cambridge university press, 2004.
Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara. Dark experience for general continual learning: strong, simple baseline. Advances in neural information processing systems, 13:15920-159300 2020.
V. Chankong and Y. Y. Haimes. Multiobjective decision making: theory and methodology. Courier Dover Publications, 2008.
A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny. Efficient lifelong learning with gem. ArXiv, abs/1812.004200, 2018. URL httppe////pp..sennnncccccccoaatt orwpaa/tt..cccl.....oooooooooop. org/CorpusII: 54443381
A. Chrysakis and M.-F. Moens. Online bias correction for task-free continual learning. ICLR 2023 at OpenReview, 2023.
10

---

M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuyte- laars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366-333 2021.
R. M. French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4): 128-135 1999.
C. Gong and X. Liu. Bi-objective trade-off with dynamic barrier gradient descent. NeurIPS 2021, 2021.
N. Gunantara. A review of multi-objective optimization: Methods and its applications. Cogent Engineering, 5(1):1502242, 2018.
G. Gupta, K. Yadav, and L. Paull. Look-ahead meta learning for continual learning. Advances in Neurul Information Processing Systems, 33: 1588-11596 2020.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
Huszár. Note on the quadratic penalties in elastic weight consolidation. Proceedings of the National Academy of Sciences, 115(1))E2496-E244) 2018.
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, Quan, T. Ramalho, A. Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 114(13):3521-335
2017.
A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Lai, Z. Zhao, F. Zhu, X. Lin, Q. Zhang, and G. Meng. Pareto continual learning: Preference- conditioned learning and adaption for dynamic stability-plaaticity trade-off. In AAAI Confer- ence on Artificial Intelligence, 2025. URL URL http::ttt///t //api. www.nticscooolaattt..e..i www.:olla.rrrrroooooooo ArrgCorpusIDD 277452206
Z. Li and D. Hoiem. Learning without forgetting. machine intelligence, 40 40(12):2935-2947, 2017.
IEEE transactions on pattern analysis and
D. Lopez-Paz and M. Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Z. Mai, R. Li, J. Jeong, D. Quispe, H. Kim, and S. Sanner. Online continual learning in image classification: An empirical survey. Neurocomputingg 469:28-51, 2022.
M. McCloskey and J. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.
K. Miettinen. Nonlinear multiobjective optimization, volume 12. Springer Science Business Media, 1999.
S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert. icarl: Incremental classifier and rep- resentation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001-2010, 2017.
D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne. Experience replay for continual learning. Advances neural information processing systems, 32, 2019.
A. Soutif-Cormeria A. Carta, A. Cossu, J. Hurtado, Lomonaco. J. Van de Weijer, and H. Hemati. A comprehensive empirical evaluation online continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3518-3528 2023.
E. Verwimp, R. Aljundi, S. Ben-David, M. Bethge, A. Cossu, A. Gepperth, T. L. Hayes,
E. Hüllermeier, C. Kanan, D. Kudithipudi, et al. Continual learning: Applications and the road forward. arXiv preprint arXiv:2311. 12111119900 2023.

---

Q. Wang, R. Wang, Y. Wu, X. Jia, and D. Meng. Cba: Improving online continual learning via continual bias adaptor In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19082-19092, 2023.
Z. Wang, Y. Li, L. Shen, and H. Huang. A unified and general framework for continual learning. arXiv preprint arXiv:2403.13244 2024.
Y. Wu, L.-K. Huang, R. Wang, D. Meng, and Y. Wei. Meta continual learning revisited: Implicitly enhancing online hessian approximation via variance reduction. In The Twelfth international conference on learning representations, volume 2, 2024a.
Y. Wu, H. Wang. P. Zhao, Y. Zheng, Y. Wei, and L.-K. Huang. Mitigating catastrophic forgetting in online continual learning by modeling previous task interrelations via pareto optimization. In porty-first International Conference on Machine Learning. 2024b.
F. Ye and A. G. Bors. Task-free continual learning via online discrepancy distance learning. Ad- vances Neural Information Processing Systems, 25:23675-233688 2022.
Zenke, B. Poole, and Ganguli. Continual learning through synaptic intelligence. Interna- tional conference machine learning, pages 3987-3995. PMLR, 2017.
Y. Zhang. B. Pfahringer, Frank, A. Bifet, N. S. Lim, and Y. Jia. A simple but strong baseline for online continual learning: Repeated augmented rehearsal. Advances in Neural Information Processing Systems, 35:14771-14783, 2022.
12

---

In this appendix, we mainly provide:
Section A.1 and A.2 (hhooretical Analysis): Detailed proofs for the propositions regard-
ing the c-oonstriint perseettive of ER and the theoretical properties of the GEC method.
(Detailed Related Works): A review of existing literature in Continual Learn- ing, Online Continual Learnigg, and relevant Multi- culti-Obbecctve Optimization methods.
Section B
Section C
Addittinal Results): Supplementary experimental results.
Section D
Algorithm Details): A pseudo-code implementation of the GEC algorithm.
Section (Experimental Setup Details): A comprehensive description of the datasets, implementation specifics, and baseline methods used in the empirical evaluation.
Section F
(Limitations): A discussion limitations of the proposed GEC method.
Section G
Broader Impact): A consideration of the potential broader impact and societal implications of this research in Online Continual Learning.
A Theoretical Analysis
Notation Throughout this appendix, use the following notation: vectors denoted by lower- case bold letters (e.g., E IR"), matrices by uppercase bold letters (e.g.. X E Rexd), and functions by bold letters when they are veclur- (e.g.. f R + I²²). Scalars are denoted by regular letters. The transpose of a vector or matrix is denoted by (-)
A.1 Proofs of Propositions
Proof of Proposition Let 0" be solution the Experience Replay (ER) optimization problem given by Eq. (4)
min (lelf(Xe:: (), Y), Ya) + (fx) YM))
where À > This means for any 8: (x(s(Xe;0)) 0%), + (exr 0*), Ym) Le([((X Y.) (M; Yas) Let (0) (o(f(xoi 0), and f2(0) (m Ym). The ER objective to minimize
(14)
+ (1(0)+ A
We want show that 0* is also solution to the following e-constraint problem as defined in Eq. 5
min (1)
(15)
s.t. [2( - [2(0 S (k)
where the slack is chosen as E (k) = f2(00) f2)0=-1). The constraint can be written as f2(0) S f2(0").
Assume, for the sake of contradiction, that 8" is not a solution to this e-constraint problem. Then, there must exist some 0 such that:
fi(0) < fi(0))
(16)
T2 f2(0a-1) S (k)
Now, consider the ER objective function for 0:
f1(0) + (f) < f1(()) Af2 < (0") + (f²
(17)
This implies (0) + (f) f00) \02) which contradicts the assumption that 0* is
solution to the ER optimization problem (Eq. (14)). Therefore, #" must be solution to the specified c-constraint problem with E (k) Em (f(X Mr: 8*), YMr) Exs (f fx-1), Ya.).
Assumptions for Proposition 2
13

---

1. The loss function for the current task, (c(f(Xx;:0) Y2), denoted as Ec(0), is convex and differentiable function of 6.
2. The average loss function memory samples, (f YmJ), denoted as las (0), is a convex and differentiable function 0.
3. A suitable constraint qualification (e.g., Slater's condition) holds for the e-constraint opti- mization problem defined in Eq. (6). Slater's condition implies that there exists a 0 such that As Mi 0), YM) Eas (f (f(X 011),, Yaa)= (k))
Proof of Proposition Let H* be a solution to the e-constraint optimization problem given by Eq. 6
())
s.t. (f KM) - (f S Ek)(k)
Let fi(0) ( 0), Y2) and 91 (0) ( Yms) The problem is ming (0) subject 91 (0) 0
MM) -M (k).
Under the stated assumptions (convexity, differentiability. and constraint qualification), the Karush- Kuhn-Tucker (KKT) [Boyd and Vandenberghe 2004) conditions are necessary and sufficient for optimality. Thus, there exists a Lagrange multiplier
2 such that for 0":
1. Stationarity: V/((0)) + (0")
2. Primal feasibility: 91 (1²) S
3. Dual feasibility: X
4. Complementary slackness: (09) =
The Lagrangian function for this problem is L(0, A²) = (1(0)+X')(00)) Substituting the definitions of (0) and (0):
L(O, = (0 (Ox10 (k)
The stationarity condition (0)
Vlo(0)) + A ( =0
Since (0) and Emr()) are convex, and X" 2 0, the function L(O, A*) ) is convex in 0. For a convex function, the stationarity condition VoL(O", . 0 sufficient for 0" be global minimizer of L(O, X*). Thus, 0* minimizes (0) + (0) - -Om(O=-)) =k)(k)). This is equivalent to (k).. Since (0x-) and x (k) are constants for the current optimization step k, the terms -A and -X (k) are also constants. Therefore, 8* also minimizes the function (0) + X (O). which is the objective function of Experience Replay (Eq. 4
minimizing (0) + (0) (0x-1
A.2 Properties of the GEC
We analyze the behavior of the GEC update mechanism by considering continuous -ime analogue, where the parameter update follows di, Ut, with i2 being the solution to the QP in Eq. 10)
The GEC update direction is 82 (Ox) + d VL.. (fx). Let h(,,) Lm (0.) em (0k-1)
be the constraint function, and (0,) = (h,) be the dynamic barrier. For brevity, we use (0.). I. (9,), Vle, Vmn. d and vt. The KKT conditions for the QP (Eq. (10) imply that d 0 and
Xe((Vem (0.), vz) - (0.)) = O.
Following [Gong and Liu 2021], we define penalty function (0t) (o.) + m(h(0,)) where [x]+ a 0) and f 2 The time derivative of Pa (0.) can shown to be:
d
di P. < || (n Ax) John (0:)l+ - de -e
(18)
This can be written as मे P. - (0,, Ao) where (,, Ae) = ||vell + (ohn (0.) de (0)], is KKT-like score function.
14

---

Property 1: Constraint Violation Reduction. The thresholded constraint function [h(O:)]+ is non-increasing with time
Proof of Constraint Violation Reduction. Consider the derivative of the penalty function (0s) = (o((:) + plh(6:)]++ If
((t)) > O, then [h(Bt]]+ = h(@,)) and Φn (8.) = ac((.)) > O, so Φ (0:))+
on (0c) and (0)) +
The derivative is de (((e) (V)) + v (0:), vt). Using
(l.(0)) = et - Xe Ven m (Oa):
d
P. ((.)
\. (0) + (0)) vz)) -0 = -1luee1 - (V)) vz)
di
From the QP constraint, (vem (0z), vx) 2 (0)). Ar > KKT implies (Vém (0t), 02) = =0,)
If Xa = 0, then (Vem (0,), 0x) 2 ob (0,) still holds. So, & (0.))) S 11102|12 2 (n Xe) (0.)
-11va]1 12 (1+ (0) (0+)]+
If h(0,) < 0, then [h(00)]+ 0, so Po() = lc(Ot). Also, (l.) S 0. so (0.)l+ 0 and
| (0,)l++ = -n (0.).
dt (()))) = dt (c(e)) = - (VVE((O)). mn)
= =-1lual12 + >x< (0.), Ut}
If A >0. (Vem (0,), 0²) (0.). A = 0. 4 = (f.), and (x(+) = 0 = (O)). So this holds generally. ((.)) = -1 1|2 + Xeds, = -
A (0+)l+
Combining both cases (h(O)) > and ((O,) < 0), we obtain the unified incquality in Eq. (8)) which holds for all values of 0,. Now, to show [h()+]] is non-increasing, we can analyze its derivative:
[h(0))] d
d h(0,)
if h(0,) >
dt
= 0
if 1(0g) < 0
(19)
Alternativel,, from Eq. 18). dividing by and taking
+ +00:
lim 1000 pa dt (0.) = dt [b((0)))
And from the RHS of Eq. (18)
(- ||va||2 ( 2x) (obn (0:)) - de ( (00)) -(dd)(00)]+ lim 11000 p
1
Thus,
d [h(00)++ S (())
dt
Since on (0.) = (O:):):
(((.)) > 0, then th (0:) > 0, so (dh (0:)] > 0. This means 2: [A((0:)] + < 0 when h((0)) > 0. If h(0))) S O, then Φn (0:) - 0, so (dh, (6+)]+ = O. This means dt [h((0)) S O.
Therefore, [=. always non-increasing. This implies that if the constraint is violated (h(O.) >
0). the algorithm works to reduce the violation. If the constraint is satisfied (h(O,) S 0), it does not become violated.
Property 2: Current Task Optimization when Constraint is Satisfied. If the constraint is satisfied (i.c., ((.))) < 0), the current task objective (0.)) non-increasing with time t.
Proof of Current Task Optimization when Constraint is Satisfied. When I((Oe) < 0, we have
6n(0,) 0. This implies [dh (b))0 0. Consider Eq. 18 with μ 0. In this case, Po(O.) = (O.).
d (o(1)) < - -|/1211 - (0- - Xx) (ox - de (m)))
dt
=-1/vel1 2 + >x 0 d (bb))
= =- - >x (0.04
15

---

Since A 2 by definition (Eq. 13 ) follows that:
l 2 0, and [-))(6.)] 2 0 (because d% (0:) - 0), it
d (0.) 0
Thus, when the memory constraint h((;)) S is satisfied, the loss the current task (o(O,) is non-increasing, meaning GEC focuses on optimizing the current task.
The theoretical properties established above naturally extend to the multi-constraint formulation used our implementation. When considering per-task constraints h²( = (, eaet for each past task f, the constraint adherence property holds independently for each task, ensuring that [hr (0)]+ is non-increasing for all t. The objective minimization property similarly extends: when all constraints are satisfied (i.e., h, (0) S 0 for all t), the algorithm prioritizes minimizing the current task loss. The multi-constraint QP solution aggregates the per-task adjustments through the La- grange multipliers X providing a principled way to balance multiple competing constraints while maintaining the convergence guarantees of the single-constrain case.
B Detailed Related Works
B.1
Continual Learning
Continual Learning (CL) aims to enable models to learn sequentially from a stream of data without catastrophically forgetting previously acquired knowledge (De Lange et al. 2021). Various strate- gies have been proposed to address this challenge, broadly categorized into regularization-bast rehearsal-basse and architecture- baasee methods. Our work primarily intersects with rehearsal- based methods in the context of Online Continual Learning (OCL).
B.1.1 Regularization-basee Methods
Regularization-baseee approaches introduce additional terms to the loss function penalize changes parameters deemed important for previous tasks. Elastic Weight Consolidation (EWC) (Kirk- patrick et al. 2017] computes Fisher Information Matrix to estimate parameter importance. Synap- tie Intelligence (SI) Zenke et al. [017]] calculates importance weights online based on the sensitivity of the loss function to parameter changes. Learning without Forgetting (LwF) (Li and Hoiem| 2017 uses knowledge distillation, forcing the model to preserve the outputs of the previous model on current task data. While effective in some scenarios, these methods can be conservative and may struggle with long task sequences or significant domain shifts.
B.1.2 Replay-based Methods
Replay-based (or rehearsal-based) methods store a small subset of samples from past tasks in memory buffer and interleave them with current task data during training [Rolnick et al. 2019] Ex- perience Replay (ER) [Rolnick et al. 2019] is a foundational technique that simply adds the loss on replayed samples to the current task loss. More advanced methods build upon ER. Dark Experience Replay (DER) [Buzzega al. 2020 and its variant DER++ enhance ER by also replaying logits. CLSER Arani et al. 2022 focuses on improving sample selection for the replay buffer.
Constraint- based replay methods, such as Gradient Episodic Memory (GEM) [Lopez-Paz and Ran- 2017) and Averaged GEM (A-GEM) [Chaudhry al. 2018). use memory samples to constrain the gradient updates for the current task, aiming prevent interference with past task knowledge. These methods typically enforce that the loss on past tasks does not increase. As discussed in our main paper, our proposed GEC method offers a new perspective by explicitly formulating the OCL update as an e-constraint optimization problem, dynamically adjusting the trade-off between current task learning and memory constraint satisfaction, which contrasts with the fixed weighting of ER or the hard constraints of GEM/A-GEM.
B.1.3 Online Continual Learning
Online Continual Learning (OCL) represents particularly challenging yet realistic CL scenario MMi et al., 2022 Aljundi et al.] 2019]. In OCI,, data arrives a stream, often as small mini- batches, and is typically processed in a single pass. Memory for storing past data is strictly limited.
16

---

This setting magnifies the problem of catastrophic forgetting and demands methods that are both computationally efficient and effective at balancing plasticity (learning new tasks) and stability (re- taining old knowledge). The significance of OCI. lies in its direct applicability to real-world systems where data is inherently sequential and non-stationary, and computational resources are constrained. Our GEC method is specifically designed for the OCL setting.
B.2 Multi-Objective Optimization Methods
Continual learning can be viewed as multi multi-objecctvv optimization problem (MOP) where the goals are minimize the loss on the current task and to minimize forgetting of past tasks. These objectives are often conflicting. Our work leverages concepts from MOP, particularly the e-constraint method.
B.2.1 Preliminaries for Multi-Ohjective Optimization
A general multi-objective optimization problem (MOP) can be formulated as:
minimize
(f(x) (x)) } subject to S.
Here, k 2 2 is the number of objective functions fi R² + R. The vector x = (Æ1, T2.... IT,)² consists of n decision variables, belonging to non-empty feasible region C RR.. The
f(S) = [f(x) x S} S RR is the feasible objective region in the objective space. An element z f(x)
an objective vector.
Definition 3 (Dominance). An objective vector z² = f(x+) is said to dominate another objective vector x²2 f(x²) (denoted z x²) f((xt) S [x(x2) { for all I,...,k k and [x(x²) (x²2 for at least one index j.
Definition 4 (Pareto Optimality). A decision vector x" E S is Pareto optimal if there is no other decision vector x E S such that f(x) dominates f(x*). Equivalently, x" is Pareto optimal if there is S such that fi(x) S fx(x)) for all i 1....,k and fx( < fy for at least one index j. The corresponding objective vector z = f(x") is a Pareto optimal objective vector. The set of all Pareto optimal objective vectors forms the Pareto front.
Definition 5 (Ideal Objective Vector). The ideal objective vector x² = (zi....; =t) is formed by components zi, where each z is the minimum value of the i-th objective function fi(x) considered
independently over the feasible region S, i.e. 2 minges f((x) for 1,...,h. k. The ideal objective vector is generally not attainable simultaneously for all objectives.
Definition 6 (Utopian Objective Vector). utopian objective vector z** is an infeasible vector strictly better than the ideal objective vector. Its components are typically defined as 2," 2 for all I.....k, where > 0 are small positive scalars.
B.2.2 Scalarization Techniques
Scalarization methods transform a MOP into single-objective optimization problem (or a series of them). Common techniques include the linear weighting method, the Chebyshev method, and the e-constraint method.
Linear Weighting Method This method combines all objective functions into single scalar ob- jective by assigning a non non-negative scalar weight w; E IR to each objective fx().. The problem is formulated as:
minimize
Wi (x()) subject to x E S.
The weights w, E R are non-negative scalars (w;; 0) and are typically normalized such that w; Σx=
Different weight vectors can yield different Pareto optimal solutions. This method simple but may not find all Pareto optimal solutions if the Pareto front is non-convex. As shown in our paper (Proposition 2), under certain conditions, an e-constraint problem solution can be found by an ER-like weighted sum.
17

---

Chebyshev Method The Chebyshev method [Miettinen 1999| aims to find solution that mini- mizes the maximum weighted deviation from reference point often the utopian objective vector 2" (or ideal x²), The problem is:
minimize
max [[x((((x)- -[()] subject to E S.
1=
Here, w; 0 are positive weights, and ztd is the reference point (c.g., z** z²). If = z=* then f((x) 21 t is positive for feasible solutions. This can be reformulated as a differentiable problem:
minimize
a
subject to wi(f((x) - 210) for all i 1, I,...,
E
where a E IR is an auxiliary variable. The Chebyshev method can find any Pareto optimal solution, regardless of the convexity of the Pareto front, by varying weights and the reference point.
c-Constraint Method The e-constraint method (Miettinen) [999 which is central our GEC approach, optimizes one objective function fx(x) while treating the other k 1 objective functions as constraints, bounded by user-defined values Ey:
minimize
fe(x)
subject to
(x) S x S.
for all j (l,...,k).2 + t
By systematically varying the choice of fi and the values of Ej. different Pareto optimal solutions can generated. This method is also capable finding Pareto optimal solutions in non-convex problems and offers direct control over the trade-offs via the Ey values. Our paper (Proposition IT) shows that ER can be interpreted as implicitly solving an e-constraint problem. GEC makes this explicit.
C Additional Results
In this section, we provide supplementary results for Average Anytime Accuracy (AAA) and Av- erage Accuracy (Acc) that include the standard deviations over 5 runs. These tables complement Table [ in the main paper, presenting the mean standard deviation. To further validate the robust- ness of GEC, we also conduct experiments in more challenging scenarios, including label noise and longer task sequences. The results consistently demonstrate GEC's superior performance.
Table 4: Average Anytime Accuracy (AAA) comparison with baseline methods Seq- Teq-CIFARIO Seq-CIFARI00, and TinyTTllemmeeeer using Reduced ResNet-18. Results are averaged over runs
(mean zs0h.
Seq-CIFARIO (N=5)
Seq-CIFAR100 (N=20)
Seq- TogTTinllmageNe (N=20)
Method
MI 0.6k
[M] 1k
[M]
[M
5k
[ 2k
M 5k
SGD
34.04.310
34.04.3.00
9.67
9.67
7.63
7.63
54.68
54,91
17.8610.8
20.67 10.8
16.27
16.101.24
Refresh CL
62,34
65.81
24.32
35.61 1:000
18,591100
22.68
DER
49.06
48.25
10,9610.00
10.47 10.32
8 8.04
7.65
DER+
57.17
61.01
17.30
17,08
12.42
11.93
CLSER
61.64
63.27 1:711
22.58
23.25+1..4
18.50=000
18.88
CBA
63.41
65.47 2007
22.462051
23.07
18.79 2005
18.98
POCL
63.624226
66.23
26.68
36.34
21. 21.56=244
25.482007
EWC
36.51
36.51 10:5
9.87
9.87
7.962a12
7.961am
GEM
37.78
37.00
13.43
13.71
10.17.00
10.27.10m
A-GEM
37.67
37.62.1.53
10.61 1:.00
10.80
7. 7.66.
7.79
GEC(Ours)
65.21.220
69.42.1.00
29.6310.0
38.11 11.10
22.45 11.45..1.00
11.40
27.66
18

---

Table 5: Performance on 40 tasks sequence of Seq-
Table Performance under various label noise settings on Seq-
CIFARI00 (IM) = 5k).
TinylmageNer.
Flip1 (0.05)
Flipi (0.2)
Flip2
(0.05)
Uniform (0.2)
Method
Method
AAA
Acc
AAA
Ace
AAA
Acc
AAA
Acc
AAA
Acc
SGD
4,95=006
1.19
DER++
24.16
15.40
21.80
12.84
22.88
14.83
20.94
12.28
ER
13.75
6.82ian
CLSER
21.69
13.34
19.18
10.69
21.96
10.53
19.62
10.47
DER+
2.39
3,76
POCL
22.38
15.41
18.99
11.41
21.59
11.30
18.82
10.06
CLSER
14.931cm6
7.74
POCT.
18.41144
10.64gem
GEC (Ours)
26.76
16.50
22.11
11.37
25.88
15.63
21.05
11.53
GEC (Ours)
25.37
11.89.um
Table 7: Average Accuracy (Acc) comparison with baseline methods on CIFACIFARIO, Seq-
CIFAR100, and Seq- TinylmageNet using Reduced ResNet- 18. Results averaged over 5 runs
(mean znd.
Seq-CIFARIO (N=5)
Seq-CIFARI00 (N=20)
Seq- TinyemageNet (N=20)
Method
[M] 0.6k
|M| 1k
[M| = 1k
[M|
5k
[M] 2k
[M] 5k
SGD
16.681027
16,681027
3.24
3.24
2.172007
2. 2.17:
ER
39.43
42.041433
11.89
14.87
10.52
11.892mm
Refresh CL
51.42.120
55.57
17.64
32.33
12.83
18.72
DER
25.801ns1
23.9010.77
3.71
3.68
2.46.a0s
2.04
DER+
47.03+
+1.50
8.72+a52
8.98
5.57=an
5,26#021
CLSER
50.36
53.06#1.58
15.68
16.42
10.03
11.61
CBA
51.47+137
52.08
15.54
15.87
11.43
10.98
POCI.
53.42
58.50
16.5420.4
33.3621199
12.69
19.40=a9
EWC
18.37
18.37 20:30
2.77
2.77
2.43
2.43
GEM
18.84
18.73
2.04
6.4610m
3.702a4
3.81
A-GEM
18.51+007
18.03+404
3.75+0.18
3.52+0.13
2.33+a1s
2.40+0.2
GEC(Ours)
54.08
59.12++200
17.78
34.22+1.50
12.77
20.25
D
Algorithm Details
This section provides the pseudo-code for the proposed GEC method and list of key symbols used. Table lists the key symbols used in the GEC algorithm.
19

---

Table 8: List of Key Symbols in GEC Pseudo-code
Symbol
Description
Model parameters. Learning rate.
7
OF
Barrier strength parameter for GEC.
8
Slack tolerance for the memory constraint. Stability constant for GEC denominator, Memory buffer storing past samples.
6
M
T
Total number of tasks.
k
Index for the current update step/iteration. Mini-batch of data from the current task.
(Xe,
(Xm: Ym)
Mini-bated of data sampled from the memory buffer M.
E(0; X,
Loss function evaluated on data (X, Y) with parameters Gradient of the loss on the current task batch, VoL(O; X Yo).
Ve.(0)
(0)
Average loss on the memory batch, AvgLoss((:; Xm: Ym))
(l)
Gradient of the average loss on the memory batch, Vo (0).
gee
Reference average memory loss (value from the start of the previous update step). Constraint violation measure at step k: Dynamic barrier value step k.
A
Dynamically computed coefficient for the memory gradient at step k.
Pk
Final update direction at step k. Task labels stored in memory buffer.
Tm
T
Set of unique past tasks in the current memory batch.
Ve,(0)
Gradient of loss for past task t.
Oh,t
Per-task dynamic barrier value.
Per-task Lagrange multiplier from multi-constraint QP.
N
Number of warm-up steps.
Algorithm Daciiimmoutttnii outlines the training procedure for the GEC method.
20

---

Algorithm Gradient-Guided Epsilon Constraint (GEC)
Require: Learning rate strength parameter a >> 0, tolerance s, constant 5 > 0.
Ensure: Final model parameters O.
1: Initialize model parameters 6.
2: Initialize memory buffer M < Ø.
3: Initialize per-task reference losses ( --11
St=0 {0}.
4: Initialize step counter A: +- O.
5: for each task Teureet 1, ..... T do
6:
for each incoming batch (Xe, Ye) from task Teurrent do +- ++1
7:
8:
if k S Na or M is empty then
9:
Compute (l) and update 0
(()
10:
else
Sample memory batch (Xm, mm Tm) from M Identify unique past tasks T = unique(Tm) for each past task E T do
12:
13:
14:
Extract task-t samples: (X Mm) {x,, y) : Tm 1 Compute per-task look-ahead gradient (() and loss (0)
15:
16:
Compute barrier: ob,t + a(l((0) C.at end for
17:
18:
Compute current task gradient (l.(0)
19:
Solve multi-constraint QP for {}e |ET
20:
ming / 2/1e ( s.t. (f), 0) 2 th,t, Vt
21:
Compute update: v +- (O) + 2IET de (() Update parameters:
22:
0 170
23:
Update reference losses: frel + (0), VIET
24:
end if
25:
Add (Xe: Ye, Teureet) to memory M via reservoir sampling end for
27:
Record task completion loss for threshold calibration
end for
29:
return A.
E Experimental Setup Details
E.1
Datasets
We evaluate our GEC method on three standard Online Continual Learning (OCL) benchmarks. The details of these datasets and their configuration for our experiments are summarized in Table all datasets, standard data augmentation techniques are applied during training.
9 For
Table 9: Details of datasets
Dataset
Total Classes
Tasks (N) Classes per Task
Training Images
Test Images
Resolution
Seq CIFARIO
10
2
50,000
10,000
32 x 32
Seq- Ceq-CIFARIIE 100
100
5
50,000
10,000
32
32 x
Seq- TinylmageNet
200
100,000
10,000
32 x 32
x
E.2 Implementation Details
Consistent with recent works [Buzzega et al. 2020 Chrysakis and Moens 2023], we use Reduced ResNet-18 (He et al. 2016) as the backbone network for experiments. The models are trained using SGD. The learning rate is set to 0.03 for GEC and all baseline methods. The streaming batch size and replay batch size are set to for all experiments. The memory buffer M updated using reservoir sampling.
21

---

GEC Hyperparameters. For our GEC method, the barrier strength parameter is set to 0.5, and the stability constant is 10 We employ dynamic threshold mode for E: the completion of each task, the average training loss is recorded as a reference, and the threshold is set to allow a small margin (5%) of degradation from this reference value. This adaptive calibration naturally accommodates the varying loss scales across different tasks.
Calti-Consttrint Extension. In our implementation, we extend the single-constran formulation presented in Section to multi-constrail setting for improved practical performance. Specifically, when task labels are stored alongside samples in the memory buffer, we compute separate gradients This leads to multi-constraint quadratic
and constraints for each past task /. E 10.,,.... --11 11. program:
U/- arg min {
-
s.t. (V((0))))) (0k, 4A < T current
(20)
VERd
(fk) is the per-task dynamic barrier. This multi-constraint QP is solved efficiently using standard quadratic programming solvers. The solution takes the form V)- = (l((0)) E, AR. (&)())), where A 2 0 are the Lagrange multipliers obtained from the QP solution. This extension preserves the theoretical properties discussed Appendix AA.2 while pro- viding finer-grained control over per-task forgetring.
where (0k))
Following [Wu et al. 2024b Gupta al., 2020 Wu et al. 2024a], we adapt a look-ahead gradient computation strategy to better assess the effect of updates memory performance. Specifically, we perform an inner-loop gradient step on task-specific data before computing the outer-loop gra- dient used for the constraint evaluation, which provides more accurate gradient information for the constraint satisfaction mechanism, particularly in the early stages of learning each task. Since the dynamic threshold mechanism requires stable reference loss for calibration, warm-up phase using standard experience replay for the first update steps when learning each new task. This allows the memory buffer to accumulate sufficient samples and establishes stable gradient estimates before the full GEC constraint mechanism is activated.
apply brief
To ensure numerical stability, gradients are clipped before storage and computation. This prevents extreme gradient values from destabilizing the QP solution. All experiments are conducted over independent runs with different random seeds, and we report the mean and standard deviation of evaluation metries. The experiments were performed on 8 NVIDIA A100 GPUs.
F.3 Baseline Method Descriptions
We compare GEC with a comprehensive set of baseline methods. Key rehearsal- based and constraint-based methods include:
E.3.1
Foundational and Regularization Methods
SGD: Standard Stochastic Gradient Descent training current task data without any spe- cific mechanism to mitigate forgetting. This serves as lower-bound baseline.
Elastic Weight Consolidation (EWC) (Kirkpatrick al., 2017 Huszár 2018: EWC is a regularization-ba based method that penalizes changes to parameters deemed important for previously learned tasks. Importance is estimated using the diagonal of the Fisher Information Matrix.
E.3.2 Replay-based Methods
Experience Replay (ER) [Rolnick et al. 2019: ER trains on combination of current task data and data sampled from memory buffer that stores examples from previous tasks. The model is typically trained on composite loss, often a weighted sum of the current task loss and the loss the replayed memory samples.
Dark Experience Replay (DER & DER++) [Buzzega et al. 2020 DER and its variant DER++ enhance ER by additionally replaying network logits (outputs before the softmax layer) from past tasks. This is usually achieved by incorporating a knowledge distillation loss based on the stored logits, aiming to better preserve the previous model's predictions on past data.
22

---

CLSER |rani et al. 2022 CLSER proposes a dual memory experience replay (ER) method which maintains short-term and long-term semantic memories that interact with the episodic memory.
Refresh CL [Wang et al., 2024]: : Refresh CL proposes general framework for CL and
introduces refresh learning mechanism (unleam-releaam) inspired by neuroscience. aims to shed outdated information to improve retention of crucial knowledge and facilitate new learning, acting as a plug-in for existing CL methods.
E.3.3 Constraint-based Methods
Gradient Episodic Memory (GEM) [Lopez-Paz and Ranzato 2017]: GEM uses samples from the memory buffer to constrain the gradient updates of the current task. aims to prevent any increase in the average loss memory samples by projecting the current task's gradient if it conflicts with the objective of not increasing memory loss (i.e., the dot product between the current task gradient and memory gradient negative).
Averaged GEM (A-GEM) [Chaudhry et al.) 2018]: A-GEM is a more computationally efficient variant of GEM. It computes a single constraint based on the average loss over all samples in the memory buffer, rather than per-task constraints, and projects the current task gradient it would increase this average memory loss.
E.3.4 Online Continual Learning Methods
Continual Bias Adaptor (CBA) [Wang al. 2023 CBA introduces a module to adapt to distribution shifts during training in online CL. It augments the classifier to handle catas- trophic distribution changes and can be removed during testing, incurring no extra compu- tational cost at inference.
Pareto Optimized Continual Learning (POCL) Wu et al. 2024b): POCL explicitly treats OCL as a multi-objective optimization problem, aiming to find solutions on the Pareto front that balance plasticity and stability. It often involves techniques to steer the optimization towards desired trade-off regions.
F
Limitations
While the GEC method demonstrates strong performance in OCL benchmarks, it has certain limita- tions. One primary consideration the computational overhead introduced by solving QP problem at each training step to determine the update direction UY. Although the QP is relatively small and has a closed-form solution as shown in Eq. (13), this still involves additional computations compared to simpler methods like ER. Future work will focus on investigating more computationally efficient approximations to achieve dynamic constraint- guided update with reduced overhead, making GEC even more suitable for resource-constraing online learning scenarios.
G Broader Impact
The proposed GBC method contributes to the field of Online Continual Learning, aiming to develop AI systems that learn sequentially and adapt to evolving data streams more effectively. Success in this area has significant positive implications for real-world applications where data is inher- ently dynamic and non-stationary. For instance, GEC could enhance the capabilities of autonomous systems (e.g., robotics, self-driving cars) to learn from new experiences in real-time without forget- ting previously learned skills. personalized learning or recommendation systems, could allow models to adapt to changing user preferences over time. In healthcare, it could enable diagnos- tic models to incorporate new medical knowledge or patient data continuously. By improving the stability stability-plastiiity trade-off, GEC can lead to more robust, reliable, and adaptable AI. While the di- rect societal risks of this specific algorithmic improvement are low, as with any advancement in AI, responsible development and deployment are crucial. The goal of this research is to advance Al's learning capabilities, which can be a powerful tool for societal benefit when applied thoughtfully and ethically.
23

---

NeurIPS 2025 Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly articulate the core idea of the GEC method and its advantages in OCL.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The discussion of computational efficiency and strong assumptions of GEC methods are in Appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech- speech-to-text system might not be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
24

---

Answer:
[Yes]
Justification:
The paper presents Propositions I and 2 in Section 2, stating that proof' are
provided in Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data provided or not)?
Answer: [Yes]
Justification: Section 5.1 details the experimental setup, including datasets, evaluation met- rics, backbone network, optimizer and other Key hyperparameters for GEC implementaion.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurlPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
25

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have provided instructions in the paper on data access and we will release the code implementation.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps::/niis.cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" am acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.. /nips.cc/publl cc/ppblic/guidesss Sodde/dSddes/odeuumm wwtp///odduiisiiooooooooooo ssionnolicy)) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please kindly refer to Section 6.1.
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification: All results include values with +, implying statistics over multiple runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
26

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g,, Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please kindly refer to Section 6.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///aaarrp.cccppuuuc.cccccaa
Answer: [Yes]
Justification: The research does not involve human subjects, does not directly handle sen- sitive personal data (uses public, processed datasets), and has no apparent goal of causing direct societal harm (e.g.. weapon development, discriminatory applications).
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We have include the discussion of broader impacts in Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
27

---

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The proposed model do not fall into the category of pre-trained language models image generators with high misuse risk, thus not requiring specific safeguards.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have mentioned the datasets used (Seq-CIFARIO, CIF-CIR,,, Seq- TinylmageNet) and the backbone network (Reduced ResNet- 18), referencing relevant works.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
28

---

If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, wwtpppppppppoooooo
com/datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification:
The main contribution of the paper is new algorithm.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects. and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research presented in the paper does not involve human subjects and therefore does not require IRB approval or equivalent review.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
29

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs are not component of our core methodology.
Guidelines:
The answer NA means that the method development in this rescarch does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy lhttps://neurss cctcoonfeeeecccs/225///////////e for what should or should be described.
30


Reviews:
[{"review_id": "6TEY2xpCmX", "reviewer": "Reviewer_bPNr", "summary": "This work investigates the online continual learning (OCL) problem and revisit the classical strategies of Experience Replay (ER), and constraint-based optimization methods such as GEM and A-GEM. The paper first show that ER can be formulated as a soft-constraint optimization problem, revealing its underlying behaviours of why it could achieve a better performance. From there, the authors proposed Gradient Guided Epsilon Constraint (GEC) which dynamically adjusts the slack variable within this optimization framework. Experiments on common OCL benchmark show encouraging results of GEC.", "questions": "- How would GEC perform on more challenging benchmarks?\n- Could you clarify the interaction between GEC and BN and compared it to ER?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n- I like the theoretical analysis in this work. Although not too complex, it is original, easy to follow, and provided useful insights. \n- The proposed GEC algorithm is well-motivated and seems to work well on common benchmarks.\n- The authors conducted extensive ablation studies and also reported the training complexity.\n\nWeaknesses\n- My major concern of this work is the benchmarks considered are quite outdated, and small in scale. It would be more interesting to validate the algorithm on more challenging scenarios, such as the long-tail setting [A], on larger datasets (e.g. ImageNet), or in NLP/RL/Robotics domains.\n- How does GEC interact with the BatchNorm (BN) component in during OCL? Specifically, the original ER mixes current and previous data samples in a mini batch and perform a single forward to update. In contrast, GEC performs two forward passes on the current task and previous task data, which results in different updating dynamic of the BN's global statistics than ER. A prior study [B] showed that this difference may impact OCL. \n\n[A] Kim, Chris Dongjoo, Jinseo Jeong, and Gunhee Kim. \"Imbalanced continual learning with partitioning reservoir sampling.\" Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16. Springer International Publishing, 2020.\n\n[B] Pham, Quang, Chenghao Liu, and Steven Hoi. \"Continual normalization: Rethinking batch normalization for online continual learning.\" ICLR, 2022.", "was_revised": true, "final_justification": "Overall a solid work, and the author addressed my concerns in the rebuttal. Therefore, I recommend acceptance.", "rebuttal": "We sincerely thank the reviewer for providing valuable feedback. We are encouraged that the reviewer found our theoretical analysis \"original, easy to follow, and provided useful insights\" and the GEC algorithm well-motivated. We detail our response below point by point.\n\n**W1/Q1: Performance on more challenging benchmarks**\n\nWe thank the reviewer for this insightful suggestion. To demonstrate GEC's robustness, we conducted extensive new experiments in three challenging settings. Blew are the three experimental settings:\n\n* Following [4], we introduced long-tail distributions to Seq-CIFAR10 and Seq-CIFAR100 with varying imbalance factors ($\\gamma=2$ and $\\gamma=5$) and distribution orders (Normal, Reversed).\n* Following [3], we introduced three types of label noise ('flip1', 'flip2', 'uniform') at different corruption ratios to evaluate robustness.\n* Following [5], we re-split Seq-TinyImageNet from 20 to a more challenging **40-task** sequence, denoted as **Seq-Tiny-l**.\n\nWe compare GEC against strong SOTA baselines (DER++[1], CLSER[2], POCL[3]).\n\nTable 1: Performance on Long-Tail Distributions\n| Setting (γ) | Dataset | Metric | DER++ | CLSER | POCL | **GEC** |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| **γ=2** | Seq-C10, Normal | AAA | 61.97 | 61.87 | 66.87 | **67.11** |\n| | | Acc | 44.04 | 48.04 | 54.82 | **55.43** |\n| | Seq-C10, Reversed | AAA | 58.52 | 55.32 | 64.45 | **64.47** |\n| | | Acc | 39.43 | 42.38 | 58.79 | **58.94** |\n| | Seq-C100, Normal | AAA | 23.45 | 22.57 | 24.13 | **26.98** |\n| | | Acc | 13.77 | 12.61 | 15.20 | **18.20** |\n| | Seq-C100, Reversed| AAA | 21.19 | 16.41 | 21.58 | **23.41** |\n| | | Acc | 14.54 | 11.47 | 14.16 | **15.43** |\n| **γ=5** | Seq-C10, Normal | AAA | 49.64 | 53.83 | 54.32 | **56.17** |\n| | | Acc | 41.22 | 35.72 | 42.13 | **44.06** |\n| | Seq-C10, Reversed | AAA | 49.16 | 51.37 | 52.52 | **54.15** |\n| | | Acc | 39.42 | 40.19 | 36.10 | **44.72** |\n| | Seq-C100, Normal | AAA | 22.84 | 19.63 | 21.70 | **24.99** |\n| | | Acc | 16.17 | 9.65 | 14.67 | **17.12** |\n| | Seq-C100, Reversed| AAA | 14.63 | 13.11 | 17.54 | **20.51** |\n| | | Acc | 11.14 | 7.34 | 12.58 | **16.03** |\n\nTable 2: Performance under Label Noise\n| Noise Type | Dataset (Ratio) | Metric | DER++ | CLSER | POCL | **GEC** |\n| :--- | :--- | :--- | :--- | :--- | :--- | :--- |\n| **flip1** | Seq-C10 (0.05) | AAA | 60.93 | 62.62 | 61.14 | **63.20** |\n| | | Acc | 49.60 | 52.63 | 51.17 | **52.75** |\n| | Seq-C10 (0.1) | AAA | 58.58 | 58.45 | 59.20 | **63.13** |\n| | | Acc | 44.84 | 44.30 | 47.96 | **52.37** |\n| | Seq-C100 (0.05) | AAA | 24.16 | 21.69 | 22.38 | **26.76** |\n| | | Acc | 15.40 | 13.34 | 15.41 | **16.50** |\n| | Seq-C100 (0.2) | AAA | 21.80 | 19.18 | 18.99 | **22.11** |\n| | | Acc | **12.84** | 10.69 | 11.41 | 11.37 |\n| **flip2** | Seq-C10 (0.05) | AAA | 63.21 | 61.19 | 62.11 | **66.03** |\n| | | Acc | 49.31 | 48.47 | 49.04 | **54.69** |\n| | Seq-C10 (0.1) | AAA | 60.66 | **65.15** | 60.90 | 62.90 |\n| | | Acc | 41.40 | **52.68** | 42.18 | 49.67 |\n| | Seq-C100 (0.05) | AAA | 22.88 | 21.96 | 21.59 | **25.88** |\n| | | Acc | 14.83 | 10.53 | 11.30 | **15.63** |\n| | Seq-C100 (0.2) | AAA | 22.26 | 19.15 | 19.83 | **22.32** |\n| | | Acc | 11.20 | 10.68 | 11.01 | **15.36** |\n| **uniform**| Seq-C10 (0.05) | AAA | 63.34 | 63.19 | 62.88 | **64.94** |\n| | | Acc | 49.18 | 50.87 | 49.21 | **51.72** |\n| | Seq-C10 (0.1) | AAA | 56.56 | 59.63 | 58.42 | **62.83** |\n| | | Acc | 45.00 | 45.37 | 43.11 | **48.12** |\n| | Seq-C100 (0.05) | AAA | 24.37 | 22.38 | 22.43 | **25.73** |\n| | | Acc | 15.22 | 12.00 | 12.87 | **17.30** |\n| | Seq-C100 (0.2) | AAA | 20.94 | 19.62 | 18.82 | **21.05** |\n| | | Acc | **12.28** | 10.47 | 10.06 | 11.53 |\n\nTable 3: Performance on a Longer Task Sequence (Seq-Tiny-l, N=40)\n| Method | AAA | Acc |\n| :--- | :--- | :--- |\n| SGD | 4.95±0.16 | 1.19±0.27 |\n| ER | 13.75±0.12 | 6.82±0.11 |\n| DER++ | 9.39±0.07 | 3.76±0.81 |\n| CLSER | 14.93±0.36 | 7.74±0.91 |\n| POCL | 18.41±0.14 | 10.64±0.33 |\n| **GEC** | **25.37±0.16** | **11.89±0.31** |\n\n* GEC's constraint is on the memory loss ($\\bar{\\ell}_m$) with a slack tolerance ($\\bar{\\varepsilon}$). This design naturally dampens the impact of high-variance gradients from imbalanced or noisy samples in the buffer. Unlike fixed-weighting methods (e.g., ER) which can be easily misguided, GEC's updates are more stable and principled (**table1&2**).\n* The compelling lead in the 40-task scenario, as shown in **table3**, highlights GEC's key advantage. By explicitly monitoring and correcting for forgetting, GEC prevents the gradual error accumulation that plagues methods with fixed trade-offs over long horizons.\n\nThese results confirm that GEC's is also effective for complex, realistic CL settings. We will include these results into the final paper.\n\n**W2/Q2: Interaction with BatchNorm (BN)**\n\nThis is a sharp and important observation. To rigorously isolate GEC's algorithmic contribution from potential BN effects, we conducted two controlled experiments.\n\n* We created a baseline, *ER (Dual-Fwd)*, that mimics GEC’s dual forward pass structure, ensuring identical BN update dynamics for a fair comparison.\n\n* We also replaced BatchNorm with the stateless GroupNorm (GN) for both ER and GEC like [6].\n\nTable 4: Performance comparison isolating the effect of BN dynamics.\n| Dataset | Metric | ER | ER (Dual-Fwd) | **GEC** |\n| :--- | :--- | :--- | :--- | :--- |\n| Seq-C10 (1k) | AAA | 54.91 | 55.70 | **69.42** |\n| | Acc | 42.04 | 44.67 | **59.12** |\n| Seq-C100 (5k) | AAA | 20.67 | 20.99 | **42.47** |\n| | Acc | 14.87 | 15.59 | **37.12** |\n| Seq-TinyNet (5k)| AAA | 16.10 | 16.88 | **31.36** |\n| | Acc | 11.89 | 13.11 | **20.25** |\n\n\nTable 5: Performance comparison with Group Normalization (GN) on Seq-CIFAR100 (|M|=5k).\n| Method | AAA | Acc |\n| :--- | :--- | :--- |\n| ER-GN | 19.85 | 14.42 |\n| GEC-GN | **41.03** | **35.88** |\n\n* **Table 4** shows that matching BN dynamics (*ER (Dual-Fwd)*) provides marginal gains over standard ER. This proves GEC's substantial performance leap stems from its core *gradient-guided constraint mechanism*.\n* **Table 5** further solidifies this. With GroupNorm, GEC still maintains its significant lead, demonstrating its effectiveness is independent of the normalization layer's statefulness.\n\n---\nWe hope these experiments have fully addressed your concerns. Your feedback has been invaluable in strengthening our paper, and we will incorporate these results into the appendix.\n\n[1] Dark experience for general continual learning: a strong, simple baseline. NeurIPS 2020\n\n[2] Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System. ICLR 2022\n\n[3] Mitigating Catastrophic Forgetting in Online Continual Learning by Modeling Previous Task Interrelations via Pareto Optimization. ICML 2024\n\n[4] Imbalanced continual learning with partitioning reservoir sampling. ECCV 2020.\n\n[5] Meta Continual Learning Revisited: Implicitly Enhancing Online Hessian Approximation via Variance Reduction. ICLR 2024\n\n[6] Continual normalization: Rethinking batch normalization for online continual learning. ICLR 2022."}, {"review_id": "xMcxOYHRs8", "reviewer": "Reviewer_ZVdS", "summary": "The paper reveals that experience replay (ER) in Online Continual Learning (OCL) implicitly employs a soft constraint with a fixed slack, thereby risking suboptimal updates and overfitting. We analyze ER as an ε-constraint optimization problem, revealing its limitations. We propose Gradient-Guided Epsilon Constraint (GEC), explicitly formulating OCL updates as an ε-constraint problem: minimize current loss while enforcing stability constraints on memory data. GEC dynamically adjusts the update direction using gradients: if memory loss exceeds a slack tolerance, it prioritizes constraint satisfaction; else, it focuses on the current task while controlling memory loss increase. Evaluations show GEC achieves a superior stability-plasticity trade-off, improving overall performance.", "questions": "1.How does the QP formulation scale with increasing task complexity or dataset size? Are there approximations (e.g., closed-form solutions) to reduce computational overhead?\n2.How does GEC perform in non-convex scenarios? What modifications are needed to handle non-convexity?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.The paper is well-organized, with clear theoretical derivations, intuitive visualizations and a logical flow from problem formulation to evaluation.\n2.The paper provides a novel perspective by reinterpreting Experience Replay (ER) as an ε-constraint optimization problem. This insight bridges the gap between rehearsal-based methods and constraint-based approaches, offering a unified framework for understanding trade-offs in online continual learning\n3.The proposed Gradient-Guided Epsilon Constraint (GEC) method introduces a dynamic, gradient-aware mechanism to balance plasticity and stability. By adjusting the update direction based on constraint violation, GEC avoids overfitting to the memory buffer while mitigating catastrophic forgetting. This design is both theoretically grounded and empirically validated.\n4.The experiments on standard OCL benchmarks demonstrate consistent improvements over state-of-the-art methods like ER, POCL, and CBA. The ablation studies and analysis of generalization gain vs. forgetting measures strengthen the claims.\n\nWeaknesses:\n1.The quadratic programming (QP) formulation in GEC may introduce computational overhead compared to simpler methods like ER. The paper briefly mentions training time comparisons (Table 3) but does not analyze how this overhead scales with larger datasets or more complex models.\n2.The analysis assumes convexity and differentiability of the loss functions. In real-world non-convex scenarios, the guarantees for constraint adherence and convergence may not hold.", "was_revised": true, "final_justification": "My concerns are well addressed.", "rebuttal": "We sincerely thank the reviewer for the insightful and positive feedback. We are encouraged that you recognized our paper's clear organization, the novelty of our $\\epsilon$-constraint perspective, and the strength of our proposed method and empirical results. You have raised two crucial points regarding computational overhead and the convexity assumption, which are vital for practical applications. We appreciate this opportunity to provide a detailed clarification.\n\n**W1/Q1: QP Computational Overhead and Scalability**\n\nThis is a critical point regarding the potential computational overhead of the QP formulation. We would like to clarify and emphasize a key aspect of our method that directly addresses this concern: the specific QP problem in GEC, as defined in Eq. (8), admits a closed-form analytical solution, which we present in Eq. (11).\n\nThis means we do not need to run an iterative QP solver, and no further approximations that might compromise performance are necessary. The computation of the optimal update direction *v_k* involves only vector operations that scale linearly with the number of model parameters (d).\n\nTo further quantify this and empirically validate your intuition, we conducted a surgical experiment on Seq-CIFAR10 to precisely isolate the costs of GEC's components. We compared standard ER, our full GEC method, and a variant *GEC (w/o QP)* which performs the dual forward/backward passes but uses a simple gradient addition instead of our dynamic QP logic.\n\nTable 1: Ablation on Computational Cost of GEC Components on Seq-CIFAR10 (|M|=1k)\n| Method | Pass | QP | Training Time (s) | Acc |\n| :--- | :--- | :--- | :----------- | :------ |\n| ER | Single | No | **86.31** | 42.04 |\n| GEC (w/o QP) | Dual | No | 132.89 | 45.17 |\n| **GEC (Ours)** | Dual | Yes | 147.62 | **59.12** |\n\nAnd we can see that the primary increase in training time arises from the dual forward/backward passes, which are essential to compute the constraint violation $h(\\theta_k)$. The additional cost of solving the QP via its closed-form solution is negligible. This efficiency is also reflected in Table 3 of our main paper, where GEC's training time is substantially lower than other advanced OCL methods like POCL and CBA. We will add more complexity analysis in Appendix in the final manuscript.\n\n**W2/Q2: Convexity Assumption and Non-Convex Performance**\n\nThis is another insightful point, and we appreciate the opportunity to clarify the precise and limited role of the convexity assumption in our work.\n\nYou are correct that our theoretical analysis uses a convexity assumption. However, it's crucial to distinguish where and why it is used:\n\n* First, the assumption of convexity is used **unidirectionally** in Proposition 2 to formally establish the equivalence **from** a standard ε-constraint problem **back to** ER's weighted-sum objective. This proof relies on the Karush-Kuhn-Tucker (KKT) conditions, where convexity is required for the conditions to be sufficient for global optimality [1]. This step serves to provide a rigorous theoretical grounding for ER's behavior within the established optimization literature.\n\n* The analysis of our proposed GEC method's properties (Section 5.1 and Appendix B.2) does **NOT depend on the convexity**  of the loss functions $\\ell_c(\\theta)$ and $\\ell_m(\\theta)$. Our theoretical guarantees (Constraint Adherence, Objective Minimization) are derived from the KKT conditions of the local QP problem defined in Eq. (8). This QP, which seeks the optimal update direction $v_k$ at each step, is always convex by its own construction (a quadratic objective with a single linear constraint), irrespective of the global non-convexity of the underlying deep learning loss landscape. This analytical approach is a standard tool for studying the behavior of optimization algorithms in non-convex settings [2] [3].\n\nTherefore, GEC as presented is designed for and validated in the practical, non-convex setting of deep neural networks, and thus no specific modifications are needed. Moreover, the SOTA performance of GEC across all our experiments using the non-convex ResNet architecture (Tables 1, 2 and results in our response to Reviewer bPNr) serves as the validation of its effectiveness in these non-convex deep learning scenarios.\n\n\n[1] Boyd S P, Vandenberghe L. Convex optimization[M]. Cambridge university press, 2004.\n\n[2] Su W, Boyd S, Candes E J. A differential equation for modeling Nesterov's accelerated gradient method: Theory and insights[J]. Journal of Machine Learning Research, 2016, 17(153): 1-43.\n\n[3] Miettinen K. Nonlinear multiobjective optimization[M]. Springer Science & Business Media, 1999."}, {"review_id": "pXeVNaR2pW", "reviewer": "Reviewer_b9bv", "summary": "The paper introduces the Gradient Guided Epsilon Constraint (GEC), an experience replay-based online continual learning method that formulates single-step parameter updates as an $\\epsilon$-constraint optimization problem. This enables the learning algorithm to dynamically adjust the strength of the replay buffer gradient during optimization to appropriately prioritize stability vs plasticity. The paper attains strong results on various class-incremental benchmarks.", "questions": "- Would the theoretical findings hold when $\\bar{l}_M$ is approximated with Monte Carlo as is the case in practice?\n- What were the values of $\\epsilon$ for the experiments, and how should they be set generally?\n- Do you have any idea why Figure 3a has GEC underperforming ER in plasticity early on for Seq-CIFAR100 while Figure 3c has GEC outperforming ER in plasticity for Seq-TinyImageNet?\n- What do the dashed lines mean in Figure 1a?", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper tackles the salient problem of dynamically balancing of stability and plasticity during online continual learning, which is important in the field of online continual learning that generally focuses on either stability or plasticity.\n- The paper's $\\epsilon$-constraint perspective of experience replay is not just novel but also useful, as it results in a strong method.\n- The paper is very well-written and has strong empirical results on class-incremental OCL benchmarks.\n- The experiment section is comprehensive, containing various benchmark results, breakdown of GEC's effects on stability and plasticity, computation details, and more.\n\nWeaknesses\n- The main text should refer to various appendix sections, as many useful details such as proofs and algorithms are present there but it is hard to reference them while reading the paper without direct pointers. In addition, a simplified related work section from the appendix should be included in the main text after the experiment section.\n- The paper should discuss and, time permitting, compare against recent work that optimizes the experience replay objectives by formulating a single parameter update step as a constrained proximal update [1,2]. These methods also employ the experience replay loss and slow and/or accelerate optimization in parameter directions that do not interfere with model predictions on the replay buffer data. One advantage of GEC that stands out is that it does not involve preconditioning matrix inversion.\n- MER baseline results are missing for Seq-CIFAR100 and Seq-TinyImageNet experiments in Tables 1 and 2.\n- Nitpicks\n  - The paper refers to the term \"posterior\" on multiple occasions, and it does not seem to be related to the Bayesian posterior. Please define this.\n  - Figure 3c caption should be Seq-TinyImageNet Plasticity, not Seq-TinyImageNet Stability.\n\n\nReferences\n\n[1] Yoo, J., Liu, Y., Wood, F. & Pleiss, G.. (2024). Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:57199-57216.\n\n[2] Urettini, E., & Carta, A. (2025). Online Curvature-Aware Replay: Leveraging 2nd Order Information for Online Continual Learning. arXiv preprint arXiv:2502.01866.", "was_revised": true, "final_justification": "My concerns and questions have been adequately addressed .", "rebuttal": "We are grateful for your thorough assessment and encouraging feedback. We are delighted that you found our $\\epsilon$-constraint perspective \"novel and useful,\" the paper \"very well-written,\" and the empirical results \"strong.\" Your insightful comments will help us improve the quality of our work. We address each of your points below.\n\n**W1: Appendix Referencing and Related Work**\n\nThis is an excellent suggestion to improve the paper's readability and self-containment. We fully agree and will implement the following in the final version:\n\n* We will add explicit pointers (e.g., \"*(see Appendix A.1 for proof)*\") throughout the main text to directly link to proofs, algorithms, and other details.\n* Also, we will add a new \"Related Work\" section to the main paper before the Conclusion. It will distill the most critical comparisons and provide context, including the discussion of the works you suggested [1][2].\n\n**W2: Comparison with Constrained Proximal Update Methods [1, 2]**\n\nWe sincerely thank you for bringing these recent papers to our attention. They represent an important and complementary direction in OCL, and a direct comparison helps clarify GEC's contributions. We will add a comparison in our new \"Related Work\" section:\n\n* First, GEC stems from a MOO perspective, explicitly balancing plasticity and stability via a dynamic $\\epsilon$-constraint [3]. In contrast, LPR [1] and OCAR [2] approach the problem from an optimizer design perspective, using proximal methods and second-order information to stabilize the optimization process.\n\n* Second, the GEC's update is dynamic. The coefficient $\\lambda_k$ is recalculated at every step based on the current forgetting level $h(\\theta_k)$. When forgetting is low ($h(\\theta_k) \\le 0$), $\\lambda_k$ can be zero, focusing entirely on plasticity. When forgetting is high ($h(\\theta_k) > 0$), $\\lambda_k$ increases to prioritize stability. In contrast, the preconditioning in LPR/OCAR remains fixed between updates.\n\n* As you pointed out, GEC is highly efficient. Our update has a *closed-form solution* requiring no matrix inversion. This is a significant practical advantage over methods that require inverting matrices.\n\n**W3: Missing MER baseline results**\n\nThe MER baseline was omitted for Seq-CIFAR100 and Seq-TinyImageNet due to its significant computational demands. As a MAML-based method, MER lead to prohibitive training times and instability on these longer benchmarks. We will add a note in the experimental setup to clarify this decision, citing [4].\n\n**Nitpicks**\n\n* Thank you for this sharp-eyed catch of the use of \"posterior\". Our use was in the non-Bayesian sense of \"after-the-fact.\" To avoid ambiguity, we will replace all instances with the more precise term **\"post-hoc\"**.\n* We are grateful for your meticulous reading of Figure 3c'scaption. The caption is incorrect and should be **\"Seq-TinyImageNet - Plasticity (Acc-P)\"**. We will correct this.\n\n**Q1: Would the theoretical findings hold when approximated with Monte Carlo?**\n\nThis is a very insightful question. Our theoretical analysis is presented in a deterministic setting for motivational clarity. In practice, where gradients are Monte Carlo estimates, the key insight holds because the stochastic gradients are unbiased estimates of the true gradients [5,6]. Over iterations, we expect the optimization trajectory to converge in a way that respects the constraints on average. Our empirical results, obtained entirely with mini-batches, serve as a validation that GEC's core idea is effective and resilient to the stochasticity of the OCL setting.\n\n**Q2: What were the values of $\\alpha$ and $\\epsilon$, and how should they be set?**\n\nThis is a very insightful question about the practical application of GEC. We used $\\alpha = 0.5$ and $\\epsilon = 0.05$ for all experiments, which proved robust across all benchmarks. Here is a general guide to setting them:\n\n*  $\\epsilon$ is the principal hyperparameter controlling the stability-plasticity trade-off. A smaller $\\epsilon$ enforces stricter forgetting constraints (leading to more stability), while a larger $\\epsilon$ allows more flexibility to learn the new task (more plasticity). As illustrated in our Figure 4, varying $\\epsilon$ allows a user to effectively trace a Pareto front of high-performing solutions to match their specific needs.\n* $\\alpha$ controls the strength of the corrective action when the constraint is violated. We found that the performance is quite robust to the choice of $\\alpha$. To demonstrate this empirically, we conducted a sensitivity analysis on Seq-CIFAR100.\n\nTable 1: Different $\\alpha$ on Seq-CIFAR100 (|M|=5k)\n| $\\alpha$ | AAA | Acc |\n| :--- | :--- | :--- |\n| 0.2 | 41.95 | 36.80 |\n| 0.5 | 42.47 | 37.12 |\n| 0.8 | 42.11 | 36.91 |\n\nAs the **table1** shows, performance remains high and very stable across a sensible range of $\\alpha$ values. The variation in both AAA and Acc is minimal. We will add this ablation study to the appendix.\n\n**Q3: Why does GEC's plasticity differ between Seq-CIFAR100 and Seq-TinyImageNet in Figure 3?**\n\nThis is a good observation. Our hypothesis centers on the varying forgetting pressure of the datasets[7]:\n\n*  On Seq-CIFAR100, with high inter-task confusion, the constraint ($h(\\theta) > 0$) is triggered more often, especially early on. GEC correctly prioritizes **stability**, sometimes at the cost of short-term plasticity. This is GEC working as intended: defending against severe forgetting.\n*  But on Seq-TinyImageNet, with more distinct classes, the constraint is more easily satisfied ($h(\\theta) \\le 0$). Here, GEC's advantage shifts to preventing overfitting to the memory buffer. Unlike ER's fixed weighting, GEC can dedicate its full update to the current task once the constraint is met, leading to superior plasticity.\n\n**Q4: What do the dashed lines mean in Figure 1a?**\n\nThank you for seeking clarification. The dashed lines represent the **iso-objective contours** for ER's scalarized loss function, i.e., $\\mathcal{L}_c + \\lambda \\cdot \\mathcal{L}_m = c$, where $c$ is a constant. The slope of these lines is $-1/\\lambda$. The optimal solution for a given $\\lambda$ is found at the point where one of these lines is tangent to the achievable Pareto front. We will add this explanation to the figure caption to make it clear.\n\n[1] Yoo, J., Liu, Y., Wood, F. & Pleiss, G.. (2024). Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:57199-57216.\n\n[2] Urettini, E., & Carta, A. (2025). Online Curvature-Aware Replay: Leveraging 2nd Order Information for Online Continual Learning. arXiv preprint arXiv:2502.01866.\n\n[3] Miettinen K. Nonlinear multiobjective optimization[M]. Springer Science & Business Media, 1999.\n\n[4] Wu Y, Wang H, Zhao P, et al. Mitigating catastrophic forgetting in online continual learning by modeling previous task interrelations via pareto optimization[C]//Forty-first international conference on machine learning. 2024.\n\n[5] Robbins H, Monro S. A stochastic approximation method[J]. The annals of mathematical statistics, 1951: 400-407.\n\n[6] Bottou L. Stochastic gradient descent tricks[M]//Neural networks: tricks of the trade: second edition. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012: 421-436.\n\n[7] Goldfarb D, Evron I, Weinberger N, et al. The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting--An Analytical Model[J]. arXiv preprint arXiv:2401.12617, 2024."}]

Author rebuttal:

Meta-review: This paper proposes an experience replay-based online continual learning method (called Gradient guided Epsilon Constraint, GEC) that formulates single-step parameter updates as an ϵ-constraint optimization problem. I would suggest the authors to take the time to incorporate the reviewers feedback, together with the additional experiments on more challenging benchmarks in the final version of the manuscript. Congratulation on getting your paper accepted.
Decision: Accept (poster)