Model Sensitivity Aware Continual Learning
Authors: Zhenyi Wang
Heng Huang
Venue: NeurIPS 2024 poster (2024)
Keywords: Continual Learning
Primary area: other

Abstract: Continual learning (CL) aims to adapt to non-stationary data distributions while retaining previously acquired knowledge. However, CL models typically face a trade-off between preserving old task knowledge and excelling in new task performance. Existing approaches often sacrifice one for the other. To overcome this limitation, orthogonal to existing approaches, we propose a novel perspective that views the CL model ability in preserving old knowledge and performing well in new task as a matter of model sensitivity to parameter updates. \textit{Excessive} parameter sensitivity can lead to two drawbacks: (1) significant forgetting of previous knowledge; and (2) overfitting to new tasks. To reduce parameter sensitivity, we optimize the model's performance based on the parameter distribution, which achieves the worst-case CL performance within a distribution neighborhood. This innovative learning paradigm offers dual benefits: (1) reduced forgetting of old knowledge by mitigating drastic changes in model predictions under small parameter updates; and (2) enhanced new task performance by preventing overfitting to new tasks. Consequently, our method achieves superior ability in retaining old knowledge and achieving excellent new task performance simultaneously.
Importantly, our approach is compatible with existing CL methodologies, allowing seamless integration while delivering significant improvements in effectiveness, efficiency, and versatility with both theoretical and empirical supports.

Model Sensitivity Aware Continual Learning
Zhenyi Wang and Heng Huang
Department of Computer Science, Institute of Health Computing
University of Maryland College Park
College Park, MD, 20742
zwang@umd. edu heng@umd edu
Abstract
Continual learning (CL) aims to adapt to non-stationary data distributions while retaining previously acquired knowledge. However, CL models typically face trade-off between preserving old task knowledge and excelling in new task perfor- mance. Existing approaches often sacrifice one for the other. To overcome this limitation, orthogonal to existing approaches, we propose a novel perspective that views the CL model ability preserving old knowledge and performing well in new task as a matter of model sensitivity to parameter updates. Excessive parameter sensitivity can lead two drawbacks: (1) significant forgetting of previous knowl- edge; and (2) overfitting to new tasks. To reduce parameter sensitivity, we optimize the model's performance based on the parameter distribution, which achieves the worst-case CL performance within a distribution neighborhood. This innovative learning paradigm offers dual benefits: (1) reduced forgetting of old knowledge by mitigating drastic changes in model predictions under small parameter updates; and (2) enhanced new task performance by preventing overfitting to new tasks. Consequently, our method achieves superior ability in retaining old knowledge and achieving excellent new task performance simultaneously. Importantly, our approach is compatible with existing CL methodologies, allowing scamless inte- gration while delivering significant improvements in effectiveness, efficiency, and versatility with both theoretical and empirical supports.
Introduction
Continual learning (CL) embodies a dynamic approach aimed at adapting to non-stationary data distributions that evolve over time. However, in pursuit of this goal, CL encounters a significant challenge: the trade-off between preserving previously acquired knowledge and effectively learning new knowledge. As the model assimilates new information, it often swiftly erases previously learned knowledge, resulting in catastrophic forgetting (CF) past tasks [4] [44. Effectively addressing CF during CL is essential to preserve previously acquired information. On the other hand, effectively learning new information is equally crucial for CL models adapt to new tasks and environments.
Existing approaches CL often face a dilemma: they either prioritize preserving old knowledge or excelling in new task performance, often at the expense of the other. When CL model prioritizes preserving old knowledge. it focuses on retaining information from previous tasks while minimizing interference or forgetting. However, excessive emphasis on old knowledge can limit the model's s ability to adapt to new tasks. Conversely, when a model prioritizes new task performance, aims to quickly adapt to new tasks or data distributions. Yet, this emphasis on new tasks can potentially degrade performance on previously learned tasks.
To overcome the aforementioned limitations, orthogonal to existing approaches, we introduce the concept of model sensitivity and approach the challenge of balancing old knowledge retention and new task performance in CL from the perspective of model parameter sensitivity. When CL
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

model exhibits high sensitivity to parameter changes. it leads to two significant issues: (1) Increased Forgetting: Excessive sensitivity in model parameters can cause abrupt and substantial changes in model predictions with minor parameter adjustments during CL. This phenomenon results in significant forgetting of previous tasks. (2) Diminished New Task Performance: High sensitivity in model parameters can also result in severe overfitting on new tasks. Overfitting occurs when a model memorizes the training data instead of generalizing patterns that can be applied to unseen data. High parameter sensitivity means that even minor alterations in the training data can induce substantial modifications in the learned model. This renders the model excessively tailored to the training data and reduces its adaptability to new, unseen data, consequently leading to suboptimal performance on new tasks.
To reduce the CL model parameter sensitivity under model updates. we aim to ensure that even minor alterations in model parameters do not substantially impair CL model performance. This is accomplished by optimizing the model's performance based on the worst-case scenario of parameter distributions within distribution neighborhood. However, finding the optimal worst-case CL model parameter distribution is challenging since the space of all possible distributions within the neighborhood is an dimmite-dimensionall space [32]. To efficiently solve this problem, we parameterize the optimal worst-case CL model parameter distribution Gaussian distribution. We propose natural- natural-gradient descent (NGD) method to efficiently inference the mean and covariance of the Gaussian distribution since NGD incorporates the information geometry of the parameter space by adapting the step size based on the curvature of the cost function. This adaptive approach leads to faster convergence compared to conventional gradient descent methods, particularly in high- dimensional spaces where the curvature exhibits notable variations. This is especially beneficial for CL models. However, calculating the natural gradient is computationally expensive due the explicit calculation of Fisher information matrix (FIM). We thus update the worst-case CL parameters in the expectation parameter space, rather than the traditional natural parameter space, of the Gaussian distribution, thereby eliminating the need for explicit calculation of the FIM.
Our method offers dual benefits: (1) Reduced Forgetting: By mitigating parameter sensitivity and avoiding drastic changes in model prediction, our approach effectively reduces the loss of previously learned task knowledge. (2) Improved New Task Performance: Through decreased parameter sensitivity, the model becomes less susceptible to overfitting on new task training data. This reduced vulnerability minor fluctuations fosters the learning of more generalized patterns rather than memorizing specific examples. As a result, the model demonstrates enhanced generalization capabilities new tasks. Therefore, method simultaneously achieves superior performance in retaining previously learned knowledge and excelling in new task performance.
We provide thorough theoretical analysis for our method. Firstly, the theory illustrates that our approach implicitly reduces the variance of loss against different parameter variations, thereby indicating reduced model parameter sensitivity. Secondly, our method tightens the generalization bound of CL models, suggesting enhanced generalization. Furthermore, our extensive experiments across multiple datasets, compared to various state-of- ohet--art- (SOTA) baseline methods, reveal substantial enhancements in overall performance across all learned tasks, backward transfer, and new task test accuracy. These results indicate significantly enhanced CL model ability in preserving old knowledge and achieving better performance on new task with our method. Additionally, our proposed approach seamlessly integrates with existing CL methodologies. functioning as a versatile plug-in. This demonstrates the effectiveness, efficiency, and versatility of our method.
Our contributions can be summarized as follows:
We tackle the challenge of both retaining old task knowledge and excelling in new task in CL from a novel perspective by mitigating model parameter sensitivity.
We introduce a novel CL approach aimed at reducing model parameter sensitivity by optimizing CL model performance under the worst-case parameter distribution within a distribution neighborhood. Additionally, we propose an efficient learning algorithm to identify the worst-case parameter distribution.
We provide comprehensive theoretical analyses that substantiate our method's ability to decrease model parameter sensitivity and improve model generalization.
Extensive experiments conducted across multiple datasets demonstrate the efficacy and versatility of our proposed method.
2

---

2
Related Works
CL aims learn non-stationary data distributions without forgetting previously learned knowledge. The CL scenarios can be further categorized into three scenarios: insk-incremental learning (Task-I)), domain-incremental learning (Domain-Il.) and class-incremental learning (Class-II.) 167] Task- II and Class-II. are most representative scenarios in CI., we thus focus on these two scenarios. Existing approaches for CL. can be categorized into five classes: (1) regularization-buaeed methods incorporate regularization terms either in model weights or outputs into the loss function to mitigate
catastrophic forgetting when learning new tasks, including [28 62 84 55 [
I 22 TO 39]; (2)
memory replay-based methods address the challenge of catastrophic forgetting by explicitly storing and replaying subset of past experiences (samples from previous tasks) while learning new tasks, 72 (3) gradient-projecte projection-
including 44 [37 15 7 [5T 68
3
8 75 74 76 61
78 77
83
36
73
based methods aim to mitigate catastrophic forgetting by projecting gradient updates onto subspaces that minimize interference with previously learned tasks, including [33 17
60
7
38
52 82];
(4) architecture-base architecture to accommodate new tasks while preserving performance on previously learned tasks, including [4 [3 34 23:: (5) Bayesian-based methods leverage principles from Bayesian inference
methods involve dynamically adapting and modifying the neural network
to manage the uncertainty and learning of new tasks while preserving knowledge from previous tasks, 30
including 48 58
25
21
66 59].
In contrast to existing methods, which often necessitate a trade-off' between retaining old knowledge and learning new knowledge, sacrificing one for the other, our approach takes a different path. It sets itself apart from these existing methods by offering an orthogonal solution that preserves old task knowledge while simultancously enhancing new task performance. This novel perspective is achieved by reducing parameter sensitivity.
Connection with existing flt-minina/SSAAD approaches: (1) Connection and difference with related approach: Our method is fundamen- tally different from SAM-based CL in two aspects. (i) Deterministic vs. Probabilistic Approach: SAM uses a fixed deterministic neighborhood, which can be restrictive in practice since updates are constrained within fixed ball. In contrast, our method employs a probabilistic distributional approach, offering two distinct advantages: (a) The distributional neighborhood is more flexible and covers a broader range of parameter variations by sampling from a neighborhood distribution, and (b) Stochastic Gradient Descent (SGD) introduces noise during CL. Our distributional approach accounts for this noise, making it more realistic model in practice and providing stronger guarantees against parameter sensitivity. (ii) Uniform vs. Parameter-specifie sensitivity without explicit calculation of FIM: SAM uniformly updates all parameters, overlooking the varying importance and sensitivity of each parameter in the context of CL. Our method, on the other hand, considers these differences and treats parameters uniquely through the natural gradient without needing to explicitly calculating the FIM. This distinction is crucial for CL, each parameter has different sensitivity to forgetting- factor that SAM does not address. (2) Connection and difference with model averaging flatness seeking approach: SWA (24) and SWAD (9, which aim to achieve flatter minima by averaging multiple models during training. However, these approaches are memory-intensive and inefficient for CL, as they require storing multiple sets of model parameters, which compromises memory efficiency.
sharpness-aware minimization (SAM) [8] 27 45)
3 Method
In this section, we first present the preliminary in section 3.1 and then present the model sensitivity aware continual learning in section 3.2
3.1 Preliminary
Continual Learning Setup The standard CL problem involves learning a sequence of T tasks, represented as " [Dir Dir, D ). The training dataset D for the 4 task contains collection of triplets: (xf v. ) where æt denotes the 7th data example specific to task k. vž represents the associated data label for at. The primary objective is to train neural network function, parameterized by 0, denoted as go (z). The goal is to achieve good performance on the test datasets from all the learned tasks, represented as Dee [D|e Di: Df J. while ensuring that knowledge
3

---

acquired from previous tasks is not forgotten. The CL loss function is defined as the following:
CCL == (ce( y: 0) + (C)(0)
where (a y: 0) is the current task entropy loss function. ( is the forgetinn-mittiation loss, e.g., memory-replay, weight-regularizztn and prriiecctoooootioonn loss, etc. Ç is constant that
balances the weight between the loss of the new task and the loss of the previous tasks.
Exponential Family of Distributions
The exponential family distribution 70 is defined as:
Ps == h(0))erp((e)) (Φ. 22(0)) - 2())
Where == denotes a definition. In existing literature 70 & are called the natural parameters for defining the distribution, (0) h())) is the base measure, 12(0) is the sufficient statistic, ((6) == log J ((0)ezp((d) (2())) )d is the log-partition function, (.) denotes the dot product between two vectors. We denote the expectation parameters X (o) We can write multivariate Gaussian distribution as canonical form of exponential family as:
H.)
e.pp (0 1)) - (0 11)
(2T) dell
expp 10
---2--7 -0 log 2 + log > + - n4]
1
(4)
Therefore, the correspondence between f(0: μ1 Σ) and exponential family distribution in Eq. be expressed as the following:
can
Φ
E
E )-
22(0)
(0"
(5)
2
A 000, 14 X == 20 14 T E
Derivations details of 6 can be found in Appendix B.1 exponential family distributions to parameterize the worst-case of CL model parameter distribution since this enables to efficiently caleulate the natural gradient in the expectation parameter space without needing to explicitly calculate the Fisher information matrix (FIM) in natural parameter space Φ.
In the following section, we
3.2 Model Sensitivity Aware Continual Learning
Learning Objective Specifically, we propose the following CL learning objective to reduce the CL parameter sensitivity under model parameter updates:
min max CCL / Vel
s.t. U (U: (K)
where U/ denotes the uncertainty set. DRL (U, V) denotes the KL divergence between the current CL model parameter distribution V and the neighbour CL model parameter distribution U. is a small constant. maxueu Ee~ CCL (0) aims find the worst-case CL model parameter distribution within neighbourhood. We choose probabilistic distributional neighbourhood due to two-fold reasons: (I) the distributional neighbourhood covers more flexible parameter space: and (2) widely used SGD method incurs update noise during CL, thereby distributional neighbourhood provides stronger guarantee against parameter sensitivity. It is important note that the outer minimization is performed with respect to μu, the expectation of 0, since during inference, only & is used as the model parameter for predictions.
Objective for Learning the Worst-Cass CL Parameter Distribution We convert the constrained inner maximization optimization in Eq. ( into the following unconstrained optimization find the worst-case CL model parameter distribution.
arg min| (U) = -Eo-vO CLL (U,
(8)
where a > is a constant. However, solving Eq.
intractable since the optimization target function space [32) For computation efficiency, we set the current CL
in

---

model parameter distribution as V(0)= = (O En), where Mo and Eg denote the mean vector and covariance matrix, respectively. We set the neighbourhood distribution as D()) = (N 2) where and Σ denote the mean vector and covariance matrix, respectively. To further improve computational efficiency, we constrain the covariance matrix to be diagonal matrix, i.e., E = diag (a and Dg = diag (p*). We denote the density function of (()) and V(0) u()) and v(0), respectively. We express the loss function in Eq. S as the following:
HUU) Eg [L( Σ) - a llg u log (0)]]
By parameterizing the distribution U as exponential family distribution in Eq. our goal is to learn the parameters & in Eq. (5) with natural gradient descent (NGD) [2] as the following equation:
Φ 0. nF VV (()))
(10)
where F is the FIM. We opt for NGD because it adjusts the step size according to the curvature of the cost function, making convergence faster than traditional gradient descent methods. This is especially advantageous in nigh-dimensional spaces where the curvature and parameter-wise sensitivity vary significantly, benefiting CL. models. However, computing the natural gradient is computationally intensive due to the need to calculate the FIM. To address this, we develop an efficient update method in the dual space, specifically the expectation parameter space X, rather than the natural parameter space Φ, eliminating the need for explicit FIM calculation. In the following, we will use ((Φ) and C(X) interchangeably. as they represent the same loss function only parameterized in different spaces. We leverage the relation between NGD in natural parameter space and gradient descent in expectation parameter space (in Appendix I.T), NGD can be performed without explicitly computing the FIM. This update in its dual space leads to significantly more efficient parameter updates and promising computational advantages.
NGD for Efficiently Finding the Worst-Case Gaussian Distribution In the following, we present specific algorithms for updating the μ and Σ with NGD to find the worst-case Gaussian distribution, i.e., U arg miny (U)). We can get the following updates for mean μa and diagonal covariance Σ = diag( ()2) (detailed derivations can be found in Appendix B
M++1 = p4 + 1 V CL (0)
a(e 14) Eg
(11)
Eز = (1 - n1- Σ + V- ( aZp
(12)
By plug-in Σ diag(o²) and Σn diag(p²) into the above equations, we can obtain the following updates:
Mi+1 = pù + 11 i+1 V Vo (0)) - alu; -0)) o 0 (1 01 + V (0.) + tap ==
(13)
(14)
In practice, we set diagonal Hessian matrix Co (0) in Eq. (14) is a computationally challenging task. Following
1.0 to reduce the reliance on hyperparameters. However, computing the
42 we efficiently approximate the Hessian as the following:
N V oo C (0))
VoL CL)
(0)
(15)
N
where N is the number of training data points for the current task, (G) denotes the loss function for the data point j. 0* denotes the tth element of the model parameter O. It is crucial to note that this Hessian approximation is computed only once after learning each task and involves calculating only the diagonal elements, i.e., Σ = diag(o") As a result, the overall computational cost throughout the continual learning process remains low. Additionally, this update mechanism maintains the same number of learnable parameters as existing methods, ensuring fair comparisons. This is because, during the learning of each task, only the mean parameters of the Gaussian distribution are updated.
Learning Algorithm We name our method as Model sensitivity Aware Continual Learning (MACL). The detailed algorithm is present in Algorithm [
5

---

Algorithm Model Sensitivity Aware Continual Learning
I: REQUIRE: model parameters 0, CL model learning rate 3, worst-case Gaussian learning rate number of CL tasks T', number of CL steps K for each task, distribution neighbourhood regularization strengths a 1.0. Randomly initialized diagonal covariance matrix, i.e. diag(o")) ).²).
for = to do
3: for i= = to K do
4:
calculate the CL loss function according Eq. update the worst-case Gaussian mean js (i.e., 0) by 0 O, + no VoL (0; 00)p
5:
6:
sample parameters from the worst-case CL model parameter distribution. O' 0 + on where C C~ (~, I)
7:
update CL model parameters 0:+1 0' VVoc"((8
end for
9: update the worst-case Gaussian covariance a by o = (1 1)00n" 2 - C00
10: where the Hessian is calculated by Vorer cC (0) N Ej=1 (oo (0))² according to Eq. 155
11:
end for
4 Theoretical Analysis
In this section, we build the theoretical connection between MACL and parameter sensitivity in Theorem 4.2 and the generalization analysis in Theorem Due to the space limitations, we provide Let's first look at the inner maximization problem in Eq. 7
the theorem proof in Appendix A.2
max VEl
C (O))U(O)
s.t. 2= (U (U,
(16)
Lemma 4.1. DKL V)
(()) log
m0 v(8)
d0 S
(u(0)-v(01)' v(0)
do
Theorem 4.2. Assume
00 1ood0- S M, we can obtain the following conclusion for Eq.
16):
cE(CCL (0) (C)2
max UEU
(O)(U)) (CI +
(17)
M
where (6) I (O)(V()) Var(CcL(0))
denotes the variance of (CL with re- spect to different model parameters variations, i.c., (ar((C((()) = B(C (0) ((CC)(0) (0) 2 d0. In this context, Var(C*) serves as a measure of the CL model's sensitivity to parameter updates. Essentially, a smaller loss variance indicates lower parameter sensitivity in the CL model. However, directly optimizing the loss variance within the parameter distribution neighborhood
impractical, as requires computing the loss variation across a large number of different sets of CL model parameters and training data points. In contrast, our method (MACL) offers an efficient and effective alternative. MACI. implicitly minimizes the loss variance across different model parameter variations by optimizing CL performance solely on the worst-case CL model parameter distribution. In the following, inspired by UDIL [64,, we further provide the following generalization bound for CL:
Theorem 4.3 (Generalization bound of MACI). Let q be the number of Cl. model parameters and be the number of training data points. The CL. loss L (CL( < C (C is constant). With high probability of 8, the following bound holds:
T=T
E Σ
(CI( S max Bo U. CC CC
+ Nr+ + SEi-T-1 N;
(18)
Veul
No+S Sints No
~2( + 2 2ogg(N + SEEE N + R 2 2log(
4(NT + + N;
Where T is constant. We denote the number of data examples for task 1.... T in the memory buffer M during training on task T as N1. N2...., ,Nr-1 when using memory replay based approach or the number of training data points when using regularization based approuch. CCL denotes the CL. loss on the data from data distribution D, of task i (generalization error), i.e., it is defined as: CEL : y L(x y. 0). (CL(O) denotes the empirical CL loss as Eq. d7. N(,, E) denotes the CL model parameter posterior distribution parameterized with Gaussian distribution.

---

Generalization bound implication: (1) When using memory-rephee approach, the number of samples from new tasks often exceeds the number of samples the memory buffer, causing data imbalance. This imbalance, where fewer samples from previous tasks are stored, affects the second and third terms the generalization bound. The bound suggests that as the number of samples in the memory buffer increases (i.e., E:= N; 1), these terms tighten, leading to a tighter generalization upper bound. This is because 1i [M(z)= == log 21 = O, meaning the generalization improves with larger buffer, aligning with the intuition that more memory buffer data leads to better performance. (2) In the regularization- based approach, < <E N; is treated as the effective sample size for previous tasks since the loss is approximated in the absence of earlier data. The parameter controls the trade-off between learning the new task and retaining knowledge from past tasks. A larger increases regularization, preventing the model from deviating too much from the parameters learned on previous tasks. This leads to higher empirical loss on the new task (first term), but tighter bounds (second and third terms), indicating that knowledge from previous tasks retained effectively. This prioritizes stability over learning flexibility for the new task.
5 Experiments
5.1 Sctup
Datasets We conduct experiments on several datasets, including CIFARIO (10 classes), CIFAR100 (100 classes) [29]. and Tiny-ImageNet (200 classes) [80]. to assess the effectiveness of MACL in task incremental learning (Task-IL) and class incremental learning (Class-IL). In addition, we also conduct experiments
5-dataset 779 5). CUB200 [69] and ImageNet-R [20] (in Appendix). Following the approach in 77 we split the CIFAR-10 10 dataset into five tasks, each with two distinct classes. We divided the CIFAR-100 100 dataset into ten tasks, each containing ten classes. We split the
Tiny-ImageNet dataset into ten tasks, each comprising twenty classes. More dataset statistics can be found in Appendix E.
Baselines We compare to the following various SOTA CL methods. (1) Regularization methods, including oEWC (22, synaptic intelligence (SI) [84) Learning without Forgetting (LwF)
[35, Classifier- Projettin- Regularization (CPR) [IO]. (2) Bayesian-based methods, including NCL
25 (3) Architecture-basee based methods, including HAT 663 (4) Memory-based methods, including
ER 15). A-GEM [4], i55RL(55), GSS (2], HAL L2) DER++ 77 ER-ACE 8 and LODE [36].
(5) Gradient- conptttrrooooooonoobneee based methods: Gradient Projection Memory (GPM) 160
Implementation Details Following 7. we use ResNet18 [9] as the backbone network for all the CL datasets and compared baseline methods. For the baselines that are included in the open-source code of DER++ 77, we use the same hyperparameters provided in DER++ 7 for the compared methods. For the baselines not included in the open-source code of DER++ e.g.. GPM, LODE, etc, we use the open-source code from their original paper for comparisons sona..ssooo.noooosoonoos For the hyperparameters in our method, hyperparamcters. For
we set 1.0 across all the datasets to minimize the model's dependence 77. we set 17 = 1e 5 for CIFARIO and CIFARI00, and 1 = 1e 6 for Tiny- Tin--ImaeeNet. The 17 is selected from the range of [le 4, le 5, le 6, le 7]. Following 7 [4], the hyperparameter is determined through the validation sets split from the training sets from the first three tasks. Similar to 77) we train all the CL models using the standard SGD optimizer to update the CL model. The batch size and replay buffer batch size are set to 32. We use a single NVIDIA A5000 GPU with 24GB memory to run the experiments. Each experiment result is averaged for 10 runs with mean and standard deviation.
5.2 Results
We evaluate the performance of different CL methods with (1) overall accuracy; (2) new task accuracy; and (3) backward transfer in the following.
Overall Accuracy (ACC) ACC is the average accuracy across the entire task sequence. We present the results on CIFARIO 10, CIFAR- 100 and Tiny-ImageNet Table [ We can observe that our method substantially improve over various SOTA baseline methods up to 3% to 4% on CIFAR 100, Tinylma geNet by integrating MACL with existing CL. methods. This overall performance improvement attributed to the reduced parameter sensitivity.
7

---

Table 1: Task-II. and class-IL. overall accuracy on CIFAR1O, CIFAR-100 and Tiny-ImageNer respectively with memory size 500. indicates not applicable/availabe..
CIFAR-10
CIFAR-100
Tiny- Mla--imaeNeet
Method
Class-IL
Task-IL
Class-IL
Task-IL
Class-111
Task-IL
line-tuning
19.62 + 0.05
61.02 + 3.33
9.29 + 0.33
33.78 + 0.42
7.92 = 0.26
18.31
0.68
Joint train
92.20 + 0.15
98.31 + 0.12
71.32
0.21
91.31 + 0.17
59.99 + 0.19
82.04
0.10
SI
19.48 1 0.17
68.03 1: 5.01
9.41 4: 0.24
31.08 1: 1.65
6.58
1 0.31
36.32
0.13
LWF
19.61 + 0.05
63.29 + 2.35
9.70 + 0.23
28.07 + 1.96
8.46
0.22
15.85
0.58
NCL
19.53 0.32
64.49
8.12 0.28
20.92 2.32
7.56
0.36
16.29
0.87
GPM
90.68 3.29
72.48 0.40
UCB
79.28 = 1.87
57.15 +: 1.67
HAT
92.56 +: 0.78
72.06 +: 0.50
A-GEM
22.67 +: 0.57
80 40 +: 1.45
9.30 +: 0.32
48,06 +: 0.57
8.06 + 0.04
25.33 0.49
GSS
49.73 + 4.78
91.02 + 1.57
13.60 + 2.98
57.50 + 1.93
HAL
41.79 1: 4.46
84.54
2.36
9.05 2.76
42.94 1: 1.80
OEWC
19.49 + 0.12
64.31
+ 4.31
8.24 + 0.21
21.2 + 2.08
7.42 + 0.31
15.19
0.82
OEWC+MACL
20.55 +: 0.71
66.95
2.46
8.82 +: 0.50
23.42 +: 1.93
7.86
0.23
17,43
0.93
CPR(EWC)
19.61 + 3.67
65.
+ 3.87
8.42 + 0.37
21.43 +: 2.57
7.67 # 0.23
15.58
0.91
CPR(EWC)+MACT.
20.58
2.56
67.28
3.75
9.15
0.63
22.87 1.78
8.10
0.49
17.96
0.82
GPM
72.48 + 0.40
30.72 0.27
GPM+MACI.
74.51 + 0.36
35.06 0.38
iCaRL
44.16 + 1.53
84.06 + 0.42
23.71 + 0.23
59.24 0.16
iCARL
48.27 0.95
84.55 0.51
24.18 0.58
59.45 0.32
ER
57.74 0.27
93.61 + 0.27
20.98 + 0.35
73.37 + 0.43
9.99
0.29
48.64 0.46
ER+MACL
63.74 + 1.24
93.78 0.36
22.18
0.27
74.87 + 0.51
9.87 0.15
51.25
DER+
72.70 1.36
93.88 1 0.50
36.37 + 0.85
75.64
19.38 + 1.41
51.91 0.68
DER+++MAL
74.53 = 0.79
94.72 0.65
0.82 39.42
77.53 - 0.89
20.17 + 1.56
54.03 0.79
ER-ACE
11.83:1:.3
94.12 0.61
37.05
0.36
75.97 +: 0.69
20.43 1: 0.97
52.59
0.75
ER-ACE+MACL
73.21 =: 0.96
94.98 0.72
0.39 40.28
77.65 = 0.76
21.99 - 0.83
53.95 0.78
LODE
75.45 0.90
94.41 1 0,22
38.95
0.93
78.92 1 0.67
19.87 1 0.72
60.18 0.65
LODE&MACL
76.41 +: 0.67
94.32 + 0.24
40.67
40.03 0.51
21.09 + 0.97
61.79
New Task Accuracy To evaluate the new task performance of the proposed CL method, evaluate the new task performance during CL by integrating MACL with DER++ and GPM Figure The results show that MACL can significantly improves the new task performance for different CL methods, indicating that reducing the model parameter sensitivity is beneficial to improve new task performance during CL.
Backward Transfer Backward trans- fer (BWT) quantifies the degree of forgetting observed on previously learned tasks. When BWT 0, it
DER++
DER+ + +MACL
GPM
GPM+MACL
New Task Test Accuracy Along the Continual Learning Process
New Task Test Accuracy (%)
indicates that learning the current new task positively influences the perfor- mance on previously learned tasks.
90
Conversely, when BWT < it sig-
nals that learning the current new task may result in forgetting previ- ously acquired knowledge. We eval-
80
75
uate BWT in Table 2 We can ob- serve that our method significantly im-
70
proves BWT by up 5% through integrating MACL with existing CL methods. This indicates that reduc- ing parameter sensitivity can substan- tially reduce forgetting on previously learned knowledge. These empirical analysis also verify our theoretical analysis that our method implicitly improves the stability by reducing loss variance.
Task Sequence on CIFAR100
Figure new task performance during CL.

---

Table 2: Backward Transfer of different CL methods with memory size 500.
CIFARIO
CIEARIO
Tiny- Iny-ImaeeNet
Method
Class-I1.
Task-II.
Class-II.
Task-It.
Class-II.
Task-II.
fineturing
96.39
-46.24 + 2.12
-89.68
0.96
-62.46
0.78
-8894 0.81
-67.44
AGEM
91.01 1.16
-14.26 + 1.18
-88.5
1.36
-15.43
2.32
-78.03 0.78
-09.28 1.08
GSS
62. + 2.67
-7.73
+ 3.90
-82.17
4.16 4.16
333998
1.54
HAL
62.21 4.34
-..1
-99.29 2.82
-13.60 1.01
ER
45.35 0.07
-3.54 0.35
-74.84 1: 1.38
-16.81
0.97
-75.24 =: 0.76
-31.98 4: 1.35
ER+MACL
-34.43 0.82
0.32 -3.31
-73.17 0.69
-15.73 0.78
-75.29
0.37
-29.32 = 0.42
DER++
22.38 4.41
-4,66 1.15
-53.89 1.85
-14.72
0.96
-64.6
-27.21 1.23
DER+ MACL
-21.87 = 1.67
-3.09 +: 1.31
-88622 +: 1.56
-13.62 2: 0.35
-62.23 = 0.78
-27.10 + 0.43
ER ACE
-13.64 1: 0.95
.3,28
0.83
39.51
1.23
-1.55
0.39
46.07
0.83
.2835 1 0.16
ACE-MALL
-12.76 1.23
-3.15 1: 0.57
-3386 1.37
-13.89 0.57
-42.29
0.50
-28.41 1 0.25
LODE
-6..37 + 0.67
.2.93 0.19
-53.23 + 1.72
-15.24 + 0.76
-55,89 + 0.98
-19.13 + 0.56
LODE+MACL
-16.25 = 0.73
-3.16 0.45
-52.67 1.35
-15.11 0.53
-55.61 1.18
-18.17 + 0.83
5.3 Ablation Study
Hyperparameter Analysis We evaluate the sensitivity of the hyperparameters 1j in Table 5 in Appendix [.I] Our observations indicate that when parameter sensitivity is not reduced, i.e., 1/ 0. the CL model performs poorly. As we gradually increase the reduction of parameter sensitivity, the CL model's performance improves. This improvement is because appropriately reducing parameter sensitivity helps mitigate forgetting and enhances learning for new tasks, thus boosting overall CL performance. However, if the reduction in parameter sensitivity is increased excessively, the model's performance deteriorates. This because an overly constrained model, while minimizing forgetting. struggles to learn new tasks effectively, resulting in worse performance.
Effect of Memory Size To assess the impact of varying memory buffer sizes, we present the results in Table 3 The results demonstrate that compared to different baseline methods, our MACL plug-in also enhances the performance of baseline methods with memory size of 2000.
Table 3: Task-IL and class-IL overall accuracy on CIFAR- and Tiny-mmageNet, respectively with memory size 2000.
Algorithm Method
CIFAR-100
Tiny-ImageNet
Task
Class-IL
Task
ER
36,06 1 0.72
81.09 1 0.45
15.16 0.78
58.19 1 0.69
ER+MACL
37,83 :: 0.94
83.37 : 1.35
17.08 = 0.73
59.51 = 0.53
DER+
50.72 0.71
62.43 4 0.38
24.21 +: 1.00
62.22
0.87
MERL+
52.79 0.85
84.07 1 0.79
27.55
1.43
64.28 0.95
0.56
85.79
31.03
1.27
70.05
0.59
LODE-MACL
54.76
0.68
86.53
+ 0.58
32.16 = 1.12
69.79 = 0.53
Benefit of NGD To evaluate the benefits of using NGD over gradient descent (GD) for calculating the worst-case Gaussian distribution, we present comparison results in Table 6 in Appendix The results show that NGD outperforms GD because NGD better captures parameter importance, which helps preserve old knowledge while effectively adapting to new tasks.
Efficiency Evaluation To assess the efficiency of our proposed method, we compare the running time of integration of different CL methods with MACL and corresponding CL methods alone on CIFAR 100, as shown in Table 15 in Appendix D.8 The results indicate that incorporating MACL increases the computational cost by only 55% to 61% compared to the corresponding CL methods alone. This demonstrates the high efficiency of our method, as it introduces only small additional training cost.
Effect of Different Architectures To evaluate the impact of different architectures, we compared various approaches using both ViT and ResNet32. For the ResNet32 experiments, followed the setup in [85, integrating MACL with MEMO [66 and comparing it to MEMO alone, using memory buffer size of 2000 on CIFAR100. Additionally, we conducted experiments with a pre-trained Vision Transformer (ViT) [6]. specifically the bit-base-patch16 P-24 model pre-trained on ImageNetIK. K. On CIFAR 100, we integrated MACL with DER++, using a memory size 500, and demonstrated that using a pre-trained ViT significantly improves CL performance. Moreover, combining MACL, with + further enhances CL performance with the pre-trained VIT. The results are presented in the Appendix.

---

Long Task Sequence To assess the effectiveness of the proposed approach across varying task lengths, we conducted experiments by splitting Tiny-ImageNet into sequences of 10 and 20 tasks. The Task-II. and Class-II. results for integrating DER++ with MACI,, using memory buffer size of 500, are presented Table f These results demonstrate that even with longer task sequences, our method still significantly outperforms DER++
Table 4: Overall accuracy of integrating DER++ with MACL using memory buffer of 500 and longer task sequence on Tiny-ImageNet.
number of tasks
10
20
Class-IL IL
19.38 + 1.41
15.02 + 0.53
Class-II++ MACI.
20.17 + 1.56
16.08 + 0.81
Task-II.
51.91
0.68
51.65 + 1.36
Task-IL IL MACL
54.03 0.79
54.96 0.72
Online CL Under the online CL setting, evaluate the effectiveness of the proposed approach CIFARIO0 and Tiny-ImageNet by comparing with MKD (46) and PCR (37 The results are put in the Appendix.
5-datasets results To assess the effectiveness of MACL on the 5-Datasets benchmark 79 5].
which includes CIFAR-10 10, MNIST [33], Fashion-MNIST [8I], SVHN [47), and notMNIST 66 conducted experiments. This dataset provides diverse range CL tasks. We performed experiments on 5 5-Datasets, using memory buffer size of 500, with MACL. The detailed results are provided in the Appendix.
ImageNet-R and CUB200 results We further evaluate the effectiveness of MACL on CUB200 66 and ImageNet-R 200). the results are shown the Appendix.
Conclusion
In this paper, address the challenge of balancing learning tasks while preserving knowledge from previous ones continual learning. We propose a model sensitivity-awar continual learning method that enhances both the model's ability retain old knowledge and improve performance on new tasks. Specifically, our goal is to reduce model parameter sensitivity by optimizing CL performance for the worst-case parameter distribution within the neighhorhood of the current model's parameter distribution. This approach improves stability in preserving old knowledge and mitigates overfitting
tasks. We provide comprehensive theoretical analysis of the proposed method, and extensive experiments on multiple datasets demonstrate its effectiveness, efficiency, and versatility.
Limitation Discussion Our method introduces additional training cost compared to existing continual learning approaches.
Broader Impacts
Our work advances continual learning, which beneficial develop more adaptable and efficient AI. Our work has no negative societal impacts.
Acknowledgments
This work was partially supported by NSF IIS 2347592, 2347604, 2348159, 2348169 DBI 2405416. CCF 2348306, CNS 2347617.
10

---

References
[1] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Mareus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. Proceedings of the European conference on computer vision, pages 139-154, 2018.
[2] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances in Neural Information Processngg Systems 30, 2019.
[3] Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based complementary learning system. In International Conference on Learning Representations, 2022.
[4] Lorenzo Bonicelli, Matteo Boschini, Angelo Porrello, Concetto Spampinato, and Simone Calderara. On the effectiveness of lipschitz-driver rehearsal in continual learning. Advances in Neural Information Processing Systems, 35:31 0183188863190,, 2022.
[5] Jorg Bornschein, Alexandre Galashov. Ross Hemsley, Amal Rannen-Trike Yutian Chen, Arslan Chaudhry, Xu Owen He, Arthur Douillard, Massimo Caccia, Qixuan Feng, et al. Nevis' 22: A stream of 100 tasks sampled from 30 years of computer vision research. Journal of Machine Learning Research, 24(308):1-77, 2023.
[6] Yaroslav Bulatov. Notmnist dataset. 2011.
[7] Pictro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple baseline. Advances in neural information processing systems, 13:15920-15930, 2020.
[8] Lucas Caccia, Rahaf Aljundi, Nader Asadi. Tinne Tuytelaars. Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In International Conference on Learning Repreventations, 2022.
[9] Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Man-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 14:22405-22415 2021.
[10] Sungmin Cha. Hsiang Hsu, Taeback Hwang, Flavio Calmon, and Taesup Moon. Cpr: Classifier-projection regularization for continual learning, International Conference on Learning Representations, 2021.
[11] Arslan Chaudhry, Puneet Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European conference on computer vision, pages 532-547, 2018.
[12] Arslan Chaudhry, Albert Gordo, Puncet K Dokania, Philip HS Torr, and David Lopez-Paz. Using hindsight to anchor past knowledge in continual learning. Association for the Advancement of Artificial Intelligence, 2021.
[13] Arslan Chaudhuy, Naeemullah Khan, Puneet Dokania, and Philip Tom. Continual learning in low-rank orthogonal subspaces, Advances Neural Information Processing Systems, 33:9900-999 1911,991111111 2020.
[14] Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. Proceedings of the International Conference on Learning Representations, 2019.
[15] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puncct K. Doka- nia, Philip H. S. Torr, and Marc' Aurelio Ranzato. Continual learning with tiny episodic memories. http:://arrivvon//////0000.. 10480 2019.
[16] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[17] Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics, pages 3762-3773. PMLR, 2020.
[18] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness- aware minimization for efficiently improving generalization. In International Conference Learning Representations, 2021.
11

---

[19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[20] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang. Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEF/CVF international conference on computer vision. pages 8340 8349, 2021.
[21] Christian Henning, Maria Cervera, Francesco D' Angelo, Johannes Von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F Grewe, and João Sacramento. Posterior meta-replay for continual learning. Advances in Neural Information Pricessing Systems, 34:14135-14149, 2021.
[22] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rehalamcing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 831 2019.
[23] Ching- Yi Hung, Cheng- Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. Advances Neural Information Processing Systems, 32, 2019.
[24] P Izmailov, AG Wilson, D Podoprikhin, D Vetrov, and T Garipov, Averaging weights leads to wider optima and better generalization. In 34th Conference on Uncertainty in Artificial Intelligence, pages 876-885, 2018.
[25] Ta-Chu Kao, Kristopher Jensen, Gido van de Ven, Alberto Bernacchia, and Guillaumc Hennequin. Natural continual learning: success is journey, not (just) a destination. Advances neural information processing systems, 14:28067-28077 2021.
[26] Mohammad Khan, Didrik Nielsen, Voot Tangkaratt, Wu Lin, Yarin Gal. and Akash Srivastava. Fast and scalable bayesian deep learning by pertuutpeturbation in adam. In International conference on machine learning, pages 2611 1-2620. PMLR, 2018.
[27] Minyoung Kim, Da Li. Shell X Hu, and Timothy Hospedales. Fisher sam: Information geometry and sharpness aware minimisation. In International Conference on Machine Learning, pages 11148-11161. PMLR, 2022.
[28] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barkkks-Baiika,, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences. 114(13):3521 -526. 2017.
[29]
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
[30] Richard Kurle, Botond Cseke, Alexej Klushyn, Patrick Van Der Smagt, and Stephan Günnemann. Continual learning with bayesian neural networks for non-stationary data. In International Conference on Learning Representations, 2019.
[31] Beatrice Laurent and Pascal Massart. Adaptive estimation of a quadratic functional by model selection. Annals of statistics, pages 1302-1338, 2000.
[32] Peter D Lax. Functional analysis, volume 55. John Wiley & Sons, 2002.
[33] Yann LeCun. Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-) 2324, 1998.
[34] Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: continual structure learning framework for overcoming catastrophic forgetting. In International Conference on Machine Learning, pages 3925-3934. PMLR, 2019.
[35] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEF Transactions on Pattern Analysis and Machine Intelligence, 2018.
[36] Yan-Shuo Liang and Wu-Jun Li. Loss decoupling for task-agnostic continual learning. Advances in Neural Information Processing Systems, 36, 2023.
[37] Huiwei Lin, Baoquan Zhang, Shanshan Feng, Xutao Li, and Yunming Ye. Per: Proxy-based contrastive replay for online class incremental continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14246-24255 2023.
12

---

[38] Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. Trgp:
Trust region gradient projection for continual learning. In International Conference on Learning Representations, 2022.
[39] Chenxi Liu,, Zhenyi Wang, Tianyi Xiong, Ruibo Chen, Yihan Wu, Junfeng Guo, and Heng Huang. Few- shot class incremental learning with attention-aware self-adaptive prompt. In European Conference on Computer Vision, 2024.
[40] David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
[41] Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7765-7777 (6777773,22 2018.
[42] James Martens. New insights and perspectives on the natural gradient method. Journal of Machine Learning Research, (1(1))),-76, 2020.
[43] David McAllester. Pac- bayesinn model averaging. Proceedings of the twelfth annual conference on Computational learning theory, pages 164-170, 1999.
[44] Michael MeCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.
[45] Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investigation of the role pre-training in lifelong learning. Journal of Machine Learning Research, 24(214):1550 2023.
[46] Nicolas Michel, Maorong Wang. Ling Xiao, and Toshihiko Yamasaki. Rethinking momentum knowledge distillation online continual learning. Forty-first International Conference on Machine Learning. 2024.
[47] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng. et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature feature
learning, volume 2011, page 4. Granada, 2011.
[48] Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, and Richard E. Turner. Variational continual learning. In International Conference on Learning Representations, 2018.
[49] Pingbo Pan. Siddharth Swaroop, Alexander Immer, Runa Eschenhagen, Richard Turner, and Mohammad Emtiyaz E Khan. Continual deep learning by functional regularisation of memorable past. Advances in Neural Information Processing Systems, 33:4453 4464, 2020.
[50] Kaare Brandt Petersen, Michael Syskind Pedersen, et al. The matrix cookbook. Technical University of Denmark, 7(15):100, 2008.
[51] Quang Pham, Chenghao Liu, and Steven HOL. Dualnet: Continual learning, fast and slow. In A. Beygelz- imer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[52] Jingyang Qiao, Xin Tan, Chengwei Chen, Yanyun Qu, Yong Peng, Yuan Xie, al. Prompt gradient projection for continual learning. The Twelfth International Conference on Learning Representations, 2024.
[53] Garvesh Raskutti and Sayan Mukherjee. The information geometry of mirror descent. IEEE Transactions on Information Theory, 61(3):155-11557 2015.
[54] Roger Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285, 1990.
[55] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001-2010, 2017.
[56] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap- proximate inference in deep generative models. In International conference on machine learning, pages 1278-1286. PMLR. 2014.
[57] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu. and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Representations, 2019.
13

---

[58] Hippolyt Ritter. Aleksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. Advances in Neural Information Processing Systems, 31, 2018.
[59] Tim GJ Rudner, Freddie Bickford Smith, Qixuan Feng. Yee Whye Teh, and Yarin Gal. Continual learning via sequential function-space variational inference. In International Conference on Machine Learning, pages 18871-18887. PMLRR, 2022.
[60] Gobinda Saha, Isha Garg. and Kaushik Roy. Gradient projection memory for continual learning. ICLR, 2021.
[61] Fahad Sarfraz, Elahe Arani, and Bahram Zonooz. Error sensitivity modulation based experience replay: Mitigating abrupt representation drift in continual learning. In The Eleventh International Conference on Learning Reprexentations 2023.
[62] Jonathan Schwarz, Jelena Luketina, Wojeiech M. Czarnecki. Agnieszka Grabska-Baree Ba--wBawwin... Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress and compress: A scalable framework for continual learning. In Proceedings of the International Conference on Machine Learning, 2018.
[63] Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference on machine learning, pages 4548 4557. PMLR, 2018.
[64] Haizhou Shi and Hao Wang. A unified approach to domain incremental learning with memory; Theory and algorithm. Advances in Neural Information Processing Syxtems, 36, 2023.
[65] James Seale Smith, Leonid Karlinsky. Vyshnavi Gutta, Paola Cascante- Boniilaa, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Contiqual decomposed attention- based prompting for rehearsal- free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11909-1 1919, 2023.
[66] Michalis K. Titsias, Jonathan Schwarz, Alexander G. de G. Matthews, Razvan Pascanu, and Yee Whye Teh. Functional regularisation for continual learning with gaussian processes. In International Conference on Learning Representations, 2020.
[67] Gido M van de Ven, Tinne Tuytelaars, and Andreas S Tolias. Three types of incremental learning, Nature Machine Intelligence, 4(12):1185-1197 185-1197, 2022.
[68] Eli Verwimp. Mathhis De Lange, and Tinne Tuytelaars. Rehearsal revealed: The limits and merits of revisiting samples in continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9385-9394, 2021.
[69] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- birds-200-2011 dataset. 2011.
[70] Martin J Wainwright, Michael Jordan, et al. Graphical models, exponential familics, and variational inference. Foundations and Trends@ Machine Learning, 1(1-2):1 22:1- 10051-35,22000 2008.
[71] Shipeng Wang. Xiaorong Li, Jian Sun, and Zongben Xu. Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 184-193, 2021.
[72] Zhenyi Wang. Yan Li. Li Shen, and Heng Huang. A unified and general framework for continual learning. In The Twelfih International Conference on Learning Representations, 2024.
[73] Zhenyi Wang. Li Shen, Tichang Duan, Qiuling Suo, Le Fang. Wei Liu, and Mingchen Gao. Distributionally robust memory evolution with generalized divergence for continual learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[74] Zhenyi Wang, Li Shen, Tichang Duan, Donglin Zhan, Le Fang, and Mingehen Gao. Learning learn and remember super long multi multi-domain task sequence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 7982-7992, June 2022.
[75] Zhenyi Wang, Shen, Le Fang, Qiuling Suo, Tichang Duan, and Mingehen Gao. Improving task-free continual learning by distributionally robust memory evolution. In International Conference on Machine Learning, pages 22985-22998. PMLR, 2022.
[76] Zhenyi Wang, Li Shen, Le Fang, Qiuling Suo, Donglin Zhan, Tichang Duan, and Mingchen Gao. Meta- learning with less forgetting on aurg-sscale non non-statoon statiooiiyn task distributions. In European Conference on Computer Vision, pages 221-238 Springer. 2022.
14

---

[77] Zhenyi Wang, Li Shen, Donglin Zhan, Qiuling Suo, Yanjun Zhu, Tiehang Duan, and Mingchen Gao. Metamix: Towards corruption-robust continual learning with temporally self-adaptive data transformation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24521- 24531, 2023.
[78] Zifeng Wang, Zheng Zhan, Yifan Gong. Yucai Shao, Stratis Ioannidis, Yanzhi Wang, and Jennifer Dy. Dualhsic: Hsic-bottleneck and alignment for continual learning. In International Conference on Machine Learning, 2023.
[79] Zifeng Wang, Zizhao Zhang, Chen Yu Lee, Han Zhang. Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF conference computer vision and pättern recognition, pages 139 149, 2022.
[80] Jiayu Wu, Qixiang Zhang, and Guoxi Xu. Tiny imagenet challenge. Technical report, 2017.
[81] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion- mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint M.Xiv11708.077777 2017.
[82] Mingging Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Hebbian learning based orthogonal projection for continual learning of spiking neural networks. In The Twelfth International Conference Learning Representations, 2024.
[83] Enneng Yang. Li Shen, Zhenyi Wang. Tongliang Liu, and Guibing Guo. An efficient dataset condensation plugin and its application to continual learning. Advances in Neural Information Processing Systems, 36, 2023.
[84] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International conference on machine learning, pages 3987-3995. PMLR, 2017.
[85] Da- Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwci Liu. leamning: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[86] Wei Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. A model 603 exemplars: Towards memier-efficient glass-incremental learning. The Eleventh International Conference on Learning Representations, 2023.
15

---

A Theorem Proof
A.1 Duality in Natural Gradient Descent for Exponential Family Distribution
Theorem A.1. Gradient of the loss [()) with respect to the expectation parameter X, i.e. (x)(A). is equal the natural gradient with respect natural parameter Ø, i.e., F V V(L)). This can be expressed as the following:
VxC(A) (oC()(
(19)
In particular, NGD in natural parameter space can equivalently performed through gradient descent with respect the expectation parameters the following:
Φi+1 = - VVC -V:
(20)
where ni is the learning rate and F is the Fisher information matrix (FIM).
Proof. The exponential family distribution is defined as the following:
(0) exp( (6, s2(0)) Z(Φ))
(21)
According to the expectation of the score function is O, we can obtain the following
0 (0) V log P& PP (0) (2( [ = À V <
(22)
Therefore,
A = V Z(6)
(23)
where the first equality is due to the fact that the expectation of the score function is zero
We then derive the Fisher information matrix (FIM) the following:
F()) == Ep (p) V log (0)
(24)
PP) V log (e)))
(25)
EP,( - VV [()))]
(26)
(V
(27)
V&A
(28)
V (()
(29)
Z(Ø))
(30)
where: := denotes defined as. Then,
V ==\= V32(0- F
(31)
Next,
=
(32)
More general results on manifold can be found in 53
A.2 Theoretical and Generalization Analysis of MACL
Lemma A.2. DRI.UU (U, V) = =f u(() log( ((0)
(0) dd
<
v(0))²
d0
w(0)
16

---

Proof.
u1
(KI(U,,) - w(0) log(
v(0) )do
(33)
u())
- log
v(0) d0 (by Jensen's inequality)
(34)
u())²
1d0 (log(1 + x) - x))
(35)
v())
(u(0) v v()))²
d0
(36)
v(0)
where the last equality is because
/ (u(O) v0) de
e0))2
2())² v(0)
2 uu())d0 + 0(0)d0= u(0)²
(37)
v(0)
Since f u()) = v(00)0== =1
Theorem A.3. Assume f
||l= 1oo < M, we can obtain the following conclusion for Eq.
16):
(0)(())) = CC) + CE(CCL CCL(O))
max UEU
(38)
M
where (CL(O) = [ We denote the variance of the random variable (CL( as ((C [C).(0))
Var (ar(c)(())) -0)
Proof. We define new distribution Z == U -- V.
I c (O)dU))) = (0) = (CL(())+ /
CC (O)dZ(0)
(39)
(CL(()) (C CCL(0))Z2()) (CL(0))Z(())
(CCCL()ZZ0)
(40)
By Lemma 4.1 and Hölder's inequality, we can obtain the following:
O (())
(u(O) v()))²
DKL(U, (U,V)
= u()) log
d0 <
(41)
(
/ (u(0) v(0)) de /
1
100
(42)
(())
/ (u(0) 2 <e
(43)
Therefore,
/ (u(()) v0))) - M
(44)
f (0110)))-00)))0000 (COL (C001))000(0))(00)) v())dd
(45)
([c0110)- -cc)(0)) d0
(u(8) 20 (Cauchy-Schrre Schwarz inequality)
(46)
(B(CCL(() (C)(0))2
(47)
M
17

---

The equality holds when the following condition holds:
( - -(()) -
(48)
where a is constant.
(0)(0))
(C)()) v(0))d0
(49)
/ (u(() [CI(0)
v())
(50)
= 0
(51)
The last equality is because ((u(0)- - do = d0) - f v())d 1-1
Therefore, obtain the following conclusion:
<B(LCL(O) - (O))2
max VEN
(O)(U)) -
(CL(
(52)
M
In this context, the CL loss variance across various sets of model parameters (ar(LCL(0) serves as measure of the CL model's sensitivity to parameter updates. Essentially, smaller loss variance indicates lower parameter sensitivity the CL model. However, directly optimizing the loss variance within the parameter distribution neighborhood is impractical, as it requires computing the loss variance across large number of different sets of CL model parameters and training data points, In contrast, our method (MACL) offers an efficient and effective alternative. MACL implicitly minimizes the loss variance across different model parameter variations by optimizing CL performance solely on the worst-case CL model parameter distribution.
We denote the prior distribution as V(0) = =N(upp Ep) and posterior distribution as U(0) NH,, E,)
Theorem A.4 (Generalization bound of MACL). Let be the number of CL model parameters and be the number of training data points. The CL loss (CL(O) S C (C' constant), With high probability of - 6, the following bound holds:
:- Ee-N(H Σ (O) S max EØ BO-uC CL
(53)
DEu
NT + (E -I N;
+2(V+ v 2Nog(Nr SE T- N)) + R +
2log(
Nr+t 1 N
+ + CE 1,
Where is a constant. We denote the number of data examples for task 1,..- ,T memory buffer M during training on task T as N1, N2,.... _Nr-1 when using memory replay based approach or the number of training data points when using regularization based approach. L (0) denotes the CL loss on the data from data distribution D, (generalization error), i.e., it is defined as: (CL(() : yyg L(x., y, 0). (C)( denotes the empirical CL loss as Eg. (7) N(,, E) denotes the Cl. model parameter posterior distribution parameterized with Gaussian distribution.
in the
Proof. We apply the PAC-Bayes theorem 43 that for any prior distribution, with probability 1 over the CL training dataset T, the following bound holds:
DKL. (U((())))((())) - log())
E ICEL (2)
(54)
The KL divergence between posterior and prior distribution can be calculated as the following:
18

---

(KL(U(O))
= Ba-ure (U)(( - log(V(0)))
(U()))
(55)
2p
= log
2 B (0 - (2 1 "i (0 1 (u,) + E0 =p))
(0 -4p)
(56)
Σ
p
= 2
log
Σ
q + (1s -kp) (Ms Up) + Tr(Ep Eg)l
(57)
We assume the following inequality:
3p log
Tr(Ep Ex) S RR R20
(58)
Σ,
Therefore,
DKL (U(O)) S R (1s Ep (1s 14p)
(59)
According to 500 we have the following identity:
For random variable O (N, Σ)
Eg~N (0 1²)² A(0 p²) (1= +
(60)
where Tr denotes the trace of A matrix. Therefore, according to Eq. (60 we have the following two equations 61 and 62
Ea-u (0 1-'
(0 us)) = (Ms 11,)7 (Hs - (a)) + +TT(EE Ex) =
(61)
E - u1p)) "xpp))"",""""" (0- 1p)) = (us - mp) E (ps - 1p) + Tr(E EE,)
(62)
(u= We set 7 = Ep (+s -p)). Then, ||yl| = (0m1))) (M= - pp)
If ~ N(O, 2T), according to 31 we have the following inequality with probability of 1
||^l < (g + 2 log n + log S <²( + 22lon n)²
(63)
Then we partition the space of μ., into two disjoint area that satisfy (μs - 1p)) T D (ms 1p) S
2c R and (u,- ppp) E (m,, mp) 2e R.
(1) In the case of (1,, pp) T (m, 1p) s 2c R, we take the maximum loss 14. we have the following inequality:
E (CF <
max (n,- Σ (M,, 522
Eorro (0)
(64)
(2) For the case of (js, 14,)
) ",,)"""\(Ha 14)) 2 R, we have CCL (0)
Combining case (1) and (2), we can obtain the following generalization bound:
DRI(UU,) <
- [1,. 14p) "T, (m. 14)) + R + < 11m11 + R
(65)
< 1 ( + v 1lg + R
(66)
We have the following bound with probability of 4:
19

---

(7
(1 -
max 1p--Ma)22--R
Eg 1
C
(67)
pp)) 2
1 max (UL(U
C
S n
Ea- (0)
(68)
n
Then, we can obtain the following generalization bound with probability of 1. n
T=
C
(H. E) Σ
(0)
max DEU
(O + Nr + SE N;
(69)
T²(V9 + v 22lg( (T + < <E 1i=--11 N2))² + R+ + 2 2log(
NT N N;
4(Nr + + N₂ N2
In this theorem, we provide the theoretical guarantee for the generalization analysis of our pro- posed method. This bound indicates by optimizing the MACL loss, method tighten/reduce the generalization error of the CL method, thus improving the overall performance of our method.
B
Equation Derivation
B.1 Exponential Family Distribution Details
According to the definition of expectation, we can obtain the following equation:
A Ex(e=w)) 1
(70)
According to the definition of covariance matrix,
Σ [ 10 [=)
(71)
E
2p0 1u1")
(72)
= E[00"] 14 T
(73)
By rearranging the above equation, we can obtain the following:
E[000]= = pp + Σ
(74)
Then, the conclusion follows:
A = E 14, X2 == E) 1412 +
(75)
B.2 Worst-Case Gaussian Distribution NGD Derivations
Gradient of Loss ((A) With Respect to X
Taking gradient with respect to X as the following:
Op TA + V
Cx. = V C(A)
VuC(A)- - 2
(76)
OA
In Eq. (76), the second equality is because the identity:
8u ax.
1,
0Σ
33
2p,. (by Eq.
DAT
8u Vx=C(A)= = V()(())
2
VE((A)
2X2 VxC(A)
(77)
OX
Eq. 77
the second equality is because the identity:
8u σx
0,
= (by Eq.
75
According to Eq. ( we set the natural parameters as:
Φ² : $²:= ==
--1
(78)
2
20

---

(1) NGD with respect to x²: According to Eq. 20 and 77 obtained as:
NGD with respect to Ø² can be
1 - - = 2 _ - Vn)x((A
2 Then, obtain the following update:
(79)
E = V +
(80)
(2) NGD with respect to Φ¹: Similarly, according to Eq. 20 and 76 Φ can be obtained as:
NGD with respect to
p = E; p4
- -Vz((A))
(81)
By simplifying and rearranging Eq. (T). the following update for ja:
μ4 V
(82)
Mean and Covariance Updates Derivations Following the results in 56 26 we can obtain the following equation:
V n L(H, Σ) ( VoC(u, E)
(83)
1 V Vy C(p, Σ) = -E V0 Σ)
(84)
2
Then, we only need to calculate E VoC(u, E) and Ee C0, E). Here, since we assumed a general CL Gaussian distribution for the current CL parameter distribution, i.e., V(0) = (0 Eo) and neighbourhood distribution, i.e. U(0) (O E). The detailed derivations for the gradient are present in the following:
V (0) ((p, ( VoL VoL a Vo log u(O) Vo log v0)
(85)
- (L) aF (0) E + a aT (4) En1
(86)
= Vo alp - uo)
(87)
1
VylEo~u(0)
- B C (0) + aBo (log u (0)) - log v())
(88)
2
Ba V L cl (0) + マ B V log u(0) - Vàe log v (0)
2
(89)
Ea V CCL (0) + E ( Eō'l
---++
(90)
1
2 Ea-u(o;1
C (0)
ax +aEp11
(91)
00
Plug-in the gradient derivation into Eq. and 80
we can obtain the following results:
E11 E-1 T ())
CL (0) a..1 + a.
(92)
= (1 - -)00) Σ +
n B
V2 (0) + czE,
(93)
Po+1 = μ4 1| E,+ E Cl + alm - -100) (0))
(94)
Finally, by using single sample from distribution U with density 0 ~ u(0) to approximate the expectation. By plug-in Σ diag(o") and E. = diag(p²) into the above equations, we can obtain the following updates:
M1+1 = + 11 +1 V V (0,) - alm; -mopp* 21
(95)
0 (1 n 2 + n| CLL ap 21
2
(96)
21

---

C Baseline Details
EWC [28]: EWC endeavors to alleviate forgetting in continual learning through the uti- lization of weighted weight regularization technique based on the Fisher information matrix.
CPR Drawing on neural networks with wide local minima and principles from information theory, CPR introduces extra regularization term. This term aims to maximize the entropy of a classifier's output probabilities, thereby reaching wider local minima to enhance generalization.
GPM [60] A CL model acquires new skills by adjusting its parameters through gradient steps that move orthogonal to the gradient subspaces considered vital for previous tasks. The Gradient Projection Memory (GPM) establishes these subspaces by analyzing network activations following the completion of each task using Singular Value Decomposition (SVD), then preserves them in memory.
HAT is task-driven hard attention mechanism that retains information from prior tasks while ensuring it doesn't interfere with the current task's learning process.
НAT 63::
AGEM aims to guarantee that, at every training step, the average loss of episodic memory over past tasks does not rise, thus mitigating the risk of forgetting previously acquired knowledge.
A-GEM [4]
The goal populate the memory buffer with diverse set of examples, using the data gradient as a feature for sample selection. For comparison, we opt for the efficient GGSS-Greed version.
Gradient-Based Sample Selection (GSS-Greedy) 2
This method stores a subset of examples from previous tasks using reservoir sampling [15]. During each iteration, we randomly replay a subset of examples from the memory buffer.
ER (15]:
DER++ [7]:
This method combines experience replay with knowledge distillation further improve the effectiveness of experience replay.
They discovered that ER causes significant overlap between the representa- tions of newly added classes and previous ones, resulting in highly disruptive parameter updates. From this empirical analysis, they proposed new method to address this issue by protecting the learned representations from drastic adaptations when incorporating new classes. Their approach uses an asymmetric update rule that pushes new classes to adapt the older ones, rather than the reverse. This technique particularly effective at task boundaries, where much of the forgetting typically happens.
ER-ACE [
They conducted an in-depth analysis of the impacts of distinguishing between and old classes, well as among new classes, finding that these learning objectives result in varying degrees of forgetting. Consequently. combining these objectives negatively affects the performance of the CL model. To address this, LODE separates the two objectives for new tasks by decoupling the loss associated with them. This approach allows LODE assign different weights to each objective, leading to better performance compared to methods that use coupled loss.
LODE (6):
D More Experimental Results
D.1 Hyperparameter Sensitivity Analysis
Table 5: Analysis of hyperparameter 7 on CIFARI00 and Tiny-ImageNet in the setting of task-IL.
7
0.0
le-7
le-6
le-5
3xle-5
CIFAR100
75.61=0.60
77.16=0.42
77.69+0.37
77.53=0.89
76.97+0.46
ImageNe
51.91+0.68
52.12+0.82
54.03+40.79
54.46-0.91
51.62+0.55
22

---

D.2
Benefit of IND
Table 6: Benefit of MACL-NGD vs. MACL-GD on CIFAR100 and Tiny-ImageNet in the setting of task-IL.
method
DER++
DER+HMACL-NGG) (EACL(MACLGGD)
CO0
75.64++0.60 51.9110.68
77.53+0.89 54.03
76.62+00.53
Tiny-enageNel
52.9710.71
D.3
Online CL results
Table 7: Online CL Results on CIFARI00 under the blurry boundary setting
Memory Size
1000
2000
5000
MKD(PCR)
0.66
44.95 = 0.42
54.87 0.39
MKD(PCR) MACL
37.2 0.53
46.17
0.51
56.21 0.43
Table Online CL Results on Tiny-ImageNet under the blurry boundary setting
Memory Size
2000
5000
10000
MKD(PCR)
17.33 + 1.28
29.58
0.60
38.02 + 1.64
MKD(PCR) MACI.
18.21 1.32
30.69 0.71
38.73 1.56
D.4 Prompt-based CL results
We conducted an experiment integrating MACL with the SOTA prompt-based CL method, CODA- Prompt [65] Our method operates on the parameters of prompt components and corresponding keys/attention vectors.
Table 9: CODA Prompt Results on ImageNet-R
Number of Tasks
10
20
CODA-P
75.45 1 0.56
72.37 = 1.19
CODA-P MACL
76.39 = 0.67
73,42 = 1.23
D.5
5-datasets results
Table 10: Comparison of methods on Class-IL and Task-IL on 5-datasets.
Method
Class
Task-IL
ER
66.03 = 1.37
92.58 + 1.26
ER+MACL
67.32 + 1.18
93.21
DER+
85,92 0.33
87.16 0.21
DER+MACL
87.23 0.51
87.51 0.30
D.6 Effect of Different Architectures
Table 11: Overall accuracy with ResNet32 using a memory buffer of 2000 by integrating with MEMO.
MEMO
MEMO+MACL
accuracy
58.49
59.61
Table 12: Overall accuracy with ViT using a memory buffer of 500 by integrating DER++ with MACL.
Class-II.
Task-II.
DER+
76.21 + 0.67
96.72 + 0.31
DER+ MACL
77.83 + 97.31 + 0.46
23

---

D.7 ImageNet-R and CUB200 results
We conducted experiment on the recent CL datasets of ImageNet-R and CUB200 with pre-trained Vision Transformer (ViT), ie.. vit- tt--bse--aate base--atchhh6..... patch11-2224 as the backbone following the codebase of + The results (memory size of 500) are shown in the following table.
Table 13: ImageNet-R Results
Method
Class-Il II
Task-IL
DER+
58.29 + 1.78
86.93
0.32
MER+
60.51 1: 1.65
87.56
0.41
LODE
74.98 + 0.21
90.22
0.39
LODE+MACT.
75.51 0.26
90.81
0.28
Table 14: CUB200 Results
Method
Class-IL IL
Task-IL
DER+
41.81 + 1.69
87.16
1.09
MACL
43,07 1: 1.53
88.03
0.97
LODE
66.87 0.35
93.12 0.56
LODE-MACL
67.53 + 0.51
93.42 0.37
D.8 Efficiency Evaluation
Table 15: Running efficiency of MACL on CIFAR100 by training for a single epoch on CIFAR100.
method
MACL
MACL
DER+
8.7
13.5
ER-ACE
6.3
10.2
LODE
13.2
20.8
E Experiment Setup
E.1 Dataset Statistics
Table 16: Dataset Statisties
Dataset
Seq-CIFARI0 Seq-CIFAR100
Number of Tasks
10
Number of Classes
10
100
200
Number of Training Samples
50,000
50,000
100,000
Number of Test Samples
10,000
10,000
10,000
24

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and introduction summarize the main contributions in our paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We dicussed the limitations of our work after conclusion.
25

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faireess
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We provide the full set of assumptions and complete (and correct) proof in Appendix.
Guidelines:
The answer NA means that the paper does include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our contribution is a new continual learning algorithm. We described full implementation details for our proposed algorithm.
Guidelines:
The answer NA means that the paper does not include experiments.
26

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
Code will be released.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ /ttppp.cc/pbllc/guideeee http://oeeubiiisssoooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
27

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We provided detailed implementation details regarding training and test details, data splits, hyperparameters and type of optimizer.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We provided results standard deviation with multiple experiment runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split. initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative еттог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provided sufficient information on the computer resources in implementa- tion details.
Guidelines:
The answer NA means that the paper does not include experiments.
28

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed the societal impacts afer conclusion.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses such risks.
29

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have cited the code package produced by DER++ and the dataset used, e.g., CIFARIO, CIFARIO0,
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pww.::s...aaseiiiia...... thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
30

---

Answer:
[NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "bHGRxT5EUW", "reviewer": "Reviewer_1NyD", "summary": "The paper proposes a new perspective on stability-plasticity trade-off in continual learning, centered on controlling model sensitivity to model updates. The goal is then to ensure that alteration in model parameters does not negatively impact the CL performance of the model. \n\nIn order to solve this challenging task, the authors propose to optimise model’s performance based on the worst-case scenario of parameter distributions within a distribution neighborhood. Given the intractability of this problem due to the infinite dimension of the space of all possible distribution within this neighborhood. The authors solve this issue by modelling the model parameters by a Gaussian distribution. They then propose to solve a min-max problem, with a CL objective equal to the sum of a cross entropy loss and  a forgetting mitigation loss (which can come from a replay method, a regularization term, a gradient projection loss, ...) within a neighborhood based on the KL-divergence with the current CL parameter distribution. In order to efficiently solve this optimization problem: \n* The authors leverage natural gradient descent in the distribution parameter space. \n* They develop an mirror-descent method in the dual space alleviating the need for computing the Fisher information matrix. \n\nThe authors support their method with a theoretical analysis and some experiments. The theoretical analysis shows a reduction in loss variance, indicating a lower model parameter sensitivity, and tighter generalization bounds. The experiments were conducted using a ResNet18 backbone, Split CIFAR and Tiny-ImageNet benchmarks, and adding the proposed approach to a large number of CL algorithms. They also include ablation study to analyse the hyperparameters, study the effect of memory size, comparing natural gradient descent to SGD and analysing the method efficiency.", "questions": "* Is modelling the current CL parameters with a standard normal distribution realistic? Isn't it too restrictive? What if we start the CL process with an already pretrained model? \n\n* Can this modelling assumption be changed with a more general Gaussian distribution? What would be the impact on the derivation? \n\n* Can the method be added to the recent family of learning to prompt techniques for CL?", "limitations": "The work is of foundational nature. I think discussing potential negative societal impact is out of scope.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The approach takes a novel and innovative (to the best of my knowledge) perspective on solving the stability-plasticity trade-off. \n\n* The method is based on solid theoretical development. It theoretically leads to the intended effect, and experimental results confirm this theoretical insights. \n\n* The method is easily applicable on top of existing CL algorithms, for what seems to be a limited computational cost. It also seems to be applicable in many CL scenarios. \n\n* The method is tested with a large number of CL algorithms, showing interesting boost in performance. I also appreciate the ablation study which provides interesting insights. \n\n* The authors provide a detailed description of the implementation, which increases the reproducibility of the work.", "weaknesses": "The main weakness of the paper is the lack of diversity in the evaluation settings, and this in multiple aspects: \n\n* The evaluation is limited to benchmarks that lack in diversity in the data distribution. I invite the authors to look at other benchmarks (e.g. 5-datasets (used in different recent works, e.g. Learning to Prompt for Continual Learning, Wang et al. 2022), Nevis'22 (Bornschein et al. 2023), ... \n\n* The evaluation is limited to relatively short sequences, it would be interesting to analyse the effect of the sequence length. \n\n* The evaluation is limited to the task aware scenario, although it is a priori also suitable for the more challenging Online Continual Learning setting. \n\n* The evaluation is limited to a single architecture that is trained from scratch. It would be interesting to analyse the impact of architecture, and of initialization on the method. \n\nThe authors also analyse the efficiency of the method in one setting only. It would be interesting to add other methods, as the additional cost can be a fix one, and the small additive cost in the analyzed setting can be due to the high cost of the original CL algorithm. \n\nDespite these limits, I think the work already provides interesting perspective and insights.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere appreciation for your constructive feedback!\n\n**Q1** 5-datasets evaluation\n\n**A1** Thank you for your suggestions! We perform experiments on 5-datasets (memory buffer size of 500) with our method (MACL) as below.\n\n| Method | Class-IL | Task-IL |\n| -------- | -------- | -------- |\n| ER     |  66.03 $\\pm$ 1.37    |    92.58 $\\pm$ 1.26 |\n| ER+MACL     |  **67.32 $\\pm$ 1.18**    |    **93.21 $\\pm$ 1.08** |\n| DER++     | 85.92  $\\pm$ 0.33    |   87.16  $\\pm$ 0.21    |\n| DER++MACL    | **87.23  $\\pm$ 0.51**    |   **87.51  $\\pm$ 0.30**    |\n\n**Q2** analyse the effect of the sequence length or long sequence.\n\n**A2**  We invite you to refer to the **global response Q2 and A2**.\n\n**Q3**  Online Continual Learning setting evaluation.\n\n**A3**  Thank you for your suggestions! We invite you to refer to the **global response Q1 and A1**.\n\n**Q4**  analyse the impact of architecture, and of initialization on the method.\n\n**A4**  Thank you for your question! We conducted an additional experiment using a pre-trained Vision Transformer (ViT), specifically the vit-base-patch16-224 model pre-trained on ImageNet1K, following existing CL literature settings. The results, shown in the following table for CIFAR100 with DER++ and a memory size of 500, demonstrate that using a pre-trained ViT significantly improves CL performance. In addition, integrating MACL with DER++ further enhances the CL performance with the pre-trained ViT.\n\n| architecture | ResNet | ViT |\n| -------- | -------- | -------- |\n| Task-IL    |  75.64 $\\pm$ 0.60   |   96.72 $\\pm$ 0.31   |\n| Task-IL + MACL    |   **77.53 $\\pm$ 0.89**             |   **97.31 $\\pm$ 0.46**   |\n| Class-IL   | 36.37 $\\pm$ 0.85     |   76.21 $\\pm$ 0.67   |\n| Class-IL + MACL  | **39.42 $\\pm$ 0.82**        |   **77.83 $\\pm$ 0.80**              |\n\n**Q5**  analyse the efficiency of other CL methods\n\n**A5** Thank you for pointing this out! We conducted experiments to compare the running efficiency by integrating our proposed method with multiple CL approaches, including DER++, ER-ACE, and LODE. The training time (in seconds) for a single epoch on CIFAR100 is shown below. Our method increases the training time by approximately 55% to 61%.\n\n**running time on CIFAR100 (seconds)**\n\n| Method | w/o MACL | w/ MACL |\n| ------ | -------- | ------ |\n| DER++  | 8.7      | 13.5   |\n| ER-ACE | 6.3      | 10.2  |\n| LODE   | 13.2     | 20.8   |\n\n**Q6** Is modelling the current CL parameters with a standard normal distribution realistic? Isn't it too restrictive? What if we start the CL process with an already pretrained model?\n\n**A6** Thank you for your question! We acknowledge that this assumption might be restrictive in certain scenarios. When starting the CL process with an already pretrained model, the parameters may not be initially distributed according to a standard normal distribution. In such cases, our method can be adapted to account for the existing parameter distribution of the pretrained model. We initialize our approach using the pretrained model's parameter statistics to estimate a more general Gaussian distribution, ensuring a more accurate and realistic modeling process.  Furthermore, we invite you to the **Q7** and **A7** for a more general Gaussian distribution derivation. By doing so, our method retains its effectiveness while being flexible enough to accommodate different starting conditions, whether the model begins from scratch or with pretrained parameters. This adaptability helps maintain the generalizability and flexibility of our approach across various settings.\n\n**Q7** change assumption with a more general Gaussian distribution? \n\n**A7** Thank you for your suggestion! Here, we assume a more general Gaussian distribution to model current CL parameter distribution, $V(\\theta) = \\mathcal{N}(\\theta|\\mu_0, \\Sigma_0)$, where $\\mu_0$ and $\\Sigma_0$ denote the mean and covariance matrix of current CL model parameters, respectively.  We can obtain the following update equation:\n\n\n$\\Sigma^{-1}_{i+1}= (1 - \\eta)\\Sigma^{-1}_i + \\eta \\mathbb{E} _{\\theta\\sim u(\\theta)} [-\\nabla _{\\theta\\theta}^2 \\mathcal{L} ^{CL}(\\theta) + \\Sigma_0^{-1}]$ \n\n$\\mu_{i+1} = (1 - \\eta)\\mu_{i} + \\eta \\Sigma _{i+1} \\mathbb{E} _{\\theta \\sim u(\\theta)} [\\nabla _{\\theta} \\mathcal{L}^{CL}(\\theta)+ \\mu_0 \\Sigma _0^{-1}]$\n\nCompared to standard normal distribution modeling assumption. Here, the above more general Gaussian parameter distribution incorporates one additional term of $\\mu_0 \\Sigma_0^{-1}$ and $\\Sigma_0^{-1}$ for $\\mu$ and $\\Sigma$ update process, respectively. This allows for greater flexibility and adaptability in the update process.\n\n**Q8** add to learning to prompt techniques for CL?\n\n**A8**  Thank you for your suggestions! We conducted an experiment integrating our proposed method (MACL) with the SOTA prompt-based CL method, CODA-Prompt [1]. Our method operates on the parameters of prompt components and corresponding keys/attention vectors. The results on ImageNet-R are shown in the following table. The improvement is due to reducing the sensitivity of these parameters. This way, when the model learns new tasks, parameter changes do not significantly increase the loss. During inference, the weighted prompt remains more stable, mitigating forgetting. Additionally, our method reduces overfitting to new tasks, thus improving overall performance.\n\n**CODA Prompt Results on ImageNet-R**\n| number of tasks | 10 | 20 |\n| -------- | -------- | -------- |\n| CODA-P    |  75.45 $\\pm$ 0.56   |  72.37 $\\pm$ 1.19   |\n| CODA-P + MACL    | **76.39 $\\pm$ 0.67**     | **73.42 $\\pm$ 1.23**    |\n\nReference:\n\n[1] CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning, CVPR 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AB6JcIW5bd", "reviewer": "Reviewer_YRBc", "summary": "This paper presents a min-max optimization framework targeting at the model sensitivity for continual learning, where the authors claim that it can mitigate the abrupt change of the model parameters and thus simultaneously alleviates the problem of catastrophic forgetting of the past knowledge and overfitting to the current task. By assuming the posterior distribution of the model parameters as multivariate gaussian, it achieves effective min-max optimization. Some experimental results are are presented to demonstrate the model's better performance.", "questions": "1. I find the formulation of the fundamental optimization problem in this work confusing. In Eq. 7,\n$$\n\\min_\\theta\\max_{U\\in\\mathcal{U}} E_{\\theta\\sim U(\\theta)}[\\mathcal{L}^{\\text{CL}}(\\theta)],\n$$ \n$$ \n\\text{s.t.}\\quad \\mathcal{U} = \\\\{U: D_{\\text{KL}}(U, V) < \\epsilon \\\\},\n$$\nthe outer minimization over $\\theta$ is not taking any effect in the inner maximization as it is integrated out over $U$. I assume this $V$ should be dependent on $\\theta$? But in the main paper, the authors claim $V=\\mathcal{N}(0, I)$. Please elaborate on this.", "limitations": "Yes. The authors addressed the limitations in Section 6.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The motivation of mitigating the catastrophic forgetting by addressing the model sensitivity is promising and there exists some work showing this point. \n2. The proof of the main claims in the paper are paired with detailed derivation, which provides additional source of reference for the readers.", "weaknesses": "1. The paper starts with treating the current continual learning objective as a whole without discussing the interplays among the different tasks or the data scarcity problem of the previous tasks, which is not a usual case. The oversimplification of assuming a $\\mathcal{L}^{\\text{CL}}(\\theta)$ can be problematic in the sense that we generally want to treat different tasks differently (past vs. current) so that old knowledge retention and new knowledge acquisition can be achieved at the same time. To further illustrate my point, consider some other arbitrary learning scenarios, this \"model sensitivity awareness\" can be seamlessly applied. So please elaborate on why CL specifically benefits from this algorithm. \n2. It is not clear what the generalization bound provided by theorem 4.3 implies. It still contains the proxy training target for continual learning $\\mathcal{L}^{\\text{CL}}$ and seems not very informative about the CL algorithm.\n3. The organization and presentation of the paper is not confusing, especially for Section 3.2. There are theorems, together with too detailed derivation of the intermediate results, I would suggest the author re-organize the material and only summarize the most important results in the main paper, and put the not-so-relevant content in the appendix. \n4. The experimental results seem not rigorously verified. As far as I know, the CIL results on CIFAR100 dataset should start with 40-50% overall accuracy but not this low in the paper (too many references and I will just skip providing them). Please provide more demonstration on this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our sincere gratitude for your valuable comments.\n\n**Q1** treating current continual learning objective as a whole without discussing the interplays among the different tasks or the data scarcity problem of the previous tasks.  why CL specifically benefits from this algorithm.\n \n **A1**  Thanks for your question! \n \n * We would like to clarify that our notation of $L^{CL}(\\theta)$ does not explicitly treat the continual leanring process as a whole. Instead, this is an abstract and unified loss notation that can represent the memory-based loss, regularization-based loss, etc. It simplifies the derivation process and guarantees the general applicability of our proposed method. Specifically, $L^{CL}(\\theta) = L_{CE}(\\theta) + L_{f}(\\theta)$. Here, the new task loss is $L_{new} = L_{CE}(\\theta)$ and the old task loss approximation is $L_{old} = L_{f}(\\theta)$. Thus, it can be further formulated as $L^{CL}(\\theta) = L_{new}(\\theta) + L_{old}(\\theta)$, which describes the interplay between new and old tasks.\n \n* The data scarcity problem of the previous tasks is incorporated into the loss of $L_{old}(\\theta)$. For example, if adopting a memory-replay-based method to mitigate forgetting, $L_{old}(\\theta) = L_{(x,y) \\sim M}(x, y, \\theta)$, i.e., the memory-replay loss, where $M$ is a memory buffer that stores a small number of data from previous tasks. If adopting a regularization-based approach, $L_{old}(\\theta) $ is a regularization term to mitigate the parameter drift from previous task parameters. \n \n* Our \"model-sensitivity-aware\" loss operates on both $L_{old}(\\theta)$ and $L_{new}(\\theta)$ at the same time. Operating on $L_{old}(\\theta)$ reduces the parameter sensitivity on old tasks to mitigate forgetting on old tasks. Operating on $L_{new}(\\theta)$ reduces the parameter sensitivity on new tasks to reduce overfitting on the new tasks, thus leading to better performance on new tasks. Therefore, our method mitigates the trade-off between old task knowlege retention and new task learning by improving the performance on new and old tasks simultaneously. \n \n**Q2** what the generalization bound in theorem 4.3 implies. \n\n**A2**  \n* **Generalization bound**: \n    * For the first term on the right-hand side (RHS) of the generalization bound, in traditional CL, the training loss does not bound the generalization error on unseen test data. In contrast, by adopting our model sensitivity-aware CL approach, the model sensitivity-aware training loss (the first term) plus an additional complexity term (the third term) can bound the generalization error on unseen test data. Therefore, this implies that by minimizing the model sensitivity-aware loss, i.e., $\\max _{\\mathbb{U} \\in \\mathcal{U}} \\mathbb{E} _{\\theta \\sim \\mathbb{U}} L ^{CL} _{T}(\\theta)$, we can indeed reduce the generalization error. \n    * The third term in the generalization bound is the model complexity term, i.e., $\\sqrt{\\frac{\\tau^2(\\sqrt{q} + \\sqrt{2\\log n})^2 + R + 2\\log(\\frac{n}{\\delta})}{4(n-1)}}$, represented by the KL divergence between the posterior distribution 𝑄 and the prior distribution 𝑃 and simplifid to the current form. This term quantifies the complexity of the posterior distribution relative to the prior. A higher KL divergence indicates that the posterior distribution 𝑄 deviates significantly from the prior 𝑃, implying a more complex model. This complexity term acts as a regularizer. It penalizes models that are too complex and deviate significantly from the prior, thereby encouraging simpler models that are more likely to generalize well, thereby mitigating overfitting issue.\n\n\n**Q3** The organization and presentation. \n\n**A3** Thank you for your suggestions! We revised the contents in the revision. \n\n**Q4** The experimental results seem not rigorously verified. As far as I know, the CIL results on CIFAR100 dataset should start with 40-50% overall accuracy but not this low in the paper. \n\n**A4** Thank you for pointing out this! \n\n* We would like to clarify that as mentioned in lines 243-244 on page 7, the CIFAR100 dataset is divided into **10** tasks, each containing 10 classes, following the same setting as [1]. [1] reports the CIFAR100 accuracy as 37.13 for DER++ and 36.48 for ER-ACE under a memory buffer size of 500 but did not provide an error bar. Our results show DER++ at 36.37 $\\pm$ 0.85 and ER-ACE at 37.05 $\\pm$ 0.36, which are very similar to [1] with the same settings.\n\n* According to [2], they split CIFAR100 into **5** disjoint tasks with each task having 20 classes, reporting the accuracy of DER++ in class incremental learning as 42.08. The performance difference is **due to different task splits**: we split CIFAR100 into **10** tasks, while [2] splits it into **5** tasks. We chose the more challenging 10-task scenario over the 5-task scenario.\n\nReference:\n\n[1] On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning, NeurIPS 2022\n\n[2] Loss Decoupling for Task-Agnostic Continual Learning, NeurIPS 2023\n\n**Q5**  In Eq. 7, the outer minimization over is not taking any effect in the inner maximization. \n\n**A5**  Thanks for your question! We would like to clarify that the inner expectation over 𝑈 is achieved by reparameterizing the Gaussian distribution as follows: $\\mu + \\sigma \\zeta$, where $\\zeta \\sim \\mathcal{N}(\\textbf{0}, \\textbf{I})$. Therefore, the expectation for $U$ can be converted to the expectation with respect to the Gaussian distribution $\\zeta$. Therefore, the outer minimization over $\\theta$ still takes effect on $\\theta$ after the inner max expectation. The details are illustrated as the following equation:\n\n$\\max _{\\mathbb{U}\\in \\mathcal{U}}  \\mathbb{E} _{\\theta \\sim \\mathbb{U}}  L^{CL} _{T}(\\theta) \\approx \\mathbb{E} _{\\zeta \\sim \\mathcal{N} (\\textbf{0}, \\textbf{I})}  L ^{CL} _{T}[(1 - \\eta)\\theta _{i}   + \\eta \\sigma _{n} ^2 [\\nabla _{\\theta} L ^{CL}(\\theta _i)]+ \\sigma_n \\zeta]$", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mQlymGnWc8", "reviewer": "Reviewer_kb2F", "summary": "The paper presents a new method to address catastrophic forgetting and improve the learning ability of a new task in continual learning. They attribute these two objectives to parameter sensitivity. To address this problem, they propose to minimize the performance on the worst-case CL parameter distribution within the neighborhood of the current CL model. The approach is integrated with existing CL methods and evaluated on three continual learning benchmarks CIFAR10, CIFAR100, and Tiny-ImageNet.", "questions": "- It seems that you mostly focus on integrating the method with replay-based approaches. Is there a motivation behind it?\n\n- Do the same findings generalize to a long sequence of tasks?\n\n- From Table 1 and Table 2, it seems that the methods improve new task learning more than reducing catastrophic forgetting. Do you have some thoughts on that?", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Strength:\n\n- To the best of my knowledge, it is a novel perspective to address the continual learning problem.\n- Integrating the approach with current CL methods improves performance.\n- The method is evaluated on two scenarios: task incremental learning and the more challenging one, class incremental learning.\n- Multiple CL metrics are evaluated and analysis is provided.", "weaknesses": "- It is not clear whether the assumption of CL model parameters following a normal distribution could be valid for all models.\n\n- I found it hard to easily understand the relation between the parameter sensitivity and the worst-case CL model. \n\n- The presentation could be improved. Some sections are wordy. Those can be concise and some of the theoretical proofs in the appendix can be moved to the main paper to easily follow.\n\n- I would expect to see forgetting reported in Table 3.\n\n- Most baselines are a bit old.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate the thoughtfulness of your feedback and support!\n\n**Q1**  whether the assumption of CL model parameters following a normal distribution could be valid for all models.\n\n**A1** Thank you for your question! \n\n* The assumption that model parameters follow a Gaussian distribution is intended to balance good performance with computational efficiency. This assumption is widely adopted in Bayesian inference for neural network parameters because of its simplicity and scalability. The Gaussian distribution is computationally efficient to work with and can be easily integrated into various optimization and inference algorithms. This balance between performance and efficiency makes the Gaussian distribution a practical choice for modeling neural network parameters, particularly in complex tasks such as continual learning. \n\n* We extend the standard normal distribution of current CL model parameters into a more general Gaussian distribution update. Here, we assume a more general Gaussian distribution to model current CL parameter distribution, $V(\\theta) = \\mathcal{N}(\\theta|\\mu_0, \\Sigma_0)$, where $\\mu_0$ and $\\Sigma_0$ denote the mean and covariance matrix of current CL model parameters, respectively.  We can obtain the following update equation:\n\n   $\\Sigma^{-1}_{i+1}= (1 - \\eta)\\Sigma^{-1}_i + \\eta \\mathbb{E} _{\\theta\\sim u(\\theta)} [-\\nabla _{\\theta\\theta}^2 \\mathcal{L} ^{CL}(\\theta) + \\Sigma_0^{-1}]$ \n\n   $\\mu_{i+1} = (1 - \\eta)\\mu_{i} + \\eta \\Sigma _{i+1} \\mathbb{E} _{\\theta \\sim u(\\theta)} [\\nabla _{\\theta} \\mathcal{L}^{CL}(\\theta)+ \\mu_0 \\Sigma _0^{-1}]$\n\n    Compared to standard normal distribution modeling assumption. Here, the above more general Gaussian parameter distribution incorporates one additional term of $\\mu_0 \\Sigma_0^{-1}$ and $\\Sigma_0^{-1}$ for $\\mu$ and $\\Sigma$ update process, respectively. This allows for greater flexibility and adaptability in the update process.\n\n* Our proposed framework is both flexible and extendible. It can be adapted to handle more general and complex parameter distributions to capture a wider range of parameter variability. This flexibility enhances its applicability to more complex distributions.\n\n**Q2**  I found it hard to easily understand the relation between the parameter sensitivity and the worst-case CL model.\n\n**A2** Parameter sensitivity implies that small changes in parameters can lead to significant drops in performance. By focusing on the worst-case scenario, we are minimizing the potential performance degradation that can occur due to parameter updates. The model is designed to be more stable against the worst possible updates in parameters. If a model can perform well under the worst-case conditions, it will naturally perform well under less extreme conditions.\n\nThis approach ensures that the model does not rely on any specific set of parameters to perform well, but rather maintains good performance across a wide range of possible parameter values to minimize performance degradation.  By preparing for the worst-case, the model minimizes the variability in its performance. This means that small changes in parameters are less likely to cause significant deviations in performance, thereby reducing parameter sensitivity.\n\n**Q3**  presentation could be improved. \n\n**A3** Thank you for your suggestions! We followed your advice and revised presentation in the revision. \n\n**Q4**  I would expect to see forgetting reported in Table 3.\n\n**A4**  \n|   | **CIFAR-100 Class-IL** | **CIFAR-100 Task-IL** | **Tiny-ImageNet Class-IL** | **Tiny-ImageNet Task-IL** |\n|---------------|------------------------|-----------------------|----------------------------|--------------------------|\n| **ER**        | -51.29 ± 0.43 | -8.17 ± 0.25  | -63.76 ± 0.38 | **-17.12 ± 0.26** |\n| **ER+MACL**   | **-50.71 ± 0.58** | **-7.28 ± 0.19** | **-62.53 ± 0.46** | -17.59 ± 0.33 |\n| **DER++**     | -34.26 ± 0.31 | -7.83 ± 0.29  | -43.05 ± 0.42 | -14.56 ± 0.78 |\n| **DER++ + MACL** | **-33.32 ± 0.39** | **-7.56 ± 0.31** | **-42.27 ± 0.53** | **-13.82 ± 0.65** |\n| **LODE**      | **-21.15 ± 1.06** | -2.72 ± 1.32  | -37.32 ± 1.25 | -9.19 ± 1.07 |\n| **LODE + MACL** | -21.80 ± 1.23 | **-1.93 ± 1.17** | **-36.38 ± 1.03** | **-8.02 ± 1.68** |\n\n**Q5** Most baselines are a bit old.\n\n**A5**  Thank you for pointing out this! We added one SOTA class-incremental learning (CIL) MRFA [1], one SOTA online CL baseline [2] and prompt-based baseline [3]. We invite you to refer to the **global response Q1, Q3, Q4** and **A1, A3, A4**\n\n**Q6** motivation for focus on integrating the method with replay-based approaches.\n\n**A6** Thank you for your question! This is because memory-replay-based approaches usually achieves better performance and are more widely studied than other continual learning methods. \n\n**Q7** generalize to a long sequence of tasks?\n\n**A7** Thank you for your question, we invite you to refer to the **global response Q2** and **A2**. \n\n**Q8** From Table 1 and Table 2, it seems that the methods improve new task learning more than reducing catastrophic forgetting. \n\n**A8** Thank you for your  question! We believe that average accuracy is more easily improved because it benefits directly from the effective learning of new tasks and preservation of old task knowledge, which is often the primary focus of many continual learning methods. In contrast, reducing forgetting and improving backward transfer requires maintaining the performance of all previously learned tasks, which is a more complex and challenging problem.\n\nReference:\n\n[1] Multi-layer Rehearsal Feature Augmentation for Class-Incremental Learning, ICML 2024\n\n[2] Rethinking Momentum Knowledge Distillation in Online Continual Learning, ICML 2024\n\n[3] CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning, CVPR 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "J4ROcnILHM", "reviewer": "Reviewer_g3Gj", "summary": "The paper \"Model Sensitivity Aware Continual Learning\" introduces a novel approach to continual learning (CL) that addresses the trade-off between retaining previously acquired knowledge and excelling in new task performance. Traditional CL models often face the dilemma of catastrophic forgetting or overfitting to new tasks. The proposed method mitigates model parameter sensitivity to updates, ensuring that minor parameter changes do not significantly impact the model's performance. This is achieved by optimizing the model's performance based on the worst-case scenario within a parameter distribution neighborhood. The approach is compatible with existing CL methodologies, offering substantial improvements in retaining old knowledge and enhancing new task performance, supported by theoretical and empirical evidence.", "questions": "Can the authors provide more insights into the computational overhead introduced by the proposed method, particularly in large-scale datasets or real-time applications?\nHave the authors considered any strategies for optimizing hyperparameters in a more automated fashion to improve the method's usability across different datasets and tasks?\nAre there any plans to extend this approach to online continual learning scenarios? If so, what challenges do the authors anticipate?", "limitations": "The authors have adequately addressed the limitations of their work, particularly in the context of offline CL scenarios. However, exploring the method's applicability to online CL could provide a more comprehensive understanding of its potential. Additionally, a more detailed discussion on the computational complexity and strategies to mitigate it would be beneficial.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality: The concept of reducing model parameter sensitivity to improve both retention and new task performance is innovative and orthogonal to existing methods.\nQuality: The methodological rigor and comprehensive theoretical analysis are commendable.\nClarity: The paper is well-structured, with clear and precise explanations of the proposed method and its advantages.\nSignificance: The ability to improve CL models' performance without sacrificing old knowledge retention is a crucial advancement in the field.", "weaknesses": "The computational complexity of the proposed method, particularly the calculation of the Fisher Information Matrix, might be a concern for large-scale applications.\nWhile the method is demonstrated to be effective in offline CL scenarios, its applicability to online CL is not explored.\nThe hyperparameter sensitivity, especially regarding the learning rate η, requires careful tuning, which might limit the method's usability in different settings without additional hyperparameter optimization.\nThe datasets used in this paper are outdated and could consider more recent datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your valuable suggestions and would like to express our gratitude.\n\n**Q1**  computational complexity of the proposed method and Fisher Information Matrix (FIM) for large-scale applications.\n\n**A1** Thank you for your question! Although the FIM calculation is costly, our method circumvent this difficulty by calculating the FIM in the dual parameter space of exponential family distribution, i.e., the expectation parameter space. According to theorem 3.1, the FIM in the natural parameter space is equal to the *gradient in its dual space, i.e., the expectation parameter space*. Therefore, the computation cost of FIM is not high. In addition, we conducted experiments to compare the running efficiency by integrating our proposed method (MACL) with multiple CL approaches, including DER++, ER-ACE, and LODE. The training time (in minutes) for a single epoch on ImageNet-R with ViT as backbone is shown below. Our method only marginally increases the training time by approximately 56% to 64%. This indicates that our method only introduces small computation cost and is efficient in practice.  \n\n**running time on ImageNet-R (minutes)**\n| Method | w/o MACL | w/ MACL |\n| ------ | -------- | ------ |\n| DER++  | 1.52      | 2.46   |\n| ER-ACE | 1.16      | 1.81 |\n| LODE   | 1.98   |   3.25  |\n**Q2** Application in online CL \n\n**A2** Online Continual Learning (CL) focuses on learning from a data stream with unclear task boundaries. This poses the challenge of not being able to utilize task identity to calculate the FIM. Additionally, we need to maintain  computational efficiency. To address these challenges, we explore calculating the FIM in an online CL setting using a moving average approach. Specifically, we approximate the diagonal Hessian matrix at each time step using the gradient outer product, i.e., $H_t \\approx g_tg_t^T$. To improve training efficiency, we calculate the FIM only in the first two residual network blocks (closer to the inputs) of ResNet (which consists of four residual network blocks). This is based on existing studies that show earlier layers, which calculate the input data representations, are more sensitive to task drift. We invite you to refer to the **global response Q1 and A1** for the results in online CL setting.\n\n**Q3**  hyperparameter optimization. \n\n**A3**   Thank you for your suggestions! We have explored automatic hyperparameter optimization to enhance the method's usability by utilizing the SMAC3 framework [1], a sample-efficient and automatic Bayesian optimization hyperparameter search strategy. The primary goal of Bayesian optimization for hyperparameter search is to optimize a black-box function. In the context of CL, the objective function is the average validation accuracy on the first three tasks, following [4].\nWe set the hyperparameters to be optimized, specifically the learning rate $\\eta$, within the range of $[1e-6, 1e-2]$. SMAC3 then performs the hyperparameter optimization process, iteratively suggesting new configurations to evaluate and updating its model based on the results. The selection of which hyperparameter configuration to evaluate next is determined by an acquisition function, which balances exploration (trying new configurations) and exploitation (focusing on configurations that are known to perform well). Initially, we perform hyperparameter optimization on the first three tasks, then use these optimized hyperparameters for subsequent tasks, following existing CL works [4]. The performance of the searched optimal hyperparameters on TinyImageNet with a memory buffer size of 500 is shown in the following table.\n\n**Hyperparameter Optimization for $\\eta$ on Tiny-ImageNet**\n\n|  | Class-IL | Task-IL |\n| -------- | -------- | -------- |\n|  Manually Selected    |  20.17 $\\pm$ 1.56    |   54.03 $\\pm$ 0.79   |\n|  Automatically Selected    |  20.39 $\\pm$ 1.32    |   54.27 $\\pm$ 0.83   |\n\n**Q4**  Datasets outdated   \n\n**A4** Thank you pointing out this! We conducted experiment on the recent CL datasets of ImageNet-R [2] and CUB200 [3] with pre-trained Vision Transformer (ViT), i.e., vit-base-patch16-224 as the backbone following the codebase of DER++. The results (memory size of 500) are shown in the following table.\n\n **ImageNet-R Results**\n | Method | Class-IL | Task-IL |\n | -------- | -------- | -------- |\n | DER++     |  58.29 $\\pm$ 1.78  |     86.93 $\\pm$ 0.32  |\n | DER++MACL       | **60.51 $\\pm$ 1.65**   |  **87.56 $\\pm$ 0.41**      |\n | LODE       | 74.98 $\\pm$ 0.21     |   90.22 $\\pm$ 0.39    |\n | LODE+MACL       | **75.51 $\\pm$ 0.26**    | **90.81 $\\pm$ 0.28**    |\n\n **CUB200 Results**\n | Method | Class-IL | Task-IL |\n | -------- | -------- | -------- |\n | DER++     | 41.81 $\\pm$ 1.69    |  87.16 $\\pm$ 1.09    |\n | DER++MACL       | **43.07 $\\pm$ 1.53**  | **88.03 $\\pm$ 0.97**        |\n | LODE       |  66.87 $\\pm$ 0.35    | 93.12 $\\pm$ 0.56     |\n | LODE+MACL       |  **67.53 $\\pm$ 0.51**     |    **93.42 $\\pm$ 0.37**   |\n\n**Q5** Can the authors provide more insights into the computational overhead introduced by the proposed method, particularly in large-scale datasets or real-time applications? Have the authors considered any strategies for optimizing hyperparameters in a more automated fashion to improve the method's usability across different datasets and tasks? Are there any plans to extend this approach to online continual learning scenarios? If so, what challenges do the authors anticipate?\n\n**A5**  Thank you for your questions! We invite you to refer to the question and answer in **Q1 to Q4** and  **A1 to A4**.\n\nReference:\n\n[1] SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization, JMLR 2022\n\n[2] The many faces of robustness: A critical analysis of out-of-distribution generalization. ICCV 2021\n\n[3] The caltech-ucsd birds-200-2011 dataset, 2011\n\n[4] Efficient Lifelong Learning with A-GEM, ICLR 2019", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Global Response


**Q1**   Online Continual Learning setting evaluation.

**A1**  Thank you for your suggestions! We conduct experiment in the online continual learning setting on the SOTA approach,  MKD with PCR [1,2]. We follow the same dataset split and hyperparameter setting as [1]. The results by integrating our method (MACL) with MKD on CIFAR100 and Tiny-ImageNet are shown in the following tables. MACL further improves the online CL performance. 


**Online CL Results on CIFAR100 under the blurry boundary setting** 
| Memory Size | 1000  | 2000  | 5000|
| ----------- | ---- | ---- | --- |
| MKD(PCR)         | 35.6 $\pm$ 0.66     | 44.95 $\pm$ 0.42    |  54.87 $\pm$ 0.39   |
| MKD(PCR) + MACL       | **37.2 $\pm$ 0.53**       |  **46.17 $\pm$ 0.51**   |**56.21 $\pm$ 0.43**     |

**Online CL Results on Tiny-ImageNet under the blurry boundary setting**
| Memory Size | 2000  | 5000  | 10000|
| ----------- | ---- | ---- | --- |
| MKD(PCR)         |  17.33 $\pm$ 1.28    |  29.58 $\pm$ 0.6   |  38.02 $\pm$ 1.64   |
| MKD(PCR) + MACL       | **18.21 $\pm$ 1.32**     |  **30.69 $\pm$ 0.71**    | **38.73 $\pm$ 1.56**    |


**Q2** analyse the effect of the sequence length or long sequence.

**A2** We conducted experiments by splitting **Tiny-ImageNet** (200 classes) into sequences of different lengths, specifically 10 and 20 tasks. The results for task-incremental learning (Task-IL) and class-incremental learning (Class-IL) using DER++ with a memory size of 500 are shown in the following table. These results indicate that even with longer task sequences, our method (MACL) still provides improvements over the compared methods, i.e., **more than 3.3\% in Task-IL on a sequence of 20 tasks**.

| number of tasks | 10   | 20   |
| --------------- | ---- | ---- | 
| Class-IL        |$19.38\pm1.41$      |   15.02 $\pm$ 0.53   |      
| Class-IL + MACL |**20.17 $\pm$ 1.56**    |   **16.08 $\pm$ 0.81**   |      
| Task-IL         | $51.91\pm0.68$ |51.65 $\pm$ 1.36  |  
| Task-IL  + MACL |**54.03 $\pm$ 0.79**  |**54.96 $\pm$ 0.72** |


**Q3** Apply the proposed method (MACL) for learning to prompt techniques for CL?

**A3**  Thank you for your suggestions! We conducted an experiment integrating our proposed method (MACL) with the SOTA prompt-based CL method, CODA-Prompt [3]. MACL operates on the parameters of prompt components and corresponding keys/attention vectors. The results on ImageNet-R are shown in the following table. The improvement is due to reducing the sensitivity of these parameters. This way, when the model learns new tasks, parameter changes do not significantly increase the loss. During inference, the weighted prompt remains more stable, mitigating forgetting. Additionally, MACL reduces overfitting to new tasks, thus improving overall performance.

**CODA Prompt Results on ImageNet-R**
| number of tasks | 10 | 20 |
| -------- | -------- | -------- |
| CODA-P    |  75.45 $\pm$ 0.56   |  72.37 $\pm$ 1.19   |
| CODA-P + MACL    | **76.39 $\pm$ 0.67**     | **73.42 $\pm$ 1.23**    |


**Q4** More state-of-the-art (SOTA) baseline. 

**A4** We added one SOTA class-incremental learning (CIL) MRFA [4], compared the performance on ImageNet100 with memory size of 2000. "10-10" means there are 10 classes in the base task, and each following incremental task also has 10 classes. "50-10" means there are 50 classes in the base task, and each incremental task has 10 classes. The results are shown below. 

| Method | 10-10 | 50-10 |
| -------- | -------- | -------- |
| MRFA(iCaRL)    |   67.34    |   65.15    |
| MRFA(iCaRL) +MACL   | **67.82**       | **65.76**     |

Reference

[1] Rethinking Momentum Knowledge Distillation in Online Continual Learning, ICML 2024

[2] PCR: Proxy-based Contrastive Replay for Online Class-Incremental Continual Learning, CVPR 2023

[3] CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning, CVPR 2023

[4] Multi-layer Rehearsal Feature Augmentation for Class-Incremental Learning, ICML 2024

Meta-review: This work considers model sensitivity for improving continual learning. An optimization approach based on parameter distribution is introduced to reduce forgetting and overfitting. While there seems to be a room for improvement, the authors provided sufficient updates to address the concerns regarding empirical evidence. Therefore, an acceptance is recommended. 

For the camera-ready submission, please make sure to improve the manuscript as much as possible to reflect the reviewers' feedback and concerns. In particular, please make sure to have the data distribution $D$ well defined and to have the generalization bound addressed sufficiently comprehensively. Updated empirical results should be reflected at least in the appendix with proper comments in the main text. 

Even though not commented by the reviewers, it might be also helpful to address the connection of this work to the existing flat-minima/SWAD related works.
Decision: Accept (poster)