Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Authors: Boqi Li
Youjun Wang
Weiwei Liu
Venue: ICML 2025 poster (2025)
Keywords: Representation Learning
Primary area: theory->learning_theory
TL;DR: We theoretically analyze catastrophic forgetting in two-layer CNNs under a multi-view model, revealing how learning dynamics cause forgetting.

Abstract: Continual learning (CL) focuses on the ability of models to learn sequentially from a stream of tasks. A major challenge in CL is catastrophic forgetting (CF). CF is a phenomenon where the model experiences significant performance degradation on previously learned tasks after training on new tasks. Although CF is commonly observed in convolutional neural networks (CNNs), the theoretical understanding about CF within CNNs remains limited. To fill the gap, we present a theoretical analysis of CF in a two-layer CNN. By employing a multi-view data model, we analyze the learning dynamics of different features throughout CL and derive theoretical insights. The findings are supported by empirical results from both simulated and real-world datasets.

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Boqi Li
Youjun Wang'
Weiwei Liu
Abstract
convolutional neural networks (CNNs) (Goodfellow et al., 2015), transformers (Kotha et al., 2024), and others, across different CL scenarios, such as task-increment CL (Sun et al.. 2023), class-incremental CL (Babakniya et al., 2023), CL (Jeeveswaran et al., 2024) and so
Continual learning (CL) focuses on the ability of models to learn sequentially from a stream of tasks. A major challenge in CL is catastrophic forgetting (CF). CF is phenomenon where the model experiences significant performance degra- dation on previously learned tasks after training on new tasks. Although CF is commonly ob- served in convolutional neural networks (CNNs), the theoretical understanding about CF within CNNs remains limited. To fill the gap, we present theoretical analysis of CF in two-layer CNN. By employing a multi-view data model, we ana- lyze the learning dynamics of different features throughout CL and derive theoretical insights. The findings are supported by empirical results from both simulated and real-world datasets.
on.
Alleviating CF is core challenge in CL, and many methods (Kirkpatrick et al., 2017; Serrà et al., 2018; Gao & Liu, 2023; Wu et al., 2024; Gao & Liu, 2025b) have been proposed to address CF in recent years. Despite tangible improvements in the field of CI., the theoretical understanding of CF re- mains underexplored. Recent theoretical works investigate CF from the perspectives of optimization (Doan et al., 2021; Lin et al., 2023; Evron et al., 2022) and statistics (Goldfarb & Hand, 2023; Zhao et al., 2024). However, few of these studies analyze the learning dynamics in non-linear models, particularly CNNs.
To bridge the gap, we investigate CF in a two-layer CNN model. Following Allen-Zhu & Li (2023); Shen et al. (2022), we employ a multi-view data model and focus on task- incremental binary-llasiffiattonn CL. In multi-view data model, the data consists of features and noises. By an- alyzing the learning dynamics of the features and noise, we identify which components the model ultimately learns after several time steps. Our data model includes four com- ponents: task-specific features, general features, random features, and background noise. Using gradient descent (GD) update the model, we analyze the learning process of the model. Under mild assumptions, our theoretical re- sults offer insights into the underlying causes of CF. Our findings suggest that when task-specific features are learned more rapidly than others, the learning of general features with low signals is hindered. Additionally, when training on new tasks, we observe that the random feature has a large signal, CF occurs, causing the model forget knowledge from previous tasks. We also offer theoretical insight into the effecttivnness of replay-based methods (Rolnick et al., 2019; Jeeveswaran et al., 2023), which are commonly used to mitigate CF.
1. Introduction
In recent years, large volumes of data are generated. De- signing general machine learning model to adapt quickly to changing environments becomes a significant concern (Ebrahimi et al., 2021; Goldfarb et al., 2024). One of the popular approaches is to train the model on a sequence of multiple tasks, which is called continual learning (CL) (Parisi et al., 2019; Chen Liu, 2018; Wang et al., 2024; Gao & Liu, 2025a). A major challenge in CL is that after training few new tasks, the model "forgets" knowledge learned from previous tasks with a significant performance degradation, which is also known as catastrophic forgetting (CF) (McCloskey & Cohen, 1989; Goodfellow et al., 2015; Shi et al., 2021; Korbak et al., 2022). CF is widely observed in various models, such as linear models (Evron et al., 2022),
'School Computer Science, Wuhan University National Engineering Research Center for Multimedia Software, Wuhan University Institute of Artificial Intelligence, Wuhan University Hubei Key Laboratory of Multimedia and Network Communica- tion Engineering, Wuhan University. Correspondence to: Weiwei Liu <liuweiwei8636 gmail.com>..
We perform numerical analysis and conduct experiments on real-world datasets to further validate our theoretical find- ings. In numerical analysis, we validate the key components
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
derived from our theoretical results. The experiments in real-world datasets further support our theoretical analysis.
the above works, we study CF in a two-layer CNN using a multi-view data model.
The contributions of our work can be summarized as fol- lows:
Theoretical Analysis in Two-Layer CNN Model. Re- cently, lot of works study the learning dynamics in a two-layer CNN using a multi-view data model under an over peeeeerramttiiniioonnn regime. The multi-view data model is firstly proposed by Allen Zhu & Li (2023), with a frame- work study the learning dynamics in ensemble learning, knowledge distillation, and seff-distillation. Within similar framework, Jelassi & Li (2022) investigate the effect of mo- mentum in (S)GD for improving generalization of CNNs, Shen et al. (2022) study the effect of data augmentation, Huang et al. (2024) investigate the federated learning. Li & Liu (2024) investigate the backdoor poisoning attacks and Kou et al. (2023); Meng al. (2024) study benign overfitting in CNN models.
We provide novel theoretical framework to analyze CF in a two-layer CNN, focusing on the learning dy- namics of different features and noises.
Theoretical findings reveal the existence of CF for two reasons: the task-specific feature has a larger signal than the general feature, and the task-specific feature appears as random feature with strong signal in other tasks.
We validate our theoretical results using experiments on both simulated and real-world datasets, demonstrat- ing the practical relevance of our findings.
However, previous works mainly focus on learning sin- gle task, while we analyze the learning dynamics across a sequence of tasks. A major difficulty in our setting is that, at the beginning of learning a new task, the weights of the model are not randomly initialized, which is not discussed in the above works. Therefore, it is imperative study the dynamics of CL in two-layer CNN model.
2. Related Works
Theoretical Analysis on Catastrophic Forgetting, A few works study CF in linear regression problems. Evron et al. (2022) show connections between CL in the linear setting and alternating projections and the Kaczmarz methods. The authors also study the effect of task orderings and similar- ity on CF. Li et al. (2023) investigate regularized CL with two linear regression tasks. Goldfarb & Hand (2023) hibit the effect of over- aneeterramttiiznttionn on performance degradation due to CF. Lin et al. (2023) provide an explicit characterization of CF and generalization error to analyze how over paramttrrizaaionn task similarity, and task order- ing are relevant to CF and generalization error. Zhao al. (2024) provide a statistical analysis of regularization-base CL a sequence of tasks, and study the effect of regu- larization terms on model performance. Li et al. (2025) provide a theoretical study of mixture-of-experts models in CL. (Ding et al., 2024) analyze factors contributing to forgetting under linear models in CL. Zheng et al. (2025)
3. Preliminaries
Consider a standard task-incremental CL problem, where a sequence of tasks indexed by t = 1,..., T. Each task T is binary classification problem over dataset S(F) n-
(xf y3
(7) }
i-1 Let D(-T) be the distribution of 2(r)
(x yO over 2 = X'x In this work, we consider the task lask-incrementll scenario with T 22, meaning the model sequentially trains on a pair of tasks.
Data. In each task т, we assume the data points are drawn from a multi-view data model (Allen-Zhu & Li, 2023). In our multi-view data model, each data point x consists of P + 3 non-overlapping patches x = (x1,..., xp+3) E Rdx(P+3), and each patch is a vector of dimension d. We suppose that there exists a general feature space O E Rd spanned by three basis vectors Features (ei, 622 Crob] In each task, the task-specific, general, and random features are defined as u a.e,, v(+) Broh, ((7) 0re Additionally, P background noise vectors p p=1 exist in x, and the distribution of {(r) is denoted by De. We define the feature-noise multi-view data model as follows:
examine memory- based CL under overparameterized linear models.
There are also some works that investigate CF in neural networks. Doan et al. (2021) analyze CF under the Neural Tangent Kernel (NTK) regime, and they propose measuring task similarity using an NTK overlap matrix. Andle & Sekeh (2022) show a theoretical analysis of information flow through layers in linear networks for task sequences, and the effect of information flow on learning performance. Cao et al. (2022a) propose a provable CL algorithm that maintains and refines the internal feature representation. Benjamin et al. (2024) introduce the concept of NTE in CL. reformulating a single neural network as an ensemble of fixed classifiers. Their analysis focuses on the linear networks with non- linear activation fucntion. Different from
Definition 3.1. In task r. given vectors u((), v(T), ((T), and the noise distribution Dz. let U be the uniform distribution and [n] is used to denote the set {l...., n). A data point z = (x, y) is drawn from the distribution D. which is defined as follows:
2

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
1. Draw two Rademacher U (+1,-11²
variables
(y.e) e)
use q = 3 while our results can be extended to any q 2 3. The CNN predicts label y sign(F(x)), where sign()) denotes the sign function.
2. Given e,e, arbitrarily choose three distinct patches p", p and p.. The feature patches are set as { {xpx, Xp', X = yy,,
Training Process. The entire training process can be split into two stages:
In the first stage, the model begins with initialization. We use Gaussian initialization to initialize the weights of the first layer, i.e. Ve E [C], w (1) (0) ~ (fla), while the last layer is fixed and set as the all-one vector, i.c. Ve E [C], A = 1. We then use the training data from the first task Sir to train the model. After Ti epochs, we stop training the model, and the model at the end of the first stage is denoted by F = F/! with parameters w: (1) (Ti).
3. Each remaining background patch pf E [P + 3] \ {p",p" ""',",'' ,6) of is set as Xfx = &. where & Dg-
Remark 3.2. In our data model, each task has both task- specific and general features. In a pair of tasks, v()) = v(2) v is an aligned and robust feature that allows the model to achieve high accuracy on both tasks. Using task- specifie features u (r)), the model can only correctly classify the data points in task т. Since ((3-T) = a_ (T)) if the model has not learned u (3- and a is large, the sign of the model's s output in the second task matches the sign of c, which is independent of y. The presence of a random feature ensures that the task-specific feature can only used for classification within its corresponding task.
In the second stage, the model is initialized as (2) (0) w( (Ti), and we use the training data of the second task tr to train the model for additional T2 epoch. The model at the end of the second stage is denoted by F
((2)
p(2) with parameters w! (T2).
To ilustrate the data model, we consider an example in two- task incremental learning based on Definition 3.1:
We utilize the logistic loss ((((x)))) ()(x),,)= = log (1 + e -)((()) as the loss function and apply gradient descent (GD) to optimize the parameters. Throughout the learning process, the last layer remains fixed. Given learning rate n, round t of stage E {1, the parameters of the network are updated as follows:
Task I: Data x with label y contains features (yagel, ya, e and noises.
Task 2, Data x with label y contains features (yo,e e2, yO (0(ee) } and noises.
(+) +1) - (1)
Remark 3.3. In such an example, all of (el, Grob: e2) appear in each task, but only ya,c, and Cro are correlated with the true label in task 5, which means the two features can be used for classification in task 60ge3-r is called random feature, which is uncorrelated with the true label. In such a model, the task-specific feature is only used in its respective task. Our Ote, Og controls the strength of the features.
7
Σ ((F(x)"), 1/ " (FF(((()))) ())
"
a
P43
"
(7) X
(F(?")
3o ( (w!
Mi
1=pp=1
The CF is formally defined as follows:
Definition 3.4 (Catastrophic Forgetting). We say that CF occurs in the training process when the following conditions are satisfied:
In this paper, we assume De is zero-mean Gaussian distri- bution (1 of, (Ia cf czee Crob (rob)Tob)) )). We use I to denote the index set of S. The training set S! con- sists of (T) independent and identically distributed (i.i.d.) data points drawn from Dr. In this work, we suppose that VT, nT n.
1. At the end of the first stage, with a high probability 1 - ō1, the model correctly classifies the test data from the first task:
Model. We use a patch-wise CNN architecture F(x) with C' channels, which is defined as
P
(xx)))=() (x)())()) > 0 > 1 1 81.
2. At the end of the second stage, with high probability 1 82, the accuracy on the first task degrades signifi- cantly:
C
F(x)= = Σ X Σ & ((Wexx))),
P (x.))~"()) (x) > 0] < 1/2 82.
where {wi,..., } are the parameters in the first layer, [AI...,,Act }c} are the parameters in the last layer, and Ø(z) 24 is the activation function. In this work, we
Remark 3.5. For a binary classification task, an algorithm that outputs randomly would achieve approximately 50%
3

---

We first simplify our CNN model as C== C=1 and consider four ideal models. Let ε 0 be a large constant. First, we consider task-speeific models. Using Wi = x)) the model achieves low training and test error in the first task but has low accuracy for data drawn from the second task. Similarly, using w1 = x2) the model only achieves high training and test accuracy in the second task and misclassi- Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
accuracy. The above definition implies that when CF occurs, the model's performance on the first task is close random guessing. By fixing the last layer, we can focus on the learning dynamies of features to find the hidden reason behind CF. It is crucial for identifying whether and how certain features contribute to CF.
In contrast, we consider two task-robust models. Firstly, employing the robust vector v(1) v(2) allows for accu- rate data classification, expressed as Wi = w = = successfully categorizing both training and test datasets in both tasks. Secondly, the model can be built using (2) and u(2), represented as W1 = w-en = -ku()) + ku!2), When
ac < aua both vectors u1)) and (2)) can effectively classify the data across both tasks.
We use the standard asymptotic notations O O Q in this pa- per, and O θ O are used to hide the log factors in O, O, Ω respectively. We use f I o(g) to denote that for every a > 0, there exists xo such that for all x > To we have f(x) S ag(x)) We use f 2 w(g) to denote that for every > 0, there exists zy such that for all x > zo we have [()) 2 og(x)) We suppose the following conditions hold in our analysis.
Among the four models outlined previously, the goal of the user is to develop model with the ability of generalization, akin to the last two models wrob and w-gen. We then ana- lyze the gradient descent dynamics involved in sequential task learning and investigate the conditions under which generalized models can be acquired. For the remainder of this section, we assume C' > 1 and set ag = 0. The condi- tions involving og in Condition 3.6 can be ignored in this section.
Condition 3.6. In the training process, for any T E {1, we have
1. The feature vectors u(r), v(r), ((T) are mutually or- thogonal, and the norms of (T)), v(r) c(T)) satisfy
4.1. Features Learned by the Model in the First Stage
S Ce. ag < au (1).
At the beginning of the first stage, the weights of F are closer to the initialization. The following lemma shows that the inner products of weight vectors and patch vectors are all small, meaning that none of the channels capture the features at initialization.
2. The network is over-parameterized, and nP S o (va) The number of channels C' is of the order of logarithm of d, i.e., C = O(log d).
Lemma 4.1. Given the weights w() initialized as w: (1 (0) ~ (O, ouTa), at the beginning of the first stage, with a probability of - O polycii ). we have
3. The network initialized with a small variance, i.e., Poo S (1) and the variance of background noise is large, i.c., (1) S og S (1/ (Poo)).
4. The learning rate is not large, i.e., 17 S.OO 0 ())
Vee fucl v(), (())
w (0), )) Sō ( oo)
mmxx CE|C1
5. Signal-to-Noise Ratio (SNR) is sufficiently large to ensure that the model does not fit the noise during the training process, i.e., / w (1).
Yee ful), v()) c(0)
max CE|C]
w c (0),e)
2 ((ee
Vi |n] pf
(0), (1))
< (
max cE(C]
Remark 3.7. In Condition 3.6, the condition on dg is to ensure that at the beginning of the training process, the network cannot casily classify the data with the speciffe feature. We use small do to ensure that the output of the network after initialization is the order of o(1). The last condition is lower bound for SNR3 = / and similar condition of SNR is also shown in Cao et al. (2022b)
W Pf max
(w!
(())=e..) 2 (()oo))
eE|C]
The proof of Lemma 4.1 can be found in Appendix C. We then analyze the dynamics of (w! (1) .u)))
d (w!". di
(1) (u())
4. Theoretical Insights in Learning Process
(1) E ((w5", x10) ( (.g.uuy)
Vi
n
1 n At initialization, Vi E [n]. F(x,) = o(1), and & el e(y ( F(x(1)) x 1/22. The dynamic reduces to
Σ E ((w5), ( ||
2
fies half of the data points drawn from the first task. Both of these models fail to generalize robustly to unseen tasks.
4

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
an ODE. Let g(t) = (w?") (2) : I we have gt) x
Note that u(2) oc (2), we have
11 "f (((())) When g(() S 1, we have (0(())1)) = oz due to the definition of Φ. Then at T = 1 o250
(1) d d(w (w "Ww'ule .u(3)
di
@ 1 ). we yield max. (w (1) u(())
((1), which
= ((n=)) -
au 222) ø ((i)) |
implies that u has been captured by the NN.
By replacing v(1) to n we yield that
The time step that max.
We ul?)) 2 22(1) is also at
d (w!") v(y)
least = -
7
Finally,
after the task task-specific feature u or general feature v(1) is captured by the model, the condition that Vi E [n]] F(x:) o11) and f ll (x))(x)) e -122 no longer hold, and the update speed of each component decreases.
dt
1
Σ
Vi (1 1/ ((w?9)
(x10) way)
o ((w(),vcu)) /
To summarize, in the first stage, if the general feature (1) has a minimal signal, the model only learns the task- specific feature u and the updates of (we v(1) and
We (a)) are both slight. From a practical perspective, the existence of the general feature is difficult to meet in two randomly sampled tasks. Our experiments demonstrate that the general feature does not exist in the majority of cases. When (w .n1)) 2 52 (1). we stop training the model in the first stage.
1 ). we yield max. (we") (1) v()) 0002) At T = 0 (mioz)
2(1). When T!!) < T!", which means 0h & Ouu the feature (2) cannot be captured by the CNN model. The above analysis shows that u is easier to learn than (!)) when (1) has a stronger signal than v11)
As for (()), let Iit) and II(T) denote the sets {i: E I07)8) yi = 4i and { y + at). respec- tively. m(!)) and ni are used to denote the sizes of I(r) and I(r) respectively. We have
4.2. Model Forgets Task-Specific Feature in the Second Stage
In the second stage, we show that the model tends to forget u when the norm of random feature x(2) becomes large.
Let Ae (c(²) = (w! + 1). ((() (w ww (T) (t),
d We (
since d²(.) is an even function, we have
dl
Ae (c(1)) ne 11511 28 ((w!" (w."().s) (g - -g(2))
n
n- Σ>
E/a ((w!))x/1)
(1) (/00550))
Vi
p=1
n
- where G(T) = - (F((x)) v) and g Eiezy> (F(((x))) wd. The update direction of (wor C)
1
Σ
14- Σ (.) o' ((m2",<=1) (
1EZ=
1EI
depends on the sign of g(2) (g - gg)), which is different from the task-specific and general features. Let (2)
=0 (1) - n14i) Φ. ((w5), 1
h (e) = -ceeer Φ (( (w!?) .e)). we rewrite e' as
Since e is a sub-Gaussian random variable, with a high probability, we yield - -1)[)) I ÖVv).. Even if aç au. the update speed of max. ( (we", (1) (a))
E' (F(xx)) = (1+ + exp (BFF (xx)))-
(1 +exp (iih (111(2) tysh (vsv(2) +yih (A5(2)))
still slower than max. ( we un)). The time step
(1 +exp (1 (112)) +h (v(2)) + +yscch ({(2))
that maxe (w:") ()) )())) reaches 22 (1) is at least T.))) =
[+eexp 1 (((1))) +h (v(2)) +/ (c00)
, Z(4);
12 (1) -n )o00 1 n
2 l
(1+exp h (1(2))+ + (v(2)) - h ((2))
, T(2)
5

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Since (211)/((22) + (exp (22 k1)), we have
Given ñi data points from the first task, the update rule for Ac(u()) and A.(u()) can be rewritten
/ (2) = (n (2) (2)
4 exp (-2h ((()))
(c (0:0))mmu00 (2) ( + o + -a.00)acch - ac
=0 n (2)
nz
-1 exp (-2aça 3/ (u (u))
(u =m(u (2) (a( - a + + + -g/)..
where m(c) = (n + 7)) de ||e|| o' ((wa(t), e)).
On the one hand, when og I o (au)) yield that max. we (2) (())) I o (1), and & (((F(x))
Note that T() e (
e ( while gl
-1 -1/2 + o (1). In such case,
+
(n), and og O (a.). We can replay
at least 2 9 (1) data points from past tasks to ensure that g go) + + - 0
g(e)| = 0 (1== -n=)) S 50(v))
(2) > 0,
and the update of (w?²), . (())) has a relatively slow speed. (2)
which suppresses CF in the second stage. Moreover, we should add ñ S O (m) to ensure that 12 can also be suc- cessfully captured by the model. Adding data points from the first task enables the model to correctly classify data points from both tasks.
On the other hand, for some ag 2 (awu), we have max. (e <(0)) 2 (1) and / can achieve the order of (2 At the beginning of the second stage, (u(2) = a 3h h (c S (1), h (y(2)) I (1), we yield that - (F(x:), v)) S (1) for i E I while (F(x))) v) 2 (1) fori E I We then have that
(2)
5. Main Results
In this section, we show the theoretical results in this work. The proofs of Lemma 5.1 and Theorem 5.2 are deferred in Appendix D and the proofs of Corollaries 5.3 and 5.5 and Theorem 5.4 can be found in Appendix E.
g = -0 (nz) = 2 (n),
and the model forgets u())
The following lemma shows that the background noises are not fitted by the network during the learning process.
The speed of forgetting depends on og. As ogg increases, max. (we, ) decreases to a low value. Additionally, as the task-specific feature (2)) is gradually learned by the model, the speed of forgetting also decreases. As a result, after sequentially training on two tasks, the CNN model gains the ability of generalization in two ways:
Lemma 5.1. In the first stage, under Condition 3.6, and given T 2 Ω (nTTOT)) ). for any 0- <<< t S T,i E I , E
Pf. we have following:
max cEC'
( (1)
max dEC
w (.) (1)
So (oo0g).
1. When ar 2 (aw), the model learns the general fea- ture (11 (2), and behaves like F' with wob.
Moreover, we have
w ()), (.! S (
max REC
2. When 0g I 0 (a.). the model learns both (1)) and 1 and behaves like F' with wgen
In the first stage, we show that when the task-specific feature has a larger norm than the general feature, the model only learns (1) at the end of the first stage.
CF does not occur in the above two cases. Instead, CF occurs when a, < o (a,) and ag 2 Ω (a.). In such a case, the model ends up classifying the data using only feature u(2), resulting in a high error on the first task. We then show the formal results in Section 5.
Theorem 5.2. In the first stage, given a training set ((1)
with size n, a S o (aw), there exists T()) =Ō gopaa
such that for any T(1) 2 T(1), the network Froo fits all training data points with high probability:
4.3. Replay-Basee Methods Can Suppress CF
nn p22
F(L) Vi E s(D), De Tiy
(x,) 2000
When 0e I (a.) and ag 2 Ω (aw), our analysis im- plies that CF occurs with a high probability. The remaining question is how to suppress CF. In the second stage, replay- based methods add data points from previous tasks into the training set to suppress CF. We then analyze the effect of replay-based methods.
poly(d)
Moreover, F(1) achieves a high accuracy on test data points at T
P ((.N))-! (opee 2 21-0 'Pp²C poly(d)
6

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
We stop training the model at (()) in the first stage. At the beginning of the second stage, we initialize Fd²) = =()) T
1. The general and robust feature v(1) has a low sig- nal, while the task-specific feature u(l) has a rela- tively large signal. As shown in Corollary 5.3, if 0y S o (au), after training in the first stage, only u()) is learned by the model.
The parameters are set as Ve E [C), we (0) We
(2) (T()).
We have the following corollary.
Corollary 5.3. At the beginning of the second task, if au 2 (at.), we have
The task-specific feature from the first task manifests as a random feature with a strong signal in the second task. If ag 52 (aw). Corollary 5.5 shows that the model forgets u (1) while learns u (2) in the second stage.
W (2)) 11 (2)
= 3 (apau),
max cE(C]
c
(0), (2)
= =6 (ooas).
max cE[C]
Remark 5.6. Based on our analysis, CF can be mitigated by violating the two conditions identified above, which is empirically supported by simulated data in Figure 2 of Sec- tion 6. Furthermore, our findings indicate that CNNs trained with (S)GD tend to learn the features with the strongest signal rather than the most robust ones. When the robust feature has a weak signal, the CNNs fail to capture the ro- bust feature during training, which highlights the necessity of developing robust training algorithms that encourage the learning of robust features to prevent CF. In addition, by fixing an and ag and increasing ap, the similarity between tasks increases, and our results suggest that CF is less likely to occur when learning on such similar tasks. We hope that our findings will inspire future research to investigate CF from the perspective of feature learning.
(0), u (1) 2 (1).
max cE[C]
In the second stage, the following theorem shows that if the has a significant norm, CF occurs, and the model cannot achieve high accuracy in the first task again.
Theorem 5.4. In the second stage, given a training set S!P) with size n, there exists T(2) ① ( such that for T(2) 2 T(2), the network Fres fits all training data points with a high probability:
pe 1 poly(dd polytn)
P P Ur
FTP ( 2 ((11)21-0
Moreover, FTON achieves a high accuracy on test data sam- from the second task:
6. Experiments
xx) > 21-0 2 (potlaa) +
n poly(d) polyín)
P ((x))=(())
In this section, we conduct experiments on both simulated and real-world datasets to validate our findings.
If o. S (au), and og 2 52 (arx), Frrey achieves a low accuracy on test data sampled from the first task
6.1. Numerical Analysis on Simulated Dataset
nP2C
We conduct numerical analysis simulated data, drawn from our data model, and use the CNN model defined in Equation (1). To analyze the feature extraction capabilities of the model, we fix the last layer of the CNN.
P (xx>>>f yf ( ( >0|= 2 +0
poly(d) polyto
We stop to train the model at (2)) = (2) in the second stage. The following corollary demonstrates that the model forgets the task-specific feature of the previous task.
To investigate which features the model captures during the two stages of training, we generate images containing only u, v, and Ç. We evaluate the model at the end of both the first and second stages, and the results are shown in Figure 1. We set a, = 1, a, = 0.1, and a 0.9. In the first stage, as shown in the first column, we find that only n()) is captured by the model using two channels, while the learning of (1) and (1)) progresses slowly, which aligns with Corollary 5.3. In the second stage, as shown in the last two columns of the first row, we observe that u!!) is forgotten by the model, and (2)) dominates the model's output, which aligns with Corollary 5.5. Moreover, the images in the last row of Figure illustrate the model's 's output with different inputs. We find that the output of the model is dominated by (!)) in the first stage, while dominated by (2) in the second stage.
Corollary 5.5. At the end of the second task, for T(2) a 2 w (aw) and ag 2 Q (au), we have T )
w!!" (e) (1) <o(),,
max cE(C]
( (2) (T()), ) v(2) ē (ooar),
max eE[C]
( (2) (T(2)), (T(2)) (2) 2 (1).
max
<E|C]
Through our theoretical analysis on the entire learning pro- cess in CL, we show that CF occurs during the learning process due to the following reasons:
7

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Stage Task
Stage Task 2
Stage Task
Stage Task
Stage Task
Stage Task
Stage Task
Stage Task
Nlote
Antiooaa Took Datr Num
Apta,
Addioonal Tosk Data Num
Alptan
Aatiooaa THNE TiMes
Festure
Patahooonal Tooks
Dota
Dara
Figure 2. The effect a and ag on the model's performance. First Row: Performance of the model in standard CL setup. Second Row: Performance of the model when additional samples from the previous task are added in the second stage to suppress CF. The X-axis represents the number of data points n, and the Y-axis represents the number of additional data points from Task-
6.2. Experiments Real-World Datasets
opoch
opoch
epoch
opoch
In real-world datasets, it is challenging decompose im- ages into distinct features. Therefore, we focus on studying the features extracted by the model in the representation space. We conduct experiments on CIFAR-10, CIFAR 100 (Krizhevsky et al., 2009), and Tiny- ImageNet (Deng et al., 2009) to evaluate the model's performance on real-world datasets. For each dataset, we split the data into K/2 binary tasks, where K is the number of classes. We then sequen- tially select pair of tasks, and train the model on the pair of tasks.
Figure 1. The output of each layer in the CNN model using differ- ent inputs. Top three rows: The output of each channel the first layer. Bottom row: The output of the last layer.
We then set the norm of au = 1, and vary on and ogg within the range of [0, 1] study the condition under which CF occurs. In the first row of Figure 2, the results in the first column demonstrate that at the end of the first stage, the model correctly classifies the data points drawn from the first task. At the end of the second stage, as shown in the last column, the model also correctly classifies the data points drawn from the second task. However, the results in the third column show that the value of ar and aç affects the performance of the model on the first task in the second stage. For a small aee, as ag increases, the model struggles to achieve high accuracy on both old and new tasks. We also observe that as the norm the general feature (2) increases, CF is suppressed even when aç large. The findings support Corollary 5.3. Additionally, in the second column, we observe that when a, oa 1, the model successfully extracts the general feature, achieving high accuracy on the second task without needing to train on the second task's 's data points.
Our first empirical observation is that, in practice, the model is easier to extract non-robust features that do not generalize to unseen tasks in most cases, as shown in Figure 3. This implies that for independently sampled tasks, achiev- ing shared general feature is difficult. In CIFAR-1O, 10, for example, when the model is trained on Task-0, the model successfully extracts the general feature which can be used in Task-4. However, in most cases, the model tends to ex- tract task-specific features for each task. Therefore, this empirical result aligns with the condition a₂ S (a.)).
CIFAR10
CIFAR100
TINY WAGENET
We further investigate the effect of the replay-bases method. We fix a, 1, a, 0.5, aç- 0.8 and show the results in the second row of Figure The third column in the last row shows the performance of the model on the first task at the end of the second stage. The result demonstrates that when no additional data is used, CF occurs in the second stage. However, by incorporating data points from previous tasks into the training set, CF is suppressed. These results further validate our theoretical insights, as shown in Section 4.3.
Figure 3. Overview CF in different datasets: CIFAR-10 (Left), 110 (Middle), and ImageNe (Right). We record the model's performance on the second task at the end of the first stage. A deeper color means better performance of the model.
8

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Partaro Speee
Deta Them Teak lan skall
Ini in
Deta fre Teek
7. Conclusion
In this work, we theoretically analyze the condition that CF occurs in two-layer CNN model using multi-view data model. We consider task-incremental CL scenario. Our theoretical results demonstrate that, in pair of tasks, if the general feature is either absent or has low signal, the model will learn the task-specific feature during the first stage. Moreover, if the task-specific feature has large norm in the second task, CF manifests in the second stage. We also provide theoretical insights into the effectiveness of replay- based methods. Finally, experiments on both simulated and real- nol-world datasets validate our findings.
Peatura Spaos: Stape
Spaos: Stage
fra Tade
Data fra Tade
2 Haxs
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
A Acknowledgments
Figure 4. Using T-SNE visualize the feature space at different stages of CL when sequentially train the model on the Task-0 and Task-3 in CIFAR-IO. First row. The model randomly initialized. Second row. At the end of the first stage. Third row. At the end the second stage.
We would like to thank Xin Zou for helpful suggestions. This work is supported by the Key R&D Program of Hubei Province under Grant 2024BAB038, the National Key R&D Program of China under Grant 2023YFC3604702, the Fun- damental Research Funds for the Central Universities under Grant 2042025kf0045.
We then use T-SNE (van der Maaten Hinton, 2008) to visualize the feature in both the first and second stages of CL. As shown in the second row of Figure 4, when using the same feature extractor, the features from the first task are clustered, while the features from the second task are not. This suggests that the features extracted in the first stage are task-specific. As features from task I and task exhibit significant overlap, which aligns the condition 0g 2 Ω (oxw)- Additionally, we calculate the maximal singular vector in the feature space. The result, shown in the last column of Figure 5, demonstrates that in the direction of the maximal singular vector, the features from the first task have weak signal in the second stage, implying that the model forgets the learned features.
References
Allen-Zhu, Z. and Li, Y. Towards understanding ensem- ble, knowledge distillation and self-distillation in deep learning. In ICLR, 2023.
Andle, J. and Sekeh, S. Y. Theoretical understanding of the information flow on continual learning performance. In ECCV, volume 13672, pp. 86-101, 2022.
Babakniya, S., Fabian, 7., He, C., Soltanolkotabi, M., and Avestimehr, S. A data-free approach to mitigate catas- trophic forgetting in federated class incremental learning for vision tasks. In NeurIPS, 2023.
Benjamin, A. S., Pehle, C., and Daruwalla, K. Continual learning with the neural tangent ensemble. In NeurIPS, 2024.
8.10
8.05
3.00
Cao, X., Liu. W., and Vempala, S. S. Provable lifelong learning of representations. In AISTATS, volume 151, pp. 3334-6356, 2022a.
Figure 5. Inner product of the features with the maximal singular vector at different stages of CL. The label Ta-Cb indicates that the data drawn from class-b in th task. We choose Task-O and Task-3 in CIFAR-10 as the first and second task.
Cao, Y., Chen, Z., Belkin, M., and Gu, Q. Benign overfitting in two-layer convolutional neural networks. In NeurIPS, 2022b.

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Cao, Y., Chen, Y., and Liu, W. Prevalence of sim- plex compression in adversarial deep neural networks. Proceedings of the National Academy of Sciences, 122 (17):22421593122, 2025.
Gong, X., Yuan, D., and Bao, W. Understanding partial multi-label learning via mutual information. In NeurlPS, pp. 4147-4156, 2021.
Gong, X., Yuan, D., and Bao, W. Partial label learning via label influence function. In ICML. volume 162, pP- 7665-7678, 2022.
Chen, Y. and Liu, W. A theory of transfer-based black-box attacks: Explanation and implications. In NeurIPS, 2023.
Chen, Z. and Liu, B. Lifelong Machine Learning, Second Edition. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, 2018.
Gong, X., Yuan, D., and Bao, W. Discriminative metric learning for partial label learning. IEEE Transactions on Neural Networks and Learning Systems, 34(8):4428- 4439, 2023a.
Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248-255, 2009.
Gong, X., Yuan, D., Bao, W., and Luo, F. A unifying probabilistic framework for partially labeled data learn- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):8033-8000
Ding, M., Ji, K., Wang, D., and Xu, J. Understanding forgetting in continual learning with linear regression. In ICML, 2024.
2023b.
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., and Bengio, Y. An empirical investigation of catastrophic forgetting in gradient-based neural networks. CoRR, abs/1312.6211, 2015.
Doan, T., Bennani, M. A., Mazoure, B., Rabusseau, G., and Alquier, P. A theoretical analysis of catastrophic forgetting through the NTK overlap matrix. In AISTATS, volume 130, pp. 1072-1080, 2021.
Huang, W., Shi, Y., Cai, Z., and Suzuki, T. Understand- ing convergence and generalization in federated learning through feature learning theory. In ICLR, 2024.
Ebrahimi, S., Petryk, S., Gokul, A., Gan, W., Gonzalez, J. E., Rohrbach, M., and Darrell, T. Remembering for the right reasons: Explanations reduce catastrophic forgetting. In ICLR, 2021.
Jeeveswaran, K., Bhat, S., Zonooz, B., and Arani, E. Birt: Bio-inspired replay in vision transformers for continual learning. In ICML, volume 202, pp. 14817-14835, 2023.
Evron, I., Moroshko, E., Ward, A., Srebro, N., and Soudry, D. How catastrophic can catastrophic forget- ting be in linear regression? In COLT, 2022.
Jeeveswaran, K., Arani, E., and Zonooz, B. Gradual diver- gence for seamless adaptation: novel domain incremen- tal learning method. In ICML, 2024.
Frei, S., Chatterji, N. S., and Bartlett, P. L. Benign overfit- ting without linearity: Neural network classifiers trained by gradient descent for noisy linear data. In COLT, vol- ume 178, pp. 2668-2703, 2022.
Jelassi, S. and Li, Y. Towards understanding how momen- tum improves generalization in deep learning. In ICML, volume 162, pp.
2022.
Gao, R. and Liu, W. DDGR: continual learning with deep diffusion-based generative replay. In ICML, volume 202, pp. 10744-10763 2023.
Kirkpatrick, J., Pascanu, R., Rabinowitz, N. C., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ra- malho, T., Grabska-Barwinske A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R. Overcoming catas- trophic forgetting in neural networks. Proceedings of the national academy of sciences, 11((1))3551-3523 2017.
Gao, R. and Liu, W. Defying catastrophic forgetting via influence function. Artificial Intelligence, 339:104261, 2025a.
Gao, R. and Liu, W. Red alarm: Controllable backdoor at- tack in continual learning. Neural Networks, 188:107479, 2025b.
Korbak, T., Elsahar, H., Kruszewski, G., and Dymetman, M. Controlling conditional language models without catastrophic forgetting. In ICML, volume 162, pp. 11499- 1 11528, 2022.
Goldfarb, D. and Hand, P. Analysis of catastrophic forget- ting for random orthogonal transformation tasks in the overparameteriiee regime. In AISTATS, volume 206, pp. 2975-2993, 2023.
Kotha, S., Springer, M., and Raghunathan, A. Under- standing catastrophic forgetting language models via implicit inference. In ICLR, 2024.
Goldfarb, D., Evron, I., Weinberger, N., Soudry, D., and Hand, P. The joint effect of task similarity and overpa- rameterization on catastrophic forgetting an analytical model. In ICLR, 2024.
Kou, Y., Chen, Z., Chen, Y., and Gu, Q. Benign overfitting in two-layer relu convolutional neural networks. In ICML, volume 202, pp. 11615-17669, 2023.
10

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Sun, W., Li, Q. Zhang, J., Wang. W., and Geng, Y. Decou- pling learning and remembering: bilevel memory frame- work with knowledge projection for task-incremental learning. In CVPR, pp.
Laurent, B. and Massart, P. Adaptive estimation of quadratic functional by model selection. The Annals of Statistics, 28(5):1302 1338, 2000.
2023.
van der Maaten, and Hinton, G. Visualizing data using t- Journal of Machine Learning Research, 988):2577-
Li, B. and Liu, W. A theoretical analysis of backdoor poison- ing attacks in convolutional neural networks. In ICML, 2024.
2605, 2008.
Wainwright, M. High-dimensional statistics: non-asymptotic viewpoint, volume 48. Cambridge uni- versity press, 2019.
Li, H., Wu, J., and Braverman, Fixed design analysis of regularization-baser continual learning. In CoLLA, volume 232, pp. 513-533, 2023.
Wang. L., Zhang, X., Su, H., and Zhu, J. A comprehen- sive survey continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8):5362-53833 2024.
Li, H., Lin, S., Duan, L., Liang, Y., and Shroff, N. B. Theory on mixpure-of-experts in continual learning. In ICLR. 2025.
Wu, Y., Wang, H., Zhao, P., Zheng. Y., Wei, Y., and Huang, Mitigating catastrophic forgetting in online contin- ual learning by modeling previous task interrelations via pareto optimization. ICML, 2024.
Lin, S., Ju, P., Liang Y., and Shroff, N. B. Theory on forgetting and generalization continual learning. In ICML,, volume 202, pp. 21078-21100 100, 2023.
Zhao, X., Wang, H., Huang, W., and Lin, W. A statisti- cal theory of regularization-base continual learning. In ICML, 2024.
Liu, W., Shen, X., Du, B., Tsang, W., Zhang, W., and Lin, X. Hyperspectral imagery classification via stochastic hhsvms. IEEE Transactions Image Processing, 28(2): 577-588, 2019.
Zheng, G., Wang, P., and Shen, L. Towards understanding memory buffer based continual learning. 2025.
McCloskey, M. and Cohen, N. J. Catastrophic interfer- ence in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24: 109-165, 1989.
Zou, X. and Liu, W. On the adversarial robustmess of out-of- distribution generalization models. In NeurlPS, 2023a.
Zou, and Liu, W. Generalization bounds for adversar- ial contrastive learning. Journal of Machine Learning Research, 24:1 14:1-114::555 2023b.
Meng, X., Zou, D., and Cao, Y. Benign overfitting two- layer relu convolutional neural networks for XOR data. In ICML. 2024.
Parisi, G. I., Kemker, R., Part, L.,, Kanan, C., and Wermter, S. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-7: 2019.
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T. P., and Wayne, G. Experience replay for continual learning. In NeurlPS, pp. 348-358, 2019.
Serrà, J., Suris, D., Miron, M., and Karatzoglou, A. Over- coming catastrophic forgetting with hard attention to the task. In ICML, volume 80, pp. 4555 2018.
Shen, R., Bubeck, S., and Gunasekar, S. Data augmenta- tion as feature manipulation. In ICML, volume 162, pp. 19773-19808, 2022.
Shi, G., Chen, J., Zhang, W. Zhan, L., and Wu, X. Over- coming catastrophic forgetting in incremental fho- learning by finding flat minima. In NeurIPS, pp. 6747- 6761, 2021.

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
A. Useful Lemmas
first show the important lemmas, which are useful our proof.
Lemma A.1 (Lemma in Laurent Massart (2000)), Suppose Xi... Xn are T. i..dd Gaussian random variables with mean 0 and variance Let .1...., an be non-negative. We set
n
|a|=0 sup lal |a Σ 0f.
Let
7 = (x - 1).
Then, the following inequalities hold for any positive t:
P 2 2 2 |a/2 Vt + 2/ { exp(-t).
p p -2552/ag vi S exp(-().
Lemma A.2 (Lemma in Shen et al. (2022)). Consider independently sampled Gaussian vectors 21 ~ (0, and 82 ~ NO, TāI.). For any 8 E (0, 1) and a large enough d, there exists constants 61, 02 such that
P ((21,22) S (10102//0og(2//) 2 11
P (2), 22) 020102Vd 2 1/4.
Lemma A.3 (Proposition 2.5 Wainwright (2019)). Suppose that the variables X, = 1,..., n, are independent, and X; has mean μ4 and sub- Gaussiass parameter di. Thenfor all r 2 0, we have
P
Σ
mb) 2t s exp (X,
2 -i=1
Lemma A.4 (Lemma A.1 in Zou & Liu (2023a)). XfX NN(H, Σ) where X E R then for any A E Ruxn we have:
AX ~ N(Ap, AΣA").
(2)
Lemma A.5 (Fact A.2. Frei et al. (2022)). Let g(2) (2) =-1(11 exp(z)), any 21, 22 E R, we have
g(21)
< max (2, 2 exp(-21)
9(22)
exp(-2>) (22)
The following lemma shows lower bound for g(=)) 9
which is inspired by Lemma C.6 in Kou al. (2023).
Lemma A.6. Let g(2) = fz(2) = -1/(1 + exp(2)) Let C1, C2 E IR be two constants and suppose c S 21 S 02 2/ E R.
Then, there exists a constant c' > O such that
g(21) ((22) 2 d exep(2s - 21).
Proof. We begin by rewriting the ratio
g(21) 1 + exp(22)
exp(22) exp(z1)
exp(22 - 21) -
g(22) 1 1 exp(z1)
1 exx((1))
1 1 exp( -21)
We first consider the case where 22 21. Since exp(22 21) 2 1, we obtain
0(21)
exp(22 21) -1 + exp(-c1)
exp(22 21) exp(-c1) dexp((2 - -2),
2 1
g(22)
1 1 exp(- (c()
12

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
where d' 1+expl -c]
Next, consider the case where 22 < 21, so that exp(22 - 21) < 1. We have
((21)
2 1 + exp(22 21) -1 exp(22 z1) + exp(-c22
2 d' exp(22 -21),
(22)
cxp(-c2)
1 exp(-c2)
where d' 1+expl- c
Combining both cases, the desired inequality holds for all 21 E ((11, and 22 E R. which completes the proof.
Proposition A.7. Given standard Gaussian variable Z ~ (0, 1), then we have PP Z 2 1 2/44 1/4.
Lemma A.8. Given a training dataset ((r) containing data drawn from the data model defined Definition 3.1. Define the index sets
I(T) = i Ui (T) -
I = {i: (T) #c $4)
Then, with probability at least
O
poly(ns) ) we have
AA E 1 [
[
=ō (nt11)
(3)
and
S 11+ō (n-1/2)
max
(4)
Proof. By Lemma A.3, with probability at least
5, we have the following concentration bound for both subsets:
VI E [ T
|
n 2
s log2/))
Setting 8 500 (nll(nn) we have Equation (3).
Let A = (2nlog(2/)) We analyze the size ratio through the following steps:
I+
I=
+ A
max
max
I
IE (T=
n
7
A
2A
(
2A
-1
max IE II
1+
n
n
4A A/n
<1 1 22 A//n
4 2 (2/8)
1
Vn
2 2log(2/5)
Setting 8 = n - for constant 0, we have Vlog(2/0) = v log flog Ō(1) Thus:
4 log(2/0) = =ō (n-1/2)
- 1=2 -1
Vn
Therefore, the ratio bound simplifies to ((n-1111)) By a union bound over both subsets, the total failure probability is 25 (n) O(1/poly(1))
13

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
B. Update Rule for Each Components
Given a learning rate 7, the parameters are optimized as
wolt +1 w.(()
v((F(x;),
7
P
= =wo(t)- 17<c
yell (F(x)), vi) ((welt) x..p)) Xt.p
p=1
= -w.(t)- 7
(F(())), vi) (ww((t) Dd
e' (F(x))) (w.(t).
i=1
(F((()))) ((.(()) Eip
7
n
PEpt
(5)
Equation (5) due to the decomposition of x. We then analyze the update of each component.
Lemma B.1. "Task-Specife and General Features]. stage t, Let Dr (u(*)) = ( w + 1), u())) - (w?? (1), (()))
and Do (v(r)) = (w (t+ + 1), - w. (t))((),, (()))(()) we have
De (u(t))
n
e' (F((x) (1) )o ((w5"(())u()))) ((07)))\\(()))) |
1=1
Do (v())=
((F((("))))) Do ((w/)00,v0)))
n
Proof. We can rewrite Ae (u(t)) as
Ae (1(T) (1 1)). -(w)(()))))))) (
T
(Fx(()))))
i=1
7
(((((
( (wP70))) (wl)),
(v67,,()))
n
Me
( (al(F(( (WPP)))))))) (c (umm
M
n
A
(F((((())) (( (m
PEPS
E (F(x!")), 0 ((W90))P/u))))
|ucoo "
14

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Similarly, we have
(v(+)) (07))=((w)) (w 1), 1))) - (w!"))))
4o
n
UFFxC(F(x (3) u. (( ((wAP())e (u(t), v(t)
iml
7 (IF((("))))()))) (WWP))) (o0
π
n
(F(((((()))))) (( ( (c (ian
n
Σ (Pe((((())dd)))) (w ((1()))))) (())
PEPS
nA (FF(())))
70 ((w!"()))((())))) ( 2llnn
We conclude our proof.
Note that ø' 2 0, < 0, Ae (u) is increasing because de 1. (t+ + 1), - (w))())))()) if (.) is an
Lemma B.2. [Random Feature]. In any stage T, let Ae (5(1)) (w!
even function, we have
A. (5(1)) nd < 12 Φ ((({)
g(t),
where g(t) ((()=_oo Diggme (-e(F) (x("")) ). ))
E Einn (-e(F(x(()) (xx(")))?")) ))
Proof. We ignore T in expression, and rewrite Do (5)
Ae (5 (1,, (w!")
(1,+1))((())
ae
(1) ((F(((() " ( ( (u()))) ((0))c())
de
Ui
"
(FF (W1P0))))))))) (a (())))))))
n
Σ v (F(()((()) ((WP)))))))) (4))))))
PEPS
( ((F(
D (
(50))()))
Mi
n
ndo (<())
2
0' ((w! (t), (())(()))) g(T)
2
where ((+)= = Eiegzer (-e(F) (x(((())))
(x{")) ). ())))))) - Emnn=a (-e(r (=((((x)))) (xx"), "i We conclude our proof.
15

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Lemma B.3 (Background Noises). We use A(E ) to denote (w:" (T + (t - > and have
(t),
A pp = T
17) (x )/ ((w!7(),)!?)) (2())))))
1=1 PEP
Proof. We can rewrite A Ap5/"p)) as
A(E =
n7
Σ Vi
((F( (u))
1=1 pEP
17
Σ
" Vi
10 (P( ((PP)))))))) (v())) (.
yi
11
pEpS
Σ (F
(50) (20
n
PEPS
n
Σ
(F(((()))
((P)))))))))))) ((PO.)))) (1
n
PEPS
Σ
9i (F(((()))))) )o (( ((
n
PEPS
We conclude our proof.
C. Theoretical Results at the Initialization
With the lemmas shown in Appendix A, we can individually analyze the inner product of different components. In task the vectors
{ (u(t), v(T), <(>) are mutually orthogonal in our assumption, have (u(7) v(()) ((()){(())
(vO), ((7)) O. We then analyze the inner product of two noise vectors.
Lemma C.1. Let I(-)) {(x;, )=) be i...d samples drawn from the distribution D(7)) defined in Definition 3.1. Then, with probability at least 1 O p² p2 ) the following holds simultaneously:
Vi, i' E [n], P.P' E PS, (i.p) + (1',p',
Kelp <// S 2 V(d-V)dddaa)) (dd-))loggel log(dd)
(6)
2 S of (V log 2 logd).
Vi E [n], pS, oेt (d-3 2/ 3) log d
(7)
Proof. We construct orthonormal basis fer, e2, ero. ē ei.....
e in Rdd and define two row-orthonormal matrices:
M = =e,....,, ed- E (dd-))xd
Mi [el, e2, @rab] E 3xdd
(7) Then for any i E [n], p E Pf, we decompose the Gaussian vector St.p as:
< - MM Me(p) M M M
Based on our definition, each <!p projected components satisfy:
N'0, od (la- elel - e - erobeTT))). Therefore, by using Lemma A.4, the
M od M1 =0311
16

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Consequently, the inner product between any pair of distinct vectors <ip and S/ satisfies:
(ME3 McN + (M 2 M M1 (M ME
since both projections onto M are zero.
According to Lemma A.2, for any distinct pair (i, * (i', ,'), there exists a constant a' such that:
P ( M |l d'o'd (d-33) 21 5.
Applying the union bound over all nP(nP- 1) distinct index pairs, we conclude that with probability at least 1 nP(nP 1 1) the following inequality holds uniformly:
Ke 5 ) s a" o () v(i. # (d',p')
for some constant a" > 0. Substituting 5 = 1/d yields the desired bound in (6) with constant 01, with overall failure probability bounded by 0 (n² /d).
To prove (7), since the distribution of ( is NO, od
we have
2
Sp
2d - 3),
pdd-1
where (d) denotes the chi- chi-squared distribution with d dd-3 3 degrees of freedom. Applying Lemma A.l with a; 1, we yield:
P (T) 2 2 of (d-3+ 2--322// (d-3) 3) + 2og d
- d'
IP (
2 < o (d-3 2 (d (d- 3) log
d'
Finally, applying a union bound over all nP such vectors shows that the inequalities in (7) hold simultaneously with probability at least 1 - 0 (nP/d).
( (0) (N, the analysis of the inner product of the weight vector and the patch vector Since w is initialized as w! is similar to the proof of Lemma C.I.
Lemma C.2. Let the network weights be initialized as we (0) ~ NO la) for each class e E (C), and let S/77)
((x)") (T) (T)
Mi ()) be i.i.d. samples from the distribution D2 defined in Definition 1. IfC = θ (log d), then with probability
at least 0 -0 ("PC), there exists constants 01, such that following inequalities hold:
Ve E (1) (())
(w (0)) e) S V Vlog d |ellz To
max (w: (1) (0). 2 2 [|ell do
max
cE(C]
=E(C)
We p P
(0))))ppp))) S 5a1000 / i- - 3) log (2d),
max max(w!!) (1))) (0), <( 2 02 cE(C]
max <E(C]
Proof. Since each w ! (0) is sampled from a spherical Gaussian, the inner products (w. (2) (0), e) for any fixed vector E [u,v,, follow the distribution (o ||e og). By using standard sub- Gabb-annsiin tail bounds in Lemma A.3, we have:
e> /2logd | 0o s0 (2)
P f [C] (w (0). e> 2 /2logd| ou
S
P [ 11) (0).
(8)
Moreover, applying Proposition A.7, yield:
P max/w (lnx/w."))0 (0). $ ||e|| 00
(w:
(1) (0). e) s b
$ (2) -
-C/4
C/4
(9)
[E|C]
17

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Inequalities (8) and jointly establish the first line of the lemma.
For the second line, consider the noise vectors St. N of, - e [2ee
erohe
Let
{ei, e2, Gzob. ....... ea-s) be orthonormal basis of IR and define two projection matrices:
M = @ .i...., ēa-a]" &R(d-a)x
= ee,, e2, @rob] T E Raxd
By Lemma A.4, we have
~ (w(
M ~ NN0, ( Ia-a), Mi cT - -3x1-
Thus, write
(w 1 (0). - (Mw! ()) Me!
+ (M.w M1) (O) = (Mw (O), M
Applying Lemma A.2 and taking union bound over nC such vectors yields:
(we (0)) S 01000 / Vd= 3) (og((d) 2 21- ( nPC
P
(10)
Lp.c
d
where d1 is constant. Likewise, applying the lower bound in Lemma A.2, we have:
min max (w! (1) (0), a2 a2 dd 3) jelp] cE|c]
21
(11)
where 02 is constant. Finally, combining (8) (11) and applying a union bound completes the proof.
At the initialization, we have the following result:
Lemma 4.1. Given the weights (1) initialized as we (0) ~ (oO0, at the beginning of the first stage, with
probability of - p=²p²C we have
polyfd)
Vee fu v(),, v()),(1)) (())
(0),e) =ō (1lell))) oo).
max CE[C]
V fua) r(1), (())
(o) e) 2 0 ([eel2 00),
max
CE|C]
vie [n]. E P
(0), ) (
max <E(C]
Vie [n],p E P
w (1)
(0), ( 2 Ω (auog))
max
cE(C]
Proof. The results can be obtained by using Lemma C.2, Recall that = Olog d) Condition 3.6, using union bound, the probability of the inequalities hold least 1 1-0 0 n²p²c) poly(d)
D. Learning Dynamics in the First Stage
In this section, we study the learning process of the first task, which can be seen as single-task learning. If the signal of the task-specific feature is larger than other components, the feature can be captured by CNN.
Lemma 5.1. In the first stage, under Condition 3.6, and given T 2 Ω ( for any 0 < <L5 T (1) .D E P. have following::
w (1) (1) ((t++),, 1). w (:!) ( So(
max REC
max REC
Moreover, we have
(1) w (1),
< Ō((000))
max GEC
18

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Proof. We analyze the change in the projection Lemma B.3:
We
(1) (t). (1))) induced by the gradient update, denoted by A (p)) from
A(E w
Tp
Σ
Ve D (F(x(").v()) o
"ew.:/")"(/()))...).m ((wP(())(())
n
PEp?
We separate the summand with (i.p) = (f',');
<((//p) = Mi
E
e F(xx")) o' ((w!"())s!)) ((m"" (ehp-19p))
sElm), (1p))(((''>
nd yt e
u.2 & ((w! |
We now upper bound both terms. Note that |6"1 S 1. Lemma C.1 implies that
A
(1)
Thee n
Σ ((w!(())()) (4pp.
(1) < >
n | nd
||
iE|n], PEP! (i.p) +F)
(ww.(())))))) of (Va+t (Pa (
(P
h)
S O
max c.t
Vd
Given that is sufficiently large under Condition 3.6, we have P/Va = o(1/n). Thus, the bound simplifies to
1) m 08 ((w! o
((E )
& <x,0
We proceed by induction. At t = 0, by Lemma 4.1 and 17 n 0 (0oaa)) for all i I .p Ps,cE [C), we have
(0) < < Ö(ooo))
Suppose this upper bound holds at step t. Then, the update satisfies
noior
|(El =ō
o(
n
since 7 S (0(1 (0002)), na 2 w (1) and d 11
Thus, we obtain
(w ( 1), 1)), - (w + A(E!)) S Ö(gvog) + +o(oog)) = (ang))
By induction, we conclude that for all 0 SS
((). (1))) < Ö(oog)) and A
max
o(angg))
eE(C]
as claimed, We conclude our proof.
Lemma D.1 (Learning the task- specific feature). Under Condition 3.6. suppose that for all iterations t E 10, T] with some
T > Ω ( ) the weights satisfy
w! (1), e) so (C-1/)).
max CE|C] cE ( (,00 w0a
max
19

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Then there exists an iteration T < Ō
such that
w (T( 2 2 (C-1/1)))
max cE|C]
Proof. Suppose that for all t E [0,
max ec[C] es (w0) w. (w. s o (C-1/s),
max
where T 2 St (1/(100a2))) Then for each i E [n], the margin satisfies
y/4 (1) rx" (x)))
0(1)
which implies that
2(1) S 1 1
Vi e [m].
(12)
We can both upper and lower bound the growth in alignment with the task-spec feature u as
max cE|C]
w (1)
(t
1), u()
max cE (t)
4) d' (( = (1azo (((x)()),u))))
=
ma
na
(13)
where the last equality follows from Equation (12).
Equation (13) shows that the sequence max (w!")( (1), ) t= have the initialization
is monotonically increasing. By Lemma 4.1,
(0) u (1) = θ (apaa).
max cE|C]
Let (1)) be the first time / such that max.eo
w (1), (1) 2 Ω (C-1/3). Starting from some max,Ec ((), n
the number of iterations it takes reach max.gc
(t) u())
2 maxecc ( (wo" (t), is at most
max.eo (w (t)"()) ga? ( (w. (1),(())))
Then, starting from θ (ana.), it takes at most
20000
T()) SO
(14)
na? (2'amaaa)
times steps to reach max,ec
w 2) (t) 2 (C-1133) This completes the proof.
Theorem 5.2. In the first stage, given a training set s(!) with size n, a S o (aw), there exists 7()) < δ nooon that for any T(1) 2 T(1,, the network Fros fits all training data points with high probability:
(
such
n _2P2C ppll(dd)
Vi E S() F.F TO (x:) ((1))
Moreover, PO)) achieves high accuracy on test data points at (
(xx,)>=!))) (yF(1) > 0] 2 21-0
nP22 pool(d))
20

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
At / 0, by Lemma 4.1, have
max cE(c]
(O), u4) ) ((ppa)),
max cE|C]
w (0), v()) =
max ( = =
w
DE
Let T* be the final time such that max We (t), u - 1/0))) Suppose that at some time t S T we have
(w (t), v()) ((ona)),
w (t). @(ooac),
max cE[C]
max eE
and additionally,
( (w (1) (((0))0))((0013)))
max <E|C]
Then the margin satisfies
( F (x, (1) S O(1),
This implies that
2(1) < -1.51.
(15)
We now analyze the update of max ( (w: (w (t), v()).
At time i, we have
w ( + 1), v())
(())))a))
max cE(C]
max
cE/Cl
na
Σ +) ((-e))
. (0((())a01)) = = (ww!((t))())))
n
and { maxxeej0)
(1)) (we (t), m())) is an increasing sequence, hence
(e
( + 1), ) 2 max (w!"((), va)) 2 (((oaa))
max <E(C]
(16)
<E(c)
Since max,e[c] w! ( (t), v()) s (1 (oga.)) and ar S o (a.). follows that
max (1) (t + 1), v()) cE|C]
(w) v(2)) Omaa s Ō(vva.) + o(oga.) Ō(ova)).
max
(17)
<E(C]
Combining Equations (16) and (17), we yield
w (2) (t 1), vaj) =onaa..
max
eE
We now analyze the alignment with the random feature (1))_ For any t, we have
mi | 1 & (maxcetcy)
< (w. (2)))
w! () (t++1).
w: (t))
|gm)
max
max cE(C]
dE[C]
n
Define sequence (o(1))TT by
(O), c(ii\/-
((0) = max <E|C]
|/00 2 Φ (maxestcy
(w.0 (0,00,,50))
|g
Φ(1 1) x(()
n
21

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
By the bounded margin condition in (15) and Lemma A.8, have
| (0ml==nn1) n4)) =0(nn11))
-
so
((1+ 4(1) + nd c .2' (ww2(0),5"))
Vn
Since (tt) S (((aa))) 1 0 (1/ (m follows that
@(t + 1) - (t) + Ona -1/2)= S ((t) + o(ooas) S Ō(ovae).
and
( + 1) (1) - 0 (nagoonn 11/2/ 2 (() - o(onac) 2 )((ooa))
Then for all / E [0, JM, we have
max cE|C]
(t), ((oaa))
max c
ww (t) @(ooac).
(18)
By Lemma D.I, there exists T < (( such that
W (TT u 2 1/3)
max
(19)
cE|O]
At At 2 T()) Ti( the output for any training sample (x,; vi) E 5 can be expressed
C
C
yi Fu (xi)
(x,
Σ ded
((w!)(T()))
(ii)) + Σ dox ((w() )(l))
c=
c=1
Σ No
(ww()
ΣΣ Φ (T)
ysci
+ yi
c=1 pEpS
By Lemma 5.1, we have
(w() ).(.)) < o(
max max cE|C] pEp.
(20)
Combining Equations (18) to (20), we yield
yi 1 Tas (x) (x:) 2 max ded
((w!)(T()) u("))) + (C - 1) min da ((w!)(T()) )(((((())){)))))))
u(")))
cE(C)
eE(C]
(ww!" v())i)
max deo ((w" )1)i)
- max ded cE(C)
C cE|O)
CP max max 6 (< <!0i)) cE(C) PEPS
2 -0110) -OO()a -Coa----------------------------
2 Ā(1).
(21)
With probability at least 1 - On PonnnppC//pppll //olyydy))) the bound in (21) holds, showing that the model correctly classifies all training samples with a significant margin.
22

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Now consider a test sample (x,y) from the same data model. With probability least 1 - (nlnP(//poly(() have
yF (x) =
Σ N
Σ
((w!
(((()))))))))))))))
+
c=1
c=1
+ ye
Σ Xoo ((w!)((())
Σ
E
((w .(1i) 2 2((1.
Φ
(22)
PEP:
This completes the proof.
E. Learning Dynamics in the Second Stage
In the second task, recall that u an ((2) (22) v(1), aça (u() Let ((( (T.) at Tu. the following lemma shows the status of w: Corollary 5.3. At the beginning of the second task, if ou 2 w (o..). we have
(2) T = 0.
w ()) u u22) ) θ (apora)
max rE(C]
(!? ?w!?) (0), v H (agar).
max cE(C]
(w(") (0), u 41) 2 (1).
max cE(C]
Proof. Lemma D.I shows that at the end of first task, we have max.eic1 (w:" (T()(Tu)))) w 2 02 (C-1/3). Moreover, v()) = H (maxcstc) we (0), (w),)a)) = θ (apa.)
Equation (18) shows that at T(1,, we have max
we (T
and max We (2) (T()), <a>) = @ (maxcetor ( ( 2 (0))c(1) = H (ovaas). Using the definition of (()), _ we conclude our proof.
Corollary F.1. In the second stage, under Condition 3.6, given T 126 52 1000
for any 0 < T, we have
max
w (2))
(1,
(2)
max CEC
w (1), S0(000))
cEC
hold for any i,p. Moreover, we have
Vi.p p
max
w (t). (2) < مŌ (ov0g).
eEC
Proof. Similar to the analysis in Lemma 5.1 for the first stage, we study the change the projection (1)
c(2) ) due
to the gradient update. Denoting this change by ((E!?) ) we have
A
(2
n Σ 3 (((x("),, (/') 06 ((w:"()) ) E?)
PEP$
nde (2) n ve
(F(x()) ). (2) 0' ('(( 2 ))
(2)
We now upper bound the magnitude of the update. Using the fact that
[e'] < 1, we obtain
A (2) (
< Te
Σ PEPS (()))
06 (( (w (t).
n 0' ((w! (w: (1) (2) )) { ||
n
n
< O. δ
mmax o' (w, Giip
of ( P
(va+t))
+ n (
11
Vd
23

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Given that is sufficiently large under Condition 3.6, we have P/vd o(1/n). Thus, we obtain
A(EP p') S de
Under the condition that n 2 w (ofam)) , (1/ ())()) and A = 1, this bound simplifies to
A 3 = =((0os))
( (we (1). across all steps. At 0, we (0) = w. (T
We then proceed by induction to bound the projection by the definition, Lemma 5.1 implics that
w (0), $ Ō(ono))
Assume the inductive hypothesis holds at step f, i.e.
(0.5F))) S ((ooe))
Then, by the bound derived earlier in the proof, we have
A A = o(
Therefore, the updated projection satisfies
w (2) c (1+ (2) ( w! (w! - A S Ö(roag) + o( s Ō(
By induction, we conclude that for all 0 5st
(1), ) S Ö((oog), and A s d(gogE))
max
cE(C)
as claimed. This completes the proof.
Recall that the update (w("), ((T)) is Ae (c(²) = gd (1)) ((! g(r), The direction (w.
depends on the sign of g(2), which is studied in the following lemma.
Lemma E.2. In the second stage, let (22) (c(2)) Ecelor o ((w!") ((2))) ((w?"<<3))
given T T>2 ÿ 1oo0
suppose
max es cE(Cle ( v(()) ( (w: (w:?(t),e)) < <0(c-133)
max
holds for all t S T. Then we have
g( <0, if h12)
(2) > ñ -1/2 + CPoj
( 0, if 112) ((2))
< J n CPo302
24

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Proof. Since A. (5(T)
ndo ((w!" (0)5())) g(t), we show the upper bound and lower bound for g Note that
I(2) = I UI(2), we have
*
Σ (F(((x) (2) ),
(0)
Σ (-C(F((()).))) )
(E
(EI
Σ (
1 exp
h12)) (1(2) (2(2)) (v(2)) +h120 (5(2))
Vi
(2)
Σ 1220
(1.)))
pEp!
1 exp
11(2) (1(2) 12(2) (vle)) 12 (c(2)
+ yi
(2)
Σ
11(0
(5.P)
(
pEP
12121 (1(e)) + +1(22) (vee)) + +1112) (5(2)) CP
& (Kw:?"). -1:0
1 exp
exp
max
aEICREZZ PEpt
Σ
exexp 1
(112)
(u()) + (2) (v!21) - -h(2) (512) + CP
max cE(ClEEZ PEPS
& ((ww??
1211 (2nn) + (2121 (vee) + +1123) (1:2)) - -CP
& (((m(?) -1())
exp
+ 1
max
cE(C]EEZ pEP!
+
(2)
1
12(2) exe exp
(1(2) + 12) (v(t)) - 12(21 (c())
CP
& (K(m? -..0 max dE(C]EZZ pEPS
Let = -n 1 ((we",<03)
CP maxcip 2 (K (2) <100)) )
we yield
Σ (F(((((()) 1( - Σ (-)(F(x(()) (121))
(
(EI
n2 - 4
(2)
+ exp (h(2) (1())) + (122) (lle1) (2 exp exp( (2) exp (7)
(1 + exp
exp (h(2) (u(2)) + ((2)) (((2) +7) (1 exp (1(22 (u 22) (v(2)) - 77)
+
(-7) (-n exp (2) (2) (u(2) + (2) (v(2)) exp
eexp (27)+ + 2 exp exp 2) (u +1(2) (v(2))) + (n()) (f)) exp (1))
exp
(1+exp (1(2) (u +h(22 (v(2) ) (M22 (u(z)) +h(2) (v(2)) -7))
We fix the values of (2) (n(2) and (2) (v(2),, and consider the equation
h
(2)
u (2)
+ 22) (v(2)) exp (27) + nl exp (B11) (u(21) + 22) (w(e)) + (n(2) - n exp (7) = 0,
-n
cxp
where the solution is given by
n - 4 + (n= - n (2))2 + 4m no exp (2h(3) (i(in) + 2h(2) (l1i)))
(2)
7" = log
2n exp (h(22 (u(²)) + n(2) (v(21))
This implies that (( > requires
22 (5(2) S Sn" + CP
max & (w.))<1)))
c,tp
25

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
sufficient condition. Since va² 12 |a| + [b), we further upper bound 7
(2)
+
+ (n= n
(2) + 4m (2) nh exp (2h(2) (u(?)) + 2h(2) (v(2)))
7" = log
2m (2)
exp (h12) (u (2) + h(2) (v(2))
n
(2)
+ n= exp (2h(2) (u(2) + 21(2) (v(2))
(2) (2)
< log
4 exp (2(2) (1(2) h22) (v(2))
(2)
n ( (2)
n
7=
< log
4 exp (h(2) (u(2)) + 1(2) (v(2)))
(2)
m - n
=
(2)
(2) n exp (h(2) (u(2)) + h(2) (v(2)))
(2)
n
Note that (1(22) (u(2)) + (2) (v(2)) 1 0 (1), by using Equations (3) and (4) in Lemma A.8, with a probability of
1 - 0 (ull(n)) ). we yield
(2
n
(2)
(2) (2)
n
$ s0 (n (12
00
4 exp (h(2) (u(2)) + h(2) (v(2)))
n x
and
( n=
15 V 1 0 - 1 5 (n-1/4)
Hence, we have
7" 5 20(1: (n-1/2
Therefore, if
(2)) (c(3) 2 δ (n11/2 + CPoCo?
we yield g(2) <0 0.
26

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
On the other hand, we upper bound- -g(2) as
Σ
(F((x)
(2) y. )
Σ (-
(EIL
Σ ( (2)
1 exp 11(2) (u + 11(2) (v(t)) - -1(23) (5(2))
Σ
1((3)
(5:37
+ y/
Σ (E
1 cxp
h122) (1(2) + 1222) (v(2)) + 12)2) (510)
+ Yi Σ (2)
(E!.0)
PEP!
1122 (u?)) + (222) (v22)) - 12)2) (c(2)) CP
& (K<=2 -1:3
+ exp
max
cE|OREZ PEPS
1 1 exp (1(2) (1 + (2) (v12)) + (2) (5(2)
& (Kww") ((w(3).()))
+
CP
max cE(O)iETO),eET PEPS
iEz!")
= (2)
1 exp 11(2) (n(3) + =1(22 (vee)-
- -1(0) (5(23) CCP
@ (((m?) (
max
cE(C)iEZ(0) PEPt
- nn
1 ( 1/(2) (u(21) + 12(2) (v(z) + +1(22 (c12)) + CP
& (((m??)
max
eE(C]EEZ()) MEPt
Let - - @ (2)
(w=",<(3))) ((31) - CP max 1 ((w (2) _ )).
we yield
Σ (F(((x) (2))2 (+) (ET(9)
Σ (-E(F(x(()
(2) 11
). 1
(E
(2) (1 + exp (M(2) (1()) + (1(2) (v(2))
mz (2) exp (- n exp (7)
m
(1 + exp (h(2) (u(2) + (22) (v(2) + ()) (1 + exp (h(2) (u(2) + h(2) (v(2) - 7)
exp (-+) (-1@eex) exp (1) (1 + (2) (v(2)) exp (25) +n (2) exp (1(²) (u 11(2) (v(22) + (n (2) -n()) Jexp (5))
)
ee+e (1)22) (u(²) +1(2) (v(2) + (1 expp (1))2 (u(2)++(() (v(2)) -7))
Similarly, we fix the values of h (u!²)) and (1 (v(2)), and consider the equation
_n exp (2 (1(2) + (2) (v(2)) exp (27) + (2) exp (1 (2) + ()) (v(e)) + (n
(1 - nn2)) exp (7) = 0,
where the solution is given by
(2) 4 -
-! + (n (2) 2))2 + 4n (2) n (2)
exp (2h(2) (u(2)) + 2h(2) (v(2)))
n
5" log
(2) 2n exp (h(2) (1(2)) 22(2 (v(2))
(23)
We then use a similar technique in bounding y* Since (2 (u!2)) + ( (v(2) S (1), by using Equations (3) and (4)
in Lemma A.8, with a probability of 1 - O ( we yield
m n
(2)
n 2 1 150 (n-12) + (1= 1500n-112)
5" (2) n exp (h(2) (y())) + 1222) (v(2)))
27

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
This implies that G(2) requires
1/2) (5
-CP max (K( w! (())) 2 -0(1=1++cPo80? CoP
as a sufficient condition. Therefore, if 12(2) (c(e)) S - 1112 + *Poooo?). then it follows that g( > 0.
We conclude our proof.
Lemma E.3. Under Condition 3.6, given T 2 . (ooon))
(modno) ). we suppose
(w/9(0,.))| =000))11)0(00)))
max cE(O] EE (ui2),
max
holds for all S T. We let a; = O (au). Then there exists T(2) s <ō =((obaa) such that
gona.
(T())
(((0)((21) S so()).
max <E(C]
w
Proof. Att t = O, when oç = θ (aw), By using Corollary 5.3, we yield
(wm?),((0)) 2 0 (a - _()) CP
2 ((w!9).10)) s (CPojOo) so()). eee
max
eE|C]
We then have
Σ $ ((w5))<()) 2 w (n' (1-1/++ + cCPojo?). eE|C|
which means A (c
S 0 due to Lemma E.2. By rewriting G( we yield
g Σ
(F(((x)) - Σ (-((F(x))))))
= Σ
1 + exp 11(2) (1121) + (2) (vi2)) + +1) (c12))
+ y/ Σ (2)
Si.p
(
DEP$
Σ
1 exxp
1122) (u(2) + (2) (v12) - 2) (5
+ y/ Σ
h () (2)
(
pEP$
By the assumption, for any i, we have
(2) (1(2) + (2) (v(2)) + yi Σ h22) (E!00
I 0(1).
PEP$
At the beginning of the second stage, we have ((12) 2 52 (maxxelcy ( (w) (2))) > (1). Then, for some 4
if ((()) 2 (1), it is clear to show that there must exist a = θ (1) such that for all aç 2 a, we have
0 (n) S g < S (m).
28

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
(w: ( (t), )
We consider the update rule for m
=-0 w (1++1)), -( (1), - (n>e |0 ((wP))))()))
(24)
This shows that ( (w?") (w((()))
(1), ((²) decreases monotonically for all E [C]. Consequently, (((2)) also decreases. Let T(()
be the first time such that h (22) ((2)(((2) (c) drops below (1). Then, the monotonic decrease described in Equation (24) holds for any 0 R ( (2)
Given 0 S 1' < E T( starting from some
(wé (ee)(t"), (()) the number of iterations it takes to reach
2 max ( (w! ( (t). ( is most O ( max (w! (t)))))))))
max (w we
(2)) (L), (1),(())
Then, starting
gaw meax (w (()))))))))
from θ (1), it takes at most
Σ 2-
2r
5
(25)
? 2i
17a
time steps to reach max
w! (2) (1))
s 2 max,e(c]
We
(t') (( Setting
(2)
log(1/ooau) log(2)
obtain
T (2) s Ō n0ga
such that
w c(2)) so (c-110)= = (1),
so(1).
max CE(C]
Then, the update magnitude
(w: (t 1), ( -
(w:" (1), (1), (2)) is small, and according to Lemma E.2, (((2) (c(2))
cannot decrease to - 22 (1) because we have g(2) > 0 (2) (<12) - - (n-1/2 + +Pogaa?) Hence, we conclude that
1 ((2)) s s0())
at some T(() <0 Ō
We then analyze the change of (w?
(2) u and (w!?), v(t)) in the second task.
Lemma E.4 (Learning the Task-Specific and General Features). Under Condition 3.6, starting from (222 = 0, there exists
T:(2 <Ō(
such that
w (T)(TT( u 20 (c-1//) w! (T(()) v(0)) < <
CE(C]
max <E|C
Proof. (2) = 0, for e [C], we have
w (2) (0), (0), mn 2 >2w((!! (O) ()) 2 - (
where the first inequality follows that { ww! (1) (1), n T() is an increasing sequence. The second inequality holds
== because w!') (0) is drawn from an isotropic Gaussian distribution and Lemma 4.1. It follows that
Vc [C], ( w!? (0)).<(2) a,a (0). 2 =
29

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
and similarly,
Ve E [C], ( (w: (0), v()) = (w: (0), v()) 2 (000))) (
- (t),
(26)
where the last inequality uses the monotonicity of (w (t), w(y) } ) and Lemma 4.1. By Equation (18) and Corollary 5.3, we obtain
(w/(())))) s so
mAx cE|C eE ((()), ((())
max
We proceed by induction. Suppose that for some > 0, we have
(2)2 |100 (c(0) s 0()). (w." o) =0(c-10)) and
max <E|C] oc f (u(t))
max
Then by Lemma E.3, ((2) (<:3) is non-increasing, and at t + 1, we still have
(2) (5()) s (1).
This ensures that
2(1) < 11, [e
which further implies
W ( 1),
max CEC' (w! (1)) = = (nn'nan na?
Σ fg o' ([w((0).ee)))
n
max CEC'
shows that maxxcejo| (w: (t), v(e)) an increasing sequence, hence
max
( w!? ( (1+ (1, v(2))
max
(1) 2 (o0a..)
(27)
rE(C]
<E(C)
Since max,c[[] (t), (t), v(2)) 0((00a)), 17 7 (1/ (ova and a. so (au). it follows that
( + 1), v2)) cE|C]
max
< max <E[C]
(1), (()) + O(nato)) s Ō(oa),)) + olooar) S ((gga)).
(28)
Combining Equations (27) and (28), we yield
max
w (2) ( 1), v(2 = =((ona))
(29)
Thus, combining Equations (26) and (29), we maintain
max
( (w (w (t 1), (t), v22) < (0)0).))
Meanwhile, for ? we have
w (2) (t + 1), (2),( )
w (t!2(tt) (2) )
max rEC
maxx CEC
=0 n 2
Σ Ee o.
1)) 0 (naïd' (((w?)(0) u0),))))))) D)
30

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
It shows that { max.e(cy We (2) (t), (t), } is an increasing sequence. Let T::²) be the first time t such that
max.co we (2) (t). 2 2 (C-1/3). Then for all 0 I T(2)
< T we have
(w! (T(2), )(8)) v()))
s 0 (o)ce,, and |100 (5(3))
S SO().
max cE|c]
and
max <EC
w (2) (T(2)
+ +1), u(2))
max nEC w (2( (T(2() ul()) = (nato' (1 (Iww ul2)) ))
(30)
Corollary 5.3 implies that max We (0), u u(2)) - (anau)-
Starting from some max,cd (w! ( (1), ((2))
the number of iterations takes reach max.ec (w: ( (t), u(3)
2maxe (w: ( (t) (t'),(()) u(() is at most
maxcco (1), na? ( w/ (1),
Then, starting from ê (anau), ). it takes at most
T < Ō
200aa
<ō
(31)
na? (2'o)au))
aoodn
times steps to reach max,cc
We (2) (t), u(2)
2 1(-33)
This completes the proof.
After training the second task, the following theorem and corollary show that at the end of the second stage, CF occurs if ag 2 Q (aw), i.c., at T(2) = 6 1 ( only (2) can be used for classification.
Theorem 5.4. In the second stage, given a training set s(2) with size n, there exists T12) H (( (nopac) such that for T(2) > 7(2,, the network Fres fis all training data points with a high probability:
n² P2C
P ViES
(2) y. P. (x.) 25(1) (111))2100 21-0
poly(d) poty(n)
Moreover, Fre, achieves a high accuracy on test data sampled from the second task:
(x fe (x) 0) 21 2 poly(d)
nP'C
polyba
a. S 2o (aw), and ag 2 Ω (aw), Fre achieves a low accuracy on test data sampled from the first task
nPeeC + poly(d)
P
(xx)>-o y ( > 0] S +0
'polyn) +
Proof. For any training sample (x,, vi) from the second task, the output can be rewritten as:
C
y1 Fycr (x:) -
ded
((())))))))) + 91 Σ ded
(w! ((w!))ev())))
)ivee)))
c=1
+ Vi Σ Xo (( (w! ((?(),c((22)
+ Σ Σ
0 w!!) (())
c=1
==1
pEpp
c
Σ 100 (( ( (0(l) + Σ ded ( (mm(2))))
C Σ
2o ( ((w1?),(1))
+ V/1 Σ Σ
+ yic,
1
c=1
pEpS
31

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
By using Lemmas E.3 and E.4, there exists some time T:22 = 0 'yaoa!
such that for T:(2) > T(2), we have
BCF (xx) 2 max ded ((m?", ((())) + +C- 1) cE(C]
min 1e ((m(?,(²))
cElC]
-C' max d (1(w!?) w(2)/) - 1o0 (K ((00))))))))
-CP max max @ (((w??))))) cE(0] PEP;
eE|C]
20(1CC) - Co (oga c0 (ojaa) Co - CPO (0i0 2 Ω(1).
(32)
The probability of Equation (32) hold is 0
""p²cc
+ pobin:) For a test sample (x. y) drawn from the second task,
poly(d)
with a probability of 1 - 0 puly(d)
C
+ poly(n)) we have
C
y FT (x) y Σ Xod (( (w?")))l))))
+ y Σ 1o0
((w(")
c=1
c
Σ Nooo (( (w!?))
ΣΣ
2 ((w!)) -50
ty
+
PEPi
21)
Additionally, for test data vx drawn form the first task, let as the definition in Definition 3.1, we have
Σ 1o0 (( w!?), )n())) + Σ ded
(( (w!?) (w?))r(()) ))r
(FFraa =c
C
E des (( (w!?) (w()),
+ 2
Σ Φ ( (w?") :
+
c=1
pEpp
c
do (( (w!"), ) 1e a 10 (w/(),(0)) yc
=yc
C
+
1 n 0
3
( (01)) +
Σ
Σ Φ (w!e), ((w!2)_!??)
c=1
=
pep$
Aace -3 ago (( (w?))
_2²) ) +((-1))
- age (( (w("),((a)) -C
2 max <E(C]
min nE(C]
max ded cE(C)
c
- a
Σ
Na (w!!)) - 10 CCP
max oE(C),PEP
& (((w?),.(())
a 30
20
3 - a
(o)aa)))))))))- - co -co (oia²) - Ca -Ca -
25(
Note that P (c y) 1/2 using the union bound, we conclude our proof.
Corollary 5.5. At the end of the second task, for (2) = - 1
(nooat)) ). ifa 2 (ax) and as 2 Q (a,), we have
w (T(22), ), ) So(1),
max
cE(C]
w (7(2)) ) v(2) H (0oaw),
max cE[C]
w:!2) (T(2)) (2))
2 (1).
max cE|c]
32

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Proof. Lemma E,4 implies that at 7'22 = G
we have
1000
w (t), u (2) 2 (c-11/3)
max cE|C]
(w = (1), (0), θ (aram)
max cE|C]
Moreover, Lemma E.3 implies that
(T(2)), (( S (1)).
max
CE(C
Since T = agaa ((2)) and ore 2 Q (a.,), we conclude the proof.
F. Additional Experiments
F.I. Simulated Dataset
For simulated dataset, we use PyTorch the deep learning framework for training. The CNN model used is defined as Equation (1) with C' 10, and the model is optimized using GD. The learning rate is set to 0.1. In each stage, the model is trained for 100 epochs. The hyperparameters are set as follows: P = 1, og = 0.5vd, and d = 50. The value of a, is fixed takes values in the range {0, 0.1,
at 1, while a, and og are varied across discrete values. Specifically, ag takes values in the range {0, 0.1, 0.2, 0.3, 1.0]. For each combination of a, and ocç, In Figure 2, we perform 20 independent experiments to calculate the average accuracy of both tasks at the completion the first and second stages.
and
We then include additional experimental results for simulated datasets. Figure 6 shows the case that both ( and u are captured by the model. Since the signal of ((2) is slight, CF does occur in this case. Moreover, Figure 7 shows the case that (1) (2) is learned by the model while none of the other components are fitted by the model since has large norm in this experiment.
F2. Real-World Datasets
In real-world datasets, use ResNet- 18 as the CNN model, which is trained for 100 epochs in both the first and second stage. We show the full results for Figure 3. Figures 8 to 10 show the performance of the model on both first and second tasks in different stages in CIFAR-IO, CIFAR-100, and Tiny-ImageNet, respectively.
We then use T-SNE (van der Maaten & Hinton, 2008) to visualize the feature in both the first and second stages of CL in CIFAR-100 and Imag-en The results in CIFAR-100 and Imag- are shown in Figures 11 and 12, respectively. Additionally, we calculate the maximal singular vector in the feature space in CIFAR- 100 and Imageen, and the results are shown in Figures 13 and 14.
33

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Stage 1ask
Stage Teak
Slage Task
Task
1.8
Cranrei-D
1.75
1.15
1.6
Cratnel-3
1.2
1.25
Charnol-4
1.00
1.00
NeatareU
Cramae-6
Chappeh-8
0.5
Cramnel-6
0.3
0.23
0.25
26
60
100
40
80
20
100
opoct
9p1.h
opect
op3c1
Crannel-6
1.35
130
Cratreh-2
Chanrol-1
1.00
2:00
1.53
Cramnel-3
Channol-S
1.75
Chamnol-6
0.75
3.75
0.75
Cramreh-7
Cramrel-
0.23
0.25
1.00
0.00
3.00
0.50
100
Crarnai-0
1.3
1.75
122
Charnol-3
123
1.2
3.25
Crarroi-4
Feature-zeta
Cramrai-5
155
Crattel-7
Cramnal-B
0.5
3.50
6.53
Cratrel-9
0.23
25
-035
2.25
-0.3
300
60
80
100
10:th
62007
00301
Dila-i
Dela
5-
ants
Figure The output each layer in the CNN model using different inputs. Top three rows: The output of each channel the first layer. Bottom row: The output of the last layer. (a,, 1, 0.9, and og- 0.1)
34

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Stage 1 T
Stage 1 Thikk
Stage Task
Stage Tust
Chamnal-0
Chatnel-2
C1atneh-3
Statrei44
Restured
1.-
Chemee-5
1.3
1.-
3.2
Cratral-7
Charrol-8
Crarrei-9
26
46
61
102
20
60
100
25
40
80
105
20
92
60
80
160
193/h
6pom
Ap3.11
2.0
Charnot-0
2.0
Cramreh1]
Stameh-2
C1ateal-4
Teatured/
Chatral-5
Cratrei-
Cramaak-7
0.5
0.-
6.3
0.0-
6.0
2.
Coannel-1
15-
Cramnah-2
Sratneh-3
155
3.5
Cramnah-4
1.0
3.0
100
3.0
Chatrel-7
Cratrel-8
Cramnal-0
20
10c
20
50
:00
40
03
100
20
42
20
20
100
spoc
60001
Cata-U
T-
6-
5--
Ourput
4-
apart
303.h
63301
Figure output each layer in CNN model using different inputs. Top three rows: The output of channel the first layer. Bottom row: The output of the last layer. (a,, 1, 1. and agę 0.8)
Stage 1 Task 1
Stage
Task 2
Stage
Task 1
Stage 2 Task 2
1
12
122
52
3-
4-
4-
2
Figure Overview of in CIFAR-10.
35

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Stage
Task
Stage
Task 2
Stage 2 Task 1
Stage 2 Task
20
20
20
20
40
40
10
30
20
40
Figure 9. Overview of CF in CIFAR-100.
Stage Task
Stage Task 2
Stage 2 Task 1
Stage 2 Task
20
20
20
40
40
60
60
60
80
80
20
80
60
Figure 10. Overview of CF in Tiny-Imagehe
36

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Initialization
Data from Task 1
Data from Task 2
10.0
Task Class
10.0
Task Class
Task Class
Task Class
Task Class
4-
2.5
2.5
2
0.0
0.0
-2.5
-2.0
-7.5
-7.5
-10.0
-10.0
-0
-50
20
30
-30
-20
20
90
-30
-20
-10
20
30
Stage
Data fr Task
Data from Task 2
Task
Task
Task Class
Taak
Task Class
-00
Task 2 Class
-00
-10
Task 2 ass
-30
-20
16
20
30
-30
10
20
30
-30
-20
-10
10
26
Stage 2
Data from Task
Data from Task 2
Task
Task Class
Tesk Class
Teak Class
Task 2 Class
Task 2 ass
-2
2
Task Class
-6
Task 2 Class
-10
20
-20
-20
-10
-30
-20
Figure 11. Using T-SNE to visualize the feature space at different stages of CL when sequentially train the model on the Task-O and Task-2 in CIFAR-100. First row. The model randomly initialized. Second row, At the end the first stage. Third row. At the end the second stage.
37

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Initial ization
Data from Task
Data from Task 2
Task
Task
Task Class
Task Class
10-
5-
5-
0-
-5-
-5
-10
-10-
Task Class
Task 2 Class
Task 2 lasss
-7.5
-5.0
-2.6
0.0
5.0
7.6
-2.5
0.0
5.0
-7.5
-2.5
0.0
5.0
7.5
Stage
Data from Task
Data from Task
Task Class
Task Class
Task Class
Task 2 Class
2
Task 2 Class
-1
-3
-15
-10
10
20
-15
-10
10
20
15
20
Stage 2
Data from Task
Data from Task
Task
Task
Task Class
Task Class
Task Class
Task 2 Class
Task 2 lasss
0-
0-
-2
-15
20
-15
-10
-15
10 15 20
Figure 12. Using T-SNE to visualize the feature space at different stages of CL when sequentially train model on the Task-O and Task-3 in Tiny- Imageeet First row. The model is randomly initialized. Second row. At the end of the first stage. Third row. At the end of the second stage.
38

---

Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks
Initial ial izat on
Stage
Stage 2
30
0. 15
30
20
0. 10
20
0.
10
0.00
0-
0
-0. 05
-10
-10
-0. 10
-20-
-20
T1-C1
T1-C2
T2-C1
T2-C2
T1-C1
T1-C2
T2-C1
T2-C2
T1-C1
T1-C2
T2-C1
T2-C2
Figure Inner product the features maximal singular vector at different stages of label Ta-Cb indicates that data drawn from class-b in a th task. We choose Task-0 and Task-2 in CIFAR-100 the first and second task.
Initial izat ion
Stage
Stage
30-
0. 15
30
20
10-
0.05
10
0-
0.00
-0.00 05
-10-
-10
-0.
-20
-20
T1-C1
T1-C2
T2-C1
T2-02
T1-C1
T1-C2
T2-C1
T2-02
T1-C1
T1-C2
T2-C1
T2-C2
Figure 14, Inner product of the features with the maximal singular vector different stages of CL. The label Ta-Ch indicates that the data drawn from class-b a th task. We choose Task-0 and Task-3 in Tiny-ImageNe as the first and second task.
39


Reviews:
[{"review_id": "llQSaBtN5L", "reviewer": "Reviewer_iCej", "summary": "The paper aims to provide a theoretical understanding of catastrophic forgetting in a two layered CNN using a multiview data model to understand the learning dynamics of different features. The key theoretical insights are that task specific features have a larger signal than the general features and task specific features of prior tasks appear as a random feature with a strong signal in other tasks. Authors provide theoretical proof and test their findings on both simulated and real world datasets.\n\n## Update after rebuttal\n\nI would like to thank the authors for addressing the questions and raised concerns. The clarifications in the rebuttal increases my confidence in the work and I would increase my score accordingly. I would still strongly request the authors to improve the readibilty of their work and provide more intuition for their main theorems so that the paper reaches a wider audience and has more impact on the commmunity.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the thoughtful and insightful review, and we answer the concerns as follows:\n\n**Q1**. Generalizing the simple theoretical model to complex settings:\n\n**A1**. Studying the learning dynamics in a two-layer CNN and multi-view model is a widely used approach for understanding deep learning. Despite its simplicity, our results are supported by our numerical and real-world results.\n\nWe agree with the reviewer that task-IL is less challenging. However, **even within task-IL, a theoretical understanding about CF remains under-explored, particularly in CNNs**. Moreover, we believe our analysis provides a novel perspective on CF by analyzing feature learning. Our framework is flexible and can be extended to more complex settings, please refers to **Q1** in our response to Reviewer QA6F.\n\n**Q2.** Why analyze feature extraction.\n\n**A2.** By fixing the last layer, we can focus on the learning dynamics of features to find the hidden reason behind CF. **It is crucial for identifying whether and how certain features contribute to CF**. Our setup of numerical empiriments follows the theoretical framework.\n\n**Q3**. Intuitions for theorems and assumptions.\n\n**A3**. Yes, the intuition behind our assumptions is shown in Remark 3.6. We additionally provide intuitions:\n\n**An example on two-task-IL to illustrate our data model** ($\\epsilon \\sim U(\\\\{+1,-1\\\\})$):\n   - Task 1: Data $\\mathbf{x}$ with label $y$ contains features $\\\\{ y \\alpha _u \\mathbf{e} _1,  y \\alpha _v \\mathbf{e} _{rob}, \\epsilon \\alpha _{\\zeta}\\mathbf{e} _2 \\\\}$ and noises.\n   - Task 2, Data $\\mathbf{x}$ with label $y$ contains features $\\\\{ y \\alpha _u \\mathbf{e} _2,  y \\alpha _v \\mathbf{e} _{rob}, \\epsilon \\alpha _{\\zeta}\\mathbf{e} _1 \\\\}$ and noises.\n\n**Note**: All of $\\\\{ \\mathbf{e} _1, \\mathbf{e} _{rob}, \\mathbf{e} _2 \\\\}$ appear in each task, but only $y\\alpha _u\\mathbf{e} _{\\tau}$ and $y\\alpha _v\\mathbf{e} _{rob}$ are correlated with the true label in task $\\tau$, which means the two features can be both used for classification in task $\\tau$. $\\epsilon\\alpha _{\\zeta}\\mathbf{e} _{3-\\tau}$ is called random feature, which  is uncorrelated with the true label. In such a model, **the task-specific feature is only used in its respective task**. $\\alpha _u, \\alpha _v, \\alpha _{\\zeta}$ control the strength of the features.\n\nTo analyze the learning dynamics, we quantify the learning speed of feature components in the first and second stages. \n1. **Initialization**, Lemma 4.1 shows the order of $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{u}^{(\\tau)} \\rangle, \\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{v} \\rangle, \\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{\\zeta} \\rangle, \\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{\\xi} \\rangle.$\n2. **First stage**, we analyze the learning speed of each component and find that when $\\alpha _v \\leq o(\\alpha _u)$, $\\mathbf{u}^{(1)}$ is captured by the model. \n3. **Second stage**, Corollary 5.3 shows the order of each component at the beginning of the second stage. We then analyze the learning speed of each component and observe that $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{\\zeta} \\rangle$ decreases with the rate depending on $\\alpha _\\zeta$. Specifically, when $\\alpha _\\zeta \\geq \\Omega(\\alpha _u)$, the decline becomes significant, leading to forgetting.\n4. **Conclusion** We identify two key conditions leading to forgetting:\n   1. $\\alpha _v \\leq o(\\alpha _u)$. The common feature is absent or has weak signal.\n   2. $\\alpha _\\zeta \\geq \\Omega(\\alpha _u)$. The random feature has a strong signal.\n\n**Q4**. Inconvincing evidence on real-world datasets.\n\n**A4**.  The reviewer misunderstands our feature definitions; we clarify this in **Q3** in this response. Based on our definition, we **evaluate the performance of the second task at the end of the first stage** (Figure 3) to study the learned feature at the end of the first stage is task-specific or general:\n- If the model learns a general feature on first task, it can achieve a great performance on the unseen second task (Task-0 and Task-4 in CIFAR-10). \n- If the model learns the task-specific feature in first task, it can not achieve a great performance on the unseen second task (Most Case in Figure 3). **This aligns with the condition $\\alpha _v \\leq o(\\alpha _u)$**.\n\n**Note:  We do not compare the performance of model on current and unseen task**\n\nAdditionally, Figures 4 and 5 analyze learning on Task-0 and Task-3 in CIFAR-10, where forgetting occurs:\n- Figure 4 provides evidence for the existence of random features in real-world datasets, as features from task 1 and task 2 exhibit significant overlap, **which aligns the condition $\\alpha _\\zeta \\geq \\Omega(\\alpha _u)$**.\n- Figure 5 demonstrates that in the second stage, the model relies on task 2's task-specific feature for classification, and task 2's task-specific feature is helpless for task 1.\n\n**Q5**. Impact on CL research.\n\n**A5**. Please refers to **Q1** in our response to Reviewer g9HH.", "claims_and_evidence": "The theoretical model is simplified (two-layer CNN, task-incremental setting with only two binary classification tasks). While insightful, it’s unclear if the findings generalize to deeper CNNs or more complex continual learning settings (e.g., class-incremental CL).", "methods_and_evaluation_criteria": "Task-incremental CL is less challenging than class-incremental CL. Results may not generalize. Only two tasks are studied—how does CF evolve across many tasks?", "theoretical_claims": "Reviewer was not able to validate the correctness of the proofs and theoretical claims.", "experimental_designs_or_analyses": "The numerical analysis in section 6.1 states that the last layer of the CNN s fixed but authors don’t provide a reason for why it is necessary to analyze the feature extraction capabilities of the model.\n\nThe experiments on real world datasets are hard to follow and the authors do not provide sufficiently convincing evidence for their claims. The expectation that models should be able to learn general features from earlier tasks that allows them to generalize on unseen tasks without even seeing these classes or training the model is unfounded. Similarly, using the performance on the current task and comparing it to unseen tasks to make the claim that the model tends to extract task-specific features isn’t convincing. The authors seem to discount the difference between representation learning and learning classifiers on top of it to learn a decision boundary. Even if the model is able to learn general features in Task 0, we cannot expect the model to be able to classify objects in task 4 without ever training the model on them. The same expectation is carried over to the T-SNE plots. Overall the reviewer did not find section 6.2 technically sound and convincing.", "relation_to_broader_scientific_literature": "The paper provides theoretical insights to understand catastrophic forgetting in two layered CNN. However it lacks suggestions on how these insights can be utilized to design a CL method and there is limited evidence to suggest the findings will generalize to commonly used architectures on complex datasets.", "essential_references_not_discussed": "Not to my knowledge.", "other_strengths_and_weaknesses": "Strengths:\n\n- The paper aims to understand catastrophic forgetting in CL and provides interesting insights which might be useful for the CL research community. \n- Useful framework for analyzing feature learning in CL.\n\nWeaknesses:\n- In addition to the concerns raised earlier, the paper is difficult to follow and would benefit a lot from major restructuring and better contextualization of theoretical insights and findings. The authors can provide more intuition for their theorems and assumptions before delving into theoretical proofs. This would help a broader audience to understand the key takeaways of the paper.\n\n- Missing discussions on other CL approaches (e.g., regularization-based, architectural methods).", "other_comments_or_suggestions": "The analysis would be more useful for the research community if the authors can add a discussion on how these insights help us design better architectures/methods to improve CL ability of the model.", "questions_for_authors": "Q1) Can the authors comment on how they believe the insights from their analysis can benefit the CL research community and if these can be used to design better CL methods or benchmarks. Reviewer believe this is an important section missing in the paper. \n\nQ2) Do the authors believe their findings would generalize to deeper architectures and more complex CL settings (e.g. Class-IL)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WqItDF5gxv", "reviewer": "Reviewer_g9HH", "summary": "This paper investigates the catastrophic forgetting (CF) phenomenon in continual learning (CL) for convolutional neural networks (CNNs) from the perspective of training dynamics. The paper considers a multi-view data model with four components: task-specific features, general features, random features, and background noise. The theoretical results in this paper show some reasons for the occurrence of CF: (1) the task-specific feature has a larger signal than the robust feature, which can genearize well in both tasks, causing the model to learn the task-specific feature rather than the robust feature; (2) the task-specific feature from one task acts as a random feature for another task, causing the model to forget the learned task-specific feature for task 1 while learning task 2. Experiments conducted on both simulated and real-world data sets validate the theoretical claims.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer g9HH for the careful reading and insightful suggestions. We will fix typos and improve our paper based on these suggestions in our revised version. We answer the major concerns as follows:\n\n**Q1**. The main implication of the theoretical results remains unclear.\n\n**A1**. Intuitively, our results suggest that CF occurs under two key conditions::\n- **The general feature is absent or has a weak signal while the task-specific feature has a relatively large signal**. If $\\alpha _v \\leq o(\\alpha _u)$, after training in the first stage, only $\\mathbf{\\mathbf{u}}^{(1)}$ is learned by the model.\n- **The task-specific feature from the first task manifests as a random feature with a strong signal in the second task**. The  model forgets $\\mathbf{\\mathbf{u}}^{(1)}$ with a fast speed while learns $\\mathbf{\\mathbf{u}}^{(2)}$ in the second stage.\n\nFuthermore, it implies that\n\n1. **We can overcome CF by breaking these two conditions**. To empirically validate this, we conduct experiments in Figure 2. As shown in the first row, third column of Figure 2, when $\\alpha _\\zeta$ decreases or $\\alpha _v$ increases, CF can be mitigated.\n2. **CNN tends to learn the strongest feature rather than the most robust feature** with (S)GD, which suggests the need for designing robust training algorithms to prevent CF.\n3. **CF is less likely to occur when learning on similar tasks**. We can quantify the similarity through the parameters $\\alpha _u,\\alpha _v,\\alpha _\\zeta$. For example we fix $\\alpha _u,\\alpha _\\zeta$,  as $\\alpha _v$ increases, the tasks become more similar. Our results suggest that CF is less likely to occur when learning on such similar tasks.\n4. **We provide a novel perspecitve for understanding CF**. We believe our works can inspire future works to think CF from the perspective of learning features with interesting ideas.\n\nWe will include the discussion in our revised version.\n\n**Q2**. It is unclear how their analysis relates to over-parameterization, task similarity, and other factors identified in earlier research.\n\n**A2**. **Our study is conducted in the over-parameterized regime** as we assume $n \\ll d$. Regarding task similarity, **we can quantify the similarity through the parameters $\\alpha _u,\\alpha _v,\\alpha _\\zeta$**, as we mentioned in **A1** in this response.\n\n**Q3**. In the right row of line 231, it lacks a theorem for $\\ell^{\\prime}(\\mathbf{z} _1)/\\ell^{\\prime}(\\mathbf{z} _2)$.\n\n**A3**. The formal theorem is provided in Lemmas A.4 and A.5 in Appendix A. We will clarify this in the main body of the paper in the revision.\n\n**Q4**. About the dimension of $e$'s.\n\n**A4**. We will rewrite it as \"$\\mathbb{O}$ is a subspace in $\\mathbb{R}^{d}$\".", "claims_and_evidence": "Yes, the central claims are supported by evidence. The theoretical claims in this paper are all supported by strict proofs. The results are also validated by experiments on both simulated and real data sets.", "methods_and_evaluation_criteria": "Yes. For the theoretical part, the paper uses the classification accuracy to measure the performance of a model, it is very reasonable. For the experimental part on the real-world dataset, it is hard to decide which feature has a stronger signal, the paper uses the singular vector with maximal singular value, I think it makes sense.", "theoretical_claims": "Yes, I have checked the proofs of the main theoretical results in this paper. I did not find any explicit errors in the proofs, and they appear to be correct.", "experimental_designs_or_analyses": "Yes, the experimental design is reasonable. The main purpose of the experiments in this paper is to validate the findings derived from the theoretical results. In addition to the experiments on simulated data, the experiments on the real-world datasets are also very important, which makes the theoretical findings meaningful in the real-world.", "relation_to_broader_scientific_literature": "The contribution of this paper lies in two parts. Firstly, for the CL community, this paper shows reasons for CF in CL, which provides a theoretical explanation for CF. Secondly, for the feature learning community, most works study cases where the parameters of the models are initialized according to a Gaussian distribution, while this paper also studies another scenario where the parameters are not randomly initialized (in the second stage). So this paper contributes to both CL and feature learning communities.", "essential_references_not_discussed": "I do not find any essential references that are not discussed.", "other_strengths_and_weaknesses": "Strengths\n\n- The paper investigates CF in CL from a training dynamics perspective, it is a novel view from feature learning.\n- The paper explains the reason of CF in CL through theoretical analysis. The results of the paper are meaningful and insightful, and are consistent with the validation experiments in this paper.\n- The experimental results support the theoretical claims, making the results more convincing.\n- The writing is kind to readers, the authors show the insights of the learning process, which helps to understand the main ideas of this paper. \n\nWeaknesses\n\n- Although theoretical results identify the condition that leads to the CF, the main implication of the theoretical results remains unclear.\n\n- This work studies CF based on learned features, but it is unclear how their analysis relates to over-parameterization, task similarity, and other factors identified in earlier research.\n\n- In the right row of line 231. The authors claim that $\\ell^{\\prime}(\\mathbf{z}_1)/\\ell^{\\prime}(\\mathbf{z}_2)$ has a same order with $\\exp(\\mathbf{z}_2 - \\mathbf{z}_1)$, but lacks a theorem to illustrate it.", "other_comments_or_suggestions": "Typos:\n  - On line 99, the left column, \"a lot of works study\" but not \"a lot of works studies\".\n  - In the second part of Definition 3.3, it should be $(x,y) \\sim \\mathcal{D}_ {z}^{(1)}$ but not $(x,y) \\sim \\mathcal{D}_{z}^{(2)}$.\n  - On line 79, the right column, it might be \"for any $T^{(2)}$\" but not \"for $T^{(2)}$\".\n  - In the second and third equations in Theorem 5.4, $(x,y)$ is better than $(x_i,y_i)$.\n\nSome Minor Issues:\n  - **The dimension of $e$'s**: On line 80, the right column, since $\\mathbb{O}$ is a subspace in $\\mathbb{R}^{d \\times (P+3)}$ and $\\\\{ e_1, e_2, e_{rob} \\\\}$ span $\\mathbb{O}$, they should be of dimension $d \\times (P+3)$. While on line 116, the left column, the expression $I_d - e_1 e_1^T$ shows that $e_1$ is of dimension $d$.\n  - **The definition of $\\ell$**: On line 151, the left column, the logistic loss is defined as a function of two variables $\\ell(F(x),y)$. While on line 159, the left column, the derivative of $\\ell$ is used as a scalar, which is confusing. I think a good way is to define the logistic loss as $\\ell(F(x),y) = L(yF(x)) = \\log \\left( 1 + \\exp{(-y F(x))} \\right)$ so on line 159, we can use $L^\\prime(y_i F(x_i))$.\n  - **The expression in Definition 3.3**: the expression \"with a high probability $\\delta_1$\" is not proper, since correctly classifying the test data  with a high probability requires $1-\\delta_1$ is close to $1$. So is can be corrected as \"with a high probability $1-\\delta_1$\". The expression in the second part faces the same problem, it might be better to change \"with a high probability $\\delta_2$\" into \"for a small $\\delta_2$ that is close to $0$\".\n  - **The definition of $\\mathcal{I}$**: On line 117, the left column, $\\mathcal{I}$ is the index set of $\\mathcal{S}$, do you mean that $\\mathcal{I} = [n]$?\n  - **The assumptions in Condition 3.5**: The usual concept of over-parameterization refers to the scenario where the number of parameters of the model exceed the size of the training dataset, in this paper, it means that $n \\le d\\times(P+3)$, which is a little far from $nP \\le o(\\sqrt{d})$. Maybe the order of the expression should be changed, for example, you can say \"$nP \\le o(\\sqrt{d})$, i.e., the network is over-parameterized\".", "questions_for_authors": "See the \"Weaknesses\" and \"Other Comments Or Suggestions\".", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A6s6cNVbCA", "reviewer": "Reviewer_QA6F", "summary": "Continual learning is an important area in machine learning, and catastrophic forgetting is the most important problem in continual learning. Despite the empirical efforts on suppressing forgetting in continual learning, the theoretical understanding of catastrophic forgetting remains less studied, especially in CNN. In this work, the authors propose a novel framework  to theoretically study catastrophic forgetting in a simple CNN model. Their analysis shows that the different distribution of features in different tasks causes catastrophic forgetting. They conduct experiments to verify their findings.\n\n--------------------after rebuttal----------------------\nThanks the author for the reply. They have sufficiently addressed my questions and concerns. After checking other reviewer's comments and the response, I think this paper takes a nice step on understanding the CL. Futher works may be inspired by this paper. Thus, I increase my score.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your positive comments. We answer the questions as follows.\n\n**Q1**. The authors only study the case of two-task continual learning.\n\n**A1**. Our framework can be naturally extended to $M$-class CL. Our framework can be extended to $M$-task scenarios. A key modification is to assume $m+1$ basis vectors $\\\\{\\mathbf{e} _{(\\tau)}\\\\}^{M} _{\\tau=1} \\cup \\\\{\\mathbf{e} _{rob}\\\\}$ exists in common feature space. In task $\\tau$, the data $\\mathbf{x}$ with label $y$ contains features $\\\\{ y\\alpha _u \\mathbf{e} _{(\\tau)}, y\\alpha _v \\mathbf{e} _{rob} \\\\} \\cup \\\\{\\epsilon\\alpha^{(m,\\tau)} _\\zeta\\mathbf{e} _{(m)}\\\\} _{m \\in [M], m \\ne \\tau}$ along with background noise, where $\\epsilon$ is defined in Definition 3.1.\n\n**Proof sketch**:\n- Step 1. Show that at the initialization, $\\forall \\tau \\in [M],$\n$$\n\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{u}^{(\\tau)} \\rangle = \\widetilde{\\Theta}(\\sigma _0\\alpha _u), \\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{v} \\rangle = \\widetilde{\\Theta}(\\sigma _0\\alpha _v), \\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{\\zeta}^{(\\tau)} \\rangle = \\widetilde{\\Theta}(\\sigma _0\\sigma _\\zeta), \\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{\\xi} \\rangle = \\widetilde{\\Theta}(\\sigma _0\\sigma _\\xi).\n$$\n- Step 2. For any $m \\in [M]$, iteratively analyze the learning speed of each component and derive the order of $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{u}^{(\\tau)} \\rangle$ for any $\\tau$, as well as $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{v} \\rangle$, $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{\\zeta}^{(\\tau)} \\rangle$ and $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{\\xi} \\rangle$ at the end of the stage $m$. Those orders depend on $\\alpha^{(m,\\tau)} _\\zeta, \\alpha _v, \\alpha _u$. The technique is similar within two-task scenario.\n- Step 3.  Identify the conditions under which catastrophic forgetting (CF) occurs.\n\nAs a result, we believe **the conclusions and implications are similar to the two-task scenario**.\n\nSimilarly, our framework can be extended to **Class-IL**: We can **modify our loss from logistic loss to cross entropy loss** for multi-class classification. The main idea remains the same—analyzing the learning dynamics of different features and determining the order of each component at the end of each stage, just as in task-IL.\n\n**Q2**. Distinguish the task-specific and general feature in real-world dataset.\n\n**A2**. It is a good question. In real-world datasets, our empirical study on Figure 3 shows that the existence of general feature is hard to satisfy. In most case, the model learns the task-specific feature. We believe that distinguishing features is a challenging problem to be solved in future works.\n\n**Q3**. More detailed explanation about $\\sigma _\\xi$ \n\n**A3**. $\\sigma _\\xi$ reflects the strength of noises in an image. Our condition about  $\\sigma _\\xi$ is that $\\omega(1)\\leq \\sigma _\\xi \\leq o(P^{-1}\\sigma _{0}^{-1})$ and $n\\alpha^{3} _{u}/\\sigma^{3} _{\\xi}\\geq \\omega(1)$. For the first condition, $\\sigma _\\xi \\leq o(P^{-1}\\sigma _{0}^{-1})$ is to ensure that at the initilization, $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\sigma _{\\xi} \\rangle$ is small enough. $\\sigma _\\xi \\geq \\omega(1)$ is to ensure the learning of features is hard under the noises. For the second condition, it provides  a lower bound for $\\text{SNR}^3$, which is also shown in Cao et al.(2022b). \n\n**Q4**. About Figure 5.\n\n**A4**. First, Figure 5 demonstrates that in the second stage, the model relies on task 2's task-specific feature for classification. Additionally, in the bottom row, middle column of Figure 4, we observe that the data from the first task is no longer separable, unlike in the middle row, middle column of Figure 4. These empirical findings provide evidence that the model forgets the learned feature from the first task.\n\n**Q5**. Why do some results, such as Corollaries 5.3 and 5.5, show only the order of the largest inner product among $C$ channels?\n\n**A5**. We have shown that both $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{u}^{(\\tau)} \\rangle$ and $\\max _{c\\in[C]} \\langle \\mathbf{w} _{c}, \\mathbf{v} \\rangle$ monotonically increases. The analysis of the largest inner product among $C$ channels demonstrates that **at least one channel learns the component**.\n\n**Q6**. Motivation and Intuition behind the random noise in the multi-view data model?\n\n**A6**. **The existence of random noise ensures that the task-specific feature can only be used for one task**. For this reason, we let the random feature be $\\epsilon \\alpha _{\\zeta} \\mathbf{e} _{3-\\tau}$ rather than $y \\alpha _{\\zeta} \\mathbf{e} _{3-\\tau}$. Moreover, we use $\\alpha _\\zeta$ to control the strength of the random feature. When $\\alpha _\\zeta$ is zero, the random feature disappears; and as it increases, forgetting occurs.", "claims_and_evidence": "The main claim of this paper is presented in Section 5, where the authors identify the condition leading to catastrophic forgetting as follows:\n\n1. The task-specific feature has a relatively larger signal than the general feature.\n2. The task-specific feature from one task appears as a random feature in the second task with a strong signal.", "methods_and_evaluation_criteria": "1. Theoretically, the authors propose a framework to show the evidence for the main claim. They formally define catastrophic forgetting in Definition 3.3.\n\n2. Experimentally, they present a numerical analysis with simulated data. By evaluating the performance of model on the old and new tasks after training, they find that the signals of the general feature and the task-specific feature are relevant to catastrophic forgetting. Furthermore, they visualize the feature space in a real-world data set to show that the learned features after learning the old and new tasks are different.", "theoretical_claims": "The theoretical claim in this paper consists of two sections: In Section 4, the authors study catastrophic forgetting in a simplified setting, and in Section 5 they show the main results. I have read the proofs in both sections, and their theoretical claim seems correct to me.", "experimental_designs_or_analyses": "I have reviewed the experimental designs and analyses presented in Section 6, and most of the analyses are reasonable and convincing to me.", "relation_to_broader_scientific_literature": "The main contribution of this paper is that they analyze the catastrophic forgetting problem in a specific CNN model, which has not been studied in recent works. As for the results in this paper, the authors analyze the reason behind the catastrophic forgetting of the features stored in the data. In the previous works, they show the reason behind the catastrophic forgetting from task dissimilarity, task order, and so on.", "essential_references_not_discussed": "Most of the important related works are included in this work. I have not found any other important references that should be discussed in this work.", "other_strengths_and_weaknesses": "**Strengths**\n\n1. Catastrophic forgetting is a valuable and important problem to study in continual learning. The authors propose a novel framework and study this problem in the CNN model, which is less explored.\n\n2. The authors analyze the training dynamics of continual learning in two stages, which is different from previous studies on studying the learning dynamics in the two-layer CNN model.\n\n3. The authors conduct experiments on both simulated and real-world datasets to verify the theoretical results.\n\n\n**Weaknesses**\n\n1. The authors only study the case of two-task continual learning, which limits the contribution of the work. \n\n2. In the section 6.2, the authors should add more description to illustrate how to distinguish the task-specific feature and general feature in real-world dataset.\n\n3. The remarks about Condition 3.5 could benefit from further clarification. The authors argue that the condition on $\\sigma_\\xi$ is to ensure that ***\"at the beginning of the training process, the network cannot easily classify the data with the task-specific feature\"***, this claim requires a more detailed explanation.\n\n4. The authors use Figure 5 to show that after training on the second task, the model forgets the learned feature from the first task. It is not convincing to me, because the signal of the feature in the second task may be larger than in the first task, and the authors only show the inner product of the features with the largest singular vector.", "other_comments_or_suggestions": "1. Line 79, right column, $\\mathbb{O} \\in \\mathbb{R}^{d \\times (P+3)} \\to \\mathbb{O} \\subseteq \\mathbb{R}^{d \\times (P+3)}$. \n2. Line 83, right column, $\\mathbf{\\xi}(\\tau) \\to \\mathbf{\\xi}^{(\\tau)}$.\n3. The notation can be improved. For example, the authors use both $n_{=}$ and $\\vert \\mathcal{I}_ {=} \\vert$ to denote the size of $\\mathcal{I}_{=}$.", "questions_for_authors": "1. Why do some results, such as Corollaries 5.3 and 5.5, show only the order of the largest inner product among $C$ channels?\n\n2. What is the motivation and intuition behind the random noise in the multi-view data model?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UGg0ppDuMA", "reviewer": "Reviewer_ZF1H", "summary": "This paper presents a theoretical analysis of catastrophic forgetting in continual learning using a two-layer neural network model. It constructs a multi-view data structure consisting of task-specific, general, and random features and examines the learning dynamics of these features. The study identifies two potential causes of catastrophic forgetting:\n- Task-specific features have a stronger signal than general features.\n- Task-specific features from previous tasks are treated as random features in future tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer ZF1H for the insightful comments. We find the reviewer misunderstanding our framework, and we show a illustration for our framework in **Q3** in our response to reviewer iCej. We the answer the questions as follows.\n\n**Q1**. Ambiguity in Feature Representation\n\n**A1**. Yes, both $\\mathbf{e} _1$ and $\\mathbf{e} _2$ appear in both tasks, please see our example in **Q3** in our response to reviewer iCej. **The existence of random noise ensures that the task-specific feature can only be used for one task**. For this reason, we let the random feature be $\\epsilon \\alpha_{\\zeta} \\mathbf{e}_{3-\\tau}$ rather than $y \\alpha_{\\zeta} \\mathbf{e}_{3-\\tau}$. Our definition of task-specific feature is reasonable. Furthermore, the task 2's feature $\\mathbf{e} _2$ **is not** learned in Task 1's training phase, as shown in Corollary 5.3.\n \n\n**Q2**. The proof relies heavily on specific choices of $\\alpha _{u}, \\alpha _{v}, \\alpha _{\\zeta}$.\n\n**A2**. Please see **Q3** in our response to reviewer iCej for clarification of our main ideas. While our theoretical results shows the conditions leading to CF depends on $\\alpha _{u}, \\alpha _{v}, \\alpha _{\\zeta}$, they help reveal the underlying causes of CF, as shown on Page 7, left column, lines 341-453. Empirical studies of $\\alpha _{u}, \\alpha _{v}, \\alpha _{\\zeta}$ in Figure 2 support our theoretical findings.\n\n**Q3**. About contradiction from $T$.\n\n**A3**. We want to clarify that at the beginning of the second stage we set $\\mathbf{w} _{c}^{(2)}(0) = \\mathbf{w} _{c}^{(1)}(T^{(1)} _{u})$. Here, $T^{(1)} _{u}$ refers to the time in the first stage while $T^{(2)} _{u}$ refers to the time in the second stage, **but not the total time in the first and seond stages**. The entire process can be described as follows:\n1. Initially, it takes $T^{(1)} _{u}$ epochs to reach the end of the first stage, after which the process transitions to the second stage. Theorem 5.2 holds in the end of the **first** stage.\n2. After an additional $T^{(2)} _{u}$ epochs, the second stage concludes. Theorem 5.4 holds in the end of the **second** stage.\n\n**There's no contradiction** for Theorems 5.2 and 5.4.\n\n**Q4**. Setting  $\\sigma _\\xi = 0$ is invalid under the last condition 3.5.\n\n**A4**. Thanks for pointing this out. First, $\\sigma _\\xi = 0$ is only used for simplicity in section 4 and **does not impact the analysis of our main results**. Additionally, when setting $\\sigma _\\xi = 0$ in Section 4, all conditions involving $\\sigma _\\xi$ in Condition 3.5 should be removed. We will clarify this in our revised version.\n\n**Q5**. The paper lacks discussions on several theoretical works.\n\n**A5**.  Thank you for your suggestions. Benjamin et al. (2024) introduce the concept of NTE in CL, reformulating a single neural network as an ensemble of fixed classifiers. Li et al. (2024) provide a theoretical study of mixture-of-experts models in CL. Ding et al. (2024) analyze factors contributing to forgetting under linear models in CL. Li et al. (2023) investigate $\\ell _2$-regularized CL with two linear regression tasks. Zheng et al. (2024) examine memory-based CL under overparameterized linear models. **Our study focuses on CF in a two-layer CNN using multi-view data, which differs from these works.** We will include discussions on these related works in our revised version.\n\n**Q6**. Confusion in the Simplified Model.\n\n**A6**. We clarify that **the reviewer miss the condition** $\\alpha _v \\gg \\alpha _u$ for learning $\\mathbf{v}$ in the first stage. If $\\alpha _v \\gg \\alpha _u$, it implies that $T^{(1)} _{v} \\gg T^{(1)} _{u}$, and the model will **only capture the general feature $\\mathbf{v}^{(1)}$ and not the task-specific feature** $\\mathbf{u}^{(1)}$, and   $\\mathbf{u}^{(1)}$ will not contribute to classification in task 1. In the right column of Line 200, we similarly shows that if $\\alpha _u \\gg \\alpha _v$, the model will only capture the task-specific feature $\\mathbf{v}^{(1)}$ and not the general feature in the first stage, and $\\mathbf{v}^{(1)}$ will not contribute to classification in task 1.\n\n\n**Q7**. Limited Generalization.\n\n**A7**. Please see our response to **Q1** to reviewer QA6F.", "claims_and_evidence": "Ambiguity in Feature Representation:\n- The data model defines over Features $=\\left[e_1, e_2, e_{\\text {rob }}\\right]$, where the random feature for each task is expressed as $\\alpha_\\zeta e_{3-\\tau}$.\n- This implies that the random feature for Task 1 is $\\alpha_\\zeta e_2$, which coincides with the task-specific feature of Task 2.\n- Consequently, the model appears to learn Task 2's feature during Task 1's training phase, and also learn Task 1's feature during Task 2's training phase, potentially confusing the conclusions.\n\nDependence on Coefficient Choices:\n- The proof relies heavily on specific choices of $\\alpha_u, \\alpha_v, \\alpha_\\zeta$.\n- However, Theorems 5.2 and 5.4 use identical values for $T_u^{(1)}$ and $T_u^{(2)}$, suggesting that for some $T \\geq T_u^{(1)}, T_u^{(2)}$, both conclusions hold simultaneously--leading to a contradiction.\n\nLine 191: Setting $\\sigma_{\\xi}=0$ is invalid under the last condition 3.5.", "methods_and_evaluation_criteria": "N/A", "theoretical_claims": "I reviewed the main framework of the proof but did not go through the step-by-step details.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "- The paper introduces a novel framework to analyze catastrophic forgetting in continual learning, offering valuable theoretical insights into its underlying causes.\n- The data model distinguishing task-specific and robust (general) features provides a reasonable perspective for understanding catastrophic forgetting.", "essential_references_not_discussed": "The paper lacks discussions on several theoretical works related to continual learning, including:\n\n- Zheng et al. (2024) – Understanding memory buffer-based continual learning.\n- Li et al. (2023) – Fixed design analysis of regularization-based continual learning.\n- Ding et al. (2024) – Understanding forgetting in continual learning with linear regression.\n- Li et al. (2024) – Theory on mixture-of-experts in continual learning.\n- Benjamin et al. (2024) – Continual learning with the neural tangent ensemble.", "other_strengths_and_weaknesses": "Two Additional Limitations:\n\nConfusion in the Simplified Model (Section 4):\n- The analysis in Section 4 is somewhat unclear.\n- For instance, if $v_1=v_2$, the paper claims that the model performs well on both tasks during training and testing.\n- However, this seems counterintuitive, as it suggests that task-specific vectors do not contribute meaningfully to classification.\n- Are there missing conditions on task-specific vectors that would clarify their role in the model's performance?\n\nLimited Generalization: The analysis is restricted to a two-task setting, making it unclear how well the results generalize to m-task scenarios.", "other_comments_or_suggestions": "Above.", "questions_for_authors": "Above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This submission studies the catastrophic forgetting of continual learning in two-layer convolutional neural networks. The authors provide principled analysis under this regime by means of the multi-view model context. Extensive experiments support the theoretical findings and claim.

The submission received the review of four reviewers, which respectively recommended 3, 4, 5, 3. Basically, their initial concerns place on the unclear implication of the theoretical results, requiring more clarity in the technical deduction and missing related works with the proper discussion. After the authors's substantial rebuttal, most concerns have been well addressed. AC have checked the reviewer's questions and the authors' reply in the interaction. Overall, AC agrees with the reviewers' suggestion and tends to recommend "Acceptance". Hope the authors carefully incorporate the reviewers' suggestion into the main body of the manuscript.
Decision: Accept (poster)