Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting
Authors: Ryo Karakida
Shotaro Akaho
Venue: ICLR 2022 Poster (2022)
Keywords: continual learning
neural tangent kernel
statistical mechanics

Abstract: Sequential training from task to task is becoming one of the major objects in deep learning applications such as continual learning and transfer learning. Nevertheless, it remains unclear under what conditions the trained model's performance improves or deteriorates. To deepen our understanding of sequential training, this study provides a theoretical analysis of generalization performance in a solvable case of continual learning.  We consider neural networks in the neural tangent kernel (NTK) regime that continually learn target functions from task to task, and investigate the generalization by using an established statistical mechanical analysis of kernel ridge-less regression. We first show characteristic transitions from positive to negative transfer. More similar targets above a specific critical value can achieve positive knowledge transfer for the subsequent task while catastrophic forgetting occurs even with very similar targets. Next, we investigate a variant of continual learning which supposes the same target function in multiple tasks. Even for the same target,  the trained model shows some transfer and forgetting depending on the sample size of each task.  We can guarantee that the generalization error monotonically decreases from task to task for equal sample sizes while unbalanced sample sizes  deteriorate the generalization. We respectively refer to these improvement and deterioration as self-knowledge transfer and forgetting, and empirically confirm them in realistic training of deep neural networks as well. 

Published as a conference paper at ICLR 2022
LEARNING CURVES FOR CONTINUAL LEARNING IN NEURAL NETWORKS: SELF-KNOWLEDGE TRANSFER
AND FORGETTING
Ryo Karakida & Shotaro Akaho
National Institute of Advanced Industrial Science and Technology (AIST), Japan Ikarakida dakidd..a ryo, s akaho (eaist.go. jp
ABSTRACT
Sequential training from task to task is becoming one of the major objects in deep learning applications such as continual learning and transfer learning. Nevertheless, it remains unclear under what conditions the trained model's performance improves or deteriorates. To deepen our understanding of sequential training, this study provides theoretical analysis of generalization performance in solvable case of continual learning. We consider neural networks in the neural tangent kernel (NTK) regime that continually learn target functions from task to task, and investigate the generalization by using an established statistical mechanical analysis of kernel ridge-less regression. We first show characteristic transitions from positive negative transfer. More similar targets above specific critical value can achieve positive knowledge transfer for the subsequent task while catastrophic forgetting occurs even with very similar targets. Next, we investigate a variant of continual learning which supposes the same target function multiple tasks. Even for the same target, the trained model shows some transfer and forgetting depending on the sample size of each task. We can guarantee that the generalization error monotoni- cally decreases from task to task for equal sample sizes while unbalanced sample sizes deteriorate the generalization. We respectively refer to these improvement and deterioration as self-knowledge transfer and forgetting, and empirically confirm them in realistic training of deep neural networks as well.
INTRODUCTION
As deep learning develops for single task, it enables to work more complicated learning frame- works where the model is sequentially trained on multiple tasks, e.g.. transfer learning, curriculum learning, and continual learning. Continual learning deals with the situation in which the learning machine cannot access previous data due to memory constraints or privacy reasons. It has attracted much attention due the demand on applications, and fundamental understanding and algorithms are being explored (Hadsell et al., 2020). One well-known phenomenon is catastrophic forgetting; when network is trained between different tasks, naive training cannot maintain performance on the previous task (McCloskey & Cohen, 1989; Kirkpatrick et al., 2017).
remains unclear in most cases under what conditions trained model's performance improves or deteriorates in sequential training. Understanding its generalization performance is still limited (Pentina & Lampert, 2014; Bennani al., 2020: Lee et al., 2021). For single-task task training, however, many empirical and theoretical studies have succeeded in characterizing generalization performance over-parameterized neural networks and given quantitative evaluation on sample-size dependencies, e.g., double descent (Nakkiran et al.. 2020). For further development, it will be helpful to extend the analyses on single-task training to sequential training on multiple tasks and give theoretical backing.
To deepen our understanding of sequential training, this study shows theoretical analysis of its generalization error in the neural tangent kernel (NTK) regime. In more details, we consider the NTK formulation of continual learning proposed by Bennani al. (2020); Doan et al. (2021). By extending a statistical mechanical analysis of kernel ridge-less regression, we investigate learning curves, i.e., the dependence of generalization on sample size number of tasks. The analysis

---

Published as a conference paper at ICLR 2022
focuses on the continual learning with explicit task boundaries. The model learns data generated by similar teacher functions, which we call target functions, from one task to another. All input samples are generated from the same distribution in i..d. manner. and each task has output samples (labels) generated by its own target function. Our main contributions are summarized as follows. First, we revealed characteristic transitions from negative positive transfer depending on the target similarity. More similar targets above a specific critical value can achieve positive knowledge transfer (i.e., better prediction on the subsequent task than training without the first task). Compared to this, backward transfer (i.e., prediction on the previous task) has a large critical value, and subtle dissimilarity between targets causes negative transfer. The error can rapidly increase, which clarifies that catastrophic forgetting is literally catastrophic (Section 4.1).
Second, we considered a variant of continual learning, that is, learning of the same target function in multiple tasks. Even for the same target function, the trained model's performance improves or deteriorates in a non-monotonic way. This depends on the sample size of each task; for equal sample sizes, we can guarantee that the generalization error monotonically decreases from task to task (Section 4.2 for two tasks & Section for more tasks). Unbalanced sample sizes, however, deteriorates generalization (Section 4.3). We refer to these improvement and deterioration of generalization as self-knowledge transfer and forgetting, respectively. Finally, we empirically confirmed that self- knowledge transfer and forgetting actually appear in the realistic training of multi-layer perceptron (MLP) and ResNet-18 (Section 5. Thus, this study sheds light on fundamental understanding and the universal behavior of sequential training in werpparameteiize learning machines.
RELATED WORK
Method of analysis. As analysis tool, use the replica method originally developed for statistical mechanics. Statistical mechanical analysis enables us rypical-cass evaluation, that is, the average evaluation over data samples or parameter configurations. It sometimes provides us with novel insight into what the worst-case evaluation has not captured (Abbaras et al., 2020; Spigler et al., 2020). The replica method for kernel methods has been developed in Dietrich al. (1999), and recently in Bordelon et al. (2020); Canatar et al. (2021). These recent works showed excellent agreement between theory and experiments on NTK regression, which enables us to quantitatively understand sample-size dependencies including implicit spectral bias, double descent, and multiple descent.
Continual learning. Continual learning dynamics in the NTK regime was first formulated by Bennani al. (2020); Doan et al. (2021), though the evaluation of generalization remains unclear. They derived an upper bound of generalization via the Rademacher complexity, but it includes naive summation over single tasks and seems conservative. In contrast, our typical-case -case evaluation enables to newly find such rich behaviors as negative/positive transfer and self- knowleewweee transfer/forgetting. The continual learning in the NTK regime belongs to so-called single-head setting (Farquhar & Gal, 2018), and it allows the model to revisit the previous classes (target functions) in subsequent tasks. This is complementary to earlier studies incremental learning of new classes and its catastrophic forgetting (Ramasesh et al., 2020; Lee et al., 2021), where each task includes different classes and does not allow the revisit. Note that the basic concept of continual learning is not limited incremental learning but allows the revisit (McCloskey & Cohen, 1989; Kirkpatrick et al., 2017). Under the limited data acquisition or resources of memory, we often need train the same model with the same targets (but different samples) from task to task. Therefore, the setting allowing the revisit seems reasonable.
3 PRELIMINARIES
3.1 NEURAL TANGENT KERNEL REGIME
We summarize conventional settings of the NTK regime (Jacot et al., 2018; Lee al., 2019). Let us consider a fully connected neural network = un given by
- o Wihe- 1/ / /M obbr, hu (wm) 1I.,,..,,
(1)
where we define weight matrices W2 E M bias terms by MR, and their variances o and of. We set random Gaussian initialization Wi,s, bú, ~ N/ 1) and focus on the mean squared error loss: L())= Σn= lly" (Il where the training samples {2l yn } composed of
2

---

Published as a conference paper at ICLR 2022
imputs x" RR" normalized by ||2|||2 1 and labels y" ERC. The set of all parameters is denoted as 0, and the number of training samples is N.. Assume the anfinite-width limit for hidden layers (M, - 30), finite sample size and depth. The gradient descent dynamics with certain learning rate then converges to a global minimum sufficiently close to the random initialization. This is known as the NTK regime, and the trained model is explicitly obtained as
(2) We denote the NTK matrix as θ arbitrary input samples (including test samples) as x', and the set of training samples as X. The indices of f mean 0 for the model at initialization and for the head of the network. Entries of NTK 0(z', æ) are defined by "afol=")Vofoo(""" abbreviation for XX) The trained network is equivalent to linearized model around random
(x²) (x') + 0(z', (X)(()) "y/s) - (x)) (c=
We write ((X) as an
initialization (Lee et al., 2019), that is, f()) = 1 + Vofó AO with
C
A = 0 - 00- Σ Volo
( (x) T
(X).
While ver-parameterized models have many global minima, the NTK dynamics implicitly select the above 0, which corresponds to the 1.2 min-norm solution. Usually, we ignore fo by taking the average over random initialization. The trained model (2) then equivalent to the kernel ridge-less regression (KRR).
NTK regime also holds in various architectures including ResNets and CNNs (Yang & Littwin, 2021), and the difference only appears in the NTK matrix. Although we focus on the fully connected network in synthetic experiments, the following NTK formulation of sequential training and our theoretical results hold in any architecture under the NTK regime.
NTK formulation of Continual learning. We denote the set of training samples in the 1-th task as (Xn, Va) (n = 2, K), model trained in a sequential manner from task I to task as fn and its parameters as 0n. That is, we train the network initialized at 0a for the n-th task and obtain fn- Assume that the number of tasks K is finite. The trained model within the NTK regime is then given as follows (Bennani et al., 2020; Doan et al., 2021):
(xx) ) = fn-1(x') + O(x', (x,, (ya fn-1(Xn))
(4)
on - 0 =
(X,))
(5)
We omit the index because each head is updated independently. The model fn completely fits the n-th task, i.c., yn (X.). The main purpose of this study is to analyze the generalization performance of the sequentially trained model (4). At each task, the model has an inductive bias of solution in the parameter space. The next task uses
KRR in the function space and L2 min- this inductive bias as the initialization of training. The problem is whether this inductive bias helps improve the generalization in the subsequent tasks.
Remark (independence between different heads). For a more accurate understanding of the continual learning in the NTK regime, it may be helpful to remark on the heads' independence, which previous studies did not explicitly mention. As in Eq. (2), all heads share the same NTK, and f(e) depends only on the label of its class y(e), While the parameter update (3) includes information of all classes, the e-th head can access only the information of the e-th class' For example, suppose that the n-th task includes all classes except 1, i.c., {2, C}. Then, the model update on the the class 1 at the n-th task, i.c., fa fn-1, becomes
Volo" (x)) 0n-1)
V (x)) EVol (X.. ((x)) 1121
because Vo V flo' 0 (c + c') in the infinite-width limit (Jacot al., 2018: Yang. 2019). Thus, we can deal with each head independently and analyze the generalization by setting C' 1 without loss of generality. This indicates that in the NTK regime, interaction between different heads do not cause knowledge transfer and forgetting. One may wonder if there are any non-trivial knowledge transfer and forgetting in such a regime. Contrary to such intuition, we reveal that when the subsequent task revisits previous classes (targets), the generalization shows interesting increase and decrease.
We use the term "class", although the regression problem is assumed in NTK theory. Usually, NTK studies solve the classification problem by regression with target (r) = {00 1).
3

---

Published as a conference paper at ICLR 2022
3.2 LEARNING CURVE ON SINGLE- TASK TRAINING
To evaluate the generalization performance of (4), we extend the following theory to our sequential training. Bordelon al. (2020) obtained analytical expression of the generalization for NTK regression on a single task (2) as follows. Assume that training samples are generated in an i.i.d. manner x p(z)) and that labels are generated from a square integrable target function f:
f(x) - Σ w,d((x), y" = f(x²) + et
(p=1, [,.1,..,,N)
(6)
where vi; are constant coefficients and e represents Gaussian noise with (ett") = 6oar? We define (x)x) (x) with basis functions (Ir) given by Mercer's decomposition:
[ dd'' (x') θ (zz, x') Φ (z')= = (z))
01 00).
(7)
Here, TI denotes NTK's eigenvalue and we assume the finite trance of NTK E, Th <c We set !/o = O in the main text to avoid complicated notations. We can numerically compute eigenvalues by using the Gauss Geeeesbeeeeb quadrature. Generalization error expressed by
oc.
where f" is trained model and (.")) is the average over training samples. Bordelon et al. (2020) derived typical-cass evaluation of the generalization error by using the replica method: for sufficiently large N, we have asymptotically
Ej (S drp(z) ff(x))(())))
-
1
K
E E
(8)
(K+N
Although the replica method takes a large sample size N, Bordelon et al. (2020); Canatar al. (2021) reported that the analytical expression (8) coincides well with empirical results even for small N. The constants k and y are defined as follows and characterize the increase and decrease of E₁:
T
1=
K
(n + Nn,)
1=0
i=0
The is positive solution of the first equation and obtained by numerical computation. The ? satisfies 0 < y < 1 by definition. For N = oD' (a > 0, O. N, D> 1), we can analytically solve and obtain more detailed evaluation. For example, a positive decreases
the sample size increases and E₁ also decreases for o² = O. For o² > O, the generalization error shows multiple descent depending on the decay of eigenvalue spectra. Since multiple descent is not main topic of this paper, we briefly summarize it in Section A.5 of the Supplementary Materials.
4 LEARNING CURVES BETWEEN TWO TASKS
In this section, we analyze the NTK formulation of continual learning (4) between two tasks (K 2). One can also regard this setting as transfer learning. We sequentially train the model from task A to task B, and each one has target function defined by
1 P).
[a(x) Σ EAA.b.a))) To(x) ".B.vi(a)) [WA, WB.) ~ NO m
(10)
The target functions are dependent on each other and belong to the reproducing kernel Hilbert space (RKHS). By denoting the RKHS by H. can interpret the target similarity p the inner product fBAIf/ ([IffA wlllBlllll = p. These targets have dual representation such as f(x) =
E, (xx,, with i.i.d. Gaussian variables o; (Bordelon et al., 2020), as summarized in Section E.1. We generate training samples by VA fa(=*) + e/A (p 1, ..., NA) and y = f=())) + tt (,=1,... 1,... N)) although we focus on the noise-less case (a = 0) in this section. Input samples A and 2ij are i..dd and generated by the same distribution p(a:). We can measure the generalization error two ways: generalization error on subsequent task B and that on previous task
(/ (A- dxp(=)(fe(x)) (offf(()))f--((3))) fa)B(x))²
EA)B(P)
(11)
AAB (p) ( ddrp(o)((A(s>) - fa-D(10)) (
(12)
4

---

B Published as a conference paper at ICLR 2022
where write f2 fA-B to emphasize the sequential training from A to B. The notation E (p) is referred to as backward transfer (Lopez-Paz Paz & Ranzato, 2017). Large negative backward transfer is known catastrophic forgetting. We take the average (---) over training samples of two tasks (DA, , Dn) }. and target coefficients vo. In fact, we can set ii as constants, and it is unnecessary to take the average. To avoid complicated notation, we take the average in the main text. We then obtain the following result.
Theorem 1. Using the replica method under sufficiently large NA and NB, for a 0, we have
Ea-B(p)= = Σ
2(1-p) (1--)) (1-114 -A.1) 1
EB,b
(13)
VAA
8
22.
E (p) Σ
2(1 - p) + F9s. +
EB,
(14)
YA
8
where define qA KA/(KA + NAT)), QB KB/(KB + NBm), EB,, TB. / and Fyn (a. b(a 2) 62 (11
The constants Kл and V (Kn and yn, respectively) are given by setting N = (N(RR) in (9). The detailed derivation is given in Section A. Technically speaking, we use the following lemma:
Lemma 2. Denote the trained model (2) on single task A as SA Ee tho and define the following cost function: E= EE ds(w"A, - -us)²) for arbitrary constants oi and us. Using the replica method under sufficiently large NA. we have
E = Σ [MAAs- u.)² 28 ( - u:)gA, +
TRAA 1²
(EA + +00)
d'. oi.
+
where EA denotes generalization error E1 on single task A.
For example, we can see that EA is special case of E with d; = ny and u = A, and that EA- reduced Φ; - " / 10/1(11000 77B) and Wg after certain calculation.
Spectral Bias. The generalization errors obtained in Theorem I are given by the summation over spectral modes like the single-task case. The EB.; corresponds to the i-th mode of generalization error (8) on single task B. The study the single task (Bordelon et al., 2020) revealed that as the sample size increases, the modes of large eigenvalues decreases first. This is known as spectral bias and clarifies the inductive bias of the NTK regression. Put the eigenvalues in descending order, i.e. A 2 A++1- When D is sufficiently large and NB = oD' (a > 0, l E N), the error of each mode asymptotically given by EB,-
(i < 1) and nf (i> We see that the spectral bias also holds in EA-B because it is weighted sum over EB,, In contrast, Eback includes a constant term 2(1 p)nî- This constant term causes catastrophic forgetting,
show later.
4.1 TRANSITION FROM NEGATIVE TO POSITIVE TRANSFER
We now look at more detailed behaviors of generalization errors obtained in Theorem 1. We first discuss the role of the target similarity P for improving generalization. Figure 1(a) shows the comparison of the generalization between single-task training and sequential training. Solid lines show theory, and markers show experimental results of trained neural networks in the NTK regime. We trained the model (I) with ReLU activation, L = 3, and M, 4, 000 by using the gradient descent over 50 trials. More detailed settings of our experiments are summarized
Section E. Because set NA NB= 100, have EA- Eg. The point is that both Ea-B(p) (p) and Ex (p) are lower than EA(= EB) for large p. This means that the sequential training degrades generalization the targets are dissimilar, that is, negative transfer. In particular, Ebookk (p) rapidly deteriorates for the dissimilarity of targets. Note that both E and Elack i are linear functions of p. Figure (a) indicates that the latter has a large slope. We can gain quantitative insight into the critical value of p for the negative transfer as follows.
Knowledge transfer. The following asymptotic equation gives us the critical value for FAA-B:
EA- 2(1 p) for NA > NB.
(15)
5

---

Published as a conference paper at ICLR 2022
(a) 10"
(b)
24
A-B
10
A/B
1.2
10'3
48 0.0
0.6
10
0.4
02
10
10
10"1 10P
10'
10
Figure 1: (a) Transitions from positive to negative transfer caused by target similarity We set NA NB. (b) Learning curves show negative transfer (FA-B/ (/3-B/EB) EB) highly non-linear way depending on unbalanced sample sizes. We changed NA and set NB = 10
A straightforward algebra leads to this (Section A.3). In the context of transfer learning, it is reasonable that the target domain has a limited sample size compared the first task. For p > 1/2, we have FA--B Ep, that is, previous task A contributes to improving the generalization performance on subsequent task (i.e., positive transfer). For / < 1/2, however, negative transfer appears.
The following sufficient condition for the negative transfer is also noteworthy. By evaluating EA-BB EB. we can prove that for any N A and NB, the negative transfer always appears for
p" == VTa/(1+ V7A).
(16)
This is just a sufficient condition and may be loose. For example, we asymptotically have the critical target similarity p= 1/2 p² for NA > NB. Nevertheless, this sufficient condition attractive in the sense that it clarifies the unavoidable negative transfer for the small target similarity.
Backward transfer. The Bback (0) includes the constant term E, TH independent of sample sizes. Note that Q4 and 9 decrease to zero for large sample sizes (Bordelon et al., 2020), and we have Eback (p) Σ; n (1 p). In contrast, ΣΛ converges zero for large NA. Therefore, the intersection between Eback (p) and FA reach p = as NA and NB increase. This means that when we have a sufficient number of training samples, negative backward transfer (E (p) > EA) occurs even for very similar targets. Figure I(a) confirms this result, and Figure 6 in Section A.6 shows more detailed learning curves backward transfer. Catastrophic forgetting seems literally "catastrophic" in the sense that the backward transfer rapidly deteriorates by the subtle target dissimilarity.
4.2 SELF-KNOWLEDOK TRANSFER
We have shown that target similarity is a key factor for knowledge transfer. We reveal that the sample size is another key factor: To clarify the role of sample sizes, we focus the same target function (p = 1) in the following analysis. We refer to the knowledge transfer this case as self- knfoleeeee tansser to emphasize the network learning the same function by the same head. As is the same in < 1, the knowledge obtained in the previous task is transferred as the network's 's initialization for the subsequent training and determines the eventual performance.
Positive transfer by equal sample sizes. We find that positive transfer is guaranteed under equal sample sizes, that is, NA NB. To characterize the advantage of the sequential training, we compare it with model average: (f^ + f2)/2,, where fA (f') means the model obtained by a single-task training on A (B). Note that since the model linear function of the parameter in the NTK regime, this average is equivalent to that of the trained parameters: (0. + 0g) )/2. After straightforward calculation Section D. the generalization error of the model average given by
Eave (1 2n/2) En-
(17)
Sequential training and model average include information of both tasks A and B; thus, it is interesting to compare it with EA-AB- We find Proposition 3. For p = 1 and any NA = NB.
FA--((1) (1) Frave FA En-
(18)

---

Published as a conference paper at ICLR 2022
The derivation is given in Section D. This proposition clarifies the superiority of sequential training over single-task training and even the average. The first task contributes to the improvement on the second task; thus, we have positive transfer.
Negative transfer by unbalanced sample sizes. While equal sample size leads to positive transfer, the following unbalanced sample sizes cause the negative transfer of self- seff-knowledge:
ΣA-B (1) EB 1/(1-7A) for NB >> NA.
(19)
The derivation is based on Jensen's 's inequality (Section A.3). While EA-B and EB asymptotically decrease to zero for the large NB, their ratio remains finite. Because 0 >A < 1, F1- Eg- indicates that the small sample size of task A leads to a bad initialization of subsequent training and makes the training on task B hard to find better solution.
Figure 1(b) summarizes the learning curves which depend on sample sizes in a highly non-linear way. Solid lines show theory, and markers show the results of NTK regression over 100 trials. The figure shows excellent agreement between theory and experiments. Although we have complicated transitions from positive negative transfer, our theoretical analyses capture the basic characteristics of the learning curves. For self-knowlede ledge transfer (p = 1), we can achieve positive transfer at NA/NB 1, as shown in Proposition 3, and for large NA/NB as shown in (15). In contrast, negative transfer appears for small N W/NNB, as shown in (19). For p < 1, large N NA/NB produces positive transfer for 1/2, as shown in (15).
If we set a relatively large > 0, the learning curve may become much more complicated due to multiple descent. Figure 5 in Section A.5 confirms the case in which multiple descent appears in EA-BB The curve shape is generally characterized by the interaction among target similarity, self-knowledge transfer depending on sample size, and multiple descent caused by the noise.
4.3 SELF-KNOWLEDGE FORGETTING
We have shown in Section 4. that backward transfer is likely to cause catastrophic forgetting for p < 1. We show that even for / = 1, sequential training causes forgetting. That is, the training on task B degrades generalization even though both tasks A and B learn the same target.
10"
Error
We have A tion 3 tells us that EA-B EA- Therefore, forgetting appears for equal sample sizes. In contrast, we have
(1) = EA-B (1) by definition, and Proposi-
10
1077
F
El-B(L)/Ea 1/(1 7B) for NA NA>ND NB.
(20)
0% fixco)
One can obtain this asymptotic equation in the same manner as (19) since (A) is a symmetric function in terms of indices A and B. Combining (20) with (15), we have ΣA < E (1) < Eg. Sequential training is better than using only task B, but training only the first task is the best. One can say that the model forgets the target despite learning the same one. We call this self-knowledge forgetting. In the context of continual learning, many studies have investigated the catastrophic forgetting caused by different heads (i.e.. in the situation of incremental learning). Our results suggest that even the sequential training on the same task and the same head shows such forgetting. Intuitively speaking, the self-knowledge forgetting caused by the limited sample size of task Note that we have E1+ Σ, EB.: - 7A). The generalization eттor of single-tast task training on task B (En.s) takes large value for a small NB and this causes the deterioration of E as well. Figure 2 confirms the eeof-knoweedge forgetting in NTK regression. We set N NB as the red line and NB 100 as the yellow line. The dashed line shows the point where forgetting appears.
10"
10¹
10a
10a
10*
Figure 2: Self-knowledge forget- ting: unbalanced sample sizes de- grade generalization.
LEARNING CURVES OF MANY TASKS
We can generalize the sequential training between two tasks to that of more tasks. For simplicity, let us focus on the self- knfwledwwede transfer (p = 1) and equal sample sizes. Applying Lemma 2 recursively from task to task, evaluate generalization defined by F,, (J ddr(()) f=()))2)
7

---

Published as a conference paper at ICLR 2022
Theorem 4. Assume that (i) (Xnx Vn.) (n I,...,
(......K K) are given by the same distribution Xn P(X)
and target Yn = E, 10,30(X, (xn) ) + En- (ii) sample sizes are equal: N₂ = -N. For n = 1,... K,
- Ent1 (1- v 2 - Ra+10²,
1
T Q d diag (4²) (1-7)x2
N
(21)
where 9; = 4K//(1 TT, 9i = n and diag(a") denotes a diagonal matrix whose entries are qf- The noise term Ru is positive constant. In the noise-less case far = 0), the learning curve shows monotonic decrease: Ex+1 I En. afll eigenvalues are positive, we have
En+1 < E,
= 1,
(22)
See Section B for details of derivation. The learning curve (i.e., generalization error to the number of tasks) monotonically decreases for the noise-less case. the monotonic decrease comes from Amaz( (2) < 1. This result means that the self-knowledge is transferred and accumulated from task to task and contributes in improving generalization. also ensures that no self-knowledge forgetting appears. We can also show that Rt converges to positive constant term for a large and the contribution of noise remains as a constant.
KRR-like expression. The main purpose of this work was address the generalization of the continually trained model fn. As side remark, we show another expression of fn:
θ (X,)
1/1
+ (X2,
(X2)
O (1' 0 (x),
(23)
3/1
θ @X X1)
+ (X,))
This easily comes from comparing the update (4) with a formula for the inversion of triangular block matrices (Section C). One can see this expression as an approximation of KRR, where the upper triangular block of NTK is set zero. Usual modification of KRR is diagonal, e.g., L2 regularization and block approximation, and it seems that the generalization eπror of this type of model has never been explored. Our result revealed that this model provides non-trivial sample size dependencies such as self-knowledge transfer and forgetting.
5.1 EXPERIMENTS
Although Theorem 4 guarantees the monotonic learning curve for equal sizes, unbalanced sample sizes should cause non-monotonic learning curve. We empirically confirmed this below.
Synthetic data. First, we empirically confirm F,, on synthetic data (10). Figure 3(a1) confirms the claim of Theorem 4 that the generalization error monotonically decreases for = 0 as the task number increases. Dashed lines are theoretical values calculated using the theorem, and points with error bars were numerically obtained by fn (4) over 100 trials. For O, the decrease was suppressed. We set 6² = {0, 10-3 10-4, 10-3} and N; 100. Figure 3(a2) shows self-Knowledge forgetting. When the first task has a large sample size, the generalization error by the second task can increase for small subsequent sample sizes N₁. For smaller N,, there was a tendency for the error to keep increasing and taking higher errors than that of the first task during several tasks. In practice, one may face a situation where the model is initialized by the first task training on many samples and then trained in a continual learning manner under memory constraint. The figure suggests that if the number of subsequent tasks is limited, we need only the training on the first task. If we have a sufficiently large number of tasks, generalization eventually improves.
MLP on MNIST/ / ResNet-18 on CIFAR-10. We mainly focus on the theoretical analysis in the NTK regime, but it will be interesting to investigate whether our results also hold in more practical settings of deep learning. We trained MLP (fully-connected neural networks with 4 hidden layers) and ResNet- 18 with stochastic gradient descent (SGD) and cross-entropy loss. We set the number of epochs sufficient for the training error to converge to zero for each task. We confirmed that they show qualitatively similar results as in the NTK regime. We randomly divided the dataset into tasks without overlap of training samples. Figures 3(b1,cl) show the monotonic decrease for equal sample size and that the noise suppressed the decrease. We set N, = 500 and generated the noise by the label corruption with corruption probability {0, {0.0.2 0.2, ..., 0.8) (Zhang et al., 2017). The vertical axis means the error, i.e., 1 - (Test accuracy [%]]/100. Figures 3b2,c2) show that unbalanced sample sizes caused the mo-monottoicc learning curve, similar to NTK regression.
8

---

Published as a conference paper at ICLR 2022
NTK
MLP
R8
(a1) 10
(b1)
(c1)
06
Error
WB. Potce
10
(a2)
(b2) J.M5
(c2)
2000
0.04
0.4
2.055
Error
0.28
0.
2023
0.2
0.2
10 04
12'
10'
Task
Task
Task
Figure 3: (al)-(c1) Learning curves for equal sample sizes. For noise-less case, they monotonically decreased (orange lines). For noisy case, decrease was suppressed (grey lines; we plotted learning curves for several 0² and those with larger test errors correspond to larger o²). We trained MLP on MNIST and ResNet-18 18 on CIFAR-10 10. (a2) Learning curves for unbalanced sample sizes were non-monotonic (N1 4, 000 for NTK regression, Nı 10a for SGD training of MLP and ResNet-18), Numbers in the legend mean N,, (n 2 2).
DISCUSSION
We provided novel quantitative insight into knowledge transfer and forgetting in the sequential training of neural networks. Even in the NTK regime, where the model is simple and linearized, learning curves show rich and non-monotonic behaviors depending on both target similarity and sample size. In particular, learning on the same target shows successful self- self-knowledge transfer or undesirable forgetting depending on the balance of sample sizes. These results indicate that the performance of the sequentially trained model is more complicated than we thought, but we can still find some universal laws behind it.
There are other research directions to be explored. While we focused on reporting novel phenomena on transfer and forgetting, it is also important to develop algorithms to achieve better performance. To mitigate catastrophic forgetting, previous studies proposed several strategies such as regularization, parameter isolation, and experience replay (Mirzadeh et al., 2020). Evaluating such strategies with theoretical backing would be helpful for further development of continual learning. For example, orthogonal projection methods modify gradient directions (Doan et al., 2021), and replay methods allow the reuse of the previous samples. We conjecture that these could be analyzed by extending our calculations in a relatively straightforward way. It would also be interesting to investigate richer but complicated situations required in practice, such as streaming of non-i.i.d. data and distribution shift (Aljundi et al., 2019). The current work and other theories in continual learning or transfer learning basically assume the same input distribution between different tasks (Lee et al., 2021: Tripuraneni et al., 2020). Extending these to the case with an input distribution shift will be essential for some applications including domain incremental learning. Our analysis may also be extended to topics different from sequential training. For example, self-distillation uses trained model's outputs for the subsequent training and plays an interesting role of regularization (Mobahi et al., 2020).
While our study provides universal results, which do not depend on specific eigenvalue spectra or architectures, it is interesting to investigate individual cases. Studies on NTK eigenvalues have made remarkable progress, covering shallow and deep ReLU neural networks (Geifman et al., 2020; Chen & Xu, 2020), skip connections (Belfer et al., 2021), 1), and CNNs (Favero et al., 2021). We expect that our analysis and findings will serve as foundation for further understanding and development theory and experiments of sequential training.
9

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
The main contributions of this work are theoretical claims, and we clearly explained their assumptions and settings in the main text. Complete proofs of the claims are given in the Supplementary Materials. For experimental results of training deep neural networks, we used only already- known models and algorithms implemented in PyTorch. All of the detailed settings, including learning procedures and hyperparameters, are clearly explained in Section E.
ACKNOWLEDGMENTS
This work was funded by JST ACT-X Grant Number JPMJAX190A and JSPS KAKENHI Grant Number 19K20366.
REFERENCES
Alia Abbaras, Benjamin Aubin, Florent Krzakala, and Lenka Zdeborová. Rademacher complexity and spin glasses: A link between the replica and statistical theories learning. In Mathematical and Scientific Machine Learning (MSML), pp. 27-54. PMLR, 2020.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11244 1263, 2019.
Yuval Belfer, Amnon Geifman, Meirav Galun, and Ronen Basri. Spectral analysis of the neural tangent kernel for deep residual networks. arXiv preprint ariiv:2040000 2021.
Mehdi Abbana Bennani, Thang Doan, and Masashi Sugiyama. Generalisation guarantees for continual learning with orthogonal gradient descent. arXiv preprint arXiv:2006. 11942, 2020.
Blake Bordelon, Abdulkadir Canatar, and Cengiz Pehlevan. Spectrum dependent learning curves in kernel regression and wide neural networks. International Conference on Machine Learning (ICML), pp. 1024-1034. PMLR, 2020.
Abdulkadir Canatar, Blake Bordelon, and Cengiz Pehlevan. Spectral bias and task-model align- ment explain generalization in kernel regression and infinitely wide neural networks. Nature communications, 12(1):1-12, 2021.
Lin Chen and Sheng Xu. Deep neural tangent kernel and Laplace kernel have the same RKHS. In International Conference on Learning Representations (ICLR), 2020.
Rainer Dietrich, Manfred Opper, and Haim Sompolinsky. Statistical mechanics of support vector networks. Physical review letters, 82(14):2975, 1999.
Thang Doan, Mehdi Abbana Bennani, Bogdan Mazoure, Guillaume Rabusseau, and Pierre Alquier. A theoretical analysis of catastrophic forgetting through the NTK overlap matrix. In International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 1072-1080. PMLR, 2021.
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.09733, 2018
Alessandro Favero, Francesco Cagnetta, and Matthieu Wyart. Locality defeats the curse of dimen- sionality in convolutional leacher-student scenarios. arXiv preprint arXiv:2106.08619, 2021.
Amnon Geifman, Abhay Yadav, Yoni Kasten, Meirav Galun, David Jacobs, and Ronen Basri. On the similarity between the Laplace and neural tangent kemels. In Advances in neural information processing systems (NeurIPS), pp. 1451-1461, 2020.
Raia Hadsell, Dushyant Rao, Andrei A Rusu, and Razvan Pascanu. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences, 2020.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. Advances in neural information processing systems (NeurIPS), pp. 8871-8580, 2018.
10

---

Published as a conference paper at ICLR 2022
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 (13):3521-3526, 2017.
Grabska-Barwinska Barwinska, et al. Overcoming
Jaehoon Lee, Lechao Xiao, Samuel S Schoenholz, Yasaman Bahri, Jascha Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. In Advances in neural information processing systems (NeurIPS), pp. 8772-8583, 2019.
and Jeffrey
Sebastian Lee, Sebastian Goldt, and Andrew Saxe. Continual learning in the teacher-student setup: Impact of task similarity. International Conference on Machine Learning (ICML), pp. 6109- 6119. PMLR. 2021.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems (NIPS), pp. 6467-6476, 2017.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Scyed Iman Mirzadeh, Mchrdad Farajtabar, Dilan Gorur, Razvan Pascanu, and Hassan Ghasemzadeh. Linear mode connectivity in multitask and continual learning. International Conference on Learning Representations (ICLR), 2020.
Hossein Mobahi, Mehrdad Farajtahar, and Peter I. Bartlett. Self-distillation amplifies regularization in Hilbert space. In Advances in Neural Information Processing Systems (NeurIPS), pp. 3351-3361, 2020.
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. In International Conference on Learning Representations (ICLR), 2020.
Anastasia Pentina and Christoph Lampert. A PAC-Bayesian bound for lifelong learning. In International Conference Machine Learning (ICML), pp. 991-999. PMLR, 2014.
Vinay Venkatesh Ramasesh, Ethan Dyer, and Maithra Raghu, Anatomy of catastrophic forgetting: Hid- den representations and task semantics. In International Conference on Learning Representations (ICLR), 2020.
Stefano Spigler, Mario Geiger, and Matthieu Wyart. Asymptotic learning curves of kernel methods: empirical data versus teacher-student paradigm. Journal of Statistical Mechanics: Theory and Experiment, 122001, 2020.
Nilesh Tripuraneni, Michael Jordan, and Chi Jin. On the theory of transfer learning: The importance of task diversity. In Advances in Neural Information Processing Systems (NeurIPS), pp. 7852-7862, 2020.
Greg Yang. Scaling limits of wide neural networks with weight sharing: Gaussian process behavior, gradient independence, and neural tangent kernel derivation. arXiv preprint 142446,90,.04760, 2019.
Greg Yang and Etai Littwin. Tensor programs IIb: Architectural universality of neural tangent kernel training dynamics. In International Conference on Machine Learning (ICML), pp. 11762-11772. PMLR, 2021.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understand- ing deep learning requires rethinking generalization. In International Conference on Learning Representations (ICLR), 2017.
11

---

Published as a conference paper at ICLR 2022
Supplementary Materials
A SEQUENTIAL TRAINING BETWEEN TWO TASKS
A.1 NOTATIONS
The kernel ridge(-less) regression given by
N Σ ==1
(f(x") 1 (f, fr y²1 + 1
f = arg min
(S.1)
SEN 2A
where N is the sample size and H. is the reproducing kernel Hilbert space (RKHS) induced by the neural tangent kernel O We assume that the input samples xL are generated from a probabilistic dis- tribution p(z) and that the NTK has finite trace, i.e., I drp(z))(Iz x) < oc. Mercer's decomposition of θ is expressed by
J dd'p(z') (x, (z') = (2)
(i 0, 1, 0, 00).
(S.2)
In other words, we have ((x',z) = Ei (z)( Note that function belonging to RKHS given by
f(x) = (x)
(S.3)
1=0
with 1/ = Ei af 0c. Then, using the orthonormal bases { the solution of the regression is given by
(* Ewfth(t), w' = -(VV + Vy
(S.4)
where (x) (x). Each column of v is given by 6(x*) (u L..,,, N). We focus on the
NTK regime and take the ridge-less limit (A + 0).
The sequentially trained model (4) between tasks and B is written as
fa-B(*) fa(x) (cx (yB fa(BB))
(S.5)
fa(x) (lx (XA)) VA.
(S.6)
The model fA is trained on single task A and represented by
fa(x) wAA (xx
(S.7)
with
W lim argmin wa Ha(wa), 2
(S.8)
1 NA
HA(WA)
(wTv(=²)= -2))² =
(S.9)
2Λ
Eq. (S.9) is the objective function equivalent to (S.1 ) because (ff., fA)H = || Similarly, we can represent fA-B by the series expansion with the bases of RKHS. Note that the right-hand side of (S.5) is equivalent to the kernel ridge-less regression on input samples X and labels yB (B). We have
fA-B(I) (wi wTn (xx))
(S.10)
with
w 1/3 lim argmin,,, H(WB, wx),
(S.11)
0
NB
HWB, w
(wbv(*))
(1/ w (())))) + wBBB
(S.12)
12

---

Published as a conference paper at ICLR 2022
A.2 PROOF OF THEOREM 1.
A.2.1 KNOWLEDGE TRANSFER EA-AR
First, we take the average over training samples of task B conditioned by task A, that is,
EA-BIA == ( dzp(=) - (wi + w {x(x))3 +
(S.13)
= ( drp(z)((wn - (wB (w
=
wi)) (xx))
(S.14)
DB
where the set task B's training samples is denoted by Dg. We have
EA-B (FA-BBA)DA-
(S.15)
Note that the objective function H (wB, wx) transformed to
No
H(WB, w = 2A ==1
1
Σ
(wB-
- (ØB wi)) ((rl))) (
+ 11
(S.16)
that the average task B conditioned by task A is equivalent to single-task training with the target function (w w (b))). Therefore, by using (8), we immediately obtain
Comparing (S.14) and (S.16);
EA-BIA
Σ
( - -'A,)²
KB
BB
0²
(S.17)
1- YB
RB + NBm:
BB
=: Σ
( w +
BB a².
(S.18)
7B
Next, we take the average of (S.18) over task A. Only / depends on the task and determined by the single-task training (S.8) This corresponds to Lemma with u we and di n/ TB). We obtain
EA+B-
Σ
(WA. @B,))² 2D AWA, WB..)AA
WA,i +
N:NA
Σ
A,
2 CA.1
TB
1 7A
7
02
1
NA
The
B,
+
BB
(S.19)
1-
TBB
Although this is general result that holds for any W and wp it is a bit complicated and seems not easy to give an intuitive explanation. Let us take the average over
1 p p). P
[WA, WB.l
NO m
1
(S.20)
The generalization error is then simplified
EA-B(P) = (EA+B)au
(S.21)
22.6
Σ
2(1 (1-9A)) + 1- -AA
EB,
(
+ 0²
NA Σ n1 92 +
BB
(S.22)
(1 11-7A BB 13
YB
where used (w2.)) (wwb,)) ?/i and (WA,i WB..) p.
13

---

Published as a conference paper at ICLR 2022
A.2.2 BACKWARD TRANSFER
Backward transfer is measured by the prediction on the previous task A:
( dxp(a))(Aa(t)) -Sa-)((x))² = (f drp(a)((wB (wA w2)) (xx)))²) =2a)]((xx)333
E
(S.23)
(S.24)
First, we take the average over task B,
( dep(=)((w" - (wA T {x))))
Eback A-+BIA
(S.25)
Dn
This corresponds to Lemma 2 with the replacement Φ; +- 77s and u WA w Recall that the objective function NTK regression was given by (S.16). The target W Lemma is replaced as WA* uh w We then have
Eback BIA
Σ
(BB,+ WA.)² - 2( - wA,)(BB - WA.)gB.
MB.B
(WB,i w/,))² + - 7B
nH m - 3 + 0
2
99,, 171
KB
1=0
(S.26)
TB Σ n(BB. w + Σ
B
WA, (WB,
1 1 1-7B
(WB,
1 (..i))
7B
+ 7B o²
(S.27)
BB
Next, we take the average over DA. Since the first and third terms of (S.27) ) are independent of DA we need evaluate only the second term. The second term corresponds Lemma with Φ; /1 7n) and u, WB.; - (1 (0///////e (7))//,,(wwn)) A..). We obtain
bback
= =(E
(EXBBAA) DA
back
BB Σ - ΣΣ - -B,,) ({ 2B)))
20 WBB.) -BB))9AA9BB
1hi +
1 NA
n 9 + +
174
1 7A x2
1 BB
+ 2
NA
Σ
M/
2
4A.i
AB.
(S.28)
1 7A
7B 2
7B
14

---

Published as a conference paper at ICLR 2022
Finally, by taking the average over (S.20), ), we have
Eback A->B(p) (p)
{E Bback
A-
2
Σ BYBB () + -TB))²
(1 p)
1- TB
2 ni (qB,
+ 7BB)
2R. (1 p) +
1
1
173 2
1
YB
7/A
7B
+ 0
1
1
NA
BB, +
2
7BB
1-7A VA A
7B R²
1 7B
2(1 p) E
1+ 1+ -2)
BB (1 - 9A,i)
1
1 7B
7/
7B
Σ n. +
1 NA Σ
m TB.
7B
(S.29)
1 1- 7/ 1 111B RAA
1
YB
A.2.3
LEMMA 2
Lemma 2. Suppose training on single task A, the target which given by f E, WA,bh and denote the trained model (2) f E, w Ant Define the following cost function:
E Σ di uz)²
(S.30)
DA
for arbitrary constants di and us. Using the replica method under a sufficiently large NA, we have
E =
2
- A.
u,)4n,
2
N AA
2 A,
M) A + +z))
TAA si.
(S.31)
7A
Proof. The process of derivation is similar Canatar al. (2021), but have additional constants 6; and u, They cause several differences in the detailed form of equations.
Define
ZJ /
BN dw exp(=BHA(WA)) + E(wa))
(S.32)
We omit the index A in the following. We have where E(wA) (E, @r u,, D.
lim 8
(log Z(J
(S.33)
BN aJ
J=0
To evaluate (log Z). we use the replica method (a.ka. replica trick):
(log 2) = lim ((Z)) - 1)
(S.34)
--+0
The point of the replica method is that we first calculate n for n E N then take the limit of n to zero by treating it real number. In addition, calculate the average (Zm) under replica symmetric ansätz and Gaussian approximation by following the calculation procedure of the previous works (Dietrich et al., 1999; Bordelon al., 2020; Canatar al., 2021).
15

---

Published as a conference paper at ICLR 2022
We have
( / dWn, exp
(-2
3 Σ
|0011 1u01|2+
BJN
(wo u)² (w0 a u)
(Q) Na
(am
2
(S.35)
Q exp
3
(ww) wA)
( (b((t)t)))))))))))))) e
(S.36)
2\
where we define dWn, = II dw and a diagonal matrix whose diagonal entries given by 9;. We take the shift of w0 + w² + Then,
/ dW, exp( - n
|10012 + nBJN
1 D
(w 1w-w) D(w T(w u)
21()
B
BJN
eexp
112
wa Ow - Bk wkTTT)) (0) (a)
(S.37)
2
2
Q exp(
(waT (x*)) =2)
(S.38)
2A
k き - JN u)
(S.39)
First, let us calculate (Q)(2 (2m_ee3 This term is exactly the same as appeared in the previous works (Bordelon et al., 2020; Canatar al., 2021). To describe notations, we overview their derivation. Define y"h = wa rb (x) which are called order parameters. We approximate the probability distribution g" by a multivariate Gaussian:
1 ( (8" - (0 (d' (2)) 12))
P({x*})
(2 del(C) exp
( a.
(S.40)
with
11 = (19) = (x)))) + (6)= = vo
(S.41)
Cab :=b:= (9 (2,e) - (b(x))(x)) ) + (8"e²) wa Aw + Σab
(S.42)
where Lab odb., (....), denotes the average over p(x) and A is diagonal matrix whose entries are T/. We have /noug in (S.41 since no corresponds to the constant shift Φu (z( = 1. Training samples are i.i.d. and we omitted the index μ.. We then have
Gdab.
(Q) exp (-2 log det (
3 (T+
( (T+ x x
x
C
11
(S.43)
2x
where / denotes the identity matrix. Define conjugate variables {ât, Caaby by the following identity:
1 c
Il a2b
du dis aC ddmb
X exp
-N Σ f -n--wa w w%,oV 7o) - N N cbab (Cab wa Au Dab)
(S.44)
a2b
where C denotes an uninteresting constant and we took the conjugate variables on imaginary axes.
Next, we perform the integral over W.. Eq. (S.37) ) becomes
(
=c7(J) / dW, II ah exp( exp(-N() -NO
10 1ta +
Era Caa((Cub _ -xah)) exp[-NG Gs]
a2b
b2b
(S.45)
16

---

Published as a conference paper at ICLR 2022
where ds2" du dif dddddi"CCcdhee dCua and define
1 log det 1+
(0)
B u
(1+ (1 [
G = 2
2x 22x"
μ.
(S.46)
3
BJN
exp(-Gs) expl
||w011
wa &w
.kk 3kTww) w²))
x exp
N Σ
w + N Σ
Erab wa Awo
(S.47)
a2b
We can represent f dW,, exp( -Gs)
/ dW,, exp(-Gss Gs) / dr, exp (- Qi:; Bbg xi
(S.48)
where D; IR" denotes vector (w!.... ,..,, w.f ..... u and i is the index of kernel's eigenvalue mode
(i = O, ...). We defined
nNN
Q = (1=11-1,00 D.JNN,
(Ö + diag (C)),
(S.49)
B
b; ki
(i 1),
(S.50)
N T00
bo k
B P.
(S.51)
where I, is identity matrix and 1, is n-dimensional vector whose all entries are 1. The Gs term includes á and e that are specific to our study. When e and u = w. it is reduced to previous works (Bordelon et al., 2020; Canatar et al., 2021). Taking the integral over [r;}, we have
exp f dW,, exp( GGs) = II 1=0
b,)// Qet
b 2
(S.52)
That is,
Gg log det Q
8
6g
Qi'b. b..
(S.53)
2 1=0
REPLICA SYMMETRY AND SADDLE-POINT METHOD
Next, we carry out the integral (S.47) by the saddle-point method. Assume the replica symmetry:
μ = n²,,, r= C0aa c = C j poa F == -0a, ? Õazb
(S.54)
(S.55)
The following three terms the same as in the previous works:
= 1 (r oo] (r-=)]
1 1+ A + 3 (r (c--)) )
3e
det
1+
c -1
(S.56)
8 10)
1
(
X 3 (r c) + n13c 11 11")
Bc
(S.57)
1 (r- (r
jT Crab( - sahh) (in+ f(r - 0
1 560 - -0?)
(S.58)
a2b
where 11 denotes a matrix whose all entries are 1. Regarding the leading term of order (n -> 0),
log det (++cc = n log
3
1 1 _ (r o)) + n x+ ( c)
3C
(S.59)
-1
211²
uT
1
11
1 + 2 (r-
(S.60)
17

---

Published as a conference paper at ICLR 2022
Furthermore. we have
n.N
N:
Q = (1 6,JN
(2F
-))
In
(S.61)
B
After straightforward algebra, the leading terms become
log det Q In 0:
(S.62)
9:3
1 1 In
(S.63)
gi
with
N
9i : - JNJN-
(2F c).
(S.64)
3
Substituting these leading terms into (S.45) ), we obtain
( C dd cix((=)((N()) with
3J
S(0) = 2 (m u) ②( u)
iup + ffrr - 0²) 2 c(c - (x2- +
1
log 3 (r ()-c) c+
m/:
+
(Ing:
+ Bb..)/00 /
(-2hooiVTo + NNnuu
(S.65)
2N
B
2go
1=0
where 0 denotes set of variables {r, e, ê, n. û.. Here, we take N > I and use the saddle-point method. We calculate saddle-point equations aS(A*) / = 0 and obtain
1
(S.66)
c
1
c+
12
(S.67)
2
(r c)2
r -C)
c+ x2
à
(S.68)
(r c)2
NnØA
Koo
Tiu
(S.69)
90,8
9o
TI Nè
No
3k
2koo
Nunji² 33 + 0²,
(S.70)
9
3
N:N
1
NoNN
3k2)
2koou + Nrnua3) + o
(S.71)
B
B
3
96
==0
i=0
From (S.67) and (S.68) )
(S.72)
2F
From (S.70) ) and (S.71) ), we also have
3(r-c) c)
Σ 1
K.
(S.73)
9s
Substituting (S.64) ) and (S.72) ) into (S.73), ), We obtain the implicit function for x:
= Σ
' W(o.) == s01 J.JN) + n.N
(S.74)
1=0
18

---

Published as a conference paper at ICLR 2022
Using K and W, the saddle-poing equations become
By
á
(S.75)
K
f
1
c
C+1
B²
8
(S.76)
2
R²
32
è
(c + 1²),
(S.77)
x2
Vukion
14 Nuoo + ((o))
(S.78)
k
r c
c+
(S.79)
3
Substituting back these quantities into (S.65) ), we obtain
S(0")
BJ
2 (w u) Φ( T
u) +
In +
1
In
B
Σ
R2
2N
2N
9:
i=0
(=0
K
B 2x
+
0² + const.
(S.80)
2W (do) Nooo + W (0o)
Recall (D")D eex((-nNSS(0)) We have
(log -N5(()).
(S.81)
GENERALIZATION ERROR
We evaluate
2 8 (log Z)
2 à
E lim
= lim
S(B*)
(S.82)
8
3N OJ
IJ= B
a.
J=
Note that W, 9. x and depend on J. At the point of J = O.
(W
Nx DJ
(S.83)
àJ
0g.
Ōx
-(0:
+
n² 0J 1k: 1
(S.84)
JJ
0k N N
d
(S.85)
0J
(k + T/s Ny
0h.
No. uz).
(S.86)
aJ
Substituting (S.83 -S.86) into (S.82), ), we obtain
N/NN
(10: uz)² 200 uz)4 +
u o3
02)
Φi + Eo.
7
R2
1=0
(S.87)
where Eo is
Eo = now! ( (< + Wo)²
Nno + 2W,
Wo(Nno + Wo) )
1
N
Σ no
Nno + 2W,
No 10 un)
- Nx2
+ 21o
(S.88)
W + Wo)²
Wo + Wo)
with Wo = K + Nojo- When 7/u- = 0, we have Eu = 0. Note that the additional term &u is not specific to our case but also appeared in the previous work (Canatar et al., 2021). We have E (11 for a large no and &n is often negligibly small.
19

---

Published as a conference paper at ICLR 2022
A.3 EQUATIONS FOR UNDERSTANDING NEGATIVE TRANSFER
A.3.1 DERIVATION OF EQUATION 15
First, let us evaluate the term including 9A,i RA/(RA + Nam)). Note that RAA is monotonically decreasing function of NA because
A A
Σ
T (KA + Nam)?
Σ
of
< 0.
(S.89)
(KA + Nam.)
which comes from the implicit function theorem. Let us write KA at finite NA as (c).). We then have
Ka(c)
AAA S
(S.90)
NAw
for NA c. Next, we evaluate
Ea-B(p)= -2(1-)))n 2(1 p)
Σ EB, + Σ
11-YA
EB,
(S.91)
The second term is negligibly small for sufficiently large NA > because
(a(c)
Σ n RA(C)
Σ FA. S
KB.
(S.92)
NA
1-7B 7B
NA
Note that we have - 7 = K
Ei=o /// + n,N.
The third term
Σ 2. EB,i
EA
7A A 1--A NA
(S.93)
I-7B
YB
YB
where used QB S 1. It decreases to zero NA increases. Thus, have EA+B(() 2(1 - EB
A.3.2 DERIVATION OF EQUATION 19
(1-
Ex 2 BB
(S.94)
EB
1- 7A
Σ. E.9..T
RA
(S.95)
1-74 Ta
KA E, G. B. 2 N
where the second line comes from Jensen's inequality. Using (S.90) ), we have
(B(c))
Σ
Σ n
(S.96)
B
for NB 2 c. Since we assumed the finite trance of NTK (i.e., Σ, < 0o), the left-hand side of (S.96 is finite and converges to zero for sufficiently large Ng. Therefore, we have
(1+
1
(S.97)
EB
1-7A
In contrast, FA-B/FBB 11/(1 714) since AA S Thus, the ratio sandwiched by 1/(1 - 7A).
A.3.3 SUFFICIENT CONDITION FOR THE NEGATIVE TRANSFER
Let denote the i-th mode of FA-BB EA- and that of Eg as EB,-- From EA-BBA EB,
we have
of 1-7 11 1=0,
f(g:) = 2(1 - (1 4.) +
(S.98)
where f'(q) is quadratic function and takes the minimum at = p)(1 - >). The condition f(q) holds if and only if
f(q") = - -11-))) - 2xp + 7 >
(S.99)
Solving this, find p < /5/(1 + v7)-
20

---

Published as a conference paper at ICLR 2022
A.4 PROOF OF PROPOSITION 3.
For NA Nn, we have yA,i - q8,i qi. Then,
Ex-B(1) = (1 (1-7) 7)² Σ No
(S.100)
and the generalization error of single-task training is given by
== Σ of ms.
(S.101)
Model average is obtained in Section D and we have Lave = (1- < E because 0 < y < 1. Thus, we only need evaluate the relation between Eave and EA-BB
2 Eave - -1 2
= (1-
4
1
1
Σ
(S.102)
(1-7)²
(1
2
1
k²
Σ
a;
2
(S.103)
2
(1 7)2
(1 7
K²
Σ N2 a2 Na.)
7 -
2
1
(S.104)
=:F
where we defined
7 K + T N
0g
(S.105)
and used qi -1a;; N and = N N oZ. We can provide a lower bound of F by using the following cubic function:
G(a.) = N a2 Na:).
We have 0 < a; < 1/N by definition. Let us consider lower bound of G' by using its tangent line at a, I/N (0 sostsll 1), that is, Nt(4 - 3()a; - 26 (1 - t). Define
Hass) = G(as) - (Ni(4- - 3/)a; 21 - (1)
112). we need S 1/2 to guarantee (a.) > 0 for all 0 S < 1/N. Here, note that // is cubic function of a, and has two fixed points a; = I/N and (4 33)((3 We can see that for S 1/2, H has the local minimum at a, t/N and
have ((0) 21 - 1) 2 Since H(1/N) = -2(t -2(t 2)
H(INN) (2 - (12 31) - 1))
=0
(S.106)
Therefore, we have (a,)) 2 for 0 S a; S I/N when the fixed constant satisfies t S 1/2. Thus, we have the lower bound of G:
G(a.) 2 Nt(4 3t)a; - 22 - f).
(S.107)
Using this lower bound, we have
F Σ a(G(a) 2 (3 (3-
1
2 7044 - 3/) - 21 - -1) - 23 - -7),
(S.108)
where we used 7 N, o3 and Σ, 0; 1. By setting = 1// obtain
2 (4 31-31/2) - 72 (1 7/2) 2 2 7)
1
F
0.
(S.109)
Therefore, have Eave EA-BB
A.5 MULTIPLE DESCENT
21

---

Published as a conference paper at ICLR 2022
x
x10
10
10
10a
1
B
B
2
102
102
2
102
10a
10
102
10
10
N
Figure 5: Theoretical values of (1) (Left) noise-less case (a = 0). (Right) noisy case
(a 10-9). We set D 100 and other settings the same in Figure 1. In noise-less generalization error monotonically decrease as N or NB increases. In contrast, when noise is added, shows multiple descents.
10
We overview the multiple descent of Ej investigated in Canatar et al. (2021). It appears for >> 0. Let us set NB aD (a > 0, 1 E N. D > 1), which is called the I-th learning stage. We have EB,; En,(a) is function of a, and becomes a one-peak curve depending the noise and the decay of kernel's eigenvalue spectrum. Because each learning stage has a one-peak curve, the whole learning curve shows multiple descents the sam- ple size increases. Roughly speaking, the appearance of the multiple descent is controlled by 7. The 7 determined by the kernel's s spectrum and sample size N. For example, previous studies showed that if we assume the (-th learning stage, 7 is non-monotonic function of a, the maximum of which is
(i 1), EB,(a) and ni > l).
10
10"3
10
10a
10a
N
Figure 4: Typical behaviors of
and 7.
1/(vA + Vt + 1)² for constant \ EE>r 7t/ / where vj is normalized eigenvalue of the kernel (Canatar et al., 2021). This tells us that y repeats the increase and decrease depending the increase of the learning stage as is shown in Figure 4. Roughly speaking, this non-monotonic change of leads to the multiple descent.
In sequential training, (B,(a) can similarly cause multiple descent as in single tasks because E a weighted summation over Ep,; Figure 5 an example in which noise causes multiple decreases and increases depending on the sample sizes. We set p and obtained the theoretical values by (S.22) ). While B (1) is symmetric function of indices A and B for 0, it is not for o > 0. Depending on NA and N B. the learning "surface" becomes much more complicated than the learning curve of single task.
A.6 ADDITIONAL EXPERIMENT BACKWARD TRANSFER
Figure shows the learning curves of backward transfer. Solid lines show theory, and markers show the mean and interquartile range of NTK regression over 100 trials. The figure shows excellent agreement between theory and experiments. As is shown in Section 4. subtle target dissimilarity
(p < 1) causes negative backward transfer (i.e., catastrophic forgetting). For a large NA. F < 1) approaches a non-zero constant while Σ approaches zero. Thus, we have Ebook > E even for the target similarity close to 1. When sample sizes unbalanced (N >> NB). the learning curve shows negative transfer even for p p= 1. This is the self knollenowedge forgetting revealed in (20). The ratio F ( EA takes constant larger than 1, that is, 1/(1 -)B).
22

---

Published as a conference paper at ICLR 2022
10
10
0,9
0.999
bbok /E
10
B
1.5
10a
10
0.5
10-2
10"1
10
10 N /N
10
10
10
10
Figure 6: Negative backward transfer easily appears depending on target similarity and sample sizes. We changed NA and set NB 102. We set D 20 and other experimental details are the same as in Figure 1(b).
B PROOF OF THEOREM 4.
B.1 LEARNING CURVE OF MANY TASKS
Eq. (4) written
Jn(x) = Σ Oc, (x)) - (
(S.110)
Each term of this summation is equivalent to the kernel ridge-less regression on input samples X and labels yk fx=1 Therefore, represent fn. by
k=1 fn (x)
w (x(x)
(S.111)
with the minimization of the objective function H:
" lim argmin H(w,, wi:(n- 11)s
(S.112)
N,
n-1
H(w,, wi:(n-11)
wh (Ix) - (yll Σ wx (x(l))
+
(S.113)
21
==1
N,
n-1
2
(wn (10 -
Σ
wx)) ( (b)")))))) - e/
+
(S.114)
21
1=1
The generalization error is
(daplt))((()) fmtt)))))))( ((xxt))ftt)))nn11))))
En+1 =
(S.115)
D
(f drp(z)((wi+z (ū Σ wi)) b(z))²
(S.116)
where D = (D, D,).
Since the data samples are independent of each other among different tasks, we can apply Lemma sequentially from Wx+1 to Wi. First, we take the average over the (n + 1)-th task, that is,
= ( dep())((uui+ (w
Ent1j1:n =
Σ wf)) Twt))<<<))>
(S.117)
k=1
Da+1
23

---

Published as a conference paper at ICLR 2022
This corresponds to Lemma 2 with Φ 17 and
WA 00 -Ek=1 wx- We have
n
Fm+1|1m
Σ
w )² + R1o
(S.118)
k=1
with
11,1 71 1,
7n+1
Rí
(S.119)
7n+1
7x+1
Here, Ku. Tn and fn,k are determined by Na. Next, we take the average over the n-th task:
Pn+1[1((n-1) (FnttIlim DaIEn
(S.120)
Σ $1,((1 (w; Σ
w
+
Rio
(S.121)
1=1
Dr
This corresponds to Lemma 2 with & = Φ1 and u = WA = w
Wk.
We obtain
n-1
1m+1[1((m-1)
Σ
Σ
wfr + R20
(S.122)
i=1
kml
with
N
22. 21,iinn +
Te Σ n
(S.123)
1in (1 - 7n)
Nn
Ra = Rı +
Σ Mi ni...
(S.124)
(1 7a)
Similarly, we can take the averages from the (n 1)-th task to the first task, and obtain
Ent1
Σ
@uti, + Rnt1o²
(S.125)
1=1
N m-
0m+1, 2 2
m +
11 +1,
Σ ny m. pa-m+
k² R m-m+1)
(S.126)
for 1,... n, and
N M
Rn
Σ
E Sh
(S.127)
k
m+
mn-m+1)
They general results for any Nn. By setting N,, N for all n, obtain simpler formulation. In vector representation, Φn+1 is explicitly written as
N
2
diag(a?) + (1-
99
g.
(S.128)
Finally, by taking the average over i. ~ N m:) which is the one mieddimenioonal case of (S.20 ), we have
Ent1
(1- "1-
- diag(g") + (1- )x²
99
q Rat10²
(S.129)
with
Ro+1
N
Σg ④
diag( ( +
N
77
m-1
ÿ +
?
(S.130)
x2
72
(1 - n
1
m
24

---

Published as a conference paper at ICLR 2022
B.2 MONOTONIC DECREASE OF En
For x² = O, we have
Ex+1 (1 7)2 Q"-²q
(S.131)
Note that Q is positive semi-definite matrix. If the maximum eigenvalue is no greater than En is monotonically non-increasing with n. We denote the infinite norm as || max; E, AAjl Because the infinite norm bounds the eigenvalues, we have
(2) S max
Qiyl
(S.132)
N Th
max
1 1
(S.133)
Tj
C=((m)
where G(n) is monotonically decreasing for 1j 2 0 and G(0) 1. Therefore, the largest eigenvalue A1 (Q) is upper-bounded by 1, and E, becomes monotonically decreasing. particular. if 7J for all i, we have (2) < and obtain En+1 < En.
For A1 (2) < 1, also easy to check that the series (S.130) in Rn+1 converges to a constant for large
C DERIVATION OF KRR-LIKE EXPRESSION
KRR-like expression comes from the inverse formula of a block triangular matrix. Let us write (23)
as
|1:(n-1)
fn(x') (0('' 1: (n 1)) O(a', Z)
(S.134)
Un
where Zn is block triangular matrix recursively defined by
Zn-1 0
2n 1: 1)) O(n)
(S.135)
Then, (S.134) is transformed
1 [ -0) 1: - 1))2,"
0 y1:(n-1)
(0 1: (n - 1)) 0(x', [[]
(S.136)
((n)
[ Yn
=(2z, 1: (n-
+ n)(n)) (mn (nn, (n 1)) 1811(nn-11)
(S.137)
fn-1(x') 0 m)@(n) (vn (X,) Thus, one can see that the KRR-like expression is equivalent our continually trained model.
(S.138)
D DERIVATION OF MODEL AVERAGE
For clarity, we 0, NA Na. and p (that is, WA wn). Generalization error of a model average is expressed by
( dxp(x)
(Fu(x)
fA(x) + FF(())
Eave
(S.139)
2
( dxp(x)
WA + WB
((x)
(S.140)
2
D
+
WB
WA + WB
Wp
A
Wp
(S.141)
2
2
25

---

Published as a conference paper at ICLR 2022
We evaluate Eave in similar way to EA-BB First, take the average over Dg. We define
A 4 (((wB - (2@B wA)) A(wB (2 - wA))) DB
142)
This average corresponds to Lemma 2 with replacement from the target task A to B, Φ + and t B WA. Then, we have
EavelA
Σ
(was @B,))² 2 WB,))B,;+ +
B
2
EB
BB
2
%/
KBB
1=0
(S.143)
Σ (WAN - (1 + (B.))BB)) + YB EB-
(S.144)
Next, we take the average over DA. The first term of (S.144) ) corresponds to Lemma with replacement ái 1) and (1 qn,;) n. Then, get
Love (BavelA)Ds
(S.145)
Σ (LL (1- qA;) - (1 (B.)) + 7A 1=0
+ BB EB. 4
EA
(S.146)
4
For NA = Na, we have 4Λ = qB, 74 .B, and Ea En- In addition, since we focused on WA A W. have
Eave
Σ
1/
BB,
2
+
BB En
(S.147)
2
1=0
1
BB
EB
(S.148)
E EXPERIMENTAL SETTINGS
E. DUAL REPRESENTATION
In dual representation, the targets (10) are given by
[1 p)
an. ~ p IP).
(S.149)
P
p'
= = EaasO(es,))) [b( Σ LB ((1),,))
(S.150)
for a sufficiently large P'. We sampled x' from the same input distribution p(z) and set P' = 10 in all experiments. is easy to check that targets (10) and (S.150) ) are asymptotically equiv- alent for large P'. Because (ar,, E, (x))))))) we have Σ; a. ((rg..)) E, Ej (d)))))) E, (x)) where w; sampled from (O,
E.2 SUMMARIES ON EXPERIMENTAL SETUP
Figure 1(a). We trained the deep neural network (1) with ReLUU, I. 3, and M; 4, 000 by the gradient descent. We set learning rate 0.5 and trained the network during 10,000 steps over 50 trials with different random seeds. The target is given by (S.150) with = D 10, and Gaussian input samples J; (0 1). We initialized the network with (o", o (2, 0), set N NB 100, and calculated the generalization error over 4, 000 samples. Each marker shows the mean and interquartile range over the trials. To calculate the theoretical curves, need eigenvalues 7) of the NTK. We numerically compute them by the Gauss- Gegeenbuee quadrature following the instruction of Bordelon et al. (2020). Its implementation is also given by Canatar al. (2021). Since the input
26

---

Published as a conference paper at ICLR 2022
samples defined hyper-sphere s and the NTK is dot product kernel (i.e., O .)) can be represented by (xx" x)), the NTK can be decomposed into Gegenbauer polynomials:
80
( Σ (.N().)).(2),
(S.151)
where {Qi} are the Gegenbauer polynomials and ((d, i) are constants depending d and Since the Gegenbauer polynomials are orthogonal polynomials, we have
7/: - : [ (z))
(S.152)
for a certain constant e and dr(2) = (1 (1-23 2d 32 The Gauss-Gegenbaur quadrature approxi- mates this integral by
Yi 2
wy ( (i()))
(S.153)
where wy are constant weights and 2, are the roots (+) The definitions of constants (N i). and wy) are given in Section 8 of Bordelon et al. (2020) and we set 1000 as is the same in this previous work. We computed 7/: 1, 1000) and substituted them to the analytical expressions.
Figure 1(b). This figure shows the results of NTK regression: (2) for the single task and (4) for sequential training. We set NB 000, D and other settings were the same as in Figure (a)).
Figure 2. This figure shows the results of NTK regression; we set D = 20, and other settings are the same as Figure ((a).
Figure 3. (NTK regression) This figure shows the results of NTK regression; We set D = 10 and show the means and error bars over 100 trials. Other settings were the same as Figure 1(a).
(MLP) We trained the deep neural network (1) with ReLU, L = 5, and Mj- = 512 by SGD with learning rate 0,001, momentum 0.9, and mini-batch size 32 over 10 trials with Pytorch seeds 1-10, We set the number of epochs to 150 and scaled the learning rate x x1/10 at the 100-th epoch, confirming that the training accuracy reached for each task.
(ResNet-18) We trained ResNet- 18 by SGD with learning rate 0.01, momentum 0.9, and
size 128 over 10 trials with Pytorch seeds 1-10. We set the number of epochs to 150 and scaled the learning rate x1/10 at the 100-th epoch, confirming that the training accuracy reached 1 for each task.
Note that the purpose of these experiments not to achieve performance comparable to state-of- the-art but to confirm self- enowleneeedge transfer and forgetting. Therefore, we did not apply any data augmentation regularization method such weight decay and dropout.
27


Reviews:
[{"review_id": "l0z4k051rLG", "reviewer": "Reviewer_spQ7", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Theoretical studies are crucial for understanding how deep neural networks work on the continual learning/sequential learning problem. NTK is one of the powerful theoretical tools for analyzing the behavior of neural networks. This paper extends the studies of NTK, specifically the Spectrum Dependent Learning Curves [1], on the continual learning setting via studying the generalization between the sequential tasks. The analyses reveal the relationship between the transfer/forgetting and the dataset for standard neural networks. However, there are still some drawbacks that may limit the significance of the paper:\n1. The paper focuses on studies of the behaviors of the standard networks (NTK) trained under standard strategies. Thus the results mainly reflect the influence of the data characteristics and show fewer insights related to the networks and the training strategies. Given the original results in [1], the results in the paper are obvious, further limiting the paper's significance. The results in the paper so far give limited guidance related to model designing, in my opinion. \n2. The studies are restricted to continual learning with explicit task boundaries. It is acceptable. But I suggested the authors claim the settings more explicitly at the beginning of the paper/method for clarity. \n\n[1] Bordelon, Blake, Abdulkadir Canatar, and Cengiz Pehlevan. \"Spectrum dependent learning curves in kernel regression and wide neural networks.\" In International Conference on Machine Learning, pp. 1024-1034. PMLR, 2020.", "summary_of_the_paper": "This paper studies the generalization/transfer behavior of the continual learning/sequential learning under the neural tangent kernel (NTK) regime. It gives the formulation of the generalization error between two tasks in forwarding and backward ways and analyzes the influences of target similarity and sample sizes. Then the self-knowledge transfer case with the same target function is studied, which shows the universality of the catastrophic forgetting and the influence of the sample size. The results are then generalized to many tasks. ", "summary_of_the_review": "Although the paper's significance is a little limited by the relationship with previous work such as [1] and the restricted scenario, the results (especially the self-knowledge transfer case) are interesting. It could be a starting point for more general and informative analyses of deep continual learning. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "gBZJEZuAYd", "reviewer": "Reviewer_aqmt", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper provides some theoretical insights on the forward and backward transfer in sequential training.  My main concern is that I have some doubts regarding its strong assumptions on the data distribution. As stated in the last paragraph on page 4, the input samples of task A and task B are i.i.d. and generated by the same distribution. This assumption cannot be satisfied in most cases of continual learning, especially for class incremental learning the data distribution of each task has significant differences.  In this sense, I think the claims of this work are more about sequential training on a large dataset rather than continual learning.  And the experiments are insufficient to verify the theoretical results in typical continual learning benchmark tasks. The experiments on CIFAR10 are basically sequential training on chunks of a dataset. I'm wondering how much of these theoretical results can be conveyed to typical continual learning tasks. As in most benchmarks, the sample size of each task is the same. Even when the target function is the same, such as in domain incremental learning, the forgetting would like to happen (e.g. permuted or rotated MNIST tasks). But in Sec.4.3, the authors claim no forgetting appears for equal sample sizes, which is not the case for most continual learning tasks in my experience. \n\nSome detailed issues:\n1. Fig.2 does not match the description as the blue line E_A is always larger in the figure. \n2. What's the error in Fig.3? The generalization error over the last task? \n3. Most experiments results are about forward transfer, I think it would be better to show backward transfer as well.\n", "summary_of_the_paper": "This paper provides a theoretical analysis for sequential training based on related NTK results.  It shows the similarity between the target functions is a key factor for forward and backward transfer, and when the target functions are the same the samples size of each task is a key factor to forward and backward transfer. It also shows even a slight difference between the target functions can cause catastrophic forgetting and the forgetting might still happen for the same target function when the sample size of a later task increases. ", "summary_of_the_review": "The theoretical results provided in this paper are kind of interesting but I think they are more about sequential training rather than continual learning. The experiments part is a bit weak to support the claims. \nAs I'm not familiar with the NTK related work, I didn't check through the proofs of those theorems. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "c6pGJsE4jmm", "reviewer": "Reviewer_2847", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nThe paper is well written and well structured. Each section is clear in what the authors try to explain demonstrate.\nNevertheless, the vocabulary used is a bit different from usual continual learning papers, which might be a bit misleading.\nA head in CL literature refers often to a group of output neurons while it seems that in this paper it refers only to one output neuron.\n\"Target function\" is also unusual to refer to the function specific to a group of targets. This terminology does not hurt but it should be explained earlier in the paper to avoid confusion. \n\nI acknowledge the use of a theoretical approach to continual learning to better define and understand the problem we are solving.\nOn the other hand, it is a bit difficult to link the paper setting, with continual learning classical settings.\nContinual learning is usually about learning in a non-iid setting with several tasks, where each task might lead to forgetting of the previous one.\nIn this paper, two different tasks are not supposed to interfere with the other as long as they have different target functions \"in the NTK regime, the interaction between different heads do not cause knowledge transfer and forgetting\".\nThe concept of \"self-knowledge transfer\" is not very intuitive, if I understand correctly is about learning again what the model is supposed to already know. \nBut it does not learn it from himself (i.e. from its own knowledge) but still from the data... that why \"self-knowledge transfer\" terminology seems to not fit the phenomenon that the authors want to describe. \"self-knowledge transfer\" would better fit to a pseudo rehearsal method or to a generative replay method.\n\n\nQuestions:\n- Section 4.3 : \"the training on task B degrades generalization even though both tasks A and B learn the same target.\"\n How is it possible? is it overfitting of the target function or just that A and B tasks are about two different modes of the same distribution?\n is it a setting similar to domain incremental settings?\n\n- Section 4.2 : \"positive transfer is guaranteed under equal sample sizes\" I believe sample size is important but if N_A == N_B and all the samples are the same,\n there is no reason for enhancement, i.e. positive transfer. On the other hand, if you have the same number of samples but with a lower variability, you can have a negative backward transfer.\nWhat am I missing here?", "summary_of_the_paper": "The paper is a theoretical paper about continual learning, that studies 2 tasks settings in the NTK regime. \nThe paper study in particular transfer from tasks with the same target functions and introduce the concept of \"self-knowledge transfer\", i.e. transfer from two tasks with the same target function.", "summary_of_the_review": "To conclude, I like the aim at theoretically approaching continual learning but it seems there is still a consequent gap to make the theoretical setting proposed fit an actual continual learning setting.\nI still will grade this paper above the acceptance threshold, because I think the paper can be useful for the community. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "6G0KxSFlRwW", "reviewer": "Reviewer_ZHBA", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a very well written paper. I never learned the replica trick and I'm not an expert of NTK theory. But the authors provide just enough background detail on the NTK and the base Bordelon et al. (2020) and Canatar et al. (2021) papers to have a grasp of the main tools used to derive the new results. These new results are concisely presented and appropriately discussed. Overall, I found the paper a pleasure to read.\n\nThe theory is approximate: it is developed for infinitely-wide networks, and it is not exact even for such networks, as it rests upon some \"tricks\" that are part of the replica method. The authors therefore carefully validate their semi-analytical results with actual neural network simulations. The theory is in impressive agreement with the large-network simulations. The fit of Fig. 1b, which shows rich nonlinear behavior, is particularly remarkable.\n\nThe paper contains a number of interesting results. It justifies the adjective \"catastrophic\" in \"catastrophic forgetting\" and it discovers complex transfer behavior as the relative training set size and teacher noise levels are varied. I think that the simulations alone are worth publishing; the fact that they can be described with semi-analytical methods is a strong plus.\n\nA weakness of the paper, which limits the generality of its conclusions, is its focus on the restricted case where p(x) is assumed to be the same for every task.\n\nLike other studies in the NTK regime, another obvious weakness of the paper is the lack of experiments that try to validate to which extent the theory holds when the network width becomes smaller.\n\nI was not capable of reviewing the calculations presented in the appendix. Therefore I cannot stand for the correctness of the results presented in the paper. As I am not an expert in this field, it is also possible that I missed some important related work that should have been cited.\n\nSome other small remarks:\n- It would be useful to provide some more detail as to how the $\\eta_i$ are computed.\n- There are typos and small English mistakes everywhere that should be fixed.\n- The discussion section would benefit from some comments on which (if any) existing continual learning algorithms that are designed to reduce forgetting might be analyzed in the NTK regime, with similar tools as those used here.\n", "summary_of_the_paper": "The paper presents a study of continual learning of neural networks in the neural tangent kernel (NTK) regime. The authors extend recent work which estimates the generalization error of such networks using a statistical mechanics technique known as the replica trick. The theory makes a number of predictions regarding backward and forward transfer learning which describe well the behavior of finite neural networks trained with gradient descent.", "summary_of_the_review": "This is a well-presented theoretical study that applies previously developed techniques to analytically describe transfer/continual learning in neural networks. The results are to the best of my knowledge novel. The theory reveals interesting rich phenomena that occurs in actual neural network simulations.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: After carefully reading the reviews and rebuttal, I believe this work is of sufficient quality for acceptance. Understanding continual learning from a theoretical stand point is a very important topic. I find that one of the main issue raised by reviewers was about the exact meaning of Continual learning, and whether what the authors studied was more akin to sequential learning. While I don't mind the term sequential learning, and is quite descriptive of the work as well, I disagree that the considered setup is not continual learning.
Decision: Accept (Poster)