Disentangling and mitigating the impact of task similarity for continual learning
Authors: Naoki Hiratani
Venue: NeurIPS 2024 poster (2024)
Keywords: Continual learning
Teacher-student model
Lifelong learning
Brain-inspired
Primary area: online_learning
TL;DR: Continual learning is difficult when the two tasks share more input features than readout patterns, but there are ways to mitigate it.

Abstract: Continual learning of partially similar tasks poses a challenge for artificial neural networks, as task similarity presents both an opportunity for knowledge transfer and a risk of interference and catastrophic forgetting.
However, it remains unclear how task similarity in input features and readout patterns influences knowledge transfer and forgetting, as well as how they interact with common algorithms for continual learning.
Here, we develop a linear teacher-student model with latent structure and show analytically that high input feature similarity coupled with low readout similarity is catastrophic for both knowledge transfer and retention. 
Conversely, the opposite scenario is relatively benign. 
Our analysis further reveals that task-dependent activity gating improves knowledge retention at the expense of transfer, while task-dependent plasticity gating does not affect either retention or transfer performance at the over-parameterized limit. 
In contrast, weight regularization based on the Fisher information metric significantly improves retention, regardless of task similarity, without compromising transfer performance. Nevertheless, its diagonal approximation and regularization in the Euclidean space are much less robust against task similarity. 
We demonstrate consistent results in a permuted MNIST task with latent variables. Overall, this work provides insights into when continual learning is difficult and how to mitigate it.

Disentangling and mitigating the impact of task similarity for continual learning
Naoki Hiratani
Department of Neuroscience Washington University in St Louis
St Louis, MO 631 10 hiratani@uustl.cmm
Abstract
Continual learning of partially similar tasks poses a challenge for artificial neural networks, task similarity presents both opportunity for knowledge transfer and a risk of interference and catastrophic forgetting. However, it remains unclear how task similarity in input features and readout patterns influences knowledge transfer and forgetting, as well as how they interact with common algorithms for continual learning. Here, develop a linear teacher-student model with latent structure and show analytically that high input feature similarity coupled with low readout similarity catastrophic for both knowledge transfer and retention. Conversely, the opposite scenario is relatively benign. Our analysis further reveals that task- dependent activity gating improves knowledge retention the expense of transfer, while wask-dependent plasticity gating does not affect either retention or transfer performance at the over-pur peee-parameterzeed limit. In contrast, weight regularization based on the Fisher information metric significantly improves retention, regardless of task similarity, without compromising transfer performance. Nevertheless, its diagonal approximation and regularization in the Fuclidean space are much less robust against task similarity. We demonstrate consistent results in permuted MNIST task with latent variables. Overall, this work provides insights into when continual learning is difficult and how to mitigate it.
Introduction
Artificial neural networks surpass human capabilities in various domains, yet struggle with continual learning. These networks tend to forget previously learned tasks when trained sequentially- problem known as catastrophic forgetting [44, 16, 23, 32). This phenomenon affects not only supervised training of feedforward networks but also extends to recurrent neural networks [10], reinforcement learning tasks [30], and fine-tuning of large language models [41]. Many algorithms for mitigating catastrophic forgetting have been developed previously, including rehearsal techniques [48, 49, 59], weight regularization [30, 36, 67]. and activity-gaii gating methods [15, 54, 42, 56), among others [57, 50, 63, 21]. However, these methods often hinder forward and backward knowledge transfer [28, 39, 29], and thus it remains unclear how to achieve knowledge transfer and retention simultaneously.
key factor for continual learning task similarity. If two subsequent tasks are similar, there is potential for knowledge transfer from one task to another, but the risk of interference also becomes high [47, 28, 35, 11, 39]. The impact of task similarity on transfer and retention performance particularly complicated because two tasks can be similar in different manners [65, 32]. Sometimes familiar input features need to be associated with novel output patterns, but at other times, novel input features need to be associated with familiar output patterns. Previous works observed that these
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

two scenarios influence continual learning differently [35], yet the impact of the input and output similarity on knowledge transfer and retention has not been well understood. Moreover, remains unknown how the task similarity interacts with algorithms for continual learning such as activity gating or weight regularization.
To gain insight into these questions, in this work, we investigate how transfer and retention perfor- mance depend on task similarity, task-dependent gating, and weight regularization in analytically tractable teacher-student models. Teacher-student models are simple, typically linear, neural networks in which the generative model of data is specified explicitly by the teacher network [18, 66, 3]. These models have provided tremendous insights into generalization property [55, 45, 19, 1], con- vergence rate [61, 58, 38], and learning dynamics [51, 52, 4, 25] of neural networks, due to their analytical tractability. Several works also studied continual learning using teacher-studee settings [2, 35, 27, 1 11, 20, 37, 13] (see Related works section for details).
We develop a linear teacher-student model with a low-dimensional latent structure and analyze how the similarity of input features and readout patterns between tasks affect continual learning. We show analytically that combination of low feature similarity and high readout similarity is relatively benign for continual learning, as the retention performance remains high and the transfer performance remains non-negative. However, the opposíte, a combination of high feature similarity and low readout similarity is harmful. In this regime, both transfer and retention performance become below the chance level even when the two subsequent tasks are positively correlated. Furthermore, transfer performance depends on the feature similarity non-monotonically, such that, beyond a critical point, the higher the feature similarity is, the lower the transfer performance becomes.
We further analyze how common algorithms for continual learning, activity and plasticity gating [15, 42, 46], activity sparsification [57], and weight regularization [30, 36, 67], interact with task similarity in our problem setting, deriving several non-trivial conclusions. Activity gating improves retention at the cost of transfer when the gating highly sparsifies the activity, but helps both transfer and retention on average if the activity is kept relatively dense. Plasticity gating and activity sparsification, by contrast, do not influence either transfer or retention performance at the over parameterized limit. Lastly, weight regularization in the Fisher information metric helps retention without affecting knowledge transfer and achieves perfect retention regardless of task similarity in the presence of low-dimensional latent. However, its diagonal approximation and the regularization in the Euclidean metric are much less robust against both task similarity and regularizer amplitude.
Furthermore, test our key predictions numerically in a permuted MNIST task with a latent structure. When the input pixels are permuted from one task to the next, the retention performance remains high. However, when the mapping from the latent to the target output is changed, both the retention and transfer performance go below the chance level, as predicted. Random task-dependent gating of input and hidden layer activity improves retention at the cost of knowledge transfer, but adaptive gating mitigates this tradeoff. We also show that in fully-connected feedforward network, there exists an efficient layer-wise approximation of the weight regularization the Fisher information metrie, which outperforms its diagonal approximation and the regularization in the Euclidean metric. Nevertheless, the performance of the diagonal approximation is much closer to the wise approximation of the Fisher information metric than to the Euclidean weight regularization.
Our theory thus reveals when continual learning is difficult, and how different algorithms mitigate these challenging situations, providing basic framework for analyzing continual learning in artificial and biological neural networks.
2
Related works
Previous works on continual learning in linear teacher-student models found that forgetting is most prominent at the intermediate task similarity [11, 40, 13] as observed empirically [47]. However, these works did not address the tradeoff between forgetting and knowledge transfer, and these simple settings did not disentangle the effect of the similarity in input feature and readout pattern. Forward and backward transfer performance in continual learning were also analyzed in linear and deep-linear networks [33, 8), yet its relationship with catastrophic forgetting has not been well characterized. Lee et al. [35] analyzed dynamics of both forgetting and forward transfer one-hidden layer nonlinear network under multi-head continual learning setting. However, their analysis of readout similarity
2

---

B
Tesk D
A
W
Readout similrity
y
C
Tusk 2
Task
2
0.0
0.0
0
Feature similarity
Figure 1: (A) Schematic representation of the continual linear regression model with low- -iw-dimensional latent variables. (B-D) Examples of continual learning with two tasks in the MNIST setting. The tasks have low feature similarity in panel C (input pixels are partially permuted) and low readout similarity in panel D (output labels are partially permuted). Panels C and D correspond to the green and orange points in panel B, respectively.
and its comparison to feature similarity were conducted numerically and it did not address the effect of common heuristics, such as gating or weight regularization, either.
By comparison, we introduce a Now-dimensiones latent structure into a linear teacher-studen model, which enables us decouple the influence of feature and readout similarity on knowledge transfer and retention. Moreover, this low-dimensionality assumption on the latent enables us to evaluate the transfer and retention performance analytically even in the presence of gating or weight regularization in the Fisher information metric.
Continual learning has been studied from many thcoretical frameworks beyond teacher-student modeling, including neural tangent kernel [5. 9, 27], PAC learning [6, 60), and computational complexity [31]. Learning of low-dimensional latent representation has also been studied in the context of multi-task learning [43, 62]. In addition, several works investigated the effect of weight regularization on continual learning in analytically tractable settings [30, 12, 24]. In particular, Evron et al. [12] investigated effect of weight-regullrizaaioo in Fisher- informatin metric in continual linear regression scenario.
3 Teacher-student model with low-dimensional latent variables
Let us consider maskiincremental continual learning of regression tasks. For analytical tractability, we consider teacher-student setting where the target outputs are generated by the teacher network (Fig. IA). We define the student network, which learns the task, as a linear projection from a potentially nonlinear transformation of the input, y W (zx). where æ & is the input, y E R&%, is
the output, W E R.,x,,, is the trainable weight matrix, and (xx)) N= - R. is an input transformation. We use (xx))= for the task-dependent gating model, and (x) = sgn(x) ) max (0, |z| h) for the softtthresholding model, where g and h are gating and thresholding vectors, respectively. Here, g @ is element- wise multiplication, and sgn(x) is function that returns the sign of the input. Throughout the paper, use bold-italic letters for vectors, capital-italie letters for matrices.
for the vanilla and weight regularization models, vxxx) g
We generate the input x and target output y of the 4-th task using a latent variable E NN;
8 - N I,), x x= Aps, y B,,,
for μ 1, where A E NNN,, and B, RN,, are mixing matrices hidden from the student network, and I, is the size N₂ identity matrix. We introduce this latent structure, 8, to decouple the effect of feature and readout similarity the transfer and retention performance. Below, set the latent space to be low-dimensional compared to the input space (i.e., N, < N₂). This is motivated by the presence of low-dimensional latent structure in many machine learning datasets [64, 7] and the tasks used in neuroscience experiments [17, 26), but also aids analytical tractability.
We generate the mixing matrices for the first task, A and B,, by sampling elements independently from a Gaussian distribution with mean zero and variance 1. v. The subsequent task matrices A2 and
B2 are also generated randomly, but with element- wise correlation with the previous matrices A and B1 (see Appendix A for the details). We denote the clement-wise correlation between A₁ and
3

---

A
simitrity A,
Thanster de..
Error
044
Ader
Aage
2
-12
0.10
A:106
E
-100
Av-00
-275
110
64
16 Go
0A
10
Iterations
Foalure simiarlly P,
Feature simiarty
Ritemton dem
Reatad smbberty
c23
Rolontoon dee
Resttout simlarty
2.00
R005
8:00
1.
A5104 22 00100
2.5 626
Feature similarly Pa
Feature siialllllyy...
simiafly Ps
Feelure similarity P.
Transfer and retention performance of the vanilla model. (A) Illustration of ÂETF and Aeg Red and blue lines represent the error on task and task 2, respectively. Here, the model was trained task I for 100 iterations and then trained on task 2 for another 100 iterations. (B, C) Transfer performance under various task similarity. Points in panel B are numerical results (the means and the standard deviations over ten random seeds), while solid lines are analytical results (Eq. 4). (D-G) Retention performance under various task similarity. Panel G magnifies the 0.9 S ps S 1.0 region of panel E, and the white dashed line in panel G represents local minima/maxima.
Figure 2:
Ag by Pa and refer as feature correlation, because A₁ and A₂ specify the input features. Similarly, we denote the correlation between B1 and B2, the readout correlation, by Pb. Figs. 1B-D illustrate the difference between Pa and ps in an image classification setting. When Pa < 1.0 and Ps 1.0. input features are partially modified in the second task compared to the first task (Green point on Fig. IB and Fig. IC). For example, permuted MNIST task [22] corresponds to this low feature similarity setting with Pa 0.0 and Pb 1.0. By contrast, when Pa and ps < 1, the readouts are changed partially (Orange point on Fig. 1B and Fig. ID). When P. = 1 and ps = -1, the same input features associated with the opposite readout in the new task, which is known the reversal learning paradigm. In this reversal scenario, the network fails to achieve knowledge transfer rather trivially. Thus, we focus the scenario when tasks have non-negative correlation in terms of both input features and readout (i.e., 0 S Pa. Pb I 1) below.
We measure the performance of the student network with weight W on the μ-th task by mean- wean-squared error (u[W]- = N, {IIB,ss s (1,,)))) where (.) is the expectation over latent variable s. The transfer and retention performance are defined by
ÁETF (e2[W.] - e2(Wi]) A,B AERT (EI [W.] E (WW)))
Here, A. is the expectation over randomly generated task matrices A1, Ag, B, and B3 under given feature and readout correlation Pa. Pb- As illustrated in Fig. 2A, the transfer performance AcTF measures how much performance the model achieves on task 2 by learning task 1, whereas the retention performance AERT measures how well the model can perform task 1 after learning task (here, the task switch occurs at the 100th iteration). Below, we study how knowledge transfer and retention, the two key objectives of continual learning, depend on task similarity, and how optimize the performance through gating and weight regularization.
Impact of task similarity on knowledge transfer and retention
Let us first investigate the vanilla model without gating or weight regularization, to examine how task similarity affects knowledge transfer and retention. Given (0(x) = æ, at the infinite sample limit,
learning by gradient descent follows W -(W A, A, A The fixed point of this dynamics, as detailed in Appendix B. is:
W₂ w (T--[[I-UU,)) U,UU)) B,
where WT - is the weight after learning of the previous task, Up is the semi semi-orthogonal matrix from singular value decomposition (SVD) of A,, UnAn viT, and A is the pseudo-inverse inverse of matrix Ay

---

Inserting Eq. 3 into Eq. 2 and taking the expectation over randomly generated tasks A1, B1. A2. B2. the transfer and retention performance are written as (see Appendix B.1)
BCTF - Pa(2pb - Pa), ACRT = 1 2PoPb + 1).
(4)
Recall that Pa is the feature similarity defined by the correlation between A1 and Ag. while p is the readout similarity, representing the correlation between B1 and B2. The derivation of the above equations relies on (correlated) random generation of A1, B1, A2, .32 and the low-rank latent assumption: N, < Nz. These two equations, despite their simplicity, capture the transfer and retention performance in numerical simulations well (Figs. 2B, D, and F; see Appendix E. for the details of numerical estimation). Furthermore, they reveal asymmetric and non-monotonic impact of the feature and readout similarity on the performance.
Fig. 2B depicts the knowledge transfer from task to task 2, ΔETF, under various (Pa: Ph)) conditions. As expected, AerF 0 when the two tasks independent (i.e., (Pa, pb) (0. 0)), and ÁetF = when the tasks are identical (i.c.. (pa. Pb) (1, 1)). At intermediate levels of similarity. there is clear asymmetry in the influence of feature and readout similarities on transfer performance; particularly, combination of high feature similarity and low readout similarity leads to negative transfer, while the opposite scenario results in a modest positive transfer (lower-right vs upper-left of Fig. 2B).
Notably, under a fixed readout similarity pb. the knowledge transfer depends non- non-monotonically on the feature similarity Pa- When Pa < Pb. the higher the feature similarity the better transfer becomes, as implied from Eq. 4. However, once the feature similarity exceeds the readout similarity, counter-intuifveely, high feature similarity worsens the transfer performance (Figs. 2B and C). This is because, when the feature similarity is high, inputs are aligned with learned features, resulting in a large output regardless of readout similarity. Particularly, under a low readout similarity, performance becomes below zero because extracted features are mostly projected to the incorrect directions.
The retention performance also exhibits asymmetric dependence on feature and readout similarity. When feature similarity Pa is low, the network barely forgets the previous task regardless of readout similarity (Fig. 2D left). By contrast, when the feature similarity is high, the retention performance can either be positive or negative depending on the readout similarity. Moreover, in the high readout similarity regime, the retention performance is the lowest at intermediate feature similarity (Figs. 2E and F). This is because high retention is possible when either interference is low, or similarity between two tasks is high. Note that, this last point the non-monotonic dependence feature similarity under Pb = has been investigated both empirically [47] and analytically [35, 11, 13). Indeed, at Ps = AERT in Eq. coincides with Eq. (5) in [13].
Therefore, the knowledge transfer and retention performance depend on feature and readout simi- larities in asymmetric and non-monoto manner. Specifically, a combination of high feature similarity and low readout similarity is detrimental to continual learning, resulting in negative transfer and retention performance, even in the presence of positive correlation between the two tasks in both input features and readout patterns. Moreover, under a fixed readout similarity, the knowledge transfer performance depends the feature similarity in non-monotonic manner. Thus, the con- tinual learning in the vanilla model is sensitive to task similarity and limited in performance. These results make wonder whether
mitigate the sensitivity task similarity and improve the overall transfer and retention performance by modifying the learning algorithm. To this end, we next investigate how task-dependent gating and weight regularization methods, two popular strategies in continual learning, alleviate knowledge transfer and retention.
5 Task-depe gating
5.1 Random activity gating
One popular method for mitigating forgetting in continual learning is activity gating [15, 3,,44 42, 21, 56). With gating of input activity, the network is written as y W (g C x)) where gn E (0, 1] is a binary gating vector for task We first consider a random gating scenario where elements of gμ are randomly sampled from Bernoulli distribution with rate a which we denote as the gating level.
All units are active at 1, while units are active - 0 limit.
5

---

0,110
Transler Ditee
0.5
Readout Readou similarty Pb
625
Abpp Jagtuee
Pietenboon Diter
perfomante
0.
Rotendan Transter
Bum0 R
8
Rum0
A-0.0
04
64
Callin level
Feature simiirity
Gating level
Getlng level
Figure 3: Random las-ddependent activity gating model. (A) Knowledge transfer performance under
P. = 1.0. The gating level is defined as the fraction of active input neurons (i.c., a = Pr[g; 1...
(B) The transfer performance under the optimal gating level a min 2h 1}. (C) Retention
performance under Pa = 1.0. (D) Average transfer and retention performance over uniform prior on 0 S Pa, Pb < 1. Horizontal dashed lines are the performance of the vanilla model, while solid lines are the performance of the random gating model. Points are numerical estimations.
Adaptive activity gating
Average performance C
Random plasticly gaing
Input soft wheeeteeeeshiii ding
Average performence
1.0
Awarogo perfomance
Transter Ner
0.6
Retention Transfer
Pj40
Retention Transter
Resention Transeer
P2-0
14
Feature simiartty P.
Gating
Gating level
Gating
Figure 4: Transfer and retention performance of the adaptive activity gating (A, B), random plasticity gating (C), and input soft-thresolding (D) models. Dashed and solid lines in panels A and B represent the performance of the random and adaptive activity gating models, respectively.
From a parallel argument with the vanilla model, when N₂ < aNNa, the transfer and retention performance are estimated as (see Appendix B.1):
АCTF cppo(2pp apa), Aerr - 20papp + +
(5a)
(5b)
Thus, random gating scales the feature similarity from Pa apa. This scaling lowers the knowledge transfer if Ps 2 Pa because random gating reduces the fraction of input neurons active in two subsequent tasks (lime line in Fig. 3A). However, if pb Pa. gating with a Ph. enhances the transfer by reducing the effective feature similarity (blue lines in Fig. 3A; also compare Fig. 3B with Fig. 2C). The optimal gating level a for knowledge transfer is min f. 1}. indicating that the input activity typically needs to be dense. By contrast, the retention performance is optimized at a + 0 limit where tasks do not interfere with each others (Fig. 3C). this limit, we have AERT 1. Note that, in reality, a has to be non-zero optimize the retention (Eq, 5b holds only when a N
Pu
These results indicate that random activity gating improves the retention at the cost of forward knowl- edge transfer, as suggested previously [28, 39, 29]. This tradeoff between knowledge transfer and retention is especially critical when the network doesn't know the task similarities. Suppose that the task similarity is distributed uniformly over 0 I Pa. Ps < 1. Then, the average transfer performance is function of a (Fig. 3D). Thus, moderate gating (a 3/) benefits both transfer and retention average. However, retention performance in this regime is significantly below one, implying that the network cannot reliably achieve high retention performance without sacrificing the transfer performance.
maximized at &. while the average retention performance decreases monotonically
5.2 Adaptive activity gating
One way to overcome the tradeoff between transfer and retention performance is to consider adaptive gating. Let us introduce a probe trial at the beginning of the task 2, in which the model tests how well it performs if it uses the gating for task 1, I1. for task 2 as well. If the emror in the probe trial is small, the model should keep using the same gating vectors to achieve a good knowledge transfer. Otherwise, it should resample the gating vector for retaining the knowledge on task 1. Using the

---

A,113
A,110
Readout simiarity Po
Avanage parformenco mamaaa
Trandfer dit
Ratemonn Mat
12
A eewppl
Retention Transfer
14
2-402 AP00
22 CA de
24
14
03
04 05
0.3
10
02 34 35
Regularizer coefiidint
Reguirizer coefficient
Feature cimilarty P.
Regulartzer confiiiion y
Figure Performance of weight regularization in Euclidean metric. (A,B) Transfer (A) and retention
(B) performance. The amplitude of the weight regularization scales with - 1. (C) Regularizer coefficient that optimizes the retention performance. (D) Average performance over uniform task
similarity distribution S Per Pb S Horizontal dashed lines are the performance of the vanilla model.
probe error Epb. we set the probability of using the same gating as Pa = Epb/ This adaptive gating indeed improves the average transfer performance compared to the random gating (solid vs dashed lines in Fig. 4A) with only a relatively small reduction in the retention performance (Fig. 4B).
5.3 Plasticity gating and input soft-thresholding
Previous works have also explored other gating mechanisms, such as plasticity gating [46] and activity sparsification [57]. However, their benefits over the activity gating discussed above have not been fully understood. We implement task- musk-dependent plasticity gating into our problem setting by making only the synaptic weights projected from subset of input neurons plastic. Unexpectedly, we found that both transfer and retention performance are independent of the fraction of plastic synapses in the N, < N₂ limit, potentially due to weerpaarameterization (Fig. 4C; see Appendix B.2 for details).
Similarly, when the input activity is sparsified using soft-threshodiing (<x) = sgn(x) max max(0, |ar] hJ. with a fixed threshold / /2erfc (a). the transfer and retention performance remain inde- pendent of the resultant input sparsity (Fig. 4D; see Appendix B.3 for the details). These results imply that plasticity gating and input sparsification are not effective in our model, which operates in an over- parameterized regime (i.e., N₂ < N2), but do not exclude their potential benefits for many continual learning tasks.
Weight regularization
6.1 Weight regularization in Euclidean metric
Another popular method is weight regularization, which keeps the synaptic weights close to those learned from previous tasks [30, 36, 67]. Let us first consider regularization of the Euclidean distance between the current weight W and the weight learned in the previous task W 1=
f/ 1 B, WA,
No 2N.
1) |[ W Ilf
(6)
Here, we parameterize the regularizer amplitude by N. for brevity. corresponds to zero regularization, while 0 is the infinite regularization limit. Under this parameterization, N₂ & Nz, the transfer and retention performance follow simple expressions as below (see Appendix C.1):
N (2 - 1) using a non-negative parameter
ACTr (2pb - YPa),
(7a)
ARRT = - -1- + +2p2) + 27pa (ph - YPp)) - (1-)) 7)2
(7b)
The expression of the transfer performance is equivalent to Eq. 5a if we change 7 to Thus, the Euclidean weight regularization with amplitude (+ 1) is mathematically cquivalent with random activity gating with sparsity in terms of knowledge transfer (compare Fig. 5A with Fig. 3A). By contrast, the expression of the retention performance contains two additional terms compared to 5b. In particular, the last term, (1 >>)², indicates that strong weight regularization not only prevent forgetting, but also impairs task acquisition. Thus, the retention performance optimized at an intermediate regularizer strength (Figs. 5B and C).
7

---

Fiew information metric
Diegonal appresimation
Avwragg performerra
Avarngge parformanco
6.0
Retention
0.2
Recention
Releniion Transfer
0.9
Transfer
650
0.702
0,000
12
08
Regularizer comficient
Regularizer conficient
Rogulartzer conficienty y
Ragularizer conficiint y
Figure 6: Weight regularization the Fisher information metric. (A,B) The retention performance under various task similarities and regularizer coefficients. (C,D) Average transfer and retention performance under the regularization with the exact Fisher information metric (C) and its diagonal approximation (D).
Because strong regularization (i.e., small 7) impairs both retention and transfer, unlike the random activity gating model, there isn't a tradeoff between knowledge transfer and retention in terms of the average performance over 0 S Pa. < 1 (Fig. 5D). However, the retention performance at the optimal parametric regime is relatively low (Fig. 5D with Fig. 4B).
6.2 Weight regularization in Fisher information metric
Previous works proposed that the synaptic weights should be regularized in the Fisher information metric to allow flexibility in the non-overlapping weight space [30, 67]. Applying it to our model setting, the regularizer term in Eq. instead becomes 2 -1) II(W w,,-))A-11ll (see Appendix C.2). If Pay Y and N, < N2, optimization of the weight under this regularization yields
W W w 1- [4] Po44-1l + w.ii-ayJ By (4 -a
(8)
where w 1 is the weight after the previous task learning. Notably, the weight becomes independent of the regularizer's amplitude y. Furthermore, the retention performance under arbitrary task similarity (Pa, Pb) is derived as AERT = O NI Thus, under the Fisher information metric, the
retention performance is perfect as long as the condition N, < N2 Po)) is satisfied (Figs. 6A-C). Intuitively, assuming over-pane parameterization (N, < N),, the network can freeze the weight changes in low-dimensional subspace, while maintaining sufficient plasticity for new tasks, unless the two tasks share the same feature. Notably, if the first task is learned with weight regularization in orthogonal direction, the transfer performance remains the same with the vanilla model (Fig. 6C).
Importantly, this invariance no longer holds when the Fisher information metric is approximated by its diagonal component (Figs. 6D vs 6C), as is done the elastic weight methods [30]. This is because the diagonal approximation makes the metric full-rank even when the true metric has a low-rank structure. Thus, weight regularization in the Fisher information metric robustly helps retention without harming transfer, but diagonal approximation attenuates its robustness.
7 Numerical experiments
Our theory so far has revealed when continual learning is difficult and when activity gating and weight regularization can ameliorate the transfer and retention in a simple problem setting. Let us next examine how much these insights are applicable to more realistic datasets and neural architectures.
To this end, we consider the permuted MNIST dataset [34, 22], but with a latent structure (see Appendix E.2). For each output label, we constructed four-dimensional latent variable s using the binary representation of the corresponding digit (e.g.. 89 = [1, 0, 0, 11²),. The target output was then generated by random fixed projection of the latent variable: y* (s 1)). Note that, although here we introduced an explicit latent structure for a direct comparison with the theory, qualitatively similar results holds in the vanilla permuted MNIST setting depicted in Figs. IB-D (see Fig. 12). We controlled the readout similarity between tasks using the element- wise correlation between two matrices B, and B2. The feature similarity was controlled by permuting a subset of input pixels, as in the previous permuted MNIST tasks. We used one-hidden-layer fully-connected network with rectified-linee nonlinearity the hidden layer, and trained the network using stochastic gradient descent.
8

---

A-10
Roateut similatty Pe.
Readout simbantty Sy
Avwrage perfomencs
Transfer Aay
1.0
-045
9:30
-4.2
Retention
Transffr
0.8
% 02
04
2-
Fealure simlaity
Feature similantty
Gating level o
Gating loval
F
2.022
Pu P.-0.
H
1.0
FIM Daye Wiel
Toursder Darr
Retenton Mett
Rotemtion &r..
2.4
FIM (dagona))
Exc
FM Obyer wise)
FM Oayer- wie))
FIM Sayerwlent
FM (giggoe))
BM (d (ggnne))
HIM (dagonal)
Eudiidcan
+02
Eudidcan
Busiddan
1014
103
to to 10
124
19
23-
101 20
Regularizer ampiituee
Regulsrizer amprltudle
Regularizer ampltude
Regularizer amp itude
Figure 7: Permuted MNIST with latent variables. (A,B) Transfer and retention performance of the vanilla model. (C,D) Performance of random (dashed lines) and adaptive (solid lines) activity gating models. (E-H) Performance of the weight regularization in the Euclidean metric, and approximated Fisher information metrics (red: layer-wist approximation; orange: synapse-war wise/diagonal approxi- mation). Here, lines are linear interpolations and error bars are standard errors over random seeds, not standard deviations.
As predicted from theory, both transfer and retention performance, measured by the test error, exhibited asymmetric and non-monotonic dependence on the task similarity. When (Pas Ps) = (1.0, 0.0), both transfer and retention performance were below zero (lower-right of Figs. 7A and vs. Figs. 2C and E). By contrast, the model achieved high retention and zero transfer under (Pa. Pb) = (0.0, 1.0) expected (upper-left). Notably, around Ps = 0.5, the transfer performance exhibited non-monotonic dependence on in accordance with the theory. The retention performance also exhibited non-monotonie dependence on Pa under Ps ss but the dependence became monotonic after a long training (see Figs. 9C and D in Appendix). We observe consistent results even when feature and readout similarity are adjusted by permuting input pixels and output labels, respectively, provided the loss is measured by cross cross-entropy (Fig. 12B, corresponding to the examples depicted in Figs. 1C and ID).
Random activity gating, implemented in both input and hidden layers, improved the retention performance at the cost of low knowledge transfer as predicted (compare Fig. 7D and Fig. 4B). Moreover, adaptive gating based on probe trial improved the transfer performance under a low gating level, especially when the readout similarity high (solid dashed lines in Figs. 7C and D).
In a deep neural network, weight regularization using the Fisher information metric is typically computationally expensive. However, an efficient layer-wise approximation exists for fully-connected networks (see Appendix E.2). This layer-wise approximation enabled high retention performance across wide range of regularization amplitudes and consistently outperformed weight regularization in the Euclidean metric under various task similarities (red vs. gray lines in Figs. 7F-H). Nevertheless, the retention performance under the diagonal approximation was closer to that of the layer- approximation of the Fisher information metric and was slightly better when both methods performed poorly (red vs. orange lines). This is because the sparsity of hidden layer weights and activity makes the diagonal components sparse.
8 Discussion
Here, we analyzed the impact of task similarity on continual learning in a linear teacher-student model with -ow-dimeniional latent structures. We showed that, combination of high feature similarity and low readout similarity lead to poor outcomes in both retention and transfer, unlike the opposite combination. We further explored how continual learning algorithms such as gating mechanisms, activity sparsification, and weight regularization interact with task similarity. Results indicate that weight regularization in the Fisher information metric significantly aids retention regardless of task similarity. Numerical experiments in permuted MNIST task with latent supported these findings.

---

Our findings on the interaction between task similarity and continual learning algorithms have several implications. Firstly, when tasks are known have high feature similarity and low readout similarity, adaptive activity gating and weight regularization in the Fisher information metric help retention without sacrificing transfer (Figs. 4, 6, 7). In the context of neuroscience, our results indicate that simple non-adaptive activity or plasticity gating mechanisms are not sufficient for good continual learning performance, especially when familiar sensory stimuli need to be associated with novel motor actions. Indeed, a previous study reported catastrophic forgetting among rats learning timing estimation tasks [14]. Lastly, it also implies the potential importance of studying wider benchmarks beyond permuted image recognition tasks for continual learning, because image permutation belongs to class of relatively harmless task similarity according to our theory. As a simple extension, here we developed an image recognition task with latent variable and showed that vanilla deep network exhibits more forgetting under readout remapping than under input permutation (Figs. 7A and B).
Limitations Our theoretical results from a teacher-student model come with limitation in direct applicability. The presence of low- fow-dimensional latent generating both input and target output is a reasonable assumption for many real- weal-world tasks [7, 26], but the linear projection assumption is not. While we replicated most key results in a deep nonlinear network solving permuted MNIST task, we found that the diagonal approximation of Fisher information metric performs much better than the prediction from the teacher-student model. This is potentially because sparse activity at the hidden layer effectively makes the regularization low-rank even under the diagonal approximation. More generally, the presence of hidden layers should enable distinctive adaptation for feature and readout similarity, which is an important future direction. It is also important to apply our thcoretical framework for analyzing continual learning in more complicated neural architectures and datasets.
Acknowledgments
The author thanks Liu Ziyin and Ziyan Li for discussion. This work was partially supported by The Swartz Foundation.
References
[1] M. S. Advani, A. M. Saxe, and H. Sompolinsky. High-dimensional dynamics of generalization error in neural networks. Neural Nenworks, 132:428-444 446, 2020.
[2] H. Asanuma, S. Takagi, Y. Nagano, Y. Yoshida, Y. Igarashi, and M. Okada. Statistical mechanical analysis of catastrophic forgetting continual learning with teacher and student networks. Journal of the Physical Society of Japan, 10:100001, 2021.
[3] Y. Bahri, Kadmon, Pennington, S. S. Schoenholz, Sohl- Dickkstiin, and Ganguli. Statistical mechanics of deep learning. Annual Review of Condensed Matter Physics, 11:501-528, 2020.
[4] Baity-Jesi, L. Sagun, M. Geiger. Spigler, G. B. Arous, C. Cammarota, LeCun, M. Wyart, and G. Biroli. Comparing dynamics: Deep neural networks versus glassy systems. In International Conference on Machine Learning, pages 314-323. PMLR, 2018.
[5] M. A. Bennani, T. Doan, and M. Sugiyama. Generalisation guarantecs for continual learning with orthogonal gradient descent. arXiv preprint arXiv:2006. 11942, 2020.
[6] X. Chen, C. Papadimitriou, and B. Peng. Memory bounds for continual learning. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS), pages 519-530. IEEE, 2022.
[7] U. Cohen, S. Chung. D. Lee, and H. Sompolinsky. Separability and geometry of object manifolds in deep neural networks. Nature communications, 11(1):746, 2020.
[8] O. Dhifallah and Y. M. Lu. Phase transitions in transfer learning for high- -igh-dimensional perceptrons. Entropy, 23(4):400, 2021.
[9] T. Doan, M. A. Bennani, B. Mazoure, G. Rabusseau, and P. Alquier. A theoretical analysis of catastrophic forgetting through the ntk overlap matrix. In International Conference on Artificial Intelligence and Statistics, pages
PMLR, 2021.
[10] B. Ehret, C. Henning, M. R. Cervera, A. Meulemans, J. Von Oswald, and B. F. Grewe, Continual learning in recurrent neural networks. arXiv preprint 121199 2020.
10

---

[11] I. Evron, E. Moroshko, R. Ward, N. Srebro, and D. Soudry. How catastrophic can catastrophic forgetting be in linear regression? In Conference Learning Theory, pages 4028-4079. PMLR, 2022.
[12] I. Eyron, E. Moroshko, G. Buzaglo, M. Khriesh, B. Marjieh, N. Srehro, and D. Soudry. Continual learning in linear classification on separable data. In International Conference on Machine Learning, pages 9440-9484. PMLR, 2023.
[13] I. Evron, D. Goldfarb, N. Weinherger, D. Soudry, and P. Hand. The joint effect of task similarity and overparameterization on catastrophic forgetting-an analytical model. arXiv preprint arX1v:2401.12617, 2024.
[14] R. French and A. Ferrara. Modelling time perception in rats: evidence for catastrophic interference in animal learning. In 21st Congress of Cognitive Sciences, 1999.
[15] R. M. French. Using semi-distributed representations to overcome catustrophic forgetting in connectionist networks. In Proceedings of the 13th annual cognitive science society conference, volume 1, pages 173-178, 1991.
[16] R. M. French. Catastrophic forgetting in connectionist networks. Trends cognitive sciences, 3(4): 128-135, 1999.
[17] P. Gao, E. Trautmann, B. Yu, G. Santhanam, S. Ryu, K. Shenoy, and S. Ganguli. A theory of multineuronal dimensionality, dynamics and measurement. BioRvir, page 214262, 2017.
[18] E. Gardner and B. Derrida. Three unfinished works on the optimal storage capacity of networks. Journal of Physics A: Mathematical and General, 22(12):1983, 1989.
[19] B. Ghorbani, Mei. Misiakiewicz. and Montanari. Limitations lazy training of two-layers neural network. Advances in Neural Information Processing Systems, 32, 2019.
[20] D. Goldfarb and P. Hand. Analysis of catastrophic forgetting for random orthogonal transformation tasks in the overparameterized regime. In International Conference on Artificial Intelligence and Statistics, pages 2975-2993, PMLR, 2023.
[21] S. Golkar, M. Kagan, and K. Cho. Continual learning via neural pruning. arXiv preprint 100.v:1903.044:: 2019.
[22] I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y. Bengio. An empirical investigation of catastrophic forgetting in gradient-based based neural networks. arXiv preprint
2013.
[23] R. Hadsell, D. Rao, A. A. Rusu, and R. Pascanu. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences, 24(12):1028 1040, 2020.
[24] R. Heckel. Provahle continual learning via sketched jacohian approximations. In International Conference on Artificial Intelligence and Statistics, pages 10448-10470. PMIR, 2022.
[25] N. Hiratani, Y. Mchta, T. Lillicrap, and E. Latham. On the stability and scalability of node perturbation learning. Advances in Neural Information Processing Systems, 19:31129-31944 2022.
[26] M. Jazayeri and S. Ostojic. Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity. Current opinion in neurobiology, 70:113- 2021.
[27] R. Karakida and S. Akaho. Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting. In International Conference on Learning Representations, 2021.
[28] Z. Ke, B. Liu, and X. Huang. Continual learning of a mixed sequence of similar and dissimilar tasks. Advances in neural information processing systems, 33:18493 8493-188544, 2020.
[29] 7. Ke, B. Liu, W. Xiong, A. Celikyilmaz, and H. L... Sub-network discovery and soft-masking for continual learning of mixed tasks. arXiv preprini arXiv:2310.09450 2023.
[30] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, Quan, T. Ra- malho, A. Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, (1331))352133356, 2017.
[31] J. Knoblauch, H. Husain, and T. Diethe. Optimal continual learning has perfect memory and is np-hard. In International Conference Machine Learning, pages 5327-5337. PMLR, 2020.

---

[32] D. Kudithipudi, M. Aguilar-Simon, J. Babb, M. Bazhenov, D. Blackiston, J. Bongard, A. P. Brna.
S. Chakravarthi Raja, N. Cheney, J. Clune, et al. Biological underpinnings for lifelong learning chines. Nature Machine Intelligence, 4(3):196-110, 2022.
[33] A. K. Lampinen and S. Ganguli. An analytic theory of generalization dynamics and transfer leaming in deep linear networks. arXiv preprint arXiv:1809.1077 10374, 2018.
[34] Y. LeCun, L. Bottou, Y. Bengio, and Haffner. Gradient-based learning applied document recognition. Proceedings of the IEEE, 86(1 1)22788-232, 1998.
[35] S. Lee, S. Goldt, and A. Saxe. Continual learning in the teacher-student setup: Impact of task similarity. In International Conference on Machine Learning, pages 6109-6119 19. PMLR, 2021.
[36] S.-W. Lee, J.HH. Kim, Jun, J.-W. Ha, and B.-T. Zhang. Overcoming catastrophic forgetting by incremental moment matching. Advances in neural information processing systems, 30, 2017.
[37] C. Li, Z. Huang. W. Zou, and H. Huang. Statistical mechanics continual learning: Variational principle and mean-field field potential. Physical Review E, 108(1):014309 2023.
[38] Y. Li, Ma, and R. Zhang. Learning over- parametrretri two-layer neural networks beyond ntk. In Conference on learning theory, pages 2613-2682. PMLR, 2020.
[39] S. Lin, L. Yang, D. Fan, and J. Zhang. Beyond not- forgettgt::: Continual (entinuallaarinng with backward knowledge transfer. Advances in Neural Information Processing Syxtems, 35:16165-16177, 2022.
[40] S. Lin, Ju, Y. Liang, and Shroff. Theory on forgetting and generalization of continual learning. In International Conference on Machine Learning, pages 21078-21100. PMLR, 2023.
[41] Y. Luo, Z. Yang, F. Meng. Y. Li, J. Zhou, and Y. Zhang. An empirical study of catastrophic forgetting in large language models during continual fine- mhn-tuning. arXiv preprint arXiv:2308. 08747, 2023.
[42] N. Y. Masse, G. D. Grant, and D. J. Freedman. Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization. Proceedings of the National Academy of Sciences, 115(44):E10467- E10475, 2018.
[43] A. Maurer, M. Pontil, and B. Parere-Paredes. The benefit of multitask representation learnin.. Journal of Machine Learning Research, 17(81):1-33 2016.
[44] M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.
[45] A. Ndirango and T. Lee. Generalization in multitask deep neural classifiers: statistical physies approach. Advances in Neural Information Processing Systems, 32, 2019.
[46] S. Özgün, A.M Rickmann, A. G. Roy, and C. Wachinger. Importance driven continual learning for segmentation across domains. Machine Learning in Medical Imaging: 11th International Workshop. MLMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings 11, pages 423-433. Springer, 2020.
[47] V. V. Ramasesh, E. Dyer, and M. Raghu Anatomy of catastrophic forgetting: Hidden representations and task semantics. arXiv preprint 2xXi:2007.077400 2020.
[48] A. Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2)) 146.
1995.
[49] D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne. Experience replay for continual learning. Advances in neural information processing systems, 32, 2019.
[50] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer. J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and 2016.
R. Hadsell. Progressive neural networks. arXiv preprint
[51] D. Saad and S. A. Solla. On-line learning in soft committee machines. Physical Review E, 52(4))4225 1995.
[52] A. M. Saxe, J. L. McClelland, and S. Ganguli. Exact solutions to the nonlincar dynamics of learning in deep linear neural networks. arXiv preprint 3rXiv:1312.6/200 2013.
[53] A. M. Saxe, J. L. McClelland, and S. Ganguli. A mathematical theory of semantic development in deep neural networks. Proceedings the National Academy of Sciences, 116723) -1546, 2019.
12

---

[54] J. Serra. D. Suris, M. Miron, and A. Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference machine learning, pages 4548 4557. PMLR, 2018.
[55] H. S. Seung. H. Sompolinsky, and N. Tishby. Statistical mechanics of learning from examples. Physical review A, 45(8):6005, 1992.
[56] E. Sezener, A. Trabska-Barwiisske D. Kostadinov, M. Beau, S. Krishnagopal, D. Budden, M. Hutter, J. Veness, M. Botvinick, C. Clopath, al. A rapid and efficient learning rule for biological neural circuits. BioRxiv, pages 2021 03, 2021.
[57] R. K. Srivastava, J. Masci, S. Kazerounian, F. Gomez, and J. Schmidhuber. Compete compute. Advances in neurul information processing systems, 26, 2013.
[58] Y. Tian. An analytical formula of population gradient for two-layered relu network and its applications in convergence and critical point analysis. In International conference on machine learning, pages 3404-311.. PMLR, 2017.
[59] G. M. Van de Ven, T. Siegelmann, and Tolias, Brain-inspired replay for continual learning with artificial neural networks. Nature communications, (11):4009, 2020.
[60] L. Wang, X. Zhang, Q. Li, Zhang, H. Su, Zhu, and Zhong. Incorporating neuro-inspired adaptability for continual learning in artificial intelligence. Nature Machine Intelligence, 5(12):1356-1366 2023.
[61] Werfel, Xie, and H. Seung. Learning curves for stochastic gradient descent linear feedforward networks. Advances in neural information processing systems, 16, 2003.
[62] Z. Xu and A. Tewari. On the statistical benefits curriculum learning. In International Conference on Machine Learning, pages 24663-24682. PMLR, 2022.
[63] Yoon, E. Yang, J. Lee, and S. J. Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint Xrviv: (0x::770.00157,, 2017.
[64] X. Yu, T. Liu, X. Wang, and Tao. On compressing deep models by low rank sparse decomposition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7370-7379, 2017.
[65] A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE conference computer vision and puttern recognition, pages 3712-3722, 2018.
[66] L. Záeborová and F. Krzakala. Statistical physics of inference: Thresholds and algorithms. Advances in Physics, 65(5):453-555 552, 2016.
[67] F. Zenke, B. Poole, and S. Ganguli. Continual learning through synaptic intelligence. In International conference machine learning, pages 3987-3995. PMLR, 2017.
13

---

A Problem setting: Linear teacher-student model with a latent variable
Let us consider continuous learning of Neess tasks. We generate the input R and the target output y² E RN,, of the f-th task by
(9) where 8 E R. is the latent variable, Is is the size N₂ identity matrix, and A,. E RNN.NN, and Bp E NN,NN, are mixing matrices that map the latent variable s into the input space and output space, respectively. Throughout the paper, the vector is defined as column vector, and its transpose x is a row vector.
- N I,), I == y = =B,8,
We consider sequential learning of these tasks with neural network defined by Wxb where W E &,,,,N, is the plastic weight. We set the function vb R - RR to ()(x)) = æ in the vanilla and weight regularization models, (xx) 9 in activity and plasticity gating models, and (xx))= = (x) O max(0, | h} in the ooftthheesholdin model. Here, @ x represents an element- wise multiplication of two vectors, and sgn(x) is function that returns the sign of the input x. The goal of the student network, y = W)(()) is to mimic the teacher network that generates both input and target output [18, 66, 3). Below, focus learning of N 2 tasks. To analyze the transfer and retention performance, we introduce the following two assumptions on task structure.
Assumption I: Random task assumption We consider random generation of task matrices Ans By but with correlation between subsequent tasks. We sample elements of the mixing matrices for the first task, A₁ and B1, from Gaussian distribution with mean zero and variance 1 independently. N,
For the subsequent tasks, generate mixing matrices Ay by
An-1,3
(with probability Pa) (otherwise)
Au.i
(10)
An.ss
where Aμ is a random matrix whose elements are sampled independently from a Gaussian distribution with mean zero and variance N We generate B, in the same manner, but with correlation Pb- Although this hidden weight generation doesn't exactly follow the correlated Gaussian assumption, the difference appears only in higher-order terms, which negligible under N, >> N₂ assumption we introduce below.
Assumption II: Low Low-dimensional latent assumption The second assumption is the relative low dimensionality of the latent space with respect to the input space: N, < Nz. This assumption of vve-parramttrrizationn in the input space simplifies the analysis (see Appendix D).
Evaluation of the transfer and retention performance We evaluate the task performance of student model with weight W on task μ by mean-squared error:
4 [
BB.s (1AAs)))11)
(11)
N,
Here, (), the expectation over latent 8 ~ NO0 Is). Using en [W], the degree of forward knowledge transfer and retention are evaluated by
TF (c W.] cn
A.B
(12a)
Ae RT (c [Wo] - En (1)+))))))
(12b)
where W,, is the weight after training on the 1--th task. Note that, the retention performance is defined by the difference with the initial error ',, (W.). This means that, if the network fails to learn the task in the first place (i.c., if [W,] > 0), the retention performance becomes sub-optimal even if the network doesn't 't forget the learned task. We used this definition for our problem setting because the network is able to learn the task perfectly unless strong regularization is added to the network.
Learning procedures We consider the gradient descent learning from infinite samples at the gradient flow limit and analyze the performance at the convergence. Below, we investigate how TeP TF and RT depend on the task similarity and hyper-parameters under various algorithms for continual learning.
14

---

B Task-dependent gating model
B.1 Activity gating
Let us first consider task-dependent activity gating model. The student network (the network that learns the task) is given by:
W Wlg O æ]},
(13)
where 9μ, is the gating vector that depends the task id μr, but independent of input Note that, by setting Iμ 1, we recover the vanilla model y = Wx.
The performance of this network on the 5-th task is written
1 cu [ = Ny (I g @ (A,,)))
Ny BB W WDDAA
(14)
where D2 diaglgu] is diagonal matrix whose (i, i)-th element is 9p.i. and ||-11 is the Frobenius norm.
Solution of gradient descent learning follows
Under the gradient descent learning, the weight dynamics
[W]
2n N, Bp) (B))
((I) -1 aW
(15)
From singular value decomposition (SVD). A is rewritten as D₂ = U.E.
(16)
where Up N.N. and V R.N,NN, are oomioorahoooomal matrices (i.e., UTU = VTV, = lo). En E Nox a positive diagonal matrix, N₂ is the number of non-zero singular values Dr Using this decomposition, the gradient descent dynamics is rewritten
2n N, (WU,E,, B, E k.
W(t)
(17)
Thus, the weight matrix (1) at arbitrary is written as
W(I) W(L 0) + Q())
(18)
where E N,N, is a time-dependent matrix. At the fixed point of this learning dynamics, we have
(W + QUA U,E., B,V,)) E, =0
(19)
where O the zero matrix and W W(t 0) the weight after learning of the previous task. Because Σ, is positive diagonal matrix, the equation above has unique solution:
Q (B - W Ep Em W W 111 (Le - Bn(D,A,)*
(20a)
(20b)
where
is the pseudo-inverse of A and Ia is the size N₂ identity matrix. From Eqs. 20b and 14, we have
1 N,
c W
| (V.VT 11)
(21)
Moreover, by setting the initial weight W, (the weight before the first tasks) to zero, we have
co = N ||
(22)
and W, and W2 follow
Wi = (D
W2 = (i( -U,U?) + B2(D,A))*
(23)
15

---

The error in task 2 after learning of task is thus written as
1 N
e2 e2[Wi]
B, - (.DAA)
(24)
Similarly, the error task after learning of task is
a [22l
B1 (I. (DzA1) - D B2(D2A2)* D1A1 D
(25)
N2
Note that the results so far does not rely on Assumptions I and II, making them applicable arbitrary task matrices (A, B).
The weight matrix after learning depends the pseudo-inverse, (DA)+ VΣ-LUT which is typically complicated function of the original matrix DA. However, when a.2 > No. it can be N₂ (DA) (see Eq. 101).
approximated by a scaled transpose, N.
Random and adaptive activity gating In the adaptive activity gating model, we generate input gating units {o randomly, but with correlations between subsequent tasks. We sample the gating units for the first task with gf < Bernoulliaa). where a is the sparsity the gating units. The gating
units for the + task, {g] 2+ }. are then generated by
of 0+ g (Brnoulli(a)
(with probability pp++1 (otherwise)
(26)
Here, = i is the correlation between the gating units for the μ-th and μ + 1-th tasks. The gating correlation p% is set to zero in the case of random task-dependent gating, whereas p is adjusted based on the model performance right after a model switch in the adaptive task desk-depnndent gating. To this end, we introduce probe trial where the model solves the new task using the gating vector for the old task. The error on the probe trial is
Epb
B2 Will
(27)
Ng
We then set the probability of using the same gating elements by
{ 0, Set
Pg max
(28)
where 6, = - BB the baseline error on the task keeps the gating the same if ens =
Ny
while it resamples all gating elements in case 6pb 2 fo.
Transfer performance
The average transfer performance Acrr from the first the second tasks
is:
ACTF
1
(IB2l1e)
Ny <IB= ( (DiA))+DDAall?)
1
N,
1
2tr [B] Bi (D) D2A2] - (i (B(()))))+D2aal1>>
N
2ph
N, (tr (D((
N, (I(
(29)
In the last line, we took the expectation over the correlated random matrices B1 and Bz. Using the approximation (DA)+ ai, (DA)T (Eq. 101), and then taking the expectation over AI, A2, D1. and D2. the first term becomes
(u((D)A))* DrAel) = Ny ((
aN,
N.
(1) (1)
95
(2))
=apNNs:
(30)
a.
where a represents the (j,i)-th element of A,, and ā is defined by
== Pg + (1 - pg)
(31)
16

---

The second term of Eq. 29 follows
2
N₂
aa:
N
(2)
aNo
i,
j
N,
1
9j 9t
9
2
lo + ozt + 6
2 N2
1,k
j.
NN
N. p +
aNr
1
+
N.
(32)
The third line follows from Isserlis' theorem, from which we can decompose the higher-order correlation as below:
(1) (2) (2) (1) (i Ojk Oik die Ojf (9
(1) (
( 0ix aij
(1) au
(1)
(
(
(2)
(1) (2)
(2) (1)
aik
aik
ajx
0ik
ajs ai
dz t L dz1
(33)
Summing up the equations above, at
N₂ oN= 0 limit, we have
ΔCTF apa (2pb apa).
(34)
Note that when Ag is generated by random replacement, instead of sampling from corre- lated Gaussian distribution, (I (I(()aa)** D2A2ll=) term instead follows (I((Daa)* D2A2lle) ~
N₂ @² pa
1+ ft (2 - p.a which converges to Eq. 32 at N, < N₂ limit. aN=
aN-
Retention performance Let next analyze the retention performance, ACRT [W.] ei (W2)) A, which characterizes how well the network performs the task after learning task 2. At aN2 > N₂ regime, (ei(Wol) A,B = 1. Inserting Eq. 23 into Eq. 14, we get
1
[ =
| - (BI(DIA)) - U2U3) + B2(D2A2)* |
N₂
1 [B((DIA))UUSS D1A, B2 D1 -
(35)
Ny
Thus, AeRT follows
AERT 1
( [ (1:)
Ny (IB(DIA))FOCEELAAI))) 1 (IB(CD.A))DDAA))?
1
N
N,
2
(r [BT (UDDAA)) ((DAA)))))
+ Ng
1 1 (I(D2A))+DAA111> (I(DDA))** DIAI 1F) N.
N,
+
2pb
N. (DDAA)))UUU ((DAA)))))
(36)
Because D₂AgA D2 U2E3US at the N. < aN, limit, U2U is approximated by (see Eq. 102)
N. U,U aN,
D2A2AD Dz-
(37)
17

---

A.4.75 Av08
1.49
Reedout simsbrri
Bupeo
pala BurWe
on Blerr
BWAX Busto
Retantoon
0
06 13
00
622
64
65
Feature simibrity P.
Transfer Ars
berr
Figure 8: The gating level dependence of the transfer and retention performance (A) Phase diagram of the gating level dependence. (B-D) Transfer and retention performance as function of the gating level representative point of each phase.
Using this approximation and (DA)+ aN (DA)) (Eq. 101), obtain analytical expression for Eq. 36. Taking the expectation over D and A, the second term of Eq. 36 becomes
1
N₂
(1 A2
D2
D2
N₂
a.o
ΣΣ
(1)
2
(2)
(1) (2) (2) (1) (1) (2) (1)
9j 9i
In
94
9j
In
9q aj Ojk Oik am Orm np ap Oqi
sjkl manp
N ΣΣ fjhl wang
91
94
91
(2)
In 9
22
x ( + Sepdigmm + okm Jn 8 Pa
1ōad ung + 6km + pmpd + Speden (0 P..)
+ 0
=ā -a
Pa
ON, aN.
N₂ a (4p: 2p +0
(38)
Here, we applied Isserlis' theorem again, then retained the terms up the next-to-leading order with N,
Similarly, the third term of Eq. 36 becomes respect a..-
1
.2 (tr (DLADD)DDAA&DDDDAA))))=) =á?
NN, aN, No (1+
(39)
N.
aN..
Lastly, the cross-term is evaluated as
N₂
N.
(N..)
a3N,
1
N3
ΣΣ<
" 91
(1)
9m
93
(2)
(2) 91
gn
(6IRdRmPh + durrrpa + 8indrpa + 8mtdjnpa) +
aN.a
Ivan
á (2p. p
(40)
O...)
Therefore, up to the leading order, the retention performance follows
E 1 à² pa (a² pa -2āpuPs + +1 + a.
(41)
Vanilla model In the vanilla model, we have à = Therefore, at No < N₂ limit, from Eqs. 34 and 41, the transfer and retention performance follow
Actr (2pp Pa):
(42a)
AERT 1 1p 2paPo + 1).
(42b)
Random activity gating Under the random task-dependent activity gating, the gating level â a constant. N, & aN, the transfer and retention performance are written by
AETF (2ph -apa),
(43a)
ACRT = 1 - a²p²(a" p -2apaPs + +
(43b)
18

---

Note that, this expression does not hold at the sparse limit No N.
Depending on (pa. Pb) combination, the gating level influences the transfer and retention performance in three different manners (Fig. 8). When Ps > Pa. unless both Pa and Ps large, there is monotonic tradeoff between the transfer and retention performance (Fig. 8B). In the red region where pb 2 20= + 3pe or Pb 2 2v/2 A Pa 2 /2 large gating level above threshold improves the retention performance (Fig. 8C). When ph < Pa. high gating level, or > 2. lower both transfer and Pa retention performance, thus there is no benefit of choosing large a (Fig. 8D).
3
Adaptive activity gating probe trial at the beginning of the task 2 to test the model performance for the new task. Under an adaptive gating with Pg = max 0,
the adaptive task- activity gating model, we introduced a
Spb the probe error epb follows:
1
(Epb) A.B
B2
Wi
N
A.B,3
1
N.
B2
Bi A1) D A2
N
aN,,
A.B.9
N2
1 - 2PaPb + +2o + +
(44)
aN
Thus, N₂ & aN region, Pg max {0, Pa(ph pa)}. and the effective gating level becomes
a + a) (2ps pa) (if 2ps pa) (otherwise)
(45)
a
meaning that if the transfer performance is positive in the vanilla model, there should be overlap the gating, but otherwise, the gating vector for the next task should be sampled independently.
Average error under uniform prior on task similarity Gating influences the performance differently depending on the feature and readout similarity. Thus, ideally the model should adjust the gating level based on the task similarity, but most practical settings, the model doesn't 't know the similarity priori. Therefore, it is important to analyze how it performs average under randomly chosen task similarity. To this end, here we measure the average transfer and retention performance assuming uniform prior on 0 S Pa. Pb. 1.
In the vanilla model, we have
ΔETF = 1'' dpa jo [ dpo (pa(2ps - -pa)) 1
(46a)
6'
[- dpc (1 (1-p2 [p? 2paPb 1]) AiTT [ dpa Jo
43
(46b)
60
Red and blue dash lines in Figs. 3D, 4C, 4D, SD. 6C, and 6D represent these baseline performance of the vanilla model: A & and A = 13
By contrast, under random task-dependent gating,
60
AEFF a
a
AR -
(47)
3
5
Solid lines in Fig. 3D and dashed lines in Fig. 4B plot the result above. In the case of adaptive gating, the expression of the average performance becomes complicated, yet still numerically tractable.
Note that the average transfer performance is expected to be lower than the retention performance, because forward knowledge transfer requires a high similarity. Even we use the optimal gating level for transfer, a" (Pa. Pb) min(1, P ), the average transfer performance becomes P.
S." dpa S dpeo (pa, pb) - a (PaaPt))a) 4'
(48)
19

---

B.2
Plasticity gating
An alternative strategy to gate plasticity at certain synapses desendepn manner while keeping the activity intact. We implement this method by making synapses from only a subset of input neurons plastic for each task. Given a linear regression model y W x. the learning dynamics follows:
W(t) = ((t)=-((WA) (W A,, (D,
(49)
where D, - diag(g,) (r) is a diagonal matrix specifying which synapses are gated. As in the activity gating, 9μ a gating vector whose elements take either one or zero task-dependent manner. We sample elements of 9μ from a Bernoulli distribution with probability a as before.
Let us denote SVD of A by An = UnAnV/ VT. Then, from a parallel argument with Eqs. 18 and 19, have
W W + (BA Ww
(50)
Notably, unlike Eq. 20b. gating term D₂ appears both inside and outside of the inverse term (UT) D. D,U,,)= and U/ D.). Thus, up to the leading order, gating level a does not affect the transfer and retention performance.
B.3 Input soft-thresoolding
Input-dependent gating is an another method proposed for efficient continual learning. In a regression setting, corresponds soft-thresholding of the inputs. Let introduce a soft- teeethhrehhhdngg function (p()) by
(z(x) (2) max |x| 1
(51)
where sg)( represents the sign of æ. This nonlinearity makes the analysis difficult, but if N, > 1 in addition Assumptions and II, the error becomes tractable. Note that N₂ >> assumption is required only in this subsection. Under the student network defined by
(52)
the mean-squared error follows
2 N, 1 (IW 1 N₂ [BBal') + N,
(ir [BBs<(As)
(53)
Let us denote m, Eynn, A1j8j. then we have
N. Ab R 1, (symi) Aij.
(m,) 0, (m Σ
(54)
j=1
Thus, sy and m, approximately follows joint Gaussian distribution:
(() Age
(1, 11)) 4r))
9
~
(55)
m,
Denoting A₁: = p for brevity, we have (8)p(m=))
h
dm(m + hi) + 1 dm(m -hi)
1
ds
exp
[s2 + +m² - 22psm)
2 - p²
2(1 p2)
ds
dm(m + + 1.a dm(mm - hi)
-hi
8
1 + psm + o² (s(2 - 1mmm² - 1) + O(03)
2
00
2T
52 + m
x exp
2
erfe
h V2
p +00')
(56)
20

---

In the third line, we performed Taylor expansion around = 0. Similarly, the expectation of ((mm))((m) over s written as
dm(m + hh) + 1-a dm(m - 1)) ( dm'(m' + + /.
(p((m))p(m)) = 2x
dm(m--i)
dm'(m' - h)
+ Parmmn + f (m2 10)-11mm - 11) 0(03) exp (- m2 + m
(-
x
((erfe
[ (/2) Pe + O(:)),
(57)
where P. = Eve Ank Therefore, if |p] < and Pel < 1, the error c[W] written as
2 |BI tr WWAA WT
2a
<W] = N
trAA"
(58)
Ng
N
where the imput activity sparseness a follows erfe descent from W = W. converges to
Under this loss function, gradient
v2
1
W = Wo(1 -U?) + BA*
(59)
a
where A UAVT is the singular value decomposition of A. Therefore, from W, O, the weights after task I and task 2 become
W = - Wi: W₂ = W2
(60)
where
B.A
W2 BiAt [I - U2U?] + B2A3.
(61)
This means that for both tasks (11, V/ 1,2)
C cu(Wa]
1 N₂
1 1 Ny
W
W W A.
2 N₂
tr Br W.1,
(62)
implying that the error is invariant against input sparseness a. Thus. in problem setting, input sparsification via softttheeshoding doesn't influence knowledge transfer or retention. It also implies that from energy efficiency perspective, soft-thresholding is beneficial because it reduces the number of active neurons while preserving its learning proficiency.
C Weight regularization
C.1 Weight regularization in Euclidean metric
Let us next consider a weight regularization approach by introducing L2 regularization with respect to the weight learned in the previous task. The loss function Ey for the μ-th task is then given by
Ey BB WApl? A II W 11-11
(63)
When À >> 0, there exists unique solution:
W = (B,." AW AW 21
(64)
Thus, from zero-initializztion, W, and W2 become
W1 B,AT (A,A + A
(65a)
W2 (B2A7 AB AT XL (AzA? -1
(65b)
Under N, >> N., the inverse term is approximated by
1
1
(AA" + Aa))
T
A
I +
ATA A
AT
X²
X
N
I
AAT
(66)
N₂ XN
21

---

Receoout amilnntyy
Readod simtarty
Reacout Pr
Readout simitritt
24
04
1000 rosennn
6.30
675
04
52
C4 64
06
10
62
Feoture aiborrty Pa
Feature simliarty Pr.
Festure simbarly Pa
Fasture wimberlly Pa
Figure 9: Weight regularization in Euclidean metric. (A,B) Optimal regularizer coefficient that maximizes the transfer performance (A), and the maximum performance at the optimal ? (B) under various (Pa, pb) pairs. (C,D) Optimal regularizer coefficient for retention (C), and the resultant performance (D). Panel is the same with Fig. 5C (replicated for completeness).
In the first line, we used Woodbury matrix identity, and in the second line, we used A A R I (see Eq. 99). Let us introduce normalized regularization amplitude 7 and f by
N₂ Nº + AN
N N₂ + XN,
7=
(67)
Under these approximations, W1 and W2 simplify
W1 X B (1 - -A)44) ~ BB
(68a)
W2 = x BB44 + x) (L, - -A245) = (BIAf + -244)) Bf
(68b)
Here, we further applied A A 1. Note that, unlike Eq. 20b, the result above holds only under Assumptions & II.
Transfer performance
The transfer performance AETF follows
1
( (11B211)) - N₂ -
RB AT A2 A2llF) (
AETF
Ny
25
(ur(²) A2l))
722
(I|BIAf A2
Ny
Ny
25pb
[olAfA2)]
52
N. (1AffAbl?))
No
-Pa (2pb - YPa) +0
N.
(69)
N.
In the last line, we used Eq. 33 to estimate (1IA4 A Aallzt) Notably, the expression of the transfer performance, ACTF, coincides with Eq. 5a under 7 + a, indicating that in terms of transfer performance, the weight regularization with amplitude N₂ gating with gating level a. Figs. 9A and show the optimal regularizer coefficient y that maximizes transfer performance and the resultant performance.
N_ (1 -1 is equivalent to random activity
Retention performance
The average error on task I after learning task 2 is
(E, (G]) A.B
|B| (FIBIAT +
N
1
(IBi(I-
-AA -AAAA))))))
27
B2)A3A11
Ny
No
72
(IIGBBAFAA B2)43 A11
(70)
Ny
22

---

Taking the expectation over A1. A2, B1, B2, up to the leading order with respect to No, N, we have
N2 (11: -4 A 111)
1
(IIB.I1 2 BiAf + BIA A A1ll
N₂
25 (tr[Af
N₂ (1144A11²)
-1-
N
= (1-7)² + O
(71)
N.
Similarly, we have
3
(tr
B2) A1))
N2
7
(tr [(T - (1) -2- PoT)
No
= Pan(l1 - pb) +0
No
(72)
No
and
([6BIATT - A1)
N₂
72
N. [1]AAA243 A1) - 2ipbtr[A] A2A
AAAAAAAAAA All)
= =p?? (1 - 2paP67 + P2722) + +0 N.
(73)
Therefore, the error written
(g(W))) = (1 -
2xpa (ps PPaa) + + - 2yPaPb + P²)).
(74)
The optimal y for each task similarity (Pas Pb) and the resultant retention performance are plotted in Figs. 9C and D.
C.2 Elastic weight regularization
In the previous section, we regularized the change in the weight matrix in the Euclidean space. However, previous works indicate that the weight should be regularized using the Fisher information matrix (FIM) as the metric [30, 67]. Let us construct an inference model by y = Wx + og, where is a zero mean Gaussian random variable. Given input-target pair æ; the likelihood of weight W is
p(W|,: y) oc p(x, (PW))(WW> x exp (-202
|| We - 211
(75)
20
Thus, (ij, khh component FIM becomes
@² Owsswa
log (W [z, y))
202 8z
ΣΣ ajn Oin Sm
20 5ck Σ
aan
(76)
and the quadratic weight regularization in the Fisher information metric follows
ΣΣ
8ak Σ ajn
(wsj - wiy) wLi) IIW AI)
(77)
Therefore, the loss function for this weight regularizer is written as
l 1 1Bp Bp W W. & I[W - W 1
(78)
As we will see, this loss function has different forms of solution depending on whether Pa < 1 or Pa 1. We thus consider these two conditions separately below.
23

---

C.2.1 Variable features (pa < 1)
Taking gradient with respect to W, we have
Dl
aw (B W A X(W W -n-1))A-1147
(79)
indicating that W is written as W = Wp-1 + QAT where Q is No 2N, matrix and Ā H6 aw
=
[Ap-1, Aul is an Nr x 2N, matrix. Solving
0, we get
(MA77 AA--AA Ap
A
AT
AAAA
" Ap-1
AT
Ap
(O By W Ap-1Ap)
17
p-1
(80)
Multiplying both sides with (Ap-1 An)
from the right side,
AAT
Ap-1
AT
A
AT H
An-1
AT AnT
AA Ap-1
AT
A
Ap
ATAAp
(AT_1Ap-1
(AT
AT_1Ap
AT
= (0 B wp-14
(81)
ATA
AT
X 0 and Pa < 1, two square matrices in the above equations are almost surely invertible under assumptions I & II. Therefore, we have
AAT
AT
Q (0 By
W 1A
(82)
Ap-1
AT Ap
Under N₂ >> N., from Eq. 99, the inverse matrix is approximated by
AAH Ap-1
AT AT-AA Ap
N.
Alg Apals
Pats)
N
x
Is
AATAp- A Ap
N₂
I,
N₂ p2)
-Pal_
Ig
(83)
Therefore, W WW + QÀT follows
N
N.
W = W N₂ - Am [4
N2 B₁ (A7 PA (84)
Notably, the equation above doesn't depend on the regularizer amplitude X, except for X + 0 condition. At - 0 limit, the inverse matrix term Eq. 82 effectively becomes singular, and thus the equation above no longer holds.
Let us suppose the first task is learned without any weight regularization, learned with weight regularization in the Fisher information metric imposed by an uncorrelated task. Then, we have N. BI Hence, the transfer performance becomes the same with the vanilla model. The weight after the second task becomes
W1 = N.
W2 N BIA (I-N.L -2) A2IA2 PoAft)
N. N=
N. N₂ (2)
B2(AR -P.A)). (85)
N2
Thus, the retention performance under W2 follows
(IBB1122)
ACIe
Ny
NN. BI BB) (A3 PaAT) 2
Bi (1-
N No
ATAI) + N.(1-25
N.
N
Ny
(86)
Taking expectation over Bi and B2, Aenr rewritten as
|1EE
AERT = 1 1
No
AF
A1
N
(tr (I
N. Af
(NE A2 pol) (Az - PP AIl)
NI ([---p)) p.
N.
2
N= A2 po (oAf))A 1| |1(A5 - PaAAf) llF)
N.
N-ll-pp)
20h + N-
N₂
(tr
[Af A A -PoA4)A
(87)
24

---

The second term is rewritten as
N₂ AT A1 F ) =1
2 NN (H[TAAI) + N. N: (u(AAT
N,+1
(88)
N=
Moreover, (AH pa )A1 term also cancels out up to the leading order term because
2 (I((A3 - A1
1 No
N. (1-pp)))
0j1 (a Pall (1)
( (2)
(a Pa 2 (
)ali")
N, (=(1-pa))
N2
(1)
((aft (2) PPa (1) (a - Paalik) (1))
= N, (1-pp)
Jj Oi
17kl
N.
EJ N111-
N
(89)
N.
(1-1-
sjkl
Similarly, the trace terms also cancelled out up the leading order term:
N. N₂ (x(1-pp))
)
(ur[A A1) A - Po
N.
2
Σ
(f Paaje (2)) (2) (1)
jaik ja aim (a Pa 20
N.
(2(1-12)
ijklm
N
N2
11 aik (1a)h Paajk (1) (a - Pa Paa (
(a))
(1)
N. N.(1-p8)
am n9
kjkimm
1
N,
Σ
1 N, Sjn (1 1-ppp N3 (Po + d
N, N,(1-02)
igkimmn
N.Po N.(1-P
1 No
(90)
and
Ir([I
No AAAA A AT PoT)(AT - (oAA))aa)
No(1-22) (2)
No
No onj ank pudsa) ) (aft Pa (1)
813
N. Σ
(1)
(1)
N. Σ
(1) (2)
(2)
0.
Na(1-p)) P2)
N.
mi
Eikl
(91)
Thus, we get Aegi = 1 O regardless of task similarity Pa and pb.
N₂ (.(1-p2)
Therefore, if N, < (1 Na) we have Aegr ~ 1
C.2.2 Fixed features (pa= 1)
When Pa = 1, Ap-1 = A A. The gradient follows
Øl,, = [ (B, WA) + A(W - W,-1)44]43
(92)
aW
and the weight W is written as W = Wn-1 + QUT, where U is defined by SVD A: = UAVT. at,- Solving SW O, we get
W W 1 -1- 1 UUT)
1
+
B,A
(93)
I+X
Thus, the weight after task and 2 follow
W1 1+À
B,
W₂ 1+
B
(1- UUT)
1+ B2t
(94)
25

---

Therefore, the transfer performance is
ATTF
1
(11Bal?)
Ny ( (1B2 -WIA211))
Ny
2pb
(1+ 1)2 = 7/ (2pb - ry).
(95)
1+
In the last line, defined yy by 71
1+x Similarly, the retention performance is written as
ACRT
Ny [[Bill8)
1
N₂ (I - W2
1 [BB111
Ny (I()-
(1 A)² (11
A
B
B2
Ny
1 X
2x (1 75) 7 + 2y (1-75)² - 7
(96)
If the first task is learned without weight regularization, the retention performance instead becomes:
1 Ng
(IBii1?)
N₂ (1++210 B1 B2) = - 1-22i -Pb).
ARRT
(97)
D Properties of very tall zero-mean random matrices
Our simple analytical results rely on the assumption that matrix A E NNN,N is very tall (i.e., N₂ >> N₂), and each element of sampled independently from normal distribution with mean zero and a finite variance.
Given Aij ~ NO, N.), we have A A A
N. 1..
The element-wise deviation from the mean
evaluated as
((N. A
1 1 8ij
No i/
1
(98)
N
1 implying that N₂ A
No (1 N.).
Therefore, at N2 >> N, limit, N, ATA A
N,
approximately follows
1 A A A
(99)
Even in the presence random gating, if the effective matrix is still tall (i.e., aN₂ > No), sim- ilar approximations hold. Given diagonal matrix D whose diagonal components are sampled independently from Bernoulli distribution with rate a, at aN,, >> N₂ limit, we have
(DA) N, L.
(100)
This result implies that,
(DA)+ R Nº
(DA) ON.
(101)
Moreover, denoting the SVD of DA by DA U.VT at N. > N, limit, we have
U aN.
D.ATD.
(102)
This is because, at aN N, limit, from Eq. 100, the eigenspectrum of matrix oN, DA(DA) is concentrated at À = N. and thus,
N DAATD AN NUUUPU SUUT
(103)
26

---

4-10
Readoot eimilerit Pu.
0.3
Taak Task
Trensfer
Retenton
0.0
125
T+10
1y-1
Ag-0 A-d
Epochs
Feature
simiartty P.
simiariy P.
U6 reg.reraaiee amplib.on m
(Myaar-fsse
FIM (diagonal)
W) regubarice amplituee
Retentee
18
W. www.darrrterrrooomom ampitude
W. www.fririze smpliudl
Figure 10: Permuted MNIST with latent variable. (A) Learning curves under Pb) = (0.8, 0.2). (B) Knowledge transfer under various task similarities.
(C) Comparison of the retention performance (D) Retention performance after 100 epochs of training with task 2. After 100 epochs of training, the system still exhibits strong asymmetric dependence on the feature and readout similarities, but the non-monotonic dependence on the feature similarity under Ps 1 region disappears. (E,F) Retention performance under the weight regularization the Fisher information metric (FIM) with layer-wise approximation (panel E) and diagonal approximation (panel F), under (Pa: Pb) (0.5, 0.5). Unlike Figs. 7F-H in the main figure, we adjusted the regularizer amplitude of two layers independently. As in Fig. 7, error bars in panel A-C represent standard errors 10 random seeds.
measured after 10 epochs (black) and 100 epochs (gray) of training.
E Numerical methods
Numerical experiments were conducted in standard laboratory GPUs and CPUs. Source codes for all numerical results are made publicly available at https://githut commhhiratani/traasttt retention_moder model.
E.1 Implementation linear teacher-student models
Numerical results in Figs. 2-6 and 8-9 were implemented as below. We set the latent variable dimensionality N, = 30, the input width N₂- = 3000, and the output width N, = 10. The student weight W was initialized the zero matrix, and updated with the full gradient descent with learning rate 17 = 0.001. We trained the network with the first task for Nepoch epochs, and then retrained with the second task for another N Neeooch epochs. We used Nepoch 100 for the vanilla model and the activity gating models, but we used Nepoch = 500 for other models. Unless stated otherwise, eтror bars represent standard deviation over 10 random seeds. Average performances in Figs. 3D, 4B-D, 5D, and 6CD were estimated by taking average over 100 pairs of (Pa. Pt). sampled uniformly from Pa. Pb E (0, 1).
In the input soft-thresholding model, we estimated the gradient a sample-based manner because the exact gradient is not tractable due to nonlinearity. We estimated the gradient from 10000 random samples at each iterations, then updated the model for 5000 iterations with learning 7 = 0.01.
E.2 Implementation permuted MNIST with latent
Data generation We used permuted MNIST dataset [34, 22], a common benchmark for continual learning, but with addition of the latent space. We constructed a four dimensional latent space s using the binary representation of digits, su [0, 0, 0, 0 $1 = [0, 0. O, 117 and so on. The target output was generated by projection of the latent variable s to ten-dimensional space, y B(s 1 1)) where B is 10 x 4 matrix generated randomly. The elements of matrix B, for the first task were sampled independently from a Gaussian distribution with mean zero and variance one. For the second task, we introduced readout similarity by keeping some of the elements while resampling other elements. We defined the readout similarity Ps by the fraction of the elements kept the same between two tasks.
27

---

A,
A,00.0000.
AL-055 Ay-0
A, P.-0.
(ityeewBo)) FiM (dkagores)
(byer wloe)
Thonater Bayr
Retentoo Aee
Ratentio Akge
Exolidaan
82%
689
-225
(byenwtes)
(ggannl)
Regultrize ampliade
Regularizer ampliude
Regularize amplitdde
Regularizer empliude
Figure 11: Effect of myyer-wise weight regularization in deep feedforward networks solving permuted MNIST. (A) The transfer performance under three forms of weight regularization as function of the regularization amplitude. (B-D) The same A, but the retention performance were plotted. The panels are parallel to Fig. 7E-H, but here, we used network with 3 hidden layers instead of a one-hidden layer network used for Fig. 7. When the regularizer amplitude is larger than 102, we observed instability in learning dynamics.
Feature similarity was introduced by permuting the input pixels as done previously [22]. We used the vanilla MNIST images for the first task and permuted some of the pixels in the second task depending on the feature similarity Pa-
Model implementation We used one hidden layer neural network with ReLU non-linearity in the hidden layer: y = b2 + W2ReLU [b, Wa We set the hidden layer width to N₂, 1500. The input and output widths were set to N₂ = 784 and N, 10. We initialized the first weight W1 and the bias b, by sampling each weight independently from a Gaussian distribution with mean zero We initialized the second weight W22 and the bias b2 in the same manner, but with and variance N
variance N3. Here, we set the amplitude of the initial weights small to operate the learning in the rich regime [53]. We set the mini-batch size to 300 and the learning rate to v/ = 0.01, and trained the network for 100 epochs per task. The retention performance was measured after 10 epochs of training with task except for Figs. 10 C, 10 D and 11 A-D where the retention was evaluated after 100 epochs. In the learning of task 2, the test error typically drops significantly in the first 10 epochs and shifts to gradual improvement afterward (red line in Fig. 10A). Notably, the asymmetric task similarity dependence of the retention performance was observed consistently both after a short and long training on the second task (Figs. 7B and 10D).
Weight regularization in the Fisher information metric From the same argument made in section C.2. FIM of noise-free model is approximated by the Hessian of the mean squared error. Let us consider vanilla feedforward neural network with depth K. where hidden layer activity h1...., hx-1 and the output y follow
h4 - (b, + Wehh-1), for k = I,..,, K y= bk + Wgha-1,
(104)
(105)
where @ is an element-wit wise rectified linear function. For brevity, we denote V as a diagonal matrix where the diagonal components represent element-wise derivative o' (br. Wa -1) Given loss function i yyy - "1 the Hessian is estimated as
8
ae
(Vo. -.. 100--")],, y²) hv-1,)
du Owll
O [w]
Wo WT 4+1 VOK- Wo Wp+1 V ho
Ip > v/+ Vo.W Vo,-1 Vo p+1 W yy yy']] by-
+ <] VV., Tet,W [ob WF y²)] hp-1.
M (m.0)
(106)
Gj!
In the second line, we omitted the second-order derivative d² as this term is effectively negligible under ReLU activation function.
28

---

Thus, distance between two weights oW = W W. in the metric defined by this approximated Hessian written
ΣΣ t5kt
Sw dw
(1)
(((e Ligh
Σ H.M
h OW Vo W AT.... VOR. W V VoroW,
T T "T y"
2
(107)
We define a layer-wise approximation of the weight regularization in the Fisher information metric by taking l components of the distance defined above:
R
[
(108)
Here, the expectation is taken over training data (æ,, y²) ) of the previous task (i.e., the task the network shouldn't t forget). Taking the derivative with respect oWa for k 1,-., K, have
0R
#+1 Vok- Vo. VOK- 1+1 WW hh
asw,
(Vo Vo.
Von) ow, (hp-1h
(109)
This layer-wise approximation captures the Fisher information metric more accurately than element- wise approximation implemented previously, while computationally less heavy than the estimation of the true metric. In the numerical estimations, we scaled the regularization terms the k-the layer by
Z = (VVo,, M+1 VOk- Vo..
(
(110)
where |1-112 is the spectral norm. We did not impose weight regularization the bias parameters by- In the case of synapse-wise approximation (i.e., elastic regularization), the regularizer is instead written
1
08e
R [W]
(ōutt)
2 1
(o
WT
VOK-IW VOK- Vonl 12 11-11) (6mo (111)
2
ut1
Thus, the gradient of R with respect to SW, follows
ORsw
(iag I Vok- Vo
Vor (hp-1 2 hn-1) @ oW
((W,))
(112)
We scaled the regularization term by the largest coefficient in layer-wise manner. In the of one-hidden layered networks, the layer-wise regularizer is written
1 RIw [W, Wal = 22, (1W2V08W,, 1?)
< <15W2012))
(113)
2Zz
where the normalization constants follow E1 = ||W233 ||, (<xx"))l contrast, the synapse-wise regularizer is given
Zz = [1(oo))l> By
Rew Wz]
Wki H4zz 2 (010 (1))
+
1
Σ
/3 tv (2))
(114)
27,
27,
k
13
where Zi maxij Ex (w (2)
),,,) and Zz max, (d)). To see the potential effect of different normalization methods used for the layer-wise and diagonal approximations, in Figs. 10 E and F,
29

---

Clarsiiction acouray
Crose-ertrony lees
1.3
Readout simi aimilatlyy Pi
Readout simbarity
Aayy segumel
Reedout simstarity
Rodontion Mar
92
-E.
0.
90
2.
935
1.00
similartty..
smilantt P.
simiartty
siiiarty P.
A,100
Prr32 A,40a
A,022,,000
A,055 A-05
Trancfor dkee
Dler
Retention daee
5:59
192
Retertion
A-885
P-110
A-0.0
adeptiee aandom
Waye (dlagona) Eucldeaan
(spar-stsa) (Glapoia))
E.05
Buckdean
FIM (ayer-aaa))
FIM (daggoae))
-33
12-7
10
10
Callig lovel a
Regulartber amplitude
Regularterr ampliude
Regularizer ampltude
Figure 12: Permuted MNIST with pixel and label permutations. (A,B) Transfer and classification performance measured by the classification accuracy (A) and the cross-entropy loss (B). (C) Transfer performance of random (dashed lines) and adaptive (solid lines) activity gating models. (D-F) Perfor- mance of the weight regularization in the Euclidean metric, and approximated Fisher information metrics.
independently changed the regularizer amplitude for W, and W2 and measured the retention perfor- mance under the two approximation methods. Even in this setting, the layer-wise approximation of the cesher-inffrmattin metric robustly outperformed the diagonal approximation, when the regulazier amplitude for W2 sufficiently large.
In Fig. 11, implemented this method where hidden layer widths are set to be 784-1000-333 1010000000-10--100 We used the learning rate 7) 0.01, size 300, and trained the network for 100 epochs per task. The weight regularization based on layer-wiss approximation of the Fisher-information metric, robustly outperformed both diagonal approximation of the metric and weight regularization in the Euclidean space even in this deep network (Fig. 11B-D).
feedforward network with three all-to-all hidden layers,
E.3 Implementation of permuted MNIST
In Fig. 12, we implemented permuted MNIST without any explicit latent structure. We modulated the input feature similarity by permuting subset of input pixels between the first and the second tasks, and modulated the readout similarity by permuting subset the output labels as shown in Figs. 1C and ID. We used the same one-hidden layer network with the model above, but introduced the softmax at the output units and estimated the loss using cross-entropy loss. We set the mini-batch size to be 300, and the epoch per task as 100. The retention performance was evaluated at the end of the training of the second task using Fq.
When transfer and retention performance were assessed based on differences in classification accuracy. both measures exhibited positive monotonic relationship with feature and readout similarity (Fig. 12A). However, when evaluated using cross-entropy loss, the training loss function, transfer and retention performance showed non-monotonic dependence on feature and readout similarities (Fig. 12B). Specifically, under low readout similarity, increasing feature similarity negatively impacted transfer performance.
Adaptive gating with a probe trial improved transfer performance (Fig. 12C vs. Figs. 7C and 4A). Additionally, weight regularization in the layer-wise approximation of the Fisher information metric yielded better retention performance compared to its diagonal approximation and weight regularization in Euclidean space. (Figs. 12D-F).
30

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in the fourth, fifth, and sixth paragraphs of Introduction were supported by sections 4, 5 and 6, and 7, respectively.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors? Answer: [Yes]
Justification: Please see the paragraph titled "Limitations" in section 8.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All assumptions and derivations of the theoretical results are provided in Appendix A-D.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please Appendix E.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Source codes are made publicly available at https://github. com/ httpataamm/taasseeeeetttttttooot
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see Appendix E.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please see the figures and the legends of Figs. and 6.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Please Appendix E
31

---

9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurips.cem httperrrpp..ccpppaaa/ccaiaaaaaaa
Question:
Answer: [Yes]
Justification: The research does not contain any element that requires ethical concern.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The work does not have any direct socictal impact.
Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA].
Justification: The paper does not contain any data or models that have risk for misuse.
12. Licenses for existing assets
Are the creators or original owners of assets g.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: The paper does not use existing licensed assets
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects. 15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
32


Reviews:
[{"review_id": "jUIPXAFfV0", "reviewer": "Reviewer_dowD", "summary": "The paper investigates the challenge of continual learning in artificial neural networks, particularly when learning tasks with partial similarity. Task similarity can both facilitate knowledge transfer and increase the risk of interference and catastrophic forgetting. The authors develop a linear teacher-student model with latent structure to analyze how input feature similarity and readout pattern similarity affect knowledge transfer and forgetting. They find that high input feature similarity with low readout similarity is detrimental, while the opposite is relatively benign. The study also explores how different continual learning algorithms, such as task-dependent activity gating and weight regularization based on the Fisher information metric, interact with task similarity.", "questions": "Please refer to the Weaknesses above.", "limitations": "Please refer to the Weaknesses above.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper stands out for its originality in several key areas: 1) Problem Formulation: The nuanced exploration of task similarity's impact on continual learning is a critical perspective. This focus on partial task similarity and its dual role in knowledge transfer and interference is novel and highly relevant. 2) Analytical Framework: The use of a linear teacher-student model with latent structure to dissect the effects of task similarity is an innovative approach. This model provides a clear, theoretical basis for understanding complex interactions in continual learning.\nThe quality of the research is evident in multiple dimensions: 1) Theoretical Rigor: The paper offers an analytical examination of task similarity's effects, supported by theoretical foundations. The linear teacher-student model is well-articulated and used to derive insightful conclusions. 2) Methodological Soundness: The evaluation of different continual learning algorithms, including task-dependent activity gating and weight regularization based on the Fisher information metric, is methodologically sound. These evaluations are carefully designed to highlight the algorithms' interactions with task similarity. 3) Empirical Validation: The empirical results on the permuted MNIST dataset provide strong support for the theoretical findings. The experiments are well-executed, with results that convincingly demonstrate the practical implications of the theoretical insights.", "weaknesses": "1. Limited Scope of Experimental Validation\nWhile the paper provides theoretical insights and validates them on the permuted MNIST dataset, the experimental scope is somewhat limited. Validate the findings on a wider variety of datasets, including more complex and diverse datasets such as CIFAR-100, ImageNet, or non-visual tasks like language modeling or reinforcement learning environments. This would strengthen the generalizability of the results.\n2. Insufficient Comparison with Existing Methods\nThe paper lacks a comprehensive comparison with a wider range of existing continual learning algorithms, particularly recent advancements in the field. Compare the proposed methods and insights with more recent state-of-the-art continual learning algorithms [1-6]. This includes methods that have been introduced in the past year, ensuring the evaluation is up-to-date.\n[1]A comprehensive survey of continual learning: theory, method and application, 2024\n[2]Lee S, Goldt S, Saxe A. Continual learning in the teacher-student setup: Impact of task similarity[C]//International Conference on Machine Learning. PMLR, 2021: 6109-6119.\n[3]Lin S, Ju P, Liang Y, et al. Theory on forgetting and generalization of continual learning[C]//International Conference on Machine Learning. PMLR, 2023: 21078-21100.\n[4]TRGP: Trust Region Gradient Projection for Continual Learning (ICLR ‘22)\n[5]The Ideal Continual Learner: An Agent That Never Forgets (ICML ‘23)\n[6]A Theoretical Study on Solving Continual Learning (NeurIPS ‘22)\n3. Limited Discussion on Practical Implementation\nThe paper primarily focuses on theoretical insights and empirical validation but provides limited guidance on the practical implementation of the proposed methods. Include a section or supplementary material that provides detailed guidelines for implementing the proposed methods in practical scenarios. This could include code snippets, hyperparameter settings, and best practices.\n4. Lack of In-Depth Analysis of Algorithm Interactions\nWhile the paper evaluates task-dependent activity gating and weight regularization, it does not provide an in-depth analysis of how these methods interact with each other and with different levels of task similarity. Conduct experiments to analyze the interaction effects between different continual learning algorithms and varying degrees of task similarity. This would provide a deeper understanding of the conditions under which these methods are most effective.\n5. Clarification on Theoretical Assumptions\nThe theoretical analysis is based on specific assumptions that may not always hold in practical scenarios. The paper could benefit from a more detailed discussion of these assumptions and their implications. Clearly state the assumptions underlying the theoretical model and discuss their limitations. Provide insights into how these assumptions might impact the applicability of the findings in real-world settings. Perform robustness analysis to test the sensitivity of the results to deviations from these assumptions. This could involve experimenting with variations in model architecture, data distribution, and task difficulty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments. Please find our replies to your comments on the weaknesses below.\n\n1. In the attached figure, we demonstrated numerically that the feature and readout similarity influence the continual learning performance in a manner predicted by our theory even in a classification setting. We leave its application to more complex datasets and network architectures to future work. \n\n2. We would like to seek clarification regarding this suggestion of comparing our work with “recent state-of-the-art continual learning algorithms [1-6]” and ensure our evaluation is up-to-date. Specifically, among the papers you cited, the first one is a review, while citations 2, 3, 5, and 6 are theoretical works that do not propose new algorithms. Could you please provide more details or specify the aspects of our work you believe need to be compared with these references?\n\n3. Although proposing a new algorithm is not our primary focus, we did introduce an alternative approximation of weight regularization in the Fisher information metric that does not rely on diagonal approximation, unlike the elastic weight algorithm (Kirkpatrick et al., PNAS, 2017). We observed that this algorithm outperforms the elastic weight regularization method in a one-hidden-layer neural network when solving the MNIST task (Fig. 6E-H). While we presented the derivation of the alternative approximation specifically for a one-hidden-layer neural network (Eqs. 105-108 in Appendix E), this approach is applicable to fully-connected feedforward neural networks of any depth. We will provide a more general formulation in the revised manuscript to help practical implementation of the algorithm. Application of this approximation method to other neural architectures, such as convolutional, recurrent, or self-attention networks, is non-trivial and therefore will be addressed in future work.\n\n4. While we did not explore the interaction between activity gating and weight regularization in this work, one interesting finding we obtained is their interchangeability in terms of knowledge transfer. Specifically, we demonstrated that Euclidean weight regularization with amplitude $\\frac{N_x}{N_s} \\left( \\frac{1}{\\gamma} - 1 \\right)$ is equivalent to random activity gating with sparsity $\\gamma$ (please see Sec. 6.1 for details). Here, $N_x$ and $N_s$ represent the widths of the input layer and latent source, respectively. This result suggests that combining gating and regularization is unlikely to provide additional benefits for knowledge transfer, but it may help prevent forgetting.\n\n5. In Appendix A, we listed the assumptions introduced in our analysis. The first assumption, the random task assumption, posits that inputs and targets are generated randomly with a specific correlation structure. The second assumption, the low-dimensional latent assumption, imposes the existence of a low-dimensional latent space that generates both inputs and target outputs. Our derivation holds asymptotically, assuming that the input dimensionality is significantly larger than that of the latent space. We will provide further clarification in the main text of the revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tn3MzaRxf3", "reviewer": "Reviewer_Uj69", "summary": "The paper systematically investigated how feature similarity and readout similarity affect knowledge transfer and retention, under different gating scenarios, showing weight regularization based on Fisher information metric improves retention without compromising transfer. These are done with both linear teacher-student models and permuted MNIST task.", "questions": "1) Could the authors please explain what they mean by 'element-wise correlation' for $\\rho_a$ and $\\rho_b$? When reading the task sampling procedure in appendix A.1, $\\rho_a$ and $\\rho_b$ are the probability of entries in the mixing matrices being identical; when deriving the analytical solutions, they seem to mean column-wise correlation? I could see either version serving as a working definition for feature similarity/readout similarity but just getting confused on having different definitions of key concepts in one paper. \n2) Relatedly, could the authors comment on/discuss how different similarity definitions affect the conclusions in a broader setting? A third similarity definition seems to be used for the permuted MNIST task but similar trend could be observed. It seems worthwhile to explicitly address the stringiness of similarity definition in the discussion.", "limitations": "See question.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The paper addressed an important question in continual learning - task similarity - with very comprehensive experiments and successfully delineated the effect of several factors clearly; 2) the paper is clearly written; 3) although the bulk of the paper is done on linea teacher-student setting, the authors were able to solidify the majority of their results with permuted MNIST task.", "weaknesses": "I'm confused on the paper's definition of 'similarity' in general, for both feature similarity $\\rho_a$ and readout similarity $\\rho_b$. Since they are key concepts in the paper, they seem to deserve more careful treatment. See question.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments. \n\n__Weaknesses:__\nIn panel A of the attached additional figure, we clarified the definitions of feature similarity and readout similarity in a classification setting. The green point in the left panel represents a scenario where two tasks have low feature similarity and high readout similarity: the input pixels are partially permuted while the output labels remain unchanged (Panel A, top). In contrast, the orange point indicates a scenario where tasks have high feature similarity and low readout similarity: the input pixels stay the same, but the output labels are partially permuted. The partial permutation of input pixels was achieved by randomly selecting a subset of input pixels with a probability of $1-\\rho_a$, and then randomly permuting these selected pixels while keeping the remaining pixels fixed. The output permutation was performed in the same manner. A partial shift in target labels is expected to be common in real-world continual learning tasks in dynamic environments. Therefore, we believe it is crucial to understand how readout similarity influences continual learning performance alongside feature similarity.\n\nIn panels B and C, we assessed the performance of knowledge transfer and retention for a continual classification task with partial input and label permutations. For this assessment, we employed the one-hidden layer model from Fig. 6 of the manuscript, with some modifications: a softmax function was added to the final layer, and cross-entropy loss was used for gradient calculation instead of mean-squared error. We observed that, even in this permuted continual classification task, our theory qualitatively explains the task similarity dependence of knowledge transfer and retention when they are measured by the cross-entropy loss (Panel C).\n\n__Question 1:__\nThank you for pointing out this issue. The original manuscript indeed lacked clarity on this point. In our analytical estimation, we assumed that the elements of the input-projection matrices $A_1$ and $A_2$ were sampled jointly from a correlated Gaussian distribution. However, in our numerical verification, we generated $A_1$ and $A_2$ differently: we first sampled $A_1$ and then generated $A_2$ by replacing randomly selected elements of $A_1$ with independently sampled values from the same distribution, as depicted in Equation 10. Despite these differing sampling methods, they produce the same macroscopic behaviors due to the large $N_x$ assumption. For example, under Equation 10, the next-order term of Equation 32 changes slightly, resulting in the expression:\n$$\\langle \\lVert (D_1 A_1)^+ D_2 A_2 \\rVert^2_F \\rangle \\approx\nN_s \\left( \\tilde{\\alpha}^2 \\rho_a^2 + \\frac{\\tilde{\\alpha} N_s}{\\alpha N_x} \\left[ 1 + \\frac{\\rho_a}{N_s} (2 - \\rho_a \\alpha \\tilde{\\alpha}) \\right] \\right)$$\nNevertheless, because the leading-order terms remain unchanged, our analytical results still align with the numerical results. We will clarify this point in the revised manuscript.\n\n__Question 2:__\nOur empirical results indicate that the derived analytical expression is robust against the choice of probability distribution for sampling matrices $A_1$ and $A_2$, provided they have zero mean and fixed covariance. However, higher-order moments appear in the analytical estimations of knowledge transfer and retention. Consequently, it remains uncertain whether the obtained result has a universality. \n\nPlease note that the partial permutation introduced to MNIST tasks is methodologically similar to the resampling method depicted in Equation 10. Specifically, in the classification problem shown in the additional figure, we can set the latent space to be a 10-dimensional label space. In this scenario, the target projection matrix for the first task $B_1$ is a 10-dimensional identity matrix, while the matrix for the second task $B_2$ is generated by permuting a randomly selected submatrix of $B_1$.\n\nWe will elaborate on these points in the discussion section of the revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rG8hW1Ullv", "reviewer": "Reviewer_LdFf", "summary": "This paper analyzes the impact of task similarity on Continual Learning (CL) within a linear teacher-student model that incorporates low-dimensional latent structures. The findings indicate that high input feature similarity combined with low readout similarity is detrimental to both knowledge transfer and retention", "questions": "Please refer to the weaknesses.", "limitations": "The authors have included a limiation discussion in Sec. 8.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper is well-motivated, addressing the important problem of understanding the impact of task similarity on continual learning.\n+ The authors provide a thorough analysis of the impact of task similarity on CL using a linear teacher-student model with low-dimensional latent structures. The results demonstrate that high feature similarity and low readout similarity are catastrophic for both knowledge transfer and retention.\n+ The paper tests its predictions numerically using the permuted MNIST task, further validating the theoretical findings.\n+ The authors provide the code, which enhances the reproducibility of their results.", "weaknesses": "+ The current theory is constrained to task incremental CL involving two regression tasks. It would be beneficial to explore whether the theory can be generalized to more tasks or other settings, such as classification tasks.\n+ How can readout similarity be understood in other CL tasks, such as classification tasks?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments.\n\nWe have included an additional figure to clarify the concept of readout similarity in a classification setting and to demonstrate the applicability of our framework to such settings. The lower half of panel A illustrates two tasks with low readout similarity. In the first task, we use the standard MNIST dataset, where digit images are classified into one of ten labels. For the second task (of a two-task continual learning), we permute half of the labels, resulting in a low readout similarity between the two tasks ($\\rho_b=0.5$). Although the partial label permutation introduced here is somewhat artificial, a partial shift in target labels is expectedly a common scenario in real-world continual learning tasks in dynamic environments. Therefore, we believe it is crucial to understand how feature and readout similarity influence continual learning performance.\n\nIn panels B and C, we evaluated the knowledge transfer and retention performance under this continual classification task with partial input and label permutations. For this evaluation, we used the one-hidden layer model depicted in Fig. 6 of the manuscript, with modifications: we added a softmax function in the last layer and employed cross-entropy loss for gradient calculation instead of mean-squared error.\n\nPanel B illustrates the knowledge transfer and retention between the two tasks based on changes in classification accuracy, while panel C shows the transfer and retention performance based on changes in cross-entropy loss between the network output and the target. Since classification accuracy is effectively lower-bounded by the chance level, panel B did not exhibit any negative impact of high feature similarity. However, when measuring performance by changes in cross-entropy loss, we observed results that align qualitatively with our theoretical predictions (panel C vs. Figs. 1D and 1F in the manuscript). Specifically, the combination of high feature similarity and low readout similarity resulted in negative knowledge transfer and retention. These effects were more pronounced when feature similarity was higher, given a fixed readout similarity, as predicted by our theory. Since the gradient was calculated using cross-entropy loss in the model, this observed effect is relevant to the continual learning of classification tasks, implying that our results are generalizable beyond regression conditions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thank you all for your valuable comments. Based on your feedback, we have added a figure that explains the feature and readout similarity in the context of a classification task (panel A) and shows their impact on knowledge transfer and retention (panels B and C). We found that when performance is measured by cross-entropy loss, our theoretical prediction explains the task similarity dependence of transfer and retention performance qualitatively even in a classification problem. Please see the individual replies for further details.

Meta-review: This paper systematically investigates the impact of **feature similarity** and **readout similarity** on knowledge transfer and retention under different gating scenarios. It demonstrates that weight regularization based on the Fisher information metric can enhance retention without sacrificing transfer. The study is conducted using both linear teacher-student models and the permuted MNIST task.

The reviewers unanimously agree that the paper is well-motivated, with a novel theoretical formulation for the community. The analyses provided are thorough and valuable for understanding knowledge transfer dynamics. Given its contributions, the paper is considered above the acceptance threshold.

However, to further improve the paper's quality, the authors are requested to (1) provide a detailed clarification on the concepts of feature similarity and readout similarity to enhance understanding and (2) include additional experimental results in classification settings to validate the findings further.
Decision: Accept (poster)