Title: Disentangling and mitigating the impact of task similarity for continual learning
Venue: NeurIPS 2024
Accepted: yes
Keywords: Continual learning

Teacher-student model

Lifelong learning

Brain-inspired
Abstract: Continual learning of partially similar tasks poses a challenge for artificial neural networks, as task similarity presents both an opportunity for knowledge transfer and a risk of interference and catastrophic forgetting.
However, it remains unclear how task similarity in input features and readout patterns influences knowledge transfer and forgetting, as well as how they interact with common algorithms for continual learning.
Here, we develop a linear teacher-student model with latent structure and show analytically that high input feature similarity coupled with low readout similarity is catastrophic for both knowledge transfer and retention. 
Conversely, the opposite scenario is relatively benign. 
Our analysis further reveals that task-dependent activity gating improves knowledge retention at the expense of transfer, while task-dependent plasticity gating does not affect either retention or transfer performance at the over-parameterized limit. 
In contrast, weight regularization based on the Fisher information metric significantly improves retention, regardless of task similarity, without compromising transfer performance. Nevertheless, its diagonal approximation and regularization in the Euclidean space are much less robust against task similarity. 
We demonstrate consistent results in a permuted MNIST task with latent variables. Overall, this work provides insights into when continual learning is difficult and how to mitigate it.

Disentangling and mitigating the impact of task similarity for continual learning

Abstract

Continual learning of partially similar tasks poses a challenge for artificial neural networks, as task similarity presents both an opportunity for knowledge transfer and a risk of interference and catastrophic forgetting. However, it remains unclear how task similarity in input features and readout patterns influences knowledge transfer and forgetting, as well as how they interact with common algorithms for continual learning. Here, we develop a linear teacher-student model with latent structure and show analytically that high input feature similarity coupled with low readout similarity is catastrophic for both knowledge transfer and retention. Conversely, the opposite scenario is relatively benign. Our analysis further reveals that taskdependent activity gating improves knowledge retention at the expense of transfer, while task-dependent plasticity gating does not affect either retention or transfer performance at the over-parameterized limit. In contrast, weight regularization based on the Fisher information metric significantly improves retention, regardless of task similarity, without compromising transfer performance. Nevertheless, its diagonal approximation and regularization in the Euclidean space are much less robust against task similarity. We demonstrate consistent results in a permuted MNIST task with latent variables. Overall, this work provides insights into when continual learning is difficult and how to mitigate it.

1 Introduction

Artificial neural networks surpass human capabilities in various domains, yet struggle with continual learning. These networks tend to forget previously learned tasks when trained sequentially-a problem known as catastrophic forgetting [44, 16, 23, 32]. This phenomenon affects not only supervised training of feedforward networks but also extends to recurrent neural networks [10], reinforcement learning tasks [30], and fine-tuning of large language models [41]. Many algorithms for mitigating catastrophic forgetting have been developed previously, including rehearsal techniques [48, 49, 59], weight regularization [30, 36, 67], and activity-gating methods [15, 54, 42, 56], among others [57, 50, 63, 21]. However, these methods often hinder forward and backward knowledge transfer [28, 39, 29], and thus it remains unclear how to achieve knowledge transfer and retention simultaneously.

A key factor for continual learning is task similarity. If two subsequent tasks are similar, there is a potential for a knowledge transfer from one task to another, but the risk of interference also becomes high [47, 28, 35, 11, 39]. The impact of task similarity on transfer and retention performance is particularly complicated because two tasks can be similar in different manners [65, 32]. Sometimes familiar input features need to be associated with novel output patterns, but at other times, novel input features need to be associated with familiar output patterns. Previous works observed that these

two scenarios influence continual learning differently [35], yet the impact of the input and output similarity on knowledge transfer and retention has not been well understood. Moreover, it remains unknown how the task similarity interacts with algorithms for continual learning such as activity gating or weight regularization.

To gain insight into these questions, in this work, we investigate how transfer and retention performance depend on task similarity, task-dependent gating, and weight regularization in analytically tractable teacher-student models. Teacher-student models are simple, typically linear, neural networks in which the generative model of data is specified explicitly by the teacher network [18, 66, 3]. These models have provided tremendous insights into generalization property [55, 45, 19, 1], convergence rate [61, 58, 38], and learning dynamics [51, 52, 4, 25] of neural networks, due to their analytical tractability. Several works also studied continual learning using teacher-student settings [2, 35, 27, 11, 20, 37, 13] (see Related works section for details).

We develop a linear teacher-student model with a low-dimensional latent structure and analyze how the similarity of input features and readout patterns between tasks affect continual learning. We show analytically that a combination of low feature similarity and high readout similarity is relatively benign for continual learning, as the retention performance remains high and the transfer performance remains non-negative. However, the opposite, a combination of high feature similarity and low readout similarity is harmful. In this regime, both transfer and retention performance become below the chance level even when the two subsequent tasks are positively correlated. Furthermore, transfer performance depends on the feature similarity non-monotonically, such that, beyond a critical point, the higher the feature similarity is, the lower the transfer performance becomes.

We further analyze how common algorithms for continual learning, activity and plasticity gating [15, 42, 46], activity sparsification [57], and weight regularization [30, 36, 67], interact with task similarity in our problem setting, deriving several non-trivial conclusions. Activity gating improves retention at the cost of transfer when the gating highly sparsifies the activity, but helps both transfer and retention on average if the activity is kept relatively dense. Plasticity gating and activity sparsification, by contrast, do not influence either transfer or retention performance at the over-parameterized limit. Lastly, weight regularization in the Fisher information metric helps retention without affecting knowledge transfer and achieves perfect retention regardless of task similarity in the presence of low-dimensional latent. However, its diagonal approximation and the regularization in the Euclidean metric are much less robust against both task similarity and regularizer amplitude.

Furthermore, we test our key predictions numerically in a permuted MNIST task with a latent structure. When the input pixels are permuted from one task to the next, the retention performance remains high. However, when the mapping from the latent to the target output is changed, both the retention and transfer performance go below the chance level, as predicted. Random task-dependent gating of input and hidden layer activity improves retention at the cost of knowledge transfer, but adaptive gating mitigates this tradeoff. We also show that in a fully-connected feedforward network, there exists an efficient layer-wise approximation of the weight regularization in the Fisher information metric, which outperforms its diagonal approximation and the regularization in the Euclidean metric. Nevertheless, the performance of the diagonal approximation is much closer to the layer-wise approximation of the Fisher information metric than to the Euclidean weight regularization.

Our theory thus reveals when continual learning is difficult, and how different algorithms mitigate these challenging situations, providing a basic framework for analyzing continual learning in artificial and biological neural networks.

2 Related works

Previous works on continual learning in linear teacher-student models found that forgetting is most prominent at the intermediate task similarity [11, 40, 13] as observed empirically [47]. However, these works did not address the tradeoff between forgetting and knowledge transfer, and these simple settings did not disentangle the effect of the similarity in input feature and readout pattern. Forward and backward transfer performance in continual learning were also analyzed in linear and deep-linear networks [33, 8], yet its relationship with catastrophic forgetting has not been well characterized. Lee et al. [35] analyzed dynamics of both forgetting and forward transfer in one-hidden layer nonlinear network under a multi-head continual learning setting. However, their analysis of readout similarity

0

1

2

3

4

5

6

7

8

9

0

1

2

3

4

5

6

7

8

9

9

1

3

6

4

5

0

7

8

2

B

Task 2

0

1

2

3

4

5

6

7

8

9

Task 1

Task 2

Feature similarity

!

!

Readout similarity

!

"

0.0

1.0

1.0

0.0

C

A

Task 1

D

and its comparison to feature similarity were conducted numerically and it did not address the effect of common heuristics, such as gating or weight regularization, either.

By comparison, we introduce a low-dimensional latent structure into a linear teacher-student model, which enables us to decouple the influence of feature and readout similarity on knowledge transfer and retention. Moreover, this low-dimensionality assumption on the latent enables us to evaluate the transfer and retention performance analytically even in the presence of gating or weight regularization in the Fisher information metric.

Continual learning has been studied from many theoretical frameworks beyond teacher-student modeling, including neural tangent kernel [5, 9, 27], PAC learning [6, 60], and computational complexity [31]. Learning of low-dimensional latent representation has also been studied in the context of multi-task learning [43, 62]. In addition, several works investigated the effect of weight regularization on continual learning in analytically tractable settings [30, 12, 24]. In particular, Evron et al. [12] investigated effect of weight-regularization in Fisher-information metric in continual linear regression scenario.

3 Teacher-student model with low-dimensional latent variables

Let us consider task-incremental continual learning of regression tasks. For analytical tractability, we consider a teacher-student setting where the target outputs are generated by the teacher network (Fig. 1A). We define the student network, which learns the task, as a linear projection from a potentially nonlinear transformation of the input, y = Wψ ( x ) , where x ∈ R N x is the input, y ∈ R N y is the output, W ∈ R N y × N x is the trainable weight matrix, and ψ ( x ) : R N x → R N x is an input transformation. We use ψ ( x ) = x for the vanilla and weight regularization models, ψ ( x ) = g ⊙ x for the task-dependent gating model, and ψ ( x ) = sgn ( x ) ⊙ max { 0 , | x |-h } for the soft-thresholding model, where g and h are gating and thresholding vectors, respectively. Here, g ⊙ x is an elementwise multiplication, and sgn ( x ) is a function that returns the sign of the input. Throughout the paper, we use bold-italic letters for vectors, capital-italic letters for matrices.

We generate the input x and target output y ∗ of the µ -th task using a latent variable s ∈ R N s :

for µ = 1 , 2 , where A µ ∈ R N x × N s and B µ ∈ R N y × N s are mixing matrices hidden from the student network, and I s is the size N s identity matrix. We introduce this latent structure, s , to decouple the effect of feature and readout similarity on the transfer and retention performance. Below, we set the latent space to be low-dimensional compared to the input space (i.e., N s ≪ N x ). This is motivated by the presence of low-dimensional latent structure in many machine learning datasets [64, 7] and the tasks used in neuroscience experiments [17, 26], but also aids analytical tractability.

We generate the mixing matrices for the first task, A 1 and B 1 , by sampling elements independently from a Gaussian distribution with mean zero and variance 1 N s . The subsequent task matrices A 2 and B 2 are also generated randomly, but with element-wise correlation with the previous matrices A 1 and B 1 (see Appendix A for the details). We denote the element-wise correlation between A 1 and

A 2 by ρ a and refer it as feature correlation, because A 1 and A 2 specify the input features. Similarly, we denote the correlation between B 1 and B 2 , the readout correlation, by ρ b . Figs. 1B-D illustrate the difference between ρ a and ρ b in an image classification setting. When ρ a < 1 . 0 and ρ b = 1 . 0 , input features are partially modified in the second task compared to the first task (Green point on Fig. 1B and Fig. 1C). For example, permuted MNIST task [22] corresponds to this low feature similarity setting with ρ a = 0 . 0 and ρ b = 1 . 0 . By contrast, when ρ a = 1 and ρ b < 1 , the readouts are changed partially (Orange point on Fig. 1B and Fig. 1D). When ρ a = 1 and ρ b = -1 , the same input features are associated with the opposite readout in the new task, which is known as the reversal learning paradigm. In this reversal scenario, the network fails to achieve knowledge transfer rather trivially. Thus, we focus on the scenario when tasks have non-negative correlation in terms of both input features and readout (i.e., 0 ≤ ρ a , ρ b ≤ 1 ) below.

We measure the performance of the student network with weight W on the µ -th task by mean-squared error ϵ µ [ W ] ≡ 1 N y 〈 ∥ B µ s -Wψ ( A µ s )] ∥ 2 〉 s , where ⟨·⟩ s is the expectation over latent variable s . The transfer and retention performance are defined by

Here, ⟨·⟩ A,B is the expectation over randomly generated task matrices A 1 , A 2 , B 1 and B 2 under a given feature and readout correlation ρ a , ρ b . As illustrated in Fig. 2A, the transfer performance ∆ ϵ TF measures how much performance the model achieves on task 2 by learning task 1, whereas the retention performance ∆ ϵ RT measures how well the model can perform task 1 after learning task 2 (here, the task switch occurs at the 100th iteration). Below, we study how knowledge transfer and retention, the two key objectives of continual learning, depend on task similarity, and how to optimize the performance through gating and weight regularization.

4 Impact of task similarity on knowledge transfer and retention

Let us first investigate the vanilla model without gating or weight regularization, to examine how task similarity affects knowledge transfer and retention. Given ψ ( x ) = x , at the infinite sample limit, learning by gradient descent follows ˙ W = -( WA µ -B µ ) A T µ . The fixed point of this dynamics, as detailed in Appendix B.1, is:

where W µ -1 is the weight after learning of the previous task, U µ is the semi-orthogonal matrix from singular value decomposition (SVD) of A µ = U µ Λ µ V T µ , and A + µ is the pseudo-inverse of matrix A µ .

Inserting Eq. 3 into Eq. 2 and taking the expectation over randomly generated tasks A 1 , B 1 , A 2 , B 2 , the transfer and retention performance are written as (see Appendix B.1)

Recall that ρ a is the feature similarity defined by the correlation between A 1 and A 2 , while ρ b is the readout similarity, representing the correlation between B 1 and B 2 . The derivation of the above equations relies on (correlated) random generation of A 1 , B 1 , A 2 , B 2 and the low-rank latent assumption: N s ≪ N x . These two equations, despite their simplicity, capture the transfer and retention performance in numerical simulations well (Figs. 2B, D, and F; see Appendix E.1 for the details of numerical estimation). Furthermore, they reveal asymmetric and non-monotonic impact of the feature and readout similarity on the performance.

Fig. 2B depicts the knowledge transfer from task 1 to task 2, ∆ ϵ TF , under various ( ρ a , ρ b ) conditions. As expected, ∆ ϵ TF = 0 when the two tasks are independent (i.e., ( ρ a , ρ b ) = (0 , 0) ), and ∆ ϵ TF = 1 when the tasks are identical (i.e., ( ρ a , ρ b ) = (1 , 1) ). At intermediate levels of similarity, there is clear asymmetry in the influence of feature and readout similarities on transfer performance; particularly, a combination of high feature similarity and low readout similarity leads to negative transfer, while the opposite scenario results in a modest positive transfer (lower-right vs upper-left of Fig. 2B).

Notably, under a fixed readout similarity ρ b , the knowledge transfer depends non-monotonically on the feature similarity ρ a . When ρ a < ρ b , the higher the feature similarity the better transfer becomes, as implied from Eq. 4. However, once the feature similarity exceeds the readout similarity, counter-intuitively, high feature similarity worsens the transfer performance (Figs. 2B and C). This is because, when the feature similarity is high, inputs are aligned with learned features, resulting in a large output regardless of readout similarity. Particularly, under a low readout similarity, performance becomes below zero because extracted features are mostly projected to the incorrect directions.

The retention performance also exhibits asymmetric dependence on feature and readout similarity. When feature similarity ρ a is low, the network barely forgets the previous task regardless of readout similarity (Fig. 2D left). By contrast, when the feature similarity is high, the retention performance can either be positive or negative depending on the readout similarity. Moreover, in the high readout similarity regime, the retention performance is the lowest at an intermediate feature similarity (Figs. 2E and F). This is because high retention is possible when either interference is low, or similarity between two tasks is high. Note that, this last point on the non-monotonic dependence on feature similarity under ρ b = 1 has been investigated both empirically [47] and analytically [35, 11, 13]. Indeed, at ρ b = 1 , ∆ ϵ RT in Eq. 4 coincides with Eq. (5) in [13].

Therefore, the knowledge transfer and retention performance depend on feature and readout similarities in an asymmetric and non-monotonic manner. Specifically, a combination of high feature similarity and low readout similarity is detrimental to continual learning, resulting in negative transfer and retention performance, even in the presence of a positive correlation between the two tasks in both input features and readout patterns. Moreover, under a fixed readout similarity, the knowledge transfer performance depends on the feature similarity in a non-monotonic manner. Thus, the continual learning in the vanilla model is sensitive to task similarity and limited in performance. These results make us wonder whether we can mitigate the sensitivity to task similarity and improve the overall transfer and retention performance by modifying the learning algorithm. To this end, we next investigate how task-dependent gating and weight regularization methods, two popular strategies in continual learning, alleviate knowledge transfer and retention.

5 Task-dependent gating

5.1 Random activity gating

One popular method for mitigating forgetting in continual learning is activity gating [15, 54, 42, 21, 56]. With gating of input activity, the network is written as y = W ( g µ ⊙ x ) , where g µ ∈ { 0 , 1 } N x is a binary gating vector for task µ . We first consider a random gating scenario where elements of g µ are randomly sampled from a Bernoulli distribution with rate α which we denote as the gating level. All units are active at α = 1 , while no units are active at α → 0 limit.

From a parallel argument with the vanilla model, when N s ≪ αN x , the transfer and retention performance are estimated as (see Appendix B.1):

Thus, random gating scales the feature similarity from ρ a to αρ a . This scaling lowers the knowledge transfer if ρ b ≥ ρ a because random gating reduces the fraction of input neurons active in two subsequent tasks (lime line in Fig. 3A). However, if ρ b < ρ a , gating with α ≥ ρ b ρ a enhances the transfer by reducing the effective feature similarity (blue lines in Fig. 3A; also compare Fig. 3B with Fig. 2C). The optimal gating level α ∗ for knowledge transfer is min { ρ b ρ a , 1 } , indicating that the input activity typically needs to be dense. By contrast, the retention performance is optimized at α → 0 limit where tasks do not interfere with each others (Fig. 3C). At this limit, we have ∆ ϵ RT → 1 . Note that, in reality, α has to be non-zero to optimize the retention (Eq. 5b holds only when α ≫ N s N x ).

These results indicate that random activity gating improves the retention at the cost of forward knowledge transfer, as suggested previously [28, 39, 29]. This tradeoff between knowledge transfer and retention is especially critical when the network doesn't know the task similarities. Suppose that the task similarity is distributed uniformly over 0 ≤ ρ a , ρ b ≤ 1 . Then, the average transfer performance is maximized at α = 3 4 , while the average retention performance decreases monotonically as a function of α (Fig. 3D). Thus, a moderate gating ( α ≈ 3 4 ) benefits both transfer and retention on average. However, retention performance in this regime is significantly below one, implying that the network cannot reliably achieve high retention performance without sacrificing the transfer performance.

5.2 Adaptive activity gating

One way to overcome the tradeoff between transfer and retention performance is to consider adaptive gating. Let us introduce a probe trial at the beginning of the task 2, in which the model tests how well it performs if it uses the gating for task 1, g 1 , for task 2 as well. If the error in the probe trial is small, the model should keep using the same gating vectors to achieve a good knowledge transfer. Otherwise, it should resample the gating vector for retaining the knowledge on task 1. Using the

probe error ϵ pb , we set the probability of using the same gating as ρ g = 1 -ϵ pb /ϵ o . This adaptive gating indeed improves the average transfer performance compared to the random gating (solid vs dashed lines in Fig. 4A) with only a relatively small reduction in the retention performance (Fig. 4B).

5.3 Plasticity gating and input soft-thresholding

Previous works have also explored other gating mechanisms, such as plasticity gating [46] and activity sparsification [57]. However, their benefits over the activity gating discussed above have not been fully understood. We implement task-dependent plasticity gating into our problem setting by making only the synaptic weights projected from a subset of input neurons plastic. Unexpectedly, we found that both transfer and retention performance are independent of the fraction of plastic synapses in the N s ≪ N x limit, potentially due to over-parameterization (Fig. 4C; see Appendix B.2 for details).

Similarly, when the input activity is sparsified using soft-thresholding φ ( x ) = sgn ( x ) max { 0 , | x | -h } , with a fixed threshold h = √ 2 erfc -1 ( α ) , the transfer and retention performance remain independent of the resultant input sparsity α (Fig. 4D; see Appendix B.3 for the details). These results imply that plasticity gating and input sparsification are not effective in our model, which operates in an over-parameterized regime (i.e., N s ≪ N x ), but do not exclude their potential benefits for many continual learning tasks.

6 Weight regularization

6.1 Weight regularization in Euclidean metric

Another popular method is weight regularization, which keeps the synaptic weights close to those learned from previous tasks [30, 36, 67]. Let us first consider regularization of the Euclidean distance between the current weight W and the weight learned in the previous task W µ -1 :

Here, we parameterize the regularizer amplitude by N x N s ( 1 γ -1) using a non-negative parameter γ for brevity. γ = 1 corresponds to zero regularization, while γ = 0 is the infinite regularization limit. Under this parameterization, if N s ≪ N x , the transfer and retention performance follow simple expressions as below (see Appendix C.1):

The expression of the transfer performance is equivalent to Eq. 5a if we change γ to α . Thus, the Euclidean weight regularization with amplitude N x N s ( 1 γ -1) is mathematically equivalent with random activity gating with sparsity γ , in terms of knowledge transfer (compare Fig. 5A with Fig. 3A). By contrast, the expression of the retention performance contains two additional terms compared to 5b. In particular, the last term, (1 -γ ) 2 , indicates that strong weight regularization not only prevent forgetting, but also impairs task acquisition. Thus, the retention performance is optimized at an intermediate regularizer strength (Figs. 5B and C).

Because strong regularization (i.e., small γ ) impairs both retention and transfer, unlike the random activity gating model, there isn't a tradeoff between knowledge transfer and retention in terms of the average performance over 0 ≤ ρ a , ρ b ≤ 1 (Fig. 5D). However, the retention performance at the optimal parametric regime is relatively low (Fig. 5D with Fig. 4B).

6.2 Weight regularization in Fisher information metric

Previous works proposed that the synaptic weights should be regularized in the Fisher information metric to allow flexibility in the non-overlapping weight space [30, 67]. Applying it to our model setting, the regularizer term in Eq. 6 instead becomes 1 2 ( 1 γ -1) ∥ ( W -W µ -1 ) A µ -1 ∥ 2 F (see Appendix C.2). If ρ a , γ < 1 and N s ≪ N x , optimization of the weight under this regularization yields

where W µ -1 is the weight after the previous task learning. Notably, the weight becomes independent of the regularizer's amplitude γ . Furthermore, the retention performance under arbitrary task similarity ( ρ a , ρ b ) is derived as ∆ ϵ RT = 1 -O ( N s N x (1 -ρ 2 a ) ) . Thus, under the Fisher information metric, the retention performance is perfect as long as the condition N s ≪ N x (1 -ρ 2 a ) is satisfied (Figs. 6A-C). Intuitively, assuming over-parameterization ( N s ≪ N x ), the network can freeze the weight changes in a low-dimensional subspace, while maintaining sufficient plasticity for new tasks, unless the two tasks share the same feature. Notably, if the first task is learned with weight regularization in an orthogonal direction, the transfer performance remains the same with the vanilla model (Fig. 6C).

Importantly, this invariance no longer holds when the Fisher information metric is approximated by its diagonal component (Figs. 6D vs 6C), as is done in the elastic weight methods [30]. This is because the diagonal approximation makes the metric full-rank even when the true metric has a low-rank structure. Thus, weight regularization in the Fisher information metric robustly helps retention without harming transfer, but diagonal approximation attenuates its robustness.

7 Numerical experiments

Our theory so far has revealed when continual learning is difficult and when activity gating and weight regularization can ameliorate the transfer and retention in a simple problem setting. Let us next examine how much these insights are applicable to more realistic datasets and neural architectures.

To this end, we consider the permuted MNIST dataset [34, 22], but with a latent structure (see Appendix E.2). For each output label, we constructed a four-dimensional latent variable s using the binary representation of the corresponding digit (e.g., s 9 = [1 , 0 , 0 , 1] T ). The target output was then generated by a random fixed projection of the latent variable: y ∗ = B µ ( s -1 2 ) . Note that, although here we introduced an explicit latent structure for a direct comparison with the theory, qualitatively similar results holds in the vanilla permuted MNIST setting depicted in Figs. 1B-D (see Fig. 12). We controlled the readout similarity between tasks using the element-wise correlation between two matrices B 1 and B 2 . The feature similarity was controlled by permuting a subset of input pixels, as in the previous permuted MNIST tasks. We used a one-hidden-layer fully-connected network with rectified-linear nonlinearity at the hidden layer, and trained the network using stochastic gradient descent.

As predicted from our theory, both transfer and retention performance, measured by the test error, exhibited asymmetric and non-monotonic dependence on the task similarity. When ( ρ a , ρ b ) = (1 . 0 , 0 . 0) , both transfer and retention performance were below zero (lower-right of Figs. 7A and B vs. Figs. 2C and E). By contrast, the model achieved high retention and zero transfer under ( ρ a , ρ b ) = (0 . 0 , 1 . 0) as expected (upper-left). Notably, around ρ b = 0 . 5 , the transfer performance exhibited a non-monotonic dependence on ρ a in accordance with the theory. The retention performance also exhibited non-monotonic dependence on ρ a under ρ b ≈ 1 , but the dependence became monotonic after a long training (see Figs. 9C and D in Appendix). We observe consistent results even when feature and readout similarity are adjusted by permuting input pixels and output labels, respectively, provided the loss is measured by cross-entropy (Fig. 12B, corresponding to the examples depicted in Figs. 1C and 1D).

Random activity gating, implemented in both input and hidden layers, improved the retention performance at the cost of low knowledge transfer as predicted (compare Fig. 7D and Fig. 4B). Moreover, adaptive gating based on a probe trial improved the transfer performance under a low gating level, especially when the readout similarity is high (solid vs dashed lines in Figs. 7C and D).

In a deep neural network, weight regularization using the Fisher information metric is typically computationally expensive. However, an efficient layer-wise approximation exists for fully-connected networks (see Appendix E.2). This layer-wise approximation enabled high retention performance across a wide range of regularization amplitudes and consistently outperformed weight regularization in the Euclidean metric under various task similarities (red vs. gray lines in Figs. 7F-H). Nevertheless, the retention performance under the diagonal approximation was closer to that of the layer-wise approximation of the Fisher information metric and was slightly better when both methods performed poorly (red vs. orange lines). This is because the sparsity of hidden layer weights and activity makes the diagonal components sparse.

8 Discussion

Here, we analyzed the impact of task similarity on continual learning in a linear teacher-student model with low-dimensional latent structures. We showed that, a combination of high feature similarity and low readout similarity lead to poor outcomes in both retention and transfer, unlike the opposite combination. We further explored how continual learning algorithms such as gating mechanisms, activity sparsification, and weight regularization interact with task similarity. Results indicate that weight regularization in the Fisher information metric significantly aids retention regardless of task similarity. Numerical experiments in a permuted MNIST task with latent supported these findings.

Our findings on the interaction between task similarity and continual learning algorithms have several implications. Firstly, when tasks are known to have high feature similarity and low readout similarity, adaptive activity gating and weight regularization in the Fisher information metric help retention without sacrificing transfer (Figs. 4, 6, 7). In the context of neuroscience, our results indicate that simple non-adaptive activity or plasticity gating mechanisms are not sufficient for good continual learning performance, especially when familiar sensory stimuli need to be associated with novel motor actions. Indeed, a previous study reported catastrophic forgetting among rats learning timing estimation tasks [14]. Lastly, it also implies the potential importance of studying wider benchmarks beyond permuted image recognition tasks for continual learning, because image permutation belongs to a class of relatively harmless task similarity according to our theory. As a simple extension, here we developed an image recognition task with a latent variable and showed that a vanilla deep network exhibits more forgetting under readout remapping than under input permutation (Figs. 7A and B).

Limitations Our theoretical results from a teacher-student model come with limitation in direct applicability. The presence of low-dimensional latent generating both input and target output is a reasonable assumption for many real-world tasks [7, 26], but the linear projection assumption is not. While we replicated most key results in a deep nonlinear network solving permuted MNIST task, we found that the diagonal approximation of Fisher information metric performs much better than the prediction from the teacher-student model. This is potentially because sparse activity at the hidden layer effectively makes the regularization low-rank even under the diagonal approximation. More generally, the presence of hidden layers should enable a distinctive adaptation for feature and readout similarity, which is an important future direction. It is also important to apply our theoretical framework for analyzing continual learning in more complicated neural architectures and datasets.

9 Acknowledgments

The author thanks Liu Ziyin and Ziyan Li for discussion. This work was partially supported by The Swartz Foundation.

References

M. S. Advani, A. M. Saxe, and H. Sompolinsky. High-dimensional dynamics of generalization error in neural networks. Neural Networks , 132:428-446, 2020.

H. Asanuma, S. Takagi, Y. Nagano, Y. Yoshida, Y. Igarashi, and M. Okada. Statistical mechanical analysis of catastrophic forgetting in continual learning with teacher and student networks. Journal of the Physical Society of Japan , 90(10):104001, 2021.

Y. Bahri, J. Kadmon, J. Pennington, S. S. Schoenholz, J. Sohl-Dickstein, and S. Ganguli. Statistical mechanics of deep learning. Annual Review of Condensed Matter Physics , 11:501-528, 2020.

M. Baity-Jesi, L. Sagun, M. Geiger, S. Spigler, G. B. Arous, C. Cammarota, Y. LeCun, M. Wyart, and G. Biroli. Comparing dynamics: Deep neural networks versus glassy systems. In International Conference on Machine Learning , pages 314-323. PMLR, 2018.

M. A. Bennani, T. Doan, and M. Sugiyama. Generalisation guarantees for continual learning with orthogonal gradient descent. arXiv preprint arXiv:2006.11942 , 2020.

X. Chen, C. Papadimitriou, and B. Peng. Memory bounds for continual learning. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS) , pages 519-530. IEEE, 2022.

U. Cohen, S. Chung, D. D. Lee, and H. Sompolinsky. Separability and geometry of object manifolds in deep neural networks. Nature communications , 11(1):746, 2020.

O. Dhifallah and Y. M. Lu. Phase transitions in transfer learning for high-dimensional perceptrons. Entropy , 23(4):400, 2021.

T. Doan, M. A. Bennani, B. Mazoure, G. Rabusseau, and P. Alquier. A theoretical analysis of catastrophic forgetting through the ntk overlap matrix. In International Conference on Artificial Intelligence and Statistics , pages 1072-1080. PMLR, 2021.

B. Ehret, C. Henning, M. R. Cervera, A. Meulemans, J. Von Oswald, and B. F. Grewe. Continual learning in recurrent neural networks. arXiv preprint arXiv:2006.12109 , 2020.

I. Evron, E. Moroshko, R. Ward, N. Srebro, and D. Soudry. How catastrophic can catastrophic forgetting be in linear regression? In Conference on Learning Theory , pages 4028-4079. PMLR, 2022.

I. Evron, E. Moroshko, G. Buzaglo, M. Khriesh, B. Marjieh, N. Srebro, and D. Soudry. Continual learning in linear classification on separable data. In International Conference on Machine Learning , pages 9440-9484. PMLR, 2023.

I. Evron, D. Goldfarb, N. Weinberger, D. Soudry, and P. Hand. The joint effect of task similarity and overparameterization on catastrophic forgetting-an analytical model. arXiv preprint arXiv:2401.12617 , 2024.

R. French and A. Ferrara. Modelling time perception in rats: evidence for catastrophic interference in animal learning. In 21st Congress of Cognitive Sciences , 1999.

R. M. French. Using semi-distributed representations to overcome catastrophic forgetting in connectionist networks. In Proceedings of the 13th annual cognitive science society conference , volume 1, pages 173-178, 1991.

R. M. French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences , 3(4): 128-135, 1999.

P. Gao, E. Trautmann, B. Yu, G. Santhanam, S. Ryu, K. Shenoy, and S. Ganguli. A theory of multineuronal dimensionality, dynamics and measurement. BioRxiv , page 214262, 2017.

E. Gardner and B. Derrida. Three unfinished works on the optimal storage capacity of networks. Journal of Physics A: Mathematical and General , 22(12):1983, 1989.

B. Ghorbani, S. Mei, T. Misiakiewicz, and A. Montanari. Limitations of lazy training of two-layers neural network. Advances in Neural Information Processing Systems , 32, 2019.

D. Goldfarb and P. Hand. Analysis of catastrophic forgetting for random orthogonal transformation tasks in the overparameterized regime. In International Conference on Artificial Intelligence and Statistics , pages 2975-2993. PMLR, 2023.

S. Golkar, M. Kagan, and K. Cho. Continual learning via neural pruning. arXiv preprint arXiv:1903.04476 , 2019.

I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y. Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 , 2013.

R. Hadsell, D. Rao, A. A. Rusu, and R. Pascanu. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences , 24(12):1028-1040, 2020.

R. Heckel. Provable continual learning via sketched jacobian approximations. In International Conference on Artificial Intelligence and Statistics , pages 10448-10470. PMLR, 2022.

N. Hiratani, Y. Mehta, T. Lillicrap, and P. E. Latham. On the stability and scalability of node perturbation learning. Advances in Neural Information Processing Systems , 35:31929-31941, 2022.

M. Jazayeri and S. Ostojic. Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity. Current opinion in neurobiology , 70:113-120, 2021.

R. Karakida and S. Akaho. Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting. In International Conference on Learning Representations , 2021.

Z. Ke, B. Liu, and X. Huang. Continual learning of a mixed sequence of similar and dissimilar tasks. Advances in neural information processing systems , 33:18493-18504, 2020.

Z. Ke, B. Liu, W. Xiong, A. Celikyilmaz, and H. Li. Sub-network discovery and soft-masking for continual learning of mixed tasks. arXiv preprint arXiv:2310.09436 , 2023.

J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences , 114(13):3521-3526, 2017.

J. Knoblauch, H. Husain, and T. Diethe. Optimal continual learning has perfect memory and is np-hard. In International Conference on Machine Learning , pages 5327-5337. PMLR, 2020.

D. Kudithipudi, M. Aguilar-Simon, J. Babb, M. Bazhenov, D. Blackiston, J. Bongard, A. P. Brna, S. Chakravarthi Raja, N. Cheney, J. Clune, et al. Biological underpinnings for lifelong learning machines. Nature Machine Intelligence , 4(3):196-210, 2022.

A. K. Lampinen and S. Ganguli. An analytic theory of generalization dynamics and transfer learning in deep linear networks. arXiv preprint arXiv:1809.10374 , 2018.

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998.

S. Lee, S. Goldt, and A. Saxe. Continual learning in the teacher-student setup: Impact of task similarity. In International Conference on Machine Learning , pages 6109-6119. PMLR, 2021.

S.-W. Lee, J.-H. Kim, J. Jun, J.-W. Ha, and B.-T. Zhang. Overcoming catastrophic forgetting by incremental moment matching. Advances in neural information processing systems , 30, 2017.

C. Li, Z. Huang, W. Zou, and H. Huang. Statistical mechanics of continual learning: Variational principle and mean-field potential. Physical Review E , 108(1):014309, 2023.

Y. Li, T. Ma, and H. R. Zhang. Learning over-parametrized two-layer neural networks beyond ntk. In Conference on learning theory , pages 2613-2682. PMLR, 2020.

S. Lin, L. Yang, D. Fan, and J. Zhang. Beyond not-forgetting: Continual learning with backward knowledge transfer. Advances in Neural Information Processing Systems , 35:16165-16177, 2022.

S. Lin, P. Ju, Y . Liang, and N. Shroff. Theory on forgetting and generalization of continual learning. In International Conference on Machine Learning , pages 21078-21100. PMLR, 2023.

Y. Luo, Z. Yang, F. Meng, Y. Li, J. Zhou, and Y. Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747 , 2023.

N. Y. Masse, G. D. Grant, and D. J. Freedman. Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization. Proceedings of the National Academy of Sciences , 115(44):E10467E10475, 2018.

A. Maurer, M. Pontil, and B. Romera-Paredes. The benefit of multitask representation learning. Journal of Machine Learning Research , 17(81):1-32, 2016.

M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation , volume 24, pages 109-165. Elsevier, 1989.

A. Ndirango and T. Lee. Generalization in multitask deep neural classifiers: a statistical physics approach. Advances in Neural Information Processing Systems , 32, 2019.

S. Özgün, A.-M. Rickmann, A. G. Roy, and C. Wachinger. Importance driven continual learning for segmentation across domains. In Machine Learning in Medical Imaging: 11th International Workshop, MLMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings 11 , pages 423-433. Springer, 2020.

V. V. Ramasesh, E. Dyer, and M. Raghu. Anatomy of catastrophic forgetting: Hidden representations and task semantics. arXiv preprint arXiv:2007.07400 , 2020.

A. Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science , 7(2):123-146, 1995.

D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne. Experience replay for continual learning. Advances in neural information processing systems , 32, 2019.

A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671 , 2016.

D. Saad and S. A. Solla. On-line learning in soft committee machines. Physical Review E , 52(4):4225, 1995.

A. M. Saxe, J. L. McClelland, and S. Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120 , 2013.

A. M. Saxe, J. L. McClelland, and S. Ganguli. A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences , 116(23):11537-11546, 2019.

J. Serra, D. Suris, M. Miron, and A. Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference on machine learning , pages 4548-4557. PMLR, 2018.

H. S. Seung, H. Sompolinsky, and N. Tishby. Statistical mechanics of learning from examples. Physical review A , 45(8):6056, 1992.

E. Sezener, A. Grabska-Barwi´ nska, D. Kostadinov, M. Beau, S. Krishnagopal, D. Budden, M. Hutter, J. Veness, M. Botvinick, C. Clopath, et al. A rapid and efficient learning rule for biological neural circuits. BioRxiv , pages 2021-03, 2021.

R. K. Srivastava, J. Masci, S. Kazerounian, F. Gomez, and J. Schmidhuber. Compete to compute. Advances in neural information processing systems , 26, 2013.

Y. Tian. An analytical formula of population gradient for two-layered relu network and its applications in convergence and critical point analysis. In International conference on machine learning , pages 3404-3413. PMLR, 2017.

G. M. Van de Ven, H. T. Siegelmann, and A. S. Tolias. Brain-inspired replay for continual learning with artificial neural networks. Nature communications , 11(1):4069, 2020.

L. Wang, X. Zhang, Q. Li, M. Zhang, H. Su, J. Zhu, and Y. Zhong. Incorporating neuro-inspired adaptability for continual learning in artificial intelligence. Nature Machine Intelligence , 5(12):1356-1368, 2023.

J. Werfel, X. Xie, and H. Seung. Learning curves for stochastic gradient descent in linear feedforward networks. Advances in neural information processing systems , 16, 2003.

Z. Xu and A. Tewari. On the statistical benefits of curriculum learning. In International Conference on Machine Learning , pages 24663-24682. PMLR, 2022.

J. Yoon, E. Yang, J. Lee, and S. J. Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 , 2017.

X. Yu, T. Liu, X. Wang, and D. Tao. On compressing deep models by low rank and sparse decomposition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 7370-7379, 2017.

A. R. Zamir, A. Sax, W. Shen, L. J. Guibas, J. Malik, and S. Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3712-3722, 2018.

L. Zdeborová and F. Krzakala. Statistical physics of inference: Thresholds and algorithms. Advances in Physics , 65(5):453-552, 2016.

F. Zenke, B. Poole, and S. Ganguli. Continual learning through synaptic intelligence. In International conference on machine learning , pages 3987-3995. PMLR, 2017.

A Problem setting: Linear teacher-student model with a latent variable

Let us consider a continuous learning of N sess tasks. We generate the input x ∈ R N x and the target output y ∗ ∈ R N y of the µ -th task by

where s ∈ R N s is the latent variable, I s is the size N s identity matrix, and A µ ∈ R N x × N s and B µ ∈ R N y × N s are mixing matrices that map the latent variable s into the input space and output space, respectively. Throughout the paper, the vector x is defined as a column vector, and its transpose x T is a row vector.

We consider sequential learning of these tasks with a neural network defined by y = Wψ ( x ) , where W ∈ R N y × N x is the plastic weight. We set the function ψ : R N x → R N x to ψ ( x ) = x in the vanilla and weight regularization models, ψ ( x ) = g ⊙ x in activity and plasticity gating models, and ψ ( x ) = sgn ( x ) ⊙ max { 0 , | x | -h } in the soft-thresholding model. Here, g ⊙ x represents an element-wise multiplication of two vectors, and sgn ( x ) is a function that returns the sign of the input x . The goal of the student network, y = Wψ ( x ) , is to mimic the teacher network that generates both input and target output [18, 66, 3]. Below, we focus on learning of N sess = 2 tasks. To analyze the transfer and retention performance, we introduce the following two assumptions on task structure.

Assumption I: Random task assumption We consider random generation of task matrices A µ , B µ but with correlation between subsequent tasks. We sample elements of the mixing matrices for the first task, A 1 and B 1 , from a Gaussian distribution with mean zero and variance 1 N s independently. For the subsequent tasks, we generate mixing matrices A µ by

where ˜ A µ is a random matrix whose elements are sampled independently from a Gaussian distribution with mean zero and variance 1 N s . We generate B µ in the same manner, but with correlation ρ b . Although this hidden weight generation doesn't exactly follow the correlated Gaussian assumption, the difference appears only in higher-order terms, which are negligible under N s ≫ N x assumption we introduce below.

Assumption II: Low-dimensional latent assumption The second assumption is the relative low dimensionality of the latent space with respect to the input space: N s ≪ N x . This assumption of over-parameterization in the input space simplifies the analysis (see Appendix D).

Evaluation of the transfer and retention performance We evaluate the task performance of a student model with weight W on task µ by mean-squared error:

Here, ⟨·⟩ s is the expectation over latent s ∼ N (0 , I s ) . Using ϵ µ [ W ] , the degree of forward knowledge transfer and retention are evaluated by

where W µ is the weight after training on the µ -th task. Note that, the retention performance is defined by the difference with the initial error ϵ µ [ W o ] . This means that, if the network fails to learn the task in the first place (i.e., if ϵ µ [ W µ ] > 0 ), the retention performance becomes sub-optimal even if the network doesn't forget the learned task. We used this definition for our problem setting because the network is able to learn the task perfectly unless strong regularization is added to the network.

Learning procedures We consider the gradient descent learning from infinite samples at the gradient flow limit and analyze the performance at the convergence. Below, we investigate how ∆ ϵ TF µ and ∆ ϵ RT µ depend on the task similarity and hyper-parameters under various algorithms for continual learning.

B Task-dependent gating model

B.1 Activity gating

Let us first consider task-dependent activity gating model. The student network (the network that learns the task) is given by:

where g µ is the gating vector that depends on the task id µ , but independent of input x . Note that, by setting g µ = 1 , we recover the vanilla model y = W x .

The performance of this network on the µ -th task is written as

where D µ ≡ diag [ g µ ] is a diagonal matrix whose ( i, i ) -th element is g µ,i , and ∥·∥ F is the Frobenius norm.

Solution of gradient descent learning Under the gradient descent learning, the weight dynamics follows

From singular value decomposition (SVD), D µ A µ is rewritten as

where U µ ∈ R N x × N o and V µ ∈ R N s × N o are semi-orthonormal matrices (i.e., U T µ U µ = V T µ V µ = I o ), Σ µ ∈ R No × N o is a positive diagonal matrix, N o is the number of non-zero singular values of D µ A µ . Using this decomposition, the gradient descent dynamics is rewritten as

Thus, the weight matrix W ( t ) at arbitrary t is written as

where Q ∈ R N y × N o is a time-dependent matrix. At the fixed point of this learning dynamics, we have

where O is the zero matrix and W µ -1 ≡ W ( t = 0) is the weight after learning of the previous task. Because Σ µ is a positive diagonal matrix, the equation above has a unique solution:

where ( D µ A µ ) + = V µ Σ -1 µ U T µ is the pseudo-inverse of D µ A µ , and I x is the size N x identity matrix.

From Eqs. 20b and 14, we have

Moreover, by setting the initial weight W o (the weight before the first tasks) to zero, we have

and W 1 and W 2 follow

The error in task 2 after learning of task 1 is thus written as

Similarly, the error on task 1 after learning of task 2 is

Note that the results so far does not rely on Assumptions I and II, making them applicable to arbitrary task matrices ( A,B ) .

The weight matrix after learning depends on the pseudo-inverse, ( DA ) + = V Σ -1 U T , which is typically a complicated function of the original matrix DA . However, when αN x ≫ N s , it can be approximated by a scaled transpose, N s αN x ( DA ) T (see Eq. 101).

Random and adaptive activity gating In the adaptive activity gating model, we generate input gating units { g µ i } randomly, but with correlations between subsequent tasks. We sample the gating units for the first task with g 1 i ← Bernoulli ( α ) , where α is the sparsity of the gating units. The gating units for the µ +1 task, { g µ +1 i } , are then generated by

Here, ρ g µ +1 is the correlation between the gating units for the µ -th and µ +1 -th tasks. The gating correlation ρ g µ is set to zero in the case of random task-dependent gating, whereas ρ g µ is adjusted based on the model performance right after a model switch in the adaptive task-dependent gating. To this end, we introduce a probe trial where the model solves the new task using the gating vector for the old task. The error on the probe trial is

We then set the probability of using the same gating elements by

where ϵ o ≡ 1 N y ∥ B 2 ∥ 2 F is the baseline error on the task 2. It keeps the gating the same if ϵ pb = 0 , while it resamples all gating elements in case ϵ pb ≥ ϵ o .

Transfer performance The average transfer performance ∆ ϵ TF from the first to the second tasks is:

In the last line, we took the expectation over the correlated random matrices B 1 and B 2 . Using the approximation ( DA ) + ≈ N s αN x ( DA ) T (Eq. 101), and then taking the expectation over A 1 , A 2 , D 1 , and D 2 , the first term becomes

where a (1) ji represents the ( j, i ) -th element of A 1 , and ˜ α is defined by

The second term of Eq. 29 follows

The third line follows from Isserlis' theorem, from which we can decompose the higher-order correlation as below:

Summing up the equations above, at N s αN x → 0 limit, we have

Note that when A 2 is generated by random replacement, instead of sampling from a correlated Gaussian distribution, 〈 ∥ ( D 1 a 1 ) + D 2 A 2 ∥ 2 F 〉 term instead follows 〈 ∥ ( D 1 a 1 ) + D 2 A 2 ∥ 2 F 〉 ≈ N s ( ˜ α 2 ρ 2 a + ˜ αN s αN x [ 1 + ρ a N s (2 -ρ a α ˜ α ) ]) , which converges to Eq. 32 at N s ≪ N x limit.

Retention performance Let us next analyze the retention performance, ∆ ϵ RT ≡ ⟨ ϵ 1 [ W o ] -ϵ 1 [ W 2 ] ⟩ A,B , which characterizes how well the network performs the task 1 after learning task 2. At αN x ≫ N s regime, ⟨ ϵ 1 [ W o ] ⟩ A,B = 1 . Inserting Eq. 23 into Eq. 14, we get

Thus, ∆ ϵ RT follows

Because D 2 A 2 A T 2 D 2 = U 2 Σ 2 2 U T 2 , at the N s ≪ αN x limit, U 2 U T 2 is approximated by (see Eq. 102)

Using this approximation and ( DA ) + ≈ N s αN x ( DA ) T (Eq. 101), we can obtain an analytical expression for Eq. 36. Taking the expectation over D and A , the second term of Eq. 36 becomes

Here, we applied Isserlis' theorem again, then retained the terms up to the next-to-leading order with respect to N s αN x . Similarly, the third term of Eq. 36 becomes

Lastly, the cross-term is evaluated as

Therefore, up to the leading order, the retention performance follows

Vanilla model In the vanilla model, we have ˜ α = 1 . Therefore, at N s ≪ N x limit, from Eqs. 34 and 41, the transfer and retention performance follow

Random activity gating Under the random task-dependent activity gating, the gating level ˜ α = α is a constant. If N s ≪ αN x , the transfer and retention performance are written by

Note that, this expression does not hold at the sparse limit α < N s N x .

Depending on ( ρ a , ρ b ) combination, the gating level influences the transfer and retention performance in three different manners (Fig. 8). When ρ b > ρ a , unless both ρ a and ρ b are large, there is a monotonic tradeoff between the transfer and retention performance (Fig. 8B). In the red region where ρ b ≥ 2 ρ a 3 + 1 3 ρ a or ρ b ≥ 2 √ 2 3 ∧ ρ a ≥ 1 √ 2 , a large gating level above a threshold improves the retention performance (Fig. 8C). When ρ b < ρ a , high gating level, α > ρ b ρ a , lower both transfer and retention performance, thus there is no benefit of choosing large α (Fig. 8D).

Adaptive activity gating In the adaptive task-dependent activity gating model, we introduced a probe trial at the beginning of the task 2 to test the model performance for the new task. Under an adaptive gating with ρ g = max { 0 , 1 -ϵ pb ϵ o } , the probe error ϵ pb follows:

Thus, at N s ≪ αN x region, ρ g = max { 0 , ρ a (2 ρ b -ρ a ) } , and the effective gating level becomes

meaning that if the transfer performance is positive in the vanilla model, there should be an overlap in the gating, but otherwise, the gating vector for the next task should be sampled independently.

Average error under a uniform prior on task similarity Gating influences the performance differently depending on the feature and readout similarity. Thus, ideally the model should adjust the gating level based on the task similarity, but in most practical settings, the model doesn't know the similarity a priori. Therefore, it is important to analyze how it performs on average under a randomly chosen task similarity. To this end, here we measure the average transfer and retention performance assuming a uniform prior on 0 ≤ ρ a , ρ b ≤ 1 .

In the vanilla model, we have

Red and blue dash lines in Figs. 3D, 4C, 4D, 5D, 6C, and 6D represent these baseline performance of the vanilla model: ∆¯ ϵ TF = 1 6 and ∆¯ ϵ RT = 43 60 . By contrast, under a random task-dependent gating,

Solid lines in Fig. 3D and dashed lines in Fig. 4B plot the result above. In the case of adaptive gating, the expression of the average performance becomes complicated, yet still numerically tractable.

Note that the average transfer performance is expected to be lower than the retention performance, because forward knowledge transfer requires a high similarity. Even if we use the optimal gating level for transfer, α ∗ ( ρ a , ρ b ) = min(1 , ρ b ρ a ) , the average transfer performance becomes

B.2 Plasticity gating

An alternative strategy is to gate plasticity at certain synapses in a task-dependent manner while keeping the activity intact. We implement this method by making synapses from only a subset of input neurons plastic for each task. Given a linear regression model y = W x , the learning dynamics follows:

where D µ = diag ( g µ ) is a diagonal matrix specifying which synapses are gated. As in the activity gating, g µ is a gating vector whose elements take either one or zero in a task-dependent manner. We sample elements of g µ from a Bernoulli distribution with probability α as before.

Let us denote SVD of A µ by A µ = U µ Λ µ V T µ . Then, from a parallel argument with Eqs. 18 and 19, we have

Notably, unlike Eq. 20b, gating term D µ appears both inside and outside of the inverse term ( ( U T µ D u U µ ) -1 and U T µ D µ ). Thus, up to the leading order, gating level α does not affect the transfer and retention performance.

B.3 Input soft-thresholding

Input-dependent gating is an another method proposed for efficient continual learning. In a regression setting, it corresponds to soft-thresholding of the inputs. Let us introduce a soft-thresholding function φ ( x ) by

where sgn ( x ) represents the sign of x . This nonlinearity makes the analysis difficult, but if N s ≫ 1 in addition to Assumptions I and II, the error becomes tractable. Note that N s ≫ 1 assumption is required only in this subsection. Under the student network defined by

the mean-squared error follows

Let us denote m i ≡ ∑ N s j =1 A ij s j , then we have

Thus, s j and m i approximately follows a joint Gaussian distribution:

Denoting A ji = ρ for brevity, we have

In the third line, we performed a Taylor expansion around ρ = 0 . Similarly, the expectation of φ ( m j ) φ ( m i ) over s is written as

where ρ c = ∑ N s k =1 A ik A jk . Therefore, if | ρ |≪ 1 and | ρ c |≪ 1 , the error ϵ [ W ] is written as

where the input activity sparseness α follows α = erfc [ h √ 2 ] . Under this loss function, gradient descent from W = W o converges to

where A = U Λ V T is the singular value decomposition of A . Therefore, from W o = O , the weights after task 1 and task 2 become

where

This means that for both tasks ( µ, ν = 1 , 2 )

implying that the error is invariant against input sparseness α . Thus, in our problem setting, input sparsification via soft-thresholding doesn't influence knowledge transfer or retention. It also implies that from energy efficiency perspective, soft-thresholding is beneficial because it reduces the number of active neurons while preserving its learning proficiency.

C Weight regularization

C.1 Weight regularization in Euclidean metric

Let us next consider a weight regularization approach by introducing L2 regularization with respect to the weight learned in the previous task. The loss function ℓ µ for the µ -th task is then given by

When λ > 0 , there exists a unique solution:

Thus, from zero-initialization, W 1 and W 2 become

Under N x ≫ N s , the inverse term is approximated by

In the first line, we used Woodbury matrix identity, and in the second line, we used A T A ≈ N x N s I s (see Eq. 99). Let us introduce normalized regularization amplitude γ and ˜ γ by

Under these approximations, W 1 and W 2 simplify to

Here, we further applied A T A ≈ N x N s I s . Note that, unlike Eq. 20b, the result above holds only under Assumptions I & II.

Transfer performance The transfer performance ∆ ϵ TF follows

In the last line, we used Eq. 33 to estimate 〈 ∥ ∥ A T 1 A 2 ∥ ∥ 2 F 〉 . Notably, the expression of the transfer performance, ∆ ϵ TF , coincides with Eq. 5a under γ → α , indicating that in terms of transfer performance, the weight regularization with amplitude N x N s ( 1 α -1 ) is equivalent to random activity gating with gating level α . Figs. 9A and B show the optimal regularizer coefficient γ that maximizes transfer performance and the resultant performance.

Retention performance The average error on task 1 after learning task 2 is

Taking the expectation over A 1 , A 2 , B 1 , B 2 , up to the leading order with respect to N s N x , we have

Similarly, we have

and

Therefore, the error is written as

The optimal γ for each task similarity ( ρ a , ρ b ) and the resultant retention performance are plotted in Figs. 9C and D.

C.2 Elastic weight regularization

In the previous section, we regularized the change in the weight matrix in the Euclidean space. However, previous works indicate that the weight should be regularized using the Fisher information matrix (FIM) as the metric [30, 67]. Let us construct an inference model by y = W x + σ ξ , where ξ is a zero mean Gaussian random variable. Given input-target pair x , y , the likelihood of weight W is

Thus, ( ij, kl ) -th component of FIM becomes

and the quadratic weight regularization in the Fisher information metric follows

Therefore, the loss function for this weight regularizer is written as

As we will see, this loss function has different forms of solution depending on whether ρ a < 1 or ρ a = 1 . We thus consider these two conditions separately below.

C.2.1 Variable features ( ρ a < 1 )

Taking gradient with respect to W , we have

indicating that W is written as W = W µ -1 + Q ˜ A T where Q is a N y × 2 N s matrix and ˜ A ≡ [ A µ -1 , A µ ] is an N x × 2 N s matrix. Solving ∂ℓ µ ∂W = 0 , we get

Multiplying both sides with ( A µ -1 A µ ) from the right side,

̸

If λ = 0 and ρ a < 1 , two square matrices in the above equations are almost surely invertible under assumptions I & II. Therefore, we have

Under N x ≫ N s , from Eq. 99, the inverse matrix is approximated by

Therefore, W = W µ -1 + Q ˜ A T follows

̸

Notably, the equation above doesn't depend on the regularizer amplitude λ , except for λ = 0 condition. At λ → 0 limit, the inverse matrix term in Eq. 82 effectively becomes singular, and thus the equation above no longer holds.

Let us suppose the first task is learned without any weight regularization, or learned with weight regularization in the Fisher information metric imposed by an uncorrelated task. Then, we have W 1 = N s N x B 1 A T 1 . Hence, the transfer performance becomes the same with the vanilla model. The weight after the second task becomes

Thus, the retention performance under W 2 follows

Taking expectation over B 1 and B 2 , ∆ ϵ RT is rewritten as

The second term is rewritten as

Moreover, ( A T 2 -ρ a A T 1 ) A 1 term also cancels out up to the leading order term because

Similarly, the trace terms are also cancelled out up to the leading order term:

and

Thus, we get ∆ ϵ RT = 1 -O ( N s N x (1 -ρ 2 a ) ) . Therefore, if N s ≪ (1 -ρ 2 a ) N x , we have ∆ ϵ RT ≈ 1 regardless of task similarity ρ a and ρ b .

C.2.2 Fixed features ( ρ a = 1 )

When ρ a = 1 , A µ -1 = A µ = A . The gradient follows

and the weight W is written as W = W µ -1 + QU T , where U is defined by SVD of A: A = U Λ V T . Solving ∂ℓ µ ∂W = 0 , we get

Thus, the weight after task 1 and 2 follow

Therefore, the transfer performance is

In the last line, we defined γ f by γ f ≡ 1 1+ λ . Similarly, the retention performance is written as

If the first task is learned without weight regularization, the retention performance instead becomes:

D Properties of very tall zero-mean random matrices

Our simple analytical results rely on the assumption that matrix A ∈ R N x × N s is very tall (i.e., N x ≫ N s ), and each element of A is sampled independently from a normal distribution with mean zero and a finite variance.

Given A ij ∼ N (0 , 1 N s ) , we have 〈 1 N x A T A 〉 A = 1 N s I s . The element-wise deviation from the mean is evaluated as

implying that 〈 ∥ ∥ ∥ 1 N x A T A -1 N s I s ∥ ∥ ∥ 2 F 〉 A = 1 N x ( 1 + 1 N s ) . Therefore, at N x ≫ N s limit, 1 N x A T A approximately follows

Even in the presence of random gating, if the effective matrix is still tall (i.e., αN x ≫ N s ), similar approximations hold. Given a diagonal matrix D whose diagonal components are sampled independently from a Bernoulli distribution with rate α , at αN x ≫ N s limit, we have

Moreover, denoting the SVD of DA by DA = U Σ V T , at αN x ≫ N s limit, we have

This is because, at αN x ≫ N s limit, from Eq. 100, the eigenspectrum of matrix 1 αN x DA ( DA ) T is concentrated at λ = 1 N s , and thus,

This result implies that,

E Numerical methods

Numerical experiments were conducted in standard laboratory GPUs and CPUs. Source codes for all numerical results are made publicly available at https://github.com/nhiratani/transfer_ retention_model .

E.1 Implementation of linear teacher-student models

Numerical results in Figs. 2-6 and 8-9 were implemented as below. We set the latent variable dimensionality N s = 30 , the input width N x = 3000 , and the output width N y = 10 . The student weight W was initialized as the zero matrix, and updated with the full gradient descent with learning rate η = 0 . 001 . We trained the network with the first task for N epoch epochs, and then retrained it with the second task for another N epoch epochs. We used N epoch = 100 for the vanilla model and the activity gating models, but we used N epoch = 500 for other models. Unless stated otherwise, error bars represent standard deviation over 10 random seeds. Average performances in Figs. 3D, 4B-D, 5D, and 6CD were estimated by taking average over 100 pairs of ( ρ a , ρ b ), sampled uniformly from ρ a , ρ b ∈ [0 , 1] .

In the input soft-thresholding model, we estimated the gradient in a sample-based manner because the exact gradient is not tractable due to nonlinearity. We estimated the gradient from 10000 random samples at each iterations, then updated the model for 5000 iterations with learning η = 0 . 01 .

E.2 Implementation of permuted MNIST with latent

Data generation We used permuted MNIST dataset [34, 22], a common benchmark for continual learning, but with addition of the latent space. We constructed a four dimensional latent space s using the binary representation of digits, s 0 = [0 , 0 , 0 , 0] T , s 1 = [0 , 0 , 0 , 1] T , and so on. The target output was generated by a projection of the latent variable s to a ten-dimensional space, y ∗ = B ( s -1 2 1 ) , where B is a 10 × 4 matrix generated randomly. The elements of matrix B 1 for the first task were sampled independently from a Gaussian distribution with mean zero and variance one. For the second task, we introduced readout similarity by keeping some of the elements while resampling other elements. We defined the readout similarity ρ b by the fraction of the elements kept the same between two tasks.

Feature similarity was introduced by permuting the input pixels as done previously [22]. We used the vanilla MNIST images for the first task and permuted some of the pixels in the second task depending on the feature similarity ρ a .

Model implementation We used one hidden layer neural network with ReLU non-linearity in the hidden layer: y = b 2 + W 2 ReLU [ b 1 + W 1 x ] . We set the hidden layer width to N h = 1500 . The input and output widths were set to N x = 784 and N y = 10 . We initialized the first weight W 1 and the bias b 1 by sampling each weight independently from a Gaussian distribution with mean zero and variance 1 2 . We initialized the second weight W 2 and the bias b 2 in the same manner, but with variance N 2 y . Here, we set the amplitude of the initial weights small to operate the learning in the rich regime [53]. We set the mini-batch size to 300 and the learning rate to η = 0 . 01 , and trained the network for 100 epochs per task. The retention performance was measured after 10 epochs of training with task 2 except for Figs. 10 C, 10 D and 11 A-D where the retention was evaluated after 100 epochs. In the learning of task 2, the test error typically drops significantly in the first 10 epochs and shifts to gradual improvement afterward (red line in Fig. 10A). Notably, the asymmetric task similarity dependence of the retention performance was observed consistently both after a short and long training on the second task (Figs. 7B and 10D).

N h 1

Weight regularization in the Fisher information metric From the same argument made in section C.2, FIM of a noise-free model is approximated by the Hessian of the mean squared error. Let us consider a vanilla feedforward neural network with depth K , where hidden layer activity h 1 , ..., h K -1 and the output y follow

where ϕ is an element-wise rectified linear function. For brevity, we denote ∇ ϕ µ as a diagonal matrix where the diagonal components represent element-wise derivative ϕ ′ ( b k + W k h k -1 ) . Given loss function ℓ = 1 2 ∥ y -y ∗ ∥ 2 , the Hessian is estimated as

In the second line, we omitted the second-order derivative ϕ ′′ as this term is effectively negligible under ReLU activation function.

Thus, distance between two weights δW = W -W o in the metric defined by this approximated Hessian is written as

We define a layer-wise approximation of the weight regularization in the Fisher information metric by taking µ = ν components of the distance defined above:

Here, the expectation is taken over training data ( x , y ∗ ) of the previous task (i.e., the task the network shouldn't forget). Taking the derivative with respect to δW k for k = 1 , .., K , we have

This layer-wise approximation captures the true Fisher information metric more accurately than element-wise approximation implemented previously, while computationally less heavy than the estimation of the true metric. In the numerical estimations, we scaled the regularization terms at the k-the layer by

where ∥·∥ 2 is the spectral norm. We did not impose weight regularization on the bias parameters b k . In the case of synapse-wise approximation (i.e., elastic regularization), the regularizer is instead written as

Thus, the gradient of R sw with respect to δW µ follows

We scaled the regularization term by the largest coefficient in a layer-wise manner.

In the case of one-hidden layered networks, the layer-wise regularizer is written as

where the normalization constants follow Z 1 = ∥ ∥ W 2 W T 2 ∥ ∥ 2 ∥ ∥ 〈 xx T 〉∥ ∥ 2 , Z 2 = ∥ ∥ 〈 ϕϕ T 〉∥ ∥ 2 . By contrast, the synapse-wise regularizer is given as

where Z 1 = max ij 〈 ∑ k ( w (2) ki ϕ ′ i x j ) 2 〉 and Z 2 = max j 〈 ϕ 2 j 〉 . To see the potential effect of different normalization methods used for the layer-wise and diagonal approximations, in Figs. 10 E and F, we

independently changed the regularizer amplitude for W 1 and W 2 and measured the retention performance under the two approximation methods. Even in this setting, the layer-wise approximation of the Fisher-information metric robustly outperformed the diagonal approximation, when the regulazier amplitude for W 2 was sufficiently large.

In Fig. 11, we implemented this method to a feedforward network with three all-to-all hidden layers, where hidden layer widths are set to be 784-1000-300-100-10. We used the learning rate η = 0 . 01 , mini-batch size 300, and trained the network for 100 epochs per task. The weight regularization based on a layer-wise approximation of the Fisher-information metric, robustly outperformed both the diagonal approximation of the metric and weight regularization in the Euclidean space even in this deep network (Fig. 11B-D).

E.3 Implementation of permuted MNIST

In Fig. 12, we implemented permuted MNIST without any explicit latent structure. We modulated the input feature similarity by permuting a subset of input pixels between the first and the second tasks, and modulated the readout similarity by permuting a subset of the output labels as shown in Figs. 1C and 1D. We used the same one-hidden layer network with the model above, but introduced the softmax at the output units and estimated the loss using cross-entropy loss. We set the mini-batch size to be 300, and the epoch per task as 100. The retention performance was evaluated at the end of the training of the second task using Eq. 2.

When transfer and retention performance were assessed based on differences in classification accuracy, both measures exhibited a positive monotonic relationship with feature and readout similarity (Fig. 12A). However, when evaluated using cross-entropy loss, the training loss function, transfer and retention performance showed a non-monotonic dependence on feature and readout similarities (Fig. 12B). Specifically, under low readout similarity, increasing feature similarity negatively impacted transfer performance.

Adaptive gating with a probe trial improved transfer performance (Fig. 12C vs. Figs. 7C and 4A). Additionally, weight regularization in the layer-wise approximation of the Fisher information metric yielded better retention performance compared to its diagonal approximation and weight regularization in Euclidean space. (Figs. 12D-F).

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The claims made in the fourth, fifth, and sixth paragraphs of Introduction were supported by sections 4, 5 and 6, and 7, respectively.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Please see the paragraph titled "Limitations" in section 8.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All assumptions and derivations of the theoretical results are provided in Appendix A-D.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Please see Appendix E.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Source codes are made publicly available at https://github.com/ nhiratani/transfer_retention_model .

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Please see Appendix E.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Please see the figures and the legends of Figs. 1 and 6.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Please see Appendix E

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research does not contain any element that requires ethical concern.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The work does not have any direct societal impact.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA] .

Justification: The paper does not contain any data or models that have a risk for misuse.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing licensed assets

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Peer reviews:
comments: SUMMARY:
This paper analyzes the impact of task similarity on Continual Learning (CL) within a linear teacher-student model that incorporates low-dimensional latent structures. The findings indicate that high input feature similarity combined with low readout similarity is detrimental to both knowledge transfer and retention

STRENGTHS:
+ The paper is well-motivated, addressing the important problem of understanding the impact of task similarity on continual learning.
+ The authors provide a thorough analysis of the impact of task similarity on CL using a linear teacher-student model with low-dimensional latent structures. The results demonstrate that high feature similarity and low readout similarity are catastrophic for both knowledge transfer and retention.
+ The paper tests its predictions numerically using the permuted MNIST task, further validating the theoretical findings.
+ The authors provide the code, which enhances the reproducibility of their results.

WEAKNESSES:
+ The current theory is constrained to task incremental CL involving two regression tasks. It would be beneficial to explore whether the theory can be generalized to more tasks or other settings, such as classification tasks.
+ How can readout similarity be understood in other CL tasks, such as classification tasks?

QUESTIONS:
Please refer to the weaknesses.

LIMITATIONS:
The authors have included a limiation discussion in Sec. 8.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper systematically investigated how feature similarity and readout similarity affect knowledge transfer and retention, under different gating scenarios, showing weight regularization based on Fisher information metric improves retention without compromising transfer. These are done with both linear teacher-student models and permuted MNIST task.\n\nSTRENGTHS:\n1) The paper addressed an important question in continual learning - task similarity - with very comprehensive experiments and successfully delineated the effect of several factors clearly; 2) the paper is clearly written; 3) although the bulk of the paper is done on linea teacher-student setting, the authors were able to solidify the majority of their results with permuted MNIST task.\n\nWEAKNESSES:\nI'm confused on the paper's definition of 'similarity' in general, for both feature similarity $\\rho_a$ and readout similarity $\\rho_b$. Since they are key concepts in the paper, they seem to deserve more careful treatment. See question.\n\nQUESTIONS:\n1) Could the authors please explain what they mean by 'element-wise correlation' for $\\rho_a$ and $\\rho_b$? When reading the task sampling procedure in appendix A.1, $\\rho_a$ and $\\rho_b$ are the probability of entries in the mixing matrices being identical; when deriving the analytical solutions, they seem to mean column-wise correlation? I could see either version serving as a working definition for feature similarity/readout similarity but just getting confused on having different definitions of key concepts in one paper. \n2) Relatedly, could the authors comment on/discuss how different similarity definitions affect the conclusions in a broader setting? A third similarity definition seems to be used for the permuted MNIST task but similar trend could be observed. It seems worthwhile to explicitly address the stringiness of similarity definition in the discussion.\n\nLIMITATIONS:\nSee question.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the challenge of continual learning in artificial neural networks, particularly when learning tasks with partial similarity. Task similarity can both facilitate knowledge transfer and increase the risk of interference and catastrophic forgetting. The authors develop a linear teacher-student model with latent structure to analyze how input feature similarity and readout pattern similarity affect knowledge transfer and forgetting. They find that high input feature similarity with low readout similarity is detrimental, while the opposite is relatively benign. The study also explores how different continual learning algorithms, such as task-dependent activity gating and weight regularization based on the Fisher information metric, interact with task similarity.\n\nSTRENGTHS:\nThe paper stands out for its originality in several key areas: 1) Problem Formulation: The nuanced exploration of task similarity's impact on continual learning is a critical perspective. This focus on partial task similarity and its dual role in knowledge transfer and interference is novel and highly relevant. 2) Analytical Framework: The use of a linear teacher-student model with latent structure to dissect the effects of task similarity is an innovative approach. This model provides a clear, theoretical basis for understanding complex interactions in continual learning.\nThe quality of the research is evident in multiple dimensions: 1) Theoretical Rigor: The paper offers an analytical examination of task similarity's effects, supported by theoretical foundations. The linear teacher-student model is well-articulated and used to derive insightful conclusions. 2) Methodological Soundness: The evaluation of different continual learning algorithms, including task-dependent activity gating and weight regularization based on the Fisher information metric, is methodologically sound. These evaluations are carefully designed to highlight the algorithms' interactions with task similarity. 3) Empirical Validation: The empirical results on the permuted MNIST dataset provide strong support for the theoretical findings. The experiments are well-executed, with results that convincingly demonstrate the practical implications of the theoretical insights.\n\nWEAKNESSES:\n1. Limited Scope of Experimental Validation\nWhile the paper provides theoretical insights and validates them on the permuted MNIST dataset, the experimental scope is somewhat limited. Validate the findings on a wider variety of datasets, including more complex and diverse datasets such as CIFAR-100, ImageNet, or non-visual tasks like language modeling or reinforcement learning environments. This would strengthen the generalizability of the results.\n2. Insufficient Comparison with Existing Methods\nThe paper lacks a comprehensive comparison with a wider range of existing continual learning algorithms, particularly recent advancements in the field. Compare the proposed methods and insights with more recent state-of-the-art continual learning algorithms [1-6]. This includes methods that have been introduced in the past year, ensuring the evaluation is up-to-date.\n[1]A comprehensive survey of continual learning: theory, method and application, 2024\n[2]Lee S, Goldt S, Saxe A. Continual learning in the teacher-student setup: Impact of task similarity[C]//International Conference on Machine Learning. PMLR, 2021: 6109-6119.\n[3]Lin S, Ju P, Liang Y, et al. Theory on forgetting and generalization of continual learning[C]//International Conference on Machine Learning. PMLR, 2023: 21078-21100.\n[4]TRGP: Trust Region Gradient Projection for Continual Learning (ICLR ‘22)\n[5]The Ideal Continual Learner: An Agent That Never Forgets (ICML ‘23)\n[6]A Theoretical Study on Solving Continual Learning (NeurIPS ‘22)\n3. Limited Discussion on Practical Implementation\nThe paper primarily focuses on theoretical insights and empirical validation but provides limited guidance on the practical implementation of the proposed methods. Include a section or supplementary material that provides detailed guidelines for implementing the proposed methods in practical scenarios. This could include code snippets, hyperparameter settings, and best practices.\n4. Lack of In-Depth Analysis of Algorithm Interactions\nWhile the paper evaluates task-dependent activity gating and weight regularization, it does not provide an in-depth analysis of how these methods interact with each other and with different levels of task similarity. Conduct experiments to analyze the interaction effects between different continual learning algorithms and varying degrees of task similarity. This would provide a deeper understanding of the conditions under which these methods are most effective.\n5. Clarification on Theoretical Assumptions\nThe theoretical analysis is based on specific assumptions that may not always hold in practical scenarios. The paper could benefit from a more detailed discussion of these assumptions and their implications. Clearly state the assumptions underlying the theoretical model and discuss their limitations. Provide insights into how these assumptions might impact the applicability of the findings in real-world settings. Perform robustness analysis to test the sensitivity of the results to deviations from these assumptions. This could involve experimenting with variations in model architecture, data distribution, and task difficulty.\n\nQUESTIONS:\nPlease refer to the Weaknesses above.\n\nLIMITATIONS:\nPlease refer to the Weaknesses above.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}