Paper: The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting — An Analytical Model (ICLR-2024)

Reviewer comment (Theory):
The proof is difficult to follow, I can tell why the three terms in Lemma 6 can be replaced by the long formulas.

Author response:
**Weakness 3 & Question 1:**  
> The proof is difficult to follow.  
  
  **Response:** Deriving exact expressions for our analysis was indeed difficult and required 60+ pages of complicated proofs. We originally tried to make our proof’s idea accessible using the Proof Sketch that appears right after Theorem 3.  
  Following the reviewer’s comment, we will further revise the manuscript and add an entire page presenting a Proof Outline in a new appendix for the final version (this requires a lot of time and we want the reviewers-authors discussion to start in the meanwhile).

Reviewer comment (Novelty):
I think the paper 'Analysis of catastrophic forgetting for random orthogonal transformation tasks in the overparameterized regime' discusses a similar task. It is difficult to tell what the difference and improvement compared to that.

Author response:
Moreover, in Weakness 4 the reviewer wrote that they believe that Goldfarb and Hand (2023) already addressed a similar research question.  
We have to respectfully disagree, since there are important differences between our work and theirs, which we have already thoroughly discussed in multiple places in our paper (e.g., in the Introduction, Learning Dynamics of Section 2.2, and in the Simulations of Section 2.4). Specifically, Goldfarb and Hand (2023) only study the effect of overparameterization without considering any notion of task similarity. They analyze the regime of *low* task similarity, i.e., applying completely random transformations to the first data matrix ($\alpha=1$), while our main result analyzes the *full* spectrum of task similarity from $\alpha=0$ to $1$.  
Additionally, that paper uses restrictive data assumptions and provides an upper bound on performance, while our work does not assume any particular data model and our main result is an *exact* expression for expected forgetting.

Reviewer comment (Writing):
I have tried to read all proof, but it is difficult to follow and check all the long expressions. I think it will be better to make the proof clearer to understand.

Author response:
**Weakness 3 & Question 1:**  
> The proof is difficult to follow.  
  
  **Response:** Deriving exact expressions for our analysis was indeed difficult and required 60+ pages of complicated proofs. We originally tried to make our proof’s idea accessible using the Proof Sketch that appears right after Theorem 3.  
  Following the reviewer’s comment, we will further revise the manuscript and add an entire page presenting a Proof Outline in a new appendix for the final version (this requires a lot of time and we want the reviewers-authors discussion to start in the meanwhile).

Reviewer comment (Evaluation):
We can only mess up pixels as a standard of dissimilarity, which does not match the real scene. So it's hard to verify the conclusions on other data.

Author response:
We hypothesize that the true effect of task similarity comes from similarity in the NTK feature regime as presented in Appendix A. Pixel shuffling is just one proxy for which to generate the desired effect in the NTK feature regime but the relationship between experimental setup and the NTK features is a complex one. We leave the full uncovering of this effect for future work (we mention this in the “Future work” section of the revised manuscript).

Reviewer comment (Theory):
Is overparameterization equivalent to the learning capacity of the model? Do the conclusions of the paper apply to other data and models?

Author response:
Overparameterization is related to learning capacity but is not equivalent. Any model that surpasses the complexity of the interpolation threshold has an equal capacity to fully fit to the training set. However, more overparameterized models enjoy the beneficial properties of smoother optimization landscapes and sometimes more favorable generalization properties.

Reviewer comment (Theory):
The formulas derived in this paper are rather complicated. Can you please explain Pseudoinverse Properties and Operator Norm Properties in detail?

Author response:
The pseudoinverse property that we used in Equation (4) is that for any matrix $\mathbf{X}$ and an orthogonal matrix $\mathbf{O}$, it holds that $\mathbf{X}=\mathbf{X}\mathbf{X}^{+}\mathbf{X}$ and $(\mathbf{X}\mathbf{O})^{+}=\mathbf{O}^{+}\mathbf{X}^{+}=\mathbf{O}^{\top}\mathbf{X}^{+}$. The operator norm property refers to $\Vert\mathbf{X}\mathbf{v}\Vert\le\Vert\mathbf{X}\Vert\Vert\mathbf{v}\Vert$.  
   To improve readability, we now added these explicitly within Equation (4) and underneath it.

Reviewer comment (Experiments):
This paper only conducted experiments on the mnist dataset. Can the same conclusion be reached for other complex datasets?

Author response:
We hypothesize that the true effect of task similarity comes from similarity in the NTK feature regime as presented in Appendix A. Pixel shuffling is just one proxy for which to generate the desired effect in the NTK feature regime but the relationship between experimental setup and the NTK features is a complex one. We leave the full uncovering of this effect for future work (we mention this in the “Future work” section of the revised manuscript).

Reviewer comment (Experiments):
At first glance it seems a little bit weird to have the same $y$ for two different tasks. I think it might be motivated by the case of permuted MNIST where permutations do not change the label. It would be great if the authors could comment on that in their revision.

Author response:
**Response:** This is due to our analysis being analogous to the data permutation task benchmarks of continual learning. These tasks are especially amenable to analysis as they are each equally difficult for an MLP to solve. Thus we can fairly compare their accuracies and forgetting between tasks.  
  More generally, many practical continual learning scenarios are subject to domain shifts (e.g., sees another “region” in $\mathcal{X}$), without changing the output space $\mathcal{Y}$.

Reviewer comment (Evaluation):
Certainly there is some gap between the theory and MNIST experiments. The authors are encouraged to discuss that gap. For example, the training loss in the theory vs test error in the experiments. The least-squares loss in the theory vs classification loss in the experiments (I suppose).

Author response:
**Response:** We view the present work as an initial result in understanding the relationship between task similarity and forgetting. Nonetheless there are still clear analogies between the analysis and experiments of our work. The permutation task setting is a variant of the DOTS model where random permutation matrices are used instead of random orthogonal matrices. In continual learning practice, training error and test error are strongly correlated: when forgetting is observed in the test error then it is also observed in the training error. Additionally, studying training error allows us to loosen the data assumptions of prior work (Goldfarb, Lin) to give better insight into the problem’s worst-case performance. Regression and classification problems are also closely related. One can turn the MNIST experiments into a one-hot regression problem and expect to observe the same effects as in the classification problem.

Reviewer comment (Theory):
Is there any deeper connection between the bound in the highly overparameterized regime, $\alpha^2(1-\alpha)^2$, and the corresponding bound in Lemma 9 of Evron et al. (2022) for the case $k=2$?

Author response:
**Response:** There is indeed a deeper connection! Given no task-repetitions ($k=2$), the result from Evron et al. (2022) yields a forgetting upper bound of $\frac{1}{2} \max_{i} \{ \cos^2 (\theta_i) (1-\cos^2 (\theta_i)) \}$, where $\theta_i$ are the nonzero principal angles between the solution subspaces of the two tasks (here, they are equivalent to the principal angles between the data itself; you can see Claim 19 in their paper).  
  This expression is of course very similar to our overparameterized results, i.e., $\alpha^{2}(1-\alpha)^{2}=\left(\frac{m}{p}\right)^{2}\left(1-\frac{m}{p}\right)^{2}$.  

  Trying to explain this very briefly and intuitively: When $m=0$, our random transformation $\textbf{O}$ (applied to $\textbf{X}_1$ to create $\textbf{X}_2$) is a deterministic identity operator. This, of course, yields principal angles of $0$ between tasks, incurring no forgetting.  
  When $m=p$, the random transformation becomes completely random, and since we assume high overparameterization, it means that the two subspaces are going to be almost orthogonal w.h.p. (i.e., have angles of $90^{\circ}$, again incurring no forgetting.  
  To understand $m=\frac{p}{2}$, it is perhaps easy to imagine two random lines in $2$ dimensions. In this case it is known that the expected angle between them should be $\approx 45^{\circ}$, which corresponds to the highest forgetting in Evron et al. (2022).

  Following this review, we have now incorporated a similar discussion (on a slightly different aspect) in our “Geometric interpretation and Comparison to Evron et al. (2022)” paragraph in the discussion of the revised manuscript. We believe that it helps see deeper connections like the one that the reviewer noticed.

Reviewer comment (Theory):
While the main theorem is accomplished by algebraic calculations, I wonder whether there is a geometric proof that would achieve the same. For example, the two steps can be thought of as two projections (onto certain subspaces), and the forgetting can be bounded deterministically by some quantity related to the principal angles. With a random orthogonal transformation, the principal angles between the two subspaces would be in some sense random. Would it be easier if one analyzes the randomness of the principal angles, and could the algebraic proofs be replaced by geometric reasoning?

Author response:
**Response:** This is a good question. Following the discussion here and with reviewer [TGNt](https://openreview.net/forum?id=u3dHl287oB&noteId=TRtKnkPHKr), and the discussion we’ve added to the manuscript, it is apparent that geometric reasoning is indeed partially possible here. Specifically, analyzing worst-case expected forgetting should be related to the largest principal angle between 2 random subspaces (see, e.g., “On the largest principal angle between random subspaces”, 2006).
  However, there should be important differences, requiring different proof techniques. Specifically, our analysis allows for the entire range of $m$ (number of directions/pixels that we rotate), while the aforementioned other paper requires the two subspaces to be completely random. Thus, it seems like they can only help analyze the $m=p$ case.

Reviewer comment (Novelty):
It is unclear how DOTS can compare with the notion of similarity in related works.

Author response:
In our linear regression setting, the existing similarity notion most comparable to our DOTS should be the principal angles between the solution subspaces of the two data matrices ($\mathbf{X}_1,\mathbf{X}_2$)}, which were used to quantify task similarity in Doan et al. (2021) and Evron et al. (2022). We elaborate on this further here below, and in the discussion in our revised manuscript.

Reviewer comment (Theory):
Is the theory able to provide worst-case forgetting analysis with more general dataset assumption, or T>2 datasets?

Author response:
These are important questions and we added the following discussion to a new section on “Limitations and Future work” in the revised manuscript.  
  Our analysis has primarily examined settings with $T=2$ tasks. Extending these analytical results to $T\ge 3$ tasks poses an immediate challenge. The complexity of our analysis, which already required intricate techniques and proofs, suggests that tackling this extension may be considerably difficult. Moreover, the convergence analysis presented in a previous paper (Evron et al. 2022) for learning $T\ge 3$ tasks cyclically has proven to be notably more challenging than that for $T=2$ tasks, and was further improved in a follow-up paper (Swartworth et al. 2023).

Reviewer comment (Theory):
It is still unclear that how DOTS relates to other notions of similarity, e.g. the principle angles in Evron et al. (2022). Is that possible to plot a figure that shows the relation between DOTS and $\theta$?

Author response:
We thank the reviewer for this interesting question!  
  Evron et al. (2022) showed analytically that intermediate task similarity (a principle angle of $45^{\circ}$) is most difficult in two-task linear regression.  
  Their analysis applies to *any* two arbitrary tasks, and thus seemingly contradicts the behavior observed, e.g., in our Figure 1(b), where maximal dissimilarity is most difficult.  
  The key to settling this apparent disagreement is the *randomness* of our transformations.  
  Their analysis focuses on any two *deterministic* tasks, while our second task is given by a *random* transformation of the first, as done in many popular continual learning benchmarks (e.g., permutation and rotation tasks).  
  
  For simplicity, instead of the case that the reviewer mentioned, where $d=p-1$, let us focus on $d=1$ (in two tasks these are almost equivalent; see Claim 19 in Evron et al. (2022)).
  
  To gain a geometric intuition, consider two tasks of rank $d=1$ ($\mathbf{x}_1, \mathbf{x}_2$). Consider also a *maximal* DOTS proxy for task dissimilarity ($\alpha=\frac{m}{p}=1$), i.e., $\mathbf{x}_2 = \mathbf{O}\mathbf{x}_1$ is simply a random rotation of $\mathbf{x}_1$ in $p$ dimensions. It is known that $\mathbb{E} \big| \big\langle \frac{\mathbf{x}_1}{\Vert{\mathbf{x}_1}\Vert}, \frac{\mathbf{x}_2}{\Vert{\mathbf{x}_2}\Vert} \big\rangle \big| \approx \frac{1}{\sqrt{p}}$. 
  Near the interpolation threshold, \eg when $p= 2$ (recall that $d=1$), we get that ${\mathbb{E} \big| \big\langle \frac{\mathbf{x}_1}{\Vert{\mathbf{x}_1}\Vert}, \frac{\mathbf{x}_2}{\Vert{\mathbf{x}_2}\Vert} \big\rangle \big| \approx \frac{1}{\sqrt{2}}} \Longrightarrow {\mathbb{E}\angle(\mathbf{x}_1,\mathbf{x}_2) \approx 45^{\circ}}$, 
  corresponding to the *intermediate* task dissimilarity in Evron et al. (2022), where forgetting is *maximal*.
  On the other hand, given high overparameterization levels ($p\to\infty$), we get that ${\mathbb{E} \big| \big\langle \frac{\mathbf{x}_1}{\Vert{\mathbf{x}_1}\Vert}, \frac{\mathbf{x}_2}{\Vert{\mathbf{x}_2}\Vert} \big\rangle \big| \approx \frac{1}{\sqrt{p}} \to 0} \Longrightarrow {\mathbb{E} \angle(\mathbf{x}_1,\mathbf{x}_2)\to 90^{\circ}}$, corresponding to the *maximal* task dissimilarity in Evron et al. (2022), where forgetting is *minimal*.


  We originally compared our results to Evron et al. (2022) in the discussion section. Following the reviewer’s question, we included the discussion above in the revised manuscript to further elaborate on these aspects.

Reviewer comment (Theory):
In Evron et al.(2022), Figure 3 shows the worst-case forgetting on T=2 tasks, where p=d-1(rank(X1)=rank(X2)=d-1), which in this paper, should corresponds to $\beta=1-\frac{d-1}{d}\approx 0$, which should fall into the very less overparameterization regime. However, there results still show the descent with a large angle. Can authors provide some explanation? (And this is also one of the reasons I hope to know how DOTS relates to $\theta$)

Author response:
We thank the reviewer for this interesting question!  
  Evron et al. (2022) showed analytically that intermediate task similarity (a principle angle of $45^{\circ}$) is most difficult in two-task linear regression.  
  Their analysis applies to *any* two arbitrary tasks, and thus seemingly contradicts the behavior observed, e.g., in our Figure 1(b), where maximal dissimilarity is most difficult.  
  The key to settling this apparent disagreement is the *randomness* of our transformations.  
  Their analysis focuses on any two *deterministic* tasks, while our second task is given by a *random* transformation of the first, as done in many popular continual learning benchmarks (e.g., permutation and rotation tasks).  
  
  For simplicity, instead of the case that the reviewer mentioned, where $d=p-1$, let us focus on $d=1$ (in two tasks these are almost equivalent; see Claim 19 in Evron et al. (2022)).
  
  To gain a geometric intuition, consider two tasks of rank $d=1$ ($\mathbf{x}_1, \mathbf{x}_2$). Consider also a *maximal* DOTS proxy for task dissimilarity ($\alpha=\frac{m}{p}=1$), i.e., $\mathbf{x}_2 = \mathbf{O}\mathbf{x}_1$ is simply a random rotation of $\mathbf{x}_1$ in $p$ dimensions. It is known that $\mathbb{E} \big| \big\langle \frac{\mathbf{x}_1}{\Vert{\mathbf{x}_1}\Vert}, \frac{\mathbf{x}_2}{\Vert{\mathbf{x}_2}\Vert} \big\rangle \big| \approx \frac{1}{\sqrt{p}}$. 
  Near the interpolation threshold, \eg when $p= 2$ (recall that $d=1$), we get that ${\mathbb{E} \big| \big\langle \frac{\mathbf{x}_1}{\Vert{\mathbf{x}_1}\Vert}, \frac{\mathbf{x}_2}{\Vert{\mathbf{x}_2}\Vert} \big\rangle \big| \approx \frac{1}{\sqrt{2}}} \Longrightarrow {\mathbb{E}\angle(\mathbf{x}_1,\mathbf{x}_2) \approx 45^{\circ}}$, 
  corresponding to the *intermediate* task dissimilarity in Evron et al. (2022), where forgetting is *maximal*.
  On the other hand, given high overparameterization levels ($p\to\infty$), we get that ${\mathbb{E} \big| \big\langle \frac{\mathbf{x}_1}{\Vert{\mathbf{x}_1}\Vert}, \frac{\mathbf{x}_2}{\Vert{\mathbf{x}_2}\Vert} \big\rangle \big| \approx \frac{1}{\sqrt{p}} \to 0} \Longrightarrow {\mathbb{E} \angle(\mathbf{x}_1,\mathbf{x}_2)\to 90^{\circ}}$, corresponding to the *maximal* task dissimilarity in Evron et al. (2022), where forgetting is *minimal*.


  We originally compared our results to Evron et al. (2022) in the discussion section. Following the reviewer’s question, we included the discussion above in the revised manuscript to further elaborate on these aspects.

Reviewer comment (Theory):
The main issue that I see in the paper is the limited scope of the provided analysis. I am fine with simple experiments in a theoretical paper but the upper bound of forgetting is very specific to the linear regression task.

Author response:
We thank the reviewer for their time and for engaging with us in this discussion.

-  **Regarding the simplicity of the analyzed model**.
Respectfully, we must disagree that the simple linear regression scenario that we analyze makes our paper *"too narrow for an ICLR audience"*.
Previous theoretical CL papers, published at top venues, often analyze linear models (some even make additional data assumptions, e.g., random isotropic features, which we do not). 
For instance, see
[Doan et al. (AISTATS 21)](https://proceedings.mlr.press/v130/doan21a.html),
[Evron et al. (COLT 22)](https://proceedings.mlr.press/v178/evron22a.html),
[Goldfarb & Hand (AISTATS 23)](https://proceedings.mlr.press/v206/goldfarb23a.html),
[Evron et al. (ICML 23)](https://proceedings.mlr.press/v202/evron23a.html),
[Lin et al. (ICML 23)](https://proceedings.mlr.press/v202/lin23f.html),
[Swartworth et al. (NeurIPS 23)](https://openreview.net/forum?id=X25L5AjHig).  
  We are not aware of *any* existing theoretical CL result on CNNs, likely due to the hardness of such analysis. We thus believe that our linear model is a more suitable *starting point* for theoretical analysis on the interplay between overparameterization and task similarity in CL.

Reviewer comment (Theory):
In theorem 3, the proxy for overparametrization is defined as $1 - \frac{d}{p}$, where $d$ is the rank of the input data and $p$ is the data dimensionality. It makes sense in linear regression since the number of parameters is the same as the data dimension, i.e., $p$. This is true to some extent in the MLP layers (at least they are correlated). However, none of the derivations hold in the case of the convolutional layers where there are shared parameters and $p$ cannot be a surrogate for the number of parameters. I encourage the authors to evaluate their upper bound in CNNs.

Author response:
The reviewer also asked (Q2,Q3) whether we tried to evaluate our upper bound *“in more challenging scenarios”* (e.g., CNNs) or with different notions of task similarity. The answer is that we did not, but neither did we claim that our analysis simply extends to such cases. For example, permutation benchmarks like the one we aim to characterize, are not very suitable for CNNs, since the permutations significantly interfere with the spatial assumptions convolutional models implicitly make (shift-equivariance of representations and shift-invariance of the classification).  
Again, our aim is that our analysis can be used as a counterexample to the practical common belief that overparameterization always mitigates forgetting and as a demonstration of a more nuanced interplay between overparameterization and task similarity. More generally, when doing theoretical research, we believe that one must start from the simplest models as we did, before addressing more complex scenarios as the reviewer rightly suggested.

Following this discussion, we now include these suggestions in the "Future work" section at the end of our revised manuscript.

Reviewer comment (Theory):
The definition of task similarity is very limited to the rotation of a subset of the data dimensions. All of the derivations are based on this initial assumption. I believe a good theoretical paper on this subject should expand this definition so that more real-world cases can be included in the analysis. My comment is the same for the permuted-NIST scenario. A simple permutation of a subset of pixels is not enough to back the main message in this more challenging scenario.

Author response:
- **Regarding our notion of task similarity.** As our paper explains, the notion we analyzed (i.e., Dim. of the Transformed Subspace) *analytically* generalizes the completely random rotation model of [Goldfarb & Hand (2023)](https://proceedings.mlr.press/v206/goldfarb23a.html) (on one extreme); and relates to the principal angles notion from [Doan et al. (2021)](https://proceedings.mlr.press/v130/doan21a.html) and [Evron et al. (2022)](https://proceedings.mlr.press/v178/evron22a.html) (see our revised discussion). *Practically*, it closely characterizes the number of permuted pixels in randomly-permuted benchmarks (see Figure 2 in the seminal paper of [Kirkpatrick et al. (2017)](https://arxiv.org/abs/1612.00796)).  
We agree that it should be interesting to investigate related phenomena with other task similarity notions, and hope that our paper will pave the way for such future work.

Reviewer comment (Theory):
Is the overparametrization proxy enough to provide a similar analysis in the CNN case? i.e., can we just substitute the $p$ with the number of parameters in a CNN layer?

Author response:
The reviewer also asked (Q2,Q3) whether we tried to evaluate our upper bound *“in more challenging scenarios”* (e.g., CNNs) or with different notions of task similarity. The answer is that we did not, but neither did we claim that our analysis simply extends to such cases. For example, permutation benchmarks like the one we aim to characterize, are not very suitable for CNNs, since the permutations significantly interfere with the spatial assumptions convolutional models implicitly make (shift-equivariance of representations and shift-invariance of the classification).  
Again, our aim is that our analysis can be used as a counterexample to the practical common belief that overparameterization always mitigates forgetting and as a demonstration of a more nuanced interplay between overparameterization and task similarity. More generally, when doing theoretical research, we believe that one must start from the simplest models as we did, before addressing more complex scenarios as the reviewer rightly suggested.

Following this discussion, we now include these suggestions in the "Future work" section at the end of our revised manuscript.

Reviewer comment (Experiments):
Have the authors tried to evaluate their upper bound in more challenging scenarios?

Author response:
The reviewer also asked (Q2,Q3) whether we tried to evaluate our upper bound *“in more challenging scenarios”* (e.g., CNNs) or with different notions of task similarity. The answer is that we did not, but neither did we claim that our analysis simply extends to such cases. For example, permutation benchmarks like the one we aim to characterize, are not very suitable for CNNs, since the permutations significantly interfere with the spatial assumptions convolutional models implicitly make (shift-equivariance of representations and shift-invariance of the classification).  
Again, our aim is that our analysis can be used as a counterexample to the practical common belief that overparameterization always mitigates forgetting and as a demonstration of a more nuanced interplay between overparameterization and task similarity. More generally, when doing theoretical research, we believe that one must start from the simplest models as we did, before addressing more complex scenarios as the reviewer rightly suggested.

Following this discussion, we now include these suggestions in the "Future work" section at the end of our revised manuscript.

Reviewer comment (Experiments):
Have the authors tried to test their upper bounds when the definition of the task similarity is different? i.e., more subtle semantic distribution shifts in the data. e.g. CIFAR-100 Superclass.

Author response:
The reviewer also asked (Q2,Q3) whether we tried to evaluate our upper bound *“in more challenging scenarios”* (e.g., CNNs) or with different notions of task similarity. The answer is that we did not, but neither did we claim that our analysis simply extends to such cases. For example, permutation benchmarks like the one we aim to characterize, are not very suitable for CNNs, since the permutations significantly interfere with the spatial assumptions convolutional models implicitly make (shift-equivariance of representations and shift-invariance of the classification).  
Again, our aim is that our analysis can be used as a counterexample to the practical common belief that overparameterization always mitigates forgetting and as a demonstration of a more nuanced interplay between overparameterization and task similarity. More generally, when doing theoretical research, we believe that one must start from the simplest models as we did, before addressing more complex scenarios as the reviewer rightly suggested.

Following this discussion, we now include these suggestions in the "Future work" section at the end of our revised manuscript.

Reviewer comment (Presentation):
The model (i.e. single linear projection) and algorithm (gradient descent, full-batch gradient descent?!) are not stated in the introduction. Information about the regime to which the analytical result applies is scattered throughout the paper, which makes it difficult to contextualise the results.

Author response:
We apologize about the lack of clarity on the aspects mentioned by the reviewer. Below we address these questions and specify how the revised manuscript clarifies them.  

  We study the simplest continual learning procedure - starting from a zero initialization $\textbf{0}_p$ and iteratively minimizing the current task’s loss using (S)GD initialized with the previous task’s solution. In each iteration, we assume that learning “converges” to the limit solution, obtained by (S)GD with *any* batch size (as long as the learning rate is small enough; see Section 2.1 in Gunasekar et al. (2018)). This is a similar procedure to the one studied in several analytical papers (e.g., Evron et al. (2022); Goldfarb and Hand (2023)).

  As can be seen in our revised manuscript, we now (1) specifically mention that we study an (S)GD scheme in the introduction; (2) changed the title of Section 2.2 to “The Analyzed Learning Scheme and its Learning Dynamics”; (3) specifically write before Scheme 1 that this is the scheme that we analyze; (4) added a footnote to the analytical iterates, explaining that any batch size can be use as long as the learning rate is small enough; (5) highlight right after the scheme that we do not actually compute pseudoinverses; (6) and finally, specifically refer to Scheme 1 in our main Theorem 3.

Reviewer comment (Theory):
It would be really helpful if all assumptions of the analytical result would be stated explicitly and collected within one section of the paper. For example, the ${\bf 0}$ initialisation and resulting convergence to the global, minimum-norm solution of the first task is hidden in Scheme 1 but crucial to understand the extent of the analytical result.

Author response:
We apologize about the lack of clarity on the aspects mentioned by the reviewer. Below we address these questions and specify how the revised manuscript clarifies them.  

  We study the simplest continual learning procedure - starting from a zero initialization $\textbf{0}_p$ and iteratively minimizing the current task’s loss using (S)GD initialized with the previous task’s solution. In each iteration, we assume that learning “converges” to the limit solution, obtained by (S)GD with *any* batch size (as long as the learning rate is small enough; see Section 2.1 in Gunasekar et al. (2018)). This is a similar procedure to the one studied in several analytical papers (e.g., Evron et al. (2022); Goldfarb and Hand (2023)).

  As can be seen in our revised manuscript, we now (1) specifically mention that we study an (S)GD scheme in the introduction; (2) changed the title of Section 2.2 to “The Analyzed Learning Scheme and its Learning Dynamics”; (3) specifically write before Scheme 1 that this is the scheme that we analyze; (4) added a footnote to the analytical iterates, explaining that any batch size can be use as long as the learning rate is small enough; (5) highlight right after the scheme that we do not actually compute pseudoinverses; (6) and finally, specifically refer to Scheme 1 in our main Theorem 3.

Reviewer comment (Writing):
I think the paper could benefit from making it more explicit that the analysis focuses on the dimensionality and overlap with respect to the manifold of the data distribution, which is often called overparametereised, but should not be confused with the overparameterisation deep networks (i.e. wide hidden layers).

Author response:
We agree with the reviewer and we added this to the “Limitations and Future work” section in our revised manuscript.

Reviewer comment (Theory):
Resulting from point 3, the model can not be used to make predictions about the underparameterised regime, e.g. bottlenecked networks.

Author response:
We see two possible interpretations of this statement, and we are slightly confused by both.  
  In case the reviewer was referring to our theoretical analysis, we note that we work in the setting of linear regression, in which there is no rich/lazy regime distinction. As far as we are aware, the lazy vs rich distinction applies only to non-linear parameterizations, where the initialization scale *qualitatively* affects the final solution (e.g., minimizing the L2 norm in "lazy", vs L1 norm in "rich", in the squared linear regression model of Woodworth et al. (COLT 2020)). In contrast, as can be deduced from our Eq. 3, in linear regression, changing the initialization just shifts the final solution location (along a direction orthogonal to the data manifold) — which is not a qualitative change.   
  In case the reviewer meant to the application of our linear regression results to neural networks, then, as far as we are aware, the only formal way to prove that linear regression results directly apply to neural networks is to be in a regime where the neural network is approximately linear, such as the NTK regime. Therefore, we only claim our results can be directly applied to neural networks in this case. Perhaps some of our conclusions also apply to the rich regime, but we do not see how this could be theoretically proven.

Reviewer comment (Experiments):
I am not entirely sure if the author’s version of permuted MNIST is suitable to study changes in task similarity that are comparable to DOTS, as vanilla MNIST data is on a very low dimensional data manifold and large parts of the information is centred on the middle of the picture. I think I would prefer a simulation study that uses artificial data as in Figure 4, with well controlled DOTS in two layer networks.

Author response:
The permuted MNIST experiments are intended to illustrate the ideas of the analysis in a more realistic image classification scenario. Note that the permuted image setting is an analogous version of the DOTS simulation where random permutation matrices are used instead of random orthogonal matrices. We recognize that MNIST images lie on a low dimensional manifold and this notion is what motivated our decision in the analysis to have data of low rank d; in this case, low effective dimensionality is a similarity between MNIST and the simulated data.

Reviewer comment (Theory):
Could you please hint me at the Theorem in Gunasekar et al. (2018) from which equation (3) is derived?

Author response:
We apologize about the lack of clarity on the aspects mentioned by the reviewer. Below we address these questions and specify how the revised manuscript clarifies them.  

  We study the simplest continual learning procedure - starting from a zero initialization $\textbf{0}_p$ and iteratively minimizing the current task’s loss using (S)GD initialized with the previous task’s solution. In each iteration, we assume that learning “converges” to the limit solution, obtained by (S)GD with *any* batch size (as long as the learning rate is small enough; see Section 2.1 in Gunasekar et al. (2018)). This is a similar procedure to the one studied in several analytical papers (e.g., Evron et al. (2022); Goldfarb and Hand (2023)).

  As can be seen in our revised manuscript, we now (1) specifically mention that we study an (S)GD scheme in the introduction; (2) changed the title of Section 2.2 to “The Analyzed Learning Scheme and its Learning Dynamics”; (3) specifically write before Scheme 1 that this is the scheme that we analyze; (4) added a footnote to the analytical iterates, explaining that any batch size can be use as long as the learning rate is small enough; (5) highlight right after the scheme that we do not actually compute pseudoinverses; (6) and finally, specifically refer to Scheme 1 in our main Theorem 3.

Reviewer comment (Theory):
What assumptions are made on the size of batches and the size of the learning rate?

Author response:
We apologize about the lack of clarity on the aspects mentioned by the reviewer. Below we address these questions and specify how the revised manuscript clarifies them.  

  We study the simplest continual learning procedure - starting from a zero initialization $\textbf{0}_p$ and iteratively minimizing the current task’s loss using (S)GD initialized with the previous task’s solution. In each iteration, we assume that learning “converges” to the limit solution, obtained by (S)GD with *any* batch size (as long as the learning rate is small enough; see Section 2.1 in Gunasekar et al. (2018)). This is a similar procedure to the one studied in several analytical papers (e.g., Evron et al. (2022); Goldfarb and Hand (2023)).

  As can be seen in our revised manuscript, we now (1) specifically mention that we study an (S)GD scheme in the introduction; (2) changed the title of Section 2.2 to “The Analyzed Learning Scheme and its Learning Dynamics”; (3) specifically write before Scheme 1 that this is the scheme that we analyze; (4) added a footnote to the analytical iterates, explaining that any batch size can be use as long as the learning rate is small enough; (5) highlight right after the scheme that we do not actually compute pseudoinverses; (6) and finally, specifically refer to Scheme 1 in our main Theorem 3.

Reviewer comment (Theory):
Why does Theorem 3 have the assumption of p >= 5?

Author response:
Theorem 3 assumes that the dimension holds $p \ge 5$ (which can actually be improved to $p \ge 4$, we fixed it now). This is because many of our proofs (e.g., in Lemma 8) yield expressions involving a $(p-3)$ factor in their denominators. However, this is a mild restriction when testing for overparameterization effects like we do.

Reviewer comment (Theory):
Why can you assume that a randomly sampled data matrix X is identical to the worst case in Figure 2?

Author response:
Please notice that the data matrix $\mathbf{X}$ in Figure 2 is indeed random, but we constrain its nonzero singular values to be identical. In Eq. (4), we present an upper bound on the forgetting, and explain that it is *sharp* when the nonzero singular values are identical. This exemplifies that our analysis is tight, and that worst-case matrices are easy to generate.  
  We added an explicit reference to Eq. (4) in the caption of Figure 2.

Reviewer comment (Evaluation):
The smallest expected forgetting in the interpolation regime (except for values close to 0 DOTS) is larger than the largest expected forgetting in the overparametereised regime. Why is that the case and how is it relevant to the analysis?

Author response:
This is an interesting question. We'll try to answer it briefly and intuitively. When we are highly overparameterized, the directions randomly rotated in the 2nd task are going to be mostly orthogonal to their corresponding directions in the original first task (thus incurring little forgetting). Near the interpolation regime, randomly rotated directions will have more interference with the original directions, since they will not become orthogonal w.h.p., and forgetting would be higher.  
  We elaborated on that in the discussion section of our revised manuscript.

Reviewer comment (Theory):
Do the authors think that their results do generalise to learning more than two tasks? I think that would clearly contradict some of the assumptions and thus maybe should be listed as a limitation of the work.

Author response:
This is an important question and we added the following discussion to a new section on “Future work” in the revised manuscript.  
  Our analysis has primarily examined settings with $T=2$ tasks. Extending these analytical results to $T\ge 3$ tasks poses an immediate challenge. The complexity of our analysis, which already required intricate proofs, suggests that tackling this extension may be considerably difficult. Moreover, the convergence analysis presented in Evron et al. (2022) for learning $T\ge 3$ tasks cyclically has proven to be notably more challenging than that for $T=2$ tasks, and was further improved in a followup paper (Swartworth et al. 2023).

Reviewer comment (Presentation):
Is it maybe possible to use a log-scaled axis in Figure 3 instead of using subplots to represent the results?

Author response:
We tried that, but the result was not as good as the current scaling. However, we slightly edited the figure and we believe it looks better now.

Reviewer comment (Presentation):
Is it possible that the x-axis of Figure 3, bottom right is labelled incorrectly? Values from 0 to 0.1 seem to be missing.

Author response:
We fixed it, thank you.

Reviewer comment (Experiments):
For Figures 6 and 7, how are neural networks initialised? Do they operate in the rich or lazy regime?

Author response:
The networks are initialized with centered Gaussian weights. The MLP’s in Figure 6 and 7 are sufficiently wide to enjoy generalization benefits of lazy training.