Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
Authors: Joonkyu Kim
Yejin Kim
Jy-yong Sohn
Venue: ICML 2025 poster (2025)
Keywords: Continual Learning
Representation Learning
Representation Forgetting
Primary area: theory->deep_learning
TL;DR: We introduce a new metric, representation discrepancy, to track how internal knowledge of an AI model changes over time. Using this, we found that forgetting occurs faster in deeper layers of the network and that wider networks tend to forget less.

Abstract: In continual learning scenarios, catastrophic forgetting of previously learned tasks is a critical issue, making it essential to effectively measure such forgetting. Recently, there has been growing interest in focusing on representation forgetting, the forgetting measured at the hidden layer. In this paper, we provide the first theoretical analysis of representation forgetting and use this analysis to better understand the behavior of continual learning. First, we introduce a new metric called representation discrepancy, which measures the difference between representation spaces constructed by two snapshots of a model trained through continual learning. We demonstrate that our proposed metric serves as an effective surrogate for the representation forgetting while remaining analytically tractable. Second, through mathematical analysis of our metric, we derive several key findings about the dynamics of representation forgetting: the forgetting occurs more rapidly to a higher degree as the layer index increases, while increasing the width of the network slows down the forgetting process. Third, we support our theoretical findings through experiments on real image datasets, including Split-CIFAR100 and ImageNet1K.

Measuring Representational Shifts in Continual Learning:
A Linear Transformation Perspective
Joonkyu Kim
Yejin Kim 2
Jy-yong Sohn
Abstract
the human learning process; individual continually ac- quires new skills and knowledge throughout their lifetime. However, unlike humans, who can retain and recall good amount of memories from the past, the trained models expe- rience a significant decline in performance on the previous tasks as they adapt to the new ones. This phenomenon, known catastrophic forgetring, is described as the model losing knowledge of the previous tasks and is typically eval- uated by measuring the drop in accuracy on the previous tasks (McClelland et al., 1995; McCloskey & Cohen, 1989).
In continual learning scenarios, catastrophic for- getting of previously learned tasks is critical issue, making it essential to effectively measure such forgetting. Recently, there has been growing interest in focusing on representation forgetting. the forgetting measured at the hidden layer. In this paper, we provide the first theoretical analysis of representation forgetting and use this analy- sis to better understand the behavior of contin- ual learning. First, we introduce a new metric called representation discrepancy, which mea- sures the difference between representation spaces constructed by two snapshots of a model trained through continual learning. We demonstrate that our proposed metric serves as an effective sur- rogate for the representation forgetting while re- maining analytically tractable. Second, through mathematical analysis of our metric, we derive several key findings about the dynamics of repre- sentation forgetting: the forgetting occurs more rapidly to higher degree as the layer index in- creases, while increasing the width of the network slows down the forgetting process. Third, we sup- port our theoretical findings through experiments on real image datasets, including Split-CIFARION 100 and ImageNetIK.
Internally, the representations of the model, that is, the hid- den activations, undergo a drastic shift during continual learning as well (Caccia et al., 2021). This calls for a sepa- rate term that indicates the loss of knowledge for previous tasks in terms of the hidden representations. In this paper, we will use the term representation forgetting of task to refer to the degradation in a model's 's ability to produce high- quality features for task t. Note that we are not the first to use the term representation fortrrttatioooogtttitiiii forgetting, as it has been used in the literature to indicate representational shifts during the continual learning process in general (Davari et al., 2022: Luo et al., 2023; Zhang et al., 2022; Murata et al., 2020).
Understanding the behavior of the representation forgetting, which is measured at the level of the intermediate layers, is equally important to that of the observed forgetting (forget- ting measured through the model's final output), because (1) there are situations such as in unsupervised continual learning (Rao et al., 2019; Fini et al., 2022) where our inter- est is in the inner representations of the model rather than the model's output, and (2) understanding the representa- tion forgetting itself can lead to better understanding of the observed forgetting.
1. Introduction
Continual learning, referred as lifelong learning, involves training models a sequence of tasks with the aim of achieving good performance for all tasks (Chen & Liu, 2018; Hadsell et al., 2020; Kudithipudi et al., 2022). This framework is particularly compelling as it closely mimics
Although there have been several empirical results on the representation forgetting under continual learning (Caccia et al., 2021; Davari et al., 2022; Hess al., 2023; Luo et al., 2023), no clear justification was provided in terms of the- ory. Our work addresses this gap by offering one of the first theoretical analyses of the representation forgetting. To achieve this, we introduce a novel metric, representation discrepancy, which quantifies the minimum alignment error of two representation spaces under linear transformation. We show that this metric not only effectively measures rep-
'Department of Electrical & Electronic Engineering, Yonsei University, Scoul, South Korea 'Department of Statistics and Data Science, Yonsei University, Scoul, South Korea. Correspondence to: Jo-yong Sohn <jysohnl 108@ @ arr
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Mcasuring Representational Shifts in Continual Learning:
A Lincar Transformation Perspective
resentation forgetting, but also enables analytical feasibility.
assumes the model to be linear, with each task framed as regression problem (Ding et al., 2024; Zhao al., 2024; Evron et al., 2022; Lin et al., 2023; Li et al., 2023; Heckel, 2022). Here, the drift in the weights during the contin- ual learning process is generally used as a measure of the forgetting or the generalization error. For instance, Evron et al. (2022); Lin et al. (2023); Ding et al. (2024) exam- ines how task sequence properties (e.g., number of tasks, task ordering, task similarity) influence the error. Mean- while, Heckel (2022); Li et al. (2023); Zhao et al. (2024) focuses more on the impact of regularization on the error. Another line of work employs the well-established theo- retical teacher-student framework (Saad & Solla, 1995;;a; Yoshida & Okada, 2019; Goldt et al., 2019; Straat et al., 2022) to study continual learning (Lee al., 2021; Asanuma et al., 2021). In these works, each teacher network repre- sents distinct task, and the similarity between teacher networks is used as proxy for task similarity. These stud- ies further derive connections between task similarity and the generalization error of the student network. Other works leverage the Neural Tangent Kernel (NTK) regime, where the continual learning setup modeled as a recursive kernel regression (Bennani et al., 2020; Doan et al., 2021; Yin et al., 2020; Karakida & Akaho, 2021).
Using the representation discrepancy, we focus on com- paring the representations of two models h, and he where he represents the model trained incrementally up to task i, and analyze how the representation evolves over the continual learning process. Our main contributions are summarized as below.
Main Contributions:
We propose novel metric representation discrepancy, denoted by D (he. At). which measures the discrep- ancy of the representation spaces of models he and h4+2r (for task at layer kt), when an optimal linear transformation T is applied to align the spaces. We show that (he: At) serves as an effective surrogate for the representation forgetting of task t while also being analytically feasible.
We construct an upper bound U./ (At) on D (h,, At) using the theoretical framework proposed by Guha & Lakshman (2024) and derive three main theoretical findings. First, U. (At) exhibits two distinet phases: the forgetting phase, during which it monotonically increases with At (the number of tasks learned af- ter task t), and the saturation phase, where it grad- ually converges to the asymptotic value. Second, the asymptotic representation discrepancy Uho == Ar- U/ (At) is linearly proportional to the size' of the representation space. Third, U (At) enters the saturation phase more quickly in the upper layers (i.e., higher k) compared to the lower layers, while increas- ing the width of the model delays the transition to the saturation phase.
The most relevant to our work is Guha & Lakshman (2024), where they proposed using perturbation analysis to derive error bounds on catastrophic forgetting. Specifically, they viewed each learning step in continual learning as a pertur- bation of the weights of the model and analyzed how this affected the overall shift in the model's s output on the pre- vious tasks. By using the maximum shifted distance of the output on the previous tasks as a proxy for measuring for- getting, they established relationships between the model's ss depth and width to the upper bound on the forgetting error.
Our work is similar to Guha & Lakshman (2024) as we also view the continual learning process as a series of perturba- tion the weights. However, the key distinction between our work and Guha & Lakshman (2024) (or in fact any other works we mention here) lies in our focus on measuring the forgetting of the representations for the previous task. Simply measuring the drift in the activations or the hidden layer weights of the model during the continual learning process fails to fully capture the representation forgetting, as the internal features may change but end up producing the same output. This leads us to propose a novel metric, the representation discrepancy D/(he, At), which quantifies the minimum alignment error between two representation spaces under a linear transformation. Note that one should not confuse representation discrepancy with discrepancy distance introduced in Mansour et al. (2009). Discrepancy distance refers the distance in terms of data distributions whereas representation discrepancy refers to the distance in terms of the representation spaces. A formal definition of
We support our theoretical findings through ex- periments on real image datasets, including Split- CIFAR100 and ImageNetIK. Furthermore, we empiri- cally observe a strong linear relationship between the layer index i and the size of the representation space. Combined with our second theoretical result shown above, we conclude that the asymptotic representation discrepancy increases linearly with the layer index k.
2. Related Works
2.1. Continual Learning Theory
Several prior works have conducted theoretical analyses of continual learning. These works can be categorized ac- cording to the modeling of the continual learning setup and their analytical framework. A simple yet effective approach
'The size of the k-th layer representation space of task is the norm its biggest feature, as defined in Def. 2.
2

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
Dj (h2, At) is provided in Def. 4.
3. Problem Setup
Other popular approaches include leveraging the PAC- Bayesian framework to establish error bounds for future tasks based on observed losses on current tasks (Pentina & Lampert, 2014) or to derive lower bounds on the mem- ory requirements in continual learning (Chen et al., 2022). Additionally, some works focus on studying the continual learning setup itself, decomposing the problem into within- task prediction and out-of-diste of-iiiiiibttiinn detection (Kim et al., 2022), or connecting it with other problems such bi-level optimization or multi-task learning under specific assump- tions (Peng et al., 2023). Researchers have also explored sample complexity (Cao et al., 2022; Li et al., 2022), the impact of pruning (Andle& & Yasaei Sekeh, 2022), and the ef- feets of contrastive loss on generalization (Wen et al., 2024) in the continual learning regime.
We consider the continual learning setup where we consecu- tively learn N tasks, and the dataset for each task is denoted by Pi.... DN. The continual learner is assumed to be L-layer ReLU network, which is trained sequentially in the increasing order: from D to DN Each task involves supervised learning on D, = { y
having sam- ples with features æ; E IR and labels y E 1R4, We define X₂ = {a]] and Y2 = {3/ ) The model trained up to task is denoted as h2 and will take on the form
hy(x)) W @W. WW (x))).
where e represents the ReLU activation and W repre- sents the weight matrix at layer k. The width of the k-th hidden layer is denoted by wh- Thus, the size of each weight matrix is as follows: W E Rm xd, and WL E R while the rest of the hidden weight ma- trix has size W R for k E (2, L 1}. We mainly use superscript k: [T.] to denote the layer in- dex, and subscript t E [N] to denote the task, where we use the notation [n]] := {1, .n. throughout the paper. In addition, since we will be exclusively dealing with the intermediate layers of the model, we use h (x) to denote the output of the k-th layer of hrs when the input to the model is æ. Note that h (x) = hx(x) holds.
2.2. Representation Forgetting
The concept of representation forgetting in continual learn- ing is not yet well-established in the literature, leading the introduction of various measurements. The most relevant to our work is Davari et al. (2022), where they propose to use the drop in linear probing accuracy as a measure of the representation forgetting. Similarly, Zhang et al. (2022) de- vises protocol for evaluating representations in continual learning, which involves computing the average accuracy of the model's features across all previous tasks. Other metrics, such canonical correlation analysis and centered kernel alignment are widely used to compare the repre- sentation spaces between models (Kornblith et al., 2019). Ramasesh et al. (2020) employ these metrics to empiri- cally demonstrate that catastrophic forgetting arises primar- ily from higher-layer representations near the output layer, where sequential training erases earlier task subspaces.
Now, we formally define the representation space for task t, along with the notions of size and distance of the represen- tations spaces. Note that this will clarify the ambiguity of the word representation space while also helping us to gain a more intuitive understanding of our theorems.
Definition (Representation Space for Task t). For a given model by trained incrementally up task t', the k-th layer representation space of he for task is defined as
Additionally, several studies have explored representation forgetting in different continual learning setups. For in- stance, Caccia et al. (2021) and (Zhang et al., 2024) in- vestigates representation forgetting in either online or un- supervised continual learning scenarios, while Luo et al. (2023) and Hess al. (2023) investigates this phenomenon in the context of Natural Language Processing (NIPP mod- els. Specifically, Luo et al. (2023) introduces three metrics- overall generality destruction, syntactic knowledge forget- ting, and semantic knowledge forgetting-to assess different aspects of forgetting for NLP models.
R+(he) == (x) x X.).
Definition (Size of Representation Space). The size of the k-th layer representation space of model by for task t defined
[|R] (he )I max (ax (x) x X.).
Definition 3 (Distance between Representation Spaces). Given task index layer index k, and models hi, and hig- the distance between representation spaces RY (he,) and RA: (hr,) is defined as
should be noted that all these studies primarily focus on the empirical analysis of representation forgetting in contin- ual learning, without presenting corresponding theoretical framework. To the best of our knowledge, our work pro- vides a theoretical analysis of representation forgetting for the first time.
d(R (hr,), (Ir,)) = max (x) hr, (x) 11: x X
Intuitively, the size of the representation space (hy ) is de- termined by the norm of the biggest feature in Rt (hyr). Fur- thermore, the distance between two representation spaces
3

---

Mcasuring Representational Shifts in Continual Learning: A Lincar Transformation Perspective
Rf(h,, and R (h,g) ) is measured by the distance between the most distant representations for the same input a; where the maximization is taken over the t-th task dataset.
tively. Formally, this would be evaluating the performance difference
A P (At) (At) = P(C (X.),
To provide glimpse into our approach to analyzing the representation forgetting, we offer a remark on the relation- ship between the metric continual learning error used a related work (Guha& & Lakshman, 2024) and our definition of the distance between two representation spaces.
(2) - -P(C o (Xt)) Y:),
where P denotes a chosen performance metric, e.g., ac- curacy, and C
C(C* (Xi), Yi) and arg min
C+ CRERAEX"R
Remark 1. Guha & Lakshman (2024) use
C CH arg min L(C* o (Xt)) Y) are the opti- C*ER-yx=L
d(R) (hr,), R, (he,)) = max ( hig (z))12: x X.) (1)
mal linear classifiers (for hy and h respectively) with respect to loss function C.
Note that mathematically analyzing Eqn. 2, which requires analyzing the performances of complex multi-layer neural networks, is challenging. A practical alternative inspired by the metric proposed by Guha & Lakshman (2024) shown in Eqn. 1, involves using the maximum distance between the outputs of the two models, written as
as a surrogate for the forgetting of task t measured for the model hit At trained on additional AI tasks. The underlying idea of using equation 1 in (Guha & Lakshman, 2024) is that, if the output of b2 is close to ht for all x E Xt, then hi-at would perform well on task t. We will use a similar approach, but we measure the discrepancy of the representation space at hidden layers k instead of the final output layer L; note that equation is special case of Def. 3 when we set k = L.
max x
|1C) o (x)) - -Citar (+)a(
(3)
a proxy for the representation forgetting.
While the metric in Eqn. 3 avoids direct evaluation of the performance P. it still requires computing the optimal clas- sifiers C and C++A However, deriving these classifiers is nontrivial. Thus, there is a need for a metric that not only serves as an effective surrogate for representation forgetting but also analytically feasible. This motivates our proposed metric, formally defined as below.
4. Proposed Metric
In this section, we propose the representation discrepancy, denoted as D. (he, At). which measures the discrepancy of the k-th layer representation spaces of models he and h4+a: (before/after learning additional At tasks) for task t. Note that discrepancy between two representation spaces is measured as the distance between them after aligned through a linear transformation. We use D (hg, At) as a proxy to the forgetting of representations for task over the continual leaning process, and analyze it in the upcoming sections. We first provide the motivation for such definition in Sec. 4.1 and then provide a formal definition in Sec. 4.2.
4.2. Representation Diserepancy
Below we propose a novel metric, the representation dis- crepancy,
practical surrogate for the representation forgetting of models trained in the continual learning setup. See Fig. 1 for the illustration of our proposed metric. Definition 4 (Representation Discrepancy). Let h(x) be an L-layer ReLU Network
4.1. Motivation
W o WW with each hidden layer ie having width wi- Suppose is trained sequentially on the datasets Di,,.., DN and by represents the model trained up to D,. For a given target task t, the k-th layer representation discrepancy for model hy after trained on additional At tasks, is defined as
As the model incrementally learns from task to task N, the learner gets total of N models, denoted by h. ,..- Our goal is to mathematically analyze the extent to which the model he+ (learned up to task + At), forgets the representations useful for conducting task f, where At > O.
Dj (hr, At) = min T
(b++ss)),
Note that numerous studies have proposed different meth- ods to quantify the representation forgetting. One famous approach, introduced in Davari et al. (2022), measures the representation forgetting in terms of the degradation of the performance of the linear probing method. Specifically, to assess the representation forgetting at the k-th hidden layer of model M44 on task f, one compares the perfor- mances of optimal linear classifiers Ch E Id trained on dataset D, when placed on top of hy and h respec-
=T
where T E R applies a linear transformation to all the features in R7 (h++ar).
Intuitively, at layer k. our metric (h,, At) measures the minimum worst-case mis-alignment between the k-th layer representation spaces of h2 and h where the minimiza- tion is over linear transformation T and the maximization is over samples in task t.
4

---

Mcasuring Representational Shifts in Continual Learning: A Lincar Transformation Perspective
h
(x)
110
ReLU Network
X10
ResNet
0.8
R. (h(+
0.6
2
Mirsell))
00.4
R2 (his)
At
0.0
50
75
100
0
100
Epocc
Epoch
Figure 1. Visual interpretation of the representation discrepancy Df (he, At) = min d(Rī (lx), TRR (N++))))) defined by us "
Figure 2. An empirical result that supports Assumption 1. which is tested on two networks (a fully connected ReLU network and ResNet) using real image datasets (CIFARI00 and ImageNet. respectively). When & and E' N. we plot how the difference of W.r and TW varies as we train linear transformation T for 100 epochs, for cach k E {1, (1,...,9 9). Note that the plot is averaged across 50 random seeds. The results show I T converges to zero. This implies that there exists a linear transformation T that closely approximates two weight matrices of each layers, demonstrating that Assumption holds in practice.
in Def. 4. Here, the linear transformation T' is optimized to re- duce the distance between the representation space (hx) = (M, (x)) and R2 (1)+ (1/ (1)))
We argue that DH (h,, At) serves effective surrogate for the representation forgetting, following the logic stated in Remark 1. Note that when D (h+, At) is small, there exists a linear transformation T E 12 such that
(ar) v T (()) E
width m. Suppose h((x)) is trained sequentially on the DD and let W represent the weight ma-
datasets DI,-.. ,D, trix of layer is after learning task t. For each k [L] and for any task indices t, t' E [M] satisfying < t,, assume that there exists linear transformation T E RRw such that the following holds:
Consequently, the accuracy of the linear probing on hy can be matched by that of the linear probing on h,4 since for any linear classifier C1 E x we have
oio Ne C1 a (T* o h[tai) C2 C20hitt
for some C2 Rd, Therefore, the lower the represen- tation discrepancy D. (h,, At), the lower the representation forgetting. Equivalently, if the model undergoes high repre- sentation forgetting, then it has high value of D. (h+, At). Our empirical observations also show that there is strong linear relationship between the representation forgetting and the representation discrepancy. See Fig. in the Appendix.
TW W:
(4)
This assumption implies that for a randomly initialized L- layer ReLU network updated through the continual learning process, the weight matrices of layer at two different task index t, l' can be aligned through linear transformation T.
In Fig. 2, show experimental results that support As- sumption 1. Specifically, the left figure shows the results when a 9-layer ReLU network is trained on the CIFARION dataset split into N = 20 tasks, while the right figure shows the results when a ResNet is trained on the ImageNet dataset split into N = 50 tasks. For each case, we focus on two models, h2 and he, where the task indices are set / = 1 and t' N. For each layer index k. collect the weight matrices of both models: W and W N- Then, we train a lin- ear transformation T to minimize the loss || W TW I| for 100 epochs and report the training loss curve, averaged across 50 random seeds. As for experimenting with the ResNet model, we generally apply the same procedure, but with an additional flattening procedure to address the differ- ence in the shape of the weights of the convolutional layers; details are provided in the Appendix B.
5. Theory
In this section, we analyze the representation discrepancy (he, At) in Def. 4. First, in Sec. 5.1, we provide an upper bound (AV) on DJ (he, At), as stated in Thm. Then, in Sec. 5.2, we analyze the behavior of the upper bound U.h (At), which gives us insights on the representa- tion forgetting. including (1) the amount of representation forgetting in the asymptotic case (i.e,, when At - oc) and (2) the rate of the representation forgetting.
5.1. Bounding the Representation Discrepancy
Before stating our main result on bounding the representa- tion discrepancy DŹ (he, At), we start with an assumption we introduced for ease of analysis, which is empirically supported in our observation in Fig. 2.
While Assumption may not universally apply to all archi- tectures or scenarios, additional experiments with Vision Transformers (Dosovitskiy et al., 2020) show similar re- sults (see Fig. 11 in the Appendix). Together, these findings
Assumption 1. Let (xx) be a randomly initialized L-layer ReLU Network with each hidden layer k having the same
5

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
suggest that there exists T with TW W in practice.
(f(A
/2 (Atmu (1 + )u((oz)
Now, we introduce two terms, the layer cushion and the activation contraction, which appear in our main theorem.
Definition 5 (Layer Cushion (Arora et al., 2018)). Let k (x) denote the output of the k-th layer of the model hx(a:) for some input E Xt. The layer cushion of the model he at layer k is defined as
Forgetting Region
Saturation Region
Pr. = minfp > 0: [ Iellolll
Ant
Af
S < o(h ((ht)"(=))) Vo X1}
Figure 3. Illustration of Proposition 1. The graph of U (shown as black solid line) can be divided into two regions: (1) the forgetting region (At < Atu),, where U. (At) consis- tently increases as function of and (2) the saturation region (At Als)), where U// (At) saturates to the asymptotic value Ug
The layer cushion of the model he is defined as
H4 max PL,k-
LE[L]
Definition 6 (Activation Contraction (Arora et al., 2018)), Let h& (x)) denote the output of the k-th layer of the model hx(x) for some input x E Xt. The activation contraction of model he is defined
Note that U/ (At) can be factored out into two terms: the first term p.ea
(It))l that does not depend on Al, and the || p.ce|
second term "/ (At) (Ar))(At) that depends on AL. To
simplify the analysis of how (At) varies with At, we in- troduce the following assumption to model the relationship between w - (At) and At.
C: = mmnff 0: 111 (x)]la S cloo(h) (2))]l2 vk [L]. Vo X.).
Assumption 2. We assume that
Using these definitions, we state our main theorem below:
d(R) Rt = =(((+))))))(A)))))
(6)
Theorem Suppose Assumption holds. Let ly, the model trained up to task t. Then, the representation discrep- ancy D (he, At) defined in Def. 4 is bounded
In other words, we assume that the left-hand-side of Eq. (6) is both upper and lower bounded by a linear function of At. The upper bound has already been established to be linear with At according to Theorem 4. of (Guha & Lakshman, 2024), while the linearity of the lower bound has not yet been established in the literature.
D (h,, At) S (icellR w (At) + w (At))
w (At))
+1
(5)
Based on the above results, the following proposition ex- plains how U. (At) evolves over At.
where w-1 (At)
A(R) ((he), ((
Here,
[/2 "(h.]ll
[R (ho)]. d(R); (he), RS (he )), fr and ot are given in Def-
Proposition 1. Suppose Assumption holds. The upper bound U of representation discrepancy monotoni- cally increases with respect At, reaching peak value of (1 + x2 (l (he)ll before gradually saturating to |eal (h+)]l-
initions 2, 3, 5 and respectively.
Proof. See Appendix A.I.
Definition 7. The upper bound in the right-hand-side of Eqn. 5 is denoted by U((A)) ie.,
Proof. Note that according to Assumption d(R (he), (h(+4))))) increases linearly with At. This implies that w -A()
(At) 2 + (At)) w (AI)" +1
(A()) pace (I
d(R) (hh,) (Re+aa))
[ (
also increases with At. Since for f(x) = E'ं
Me (f:) = = fow o (A))
(*()))
5.2. Evolution of Representation Diserepancy
#-1 the shape of U.k solely
The upper bound U (At) the representation discrepancy, given in Def. 7, can be interpreted as follows: For a fixed t (the task index for which we measure the forgetting) and a fixed layer index k; the upper bound U (At) becomes a function At, which represents the number of additional tasks learned after learning task t.
depends on the shape of f. which concludes the proof.
Fig. 3 shows the implications of Proposition 1: the upper bound U on the representation discrepancy exhibits two different phases. The first phase, which we call the forget- ting phase, is where U. (At) monotonically increases as a
6

---

Mcasuring Representational Shifts in Continual Learning: A Lincar Transformation Perspective
function of AL. The second phase, which we call the salu- ration phase, is where U slowly saturates an asymptotic value.
where 7, 8 E R are positive constants?.
The above assumption implies that for each layer ht the normalized difference between the weight matrix W++ after learning task ++ and the weight matrix W. before learning task t + 1 is upper bounded by quantity that decreases as the model width m grows. This assumption allows us to formulate how the weight matrices of each layer change after learning the subsequent tasks. Under this assumption, we derive the upper bound on the convergence rate r of the representation discrepancy.
Therefore, we focus on two quantities that capture the evo- lution of the upper bound U of the representation discrep- ancy. The first quantity is the asymptotic representation discrepancy
Ui,00 :- lim (A() = pec; (ec (he)ll, At-+00
defined as (At) in the asymptotic regime of large At.
The second quantity is the convergence rate of the repre- sentation discrepancy, which is defined as how fast U, (At)
Theorem 2. Suppose Assumption 3 holds. Let Ni.v
|l denote the ratio of the spectral norms of the k-- Wf
enters the saturation region, denoted by
th layer weight matrices for task t and t', and let x = maXEe()) Ni.-- Then, the convergence rate r of the representation discrepancy is bounded as
(8)
where
sat == arg max (AI)
(9)
r S <v= (v2-11)mm 1)7m B
(Pephec) de
(11)
A1>0
the task index when the model enters the saturation region, as shown in Fig. 3.
where 7: 3 are positive constants defined in Assumption 3, and μe, Ct are constants defined in Definitions 5 and 6.
Now we investigate how the asymptotic representation dis- crepancy _ and the convergence rate rf behave as the layer index and the width m of the neural network vary.
Proof. See Appendix A.2.
First, we state how Uho varies as a function of k. Corollary 1. Let the task index t and layer index k be fixed. Then, the k-dependency of the asymptotic representation discrepancy U is fully captured by [[ (hx) defined in Def. 2, the maximum norm of the activation at layer k. In addition, 000 is linearly proportional to | (l, i.e., Uk.c0 x (bx)l-
Since the constants At, (4r Ct are positive for any I, the upper bound on the right-hand side of Eqn. 11 inereases with the layer index k. This implies that the layer closer to the input (i.e., lower k) forgets at slower pace, compared with the layer closer to the output (i.e., higher k:). Additionally, the exponent of the width m on the right-hand side of Eqn. 11 is negative, indicating that the model with larger width forgets the representation more slowly.
Note that our empirical results (plotted in Fig. 5 in Sec. 6) show that | (hx)] is linearly proportional to k. Combin- ing this with the above corollary implies that the asymptotic representation forgetting U/t is a linear function of k, i.e., the model forgets more in the deeper layers.
6. Experiments
In this section, we present experimental results that support our theoretical findings. We measured both the representa- tion forgetting and the upper bound of the representation discrepancy under the same continual Icarning setup. Due to limited spacing, we report results on representation for- getting here and defer those on representation discrepancy to Appendix C. In Sec. 6.1, we empirically observe the representation forgetting of the first task during the contin- ual learning process and check whether it coincides with the behavior deduced from our analysis, as demonstrated in Proposition In Sec. 6.2, we provide empirical results that show the linear relationship between the overall amount of representation forgetting and the size of the representation space, which supports our theoretical findings in Corollary 1. In Sec. 6.3, we investigate whether the layer index k and
Second, we analyze how the convergence rate rf of the representation discrepancy behaves as the layer index & and the width m varies. We begin with an assumption that was empirically justified in Guha & Lakshman (2024).
Assumption 3 (Assumption 4.3 in Guha & Lakshman be an
(2024)). Let h(x)) = W o (00-1...6(11.. L-layer ReLU network with each hidden layer k having the same width m, and let E [N] be any task index. Consider the continual learning setup, where the weights are updated via gradient-base based learning methods. Then, the weight matrices ( and W of the network h beforelafter training on task t - 1 satisfies the following for all layers l:
W WillF
For more details the actual values of (3 and y, refer Guha & Lakshman (2024).
S Smm"
(10)
W 2|
7

---

Mcasuring Representational Shifts in Continual Learning: A Lincar Transformation Perspective
R²
0.90
300
R² = 1.00
(A(()))
200
CIFARIO0
ImageNalK
Torgetting Region
Forpetting Region
Suturatied Region
Saturzion Region
Alsat110 10
10
100
20
30
40
20
30
40
A1
AC
100
200
300
(a) Split-CIFARI00
(b) I
k
Figure 4. The evolution of representation forgetting AP, measured on Split-CIFAR100 and ImageNet1K K datasets. We mea- sure AP (At) for t = and & L. The solid line shows (At),, while the red dashed line shows the 4-th degree poly- nomial that best fits the solid line, to check the overall tendency of the representation forgetting. Similar to the plot in Fig. 3, we divide the graph into two regions based on the point where the first local maximum of the 4-th degree polynomial has occurred. For both datascts, we see that this is when a 10, as indicated by the black dashed line. The shape of the plot for AP? is similar to the shape of the upper bound U. of the representation discrepancy we analyzed in Proposition 1.
(a) AP (At) versus | (1x)l
(b) || (lx)ll versus l:
Figure 5. The relationship between the amount of representa- tion forgetting A A the size of the representation space [| (h+)ll, and the layer index k. We set and evaluated each quantity for k = 1,... .9. The left plot shows strong linear relationship between A P,, (Aat) (ASt) and (h+))l. which coincides with Corollary 1. The right plot also shows a strong linear relationship between ||R (A+)] and Similar re- sults for ImageNet K datasets are given in Fig, 12 in Appendix.
1, At N 1,
stay within bounded region This behavior is consistent with our theoretical results on the upper bound U.t of the representation discrepancy, as described in Proposition 1.
the model width m satisfies the monotonic relationships (in accordance to Thm. 2) to the convergence rate of the repre- sentation forgetting. Before we show our results, we give an outline of the setup for our continual learning experiments.
6.2. Bigger Spaces Experience Greater Forgetting
In Fig. 5a, we provide experimental results that relate the size [ (ha)ll of the representation space and the amount of representation forgetting A Pt (At) at the final phase of the continual learning, i.e., when At = N - 1. Specifically, for each layer = 1, 2, ... .9, we calculate the difference in the linear probing accuracy of task 1. before and after learning the remaining N 1 49 tasks. We then plot this against the norm of the largest feature in R2 (h+) and assess the linearity by fitting line using linear regression. As shown in Fig. 5a, the high H²² value confirms the strong linear relationship between A Pt (At) and [|R] [l This coincides with our findings in Corollary for U/.... an upper bound on a proxy of representation forgetting Apk.
Implementation details We test on Split-CIFAR100 dataset (Ramasesh al.. 2020) and downsampled ver- sion of the original ImageNer1 dataset (Chrabaszez al., 2017). For each dataset, the classes are partitioned into N = 50 categories, with each category containing 2 classes for Split-CIFAR 100 and 5 classes for ImageNetI K, follow- ing their original indices. Our model is based on the ResNet architecture (He et al., 2016), modified to maintain con- sistent latent feature dimensions throughout the network by adjusting parameters such stride and kernel size; see detailed structure in Fig. 7 in Appendix. Specifically, we define the network width m as (# channels) x 32 x 32, and the number of layers L as the number of blocks. By de- fault, we set (# channels) = 8 and use L = blocks. After pre-training ResNet through the continual learning process, we measure the performance of each hidden layer feature using linear probing; we extract the features k-th block of ResNet, and train linear classifier for task t 1 top of the extracted features.
In Fig. 5b, we plot [ (he)ll against the layer index k. Sur- prisingly, we observe strong linear relationship between them. Combining this result with Corollary implies that the amount of asymptotic representation forgetting Ut, linearly increases with the layer index k.
6.3. Effect of Network Size on the Convergence Rate
6.1. Evolution of Representation Forgetting
We empirically show how the convergence rate of the repre- sentation forgetting A P changes as the layer index & and the network width m varies, for Split-CIFARIE dataset. We first draw A Pk as well as the 4-th degree polynomial
In Fig. 4, we plot the evolution of the representation forget- ting A At) as Al grows, when /= == and l: = 9. tested on Split-CIFAR 100 and ImageNetIK datasets, respectively. We see that the forgetting curves for both experiments ex- hibit similar properties: initially, they both increase with At, but after some threshold value, i.e, At 10, the curves
'Empirica results indicate that this steady-state behavior oc- curs while the model retains non-trivial amount of information from task 1. See Fig. in Appendix.
8

---

Mcasuring Representational Shifts in Continual Learning: A Lincar Transformation Perspective
Impact Statement
12
14
Afsat
This paper presents theoretical understanding of the repre- sentation forgetting in continual learning scenarios. Our results reveal the two-phase structure of the representation forgetting as well as the behavior of the forgetting during such phases. This discovery provides deeper understand- ing of how models lose previously learned knowledge over time and sheds light on the mechanisms behind catastrophic forgetting. It may have implications on designing safer con- tinual learning algorithms, partieularly in scenarios where retaining sensitive information is critical. By addressing the challenges associated with knowledge retention, this research contributes to advancing the reliability and robust- ness of lifelong learning systems.
11
12
10
10 1
16
32
3
7
9
# chanmels
Figure 6. The impact of # channels (Left) and & (Right) on A ban- For each I and # channels, we report At when the best fitted 4-th degree polynomial achieves its first local maximum, similar to the point indicated by the black dashed line in Fig. 4.
that best fits A Pl similar to the plot shown in Fig. 4a. Then, we get sat defined in Eqn. 9, which is estimated by the local maximum of the 4-th degree polynomial that best fits AP,. Fig. 6 shows sas for various width m (or equiva- lently, # channels in ResNet) and layer index k. Our results indicate that at decreases with the layer index k, indicat- ing that the convergence rate r 1_ addition, increasing # channels results in longer A indi- cating that increasing the width decreases the convergence rate. These results support our theoretical findings from Thm. 2. Altogether, our results indicate that the forgetting occurs more rapidly to a higher degree as the layer index k increases, while increasing the width of the network slows down the forgetting process.
References
Andle, J. and Yasaei Sekeh, S. Theoretical understanding of the information flow on continual learning performance. In European Conference on Computer Vision, pp. 86-101. Springer, 2022.
Aia increases with k. In
Arora, S., Ge. R., Neyshabur, B., and Zhang, Y. Stronger generalization bounds for deep nets via compression ap- proach. In International conference on machine learning, pp. 254-263. PMLR, 2018.
Asanuma, H., Takagi. S., Nagano, Y., Yoshida, Y., Igarashi, Y., and Okada, M. Statistical mechanical analysis of catastrophic forgetting in continual learning with teacher and student networks. Journal of the Physical Society of Japan, 90(10): 2021.
7. Conclusion
In this paper, we present the first theoretical analysis of rep- resentation forgetting in continual learning. By introducing representation discrepancy, metric measuring dissimilarity between representation spaces under optimal linear trans- formation, we characterize the evolution of forgetting as the model learns new tasks. Our analysis shows that the forgetting curve exhibits two distinct phases (forgetting and saturation) and demonstrates that forgetting intensifies in deeper layers while larger network width delays forgetting. These theoretical findings are validated through experiments on image datasets.
Bennani, M. A., Doan, T., and Sugiyama, M. Generali- sation guarantees for continual learning with orthogonal gradient descent. arXiv preprint arXiv:2006. / 1942, 2020.
Caccia, L., Aljundi, R., Asadi, N., Tuytelaars, T., Pineau, J., and Belilovsky, E. New insights on reducing abrupt representation change in online continual learning. arXiv preprint arXiv:2104.05002 2021.
Cao, X., Liu, W., and Vempala, S. Provable lifelong learning of representations. In International Conference on Artifi- cial Intelligence and Statistics, pp. 6334-6356. PMLR, 2022.
Acknowledgments
This work was supported by the National Research Foun- dation of Korea (NRF) grant funded by the Korea govern- ment (MSIT) L03445,-0 R0-2024-004080033, the MSIT (Ministry of Science and ICT), Korea, under the ICAN (ICT Challenge and Advanced Network of HRD) support program (RS- RRS-2023-00259994, RS- RS-2025-02283044) supervised by the IITP (Institute for Information & Commu- nications Technology Planning & Evaluation), the Yonsei University Research Fund (2025-22-0025).
Chen, X., Papadimitriou. C., and Peng, B. Memory bounds for continual learning. In 2022 IEEE 63rd Annual Sym- posium Foundations of Computer Science (FOCS), pp. 519-530. IEEE, 2022.
Chen, Z. and Liu, B. Lifelong machine learning. Morgan & Claypool Publishers, 2018
9

---

Mcasuring Representational Shifts in Continual Learning: A Lincar Transformation Perspective
Chrabaszez, P., Loshchilov, I., and Hutter, F. A downsam- alternative the cifar
Artificial Intelligence and Statistics, pp. 10448-10470.
pled variant of imagenet datasets. arXiv preprint arXiv: (exiv:/777.0881111 2017.
PMLR, 2022.
Hess, T., Verwimp, E., van de Ven, G. M., and Tuytelaars, T. Knowledge accumulation in continually learned represen- tations and the issue of feature forgetting. arXiv preprint arXiv:2304.0090 2023.
Davari, M., Asadi, N., Mudur, S., Aljundi, R., and Belilovsky, E. Probing representation forgetting super- vised and unsupervised continual learning. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 16712-16721, 2022.
Karakida, R. and Akaho, S. Learning curves for continual learning in neural networks: Self-knowledge transfer and forgetting, arXiv preprint arXiv:21 2021.
Ding, M., Ji, K., Wang, D., and Xu, J. Understanding forgetting in continual learning with linear regression. arXiv preprint arXiv:2405.17588 2024.
Kim, G., Xiao, C., Konishi, T., Ke, Z., and Liu, B. A theoretical study on solving continual learning. Advances in neural information processing systems, 35:5065-5079. 2022.
Doan, T., Bennani, M. A., Mazoure, B., Rabusseau, G., and Alquier, A theoretical analysis of catastrophic forgetting through the ntk overlap matrix. In International Conference Artificial Intelligence and Statistics, pp. 1072-1080. PMLR, 2021
Korbblith, S., Norouzi, M., Lee, H., and Hinton, G. Sim- ilarity of neural network representations revisited. In International conference on machine learning, pp. 3519- 3529. PMLR, 2019.
Dosovitskiy, A., Beyer, I.., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. //999, 2020.
Kudithipudi, D., Aguilar- Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A. P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al. Biolog- ical underpinnings for lifelong learning machines. Nature Machine Intelligence, 4(3):196-2) 210, 2022.
Evron, I., Moroshko, E., Ward, R., Srebro, N., and Soudry, D. How catastrophic can catastrophic forgetting be in linear regression? In Conference Learning Theory, pp. 4028-4079 PMLR, 2022.
Lee, S., Goldt, S., and Saxe, A. Continual learning in the teacher-student setup: Impact of task similarity. In International Conference on Machine Learning, pp. 6109- 6119. PMLR, 2021.
Fini, E., Da Costa, V. G. T., Alameda-Pinedas X., Ricci, E., Alahari, K., and Mairal, J. Self-supervised models continual learners. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 9621-9630, 2022.
Li, H., Wu, J., and Braverman, V. Fixed design analysis of regularization- based continual learning. In Conference on Lifelong Learning Agents, pp. 513-533, PMLR, 2023.
Goldt, S., Advani, M., Saxe, A. M., Krzakala, F., and Zde- borová, L. Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup. Advances in neural information processing systems. 32, 2019.
Li, Y., Li. M., Asif, M. S., and Oymak, S. Provable and efficient continual representation learning. arXiv preprint arXiv:2203. 02026, 2022.
Lin, S., Ju, P., Liang, Y., and Shroff, N. Theory forget- ting and generalization of continual learning, In Inter- national Conference on Machine Learning, pp. 21078- 21100. PMLR, 2023.
Guha, E. and Lakshman, V. On the diminishing turns of width for continual learning. arXiv preprint arXiv:2403.0633 2024.
Luo, Y., Yang. Z., Bai, X., Meng, F., Zhou, J., and Zhang, Y. Investigating forgetting in pre-trained rep- resentations through continual learning. arXiv preprint M.Xiv:330500563 2023.
Hadsell, R., Rao, D., Rusu, A. A., and Pascanu, R. Embrac- ing change: Continual learning in deep neural networks.
Trends in cognitive sciences, 24(12): (128-10400 2020.
He, K., Zhang, X., Ren, S., and Sun, Deep residual learn- ing for image recognition. In Proceedings of the IEEE
Mansour, Y., Mohri, M., and Rostamizadeh, Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430, 2009.
conference computer vision and pattern recognition, pp. 770-778, 2016.
McClelland, J. L., McNaughton, B. L.,, and O' O'Reilly, R. C.
Hockel, R. Provable continual learning via sketched ja- cobian approximations. In International Conference on
Why there are complementary learning systems in the hip- pocampus and neocortex: insights from the successes and
10

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
Zhang, X., Dou, D., and Wu, J. Feature forgetting in continual representation learning. arXiv preprint arXiv: 2253::2205.3335,, 2022.
McCloskey, M. and Cohen, N. J. Catastrophic interfer- ence in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, vol- ume 24, pp. 109-165. Elsevier, 1989.
Zhang, Y., Charlin, L.,, Zemel, R., and Ren, M. Integrating present and past in unsupervised continual learning. arXiv preprint arXiv:2404. 191:22224. 2024.
Murata, K., Toyo T., and Ohara, K. What is happening inside continual learning model? representation--ent based evaluation of representational forgetting. In Proceedings of the IEEE/CVF Conference on Computer Vision and 2020.
Zhao, X., Wang, H., Huang, W., and Lin, W. A statistical
theory regularization- baaddd continual learning. arXiv preprint arXiv:2406. 06210, 2024.
Pattern Recognition Workshops, pp.
Peng, L., Giampouras, P., and Vidal, R. The ideal continual learner: An agent that never forgets. In International Con- ference on Machine Learning, pp. 2023.
PMLR,
Pentina, A. and Lampert, C. pac-bayesian bound for life- long learning. In International Conference Machine Learning, pp. 991-999, PMLR, 2014.
Ramasesh, V. V., Dyer, E., and Raghu, M. Anatomy of catastrophic forgetting: Hidden representations and task semantics. arXiv preprint arXiv:2007.07400, 2020.
Rao, D., Visin, F., Rusu, A., Pascanu, R., Teh, Y. W., and Hadsell, Continual unsupervised representation leam- ing. Advances in neural information processing systems, 32, 2019.
Saad, D. and Solla, S. A. Exact solution for on-line learning in multilayer neural networks. Physical Review Letters, 74337) 1995a.
Saad, D. and Solla, S. A. On-line learning in soft committee machines. Physical Review E, 52(4))4225, 1995b.
Straat, M., Abadi, F., Kan, Z., Göpfert, C., Hammer, B., and Biehl, M. Supervised learning in the presence of concept drift: modelling framework. Neural Computing and Applications, 1111:1001-118 2022.
Wen, Y., Tan, Z., Zheng, K., Xie, C., and Huang, W. Provable contrastive continual learning. arXiv preprint arXiv:2405. 18756, 2024.
Yin, D., Farajtabar, M., Li, A., Levine, N., and Mott, A. Optimization and generalization of regularization-base continual learning: a loss approximation viewpoint. arXiv preprint arXiv:2006. 10974, 2020.
Yoshida, Y. and Okada, M. Data-dependence of plateau phenomenon in learning with neural network-statistee mechanical analysis. Advances in Neural Information Processing Systems, 32, 2019.

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
A. Proofs
A.I. Proof of Theorem
The proof of Thm. can be split into 3 parts: (1) derive an upper bound of DJ (h2, At) as an optimization problem over the linear transformation T. (2) utilize Lemma derive suboptimal solution for the optimization problem (3) apply compression approaches introduced by Arora et al. (2018) remove the dependency the weight norm of the bound.
A.1.1. DERIVING AN UPPER BOUND IN TERMS OF T
Let h(()== = W 6( (WW')) )) be randomly initialized I--layer ReLU Network with each hidden layer & having the same width m. Suppose h(x) is trained sequentially on the datasets Di, , and ha represents the model trained up to Dr. Define |[R (he)] = max max (2))l2 x E X1). h as the norm of the biggest feature in R2 (ha), and (RR((h)) R[ == max{|lz 21112
Lemma 1.
(he), E (k4), as the distance between the two most distant features sampled at each space. Then, the k-th layer representation discrepancy of task in model by when trained on additional At tasks bounded
D (he, At) S min d(R) d(R))-(h)) R4 (hi+se)) [TW | ITW;
W
(12)
where T Xws any linear transformation from the k-th representation space itself
Proof.
DL (h,, At) min max mEX;
[o (2)) (x))1
min max =EX;
TW L+ (h W (x))) (x))]l2
min max
TW ++W. (x)) TW (x))
(x)) w( (x))]la
< min max wEX.
|TW ++
TW
TW (x))
(x)) - (( (=))ll2
S min max =EX,
|TW t+ |2|
- (1:
TW f+rr 11
(13)
S min max sEX,
TTW* k
1+ A+ 242 (z)) N (x))1 TW :tr
(x)]l2
(14)
S min d(R) - (h+), R ((- TW
HAr ll2 + || (b+)) |TW -
We used the sub- sub-multipee cubbmultiplicative property of the matrix 2-norm to derive Equation (12), and 1-Lipschitz property of é to derive Equation (14).
A. 1.2. FINDING A SUROPTIMAL SOLUTION
We use the following Lemma 2 to derive a suboptimal solution for Eqn. 12.
Lemma 2. Let A E RaxX be an arbitrary matrix, and C1, C2 E Rz be any posilive constants. Then,
w + w
e||8122 + 02) X A e2l XeRaxn
min
(15)
w +
where w G
Proof. Since the l2 norm for a matrix both non-differentiable and non-convex, we derive an optimal solution to its close convex-differentiable form,
min X m Km
X 122 + c2||X Tl s
X 02 | T22
where X arg min of X |
2 X
Al
XcRmxn
Now, since f(X) = of X
+ x
Al is convex and differentiable, is enough to find X such that VS(X*) =
12

---

max Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
We claim that X A satisfies the above condition.
V/((') 20 20
20f0z
2010
= cf + 0
7+
A
0
Hence, we can now derive the upper bound as follows,
min X X
c1 | + 02||
T22 <c| 112 - 02 =2 - -2l12
q0 c2d
||
2 2
X² x² X
02 12 + A|l2
where À G1
Substituting A W Lt X TW., e (h4), RK (h++ 02 R
[[[R)-((h)) in Lemma 2, we get the
following Corollary 2.
Corollary 2. Let h(x)) W'
"er.i......'''' ((W! be a randomly initialized L-layer ReLU Network with each hidden layer k having the same width m. Suppose Assumption holds. Then, the k-th layer representation discrepancy of task in model ht when trained additional At tasks is bounded as
(W) (At) + wi (A()
(h+, Al) S IR (
(16)
w (At) +1
where (of-((A)) d(R) ((.)..R ((h++ae)
[ "(he)ll
A. 1.3. REMOVING THE DEPENDENCY ON THE WEIGHT NORM
For the final step, use Proposition 2 remove the dependency on the weight norm in Eqn. 16.
Proposition Following the setting of Corollary we have,
Pacel| (he)
#EX, || (x=)l2
(17)
where μt, are constants depending the dataset D; and the model hy
Proof. The proof follows trivially from the definitions of the layer cushion Def. 5 and the activation contraction Def. 6. Namely,
1
(x)1l2
max W mex.
112 (x))l2
x
W ll2
(1 (x))l
max W l
=eX,
max u.c.l W (h; ØeX. Mice/ [ (ha)l
W
13

---

Mcasuring Representational Shifts in Continual Learning: A Lincar Transformation Perspective
Proposition 2 trivially follows from Def. 5 and Def. 6. Using Proposition 2 to replace [|, Tr.k-10 '(fg)] (Tt)" with peo, (R) Lemma 2, get Thm.
in
Theorem Suppose Assumption holds. Let by be the model trained up to D.. Define [R, == max { (z)]l2 :
Xeh, as the norm of the biggest feature in Rt (he), and (he)) ( (hy)) == max ø2lla 21 RS ) R4 (hsz)), as the distance between the most distant features sampled each space. Then, the k-th layer representation discrepancy of task in model hy when trained on additional At tasks is bounded
(W) (4) (AI)) + w (Al)).
(he, At) S (icel(
(18)
w (At) +1
where wi (At) d(R) d (1 and pes Ct are constants depending on the dataset Dr and the model he, as [R] (ha)ll
defined in Def. 5 and 6.
A.2. Proof of Theorem 2
The proof of Thm. 2 consists of two steps. First, in Thm. 4, derive upper bound of w (At) through perturbation analysis, using techniques similar to Guha & Lakshman (2024). Then, we derive the upper bound of r by using the fact that Atsa == arg max Uf (At) holds when w (Dtma) = =22
21>0
[Wllla denote the ratio of the spectral norms of the k-th layer weight matrices for task indexes L 1W
Theorem 4. Let Xt.,-
and t', and d = max Ne Following the setting of Thm. w (At) (C, 3,R. (h++ has an upper bound
[R2 (ha)]l
as
(Defice)f
(ftt) S 7m -Att
(19)
where 3 are positive constants defined in Assumption 3, and pt, Ox are constants defined in Definitions 5 and 6, which depend on the dataset Dr and the model ht.
Proof. From the definition of w (At), we can see that,
w (At) max 11 (x) h/ he (2)
max 11 ll [1/((()) Mit
max x
Now, we prove that,
| (z)
(x:)( 2 mm-iAA (2
Σ (Auna))
max
xEX.
|| (2))l2
14

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
This can be seen from the following.
[ (x: 164 (
114((0) h (2)
| (2)))22
W (z)]l2
| (x)) W ae sh ax))ll
Pcc
[W (2)
| (x)) - W (h
(x)) + ( (x)) - ((
pcc
[W || (x)]l2
| (x)) W O (x)) + || ((h (x)) w ((t
(hh
- p4ct
||
| W W
]|12
(1 (a=) -
S p4c
(20)
| (z)]l2
[ W aellll
(x)
[
Watlllllllt 1 (x) h
S Pcc
(21)
1 (x)]l2
W WHtlllz
aelaell [|A
(
S secs
+
|
||
(a))-(()))2
|1A--1(a0
1
(
S Spcc mm
At A
(22)
|/
We used the sub-multiplicative property of the matrix 2-norm to derive Equation (19), and 1-Lipschitz property of @ to derive Equation (20). Moreover, Equation (21) comes from Assumption I and the definition of A
Note that [1) (1)
|
(M (x)
(2))l2 (e)ill
Let A4 = max
Then, from the above inequality, we get
Au S micdde (AR-1 Xx 3m=88 (m)))
S megde (1 ( (Ap_2 +
2 mm At) 2
+ 2m mm-8 an-)
dx
de
* < (% (mm Σ (eppcs) Ne
5=1
where the last inequality holds since Ao 0
Substituting At = Atsar wf (Atrs) = 2 + 1, we get our final theorem.
Theorem 5. Suppose Assumption 3 holds. Let Ni.r =
w lla
I denote the ratio of the spectral norms of the k-th layer
weight matrices for lask indices L and I', and X = ( Ni.-- Then, the convergence rate r of the k-th layer representation discrepancy of task t is bounded as
k
(Aepece)²
r s <V2 1)
Σ
(23)
X
i=1
where 3 are positive constants defined in Assumption 3, and pt, Ct are constants defined in Definitions 5 and 6, which depend on the dataset D, and the model ht.
15

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
Proof. By Thm. 4, have
w (At) (2 m= -t) E (Aepaa)
Therefore
1 Atsal
k
2 The
> (upios))
w (Aisat)
k
( -m-
(Defress
V2+ + 1
-V2 - (
m
=8
(Delece)
1=1
B. Experiments details
Tet eins
3x3 cumi:
comi
Jr3 cumit
gumi
Jis come:
3: cant:
Block
Bluck
Figure 7. ResNet model architecture illustration.
We illustrate the ResNet model used in this paper. Figure 7 depicts the network, which consists of nine blocks. The first block contains a single convolutional layer, while the remaining blocks each have two convolutional layers with residual connections. All convolutional layers use the same channel size, m = 8, as the default setting. Specifically, the second through fifth blocks employ 7 x kernel, while blocks six through nine use a an average pooling layer, followed by fully connected layer and softmax output. Notably, we use (kernel size, stride, padding) triplet of (7, 1, for the larger kernels and (3, 1, 1) for the smaller ones, ensuring the feature map dimensions remain unchanged. Table lists the hyperparameters used for continual learning setups on ImageNetIK and Split-CIFAR100 training.
kernel. The network concludes with
Table 1. Continual learning syper-parameted for ImageNet32 training
PARAMETER
VALUE
LEARNING
0.001
BATCH SIZE
512
EPOCHS
500
WARM UP STEPS
200
WORKERS
4
OPTIMIZER
ADAMW
WEIGHT DECAY
5E-4
16

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
B.1. Relationship Between Representation Discrepancy and Representation Forgetting
20
R
0.74
- - 0.88
15
AP
(At)
APk
10
0.01
0.02 (h,, At)
0.03
50
100
150
200
Dh At)
(a) Split-CHIE CIFARIO0
(b) ImageNetIK
Figure 8. Correlation between representation discrepancy D (he, At) and representation forgetting APP (At) on (a) Split- Split-CIFARIIA FINEARIIO 100 and (b) ImageNetIK. The strong linear trends (with H² values of 0.88 and 0.74 respectively) demonstrate that the proposed representation discrepancy metric D (ha, At) serves effective surrogate for measuring representation forgetting AP (At) in continual learning.
Figure 8 presents empirical validation of the proposed representation discrepancy D (h,. At) as effective surrogate for representation forgetting. On both Split-CIFAR 100 and ImageNetI we observe a strong linear relationship between Df (h,, At) and AP (At), with coefficient of determination R² 0.88 and H²² = 0.74, respectively. This high degree of correlation empirically supports that our proposed D (h,, At) can be considered as practical surrogate for the representation forgetting AP (At).
B.2. Relationship Between Representation Discrepancy and Its Upper Bound
2000
1000
2000
0
0-
5
10
15
5
10
15
At
U
D
Uk
DF
U
D
(a) Width
(b) Width 128
(c) Width 256
Figure 9. Empirical comparison between the theoretical upper bound U.* and the measured representation discrepancy DŁ across ReLU networks of width 64, 128, and 256 on Split-CIFAR100
Figure 9 presents comparison between the theoretical upper bound Uf and the empirically measured representation discrepancy D. evaluated on ReLU networks trained on the Split-CIFAR100 100 dataset. We report the results for networks with varying widths m E {64, 128, 256} at layer & = 5. Although U is analytically derived, we observe that U/ consistently captures the trend of D. across various setup. This suggests that U provides a meaningful insight of the representation forgetting dynamics in practice.
B.3. Comparison of Model Trained on the First Task and Randomly Initialized Model
We assess how far the model is from a theoretical upper bound of forgetting. Figure 10, we visualize the layerwise representation discrepancy D between the model trained only on the first task (h₂) and two baselines: a randomly initialized model (hg i.e., At 11) and the final model after training on all tasks (hv, Le., , At N-1). We observe that across
17

---

Mcasuring Representational Shifts in Continual Learning: A Lincar Transformation Perspective
all layers, the discrepancy (h hy) is consistently and significantly smaller than D (h)) This result implies that the final model retains substantial information about the first task and that the observed saturation in forgetting does not reflect total erasure of prior knowledge. Therefore, the model has not yet reached state of maximum forgetting, suggesting there is still residual task-specific information preserved even after continual learning.
Model
250
At=
At N
200
Dfth Az)
150
100
0
5
6
7
8
9
Layer
Figure 10. Discrepancy hetween the task-1 model and the final model (At N remains far below that of randomly initialized
model (At = - 11), indicating that the model retains nontrivial task-1 information and has not undergone maximal forgetting.
B.4. Additional Supplementary Experiments For Assumption
We evaluate the validity of Assumption for ResNet by analyzing the flattened weights of the second convolutional layers in each block. Specifically, the weights, originally shaped as C, x C, W x H, are reshaped into C, x (C, W H), where C, and C, represent the output and input channel dimensions, and W and are the kernel's width and height. To test the assumption, optimized using the AdamW optimizer with a learning rate of 10-2 and no weight decay. As shown in Figure 2, for every block k. we successfully identify linear transformation T such that [ - TW [2 is minimized. Our results focus on the case where f. 1, = N, and l E (1....,9).
train linear model to transform the flattened weights W into W for each block k. The model is
Figure 11 illustrates the generality of Assumption in the context of transformer-based architectures. We conduct experiments on Vision Transformer (ViT) with 9 transformer layers and single attention head, trained on the Split- CIFARIO 100 dataset with N 50 tasks. For each layer &s e (l...., .,, 9). we extract the weight matrices from the MLP block of the transformer at two checkpoints: after task t = 1 and after task t' = 50. A linear transformation T dxd is then trained to align these weights by minimizing the alignment loss W TW l1ž, averaged over 10 random seeds. We observe that the loss rapidly decreases and approaches zero within 50 epochs across all layers, further supporting the validity of the assumption in VIT models.
B.5. Additional Experiments on ImageNet1K
In this section, we report the results when we use the ImageNet32 datasets (Chrabaszcz et al., 2017), downsampled from the original ImageNet, to design a series of tasks denoted as (D; J. Each task D; includes 5 classes selected based on their original class indices. Our setup consists the first 50 tasks in this ordered split, resulting in N = 50 tasks. We train a ResNet (He et al., 2016) with modifications to architectural parameters, including stride and kernel size, to ensure consistent latent feature dimensions. The network comprises blocks (L- 9) shown in Fig. 7 in Appendix.
We observe strong linear relationship between the representation forgetting AP, (At) and [[)] Here, set 1 = 1 and /' = N focusing investigating layers k E 1). (hx)) is computed the Frobenius norm of the activated output of the residual blocks, while ( represents the linear probing accuracy difference between A/ and h/. In Fig. 12a, APA (At)) increases proportionally with || (h+) The linear regression (red line) confirms this correspondence, achieving a coefficient of determination R² of 0.84. Additionally, Figure 12b reveals that (hx)l scales proportionally with the layer index k.
18

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
1.0 xto
0.8
TW
0.6
I|
0.4
4
0.2
0.0
20
40
60
80
100
Epoch
Figure 11. Difference between W and TW across epochs for each transformer layer & in Vision Transformer (ViT) trained on Spli--CIFAR100. The results show rapid convergence of the linear transformation T, validating Assumption in the transformer setting.
B.6. Analyzing the Effect of And m on Representation Forgetting
We evaluate the impact of k and m on the convergence rate in Sec. 6.3. Using the hyperparameters listed Tab. in Appendix, we train a ResNet model on the Split-CIFAR 100 dataset. For & (1,.., 9} and m E (8 x2x32xx3) x32, 16x32x332 x2 x22xx3330 x32), The results confirm that as lt increases, af
Fig. 13 Appendix shows the evolution of &P& (At) as At grows for decreases, indicating faster convergence rate. Similarly, for varying and k = L, Figure 14 in Appendix shows the evolution of A ( (Al) as Al grows for / 1. The results indicate that as m increases, Al increases, implying slower convergence rate.
C. Experimental results on the Representation Discrepancy C.1. Evolution of U.
We observe that the empirically measured U stays within the bounded region. We empirically measure U (Af) on the Split-CIFAR 100 and I datasets for = L and 1. As shown in Figure 15, the solid blue line represents U.t (At), while the red dashed line denotes the 4-th degree polynomial that best fits the empirical curve, used to visualize the overall trend of representation forgetting. We segment the graph into two regions based the point where the first local maximum of the fitted polynomial occurs. This point is denoted as Af.. = 10 for Split-CIAAR100 and At = 15 for ImageNet1K, as indicated by the vertical black dashed lines. Note that the values of U (At) are shifted such that U (0) = 0, i.e. all values are normalized by subtracting the value at At = 0 to enable consistent comparison across datasets. The shape of the plot for Uh similar the shape of the upper bound Pk of the representation discrepancy, as shown in Figure 4.
C.2. Empirical Validation of U. And R
We examine the behavior of the upper bound U.t. which captures tractable proxy for representation discrepancy. In Figure 16, we plot UK against (R/((h))) for each layer using the Split-CIFAR 100 and ImageNetIK datasets, respectively. For both datasets, observe strong linear correlation, reflected in high R² values (0.97 and 0.99), which empirically supports the proportionality U || (he)] established in Corollary 1.
C.3. Convergence Analysis With U.K
Recall that empirically measured the convergence rate using the representation forgetting metric A P.t. shown in Figure 6. Specifically, we plotted the temporal evolution of A for different layer indices k, fit 4-th degree polynomial to each curve, and estimated Ats as the time corresponding to the local maximum of the fitted polynomial.
Using the same methodology, we observed similar trend when measuring convergence with U.h. As shown Figure 17, Atsa consistently decreases as the layer index & increases, suggesting that higher layers converge more quickly. The alignment of results based on both A APk and U reinforces the conclusion that deeper layers tend to stabilize- thus
19

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
400
20
R²= 0.84
R² 0.97
15
300
(Pk(AN (At)
[IR
10
200
5
100
100
200
300
400
5
9
||R (h+)
k
(a) (Af) versus || (he)ll
(b) || (It)]l versus
Figure 12. |[R (hx)ll explains the representation forgetting AP" (At). Each point represents each layer from the ResNet. In Fig 12a. the red line shows the linear regression fit between (hx)] and AP' (At), demonstrating a strong correlation with R² = 0.84. In Fig. 12h, || [I is shown increase proportionally with the layer index These results indicate that representation forgetting. AP(((A) is closely linked to both |2 (he)]l and the layer index k.
forget more rapidly than shallower ones.
20

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
8
86
AA
6
APX
k-9
APX(A))
K-8
APK(DE)
k-7
Forgetting Region
4
Forgetling Region
4
Forgetting Region
Saturation Region
Saturation Region
Saturation Region
2
10
sat 10
Atsat
Sa
20
30
40
10
40
At
At
At
MATrA
APk (At)
6
APK(Ae))
12
k-5
APK(AE)
k-4
Forgetting Region
Forgetting Region
Forgetting Region
Saturation Region
Saturation Region
Saturation Region
2
At 11
sts
St 12
20
30
40
10
At
At
At
4
2
APk (At)
3-3
APP(A))
2
k-2
APk(D))
k
Forgetting Region
Forgetting Region
Forgetting Region
Saturation Region
Saturation Region
Saturation Region
At 12
At 11s
Atgat
20
30
40
10
30
20
30
40
At
At
At
Figure 13, The impact of & on the convergence of representation discrepancy.
20
15
8
15
(At))
mm-22x32x12 32 Forgetting Region
AP((
10
mm-16x32x33 Forgetting Region
0 2 (AR))
32
Forgetting Region
Saturation Region
Saturation Region
Saturation Region
Atc
12
11
Atsa=10
10
20
30
40
10
20
30
40
10
20
30
40
At
At
At
Figure 14, The impact of m on the convergence of representation discrepancy.
21

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
1.0
10
(k(A))
(fttt
0.5
0
U5
=0
Fongetring Region
Fongetting Region
Saturation Regios
Soturation Regian
0.0
10
20
30
40
10
20
30
40
At
(a) SpliI-CIFARI00
(b) ImageNelIK
Figure Evolution of empirically measured upper hound U, (At) on the representation discrepancy for t and T,, evaluated on (a) Split-IIFAR 100 and (b) ImageNerIK. The solid blue line shows U.2 (At), and the red dashed line indicates the 4-th degree polynomial fit used to reveal the overall trend forgetting. Each graph is divided into two phases, separated at Ats 10 for Split- CIFAR100 and At. 15 for ImageNetIK, corresponding the first local maximum of the polynomial fit. All curves normalized such that U (0) = 0 for consistent comparison.
300
2297
250
300
R 2099
3 200
200
150
100
200
400
600
400 ||R (RR((e))
600
R k (I
(a) SPAR
(b) ImageNetIK
Figure Relationship between the U. and |[ (hr)ll, across layers = 1,...,,, for Split-CIFAR CIFAR (left) and ImageNet1 (right). Each point corresponds to specific layer. The fitted red lines and high R² values (0.97 and 0.99) confirm strong lincar relationship. supporting the proportionality Lif x | (hx) stated in Corollary 1.
22

---

Mcasuring Representational Shifts in Continual Learning: A Linear Transformation Perspective
15
14
At
11
2
3
4
5
6
7 8
9
Figure Saturation time Ars measured using U. for different layer indices Following same methodology as Figure 6, we 4-th degree polynomial to the temporal trajectory of U and estimate Atsa as the time corresponding to the local maximum of the fitted curve. The result shows that At. decreases with increasing k. indicating faster convergence deoper layers.
23


Reviews:
[{"review_id": "OzCzfPI8Wp", "reviewer": "Reviewer_1dag", "summary": "This paper introduces a novel metric for measuring representation forgetting in continual learning and derives an upper bound for this metric. The theoretical findings provide valuable insights, which are further validated through experiments on real image datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer 1dag for the detailed review and constructive suggestions. We appreciate your acknowledgements that our **proposed metric is a reasonable choice** for facilitating theoretical analysis and that this paper is **well-written**. Below we show our reply on your comments and questions.\n\n---\n**[R1-1]** `In Definition 3, the self-distance is nonzero. Does this indicate that Definition 3 is not suitable for measuring distances between representation spaces?`\n\nWe truly appreciate this comment and would like to thank all the reviewers for pointing this out. **We found that there was a typo in Def.3** (as well as Eq.1 for recalling the definition in Guha et al. 2024) and revised Def.3 as follows. We assure you that our theoretical results and the corresponding proofs remain the same.\n\n| Submitted version: | Revised version: |\n|--|--|\n| https://hackmd.io/_uploads/ByVVMGZ6kl.png | https://hackmd.io/_uploads/Sy-BI8-Tyl.png |\n\n\nPreviously, our definition of the distance between two representation spaces was based on computing the distance between the two most distant features in the respective representation spaces. Our newly revised definition only computes the maximum distance between the features of $\\mathcal{R}^k_t(h_{t_1})$ and $\\mathcal{R}^k_t(h_{t_2})$ with respect to the **same input**. \n\nThus, in our revised version, the self-distance is zero. \n\nWe will revise the manuscript accordingly. \n\n---\n**[R1-2]** `Without a discussion on the tightness of the upper bound, certain interpretations—such as the peak value in Proposition 1—become questionable. `\n\n\nWe thank the reviewer for the insightful comments. In response, we conducted experiments using a ReLU network on Split-CiFAR100 dataset to assess the tightness of the upper bound $U^k_t$ with respect to the representation discrepancy $D^k_t$, when $t=1$ and $k=5$; see the below plot.\n\nhttps://hackmd.io/_uploads/SyV_GCD6yg.png\n\nAlthough there exists a gap between $U^k_t$ and $D^k_t$, we observe that U^k_t consistently tracks the trend of D^k_t, suggesting that its analysis still yields meaningful insights on D^k_t.\n\nWe will include this discussion in the revised manuscript.\n\n---\n**[R1-3]** `Other aspects of the paper may also raise concerns about precision and tightness, such as Definitions 3 and 4. The proposed distance/discrepancy measures are only approximations based on imprecise beliefs (as indicated by the use of “ \\approx ” in Section 4.2).`\n\nWe agree with the reviewer that our distance/discrepancy measures lack formal guarantees for exactly quantifying the representation forgetting. However, similar approximations have been employed in prior work, e.g., Guha et al. (2024), which used similar measures to analyze the catastrophic forgetting, and their empirical results were well-aligned with their theoretical insights. In our case as well, the empirical trends closely align with the theoretical predictions, supporting the practical relevance of our formulations.\n\n---\n**[R1-4]** `Figures 4 and 8 display multiple peaks, which contradicts the unique peak predicted by Proposition 1. Could this be evidence that the theoretical upper bound is not sufficiently tight?`\n\nThank you for pointing this out. \n\nThe reviewer is true that the number of peaks is different from theory (in Proposition 1) and practice (in Figures 4 and 8), but we claim that the number of peaks is not an important factor in terms of figuring out the overall trend. More importantly, both curves (theory and practice) have similar trend of exibithing two phases -- forgetting phase and saturation phase -- where the amount of forgetting significantly increases at the first phase, and the amount of forgetting deviates in a smaller range in the second phase. We will mention this in the revised manuscript. \n\n\nRegarding the tightness of our upper bound, please refer to our response in [R1-2].\n\n\n---\n**[R1-5]** `How useful are the insights in this paper for improving the performance of continual learning?`\n\nOur analysis reveals two key findings with direct implications for improving performance in continual learning: (1) representation forgetting tends to occur more rapidly and severely in deeper layers, and (2) increasing network width mitigates the degree and speed of forgetting. These insights suggest practical strategies such as allocating more regularization or memory resources to deeper layers, or designing architectures with wider representations in critical layers to slow down forgetting.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No.", "experimental_designs_or_analyses": "No.", "relation_to_broader_scientific_literature": "This paper presents theories in continual learning, which helps the general understanding of lifelong learning systems.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Overall, this is a well-written paper with a clearly stated system model and key assumptions. The proposed metric is a reasonable choice, particularly for facilitating theoretical analysis. However, the interpretation of Theorem 1 is insufficient, especially regarding the tightness of its upper bound, which is not discussed.", "other_comments_or_suggestions": "NA.", "questions_for_authors": "1. How useful are these insights for improving the performance of continual learning?\n\n2. In Definition 3, the self-distance is nonzero. Does this indicate that Definition 3 is not suitable for measuring distances between representation spaces?\n\n3. Without a discussion on the tightness of the upper bound, certain interpretations—such as the peak value in Proposition 1—become questionable. Additionally, other aspects of the paper may also raise concerns about precision and tightness, such as Definitions 3 and 4. Specifically, the proposed distance/discrepancy measures are only approximations based on imprecise beliefs (as indicated by the use of “$\\approx$” in Section 4.2).\n\n4. Figures 4 and 8 display multiple peaks, which contradicts the unique peak predicted by Proposition 1. Could this be evidence that the theoretical upper bound is not sufficiently tight?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "utEubGiVlx", "reviewer": "Reviewer_9B8C", "summary": "This paper introduces a novel metric—termed representation discrepancy—to quantify the degradation of internal feature representations (i.e., representation forgetting) in continual learning. By framing the forgetting problem in terms of a minimum alignment error between hidden layer representation spaces via an optimal linear transformation, the authors provide both a theoretical analysis and empirical validation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer 9B8C for the detailed review and constructive suggestions. We appreciate your acknowledgements that our theoretical and experimental results are generally **well-presented and convincing**, and that our **proofs are methodologically sound**. Below we show our reply on your comments and questions.\n\n---\n\n**[R2-1]** `Generalizability and limitations of Assumption 1`\n\nThank you for raising this important point.\n\nWe agree that **Assumption 1** (existence of a near-perfect linear transformation $T$ aligning $W_t^k$ and $W_{t'}^k$) may not universally hold. In the revised manuscript, we clarify its scope and limitations.\n\nAs shown in **Figure 2**, Assumption 1 holds empirically for **MLPs** and **ResNets**, the latter of which includes skip connections (asked by the reviewer). To test generality, we conducted experiments on a **Vision Transformer (ViT)** (9 layers, 1 head) trained on **Split-CIFAR100**, optimizing $T$ to align $W_1^9$ and $W_{50}^9$, which showed clear convergence:\n\nhttps://hackmd.io/_uploads/SkzzjJ4p1x.png\n\nWe also compare alignment errors:\n\n| Architecture           | Final Alignment Error         |\n|------------------------|-------------------------------|\n| Multi-Layer Perceptron | $7.58 \\times 10^{-7}$         |\n| ResNet                 | $5.18 \\times 10^{-7}$         |\n| Vision Transformer     | $6.52 \\times 10^{-8}$         |\n\nWhile an approximate version of the assumption (e.g., $\\| W_t^k - T W_{t'}^k \\| \\leq \\varepsilon$) is possible, understanding its effect on theory is left for future work. This limitation will be explicitly discussed.\n\n---\n\n**[R2-2]** `Ablations across architectures or continual learning scenarios`\n\nThanks for the suggestion.\n\nWe added experiments under a **domain-incremental setup**, training ResNet on **rotated Split-CIFAR100** (same classes, different input rotations). We checked $R_t^k$ vs. $U_t^k$ for $t=1$ and $k=1,\\dots,9$, observing consistent linear trends as predicted by Corollary 1:\n\nhttps://hackmd.io/_uploads/r14Vgb_Tyg.png\n\nIn addition, we ran additional experiments for **Vision Transformer (ViT)** architecture to check the validity of Assumption 1, as shown in our response in [R2-1]. \n\n---\n\n**[R2-3]** `Clarifying proofs and discussing limitations`\n\nThank you. We revised the proofs to add explanations in non-trivial derivations:\n\n| Submitted Version                                      | Revised Version                                        |\n|--------------------------------------------------------|--------------------------------------------------------|\n| https://hackmd.io/_uploads/rkg3RbB61l.png | https://hackmd.io/_uploads/r15TpWr6ke.png  |\n| https://hackmd.io/_uploads/HJmS0WSpyl.png | https://hackmd.io/_uploads/ryHb0bH61g.png |\n\nFor discussion on potential limitations of our assumption, please refer to our response in [R2-1].\n\n---\n\n**[R2-4]** `Referencing recent unsupervised and contrastive learning works`\n\nWhile our focus is on supervised continual learning, we agree that linking to recent related work adds context. We will include the following:\n\n- Malviya et al., *J. Comput. Sci.*, 2025\n- Wen et al., *arXiv:2405.18756*, 2024\n- Zhang et al., *arXiv:2404.19132*, 2024\n\n---\n\n**[R2-5]** `Practical implications of the theoretical findings`\n\nThank you for the insightful suggestion.\n\nWhile our metric is theoretical in nature, the **insights derived from it** suggest actionable strategies for improving continual learning (see [R1-5]). We will emphasize this connection more clearly in the revision.\n\n---\n\n**[R2-6]** `Hyperparameter sensitivity and dataset diversity`\n\nThank you for the constructive suggestion.\n\nWe conducted experiments varying **batch size** ($B = 128, 256, 512$) and **learning rate** ($\\gamma = 10^{-3}, 10^{-4}, 10^{-5}$) on Split-CIFAR100 using ResNet. The linear relationship between $R_t^k$ and $U_t^k$ remained consistent, demonstrating robustness:\n\n| **Batch size** $B$                                       | **Learning rate** $\\gamma$                                  |\n|----------------------------------------------------------|-------------------------------------------------------------|\n| https://hackmd.io/_uploads/H1jp6bOpJx.png | https://hackmd.io/_uploads/HkZ06bOa1e.png |\n\nAs for dataset diversity, see [R2-2] for domain-incremental results on **rotated Split-CIFAR100**.", "claims_and_evidence": "The primary claim is that the proposed representation discrepancy is a reliable and analytically tractable surrogate for measuring representation forgetting. The authors support this claim through rigorous theoretical derivations (e.g., Theorem 1 and Theorem 2) and by demonstrating a strong empirical correlation between the discrepancy and conventional measures (such as degradation in linear probing accuracy). While the derivations are nontrivial and depend on certain assumptions (e.g., the existence of a linear transformation aligning weight matrices across tasks), the evidence—both theoretical and experimental—is generally convincing. Some aspects, particularly the sensitivity of these assumptions to different network architectures, might benefit from further discussion.", "methods_and_evaluation_criteria": "The method of aligning representation spaces using a linear transformation is well-motivated and novel in the context of continual learning. The evaluation criteria, which include comparing linear probing performance before and after task transitions, appear appropriate for assessing representation quality. The choice of benchmark datasets and the experimental design are standard for the field, lending credibility to the claims. However, additional details on hyperparameter sensitivity and broader dataset diversity could further strengthen the evaluation.", "theoretical_claims": "The paper presents several theoretical claims, including explicit upper bounds on the representation discrepancy and an analysis of the convergence rate of forgetting across layers. The proofs, particularly for Theorem 1 and Theorem 2, seem methodologically sound, though they rely on assumptions (such as Assumption 1 regarding the existence of a suitable linear transformation) that, while empirically supported, might not hold in all settings. Clarifying the limitations of these assumptions and their impact on generality would improve the theoretical discussion.", "experimental_designs_or_analyses": "The experiments are designed to validate the theoretical predictions: the evolution of representation forgetting is tracked as new tasks are learned, and the relationships between layer depth, network width, and forgetting are examined. The use of visualizations (e.g., plotting the forgetting curves and the relationship between representation space size and forgetting) helps illustrate the key findings. While the experimental setup is solid, additional ablation studies—such as varying network architectures or exploring different continual learning scenarios—could further substantiate the results.", "relation_to_broader_scientific_literature": "This work builds on a rich body of literature in continual learning and catastrophic forgetting. It is well situated relative to prior work—especially that of Guha & Lakshman (2024) and other studies addressing representation changes over time. The connection drawn between representation forgetting and linear probing performance is particularly insightful. However, integrating discussion of a few more recent works on unsupervised continual learning and contrastive methods might further contextualize the contributions.", "essential_references_not_discussed": "To my knowledge, no.", "other_strengths_and_weaknesses": "Strengths:\n1.The paper proposes a novel and theoretically grounded metric for a long-standing problem in continual learning.\n2. The theoretical analysis is detailed and offers clear predictions about how factors like layer depth and network width influence forgetting.\n3. Experimental validation on standard datasets is convincing and well-presented.\n\nWeaknesses:\n1. Some of the core assumptions (e.g., the existence of a near-perfect linear alignment between weight matrices) may not generalize across all architectures or learning scenarios.\n2. Certain proofs and derivations could benefit from additional clarity, and more discussion on potential limitations would be helpful.", "other_comments_or_suggestions": "The authors can consider providing more intuition on the practical implications of the theoretical findings—especially how this metric could potentially inform better continual learning strategies.", "questions_for_authors": "Could you elaborate on the limitations of Assumption 1 regarding the existence of a linear transformation aligning weight matrices? How might this assumption be relaxed or validated for other network architectures (e.g., non-ReLU networks or models with skip connections)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8yAbauda8c", "reviewer": "Reviewer_4iQx", "summary": "In this works, the authors introduce a novel measure of forgetting in the hidden layers of deep neural networks in continual learning setting. They derive an upper-bound for the proposed representation discrepancy measure and the convergence rate of this measure under a set of assumptions. Additionally, the authors evaluate the development of representation forgetting during learning empirically using Split-CIFAR100 and ImageNet 1K.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer 4iQx for the detailed review and constructive suggestions. We appreciate your acknowledgements that the problem addressed in this manuscript is potentially **important and interesting**, and that our results are **clearly distinct from the previous works**. Below we show our reply on your comments and questions.\n\n---\n**[R3-1]**`The introduction of a new measure for representation forgetting is motivated by the difficulty of deriving an optimal linear classifier for hidden layers. I’m confused by this argument.`\n\nWe believe that there may have been a misunderstanding regarding our motivation for introducing the measure $D^k_t$.\n\nThe term *difficulty* in our statement refers to theoretical intractability rather than computational cost. Our intention was **not** to use $D^k_t$ as a *more efficient alternative* for computing representation forgetting in practical settings. Rather, we introduced $D^k_t$ to *facilitate the theoretical analysis* of $\\Delta P^k_t$.\n\nWe will clarify this point in the revised manuscript. \n\n---\n**[R3-2]**`It remains unclear how D^k_t and P^k_t scale or whether one provides an upper bound for the other. `\n\nBelow we added plots showing $D_t^k$ versus $\\Delta P_t^k$ across different layers $k$. In both datasets, $\\Delta P_t^k$ **scales approximately linearly** with $D_t^k$, supporting its role as a surrogate. We will include these results in the revised manuscript.\n\n| **ImageNet1K**                                           | **SplitCIFAR100**                                         |\n|-|-|\n|https://hackmd.io/_uploads/BJVuwbdp1g.png|https://hackmd.io/_uploads/Sk-YD-upkl.png|\n\n---\n**[R3-3]** `The empirical results evaluate the properties of only P^k_t not D^k_t. Thus, the analytical results do not support their claims on forgetting convincingly.`\n\nWe conducted additional experiments directly evaluating $D_t^k$. As shown below, we observed a **strong linear relationship** between $D_t^k$ and $R_t^k$, similar to our observation in Fig.5 of the submitted manuscript. We also confirm the linearity between $U_t^k$ and $R_t^k$ as predicted by Corollary 1. \n\nThese results will be included in the revised manuscript.\n\n| **ImageNet1K**                                           | **SplitCIFAR100**                                         |\n|-|-|\n|https://hackmd.io/_uploads/BJ-jMLEpJg.png|https://hackmd.io/_uploads/S1RsG8Vpye.png|\n|https://hackmd.io/_uploads/SkTYGzd6Je.png|https://hackmd.io/_uploads/B1n9MGOa1l.png|\n\n---\n**[R3-4]**`The motivation behind the definition of the representation discrepancy is unclear. It is susceptible to outlier, raising concerns about its robustness.`\n\nThank you for your comment. As noted in [R3-1], our primary motivation for introducing the metric $D^k_t$ is to facilitate the theoretical analysis of $\\Delta P^k_t$. We adopted the maximum discrepancy formulation following the approach of Guha et al. (2024), who succesfully employed a similar method in their theoretical study of catastrophic forgetting in continual learning.\n\n---\n**[R3-5]**`The proof of Prop 1., claims d increases linearly with \\Delta t, citing Thm 4.1 of Guha & Lakshman (2024), but the theorem only shows this dependence for the upper bound of d. Hence, the proof and the following arguments on the two-stage dynamics is not well supported.`\n\nThank you for the sharp comment. The reviewer is correct that the linear dependency of $d$ with respect to $\\Delta t$ is not *theoretically* proven in Guha & Lakshman (2024). As such, our proof in Proposition 1 requires an additional assumption (that the lower bound of $d$ increases without a limit as a function of $\\Delta t$) for the result to hold.\n\nUnder this assumption, the unique peak described in Proposition 1 may not always be guaranteed. However, this assumption is sufficient for the two-phase dynamics to hold, as the upper bound $U^k_t$ will eventually reach a peak and then saturate.\n\nWe also note that if $d$ itself monotonically increases with $\\Delta t$, then the uniqueness of the peak as stated in Proposition 1 follows. We will clarify these assumptions and their implications in the revised version.\n \n---\n**[R3-6]**`The convergence rate in Fig. 6 is defined using P^k_t, which is different from the original definition in Eq. 7.`\n\nAs suggested, we report the convergence rate using $U_t^k$ instead of $\\Delta P_t^k$. Similar to Figure 6, we find that $\\Delta t_{\\text{sat}}$ decreases with layer index $k$, so the convergence rate $r_t^k = 1 / \\Delta t_{\\text{sat}}$ increases with $k$, consistent with Theorem 2.\n\nhttps://hackmd.io/_uploads/r1VnzXOp1l.png\n\n---\n**[R3-7]**`Naively thinking, Assumption 1 shouldn't hold when w_k < w_{k-1}. Why is that the case?`\n\nAssumption 1 compares across different tasks $t$ and $t'$ for the same layer $k$, not across different layers. Thus, the width $w_k$ remains fixed for each comparison, and the condition holds regardless of whether $w_k < w_{k-1}$. This is why Assumption 1 is satisfied in Figure 2.", "claims_and_evidence": "1. The manuscript doesn’t clearly justify how the proposed measure of representation discrepancy $D^k\\_t$ relates to forgetting, despite claiming it as an effective surrogate for representation forgetting. While the authors demonstrate that when $D^k\\_t$ is small, $P^k\\_t$ is also small, it remains unclear how these values scale or whether one provides an upper bound for the other. This is particularly concerning, because the empirical results evaluate the properties of only $P^k\\_t$, not $D^k\\_t$. Furthermore, the convergence rate in Fig. 6 is defined using $P^k\\_t$, which is different from the original definition in Eq. 7. Thus, I find that the analytical results do not support their claims on forgetting convincingly, and there remains discrepancy between analytical and empirical findings.\n\n2. In the proof of proposition 1, the authors state that $d$ increases linearly with $\\Delta t$, citing Theorem 4.1 of Guha & Lakshman (2024). However, in the original theorem, this linear $\\Delta t$ dependence is shown for the upper bound of $d$, not $d$ itself, making this proof and the following arguments on the two-stage dynamics not well supported.\n\n3. The motivation behind the definition of the representation discrepancy is unclear. Because the definition relies on the maximum discrepancy over all samples rather than the distance between two activity distributions, it is susceptible to outlier, raising concerns about its robustness.\n\n4. In section 4.1, the introduction of a new measure for representation forgetting is motivated by the difficulty of deriving an optimal linear classifier for hidden layers. I’m confused by this argument particularly because the proposed measure of representation discrepancy requires derivation of the optimal linear transformation between two hidden layer representations, which is typically computationally heavier than the derivation of the optimal linear classifier (if $d_y < w_k$).", "methods_and_evaluation_criteria": "See the comment 1 above.", "theoretical_claims": "See the comment 2 above.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "While the manuscript is built heavily upon Guha & Lakshman (2024), the results presented here are clearly distinct from that previous work.", "essential_references_not_discussed": "Not to my knowledge.", "other_strengths_and_weaknesses": "The question addressed in this manuscript, forgetting in hidden layers of ANNs, is potentially interesting and important.", "other_comments_or_suggestions": "Naively thinking, Assumption 1 shouldn't hold when $w_k < w\\_{k-1}$, but Figure 2 suggests it does. Why is that the case?", "questions_for_authors": "Please see the comments above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o22qxhXFW0", "reviewer": "Reviewer_cno5", "summary": "This paper focuses on the theoretical analysis of representation forgetting in Continual Learning scenarios. While representation forgetting has been introduced previously, the focus was largely experimental and the main contribution of the proposed work is to focus on the theoretical aspect of representation forgetting. Therefore, the authors justify the usage of a new metric, Representation Decrepancy, as a surrogate for traditional representation forgetting measures. Consequently, this paper analyzes the theoretical behavior of representation forgetting, showing its dependence on layer index, and network depth, and identifying various regimes of representation forgetting during training. Such findings are supported by experiments.\n\n## update after rebuttal\n\nI thank the authors for their detailed responses and clarifications.\n\nThe figure given in response R4-3 is interesting and I would suggest including it in the final version of the manuscript (or in appendix).\n\nRegarding R4-2, I should have been more precise in my comment, but I believe the main experimental result of the paper lies in Figure 4, which is what I would like to see regarding the evolution of $D^k_t$. Although as stated, I understand that $D^k_t$ and $\\Delta P^k_t$ are correlated so similar results should be observed. Nonetheless, the authors should display the results with $D^k_t$ since the theoretical analysis is made with this quantity.\n\nFor these reasons, I will maintain my current score of 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer cno5 for the detailed review and constructive suggestions. We appreciate your acknowledgements that the theoretical aspects of this paper are **clear and adequately justified**, and that the findings are **interesting and novel**. Below we show our reply on your comments and questions.\n\n---\n\n**[R4-1]**`In Section 6.1, why compute the representation forgetting \\Delta P^k_t instead of directly computing your newly introduced metric D^k_t? I understand that the two are correlated as stated in 4.2 but all the theoretical analysis is made on D^k_t, which should be computable`\n\nAs noted in [R3-3] and [R3-6], we evaluated $D_t^k$ directly and observed results consistent with Figures 5 and 6. These will be included in the revised manuscript.\n\n---\n\n**[R4-2]**`I am unsure how much should the analysis of \\Delta P^k_t gives insight on the behavior of the maximum value of D^k_t. Could you provide more explanation on this choice and its consequences?`\n\nThank you for your insightful comments. Our original intent in conducting experiments with $\\Delta P^k_t$ was to demonstrate that the theoretical results derived from $D^k_t$ also holds in $\\Delta P^k_t$. However, we agree that experiments based solely on $\\Delta P^k_t$ are insufficient to support the theoretical claims concerning $D^k_t$. Accordingly, we have conducted additional experiments using $D^k_t$, as detailed in [R3-3] and [R3-6].\n\n---\n\n**[R4-3]**`How far is the theoretical upper bound U^k_t,\\infty from an experimental upper bound? For example, what is the discrepancy between the model trained on the first task and a randomly initialized model? Such information could be interesting to understand how much the model can forget, as for now it is not clear if this saturation is the consequence of \"not being able the forget more\" or if it occurs before reaching such a stage, therefore maintaining some of the previous knowledge (and if so, how far are we from \"maximum forgetting\"?).`\n\nWe thank the reviewer for the insightful question. To clarify, at each layer $k$, the below plot reports the representation discrepancy $D_t^k$ between the task-1 model $h_1$ and two models:\n* a randomly initialized model $h_0$, i.e., $\\Delta t = -1$\n* the final model after training on all tasks $h_N$, i.e., $\\Delta t = N-1$\n\nOur results show that $D^k_1(h_1, N-1)$ is significantly smaller than $D^k_1(h_1, -1)$, indicating that the saturation does not correspond to complete forgetting. Thus, the model retains a nontrivial amount of task-1 information even after learning all subsequent tasks.\n\nhttps://hackmd.io/_uploads/ByCfAQOTke.png\n\n---\n\n**[R4-4]**`The increase of forgetting in latest layers is intuitive and expected. This has similarly been discussed in previous studies such as [1]. While the theoretical findings are interesting, further discussion with previous studies would be included.`\n\nThank you for the suggestion.\n\nWe will include a discussion comparing our results with [1], which empirically observed greater drift in deeper layers. Our work complements this by providing the first theoretical explanation of this phenomenon (see Corollary 1 and Fig. 5).\n\n---\n\n**[R4-5]**`I would like the code to be shared`\n\nYou can find code here: [link](https://anonymous.4open.science/r/representational-shifts-in-cl-F1F5/README.md)\n\n---\n\n**[R4-6]**`From a theoretical perspective, it seems to me that such analysis can be adequately applied beyond Class-Incremental Learning scenarios (which \nis the only one considered here). I would appreciate discussion in this regard. While I believe the current experiments to be sufficient, experiments in Domain Incremental Learning cases would further strengthen the theoretical findings.`\n\nWe agree that our framework can extend to domain-incremental learning (DIL), provided an appropriate modeling of weight perturbation as done in Assumption 2 for class-incremental learning. While a formal theory for DIL is left for future work, we provide preliminary empirical evidence in [R2-2] using rotated Split-CIFAR100, where Corollary 1 continues to hold.\n\n---\n\n**[R4-7]**`What about when the representations are normalized? Such strategy is very common in representation learning, how would the current analysis be impacted?`\n\nWhile we do not yet have a formal analysis under normalization, we offer the following intuition: normalization compresses the representation space by reducing feature norms, which in turn may reduce the magnitude of $D_t^k$. If the shift in linear structure is preserved under normalization, our discrepancy measure would still reflect representational drift, but with smaller scale. We will mention this in the revised manuscript.\n\n---\n\n**[R4-8]**`Why not use centroid distances for definition 3, instead of the most distant features?`\n\nWe appreciate the suggestion. We follow the max-discrepancy formulation as in Guha et al. (2024), who adopted a similar worst-case approach in their theoretical analysis of catastrophic forgetting.", "claims_and_evidence": "The theoretical aspect of the paper is clear and adequately justified. I did not find any major flow in the presented theorems or assumptions.\n\nMy main concern resides in the experimental section. While the author introduces a new metric, representation discrepancy, they somehow never compute it in the experiments to the benefit of representation forgetting as defined in previous work. While the authors indeed discuss how both metrics are related, I struggle to understand the justification of such a choice and I am unsure that this adequately justifies the theoretical claims.", "methods_and_evaluation_criteria": "Yes, the dataset perfectly makes sense, even though additional settings would be considered. However, the paper being mostly theoretical, I believe the current choice to be sufficient.", "theoretical_claims": "I checked the correctness of the main paper, however only briefly looked at the demonstrations in the appendix. So far, no major issues has been identified.", "experimental_designs_or_analyses": "The experimental design is clearly explained and overall makes perfect sense for demonstrating the theoretical claim. As already stated, my main concern remain the metric used in epxeriments.", "relation_to_broader_scientific_literature": "This is related to Continual Learning in general as it helps understand how neural network representation can change throughout training. Notably, while some findings have been observed experimentally in previous studies (forgetting strong in the latest layers), it gives additional insight into being able to connect it with the theory proposed in the paper. Maybe such theoretical analysis can help design new methods for reducing forgetting in intermediate layers or give additional metrics for quantifying training behaviors.", "essential_references_not_discussed": "The increase of forgetting in latest layers is intuitive and expected. This has similarly been discussed in previous studies such as [1]. While the theoretical findings are interesting, further discussion with previous studies would be included.\n\n[1] Ramasesh, Vinay Venkatesh, Ethan Dyer, and Maithra Raghu. \"Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics.\" _International Conference on Learning Representations_.", "other_strengths_and_weaknesses": "*Strenghts*\n\n- The writing quality is very good, I enjoyed reading the paper and found the presentation particularly clear\n- The findings are interesting and novel\n- Illustrations are relevant\n- The presence of experiments to support the theoretical claims is particularly appreciated\n\n*Weaknesses*\n- I would like the code to be shared\n- See questions and suggestions", "other_comments_or_suggestions": "- The increase of forgetting in the latest layers is intuitive and expected. This has similarly been discussed in previous studies such as [1]. While the theoretical findings are interesting, further discussion with previous studies would be included.\n\n- From a theoretical perspective, it seems to me that such analysis can be adequately applied beyond Class-Incremental Learning scenarios (which is the only one considered here). I would appreciate discussion in this regard. While I believe the current experiments to be sufficient, experiments in Domain Incremental Learning cases would further strengthen the theoretical findings.", "questions_for_authors": "- What about when the representations are normalized? Such strategy is very common in representation learning, how would the current analysis be impacted?\n- Why not use centroid distances for definition 3, instead of the most distant features?\n- How far is the theoretical upper bound $U_{t, \\infty}^k$ from an experimental upper bound?  For example, what is the discrepancy between the model trained on the first task and a randomly initialized model? Such information could be interesting to understand how much the model *can* forget, as for now it is not clear if this saturation is the consequence of \"not being able the forget more\" or if it occurs before reaching such a stage, therefore maintaining some of the previous knowledge (and if so, how far are we from \"maximum forgetting\"?).\n- My biggest concern is the following. In Section 6.1, why compute the representation forgetting $\\Delta P_t^k(\\Delta t)$  as defined by [2], instead of directly computing your newly introduced metric $D_t^k(h_t,\\Delta t)$? I understand that the two are correlated as stated in 4.2 but all the theoretical analysis is made on $D_t^k(h_t,\\Delta t)$, which should be computable. I am unsure how much should the analysis of $\\Delta P_t^k(\\Delta t)$ gives insight on the behavior of the maximum value of $D_t^k(h_t,\\Delta t)$. Could you provide more explanation on this choice and its consequences?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a metric, which the authors term *representation discrepancy*, to measure representation forgetting in continual learning. The authors frame forgetting as a minimum alignment error between hidden layer representations through an optimal linear transformation measuring the minimum worst-case feature misalignment between different tasks at layer $k$. The manuscript provides both theoretical analysis and empirical validation of the proposed proxy for forgetting in continual learning. They derive an upper bound for the representation discrepancy measure and study its convergence rate under specific assumptions. The theoretical focus -- especially the analysis of different phases of forgetting/saturation and the dependence of forgetting on layer and depth -- of this work distinguishes it from previous research, which primarily emphasized experimental results. The paper's theoretical findings are supported by experiments on CIFAR-100 and ImageNet1K.

The reception of the work by reviewers was initially favorable, with reviewers primarily questioning:

+ The tightness of the upper bound approximation, which the authors point out in rebuttal is common to existing proxies on representation forgetting and point to their empirical evidence supporting the validity of $D^k_t$.
+ The validity of the core Assumption 1 across network architectures and CL scenarios. In rebuttal the authors again point to the alignment of their empirical results with the theoretical predictions, and additionally provide results for a ViT backbone and a domain-incremental scenario.

During the discussion phase the authors addressed the main concerns expressed by the reviewers, providing several new results supporting both the theoretical and empirical claims made in the original paper. The general consensus is that this is a valuable and novel contribution to the general problem of representation forgetting in continual learning. The recommendation is to Accept.
Decision: Accept (poster)