Original: Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer
Revised: Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer

Original: Abstract
Revised: Abstract

Original: By learning a sequence of tasks continually, an agent in continual learning (CL) canimprove the learning performance of both a new task and ‘old’ tasks by leveragingthe forward knowledge transfer and the backward knowledge transfer, respectively.
Revised: By learning a sequence of tasks continually, an agent in continual learning (CL) canimprove the learning performance of both a new task and ‘old’ tasks by leveragingthe forward knowledge transfer and the backward knowledge transfer, respectively.

Original: However, most existing CL methods focus on addressing catastrophic forgetting inneural networks by minimizing the modification of the learnt model for old tasks.
Revised: However, most existing CL methods focus on addressing catastrophic forgetting inneural networks by minimizing the modification of the learnt model for old tasks.

Original: This inevitably limits the backward knowledge transfer from the new task to theold tasks, because judicious model updates could possibly improve the learningperformance of the old tasks as well.
Revised: This inevitably limits the backward knowledge transfer from the new task to theold tasks, because judicious model updates could possibly improve the learningperformance of the old tasks as well.

Original: To tackle this problem, we first theoreticallyanalyze the conditions under which updating the learnt model of old tasks couldbe beneficial for CL and also lead to backward knowledge transfer, based on thegradient projection onto the input subspaces of old tasks.
Revised: To tackle this problem, we first theoreticallyanalyze the conditions under which updating the learnt model of old tasks couldbe beneficial for CL and also lead to backward knowledge transfer, based onthe gradient projection onto the input subspaces of old tasks.

Original: Building on the theoreticalanalysis, we next develop a ContinUal learning method with Backward knowlEdgetRansfer (CUBER), for a fixed capacity neural network without data replay.
Revised: Building on thetheoretical analysis, we next develop a ContinUal learning method with BackwardknowlEdge tRansfer (CUBER), for a fixed capacity neural network without datareplay.

Original: Inparticular, CUBER first characterizes the task correlation to identify the positivelycorrelated old tasks in a layer-wise manner, and then selectively modifies the learntmodel of the old tasks when learning the new task.
Revised: In particular, CUBER first characterizes the task correlation to identify thepositively correlated old tasks in a layer-wise manner, and then selectively modifiesthe learnt model of the old tasks when learning the new task.

Original: Experimental studies show that CUBER can even achieve positive backward knowledge transfer on several existing CL benchmarks for the first time without data replay, where the related baselinesstill suffer from catastrophic forgetting (negative backward knowledge transfer).
Revised: Experimental studiesshow that CUBER can even achieve positive backward knowledge transfer onseveral existing CL benchmarks for the first time, where the related baselines stillsuffer from catastrophic forgetting (negative backward knowledge transfer).

Original: The superior performance of CUBER on the backward knowledge transfer alsoleads to higher accuracy accordingly.
Revised: Thesuperior performance of CUBER on the backward knowledge transfer also leads tohigher accuracy accordingly.

Original:  Introduction
Revised:  Introduction

Original: One ultimate goal of artificial intelligence is to build an agent that can continually learn a sequenceof different tasks, so as to echo the remarkable learning capability of human beings during theirlifespan.
Revised: One ultimate goal of artificial intelligence is to build an agent that can continually learn a sequenceof different tasks, so as to echo the remarkable learning capability of human beings during theirlifespan.

Original: With more tasks being learnt, the agent is expected to be able to learn a new task moreeasily by leveraging the accumulated knowledge from old tasks (forward knowledge transfer) andalso further improve the learning performance of old tasks based on the gained knowledge of relatednew tasks (backward knowledge transfer).
Revised: With more tasks being learnt, the agent is expected to be able to learn a new task moreeasily by leveraging the accumulated knowledge from old tasks (forward knowledge transfer) andalso further improve the learning performance of old tasks based on the gained knowledge of relatednew tasks (backward knowledge transfer).

Original: Such a learning paradigm is known as continual learning(CL) [5, 23], which has recently attracted much attention.
Revised: Such a learning paradigm is known as continual learning(CL) [5, 22], which has recently attracted much attention.

Original: Most existing CL methods focus on addressing the catastrophic forgetting problem [20], i.e., theneural network may easily forget the knowledge of old tasks when learning a new task.
Revised: Most existing CL methods focus on addressing the catastrophic forgetting problem [19], i.e., theneural network may easily forget the knowledge of old tasks when learning a new task.

Original: The mainstrategy therein is to avoid the model change of old tasks when learning the new task.
Revised: The mainstrategy therein is to avoid the model change of old tasks when learning the new task.

Original: For example,regularization-based methods (e.g., [12, 1, 18]) penalize the modification of important weights ofold tasks; parameter-isolation based methods (e.g., [7, 26, 31, 9]) fix the model learnt for old tasks;and memory-based methods (e.g., [3, 6, 25]) aim to update the model with minimal interferenceintroduced to old tasks.
Revised: For example,regularization-based methods (e.g., [11, 1, 17]) penalize the modification of important weights ofold tasks; parameter-isolation based methods (e.g., [7, 25, 30, 9]) fix the model learnt for old tasks;and memory-based methods (e.g., [3, 6, 24]) aim to update the model with minimal interferenceintroduced to old tasks.

Original: While effectively mitigating catastrophic forgetting, such a model-freezing strategy inevitably limits the backward knowledge transfer, by implicitly and conservatively treatingthe model update of the new task as interference to old tasks.
Revised: While effectively mitigating catastrophic forgetting, such a model-freezing strategy inevitably limits the backward knowledge transfer, by implicitly and conservatively treatingthe model update of the new task as interference to old tasks.

Original: Intuitively, careful modifications of thelearnt model of old tasks may further improve the learning performance especially when the new taskshares similar knowledge to the old tasks.
Revised: Intuitively, careful modifications of thelearnt model of old tasks may further improve the learning performance especially when the new taskshares similar knowledge to the old tasks.

Original: This work seeks to explore CL from a new perspective by going beyond merely addressing forgetting:
Revised: This work seeks to explore CL from a new perspective by going beyond merely addressing forgetting:

Original: When and how could we improve the learnt model of old tasks to facilitate backward knowledgetransfer?
Revised: When and how could we improve the learnt model of old tasks to facilitate backward knowledgetransfer?

Original: In particular, we consider a fixed neural network and the scenario where the data of oldtasks is not accessible when learning a new task, which naturally rules out the expansion-basedmethods (e.g., [31, 9]) and the experience-replay methods (e.g., [22, 4]).
Revised: In particular, we consider a fixed neural network and the scenario where the data of oldtasks is not accessible when learning a new task, which naturally rules out the expansion-basedmethods (e.g., [30, 9]) and the experience-replay methods (e.g., [21, 4]).

Original: Clearly, achieving backwardknowledge transfer is very challenging in this case as mitigating forgetting herein is already nontrivial.
Revised: Clearly, achieving backwardknowledge transfer is very challenging in this case as mitigating forgetting herein is already nontrivial.

Original: To tackle this challenge, note that the orthogonal-projection based CL methods (e.g., [25, 17]), whichupdate the model with gradients orthogonal to the input subspaces of old tasks, have demonstrated thestate-of-the-art performance under the setting above.
Revised: To tackle this challenge, note that the orthogonal-projection based CL methods (e.g., [24, 16]), whichupdate the model with gradients orthogonal to the input subspaces of old tasks, have demonstrated thestate-of-the-art performance under the setting above.

Original: Thus motivated, we propose a new orthogonalprojection based CL method to carefully modify the learnt model of old tasks for better backwardknowledge transfer.
Revised: Thus motivated, we propose a new orthogonalprojection based CL method to carefully modify the learnt model of old tasks for better backwardknowledge transfer.

Original: The main contributions of this work include 1) the quantification of the conditions under whichmodifying the learnt model of old tasks is beneficial and 2) a new CL method inspired by the analysis,so as to answer the question above.
Revised: The main contributions of this work include 1) the quantification of the conditions under whichmodifying the learnt model of old tasks is beneficial and 2) a new CL method inspired by the analysis,so as to answer the question above.

Original: More specifically, we first introduce notions of ‘sufficientprojection’ and ‘positive correlation’ based on the gradient projection onto the subspaces of oldtasks to characterize the task correlation.
Revised: More specifically, we first introduce notions of ‘sufficientprojection’ and ‘positive correlation’ based on the gradient projection onto the subspaces of oldtasks to characterize the task correlation.

Original: We theoretically show that when the task gradients aresufficiently aligned in the old task subspace, appropriately updating the learnt model of old taskswhen learning the new task could improve the learning performance and possibly lead to a bettermodel for the old tasks.
Revised: We theoretically show that when the task gradients aresufficiently aligned in the old task subspace, appropriately updating the learnt model of old tasks whenlearning the new task could improve the learning performance and possibly lead to a better modelfor the old tasks.

Original: Based on this analysis, we next propose an orthogonal-projection based ContinUal learning method with Backward knowlEdge tRansfer (CUBER), which 1) first identifiesthe positively correlated old tasks for the new task and 2) carefully updates the learnt model of selectedold tasks along with the model learning for the new task.
Revised: Based on this analysis, we next propose an orthogonal-projection based ContinUallearning method with Backward knowlEdge tRansfer (CUBER), which 1) first identifies the positivelycorrelated old tasks for the new task and 2) carefully updates the learnt model of selected old tasksalong with the model learning for the new task.

Original: The experimental results on standard CLbenchmarks show that CUBER can achieve the best backward knowledge transfer compared to allthe considered baseline CL methods, which consequently improves the overall learning performance.
Revised: The experimental results on standard CL benchmarksshow that CUBER can achieve the best backward knowledge transfer compared to all the baseline CLmethods, which consequently improves the overall learning performance.

Original: In particular, to the best of our knowledge, CUBER is the first to demonstrate positive backwardknowledge transfer on these benchmarks using a fixed capacity network without experience replay,whereas all the compared baselines still suffer from catastrophic forgetting with negative backwardknowledge transfer.
Revised: In particular, to the best ofour knowledge, CUBER is the first to demonstrate positive backward knowledge transfer on thesebenchmarks using a fixed capacity network without experience replay, whereas all the comparedbaselines still suffer from catastrophic forgetting with negative backward knowledge transfer.

Original:  Related workContinual learning.
Revised:  Related workContinual learning.

Original: Existing CL methods can be generally divided into three categories: 1)
Revised: Existing CL methods can be generally divided into three categories: 1)

Original: Regularization-based methods (e.g., [12, 1, 15, 18]) add additional regularization in the loss functionto penalize the model change of the old tasks when learning the new task.
Revised: Regularization-based methods (e.g., [11, 1, 14, 17]) add additional regularization in the loss functionto penalize the model change of the old tasks when learning the new task.

Original: For example, Elastic Weight
Revised: For example, Elastic Weight

Original: Consolidation (EWC) [12] regularizes the update on the important weights that are evaluated using the
Revised: Consolidation (EWC) [11] regularizes the update on the important weights that are evaluated using the

Original: Fisher Information matrix; a recursive gradient optimization method is proposed in [18] to modify thegradient direction for the objective function regularized by the model change during CL.
Revised: Fisher Information matrix; a recursive gradient optimization method is proposed in [17] to modify thegradient direction for the objective function regularized by the model change during CL.

Original: 2) Parameterisolation based methods (e.g., [24, 16, 31, 30, 26, 29]) allocate different model parameters to differenttasks, fix the parameters for old tasks by masking them out during the new task learning and expandthe network when needed.
Revised: 2) Parameterisolation based methods (e.g., [23, 15, 30, 29, 25, 28]) allocate different model parameters to differenttasks, fix the parameters for old tasks by masking them out during the new task learning and expandthe network when needed.

Original: For example, Additive Parameter Decomposition (APD) [30] proposes ahierarchical knowledge consolidation method to separate task-specific parameters and task-sharedparameters, where the task-specific parameters are kept intact to address forgetting.
Revised: For example, Additive Parameter Decomposition (APD) [29] proposes ahierarchical knowledge consolidation method to separate task-specific parameters and task-sharedparameters, where the task-specific parameters are kept intact to address forgetting.

Original: 3) Memory-basedmethods can be further divided into experience-replay methods and orthogonal-projection basedmethods depending on if the data of old tasks is available for the new task.
Revised: 3) Memory-basedmethods can be further divided into experience-replay methods and orthogonal-projection basedmethods depending on if the data of old tasks is available for the new task.

Original: Experience-replay methods(e.g.
Revised: Experience-replay methods(e.g.

Original: , [3, 22, 8]) store and replay the old tasks data when learning the new task, while orthogonalprojection based methods (e.g., [32, 6, 25, 17]) update the model in the orthogonal direction ofold tasks without storing the raw data of old tasks.
Revised: , [3, 21, 8]) store and replay the old tasks data when learning the new task, while orthogonalprojection based methods (e.g., [31, 6, 24, 16]) update the model in the orthogonal direction of oldtasks without storing the raw data of old tasks.

Original: In particular, [10] proposes natural continuallearning based on Bayesian learning to unify weight regularization and orthogonal gradient projection.

Original: However, [10] leverages the knowledge of the current task in an implicit way to improve on theold tasks, whereas our method attempts more explicitly to update the model of old tasks wheneverrelevant new information is available.

Original: Knowledge transfer.
Revised: Knowledge transfer.

Original: Most of CL methods focus on the catastrophic forgetting problem, and littleattention has been put on the knowledge transfer across tasks in CL with neural networks.
Revised: Most of CL methods focus on the catastrophic forgetting problem, and littleattention has been put on the knowledge transfer across tasks in CL with neural networks.

Original: Progressive
Revised: Progressive

Original: Network [24] considers the forward knowledge transfer but learns one model for each task.
Revised: Network [23] considers the forward knowledge transfer but learns one model for each task.

Original: [17]proposes trust region gradient projection (TRGP) to facilitate forward knowledge transfer fromcorrelated old tasks to the new task through a scaling matrix, but the model is still updated alongthe direction orthogonal to the input subspaces of old tasks to mitigate forgetting.
Revised: [16]proposes trust region gradient projection (TRGP) to facilitate forward knowledge transfer fromcorrelated old tasks to the new task through a scaling matrix, but the model is still updated alongthe direction orthogonal to the input subspaces of old tasks to mitigate forgetting.

Original: [11] studies theknowledge transfer in a mixed sequence of similar and dissimilar tasks, but employs a complicatedtask similarity detection mechanism where two additional networks need to be trained first for eachtask before learning the new task.
Revised: [10] studies theknowledge transfer in a mixed sequence of similar and dissimilar tasks, but employs a complicated task similarity detection mechanism where two additional networks need to be trained first for eachtask before learning the new task.

Original:  When could we improve the learnt model of old tasks?
Revised:  When could we improve the learnt model of old tasks?

Original: In continual learning, a sequence of tasks T = { t } Tt =1 arrives sequentially.
Revised: In continual learning, a sequence of tasks T = { t } Tt =1 arrives sequentially.

Original: For each task t , there is adataset D t = { ( x ti , y ti ) } N t
Revised: For each task t , there is adataset D t = { ( x ti , y ti ) } N t

Original: i
Revised: i

Original: =1 with N t sample pairs, which is sampled from some unknown distribution 101 P t .
Revised: =1 with N t sample pairs, which is sampled from some unknown distribution 96 P t .

Original: In this work, we consider a fixed capacity neural network with weights w .
Revised: In this work, we consider a fixed capacity neural network with weights w .

Original: When learning a newtask t , we only have access to the dataset D t and no datapoints of old tasks are available.
Revised: When learning a newtask t , we only have access to the dataset D t and no datapoints of old tasks are available.

Original: We furtherdenote L ( w , D t ) =
Revised: We furtherdenote L ( w , D t ) =

Original: L t ( w ) as the loss function for training, e.g., mean squared and cross-entropyloss, and w t as the model after learning task t .
Revised: L t ( w ) as the loss function for training, e.g., mean squared and cross-entropyloss, and w t as the model after learning task t .

Original: Intuitively, if the new task t has strong similarity with some old tasks, appropriate model update ofthe new task would not introduce forgetting of the similar old tasks, but can lead to a better model forthese old tasks due to the backward knowledge transfer.
Revised: Intuitively, if the new task t has strong similarity with some old tasks, appropriate model update ofthe new task would not introduce forgetting of the similar old tasks, but can lead to a better model forthese old tasks due to the backward knowledge transfer.

Original: To formally characterize this task similarity,we introduce the following conditions on the task gradients.
Revised: To formally characterize this task similarity,we introduce the following conditions on the task gradients.

Original: Definition 1 (Sufficient Projection) .
Revised: Definition 1 (Sufficient Projection) .

Original: For any new task t ∈ [1 , T ] , we say it has sufficient gradient 110 projection on the input subspace of old task j ∈ [0 , t − 1] if for some ϵ 1 ∈ (0 , 1)∥ [Equation] [Equation] w
Revised: For any new task t ∈ [1 , T ] , we say it has sufficient gradientprojection on the input subspace of old task j ∈ [0 , t − 1] if for some ϵ 1 ∈ (0 , 1)

Original: [Equation]
Revised: [Equation]

Original: 1 ) ∥ 2 .

Original: Here Proj S
Revised: Here Proj S

Original: j defines the projection on the input subspace S j of task j : Proj S j ( A ) =
Revised: j defines the projection on the input subspace S j of task j : Proj S j ( A ) =

Original: AB j ( B j ) ′
Revised: AB j ( B j ) ′

Original: forsome matrix A and B j is the bases for S j .
Revised: forsome matrix A and B j is the bases for S j .

Original: This definition of sufficient projection follows the sameline of the trust region defined in [17], which implies that task t and j may have sufficient commonbases between their input subspaces and hence are strongly correlated, because the gradient lies inthe span of the input [33].
Revised: This definition of sufficient projection follows the sameline of the trust region defined in [16], which implies that task t and j may have sufficient commonbases between their input subspaces and hence are strongly correlated, because the gradient lies inthe span of the input [32].

Original: Definition 2 (Positive Correlation) .
Revised: Definition 2 (Positive Correlation) .

Original: For any new task t ∈ [1 , T ] , we say it has positive correlation 118 with old task j ∈ [0 , t − 1] if for some ϵ 2 ∈ (0 , 1)⟨∇L j ( w j ) , ∇L t ( w t − 1 ) ⟩ ≥ ϵ 2 ∥∇L j ( w j ) ∥ 2 ∥∇L t ( w t − 1 ) ∥ 2 .
Revised: For any new task t ∈ [1 , T ] , we say it has positive correlationwith old task j ∈ [0 , t − 1] if for some ϵ 2 ∈ (0 , 1) [Equation] w [Equation]

Revised: ∇L

Original: 120 While sufficient projection suggests the possibly strong correlation between two tasks t and j ,
Revised: [Equation] w [Equation] ∥∇L

Revised: [Equation]

Revised: w

Revised: [Equation]

Revised: ∥∇L

Revised: [Equation]

Revised: ( w

Revised: [Equation]

Original:  Definition 2 goes one step further by introducing the positive correlation between tasks, in the sensethat the initial model gradient of task t is not conflicting with the model gradient when learning taskj .
Revised: − 1 ) ∥ [Equation] While sufficient projection suggests the possibly strong correlation between two tasks t and j , Definition 2 goes one step further by introducing the positive correlation between tasks, in the sensethat the initial model gradient of task t is not conflicting with the model gradient when learning taskj .

Original: In fact, it can be shown that the positive correlation implies the sufficient projection in Definition 1for some ϵ 1 ≥ ϵ 2 .
Revised: In fact, it can be shown that the positive correlation implies the sufficient projection in Definition 1for some ϵ 1 ≥ ϵ 2 .

Original: Note that both Definition 1 and 2 characterize the correlation based on the initialmodel w t − 1 , which allows the task correlation detection before learning the new task t .
Revised: Note that both Definition 1 and 2 characterize the correlation based on the initialmodel w t − 1 , which allows the task correlation detection before learning the new task t .

Original: For ease of exposition, consider the scenario with a sequence of two tasks 1 and 2.
Revised: For ease of exposition, consider the scenario with a sequence of two tasks 1 and 2.

Original: Let F ( w ) =
Revised: Let F ( w ) =

Original: L ( w , D 1 )+ L ( w , D 2 ) , g 1 ( w ) = ∇ w L ( w , D 1 ) and g 2 ( w ) = ∇ w L ( w , D 2 ) .
Revised: L ( w , D 1 )+ L ( w , D 2 ) , g 1 ( w ) = ∇ w L ( w , D 1 ) and g 2 ( w ) = ∇ w L ( w , D 2 ) .

Original: Given the model learntfor task 1 as w 1 , we consider the following two model update rules for learning task 2, where onlythe data of task 2 is available :
Revised: Given the model learntfor task 1 as w 1 , we consider the following two model update rules for learning task 2, where onlythe data of task 2 is available :

Original: • (Rule #1 ): w k +1 =
Revised: • (Rule #1 ): w k +1 =

Original: w k − α
Revised: w k − α

Original: [ g 2 ( w k ) − Proj S 1 ( g 2 ( w k ))] ,
Revised: [ g 2 ( w k ) − Proj S 1 ( g 2 ( w k ))] ,

Original: • (Rule #2 ): w k +1 =
Revised: • (Rule #2 ): w k +1 =

Original: w k − α g 2 ( w k ) .
Revised: w k − α g 2 ( w k ) .

Original: Here w 0 = w 1 and k ∈ [0 , K − 1] .
Revised: Here w 0 = w 1 and k ∈ [0 , K − 1] .

Original: Rule #1 characterizes the model update of the orthogonalprojection based methods, where no interference is introduced to task 1 as the model change isorthogonal to the input subspace S 1 of task 1 [17] (and hence the learnt model of task 1 will notbe updated).
Revised: Rule #1 characterizes the model update of the orthogonalprojection based methods, where no interference is introduced to task 1 as the model change isorthogonal to the input subspace S 1 of task 1 [16] (and hence the learnt model of task 1 will notbe updated).

Original: In contrast, the vanilla gradient descent Rule #2 for learning task 2 will inevitablymodify the learnt model w 1 for task 1 unless Proj S 1 ( g 2 ( w k ))
Revised: In contrast, the vanilla gradient descent Rule #2 for learning task 2 will inevitablymodify the learnt model w 1 for task 1 unless Proj S 1 ( g 2 ( w k ))

Original: = 0 , i.e., the extreme case where bothforgetting and knowledge transfer do not occur.
Revised: = 0 , i.e., the extreme case where bothforgetting and knowledge transfer do not occur.

Original: In what follows, we evaluate the performance ofthese update rules.
Revised: In what follows, we evaluate the performance ofthese update rules.

Original: Theorem 1.
Revised: Theorem 1.

Original: Suppose that the loss L is B -Lipschitz and H 2 -smooth.
Revised: Suppose that the loss L is B -Lipschitz and H 2 -smooth.

Original: Let α < min (cid:110) 1 H , γ ∥ g 1 ( w 0 )
Revised: Let α < min (cid:110) 1 H , γ ∥ g 1 ( w 0 )

Original: ∥
Revised: ∥

Original: HBK (cid:111)and ϵ 2 ≥ (2+ γ 2 ) ∥ g 1 ( w 0 ) ∥ 4 ∥ g 2 ( w 0 ) ∥ for some γ ∈ (0 , 1) .
Revised: HBK (cid:111)and ϵ 2 ≥ (2+ γ 2 ) ∥ g 1 ( w 0 ) ∥ 4 ∥ g 2 ( w 0 ) ∥ for some γ ∈ (0 , 1) .

Original: We have the following results:
Revised: We have the following results:

Original: (1) If L is convex, Rule #2 for task 2 converges to the optimal model w ∗
Revised: (1) If L is convex, Rule #2 for task 2 converges to the optimal model w ∗

Original: =
Revised: =

Original: arg min F ( w ) ;
Revised: arg min F ( w ) ;

Original: (2) If L is nonconvex, Rule #2 for task 2 converges to the first order stationary point, i.e.,
Revised: (2) If L is nonconvex, Rule #2 for task 2 converges to the first order stationary point, i.e.,

Original: [Equation]
Revised: [Equation]

Original: Theorem 1 indicates that updating the model with Rule #2 will lead to the convergence to theminimizer of the joint objective function F ( w ) in the convex case, and the convergence to the firstorder stationary point in the nonconvex case, when task 1 and 2 satisfy the positive correlation with
Revised: Theorem 1 indicates that updating the model with Rule #2 will lead to the convergence to theminimizer of the joint objective function F ( w ) in the convex case, and the convergence to the firstorder stationary point in the nonconvex case, when task 1 and 2 satisfy the positive correlation withϵ 2 ≥ (2+ γ 2 ) ∥ g 1 ( w 0 )

Original: [Equation]{+∥ 
 ∥ g 2 ( w 0 ) ∥ .+}
Revised: {+∥ 
 ∥ g 2 ( w 0 ) ∥ .+}

Original: That is to say, Rule #2 not only results in a good model for task 2, but can alsobe beneficial for the joint learning of task 1 and 2.
Revised: That is to say, Rule #2 not only results in a good model for task 2, but can alsobe beneficial for the joint learning of task 1 and 2.

Original: Note that since w 0 is the learnt model of task 1, ingeneral we have ∥ g 1 ( w 0 )
Revised: Note that since w 0 is the learnt model of task 1, ingeneral we have ∥ g 1 ( w 0 )

Original: ∥ <
Revised: ∥ <

Original: ∥ g 2 ( w 0 ) ∥ .
Revised: ∥ g 2 ( w 0 ) ∥ .

Original: Proof of Theorem 1 can be found in Appendix A.
Revised: Proof of Theorem 1 can be found in Appendix A.

Original: Theorem 2.
Revised: Theorem 2.

Original: Suppose that the loss L is B -Lipschitz and H 2 -smooth.
Revised: Suppose that the loss L is B -Lipschitz and H 2 -smooth.

Original: We have the following results:
Revised: We have the following results:

Original: (1) Let w c and w r be the model parameters after applying one update to some initial model w by using Rule #1 and Rule #2 , respectively.
Revised: (1) Let w c and w r be the model parameters after applying one update to some initial model w byusing Rule #1 and Rule #2 , respectively.

Original: Suppose α < min (cid:110) 1 H , γ ∥ g 1 ( w 0 )
Revised: Suppose α < min (cid:110) 1 H , γ ∥ g 1 ( w 0 )

Original: ∥
Revised: ∥

Original: HBK
Revised: HBK

Original: (cid:111)
Revised: (cid:111)

Original: , ϵ 1 ≥ (cid:113) 1+2 αH 2+ αH and ϵ 2 ≥ (2+ γ 2 )∥
Revised: , ϵ 1 ≥ (cid:113) 1+2 αH 2+ αH andϵ 2 ≥ (2+ γ 2 )

Original:  g 1 ( w 0 ) ∥ 4 ∥ g 2 ( w 0 ) ∥ for some γ ∈ (0 , 1) .
Revised: ∥ g 1 ( w 0 ) ∥ 4 ∥ g 2 ( w 0 ) ∥ for some γ ∈ (0 , 1) .

Original: It follows that F ( w r ) ≤ F ( w c ) ; (2) Let w k be the k -th iterate for task 2 with Rule #2 .
Revised: It follows that F ( w r ) ≤ F ( w c ) ;(2) Let w k be the k -th iterate for task 2 with Rule #2 .

Original: Suppose that ⟨ g 1 ( w 0 ) , g 2 ( w i ) ⟩ ≥
Revised: Suppose that ⟨ g 1 ( w 0 ) , g 2 ( w i ) ⟩ ≥

Original: ϵ 2 ∥ g 1 ( w 
Revised: ϵ 2 ∥ g 1 ( w 0 ) ∥∥

Original: ∥∥ g 2 ( w i ) ∥ for i ∈ [0 , k − 1] and α ≤ 4 ϵ 2 ∥ g 1 ( w 0 ) 
Revised: g 2 ( w i ) ∥ for i ∈ [0 , k − 1] and α ≤ 4 ϵ 2 ∥ g 1 ( w 0 ) ∥

Original: ∥ HBk 1 ..
Revised: HBk 1 ..

Original: It follows that L 1 ( w k ) ≤ L 1 ( w 1 ) .
Revised: It follows that L 1 ( w k ) ≤ L 1 ( w 1 ) .

Original: Intuitively, the first part of Theorem 2 shows that updating with Rule #2 can achieve lower lossvalue compared to Rule #1 after one step gradient update when task 1 and 2 satisfy the sufficientprojection with ϵ 1 ≥ (cid:113) 

Revised: Intuitively, the first part of Theorem 2 shows that updating with Rule #2 can achieve lower lossvalue compared to Rule #1 after one step gradient update when task 1 and 2 satisfy the sufficientprojection with ϵ 1 ≥ (cid:113)

Original: +2 αH 2+ αH and the positive correlation with ϵ 2 ≥ (2+ γ 2 )
Revised: +2 αH 2+ αH and the positive correlation with ϵ 2 ≥ (2+ γ 2 )

Original: ∥ g 1 ( w 0 ) ∥ 4 ∥ g 2 ( w 0 ) ∥ .
Revised: ∥ g 1 ( w 0 ) ∥ 4 ∥ g 2 ( w 0 ) ∥ .

Original: If the positivecorrelation condition also holds for iterates of Rule #2 when learning the task 2, the second part of
Revised: If the positivecorrelation condition also holds for iterates of Rule #2 when learning the task 2, the second part of

Original: Theorem 2 indicates that updating the model indeed leads to a better model for task 1 with respect to
Revised: Theorem 2 indicates that updating the model indeed leads to a better model for task 1 with respect to

Original: L 1 .
Revised: L 1 .

Original: In a nutshell, when 1) the task 2 has sufficient gradient projection onto the subspace of task 1 and2) the projected gradient is also aligned well with the gradient of task 1 in the subspace of task 1,updating the model along g 2 will modify the learnt model w 1 towards a favorable direction for CLand enable the backward knowledge transfer to task 1.
Revised: In a nutshell, when 1) the task 2 has sufficient gradient projection onto the subspace of task 1 and2) the projected gradient is also aligned well with the gradient of task 1 in the subspace of task 1,updating the model along g 2 will modify the learnt model w 1 towards a favorable direction for CLand enable the backward knowledge transfer to task 1.

Original: Proof of Theorem 2 is in Appendix B.
Revised: Proof of Theorem 2 is in Appendix B.

Original: It is worth noting that the conditions for Theorem 1 and 2 depend only on the initial model gradientg 2 ( w 1 ) before learning task 2 and the gradient g 1 ( w 1 ) when learning task 1, which are both easilyaccessible and calculated.
Revised: It is worth noting that the conditions for Theorem 1 and 2 depend only on the initial model gradientg 2 ( w 1 ) before learning task 2 and the gradient g 1 ( w 1 ) when learning task 1, which are both easilyaccessible and calculated.

Original: Particularly, the positive correlation can be evaluated by only storingthe gradient g 1 ( w 1 ) instead of the data of task 1.
Revised: Particularly, the positive correlation can be evaluated by only storingthe gradient g 1 ( w 1 ) instead of the data of task 1.

Original: In stark contrast, the task gradient correlationcharacterization in [3, 22] involves the gradient evaluation of old tasks with respect to the currentmodel weight, and hence requires the data of old tasks when learning the new task.
Revised: In stark contrast, the task gradient correlationcharacterization in [3, 21] involves the gradient evaluation of old tasks with respect to the currentmodel weight, and hence requires the data of old tasks when learning the new task.

Original:  Continual learning with backward knowledge transfer
Revised:  Continual learning with backward knowledge transfer

Original: Based on the theoretical analysis above, we next propose a continual learning method with backwardknowledge transfer (CUBER), by selectively updating the learnt model of old tasks when learningthe new task.
Revised: Based on the theoretical analysis above, we next propose a continual learning method with backwardknowledge transfer (CUBER), by selectively updating the learnt model of old tasks when learningthe new task.

Original: In particular, CUBER works in a layer-wise manner:
Revised: In particular, CUBER works in a layer-wise manner:

Original: Given a L -layer network, CUBERfirst characterizes the task correlation for each layer, and then employs different strategies to learn thenew task depending on the task correlation.
Revised: Given a L -layer network, CUBERfirst characterizes the task correlation for each layer, and then employs different strategies to learn thenew task depending on the task correlation.

Original: More specifically, denote the set of weights as w = { w l } Ll =1 , where w l is the layer-wise weightfor layer l .
Revised: More specifically, denote the set of weights as w = { w l } Ll =1 , where w l is the layer-wise weight forlayer l .

Original: Given the data input x ti for task t , let x tl,i be the input of layer l and x t 1 ,i = x ti .
Revised: Given the data input x ti for task t , let x tl,i be the input of layer l and x t 1 ,i = x ti .

Original: Denotef as the operation of the network layer.
Revised: Denote f asthe operation of the network layer.

Original: The output x tl +1 ,i for layer l can be then computed asx tl +1 ,i =
Revised: The output x tl +1 ,i for layer l can be then computed asx tl +1 ,i =

Original: f
Revised: f

Original: ( w l , x tl,i ) .
Revised: ( w l , x tl,i ) .

Original: Here we denote x tl,i as the representations of the input x ti at layer l .
Revised: Here we denote x tl,i as the representations of the input x ti at layer l .

Original: Given anew task t ≥ 2 , we characterize its task correlation with some old task j ∈ [1 , t − 1] for layer l intothree different regimes based on Definition 1 and 2.
Revised: Given a new task t ≥ 2 , wecharacterize its task correlation with some old task j ∈ [1 , t − 1] for layer l into three differentregimes based on Definition 1 and 2.

Original: Regime 1 (no forgetting):
Revised: Regime 1 (no forgetting):

Original: We say that task j ∈ Reg t l, 1 for layer l if the following holds:
Revised: We say that task j ∈ Reg tl, 1 for layer l if the following holds:

Original: [Equation]
Revised: [Equation]

Original: In this case, the layer-wise input subspace S jl for task j and S tl for task t are treated as nearly 184 orthogonal ((a) in Fig. 1).
Revised: In this case, the layer-wise input subspace S jl for task j and S tl for task t are treated as nearlyorthogonal ((a) in Fig. 1).

Original: As a result, there is little knowledge transfer between these two tasks, and updating the model along with ∇L t ( w l ) would not introduce much interference to task j .
Revised: As a result, there is little knowledge transfer between these two tasks, andupdating the model along with ∇L t ( w l ) would not introduce much interference to task j .

Original: To reinforce 186 the knowledge protection for task j , the model will be updated based on orthogonal projection:
Revised: To reinforcethe knowledge protection for task j , the model will be updated based on orthogonal projection :

Original: [Equation]
Revised: [Equation]

Original: Regime 2 (forward knowledge transfer) :
Revised: Regime 2 (forward knowledge transfer) :

Original: We say that task j ∈ Reg tl, 2 for layer l if the following holds:
Revised: We say that task j ∈ Reg tl, 2 for layer l if the following holds:

Original: [Equation]
Revised: [Equation]

Original: In this case, task j and task t can be strongly correlated for layer l but possibly with ‘negative’ corre- 189 lation ((b) in Fig. 1), in the sense that updating the model along with ∇L t ( w l ) would substantiallymodify the learnt model Proj S jl ( w t − 1 ) for task j in an unfavorable way and lead to the forgettingof task j .
Revised: In this case, task j and task t can be strongly correlated for layer l but possibly with ‘negative’ correlation ((b) in Fig. 1), in the sense that updating the model along with ∇L t ( w l ) would substantiallymodify the learnt model Proj S jl ( w t − 1 ) for task j in an unfavorable way and lead to the forgettingof task j .

Original: As there will be better forward knowledge transfer from the old task j to the new task 192 t for layer l , we leverage the scaled weight projection in [17] to facilitate the forward knowledgetransfer through a scaling matrix Q j,tl , whereas the model is still updated using orthogonal projectionto protect the knowledge of old tasks:
Revised: As there will be better forward knowledge transfer from the old task j to the new taskt for layer l , we leverage the scaled weight projection in [16] to facilitate the forward knowledgetransfer through a scaling matrix Q j,tl , whereas the model is still updated using orthogonal projectionto protect the knowledge of old tasks:

Original: [Equation]
Revised: [Equation]

Original: Here B jl is the bases matrix for  subspace S jl .
Revised: Here B jl is the bases matrix for the subspace S jl .

Original: Intuitively, the scaled weight projection w l B jl Q j,tl ( B jl ) ′replaces the weight projection Proj S jl ( w l ) of task j by a scaled version, which transforms theknowledge of task j to the appropriate model of the new task t through the optimization of Q j,tl .
Revised: Intuitively, the scaled weight projectionw l B jl Q j,tl ( B jl ) ′ replaces the weight projection Proj S jl ( w l ) of task j by a scaled version, whichtransforms the knowledge of task j to the appropriate model of the new task t through the optimization of Q j,tl .

Original: Regime 3 (backward knowledge transfer) :
Revised: Regime 3 (backward knowledge transfer) :

Original: We say  task j ∈ Reg tl, 3 for layer l if the following holds:
Revised: We say that task j ∈ Reg tl, 3 for layer l if the followingholds:

Original: [Equation]
Revised: [Equation]

Original: With the sufficient projection and the positive correlation, updating the model along with ∇L t ( w l )could possibly lead to a better model for continual learning and also improve the learning performanceof the old task
Revised: With the sufficient projection and the positive correlation, updating the model along with ∇L t ( w l )could possibly lead to a better model for continual learning and also improve the learning performance of the old task

Original: j ((c) in Fig. 1).
Revised: j ((c) in Fig. 1).

Original: To avoid overly-optimistic modification on the learnt model of taskj , we further regularize the projection of the model change on the subspace S jl , given that themodel projection is indeed frozen for task j to address forgetting with orthogonal projection, i.e.,
Revised: To avoid overly-optimistic modification on the learnt model of taskj , we further regularize the projection of the model change on the subspace S jl , given that themodel projection is indeed frozen for task j to address forgetting with orthogonal projection, i.e.,

Original: Proj S j l ( w t − 1 l ) 
Revised: Proj S jl ( w t − 1 l ) =

Original: [Equation]

Original: Proj S j l ( w j l ) .
Revised: Proj S jl ( w jl ) .

Original: This gives the following model update:
Revised: This gives the following model update:

Original: [Equation]
Revised: [Equation]

Original: Note that the gradient projection on the subspaces of old tasks in Regime 1 and 2 is removed from thegradient in the model update above.
Revised: Note that the gradient projection on the subspaces of old tasks in Regime 1 and 2 is removed from thegradient in the model update above.

Original: Besides, we also learn a scaling matrix Q j,tl for better forwardknowledge transfer from the old 
Revised: Besides, we also learn a scaling matrix Q j,tl for better forwardknowledge transfer from the old task j ∈ Reg tl, 3 to the new task t as in Eq.

Original: However, continuous model update with task t gradient ∇L t ( w l )will eventually lead to the task specific model for task t , whichusually differs from the model of task j in Regime 3 (Fig. 2).
Revised: However, continuous model update with task t gradient ∇L t ( w l )will eventually lead to the task specific model for task t , which usually differs from the model of task j in Regime 3 (Fig. 2).

Original: To address this problem, note that the second part of Theorem2 characterizes the condition under which updating the modelwith ∇L t ( w l ) will result in backward knowledge transfer.
Revised: To addressthis problem, note that the second part of Theorem 2 characterizesthe condition under which updating the model with ∇L t ( w l ) willresult in backward knowledge transfer.

Original: Thusmotivated, for a model update iterate w l,k at k -th iteration whenlearning the new task t , we evaluate the following condition fortask j ∈ Reg tl, 3 :
Revised: Thus motivated, for a modelupdate iterate w l,k at k -th iteration when learning the new task t , weevaluate the following condition for task j ∈ Reg tl, 3 :

Original: [Equation]
Revised: [Equation]

Original: and degenerate task j to Regime 2, i.e., remove the gradient projection on the subspace S jl from the task t gradient and stop modifying the model for task j , if thecondition (2) does not hold.
Revised: and degenerate task j to Regime 2, i.e., remove the gradient projection on the subspace S jl from the task t gradient and stop modifyingthe model for task j , if the condition (2) does not hold.

Original: Bases extraction: After learning the model w t for task t , we construct the input subspace for eachlayer l by extracting bases from its representation x tl,i based on singular value decomposition (SVD)[17].
Revised: Bases extraction: After learning the model w t for task t , we construct the input subspace for eachlayer l by extracting bases from its representation x tl,i based on singular value decomposition (SVD)[16].

Original: More specifically, given a batch of n samples and the learnt model w t , the representation matrixfor layer l is denoted as R tl =
Revised: More specifically, given a batch of n samples and the learnt model w t , the representation matrixfor layer l is denoted as R tl =

Original: [ x tl, 1 , x tl, 2 , ..., x tl,n ] .
Revised: [ x tl, 1 , x tl, 2 , ..., x tl,n ] .

Original: Since the bases of the old tasks may includeimportant bases for the new task t , we determine the bases for task t from the union of the bases ofthe old tasks and the newly generated bases.
Revised: Since the bases of the old tasks may includeimportant bases for the new task t , we determine the bases for task t from the union of the bases ofthe old tasks and the newly generated bases.

Original: Towards this end, we first concatenate the bases B jl forj ∈ [0 , t − 1] together in a matrix O tl and remove the common bases.
Revised: Towards this end, we first concatenate the bases B jl forj ∈ [0 , t − 1] together in a matrix O tl and remove the common bases.

Original: Then SVD is applied on theresidual representation matrix ˜ 
Revised: Then SVD is applied on theresidual representation matrix ˜ R tl =

Original: [Equation]
Revised: R tl − R tl O tl

Original: ( O tl ) ′ , i.e., ˜ R tl =
Revised: ( O tl ) ′ , i.e., ˜ R tl =

Original: U tl Σ tl ( V tl ) ′ , where U tl is the leftsingular matrix.
Revised: U tl Σ tl ( V tl ) ′ , where U tl is the leftsingular matrix.

Original: We construct B tl by selecting the most important bases from the pool of bases in O tland U tl depending on their eigenvalues, which yields a low rank matrix approximation of R tl .
Revised: We construct B tl by selecting the most important bases from the pool of bases in O tland U tl depending on their eigenvalues, which yields a low rank matrix approximation of R tl .

Original: To conclude, the optimization problem for learning the new task t can be summarized as follows:
Revised: To conclude, the optimization problem for learning the new task t can be summarized as follows:

Original: [Equation]
Revised: [Equation]

Original:  The key idea is that we conservatively update the model for old tasks in Regime 3 while usingorthogonal projection to preserve the knowledge of other old tasks; in the meanwhile, we leveragethe scaled weight projection to reuse the model knowledge of old tasks in both Regime 2 and 3 tofacilitate forward knowledge transfer.
Revised: [Table] The key idea is that we conservatively update the model for old tasks in Regime 3 while usingorthogonal projection to preserve the knowledge of other old tasks; in the meanwhile, we leveragethe scaled weight projection to reuse the model knowledge of old tasks in both Regime 2 and 3 tofacilitate forward knowledge transfer.

Original: It is worth to note that the task correlation is determined beforelearning the new task t , as both the strong projection condition and the positive correlation conditiononly depend on the initial model gradient for the new task.
Revised: It is worth to note that the task correlation is determined beforelearning the new task t , as both the strong projection condition and the positive correlation conditiononly depend on the initial model gradient for the new task.

Original: And this can be achieved by a simpleforward-backward pass through the initial model with a batch of new task data.
Revised: And this can be achieved by a simpleforward-backward pass through the initial model with a batch of new task data.

Original: The overview of
Revised: The overview of

Original: CUBER can be found in Algorithm 1.
Revised: CUBER can be found in Algorithm 1.

Original:  Experiments
Revised:  Experiments

Original: Datasets.
Revised: Datasets.

Original: We evaluate the performance of CUBER on four standard CL benchmarks.
Revised: We evaluate the performance of CUBER on four standard CL benchmarks.

Original: (1) Permuted
Revised: (1) Permuted

Original: MNIST: a variant of the MNIST dataset [14] where random permutations are applied to the inputpixels.
Revised: MNIST: a variant of the MNIST dataset [13] where random permutations are applied to the input pixels.

Original: Following [19, 25], we divide the dataset into 10 tasks with different permutations and eachtask includes 10 classes.
Revised: Following [18, 24], we divide the dataset into 10 tasks with different permutations and eachtask includes 10 classes.

Original: (2) Split CIFAR-100: we divide the CIFAR-100 dataset [13] into 10 differenttasks, where each task is a 10-way multi-class classification problem.
Revised: (2) Split CIFAR-100: we divide the CIFAR-100 dataset [12] into 10 differenttasks, where each task is a 10-way multi-class classification problem.

Original: (3) 5-Datasets: we consider asequence of 5 datasets, i.e., CIFAR-10, MNIST, SVHN [21], not-MNIST[2], Fashion MNIST[28],and the classification problem on each dataset is a task.
Revised: (3) 5-Datasets: we consider asequence of 5 datasets, i.e., CIFAR-10, MNIST, SVHN [20], not-MNIST[2], Fashion MNIST[27],and the classification problem on each dataset is a task.

Original: (4) Split MiniImageNet: we divide the
Revised: (4) Split MiniImageNet: we divide the

Original: MiniImageNet dataset [27] into 20 tasks, where each task includes 5 classes.
Revised: MiniImageNet dataset [26] into 20 tasks, where each task includes 5 classes.

Original: Baselines.
Revised: Baselines.

Original: In this work, we compare CUBER with the following baselines on the benchmarksmentioned above.
Revised: In this work, we compare CUBER with the following baselines on the benchmarksmentioned above.

Original: (1) EWC [12]: a regularization-based method that leverages Fisher Informationmatrix for weights importance evaluation; (2) HAT [26]: learns a hard attention mask to preserve theknowledge of old task; (3) Orthogonal Weight Modulation (OWM) [32]: learns a projector matrix to project the gradient of the new task to the orthogonal direction of input subspace of old tasks; (4)
Revised: (1) EWC [11]: a regularization-based method that leverages Fisher Informationmatrix for weights importance evaluation; (2) HAT [25]: learns a hard attention mask to preserve theknowledge of old task; (3) Orthogonal Weight Modulation (OWM) [31]: learns a projector matrix toproject the gradient of the new task to the orthogonal direction of input subspace of old tasks; (4)

Original: Gradient Projection Memory (GPM) [25]: stores the bases of the input subspace of old tasks and thenupdates the model with the gradient projection orthogonal to the subspace spanned by these bases;(5)
Revised: Gradient Projection Memory (GPM) [24]: stores the bases of the input subspace of old tasks and thenupdates the model with the gradient projection orthogonal to the subspace spanned by these bases;(5)

Original: TRGP [17]: proposes a scaled weight projection to facilitate the forward knowledge transfer fromrelated old tasks to the new task while updating the model based on orthogonal gradient projection,which demonstrates the state-of-the-art performance for a fixed capacity network; (6) Averaged GEM(A-GEM)
Revised: TRGP [16]: proposes a scaled weight projection to facilitate the forward knowledge transfer fromrelated old tasks to the new task while updating the model based on orthogonal gradient projection,which demonstrates the state-of-the-art performance for a fixed capacity network; (6) Averaged GEM(A-GEM)

Original: [3]: constrains the new task learning with the gradient calculated using the stored data ofold tasks; (7) Experience Replay with Reservior sample (ER_Res) [4]: uses a small episodic memoryto store old task samples for addressing forgetting; (8) Multitask: jointly learns all tasks once with asingle network using all datasets, which usually serves as a performance upper bound in CL [25].
Revised: [3]: constrains the new task learning with the gradient calculated using the stored data ofold tasks; (7) Experience Replay with Reservior sample (ER_Res) [4]: uses a small episodic memoryto store old task samples for addressing forgetting; (8) Multitask: jointly learns all tasks once with asingle network using all datasets, which usually serves as a performance upper bound in CL [24].

Original: Network and training details.
Revised: Network and training details.

Original: For a given dataset, we study all CL methods using the same networkarchitecture.
Revised: For a given dataset, we study all CL methods using the same networkarchitecture.

Original: More specifically, for Permuted MNIST, we consider a 3-layer fully-connected networkincluding 2 hidden layers with 100 units.
Revised: More specifically, for Permuted MNIST, we consider a 3-layer fully-connected networkincluding 2 hidden layers with 100 units.

Original: And we train the network for 5 epochs with a batchsize of 10 for every task.
Revised: And we train the network for 5 epochs with a batchsize of 10 for every task.

Original: For Split CIFAR-100, we use a version of 5-layer AlexNet by following[25, 17].
Revised: For Split CIFAR-100, we use a version of 5-layer AlexNet by following[24, 16].

Original: When learning each task, we train the network for a maximum of 200 epochs with earlytermination based on the validation loss, and use a batch size of 64.
Revised: When learning each task, we train the network for a maximum of 200 epochs with earlytermination based on the validation loss, and use a batch size of 64.

Original: For 5-Datasets, we use a reduced
Revised: For 5-Datasets, we use a reduced

Original: ResNet-18 [19] and follow the same training strategy as in Split CIFAR-100.
Revised: ResNet-18 [18] and follow the same training strategy as in Split CIFAR-100.

Original: For Split MiniImageNet,a reduced ResNet-18 is also used, and we train the network for a maximum of 100 epoches with earlytermination.
Revised: For Split MiniImageNet,a reduced ResNet-18 is also used, and we train the network for a maximum of 100 epoches with earlytermination.

Original: The batch size is 64.
Revised: The batch size is 64.

Original: Similar to [17], we select at most two tasks to be in Regime 2 and3 for each layer with the largest gradient projection norm, to reduce the performance sensitivity onthe choice of ϵ 1 .
Revised: Similar to [16], we select at most two tasks to be in Regime 2 and3 for each layer with the largest gradient projection norm, to reduce the performance sensitivity onthe choice of ϵ 1 .

Original: In the experiments, we set ϵ 1 = 0 .
Revised: In the experiments, we set ϵ 1 = 0 .

Original: To evaluate the learning performance, we consider the following two metrics, i.e., accuracy (ACC) 274 which measures the final accuracy averaged over all tasks, and backward transfer (BWT) whichmeasures the average accuracy change of each task after learning new tasks:
Revised: To evaluate the learning performance, we consider the following two metrics, i.e., accuracy (ACC) 268 which measures the final accuracy averaged over all tasks, and backward transfer (BWT) whichmeasures the average accuracy change of each task after learning new tasks:

Original: [Equation]
Revised: [Equation]

Original: where A i,j represents the testing accuracy of task j after learning task i .
Revised: where A i,j represents the testing accuracy of task j after learning task i .

Original: 5.1 Main results
Revised: 5.1 Main results

Original: As shown in Table 1, CUBER demonstrates the best performance of BWT on all datasets comparedto the baselines.
Revised: As shown in Table 1, CUBER demonstrates the best performance of BWT on all datasets comparedto the baselines.

Original: In particular, positive BWT can be obtained by CUBER on Split CIFAR-100 and 
Revised: In particular, positive BWT can be obtained by CUBER on Split CIFAR-100 and Split

Original: Split MiniImageNet, which has not been achieved by previous works in a fixed capacity networkwithout data-replay .
Revised: MiniImageNet, which has not been achieved by previous works with a fixed capacity network .

Original: For 5-Dataset, since the tasks therein are less related to each other as in GPM[25], we do not expect much knowledge transfer across tasks (but knowledge transfer still exists interms of the layer-wise features), and there is no forgetting in CUBER.
Revised: For5-Dataset, since the tasks therein are less related to each other as in GPM [24], we do not expect muchknowledge transfer across tasks (but knowledge transfer still exists in terms of the layer-wise features),and there is no forgetting in CUBER.

Original: Achieving zero-forgetting isvery difficult for Permuted MNIST, because all the tasks share one output layer and there is no taskidentifier during testing (domain-incremental) [25].
Revised: Achieving zero-forgetting is very difficult for Permuted MNIST,because a single-head network is used to learn all the tasks, i.e., all the tasks share the same classifier.

Original: However, even in this case CUBER still achieves the best BWT, i.e, nearly non-forgetting, among all methods.
Revised: However, even in this case CUBER still achieves the best BWT, i.e, nearly non-forgetting, among allmethods.

Original: Clearly, the strong performance on BWT indicates that CUBER can effectively facilitate the backward knowledge transfer by wiselymodifying the learnt model of old tasks.
Revised: Clearly, the strong performance on BWT indicates that CUBER can effectively facilitatethe backward knowledge transfer by wisely modifying the learnt model of old tasks.

Original: Benefiting from the superior performance on the backward knowledge transfer, CUBER also achievesthe best or comparable performance on the averaged accuracy.
Revised: Benefiting from the superior performance on the backward knowledge transfer, CUBER also achievesthe best or comparable performance on the averaged accuracy.

Original: More specifically, CUBER improvesaround 1% in ACC over the best prior results on Split CIFAR-100, Split MiniImageNet and Permuted
Revised: More specifically, CUBER improvesaround 1% in ACC over the best prior results on Split CIFAR-100, Split MiniImageNet and Permuted

Original: MNIST, while showing the comparable performance with the state-of-the-art method TRGP evenon 5-datasets where tasks are less correlated.
Revised: MNIST, while showing the comparable performance with the state-of-the-art method TRGP evenon 5-datasets where tasks are less correlated.

Original: Moreover, CUBER performs better than Multitask onboth 5-Dataset and Permuted MNIST, which implies the importance of studying knowledge transferin CL: Both TRGP and CUBER outperform Multitask on 5-Dataset because of the scaled weightprojection to facilitate forward knowledge transfer, whereas by facilitating backward knowledge
Revised: Moreover, CUBER performs better than Multitask onboth 5-Dataset and Permuted MNIST, which implies the importance of studying knowledge transferin CL: Both TRGP and CUBER outperform Multitask on 5-Dataset because of the scaled weightprojection to facilitate forward knowledge transfer, whereas by facilitating backward knowledge

Original: CUBER becomes the only method that outperforms Multitask on Permuted MNIST. 
Revised: CUBER becomes the only method that outperforms Multitask on Permuted MNIST. Permuted MNIST.

Original: [Table]
Revised: Moreover, ϵ 2 = 0 .

Original: [Table]
Revised: [Table]

Original: 5.2 Ablation studies
Revised: 5.2 Ablation studies

Original: [Table]
Revised: [Table]

Original: Backward knowledge transfer.
Revised: Backward knowledge transfer.

Original: The value of backward knowledge transfer indicates the averageaccuracy improvement for each old task after learning all tasks, which implies that the new tasklearning provides additional useful information for learning features of similar old tasks.

Original: Intuitively,this value should depend on the task similarity in CL.

Original: To better understand the advantage of CUBERin terms of backward knowledge transfer, we further consider a special setup which includes asequence of similar and dissimilar tasks.
Revised: To better understand the advantage of CUBER in terms of backwardknowledge transfer [Table] we further consider a special setup which includes a sequence of similar anddissimilar tasks.

Original: Specifically, different with Split-CIFAR100 where no taskshave overlapping classes, we split the first 50 classes in CIFAR100 into 7 tasks (OL-CIFAR100): Task 0-6 contain classes 0-9, 5-14, 10-19, 20-29, 25-34, 30-39, 40-49, respectively.
Revised: Specifically, different with Split-CIFAR100 where no tasks have overlapping classes,we split the first 50 classes in CIFAR100 into 7 tasks (OL-CIFAR100): Task 0-6 contain classes0-9, 5-14, 10-19, 20-29, 25-34, 30-39, 40-49, respectively.

Original: We compare theperformance of CUBER with TRGP in Table 2.
Revised: We compare the performance of CUBERwith TRGP in Table 2.

Original: As shown in Table 2a, CUBER clearly outperforms TRGP in both ACC and BWT.
Revised: As shown in Table 2a, CUBER clearly outperforms TRGP in both ACC and BWT.

Original: In particular, CUBER has a positive BWT of 0 .
Revised: In particular, CUBER has a positive BWT of 0 .

Original: 28% , where TRGPsuffers from forgetting.
Revised: 28% , where TRGP suffers from forgetting.

Original: We further analyze the task selections in Table 2b, where “selected old task”refers to the task that has the largest number of layers in Regime 3 of the new task.
Revised: We further analyze the task selections in Table 2b, where “selected old task” refers to the task thathas the largest number of layers in Regime 3 of the new task.

Original: For example,according to the setup of OL-CIFAR100, the selected old task for Task 4 should be Task 3 with ahigh probability as they have overlapping classes.
Revised: For example, according to the setupof OL-CIFAR100, the selected old task for Task 4 should be Task 3 with a high probability as they have overlapping classes.

Original: And we denote a new metric, namely BWT-S, toevaluate the backward knowledge transfer of the selected old task after learning the new task, i.e., [Equation] ,j where j is the “selected old task” of the new task t .
Revised: And we denote a new metric, namely BWT-S, to evaluate the backwardknowledge transfer of the selected old task after learning the new task, i.e., BWT-S= A t,j − A t − 1 ,jwhere j is the “selected old task” of the new task t .

Original: As shown in Table 2b, CUBER correctly identifies the correlated tasks for most new tasks except Task 5.
Revised: As shown in Table 2b, CUBER correctlyidentifies the correlated tasks for most new tasks except Task 5.

Original: This is a reasonableresult because the task correlation detection is based on the initial model gradient which is noisyin general, and hence only serves as an estimation of underlying true correlation.
Revised: This is a reasonable result becausethe task correlation detection is based on the initial model gradient which is noisy in general, andhence only serves as an estimation of underlying true correlation.

Original: However, suchan estimated task correlation characterization is indeed sufficient to effectively facilitate backwardknowledge transfer, as corroborated by the positive BWT-S and the superior performance of CUBER.
Revised: However, such an estimated taskcorrelation characterization is indeed sufficient to effectively facilitate backward knowledge transfer,as corroborated by the positive BWT-S and the superior performance of CUBER.

Original: Forward knowledge transfer.
Revised: Forward knowledge transfer.

Original: We also evaluate the forward knowledge transfer (FWT) in CUBER,compared to the best two baseline methods GPM and TRGP.
Revised: We also evaluate the forward knowledge transfer (FWT) in CUBER,compared to the best two baseline methods GPM and TRGP.

Original: Here the FWT measures the gap between
Revised: Here the FWT measures the gap between

Original: A i,i and the accuracy of learning task i only from scratch.
Revised: A i,i and the accuracy of learning task i only from scratch.

Original: For simplicity, we use the FWT of GPM asa baseline and evaluate the improvements of TRGP and CUBER over GPM.
Revised: For simplicity, we use the FWT of GPM asa baseline and evaluate the improvements of TRGP and CUBER over GPM.

Original: It can be seen from Table3 that CUBER performs even better than TRGP although CUBER follows the same strategy, i.e., thescaled weight projection, to facilitate the forward knowledge transfer, and achieves the best FWTamong the three methods in most cases.
Revised: It can be seen from Table3 that CUBER performs even better than TRGP although CUBER follows the same strategy, i.e., thescaled weight projection, to facilitate the forward knowledge transfer, and achieves the best FWTamong the three methods in most cases.

Original: The reason behind is that the characterization of Regime 3 in
Revised: The reason behind is that the characterization of Regime 3 in

Original: CUBER not only allows the modification of the learnt model of the old tasks to prompt the backwardknowledge transfer, but also relaxes the constraint on the gradient update for the new task, i.e., themodel can be now updated in the subspace of the selected old tasks for learning the new task.
Revised: CUBER not only allows the modification of the learnt model of the old tasks to prompt the backwardknowledge transfer, but also relaxes the constraint on the gradient update for the new task, i.e., themodel can be now updated in the subspace of the selected old tasks for learning the new task.

Original: Thisgradient constraint relaxation consequently leads to better model learning of the new task.
Revised: Thisgradient constraint relaxation consequently leads to better model learning of the new task.

Original: [Table]
Revised: [Table]

Original: Impact of ϵ 2 .
Revised: Impact of ϵ 2 .

Original: It is clear that the selection of layer-wise Regime 3 depends on the value of thethreshold ϵ 2 .
Revised: It is clear that the selection of layer-wise Regime 3 depends on the value of thethreshold ϵ 2 .

Original: To show the impact of ϵ 2 , we evaluate the learning performance under different valuesof ϵ 2 in Split CIFAR-100.
Revised: To show the impact of ϵ 2 , we evaluate the learning performance under different valuesof ϵ 2 in Split CIFAR-100.

Original: As shown in Table 4, the performance on ACC is comparable for allthree cases and the BWT decreases as the value of ϵ 2 increases.
Revised: As shown in Table 4, the performance on ACC is comparable for allthree cases and the BWT decreases as the value of ϵ 2 increases.

Original: Intuitively, ϵ 2 characterizes theconservatism in selecting tasks to Regime 3 and modifying the model of selected tasks.
Revised: Intuitively, ϵ 2 characterizes theconservatism in selecting tasks to Regime 3 and modifying the model of selected tasks.

Original: Specifically,with a larger ϵ 2 , we just consider the backward knowledge transfer to the old tasks that are stronglycorrelated with the new task, and only slightly modify the learnt model of these selected tasks,because the condition Eq.
Revised: Specifically,with a larger ϵ 2 , we just consider the backward knowledge transfer to the old tasks that are stronglycorrelated with the new task, and only slightly modify the learnt model of these selected tasks,because the condition Eq.

Original: (2) can be quickly violated with the model update and CUBER will stopchanging the learnt model of the selected tasks.
Revised: (2) can be quickly violated with the model update and CUBER will stopchanging the learnt model of the selected tasks.

Original: [Table]
Revised: [Table]

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we study the problem of backward knowledge transfer in CL.
Revised: In this work, we study the problem of backward knowledge transfer in CL.

Original: Different from mostexisting methods that generally freeze the learnt model of the old tasks so as to mitigate catastrophicforgetting, this study seeks to carefully modify the learnt model to facilitate backward knowledgetransfer from the new task to the old tasks.
Revised: Different from mostexisting methods that generally freeze the learnt model of the old tasks so as to mitigate catastrophicforgetting, this study seeks to carefully modify the learnt model to facilitate backward knowledgetransfer from the new task to the old tasks.

Original: To this end, we first introduce notions of strong projectionand positive correlation to characterize the task correlation, and show that when the task gradientsare sufficiently aligned in the old task subspace, appropriate model change for the old tasks could bebeneficial for CL and result in better backward knowledge transfer.
Revised: To this end, we first introduce notions of strong projectionand positive correlation to characterize the task correlation, and show that when the task gradientsare sufficiently aligned in the old task subspace, appropriate model change for the old tasks could bebeneficial for CL and result in better backward knowledge transfer.

Original: Based on the theoretical analysis,we next propose CUBER to carefully learn the model for the new task, which would carefully updatethe learnt model of the old tasks that are positively correlated with the new task.
Revised: Based on the theoretical analysis,we next propose CUBER to carefully learn the model for the new task, which would carefully updatethe learnt model of the old tasks that are positively correlated with the new task.

Original: As shown in theexperimental results, CUBER can successfully improve the backward knowledge transfer on thestandard CL benchmarks in contrast to related baselines.
Revised: As shown in theexperimental results, CUBER can successfully improve the backward knowledge transfer on thestandard CL benchmarks in contrast to related baselines.

Original: Impact and limitations.

Original: The mainstream strategy nowadays to address forgetting is to minimize theinterference to old tasks and avoid the learnt model change, which may however conflict with thegoal of CL, in the sense that the backward knowledge transfer from the new task to old tasks can berestricted without modifying the learnt model.

Original: In this work, we go beyond this strategy and shed lighton the relationship between backward knowledge transfer and model modification, by characterizingthe task correlations with gradient projection.

Original: We hope that this work will serve as initial steps andmotivate further research in CL community on the important while less explored problem, i.e., howto design algorithms that can provide targeted treatments to achieve backward knowledge transfer.

Original: However, CUBER also comes with several limitations.

Original: As in recent orthogonal-projection based

Original: CL methods, CUBER extracts the bases of the task subspaces based on SVD, which may lead tohigh computational cost for large dimensional data.

Original: How to reduce the complexity is an interestingdirection.

Original: Another limitation is that we assume that clear task boundaries exist between differenttasks.

Original: In future work, it is of great interests to extend CUBER to more general CL settings.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.
Revised: [No] The code and the data are

Original: • Did you include the license to the code and datasets?
Revised: proprietary.• Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 6.

Original: (c) Did you discuss any potential negative societal impacts of your work?[Yes] See
Revised: [No](c) Did you discuss any potential negative societal impacts of your work?

Original: Section 6. (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [No](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Theorem 1and 2.(b)
Revised: [Yes] See Theorem 1and 2.(b)

Original: Did you include complete proofs of all theoretical results?
Revised: Did you include complete proofs of all theoretical results?

Original: [Yes] See the appendix.3.
Revised: [Yes] See the appendix.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] We include the code in the supplemental material.
Revised: [Yes] We include thecode in the supplemental material.

Original: (b) Did you specify all the training details (e
Revised: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how they were chosen)?
Revised: g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] See section 5.
Revised: [Yes] See section 5.

Original: (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] See section 5.
Revised: [Yes] See section 5.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] See the appendix.
Revised: [Yes] See the appendix.

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See section 5.
Revised: [Yes] See section 5.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See the appendix.
Revised: [Yes] See the appendix.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]We include code in the supplemental material.
Revised: [Yes] We include code in the supplemental material.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]