Original: C ONTINUAL L EARNING WITH A DAPTIVE W EIGHTS (CLAW)
Revised: C ONTINUAL L EARNING WITH A DAPTIVE W EIGHTS (CLAW)

Original: A BSTRACT
Revised: A BSTRACT

Original: Approaches to continual learning aim to successfully learn a set of related tasks that arrive in an online manner.
Revised: Approaches to continual learning aim to successfully learn a set of related tasks that arrive in an online manner.

Original: Recently, several frameworks have been developed which enable deep learning to be deployed in this learning scenario.
Revised: Recently, several frameworks have been developed which enable deep learning to be deployed in this learning scenario.

Original: A key modelling decision is to what extent the architecture should be shared across tasks.
Revised: A key modelling decision is to what extent the architecture should be shared across tasks.

Original: On the one hand, separately modelling each task avoids catastrophic forgetting but it does not support transfer learning and leads to large models.
Revised: On the one hand, separately modelling each task avoids catastrophic forgetting but it does not support transfer learning and leads to large models.

Original: On the other hand, rigidly specifying a shared component and a task-speciﬁc part enables task transfer and limits the model size, but it is vulnerable to catastrophic forgetting and restricts the form of task-transfer that can occur.
Revised: On the other hand, rigidly specifying a shared component and a task-speciﬁc part enables task transfer and limits the model size, but it is vulnerable to catastrophic forgetting and restricts the form of task-transfer that can occur.

Original: Ideally, the network should adaptively identify which parts of the network to share in a data driven way.
Revised: Ideally, the network should adaptively identify which parts of the network to share in a data driven way.

Original: Here we introduce such an approach called Continual Learning with Adaptive Weights (CLAW), which is based on probabilistic modelling and variational inference.
Revised: Here we introduce such an approach called Continual Learning with Adaptive Weights (CLAW), which is based on probabilistic modelling and variational inference.

Original: Experiments show that CLAW achieves state-of-the-art performance on six benchmarks in terms of overall continual learning performance, as measured by classiﬁcation accuracy, and in terms of addressing catastrophic forgetting.
Revised: Experiments show that CLAW achieves state-of-the-art performance on six benchmarks in terms of overall continual learning performance, as measured by classiﬁcation accuracy, and in terms of addressing catastrophic forgetting.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Continual learning (CL), sometimes called lifelong or incremental learning, refers to an online framework where the knowledge acquired from learning tasks in the past is kept and accumulated so that it can be reused in the present and future.
Revised: Continual learning (CL), sometimes called lifelong or incremental learning, refers to an online framework where the knowledge acquired from learning tasks in the past is kept and accumulated so that it can be reused in the present and future.

Original: Data belonging to different tasks could potentially be non i.i.d.
Revised: Data belonging to different tasks could potentially be non i.i.d.

Original: (Schlimmer & Fisher, 1986; Sutton & Whitehead, 1993; Ring, 1997; Schmidhuber, 2013; Nguyen et al., 2018; Schmidhuber, 2018).
Revised: (Schlimmer & Fisher, 1986; Sutton & Whitehead, 1993; Ring, 1997; Schmidhuber, 2013; Nguyen et al., 2018; Schmidhuber, 2018).

Original: A continual learner must be able to learn a new task, crucially, without forgetting previous tasks (Ring, 1995; Srivastava et al., 2013; Schwarz et al., 2018; Serra et al., 2018; Hu et al., 2019).
Revised: A continual learner must be able to learn a new task, crucially, without forgetting previous tasks (Ring, 1995; Srivastava et al., 2013; Schwarz et al., 2018; Serra et al., 2018; Hu et al., 2019).

Original: In addition, CL frameworks should continually adapt to any domain shift occurring across tasks.
Revised: In addition, CL frameworks should continually adapt to any domain shift occurring across tasks.

Original: The learning updates must be incremental – i.e, the model is updated at each task only using the new data and the old model, without access to all previous data (from earlier tasks) – due to speed, security and privacy constraints.
Revised: The learning updates must be incremental – i.e, the model is updated at each task only using the new data and the old model, without access to all previous data (from earlier tasks) – due to speed, security and privacy constraints.

Original: A compromise must be found between adapting to new tasks and enforcing stability to preserve knowledge from previous tasks.
Revised: A compromise must be found between adapting to new tasks and enforcing stability to preserve knowledge from previous tasks.

Original: Excessive adaptation could lead to inadvertent forgetting of how to perform earlier tasks.
Revised: Excessive adaptation could lead to inadvertent forgetting of how to perform earlier tasks.

Original: Indeed, catastrophic forgetting is one of the main pathologies in continual learning (McCloskey & Cohen, 1989; Ratcliff, 1990; Robins, 1993; 1995; French, 1999; Pape et al., 2011; Goodfellow et al., 2014a;
Revised: Indeed, catastrophic forgetting is one of the main pathologies in continual learning (McCloskey & Cohen, 1989; Ratcliff, 1990; Robins, 1993; 1995; French, 1999; Pape et al., 2011; Goodfellow et al., 2014a;

Original: Achille et al., 2018; Kemker et al., 2018; Kemker & Kanan, 2018; Diaz-Rodriguez et al., 2018; Zeno et al., 2018;
Revised: Achille et al., 2018; Kemker et al., 2018; Kemker & Kanan, 2018; Diaz-Rodriguez et al., 2018; Zeno et al., 2018;

Original: Ahn et al., 2019; Parisi et al., 2019; Pfulb & Gepperth, 2019; Rajasegaran et al., 2019).
Revised: Ahn et al., 2019; Parisi et al., 2019; Pfulb & Gepperth, 2019; Rajasegaran et al., 2019).

Original: Many approaches to continual learning employ an architecture which is divided a priori into (i) a slowly evolving, global part; and (ii) a quickly evolving, task-speciﬁc, local part.
Revised: Many approaches to continual learning employ an architecture which is divided a priori into (i) a slowly evolving, global part; and (ii) a quickly evolving, task-speciﬁc, local part.

Original: This is one way to enable multi-task transfer whilst mitigating catastrophic forgetting, which has proven to be effective (Rusu et al., 2016b; Fernando et al., 2017; Yoon et al., 2018), albeit with limitations.
Revised: This is one way to enable multi-task transfer whilst mitigating catastrophic forgetting, which has proven to be effective (Rusu et al., 2016b; Fernando et al., 2017; Yoon et al., 2018), albeit with limitations.

Original: Specifying a priori the shared global, and task-speciﬁc local parts in the architecture restricts ﬂexibility.
Revised: Specifying a priori the shared global, and task-speciﬁc local parts in the architecture restricts ﬂexibility.

Original: As more complex and heterogeneous tasks are considered, one would like a more ﬂexible, data-driven approach to determine the appropriate amount of sharing across tasks.
Revised: As more complex and heterogeneous tasks are considered, one would like a more ﬂexible, data-driven approach to determine the appropriate amount of sharing across tasks.

Original: Here, we aim at automating the architecture adaptation process so that each neuron of the network can either be kept intact, i.e. acting as global, or adapted to the new task locally.
Revised: Here, we aim at automating the architecture adaptation process so that each neuron of the network can either be kept intact, i.e. acting as global, or adapted to the new task locally.

Original: Our proposed variational inference framework is ﬂexible enough to learn the range within which the adaptation parameters can vary.
Revised: Our proposed variational inference framework is ﬂexible enough to learn the range within which the adaptation parameters can vary.

Original: We introduce for each neuron one binary parameter controlling whether or not to adapt, and two parameters to control the magnitude of adaptation.
Revised: We introduce for each neuron one binary parameter controlling whether or not to adapt, and two parameters to control the magnitude of adaptation.

Original: All parameters are learnt via variational inference.
Revised: All parameters are learnt via variational inference.

Original: We introduce our framework as an expansion of the variational continual learning algorithm
Revised: We introduce our framework as an expansion of the variational continual learning algorithm

Original: (Nguyen et al., 2018), whose variational and sequential Bayesian nature makes it convenient for our modelling and architecture adaptation procedure.
Revised: (Nguyen et al., 2018), whose variational and sequential Bayesian nature makes it convenient for our modelling and architecture adaptation procedure.

Original: Our modelling ideas can also be applied to other continual learning frameworks, see the Appendix for a brief discussion.
Revised: Our modelling ideas can also be applied to other continual learning frameworks, see the Appendix for a brief discussion.

Original: We highlight the following contributions: (1) A modelling framework which ﬂexibly automates the adaptation of local and global parts of the (multi-task) continual architecture.
Revised: We highlight the following contributions: (1) A modelling framework which ﬂexibly automates the adaptation of local and global parts of the (multi-task) continual architecture.

Original: This optimizes the tradeoff between mitigating catastrophic forgetting and improving task transfer.
Revised: This optimizes the tradeoff between mitigating catastrophic forgetting and improving task transfer.

Original: A probabilistic variational inference algorithm which supports incremental updates with adaptively learned parameters.
Revised: A probabilistic variational inference algorithm which supports incremental updates with adaptively learned parameters.

Original: The ability to combine our modelling and inference approaches without any signiﬁcant augmentation of the architecture (no new neurons are needed).
Revised: The ability to combine our modelling and inference approaches without any signiﬁcant augmentation of the architecture (no new neurons are needed).

Original: (4) State-of-the-art results in six experiments on ﬁve datasets, which demonstrate the effectiveness of our framework in terms of overall accuracy and reducing catastrophic forgetting.
Revised: (4) State-of-the-art results in six experiments on ﬁve datasets, which demonstrate the effectiveness of our framework in terms of overall accuracy and reducing catastrophic forgetting.

Original: 1. R ELATED W ORK

Original: We brieﬂy discuss three related approaches to continual learning: (a) regularisation-based, (b) architecture-based and (c) memory-based.

Original: We provide more details of related work in Section A in the Appendix.

Original: (a) A complementary approach to CLAW is the regularisation-based approach to balance adaptability with catastrophic forgetting: a level of stability is kept via protecting parameters that greatly inﬂuence the prediction against radical changes, while allowing the rest of the parameters to change without restriction (Li & Hoiem, 2016; Lee et al., 2017; Zenke et al., 2017; Chaudhry et al., 2018;

Original: Kim et al., 2018; Nguyen et al., 2018; Srivastava et al., 2013; Schwarz et al., 2018;

Original: Vuorio et al., 2018; Aljundi et al., 2019c).

Original: The elastic weight consolidation (EWC) algorithm by

Original: Kirkpatrick et al.

Original: (2017) is a seminal example, where a quadratic penalty is imposed on the difference between parameter values of the old and new tasks.

Original: One limitation is the high level of hand tuning required.

Original: (b) The architecture-based approach aims to deal with stability and adaptation issues by a ﬁxed division of the architecture into global and local parts (Rusu et al., 2016b;

Original: Fernando et al., 2017;

Original: Shin et al., 2017; Kaplanis et al., 2018; Xu & Zhu, 2018; Yoon et al., 2018; Li et al., 2019b).

Original: (c)

Original: The memory-based approach relies on episodic memory to store data (or pseudo-data) from previous tasks (Ratcliff, 1990; Robins, 1993; 1995; Thrun, 1996; Schmidhuber, 2013; Hattori, 2014; Mocanu et al., 2016; Rebufﬁ et al., 2017; Kamra et al., 2017; Shin et al., 2017; Rolnick et al., 2018; van de

Original: Ven & Tolias, 2018; Wu et al., 2018; Titsias et al., 2019).

Original: Limitations include overheads for tasks such as data storage, replay, and optimisation to select (or generate) the points.

Original: CLAW can as well be seen as a combination of a regularisation-based approach (the variational inference mechanism) and a modelling approach which automates the architecture building process in a data-driven manner, avoiding the overhead resulting from either storing or generating data points from previous tasks.

Original: CLAW is also orthogonal to (and simple to combine with, if needed) memory-based methods.

Original:  B ACKGROUND ON V ARIATIONAL C ONTINUAL L EARNING (VCL)
Revised:  B ACKGROUND ON V ARIATIONAL C ONTINUAL L EARNING (VCL)

Original: In this paper, we use Variational Continual Learning (VCL, Nguyen et al., 2018) as the underlying continual learning framework.
Revised: In this paper, we use Variational Continual Learning (VCL, Nguyen et al., 2018) as the underlying continual learning framework.

Original: However, our methods apply to other frameworks, see Appendix (Section A.1).
Revised: However, our methods apply to other frameworks, see Appendix (Section A.1).

Original: VCL is a variational Bayesian framework where the posterior of the model parameters θ is learnt and updated continually from a sequence of T datasets, { x ( n ) t , y ( n ) t } N t n =1 , where t = 1 , 2 , . . .
Revised: VCL is a variational Bayesian framework where the posterior of the model parameters θ is learnt and updated continually from a sequence of T datasets, { x ( n ) t , y ( n ) t } N t n =1 , where t = 1 , 2 , . . .

Original: , T and N t is the size of the dataset associated with the t -th task.
Revised: , T and N t is the size of the dataset associated with the t -th task.

Original: More speciﬁcally, denote by p ( y | θ , x ) the probability distribution returned by a discriminative classiﬁer with input x , output y and parameters θ .
Revised: More speciﬁcally, denote by p ( y | θ , x ) the probability distribution returned by a discriminative classiﬁer with input x , output y and parameters θ .

Original: For D t = { y ( n ) t } N t n =1 , we approximate the intractable posterior p ( θ |D 1: t ) after observing the ﬁrst t datasets via a tractable variational distribution q t as:
Revised: For D t = { y ( n ) t } N t n =1 , we approximate the intractable posterior p ( θ |D 1: t ) after observing the ﬁrst t datasets via a tractable variational distribution q t as:

Original: [Equation]
Revised: [Equation]

Original: where q 0 is the prior p , p ( D t | θ )
Revised: where q 0 is the prior p , p ( D t | θ )

Original: = (cid:81)
Revised: = (cid:81)

Original: N t n =1 p ( y ( n ) t | θ , x ( n ) t ) , and Z t is the normalizing constant which does not depend on θ but only on the data D .
Revised: N t n =1 p ( y ( n ) t | θ , x ( n ) t ) , and Z t is the normalizing constant which does not depend on θ but only on the data D .

Original: This framework allows the approximate posterior q t ( θ ) to be updated incrementally from the previous approximate posterior q t − 1 ( θ ) in an online fashion.
Revised: This framework allows the approximate posterior q t ( θ ) to be updated incrementally from the previous approximate posterior q t − 1 ( θ ) in an online fashion.

Original: In VCL, the approximation in (1) is performed by minimizing the following KL-divergence over a family Q of tractable distributions: 
Revised: In VCL, the approximation in (1) is performed by minimizing the following KL-divergence over a family Q of tractable distributions: [Equation]

Original: [Equation]
Revised: [Equation]

Original: This framework can be enhanced to further mitigate catastrophic forgetting by using a coreset (Nguyen et al., 2018), i.e. a representative set of data from previously observed tasks that can serve as memory and can be revisited before making a decision.
Revised: This framework can be enhanced to further mitigate catastrophic forgetting by using a coreset (Nguyen et al., 2018), i.e. a representative set of data from previously observed tasks that can serve as memory and can be revisited before making a decision.

Original: As discussed in the Related Work, this leads to overhead costs of memory and optimisation (selecting most representative data points).
Revised: As discussed in the Related Work, this leads to overhead costs of memory and optimisation (selecting most representative data points).

Original: Previous work on VCL considered simple models without automatic architecture building or adaptation.
Revised: Previous work on VCL considered simple models without automatic architecture building or adaptation.

Original:  O UR CLAW A PPROACH
Revised:  O UR CLAW A PPROACH

Original: In earlier CL approaches, the parts of the network architecture that are shared among the learnt tasks are designated a priori .
Revised: In earlier CL approaches, the parts of the network architecture that are shared among the learnt tasks are designated a priori .

Original: To alleviate this rigidity and to effectively balance adaptation and stability, we propose a multi-task, continual model in which the adaptation of the architecture is data-driven by learning which neurons need to be adapted as well as the maximum adaptation capacity for each.
Revised: To alleviate this rigidity and to effectively balance adaptation and stability, we propose a multi-task, continual model in which the adaptation of the architecture is data-driven by learning which neurons need to be adapted as well as the maximum adaptation capacity for each.

Original: All the model parameters (including those used for adaptation) are estimated via an efﬁcient variational inference algorithm which incrementally learns from data of the successive tasks, without a need to store (nor generate) data from previous tasks and with no expansion in the network size.
Revised: All the model parameters (including those used for adaptation) are estimated via an efﬁcient variational inference algorithm which incrementally learns from data of the successive tasks, without a need to store (nor generate) data from previous tasks and with no expansion in the network size.

Original: 3. M ODELLING
Revised: 3. M ODELLING

Original: With model parameters θ , the overall variational objective we aim at maximising at task with index t is equivalent to the following online marginal likelihood:
Revised: With model parameters θ , the overall variational objective we aim at maximising at task with index t is equivalent to the following online marginal likelihood:

Original: [Equation]
Revised: [Equation]

Original: We propose a framework where the architecture, whose parameters are θ , is ﬂexibly adapted based on the available tasks, via a learning procedure that will be described below.
Revised: We propose a framework where the architecture, whose parameters are θ , is ﬂexibly adapted based on the available tasks, via a learning procedure that will be described below.

Original: With each task, we automate the adaptation of the neuron contributions.
Revised: With each task, we automate the adaptation of the neuron contributions.

Original: Both the adaptation decisions (i.e. whether or not to adapt) and the maximum allowed degree of adaptation for every neuron are learnt.
Revised: Both the adaptation decisions (i.e. whether or not to adapt) and the maximum allowed degree of adaptation for every neuron are learnt.

Original: We refer to the binary adaptation variable as α .
Revised: We refer to the binary adaptation variable as α .

Original: There is another variable s that is learnt in a multi-task fashion to control the maximum degree of adaptation, such that the expression b = s 1+e − a − 1 limits how far the task-speciﬁc weights can differ from the global weights, in case the respective neuron is to be adapted.
Revised: There is another variable s that is learnt in a multi-task fashion to control the maximum degree of adaptation, such that the expression b = s 1+e − a − 1 limits how far the task-speciﬁc weights can differ from the global weights, in case the respective neuron is to be adapted.

Original: The parameter a depicts unconstrained adaptation, as described later.
Revised: The parameter a depicts unconstrained adaptation, as described later.

Original: We illustrate the proposed model to perform this adaptation by learning the probabilistic contributions of the different neurons within the network architecture on a task-by-task basis.
Revised: We illustrate the proposed model to perform this adaptation by learning the probabilistic contributions of the different neurons within the network architecture on a task-by-task basis.

Original: We follow this with the inference details.
Revised: We follow this with the inference details.

Original: Steps of the proposed modeling are listed as follows:
Revised: Steps of the proposed modeling are listed as follows:

Original: • For a task T , the classiﬁer that we are modeling outputs: (cid:80) N T n =1 (cid:2) log p ( y ( n ) | x ( n ) , w T ) (cid:3) .
Revised: • For a task T , the classiﬁer that we are modeling outputs: (cid:80) N T n =1 (cid:2) log p ( y ( n ) | x ( n ) , w T ) (cid:3) .

Original: • The task-speciﬁc weights w T can be expressed in terms of their global counterparts as follows:
Revised: • The task-speciﬁc weights w T can be expressed in terms of their global counterparts as follows:

Original: [Equation]
Revised: [Equation]

Original: The symbol ◦ denotes an element-wise (Hadamard) multiplication.
Revised: The symbol ◦ denotes an element-wise (Hadamard) multiplication.

Original: • For each task T and each neuron j at layer i , α T i , j is a binary variable which indicates whether the corresponding weight is adapted ( α T i , j = 1 ) or unadapted ( α T i , j = 0 ).
Revised: • For each task T and each neuron j at layer i , α T i , j is a binary variable which indicates whether the corresponding weight is adapted ( α T i , j = 1 ) or unadapted ( α T i , j = 0 ).

Original: Initially assume that the adaptation probability α T i , j follows a Bernoulli distribution with probability p 
Revised: Initially assume that the adaptation probability α T i , j follows a Bernoulli distribution with probability p i , j 3 , α T i , j ∼ Bernoulli ( p i , j ) . Since this

Original: i , j 3 , α T i , j ∼ Bernoulli ( p
Revised: Bernoulli is not straightforward to

Original: i j )
Revised: optimise , and to adopt a scalable inference procedure based on

Original: Since this Bernoulli is not straightforward to optimise and to adopt a scalable inference procedure based on continuous latent variables  we replace this Bernoulli with a Gaussian that has an equivalent mean and variance from which wedraw α T i , j .
Revised: continuous latent variables , we replace this Bernoulli with a Gaussian that has an equivalent mean and variance from which we

Revised: draw α T i , j

Original: For the sake of attaining higher ﬁdelity than what is granted by a standard Gaussian  we base our inference ona variational
Revised: For the sake of attaining higher ﬁdelity than what is granted by a standard Gaussian , we base our inference on

Revised: a variational

Original: Gaussian estimation.
Revised: Gaussian estimation.

Revised: Though

Revised: in

Original: Though in a context different from continual learning and with different estimators, the idea 
Revised: a context different from continual learning and with different estimators, the idea of replacing Bernoulli

Original: of replacing Bernoulli with an equivalent Gaussian has proven to be
Revised: with an equivalent Gaussian has proven to be

Original: effective with dropout (Srivastava et [-al., 
-]
Revised: effective with dropout Srivastava et [-al., 
-]al.

Original: ; Kingma et al., 2015).
Revised: ; Kingma et al.

Original: The approximation of the Bernoulli distribution by the corresponding Gaussian distribution is achieved by matching the mean and variance.
Revised: The approximation of the Bernoulli distribution by the corresponding Gaussian distribution is achieved by matching the mean and variance.

Original: The mean and variance of the Bernoulli distribution are p i , j , p i , j (1 − p i , j ) , respectively.
Revised: The mean and variance of the Bernoulli distribution are p i , j , p i , j (1 − p i , j ) , respectively.

Original: A Gaussian distribution with the same mean and variance is
Revised: A Gaussian distribution with the same mean and variance is

Original: used to ﬁt α T i , j .
Revised: used to ﬁt α T i , j .

Original: [Equation]
Revised: [Equation]

Original: • The variable b T controls the strength of the adaptation and it limits the range of adaptation via:
Revised: • The variable b T controls the strength of the adaptation and it limits the range of adaptation via:

Original: [Equation]
Revised: [Equation]

Original: So that the maximum adaptation is s .
Revised: So that the maximum adaptation is s .

Original: The variable a T is an unconstrained adaptation value, similar to that in (Swietojanski & Renals, 2014).
Revised: The variable a T is an unconstrained adaptation value, similar to that in (Swietojanski & Renals, 2014).

Original: The addition of 1 is to facilitate the usage of a probability distribution while still keeping an adaptation range allowing for the attenuation or ampliﬁcation of each neuron’s contribution.
Revised: The addition of 1 is to facilitate the usage of a probability distribution while still keeping an adaptation range allowing for the attenuation or ampliﬁcation of each neuron’s contribution.

Revised: There can be a parameter p i per layer i instead of one p i , j for each neuron j at each layer i , but we opt for the latter for the sake of gaining further adaptation ﬂexibility.

Original: • Before facing the ﬁrst dataset and learning task t =1 ,
Revised: • Before facing the ﬁrst dataset and learning task t =

Original: the prior on the weights q 0 ( w ) = p ( w ) is chosen to be a log-scale prior, which can be expressed as: p (log | w | )
Revised: the prior on the weights q 0 ( w ) = p ( w ) is chosen to be a log-scale prior, which can be expressed as: p (log | w | )

Original: ∝ c , where c is a constant.
Revised: ∝ c , where c is a constant.

Original: The log-scale prior can alternatively be described as:
Revised: The log-scale prior can alternatively be described as:

Original: [Equation]
Revised: [Equation]

Original: At a high level, adapting neuron contributions can be seen as a generalisation of attention mechanisms in the context of continual learning.
Revised: At a high level, adapting neuron contributions can be seen as a generalisation of attention mechanisms in the context of continual learning.

Original: Applying this adaptation procedure to the input leads to an attention mechanism.
Revised: Applying this adaptation procedure to the input leads to an attention mechanism.

Original: However, our approach is more general since we do not apply it only to the very bottom (i.e. input) layer, but throughout the whole network.
Revised: However, our approach is more general since we do not apply it only to the very bottom (i.e. input) layer, but throughout the whole network.

Original: We next show how our variational inference mechanism enables us to learn the adaptation parameters.
Revised: We next show how our variational inference mechanism enables us to learn the adaptation parameters.

Original: 3. I NFERENCE
Revised: 3. I NFERENCE

Original: We describe the details related to the proposed variational inference mechanism.
Revised: We describe the details related to the proposed variational inference mechanism.

Original: The adaptation parameters are included within the variational parameters.
Revised: The adaptation parameters are included within the variational parameters.

Original: The (unadapted version of the) model parameters θ consist of the weight vectors w .
Revised: The (unadapted version of the) model parameters θ consist of the weight vectors w .

Original: To automate adaptation, we perform inference on p i , j , which would have otherwise been a hyperparameter of the prior (Louizos et al., 2017; Molchanov et al., 2017; Ghosh et al., 2018).
Revised: To automate adaptation, we perform inference on p i , j , which would have otherwise been a hyperparameter of the prior (Louizos et al., 2017; Molchanov et al., 2017; Ghosh et al., 2018).

Original: Multiplying w by (1 + b α ) where α is distributed according to (5), then from (4) with random noise variable (cid:15) ∼ N (0 , 1) :
Revised: Multiplying w by (1 + b α ) where α is distributed according to (5), then from (4) with random noise variable (cid:15) ∼ N (0 , 1) :

Original: [Equation]
Revised: [Equation]

Original: From (7) and (8), the corresponding KL-divergence between the variational posterior of w , q ( w | γ ) and the prior p ( w ) is as follows.
Revised: From (7) and (8), the corresponding KL-divergence between the variational posterior of w , q ( w | γ ) and the prior p ( w ) is as follows.

Original: The subscripts are removed when q in turn is used as a subscript 
Revised: The subscripts are removed when q in turn is used as a subscript for improved readability.

Revised: The variational parameters are γ i , j and p i , j .

Original: [Equation]
Revised: [Equation]

Original: where the switch from (9) to (10) is due to the entropy computation (Bernardo & Smith, 2000) of the Gaussian q ( w i , j | γ i , j ) deﬁned in (8).
Revised: where the switch from (9) to (10) is due to the entropy computation (Bernardo & Smith, 2000) of the Gaussian q ( w i , j | γ i , j ) deﬁned in (8).

Original: The switch from (10) to (11) is due to using a log-scale prior, similar to Appendix C in (Kingma et al., 2015) and to Section 4.2 in (Molchanov et al., 2017).
Revised: The switch from (10) to (11) is due to using a log-scale prior, similar to Appendix C in (Kingma et al., 2015) and to Section 4.2 in (Molchanov et al., 2017).

Original: E q ( w | γ ) log | (cid:15) | is computed via an accurate approximation similar to equation (14) in (Molchanov et al., 2017), with slightly different values of k 1 , k 2 and k 3 .
Revised: E q ( w | γ ) log | (cid:15) | is computed via an accurate approximation similar to equation (14) in (Molchanov et al., 2017), with slightly different values of k 1 , k 2 and k 3 .

Original: This is a very close approximation via numerically pre-computing E q ( w | γ ) log | (cid:15) | using a third degree polynomial (Kingma et al., 2015;
Revised: This is a very close approximation via numerically pre-computing E q ( w | γ ) log | (cid:15) | using a third degree polynomial (Kingma et al., 2015;

Original: Molchanov et al., 2017).
Revised: Molchanov et al., 2017).

Original: This is the form of the KL-divergence between the approximate posterior after the ﬁrst task and the prior.
Revised: This is the form of the KL-divergence between the approximate posterior after the ﬁrst task and the prior.

Original: Afterwards, it is straightforward to see how this KL-divergence applies for the subsequent tasks in a manner similar to (2), but while taking into account the new posterior form and original prior.
Revised: Afterwards, it is straightforward to see how this KL-divergence applies for the subsequent tasks in a manner similar to (2), but while taking into account the new posterior form and original prior.

Original: The KL-divergence expression derived in (11) is to be minimised.
Revised: The KL-divergence expression derived in (11) is to be minimised.

Original: By minimising (11) with respect to p i , j and then using samples from the respective distributions to assign values to α i , j , adapted contributions of each neuron j at each layer i of the network are learnt per task.
Revised: By minimising (11) with respect to p i , j and then using samples from the respective distributions to assign values to α i , j , adapted contributions of each neuron j at each layer i of the network are learnt per task.

Original: Values of p i , j are constrained between 0 and 1 during training via projected gradient descent.
Revised: Values of p i , j are constrained between 0 and 1 during training via projected gradient descent.

Original: 3.2. L EARNING THE MAXIMUM ADAPTATION VALUES
Revised: Input: A sequence of T datasets, { x ( n ) t , y ( n ) t } N t n =1 , where t = 1 , 2 , . . , T and N t is the size of the dataset associated with the t -th task. Output: q t ( θ ) , where θ are the model parameters. Initialise all p ( | w i , j | ) with a log-scale prior, as in (7).

Revised: for t = 1 . . .

Revised: T do

Revised: Disclose the dataset { x ( n ) t , y ( n ) t } N t n =1 for the current task t .

Revised: for i = 1 . . .

Revised: # layers do for j = 1 . . .

Revised: # neurons at layer i do

Revised: Compute p

Revised: i , j using stochastic gradient descent on (11).

Revised: Compute s i , j , t using (13).

Revised: Update the corresponding general value s i , j using (14).

Revised: end for end for end for

Revised: 3.2. L EARNING THE MAXIMUM ADAPTATION VALUES

Original: Using (6) to express the value of b i , j , and neglecting the constant term therein since it does not affect the optimisation, the KL-divergence in (11) is equivalent to:
Revised: Using (6) to express the value of b i , j , and neglecting the constant term therein since it does not affect the optimisation, the KL-divergence in (11) is equivalent to:

Original: [Equation]
Revised: [Equation]

Original: Values of a i , j arestraightforwardly learnt by minimising (12) with respect to a i , j .
Revised: Values of a i , j are learnt by minimising (12) with respect to a i , j .

Original: This subsection explains how to learn the maximum adaptation variable s i , j .
Revised: This subsection explains how to learn the maximum adaptation variable s i , j .

Original: Values of the maximum s i , j of the logistic function deﬁned in (6) are learnt from multiple tasks.
Revised: Values of the maximum s i , j of the logistic function deﬁned in (6) are learnt from multiple tasks.

Original: For each neuron j at layer i , there is a general value s i , j and another value that is speciﬁc for each task t , referred to as s i , j , t .
Revised: For each neuron j at layer i , there is a general value s i , j and another value that is speciﬁc for each task t , referred to as s i , j , t .

Original: This is similar to the meta-learning procedure proposed in (Finn et al., 2017).
Revised: This is similar to the meta-learning procedure proposed in (Finn et al., 2017).

Original: The following procedure to learn s is performed for each task t such that: (i) the optimisation performed to learn a task-speciﬁc value s i , j , t beneﬁts from the warm initialisation with the general value s i , j rather than a random initial condition; and then (ii) the new information obtained from the current task t isultimately reﬂected back to update the general value s i , j . • First divide the sample N t into two halves.
Revised: The following procedure to learn s is performed for each task t such that: (i) the optimisation performed to learn a task-speciﬁc value s i , j , t beneﬁts from the warm initialisation with the general value s i , j rather than a random initial condition; and then (ii) the new information obtained from the current task t is reﬂected back to update the general value s i , j . • First divide the sample N t into two halves.

Original: For the ﬁrst half, depart from the general value of s i , j as an initial condition, and use the assigned data examples from task t to learn the taskspeciﬁc values
Revised: For the ﬁrst half, depart from the general value of s i , j as an initial condition, and use the assigned data examples from task t to learn the taskspeciﬁc values

Original: s i , j , t for the current task t .
Revised: s i , j , t for the current task t .

Original: For neuron j at layer i , refer to the second term in (3), (cid:80) N t n =1 E q t ( θ ) (cid:2) log p ( y ( n ) | x ( n ) , θ ) (cid:3) as f t ( x , y , s i , j ) .
Revised: For neuron j at layer i , refer to the second term in (3), (cid:80) N t [Equation] [Equation] log p ( y ( n ) | x ( n ) , θ ) (cid:3) as f t ( x , y , s i , j ) .

Original: The set of parameters θ contains s as well as other parameters, but we focus here on s in the f notation since the following procedure is developed to optimise s .
Revised: The set of parameters θ contains s as well as other parameters, but we focus here on s in the f notation since the following procedure is developed to optimise s .

Original: Also, refer to the loss of the (classiﬁcation) function f as Err ( f ) =
Revised: Also, refer to the loss of the (classiﬁcation) function f as Err ( f ) =

Original: CE ( f ( x , θ ) (cid:107) y ) , where CE stands for the cross-entropy:
Revised: CE ( f ( x , θ ) (cid:107) y ) , where CE stands for the cross-entropy:

Original: [Equation]
Revised: [Equation]

Original: • Now use the second half of the data from task t to update the general learnt value s i , j :
Revised: • Now use the second half of the data from task t to update the general learnt value s i , j :

Original: [Equation]
Revised: [Equation]

Original: Where ω 1 and ω 2 are step-size parameters.
Revised: Where ω 1 and ω 2 are step-size parameters.

Original: When testing on samples from task t after having faced future tasks t + 1 , t + 2 , . .
Revised: When testing on samples from task t after having faced future tasks t + 1 , t + 2 , . .

Original: . , the value of s i , j used is the learnt s i , j , t .
Revised: . , the value of s i , j used is the learnt s i , j , t .

Original: There is only one value per neuron, so the overhead resulting from storing such values is negligible.
Revised: There is only one value per neuron, so the overhead resulting from storing such values is negligible.

Original: The key steps of the algorithm are listed in Algorithm 1.
Revised: The key steps of the algorithm are listed in Algorithm 1.

Original: Input: A sequence of T datasets, { x n t , y n ) t } N t n t = 1 , 2 , . .

Original: , T

Original: N t is the size of the dataset associated with the t -th task. Output: q t ( θ ) , θ are the model parameters. Initialise all p ( | w i , j | ) with a log-scale prior, as in (7). for t = 1 . . .
Revised: At task t , the algorithmic complexity of a single joint update of the parameters θ based on the additive terms in (12) is O ( MELD 2 ) , where L is the number of layers in the network, D is the

Original: T do

Original: Disclose the dataset { x ( n ) t , y ( n ) t } N t n =1 for the current task t . for i = 1 . . . # layers do for j = 1 . . . # neurons at layer i do
Revised: (largest) number of neurons within a single layer, E is the number of samples taken from the random noise variable (cid:15) , and M is the minibatch size.

Original: Compute p

Original: i , j using stochastic gradient descent on (11).

Original: Compute s i , j , t using (13).

Original: Update the corresponding general value s i , j using (14).

Original: end for end for end for

Original: At task t , the algorithmic complexity of a single joint update of the parameters θ based on the additive terms in (12) is O ( MELD 2 ) , where L is the number of layers in the network, D is the (largest) number of neurons within a single layer, E is the number of samples taken from the random noise variable (cid:15) , and M is the minibatch size.

Original: Each α is obtained by taking one sample from the corresponding p , so that does not result in an overhead in terms of the complexity.
Revised: Each α is obtained by taking one sample from the corresponding p , so that does not result in an overhead in terms of the complexity.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Our experiments mainly aim at evaluating the following: (i) the overall performance of the introduced
Revised: Our experiments mainly aim at evaluating the following: (i) the overall performance of the introduced

Original: CLAW , depicted by the average classiﬁcation accuracy over all the tasks; (ii) the extent to which catastrophic forgetting can be mitigated when deploying CLAW ; and (iii) the achieved degree of positive forward transfer.
Revised: CLAW , depicted by the average classiﬁcation accuracy over all the tasks; (ii) the extent to which catastrophic forgetting can be mitigated when deploying CLAW ; and (iii) the achieved degree of positive forward transfer.

Original: The experiments demonstrate the effectiveness of CLAW in achieving state-of-the-art continual learning results measured by classiﬁcation accuracy and by the achieved reduction in catastrophic forgetting.
Revised: The experiments demonstrate the effectiveness of CLAW in achieving state-of-the-art continual learning results measured by classiﬁcation accuracy and by the achieved reduction in catastrophic forgetting.

Original: We also perform ablations in Section D in the Appendix which exhibit the relevance of each of the proposed adaptation parameters.
Revised: We also perform ablations in Section D in the Appendix which exhibit the relevance of each of the proposed adaptation parameters.

Original: We perform six experiments on ﬁve datasets.
Revised: We perform six experiments on ﬁve datasets.

Original: The datasets in use are: MNIST (LeCun et al., 1998), notMNIST (Butalov, 2011), Fashion-MNIST (Xiao et al., 2017), Omniglot
Revised: The datasets in use are: MNIST (LeCun et al., 1998), notMNIST (Butalov, 2011), Fashion-MNIST (Xiao et al., 2017), Omniglot

Original: (Lake et al., 2011) and
Revised: (Lake et al., 2011) and

Original: CIFAR-100 (Krizhevsky & Hinton, 2009).
Revised: CIFAR-100 (Krizhevsky & Hinton, 2009).

Original: We compare the results obtained by CLAW to six different state-of-the-art continual learning algorithms: the VCL algorithm (Nguyen et al., 2018) (original form and one with a coreset), the elastic weight consolidation (EWC) algorithm (Kirkpatrick et al., 2017), the progress and compress (P & C) algorithm
Revised: We compare the results obtained by CLAW to six different state-of-the-art continual learning algorithms: the VCL algorithm (Nguyen et al., 2018) (original form and one with a coreset), the elastic weight consolidation (EWC) algorithm (Kirkpatrick et al., 2017), the progress and compress (P & C) algorithm

Original: (Schwarz et al., 2018), the reinforced continual learning (RCL) algorithm (Xu & Zhu, 2018), the one referred to as functional regularisation for continual learning (FRCL) using Gaussian processes (Titsias et al., 2019) and the learn-to-grow (LTG) algorithm (Li et al., 2019b).
Revised: (Schwarz et al., 2018), the reinforced continual learning (RCL) algorithm (Xu & Zhu, 2018), the one referred to as functional regularisation for continual learning (FRCL) using Gaussian processes (Titsias et al., 2019) and the learn-to-grow (LTG) algorithm (Li et al., 2019b).

Original: 4. O VERALL C LASSIFICATION A CCURACY
Revised: 4. O VERALL C LASSIFICATION A CCURACY

Original: Our main metric is the all-important classiﬁcation accuracy.
Revised: Our main metric is the all-important classiﬁcation accuracy.

Original: We consider six continual learning experiments, based on the MNIST, notMNIST, Fashion-MNIST, Omniglot and CIFAR-100 datasets.
Revised: We consider six continual learning experiments, based on the MNIST, notMNIST, Fashion-MNIST, Omniglot and CIFAR-100 datasets.

Original: The introduced CLAW is compared to two VCL versions: VCL with no coreset and VCL with a 200-point coreset assembled by the K-center method (Nguyen et al., 2018), EWC, P & C, RCL,
Revised: The introduced CLAW is compared to two VCL versions: VCL with no coreset and VCL with a 200-point coreset assembled by the K-center method (Nguyen et al., 2018), EWC, P & C, RCL,

Original: FRCL (its TR version) and LTG 4 .
Revised: FRCL (its TR version) and LTG 4 .

Original: All the reported classiﬁcation accuracy values reﬂect the average classiﬁcation accuracy over all tasks the learner has trained on so far.
Revised: All the reported classiﬁcation accuracy values reﬂect the average classiﬁcation accuracy over all tasks the learner has trained on so far.

Original: More speciﬁcally, assume that the continual learner has just ﬁnished training on a task t , then the reported classiﬁcation accuracy at time t is the average accuracy value obtained from testing on equally sized sets each belonging to one of the tasks 1, 2, .. ., t .
Revised: More speciﬁcally, assume that the continual learner has just ﬁnished training on a task t , then the reported classiﬁcation accuracy at time t is the average accuracy value obtained from testing on equally sized sets each belonging to one of the tasks 1, 2, .. ., t .

Original: For all the classiﬁcation experiments, statistics reported are averages of ten repetitions.
Revised: For all the classiﬁcation experiments, statistics reported are averages of ten repetitions.

Original: Statistical signiﬁcance and standard error of the average classiﬁcation accuracy obtained after completing the last two tasks of each experiment are displayed in Section E in the Appendix.
Revised: Statistical signiﬁcance and standard error of the average classiﬁcation accuracy obtained after completing the last two tasks of each experiment are displayed in Section E in the Appendix.

Original: As can be seen in Figure 1, CLAW achieves state-of-the-art classiﬁcation accuracy in all the six experiments.
Revised: As can be seen in Figure 1, CLAW achieves state-of-the-art classiﬁcation accuracy in all the six experiments.

Original: The minibatch size is 128 for Split MNIST and 256 for all the other experiments.
Revised: The minibatch size is 128 for Split MNIST and 256 for all the other experiments.

Original: More detailed descriptions of the results of every experiment are given next:
Revised: More detailed descriptions of the results of every experiment are given next:

Original: Permuted MNIST Using MNIST, Permuted MNIST is a standard continual learning benchmark (Goodfellow et al., 2014a; Kirkpatrick et al., 2017; Zenke et al., 2017).
Revised: Permuted MNIST Using MNIST, Permuted MNIST is a standard continual learning benchmark (Goodfellow et al., 2014a; Kirkpatrick et al., 2017; Zenke et al., 2017).

Original: For each task t , the corresponding dataset is formed by performing a ﬁxed random permutation process on labeled MNIST images.
Revised: For each task t , the corresponding dataset is formed by performing a ﬁxed random permutation process on labeled MNIST images.

Original: This random permutation is unique per task, i.e. it differs for each task.
Revised: This random permutation is unique per task, i.e. it differs for each task.

Original: For the hyperparameter λ of EWC, which controls the overall contribution from previous data, we experimented with two values, λ = 1 and λ = 100 .
Revised: For the hyperparameter λ of EWC, which controls the overall contribution from previous data, we experimented with two values, λ = 1 and λ = 100 .

Original: We report the latter since it has always outperformed EWC with λ
Revised: We report the latter since it has always outperformed EWC with λ

Original: =in this experiment.
Revised: =in this experiment.

Original: EWC with λ = 100 has also previously produced the best EWC classiﬁcation results (Nguyen et al., 2018).
Revised: EWC with λ = 100 has also previously produced the best EWC classiﬁcation results (Nguyen et al., 2018).

Original: In this experiment, fully connected single-head networks with two hidden layers are used.
Revised: In this experiment, fully connected single-head networks with two hidden layers are used.

Original: There are 100 hidden units in each layer, with ReLU activations.
Revised: There are 100 hidden units in each layer, with ReLU activations.

Original: Adam (Kingma & Ba, 2015) is the optimiser used in the 6 experiments with η = 0 .
Revised: Adam (Kingma & Ba, 2015) is the optimiser used in the 6 experiments with η = 0 .

Original: 001 , β 1 = 0 .
Revised: 001 , β 1 = 0 .

Original: 9 and β 2 = 0 .
Revised: 9 and β 2 = 0 .

Original: Further experimental details are given in Section C in the Appendix.
Revised: Further experimental details are given in Section C in the Appendix.

Original: Results of the accumulated classiﬁcation accuracy, averaged over tasks, on a test set are displayed in Figure 1a.
Revised: Results of the accumulated classiﬁcation accuracy, averaged over tasks, on a test set are displayed in Figure 1a.

Original: After 10 tasks, CLAW achieves signiﬁcantly (check the Appendix) higher classiﬁcation results than all the competitors.
Revised: After 10 tasks, CLAW achieves signiﬁcantly (check the Appendix) higher classiﬁcation results than all the competitors.

Original: Split MNIST In this MNIST based experiment, ﬁve binary classiﬁcation tasks are processed in the following sequence: 0/1, 2/3, 4/5, 6/7, and 8/9 (Zenke et al., 2017).
Revised: Split MNIST In this MNIST based experiment, ﬁve binary classiﬁcation tasks are processed in the following sequence: 0/1, 2/3, 4/5, 6/7, and 8/9 (Zenke et al., 2017).

Original: The architecture used consists of fully connected multi-head networks with two hidden layers, each consisting of 256 hidden units with
Revised: The architecture used consists of fully connected multi-head networks with two hidden layers, each consisting of 256 hidden units with

Original: ReLU activations.
Revised: ReLU activations.

Original: As can be seen in Figure 1b, CLAW achieves the highest classiﬁcation accuracy.
Revised: As can be seen in Figure 1b, CLAW achieves the highest classiﬁcation accuracy.

Original: Split notMNIST It contains 400,000 training images, and the classes are 10 characters, from A to J. Each image consists of one character, and there are different font styles.
Revised: Split notMNIST It contains 400,000 training images, and the classes are 10 characters, from A to J. Each image consists of one character, and there are different font styles.

Original: The ﬁve binary classiﬁcation tasks are: A/F, B/G, C/H, D/I, and E/
Revised: The ﬁve binary classiﬁcation tasks are: A/F, B/G, C/H, D/I, and E/

Original: J. The networks used here contain four hidden layers, each containing 150 hidden units with ReLU activations.
Revised: J. The networks used here contain four hidden layers, each containing 150 hidden units with ReLU activations.

Original: CLAW achieves a clear improvement in classiﬁcation accuracy over competitors (Figure 1c).
Revised: CLAW achieves a clear improvement in classiﬁcation accuracy over competitors (Figure 1c).

Original: Split Fashion-MNIST Fashion-MNIST is a dataset whose size is the same as MNIST but it is based on different (and more challenging) 10 classes.
Revised: Split Fashion-MNIST Fashion-MNIST is a dataset whose size is the same as MNIST but it is based on different (and more challenging) 10 classes.

Original: The ﬁve binary classiﬁcation tasks here are:
Revised: The ﬁve binary classiﬁcation tasks here are:

Original: T-shirt/Trouser, Pullover/Dress, Coat/Sandals, Shirt/Sneaker, and Bag/Ankle boots.
Revised: T-shirt/Trouser, Pullover/Dress, Coat/Sandals, Shirt/Sneaker, and Bag/Ankle boots.

Original: The architecture used is the same as in Split notMNIST.
Revised: The architecture used is the same as in Split notMNIST.

Original: In most of the continual learning tasks (including the more signiﬁcant, later ones) CLAW achieves a clear classiﬁcation improvement (Figure 1d).
Revised: In most of the continual learning tasks (including the more signiﬁcant, later ones) CLAW achieves a clear classiﬁcation improvement (Figure 1d).

Original: Omniglot This is a sequential learning task of handwritten characters of 50 alphabets (a total of over 1,600 characters with 20 examples each) belonging to the Omniglot dataset (Lake et al., 2011).
Revised: Omniglot This is a sequential learning task of handwritten characters of 50 alphabets (a total of over 1,600 characters with 20 examples each) belonging to the Omniglot dataset (Lake et al., 2011).

Original: We follow the same way via which this task has been used in continual learning before (Schwarz et al., 2018; Titsias et al., 2019); handwritten characters from each alphabet constitute a separate task.
Revised: We follow the same way via which this task has been used in continual learning before (Schwarz et al., 2018; Titsias et al., 2019); handwritten characters from each alphabet constitute a separate task.

Original: We thus have 50 tasks, which also allows to evaluate the scalability of the frameworks in comparison.
Revised: We thus have 50 tasks, which also allows to evaluate the scalability of the frameworks in comparison.

Original: The model used is a CNN.
Revised: The model used is a CNN.

Original: To deal with the convolutions in CLAW , we used the idea proposed and referred to as the local reparameterisation trick by Kingma et al.
Revised: To deal with the convolutions in CLAW , we used the idea proposed and referred to as the local reparameterisation trick by Kingma et al.

Original: (2014; 2015), where a single global parameter is employed per neuron activation in the variational distribution, rather than employing parameters for every constituent weight element 5 .
Revised: (2014; 2015), where a single global parameter is employed per neuron activation in the variational distribution, rather than employing parameters for every constituent weight element 5 .

Original: Further details about the CNN used are given in
Revised: Further details about the CNN used are given in

Original: Section C. The automatically adaptable CLAW achieves better classiﬁcation accuracy (Figure 1e).
Revised: Section C. The automatically adaptable CLAW achieves better classiﬁcation accuracy (Figure 1e).

Original: CIFAR-100 This dataset consists of 60,000 colour images of size 32 × 32.
Revised: CIFAR-100 This dataset consists of 60,000 colour images of size 32 × 32.

Original: It contains 100 classes, with 600 images per class.
Revised: It contains 100 classes, with 600 images per class.

Original: We use a split version CIFAR-100.
Revised: We use a split version CIFAR-100.

Original: Similar to Lopez-Paz & Ranzato

Original: [Figure]

Original: [Figure]

Original: l a ss i f

Original: i c a t i o n a cc u r a c y

Original: [Figure]

Original: i c

Original: [Figure]

Original: t

Original: i

Original: [Figure]

Original: n

Original: [Figure]

Original: i

Original: [Figure]

Original: t

Original: i

Original: [Figure]

Original: cc

Original: [Figure]

Original: i f i

Original: [Figure]

Original: CLAW achieves signiﬁcantly higher classiﬁcation results than the competing continual learning frameworks.

Original: Statistical signiﬁcance values are presented in Section E in the Appendix.

Original: The value of λ for EWC is 10,000 in (c), and 100 in the other experiments.

Original: Best viewed in colour. (2017), we perform a 20-task experiment with a disjoint subset of ﬁve classes per task.
Revised: Similar to Lopez-Paz & Ranzato (2017), we perform a 20-task experiment with a disjoint subset of ﬁve classes per task.

Original: CLAW achieves signiﬁcantly higher classiﬁcation accuracy (Figure 1f)
Revised: CLAW achieves signiﬁcantly higher classiﬁcation accuracy (Figure 1f)

Original: -also higher than the previous state of the art on CIFAR-100 by Kemker & Kanan (2018).
Revised: -also higher than the previous state of the art on CIFAR-100 by Kemker & Kanan (2018).

Original: Details of the used CNN are in Section C.
Revised: Details of the used CNN are in Section C.

Original: A conclusion that can be taken from Figure 1(a-f) is that CLAW consistently achieves state-of-the-art results (in all the 6 experiments).
Revised: A conclusion that can be taken from Figure 1(a-f) is that CLAW consistently achieves state-of-the-art results (in all the 6 experiments).

Original: It can also be seen that CLAW scales well.
Revised: It can also be seen that CLAW scales well.

Original: For instance, the difference between CLAW and the best competitor is more signiﬁcant with Split notMNIST than it is with the ﬁrst two experiments, which are based on the smaller and less challenging MNIST.
Revised: For instance, the difference between CLAW and the best competitor is more signiﬁcant with Split notMNIST than it is with the ﬁrst two experiments, which are based on the smaller and less challenging MNIST.

Original: Also,
Revised: Also,

Original: CLAW achieves good results with Omniglot and CIFAR-100.
Revised: CLAW achieves good results with Omniglot and CIFAR-100.

Original: 4. C ATASTROPHIC F ORGETTING
Revised: 4. C ATASTROPHIC F ORGETTING

Original: To assess catastrophic forgetting, we show how the accuracy on the initial task varies over the course of the training procedure on the remaining tasks (Schwarz et al., 2018).
Revised: To assess catastrophic forgetting, we show how the accuracy on the initial task varies over the course of the training procedure on the remaining tasks (Schwarz et al., 2018).

Original: Since Omniglot (and
Revised: Since Omniglot (and

Original: CIFAR-100) contain a larger number of tasks: 50 (20) tasks, i.e. 49 (19) remaining tasks after the initial task, this setting is more relevant for Omniglot and CIFAR-100.
Revised: CIFAR-100) contain a larger number of tasks: 50 (20) tasks, i.e. 49 (19) remaining tasks after the initial task, this setting is more relevant for Omniglot and CIFAR-100.

Original: We nonetheless display the results for Split MNIST, Split notMNIST, Split Fashion-MNIST, Omniglot and CIFAR-100.
Revised: We nonetheless display the results for Split MNIST, Split notMNIST, Split Fashion-MNIST, Omniglot and CIFAR-100.

Original: As can be seen in Figure 2, CLAW (at times jointly) achieves state-of-the-art performance retention degrees.
Revised: As can be seen in Figure 2, CLAW (at times jointly) achieves state-of-the-art performance retention degrees.

Original: Among the competitors, P & C and LTG also achieve high performance retention degrees.
Revised: Among the competitors, P & C and LTG also achieve high performance retention degrees.

Original: An empirical conclusion that can be made out of this and the previous experiment, is that CLAW achieves better overall continual learning results, partially thanks to the way it addresses catastrophic forgetting.
Revised: An empirical conclusion that can be made out of this and the previous experiment, is that CLAW achieves better overall continual learning results, partially thanks to the way it addresses catastrophic forgetting.

Original: The idea of adapting the architecture by adapting the contributions of neurons of each layer also seems to be working well with datasets like Omniglot and CIFAR-100, giving directions for imminent future work where CLAW can be extended for other application areas based on CNNs.
Revised: The idea of adapting the architecture by adapting the contributions of neurons of each layer also seems to be working well with datasets like Omniglot and CIFAR-100, giving directions for imminent future work where CLAW can be extended for other application areas based on CNNs.

Original: 4. P OSITIVE F ORWARD T RANSFER
Revised: 4. P OSITIVE F ORWARD T RANSFER

Original: The purpose of this experiment is to assess the impact of learning previous tasks on the current task.
Revised: The purpose of this experiment is to assess the impact of learning previous tasks on the current task.

Original: In other words, we want to evaluate whether an algorithm avoids negative transfer, by evaluating the
Revised: In other words, we want to evaluate whether an algorithm avoids negative transfer, by evaluating the

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: I n i t i  al - t a s k p e r f o  rm an c e (
Revised: l a ss i f i c a t i o n a cc u r a c y

Original: [Figure]
Revised: [Figure]

Revised: i c

Original: [Figure]
Revised: [Figure]

Original: n i t i 
Revised: t i [Figure] n

Revised: [Figure]

Revised: i

Revised: [Figure]

Revised: t

Revised: i

Revised: [Figure]

Revised: cc

Revised: [Figure]

Revised: i f i

Original: [Figure]
Revised: [Figure]

Revised: CLAW achieves signiﬁcantly higher classiﬁcation results than the competing continual learning frameworks.

Revised: Statistical signiﬁcance values are presented in Section E in the Appendix.

Revised: The value of λ for EWC is 10,000 in (c), and 100 in the other experiments.

Revised: Best viewed in colour.

Original: relative performance achieved on a unique task after learning a varying number of previous tasks (Schwarz et al., 2018).
Revised: relative performance achieved on a unique task after learning a varying number of previous tasks (Schwarz et al., 2018).

Original: From Figure 3, we can see that CLAW achieves state-of-the-art results in 4 out of the 5 experiments (at par in the ﬁfth) in terms of avoiding negative transfer. 
Revised: From Figure 3, we can see that CLAW achieves state-of-the-art results in 4 out of the 5 experiments (at par in the ﬁfth) in terms of avoiding negative transfer. [Figure] I n i t i a l - t a s k p

Revised: e r f o r m a n c e (

Revised: [Figure]

Original: C ONCLUSION
Revised: accuracy of the initial task is monitored along with the progression of tasks. Results are displayed for ﬁve datasets. CLAW is the least forgetful algorithm since performance levels achieved on the initial task do not degrade [Figure]

Revised: 2 refers to the learning accuracy of the last task after having learnt two tasks (an additional previous task), etc.

Revised: Overall, CLAW achieves state-of-the-art results in 4 out of the 5 experiments (at par in the ﬁfth) in terms of avoiding negative transfer.

Revised: Best viewed in colour.

Revised:  R ELATED W ORK

Revised: We brieﬂy discuss three related approaches to continual learning: (a) regularisation-based, (b) architecture-based and (c) memory-based.

Revised: We provide more details of related work in Section A in the Appendix.

Revised: (a) A complementary approach to CLAW is the regularisation-based approach to balance adaptability with catastrophic forgetting: a level of stability is kept via protecting parameters that greatly inﬂuence the prediction against radical changes, while allowing the rest of the parameters to change without restriction (Li & Hoiem, 2016; Lee et al., 2017; Zenke et al., 2017; Chaudhry et al., 2018;

Revised: Kim et al., 2018; Nguyen et al., 2018; Srivastava et al., 2013; Schwarz et al., 2018;

Revised: Vuorio et al., 2018; Aljundi et al., 2019c).

Revised: The elastic weight consolidation (EWC) algorithm by

Revised: Kirkpatrick et al.

Revised: (2017) is a seminal example, where a quadratic penalty is imposed on the difference between parameter values of the old and new tasks.

Revised: One limitation is the high level of hand tuning required.

Revised: (b) The architecture-based approach aims to deal with stability and adaptation issues by a ﬁxed division of the architecture into global and local parts (Rusu et al., 2016b;

Revised: Fernando et al., 2017;

Revised: Shin et al., 2017; Kaplanis et al., 2018; Xu & Zhu, 2018; Yoon et al., 2018; Li et al., 2019b).

Revised: (c)

Revised: The memory-based approach relies on episodic memory to store data (or pseudo-data) from previous tasks (Ratcliff, 1990; Robins, 1993; 1995; Thrun, 1996; Schmidhuber, 2013; Hattori, 2014; Mocanu et al., 2016; Rebufﬁ et al., 2017; Kamra et al., 2017; Shin et al., 2017; Rolnick et al., 2018; van de

Revised: Ven & Tolias, 2018; Wu et al., 2018; Titsias et al., 2019).

Revised: Limitations include overheads for tasks such as data storage, replay, and optimisation to select (or generate) the points.

Revised: CLAW can as well be seen as a combination of a regularisation-based approach (the variational inference mechanism) and a modelling approach which automates the architecture building process in a data-driven manner, avoiding the overhead resulting from either storing or generating data points from previous tasks.

Revised: CLAW is also orthogonal to (and simple to combine with, if needed) memory-based methods.

Revised:  C ONCLUSION

Original: We introduced a continual learning framework which learns how to adapt its architecture from the tasks and data at hand, based on variational inference.
Revised: We introduced a continual learning framework which learns how to adapt its architecture from the tasks and data at hand, based on variational inference.

Original: Rather than rigidly dividing the architecture into shared and task-speciﬁc parts, our approach adapts the contributions of each neuron.
Revised: Rather than rigidly dividing the architecture into shared and task-speciﬁc parts, our approach adapts the contributions of each neuron.

Original: We achieve that without having to expand the architecture with new layers or new neurons.
Revised: We achieve that without having to expand the architecture with new layers or new neurons.

Original: Results of six different experiments on ﬁve datasets demonstrate the strong empirical performance of the introduced framework, in terms of the average overall continual learning accuracy and forward transfer and also in terms of effectively alleviating catastrophic forgetting.
Revised: Results of six different experiments on ﬁve datasets demonstrate the strong empirical performance of the introduced framework, in terms of the average overall continual learning accuracy and forward transfer, and also in terms of effectively alleviating catastrophic forgetting.

Original: A CKNOWLEDGMENTS
Revised: A CKNOWLEDGMENTS

Original: HZ acknowledges support from the DARPA XAI project, contract # FA87501720152 and an Nvidia
Revised: HZ acknowledges support from the DARPA XAI project, contract # FA87501720152 and an Nvidia

Original: GPU grant.
Revised: GPU grant.

Original: RT acknowledges support by Google, Amazon, Improbable and EPSRC grants
Revised: RT acknowledges support by Google, Amazon, Improbable and EPSRC grants

Original: EP/M0269571 and EP/L000776/1.
Revised: EP/M0269571 and EP/L000776/1.

Original: A PPENDIX
Revised: A PPENDIX

Original: We begin by brieﬂy summarising the contents of the Appendix below:
Revised: We begin by brieﬂy summarising the contents of the Appendix below:

Original: • Related works are described in Section A, followed by a brief discussion on the potential applicability of CLAW to another continual learning (CL) framework in Section A.1.
Revised: • Related works are described in Section A, followed by a brief discussion on the potential applicability of CLAW to another continual learning (CL) framework in Section A.1.

Original: • In Section E, we provide the statistical signiﬁcance and standard error of the average classiﬁcation accuracy results obtained after completing the last two tasks from each experiment.
Revised: • In Section E, we provide the statistical signiﬁcance and standard error of the average classiﬁcation accuracy results obtained after completing the last two tasks from each experiment.

Original: • Further experimental details are given in Section C. • In Section D and Figures 4- 8, we display the results of performed ablations which manifest the relevance of each adaptation parameter.
Revised: • Further experimental details are given in Section C. • In Section D and Figures 4a- 4e, we display the results of performed ablations which manifest the relevance of each adaptation parameter.

Original: A  R ELATED W ORK
Revised: A M ORE R ELATED W ORK

Original: A complementary approach to CLAW , which could be combined with it, is the regularisation-based approach to balance adaptability with catastrophic forgetting: a level of stability is kept via protecting parameters that greatly inﬂuence the prediction against radical changes, while allowing the rest of the parameters to change without restriction (Li & Hoiem, 2016; Vuorio et al., 2018).
Revised: A complementary approach to CLAW , which could be combined with it, is the regularisation-based approach to balance adaptability with catastrophic forgetting: a level of stability is kept via protecting parameters that greatly inﬂuence the prediction against radical changes, while allowing the rest of the parameters to change without restriction (Li & Hoiem, 2016; Vuorio et al., 2018).

Original: In (Zenke et al., 2017), the regulariser is based on synapses where an importance measure is locally computed at each synapse during training, based on their respective contributions to the change in the global loss.
Revised: In (Zenke et al., 2017), the regulariser is based on synapses where an importance measure is locally computed at each synapse during training, based on their respective contributions to the change in the global loss.

Original: During a task change, the less important synapses are given the freedom to change whereas catastrophic forgetting is avoided by preventing the important synapses from changing (Zenke et al., 2017).
Revised: During a task change, the less important synapses are given the freedom to change whereas catastrophic forgetting is avoided by preventing the important synapses from changing (Zenke et al., 2017).

Original: The elastic weight consolidation (EWC) algorithm, introduced by Kirkpatrick et al. (2017), is a seminal example of this approach where a quadratic penalty is imposed on the difference between parameter values of the old and new tasks.
Revised: The elastic weight consolidation (EWC) algorithm, introduced by Kirkpatrick et al. (2017), is a seminal example of this approach where a quadratic penalty is imposed on the difference between parameter values of the old and new tasks.

Original: One limitation of EWC, which is rather alleviated by using minibatch or stochastic estimates, appears when the output space is not low-dimensional, since the diagonal of the Fisher information matrix over parameters of the old task must be computed, which requires a summation over all possible output labels (Kirkpatrick et al., 2017; Zenke et al., 2017; Schwarz et al., 2018).
Revised: One limitation of EWC, which is rather alleviated by using minibatch or stochastic estimates, appears when the output space is not low-dimensional, since the diagonal of the Fisher information matrix over parameters of the old task must be computed, which requires a summation over all possible output labels (Kirkpatrick et al., 2017; Zenke et al., 2017; Schwarz et al., 2018).

Original: In addition, the regularisation term involves a sum over all previous tasks with a term from each and a hand-tuned hyperparameter that alters the weight given to it.
Revised: In addition, the regularisation term involves a sum over all previous tasks with a term from each and a hand-tuned hyperparameter that alters the weight given to it.

Original: The accumulation of this leads to a lot of hand-tuning.
Revised: The accumulation of this leads to a lot of hand-tuning.

Original: The work in (Chaudhry et al., 2018) is based on penalising conﬁdent ﬁtting to the uncertain knowledge by a maximum entropy regulariser.
Revised: The work in (Chaudhry et al., 2018) is based on penalising conﬁdent ﬁtting to the uncertain knowledge by a maximum entropy regulariser.

Original: Another seminal algorithm based on regularisation, which can be applied to any model, is variational continual learning (VCL) (Nguyen et al., 2018) which formulates CL as a sequential approximate (variational) inference problem.
Revised: Another seminal algorithm based on regularisation, which can be applied to any model, is variational continual learning (VCL) (Nguyen et al., 2018) which formulates CL as a sequential approximate (variational) inference problem.

Original: However, VCL has only been applied to simple architectures, not involving any automatic model building or adaptation.
Revised: However, VCL has only been applied to simple architectures, not involving any automatic model building or adaptation.

Original: The framework in (Lee et al., 2017) incrementally matches the moments of the posterior of a Bayesian neural network that has been trained on the ﬁrst and then the second task, and so on.
Revised: The framework in (Lee et al., 2017) incrementally matches the moments of the posterior of a Bayesian neural network that has been trained on the ﬁrst and then the second task, and so on.

Original: Other algorithms pursue regularisation approaches based on sparsity (Srivastava et al., 2013; Kim et al., 2018).
Revised: Other algorithms pursue regularisation approaches based on sparsity (Srivastava et al., 2013; Kim et al., 2018).

Original: For example, the work in (Aljundi et al., 2019c) encourages sparsity on the neuron activations to alleviate catastrophic forgetting.
Revised: For example, the work in (Aljundi et al., 2019c) encourages sparsity on the neuron activations to alleviate catastrophic forgetting.

Original: The l 2 distance between the top hidden activations of the old and new tasks is used for regularisation in (Jung et al., 2016).
Revised: The l 2 distance between the top hidden activations of the old and new tasks is used for regularisation in (Jung et al., 2016).

Original: This approach has achieved good results, but is computationally expensive due to the necessity of computing at least a forward pass for every new data point through the network representing the old task (Zenke et al., 2017).
Revised: This approach has achieved good results, but is computationally expensive due to the necessity of computing at least a forward pass for every new data point through the network representing the old task (Zenke et al., 2017).

Original: Other regularisation-based continual learning algorithms include (Ebrahimi et al., 2019; Park et al., 2019).
Revised: Other regularisation-based continual learning algorithms include (Ebrahimi et al., 2019; Park et al., 2019).

Original: Another approach is the architecture-based one where the principal aim is to administer both the stability and adaptation issues via dividing the architecture into reusable parts that are less prone to changes, and other parts especially devoted to individual tasks (Rusu et al., 2016b;
Revised: Another approach is the architecture-based one where the principal aim is to administer both the stability and adaptation issues via dividing the architecture into reusable parts that are less prone to changes, and other parts especially devoted to individual tasks (Rusu et al., 2016b;

Original: Fernando et al., 2017; Yoon et al., 2018; Du et al., 2019;
Revised: Fernando et al., 2017; Yoon et al., 2018; Du et al., 2019;

Original: He et al., 2019; Li et al., 2019a; Xu et al., 2019).
Revised: He et al., 2019; Li et al., 2019a; Xu et al., 2019).

Original: To learn a new task in the work by Rusu et al. (2016a), the whole network from the previous task is ﬁrst copied then augmented with a new part of the architecture.
Revised: To learn a new task in the work by Rusu et al. (2016a), the whole network from the previous task is ﬁrst copied then augmented with a new part of the architecture.

Original: Although this is effective in eradicating catastrophic forgetting, there is a clear scalability issue since the architecture growth can be prohibitively high, especially with an increasing number of tasks.
Revised: Although this is effective in eradicating catastrophic forgetting, there is a clear scalability issue since the architecture growth can be prohibitively high, especially with an increasing number of tasks.

Original: The work introduced in (Li et al., 2019b) bases its continual learning on neural architecture search, whereas the representation in (Javed & White, 2019) is optimised such that online updates minimize the error on all samples while limiting forgetting.
Revised: The work introduced in (Li et al., 2019b) bases its continual learning on neural architecture search, whereas the representation in (Javed & White, 2019) is optimised such that online updates minimize the error on all samples while limiting forgetting.

Original: The framework proposed by Xu & Zhu (2018) interestingly aims at solving this neural architecture structure learning problem, while balancing the tradeoff between adaptation and stability, via designed reinforcement learning (RL) strategies.
Revised: The framework proposed by Xu & Zhu (2018) interestingly aims at solving this neural architecture structure learning problem, while balancing the tradeoff between adaptation and stability, via designed reinforcement learning (RL) strategies.

Original: When facing a new task, the optimal number of neurons and ﬁlters to add to each layer is cast as a combinatorial optimisation problem solved by an RL strategy whose reward signal is a function of validation accuracy and network complexity.
Revised: When facing a new task, the optimal number of neurons and ﬁlters to add to each layer is cast as a combinatorial optimisation problem solved by an RL strategy whose reward signal is a function of validation accuracy and network complexity.

Original: Another RL based framework is the one presented by Kaplanis et al.
Revised: Another RL based framework is the one presented by Kaplanis et al.

Original: (2018) where catastrophic forgetting is mitigated at multiple time scales via RL agents with a synaptic model inspired by neuroscience.
Revised: (2018) where catastrophic forgetting is mitigated at multiple time scales via RL agents with a synaptic model inspired by neuroscience.

Original: Bottom layers (those near the input) are generally shared among the different tasks, while layers near the output are task-speciﬁc.
Revised: Bottom layers (those near the input) are generally shared among the different tasks, while layers near the output are task-speciﬁc.

Original: Since the model structure is usually divided a priori and no automatic architecture learning nor adaptation takes place, alteration on the shared layers can still cause performance loss on earlier tasks due to forgetting (Shin et al., 2017).
Revised: Since the model structure is usually divided a priori and no automatic architecture learning nor adaptation takes place, alteration on the shared layers can still cause performance loss on earlier tasks due to forgetting (Shin et al., 2017).

Original: A clipped version of maxout networks (Goodfellow et al., 2013) is developed in (Lin et al., 2018) where parameters are partially shared among examples.
Revised: A clipped version of maxout networks (Goodfellow et al., 2013) is developed in (Lin et al., 2018) where parameters are partially shared among examples.

Original: The method in (Ostapenko et al., 2019) is based a dynamic network expansion accomplished by a generative adversarial network.
Revised: The method in (Ostapenko et al., 2019) is based a dynamic network expansion accomplished by a generative adversarial network.

Original: The memory-based approach, which is the third inﬂuential approach to address the adaptationcatastrophic forgetting tradeoff, relies on episodic memory to store data (or pseudodata) from previous tasks (Ratcliff, 1990; Robins, 1993; 1995; Hattori, 2014;
Revised: The memory-based approach, which is the third inﬂuential approach to address the adaptationcatastrophic forgetting tradeoff, relies on episodic memory to store data (or pseudodata) from previous tasks (Ratcliff, 1990; Robins, 1993; 1995; Hattori, 2014;

Original: Rolnick et al., 2018; Teng & Dasgupta, 2019).
Revised: Rolnick et al., 2018; Teng & Dasgupta, 2019).

Original: A major limitation of the memory-based approach is that data from previous tasks may not be available in all real-world problems (Shin et al., 2017; Choi et al., 2019).
Revised: A major limitation of the memory-based approach is that data from previous tasks may not be available in all real-world problems (Shin et al., 2017; Choi et al., 2019).

Original: Another limitation is the overhead resulting from the memory requirements, e.g. storage, replay, etc.
Revised: Another limitation is the overhead resulting from the memory requirements, e.g. storage, replay, etc.

Original: In addition, the optimisation required to select the best observation to replay for future tasks is a source of further overhead (Titsias et al., 2019).
Revised: In addition, the optimisation required to select the best observation to replay for future tasks is a source of further overhead (Titsias et al., 2019).

Original: In addition to the explicit replay form, some works have been based on generative replay (Thrun, 1996; Schmidhuber, 2013;
Revised: In addition to the explicit replay form, some works have been based on generative replay (Thrun, 1996; Schmidhuber, 2013;

Original: Mocanu et al., 2016; Rebufﬁ et al., 2017;
Revised: Mocanu et al., 2016; Rebufﬁ et al., 2017;

Original: Kamra et al., 2017; Shin et al., 2017; van de Ven & Tolias, 2018; Wu et al., 2018).
Revised: Kamra et al., 2017; Shin et al., 2017; van de Ven & Tolias, 2018; Wu et al., 2018).

Original: Notably, Shin et al.
Revised: Notably, Shin et al.

Original: (2017) train a deep generative model based on generative adversarial networks (GANs, Goodfellow et al., 2014b; Goodfellow, 2016) to mimic past data.
Revised: (2017) train a deep generative model based on generative adversarial networks (GANs, Goodfellow et al., 2014b; Goodfellow, 2016) to mimic past data.

Original: This mitigates the aforementioned problem, albeit at the added cost of the training of the generative model (Schwarz et al., 2018) and sharing its parameters.
Revised: This mitigates the aforementioned problem, albeit at the added cost of the training of the generative model (Schwarz et al., 2018) and sharing its parameters.

Original: Alleviating catastrophic forgetting via replay mechanisms has also been adopted in reinforcement learning, e.g. (Isele & Cosgun, 2018;
Revised: Alleviating catastrophic forgetting via replay mechanisms has also been adopted in reinforcement learning, e.g. (Isele & Cosgun, 2018;

Original: Rolnick et al., 2018).
Revised: Rolnick et al., 2018).

Original: A similar approach was introduced by Lopez-Paz & Ranzato (2017) where gradients of the previous task (rather than data examples) are stored so that a trust region consisting of gradients of all previous tasks can be formed to reduce forgetting.
Revised: A similar approach was introduced by Lopez-Paz & Ranzato (2017) where gradients of the previous task (rather than data examples) are stored so that a trust region consisting of gradients of all previous tasks can be formed to reduce forgetting.

Original: Other algorithms based on replay mechanisms include (Aljundi et al., 2019a;b).
Revised: Other algorithms based on replay mechanisms include (Aljundi et al., 2019a;b).

Original: Equivalent tradeoffs to the one between adaptation and stability can be found in the literature since the work in (Carpenter & Grossberg, 1987), in which a balance was needed to resolve the stabilityplasticity dilemma, where the latter refers to the ability to rapidly adapt to new tasks.
Revised: Equivalent tradeoffs to the one between adaptation and stability can be found in the literature since the work in (Carpenter & Grossberg, 1987), in which a balance was needed to resolve the stabilityplasticity dilemma, where the latter refers to the ability to rapidly adapt to new tasks.

Original: The works introduced in (Chaudhry et al., 2018; Kim et al., 2019) shed light on the tradeoff between adaptation and stability, where they explore measures of intransigence and forgetting.
Revised: The works introduced in (Chaudhry et al., 2018; Kim et al., 2019) shed light on the tradeoff between adaptation and stability, where they explore measures of intransigence and forgetting.

Original: The former refers to the inability to adapt to new tasks and data, whereas an increase in the latter clearly signiﬁes an instability problem.
Revised: The former refers to the inability to adapt to new tasks and data, whereas an increase in the latter clearly signiﬁes an instability problem.

Original: Other recent works tackling the same tradeoff include (Riemer et al., 2019) where the transfer-interference (interference is catastrophic forgetting) tradeoff is optimised for the sake of maximising transfer and minimising interference by an algorithm based on experience replay and meta-learning.
Revised: Other recent works tackling the same tradeoff include (Riemer et al., 2019) where the transfer-interference (interference is catastrophic forgetting) tradeoff is optimised for the sake of maximising transfer and minimising interference by an algorithm based on experience replay and meta-learning.

Original: Other recent algorithms include the ORACLE algorithm by Yoon et al.
Revised: Other recent algorithms include the ORACLE algorithm by Yoon et al.

Original: (2019), which addresses the sensitivity of a continual learner to the order of tasks it encounters by establishing an order robust learner that represents the parameters of each task as a sum of task-shared and task-speciﬁc parameters.
Revised: (2019), which addresses the sensitivity of a continual learner to the order of tasks it encounters by establishing an order robust learner that represents the parameters of each task as a sum of task-shared and task-speciﬁc parameters.

Original: The algorithm in (Titsias et al., 2019) achieves functional regularisation by performing approximate inference over the function (instead of parameter) space.
Revised: The algorithm in (Titsias et al., 2019) achieves functional regularisation by performing approximate inference over the function (instead of parameter) space.

Original: They use a
Revised: They use a

Original: Gaussian process obtained by assuming the weights of the last neural network layer to be Gaussian distributed.
Revised: Gaussian process obtained by assuming the weights of the last neural network layer to be Gaussian distributed.

Original: Our model is also related to the multi-task learning approach (Caruana, 1997; Heskes, 2000; Bakker & Heskes, 2003;  Stickland & Murray, 2019).
Revised: Our model is also related to the multi-task learning approach (Caruana, 1997; Heskes, 2000; Bakker & Heskes, 2003; Adel et al., 2017; Zhao et al., 2019a; Stickland & Murray, 2019; Zhao et al., 2019b).

Original: A. A PPLICABILITY OF CLAW TO O THER CL F RAMEWORKS
Revised: A. A PPLICABILITY OF CLAW TO O THER CL F RAMEWORKS

Original: As mentioned in the main document, ideas of the proposed CLAW can be applied to continual learning frameworks other than VCL.
Revised: As mentioned in the main document, ideas of the proposed CLAW can be applied to continual learning frameworks other than VCL.

Original: The latter is more relevant for the inference part of CLAW since both are based on variational inference.
Revised: The latter is more relevant for the inference part of CLAW since both are based on variational inference.

Original: As per the modeling ideas, e.g. the binary adaptation parameter depicting whether or not to adapt, and the maximum allowed adaptation, these can be integrated within other continual learning frameworks.
Revised: As per the modeling ideas, e.g. the binary adaptation parameter depicting whether or not to adapt, and the maximum allowed adaptation, these can be integrated within other continual learning frameworks.

Original: For example, the algorithm in Xu & Zhu (2018) utilises reinforcement learning to adaptively expand the network.
Revised: For example, the algorithm in Xu & Zhu (2018) utilises reinforcement learning to adaptively expand the network.

Original: The optimal number of nodes and ﬁlters to be added is cast as a combinatorial optimisation problem.
Revised: The optimal number of nodes and ﬁlters to be added is cast as a combinatorial optimisation problem.

Original: In CLAW , we do not expand the network.
Revised: In CLAW , we do not expand the network.

Original: As such, an extension of the work in (Xu & Zhu, 2018) can be inspired by CLAW where not only the number of nodes and ﬁlters to be added is decided for each task, but also a soft and more general version where an adaptation based on the same network size is performed such that the network expansion needed in (Xu & Zhu, 2018) can be further moderated.
Revised: As such, an extension of the work in (Xu & Zhu, 2018) can be inspired by CLAW where not only the number of nodes and ﬁlters to be added is decided for each task, but also a soft and more general version where an adaptation based on the same network size is performed such that the network expansion needed in (Xu & Zhu, 2018) can be further moderated.

Original: B S TATISTICAL S IGNIFICANCE AND S TANDARD E RROR
Revised: B S TATISTICAL S IGNIFICANCE AND S TANDARD E RROR

Original: In this section, we provide information about the statistical signiﬁcance and standard error of CLAW and the competing continual learning frameworks.
Revised: In this section, we provide information about the statistical signiﬁcance and standard error of CLAW and the competing continual learning frameworks.

Original: In Table 1, we list the average accuracy values (Figure 1 in the main document) obtained after completing the last two tasks from each of the six experiments.
Revised: In Table 1, we list the average accuracy values (Figure 1 in the main document) obtained after completing the last two tasks from each of the six experiments.

Original: A bold entry in Table 1 denotes that the classiﬁcation accuracy of an algorithm is signiﬁcantly higher than its competitors.
Revised: A bold entry in Table 1 denotes that the classiﬁcation accuracy of an algorithm is signiﬁcantly higher than its competitors.

Original: Signiﬁcance results are identiﬁed using a paired t-test with p = 0 . 05 .
Revised: Signiﬁcance results are identiﬁed using a paired t-test with p = 0 . 05 .

Original: Each average accuracy value is followed by the corresponding standard error.
Revised: Each average accuracy value is followed by the corresponding standard error.

Original: Average classiﬁcation accuracy resulting from CLAW is signiﬁcantly higher than its competitors on theexperiments. 
Revised: Average classiﬁcation accuracy resulting from CLAW is signiﬁcantly higher than its competitors on theexperiments. [Table]

Revised: Split MNIST (task 4)

Original: [Table]
Revised: [Table]

Revised: Split MNIST (task 5)

Revised: [Table]

Revised: Split notMNIST (task

Revised: [Table]

Revised: [Table]

Revised: notMNIST (

Revised: [Table]

Revised: [Table]

Revised: Split Fashion-MNIST (task 4)

Revised: [Table]

Revised: Split Fashion-MNIST (task 5)

Revised: [Table]

Revised: [Table]

Revised: Omniglot (

Revised: [Table]

Original: C O THER E XPERIMENTAL D ETAILS
Revised: [Table] CIFAR-100 (task 19) [Table] CIFAR-100 (

Revised: [Table]

Revised: [Table]

Revised: Permuted MNIST (

Revised: [Table]

Revised: [Table]

Revised: Split MNIST (task 4)

Revised: [Table]

Revised: Split MNIST (task 5)

Revised: [Table]

Revised: Split notMNIST (task 4)

Revised: [Table]

Revised: Split notMNIST (task 5)

Revised: [Table]

Revised: Split Fashion-MNIST (task 4)

Revised: [Table]

Revised: Split Fashion-MNIST (task 5)

Revised: [Table]

Revised: Omniglot (task 49)

Revised: [Table]

Revised: Omniglot (task 50)

Revised: [Table]

Revised: CIFAR-100 (task 19)

Revised: [Table]

Revised: CIFAR-100 (task 20)

Revised: [Table]

Revised: C O THER E XPERIMENTAL D ETAILS

Original: Here are some additional details about the datasets in use:
Revised: Here are some additional details about the datasets in use:

Original: The MNIST dataset is used in both the Permuted MNIST and Split MNIST experiments.
Revised: The MNIST dataset is used in both the Permuted MNIST and Split MNIST experiments.

Original: The MNIST (Mixed National Institute of Standards and Technology) dataset (LeCun et al., 1998) is a handwritten digit dataset.
Revised: The MNIST (Mixed National Institute of Standards and Technology) dataset (LeCun et al., 1998) is a handwritten digit dataset.

Original: Each MNIST image consists of 28 × 28 pixels, which is also the pixel size of the notMNIST and Fashion-MNIST datasets.
Revised: Each MNIST image consists of 28 × 28 pixels, which is also the pixel size of the notMNIST and Fashion-MNIST datasets.

Original: The MNIST dataset contains a training set of 60, instances and a test set of 10,000 instances.
Revised: The MNIST dataset contains a training set of 60, instances and a test set of 10,000 instances.

Original: As mentioned in the main document, each experiment is repeated ten times.
Revised: As mentioned in the main document, each experiment is repeated ten times.

Original: Data is randomly split into three partitions, training, validation and test.
Revised: Data is randomly split into three partitions, training, validation and test.

Original: A portion of 60 % of the data is reserved for training, 20 % for validation and 20 % for testing.
Revised: A portion of 60 % of the data is reserved for training, 20 % for validation and 20 % for testing.

Original: Statistics reported are the averages of these ten repetitions.
Revised: Statistics reported are the averages of these ten repetitions.

Original: Number of epochs required per task to reach a saturation level for CLAW (and the bulk of the methods in comparison) was 10 epochs for all experiments except for Omniglot and CIFAR-100 (15 epochs).
Revised: Number of epochs required per task to reach a saturation level for CLAW (and the bulk of the methods in comparison) was 10 epochs for all experiments except for Omniglot and CIFAR-100 (15 epochs).

Original: Used values of ω 1 and ω 2 are 0.05 and 0.02, respectively.
Revised: Used values of ω 1 and ω 2 are 0.05 and 0.02, respectively.

Original: For Omniglot, we used a network similar to the one used in (Schwarz et al., 2018), which consists of 4 blocks of 3 × 3 convolutions with 64 ﬁlters, followed by a ReLU and a 2 × 2 max-pooling.
Revised: For Omniglot, we used a network similar to the one used in (Schwarz et al., 2018), which consists of 4 blocks of 3 × 3 convolutions with 64 ﬁlters, followed by a ReLU and a 2 × 2 max-pooling.

Original: The same CNN is used for CIFAR-100.
Revised: The same CNN is used for CIFAR-100.

Original: CLAW achieves clearly higher classiﬁcation accuracy on both
Revised: CLAW achieves clearly higher classiﬁcation accuracy on both

Original: Omniglot and CIFAR-100 (Figures 1e and 1f).
Revised: Omniglot and CIFAR-100 (Figures 1e and 1f).

Original: D A BLATIONS
Revised: D A BLATIONS

Original: The plots displayed in this section empirically demonstrate how important the main adaptation parameters are in achieving the classiﬁcation performance levels reached by CLAW .
Revised: The plots displayed in this section empirically demonstrate how important the main adaptation parameters are in achieving the classiﬁcation performance levels reached by CLAW .

Original: In each of the
Revised: In each of the

Original: Figures 4- 9, the classiﬁcation performance of CLAW is compared to the following three cases: 1) when the parameter controlling the maximum degree of adaptation is not learnt in a multi-task fashion, i.e. when the respective general value s i , j is used instead of s i , j , t . 2) when adaptation always happens, i.e. the binary variable denoting the adaptation decision is always activated.
Revised: Figures 4a- 4f, the classiﬁcation performance of CLAW is compared to the following three cases: 1) when the parameter controlling the maximum degree of adaptation is not learnt in a multi-task fashion, i.e. when the respective general value s i , j is used instead of s i , j , t . 2) when adaptation always happens, i.e. the binary variable denoting the adaptation decision is always activated.

Original: 3) when adaptation never takes place.
Revised: 3) when adaptation never takes place.

Original: The differences in classiﬁcation accuracy between CLAW and each of the other three plots in Figures 4- 9 empirically demonstrate the relevance of each adaptation parameter.Tasks [Figure]
Revised: The differences in classiﬁcation accuracy between CLAW and each of the other three plots in Figures 4a- 4f empirically demonstrate the relevance of each adaptation parameter.

Original: [Figure]

Original: C l a ss i f

Original: i c a t i o n a cc u r a c y ( % )

Original: Permuted MNIST

Original: [Figure]

Original: [Figure]

Original: Tasks

Original: [Figure]

Original: [Figure]

Original: C

Original: [Figure]

Original: a ss i f i c a t i o n a cc u r a c y

Original: [Equation]

Original: Split MNIST

Original: [Figure]

Original: Tasks

Original: C l a ss i f

Original: i c a t i o n a cc u r a c y ( % )

Original: [Figure]

Original: Tasks

Original: C l a ss i f

Original: i c a t i o n a cc u r a c y ( % )

Original: [Figure]

Original: Tasks

Original: C l a ss i f

Original: i c a t i o n a cc u r a c y ( % )

Original: Omniglot

Original: [Figure]

Original: Tasks

Original: C l a ss i f

Original: i c a t i o n a cc u r a c y ( % )

Original: [Figure]

Original: E R UN - TIME
Revised: E R UN - TIME

Original: In Table 2, we report the wall-clock run time (in seconds) after ﬁnishing training in each of the six experiments: Permuted MNIST, Split MNIST, Split notMNIST, Split Fashion-MNIST, Omniglot and
Revised: In Table 2, we report the wall-clock run time (in seconds) after ﬁnishing training in each of the six experiments: Permuted MNIST, Split MNIST, Split notMNIST, Split Fashion-MNIST, Omniglot and

Original: CIFAR-100.
Revised: CIFAR-100.

Original: VCL and CLAW converge (i.e. reach the accuracy levels reported earlier) more quickly than the other methods.
Revised: VCL and CLAW converge (i.e. reach the accuracy levels reported earlier) more quickly than the other methods.

Original: CLAW was the fastest in 3 out of the 6 experiments, whereas VCL was the fastest in the other 3, but their training run-time values have always been close to each other.
Revised: CLAW was the fastest in 3 out of the 6 experiments, whereas VCL was the fastest in the other 3, but their training run-time values have always been close to each other.

Original: As reported in the earlier sections, there is a signiﬁcant difference in classiﬁcation accuracy in favour of
Revised: As reported in the earlier sections, there is a signiﬁcant difference in classiﬁcation accuracy in favour of

Original: CLAW .
Revised: CLAW .

Original: This has been achieved within reasonably acceptable run-time levels thanks to the proposed design where the whole data-driven adaptation procedure is kept as part of an amortised variational inference algorithm.
Revised: This has been achieved within reasonably acceptable run-time levels thanks to the proposed design where the whole data-driven adaptation procedure is kept as part of an amortised variational inference algorithm.

Original: RCL is the slowest since it is based on reinforcement learning, where a large number of trials are typically required.
Revised: RCL is the slowest since it is based on reinforcement learning, where a large number of trials are typically required.

Original: This has also been acknowledged and reported therein (Xu &
Revised: This has also been acknowledged and reported therein (Xu &

Original: Zhu, 2018).
Revised: Zhu, 2018).

Original: As mentioned earlier, thestatistics reported areaverages of ten repetitions.
Revised: mentioned earlier, the reported are

Original: [Table]
Revised: [Table]

Revised: [Figure]

Original: [Table]
Revised: [Figure] [Figure] [Figure] [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]