O VERCOMING C ATASTROPHIC F ORGETTING VIA M ODEL A DAPTATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Learning multiple tasks sequentially is important for the development of AI and lifelong learning systems.
However, standard neural network architectures suffer from catastrophic forgetting which makes it difﬁcult to learn a sequence of tasks.
Several continual learning methods have been proposed to address the problem.
In this paper, we propose a very different approach, called model adaptation , to dealing with the problem.
The proposed approach learns to build a model, called the solver , with two sets of parameters.
The ﬁrst set is shared by all tasks learned so far and the second set is dynamically generated to adapt the solver to suit each individual test example in order to classify it.
Extensive experiments have been carried out to demonstrate the effectiveness of the proposed approach.
I NTRODUCTION
It is well-known that neural networks (NN) suffer from catastrophic forgetting (CF) (McCloskey &
Cohen, 1989), which refers to the phenomenon that when learning a sequence of tasks, the learning of each new task may cause the NN to forget the models learned for the previous tasks.
Without solving this problem, an NN is hard to adapt to continual learning, which is important for AI.
Problem Statement: Given a sequence of supervised learning tasks T =
( T 1 , T 2 , . .
. , T N ) , we want to learn them one by one in the given sequence such that the learning of each new task will not forget the models learned for the previous tasks.
In recent years, many approaches (often called continual learning ) have been proposed to lessen the effect of CF (Parisi et al., 2018), e.g., dynamically expandable network (DEN) (Yoon et al., 2018), learning without forgetting (LWF) (Li & Hoiem, 2016)), elastic weight consolidation (EWC) (Kirkpatrick et al., 2017), incremental moment matching (IMM)
(Lee et al., 2017), gradient episodic memory (GEM) (Lopez-Paz et al., 2017), generative replay (GR) (Shin et al., 2017), etc.
These existing studies except DEN and LWF focused on learning a model parameterized by a single joint set of parameters θ ∗ which is assumed to work well for all tasks.
We call them joint parameterization (JP) methods.
DEN and LWF required constantly increasing the number of parameters and thus can result in a huge and complex model.
JP methods, however, suffer from accuracy deterioration .
Assume we have two tasks A and B that need to be learned sequentially.
Let θ ∗ A and θ ∗ B be the optimal parameters for Task A and Task B respectively where each of them is learned individually.
Let θ ∗ be the joint parameters learned by the existing approaches to perform Tasks A and B in sequence.
Inevitably, θ ∗ is different from θ ∗ A and/or θ ∗ B and is very likely to result in more errors for the two tasks than θ ∗ A and θ ∗ B individually.
In this paper, we propose a different approach, called model adaptation (MA), to deal with CF and to signiﬁcantly reduce accuracy deterioration.
MA does not learn a joint parameter set θ ∗ , but learns a parameter generator f ( · ) and a shared parameter set θ 0 .
The key idea of MA is as follows:
The overall classiﬁcation network, called the solver S , has two disjoint subsets/parts of parameters.
The ﬁrst subset is θ 0 , which is shared by all tasks learned so far.
The second subset is just a place holder H , which will be ﬁlled by parameters generated by f ( · ) for each test instance.
Speciﬁcally, in testing, given a test instance, a set of parameters will be generated by the learned parameter generator f ( · ) to replace H .
Solver S then combines θ 0 and the generated parameters for the test instance to classify it.
Clearly, unlike existing JP approaches, we do not have a network with a set of ﬁxed parameters for S .
The idea is that θ 0 contains the common features of all tasks, and the generated parameters for each test instance adapt S for each test instance in order to classify it.
Since f ( · ) and the shared parameters θ 0 will change during training for each new task, forgetting can occur for previous tasks.
To deal with it, in training f ( · ) and S for each task T i , in addition to the training data of T i , a small number of replayed samples will be generated by a data generation network for the previous tasks to ensure that the knowledge learned for previous tasks remain stable/unforgotten.
Compared with the existing generative replay methods, our method does not need labels for the replayed data because we use the replayed data only to constrain the training of S and f ( · ) rather than to treat them as surrogates of labeled training data from previous tasks.
Further, since in the existing replay methods, the labels are produced by the current learned network itself, the labels can be noisy and biased, which can result in errors being accumulated and propagated to subsequent tasks.
Our method does not have this problem.
Apart from reducing the effect of accuracy deterioration of the existing approaches, the proposed approach also has some other advantages.
First, no parameter increase or network expansion is needed to learn new tasks.
Second, no previous data needs to be stored to enable the system to remember the previous learned models or knowledge.
Experiments conducted using two image datasets (MNIST and CIFAR-10) and two text datasets (DBPedia ontology (Lehmann et al., 2015) and THUCNews (Li et al., 2006)) show that the proposed approach works well for different scenarios and different types of datasets, and outperforms the existing state-of-the art baselines markedly.
P ROPOSED M ODEL A DAPTATION F RAMEWORK
Let the sequence of supervised learning tasks be T
= ( T 1 , T 2 , . .
. , T N ) .
Each task T i is represented by T i = { x ij , y ij | j ∈ (1 , . . .
, N i ) } , where x ij is the j -th example/sample of T i , and y ij is its label.
We use ( x t , y t ) to denote a test instance/example. To simplify the notation, we will just use ( x i , y i ) to denote a training example from task T i omitting the second subscript j . Note that in the paper, we use the terms example , sample , and instance interchangeably. The proposed Model Adaptation (MA) model has three main components:
• Solver S : It is the main classiﬁcation model. As mentioned in Section 1, the parameter set of S consists of two subsets, θ 0 that is shared by all tasks (and instances) and H , a parameter place holder, which will be replaced by the generated parameters set p i (or p t ) for each training (or testing) example x i (or x t ).
p i [-} Ni =
-](or p t ) basically serves to adapt the solver S to classify the example in training or testing. This is the key idea of our approach. We adopt this parameter split as several studies (Yoon et al., 2018; Li & Hoiem, 2016; Kirkpatrick et al., 2017) have shown that only part of parameters of a neural network needs to be adjusted when learning a new task. See Section 2.
• Dynamic Parameter Generator (DPG) f
( · ) : It takes the embedding z i (or z t ) of each input training (or testing) example x i (or x t ) to generate the parameters p i (or p t ) for solver S . Note that we use z i (or z t ) rather than the raw data x i (or x t ) because the raw data’s dimension can be very high. The embedding z i (or z t ) as a low dimensional dense representation reduces the mapping space for DPG and thus reduces the difﬁculty in its parameter generation.
See Section 2.2.
• Data Generator (DG) : It has two functions. The main function is to generate a set of replayed data or samples { x (cid:48) m } Mm =1 using its decoder DG D for previous tasks to deal with catastrophic forgetting. The other function is to generate the embedding z i (or z t ) of each input training (or testing) example data x i (or x t ) using its
encoder DG E .
See Section 2.3.
2. O VERALL A PPROACH
We work backward by describing testing ﬁrst before training.
Given a test instance x t , DG E ﬁrst generates its embedding z t , which is fed to f ( · ) to generate a set of parameters p t .
Solver S then takes x t as input and uses the trained/learned shared parameters θ 0 and p t to classify x t .
θ 0 contains the common features of all tasks learned so far.
p t simply adapts S for x t in order to classify x t .
Figure 1 shows the pipeline of the proposed MA framework for training.
Given a new task T i with its data ( x i , y i ), solver S and DPG f ( · ) are jointly trained to learn T i and also not to forget the previously learned tasks.
In each iteration, a set of parameters p i is generated by the current DPG f ( z i , µ ) for each training instance x i , where µ is the set of parameters of the DPG network, which is trained.
[Figure]
In training the solver S and DPG for the new task T i , both f ( · ) and the shared parameters θ 0 will change, which can cause forgetting in DPG for previous tasks.
To keep DPG remembering the acquired knowledge for previous tasks, we minimize the variation of certain layers’ output caused by the changes of θ 0 and f ( · ) using the set of replayed samples { x (cid:48) m } Mm =1 generated by DG.
The objective of solver S including
DPG f ( z i , µ ) and θ 0 for learning task T i is deﬁned as:
[Equation]
where R can be any layer (or a set of layers 1 ) in solver S , which we will introduce later; θ ∗ i is the whole set of parameters of S , θ ∗ i = combine ( p i , θ 0 ) representing an adaption of S (see Section 2.2).
We can see that the generated replayed samples x (cid:48) m are used as constraints.
Note that DPG is optimized together with solver S .
DG has its own objective function (see Section 2.4) and is trained alternately with DPG & S .
This is because DPG takes input z i (which is produced by DG) to generate parameters, and alternating training ensures consistent convergence rates for DG and DPG.
Note also since we only have one data generator DG, it also has the forgetting problem caused by incremental training of new tasks.
To avoid forgetting in DG, we also use constraints in optimization similar to Eq. 1:
[Equation]
where θ d is the set of parameters of DG D in the optimization of the current task T i , and like R , G can be any layer (or a set of layers) in DG. More details will be given in Section 2.4.
2. D YNAMIC P ARAMETER G ENERATOR (DPG) AND S OLVER S I MPLEMENTATION
Several neural networks can be used to implement DPG for parameter generation, e.g., convolutional neural network (CNN), recurrent neural network (RNN) and multilayer perceptron (MLP).
The objective of this paper is not to explore all possible implementations.
We found that a straightforward implementation using MLP can already achieve good results.
Formally, DPG can be written as:
[Equation]
where σ is the activation function, and w D and b are the parameters of DPG and denoted by µ .
For the implementation of solver S , several deep learning networks can be used too.
Again, a
MLP is used in this work.
Each layer of the MLP is a perceptron and can be formalized by output = σ ( wx input ) , where x input denotes the input of the particular perceptron.
We also call a perceptron a basic unit 2 .
In general, for each basic unit k of the solver S , we can have a shared portion of the parameters θ 0 ,k and a generated portion of the parameters p i,k , i.e.,
[Equation]
where combine(.) is concatenation and K is the number of basic units that the solver has.
In our case,
K the number of hidden layers (basic units) of the solver MLP.
It is important to note that the above is the most general case.
In practice, there is no need to adapt all the basic units in the solver, but only a subset of them.
In our experiments (Section 3), we will see that adapting only the ﬁnal layer of the solver MLP can already achieve good results.
One concern about S , is that the proposed combination method may not have sufﬁcient capacity to adjust the solver because only part of parameters are generated.
Actually, p ik can affect all dimensions of the output of its corresponding basic unit k :
[Equation]
where x 1 input and x 2 input are the block vectors of x input .
p i,k x 2 input can be regarded as the bias and can adapt the output vector to any point in the vector space.
2. D ATA G ENERATOR (DG)
As indicated earlier, DG has two functions. First, it compresses the original input data x
i to z i using its decoder to reduce the number of dimensions of x i and consequently reduces the mapping space of DPG to make the generation of the parameters p i easier.
The compression is formulated by: [Equation]
where DG E is the encoder of DG with parameters θ e . Second, it generates the replayed data for previous tasks to deal with forgetting in DPG.
Note that
DG differs from data generators in the existing generative replay (GR) methods (Shin et al., 2017) as GR needs to generate both the replay data x (cid:48) m and their labels y (cid:48) m (using the solver learned so far), which can be noisy. DG only generates the data x (cid:48) m but not the labels (which are not needed by our approach), and then the labeling errors won’t affect our model MA, but will hurt GR.
Each replayed sample x (cid:48) m is generated by the decoder of DG, called DG D : [Equation] ) where z samplem is the m th sample sampled from the multivariate normal distribution.
3 θ (cid:48) d is the set of parameters of DG D before optimizing the current task T i . DG can be implemented with an auto-encoder, e.g., VAE-like (Variational Auto-Encoder (Kingma & Welling, 2013)) and WAE-like (Wasserstein Auto-Encoder) auto-encoders.
We use WAE in DG since it can let different examples get a chance to stay far away from each other, which promotes better reconstruction (Tolstikhin et al., 2017).
4 Note that DG also suffers from forgetting, which is dealt with like DPG and also using specially designed losses (see Section 2.4).
2. O BJECTIVE F UNCTIONS AND C ONSTRAINTS
As discussed in Section 2.1, there are two training components in MA: 1) DPG and solver S (DPG&S for short) and 2) DG.
The two components have their respective objective functions: 1) for DPG&S , cross entropy loss ( L ce ) is used to learn new classiﬁcation tasks; 2) for DG , mean square error is used as the reconstruction loss to enable its replay ability of past data, and penalized form of the
Wasserstein distance between the distribution of z i and multivariate normal distribution is added to help generate data (Tolstikhin et al., 2017) (together denoted as L wae ).
As shown in Eq. 1, we use constraints to alleviate DPG&S ’s forgetting.
Specially, we extend the knowledge distillation loss (Hinton et al., 2015) to the general situation.
That is, to keep the past learned knowledge, the output of the basic units in the solver should not change much when learning a new task with the help of the generated data.
Formally, when learning task T i , we use the following constraints (recall: x (cid:48)
m is a generated replayed sample of some past task, see Eq. 7):
[Equation]
where R ( · ) denotes the output of a basic unit in the solver.
If we do not consider the activation function, Eq. 8 can also be written as:
[Equation]
where K again denotes the number of basic units and M denotes the number of replayed samples.
The basic unit with a smaller k is at the relative lower layer of the solver network.
5 x (cid:48) m,k is the input of the k th basic unit and is calculated through forward propagation, except x (cid:48) m, 1 = DG D ( z samplem , θ (cid:48)
d ) which is the initial replayed sample x (cid:48) m generated by DG (before optimizing the current task T i ).
w ∗ i,k is the combined parameter introduced in Section 2.2.
Two questions may be asked about Eqs. 8 and 9.
(1) Since the replayed data x (cid:48) is used in Eqs.
8 and 9, does it play the same role as the original data x from previous tasks?
(2) What is the impact of the constraints on learning the new task? We answer the two questions now.
Question 1: The answer is positive. Since the purpose of the constraints is to maintain the learned effect of the old data, clearly, using the original old data x in the constraints is better.
However, we proved the positive answer to question 1, which is given in appendix.
{+Question 2: In calculating Eq. 9, if we stack all { x (cid:48) m,k } Mm =
+}
(column vectors) into a matrix
X (cid:48)
k , Eq.
9 can be regarded as constraining w ∗ i,k X (cid:48)
k to remain unchanged.
If X (cid:48) k is a row low rank matrix, w ∗
i,k can be trained to ﬁt the new tasks.
Otherwise, especially if X (cid:48) k is a row full rank matrix, w ∗ i,k cannot be trained and therefore cannot learn new tasks.
However, beneﬁting from dynamic parameter generation, our approach will not suffer from this problem.
That is because w ∗ i,k is specially generated through DPG to perform well with input X (cid:48)
k .
And the new parameters will be generated to ﬁt new tasks while w ∗ i,k is able to remain unchanged.
[Figure] DG also suffers from forgetting due to the using of stochastic gradient descent.
In addition to using the DPG method, as shown in Figure 2, we also investigated using loss functions to constrain DG to overcome its forgetting.
We describe this approach here:
[Equation]
where x (cid:48) m is the replayed data (see Eq. 7), θ e and θ d are the encoder’s and decoder’s parameters of DG in the process of learning the new task T i , respectively, and z samplem and M have the same meanings as they are in DPG&S. Eq.
10 constrains the consistency of DG’s decoder and encoder over the randomly sampled z sample m .
Eq. 11 ensures that DG’s decoder can still remember the old data.
Using Eqs.
10 and 11, we can maintain DG’s ability to reﬂect the old data. Finally, we summarize the whole training procedure in Algorithm 1, which is self-explanatory. “//” is followed by comments. The test procedure is straightforward, see Section 2.1.
Input: T where
Initial: initialize do 5:
Sample Minimize L // denote a generated parameters 10: // solvers respectively Compute p 2 Construct S n using p
Minimize ce f
; end
15: // Learning tasks do
Generate replayed samples x (cid:48)
m by
; for convergence do
Sample a mini batch from T
Minimize L wae , Eq. 10 &11 and then update DG ; // Section 2.4 Compute p n using Eqs. 3 & 6;
Construct S n using p n and θ 0 ; Minimize L ce
Eq.
then update f and for 25: end for
E XPERIMENTS
We now evaluate the proposed approach and compare it with state-of-the-art baselines using two image datasets and two text datasets.
Datasets
• Two image datasets : (1) MNIST: this dataset consists of 70,000 images of handwritten digits from 0 to 9.
We use 60,000/3000/7000 images for training/validation/testing respectively.
(2) CIFAR-10: this dataset consists of 60,000 32x32 color images of 10 classes, with 6000 images per class.
There are 50,000/3000/7000 images for training/validation/testing respectively.
• Two text datasets : (1) DBPedia ontology 6 : this is a crowd -sourced dataset (Lehmann et al., 2015) with 560,000 training samples and 70,000 test samples.
Out of the 70,000 test samples, we use 10,000 for validation and 60,000 for test.
(2) THUCNews: this dataset consists of 65, sentences of 10 classes (Li et al., 2006).
We randomly select 50,000/5000/10,000 sentences for training/validation/testing respectively.
Experiment Settings
Data Preparation:
To
simulate sequential learning
, we adopt the
same
two
data processing methods as in (Lee et al., 2017), named disjoint and shufﬂed .
• Disjoint: This method divides each dataset into several subsets of classes.
Each subset is a task.
For example, we can divide the MNIST dataset into two tasks (or subsets of classes).
The ﬁrst task consists of digits (classes) { 0 , 1 , 2 , 3 , 4 } and the second task consists of the remaining digits (classes) { 5 , 6 , 7 , 8 , 9 } .
The systems learn the two subsets as two tasks in a sequential fashion and regard them together as 10-class classiﬁcation.
In order to consider more tasks in testing, for MNIST, CIFAR-10, and THUCNews, which all have 10 classes, we created two settings of 2 tasks (5 classes per task) and 5 tasks (2 classes per task).
For DBPedia, which has 14 classes, we created three settings, 2 tasks (7 classes per task), 3 tasks (5, 5, and 4 classes for the three tasks respectively), and 5 tasks (3, 3, 3, 3, 3, and 2 classes for the 5 tasks respectively).
• Shufﬂed: This method shufﬂes the input pixels of an image with a ﬁxed random permutation. Two test settings were created: 3 tasks and 5 tasks.
In both cases, the dataset for the ﬁrst task is the original dataset.
The datasets for the rest of the tasks are constructed through shufﬂing.
Since shufﬂing of words in a sentence will change the sentence meaning and results in confusion, thus this experiment is not done on text datasets.
Baselines: We use three state-of-the-art baselines that are representative of the current approaches: 1) EWC (elastic weight consolidation)
(Kirkpatrick et al., 2017); 2) IMM (incremental moment matching)
(Lee et al., 2017); 3) GR (generative replay)
(Shin et al., 2017).
We use the open source code released by the authors or the third party for comparison.
We use Adam algorithm to update the parameters and also use Adam as a baseline to show how serious the forgetting problem is.
Training Details: For fair comparison, our proposed approach uses the same solver (or classiﬁer) as the baselines.
That is, a multilayer perceptron is adopted as the solver/classiﬁer (as the baselines all use this method), which is a 3-layer network (i.e., two basic units with each hidden layer as a unit) followed by a softmax layer.
For our approach, the total number of parameters in the solver includes both the generated parameters p and the shared parameters θ 0 .
Due to the differences among different datasets, we adopt different settings for them, see Table 5 in Appendix for details.
All baselines and our approach use the same setting for the same dataset.
We use a 3-layer perceptron (with 2 hidden layers) network (we also call it T-net ) for DPG and set the size of each hidden layer to 1000.
Each T-net can generate 100 parameters at a time.
We can parallel several T-nets in the DPG to generate more parameters when needed.
The network parameters are updated using the Adam algorithm with a learning rate of 0.001.
Results and Analysis
Figure 3 shows the test accuracy plots per 40 training steps of each method for each task as the tasks are sequentially learned. IMM is not used here because IMM works by combining well trained models of individual tasks to form one joint model for all tasks. Thus, we cannot draw an accuracy curve with increased training steps like others.
However, its ﬁnal results and those of the other systems are given in Tables 1 and 2.
Shufﬂed CIFAR-10 is also not included in the ﬁgure because our experiments show that it cannot be learned by CNN.
It is also not used in any existing paper.
Note that due to space limitations, Figure 3 only plots the results of those settings with only 2 and 3 tasks.
From Figure 3, we can make the following observations:
The proposed method consistently outperforms the baselines in overcoming forgetting, by a big margin in most cases.
EWC does not perform well for the disjoint setting.
GR is better but still poorer than our method.
Adam’s results show that forgetting is very serious for all datasets and settings.
Time efﬁciency: Due to DPG, our method uses slightly more time than baselines, under 25% for training and under 28% for testing compared with GR, which is most efﬁcient.
This is a small price to pay for the major gain in accuracy in dealing with catastrophic forgetting.
Memory usage: There is no need for our method to save data, which results in memory saving. For more details about time efﬁciency and memory usage, please refer to Appendix (Section C).
[Table] From Table 1, we can see that our method MA is consistently superior to EWC, GR, and IMM on different datasets with 2 or 3 tasks (5 task results are in Table 2).
We believe that is because our MA model can reduce the effect of the accuracy deterioration problem discussed in Section 1.
Adam’s results show that forgetting is very serious as it does not deal with the forgetting problem. EMC’s performance is poor for the disjoint case (a more realistic setting in practice), which is also reported by other researchers (Lee et al., 2017). [Table] Table 2 shows the results for 5 tasks.
EWC and Adam are not included as they performed poorly. Again, we observe that our method is markedly better than the baselines.
[Figure]
Ablation Study
Here we study how the system behaves with less and less parameters in θ 0 or more and more parameters being replaced by parameters generated by DPG.
We select the disjoint MNIST 2 task setting to conduct the experiment as it is more useful than the shufﬂed setting and also more difﬁcult.
[Table]
Table 3 shows the accuracy results (averaged in the same way as in Table 1) when only a portion of the parameters in the last layer of the solver is replaced parameters generated by DPG. We observe that the accuracy improves with increased percentages of parameters being replaced.
The best accuracy is obtained when 80% of the parameters in the last layer are replaced through DPG, and the accuracy won’t further improve with more replaced parameters.
This observation indicates that replacing a part of parameters in solver to adapt new input tasks is sufﬁcient.
We believe that is because part of parameters are shared among different tasks and thus there is no need to change them.
The same conclusion can also be made by replacing the parameters in the ﬁrst hidden layer of the solver (which has 2 hidden layers).
We ﬁx the replacing percentage of the last layer to 20%, and then increase the replacing percentages of the ﬁrst layer.
The best accuracy reaches 94.31%( ± 0.84%) when replacing 40% parameters of the ﬁrst layer, which only gains 1.87% in accuracy compared with no replacement.
This result also indicates that it sufﬁces to replace the parameters in the last layer. [Table]
Table 4 shows the contribution of different components.
We can see a signiﬁcant drop when removing DPG.
The second row gives the performance of our model when only
DG and constraints are used. The third row shows the result without using constraints (DG+label(like GR)) but replacing them with the replay method. We can see that constraints work better than predicting labels.  R ELATED W ORK
Many approaches have been proposed to deal with catastrophic forgetting (CF).
EWC (Kirkpatrick et al., 2017) quantiﬁes the importance of weights to previous tasks, and selectively alters the learning rates of weights.
Following EWC, Zenke et al.
(2017) measured the synapse consolidation strength in an online fashion and used it as regularization.
Learning without forgetting (LWF) (Li & Hoiem, 2016) feeds the old network with new training data in new tasks and regards the output as ”pseudolabels”.
In Incremental Moment Matching (IMM) (Lee et al., 2017), each trained network on one task is preserved and all networks are merged into one at the end of the sequence of tasks.
Above approaches focus on adjusting the network weights.
Another main approach is to add some data of past tasks to the new task training to prevent forgetting the past.
Gradient Episodic Memory (GEM) (Lopez-Paz et al., 2017) stores a subset of training data for every ﬁnished task, and limits the loss function on these so-called ”memories”.
Instead of keeping some real data from previous tasks, Generative Replay (GR) (2017) keeps data generators for previous tasks and learns using a mix of real data of the new tasks and replayed data of previous tasks.
Seff et al.
(2017) also proposed to solve continual generative modeling by combining the ideas of data generation and EWC.
Some other approaches include Learn++ (Polikar et al., 2001), iCaRL (Rebufﬁ et al., 2017), Pathnet (Fernando et al., 2017), Memory Aware Synapses (Aljundi et al., 2017), One Big Net for Everything (Schmidhuber, 2018), Phantom Sampling (Venkatesan et al., 2017), Active Long Term Memory
Networks (Furlanello et al., 2016), Conceptor-Aided Backprop (He & Jaeger, 2018), Gating Networks (Masse et al., 2018; Serr ` a et al., 2018), PackNet (Mallya & Lazebnik, 2017), Diffusion-based
Neuromodulation (Velez & Clune, 2017), Dynamically Expandable Networks (DEN) (Yoon et al., 2018), Progress & Compress (Schwarz et al., 2018), and Incremental Regularized Least Squares (Camoriano et al., 2017).
Most of those works suffer from accuracy deterioration as discussed in Section 1 while some dynamically expanding the network size (e.g., DEN and LWF), which results in a huge and complex model.
Our method is very different from these existing approaches. Several existing works also generate parameters, but they are for different purposes. Denil et al.
(2013) train several different architectures by learning only a small number of weights and predicting the rest.
Ha et al.
(2016) used a small network to generate the weights for a larger network.
Brock et al. (2017) learn an auxiliary HyperNet to generate weights for the main model. Our method uses the generated parameters for model adaption for continual learning.  C ONCLUSION
This paper proposed a very different approach to dealing with catastrophic forgetting.
The approach learns to build a model with two sets of parameters.
The ﬁrst set is shared by all tasks learned so far and the second set is dynamically generated to adapt the model (solver) to suit each individual test example.
As we discussed in related work, this is different from all existing methods.
Experimental results showed that the proposed approach outperformed the existing baseline methods markedly.
A P ROOF FOR Q UESTION
Proof: Our objective is to minimize ||R ( x, θ ∗ i ) − R ( x, θ ∗ i − 1 )
|| .
Since R ( · ) is differentiable and its derivative function is bounded because it uses 1-Lipschitz activation function (e
[Equation]
g., RELU or tanh) and R ( · ) satisﬁes the Lipschitz condition.
After some transformation, we obtain:
[Equation]
where the last inference is based on Lipschitz continuity, and L 1 and L 2 are Lipschitz constants.
We can see that if the solver has a small Lipschitz constant or the DG’s reconstruction error is small, minimizing Eq.
8 is consistency with constraining using the original data.
B P ARAMETER S ETTINGS
[Table]
C M ORE E XPERIMENTAL R ESULTS Memory
• The memory used by our model is around 151.49MB, by IMM is around 38.96MB * task num (task num will be 3 if there are three tasks), and by GR is around 70.13MB.
The memory required by all these systems are very small as compared to the total memory available in a modern computer.
• Our data generator DG has around 1,460,361 parameters (around 44.57MB, calculated by 32 Byte per parameter), which is ﬁxed.
Saving data would need much more memory.
Taking the MNIST dataset as an example.
It has 60,000 training samples and the size of each sample is 28*28.
The memory needed to store the data is 28*28*60,000 = 47,040,000 (around 358.89MB, calculated byByte per unit).
And this number multiplies when the number of tasks increases.
Training and Test Time
Taking the CIFAR10 dataset as an example (others are similar), the total time used by our model and baselines are shown in Table 6.
We can see that our method needs a bit more time than baselines but not too much (under 25% for training and under 28% for testing compared with GR).
This is a small price to pay for the major gain in accuracy and in dealing with the catastrophic forgetting problem.
Accuracy on the Last Task
ADAM gets the best accuracy on the last task as Adam optimizer learns the new task only and does not need to worry about the forgetting problem on the old tasks.
For the last task, Our MA system
[Table]
obtained comparable accuracy on average with the baselines designed for overcoming forgetting.
However
[Table]
we are much better at preventing forgetting of old tasks.
The results for the last task are given in Table 7.
Note that although EWC is quite good at the last task but it suffers seriously from forgetting as shown in Table 1 or Figure 3, which is consistent with results reported in other papers.
[Table]