O VERCOMING C ATASTROPHIC F ORGETTING FOR C ONTINUAL L EARNING VIA M ODEL A DAPTATION
A BSTRACT
Learning multiple tasks sequentially is important for the development of AI and lifelong learning systems.
However, standard neural network architectures suffer from catastrophic forgetting which makes it difﬁcult for them to learn a sequence of tasks.
Several continual learning methods have been proposed to address the problem.
In this paper, we propose a very different approach, called Parameter
Generation and Model Adaptation (PGMA), to dealing with the problem.
The proposed approach learns to build a model, called the solver , with two sets of parameters.
The ﬁrst set is shared by all tasks learned so far and the second set is dynamically generated to adapt the solver to suit each test example in order to classify it.
Extensive experiments have been carried out to demonstrate the effectiveness of the proposed approach.
I NTRODUCTION
It is well-known that neural networks (NNs) suffer from catastrophic forgetting (CF) (McCloskey &
Cohen, 1989), which refers to the phenomenon that when learning a sequence of tasks, the learning of each new task may cause the NN to forget the models learned for the previous tasks.
Without solving this problem, an NN is hard to adapt to lifelong or continual learning , which is important for AI.
Problem Statement:
Given a sequence of supervised learning tasks T =
( T 1 , T 2 , . .
. , T N ) , we want to learn them one by one in the given sequence such that the learning of each new task will not forget the models learned for the previous tasks.
In recent years, many approaches (often called continual learning ) have been proposed to lessen the effect of CF (Chen & Liu, 2018; Parisi et al., 2018), e.g., dynamically expandable network (DEN) (Yoon et al., 2018), learning without forgetting (LWF) (Li & Hoiem, 2016)), elastic weight consolidation (EWC) (Kirkpatrick et al., 2017), incremental moment matching (IMM) (Lee et al., 2017), gradient episodic memory (GEM) (Lopez-Paz et al., 2017), generative replay (GR) (Shin et al., 2017), etc.
These existing studies except DEN and LWF focused on learning a model parameterized by a single joint set of parameters θ ∗ which is assumed to work well for all tasks.
We call them joint parameterization (JP) methods.
DEN and LWF require constantly increasing the number of parameters and thus can result in a huge and complex model.
JP methods, however, suffer from accuracy deterioration .
Assume we have two tasks A and B that need to be learned sequentially.
Let θ ∗ A and θ ∗ B be the optimal parameters for Task A and Task B respectively when each of them is learned individually.
Let θ ∗ be the joint parameters learned by the existing approaches to perform Tasks A and B in sequence.
Inevitably, θ ∗ is different from θ ∗ A and/or θ ∗ B and is highly likely to result in more errors for the two tasks than θ ∗ A and θ ∗ B individually.
In this paper, we propose a different approach, called Parameter Generation and Model Adaptation (PGMA), to dealing with CF and to signiﬁcantly reducing accuracy deterioration.
PGMA does not learn a joint parameter set θ ∗ .
Instead, it learns a parameter generator f ( · ) and a shared parameter set θ 0 .
The key idea of PGMA is as follows:
The overall classiﬁcation network, called the solver S , has two disjoint subsets/parts of parameters.
The ﬁrst subset is θ 0 , which is shared by all tasks learned so far.
The second subset is just a place holder H that will be ﬁlled by parameters generated by f ( · ) for each test instance.
That is, given a test instance, a set of parameters will be generated by the learned parameter generator f ( · ) to replace H .
Solver S then combines θ 0 and the generated parameters for the test instance to classify it.
Clearly, unlike existing JP approaches, we do not have a network with a set of ﬁxed parameters for S .
The idea is that θ 0 contains the common features of all tasks, and the generated parameters for each test instance adapt S for each test instance in order to classify it.
Since f ( · ) and the shared parameters θ 0 will change during training for each new task, forgetting can occur for previous tasks.
To deal with it, in training f ( · ) and S for each task T i , in addition to the training data of T i , a small number of replayed samples will be generated by a data generation network for the previous tasks to ensure that the knowledge learned for previous tasks remain stable/unforgotten.
Compared with the existing generative replay methods, our method does not need labels for the replayed data because we use the replayed data only to constrain the training of S and f ( · ) rather than to treat them as surrogates of labeled training data from previous tasks.
Further, since in the existing replay methods, the labels are produced by the current learned network itself, the labels can be noisy and biased, which may result in errors being accumulated and propagated to subsequent tasks.
Our method does not have this problem.
Apart from reducing the effect of accuracy deterioration of the existing approaches, the proposed approach also has some other advantages.
First, no parameter increase or network expansion is needed to learn new tasks.
Second, no previous data needs to be stored to enable the system to remember the previously learned models or knowledge.
Experiments conducted using two image datasets (MNIST and CIFAR-10) and two text datasets (DBPedia ontology (Lehmann et al., 2015) and THUCNews (Li et al., 2006)) show that the proposed approach PGMA works well for different scenarios and different types of datasets, and outperforms the existing strong baselines markedly.
P ROPOSED PGMA F RAMEWORK
Let the sequence of supervised learning tasks be T
= ( T 1 , T 2 , . .
. , T N ) .
Each task T i is represented by T i = { x ij , y ij | j ∈ (1 , . . .
, N i ) } , where x ij is the j -th example/sample of T i , and y ij is its label.
We use ( x t , y t ) to denote a test instance/example.
To simplify the notation, we will just use ( x i , y i ) to denote a training example from task T i , omitting the second subscript j .
Note that in the paper, we use the terms example , sample , and instance interchangeably.
The proposed parameter generation and model adaptation (PGMA) architecture has three main components:
• Solver S :
It is the main classiﬁcation model.
As mentioned in Section 1, the parameter set of S consists of two subsets, θ 0 that is shared by all tasks (and instances) and H , a parameter place holder, which will be replaced by the generated parameters set p i (or p t ) for each training (or testing) example x i (or x t ).
p i (or p t ) basically serves to adapt the solver S to classify the example in training or testing.
This is the key idea of our approach
We adopt this parameter split as several
studies
Yoon et al
, 2018; Li & Hoiem, 2016; Kirkpatrick et al., 2017)
have
shown
that only
part
of parameters of a neural network
needs to be adjusted when learning a new task.
See
Section 2.
• Dynamic Parameter Generator (DPG) f
( · ) : It takes the embedding z i (or z t ) of each input training (or testing) example x i (or x t ) to generate the parameters p
i (or p t ) for solver S .
Note that we use z i (or z t ) rather than the raw data x i (or x t ) because the raw data’s dimension can be very high.
The embedding z i (or z t ) as a low dimensional dense representation reduces the mapping space for DPG and thus reduces the difﬁculty in its parameter generation.
See Section 2.2.
• Data Generator (DG) :
It has two functions.
The main function is to generate a set of replayed data or samples { x (cid:48) m } Mm =1 using its decoder DG D for previous tasks to deal with catastrophic forgetting.
The other function is to generate the embedding z i (or z t ) of each input training (or testing) example x i (or x t ) using its encoder DG E .
See Section 2.3.
2. O VERALL A PPROACH OF PGMA
We work backward by describing testing ﬁrst before training.
Given a test instance x t , DG E ﬁrst generates its embedding z t , which is fed to f ( · ) to generate a set of parameters p t .
Solver S then takes x t as input and uses the trained/learned shared parameters θ 0 and p t to classify x t .
θ 0 contains the common features of all tasks learned so far.
p t simply adapts S for x t in order to classify x t .
For training, the pipeline of the proposed PGMA framework is shown in Figure 1.
Given a new task
T i with its data ( x i , y i ), solver S and DPG f ( · ) are jointly trained to learn T i and also not to forget the previously learned tasks.
In each iteration, a set of parameters p i is generated by the current DPG f ( z i , µ ) for each training instance x
i ( z i being its embedding), where µ is the set of parameters of the DPG network, which is trained.
[Figure]
In training the solver S and DPG for the new task T i , both f ( · ) and the shared parameters θwill change, which can cause forgetting in DPG for previous tasks.
To keep DPG remembering the acquired knowledge for previous tasks, we minimize the variation of certain layers’ output caused by the changes of θ 0 and f ( · ) using the set of replayed samples { x (cid:48) m } Mm =1 generated by DG.
PGMA has two training components: (1) DPG and solver S ( DPG&S for short), i.e., DPG is optimized together with solver S .
(2) DG .
The two components have their respective objective functions (see Sections 2.2 and 2.3), and are trained alternately.
This is because DPG takes input z i (which is produced by DG) to generate parameters, and alternating training ensures consistent convergence rates of DG and DPG.
2. D YNAMIC P ARAMETER G ENERATOR (DPG) AND S OLVER S I MPLEMENTATION
Several neural networks can be used to implement DPG for parameter generation, e.g., convolutional neural network (CNN), recurrent neural network (RNN) and multilayer perceptron (MLP).
The objective of this paper is not to explore all possible implementations.
We found that a straightforward implementation using MLP can already achieve good results.
Formally, DPG can be written as:
[Equation]
where σ is the activation function, and w D and b are the parameters of DPG and denoted by µ .
For the implementation of solver S , several deep learning networks can be used as well.
Again, a
MLP is employed in this work.
Each layer of the MLP is a perceptron and can be formalized by output = σ ( wx input ) , where x input denotes the input of the particular perceptron.
We also call a perceptron a basic unit 1 .
In general, for each basic unit k of the solver S , we can have a shared portion of the parameters θ 0 ,k and a generated portion of the parameters p i,k , i.e.,
[Equation]
where combine(.) is concatenation and K is the number of basic units that the solver has.
In our case,
K is the number of hidden layers (basic units) of the solver MLP.
It is important to note that the above is the most general case.
In practice, there is no need to adapt all the basic units in the solver, but only a subset of them.
In our experiments (Section 3), we will see that adapting only the ﬁnal layer of the solver MLP can already achieve good results.
However, one may still ask whether the proposed combination method has sufﬁcient capacity to adjust the solver S in general because only part of the parameters are generated.
Actually, p ik can affect all dimensions of the output of its corresponding basic unit k :
[Equation]
where x 1 input and x 2 input are the block vectors of x input .
p i,k x 2 input can be regarded as the bias and can adapt the output vector to any point in the vector space.
To train DPG&S, we use cross entropy loss ( L ce ).
The objective function of the solver S including
DPG f ( z i , µ ) and θ 0 for learning each new classiﬁcation task T i is deﬁned as:
[Equation]
where L ce is the cross entropy loss, R ( · ) denotes the output of all basic units (which can be any layer or a set of layers 2 ) in solver S , which we will introduce later, and θ ∗ i is the whole set of parameters of
S ( θ ∗ i = combine ( p i , θ 0 ) ) representing the adaptation of S .
We can see that the generated replayed samples x (cid:48) m are used as constraints to alleviate
DPG&S’s forgetting.
Specially, we extend the knowledge distillation loss (Hinton et al., 2015) to the general situation.
That is, to keep the past learned knowledge, the output of the basic units in the solver should not change much when learning a new task with the help of the generated data.
If we do not consider the activation function, the constraints in Eq.
4 can also be written as:
[Equation]
where K again denotes the number of basic units and M denotes the number of replayed samples.
The basic unit with a smaller k is at the relative lower layer of the solver network.
3 x (cid:48) m,k is the input of the k th basic unit and is calculated through forward propagation, except x (cid:48) m,
1 = DG D
z samplem , θ (cid:48)
d ) which is the initial replayed sample x (cid:48) m generated by DG (before optimizing the current task T i ).
w ∗ i,k is the combined parameter introduced in Section 2.2.
Two questions may be asked about Eqs.
4 and 5.
(1) Since the replayed data x (cid:48) (denoting all x (cid:48) m ) is used in Eqs.
4 and 5, does it play the same role as the original data x from previous tasks?
(2) What is the impact of the constraints on learning the new task?
We answer the two questions now.
Question 1:
The answer is positive.
Since the purpose of the constraints is to maintain the learned effect of the old data, clearly, using the original old data x in the constraints is better.
However, we proved the positive answer to question 1, which is given in Appendix.
Question 2:
In calculating Eq. 5, if we stack all { x (cid:48) m,k } Mm =1 (column vectors) into a matrix X (cid:48) k , Eq.
5 can be regarded as constraining w ∗ i,k X (cid:48)
k to remain unchanged.
If X (cid:48) k is a row low rank matrix, w ∗
i,k can be trained to ﬁt the new tasks.
Otherwise, especially if X (cid:48) k is a row full rank matrix, w ∗
i,k cannot be trained and therefore cannot learn new tasks.
However, beneﬁting from dynamic parameter generation, our approach will not suffer from this problem.
That is because w ∗ i,k is specially generated through DPG to perform well with input X (cid:48)
k .
And the new parameters will be generated to ﬁt new tasks while w ∗ i,k is able to remain unchanged.
2. D ATA G ENERATOR (DG)
As indicated earlier, DG has two functions.
First, it compresses the original input data x
i to z
i using its encoder to reduce the number of dimensions of x i and consequently reduces the mapping space of
DPG to make the generation of the parameters p i easier.
The compression is formulated by:
[Equation]
where DG E is the encoder of DG with parameters θ e .
Second, it generates the replayed data for previous tasks to deal with forgetting in solver S and DPG.
Note that DG differs from data generators in the existing generative replay (GR) methods (Shin et al., 2017) as GR needs to generate both the replay data x (cid:48) m and their labels y (cid:48) m (using the solver learned so far), which can be noisy.
DG only generates the data x (cid:48) m but not the labels (which are not needed by our approach), and then the labeling errors won’t affect our model PGMA, but will hurt GR.
Each replayed sample x (cid:48) m is generated by the decoder of DG, called DG D :
[Equation]
where z samplem is the m th sample sampled from the multivariate normal distribution.
4 θ (cid:48)
d is the set of parameters of DG D before optimizing the current task T i .
DG can be implemented with an auto-encoder, e.g., VAE-like (Variational Auto-Encoder (Kingma &
Welling, 2013)) and WAE-like (Wasserstein Auto-Encoder) auto-encoders.
We use WAE in DG since it can let different examples get a chance to stay far away from each other, which promotes better reconstruction (Tolstikhin et al., 2017).
5 Note that DG also suffers from forgetting, which is dealt with like DPG and also using specially designed losses (see below).
To train DG, we use mean square error as the reconstruction loss to enable its replay ability of the past data, and add to it the penalized form of the Wasserstein distance between the distribution of z i and multivariate normal distribution to help generate data (Tolstikhin et al., 2017) (together denoted as L wae ).
[Figure]
Note also since we only have one data generator DG, it has the forgetting problem caused by incremental training of new tasks too.
To avoid forgetting in DG, we investigated using loss functions to constrain DG to overcome its forgetting as shown in Figure 2.
We describe this approach here:
[Equation]
where x (cid:48) m is the replayed data (see Eq. 7), θ e and θ d are the encoder’s and decoder’s parameters of DG in the process of learning the new task T i , respectively, and z samplem and M have the same meanings as they are in DPG&S. Eq.
8 constrains the consistency of DG’s decoder and encoder over the randomly sampled z samplem .
Eq.
9 ensures that DG’s decoder can still remember the old data.
Using Eqs.
8 and 9, we can maintain DG’s ability to reﬂect the old data.
Overall, the ﬁnal objective function for DG is composed by L wae , Eq. 8 and 9.
2. T HE COMPLETE T RAINING P ROCEDURE
Finally, we summarize the whole training procedure of PGMA in Algorithm 1, which is selfexplanatory.
“//” is followed by comments.
The test procedure is straightforward, see Section 2.1.
Input:
Learning the do 5:
Sample mini
Training DG
Minimize L wae then
DG ; // Training DPG
S below // denote a of generated parameters 10: // and adapted solvers, respectively Compute p n using Eqs.
1 & 6; // Section 2.2 Construct S n using p n and θ 0 ; // Section 2.2 Minimize L ce then update f (
S ; end for 15: //
Learning the tasks
i
N
+ do
Generate replayed samples x m by
; for all until convergence do
Sample a mini batch from i ; 20:
Minimize L wae , Eqs. 8 &9 and then update DG ; // Section 2.3 Compute p n using Eqs. 1 & 6;
Construct S n using p n and θ 0 ; Minimize L ce , Eq. 5, then update f ( · ) and S ; // Section 2.2 end for 25: end for
E XPERIMENTS
We now evaluate the proposed approach PGMA 6 and compare it with state-of-the-art baselines using two image datasets and two text datasets.
Datasets
• Two image datasets : (1) MNIST: this dataset consists of 70,000 images of handwritten digits from 0 to 9.
We use 60,000/3000/7000 images for training/validation/testing respectively.
(2) CIFAR-10: this dataset consists of 60,000 32x32 color images of 10 classes, with 6000 images per class.
There are 50,000/3000/7000 images for training/validation/testing respectively.
• Two text datasets : (1) DBPedia
ontology: this is a crowd-sourced dataset (Lehmann et al., 2015) with 560,000 training samples and 70,000 test samples.
Out of the 70,000 test samples, we use 10,000 for validation and 60,000 for test.
(2) THUCNews: this dataset consists of 65, sentences of 10 classes (Li et al., 2006).
We randomly select 50,000/5000/10,000 sentences for training/validation/testing respectively.
Experiment Settings
Data
Preparation: To simulate sequential learning, we adopt the same two data processing methods as in (Lee et al., 2017), named disjoint and shufﬂed .
• Disjoint: This method divides each dataset into several subsets of classes.
Each subset is a task.
For example, we divide the MNIST dataset into two tasks (or subsets of classes).
The ﬁrst task consists of digits (classes) { 0 , 1 , 2 , 3 , 4 } and the second task consists of the remaining digits (classes) { 5 , 6 , 7 , 8 , 9 } .
The systems learn the two subsets as two tasks in a sequential fashion and regard them together as 10-class classiﬁcation.
In order to consider more tasks in testing, for MNIST, CIFAR-10, and THUCNews, which all have 10 classes, we created two experiment settings of 2 tasks (5 classes per task) and 5 tasks (2 classes per task).
For DBPedia, which has 14 classes, we created three experiment settings, 2 tasks (7 classes per task), 3 tasks (5, 5, and 4 classes for the three tasks respectively), and 5 tasks (3, 3, 3, 3, and 2 classes for the 5 tasks respectively).
• Shufﬂed: This method shufﬂes the input pixels of an image with a ﬁxed random permutation.
Two experiment settings were created: 3 tasks and 5 tasks.
In both cases, the dataset for the ﬁrst task is the original dataset.
The datasets for the rest of the tasks are constructed through shufﬂing.
Since shufﬂing of words in a sentence will change the sentence meaning and results in confusion, thus this experiment is not done on text datasets
Baselines: We use three state-of-the-art baselines that are representative of the current approaches: 1) EWC (elastic weight consolidation)
(Kirkpatrick et al., 2017); 2) IMM (incremental moment matching)
(Lee et al., 2017); 3) GR (generative replay)
(Shin et al., 2017).
We use the open source code released by the authors or the third party for comparison.
We use Adam algorithm to update the parameters and also use Adam as a baseline to show how serious the forgetting problem is.
Training Details: For fair comparison, our proposed approach uses the same solver (or classiﬁer) as the baselines.
That is, a multilayer perceptron is adopted as the solver/classiﬁer (as the baselines all use this method), which is a 3-layer network (i.e., two basic units with each hidden layer as a unit) followed by a softmax layer.
For our approach, the total number of parameters in the solver includes both the generated parameters p and the shared parameters θ 0 .
Due to the differences among different datasets, we adopt different settings for them, see Table 5 in Appendix for details.
All baselines and our approach use the same setting for the same dataset.
We use a 3-layer perceptron (with 2 hidden layers) network (we also call it T-net ) for DPG and set the size of each hidden layer to 1000.
Each
T-net can generate 100 parameters at a time.
We can parallel several T-nets in the DPG to generate more parameters when needed.
The network parameters are updated using the Adam algorithm with a learning rate of 0.001.
Results and Analysis
Figure 3 shows the test accuracy plots per 40 training steps of each method for each task as the tasks are sequentially learned.
IMM is not shown here because IMM works by combining well trained models of individual tasks to form one joint model for all tasks.
Thus, we cannot draw an accuracy curve with increased training steps like others.
However, its ﬁnal results and those of the other systems are given in Tables 1 and 2.
Shufﬂed CIFAR-10 is also not included in the ﬁgure because our experiments show that it cannot be learned by CNN.
It is also not used in any existing paper.
Note that due to space limitations, Figure 3 only plots the results of those settings with only 2 and 3 tasks.
From Figure 3, we can make the following observations:
The proposed PGMA method consistently outperforms the baselines in overcoming forgetting, by a big margin in most cases.
EWC does not perform well for the disjoint setting.
GR is better but still poorer than our method.
Adam’s results show that forgetting is very serious for all datasets
and settings.
Time efﬁciency: Due to DPG, our method uses slightly more time than baselines, under 25% for training and under 28% for testing compared to GR, which is efﬁcient.
This is a small price to pay for the major gain in accuracy in dealing with catastrophic forgetting.
Memory usage: There is no need for our method to save data, which results in a large memory saving.
For more details about time efﬁciency and memory usage, please see Appendix (Section C).
[Table]
From Table 1, we can see that our PGMA method is consistently superior to EWC, GR, and IMM on different datasets with 2 or 3 tasks (5 tasks results are given in Table 2).
We believe that is because our PGMA model can reduce the effect of the accuracy deterioration problem discussed in Section 1.
Adam’s results show that forgetting is very serious as it does not deal with the forgetting problem.
EWC’s performance is poor for the disjoint case (a more realistic setting in practice), which is also reported by other researchers (Lee et al., 2017).
Appendix has the results for only the last task.
[Table]
Table 2 shows the results for 5 tasks.
EWC and Adam are not included as they performed poorly.
Again, we observe that our PGMA method is markedly better than the baselines.
[Figure]
Ablation Study
Here we study how the system behaves with less and less parameters in θ 0 or more and more parameters replaced by the parameters generated by DPG.
We selected the disjoint MNIST 2 tasks setting to conduct the experiment as it is more useful than the shufﬂed setting and also more difﬁcult.
[Table]
Table 3 shows the accuracy results (averaged in the same way as in Table 1) when a portion of the parameters in only the last layer of the solver is replaced by the parameters generated by DPG.
We observe that the accuracy improves with increased percentages of parameters being replaced.
The best accuracy is obtained when 80% of the parameters in the last layer are replaced through
DPG, and the accuracy won’t further improve with more replaced parameters.
This observation indicates that replacing a part of parameters in solver to adapt new input tasks is sufﬁcient.
The same conclusion can also be made by replacing the parameters in the ﬁrst hidden layer of the solver (which has 2 hidden layers).
We ﬁx the replacing percentage of the last layer to 20%, and then increase the replacing percentages of the ﬁrst layer.
The best accuracy reaches 94.31%( ± 0.84%) when replacing 40% parameters of the ﬁrst layer, which gains only 1.87% in accuracy compared with no replacement.
This result indicates that it sufﬁces to replace the parameters in the last layer.
[Table]
Table 4 shows the contribution of different components.
We can see a signiﬁcant drop if DPG is removed.
The second row gives the performance of our model when only
DG and constraints are used.
The third row shows the result without using constraints (DG+label(like GR)) but replacing them with the replay method.
We can see that constraints work better than predicting labels.
R ELATED W ORK
Many approaches have been proposed to deal with catastrophic forgetting (CF), which is one of the challenging problems of neural networks for lifelong learning (Chen & Liu, 2018).
EWC (Kirkpatrick et al., 2017) quantiﬁes the importance of weights to previous tasks, and selectively alters the learning rates of weights.
Following EWC, Zenke et al.
(2017) measured the synapse consolidation strength in an online fashion and used it as regularization.
Learning without forgetting (LWF) (Li & Hoiem, 2016) feeds the old network with new training data in new tasks and regards the output as ”pseudolabels”.
In Incremental Moment Matching (IMM) (Lee et al., 2017), each trained network on one task is preserved and all networks are merged into one at the end of the sequence of tasks.
Above approaches focus on adjusting the network weights.
Another main approach is to add some data of past tasks to the new task training to prevent forgetting the past.
Gradient Episodic Memory (GEM) (Lopez-Paz et al., 2017) stores a subset of training data for every ﬁnished task, and limits the loss function on these so-called ”memories”.
Instead of keeping some real data from previous tasks,
Generative Replay (GR) (Shin et al., 2017) keeps data generators for previous tasks and learns using a mix of real data of the new tasks and replayed data of previous tasks.
Seff et al.
(2017) proposed to solve continual generative modeling by combining the ideas of data generation and EWC.
Other existing approaches include iCaRL (Rebufﬁ et al., 2017), Pathnet (Fernando et al., 2017), memory aware synapses (Aljundi et al., 2017), phantom sampling (Venkatesan et al., 2017), active long term memory networks (Furlanello et al., 2016), conceptor-aided backprop (He & Jaeger, 2018), gating networks (Masse et al., 2018;
Serr ` a et al., 2018), dynamically expandable networks (DEN) (Yoon et al., 2018), progress & compress (Schwarz et al., 2018) (active column is distilled into the knowledge base, taking care to protect any previously acquired skills), and incremental regularized least squares (Camoriano et al., 2017).
Most of those works suffer from accuracy deterioration as discussed in Section 1 while some dynamically expanding the network size (e.g., DEN and LWF), which results in a huge and complex model.
Our method is different from these existing approaches.
Several existing works generate parameters, but they are for different purposes.
Denil et al.
(2013) trained several different architectures by learning only a small number of weights and predicting the rest.
Ha et al.
(2016) used a small network to generate the weights for a larger network.
Brock et al.
(2017) learned an auxiliary HyperNet to generate weights for the main model.
Our method uses the generated parameters for model adaptation for continual learning.
C ONCLUSION
This paper proposed a novel approach PGMA to dealing with catastrophic forgetting.
The approach learns to build a model with two sets of parameters.
The ﬁrst set is shared by all tasks learned so far and the second set is dynamically generated to adapt the model (solver) to suit each individual test example.
As we discussed in related work, this is different from all existing methods.
Experimental results showed that the proposed approach outperformed the existing baseline methods markedly.
A CKNOWLEDGMENTS
We thank Zhangming Chan and Shen Gao for helping check the code.
This work was supported by the National Key Research and Development Program of China (No. 2017YFC0804001).
Bing Liu’s work was partially supported by National Science Foundation (NSF) under grant no.
IIS 1838770, by a research contract with Huawei Technologies Co. Ltd. and by a research gift from Tencent Holdings Limited.
Rui Yan’s work was supported by the National Science Foundation of China (NSFC No.
61672058; NSFC No. 61876196), CCF-Tencent Open Research Fund and Microsoft Research Asia (MSRA) Collaborative Research Program
APPENDIX A P ROOF FOR Q UESTION
Proof: Our objective is to minimize ||R ( x, θ ∗ i ) − R ( x, θ ∗ i − 1 )
|| .
Since R ( · ) is differentiable and its derivative function is bounded (as it uses 1-Lipschitz activation function, e
[Equation]
g., RELU or tanh), R ( · ) thus satisﬁes the Lipschitz condition.
After some manipulation, we obtain:
[Equation]
where the last inference is based on Lipschitz continuity, and L 1 and L 2 are Lipschitz constants.
We can see that if the solver has a small Lipschitz constant or the DG’s reconstruction error is small, minimizing Eq.
4 is consistent with constraining using the original data
[Table]
B P ARAMETER S ETTINGS
[Table]
C M ORE E XPERIMENTAL R ESULTS Memory
• The memory used by our model is around 151.49MB, by IMM is around 38.96MB * task num (task num will be 3 if there are three tasks), and by GR is around 70.13MB.
The memory required by all these systems are very small as compared to the total memory available in a modern computer.
• Our data generator DG has around 1,460,361 parameters (around 44.57MB, calculated by 32 Byte per parameter), which is ﬁxed.
Saving data would need much more memory.
Taking the MNIST dataset as an example.
It has 60,000 training samples and the size of each sample is 28*28.
The memory needed to store the data is 28*28*60,000 = 47,040,000 (around 358.89MB, calculated byByte per unit).
And this number multiplies when the number of tasks increases.
Training and Test Time
Taking the CIFAR10 dataset as an example (others are similar), the total time used by our model and baselines are shown in Table 6.
We can see that our method needs a bit more time than baselines but not too much (under 25% for training and under 28% for testing compared with GR).
This is a small price to pay
for
the major gain in accuracy and in dealing with the catastrophic forgetting problem
Accuracy on the Last Task
[Table]
ADAM gets the best accuracy on the last task as Adam optimizer learns the new task only and does not need to worry about the forgetting problem on the old tasks.
For the last task, our PGMA system obtained comparable accuracy on average with the baselines designed for overcoming forgetting.
However, we are much better at preventing forgetting of old tasks.
The results for the last task are given in Table 7.
Note that although EWC is quite good at the last task but it suffers seriously from forgetting as shown in Table 1 or Figure 3, which is consistent with results reported in other papers
[Table] [Table]