Pseudo-Rehearsal for Continual Learning with Normalizing Flows
Abstract
Catastrophic forgetting (CF) happens whenever a neural network overwrites past knowledge while being trained on new tasks.
Common techniques to handle CF include regularization of the weights (using, e.g., their importance on past tasks), and rehearsal strategies, where the network is constantly re-trained on past data.
Generative models have also been applied for the latter, in order to have endless sources of data.
In this paper, we propose a novel method that combines the strengths of regularization and generative-based rehearsal approaches.
Our generative model consists of a normalizing ﬂow (NF), a probabilistic and invertible neural network, trained on the internal embeddings of the network.
By keeping a single NF conditioned on the task, we show that our memory overhead remains constant.
In addition, exploiting the invertibility of the NF, we propose a simple approach to regularize the network’s embeddings with respect to past tasks.
We show that our method performs favorably with respect to state-of-the-art approaches in the literature, with bounded computational power and memory overheads.
1. Introduction
One of the major open problems in deep learning is the socalled catastrophic forgetting (CF) (McCloskey & Cohen, 1989; Roger, 1990; French, 1999).
It is the tendency of a NN to forget past learned information when training on new tasks.
This problem is intrinsically connected with the continual learning (CL) property of a NN, which is the ability of a NN to learn consecutive information without forgetting previously stored knowledge.
Encoder
Head #
Head
[Figure]
Overcoming, or mitigating, CF is a key step in order to achieve a more general artiﬁcial intelligence; a system should be able to learn a sequence of tasks and remember them, following the lifelong learning paradigm (Thrun & Mitchell, 1995).
This is a key problem because real world tasks continually evolve and, often, it is not possible to train a NN from scratch.
Without efﬁcient methods to overcome
CF, online training in a lifelong learning scenario is not possible.
Recently, there has been a resurgence of interest in this area due to its importance.
One of the ﬁrst attempts to mitigate CF consisted in storing past examples and replaying them into the model while learning new information (Robins, 1995).
These rehearsal methods have been improved over the years, with more complex memory systems and hybrid approaches emerging.
In particular, many pseudo-rehearsal methods have been proposed, in which the external memory is replaced with a generative model, capable of generating endless samples from the past.
Pseudo-rehearsal algorithms, however, require generative models that are both easy to train (in order to provide high-quality data) and simple to condition on the task (in order to avoid having one generative model for each task).
Both of these conditions are extremely challenging in practice.
To overcome (or complement) these limitations, many regularization methods have been studied over the last years, in which additional loss terms are used to mitigate CF.
In elastic weight consolidation (EWC, (Kirkpatrick et al., 2017)), for example, past weights are used to regularize the training process, by slowing down the modiﬁcation of important weights, where the importance is quantifed based on a Fisher information criterion.
Alternative regularization strategies are achieved by acting on the previous outputs (Li & Hoiem, 2017) or gradients (Lopez-Paz & Ranzato, 2017).
Recently, it was also shown that state-of-the-art performance can be achieved by acting on the internal embeddings (i.e., the activations before the classiﬁcation layer)
(Pomponi et al., 2020).
A more complete overview of CL methods is provided later on in Section 2, or in (Parisi et al., 2019).
In this paper, we propose a novel CL method aiming to combine the beneﬁts of pseudo-rehearsal and regularization strategies.
Our algorithm is shown in Fig.
1a, and can be summarized in three steps:
Similarly to pseudo-rehearsal strategies, we store information on past tasks by training an auxiliary generative model.
Instead of training it on the input space, however, we train it to generate samples from the internal embeddings of the network (the output of the last convolution layer), simultaneously with the main classiﬁer.
We use a normalizing ﬂow (NF, (Papamakarios et al., 2019)) as generative model.
NFs are invertible neural networks that can perform sampling and density estimation in both directions.
In this way, the NF can be trained efﬁciently, with no need for additional components such as in generative adversarial networks.
Finally, we use the sampled embeddings from the trained NF to perform regularization with respect to the past tasks, as shown in
Fig.
1b
We claim that (i) training the generative model in the em- beddings’ space is signiﬁcantly easier (both in its design and in its optimization) compared to the input space, and (ii) that regularization makes better use of past information stored in the generative model, compared to simply augmenting the mini-batch with new data (similarly to (Pomponi et al., 2020)).
In our experimental evaluation, we validate these two claims, and we show that our method performs favorably (or better) than several state-of-the-art approaches, while requiring signiﬁcantly less memory and computational overhead.
2. Related Works
The methods for overcoming CF can be categorized in line with (Parisi et al., 2019)
and
(Maltoni & Lomonaco, 2019), in three broad groups.
We underline that the boundaries are not always deﬁned, with many methods, including ours, exploiting two or more of these strategies.
• Architectural Strategies : methods that use speciﬁc architectures, layers, activation functions and/or weights freezing/pruning, and eventually grow the architecture when needed (e.g., (Rusu et al., 2016)).
For example, Hard Attention on Task (HAT, (Serra et al., 2018)) uses an attention mechanism in order to route the information/gradient ﬂow in the NN and preserve the weights associated to past tasks.
• Rehearsal strategies : in this case, past examples are stored and later replayed in the current mini-batches to consolidate the network.
In order to avoid having to explicitly store past examples, which requires a growing memory, pseudo-rehearsal algorithms (Robins, 1995) craft them on-the-ﬂy, most notably exploiting generative models (Shin et al., 2017).
• Regularization techniques : in this case, popularized by elastic weight consolidation (EWC, (Kirkpatrick et al., 2017)), the loss function on the current task is extended with a regularization penalty to selectively consolidate past information or slow the training on new tasks.
Broadly speaking, regularization methods are easy to implement, but they require to carefully select what information is being regularized, and how.
The method we propose in this paper is at the boundary of pseudo rehearsal and regularization strategies, so we focus on these two classes below.
In fact, our method use a generative model in the embedding space and then use the generated embeddings to regularize the network while training on new tasks.
Learning Without Forgetting (LWF, (Li & Hoiem, 2017)) is one of the ﬁrst regularization methods.
It attempts to alleviate CF by stabilizing the output layer using knowledge distillation.
Others well-known regularization methods are
EWC, which applies a soft structural regularization computed between the weights’ importance relative to the past tasks and the current weights, and Synaptic Intelligence (SI, (Zenke et al., 2017)), a modiﬁcation of EWC, which uses the difference between the current weights and their trajectory calculated during the training.
Other methods include Gradient Episodic Memory (GEM, (Lopez-Paz &
Ranzato, 2017)), Averaged-GEM (Chaudhry et al., 2019), and the recently proposed Embedding Regularization (ER, (Pomponi et al., 2020)).
In GEM, the external memory is populated with past examples that are used to regularize the direction of the current gradients, in order to move the weights in a region of the space in which all the tasks are satisﬁed.
This method is capable of improving past scores, but it requires to solve a complex minimization problem at every step, which does not scale well with the number of tasks.
ER is a regularization technique in which the external memory contains past examples and their associated embeddings, extracted at the end of the training process on the associated task.
The memory is used to impose a penalty to constrain the current embeddings to lie in the vicinity of the past ones.
This method is extremely fast and requires little memory.
The approach we propose in this paper follows the philosophy of ER to act at the level of the embeddings, instead of single weights or outputs.
Another novel and interesting approach is proposed in (Ebrahimi et al., 2020), in which the authors use an Adversarial Continual Learning (ACL) approach: it aims to alleviate CF by learning a disjoint latent space representation composed of a task-speciﬁc latent space for each task and a task-invariant feature space for all tasks.
A more challenging set of methods are pseudo-rehearsal ones.
In (Shin et al., 2017), the authors proposed a method which consists of two modules: a deep generative model and a task solver.
In this way, samples from past tasks can be generated using the generative model and interleaved with information from the new tasks; the solver is used to predict the label associated to the generated images and then regularize the network.
In (Kang et al., 2020) a similar approach, but based on a Variational Autoencoder (VAE), is proposed: it consists in a VAE and an external NN, which learns to replicate the distribution of the embeddings associated to a task; this external NN can be used to generate images associated to past tasks and reduce CF.
Right now, pseudorehearsal methods were strictly evaluated on datasets of relatively low complexity.
The question on whether these generative approaches can scale up to more complex domains is still open.
The method we propose here can be considered a pseudorehearsal one, but we focus on a more recent class of generative models (Kingma & Dhariwal, 2018), and we apply them at the level of embeddings instead of in the input space.
We note that in the literature on generative models, a number of authors have considered similar combinations of autoencoders with a generative model on their latent space.
In (Rezende & Mohamed, 2015), the authors applied a NF to learn a VAE prior.
This idea has been further studied in (Kingma et al., 2016), where the authors proposed a new NF which scales well for high-dimensional embedding spaces.
More similar to our proposal, in (Guo et al., 2019)
the authors proposed a model which uses an adversarial generative model in the embedding space to generate high resolution images.
A relatively new and emerging area of study researches how to mitigate CF when the tasks’ boundaries are not known.
In this area we highlight (Aljundi et al., 2019), in which the authors proposed a task free approach to continual learning, using a regularization-based memory.
In (Zeno et al., 2018), a task agnostic Bayesian method was proposed, demonstrating the ability of probabilistic models to handle ambiguous task boundaries.
Finally, (Rao et al., 2019) has introduced the novel idea of unsupervised learning in a lifelong scenario.
Many other methods exists; for a complete review of existing methods see (Parisi et al., 2019).
3. Proposed method 3.1. Motivation
In a sense, rehearsal methods are close to optimal, because in the limit of a very large memory they recover a standard multi-task setting.
On the other hand, these methods require a memory that, usually, grows linearly with the number of tasks.
Additionally, the computational requirements increase with the number of tasks, since each past task needs to be remembered (and rehearsed) separately.
Pseudo-rehearsal methods try to overcome these limitations by substituting the memory with a generative model and doing parameter sharing on the generative model, which is incrementally trained on all the tasks and constrained to remember the information about the tasks encountered so far.
However, this creates a new set of challenges: (i) the CF problem is removed from the NN, but the generative model itself potentially suffers CF; (ii) doing parameter sharing on real-world images can be difﬁcult.
The aim of this paper is to propose a generative approach which does not work directly on the input, and that can be used to regularize the model instead of simply augmenting the dataset.
The key idea of the proposed pseudo-rehearsal embedding regularization (PRER) is to use a generative model to sample new embeddings associated to past tasks.
To regularize the network, we (i) reconstruct the images associated to the embeddings, (ii) calculate the embeddings given by the current network; (iii) force the old and the new embeddings to be as close as possible (by moving the current one in the direction of the past ones).
3.2. Problem formulation
The following formulation of the CL scenario is similar to the one proposed in (Lopez-Paz & Ranzato, 2017).
We receive a sequence of tasks t = 1 , . . .
, M , each one composed by a set of triples { ( x i , t, y i ) } Si =1 ∈ X × N + × Y , where x i is a sample, t is an integer identifying the current task, and y i is the corresponding label of x i .
Within this formulation, the tasks never intersect, and a new task is collected only when the current one is over.
The input and the labels can belong to any domain, although most benchmarks in the CL literature have considered the image domain in a classiﬁcation setting (Parisi et al., 2019).
In this paper we focus on a CL scenario in which each task has its own classiﬁer, meaning that only a portion of the NN, that we call the encoder, is shared (top violet part in Fig. 1a).
There is no class overlap among different tasks, and accuracy is computed separately for each task.
This is done by creating a leaf branch for each task, called head, which classiﬁes only images associated to that task (right part in Fig.
1a).
Mathematically, we have a model f
( x , t ) =
S t ( E ( x )) , where E ( · ) is the encoder and S t ( · ) a task-speciﬁc classiﬁer.
In a naive setting, every time we receive a new task t , we minimize a task-speciﬁc loss starting from the current encoder E and the new randomly-initialized S t :
[Equation]
where L is a suitable loss (e.g., cross-entropy).
CF appears whenever training on the current task t degrades the performances on previous tasks 1 , . . .
, t − 1 .
Such models cannot be used to classify unknown samples, since the belonging task needs to be known a priori.
These experiments are suitable for studying the feasibility of training disjoints tasks without forgetting how to solve the previous ones.
We leave explorations of alternative scenarios, such as the so-called Single Incremental Task (SIT) (Maltoni & Lomonaco, 2019), for future work.
3.3. Pseudo-rehearsal in the embedding space
As shown in Fig.
1a, we augment the classiﬁer described in
Section 3.2 with two additional networks:
Firstly, we add a decoder network ˆ x = D
( z ) to approximately invert the encoder network.
Both of the networks are trained jointly as an autoencoder network.
Secondly, we train a generative model to generate samples from p ( z | t, y ) , i.e., embeddings of a speciﬁc task and class.
This model is implemented with a NF, described in Section 3.4.
Given the data for the new task, the training process is divided in three phases:
1) Autoencoder training : In the ﬁrst stage we train the encoder and decoder networks on the current task, by mini-
mizing a reconstruction loss
[Equation]
To avoid to use all the capacity of the autoecnoder we implement it as a sparse autoencoder (Arpit et al., 2016).
In this way, by regularizing the layers of our network, not only the reconstruction improves but also the classiﬁcation scores, due to the sparsity of the produced embeddings.
1.1) Pseudo-rehearsal embedding regularization
If the task is not the ﬁrst one, we further overwrite a percentage of each mini-batch with samples drawn from the generative model.
We sample from the generative model using the same proportions of labels observed in the training sets.
In addition to using the generative model to augment the dataset, we also apply a regularization technique similar to the ER method proposed in (Pomponi et al., 2020).
This step is applied only while we train the autoencoder and only its weights are updated; this step is is depicted in Fig.
1b.
Consider a generic sample ˆ z ∼ p ( z | t, y ) from the generative model.
We ﬁrst project the embedding to the input space using the trained decoder ˆ x = D
(ˆ z ) .
Then, we regularize the autoencoder by penalizing deviations from the currently generated embeddings:
[Equation]
where d is a distance function.
Practically, when training the autoencoder, we augment the reconstruction loss in (2) by computing (3) on all the data we use to augment the dataset.
2) Generative model training : In the second stage, we train the generative model to accurately sample embeddings from the new task t .
This procedure is described in Section 3.4.
We underline that we use a single generative model conditioned on the task, in order to keep the memory overhead of our method constant.
As in the autoencoder training, if the task is not the ﬁrst one, we overwrite a percentage of each mini-batch with samples associated from the past tasks, drawn from the generative model.
In this phase, regularize the embeddings is not needed, since the NF is will be trained on new and past embeddings jointly; the training itself acts like a consolidation of past information.
3) Classiﬁer training : In the third stage, the current taskspeciﬁc head S t is trained to solve the current task t by minimizing (1). We found that splitting the training into three separate processes helps the stability and improves the results, since each block works with the others already optimized. The computational overhead is also small, since each step updates a small part of the overall architecture (we provide memory and time comparisons in our experimental evaluation). 3.4. Normalizing Flows
To complete the speciﬁcation of our PRER method, we need to describe a speciﬁc generative model to estimate samples from the embedding space.
While most generative methods could be used here, we have found normalizing ﬂows (NFs) to be particularly effective for the task.
Introduced in (Tabak & Vanden-Eijnden, 2010) and (Tabak &
Turner, 2012), and popularized by (Rezende & Mohamed, 2015) and (Dinh et al., 2014), NFs are probabilistic models describing the transformation of a probability distribution into a more complex one using a sequence of differentiable, invertible mapping functions.
Depending on the type, a
NF is capable of efﬁciently performing sampling or density estimation in either direction (Papamakarios et al., 2019).
Ignoring for a moment the conditioning on a given task, let u ∼ p
u
( u ) be a real d -dimensional distribution which is easy to sample from, and T a transformation T : R d → R d .
The key requirements for T to be a NF are: 1) T must be invertible, with an inverse denoted by T − 1 , and 2) both T and T − 1 must be differentiable.
We can obtain samples from p z ( z ) by drawing samples from the easier p u ( u ) and then computing T ( u ) (forward mapping).
Alternatively, we can ‘normalize’ a known sample z ∼ p
z ( z ) by applying the inverse transformation T − 1 ( z ) (inverse mapping).
Additionally, we can evaluate the likelihood of a known sample z as:
[Equation]
4) where J T − 1 ∈ R
d
× d is the Jacobian matrix of all partial derivatives of T − 1 .
The prior distribution p u ( u ) is generally chosen as an Isotropic Normal distribution.
T is instead obtained by composing multiple simpler, invertible transformations T 1 , . . .
, T L , resulting in T =
T L ◦ T L − 1 ◦ · · · ◦ T 1 .
The arbitrarily complex density z can be constructed from a prior distribution by composing several simple maps and then applying Eq. 4.
As stated before, a NF can perform both sampling and density estimation in both directions.
Depending on the speciﬁc
NF, not all of these operations are necessarily easy to compute (Papamakarios et al., 2019).
This is due to the fact that to achieve them simultaneously, the model needs both a simple forward mapping, a simple inverse mapping, and the Jacobians have to be easy to compute.
In general, this is not always possible, mostly due to a computational trade- off.
Hence, a NF model needs to be built depending on the application.
In our case, we need efﬁcient density estimation and sampling in the inverse mapping (for training), but only efﬁcient sampling in the forward direction (see Fig. 1b).
We use a setup similar to (Dinh et al., 2016; Kingma & Dhariwal, 2018), whose layers we brieﬂy summarize below.
Denote by u j +1 =
T j ( u ) the generic j th block of the NF.
We build the overall NF by interleaving three types of invertible transformations.
1) Coupling Layer and Afﬁne transformation : Proposed in (Dinh et al., 2014; 2016), a coupling layer consists in a powerful reversible transformation, where the forward and the inverse mappings are computationally efﬁcient.
Here, we use the version presented in (Kingma & Dhariwal, 2018).
Consider a generic split of the input vector u j = (cid:20) a [Equation]
A coupling layer is deﬁned as:
[Equation]
where f θ ( · ) is a generic transformation implemented via a
NN, and (cid:12) is the element-wise multiplication.
The log det of a coupling layer is simply (cid:80) i log s i .
The main advantage of this layer is that the function f θ ( · ) does not have to be invertible.
The main advantage is also a disadvantage, since, due to the simplicity of the afﬁne transformation, a
NF implemented with this technique needs multiple layers and blocks in order to have enough expressive power to transform any input into p u ( u ) .
2) Random Permutation : since only a portion of the input is modiﬁed in each block, it is required to randomly permute the output of a block in order to modify a new set of parameters in the following block, i.e., T j ( u ) =
Pu , where P is a ﬁxed permutation matrix.
Clearly, T − 1 j ( u j +1 ) =
P T u j +1 , and the transformation has a unitary determinant.
3) Invertible Batch Norm : As in (Dinh et al., 2016), we also apply batch normalization, but on the input of the coupling layer instead of the output.
It acts as a linear re-scaling of each parameter, thus it can be easily inverted and included in the Jacobian computation.
The scaling is done in the following way:
[Equation]
and the parameters are iteratively estimated as:
[Equation]
where m ∈ [0 , 1] is the momentum, while µ b and σ b are, respectively, the mean and the standard deviation of the current mini-batch.
On the ﬁrst batch, the parameters are initialized as µ = µ b and σ = σ b , and they are updated with each new batch during the training.
The log det is computed as:
[Equation]
with (cid:15) > 0 a parameter to avoid zero multiplication.
Note that, in our case, we need a NF that is conditioned on the pair ( t, y ) .
To implement this, we condition the ﬁrst coupling layer by passing the pair ( t, y ) as an additional argument to f θ , in order to be able to generate embeddings for every possible class.
The idea is similar to what is done in (Winkler et al., 2019).
Practically, the conditioning is done by building a one-hot vector with respect to the maximum number of separate classes we presume to observe.
We have found the NF to be very easy to train and condition in all our experiments, with no need for speciﬁc techniques to avoid
CF.
In Fig.
2 we show some examples of generated embeddings from the MNIST dataset, using several conditioning vectors.
Furthermore, we implement a multi scale architecture as explained in (Dinh et al., 2016; Kingma & Dhariwal, 2018).
Each mapping T j is decomposed into multiple submappings.
After each sub-level, the output is split in two, with the ﬁrst part being sent directly to the output of the level, while the other part ﬂows into the next sub-mapping for further processing.
This architecture improves the convergence and the stability, and it results in a smaller number of parameters.
Many others way of building a NF exists; for an in-depth review of the NF literature we refer to (Papamakarios et al., 2019) and (Kobyzev et al., 2020).
4. Experiments 4.1. Datasets and metrics
For evaluating the proposed method, we consider three different datasets: MNIST, KMNIST (Clanuwat et al., 2018),
SVHN (Netzer et al., 2011) and CIFAR10 (Krizhevsky, 2009).
We evaluate the methods under the previously described multi-head CL scenario; to do so, being C the classes of a dataset, these are grouped in M sets, each one making up a task containing
c m ∈ N + classes, with c m > 1 , giving: M =
Cc m (with this formulation each task contains the same number of classes, with the exception of the last one if C mod c m (cid:54) = 0 ).
In particular, we split the labels by grouping the original ones in an incremental way.
To evaluate the efﬁciency and to compare the methods two metrics from (D´ıaz-Rodr´ıguez et al., 2018) have been used.
Conditioner ...
All these metrics are calculated on a matrix R ∈ R M × M , where M is the number of tasks, and each entry R ij is the test accuracy on task j when the training on task i is completed.
This matrix of scores can be used to calculate different metrics, and we use the following:
Accuracy : it is the average accuracy on the trained tasks.
It considers the elements of the diagonal as well as the elements below it, doing so also the evolution of the scores is taken into consideration:
[Equation]
This metric aims to show the average performance of the model in every step of the training and for each task.
Backward Transfer (BWT) : it measures how much information from the old tasks is remembered during the training on the new one.
It is calculated as:
[Equation]
This metric can be greater than zero, meaning that not only the model is remembering everything about the past tasks, but it also improves the score on these.
In our scenario, this phenomenon is rare, since heads associated to past tasks are no longer trained.
We use these metrics because they embed all the important aspects of the CL problem: the ability of an approach to mitigate CF and to classify correctly the past tasks, but also the ability to train the model on the current task.
They are both important because a model with high accuracy and low BWT is a model not capable of alleviating CF; on the [Table] other hand, low accuracy and high BWT tells us that the constraints applied on the model are too restrictive, blocking the training of the current task.
4.2. Experimental setting
We compare our method to the naive approach –training on all the tasks sequentially without mitigating the CF problem– and to the most popular or related rehearsal and regularization approaches: LWF, EWC, GEM, and ER.
With regard to LWF, since the original method was designed for a 2-task scenario and it is not competitive outside it, we modiﬁed the approach by transforming it into a rehearsal approach: we have an external memory which contains past samples and the associated past predictions.
When regularizing, the current predictions on past tasks’ samples are forced to match the past prediction; in this way, this approach is competitive with the other baseline methods.
For MNIST and KMNIST we train the models using SGD with a learning rate equal to 0 .
1 , while for SVHN and CIFAR we use Adam (Kingma & Ba, 2014) with a learning rate equal to 0 .
Regarding PRER, we train both the autoencoders and the NF using Adam, with, respectively, learning rate equal to 1e − 3 and 1e − 4 .
We use batch size equal to 64 for all datasets with the exception of CIFAR10, for which we used 128 as batch size.
Each dataset have been splitted in 5 tasks by setting c m =
We ﬁne-tune all hyper-parameters based on a grid-search and the results from the original papers.
For PRER we use 2 levels, each one composed by 10 blocks (each block is composed by: batch normalization, a coupling layer, and a random permutation); each coupling layer is deﬁned as a fully connected network with 3 layers: T : R d 2 → R d →
R d 2 .
We also ﬁnd that setting the coupling layers scale as ˜ s = sigmoid ( s + 2) , so that the initial scale is near the identity, helps in stabilizing the training phase.
The encoder is composed by three convolutional layers of 12, 24, and 48 ﬁlters for MNIST and KMNIST, and twice these sizes for SVHN and CIFAR10.
Each layer has 4 ×kernel sizes and stride 2 × 2 .
Then, the output of the last convolutional layer is ﬂattened and, using a fully-connected [Table] layer, the ﬁnal embedding vector is produced: for MNIST and KMNIST we use 50 as embedding size, while we use 100 for SVHN and 200 for CIFAR10.
The head takes as input the embedding vector, and predicts the class using three fully-connected layers, each of which halves the dimension of its input; we also use a dropout layer, with probability set to 0 .
2 , between consecutive layers.
The activation function used in all the models is the ReLU.
We repeat each experiment for 5 times, each time changing the split of the dataset (balanced split based on labels, with proportion 80% train and 20% test) as well as the initial weights of the models.
4.3. Results
In Table 1 we summarize the results concerning accuracy and BTW, while Table 2 shows the required time and memory space for each method.
Results in Table 2 are shown for brevity only on MNIST, but they are similar on the other two datasets.
First, we look to the baseline methods.
We can see that, in this multi-head scenario, all methods perform better that the
Naive approach.
LWF and EWC perform better than the naive method but not by much, and they are also more sensi- tive to the starting weights (as can be seen by the variance) and, in the case of LWF, also to the images stored in the external memory; furthermore, LWF performance degrades when the dataset becomes more complex, which leads us to conclude that it is not a reliable method for alleviating the CF problem.
The others baseline methods, GEM and
ER, performs better than LWF and EWC, with ER being slightly better in both BWT and accuracy.
If we consider also the required time and the memory requirements, which are summarized in Table 2, we can conclude that ER is the best baseline method, because it requires less memory and less time than GEM (as advocated in (Pomponi et al., 2020)).
The method we propose, PRER, achieves better BWT score than all the others methods (by a signiﬁcant margin), and comparable accuracy to the best one; it is also more robust to the weights’ initialization.
Looking at the required memory space, PRER achieves the best results w.r.t.
the others baseline methods, because, given a NF which is large enough, the generative model is capable of generating all the classes encountered during the training, requiring constant memory.
In terms of time, it depends mostly on the dimension of the generative model, which needs to be trained separately: in the MNIST example, shown in Table 2, the required time is competitive to the other methods due to the small NF that works directly on the embeddings.
5. Conclusion
In this paper we introduced PRER (Pseudo-Rehearsal Embedding Regularization), a pseudo-rehearsal method that, by working on the embedding space, is able to generate past embeddings and use them to protect past information while learning new tasks.
This approach is a different point of view on the pseudo-rehearsal methods, which usually work on the input space.
By working on a lower complexity space, the required time is reduced as well as the dimension of the generative model.
Once combined with a decoder, the generative model can be used to generate images associated to past tasks, and to constrain the model by regularizing the embeddings’ deviation.
We believe that this set of methods can be further investigated, leading to a different view of the pseudo-rehearsal approaches, which, right now, are feasible only for low complexity datasets.
We leave an investigation on how the PRER method scales to more complex datasets to future work.