U NCERTAINTY - GUIDED
L IFELONG
L EARNING
IN
B AYESIAN N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
The ability to learn in a setting where tasks arrive in a sequence without having access to previous data is vital for continual learning algorithms when restricted in capacity.
In this lifelong learning setting a single model is challenged to learn a new task, while at the same time using what it deems important to avoid forgetting about previous tasks as well as freeing up capacity for future tasks.
We argue that the ability to identify the most contributing network parameters in a learned task, plays a critical role to decide which ones to remember.
In this work we propose to rely on Bayesian Networks, which inherently model the distribution of network parameters to formulate a continual learning framework, using the parameter uncertainty for two lifelong learning directions; i.e. 1) eliminating catastrophic forgetting using weight pruning, where a hard selection is made on the most certain weights saved as a task-based binary masks, 2) reducing catastrophic forgetting by adaptively correlating the learning rate of the parameters with their uncertainty allowing for a task-agnostic model which does not require to know when a new task arrives.
We show the beneﬁt of our approach using diverse object classiﬁcation datasets in both cases compared to various baselines.
I NTRODUCTION
Humans can easily accumulate and maintain the knowledge gained from previously observed tasks and continuously learn to solve new problems or tasks.
Inspired by this capability of humans,
Lifelong Learning (LLL) intends to learn sequential tasks without forgetting leaned concepts as new tasks are being introduced.
Unlike the powerful capabilities observed in mammalian brains in acquiring and preserving the knowledge, artiﬁcial agents still struggle in continual learning.
One of the main challenges for a lifelong learning agent is known as catastrophic forgetting (McCloskey & Cohen, 1989;
McClelland et al., 1995) which refers to the signiﬁcant drop in the performance of a learner when switching from a trained task to a new one.
This phenomenon occurs because trained parameters on initial task require to change in favor of learning new objectives.
This is the reason that naive ﬁnetuning intuitively suffers the most from catastrophic forgetting .
Given limited capacity of a network, one way to see this problem as to deciding which part of network is best to keep and which part is less important and can thus be used by future tasks without signiﬁcantly effecting current task performance.
In this work we propose to turn to Bayesian Networks as introduced by Blundell et al.
These networks represent each parameter with a distribution deﬁned by a mean and variance which more naturally allows to determine which parameters are most important for a task.
We integrate this idea with two different LLL paradigms:
Hard pruning of part of the network and soft regularization based importance weighting of network parameters.
To avoid adding capacity when learning new tasks, one can ﬁrst prune or compress the network (Liu et al., 2015; Aghasi et al., 2017) and then learn new tasks using free parameters as proposed by (Mallya & Lazebnik, 2017; 2018).
Here a part of the network is frozen, while the remaining network is free to adapt for future tasks.
While this eliminates forgetting over time, it reduces ﬂexibility when learning large number tasks as the network ﬁlls up over time as the percentage which is frozen by previous tasks increases over time.
The second paradigm we study is regularization based LLL where forgetting is reduced by introducing a loss which penalizes when important weight parameters are changed while learning new tasks.
In contrast to the hard pruning this is “softer” way to avoid forgetting.
The main challenge for the approaches in this domain is how to deﬁne and measure importance.
Different techniques have explored different quantities to measure importance with Kirkpatrick et al.
; Zenke et al.
(2017); Aljundi et al.
While we go over more details about these methods in section 4.2 we argue that the externally deﬁned importance deﬁnition used in these woks might yield sub-optimal regularization performance.
Moreover, importance computation in these methods requires an extra computational step (whether online or ofﬂine).
Despite of the difﬁculties in deﬁning and measuring importance, these approaches have the beneﬁt of good scalability as well as high efﬁciency.
Most of the work in lifelong learning have been proposed within the realm of non-Bayesian models where model parameters tend to overconﬁdently ﬁt to training data and there is no internal deﬁnition of weight importance.
In contrast, Bayesian models, enhanced by their statistically-grounded theoretical framework, provide clear deﬁnitions of uncertainty in model weights and/or predictions.
Although exact Bayesian inference remains intractable in neural networks, variational approximations have provided a powerful alternative to this problem.
In this work we exploit an efﬁcient back-propagation-compatible Bayesian formulation in neural networks, called Bayes-by-Backpop (BBB) Blundell et al.
to obtain weights’ uncertainty as a guidance to perform continual learning.
We show how lifelong learning using pruning and regularization can be enhanced when accurate weights importance are present to identify task-speciﬁc parameters.
The main intuition behind is that Bayesian neural network provide precise uncertainty measurements in weights which is the bottleneck for regularization methods.
R ELATED W ORK
Here we list some of the previously introduced approaches taken in lifelong learning that mitigate catastrophic forgetting to different extents.
Conceptually, these approaches can be divided into the following categories.
We list pros and cons for each and explain why (not) our approach is inspired or build upon them.
Dynamic architectural methods : In this setting the architecture will grow such that it carries over the past knowledge as well as storing new gained knowledge in different forms such as additional layers, nodes, or dual-architecture.
In this approach the objective function remains ﬁxed whereas the model capacity grows often exponentially with the number of tasks.
Progressive networks Rusu et al.
; Schwarz et al.
(2018) was one of the earliest work in this path which was successfully applied on reinforcement learning problems where the base architecture is duplicated and lateral connections are added in response to new tasks.
Roy et al.
(2018) also uses hierarchical network, where CNNs at multiple levels, could grow in tree-shape manner to accommodate new tasks.
Dynamically Expandable Network (DEN)
Yoon et al.
(2018) also expands the network by selecting the drifting units and retraining them separately based on the current tasks in hand.
Despite the ability of these methods in preventing forgetting, they have large overheads upon arrival of new tasks.
We have completely avoided using architectural growth in our method due to this reason.
Memory-based methods: In this regime, previous information is partially stored in different fashions to be used later as a form of rehearsal Robins (1995).
GeppNet Gepperth & Karaoguz (2016) and its variant GeppNet-STM were introduced in this category for class incremental learning settings.
They use a self-organizing map (SOM), a linear regression layer, to re-structure the input data topographically.
Once the SOM is trained, it will be only updated when a an input belong to a new class arrive.
This example is stored by a short-term memory buffer to prevent forgetting it in the future.
iCaRL Rebufﬁ et al.
uses this idea to perform class incremental learning through nearest-mean-of-exemplars using current data and stored examples of previously seen classes.
It was shown to be capable of incrementally learning large scale datasets such as ImageNet.
However, this method can not be used in learning multiple sequential tasks from different datasets.
Gradient episodic memory (GEM) (Lopez-Paz et al., 2017) was another work for class incremental learning where data is stored at the end of each episode to be used later to restrict gradient updates from deviating away from their previous values.
GEM is not only able to mitigate forgetting, but also allows for positive backward knowledge transfer which is referred to improvement on previously learned tasks.
Another value of this work was that it was the ﬁrst method capable of learning with seeing each training example only once.
Since storing examples can be costly and sometimes unpractical, a generative model was employed to encode/decode pseudo information Draelos et al.
Shin et al.
; Nguyen et al.
However, in practice generative models are difﬁcult to train specially in sequential learning scenarios.
Regularization methods : These approaches do not require storing data while retaining the whole network.
Catastrophic forgetting is alleviated via preventing signiﬁcant changes to representation learned for previous tasks.
This can be performed through regularizing the objective function, or directly applied on weight parameters such that important weights to one task are not allowed to change in the future.
Various methods have been explored in both weight importance measurements as well as regularization techniques.
Elastic weight consolidation (EWC) method Kirkpatrick et al.
(2017) imposes a quadratic penalty on changes in weights that are found to be important to prior tasks.
However, the importance is measured through computing the diagonal of the Fisher information metric over parameters of the old task resulting in inefﬁciency on high-dimensional output spaces.
Synaptic Intelligence (SI) suggested an online way of computing importance which had comparable results to EWC while being more efﬁcient.
Similar to SI, Memory-aware Synapses (MAS)
Aljundi et al.
(2017) proposed an online way of computing importance adaptive to the test set using the change in the model outputs w.r.t the inputs.
PackNet Serr`a et al.
(2018) used iterative pruning to fully restrict gradient updates on important weights via saving binary masks.
While applying this binary mask at inference results in zero forgetting of the initial performance, it requires to know which task we are performing to use the appropriate mask.
PackNet also ranks the weight importance by their magnitude which is not guaranteed to be a proper importance indicative.
A follow-up work to PackNet Mallya & Lazebnik (2018) suggested learning these binary masks end-to-end using a meta-learner.
B ACKGROUND : V ARIATIONAL B AYES - BY -B ACKPROP
In this section we review the Bayesian framework we use to be able to learn sequential tasks by remembering task-speciﬁc parameters.
The Bayesian formulation used here was ﬁrst introduced by
Blundell et al.
(2015) that learns the probability distribution over network parameters.
In the original paper, it was shown that this novel back-propagation-compatible algorithm acts as a regularizer and yields comparable performance to dropout on MNIST dataset.
In the next section (Section 4) we show how to use the derived weight uncertainty to measure the importance in performing the task in hand and use the rest of the network to learn later tasks with.
3. V ARIATIONAL INFERENCE IN NEURAL NETWORKS
In Bayesian models, latent variables are drawn from a prior density p ( w ) which are related to the observations through the likelihood p ( x | w ) .
During the inference, the posterior distribution p ( w | x ) is computed conditioned on the given input data.
However, in practice this probability distribution is intractable and is often estimated through approximate inference.
Markov chain Monte Carlo (MCMC) sampling (Hastings, 1970) has been widely used and explored for this purpose (see Robert & Casella (2013) for different methods under this category).
However, MCMC algorithms, despite providing guarantees in ﬁnding asymptotically exact samples from the target distribution, they are not suitable for large datasets and/or large models as they are bounded by speed and scalability issues.
Alternatively, Variational inference provides a faster solution to the same problem in which the posterior is approximated using optimization rather than being sampled from a chain Peterson (1987); Hinton & Van Camp (1993); Jaakkola & Jordan (1996; 1997).
Similar to any other approximation algorithms, there will be trade-offs between the precision of the results and the efﬁciency of the optimization.
However, variational inference methods always take advantage of fast optimization techniques such as citestochastic methods, distributed methods, which allows them to explore data models faster to ﬁnd the best approximation.
See Blei et al.
(2017) for a complete review of the theory.
3. B AYES BY B ACKPROP
Let x ∈ R n be a set of observed variables and w be a set of latent variables parametrized by θ sharing a joint density of p ( w , x ) .
Given the observations x and by using the set of weight parameters w , a neural network, as a probabilistic model, p ( y | x , w ) can output a probability associated with a possible output y ∈ Y .
Variational inference aims to calculate this conditional probability distribu- tion over the latent variables by ﬁnding the closest proxy to the exact posterior through solving an optimization problem.
We ﬁrst propose a family D of probability densities over the latent variables ( q ( w | θ ) ∈ D ) .
We then ﬁnd the closest member of this family to the true conditional probability of interest p ( w | x ) by minimizing the Kullback-Leibler (KL) divergence between q and p :
[Equation]
Once solved, q ( · ) would be the closest approximation to variational posterior parametrized by θ ∗ to construct the parameters of the network.
We can unpack Eq.
and call it a loss function commonly known as variational free energy or expected lower bound :
[Equation]
Blundell et al.
(2015) showed that Eq.
2 can be approximated using i Monte Carlo samples from the variational posterior:
[Equation]
We assume q ( w | θ ) to have a Gaussian pdf with diagonal covariance.
θ is parametrized point-wise with a mean µ and standard deviation of σ = log(1 + exp( ρ )) .
Variational posterior parameters will have a form of µ + log(1 + exp( ρ )) ◦ (cid:15) where (cid:15) is a sample drawn from posited q .
For the prior, as it was suggested in Blundell et al.
(2015), a scale mixture of two Gaussian pdfs are chosen which are zero-centered while having difference variances of σ 21 and σ 22 .
The uncertainty obtained for every weight parameter, as opposed to hidden units Rezende et al.
(2014); Kingma & Welling (2013) has been successfully used in model compression Blundell et al.
(2015), uncertainty-based exploration in reinforcement learning application Blundell et al. (2015); Lipton et al. (2016).
In this work we want to use this framework to learn sequential tasks without forgetting using per-weight derived uncertainties.
L IFELONG LEARNING USING B AYES BY B ACKPROP ( BLLL )
Unlike regular deep neural networks, in BBB model, weight parameters are represented by probability distributions parametrized by their mean and standard deviation.
In order to take into account both mean and variance we use the quantity of signal-to-noise ratio (SNR) for each parameter deﬁned as
[Equation]
SNR is a commonly used measure in signal processing to distinguish between “useful” information from unwanted noise contained in a signal.
In the context of neural models, SNR can be thought as an indicative of parameter importance; the higher the SNR, the more effective or important the parameter is to the model predictions for a given task.
In our method called, Bayesian Lifelong Learning ( BLLL ), we have considered two common scenarios of lifelong learning: 1) sequential learning of multiple datasets; 2) class-incrementally learning of a single or two similar datasets together.
We perform continual learning using weight pruning ( BLLL-PR ), and adaptive scaled gradient update method based upon the Bayesian uncertainty deﬁned per weight parameters.
4. BLLL WITHOUT FORGETTING USING WEIGHT PRUNING ( BLLL-PR )
Weight pruning is a long standing solution to reduce inference computation or network compression (Liu et al., 2017; Molchanov et al., 2016).
Moreover, it has been recently used to perform lifelong learning (Mallya & Lazebnik, 2017), where the goal is to sequentially learning multiple tasks using a single network’s capacity.
In (Mallya & Lazebnik, 2017) this has been accomplished by freeing up parameters deemed to be unimportant to the current task according to their magnitude.
Forgetting is prevented in pruning by saving a task-speciﬁc binary mask of important vs. unimportant parameters.
Require: hyper parameters for BBB-only: σ , σ 1 , σ 2 , ρ , µ , π , n
Require: hyper parameters for training: learning rate α 1: (cid:15) ∼ N (0 , I ) 2: σ = log(1 + exp( ρ ))
(cid:46) Ensures σ is always positive 3: w = t ( θ, (cid:15) )
=
µ + σ ◦ (cid:15) (cid:46) w := a posterior sample of weights 4: while not done do 5: l 1 ← (cid:80) ni
=1 log N ( w i | µ, σ 2 ) (cid:46) log-posterior 6: l 2 ← (cid:80) ni
=1 log (cid:0) π N ( w i | 0 , σ 21 ) + (1 − π ) N ( w i | 0 , σ 22 ) (cid:1) (cid:46) log prior 7: l 3 ← (cid:80)
ni =1 log( p ( x | w ))
(cid:46) log-likelihood of the data 8:
L BBB ← 1 M ( l 1 − l 2 ) − l(cid:46) M := number of minibatches 9: if BLLL-PR then 10:
SNR i ← | µ i | σ i (cid:46) signal-to-noise ratio for every weight 11: for all layers do 12: for all parameters in layer do 13: if SNR i ∈ Top p portion of SNR values in layer then 14: α =15: µ (cid:48) ← µ − α ∇L BBB µ (cid:
) Update parameter using gradient descent 16: ρ (cid:48) ← ρ − α ∇L
BBB ρ (cid:46) Update parameter using gradient descent 17: if BLLL-LR then 18: β ← σ × α 19: µ (cid:48) ← µ − β ∇L BBB µ ρ (cid:48) ← ρ − α ∇L BBB ρ (cid:46)
Update the remaining uniformly using gradient descent Our work is signiﬁcantly different from (Mallya & Lazebnik, 2017) in the different criterion we propose to use for importance measurement; the statistically-grounded uncertainty deﬁned in Bayesian neural networks.
BLLL-pruning is performed as follows: for every layer, convolutional or fully-connected, the parameters are ordered by their SNR value and those with the lowest importance are pruned (set to zero).
The pruned parameters, are “saved” using a binary mask so that they can be used later in learning new tasks whereas the important parameters remain ﬁxed throughout training on future tasks.
Once a task is learned, an associated binary mask is saved which will be used at inference to recover key parameters to the desired task.
The overhead memory caused by saving the binary mask is negligible given the fact it completely eliminates the forgetting by recovering the exact performance achieved on a previously learned task.
4. BLLL WITH REDUCED FORGETTING
Another strategy to perform continual learning is to reduce forgetting by controlling further changes in a representation learned for a task.
This can be enforced in a form of regularization in the objective function or through a varying learning rate for weight parameters.
Adaptive learning rate ( BLLL-LR ): In order to preven drastic changes to the predicted posterior for a learned task, we propose to condition the changes in the mean of each parameter’s distribution on its uncertainty i.e., once a task is learned, the more certain a mean of a distribution is, the less it should be updated when a new task arrives.
We impose this through means of the parameters distributions, µ i , to be adaptive to their uncertainty represented by σ i .
The key beneﬁt of the learning rate-based Bayesian continual learning is that it neither requires additional memory, as opposed to pruning technique, nor tracking the change in parameters with respect to the previous learned task, as needed in the weight regularization methods.
More importantly, this method does not need to be aware of task switching as it only needs to adjust the learning rates of the means in the posterior distribution based on their current standard deviations.
Complete algorithm for BLLL with pruning ( BLLL-PR ), regularization ( BLLL-Reg ) and, adaptive learning rate ( BLLL-LR ).
is shown in
Algorithm 1.
E XPERIMENTAL S ETUP
Datasets: Toy benchmarks such as MNIST(LeCun et al., 1998) and CIFAR10/100 (Krizhevsky &
Hinton, 2009) have been widely used to measure forgetting and accuracy of various lifelong learning approaches.
As a result, we include them as part of our evaluation.
We use MNIST split and permuted MNIST (Srivastava et al., 2013) for class incremental learning with similar experimental settings used in Serr`a et al.
Lopez-Paz et al.
(2017) in terms of data splits.
Furthermore, to have a better understanding of our method, we also evaluate our approach on sequentially learning of various datasets belonging to different distributions similar to the sequence of 8 tasks used in (Serr`a et al., 2018) including FaceScrub (Ng & Winkler, 2014), MNIST, CIFAR100, Not NotMNIST (not),
SVHN (Netzer et al., 2011), CIFAR10, TrafﬁcSigns (Stallkamp et al., 2011), and FashionMNIST (Xiao et al., 2017).
Details of each are summarized in Table 1.
In all the experiments we resized images to 32 × 32 × 3 if necessary.
For datasets with monochromatic images, we replicate the image across all RGB channels.
No data augmentation of any kind has been used in our analysis.
[Table]
Baselines: Within Bayesian framework, we have three reference baselines of ﬁne-tuning, feature extraction, and joint training.
In ﬁne-tuning ( BLLL-FT ), training with regular SGD optimization continues upon arrival of new tasks without any forgetting avoidance strategy.
Feature extraction, denoted as ( BLLL-FE ) in our experiments, refers to freezing all layers in the network except for the last layer when a new task arrives.
In joint training ( BLLL-JT ) we learn all the tasks in a multitask learning fashion which serves as the upper bound for average accuracy on all tasks.
We have compared these three Bayesian references with their counterparts using an ordinary (non-Bayesian) network referred to as ORD-FT , ORD-FE , and ORD-JT .
We have compared them on Split MNIST class incremental learning experiments.
Along with the BLLL variants of reference baselines, we compare with the state-of-the-art approaches including EWC Kirkpatrick et al.
IMM (Lee et al., 2017), learning without forgetting (LWF) (Li & Hoiem, 2017), less-forgetting learning (LFL)
(Jung et al., 2016), PathNet (Fernando et al., 2017), PNNs (Rusu et al., 2016), and Hard attenstion mask (HAT) (Serr ` a et al., 2018) using implementations provided in (Serr ` a et al., 2018).
On Permuted MNIST dataset average accuracies are directly reported from GEM (Lopez-Paz et al., 2017),
SI (Zenke et al., 2017), and VCL (Nguyen et al., 2017) without re-implementation.
Network architecture:
For Split MNIST and Permuted MNIST experiments we have used a multilayer perceptron with three hidden layers with 660 units whereas other baselines use a two-layer
MLP with 800 units in each.
Because there exists double number of parameters in our bayesian network compared to its equivalent regular neural net, we ensured fair comparison by matching the total number of parameters between the two to be 1 .
9M unless otherwise is stated.
For multiple datasets learning scenario as well as alternating incremental CIFAR10/100 datasets we have used a ResNet18 Bayesian network with 7 . 1 - 11 .
3M parameters depending on the experiment.
Note that the majority of the baselines provided in this work were originally developed using some variants of
AlexNet structure.
We tried change that to ResNet type of network, however, it resulted in degrade in their reported and experimented performance.
On the other hand, best results for our Bayesian network were achieved using ResNet18.
Therefore, we kept the architectures as is and only matched their number of parameters to ensure having equal capacity across different approaches.
Hyperparameter tuning: We have used the approach introduced in (Anonymous, 2019) speciﬁc to continual learning setting for hyperparameter tunning.
Unlike commonly used tuning techniques which use a validation set composed of all the classes in the dataset, this method realistically assumes to have access to a few classes only on which we can tune the hyperparameters on a held-out validation set.
In all our experiments we consider 0 .
15 split for validation set on the ﬁrst two tasks in all the learning sequences.
Training details: It is important to note that in all our experiments, no pre-trained model is used.
We used stochastic gradient descent with learning rate of 0 .
01 decaying by a factor of 0 .
3 upon being plateau for 5 consecutive epochs.
Batch size of 64 was consistently used throughout all experiments.
Dataset splits and batch shufﬂe has been performed identically across BLLL experiments and all the baselines.
Performance measurement: Let n be the total number of tasks.
Once all are learned, we evaluate our model on all n tasks.
ACC is the average test classiﬁcation accuracy across all tasks.
To measure forgetting we report backward transfer, BWT, which indicates how much learning new tasks has inﬂuenced the performance on previous tasks.
While BWT < 0 directly reports catastrophic forgetting , BWT > 0 indicates that learning new tasks has helped with the preceding tasks.
In mathematical notation BWT and ACC are as follows:
[Equation]
where R i,n is the test classiﬁcation accuracy on task i after sequentially ﬁnishing learning the n th task.
R ESULTS & D ISCUSSION
We present our results for class incremental learning of single datasets (Split MNIST and Permuted
MNIST) as well as sequential learning of 8 distinct tasks using variant of BLLL approach compared against various recent lifelong learning approaches.
6. S PLIT MNIST
We ﬁrst present our results for class incremental learning of MNIST (Split MNIST) in which we learn 0 − 9 digits in two tasks with 5 randomly shufﬂed classes at a time.
All presented methods in this setup have 1 . 9M parameters.
Figure 1a shows the results for reference baselines in
Bayesian and non-Bayesian networks including ﬁne-tuning ( BLLL-FT , ORD-FT ), feature extraction ( BLLL-FE , ORD-FE ) and, joint training ( BLLL-JT , ORD-JT ) averaged over 5 runs.
Although
MNIST dataset is too easy to draw broad conclusions on it, we observe throughout all experiments that Bayesian ﬁne-tuning and joint training, perform signiﬁcantly better than their counterparts
ORD-FT and ORD-JT that use a regular network where forgetting in BLLL-FT is comparable and even better than some of the baselines presented in literature.
We also compare against HAT (Serr`a et al., 2018) and LWF (Li & Hoiem, 2017) where HAT has similar performance to our BLLL-PR with zero forgetting while BLLL-LR is able to increase the ACC by 2% at the cost of 0 .
1% drop in forgetting.
Figure 2a shows how the accuracy carries from the ﬁrst half of the dataset to the second half.
Ordinary ﬁne-tuning is the only reference method that experience a performance boos upon change in the task whereas both BLLL-FE and ORD-FE degrade after new task arrives which is fully expected.
All other baselines as well as our BLLL invariants steadily perform above ≈ 99% on both tasks.
6. P ERMUTED MNIST
Permuted MNIST is a popular variant of MNIST dataset to evaluate continual learning approaches in which each task is considered as a random permutation of the original MNIST pixels.
Following the literature, we learn a sequence of 10 random permutations and report average accuracy at the end.
Figure 1b shows ACC and BWT performance of BLLL-LR and BLLL-PR against that of the state of the art models.
We can again observe that BLLL-FT , despite having no intention in preventing forgetting, exhibits reasonable negative BWT values which makes it a successor over
IMM (Lee et al., 2017) and LWF (Li & Hoiem, 2017) baselines.
While HAT (Serr`a et al., 2018) and
BLLL-PR have a similar ACC performance with no forgetting, BLLL-LR shows a slight increase in the tendency to forget ( − 0 .
8% ) in exchange for a negligible boost in ACC ( 0 . 4% ).
Accuracy evolution over 10 tasks is shown in 2b.
[Figure]
[Figure]
[Figure] [Figure] [Figure] [Figure]
[Figure] [Figure]
[Figure]
[Figure] [Figure]
[Figure]
CIFAR10/
[Figure]
n c e
[Figure] [Figure]
[Figure] [Figure]
(d
[Figure]
Multiple datasets ( 8 tasks)
Results in Fig.
1b are computed on a 1 . 9M network.
However ACC for this experiment is also reported in HAT (Serr ` a et al., 2018), VCL (Nguyen et al., 2017), EWC (Kirkpatrick et al., 2017), SI (Zenke et al., 2017), and GEM(Lopez-Paz et al., 2017) using a smaller MLP with 0 .
1M parameters.
ACC for these approaches are given as 91 . 0% , 90% , 88 .
6% , respectively.
However,
BLLL-LR achieves 92 .
2% with BWT of − 0 .
4% under the same conditions.
6. A LTERNATING CIFAR10 AND CIFAR
In this experiment we alternate between class incrementally learning of CIFAR10 and CIFAR100.
Both datasets are divided into 5 tasks each with 2 and 20 classes per task, respectively.
Figure 1c presents ACC and BWT obtained with BLLL-PR , BLLL-LR , and three BLLL reference methods compared against various continual learning baselines.
It is interesting to note that in this setup, some baselines (LWF and PathNet) do not perform better than the naive accuracy achieved by feature extraction .
LFL is able to marginally outperform feature extraction in ACC at the cost of twice of the forgetting we can have in ﬁne-tuning while IMM performs only as well as ﬁne-tuning in both
ACC and BWT.
PNN, EWC, and HAT are the only baselines that perform between the minimum expected ACC ( BLLL-FE ) and maximum acceptable negative BWT ( BLLL-FT ) bounded by the high ACC and near zero forgetting achieved in joint training ( BLLL-JT ) for this setup.
Among the three acceptable baselines, PNN is the only zero-forgetting-guaranteed approach and hence can be properly compared with our BLLL-PR which also never forgets by deﬁnition.
Note that PathNet is also a zero-forgetting approach which suffers from bad pre-assignment of network’s capacity per task which causes poor performance on the beginning task from which it never recovers.
BLLL-PR consistently outperforms PNN by overall 10% better ACC where the gap between them is more pronounced on CIFAR100 tasks which are more difﬁcult compared to binary classiﬁcation tasks of CIFAR10.
EWC and HAT are both allowed to forget by construction, however HAT shows zero forgetting behavior.
While EWC is outperformed by both of our BLLL variants, HAT exhibits 1% better ACC over BLLL-PR .
BLLL-LR is able to compensate for that at a cost of 0 .
7% drop in forgetting.
A trade off between ACC and BWT exists when choosing between BLLL-PR and
BLLL-LR depending on what is needed in the problem.
6. M ULTIPLE DATASETS LEARNING
Finally we present our results for sequentially learning of 8 tasks using BLLL-PR and BLLL-LR .
Similar to the previous experiments we look at both ACC and BWT obtained for BLLL-PR ,
[Figure]
not able to do better than BLLL-FT including LFL, PathNet, LWF, IMM, and EWC.
PNN and HAT remain the only considerable baselines for our BLLL-PR and BLLL-LR approaches.
BLLL-PR continues to outperform PNN by 5% in ACC while the two ensure zero forgetting.
HAT exhibits − 0 .
1% BWT while being overcome by BLLL-LR in ACC by 3% .
We believe the main insight of our Bayesian approach is that instead of computing extra measurements of importance, which are often task, input or output dependent, we are able to use the predicted weights uncertainty as a bi-product of the framework to deem the most contributing parameters in the overall posterior distribution and choose to freeze them using a binary mask, as in BLLL-PR , or to enforce minimum changes in them conditioned on their current uncertainty, as in BLLL-LR , allowing for more capacity to learn with and most importantly developing a task-agnostic model that does not require to be warned of task arrival.
This makes our approach an excellent candidate for applications in which such information is inevitably unavailable.
C ONCLUSION
In this work we propose a lifelong learning formulation for Bayesian networks, called B-LLL , that uses the Bayesian-embedded uncertainty predictions as a guidance to perform lifelong learning; i.e. to identify important parameters that can be either fully preserved through a saved binary mask ( BLLL-PR ) or allowed to change conditioned on their uncertainty for learning new tasks ( BLLL-LR ).
We demonstrated how the probabilistic uncertainty distributions per weights are helpful in sequentially learning short and long sequences of benchmark datasets compared against many other approaches.
We showed that both BLLL-PR and BLLL-LR perform superior or marginally better than the state-of-the-art models across all the experiments.
In order to choose between the two one should know that BLLL-PR results in zero forgetting while requiring a small binary mask to be saved per task whereas BLLL-LR is memory-free and allows for more capacity in the network by allowing small forgetting to occur.
The key beneﬁt from BLLL-PR is that the model no longer need to know when tasks are being switched leading to a task-agnostic model that can be deployed where it is not possible to distinguish tasks in a continuous stream of the data.