Can Sequential Bayesian Inference Solve Continual Learning?
Abstract
Previous work in Continual Learning (CL) has used sequential Bayesian inference to prevent forgetting and accumulate knowledge from previous tasks.
A limiting factor to performing
Bayesian CL has been exact inference in a Bayesian Neural Network (NN). We perform sequential Bayesian inference with a Bayesian NN using Hamiltonian Monte Carlo (HMC) and propagate the posterior as a prior for a new task by ﬁtting a density estimator on HMC samples.
We ﬁnd that this approach fails to prevent forgetting.
We propose an alternative view of the CL problem which directly models the data generating process and decomposes the CL problem in task speciﬁc and shared parameters.
This method named Prototypical Bayesian CL and performs well compared to the latest Bayesian CL methods.
1. Introduction
In Continual Learning (CL) a Neural Network (NN) predictor is required to solve one task then another sequentially while maintaining performance on previously mastered tasks.
One key challenge of CL is that NN weights from previously mastered tasks get overwritten when learning a new task and as such causes catastrophic forgetting of previous tasks
French (1999).
One way to prevent forgetting hinges on using recursive applications of
Bayes’ rule; using a Bayesian NN’s posterior as a prior for a new task Husz´ar (2017); Kirkpatrick et al.
Obtaining the posterior over NN weights is intractable and we need to resort to approximations of the posterior such as using a Laplace approximation MacKay (1992) or variational inference Graves (2011);
Blundell et al.
(2015) to obtain a posterior to subsequently use as a prior for learning a new task.
When performing Bayesian CL and using an approximate inference method such as a
Laplace approximation Schwarz et al.
; Ritter et al.
(2018) or variational approximation Nguyen et al.
; Ebrahimi et al.
; Kessler et al.
; Loo et al.
(2020) most implementations will use a multi-head NN architecture.
By multi-head we mean that there is a shared feature extractor which takes in some input x ∈ X for all tasks g : x → z which maps to some embedding space z ∈ Z and then there is a head per task
h
i : z → ˆ y where ˆ y is a prediction and i ∈ { 1 , . . .
, T } , each head could be a logistic regression in the case of binary classiﬁcation.
So each head is looked up when predicting each past task and each head preserves task speciﬁc information make Bayesian CL easier to perform.
Indeed past works have noted this implementation “trick” and suggested using a single head , h as a more challenging benchmark Farquhar and Gal (2018); Van de Ven and Tolias (2019); Hsu et al.
When we compare performance using variational continual learning (VCL)
Nguyen et al.
and see that for Split MNIST that there is a drop in performance for all tasks after having learned on a particular task: this is forgetting.
What we also notice is that there is a diﬀerence in performance when using a multi-head approach versus a singlehead approach, Figure 1.
This throws up a set of interesting questions.
If we had access to the true posterior of a single-head BNN would this be enough to prevent forgetting and would the gap in performance between single-head and multi-head BNNs be closed?
Do we need to have some task speciﬁc parameters to prevent forgetting and do we need to deviate from the Bayesian CL framework by doing so?
Our contributions in this paper are to consider the above questions.
We propagate the true posterior of a BNN and use this as a prior for a new task.
We do this by using
Hamiltonian Monte Carlo (HMC) and use a density estimation on the samples from HMC and using this approximate density or posterior as a prior for a new task within the HMC sampling process.
This yields no noticeable beneﬁts over VCL.
We argue that the current formulation of Bayesian CL cannot prevent Bayesian inference from weighting the likelihood over the prior and so yielding forgetting, similarly to the Kalman ﬁlter (see Section 9).
We then propose a new probabilistic model for CL and show that by explicitly modeling the generative process of the data we can achieve good performance rather then relying solely on propagating the posterior from a previous task to prevent forgetting.
2. The Continual Learning Setup
Continual learning (CL) is a setting whereby a model must learn a set of tasks sequentially, while maintaining performance across all tasks.
In CL, the model is shown a set of T tasks sequentially T t for t = 1 , . . .
T , ( T can equal ∞ ).
Each task is comprised of a dataset such that T t : D t = { ( x i , y i ) } for i = 1 , . . .
, N t .
Although the model will lose access to the training dataset for task T t , it will be continually evaluated on all previous tasks T i for i ≤ t .
The id t can be used as a task identiﬁer informing the agent when to start training on a new task or what task to test.
For a comprehensive review of CL scenarios see Hsu et al.
(2018); Van de Ven and Tolias (2019).
3. The Current Formulation of Bayesian Continual Learning
We consider the situation in which data arrives sequentially at times, t = 1 , 2 , . . .
, T .
For the ﬁrst time step, t = 1, the model receives the dataset D 1 and the aim is to learn a model (a neural network) with parameters θ with prior p ( θ ) of the conditional distribution p ( y i | x i , θ ) for all ( x i , y i )
∈ D 1 .
Then the predictive distribution for a test point, x ∗ 1 is:
[Equation]
We note that the above requires knowledge of the posterior over parameters p ( θ |D 1 ).
For t = 2, the model is required to ﬁt p ( y i | x i , θ ) for D 1 ∪ D 2 .
The predictive distribution for a
The the ﬁrst task, the images for { 0 , 1 } are mapped to the classes { 0 , 1 } .
In the second task the images for { 2 , 3 } are mapped { 0 , 1 } .
We compare a naive NN trained with SGD (singleheaded) to VCL.
VCL is evaluated on the single-headed variant (SH) and the multi-headed variant (MH).
new test point x ∗ 2 point is:
[Equation] Thus we need to update our posterior to be able to learn this new conditional distribution.
We can use repeated applications of Bayes’ rule to calculate the posterior distribution p ( θ |D 1 , . .
, D T ), namely:
[Equation]
To aid tractability one usually resorts to approximate inference e.g. a Laplace approximation or variational inference.
In CL we loose access previous training datasets, but using repeated applications of
Bayes’ rule, Equation (3) this limitation can be circumvented.
Consider at t = 2 were we require a posterior p ( θ |D 1 , D 2 ) to calculate the predictive distribution in Equation (2), however we have lost access to D 1 , with Bayes’ rule the posterior takes the form:
[Equation]
Hence we can use the posterior at t as a prior for learning a new task at t +1.
Observation: we require that our model with parameters θ is a suﬃcient statistic of D 1 hence the motivation for using complex BNN predictors; a Bayesian linear regression model will be unable to model images datasets.
The last term is included since without conditioning on θ , the data sets D 1 and D 2 are no longer independent Husz´ar (2017).
3.1. Variational Continual Learning
CL can be decomposed into approximate sequential Bayesian updates, Variational CL (VCL)
Nguyen et al.
(2018) uses a variational BNN to perform the prediction tasks where the network weights are independent Gaussians.
The variational posterior from previous tasks is used as a prior for new tasks.
Consider learning the ﬁrst task T 1 , and θ are the variational BNN weights then the variational posterior is q 1 ( θ |D 1 ).
For the subsequent task the prior will be q 1 ( θ |D 1 ), optimization of the ELBO will yield the variational posterior q 2 ( θ |D 2 ).
Generalizing, the negative ELBO for the t -th task is:
[Equation]
When applying VCL to the problem of Split MNIST, Figure 1 we can see that singlehead (SH) VCL performs worse than multi-head VCL, despite following more closely the process of Bayesian CL described in Section 3.
Observation: using task speciﬁc parameters helps in preserving knowledge from previous tasks.
One confounder could be that since we are propagating a variational approximation of the posterior then we are actually not performing the Bayesian CL process described earlier.
Next we will try to propagate the true posterior of our BNN.
3.2. Propagating the True Posterior
To perform Bayesian inference of our BNN we use HMC which produces samples Neal et al.
(2011); Cobb and Jalaian (2021), we then use these samples and learn a density estimator which we can be used as a prior for a new task.
This resembles sequential Bayesian inference more closely than performing approximate inference and propagating a diagonal Gaussian posterior like in VCL.
Consider T 1 , to then propagate the posterior p ( θ |D 1 ) we use a density estimator to ﬁt a probability density on the samples to obtain ˆ p ( θ |D t ).
For the next task
T 2 we can use our density estimate ˆ p ( θ |D t ) as a prior for a new HMC sampling chain for a new task p t +1 ( θ ) :
= ˆ p ( θ |D t )
≈ p ( θ |D t ).
The density estimators we consider are a mixture of Gaussians (GMM) (we also tried using Normalizing Flows,
RealNVP Dinh et al.
(2016) which did not work robustly).
See Figure 2 for a diagram of this procedure.
The density estimator priors have two conditions required for use within the HMC sampling procedure.
1) That they are a valid probability density function 2) that they are diﬀerentiable with respect to the inputs, i.e. samples from the BNN which we want to create a density estimator for.
We construct a toy dataset of 10 classes where each class is generated according to a
Gaussian distribution arranged in a circle about the origin, x ∈ X ∈ R 2 and each class visibly separable.
Each task will require that we perform a binary classiﬁcation of each of
5 Task trained on 0.
A v e r a g e
A cc u r a c y
HMC
MH VCL
SH VCL
SGD
MT SGD/HMC two adjacent classes for 5 tasks, see Figure 4 for a visualization of the data.
We train alayer BNN with hidden state size of 10, (training an MLP on the same multi-task dataset gets an accuracy of 1 . 0).
For T 1 we set the prior p 1 ( θ ) =
N (0 , τ − 1 1 ) with τ = 10.
We burn-in the HMC chain for 1000 steps and sample for 5000 more steps, we run 20 diﬀerent chains to obtain samples from our posterior which we then pass to our density estimators.
We use a step size of 0 . 001 and trajectory length of L = 20. We then assess convergence and eﬀective sample size; all chains converge and the eﬀective sample size of the samples from all chains is similar to other BNNs in Cobb and Jalaian (2021).
When using a GMM to provide a density estimate for the HMC samples we optimize for the number of components by using a holdout set of samples from the HMC sampling.
We also ensure that samples from our density estimator ˆ p ( θ |D t ) provide accurate estimates for the current task T t , see Section 8 for details on the HMC sampling diagnostics.
We obtain the following results in Figure 3 which demonstrate that propagating the posterior in the form of a GMM density estimator fails to prevent forgetting.
Moreover, comparing multihead VCL with single-headed VCL or HMC (also single-headed) we see the multi-head VCL clearly outperform both methods indicating that the source of the knowledge retention is not through the propagation of the posterior but through the task speciﬁc parameters in each head.
Observation: we can conclude that just relying on sequential Bayesian inference is not suﬃcient in preventing forgetting in this setting .
We stress that we are assuming that the density estimator is able to encapsulate the HMC samples with a high degree of ﬁdelity: indeed we are able to sample from the density and recover our BNN performance,
Figure 5. This is backed up by current state of the art probabilistic methods in CL which use inducing points from previous tasks to prevent forgetting Titsias et al.
Pan et al.
4. Prototypical Bayesian Continual Learning
We have shown that solely relying on sequential Bayes cannot ensure that forgetting is prevented and we have shown that modeling the CL problem in terms task speciﬁc and global parameters like in meta-learning (see Section 10) helps prevent forgetting as well.
Hence we model the generative process of CL datasets where new tasks are comprised of new classes of images.
We use a NN to embed images of each class and maintain this embedding as we learn continually.
Each class embedding is modeled as a Gaussian; this is a probabilistic version of Prototypical Networks Snell et al.
(2017) or iCarl Rebuﬃ et al.
(2017) which both create embedding functions for diﬀerent classes which are simply arithmetic means over embeddings and perform classiﬁcation through nearest neighbors.
Our approach borrows from the meta-learning model, probabilistic clustering for online classiﬁcation Harrison et al.
and we re-cast the general generative model for probabilistic clustering in CL.
Model.
The generative model, ProtoCL, the classes are generated from a categorical distribution with a Dirichlet prior:
[Equation]
We learn a joint embedding space for our data with a NN, z = f ( x ; w ) with parameters w .
The embedding space for each class is Gaussian whose mean has a prior which is also
Gaussian:
[Equation]
By modeling a mean embedding for each class we ensure that we are modeling each new task with a speciﬁc set of parameters.
To ensure that the mean embeddings do not drift we need make use of a memory of past task data to prevent forgetting.
In practice we use 1000 data points per task, see Fig 8 for a sensitivity analysis.
This is similar to the most recent approaches in probabilistic CL Titsias et al.
Kapoor et al.
Pan et al.
We can ﬂexibly scale to large vision benchmarks as f ( · ; w ) does the representation learning and can be e.g. a CNN encoder.
For full details with regards to obtaining the log posterior of the embeddings and class labels needed for optimization, the recursive updates for Dirichlet and Gaussian priors and for making predictions see Section 11 for details.
Code to reproduce all experiments is located here https://github.com/skezle/bayes cl posterior.
The purpose of this model is not to say that we can get better results than other baselines, but rather to demonstrate that by modeling the generative process and not blindly relying on sequential Bayesian inference we can obtain strong initial results (Table 4) in comparison to other Bayesian CL methods.
[Table]
5. Discussion
We revisit sequential Bayesian inference which underpins many methods in Bayesian CL.
We show that if we were to follow Eq (5) strictly using HMC samples and assuming our density estimator captures these samples then we are unable to prevent forgetting.
This is not entirely surprising when we revisit the Bayesian ﬁltering literature where sequential Bayesian inference can be used in close form to track data, but is unable to recall past tasks (see Section 9).
Sequential Bayesian updating forgets and hence why previous probabilistic approaches to CL have also relied on multi-head architectures and coresets for instance. This is why we should alternatively focus on modeling the CL problem which is what ProtoCL does in a very simple fashion. Bayes is agnostic to the model, if one’s model is poor then Bayes will still infer the parameters of this model.
Why do some methods which rely on sequential Bayesian inference perform well?
Most work which performs sequential Bayesian inference uses variational inference.
One explanation could be that variational BNNs are known to be sparse due to variational overpruning Trippe and Turner (2018) and so the variational BNNs have more capacity to learn more tasks through Swaroop et al. ; Mallya and Lazebnik (2018).
Is sequential Bayes useless for CL?
No, it provides a form of regularization of the likelihood.
In the case of VI the regularization is explicit in the form of a KL divergence.
However, sequential Bayes on its own is not enough to prevent forgetting and ensuring past task information is not forgotten. 6. Acknowledgments
We would like to thank Sebastian Farquhar and Siddharth Swaroop for interesting discussions.
This material is based upon work supported by the United States Air Force and
DARPA under Contract No. FA8750-20-C-0002.
Any opinions, ﬁndings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reﬂect the views of the United States Air Force and DARPA.
7. The Toy Gaussians Dataset
See Figure 4 for a visualization of the toy Gaussians dataset which we use to demonstrate using a density estimator on HMC samples as a surrogate of the posterior and use this as a prior for a new task.
We construct a simple CL problem of 5 tasks of binary classiﬁcation of data arranged in a ring.
Each task is binary classiﬁcation of two adjacent Gaussian datasets, then the next task is the next two adjacent Gaussians (not including dataset see in D 1 ).
This should enable us to use small BNNs and eﬃciently perform HMC sampling and density estimation over HMC samples.
With just a 2 layer network with 10 neurons as a network can perform binary classiﬁcation over this entire 5 task binary classiﬁcation problem in a multi-task fashion.
Hence we postulate that a BNN with the same size should be able to learn all 5 binary classiﬁcation tasks continually and that the functional approximation capabilities of the network are not a constraint fro learning.
8. Density Estimation Diagnostics
We provide plots to show that the HMC chains which we run to sample from the posterior have converged in Figure 6.
The tasks are the toy Gaussians dataset described in Section 7.
We run 20 HMC sampling chains and randomly select one chain to plot for each seed (of 10).
We run HMC over 10 seeds and aggregate the results.
The posterior p ( θ |D 1 ) is approximated with a GMM and used as a prior for the second task and so forth for tasks
T 2:5 .
Our BNN is of size of 2 layers with Gaussian weights and a hidden state size of 10.
We provide empirical evidence to show that the density estimators have ﬁt to HMC samples of the posterior in Figure 5.
Where we show the number of components of the
GMM density estimator which we use as a prior for a new task, the BNN accuracy when sampling weights from our GMM and the eﬀective sample size (ESS) of the 20 chains which is a measure of how correlated the samples are (higher is better).
These ESS values are in line with previous work which uses HMC for BNN inference Cobb and Jalaian (2021).
9. Bayesian Continual Learning as Filtering
In the Bayesian ﬁltering literature the parameters { θ t } are denoted as a hidden state all relevant information about { θ 0 , θ 1 , . .
, θ k } given observations up to time k is encapsulated
[Figure]
A cc
[Figure]
s a m p l e d f r o m
G MM p o s t
e r i
o r
[Figure]
# G
MM c o m p o n e n t s i n p o s t e r i o r
[Figure]
by the posterior distribution p ( θ 0 , θ 1 , . .
, θ k | y 0 , y 1 , . . .
, y k ). We estimate this posterior distribution recursively in time; in particular the marginal distribution p ( θ k | y 0 , y 1 , . . . , y k ) denoted as the ﬁltering distribution Doucet et al. Bayesian ﬁltering has been used for applications such as target tracking Gordon et al.
or estimating the volatility of a security Pitt and Shephard (1999).
In all but the simplest case of a Gaussian state-space model: Gaussian likelihood and Gaussian parameters (or hidden state), exact inference is intractable; the seminal Kalman ﬁlter relies on sequential Bayesian updates Kalman (1960).
9.1. Sequential Bayesian Estimation of a Single Parameter
Let’s consider an even simpler setting than CL and consider data arriving online from a non-stationary distribution one point after another: y 0 , y 1 , . . . .
The likelihood is p ( y t | θ t ) =
N ( y t ; f ( · ; θ t ) , σ 2 ) such that y t = f ( · ; θ t ) + (cid:15) where (cid:15) ∼ N (0 , σ 2 ) and where the function f ( · ; θ t )
= θ t are simply the parameters.
We consider a Gaussian prior over the parameters θ such that p ( θ 0 ) =
N ( θ 0 ; 0 , σ 2 0 ).
Considering the calculation of our posterior at time t since our prior and likelihood are Gaussian then so is the posterior hence the prior and posterior are N ( θ t − 1 ; ˆ θ t − 1 , ˆ σ 2 t − 1 ) and N ( θ t ; ˆ θ t , ˆ σ 2 t ) respectively.
By using sequential Bayesian inference we can have closed form update equations for our posterior parameters:
[Equation]
We can see from our sequential estimate of the mean that the posterior is a linear combination of the observation the prior.
If the observations are non-stationary like in CL then the mean parameters will start to deviate from the prior and the posterior will eventually track the new distribution of observations.
We show this in the following notebook.
10. Multi-head Continual Learning and Meta-Learning
We perform sequential Bayesian inference with a BNN by using a GMM density estimator to ﬁt the samples of the posterior.
By ﬁtting a mixture of Gaussians as our posterior for task
T t and using it as a prior before performing HMC for task T t
+1 we approach exact sequential
Bayes on small BNNs for a toy dataset.
This is still not exact sequential Bayesian inference, however we show that by sampling from the GMM we can obtain 100% accuracy, hence the GMM is a high ﬁdelity posterior.
By using this procedure show that we are unable to prevent forgetting and multi-headed VCL performs best.
This leads us to conclude that the main mechanism to prevent forgetting is the use of diﬀerent heads for diﬀerent tasks.
The diﬀerent heads are acting to retain task speciﬁc information.
More generally, it is the use of task speciﬁc parameters to enable to preserve previous task predictors.
This is similar to the problem of meta-learning which speciﬁcally decomposes the meta-learning solutions into global shared parameters and task speciﬁc parameters.
In this section we will elaborate on this by exploring gradient-based meta-learning Finn et al.
; Grant et al.
and meta-learning probabilistic inference methods Gordon et al.
; Ravi and Beatson (2018).
The graphical model for the meta-learning is shown in Figure 7, each θ φ t y it x it N t T task speciﬁc parameter φ t is distinct and inﬂuences the prediction of y it given x it , a meta level global parameter θ captures global information about all tasks and inﬂuences φ t .
The goal of meta-learning is to learn θ which generalizes to new unseen datasets.
Gradient based meta-learning.
We learn θ in a meta-learning stage by maximum likelihood.
Consider a task inputs X t = { x ti } N t i =1 , and the set of all datasets from all tasks is { X t } Tt =1 which we denote as { X } in short hand, then without splitting our dataset into support and query sets for ease of notation and employing the factorization in Figurethen:
[Equation]
In MAML Finn et al.
; Grant et al.
the distribution of task speciﬁc parameters p ( φ t | θ ) is approximated by point estimates around ˆ φ t
= θ + α ∇ θ p ( Y t | X t , θ ) and the global parameters θ act as an initialization of the NN parameters for few shot adaptation to obtain ˆ φ t and α is the SGD step-size.
Hence the we can rewrite the marginal likelihood in Eq.
(12) as:
[Equation]
(13) which is the MAML objective.
Meta-learning probabilistic inference.
Instead of a gradient based approach to meta-learning we can meta-learn an amortization network which produces task speciﬁc parameters Gordon et al.
; Ravi and Beatson (2018).
For a test point x , a prediction is governed by p ( y | x, φ t , θ ) where the task speciﬁc parameters are generated by an amortized variational inference φ t ∼ q ψ ( φ |D t , θ ): a feed forward network with parameters ψ is learned to output task speciﬁc parameters given training D t
= { ( x ( t ) n , y ( t ) n ) } N t n =1 .
In practice a feature extractor h
θ ( x ) is shared across all tasks and feeds into a task speciﬁc linear classiﬁer head, φ t , this is learned in an end-to-end manner backpropagating back to θ and ψ via the objective:
[Equation]
in practice this can be optimized via Monte-Carlo integration Gordon et al.
Meta-learning and multi-head CL.
The meta-learning methods detailed above have a clear demarcation between shared parameters and task speciﬁc parameters.
In MAMLtype gradient based meta-learning the global parameters θ is an NN initialization which generalizes in a few steps of gradient descent to obtain the task speciﬁc parameter φ t .
For amortized inference based meta-learning approaches the shared parameters are the feature extractor and amortization network which output task speciﬁc parameter heads.
In practice this amortization approach is similar to multi-head CL models which also has a shared feature extractor and task speciﬁc heads, although none of the Bayesian CL literature explicitly models the problem in terms of shared parameters and task speciﬁc parameters.
This is the key idea we wish to highlight in this work.
It is clear that sequential Bayesian inference is not enough to solve CL problems and instead they need to model this task speciﬁc and shared parameters which they implicitly implement with task speciﬁc heads.
Simply using a single BNN to model all the tasks in the CL loop is too general of a solution and is bound to suﬀer from catastrophic forgetting.
With this in mind we tackle the CL problem with a probabilistic model which explicitly models each class and is decomposed into task speciﬁc parameters and shared parameters in Section 4.
11. Prototypical Bayesian Continual Learning
The generative model for CL problems where new tasks are comprised of new classes can be modeled as by using a categorical distribution with a Dirichlet prior
[Equation]
We learn a joint embedding space for our data with a NN, z = f ( x ; w ) with parameters w .
The embedding space for each class is Gaussian whose mean has a prior which is also
Gaussian:
[Equation]
By ensuring that we have an embedding per class and using a memory of past data to ensure that the embedding does not drift we can solve the incremental class learning problem in CL.
The posterior parameters are η t = { α t , µ 1: J,t , Λ − 1 1: J,t } .
11.1. Inference
As the Dirichlet prior is conjugate with the Categorical distribution and so is the Normal distribution with a Normal prior over the mean, then we can calculate posteriors in closed form and update our parameters as we see new data without using gradient based updates.
We optimize the model by maximizing the posterior of the data and use the softmax over class probabilities to perform predictions, we perform gradient based learning of the NN embedding function f ( · ).
The posterior parameters are η t = { α t , µ 1: J,t , Λ − 1 1: J,t } .
As the
Dirichlet prior is conjugate with the Categorical distribution and so the Normal distribution with a Normal prior over the mean, we can update these distributions in closed form.
Sequential updates.
We can obtain our posterior:
[Equation]
Concentrating on the Categorical-Dirchlet conjugacy:
[Equation]
Thus:
[Equation]
Also, due to Normal-Normal conjugacy, then the posterior for the Gaussian prototype of the embedding for each class is:
[Equation]
The update equations for the mean and variance of the posterior are:
[Equation]
[Equation]
Objective.
The likelihood we want to optimize is:
[Equation]
θ = { p 1: J , ¯ z y [Equation]
[Equation]
Where in Eq.
(32) we use § 8.1.8 in Petersen et al. [Equation]
The term p ( y ) is:
[Equation]
where we use the identity Γ( n + 1)
= n Γ( n ).
Predictions.
To make a prediction for a test point ˆ x :
[Equation]
[Figure]
r a c y where η t is a suﬃcient statistic for ( x, y ).
Preventing forgetting.
As we wish to retain the task speciﬁc embeddings and probabilistic model parameters as if we were performing multi-task learning: as if we had access to the entire dataset.
Then at the end of learning a task T t we take a small subset of the data
M t ⊂ D t as a memory to ensure that posterior parameters and prototypes do not drift.
11.2. Experimental Setup
We perform the evaluation of our model using class-incremental learning : for each new task our model needs to predict the actual class of the data in the task.
For instance for the
Split-MNIST example explored in Figure 1, for the ﬁrst task images of { 0 , 1 } are shown and the class labels are { 0 , 1 } ∈ N 10 , then for the second task { 2 , 3 } and the class labels are { 2 , 3 } ∈ N
and so on.
11.3. Prototypical Bayesian Continual Learning: Additional Results
We show the variation of in performance of ProtoCL with the size of the memory of each task in Figure 8.
We also show the variation of the Dirichlet parameters in Figure 9, the class probability priors all increase as the corresponding task datasets are seen.
[Figure]