A Unified Bayesian Framework for Discriminative and Generative Continual Learning

1 INTRODUCTION . Continual learning ( CL ) ( Ring , 1997 ; Parisi et al. , 2019 ) is the learning paradigm where a single model is subjected to a sequence of tasks . At any point of time , the model is expected to ( i ) make predictions for the tasks it has seen so far , ( ii ) if subjected to training data for a new task , adapt to the new task leveraging the past knowledge if possible ( forward transfer ) and benefit the previous tasks if possible ( backward transfer ) . While the desirable aspects of more mainstream transfer learning ( sharing of bias between related tasks ( Pan & Yang , 2009 ) ) might reasonably be expected here too , the principal challenge is to retain the predictive power for the older tasks even after learning new tasks , thus avoiding the so-called catastrophic forgetting . Real world applications in , for example , robotics or time-series forecasting , are rife with this challenging learning scenario , the ability to adapt to dynamically changing environments or evolving data distributions being essential in these domains . Continual learning is also desirable in unsupervised learning problems as well ( Smith et al. , 2019 ; Rao et al. , 2019b ) where the goal is to learn the underlying structure or latent representation of the data . Also , as a skill innate to humans ( Flesch et al. , 2018 ) , it is naturally an interesting scientific problem to reproduce the same capability in artificial predictive modelling systems . Existing approaches to continual learning are mainly based on three foundational ideas . One of them is to constrain the parameter values to not deviate significantly from their previously learned value by using some form of regularization or trade-off between previous and new learned weights ( Schwarz et al. , 2018 ; Kirkpatrick et al. , 2017 ; Zenke et al. , 2017 ; Lee et al. , 2017 ) . A natural way to accomplish this is to train a model using online Bayesian inference , whereby the posterior of the parameters learned from task t serve as the prior for task t + 1 as in Nguyen et al . ( 2018 ) and Zeno et al . ( 2018 ) . This new informed prior helps in the forward transfer , and also prevents catastrophic forgetting by penalizing large deviations from itself . In particular , VCL ( Nguyen et al. , 2018 ) achieves the state of the art results by applying this simple idea to Bayesian neural networks . The second idea is to perform an incremental model selection for every new task . For neural networks , this is done by evolving the structure as newer tasks are encountered ( Golkar et al. , 2019 ; Li et al. , 2019 ) . Structural learning is a very sensible direction in continual learning as a new task may require a different network structure than old unrelated tasks and even if the tasks are highly related their lower layer representations can be very different . Another advantage of structural learning is that while retaining a shared set of parameters ( which can be used to model task relationships ) it also allow task-specific parameters that can increase the performance of the new task while avoiding catastrophic forgetting caused due to forced sharing of parameters . The third idea is to invoke a form of ’ replay ’ , whereby selected or generated samples representative of previous tasks , are used to retrain the model after new tasks are learned . In this work , we introduce a novel Bayesian nonparametric approach to continual learning that seeks to incorporate the ability of structure learning into the simple yet effective framework of online Bayes . In particular , our approach models each hidden layer of the neural network using the Indian Buffet Process ( Griffiths & Ghahramani , 2011 ) prior , which enables us to learn the network structure as new tasks arrive continually . We can leverage the fact that any particular task t uses a sparse subset of the connections of a neural network Nt , and different related tasks share different subsets ( albeit possibly overlapping ) . Thus , in the setting of continual learning , it would be more effective if the network could accommodate changes in its connections dynamically to adapt to a newly arriving task . Moreover , in our model , we perform the automatic model selection where each task can select the number of nodes in each hidden layer . All this is done under the principled framework of variational Bayes and a nonparametric Bayesian modeling paradigm . Another appealing aspect of our approach is that in contrast to some of the recent state-of-the-art continual learning models ( Yoon et al. , 2018 ; Li et al. , 2019 ) that are specific to supervised learning problems , our approach applies to both deep discriminative networks ( supervised learning ) where each task can be modeled by a Bayesian neural network ( Neal , 2012 ; Blundell et al. , 2015 ) , as well as deep generative networks ( unsupervised learning ) where each task can be modeled by a variational autoencoder ( VAE ) ( Kingma & Welling , 2013 ) . 2 PRELIMINARIES . Bayesian neural networks ( Neal , 2012 ) are discriminative models where the goal is to model the relationship between inputs and outputs via a deep neural network with parametersw . The network parameters are assumed to have a prior p ( w ) and the goal is to infer the posterior given the observed dataD . The exact posterior inference is intractable in such models . One such approximate inference scheme is Bayes-by-Backprop ( Blundell et al. , 2015 ) that uses a mean-field variational posterior q ( w ) over the weights . Reparameterized samples from this posterior are then used to approximate the lower bound via Monte Carlo sampling . Our goal in the continual learning setting is to learn such Bayesian neural networks for a sequence of tasks by inferring the posterior qt ( w ) for each task t , without forgetting the information contained in the posteriors of previous tasks . Variational autoencoders ( Kingma & Welling , 2013 ) are generative models where the goal is to model a set of inputs { x } Nn=1 in terms of a stochastic latent variables { z } Nn=1 . The mapping from each zn to xn is defined by a generator/decoder model ( modeled by a deep neural network with parameters θ ) and the reverse mapping is defined by a recognition/encoder model ( modeled by another deep neural network with parameters φ ) . Inference in VAEs is done by maximizing the variational lower bound on the marginal likelihood . It is customary to do point estimation for decoder parameters θ and posterior inference for encoder parameters φ . However , in the continual learning setting , it would be more desirable to infer the full posterior qt ( w ) for each task ’ s encoder and decoder parameters w = { θ , φ } , while not forgetting the information about the previous tasks as more and more tasks are observed . Our proposed continual learning framework address this aspect as well . Variational Continual Learning ( VCL ) Nguyen et al . ( 2018 ) is a recently proposed approach to continual learning that combats catastrophic forgetting in neural networks by modeling the network parameters w in a Bayesian fashion and by setting pt ( w ) = qt−1 ( w ) , that is , a task reuses the previous task ’ s posterior as its prior . VCL solves the follow KL divergence minimization problem qt ( w ) = arg min q∈Q KL ( q ( w ) || 1 Zt qt−1 ( w ) p ( Dt|w ) ) ( 1 ) While offering a principled way that is applicable to both supervised ( discriminative ) and unsupervised ( generative ) learning settings , VCL assumes that the model structure is held fixed throughout , which can be limiting in continual learning where the number of tasks and their complexity is usually unknown beforehand . This necessitates adaptively inferring the model structure , that can potentially adapt with each incoming task . Another limitation of VCL is that the unsupervised version , based on performing CL on VAEs , only does so for the decoder model ’ s parameters ( shared by all tasks ) . It uses completely task-specific encoders and , consequently , is unable to transfer information across tasks in the encoder model . Our approach addresses both these limitations in a principled manner . 3 BAYESIAN STRUCTURE ADAPTATION FOR CONTINUAL LEARNING . In this section , we present a Bayesian model for continual learning that can potentially grow and adapt its structure as more and more tasks arrive . Our model extends seamlessly for unsupervised learning as well . For brevity of exposition , in this section , we mainly focus on the supervised setting where a task has labeled data with known task identities t ( task-incremental ) . We then briefly discuss the unsupervised extension ( based on VAEs ) in Sec . 3.3 where task boundaries may or may not ( taskagnostic ) be available and provide further details in the appendix ( Sec I ) . Our approach uses a basic primitive that models each hidden layer using a nonparametric Bayesian prior ( Fig . 1a shows an illustration and Fig . 1b shows a schematic diagram ) . We can use these hidden layers to model feedforward connections in Bayesian neural networks or VAE models . For simplicity , we will assume a single hidden layer , the first task activates as many hidden nodes as required and learns the posterior over the subset of edge weights incident on each active node . Each subsequent task reuses some of the edges learned by the previous task and uses the posterior over the weights learned by the previous task as the prior . Additionally , it may activate some new nodes and learn the posterior over some of their incident edges . It thus learns the posterior over a subset of weights that may overlap with weights learned by previous tasks . While making predictions , a task uses only the connections it has learned . More slack for later tasks in terms of model size ( allowing it to create new nodes ) indirectly lets the task learn better without deviating too much from the prior ( in this case , posterior of the previous tasks ) and further reduces chances of catastrophic forgetting ( Kirkpatrick et al. , 2017 ) . 3.1 GENERATIVE STORY .. Omitting the task id t for brevity , consider modeling tth task using a neural network having L hidden layers . We model the weights in layer l as W l = Bl V l , a point-wise multiplication of a realvalued matrix V l ( with a Gaussian prior N ( 0 , σ20 ) on each entry ) and a task-specific binary matrix Bl . This ensures sparse connection weights between the layers . Moreover , we modelBl ∼ IBP ( α ) using the Indian Buffet Process ( IBP ) Griffiths & Ghahramani ( 2011 ) prior , where the hyperparameter α controls the number of nonzero columns in B and its sparsity . The IBP prior thus enables learning the size ofBl ( and consequently of V l ) from data . As a result , the number of nodes in the hidden layer is learned adaptively from data . The output layer weights are denoted as Wout with each weight having a Gaussian prior N ( 0 , σ20 ) . The outputs are yn ∼ Lik ( WoutφNN ( xn ) ) , n = 1 , . . . , N ( 2 ) Here φNN is the function computed ( using parameter samples ) up to the last hidden layer of the network thus formed , and Lik denotes the likelihood model for the outputs . Similar priors on the network weights have been used in other recent works to learn sparse deep neural networks ( Panousis et al. , 2019 ; Xu et al. , 2019 ) . However , these works assume a single task to be learned . In contrast , our focus here is to leverage such priors in the continual learning setting where we need to learn a sequence of tasks while avoiding the problem of catastrophic forgetting . Henceforth , we further suppress the superscript denoting layer number from the notation for simplicity ; the discussion will hold identically for all hidden layers . When adapting to a new task , the posterior of V learned from previous tasks is used as the prior . A newB is learned afresh , to ensure that a task only learns the subset of weights relevant to it . Stick Breaking Construction . As described before , to adaptively infer the number of nodes in each hidden layer , we use the IBP prior ( Griffiths & Ghahramani , 2011 ) , whose truncated stick-breaking process ( Doshi et al. , 2009 ) construction for each entry of B is as follows νk ∼ Beta ( α , 1 ) , πk = k∏ i=1 νi , Bd , k ∼ Bernoulli ( πk ) ( 3 ) for d ∈ 1 , ... , D , where D denotes the number of input nodes for this hidden layer , and k ∈ 1 , 2 , ... , K , where K is the truncation level and α controls the effective value of K , i.e. , the number of active hidden nodes . Note that the prior probability πk of weights incident on hidden node k being nonzero decreases monotonically with k , until , say , K nodes , after which no further nodes have any incoming edges with nonzero weights from the previous layer , which amounts to them being turned off from the structure . Moreover , due to the cumulative product based construction of the πk ’ s , an implicit ordering is imposed on the nodes being used . This ordering is preserved across tasks , and allocation of nodes to a task follows this , facilitating reuse of weights . The truncated stick-breaking approximation is a practically plausible and intuitive solution for continual learning since a fundamental tenet of continual learning is that the model complexity should not increase in an unbounded manner as more tasks are encountered . Suppose we fix a budget on the maximum allowed size of the network ( no . hidden nodes in a layer ) after it has seen , say , T tasks . Which exactly corresponds to the truncation level for each layer . Then for each task , nodes are allocated conservatively from this total budget , in a fixed order , conveniently controlled by the α hyperparameter . In appendix ( Sec . D ) , we also discuss a dynamic expansion scheme that avoids specifying a truncation level ( and provide experimental results ) .

Summary: The paper proposes a continual learning framework based on Bayesian non-parametric approach.  The hidden layer is modeled using Indian Buffet Process prior.  The inference uses a structured mean-field approximation with a Gaussian family for the weights, and Beta-Bernoulli for the task-masks.  The variational inference is done with Bayes-by-backprop on a common ELBO setup.  The experiments show less diminishing accuracy on the increment of tasks on five datasets for the discriminative problem, and for generation the methods learn one digit or character at a time on MNIST and notMNIST datasets.