Continual Learning with Bayesian Neural Networks for Non-Stationary Data

1 INTRODUCTION . Continual learning ( CL ) , also referred to as lifelong learning , is typically described informally by the following set of desiderata for computational systems : the system should ( i ) learn incrementally from a data stream , ( ii ) exhibit information transfer forward and backward in time , ( iii ) avoid catastrophic forgetting of previous data , and ( iv ) adapt to changes in the data distribution ( Ring , 1997 ; Silver et al. , 2013 ; Chen & Liu , 2016 ; Ruvolo & Eaton , 2013 ; Parisi et al. , 2018 ) . The necessity to adapt to non-stationary data is often not reconcilable with the goal of preventing forgetting . This problem is also known as the stability-plasticity dilemma ( Grossberg , 1987 ) . The majority of current CL research is conducted in the context of online multi-task learning ( Nguyen et al. , 2018 ; Kirkpatrick et al. , 2017 ; Schwarz et al. , 2018 ; Rusu et al. , 2016 ; Fernando et al. , 2017 ) , where the main objective is to prevent catastrophic forgetting of previously learned tasks . This focus is reasonable since changes in the statistics of the data distribution are usually an artefact of learning different tasks sequentially . However , changes in the statistics of the data can also be real properties of the data-generating process . Examples include models of energy demand , climate analysis , financial market , or user-behavior analytics ( Ditzler et al. , 2015 ) . In such applications , the statistics of the current data distribution are of particular interest . Old data may be outdated and can even deteriorate learning if the drift in the data distribution is neglected . Consequently , CL systems for non-stationary data require adaptation methods , which deliberately forget outdated information . In this work , we develop an approximate Bayesian approach for training Bayesian neural networks ( BNN ) ( Hinton & van Camp , 1993 ; Graves , 2011 ; Blundell et al. , 2015 ) incrementally with nonstationary streaming data . Similar to variational continual learning ( VCL ) ( Nguyen et al. , 2018 ) and the Virtual Vector Machine ( VVM ) ( Minka et al. , 2009 ) , we approximate the posterior using a Gaussian distribution and a complementary memory of previous data . Both components are updated sequentially , while adapting to changes in the data distribution . Our main contributions are as follows : • We propose an online approximation consisting of a diagonal Gaussian distribution and a running memory , and we provide a novel sequential update method for both components . • We extend the online approximation by two alternative adaptation methods , thereby general- ising online variational Bayes with Bayesian neural networks to non-stationary data . We compare our sequential update method to VCL in the online-inference setting on several popular datasets , demonstrating that our approach is favorable . Furthermore , we validate our adaptation methods on several datasets with concept drift ( Widmer & Kubat , 1996 ) , showing performance improvements compared to online variational Bayes without adaptation . ∗Correspondence to richard.kurle @ tum.de 2 BACKGROUND : ONLINE INFERENCE . Consider a stream of datasets { Dtk } Kk=1 , where tk are the time points at which datasets Dtk are observed . For the moment , we assume that these datasets and the samples within are generated independently and identically distributed ( i.i.d. ) . Methods for non-i.i.d . data are considered in Sec . 4 . In the Bayesian approach to online learning , we want to infer the posterior distribution p ( w|Dt1 : tk ) of our model parameters , with the restriction that the data is processed sequentially.1 Using Bayes rule , a recursive posterior inference equation emerges naturally : p ( w|Dt1 : tk ) ∝ p ( w|Dt1 : tk−1 ) p ( Dtk |w , Dt1 : tk−1 ) = p ( w|Dt1 : tk−1 ) p ( Dtk |w ) , ( 1 ) where the last step follows from the i.i.d . assumption of the data . In this paper , we consider Gaussian and multinomial likelihoods , parametrised by a neural network with weights w and prior p ( w|∅ ) = p0 ( w ) = N ( w ; µ0 , σ0 ) . Furthermore , we consider supervised learning , where Dtk = { d ( n ) tk } n = { ( x ( n ) tk , y ( n ) tk ) } n and p ( d ( n ) tk |w ) = p ( y ( n ) tk |NN ( x ( n ) tk ; w ) ) . 2.1 ONLINE VARIATIONAL BAYES . Since exact Bayesian inference is intractable for non-trivial models , various approximations have been developed . Prominent examples include sequential Monte Carlo ( Liu & Chen , 1998 ) , assumed density filtering ( Maybeck , 1982 ) , and online variational Bayes ( Opper , 1998 ; Ghahramani , 2000 ; Sato , 2001 ; Broderick et al. , 2013 ) . Online variational Bayes ( VB ) approximates the posterior of Eq . ( 1 ) by a parametrised distribution qθtk ( w ) ≈ p ( w|Dt1 : tk ) through a sequence of projections : qθtk ( w ) = argmin qθ KL [ qθ ( w ) ||Z−1tk qθtk−1 ( w ) p ( Dtk |w ) ] , ( 2 ) where Ztk is the normalisation constant . The above minimisation is equivalent to maximising the evidence lower bound ( ELBO ) Ltk ( θ ; Dtk ) = Eqθ ( w ) [ log p ( Dtk |w ) ] −KL [ qθ ( w ) || qθtk−1 ( w ) ] . In this work , we consider diagonal Gaussian posterior approximations qθtk ( w ) for the neural network weights , similar to Nguyen et al . ( 2018 ) . 2.2 ONLINE VARIATIONAL BAYES WITH MEMORY . Online approximate Bayesian inference methods inevitably suffer from an information loss due to the posterior approximation at each time-step . An alternative approach to online learning is to store and update a representative dataset/generative model—and to use it as a memory—in order to improve inference ( Robins , 1995 ; Lopez-Paz & Ranzato , 2017 ; Shin et al. , 2017 ; Kamra et al. , 2017 ) . Memory-based online learning has also been combined with online Bayesian inference methods ( Minka et al. , 2009 ; Nguyen et al. , 2018 ) . A common property of these approaches is to represent the ( current ) posterior approximation by a product of two factors p ( w|Dt1 : tk ) ≈ qθtk ( w ) p ( Mtk |w ) ( 3 ) and update them sequentially as new dataDtk is observed . The factor p ( Mtk |w ) = ∏M m p ( m ( m ) tk |w ) is the likelihood of a set of M = |M| data points , which we refer to as running memory ; and qθtk ( w ) is a Gaussian distribution , which summarises the rest of the data D̄1 : tk = D1 : tk\Mtk . In case of VCL , the factors in Eq ( 3 ) are updated in two steps , which we refer to as ( i ) memory update and ( ii ) Gaussian update : ( i ) a new memoryMtk ⊂ Dtk ∪ Mtk−1 is selected using heuristics such as random selection or the k-center method ( a greedy algorithm that selects K data points based on geometric properties of Dtk ∪ Mtk−1 . ) ; ( ii ) the Gaussian distribution is updated with the remaining data D̄tk = Dtk ∪ Mtk−1 \ Mtk ( using Eq . ( 2 ) ) to obtain qθtk ( w ) ≈ qθtk−1 ( w ) p ( D̄tk |w ) . Note that we can not sample directly from the posterior approximation in Eq . ( 3 ) and thus we can not easily evaluate quantities such as the posterior predictive distribution . VCL therefore performs a second projection q̃θtk ( w ) = argmin qθ KL [ qθ ( w ) || Z̃−1tk qθtk ( w ) p ( Mtk |w ) ] . ( 4 ) This distribution should not be confused with the recursively updated variational distribution ( Eq . ( 2 ) ) . 1 A strict definition of online learning requires single data samples at each time step instead of batches Dtk . 3 IMPROVING MEMORY-BASED ONLINE VARIATIONAL BAYES . In this section , we focus on two problems of existing approaches using online VB with a running memory : ( i ) the memory update does not take into account the approximation error or approximation capabilities of the variational distribution ; ( ii ) the Gaussian update—performed by optimising the ELBO ( Eq . ( 2 ) ) only with data D̄tk—can fail for streaming data . This is because VB yields poor posterior approximations if the dataset is too small or the neural network architecture has too much capacity ( cf . Ghosh et al . ( 2018 ) , Fig . 1 ) . In Secs . 3.2 and 3.3 , we propose improvements to these two update methods . The mathematical background for our approach is provided in Sec . 3.1 . 3.1 PROPERTIES OF THE GAUSSIAN VARIATIONAL APPROXIMATION . There are two important properties of the Gaussian variational approximation that we will exploit later : ( i ) Gaussian approximate posterior distributions factorise into a product of Gaussian terms corresponding to the prior and each likelihood term ; ( ii ) the ELBO can be written as the sum of the approximation ’ s normalisation constant and a sum of residuals corresponding to these factors . Let p0 ( w ) = N ( w ; µ0 , Σ0 ) be a Gaussian prior and p ( D|w ) = ∏ n p ( d ( n ) |w ) be the likelihood of the observed data D. Furthermore , let qθ ( w ) = N ( w ; µ , Σ ) denote the corresponding Gaussian variational approximation with θ = { µ , Σ } . Assume that µ and Σ are the optimal parameters corresponding to a ( local ) maximum of the ELBO L ( µ , Σ ; D ) . The optimality conditions ∂µL ( µ , Σ ; D ) = 0 and ∂ΣL ( µ , Σ ; D ) = 0 can be rewritten as follows ( Knowles & Minka , 2011 ; Opper & Archambeau , 2008 ; Cseke et al. , 2013 ) ( cf . App . C ) : Σ−1µ = Σ−10 µ0 + ∑ n ( ∂µEqθ ( w ) [ log p ( d ( n ) |w ) ] − 2∂ΣEqθ ( w ) [ log p ( d ( n ) |w ) ] µ ) , ( 5a ) Σ−1 = Σ−10 − 2 ∑ n ∂ΣEqθ ( w ) [ log p ( d ( n ) |w ) ] . ( 5b ) Since the sum of natural parameters corresponds to a product in distribution space , the above equations show that—at a local optimum—the approximation qθ ( w ) factorises in the same way as the posterior p ( w|D ) . It can be written in the form qθ ( w ) = Zq−1p0 ( w ) ∏ n r ( n ) ( w ) , where the factors r ( n ) ( w ) are Gaussian functions with natural parameters given by Eqs . ( 5a ) and ( 5b ) , and where Zq = ∫ p0 ( w ) ∏ n r ( n ) ( w ) dw is the normalisation constant . These Gaussian functions r ( n ) ( w ) each correspond to the contribution of the likelihood p ( d ( n ) |w ) to the posterior approximation qθ ( w ) . The resulting factorisation implies that the ELBO L ( µ , Σ ; D ) can be written in the form ( Opper & Winther , 2005 ) ( c.f . App . D ) L ( µ , Σ ; D ) = logZq + ∑ n Eqθ ( w ) [ log p ( d ( n ) |w ) − log r ( n ) ( w ) ] . ( 6 ) If the terms p ( d ( n ) |w ) were ( diagonal ) Gaussian in w , they would each cancel with the corresponding ( diagonal ) Gaussian term , leaving only logZq . Intuitively , the residual terms in Eq . ( 6 ) can be used to quantify the quality of the Gaussian approximation .

Summary: This paper proposes a method for training Bayesian neural nets on a stream of non-stationary data. The authors propose a new way to approximate the variational distribution in the case where a memory (of past data) is used. The authors also use two forgetting mechanisms that allow the model to adapt to the changes in the data distribution.