Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning

1 Introduction . Deployed machine learning systems are often faced with the problem of distribution shift , where the new data that the model processes is systematically different from the data the system was trained on [ Zech et al. , 2018 , Ovadia et al. , 2019 ] . Furthermore , a shift can happen anytime after deployment , unbeknownst to the users , with dramatic consequences for systems such as self-driving cars , robots , and financial trading algorithms , among many other examples . Updating a deployed model on new , representative data can help mitigate these issues and improve general performance in most cases . This task is commonly referred to as online or incremental learning . Such online learning algorithms face a tradeoff between remembering and adapting . If they adapt too fast , their performance will suffer since adaptation usually implies that the model loses memory of previously encountered training data ( which may still be relevant to future predictions ) . On the other hand , if a model remembers too much , it typically has problems adapting to new data distributions due to its finite capacity . The tradeoff between adapting and remembering can be elegantly formalized in a Bayesian online learning framework , where a prior distribution is used to keep track of previously learned parameter estimates and their confidences . For instance , variational continual learning ( VCL ) [ Nguyen et al. , 2018 ] is a popular framework that uses a model ’ s previous posterior distribution as the prior for new data . However , the assumption of such continual learning setups is usually that the data distribution is stationary and not subject to change , in which case adaptation is not an issue . This paper proposes a new Bayesian online learning framework suitable for non-stationary data distributions . It is based on two assumptions : ( i ) distribution shifts occur irregularly and must be inferred from the data , and ( ii ) the model requires not only a good mechanism to aggregate data but 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . also the ability to partially forget information that has become obsolete . To solve both problems , we still use a Bayesian framework for online learning ( i.e. , letting a previous posterior distribution inform the next prior ) ; however , before combining the previously learned posterior with new data evidence , we introduce an intermediate step . This step allows the model to either broaden the previous posterior ’ s variance to reduce the model ’ s confidence , thus providing more “ room ” for new information , or remain in the same state ( i.e. , retain the unchanged , last posterior as the new prior ) . We propose a mechanism for enabling this decision by introducing a discrete “ change variable ” that indicates the model ’ s best estimate of whether the data in the new batch is compatible with the previous data distribution or not ; the outcome then informs the Bayesian prior at the next time step . We further augment the scheme by performing beam search on the change variable . This way , we are integrating change detection and Bayesian online learning into a common framework . We test our framework on a variety of real-world datasets that show concept drift , including basketball player trajectories , malware characteristics , sensor data , and electricity prices . We also study sequential versions of SVHN and CIFAR-10 with covariate drift , where we simulate the shifts in terms of image rotations . Finally , we study word embedding dynamics in an unsupervised learning approach . Our approach leads to a more compact and interpretable latent structure and significantly improved performance in the supervised experiments . Furthermore , it is highly scalable ; we demonstrate it on models with hundreds of thousands of parameters and tens of thousands of feature dimensions . Our paper is structured as follows : we review related work in Section 2 , introduce our methods in Section 3 , report our experiments in Section 4 , and draw conclusions in Section 6 . 2 Related Work . Our paper connects to Bayesian online learning , change detection , and switching dynamical systems . Bayesian Online and Continual Learning There is a rich existing literature on Bayesian and continual learning . The main challenge in streaming setups is to reduce the impact of old data on the model which can be done by exponentially decaying old samples [ Honkela and Valpola , 2003 , Sato , 2001 , Graepel et al. , 2010 ] or re-weighting them [ McInerney et al. , 2015 , Theis and Hoffman , 2015 ] . An alternative approach is to adapt the model posterior between time steps , such as tempering it at a constant rate to accommodate new information [ Kulhavỳ and Zarrop , 1993 , Kurle et al. , 2020 ] . In contrast , continual learning typically assumes a stationary data distribution and simply uses the old posterior as the new prior . A scalable such scheme based on variational inference was proposed by [ Nguyen et al. , 2018 ] which was extended by several authors [ Farquhar and Gal , 2018 , Schwarz et al. , 2018 ] . A related concept is elastic weight consolidation [ Kirkpatrick et al. , 2017 ] , where new model parameters are regularized towards old parameter values . All of these approaches need to make assumptions on the expected frequency and strengh of change which are hard-coded in the model parameters ( e.g. , exponential decay rates , re-weighting terms , prior strengths , or temperature choices ) . Our approach , in contrast , detects change based on a discrete variable and makes no assumption about its frequency . Other approaches assume situations where data arrive in irregular time intervals , but are still concerned with static data distributions [ Titsias et al. , 2019 , Lee et al. , 2020 , Rao et al. , 2019 ] . Change Point Models There is also a rich history of models for change detection . A popular class of change point models includes “ product partition models ” [ Barry and Hartigan , 1992 ] which assume independence of the data distribution across segments . In this regime , Fearnhead [ 2005 ] proposed change detection in the context of regression and generalized it to online inference [ Fearnhead and Liu , 2007 ] ; Adams and MacKay [ 2007 ] described a Bayesian online change point detection scheme ( BOCD ) based on conditional conjugacy assumptions for one-dimensional sequences . Other work generalized change detection algorithms to multivariate time series [ Xuan and Murphy , 2007 , Xie et al. , 2012 ] and non-conjugate Bayesian inference [ Saatçi et al. , 2010 , Knoblauch and Damoulas , 2018 , Turner et al. , 2013 , Knoblauch et al. , 2018 ] . Our approach relies on jointly inferring changes in the data distribution while carrying out Bayesian parameter updates for adaptation . To this end , we detect change in the high-dimensional space of model ( e.g. , neural network ) parameters , as opposed to directly in the data space . Furthermore , a detected change only partially resets the model parameters , as opposed to triggering a complete reset . Titsias et al . [ 2020 ] proposed change detection to detect distribution shifts in sequences based on low-dimensional summary statistics such as a loss function ; however , the proposed framework does not use an informative prior but requires complete retraining . Switching Linear Dynamical Systems Since our approach integrates a discrete change variable , it is also loosely connected to the topic of switching linear dynamical systems . Linderman et al . [ 2017 ] considered recurrent switching linear dynamical systems , relying on Bayesian conjugacy and closed-form message passing updates . Becker-Ehmck et al . [ 2019 ] proposed a variational Bayes filtering framework for switching linear dynamical systems . Murphy [ 2012 ] and Barber [ 2012 ] developed an inference method using a Gaussian sum filter . Instead , we focus on inferring the full history of discrete latent variable values instead of just the most recent one . Bracegirdle and Barber [ 2011 ] introduce a reset variable that sets the continuous latent variable to an unconditional prior . It is similar to our work , but relies on using low-dimensional , tractable models . Our tempering prior can be seen as a partial reset , augmented with beam search . We also extend the scope of switching dynamical systems by integrating them into a supervised learning framework . 3 Methods . Overview Section 3.1 introduces the setup and the novel model structure under consideration . Section 3.2 introduces an exact inference scheme based on beam search . Finally , we introduce the variational inference extension for intractable likelihood models in Section 3.3 . 3.1 Problem Assumptions and Structure . We consider a stream of data that arrives in batches xt at discrete times t.1 For supervised setups , we consider pairs of features and targets ( xt , yt ) , where the task is to model p ( yt|xt ) . An example model could be a Bayesian neural network , and the parameters zt could be the network weights . For notational simplicity we focus on the unsupervised case , where the task is to model p ( xt ) using a model p ( xt|zt ) with parameters zt that we would like to tune to each new batch.2 We then measure the prediction error either on one-step-ahead samples or using a held-out test set . Furthermore , we assume that while the xt are i.i.d . within batches , they are not necessarily i.i.d . across batches as they come from a time-varying distribution pt ( xt ) ( or pt ( xt , yt ) in the supervised cases ) which is subject to distribution shifts . We do not assume whether these distribution shifts occur instantaneously or gradually . The challenge is to optimally adapt the parameters zt to each new batch while borrowing statistical strength from previous batches . As follows , we will construct a Bayesian online learning scheme that accounts for changes in the data distribution . For every new batch of data , our scheme tests whether the new batch is compatible with the old data distribution , or more plausible under the assumption of a change . To this end , we employ a binary “ change variable ” st , with st = 0 for no detected change and st = 1 for a detected change . Our model ’ s joint distribution factorizes as follows : p ( x1 : T , z1 : T , s1 : T ) = T∏ t=1 p ( xt|zt ) p ( zt|st ; τ t ) p ( st ) . ( 1 ) We assumed a factorized Bernoulli prior ∏ t p ( st ) over the change variable : an assumption that will simplify the inference , but which can be relaxed . As a result , our model is fully-factorized over time , however , the model can still capture temporal dependencies through the informative prior p ( zt|st ; τ t ) . Temporal information enters this prior through certain sufficient statistics τ t that depend on properties of the previous time-step ’ s approximate posterior . In more detail , τ t is a functional on the previous time step ’ s approximate posterior , τ t = F [ p ( zt−1|x1 : t−1 , s1 : t−1 ) ] .3 Throughout this paper , we will use a specific form of τ t , namely 1In an extreme case , it is possible for a batch to include only a single data point . 2In supervised setups , we consider a conditional model p ( yt|zt , xt ) with features xt and targets yt . 3Subscripts 1 : t− 1 indicates the integers from 1 to t− 1 inclusively . capturing the previous posterior ’ s mean and variance.4 More formally , τ t ≡ { µt−1 , Σt−1 } ≡ { Mean , Var } [ zt−1|x1 : t−1 , s1 : t−1 ] . ( 2 ) Based on this choice , we define the conditional prior as follows : p ( zt|st ; τ t ) = { N ( zt ; µt−1 , Σt−1 ) for st = 0 N ( zt ; µt−1 , β−1Σt−1 ) for st = 1 ( 3 ) Above , 0 < β < 1 is a hyperparameter referred to as inverse temperature5 . If no change is detected ( i.e. , st = 0 ) , our prior becomes a Gaussian distribution centered around the previous posterior ’ s mean and variance . In particular , if the previous posterior was already Gaussian , it becomes the new prior . In contrast , if a change was detected , the broadened posterior becomes the new prior . For as long as no changes are detected ( st = 0 ) , the procedure results in a simple Bayesian online learning procedure , where the posterior uncertainty shrinks with every new observation . In contrast , if a change is detected ( st = 1 ) , an overconfident prior would be harmful for learning as the model needs to adapt to the new data distribution . We therefore weaken the prior through tempering . Given a temperature β , we raise the previous posterior ’ s Gaussian approximation to the power β , renormalize it , and then use it as a prior for the current time step . The process of tempering the Gaussian posterior approximation can be understood as removing equal amounts of information in any direction in the latent space . To see this , let z be a multivariate Gaussian with covariance Σ and u be a unit direction vector . Then tempering removes an equal amount of information regardless of u , Hu = 12 log ( 2πeu > Σu ) − 12 log β , erasing learned information to free-up model capacity to adjust to the new data distribution . See Supplement B for more details . Connection to Sequence Modeling Our model assumptions have a resemblance to time series modeling : if we replaced τ t with zt−1 , we would condition on previous latent states rather than posterior summary statistics . In contrast , our model still factorizes over time and therefore makes weaker assumptions on temporal continuity . Rather than imposing temporal continuity on a data instance level , we instead assume temporal continuity at a distribution level . Connection to Changepoint Modeling . We also share similar assumptions with the changepoint literature [ Barry and Hartigan , 1992 ] . However , in most cases , these models don ’ t assume an informative prior , effectively not taking into account any sufficient statistics τ t. This forces these models to re-learn model parameters from scratch after a detected change , whereas our approach allows for some transfer of information before and after the distribution shift .

Summary: This paper addresses the topic of changepoint detection in the event of distribution shift. A Bayesian model meant to detect such changes online, and able to discount historical information after a distributional shift, is formulated to express the problem. Algorithmic implementation is provided and evaluated across a number of data sets.