Conditioning Sparse Variational Gaussian Processes for Online Decision-making

1 Introduction . Intelligent systems should be able to quickly and efficiently adapt to new data , adjusting their prior beliefs in response to the most recent events . These characteristics are desirable whether the system in question is controlling the actuators of a robot , tuning the power output of a laser , or monitoring the changing preferences of users on an online platform . What these applications share in common is a constant stream of new information . In this paper , we are interested in efficient conditioning , meaning that we wish to efficiently update a posterior distribution after receiving new data . The ability of Gaussian process ( GP ) regression models to condition on new data in closed form has made them a popular choice for Bayesian optimization ( BO ) , active learning , and control [ 24 ] . All of these settings share similar characteristics : there is an “ outer loop ” , where new data is acquired from the real world ( e.g . an expensive simulator ) , interleaved with an “ inner loop ” , which chooses where to collect data . In BO , for example , the “ inner loop ” is the optimization of an acquisition function evaluated using a surrogate model of the true objective . Simple acquisition functions , e.g . expected improvement ( EI ) , consider only the current state of the surrogate , while more sophisticated acquisition functions “ look ahead ” to consider the effect of hypothetical observations on future surrogate states . One such acquisition function , batch knowledge gradient ( qKG ) , defines the one-step Bayes-optimal data batch as the batch that maximizes the expected surrogate maximum after the batch has been acquired [ 2 , 84 ] . Advanced acquisition functions like qKG require the surrogate to have both efficient posterior sampling and efficient conditioning on new data . GP regression has two major limitations that have prevented its large scale deployment for online decision-making . First , the computational and memory consumption of exact GPs grows at least quadratically with the amount of data [ 25 , 65 ] , generally limiting their usage to BO problems with fewer than 1 , 000 function evaluations [ 24 , 2 , 79 ] . Second , they are limited to applications that have continuous real-valued responses , enabling modelling with solely a Gaussian likelihood . Stochastic 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . responses ( b ) and non-Gaussian models ( c ) such as the Gaussian copula volatility model [ 82 ] . variational Gaussian processes ( SVGPs ) [ 33 ] have constant computational and memory footprints and are applicable to non-Gaussian likelihoods , but they sacrifice closed form expressions for updated posteriors on receiving new data . The SVGP posterior is optimized through the evidence lower bound ( ELBO ) . In the online setting , training with the ELBO has two primary difficulties : the need to specify a fixed number of observations to properly scale the ELBO gradient [ 8 ] and the need to adjust the inducing points without looking at past data [ 3 , 9 ] . Thus , we are presented with a choice between the simplicity and analytic tractability of exact GPs and the scalability and flexibility of SVGPs . In this work , we develop Online Variational Conditioning ( OVC ) to allow SVGPs to be conditioned on-the-fly , as shown in Figure 1 . In the top row of each subplot , we fit the data points shown in red , shifting to another batch of data points in the bottom row . We use an exact GP in Figure 1a , with exact conditioning shown in the bottom panel . The SVGP emulates the exact GP very well before seeing the new data and again after conditioning on the new data by using OVC ( Figure 1b ) . In Figure 1c , we consider a non-Gaussian data model ( a Gaussian copula volatility model [ 82 ] ) , where we can not use exact GPs ; the SVGP is still able to update its posterior over the latent volatility in response to new data without “ forgetting ” old observations . OVC is inspired by a new , simple rederivation of streaming sparse GPs ( O-SGPR ) , originally proposed by Bui et al . [ 9 ] . OVC makes SVGPs truly compelling models for online decision-making , augmenting their existing strengths with efficient , closed-form conditioning on new data points . In short , our contributions are : • The development of OVC , a novel method to condition SVGPs on new data without reoptimizing the variational posterior through an evidence lower bound . • OVC provides both stable inducing point initialization for SVGPs while enabling the inducing points and variational parameters to update in response to the new data . • Enabling the effective application of SVGPs , through OVC , in look-ahead acquisitions in BO for black-box optimization , controlling dynamical systems , and active learning . Please see Appendix A for discussion of the limitations and broader impacts of our work . Our code is available at https : //github.com/wjmaddox/online_vargp . 2 Preliminaries and Related Work . In this section , we first review exact inference with Gaussian processes before reviewing variational sparse Gaussian processes , introducing sparse Gaussian process regression ( SGPR ) , sparse variational Gaussian processes ( SVGP ) , and streaming sparse GPs ( O-SGPR ) . Please see Appendix B for further background and Appendix B.1 specifically for further related work . 2.1 Gaussian Processes and Exact Inference . For a full introduction to GPs , see Rasmussen and Williams [ 65 ] . We begin by reviewing GP regression , with inputs x 2 X and responses y 2 R. GP regression assumes y ⇠ N ( f , 2 ) , where f ⇠ GP ( 0 , k✓ ( x , x0 ) ) is a GP characterized by the kernel function k✓ and corresponding hyperparameters ✓ . Given Xtrain = [ x1 , . . . , xn ] > and Xtest = [ x01 , . . . , x0k ] > , we denote the corresponding function values as v = [ f ( x1 ) , . . . , f ( xn ) ] > and w = [ f ( x01 ) , . . . , f ( x0k ) ] > , respectively . Note that v and w are random variables with prior covariance Cov [ v , w ] = Kvw : = k✓ ( Xtrain , Xtest ) . A well-known identity for multivariate conditional Gaussians allows us to compute the GP predictive posterior p ( w|Xtest , D , ✓ ) = N ( µ⇤w|D , ⌃⇤w|D ) as follows : µ⇤w|D = Kwv ( Kvv + 2I ) 1y , ⌃⇤w|D = Kww Kwv ( Kvv + ⌃y ) 1Kvw , ( 1 ) where D : = ( Xtrain , y ) and ⌃y = 2I . Naïvely , computations with Kvv cost O ( n2 ) space and O ( n3 ) operations . When repeatedly computing p ( w|Xtest , D , ✓ ) at different sets of query points Xtest , it is more efficient to cache the terms which depend only on the training data.1 Specifically , we store a : = ( Kvv + ⌃y ) 1y ( the predictive mean cache ) and RR > : = ( Kvv + ⌃y ) 1 ( the predictive covariance cache ) , resulting in simplified forms for the predictive distribution , µ⇤w|D = Kwva and ⌃ ⇤ w|D = Kww KwvRR > Kvw . Adding a new observation is equivalent to adding a single row and column to Kvv and an entry to y , which enables efficient low-rank updates to the predictive caches [ 58 , 25 , 64 , 39 ] .2 2.2 Variationally Sparse Gaussian Processes . Variational sparse GPs reduce the computational burden of GP inference through sparse approximations of the kernel matrix . For further reference , see Matthews [ 50 ] and Van der Wilk [ 77 ] . These methods define a variational distribution over the inducing point values u = [ f ( z1 ) , . . . , f ( zp ) ] > , defined at inducing point locations Z = [ z1 , · · · , zp ] > , where zi 2 X . ( u ) is parameterized as a Gaussian with variational mean and covariance mu and Su . These methods assume the latent function values f ( x ) , f ( x0 ) are conditionally independent given u and x , x0 /2 Z , so as to cheaply approximate the predictive posterior p ( w|Xtest , D , ✓ ) ⇡ q ( w ) = N ( µ⇤w| , ⌃⇤w| ) . Like exact GP regression , we can compute q ( w ) in closed form ( given mu , Su ) , µ⇤w| : = KwuK 1 uumu , ⌃ ⇤ w| : = Kww KwuK 1uu ( Kuu Su ) K 1uuKuw . ( 2 ) Similarly the predictive mean and covariance caches are given by a = K 1uumu and RR > = K 1uu ( Kuu Su ) K 1uu , reducing the complexity of inference from O ( n3 ) to O ( np2 ) , which is a significant improvement if p ⌧ n. There are two common approaches to finding optimal variational parameters mu and Su . In seminal work , Titsias [ 73 ] proposed sparse GP regression ( SGPR ) , which optimizes mu and Su in closed form , resulting in a `` collapsed '' evidence lower bound3 ( ELBO ) that only depends on ✓ and Z . The computational cost of each gradient update to the remaining model parameters is still linear in n , and like exact GP regression , SGPR requires a Gaussian likelihood . Stochastic variational GPs ( SVGPs ) remedies both these limitations by using gradient-based optimization to learn mu and Su alongside Z and ✓ [ 33 , 34 ] . The SVGP objective is an “ uncollapsed ” ELBO which decomposes additively across the training examples , allowing gradients to be estimated from minibatches of data , reducing the complexity of each gradient update to O ( bp2 + p3 ) , where b is the minibatch size . We emphasize the distinction between constant-time minibatch gradients , and constant-time conditioning . Given an SVGP already trained on some existing data , conditioning jointly on both the old and new data requires storing all the data and making multiple gradient updates to the variational parameters . As the size of the dataset grows , so does the number of gradient steps needed . In contrast by constant-time conditioning we mean a procedure that takes a posterior conditioned on existing data and produces a new posterior conditioned jointly on the old and new data with a fixed amount of compute and memory , regardless of the number of past observations . One example of constant-time conditioning is found in Bui et al . [ 9 ] , who proposed streaming sparse GPs ( which we call online SGPR , or O-SGPR , to distinguish from sparse spectrum GPs [ 44 ] ) for 1Cached expressions will be written in orange 2What we use as the SGPR covariance cache is slightly different from the implementation in the prediction strategy in GPyTorch , which stores R = KvuK 1/2uu . However , they reduce to the same strategy . 3A lower bound of the true GP marginal log-likelihood Algorithm 1 Online Variational Conditioning ( OVC ) Input : Data batch ( Xbatch , y ) , SVGP with inducing points Z 0 and ( u0 ) = N ( mu0 , Su0 ) . 1 . Compute c0 , C 0 ( Eq . 5 ) . 2 . Compute ŷ = K 0u0u0C 0 1c0 and ⌃ŷ = K 0 1u0u0C 0K 0 1u0u0 ( Eq . 8 ) . 3 . Construct GP with D = ( [ Xbatch Z 0 ] , [ y ŷ ] ) and ⌃ = blkdiag ( ⌃y , ⌃ŷ ) . 4 . Compute predictive mean and covariance caches , a and RR > as in Section 2.1 . 5 . Use caches to compute conditioned GP posterior on test points , Xtest . incremental learning . We extend their work , providing an alternative , simpler derivation of their model that highlights the connection with SGPR [ 73 ] . Furthermore , our perspective enables us to construct a principled approach to updating inducing point locations as new data arrives , that prevents the “ forgetting ” of old data induced by the resampling heuristic used by Bui et al . [ 9 ] .

Summary: The authors propose online variational conditioning (OVC), a procedure for efficiently conditioning Stochastic Variational Gaussian Processes (SVGPs) in an online setting that does not require re-training through the evidence lower bound with the addition of new data. The authors propose to efficiently update a posterior distribution after receiving new data; they rely on the idea of a Sparse Gaussian Process Regression (SGPR) model trained on pseudo-data under a block-diagonal Gaussian likelihood, they also provide an extension to non-Gaussian likelihoods by means of the Laplace approximation. The authors show the application of their method in look-ahead acquisitions for Bayesian Optimisation.