Conditioning Sparse Variational Gaussian Processes for Online Decision-making
Authors: Wesley Maddox
Samuel Don Stanton
Andrew Gordon Wilson
Venue: NeurIPS 2021 Poster (2021)
Keywords: Gaussian processes
Bayesian optimization
stochastic variational Gaussian processes

Abstract: With a principled representation of uncertainty and closed form posterior updates, Gaussian processes (GPs) are a natural choice for online decision making. However, Gaussian processes typically require at least $\mathcal{O}(n^2)$ computations for $n$ training points, limiting their general applicability. Stochastic variational Gaussian processes (SVGPs) can provide scalable inference for a dataset of fixed size, but are difficult to efficiently condition on new data. We propose online variational conditioning (OVC), a procedure for efficiently conditioning SVGPs in an online setting that does not require re-training through the evidence lower bound with the addition of new data. OVC enables the pairing of SVGPs with advanced look-ahead acquisition functions for black-box optimization, even with non-Gaussian likelihoods. We show OVC provides compelling performance in a range of applications including active learning of malaria incidence, and reinforcement learning on MuJoCo simulated robotic control tasks.


Conditioning Sparse Variational Gaussian Processes for Online Decision-making
Wesley J. Maddox New York University wjm3630nyu.edd
Samuel Stanton New York University ss 111134410yu..e.m edu
Andrew Gordon Wilson New York University andrewgwdcims nyu edu
Abstract
With a principled representation of uncertainty and closed form posterior updates, Gaussian processes (GPs) are a natural choice for online decision making. However, Gaussian processes typically require at least O(n²) computations for n training points, limiting their general applicability. Stochastic variational Gaussian pro- cesses (SVGPs) can provide scalable inference for a dataset of fixed size, but are difficult to efficiently condition on new data. We propose online variational conditioning (OVC), procedure for efficiently conditioning SVGPs in an online setting that does not require re- re-training through the evidence lower bound with the addition of new data. OVC enables the pairing of SVGPs with advanced look- ahead acquisition functions for black-box optimization, even with non-Gaussian likelihoods. We show OVC provides compelling performance in range of applica- tions including active learning of malaria incidence, and reinforcement learning on MuJoCo simulated robotic control tasks.
Introduction
Intelligent systems should able quickly and efficiently adapt new data, adjusting their prior beliefs in response to the most recent events. These characteristics desirable whether the system in question controlling the actuators of a robot, tuning the power output a laser, or monitoring the changing preferences of users on online platform. What these applications share in common a constant stream of new information. In this paper, we are interested in efficient conditioning, meaning that wish to efficiently update posterior distribution after receiving new data.
The ability Gaussian process (GP) regression models condition on new data in closed form has made them popular choice for Bayesian optimization (BO), active learning, and control [24] All of these settings share similar characteristics: there is an "outer loop", where new data is acquired from the real world (e.g. expensive simulator), interleaved with an "inner loop". which chooses where to collect data. BO, for example, the "inner loop" the optimization of an acquisition function evaluated using a surrogate model of the true objective. Simple acquisition functions, e.g. expected improvement (EI), consider only the current state of the surrogate, while more sophisticated acquisition functions "look ahead" to consider the effect of hypothetical observations future surrogate states. One such acquisition function, batch knowledge gradient (gKG), defines the one-step Bayes-optimal data batch the batch that maximizes the expected surrogate maximum after the batch has been acquired 2 (4). Advanced acquisition functions like qKG require the surrogate to have both efficient posterior sampling and efficient conditioning new data.
GP regression has two major limitations that have prevented its large scale deployment for online decision-making First, the computational and memory consumption of exact GPs grows least quadratically with the amount of data [25 65 generally limiting their usage to BO problems with fewer than 1, 000 function evaluations continuous real-valued responses, enabling modelling with solely a Gaussian likelihood. Stochastic
24
79
Second, they are limited to applications that have
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Onginal @P
Condiooned GP
Original SVGP
Condbooned SVGP
True Lanent
Training Poins
X
New Ponts
9.5
wwwm
Volama
-05
0.
Volatity
0.-
3.0
0
0.0
(a) Exact GP
(b) SVGP
(c) SVGP + OVC volatility model
Figure I: An exact GP updates its predictive distribution after conditioning new data points (a, moving from top row to bottom row). With OVC, we can condition SVGPs on both Gaussian responses (b) and non-Gaussian models (c) such as the Gaussian copula volatility model [82]
variational Gaussian processes (SVGPs) 33 have constant computational and memory footprints and are applicable non- Gaussian likelihoods, but they sacrifice closed form expressions for updated posteriors on receiving new data. The SVGP posterior is optimized through the evidence lower bound (ELBO). In the online setting, training with the ELBO has two primary difficulties: the need to specify a fixed number of observations to properly scale the ELBO gradient [8] and the need to adjust the inducing points without looking at past data [3] 9 Thus, we are presented with a choice between the simplicity and analytic tractability of exact GPs and the scalability and flexibility of SVGPs.
In this work, develop Online Variational Conditioning (OVC) allow SVGPs be conditioned on-the-fly, as shown in Figure In the top row of each subplot, we fit the data points shown in red, shifting to another batch of data points the bottom row. We use an exact GP in Figure Ia with exact conditioning shown in the bottom panel. The SVGP emulates the exact GP very well before seeing the new data and again after conditioning on the new data by using OVC (Figure lb In Figure Ic we consider non-Gaussian data model (a Gaussian copula volatility model (2) where we cannot use exact GPs;; the SVGP still able update its posterior over the latent volatility response to new data without "forgetting" old observations. OVC is inspired by a new, simple rederivation of streaming sparse GPs (O-SGPR), originally proposed by Bui et [ OVC makes SVGPs truly compelling models for online decision-making augmenting their existing strengths with efficient, closed-form conditioning on new data points. In short, our contributions are:
The development of OVC, novel method to condition SVGPs new data without re- optimizing the variational posterior through an evidence lower bound.
OVC provides both stable inducing point initialization for SVGPs while enabling the inducing points and variational parameters to update in response to the new data.
Enabling the effective application of SVGPs, through OVC, in look-ahead acquisitions in BO for black-box optimization, controlling dynamical systems, and active learning.
Please Appendix Alor discussion of the limitations and broader impacts of our work. Our code available at htttts:: //github cwm/ hoom/jjmaddox wowpoex//oonnnnn/ooooeeo www/oonnnneerrreeeeeeeeeeeeaaeee __aa__
2 Preliminaries and Related Work
In this section, we first review exact inference with Gaussian processes before reviewing variational sparse Gaussian processes, introducing sparse Gaussian process regression (SGPR), sparse variational Gaussian processes (SVGP), and streaming sparse GPs (O-SGPR). Please see Appendix B for further background and Appendix B.I specifically for further related work.
2.1 Gaussian Processes and Exact Inference
For full introduction to GPs, see Rasmussen and Williams 655 We begin by reviewing GP re- gression, with inputs E X' and responses y E E. GP regression assumes y NNff where

---

f~ ">GP( (x x')) is GP characterized by the kernel function kg and corresponding hyperpa- rameters O. Given Xtrin = [x1,.... xn] and xest = [x(,..... xX] we denote the corresponding function values as V = (
respectively. Note that and are random variables with prior covariance Cov[v, w| K vww :- ka (Xrainn Xtest). ). A well-known identity for multivariate conditional Gaussians allows us to compute the GP predictive posterior p(w X cest: D,0) -N (PWID: Swip) follows:
T and w = [f(x)), f(xb) )
WDID Ww (K + o²r)-
SwID Kww (Kvr + Eny) Kuws
where D == (X y) and Ey = 1²
Naively, computations with Krr cost (n²) space and (nn)) operations. When repeatedly com- puting p[w|Xpp xesst D, 0) at different sets of query points Xtest it is more efficient to cache the terms which depend only on the training data P Specifically, we store a == (Kvv + +,y (the predictive mean cache) and RRT := (Kw rxy -1 (the predictive covariance cache), resulting simplified forms for the predictive distribution, WWIp = Kwva and w Www Kwv R.R. Kvw- Adding a new observation equivalent to adding a single row and column to Kvv and an entry to y, which
enables efficient low-rank updates to the predictive caches [58 25
39
2.2 Variationally Sparse Gaussian Processes
Variational sparse GPs reduce the computational burden of GP inference through sparse approxima- tions of the kernel matrix. For further reference, see Matthews [50] and Van der Wilk 777. These methods define a variational distribution & over the inducing point values u = defined at inducing point locations Z [z].... :pp]" where z; E X. d(u) is parameterized as Gaussian with variational mean and covariance mu and S.- These methods assume the latent function values f(x), fxx) are conditionally independent given u and x, $ Z, so as to cheaply approximate the predictive posterior p( w/ w D,0) 2 q(w) (P wa)) Like exact GP regression, we can compute q(w) in closed form (given Mu, Sw).
Pwis= == Kwn Ku mus
( (Kun Sa) Ku) Kuw- Swip == Kww - Kwu K uu
Similarly the predictive mean and covariance caches are given by a Ki mu and RR Kuu (Km ( S., Kul reducing the complexity of inference from O(n²) to O(np²), which is a significant improvement if n..
There are two common approaches to finding optimal variational parameters ma and S.. In seminal work, Titsias 73 proposed sparse GP regression (SGPR), which optimizes mu and Su in closed form, resulting in "collapsed" evidence lower bound (ELBO) that only depends on 9 and Z. The computational cost of each gradient update to the remaining model parameters is still linear in n. and like exact GP regression, SGPR requires a Gaussian likelihood. Stochastic variational GPs (SVGPs) remedies both these limitations by using muddeett-based optimization to learn m, and S. alongside Z and 333 34. The SVGP objective is an "uncollapsed" ELBO which decomposes additively across the training examples, allowing gradients be estimated from minibatches of data, reducing the complexity of each gradient update to (bp) + y²), where b is the minibatch size.
We emphasize the distinction between constant- tintt minibatch gradients, and constant-ting condi- tioning. Given an SVGP already trained on some existing data, conditioning jointly on both the old and new data requires storing all the data and making multiple gradient updates to the variational parameters. As the size of the dataset grows, so does the number of gradient steps needed. In contrast by constant-time conditioning we mean a procedure that takes posterior conditioned data and produces a new posterior conditioned jointly on the old and new data with a fixed amount of compute and memory, regardless of the number of past observations.
existing
One example of constant-time conditioning is found in Bui et al. 2, who proposed streaming sparse GPs (which we call online SGPR, or O-SGPR, to distinguish from sparse spectrum GPs [44) ) for
'Cached expressions will be written in orange
What we use as the SGPR covariance cache is slightly different from the implementation in the prediction strategy in GPyTorch, which stores R = Kvu 11/ However, they reduce to the same strategy. 3 A lower bound of the true GP marginal likeliiodoo
3

---

Algorithm Online Variational Conditioning (OVC)
Input: Data batch ( (Xhatch: y), SVGP with inducing points Z' and o(u') = Mmm ,
1. Compute c', C" (Eq. 5)
2. Compute y n C cc' and Mi K C'K (Eq. 8
3. Construct GP with D b[att z"1, ly Gl) and Σ u''
blkdiia
4. Compute predictive mean and covariance caches, a and RR as in Section 2.1
5. Use caches compute conditioned GP posterior on test points, Xuest
incremental learning. We extend their work, providing alternative, simpler derivation of their model that highlights the connection with SGPR [73]. Furthermore, our perspective enables us to construct principled approach to updating inducing point locations new data arrives, that prevents the "forgetting" of old data induced by the resampling heuristic used by Bui et al. [9].
2.3 Bayesian Optimization and Monte Carlo Acquisitions
Bayesian optimization (BO) obtains x" aegminxex by constructing probabilistic surrogate model of f. which turn is used to evaluate an acquisition function. GPs are favored for BO due to their sample-efficieccy and efficient posterior sampling that enables cheap, gradient based opti- mization of the acquisition function propose new query points (4]. Many interesting acquisition functions look ahead into the future to see how the model will change if we query specific point, procedure known as "fantasization" [32] 84 39]. Fantasization is done by drawing samples from the
current surrogate posterior at some set of points and conditioning the surrogate on those samples. For is given by
example, the batch knowledge gradient [qKG, 84 2
a(x, :
E
max X'EX
E f(x)) (p)))pt))))))) s(x))
max x'EX f(x)>=((II))
E
f(x'),
(3)
/(x)-p(-ID)
where D+x = DU f(x)). The inner expectation in the first term requires conditioning the surrogate model posterior samples x, before optimizing through predictions of the conditioned surrogate model. The goal is to simulate the effect on the model we had observed the batch of data.
Use of sparse GPs in BO: Sparse GPs have not seen wide adoption in the BO community, with only several preliminary studies that have mostly used basic acquisitions. Nickson et al. [53] and Krityakierne and Ginsbourger [43] used expected improvement (EI) with SGPR
several test problems, while McIntire et al. [51 proposed a sparse GP method using EI to tune free electron lasers
(20). Stanton al. 770 proposed WISKI, an online implementation of scalable kernel approach called SKI [83, for low-dimensional BO problems using batch upper confidence bound (qUCB) [2]
3 Methodology
We now briefly describe the key ideas behind OVC with the goal of devising an efficient and stable method for updating the variational parameters with respect to newly observed data. We begin by highlighting an alternative parameterization of SGPR that will prove useful. Then we describe the OVC update to the variational distribution from two equivalent points of view, namely the projection view and the pseudo-data view. The pseudo-data view summarized in Algorithm [ Next we address a critical detail for good performance, which is how the inducing point locations should be selected. We then demonstrate how OVC can be applied to compute updated posterior distributions, e.g.
in Eq. 3 and quantities of the posterior, during gradient- mudiinttbbsedd acquisition function optimization in BO, with reference to how this can be performed practically in Scction Finally, we discuss how to apply OVC to models with non-Gaussian likelihoods.
3.1 Updating the Variational Posterior
We assume that we have trained a SVGP model (e.g. with the ELBO) on a fixed set of data and have already trained the inducing point locations and variational parameters, mu, Sn. Instead of the traditional m,, Sn parameterization used by Titsias (73), Hensman al. [33 34), we focus for now alternative parameterization which was favored in early work on sparse GP inference

---

66 57].
The parameterization is also similar to those used in both dual space functional variational
inference AT and expectation propagation [Q. More recently, Panos et al. [6T] used similar parameterization in the context of large scale multi-label learning with SVGPs.
The SGPR predictive posterior q(w) relies
terms dependent the training data,
Kuvuv E
C= Kuv Kvu.
(4)
where By is the covariance of the likelihood (fy) The optimal mu, Su are then given by
m m + c
Su Kn SamuKm,, (Kun + C)=²kmm
which can be substituted into Eq. (2] to obtain gww). Our first observation that if Eyy is block- diagonal, then and C' are additive across blocks of observations. For some intuition, consider i.i.d Gaussian noise (i.e. Ny = I²(,), which implies
C Σ
" (z, xj) v(zi) Σ o 2 yjo((;),
Ca = Σ σ x,)(,,,,,))),,,,,,,,,,,,,,,, = o(z,) T ((x))))) o(z)),
where ø is the (potentially infinite-dimensionall feature map associated with kg. Hence the entries of and C are both inner products between projected inducing points and weighted sums of features. For fixed inducing points, Z, and hyper-paramettess 0, we can use these updates to produce streaming version of SGPR by exploiting the additive structure of e and C. Furthermore, this streaming version of SGPR is exactly Gaussian conditioning for SGPR as we show in Appendix C.1 the inducing points and hyper- paraamertssssss to vary, which we address next
We can also allow
The projection view: We assume have c' = KKrr: y' and C" -"==aa K'
C"=K''''' 23y Ky computed with inducing point locations Z' from data (X y') with kernel hyperparameters 0' (using shorthand kg- K') 5 After obtaining the next parameters Z and 0 (perhaps from gradient based optimization of the ELBO), we observe new data (Xbatch y) and would like to continue with inference. One challenge is translating c', C' (whose elements are inner products of the old features) to the new feature space associated with To resolve this challenge, we construct a representative set of responses, y = PT c' and likelihood covariance Eg PT C'P P to project from the old feature space into the feature space by passing back through data space. The choice that minimizes reconstruction error is the pseudo-inverse P = (K (K' ) K but requires storage of the full dataset, (X Y'). Instead we take P P= K resulting in the following modifications to Eq.
c Kuv y + Kuu KuwKKrll Kirle', C = Kuv Kvu Knn' (Kard C' (KF Kuu')
(6)
Note that K = - Ey'y' and K u'u' = Ey' in the special case where Xbatch = Z'. We also want to emphasize that although we have only considered two batches of data for the sake of clarity, the approach applies any number of incoming batches.
The pseudo-data view:
The above update is equivalent having an SGPR model with a Gaussian likelihood with covariance Σ = blkdiag Egg, Ey) on the data (cat(2', X bbttch) cat(y, where
y = K K C'-ld' Eg Eg-= KT-l C"
This interpretation is reminiscent of prior online variational approaches of Csató and Opper [16] and Opper [56]. That is, in the context of conditioning a SVGP, we can assume that we began with data { and are now observing the new data (Xhutch, ). See Appendix C.2 for more details.
Extending to SVGPs: SGPR computes m, and S, function of and C' in Eq. (5) However the equations can be reversed to solve for e and C given m, and Sn, allowing us to condition any variational sparse GP into SGPR model, without touching any previous observations due the
'For full generality, we allow the hyper-paramene consider varying the inducing points as that's all we need to update when computing acquisition functions. "Cached computations that depend on (Xiatta y') are highlighted in blue.
to vary: however. our BO experiments. we only
5

---

conditional independence assumptions of variationally sparse GPs Note that if the variational parameters are not the optimal solution when the variational distribution is projected back the Dseudo-data, the projection will be to the targets and likelihood for which the current variational parameters would be optimal, which may not correspond well the data that originally created the model. This potential pitfall is mitigated if the variational parameters are well optimized and is offset by the practical advantages SVGPs.
Formally, the updates described in Eqs. (6) and 7 are equivalent to the O-SGPR approach of Bui et al. (9. as we show in Appendix C.2 The original derivation of O-SGPR is very technical, and does not highlight the similarities between the batch and online SGPR variants. Both the projection and pseudo-data views we have just described provide a much more intuitive way to reason about the behavior of O-SGPR models. Our formulation also eliminates a matrix subtraction operation, which is beneficial for numerical stability.
Connection to O-SGPR [
3.2 Inducing Point Selection
Here, we describe inducing point selection during the conditioning procedure to enable better variance reduction on new inputs. While heuristics including re-sampiing [9] and data sufficient statistics [35] have been proposed, they either require the number of inducing points to grow gradually forget old observations. We show in Appendix C.4 that relying exclusively on gradient-based optimization of inducing locations works very poorly the online setting.
To update the inducing point locations during conditioning, we extend Burt et al. (IJ's batch inducing point initialization approach to heteroskedastic Gaussian likelihoods. They con- sider theoretical bounds on the marginal likelihood, finding that for homoscedastic Gaussian likelihoods a good strategy is to minimize the trace of the error of rank-p Nyström approxi-
0.87
Resampled
Piv. Cholesky
0.86
RMSE
0.85
0.84
mation, e.g. E = tr [yy
1/2 (K - Kid K ) |Ey1/2)
0.83
-- 1///ttrKwv Qvv) for Qvv KvuKnn Kuv. They follow a classical approach of Fine and Scheinberg (23 by a greedy
0.82
minimiziation strategy: choosing as inducing points the pivots of a rank pivoted Cholesky factorization of Kvv-
10000 20000 30000 Number of Data Points
Incremental learning RMSE on the UCI protein dataset. Pivoted cholesky initialization out- performs resampling.
Figure 2:
We denote the function values over the batch++seudd dataset In our case the
as v = [f(x))..... (xx), fzz), f((,), f(22,)] covariance of the pseudo-likelihood is longer homoscedastic, so the slack term becomes and hence the pivoted Cholesky decomposition is instead per- formed over Σ' 1/ to select the top p pivots of the p + Rnew matrix. When compared to re-sampling the inducing points [91, pivoted cholesky updates perform significantly better, as shown in Figure 2 on the UCI protein dataset [99] Experimental details are given Appendix D.2
tr(E 1/2 - Q++) 2x+))-112 1/2)
Application to Bayesian Optimization: In the context of BO, we condition on hypothetical observations, and the conditioned surrogates are discarded after each acquisition function evaluation. Since the SVGP will not be conditioned on more than a few batches of observations, we can sidestep the issue of updating inducing locations entirely by instead conditioning into an exact GP trained the combined pseudo-data through the pseudo-likelihood That is, we model the data as (y: y) ~ (J,, E) (Gaussian with block -ock-diagonal covariance) assuming f GP ki (;, )). We reach the same model by choosing X as the inducing points in our conditioned SGPR (Section 3.1 For small n,, an exact GP is not much slower, taking only (n, + p) computations instead of p² computations, further reduced by using low rank updates.
3.3 Local Laplace Approximations for Non-Gaussian Observations
Thus far, we have solely considered Gaussian observations. The introduction of non-Gaussian likelihood presents new challenge, since it implies that the current observation batch and the
Alternatively, we could construct SVGPs via direct optimization of and C.

---

10 Achieved
Beal Achioverd
3.1
Beat Actieved
2.5
Best Achiervdd
3.1
2.0
SVBP KG
SVGP + KG
SVGP + NEI
SVSP KG
SVGP NEI
2.9
1.5
SVGP
SVGP
Laplaoe
Exadt NEI
+I1
SVGP NE1
Random
2.8
1,0 100
2.8
100 Function Evalustions
Fundtion EvalLations
160
200
100 Functon Evalustions
25 Function Evaluations
50
75 100
(a) Hartmann6
(b) Laser
(c) Poisson-Hartmanne Hartmann6
(d) Preference Learning
Figure 3: (a) Hartmann6 test problem with constraint. Here, SVGPs with noisy expected improvement (qNEI) and KG match the performance of exact GPs with qNEI and qKG. (b) Free electron laser problem from McIntire et al. [51]: SVGPs with knowledge gradient outperforming weighted sparse GPs. (c) Constrained Hartmann6 test problem with count responses (Poisson likelihood). Only SVGPs can be used here, and qKG outperforms QNEI. (d) Preference learning; SVGPs with qKG are similar to Laplace approximations with NEI, and outperform qÑEI with SVGPs.
pseudo-data are no longer jointly Gaussian. To adapt the conditioning procedure to the non- non-Gaussian setting, we can simply perform Laplace approximation of the likelihood at the new points [65
Ch. 3]. Specifically, this gives us an approximate likelihood, ply f) = Ni; f. H. where and (") is the maximizer of pgg + fTK K-'KK1pp f. computed via Newton iteration i When conditioning on new observations y. we substitute (y) instead. That is, our new model has responses (y) v instead of (y, and the pscudo-likclihood remains Gaussian with covariance Σ = blkdiag H. 2g)). We primarily consider natural parameterizations of one-dimensional exponential families, so that H. positive, diagonal and depends solely on f. Computing V log p(Ulf) and H. is possible by hand but one can also use automatic differentiation [AD, 63 In Appendix D Figure 8 we show the effect of repeated Laplace approximations across several batches for online classification.
H. = V3 poy
4 Experiments
Our experimental evaluation demonstrates that SVGPs using OVC can be successfully used as surrogate models with advanced acquisition functions in Bayesian optimization, even in the large batch and non-Gaussian settings. In keeping with the BO literature, we will refer to the query hatch size as q (not to be confused with the variational posterior q(f) in previous sections). All SVGP models that use conditioning (or fantasization) require OVC to even be practical to implement.
Using OVC as a Building Block inside of BO
In all of our experiments, a standard BO acquisition function that requires fantasiziation. These acquisitions are generally "look-ahead" as result; specifically, qKG [2] 39]. LTSs, our version of GGIBBON which uses fantasy batch [54). and qMultiStepL.ook in OVC as the condition.c simply optimize gKG or GGIBBON with an SVGP exactly as an exact GP surrogate, by using gradient based optimizers such L-BFGS-B. In general, need to differentiate through the fantasy model with respect to the inputs and then use gradient based methods to find the optimum. Please see Balandat al. 2 and Frazier 24 for description of how a BO loop is constructed and how acquisition functions are optimized.
OVC building block enable fantasization (Algorithm[I) within
33 all use the fantasization model. After adding observations function within a BoTorch model class [2]. we can
Experimental Setup general, Bayesian optimization loop consists of the steps of training the model and then using the trained model to optimize an acquisition function to acquire new data points,
"ne could consider using the posterior covariance instead of K. Our experiments with the posterior covariance produced more extreme values of f and thus less regularization. "Specifically, we use PyTorch's functional htlpll/pp//ppoooorr..o/////////s hunctional-highe hiiherr-evee--ap. level-api
7

---

which are then added into the training data for the next model. All experiments use PyTorch [[2]. GPyTorch (25], and BoTorch [2]] Unless otherwise specified, we run each experiment 50 times and report the mean and two standard deviations of the mean.
In the first step, first train the inducing points, variational distribution, and kernel hyper-parametern using the evidence lower bound given in Eq. A.3 As all components are differentiable, we use the Adam optimizer with a learning rate of 0.1 and optimize for 1000 steps or until the loss converges, whichever is shorter. To initialize the inducing points, we compute a pivoted cholesky factorization on the initial kernel on the training data (described in Section (.2ioonnn.fooooooooo following Burt et al. (T]). The kernel hyper- parameters are initialized to GPyTorch defaults (which sets all lengthscales to one), while the variational distribution is initialized mu Su = I (again, GPy defaults). Further experimental details and dataset descriptions are in the Appendix.
4.1 Knowledge Gradient with SVGPs
These experiments use the one-shot formulation of the batch knowledge gradient (qKG) (Eq. from Balandat et al. (2). who demonstrated that qKG outperforms other acquisitions due being able to plan two steps into the future. Using and optimizing qKG has only been available for exact GPs previously. By using OVC, we have enabled SVGPs to also efficiently and tractably optimize qKG, even for non-Gaussian observations. We compare to batch noisy expected improvement [qNEI, which is myopic and does use fantasization (e.g. conditioning). Here, for the SVGPs used min (N, 25) inducing points.
Gaussian observations: We the Hartmann6 test function, with one black box constraint, Ar - P.)) subject to the constraint that c(z) = ||^||| for fixed A, P. a. We use 10 initial points and batch size of 3 optimizing for 50 iterations, comparing to SVGPs and exact GPs using QNEI. We show the results in Figure Ga where SVGPs
maximizing f(x) = =- -i-r Or
with qKG match exact GPs with both QNEI and qKG, and outperform SVGPs using NEI.
Second, we mimic the laser tuning experiment of [51] 20), demonstrating that SVGPs outperform even weighted online GPs (WOGP), which were designed for this task. Here, we use 100 initial points, with d = 14, and and wish to tune a laser's output energy as function of the magnet settings that produce the beam. Like McIntire et al. [5I]] we treat pretrained GP fit experimental data as a simulator. We use a batch size of 1, finding that SVGPs + KG outperform WOGP (Figure (bp. However, exact GPs outperform the variational approaches (Appendix Fig. Pb).
Next, we extend the knowledge gradient problems with non- non-Gaussian likelihoods. First, we take the constrained Hartmannó test function from the previous section, and use Poisson responses, y ~ Poisson( ( (f)))) repeating the same settings as for the Gaussian case. Now, the data is non-Gaussian and cannot be well-modelled by Gaussian likelihood, so we compare only SVGPs with QNEI. QNEI is outperformed by qKG, as shown in Figure Bc
Non-Gaussian likelihoods:
Second, in Figure 3d we consider a preference learning problem inspired by Lin et al. [46]. Here, the latent data is described by f(x) = - 110-1/2 1/2 E vi cos(2 for æ E [0, 1] comparing to Laplace approximations [15). Again, we see that SVGPs with qKG outperform qNEI with both
SVGPs and Laplace approximations.
4.2 Active Learning of Disease Incidence
We next present results for two active learning tasks governing the collection of disease incidence data. In both tasks the acquisition functions again require efficient conditioning hypothetical data, and the second task has Binomial responses, so exact GPs cannot be applied. In both settings, applying OVC to SVGPs gives strong results competitive with either exact GPs random forests.
We consider data from the Malaria Global Atlas [8] describing the infection rate of parasite known to cause malaria in 2017. We wish to choose spatial locations to query malaria incidence in order to make the best possible predictions on a withheld test set, the entire country of Nigeria. Following Stanton et al. [70], we minimize the negative integrated posterior variance [NIPV, 68), defined as a(x; D) == - Sx {x (D))xx again with D+x = DU [(x, y)). Intuitively, the minimizer of this acquisition will the batch of data points that when added into the model will most reduce the total posterior uncertainty across the domain,
Modelling of Malaria Incidence:
8

---

0.28
0.025
SVGP MPV
SVGP Entrapy Search
NPV
0.024
SVGP Random
0.07
SVGP- Random
NIPV
Hobopol Ancuracy
0.88
0.02
KiigngRFF Random
KigingRF Search
RMSE
Exect- Random
0.020
0.00
Random
0.84
0.018
0.06
0.04
0.82
20
40
100
Acquited
Steps
(a) Malaria incidence in Nigeria
(b) Hotspot prediction accuracy
(c) Prevalence modelling
Figure 4: (a) Active learning of malaria incidence from satellite data. Using qNIPV outperforms randomly selecting points, while the SVGPs slightly outperform both exact GPs and WISKI. (b,c) Active learning of schistomiasis incidence in Cote Ivoire from Andrade-Pachedt et al. [1] Compar- ison is to the random forest based approach using active sampling. While the GP based models somewhat less accurate at predicting hotspots (b), they are better as global model of prevalence (c).
requiring efficient conditioning to do so in a tractable manner. The results are shown for a batch size qf = across 15 trials Figure 4a where we see that each method outperforms random baselines. Perhaps due to the optimization freedom, the SVGP outperforms both the exact GP and WISKI [70].
Hotspot Modelling: We follow Andrade-Pacheco al. M and model the prevalence of schisto- miatosis in Côte d'I d'Ivo using simulated responses from 1500 villages in that country, and taking into account six other demographic variables. We model the responses y (incidence) locations x with population n(x) with Binomial likelihood p(ylS:x x) ~ Binomial(n(x) r(f)), where r(f) = (1 exp(-f)) Letting T (0, 1) be threshold on the prevalence for a location to be considered a "hotspot"
[ we compute the entropy:
1
h,(x, D) Epp p) (H (Bermoulll (ogit(r))))
Σ
logght)))) 1 (Beroooull))
K
taking the acquisition value to be the reduction in the entropy of the posterior predictive distribution over the incidence under the hotspot-focused likelihood.
a, (x, D) = [ [x', (x',,, D))dd'.
A location is given a high acquisition value if observing the incidence at that location reduces the uncertainty of the model on the predicted set of hotspots. In Figure 4b) we compare Andrade- Pacheco et al. [I] who use spatial kriging on the residuals of random forest model. Both their random baseline and their exploration based procedure (a variant of UCB) start off with higher prediction accuracy; however our SVGP models ultimately outperform the kriging approach with random selection. The SVGP is a better predictor of true prevalence, as shown in Figure 4c both cases, our acquisition function significantly outperforms random selection with a SVGP surrogate.
4.3 Rollouts within Thompson Sampling for High Dimensional BO
For our final set of experiments, we solve control problems using trust region Bayesian optimization [TurBO, (22]. Inspired by multi-step look-aheads (39 (4. we propose l--tep look-ahead Thompson sampling (LTS-h). In BO, Thompson sampling (TS) is often implemented by drawing samples from the posterior at points all over the domain, then selecting the best to form a query batch 771 (22 LTS-h extends the idea by conditioning the surrogate independently on each posterior sample in the original TS query batch, Thompson sampling again from the updated posterior with a new set of points, and appending the best sample its predecessor to form a path. The process is repeated times. Finally, we condition the original surrogate jointly on each path, then perform TS again to choose the new query batch. Informally, each path corresponds p(f allowing the inner-loop to refine its guess of the global optimum for different f. and the final round of TS chooses the query batch based on those guesses. See Appendix C.5 a formal description. Like other look-ahead acquisitions, LTSSh is only practical if posterior conditioning and samples are very efficient and numerically stable. LTTSh is conceptually similar to path sampling for ahead in Jiang et al. 39 and kriging believer (26
distinet, coherent draw from

---

Exact TS
SVGP TS
Exact LTS-4
SVGP LTS-4
Reward Threshold
.... ARS
4.0
Beal Achieved
3.5
Adhiennd
3.5
Beel Aghievedd
-300
2900
3.0
2000
2.5
2.0
2.0
200
1000
600
1.0
5000 10030 16000 20000 Fundtinn Queries
1000 2000 3000
4000
Watcook (min)
Quarlas
Funztion Querees
(a) Wall-Clock time, rover,
(b) rover,
SW swimmer,
(d) hopper, d = 33
d 60
d = 60
d 16
Figure 5: Multi-step rollouts with OVC and SVGPs provides sampic-complexity and wall-clock time improvements on high dimensional BO problems when using TurBO and LAMCTS [22] 78). SVGP rollouts time efficient (to 150 iterations) standard TS (a) on lunar rover, d = 60. (c-d) MuloCo environments using LAMCTS + TurBO. Also shown is the reward threshold (dashed grey lines) and augmented random search (ARS)'s performance (dotted red lines) [49]. The median and its 95% confidence interval are shown over 24 trials for rover and 10 trials for swimmer and hopper.
For validation, we consider tuning the 60 dimensional path that lunar rover takes across a field stacked with obstacles 779 22]. We use batches of q 100 with 200 initial points, and use trust region Bayesian optimization (TurBO) to split up the space effectively [22], comparing to TurBO with Thompson sampling (TS) as the acquisition [7]]. We show the wall clock times per iteration
in Figure 5a where we see that the TurBO + LTS approaches compare well in wallclock time to TurBO+TS, while being more function efficient (Figure (5b).
Finally, we consider MuJoCo problems using the OpenAl gym 75 7 with LTSs inside of TurBO with trust regions generated by Monte Carlo Tree Search following the procedure of Wang et al. [78]. Following Wang et al. [78]. we learn a linear policy and consider the swimmer-v2 hopper-v2 environments over 10 trials displaying the median and its 95% confidence band due to high variance. On both problems, SVGP with LTSS tend to be the most sample efficient, with SVGP + TS performing least as well on swimmer-v2. We also show the reward threshold and the performance achieved by augmented random search (ARS), which is strong baseline reinforcement learning method that uses random search to tune linear controllers [49. Our results suggest that LTSs are promising overall; however, more work needs to be done for high dimensional kernlss on these problems.
Discussion
In conclusion, we have demonstrated how to efficiently condition on new data points with stochastic variational Gaussian processes via closed form updates to the variational distribution. Our condi- tioning approach generalizes exact GP conditioning via Laplace approximations for non-Gaussian likelihoods. As a result, we have decoupled look-ahead BO acquisition functions from their depen- dence on exact GP inference through a Gaussian likelihood, increasing the range, scale, and efficiency of BO applications. In the future, we hope to extend OVC to multi-task and deep Gaussian processes for use in Bayesian optimization
Acknowledgements
WJM, SS, AGW are supported by an Amazon Research Award, NSF I-DISRE 193471, NIH RO1 DA048764-01AA NSF IIS- 1910266, and NSF 1922658 NRT-HDR: FUTURE Foundations, Transla- tion, and Responsibility for Data Science. WJM was additionally supported by an NSF Graduate Research Fellowship under Grant No. DGE- States Department of Defense through the National Defense Science & Engineering Graduate (ND- SEG) Fellowship Program. We'd like to thank Greg Benton for setting up the volatility experiment and for helpful discussions and Nate Gruver and Eytan Bakshy for helpful comments.
DGE-1839302. 839302. SS is additionally supported by the United
10

---

References
[1] Andrade-Pacheco, R., Rerolle, F., Lemoine. J., Hernandez, L... Meité, A., Juziwelo, L.,, Bibaut, A. F., van der Laan, M. J., Arnold, B. F., and Sturrock, H. (2020). Finding hotspots: development of an adaptive spatial sampling approach. Scientific reports, 10(1):1-12.
[2] Balandat, M., Karrer, B., Jiang. D., Daulton, S., Letham, B., Wilson, A. G., and Bakshy, (2020). BoTorch: A Framework for Efficient Monte-Carle Bayesian Optimization. In Advances in Neural Information Processing Systems, volume 33.
[3] Bauer, M., Van Der Wilk, M., and Rasmussen, C. (2016). Understanding probabilistic sparse gaus- sian process approximations. Advances in Neural Information Processing Systems, volume 29, pages 1533-1541.
[4] Bertsekas, D. P. (2020). Rollout, Policy Iteration, and Distributed Reinforcement Learning. Athena Scientific.
[5] Bijl, H., Schön, T. B., van Wingerden, J.-W., and Verhaegen, M. (2016). Online sparse gaussian process training with input noise. arXiv preprint arXiv 1601.08068.
[6] Boedecker, J., Springenberg.1. J. T., Wülfing, J., and Riedmiller, M. (2014). Approximate real-time optimal control based on sparse gaussian process models. 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pages 1-8. IEEE.
[7] Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang. J., and Zaremba, W. (2016). Openai gym. arXiv preprint arXiv: 1606.0055..
[8] Broderick, T., Boyd, N., Wibisono, A., Wilson, A. C., and Jordan, M. (2013). Streaming variational bayes. In Advances in Neural Information Processing Systems, volume 26.
[9] Bui, T. D., Nguyen, C. V., and Turner, R. E. (2017a). Streaming sparse Gaussian process approximations. In Advances in Neural Information Processing Systems, volume 31, pages 3301-3309, Long Beach, California, USA. Curran Associates Inc.
[10] Bui, T. D., Yan, J., and Turner, R. E. (2017b). A unifying framework for gaussian process pseudo-point approximations using power expectation propagation. The Journal of Machine Learning Research, 18(1):3649-3720.
[11] Burt, D., Rasmussen, C. E., and Van Der Wilk, M. (2019). Rates of convergence for sparse variational Gaussian process regression. In Proceedings of the 36th International Conference on Machine Learning, volume 97, pages 862-87 172.871.7 PMLR.
[12] Canu, S. and Smola, A. (2006). Kernel methods and the exponential family. Neurocomputing. 697))7144-700 New Issues in Neurocomputing: 13th European Symposium on Artificial Neural Networks.
[13] Cheng, C.-A. and Boots, B. (2016). Incremental variational sparse Gaussian process regression. In Advances in Neural Information Processing Systems, volume 30, pages 4410-4418, Barcelona, Spain. Curran Associates Inc
[14] Chowdhary, G., Kingravi, H. A., How, J. P., and Vela, A. (2014). Bayesian nonparametric adaptive control using gaussian processes. IEEE transactions on neural networks and learning systems, 26(3):537-550.
[15] Chu, W. and Ghahramani, Z. (2005). Preference learning with gaussian processes. In Proceed- ings of the 22nd International Conference on Machine learning, pages 137-144.
[16] Csató, L. and Opper, M. (2002). Sparse on-line gaussian processes. Neural computation. 14(3):641-66)
[17] Cutajar, K., Pullin, M., Damianou, A., Lawrence, N., and González, J. (2019). Deep gaussian processes for multi-fidelity modeling. arXiv preprint arXiv:1903.073:
11

---

[18] Deisenroth, M. and Rasmussen, C. E. (2011). Pilco: A model-based and data- ceta-efficceet approach to policy search. In Proceedings of the 28th International Conference on Machine Learning, pages 465 4722.
[19] Dua, D. and Graff, C. (2017). UCI machine learning repository.
[20] Duris, J., Kennedy, D., Hanuka, A., Shtalenkova, J., Edelen, A., Baxevanis, P., Egger, A., Cope, T., McIntire, M., Ermon, S., et al. (2020). Bayesian optimization of a free-electron laser. Physical
review letters, 124(12):124801.
[21] Eriksson, D. and Jankowiak, M. (2021). High-dimensional bayesian optimization with sparse axis-aligned subspaces. arXiv preprint arXiv:2103.00344
[22] Eriksson, D., Pearce, M., Gardner, J., Turner, R. D., and Poloczek, M. (2019). Scalable global optimization via local bayesian optimization. Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
[23] Fine, S. and Scheinberg, K. (2001). Efficient svm training using low-rank kereel representations. Journal of Machine Learning Research, 2(Dec):243 264.
[24] Frazier, P. I. (2018). A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811
[25] Gardner, J. R., Pleiss, G., Bindel, D., Weinberger, K. Q., and Wilson, A. G. (2018). GPyTorch: Blackbox Matrix-Matris Gaussian Process Inference with GPU Acceleration. In Advances in Neural Information Processing Systems, volume 31.
[26] Ginsbourger, D., Le Riche, R., and Carraro, L. (2010). Kriging is well-suited to parallclize optimization. In Computational intelligence in expensive optimization problems, pages 131-162. Springer.
(27] Girard, A., Rasmussen, C. E., Candela, J. Q., and Murray-Smith, R. (2002). Gaussian process priors with uncertain inputs-application to multiple-step ahead time series forecasting. In Advances in Neural Information Processing Systems, volume 15.
[28] Gómez-Bombarell R., Wei, J. N., Duvenaud, D., Hernndezz-Lbaute J. M., Sänchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguire J., Hirzel, T. D., Adams, R. P., and Aspuru-Guzin A. (2018). Automatic chemical design using a data-driven continuous representation of molecules. ACS central science, 4(2):268-22 276.
[29] Groot, P., Lucas, P., and Bosch, P. (2011). Multiple-step time series forecasting with sparse gaussian processes. Causmaecker; P. De (ed.). Proceedings of the 23rd Benelux Conference on Artificial Intelligence, pages 1-8. [SI: sn].
[30] Hagan, P. S., Kumar, D., Lesniewski, A. S., and Woodward, D. E. (2002). Managing smile risk. The Best of Wilmouf, 1:249-296.
[31] Hebbal, A., Brevault, L., Balesdent, M., Talbi, E.-G., and Melab, N. (2021). Bayesian optimiza- tion using deep gaussian processes with applications to aerospace system design. Optimization and Engineering, 22(1):321-361
[32] Hennig, P. and Schuler, J. (2012). Entropy search for information-eeffccint global optimization. Journal of Machine Learning Research, 13(6).
[33] Hensman, J., Fusi, N., and Lawrence, N. D. (2013). Gaussian processes for big data. In Proceedings of the Twetty-Ninth Conference on Uncertainty in Artificial Intelligence, UAI' 13, page 282-290, Arlington, Virginia, USA. AUAI Press.
[34] Hensman, J., Matthews, A.. and Ghahramani, Z. (2015). Scalable Variational Gaussian Process Classification. Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, pages 351-360 San Diego, California, USA. PMLR.
[35] Hoang, T. N., Hoang, Q. M., and Low, B. K. H. (2015). A unifying framework of anytime sparse gaussian process regression models with stochastic variational inference for big data. In International Conference on Machine Learning, volume 37, pages 569-578. PMLR.
12

---

[36] Huber, M. F. (2013). Recursive gaussian process regression. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 3362-3366. IEEE.
[37] Huber, M. F. (2014). Recursive gaussian process: On-line regression and learning. Puttern Recognition Letters, 45:85-91.
[38] Jankowiak, M., Pleiss, G., and Gardner, (2020), Parametric gaussian process regressors. In International Conference on Machine Learning, volume 119, pages 4702 4712. PMLR.
[39] Jiang, S., Jiang, D., Balandat, M., Karrer, B., Gardner, J., and Gamnett, (2020). Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees. In Advances in Neural Information Processing Systems, volume 33.
[40] Kapoor, S., Karaletsos, T., and Bui, T. D. (2020). Variational auto-regressive gaussian processes for continual learning. arXiv preprint arXiv:2006.05466.
[41] Khan, M. and Lin, W. (2017). Conjugate-computall variational inference: Converting variational inference in non-conjugate models to inferences in conjugate models. In Artificial Intelligence and Statistics, pages 878-887. PMLR.
[42] Knudde, N., van der Herten, J., Dhaene, T., and Couckuyt, I. (2017). GPilowOpt: A Bayesian Optimization Library using TensorFlow. arXiv preprint 171.08475
[43] Krityakierne, T. and Ginsbourger. D. (2015). Global optimization with sparse and local gaussian process models. In International Workshop on Machine Learning. Optimization and Big Data, pages 185-196. Springer.
[44] Lázaro-Gredilla M., Quinonero-Candela, J., Rasmussen, C. E.,, and Figueiras- Vineir,,--Vial,, A. R. (2010). Sparse spectrum gaussian process regression. The Journal of Machine Learning Research, 11:1865-1881.
[45] Letham, B., Karrer, B., Ottoni, G., Bakshy, E., et al. (2019). Constrained bayesian optimization with noisy experiments. Bayesian Analysis, 14(2):495-519
[46] Lin, J., Obeng, A., and Bakshy, (2020). Preference learning for real-world multi-ohjective decision making. ICML 2020 Workshop on Real World Experiment Design and Active Learning.
[47] Ling, C. K., Low, K. H., and Jaillet, P. (2016). Gaussian process planning with lipschitz continuous reward functions: Towards unifying bayesian optimization, active learning, and beyond. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30.
[48] MacKenzie, D. and Spears, T. (2014). 'the formula that killed wall street': : The gaussian copula and modelling practices in investment banking. Social Studies of Science, 4413)
[49] Mania, H., Guy, A., and Recht, B. (2018). Simple random search of static linear policies is competitive for reinforcement learning. In Advances in Neural Information Processing Systems, volume 31, pages 1805-1814.
[50] Matthews, A. G. d. G. (2017). Scalable Gaussian process inference using variational methods. PhD thesis, University of Cambridge.
[51] McIntire, M., Ratner, D., and Ermon, S. (2016). Sparse gaussian processes for bayesian optimization. In Uncertainty Artifical Intelligence, volume 32. Association for Uncertainty in Artifical Intelligence (AUAI).
[52] Moreno-Muñoz, P., Artés-Rodríguez, A., and Álvarez, M. A. (2019). Continual multi-task gaussian processes. arXiv preprint arXiv: 1.00002.
[53] Moreno-Muñoz, P., Artés-Rodríguez, A., and Álvarez, M. A. (2020). Recyclable gaussian processes. arXiv preprint arXiv:2010.02554.
[54] Moss, H. B., Leslie, D. S., Gonzalez, J., and Rayson, P. (2021). Gibbon: General-purpose information- baased bayesian optimisation. arXiv preprint 4r22v:2102.03324.
13

---

[55] Nickson, T., Osborne, M. A., Reece, S., and Roberts, S. (2014). Automated machine learning using stochastic algorithm tuning. In NIPS Workshop on Bayesian Optimization.
[56] Opper, M. (1998). bayesian approach to online learning. On-line learning in neural networks, pages 363-378.
[57] Opper, M. and Archambeau, C. (2009). The variational gaussian approximation revisited. Neural computation, 21(3):78--792.
[58] Osboree, M. A. (2010). Bayesian Gaussian processes for sequential prediction, optimisation and quadrature. PhD thesis, Oxford University, UK.
[59] Pan, P. Swaroop, S., Immer, A., Eschenhagen, R., Turner, R. E., and Khan, M. E. (2020). Contin- ual deep learning by functional regularisation of memorable past. arXiv preprint arXiv:2004. 14070.
[60] Pan, Y., Yan, X., Theodorou, E. A., and Boots, B. (2017). Prediction under uncertainty in sparse spectrum gaussian processes with applications to filtering and control. In International Conference on Machine Learning, volume 70, pages 2760-2766 PMLR.
[61] Panos, A., Dellaportas, P., and Titsias, M. K. (2018). Fully scalable gaussian processes using subspace inducing inputs. arXiv preprint arXiv: / /807.02537.
[62] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang. E., De Vito. Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019). Pytorch: An imperative style, pirh-pperfrrance deep learning library. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
[63] Pearlmutter, B. A. (1994). Fast exact multiplication by the hessian. Neural computation, 6(1) 6(1):147-160.
[64] Pleiss, G., Gardner, J. R., Weinberger, Q., and Wilson, G. (2018). Constant-Tine Time Predictive Distributions for Gaussian Processes, In Artificial Intelligence and Statistics, volume 84, PMLR.
[65] Rasmussen, F. and Williams, C. K. I. (2008). Gaussian processes for machine learning. Adaptive computation and machine learning. MIT Press, Cambridge, Mass., 3. print edition.
[66] Sæmundsson, S., Hofmann, K., and Deisenroth, M. (2018). Meta reinforcement learning with latent variable gaussian processes. 34th Conference on Uncertainty in Artificial Intelligence, volume 34, pages 642-652, Association for Uncertainty in Artificial Intelligence (AUAI).
[67] Seeger, M. W., Williams, C. K., and Lawrence, N. D. (2003). Fast forward selection to speed up sparse gaussian process regression. International Workshop on Artificial Intelligence and Statistics, pages 254-261. PMLR.
[68] Seo, S., Wallat, M., Graepel, T., and Obermayer, K. (2000). Gaussian process regression: Active data selection and test point rejection. In Neural Networks, IEEE-INNS-ENNS International Joint Conference on, volume 3, pages 3241-3241.
[69] Shi, J., Titsias, M., and Mnih, A. (2020). Sparse orthogonal variational inference for gaussian processes. In International Conference on Artificial Intelligence and Statistics, volume 108, pages 1932-1942. PMLR.
[70] Stanton, S., Maddox, W. Delbridge, I., and Wilson, A. G. (2021). Kernel Interpolation for Scalable Online Gaussian Processes. In Artificiaa Intelligence and Statistics, volume 130. PMLR.
[71] Thompson, W. R. (1933). On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3/4):285-294
[72] Titsias, M. (2008). Variational Model Selection for Sparse Gaussian Process Regression. www.aueb. www.:aelwwyweeeetttttcccpppppepp
page 20.
[73] Titsias, M. (2009). Variational Learning of Inducing Variables in Sparse Gaussian Processes. In Artificial Intelligence and Statistics, pages 567-574. PMLR. ISSN: 1938-7228.
14

---

[74] Titsias, M. K., Schwarz, J., Matthews, A. G. d. G., Pascanu, R., and Teh, Y. W. (2019). Functional regularisation for continual learning with gaussian processes. International Conference on Learning Representations.
[75] Todorov, E., Erez, T., and Tassa, Y. (2012). Mujoco: A physics engine for model-based control. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026- 5033. IEEE.
[76] Tripp, A., Daxberger, E., and Hernández-Lobato, J. M. (2020). Sample-efficient optimization in the latent space of deep generative models via weighted retraining. Advances in Neural Information Processing Systems, 33.
[77] Van der Wilk, M. (2019). Sparse Gaussian process approximations and applications. PhD thesis, University of Cambridge.
[78] Wang, I.., Fonseca, R., and Tian, Y. (2020). Learning search space partition for black-box optimization using monte carlo tree search. Advances Neural Information Processing Systems, volume 33.
[79] Wang, Z., Gchring, C., Kohli, P., and Jegelka. S. (2018). Batched large-scale bayesian opti- mization high-dimensiona spaces. In International Conference on Artificial Intelligence and Statistics, volume 84, pages 745-754. PMLR.
[80] Wang, Z. and Jegelka, S. (2017). Max-value entropy search for efficient bayesian optimization. In International Conference on Machine Learning, volume 70, pages 3627-3635. PMLR.
[81] Weiss, D. J., Lucas, T. C., Nguyen, M., Nandi, A. K., Bisanzio, D., Battle, K. E., Cameron, E., Twohig, K. A., Pfeffer, D. A., Rozier, J. A., al. (2019). Mapping the global prevalence, incidence, and mortality of plasmodium falciparum, 2000-17: spatial and temporal modelling study. The Lancet, 394( (0195))333):
[82] Wilson, A. and Ghahramani, Z. (2010). Copula Processes. In Advances in Neural Information Processing Systems, volume 23, page 9.
[83] Wilson, A. and Nickisch, H. (2015). Kernll Interpolation for Scalable Structured Gaussian Processes (KISS-GP). In International Conference on Machine Learning, pages 1775-1784. ISSN: 1938-7228 Section: Machine Learning.
[84] Wu, and Frazier, P. I. (2016). The parallel knowledge gradient method for batch Bayesian optimization. In Advances in Neural Information Processing Systems, volume 30 of NIPS' 16, pages 3134-3142, Red Hook, NY, USA. Curran Associates Inc.
[85] Xu, N., Low, K. H., Chen, J., Lim, K. K., and Ozgul, F. (2014). Gp-localize: Persistent mobile robot localization using online sparse gaussian process observation model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28.
15


Reviews:
[{"review_id": "cSdBbGqK_U", "reviewer": "Reviewer_pE77", "summary": "The streaming sparse GP (named O-SGPR in this paper) is a variational GP approximation that supports updating the variational posteriors and hyper-parameters online. This paper reinterprets O-SGPR as compressing the past observations into a small set of \"pseudo-data\". Then updating the posterior based on new observations can be simply treated as a standard GP inference based on the new observation and the pseudo-data. This paper adopts the proposed Online Variational Conditioning (OVC) over Bayesian optimization and active learning, where inferring the updated belief from new observations is important to quantify the contained information. This paper is well-written and conducts extensive experiments to verify the proposed approach.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Method\n### Novelty and Significance\nEnabling variational GPs for online posterior inference is an important problem. The streaming sparse GP (named O-SGPR in this paper) is a landmark approach that enables updating both the posterior belief and the hyper-parameters for variational GPs. This paper reinterprets O-SGPR in terms of performing GP inference over the new observations and \"pseudo data\", which I think is a novel contribution. \n\n### Relations to O-SGPR\nAs discussed in the paper, OVC is based on a new interpretation of the O-SGPR. However, I think it deserves more discussions on their differences. Based on my understanding, O-SGPR updates the variational posterior by maximizing the ELBO, which prevents it from being directly plugged into the Bayesian optimization problems. In contrast, the OVC directly obtains the posterior by the \"exact GP formulation\", enabling the explicit reasoning in Bayesian optimizations. However, OVC sacrifices the general applicability towards non-Gaussian likelihoods and turns to Laplacian approximations. \n\nI think this difference is the major contribution in this paper, given the existence of O-SGPR. And it is the key for being applied in Bayesian optimizations. But it is not obvious from reading the paper.\n\n###  Relations to standard SVGP\nImagine that we have the inducing points $Z_{t-1}$ and the variational posterior $ \\phi_{t-1}(u') = \\mathcal{N} (\\mu, S) $, then the current posterior is a Gaussian process \n$$\nGP_{t-1} = \\mathcal{GP}( k(\\cdot, Z) k(Z, Z)^{-1} \\mu, k(\\cdot, \\cdot) - k(\\cdot, Z) k(Z, Z)^{-1} k(Z, \\cdot)  ),\n$$\nGiven any new observations, we can also infer the new posterior belief under this $GP_{t-1}$.\n\nIs OVC equivalent to this approach ? If not, what is the advantage of OVC against it ? I noticed that Line147-150 might be relevant, but the two shortcomings listed there apply to OVC as well in my perspective.\n\n\n## Clarity\nThis paper is well written in general. Here are some minor comments,\n1) Line 158: define $p_t, q_t$ before using them.\n2)  Inducing Point Selection: do you select over \"[new_obs, pseudo_data]\", or  keep all \"pseudo_data\" and only select from the \"new_obs\" ? \n3) Figure 2: The captions for b,c,d  are not clear. It is not clear to me what is the difference between (c) and (d).\n4) Line 277: the notation $x$ is used in both $a(x; D)$ and $dx$.\n\n## Experiments\n1) Do you keep a fixed budget for the number of inducing points, or add a fixed number of inducing points per iteration ? \n2) What is the impact of the number of inducing points ? For example, in Figure 3(a) and Figure 5, it is not clear to me how many inducing points you used compared to the Exact GP. \n3) Do you update the kernel hyperparameters along training ? ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "To further improve the paper, I think these are potential directions,\n1) A clear discussion on the contribution compared to O-SGPR and a standard SVGP.\n2) A clear model description on \n\n (a) how the variational distributions are obtained,\n\n (b) how the hyperparameters are chosen,\n\n (c) what is the training workflow,\n\nThis paper does not lead to negative societal impacts.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "RP_y854dGl9", "reviewer": "Reviewer_DFYr", "summary": "The authors are interested in the problem of online learning with applications to online Decision making. The focus is on (sparse variational) Gaussian Processes and Bayesian optimization.\n\nThe authors 1) propose an alternative formulation of the online learning algorithm of Bui et al, which allow to ‘preserve’ what has been learned from the previous data 2) propose a way to update the posterior when adding observations in the conditioning set for non Gaussian likelihood 3) Thoroughly evaluate the resulting algorithm on various BO tasks.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nOriginality and Quality:\n\nThe paper builds up on the work of Bui et al, where the aim is to update posterior processes when adding observations to the conditioning set.\nThe authors acknowledge mostly rederiving the method in an alternative fashion with the aim to connect it to Sparse Gaussian Process Regression (SVGP in the conjugate case by Titsias).\n\nThe approach consists in essence in explicitly representing the posterior via sites that are updated in an incremental fashion. The ‘unifying’ paper of Bui would be a welcome reference here. As a rederivation rather than introduction, I m not sure it is worth a new name. Also the terminology is not very enlightening. Computing the posterior is always conditioning on observations. online conditioning is not very precise not specific to this method (also see comment below).\n\nThe proposed incremental formulation is equivalent to Bui et al, and brings practical advantages when implementing. The particular applications of SVGP to BO is original (recent work on Thompson sampling in this particular combination would be worth mentioning :Vakili 2020).\n\nI have a concern regarding the Laplace approximation. Each likelihood term is optimized independently of the ongoing posterior distribution. For a Bernoulli likelihood, the maximizer is at infinity and the hessian is ‘flat’ (being exponential family doesn't solve that). This problem is why the community has derived ways to compute sites in the context of the ongoing approximation  (khan et at 2017 for VI, or Bui et al 2017 for EP). This and other limitations of this heuristic should be discussed. The lack thereof does not make me confident or provide intuition that errors don’t accumulate. \nAn experiment focusing just on the quality of this approximation for online learning would help justify this particular choice.\n\n\nClarity:\n\nThe manuscript is fairly well written (see comments below for improvement suggestions)\n\nSignificance\n\nLook ahead objectives are strong candidates for BO and deriving practical algorithms to update posterior processes approximately and cheaply is needed.\nThough tackling an relevant problem, the contribution in the paper is a rewriting of an existing algorithm and I have concerns about the extension.\n\nMy rating of this paper reflects these mixed feelings, but I am ready to increase my score.\n\n\nRemarks:\n\nEq 3: covariance looks wrong to me. you can’t subtract a covariance from a precision, the following derivations look fine though\n\n‘Conditioning a GP into another GP’ is very informal and doesn’t help the reading. “conditioning into” is not a thing. I’d recommend using more standard formulations.\n\nThe S in acronym SVGP is used to denote both Sparse and Stochastic throughout the paper, which is confusing. \n \nA couple of typos (the prime) in section 3.1, makes it hard to parse.\n\n\nI have not thoroughly reviewed the experimental section and leave this to other reviewers.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations: the authors could be more careful in discussing and assessing the limitations of the approximation introduced for the online learning problem before applying it to decision making.\n\nnegative societal impact : None", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "LUblOfusBf", "reviewer": "Reviewer_EXgr", "summary": "The authors propose online variational conditioning (OVC), a procedure for efficiently conditioning Stochastic Variational Gaussian Processes (SVGPs) in an online setting that does not require re-training through the evidence lower bound with the addition of new data. The authors propose to efficiently update a posterior distribution after receiving new data; they rely on the idea of a Sparse Gaussian Process Regression (SGPR) model trained on pseudo-data under a block-diagonal Gaussian likelihood, they also provide an extension to non-Gaussian likelihoods by means of the Laplace approximation. The authors show the application of their method in look-ahead acquisitions for Bayesian Optimisation. ", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "$\\textbf{Originality:}$ \nThe work is novel in the way the authors associate the work of “Streaming Sparse GPs” (O-SGPR) with the well-known idea of a Sparse Gaussian Process Regression (SGPR) model. They  highlight that an O-SGPR can be seen as the SGPR  being trained on pseudo-data under a block-diagonal Gaussian likelihood. Additionally, the authors provide an extension to non-Gaussian likelihoods by means of the Laplace approximation.\n\n$\\textbf{Quality:} $\nSection 3.1 presents the main theoretical analysis and quality contribution of the paper. The section of experiments explores various important scenarios to show the advantages of the method. Nonetheless, in sections 2 and 3,  the work presents some mathematical typos and a few variables are not properly introduced. Also, the authors are probably missing a broader discussion about the results obtained. For instance: \n\n-Figure 3(a) shows that Exact GP+NEI presents the better performance than Exact GP+KG. We could intuitively expect that SVGP+NEI also presented a better performance than SVGP+KG. Although, when using SVGP+NEI we obtain the poorest performance; in contrast, the SVGP+KG achieved the best performance. It might be useful to discuss about such effect. What changes when using Exact GP+NEI and SVGP+NEI? Why does it degrade the performance?\n\nOn the other hand, Figure 1 presents an scenario that makes me wonder about the quality of the approach:\n\n-The authors claim the following regarding Figure 1c: “the SVGP is still able to update its posterior over the latent volatility in response to new data without “forgetting” old observations.”. Nonetheless, the figure 1c shows that the mean predictions and error bars differ much between the top and bottom figures for the old batch of data. Shouldn’t we expect the same mean and error bars behaviour (in top and bottom figures 1c) for the old batch of data, since the model does not “forget” old observations? This figure is lacking of a proper discussion about it. \n\n$\\textbf{Clarity:} $\nThe paper is not well organised, the writing style seems not to be homogeneous along the sections. Particularly, the way the sections 1, 2 and 3 are written differ much of the writing style of the section of experiments. There are some confusing phrases along the document, some acronyms are multiple time introduced or not even introduced. There are some inadequate mathematical notations in the paper and a possible error in equation (3) (Please see the detailed comments and recommendations  at the end after “Significance”). The work lacks of an appropriate connection between the experiments section and the previous methodology sections. For instance, section 2.3 “Bayesian Optimization and Monte Carlo Acquisitions” seems to be isolated. Equations (3), (4), (7) and (8) are never cited in the text. In the experiments sections would be important to mention the use of the Algorithm 1 presented in the paper. Likewise, it would be worthy to link such experiments sections with sections 2 and 3.  \n\n-Possible error in Eq. (3). The authors refer to the work of Titsias “Variational Learning of Inducing Variables in Sparse Gaussian Processes” for Sparse GP regression and express equations (2) and (3).  Although, the covariance in Eq. (3): \n$\\Sigma_{SGPR} = K_{\\mathbf{ww}} – K_{\\mathbf{wu}}( K_{\\mathbf{uu}}^{-1} – S_{\\mathbf{u}} )K_{\\mathbf{uw}}$, for the predictive distribution seems not to be correct in comparison to Titsias’ work. Such a covariance appears in Eq. (6) of Titsias’ work as:\n$\\Sigma_{SGPR} = K_{\\mathbf{ww}} – K_{\\mathbf{wu}}( K_{\\mathbf{uu}}^{-1} – B )K_{\\mathbf{uw}}$, where $B = K_{\\mathbf{uu}}^{-1}(S_{\\mathbf{u}})K_{\\mathbf{uu}}^{-1}$, and $S_{\\mathbf{u}}$ is the covariance of the Variational posterior (it is called $A$ in Titsias’ paper)\n\n$\\textbf{Significance:} $\nThe experiments carried out in the paper might be useful for other practitioners and researchers in the field, particularly the paper might contain unique experimental contribution like the results presented in Figure 3 associated with non-Gaussian likelihoods.    \n\n$\\textbf{Some detailed comments and recommendations for the authors:} $\n\n-The acronyms GP, OVC, SVGP, ELBO, qKG and BO were introduced many times. \n\n-The acronym SGPR appears in: “An improved conceptual understanding of O-SGPR [9] as SGPR trained...” without being introduced.\n\n-The following phrase might be better written to improve the intelligibility: “First, the compute and memory consumption of exact GPs grows at least quadratically with the amount of data [24, 60], generally limiting them to BO problems with fewer than 1,000 function evaluations.”\n\n-It is not clear the sentence: “Second, they are limited to applications that are modeled well by a Gaussian likelihood”. What does it mean “applications that are modeled well”?\n\n-There is a use of the word “compute” that is not clear, for instance: “First, the compute and memory consumption of exact GPs grows…”, also in “Stochastic variational Gaussian processes (SVGPs) [32] have constant compute and memory footprints and are applicable to non-Gaussian likelihoods”. Is it “computation”? Or “computational”? \n\n-The following phrase is not clear: “…, but they sacrifice closed form expressions for conditional posteriors.”. Is it missing additional explanation? \n\n-The caption for “Figure 1” is not very informative and a bit confusing. For instance, at the end of the caption appears: “Efficient, closed form condtioning for SVGPs was previously impractical.”, there is a typo in “conditioning” and what is the author referring to when saying “...was previously impractical”? What is the intention of such a phrase in the caption?\n\n-In section 2.1, typo in “Given $X_{train} = [x_i , . . . , x_n ]$”, change “i” for number “1”? \n\n-The following expression in section 2.1 is not clear: “Computations with $K_{\\mathbf{vv}}$ cost $O(n^2 )$ space and $O(n^3 )$ computation.”.\n\n-The matrices $K_{\\mathbf{ww}}$ and $K_{\\mathbf{vv}}$ appear in section 2.1, but they are not clearly introduced.\n\n-The matrices $K_{\\mathbf{uu}}$ and $K_{\\mathbf{uv}}$ appear in section 2.2, but they are not clearly introduced.\n\n-Typo in section 2.2, it says: “The optimal $m_{\\mathbf{u}}$, $S_{\\mathbf{u}}$, and resulting $q(\\mathbf{w})$ are”, the variable “$m_{\\mathbf{u}}$” should be bold, $\\mathbf{m_u}$.\n\n-Possible error in Eq. (3). The authors refer to the work of Titsias “Variational Learning of Inducing Variables in Sparse Gaussian Processes” for Sparse GP regression and express equations (2) and (3).  Although, the covariance in Eq. (3): \n$\\Sigma_{SGPR} = K_{\\mathbf{ww}} – K_{\\mathbf{wu}}( K_{\\mathbf{uu}}^{-1} – S_{\\mathbf{u}} )K_{\\mathbf{uw}}$, for the predictive distribution seems not to be correct in comparison to Titsias’ work. Such a covariance appears in Eq. (6) of Titsias’ work as:\n$\\Sigma_{SGPR} = K_{\\mathbf{ww}} – K_{\\mathbf{wu}}( K_{\\mathbf{uu}}^{-1} – B )K_{\\mathbf{uw}}$, where $B = K_{\\mathbf{uu}}^{-1}(S_{\\mathbf{u}})K_{\\mathbf{uu}}^{-1}$, and $S_{\\mathbf{u}}$ is the covariance of the Variational posterior (it is called $A$ in Titsias’ paper). \n\n-As a recommendation for section 3.1, it would be useful to apply some intuition to the notation. For instance, it is confusing for the reader to associate the definitions: \n“inducing function values $\\mathbf{u}^{\\prime} := [f(\\mathbf{z}_1), . . . , f(\\mathbf{z}_p )]$” and \n“$\\mathbf{u} := [f (\\mathbf{z}^{\\prime}_1), . . . ,f(\\mathbf{z}^{\\prime}_p )]$”. \nIt might be easier to associate the “$\\mathbf{u}^{\\prime}$” with the evaluations of the “$f(\\mathbf{z}^{\\prime})$” and also the “$\\mathbf{u}$” (without prime) with the evaluations of the “$f(\\mathbf{z})$” (without prime).\n\n-In section 3.1, the following notations should be corrected to make the dimensionality coincide: \n“… Eq. (6) to define a new SGPR model with pseudo-data ($\\hat{X} := [X_t \\quad Z_{t-1}]^{\\top} , \\hat{\\mathbf{y}} := [\\mathbf{y}_t \\quad \\tilde{\\mathbf{y}}_\\mathbf{u^{\\prime}}]^{\\top}$)”. \nA possible way is to write: \n\n$\\hat{X}:= [{X_t}^{\\top}$ $Z^{\\top}_{t-1}]^{\\top}$ and $\\hat{\\mathbf{y}} := [\\mathbf{y}^{\\top}_t \\quad \\tilde{\\mathbf{y}}^{\\top}_\\mathbf{u^{\\prime}}]^{\\top}$.\n \n\n-In section 3.1, the distributions “$q_t(\\cdot)$”, “$\\hat{p}(\\cdot|\\mathbf{y}_t)$” and $p_t(\\cdot)$ are not explained.\n\n-In section 3.1, there is not explanation of what is the variable “$\\mathbf{\\hat{v}}$” in the phrase: “where the distribution of $\\mathbf{y}$ is the pseudo-likelihood $N(\\mathbf{\\hat{v}}, Σ_{\\hat{\\mathbf{y}}} )$” \n\n-In section 3.1, typo word, Should it be “through”? In phrase: “projecting that distribution back into pseudo-data to be combined with a fresh batch of ground-truth data though a joint Gaussian pseudo-likelihood.”.\n\n-Equations (3), (4), (7) and (8) are never cited in the text.\n\n-There is not a clear explanation for the Figure 2. Therefore, it is difficult to understand what are the main comparisons to analyse between the Figure 2 (b), Figure 2 (c) and Figure 2 (d). Additionally, the colour bars of the figures have different ranges making difficult to distinguish their characteristics. For instance, Figure 2 (b) has a colour bar range (1.6,2.5), Figure 2 (c) has a colour bar range (1.7,2.5) and Figure 2 (d) a colour bar range (1.52,2.16). \n\n-In the caption for Figure 3, the authors refer to NEI (an acronym not introduced previously). For instance in the caption: “Preference learning; SVGPs with qKG slightly outperform Laplace approximations with NEI, and outperform qNEI with SVGPs.”. is the acronym NEI the same as qNEI?\n  \n-The authors write in the caption of Figure 3: “Only SVGPs can be used here,”. Nonetheless, there is not information of why only SVGP can be used. It might be useful to give some insight to the reader; probably the caption is not the appropriate place to give such an insight.\n\n-How did the authors select the batch sizes q? It might be useful to have an analysis of the influence of the batch size. What could it be the suggestion for a practitioner when selecting the batch size q?\n\n\n-In experiments for Hotspot Modelling it appears: \"We model the responses $y$ (incidence) at locations $x$ with population $n(x)$ with a Binomial likelihood $p(\\hat{y}|f, x)$\". is it $y$ different to $\\hat{y}$?\n\n\nCheck References:\n\n-In [3], [5], [6], [13], [14], [15], [16], [28], [30], [34], [35], [36], [37], [39], [41], [42], [45], [48], [49], [50], [53], [56], [61], [63], [68] and [79] change “gaussian” by “Gaussian”.\n\n-In [8] change “bayes” by “Bayes”.\n\n-In [20], [21], [23], [43], [48], [51], [73] and [74] change “bayesian” by “Bayesian”.\n\n-In [22] change “svm” by “SVM”.\n\n-In [44] change “lipschitz” by “Lipschitz”.\n\n-In [58] change “hessian” by “Hessian”.\n\n-In [72] change “monte carlo” by “Monte Carlo”.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors could improve their work by extending a discussion regarding the reliability of the methods, particularly in the context of Disease Incidence where the authors showed different applications. Such a discussion might enhance the usefulness of their work and the potential positive or negative societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "FwAN2tcP4eY", "reviewer": "Reviewer_XKNN", "summary": "Authors propose a novel method, named OVC for online variational conditioning of GPs under sparse approximations. Particularly, they are interested in the recursive update of the posterior GP distribution after receiving new data per time-step. The model can be used for both regression and classification tasks, bayesian optimisation and active learning as is shown in the experiments.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality and significance:** \n\nGood paper. I enjoyed quite a lot while reading it. Particularly, I want to remark the following points that are positive to me:\n\n- The idea of transforming the inducing points and their corresponding function instances into pseudo-data is interesting. It can be considered as some sort of data augmentation. Authors correctly identified the limitations of the approach presented in Bui et al. (2017).\n- Figure 1 is great. It is not that easy to obtain such performance, even if the “smoothness” of the signal changes across time. How do you treat the kernel hyperparameters? In that particular case seems like between [-2,2] the length scale is around 1.0, and later between [2,6] it is smaller (0.5 for example). Is the model well fitted to the past data because you are using the cache thing right? Otherwise would be problematic if re-training.\n- Experiments are well designed and prove the performance of the tool. I see clearly that the method outperforms Bui et al. (2017).\n- Perhaps Bui et al. (2017) could work better with non-Gaussian likelihoods, but the limitations are overcame by OVC here. I like the Laplace approximation, but maybe if considering multivariate likelihood models, things could be more complicated.. \n- I found the “banana dataset” experiment in Figure 8 of the Appendix. What a pity not having included it in the main manuscript, because the results are pretty good. \n- An extra positive point of this work is that you are able to update the model with one-single observation (in a filtering-like model) or with more than one data-point per time step (similarly as Bui et al. 2017). Great!\n\n**Quality and clarity:** \n\n In particular, I want to remark the quality of Section 2, which demonstrates the wide knowledge of authors on the online GP topic and how the decisions taken are well-motivated later. Experiments are also thorough and very well presented. Results are convincing. To sign a flaw of the paper, I would say that section 3.1 could be clearer, but this is a little detail. I’m sure the paper will have an impact on the community.\n\n**Questions and other comments:**\n\n- L106: I think that this whitened parametrisation is not that well-known, I suggest a super-short explanation, mainly because later is re-used.\n- L156: are the definitions of u’ and u in the correct order? It seems that u’ should refer to f(z’), instead of the written ones.\n- The approximation of y_u in the point 1 of Algorithm 1) makes sense to me, but maybe is not that well explained. Do you do it to use the “fixed” variational distribution S_u, right? Otherwise, would be useless at the next time-step t.\n- The cache terms a and RR look like they are gonna be critical in the paper at the beginning, but later they are not commented anymore. The task of adding new rows and columns to Kvv (L82) in an online fashion seems like a contribution of other works, is that the reason why authors do not claim it?\n- Algorithm 1 would be even better having a recursive nature, like do this every time step t. At point (4) one may get lost and do not know how to use the model again at t=t+1.\n- L192: How is the number of inducing points treated? Do authors augment them as long as new data arrives? Which is the criteria?\n- Could you add some extra details about the treatment of hyperparameters + kernel models? Are you assuming stationary functions or it may change?\n\n**References:**\n\n- Bui et al. (2017). “Streaming sparse Gaussian process approximations”. NIPS 2017.\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors adequately addresses both limitations and societal impact of the work in the section A of the Appendix. Particularly, they explained very well the limitations of the method and the cases where it might not work very well. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3,5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper contributes to the literature of sparse variational Gaussian processes applied to an online setting. The authors introduce the novel idea of treating inducing variables and inducing inputs at time t-1 as pseudo-data to be used at time t, leading to recursive expressions for the posterior distribution. The paper appears technically solid. The most critical reviewer does not oppose acceptance. Comments regarding clarity and typos highlighted by the reviewers can be addressed in the camera-ready version.
Decision: Accept (Poster)