Batch Bayesian optimisation via density-ratio estimation with guarantees
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Bayesian optimisation (BO) algorithms have shown remarkable success in applications involving expensive black-box functions.
Traditionally BO has beenset as a sequential decision-making process which estimates the utility of querypoints via an acquisition function and a prior over functions, such as a Gaussianprocess.
Recently, however, a reformulation of BO via density-ratio estimation(BORE) allowed reinterpreting the acquisition function as a probabilistic binaryclassiﬁer, removing the need for an explicit prior over functions and increasingscalability.
In this paper, we present a theoretical analysis of BORE’s regret andan extension of the algorithm with improved uncertainty estimates.
We also showthat BORE can be naturally extended to a batch optimisation setting by recastingthe problem as approximate Bayesian inference.
The resulting algorithm comesequipped with theoretical performance guarantees and is assessed against otherbatch BO baselines in a series of experiments.
INTRODUCTION
Bayesian optimisation (BO) algorithms provide ﬂexible black-box optimisers for problems involvingfunctions which are noisy or expensive to evaluate [1].
Typical BO approaches place a probabilisticmodel over the objective function which is updated with every new observation in a sequentialdecision-making process.
Most methods are based on Gaussian process (GP) surrogates [2], whichprovide closed-form analytic expressions for the model’s posterior distribution and allow for a numberof theoretical performance guarantees [3–5].
However, GP surrogates have a number of limitations,such as not easily scaling to high-dimensional domains, high computational complexity and requiringa careful choice of covariance function and hyper-parameters [2].
Non-GP-based BO methods havealso been proposed in the literature, such as BO methods based on neural networks [6, 7] and randomforests [8] regression models.
As an alternative to improving the model, Tiao et al.
[9] focus on the acquisition function, whichin BO frameworks represents the guide that takes the model predictions into account.
They showthat one can derive the acquisition function directly without an implicit model by reinterpretingthe expected improvement [4, 1] via a density-ratio estimation problem.
Applying this perspective,the acquisition function can then be derived as a classiﬁcation model, which can be representedby ﬂexible parametric models, such as deep neural networks, and efﬁciently trained via stochasticgradient descent.
The resulting method, called Bayeisan optimisation via density-ratio estimation(BORE) is then shown to outperform a variety of traditional GP-based and non-GP baselines.
Despite the signiﬁcant performance gains, BORE has only been applied to a sequential setting andnot much is known about the method’s theoretical guarantees.
Batch BO methods have the potentialto speed up optimisation in settings where multiple queries to the objective function can be evaluated simultaneously [10–13].
Given its ﬂexibility to apply models which can scale to large datasets, it is therefore a natural question as to whether BORE can be readily extend to the batch setting in acomputationally efﬁcient way.
In this paper, we extend the BORE framework to the batch setting and analyse its theoreticalperformance.
To derive theoretical guarantees, we ﬁrst show that the original BORE can be improvedby accounting for uncertainty in the classiﬁer’s predictions.
We then propose a novel method, called
BORE++, which uses an upper conﬁdence bound over the classiﬁer’s predictions as its acquisitionfunction.
The method comes equipped with guarantees in the probabilistic least-squares setting.
Weprovide extensions for both BORE and BORE++ to the batch setting.
Lastly, we present experimentalresults demonstrating the performance of the proposed algorithms in practical optimisation problems.
Related work
Since their proposal by Schonlau et al.
[14], batch Bayesian optimisation methods have appeared invarious forms in the literature.
Many methods are based on heuristics derived from estimates givenby a Gaussian process regression model [12, 11].
Others are based on Monte Carlo estimates ofmulti-query acquisition functions [10, 13].
Despite that, the prevalent approaches to batch BO arestill based on a GP regression model, which require prior knowledge about the objective function anddo not scale to high-dimensional problems.
We instead take a different approach by viewing BO as adensity-ratio estimation problem following the work by Tiao et al.
For batch design, we take anoptimisation-as-inference approach [15, 16] by applying Stein variational gradient descent, a nonparametric approximate inference method [17], which has been recently combined with GP-based
BO [18, 19].
Our theoretical results, however, are agnostic to the choice of inference algorithm.
Background
We consider a global optimisation problem over a compact search space X ⊂ R d of the form:
[Equation]
where f : X → R is assumed to be a black-box objective function, i.e., we have no access to gradientsnor analytic formulations of it.
In addition, we are only allowed to collect up to T observationsy t := f ( x t ) + (cid:15) t , for t ∈ { 1 , . . . , T } , which are corrupted by additive noise (cid:15) t .
3.1 Bayesian optimisation
Bayesian optimisation (BO) algorithms approach the problem in Equation 1 via sequential decisionmaking [1].
At each iteration, BO selects a query point x t ∈ X by maximising an acquisition functiona ( x |D t ) .
[Equation]
The acquisition function encodes information provided by the observations collected so far D t :
={ x i , y i } t − 1 i =
1 using a probabilistic model over f , typically a Gaussian process (GP) [2], conditioned onthe data.
After collecting an observation y t , the dataset is updated with the new query-observationpair D t :=
D t − 1 ∪ x t , y t .
This process then repeats for a given number of iterations T .
3.2 The expected improvement algorithm
A popular acquisition function in the BO literature is the expected improvement (EI) [4].
At iterationt ≥ 1 , one can deﬁne τ := min
i<t y t as an incumbent target.
The expected improvement is thendeﬁned as:
[Equation]
In the case of a GP prior on f |D t − 1 ∼ GP ( µ t − 1 , k t − 1 ) , for any point x where σ t − 1 ( x ) > 0 , the EIis given by:
[Equation]
where s t − 1 := τ − µ t − 1 ( x )
[Equation]
t −
[Equation]
x ) , if σ 2 t − [Equation] x ) :=
[Equation]
t − 1 ( x , x ) > 0 .
For points x ∈ X where σ t − 1 ( x ) = 0 ,i.e., there is no posterior uncertainty, we set a EI ( x |D t − 1 ) :
= 0 by convention.
Here Ψ( s ) and ψ ( s )denote, respectively, the cumulative distribution function (CDF) and the probability density function(PDF) of the standard normal distribution evaluated at s ∈ R .
Algorithm
1 for t ∈ { 1 , . . .
, T } do τ := ˆΦ − 1 t − 1 ( γ )
z i := I
[ y i ≤ τ ] , i ∈ { 1 , . . .
, t − 1 }
˜ D
t − 1 :=
{ x i , z i } t − 1 i =  ˆ π t ∈ argmin π L [ π | ˜
D t − 1 ] x t ∈ argmax x ∈X ˆ π t − 1 ( x ) y t :
= f ( x t ) + (cid:15) t 8 end
3.3 Bayesian optimisation via density-ratio estimation (BORE)
The EI acquisition function can be reformulated as a density ratio between two probability distributions under certain assumptions [20, 9].
Let (cid:96) ( x ) :
= p ( x | y ≤ τ ) represent the probability densityover x ∈ X conditioned on the observation y being below a threshold τ ∈ R .
Conversely, letg ( x ) := p ( x | y > τ ) .
For γ ∈ [0 , 1] , the γ -relative density ratio between these two probabilitydensities is:
[Equation]
noting that γ = 0 leads to the usual probability density ratio deﬁnition, ρ 0 ( x ) =
(cid:96) ( x ) / g ( x ) .
Now if wechoose τ :=
Φ − 1 ( γ ) , where Φ( s ) :
= p ( y ≤ s ) represents the cumulative distribution function of themarginal distribution of observations, 1 for s ∈ R , and then replace τ in Equation 3, Bergstra et al.[20] have shown that 2 a EI ( x ) ∝ ρ γ ( x ) , for x ∈ X .
Based on this fact, Tiao et al.
[9] showed:
[Equation]
) where π ( x ) :
= p ( y ≤ τ | x ) can be approximated by a probabilistic classiﬁer trained with a properscoring rule, such as the binary cross-entropy loss:
[Equation]
Other examples of proper scoring rules include the least-squares loss, which leads to probabilisticleast-squares classiﬁers [21], and the zero-one loss.
We refer the reader to Gneiting and Raftery [22]for a review and theoretical analysis on this topic.
BORE is summarised in Algorithm 1.
As seen, the marginal observations distribution CDFΦ( s ) :
= p ( y ≤ s ) is replaced by the empirical approximation ˆΦ t ( s ) :
= 1 t (cid:80) ti =
I
[ y i ≤ s ] andits corresponding quantile function ˆΦ − 1 t .
At each iteration, observations are labelled according tothe estimated γ th quantile τ , and a classiﬁer ˆ π t is trained by minimising the loss L [ π | ˜ D t ] over thedata points ˜ D t .
A query point x t is chosen by maximising the classiﬁer’s probabilities, which inour case corresponds to maximising the expected improvement.
A new observation is collected,and the algorithm continues running up to a given number of iterations T .
As demonstrated, noexplicit probabilistic model for f is needed, only a classiﬁer, which can be efﬁciently trained via, e.g.,stochastic gradient descent.
Analysis of the BORE framework
In this section, we analyse limitations of the BORE framework in modelling uncertainty and analyseits effects on the algorithm’s performance.
As presented in Section 3.3, at each iteration t ≥ 1 , theoriginal BORE framework trains a probabilistic classiﬁer ˆ π t ( x ) to approximate p ( y ≤ τ | x ) , where τdenotes the γ th quantile of the marginal observations distribution, i.e., p ( y ≤ τ )
=
γ .
This approachleads to a maximum likelihood estimate for the classiﬁer ˆ π , which may not properly account for theuncertainty in the classiﬁer’s approximation.
Since BORE is based on probabilistic classiﬁers, instead of regression models as in traditional BOframeworks [1], a natural ﬁrst question to ask is whether a classiﬁer can guide it to the global optimumof the objective function.
The following lemma answers this question and is a basis for our analysis.
Lemma 1.
Let f : X → R be a continuous function over a compact space X .
Assume that, forany x ∈ X , we observe y = f
( x ) + (cid:15) , where (cid:15) is i.i.d.
noise with a strictly monotonic cumulativedistribution function Φ (cid:15) : R → [0 , 1] .
Then, for any τ ∈ R , we have:
[Equation]
Proof.
As the observation noise CDF is monotonic, by basic properties of the argmax , we have:
[Equation]
which concludes the proof.
According to this lemma, maximising class probabilities is equivalent to optimising the objectivefunction when the classiﬁer is optimal, i.e., it has perfect knowledge of f .
This result holds for anygiven threshold τ ∈ R .
We only make a mild assumption on the CDF of the observation noise Φ (cid:15) ,which is satisﬁed for any probability distribution with support covering the real line (e.g. Gaussian,
Student-T, Cauchy, etc.).
To analyse BORE’s optimisation performance, we will aim to bound the algorithm’s instant regret:r t :=
f ( x t ) − f ( x ∗ ) , t ≥ 1 , (10) and its cumulative version R T := (cid:80) Tt =1 r t .
Sub-linear bounds on R T lead to a no-regret algorithm,since lim T →∞ R T T = 0 and min t ≤ T r t ≤ R T T .
Assuming that there is an optimal classiﬁer π ∗ :
X → [0 , 1] , which is such that π ∗ ( x ) = p ( y ≤τ | x , f ) , for a given τ ∈ R , we can directly relate the classiﬁer probabilities to the objective functionf values, since:
[Equation]
The existence of the inverse Φ − 1 (cid:15) is ensured by the strict monotonicity assumption on Φ (cid:15) in Lemma 1.
Under this observation, the algorithm’s regret at any iteration t ≥ 1 can be bounded in terms ofclassiﬁer probabilities:
[Equation]
[Equation] where L (cid:15) is any Lipschitz constant for Φ − 1 (cid:15) , which exists since X is compact.
Therefore, we shouldbe able to bound BORE’s regret by analysing the approximation error for ˆ π t at each iteration t ≥ 1 .
Although approximation guarantees for classiﬁcation algorithms under i.i.d.
data settings are wellknown [28], each observation in BORE depends on the previous ones via the acquisition function.
This process is also not necessarily stationary, so that we cannot apply known results for classiﬁersunder stationary processes [29].
In the next section, we consider a particular setting for learning aclassiﬁer which allows us to bound the prediction error under BORE’s data-generating process.
4.1 Probabilistic least-squares classiﬁers
We consider the case of probabilistic least-squares (PLS) classiﬁers [30, 31].
In particular, we model aprobabilistic classiﬁer π : X → [0 , 1] as an element of a reproducing kernel Hilbert space (RKHS)
Hassociated with a positive-deﬁnite kernel k : X × X → R .
A RKHS is a space of functions equippedwith inner product (cid:104)· , ·(cid:105) k and norm (cid:107)·(cid:107) k := (cid:112) (cid:104)· , ·(cid:105) k
For the purposes of this analysis, we willalso assume that k ( x , x ) ≤ 1 , for all X .
This setting allows for both linear and non-parametricmodels.
Gaussian assumptions on the function space would lead us to GP-based PLS classiﬁers [2], but we are not restricted by Gaussianity in our analysis.
If the kernel k is universal, as Φ (cid:15) is injective,we can also see that the RKHS assumption allows for modelling any continuous function.
For a given τ ∈ R , a PLS classiﬁer is obtained by minimising the regularised squared-error loss:
[Equation]
where λ > 0 is a given regularisation factor and z
i :=
I [ y i ≤ τ ] ∈ { 0 , 1 } .
In the RKHS case, thesolution to the problem above is available in closed form [33, 21] as:
[Equation]
where k t ( x ) :=
[ k ( x , x 1 ) , . .
k ( x , x t )] T ∈ R t , K t :=
[ k ( x i , x j )] ti,j =1 ∈ R t × t and z t :=[ z 1 , . . .
, z t ] T ∈ R t .
This PLS approximation may not yield a valid classiﬁer, since it is possible that ˆ π t ( x ) / ∈ [0 , 1] for some x ∈ X .
However, it allows us to place a conﬁdence interval on theoptimal classiﬁer’s prediction, as presented in the following theorem, which is based on theoreticalresults from the online learning literature [34, 35].
Our proofs can be found in the supplement.
Theorem 1.
Given τ ∈ R , assume π ( x ) :=
Φ (cid:15) ( τ − f ( x )) is such that π ∈ H , and (cid:107) π (cid:107) k ≤ b .
Let{ x t } ∞ t =1 be a X -valued discrete-time stochastic process predictable with respect to the ﬁltration{ F t } ∞ t =0 .
Let { z t } ∞ t =1 be a real-valued stochastic process such that ν t :=
z t − π ( x t ) is 1 -subGaussian conditionally on F t − 1 , for all t ≥ 1 .
Then, for any δ ∈ (0 , 1) , with probability at least1 − δ , we have that:
[Equation]
where β t ( δ ) :
=
b + (cid:112) 2 λ − 1 log( | I + λ − 1 K t | 1 / 2 /δ ) , with | A | denoting the determinant of matrix A ,and σ 2 t ( x ) :
= k ( x , x ) − k t ( x ) T ( K t + λ I ) − 1 k t ( x ) , [Equation]
4.2 Regret analysis for BORE
We now consider BORE with a PLS classiﬁer.
For this analysis, we will assume an ideal settingwhere τ is ﬁxed, possibly corresponding to the true γ th quantile of the observations distribution.
However, our results hold for any choice of τ ∈ R and can therefore be assumed to approximatelyhold for a varying τ which is converging to a ﬁxed value.
In this setting, the algorithm’s choices are:given by:
[Equation]
[Equation]
where ˆ π t is the estimator in Equation 14.
we can then apply Theorem 1 to the classiﬁer-based regretin Equation 12 to obtain a regret bound.
For this result, we will also need the following quantity:
[Equation]
[Equation]
where the maximisation is taken over the discrete set of locations { x i } N i =1 ⊂ X and K N :=[ k ( x i , x j )]
Ni,j =1 .
This quantity denotes the maximum information gain of a Gaussian process modelafter N observations.
We are now ready to state our theoretical result regarding BORE’s regret.
Theorem 2.
Under the conditions in Theorem 1, with probability at least 1 − δ , δ ∈ (0 , 1) , theinstant regret of the BORE algorithm with a PLS classiﬁer after T ≥ 1 iterations is bounded by:
[Equation]
and the cumulative regret by:
[Equation]
As Theorem 2 shows, the regret of the BORE algorithm in the PLS setting is comprised of twocomponents.
The ﬁrst term is related to the regret of a GP-UCB algorithm [see 36, Thr. 3] and itsknown to grow sub-linearly for a few popular kernels, such as the squared exponential and the Matérnclass [3, 37].
The second term, however, reﬂects the uncertainty of the algorithm around the optimum location x ∗ .
If the algorithm never samples at that location, this second summation might have amostly linear growth, which will not lead to a vanishing regret.
In fact, if we consider Equation 14and a RKHS with a translation-invariant kernel, we see that, as soon as an observation z t = 1 iscollected at a location x t (cid:54) = x ∗ , that location will constitute the maximum of the classiﬁer output.
Then the algorithm would keep returning to that same location, missing opportunities to sample at x ∗ .
It is worth noting that Theorem 2 reﬂects the regret of BORE in an idealistic setting where thealgorithm uses the optimal PLS estimator in the function space H .
However, if we train a parametricclassiﬁer, such as neural network, via gradient descent, the behaviour will not necessarily be thesame, and the algorithm might still achieve a good performance.
In the original BORE paper, forinstance, a parametric classiﬁer is trained by minimising the binary cross-entropy loss [9] and leadsto a successful performance in experiments.
Neural network models trained via stochastic gradientdescent are known to provide approximate samples of a posterior distribution [38, 39], instead ofan optimal best-ﬁt predictor, which might make BORE behave like Thompson sampling [40] (seediscussion in the appendix).
Nevertheless, Theorem 2 still shows us that BORE may get stuck intolocal optima, which is not ideal for BO methods.
In the next section, we present an extension of the
BORE framework which addresses this shortcoming.
BORE++: improved uncertainty estimates
As discussed in the previous section, the lack of uncertainty quantiﬁcation in the estimation of theclassiﬁer for the original BORE might lead to sub-optimal performance.
To address this shortcoming,we present an approach for uncertainty quantiﬁcation in the BORE framework which leads toimprovements in performance and theoretical optimality guarantees.
Our approach is based on usingan upper conﬁdence bound (UCB) on the predicted class probabilities as the acquisition function for
BORE.
Due to its improved uncertainty estimates, we call this approach BORE++.
Class-probability upper conﬁdence bounds
We propose replacing ˆ π t in Algorithm 1 by an upper conﬁdence bound which is such that:
[Equation]
which with probability greater than 1 − δ , given δ ∈ (0 , 1) .
Therefore, π t,δ ( x ) represents an upperquantile over the optimal class probability π ∗ ( x ) .
BORE++ selects x t ∈ argmax x ∈X π t − 1 ,δ ( x ) .
To derive an upper conﬁdence bound on a classiﬁer’s predictions π ( x ) , we can take a few differentapproaches.
For a parametric model π θ , a Bayesian model updating the posterior p ( θ |D t ) leads toa corresponding predictive distribution over π θ ( x ) .
This is the case of ensemble models [41], forinstance, where we approximate predictions p ( y ≤ τ t | x , D t )
≈ 1 M (cid:80)
Mi
=1 π θ i ( x ) with θ i ∼ p ( θ |D t ) .
Instead of using the expected class probability, however, BORE++ uses an (empirical) quantileapproximation for π t,δ to ensure Equation 20 holds.
Bayesian neural networks [42], random forests[43], dropout methods, etc.
[44], also constitute valid approaches for predictive uncertainty estimation.
An alternative approach is to place a non-parametric prior over π ∗ , such as a Gaussian process model[2], which allows for the modelling of uncertainty directly in the function space where π ∗ lies.
In thenext section, we present a concrete derivation of BORE++ for the PLS classiﬁer setting which takesthe non-parametric perspective and allows us to derive theoretical performance guarantees.
5.2 BORE++ with PLS classiﬁers
In the PLS setting, the result in Theorem 1 gives us a closed-form expression for a classiﬁer upperconﬁdence bound satisfying the condition in Equation 20. Given δ ∈ (0 , 1) , we set:
[Equation]
where σ t and β t are set according to Theorem 1. We then obtain the following result for BORE++.
Theorem 3.
Under the assumptions in Theorem 1, running the BORE++ algorithm with a PLSclassiﬁer π t,δ as deﬁned above yields, with probability at least 1 − δ , an instant regret bound of: [Equation]
[Equation]
and a cumulative regret bound after T ≥ 1 iterations:
[Equation]
According to Theorem 3, the regret of BORE++ vanishes if the maximum information gain ξ Tgrows sub-linearly, since lim T →∞ R T T = 0 and min t ≤ T r t ≤ R T T .
Sub-linear growth is knownto be achieved for popular kernels, such as the squared exponential, the Matérn family and linearkernels [3, 37].
This result also tells us that theoretically BORE++ performs no worse than GP-UCBsince they share similar regret bounds [3, 36].
However, in practice, the BORE++ framework offersa series of practical advantages over GP-UCB, such as no need for an explicit surrogate model,and a classiﬁer which does not need to be a GP and can therefore be more ﬂexible and scalable tohigh-dimensional problems and large amounts of data.
The connection with GP-UCB, instead, bringsus new insights into how the density-ratio BO algorithm can still share some of the well knownguarantees of traditional BO methods.
Batch BORE
This section proposes an extension of the BORE framework which allows for multiple queries tothe objective function to be performed in parallel.
Although many methods for batch BO havebeen previously proposed in the literature, we here focus on approaching batch optimisation asan approximate Bayesian inference problem.
Instead of having to derive complex heuristics toapproximate the utility of a batch of query points, we can view points in a batch as samples from aposterior probability distribution which uses the acquisition function as a likelihood.
6.1 BORE batches via approximate inference
Applying an optimisation-as-inference perspective to BORE, we can formulate a batch BO algorithmwhich does not require an explicit regression model for f .
The classiﬁer ˆ π ( x ) ≈ p ( y ≤ τ | x ) naturallyturns out as a likelihood function over query locations x ∈ X .
Since the search space X is compact,we can assume a uniform prior distribution p ( x ) ∝ 1 .
Also note that the normalisation constant inthis case is simply (cid:82) X p ( y ≤ τ | x ) p ( x ) d x = p ( y ≤ τ )
=
γ .
Our posterior distribution then becomes:
[Equation]
Therefore, we formulate a batch version of BORE as an inference problem aiming for:
[Equation]
where D KL ( q || (cid:96) ) denotes the Kullback-Leibler (KL) divergence between q and (cid:96) , and P representsthe space of probability distributions over X .
Sampling from (cid:96) would allow us to obtain the points ofinterest in the search space, including the optimum x ∗ and other locations where y ≤ τ .
However, asthe true p ( y ≤ τ | x ) is unknown, we instead formulate a proxy inference problem with respect to asurrogate target distribution ˆ p t based on the classiﬁer model.
For BORE, we set ˆ p t ( x ) ∝ ˆ π t − 1 ( x ) ,while for BORE++ the setting is ˆ p t ( x ) ∝ π t − 1 ,δ ( x ) .
In contrast to (cid:96) ( x ) ∝ p ( y ≤ τ | x ) , thenormalisation constant for the surrogate distributions is unknown, leading us to a proxy problem: [Equation]
[Equation]
The variational inference problem above can be efﬁciently solved via Stein variational gradientdescent (SVGD) [17], which we describe next.
6.2 Batch sampling via Stein variational gradient descent
In our implementation, we apply SVGD to approximately sample a batch B t :=
{ x t,i } Mi =1 of M ≥ 1points from ˆ p t . Other approximate inference algorithms could also be applied.
One of the mainadvantages of SVGD, however, is that it encourages diversiﬁcation in the batch, capturing the possiblemultimodality of ˆ p t .
Given the batch locations, observations can be collected in parallel and thenadded to the dataset to update the classiﬁer model.
SVGD is an approximate inference algorithm which represents a variational distribution q as a set ofparticles { x i } Mi =1 [17].
The particles are initialised as i.i.d.
samples from an arbitrary base distribution
[Figure]
and then optimised via a sequence of smooth transformations towards the target distribution, whichin our case corresponds to ˆ p t ∝ π t − 1 ,δ .
The SVGD steps are given by: [Equation] where k : X × X → R is a positive-deﬁnite kernel, and α > 0 is a small step size. Intuitively,the ﬁrst term in the deﬁnition of ζ t guides the particles to the modes of ˆ p t , while the second termencourages diversiﬁcation by repelling nearby particles.
Theoretical convergence guarantees [23, 24]and practical extensions, such as second-order methods [25, 26] and derivative-free approaches [27],have been proposed in the literature.
Further details on SVGD can be found in Liu and Wang [17].
6.3 Regret bound for Batch BORE++ with PLS classiﬁers
We follow the derivations in Oliveira et al.
[45] to derive a distributional regret bound for batch
BORE++ with respect to its target sampling distribution (cid:96) , which is presented in the following result.
Theorem 4.
Under the same assumptions in Theorem 1, running batch BORE++ with π t,δ set as in
Equation 21, we obtain a bound on the instantaneous distributional regret:
[Equation]
− 1 ] , t ≥ 1 , [Equation] 29) where L π := max x ∈X 1 π ( x ) , and on the cumulative distributional regret at T ≥ 1 :¯ R T
[Equation]
both of which hold with probability at least 1 − δ .
As in the case of non-batch BORE++, the distributional regret bounds for the batch algorithm alsogrow sub-linearly for most popular kernels, leading to an asymptotically vanishing simple regret.
Although different, to compare the distributional regret of batch BORE++ with the non-distributionalregret bounds for BORE++, we may consider a case where τ is set to the function minimumτ : = f ( x ∗ ) =
min x ∈X f ( x ) and the observation noise is small.
In this case, the batch samplingdistribution would converge to a Dirac at the optimum, so that E x ∼ (cid:96) [ f ( x )]
≈ f ( x ∗ ) .
Compared tothe regret of non-batch BORE++ (Theorem 3) after collecting an equivalent number of observations T (cid:48) :
= MT , the expected regret of the batch version of BORE++ after T iterations is then lower bya factor of ξ T /ξ MT , noting that ξ T ≤ ξ T (cid:48) =
ξ MT .
Therefore, batch BORE++ should be able toachieve lower regret per runtime than sequential BORE++ with an equivalent number of observations.
Experiments
This section presents experiments assessing the theoretical results and evaluating batch BORE on aseries of global optimisation benchmarks.
We compared our methods against GP-based BO baselinesin both experiments sets.
Additional experiments and details are presented in the appendix.
7.1 Theory assessment
[Figure] We ﬁrst present simulated experiments assessing the theoretical results in practice testing BOREand BORE++ in the PLS setting.
As a baseline, we compare both methods against GP-UCB.
Thisexperiment was run by generating a random base classiﬁer in the RKHS H k and then a correspondingobjective function via the inverse noise CDF Φ − 1 (cid:15) .
The search space was set as a uniformly-sampledﬁnite subset of the unit interval X :=
[0 , 1] ⊂ R .
We applied the theory-backed settings for BORE++(Section 3.3) and GP-UCB [35], while BORE employed the optimal PLS classiﬁer (Equation 14).
As the results in Figure 1 show, BORE using an optimal PLS classiﬁer simply gets stuck at a its initialpoint, resulting in constant regret.
BORE++, however, is able to progress in the optimisation problemtowards the global optimum, outperforming the GP-UCB baseline.
7.2 Global optimisation benchmarks
We evaluated the proposed SVGD-based batch BORE method in a series of test functions for globaloptimisation comparing it against other BO baselines.
In particular, for our comparisons, we ran thelocally penalised EI (LP-EI) method [11] and the Monte Carlo based q -EI method [10], which areboth based on the EI algorithm, like BORE.
Results are presented in Figure 2.
All methods ran for
T := 200 iterations and used of batch size of 10 evaluations per iteration.
Additional experimentaldetails are deferred to the supplementary material.
As Figure 2 shows, batch BORE is able to outperform its baselines on most of the global optimisationbenchmarks.
We also note that, in some case, due to its complexity the LP-EI method becomescomputationally infeasible after 100 iterations, having to be aborted halfway through the optimisation.
Batch BORE, however, is able to maintain steady performance throughout its runs.
Conclusion
This paper presented an extension of the BORE framework to the batch setting alongside the theoretical analysis of the proposed extension and an improvement over the original BORE.
Theoreticalresults in terms of regret bounds and experimental results show that BORE methods are able toobtain performance guarantees and outperform traditional BO baselines.
As future work, we plan toinvestigate BORE under different loss functions and analyse other batch design settings.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] Code is planned
to be made available after the paper submission
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] To be included in the supplement
(c) Did you report error bars (e
g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[No] Our focus is on theory assessments
rather than computational comparisons
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any
potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A](c)
Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]