Batch Bayesian optimisation via density-ratio estimation with guarantees
Abstract
Bayesian optimisation (BO) algorithms have shown remarkable success in applications involving expensive black-box functions.
Traditionally BO has been set as a sequential decision-making process which estimates the utility of query points via an acquisition function and a prior over functions, such as a Gaussian process.
Recently, however, a reformulation of BO via density-ratio estimation (BORE) allowed reinterpreting the acquisition function as a probabilistic binary classiﬁer, removing the need for an explicit prior over functions and increasing scalability.
In this paper, we present a theoretical analysis of BORE’s regret and an extension of the algorithm with improved uncertainty estimates.
We also show that BORE can be naturally extended to a batch optimisation setting by recasting the problem as approximate Bayesian inference.
The resulting algorithms come equipped with theoretical performance guarantees and are assessed against other batch and sequential BO baselines in a series of experiments.
Introduction
Bayesian optimisation (BO) algorithms provide ﬂexible black-box optimisers for problems involving functions which are noisy or expensive to evaluate [1].
Typical BO approaches place a probabilistic model over the objective function which is updated with every new observation in a sequential decision-making process.
Most methods are based on Gaussian process (GP) surrogates [2], which provide closed-form analytic expressions for the model’s posterior distribution and allow for a number of theoretical performance guarantees [3–5].
However, GP surrogates have a number of limitations, such as not easily scaling to high-dimensional domains, high computational complexity and requiring a careful choice of covariance function and hyper-parameters [2].
Non-GP-based BO methods have also been proposed in the literature, such as BO methods based on neural networks [6, 7] and random forests [8] regression models.
As an alternative to improving the model, Tiao et al.
[9] focus on the acquisition function, which in BO frameworks represents the guide that takes the model predictions into account.
They show that one can derive the acquisition function directly without an implicit model by reinterpreting the expected improvement [4, 1] via a density-ratio estimation problem.
Applying this perspective, the acquisition function can then be derived as a classiﬁcation model, which can be represented by ﬂexible parametric models, such as deep neural networks, and efﬁciently trained via stochastic gradient descent.
The resulting method, called Bayesian optimisation via density-ratio estimation (BORE) is then shown to outperform a variety of traditional GP-based and non-GP baselines.
Despite the signiﬁcant performance gains, BORE has only been applied to a sequential setting and not much is known about the method’s theoretical guarantees.
Batch BO methods have the potential to speed up optimisation in settings where multiple queries to the objective function can be evaluated simultaneously [10–13].
Given its ﬂexibility to apply models which can scale to large datasets, it is therefore a natural question as to whether BORE can be readily extended to the batch setting in a computationally efﬁcient way.
In this paper, we extend the BORE framework to the batch setting and analyse its theoretical performance.
To derive theoretical guarantees, we ﬁrst show that the original BORE can be improved by accounting for uncertainty in the classiﬁer’s predictions.
We then propose a novel method, called
BORE++, which uses an upper conﬁdence bound over the classiﬁer’s predictions as its acquisition function.
The method comes equipped with guarantees in the probabilistic least-squares setting.
We provide extensions for both BORE and BORE++ to the batch setting.
Lastly, we present experimental results demonstrating the performance of the proposed algorithms in practical optimisation problems.
Background
We consider a global optimisation problem over a compact search space X ⊂ R d of the form:
[Equation]
where f : X → R is assumed to be a black-box objective function, i.e., we have no access to gradients nor analytic formulations of it.
In addition, we are only allowed to run up to T rounds of function evaluations, where we might collect single points or batches of observations y t :
= f ( x t ) + (cid:15) t , which are corrupted by additive noise (cid:15)
2.1 Bayesian optimisation
Bayesian optimisation (BO) algorithms approach the problem in Equation 1 via sequential decision making [1].
At each iteration, BO selects a query point by maximising an acquisition function a :
[Equation]
The acquisition function encodes information provided by the observations collected so far D t :
= { x i , y i } t − 1 i =
1 using a probabilistic model over f , typically a Gaussian process (GP) [2], conditioned on the data.
After collecting an observation y t , the dataset is updated with the new query-observation pair D t :=
D t − 1 ∪ x t , y t .
This process then repeats for a given number of iterations T .
2.2 Bayesian optimisation via density-ratio estimation (BORE)
The expected improvement (EI) [4, 14] is a popular acquisition function in the BO literature and the basis for many BO algorithms.
At each iteration t ≥ 1 , one can deﬁne τ := min
i<t y t as an incumbent target.
EI is then deﬁned as:
[Equation]
In the case of a GP prior on f |D t − 1 ∼ GP ( µ t − 1 , k t − 1 ) , the EI is available in closed form as a function of the GP posterior.
However, the EI may be reformulated without the need for a prior.
Under mild assumptions, Bergstra et al.
[15] showed that the EI can be formulated as a density ratio between two probability distributions.
Let (cid:96) ( x ) :
= p ( x | y ≤ τ ) represent the probability density over x ∈ X conditioned on the observation y being below a threshold τ ∈ R .
Conversely, let g ( x ) :
= p ( x | y > τ ) .
For γ ∈ [0 , 1] , the γ -relative density ratio between these two densities is:
[Equation]
Algorithm
= ˆΦ −
−
( γ )
y i ≤ τ ] , i ∈ { 1 , . . .
, t − 1 } i ,
} t − 1 i =  noting that γ = 0 leads to the ordinary probability density ratio deﬁnition, ρ 0 ( x ) = (cid:96) ( x ) / g ( x ) .
Now if we choose τ :=
Φ − 1 ( γ ) , where Φ( s ) :
= p ( y ≤ s ) represents the cumulative distribution function of the marginal distribution of observations, 2 for s ∈ R , and then replace τ in Equation 3, Bergstra et al.
[15] have shown that 3 a EI ( x ) ∝ ρ γ ( x ) , for x ∈ X .
Based on this fact, Tiao et al.
[9] showed:
[Equation]
where π ( x ) :
= p ( y ≤ τ | x ) can be approximated by a probabilistic classiﬁer trained with a proper scoring rule, such as the binary cross-entropy loss:
[Equation]
Other examples of proper scoring rules include the least-squares loss, which leads to probabilistic least-squares classiﬁers [16], and the zero-one loss.
We refer the reader to Gneiting and Raftery [17] for a review and theoretical analysis on this topic.
BORE is summarised in Algorithm 1.
As seen, the marginal observations distribution CDF Φ( s ) :
= p ( y ≤ s ) is replaced by the empirical approximation ˆΦ t ( s ) :
= 1 t (cid:80) ti =
I
[ y i ≤ s ] and its corresponding quantile function ˆΦ − 1 t .
At each iteration, observations are labelled according to the estimated γ th quantile τ , and a classiﬁer ˆ π t is trained by minimising the loss L [ π | ˜ D t ] over the data points ˜ D t .
A query point x t is chosen by maximising the classiﬁer’s probabilities, which in our case corresponds to maximising the expected improvement.
A new observation is collected, and the algorithm continues running up to a given number of iterations T .
As demonstrated, no explicit probabilistic model for f is needed, only a classiﬁer, which can be efﬁciently trained via, e.g., stochastic gradient descent.
Analysis of the BORE framework
In this section, we analyse limitations of the BORE framework in modelling uncertainty and analyse its effects on the algorithm’s performance.
As presented in Section 2.2, at each iteration t ≥ 1 , the original BORE framework trains a probabilistic classiﬁer ˆ π t ( x ) to approximate p ( y ≤ τ | x ) , where τ denotes the γ th quantile of the marginal observations distribution, i.e., p ( y ≤ τ )
=
γ .
This approach leads to a maximum likelihood estimate for the classiﬁer ˆ π , which may not properly account for the uncertainty in the classiﬁer’s approximation.
Since BORE is based on probabilistic classiﬁers, instead of regression models as in traditional BO frameworks [1], a natural ﬁrst question to ask is whether a classiﬁer can guide it to the global optimum of the objective function.
The following lemma answers this question and is a basis for our analysis.
Lemma 1.
Let f : X → R be a continuous function over a compact space X .
Assume that, for any x ∈ X , we observe y = f
( x ) + (cid:15) , where (cid:15) is i.i.d.
noise with a strictly monotonic cumulative distribution function Φ (cid:15) : R → [0 , 1] .
Then, for any τ ∈ R , we have:
[Equation]
Proof.
As the observation noise CDF is monotonic, by basic properties of the argmax , we have:
[Equation]
which concludes the proof.
According to this lemma, maximising class probabilities is equivalent to optimising the objective function when the classiﬁer is optimal, i.e., it has perfect knowledge of f .
This result holds for any given threshold τ ∈ R .
We only make a mild assumption on the CDF of the observation noise Φ (cid:15) , which is satisﬁed for any probability distribution with support covering the real line (e.g. Gaussian,
Student-T, Cauchy, etc.).
To analyse BORE’s optimisation performance, we will aim to bound the algorithm’s instant regret:
[Equation]
and its cumulative version R T := (cid:80) Tt =1 r t .
Sub-linear bounds on R T lead to a no-regret algorithm, since lim T →∞ R T T = 0 and min t ≤ T r t ≤ R T T .
Assuming that there is an optimal classiﬁer π ∗ :
X → [0 , 1] , which is such that π ∗ ( x ) = p ( y ≤ τ | x , f ) , for a given τ ∈ R , we can directly relate the classiﬁer probabilities to the objective function f values, since:
[Equation]
The existence of the inverse Φ − 1 (cid:15) is ensured by the strict monotonicity assumption on Φ (cid:15) in Lemma 1.
Under this observation, the algorithm’s regret at any iteration t ≥ 1 can be bounded in terms of classiﬁer probabilities:
[Equation]
where L (cid:15) is any Lipschitz constant for Φ − 1 (cid:15) , which exists since X is compact.
Therefore, we should be able to bound BORE’s regret by analysing the approximation error for ˆ π t at each iteration t ≥ 1 .
Although approximation guarantees for classiﬁcation algorithms under i.i.d.
data settings are well known [18], each observation in BORE depends on the previous ones via the acquisition function.
This process is also not necessarily stationary, so that we cannot apply known results for classiﬁers under stationary processes [19].
In the next section, we consider a particular setting for learning a classiﬁer which allows us to bound the prediction error under BORE’s data-generating process.
3.1 Probabilistic least-squares classiﬁers
We consider the case of probabilistic least-squares (PLS) classiﬁers [20, 21].
In particular, we model a probabilistic classiﬁer π : X → [0 , 1] as an element of a reproducing kernel Hilbert space (RKHS)
H associated with a positive-deﬁnite kernel k : X × X → R .
A RKHS is a space of functions equipped with inner product (cid:104)· , ·(cid:105) k and norm (cid:107)·(cid:107) k := (cid:112) (cid:104)· , ·(cid:105) k
For the purposes of this analysis, we will also assume that k ( x , x ) ≤ 1 , for all X .
This setting allows for both linear and non-parametric models.
Gaussian assumptions on the function space would lead us to GP-based PLS classiﬁers [2], but we are not restricted by Gaussianity in our analysis.
If the kernel k is universal, as Φ (cid:15) is injective, we can also see that the RKHS assumption allows for modelling any continuous function.
For a given τ ∈ R , a PLS classiﬁer is obtained by minimising the regularised squared-error loss:
[Equation]
where λ > 0 is a given regularisation factor and z
i :=
I [ y i ≤ τ ] ∈ { 0 , 1 } .
In the RKHS case, the solution to the problem above is available in closed form [23, 16] as:
[Equation]
where k t ( x ) :=
[ k ( x , x 1 ) , . .
k ( x , x t )] T ∈ R t , K t :=
[ k ( x i , x j )] ti,j =1 ∈ R t × t and z t :=
[ z 1 , . . .
, z t ] T ∈ R t .
This PLS approximation may not yield a valid classiﬁer, since it is possible that ˆ π t ( x ) / ∈ [0 , 1] for some x ∈ X .
However, it allows us to place a conﬁdence interval on the optimal classiﬁer’s prediction, as presented in the following theorem, which is based on theoretical results from the online learning literature [24, 25].
Our proofs can be found in the supplement.
Theorem 1.
Given τ ∈ R , assume π ( x ) :=
Φ (cid:15) ( τ − f ( x )) is such that π ∈ H , and (cid:107) π (cid:107) k ≤ b .
Let { x t } ∞ t =1 be a X -valued discrete-time stochastic process predictable with respect to the ﬁltration { F t } ∞ t =0 .
Let { z t } ∞ t =1 be a real-valued stochastic process such that ν t :=
z t − π ( x t ) is 1 -subGaussian conditionally on F t − 1 , for all t ≥ 1 .
Then, for any δ ∈ (0 , 1) , with probability at least 1 − δ , we have that:
[Equation]
(14) where β t ( δ ) :
=
b + (cid:112) 2 λ − 1 log( | I + λ − 1 K t | 1 / 2 /δ ) , with | A | denoting the determinant of matrix A , and σ 2 t ( x ) :
= k ( x , x ) − k t ( x ) T ( K t + λ I ) − 1 k t ( x ) , x ∈ X , t ≥ 1 .
3.2 Regret analysis for BORE
We now consider BORE with a PLS classiﬁer.
For this analysis, we will assume an ideal setting where τ is ﬁxed, possibly corresponding to the true γ th quantile of the observations distribution.
However, our results hold for any choice of τ ∈ R and can therefore be assumed to approximately hold for a varying τ which is converging to a ﬁxed value.
In this setting, the algorithm’s choices are: given by:
[Equation]
where ˆ π t is the estimator in Equation 13.
we can then apply Theorem 1 to the classiﬁer-based regret in Equation 11 to obtain a regret bound.
For this result, we will also need the following quantity:
[Equation]
where the maximisation is taken over the discrete set of locations { x i } Ni =1 ⊂ X and K N :=
[ k ( x i , x j )]
Ni,j =1 .
This quantity denotes the maximum information gain of a Gaussian process model after N observations.
We are now ready to state our theoretical result regarding BORE’s regret.
Theorem 2.
Under the conditions in Theorem 1, with probability at least 1 − δ , δ ∈ (0 , 1) , the instant regret of the BORE algorithm with a PLS classiﬁer after T ≥ 1 iterations is bounded by:
[Equation]
and the cumulative regret by:
[Equation]
As Theorem 2 shows, the regret of the BORE algorithm in the PLS setting is comprised of two components.
The ﬁrst term is related to the regret of a GP-UCB algorithm [see 26, Thr. 3] and its known to grow sub-linearly for a few popular kernels, such as the squared exponential and the Matérn class [3, 27].
The second term, however, reﬂects the uncertainty of the algorithm around the optimum location x ∗ .
If the algorithm never samples at that location, this second summation might have a mostly linear growth, which will not lead to a vanishing regret.
In fact, if we consider Equationand a RKHS with a translation-invariant kernel, we see that, as soon as an observation z t = 1 is collected at a location x t (cid:54) = x ∗ , that location will constitute the maximum of the classiﬁer output.
Then the algorithm would keep returning to that same location, missing opportunities to sample at x ∗ .
It is worth noting that Theorem 2 reﬂects the regret of BORE in an idealistic setting where the algorithm uses the optimal PLS estimator in the function space H .
However, if we train a parametric classiﬁer, such as a neural network, via gradient descent, the behaviour will not necessarily be the same, and the algorithm might still achieve a good performance.
In the original BORE paper, for instance, a parametric classiﬁer is trained by minimising the binary cross-entropy loss [9] and leads to a successful performance in experiments.
Neural network models trained via stochastic gradient descent are known to provide approximate samples of a posterior distribution [28, 29], instead of an optimal best-ﬁt predictor, which might make BORE behave like Thompson sampling [30] (see discussion in the appendix).
Nevertheless, Theorem 2 still shows us that BORE may get stuck into local optima, which is not ideal for BO methods.
In the next section, we present an extension of the
BORE framework which addresses this shortcoming.
BORE++: improved uncertainty estimates
As discussed in the previous section, the lack of uncertainty quantiﬁcation in the estimation of the classiﬁer for the original BORE might lead to sub-optimal performance.
To address this shortcoming, we present an approach for uncertainty quantiﬁcation in the BORE framework which leads to improvements in performance and theoretical optimality guarantees.
Our approach is based on using an upper conﬁdence bound (UCB) on the predicted class probabilities as the acquisition function for
BORE.
Due to its improved uncertainty estimates, we call this approach BORE++.
4.1 Class-probability upper conﬁdence bounds
We propose replacing ˆ π t in Algorithm 1 by an upper conﬁdence bound which is such that:
[Equation]
which with probability greater than 1 − δ , given δ ∈ (0 , 1) .
Therefore, π t,δ ( x ) represents an upper quantile over the optimal class probability π ∗ ( x ) .
BORE++ selects x t ∈ argmax x ∈X π t − 1 ,δ ( x ) .
To derive an upper conﬁdence bound on a classiﬁer’s predictions π ( x ) , we can take a few different approaches.
For a parametric model π θ , a Bayesian model updating the posterior p ( θ |D t ) leads to a corresponding predictive distribution over π θ ( x ) .
This is the case of ensemble models [31], for instance, where we approximate predictions p ( y ≤ τ t | x , D t )
≈ 1 M (cid:80)
Mi
=1 π θ i ( x ) with θ i ∼ p ( θ |D t ) .
Instead of using the expected class probability, however, BORE++ uses an (empirical) quantile approximation for π t,δ to ensure Equation 19 holds.
Bayesian neural networks [32], random forests [33], dropout methods, etc.
[34], also constitute valid approaches for predictive uncertainty estimation.
An alternative approach is to place a non-parametric prior over π ∗ , such as a Gaussian process model [2], which allows for the modelling of uncertainty directly in the function space where π ∗ lies.
In the next section, we present a concrete derivation of BORE++ for the PLS classiﬁer setting which takes the non-parametric perspective and allows us to derive theoretical performance guarantees.
4.2 BORE++ with PLS classiﬁers
In the PLS setting, the result in Theorem 1 gives us a closed-form expression for a classiﬁer upper conﬁdence bound satisfying the condition in Equation 19.
Given δ ∈ (0 , 1) , we set:
[Equation]
where σ t and β t are set according to Theorem 1.
We then obtain the following result for BORE++.
Theorem 3.
Under the assumptions in Theorem 1, running the BORE++ algorithm with a PLS classiﬁer π t,δ as deﬁned above yields, with probability at least 1 − δ , an instant regret bound of:
[Equation]
and a cumulative regret bound after T ≥ 1 iterations:
[Equation]
According to Theorem 3, the regret of BORE++ vanishes if the maximum information gain ξ T grows sub-linearly, since lim T →∞ R T T = 0 and min t ≤ T r t ≤ R T T .
Sub-linear growth is known to be achieved for popular kernels, such as the squared exponential, the Matérn family and linear kernels [3, 27].
This result also tells us that theoretically BORE++ performs no worse than GP-UCB since they share similar regret bounds [3, 26].
However, in practice, the BORE++ framework offers a series of practical advantages over GP-UCB, such as no need for an explicit surrogate model, and a classiﬁer which does not need to be a GP and can therefore be more ﬂexible and scalable to high-dimensional problems and large amounts of data.
The connection with GP-UCB, instead, brings us new insights into how the density-ratio BO algorithm can still share some of the well known guarantees of traditional BO methods.
Batch BORE
This section proposes an extension of the BORE framework which allows for multiple queries to the objective function to be performed in parallel.
Although many methods for batch BO have been previously proposed in the literature, we here focus on approaching batch optimisation as an approximate Bayesian inference problem.
Instead of having to derive complex heuristics to approximate the utility of a batch of query points, we can view points in a batch as samples from a posterior probability distribution which uses the acquisition function as a likelihood.
5.1 BORE batches via approximate inference
Applying an optimisation-as-inference perspective to BORE, we can formulate a batch BO algorithm which does not require an explicit regression model for f .
The classiﬁer ˆ π ( x ) ≈ p ( y ≤ τ | x ) naturally turns out as a likelihood function over query locations x ∈ X .
Since the search space X is compact, we can assume a uniform prior distribution p ( x ) ∝ 1 .
Also note that the normalisation constant in this case is simply (cid:82) X p ( y ≤ τ | x ) p ( x ) d x = p ( y ≤ τ )
=
γ .
Our posterior distribution then becomes:
[Equation]
Therefore, we formulate a batch version of BORE as an inference problem aiming for:
[Equation]
(24) where D KL ( q || (cid:96) ) denotes the Kullback-Leibler (KL) divergence between q and (cid:96) , and P represents the space of probability distributions over X .
Sampling from (cid:96) would allow us to obtain the points of interest in the search space, including the optimum x ∗ and other locations where y ≤ τ .
However, as the true p ( y ≤ τ | x ) is unknown, we instead formulate a proxy inference problem with respect to a surrogate target distribution ˆ p t based on the classiﬁer model.
For BORE, we set ˆ p t ( x ) ∝ ˆ π t − 1 ( x ) , while for BORE++ the setting is ˆ p t ( x ) ∝ π t − 1 ,δ ( x ) .
In contrast to (cid:96) ( x ) ∝ p ( y ≤ τ | x ) , the normalisation constant for the surrogate distributions is unknown, leading us to a proxy problem of minimising q t ∈ argmin q ∈P D KL ( q || ˆ p t ) at each iteration t ≥ 1 .
This variational inference problem above can be efﬁciently solved via Stein variational gradient descent (SVGD) [35], described next.
5.2 Batch sampling via Stein variational gradient descent
In our implementation, we apply SVGD to approximately sample a batch B t :=
{ x t,i } Mi =1 of M ≥points from ˆ p t .
Other approximate inference algorithms could also be applied.
One of the main advantages of SVGD, however, is that it encourages diversiﬁcation in the batch, capturing the possible multimodality of ˆ p t .
Given the batch locations, observations can be collected in parallel and then added to the dataset to update the classiﬁer model.
SVGD is an approximate inference algorithm which represents a variational distribution q as a set of particles { x i } Mi =1 [35].
The particles are initialised as i.i.d.
samples from an arbitrary base distribution and then optimised via a sequence of smooth transformations towards the target distribution, which in our case corresponds to ˆ p t ∝ π t − 1 ,δ .
The SVGD steps are given by:
[Equation]
where k : X × X
→ R is a positive-deﬁnite kernel, and α > 0 is a small step size.
Intuitively, the ﬁrst term in the deﬁnition of ζ t guides the particles to the modes of ˆ p t , while the second term encourages diversiﬁcation by repelling nearby particles.
Theoretical convergence guarantees [36, 37] and practical extensions, such as second-order methods [38, 39] and derivative-free approaches [40], have been proposed in the literature.
Further details on SVGD can be found in Liu and Wang [35].
5.3 Regret bound for Batch BORE++ with PLS classiﬁers
We follow the derivations in Oliveira et al.
[41] to derive a distributional regret bound for batch
BORE++ with respect to its target sampling distribution (cid:96) , which is presented in the following result.
Theorem 4.
Under the same assumptions in Theorem 1, running batch BORE++ with π t,δ set as in
Equation 20, we obtain a bound on the instantaneous distributional regret:
[Equation]
where L π := max x ∈X 1 π ( x ) , and on the cumulative distributional regret at T ≥ 1 :
[Equation]
both of which hold with probability at least 1 − δ .
As in the case of non-batch BORE++, the distributional regret bounds for the batch algorithm also grow sub-linearly for most popular kernels, leading to an asymptotically vanishing simple regret.
Although different, to compare the distributional regret of batch BORE++ with the non-distributional regret bounds for BORE++, we may consider a case where τ is set to the function minimum τ := f ( x ∗ ) =
min x ∈X f ( x ) and the observation noise is small.
In this case, the batch sampling distribution would converge to a Dirac at the optimum, so that E x ∼ (cid:96) [ f ( x )]
≈ f ( x ∗ ) .
Compared to the regret of non-batch BORE++ (Theorem 3) after collecting an equivalent number of observations
T (cid:48) :
= MT , the expected regret of the batch version of BORE++ after T iterations is then lower by a factor of ξ T /ξ MT , noting that ξ T ≤ ξ T (cid:48) =
ξ MT .
Therefore, batch BORE++ should be able to achieve lower regret per runtime than sequential BORE++ with an equivalent number of observations.
Related work
Since their proposal by Schonlau et al.
[42], batch Bayesian optimisation methods have appeared in various forms in the literature.
Many methods are based on heuristics derived from estimates given by a Gaussian process regression model [11, 12, 43].
Others are based on Monte Carlo estimates of multi-query acquisition functions [10, 13], optimising points over GP posterior samples [44], solving local optimisation problems [45], or optimising over ensembles of acquisition functions [46].
Despite that, the prevalent approaches to batch BO are still based on a GP regression model, which require prior knowledge about the objective function and do not scale to high-dimensional problems.
We instead take a different approach by viewing BO as a density-ratio estimation problem following the
BORE framework by Tiao et al.
For batch design, we take an optimisation-as-inference approach [47, 48] by applying Stein variational gradient descent, a non-parametric approximate inference method [35], which has been recently combined with GP-based BO [49, 50].
Our theoretical results, however, are agnostic to the choice of inference algorithm.
In contrast to traditional batch BO methods, the inference approach does not require solving inter-dependent optimisation problems for each batch point, as in heuristic-based approaches [43, 11, 12], Monte Carlo integration over the
GP posterior [10, 13], nor sampling from it [44].
SVGD allows batch selection to be solved in a vectorised way, which can take advantage of hardware accelerators, such as GPUs.
Experiments
This section presents experiments assessing the theoretical results and demonstrating the practical performance of batch BORE on a series of global optimisation benchmarks.
We compared our methods against GP-based BO baselines in both experiments sets.
Additional experimental results, including the sequential setting (Appendix E), a description of the experiments setup (Appendix E), and further discussions on theoretical aspects can be found in the supplementary material.
Theory assessment.
We ﬁrst present simulated experiments assessing the theoretical results in practice, testing BORE and BORE++ in the PLS setting.
As a baseline, we compare both methods against GP-UCB.
This experiment was run by generating a random base classiﬁer in the RKHS
H k
and then a corresponding objective function via the inverse noise CDF Φ − 1 (cid:15) .
The search space was set as a uniformly-sampled ﬁnite subset of the unit interval X :=
[0 , 1] ⊂ R .
We applied the theory-backed settings for BORE++ (Section 2.2) and GP-UCB [25], while BORE employed the optimal PLS classiﬁer (Equation 13).
[Figure]
As the results in Figure 1 show, BORE using an optimal PLS classiﬁer simply gets stuck at a its initial point, resulting in constant regret.
BORE++, however, is able to progress in the optimisation problem towards the global optimum, outperforming the GP-UCB baseline.
Global optimisation benchmarks.
We evaluated the proposed SVGD-based batch BORE method in a series of test functions for global optimisation comparing it against other BO baselines.
In particular, for our comparisons, we ran the locally penalised EI (LP-EI) method [11] and the Monte
Carlo based q -EI method [10], which are both based on the EI algorithm, like BORE.
Results are presented in Figure 2.
All methods ran for T := 200 iterations and used of batch size of 10 evaluations per iteration.
Additional experimental details are deferred to the supplementary material.
As Figure 2 shows, batch BORE is able to outperform its baselines on most of the global optimisation benchmarks.
We also note that, in some case, due to its complexity the LP-EI method becomes computationally infeasible after 100 iterations, having to be aborted halfway through the optimisation.
Batch BORE, however, is able to maintain steady performance throughout its runs.
Real-data benchmarks.
Lastly, we compared the sequential version of BORE++ against BORE and other baselines, including traditional BO methods, such as GP-UCB and GP-EI [1], the Treestructured Parzen Estimator (TPE) [15], and random search, on real-data benchmarks.
In particular, we assessed the algorithms on some of the same benchmarks present in the original BORE paper [9].
Results are presented in Figure 3.
As the plots show, BORE++ presents signiﬁcantly better performance than BORE in the probabilistic least-squares (PLS) setting (i.e., β t := 0 ), as the theoretical results suggested.
In fact, it is possible to note that BORE (PLS) performs comparably to (or at
[Figure]
times worse than) random search, indicating that the optimal least-squares classiﬁer by itself is unable to properly capture the epistemic uncertainty.
By using a neural network classiﬁer trained via gradient descent and a different loss function (cross-entropy), the original BORE is still able to achieve top performance in most benchmarks.
Both BORE versions are only surpassed by traditional
GP-based BO on the racing line optimisation problem, as observed in Tiao et al.
[9], due to the inherent smoothness the problem, and in the ﬁnal iterations of the neural architecture search problem by GP-EI.
Interestingly, even though restricted to the kernel-based PLS setting, we observe that
BORE++ is able to surpass the original BORE in the NN tuning problems ( SLICE and PARKINSONS ), while maintaining similar performance in other tasks.
These results conﬁrm that improved uncertainty estimates can lead to practical performance gains.
Conclusion
This paper presented an extension of the BORE framework to the batch setting alongside the theoretical analysis of the proposed extension and an improvement over the original BORE.
Theoretical results in terms of regret bounds and experiments show that BORE methods are able to maintain performance guarantees while outperforming traditional BO baselines.
The main purpose of this work, however, was to establish the theoretical foundations for the analysis and derivation of new algorithmic frameworks for Bayesian optimisation via density-ratio estimation, equipping BO with new tools based on probabilistic classiﬁcation, instead of regression models.
As future work, we plan to investigate the theoretical properties of BORE under different loss functions and analyse other batch design strategies.
The theoretical contributions of this work can also be extended to other versions of BORE, such as its recent multi-objective version [51], and provide insights into other likelihood-free BO approaches [52].
We also consider integrating BORE++ with other probabilistic classiﬁcation models embedded with uncertainty estimates, such as neural network ensembles [53], random forests [54], and Bayesian generalised linear models, which should enable further performance improvements.
Acknowledgments and Disclosure of Funding
The ﬁrst author was supported by the Medical Research Future Fund (MRFF)
Applied Artiﬁcial
Intelligence in Health Care grant (MRFAI000097).
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[TODO] Code is planned to be made available after the paper publication.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See supplement.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] Our focus is on theory assessments rather than computational comparisons.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]