Bayesian optimization with derivatives acceleration
Authors: Guillaume Perrin
rodolphe le riche
Venue: Accepted by TMLR ()

Abstract: Bayesian optimization algorithms form an important class of methods to minimize functions that are costly to evaluate, which is a very common situation. 
These algorithms iteratively infer Gaussian processes from past observations of the function and decide where new observations should be made 
through the maximization of an acquisition criterion. 
Often, 
the objective function is defined on a compact set such as in a hyper-rectangle of the $d$-dimensional real space, 
and the bounds are chosen wide enough so that the optimum is inside the search domain.
In this situation, this work provides a way to integrate in the acquisition criterion the \textit{a priori} information that 
these functions, once modeled as GP trajectories, should be evaluated at their minima, and not at any point as usual acquisition criteria do.
We propose an adaptation of the widely used Expected Improvement acquisition criterion that
accounts only for GP trajectories where the first order partial derivatives are zero and the Hessian matrix is positive definite.
The new acquisition criterion keeps an analytical, computationally efficient, expression. 
This new acquisition criterion is found to improve Bayesian optimization on a test bed of functions made of Gaussian process trajectories in low dimension problems. 
The addition of first and second order derivative information is particularly useful for multimodal functions.

Published in Transactions Machine Learning Research (04/2024)
Bayesian optimization with derivatives acceleration
Guillaume Perrin
Université Gustave Biffel, COSYS, 5 Bd Descortes 77454 Morne-La- Vallée, France
Rodolphe Leriche
leriche Demse.f
LIMOS (CNRS, Mines Saint-El EEienne, UCA) Saint-Elee Siint-E,ienne, Fronce
Reviewed OpenReview: https: // openreview. net/ forum? id= XXXX
Abstract
Bayesian optimization algorithms form an important class of methods minimize func- tions that costly to evaluate, which is very common situation. These algorithms iteratively infer Gaussian processes from past observations of the function and decide where new observations should be made through the maximization of an acquisition crite- rion. Often, the objective function is defined on compact set such as in hyper-rectangle of the d-dimensional real space, and the bounds are chosen wide enough so that the opti- mum is inside the search domain. this situation, this work provides a way to integrate in the acquisition criterion the a priori information that these functions, once modeled GP trajectories, should be evaluated at their minima, and at any point as usual acqui- sition criteria do. We propose adaptation of the widely used Expected Improvement acquisition crilerion that accounts only for GP trajectories where the first. order partial derivatives are zero and the Hessian matrix is positive definite. The new acquisition cri- terion keeps an analytical, computationally efficient, expression. This new acquisition criterion is found to improve Bayesian optimization on a test bed of functions made of Gaussian process trajectories low dimension problems. The addition of first and second order derivative information particularly useful for multimodal functions.
Introduction
Over the last 20 years, Bayesian optimization (BO) methods have established themselves as one of the references for approximating the point(s) minimizing an popenssiv--oo-vallaae black- function, from as few calls to this function as possible. This is reflected in the existence of many reviews and tutorials on BO in the literature (see for instance Jones (2001): (Sobester et al. (2008); Shahhiiari al (2015): Gramacy (020)); Garnett (2023): Frazier (2018). as well many applications of BO in industrial applications, such aeronautics Forrester et al (2007); Lam et al. (2018) agriculture Picheny et al (2017)). Today, the
machine learning community is key contributor to BO advances, motivated by the need to optimize hyper-
parameters [Bergstra et al. (2011): 1): (nook et al. (2012): Klein et al. (2017): : Wn et al. (2019): Turner et al.
(2021) or exploration strategies in reinforcement learning Wang al (2023). More specifically, BO
concerned with minimization problems that can be writlen the following form:
x= E arg min y(x),
where is a pointwise observable function defined over the compact set X c Rd, d 1. BO assumes that y can be usefully represented as a Gaussian process (GP), whose mean and covariance functions have been identified from a limited number calls to function It then sequentially adds new observations of y points maximizing an acquisition criterion whose objective, in the scarch for the global minimum,
1

---

Published in Transactions on Machine Learning Research (04/2024)
to make a judicious trade-off between the exploration of and the exploitation of past observations. In the theory of decision under uncertainty, acquisition criteria are the expectation of utility of the possible function observations according to the stochastic model of the objective function Zilinskas & Calvin (2019)
Several acquisition criteria have been proposed. The earliest, one-dinensional, version of BO Kushner (1962) involved the probability of improvement and an upper confidence bound. The upper confidence bound was later theoretically studied in many dimensions in Srinivas et al. (2010). Another early BO acquisition critcrion was described in Močkus (1972) which is, since Frazicr & Powell (2007), called the knowledge gradient. It is a one-step-ahead expected progress in GP mean. The Expected Improvement beyond the current best observation (EI) is the most classical acquisition criterion. The El has a simple interpretation and an analytical expression deprived of parameters to tune, two features which have con- tributed to its popularity. It was first proposed in [Saltenis (1971) and popularized in [Schonla (1997): Jones et al. (1998); Močkus (2012). More recently, acquisition criteria based on information theory have been suggested which target entropy reductions in the GP model extrema Lornndndez--obattte et al (2014) or locations of extrema Villemonteix el. al. (2009); ; Hennig & Schulen (2012).
BO particularly efficient when the dimension of the search space remains limited (d & 5 to 10) and when the function is multimodal with some structure Le Riche & Picheny (2021). Several adaptations of this formalism have been proposed to extend the efficiency of these approaches to larger input spaces, by playing directly on the acquisition criterion Siivola et al (2018), on the identification of latent spaces of reduced dimensions Bouhlel et all (2016); Gaudric et al (2020), on the introduction of trust regions Diouane et. al. (2022), or by replacing the GP by a Bayesian neural network Kim el. al (2022).
It sometimes happens that the derivatives of the true function at the observed points is available (e.g.. through automatic diffcrentiation, or adjoint codes in partial differential equations solvers). It is then possible to add these derivatives as part of the observations of vectorized Gaussian process composed of the function prediction and its derivatives Laurent et al (2019). All the above acquisition criteria. could then be calculated with such a gradient-enriched underlying Gaussian Process. This has been done with the pradient-knowledge criterion in Wu et al (2017).
It is nevertheless interesting to note that all of these methods only exploit a limited part of the information conveyed by the GP (once conditioned by the observations of the true function and potentially by the observations of its gradient). In particular, they do not take into account the information that the GP derivatives could bring, whether the function y is convex or not, and even if the derivatives of y are not observed. Indeed, when y is twice differentiable, it is well known that the first derivatives of y become zero and that its Hessian matrix is positive definite at its minimum (unless the minimum lies al an edge of the domain). It is reasonable to believe that the minimization strategy can only benefit from this supplementary knowledge on derivatives. intuition. It shows three plots with GP trajectories conditioned by three observations (also known as a kriging model). The two bottom plots further condition the trajectories on their derivatives so that they have local minima at the wrong (Ieft) or right location. Because it is easier force local minima where the true function really has optimum, the information on local minima helps better discriminating the optimal from the non-mptimal region. Note also that in Figure while the first and second order derivatives of the GP trajectories are constrained, no information about the derivatives of the true function is used. This is a key difference with other work on BO with gradient knowledge such as Wu et al (2017).
Figure D provides a graphical illustration to support this
With this in mind, the main contribution of this paper is to propose an adaptation of the famous El criterion so that it integrates the information of zero derivative and positive definite Hessian matrix of the GP trajectories. In other terms, this new criterion only accounts for possible minima of the GP trajectories, as opposed to the traditional El that can confer utility to any part of trajectory. We emphasize that the proposed criterion does not imply that derivatives of the true function y(æ) be calculated. The derivatives only concern the GP.
The new criterion is meaningful if the minimum is located inside the search domain, which is a reasonable assumption in most applications where, precisely, the bounds are chosen as extremes that should not be reached. A complementary idea for cases where the bounds might be active is nevertheless given as a
2

---

Published in Transactions Machine Learning Research (04/2024)
1.5
0.5
-0.5
local optima
local
optima
1.5
1.5
0.5
0.5
-0.5
-0.5
-1.5
-1.5
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Figure 1: An illustration of the motivation for conditioning kriging trajectories with derivatives. Top: kriging trajectories in red, true function as black solid line. Bottom: kriging trajectories forcing, by Gaussian conditioning, a zero derivative and a positive second derivative at the vertical dotted bar, i.e. at the global maximum of the true function in the figure on the left, and at the global minimum in the figure the right. The difference between trajectories the maximum and minimum of the true function is more apparent when forcing local minima the right location.
3

---

Published in Transactions on Machine Learning Research (04/2024)
perspective: a method is proposed to estimate the likelihood that the minimum of y is on the edge of the domain.
Empirically investigating the effect of new idea - here adding derivatives acceleration on optimization algorithm is difficult because the performance of algorithu depends on both the function it is applied to and the tuning of its hyperparameters. The empirical tests we provide are designed to exclusively show the effects of the derivatives acceleration while avoiding all such experimental side effects. This is achieved firstly by testing on Gaussian processes whose rrper-parameters are known, therefore guarantecing the compatibility of the model and the test function. Secondly, the maximization the acquisition crileria done with extreme care, which feasible up to dimension 5 and needs to be relaxed beyond.
The outline of this paper is as follows. Scction 2 recalls the theoretical bases of the Gaussian process regression (GPR) and its use for the minimization of black-box functions. Section B introduces the ac- quisition criterion we propose for taking into account information on the derivatives of Section g then illustrates the benefits of this new acquisition criterion on simulated test functions that can be modeled as realizations of Gaussian processes. Section b describes how optima on the bounds can be handled and concludes the paper.
2 The BO general framework
For d > 1, let X be a compact subset of Rd In this work, we are interested in finding the solution(s) x' of the optimization problem defined by Eq. ① using as few pointwise observations of as possible. Anticipating the developments in the following sections exploiting the gradient y, we assunee that y is an element of C² (X, R), the set of real-valued twice contimuously differentiable functions defined on X. In addition, we treat X explicit, which means that the function y cannot be evaluated outside the search region (it is defined as product of intervals in the applications).
To solve this problem, consider Bayesian Optimization guided by the Expected Improvement (EI) acquisition criterion. Such methods are sometimes called Efficient Global Optimization algorithms in reference to Jones et al (1998), although implementations (of the GP and of the EI maximization) vary. The choice of the EI acquisition criterion is guided by simplicity: it is the most standard criterion and most importantly, it does not require GP simulations to be evaluated. Others criteria could benefit from derivatives acceleration as discussed in the perspectives of this article (Section (5.2). вO relies the evaluation of the objective function at sequence of well-chosen points as summarized hereunder and in Algorithm I
Initialization
To begin, the function y is evaluated at Ny points uniformly chosen in X (typically according to space- filling design of experiments (DoE) Fang et al. (2000); Perrin & Cannamela (2017)). We note (x("), Un := ()) No the obtained pairs. Given this available data, GP-based surrogate model is trained for To obtain convergence results, common theoretical assumption is that is particular realization of a Gaussian process Y ~ GP(,,C) whose prior mean and prior covariance functions are noted and C respectively (see Santner al. (2003); Rasmussen (2003) for more details about Gaussian process regression). In practice, it is only required that y can be observed at a finite number of points and the assumption of y being a sample of Y may not hold. The algorithm then conditions Y to interpolate the No available input-output pairs, resulting in new YN, GP(NN,, CNo), where:
(x) = p(x) + C(., (xx - -x(X), D
(2)
C2o(2,x²) = -X,,, 2, e
(3)
4

---

Published in Transactions Machine Learning Research (04/2024)
In the former expressions, := [xx(1) cf(oo) is the (No x d)-dimensional matrix that gathers the available imput points, and for any function f and g defined on X and X respectively, the following notation is adopted:
(f(x)), = =((x! (o(X, g( Im), 16n, m. & No-
(4)
Iteration
Given N > No evaluations of y, an acquisition criterion is introduced to choose at which point to carry out the (N + 1)th evaluation y. In the noise-free setting, the classical acquisition criterion is the Expected Improvement (EI). It is the expectation of a utility at æ defined as the progress below the current best observation:
EIx( = [max Umii Vv(x)))
max(0, Umin - ()
(5)
(x)))) (Z (UN(2)) + (UN(=))))
Here, (x) :- (ymin (o(=)= (x) :- v Cx(ar,)) Ymin the current minimum of the N obsorvations y. noted )(.=))).... + and & denote the probability density function (PDF) and cumulative density function (CDF) of the standard Gaussian variables, and fyw(e)(y) = Q( (x(=)))oxx()))/oxx(( the PDF of the Gaussian random variable Yv(xz) -N(u(()), where
YN = Y( y(="")...,,
(Nz
By construction, this acquisition criterion seeks a compromise between exploitation (first term) and explo- ration (second term) for the global search of the minimum, and the next evaluation point is chosen such that
x (N+1) arg max EI (x:). meX
Stopping criterion
For most existing implementations of BO, the stopping criterion is maximum number of evaluations of Indeed, unlike gradient-based approaches for minimizing convex functions, once a local minimum of has been found, there is no theoretical guarantee that it corresponds to the global minimum of y. While may be tempting, stopping the search when the expected improvement drops below a lower bound unstable in practice the El changes lot with the GP length scales.
Degrees of freedom
The performance of the BO method depends several degroes of freedom that vary between implemen- tations. The choice for a and C, the way the parameters which μ and C depend are optimized, the ratio No/budget, the way the initial DoE is constructed, the way the acquisition criterion is maximized are all important (see Le Riche & Pichemy (2021) for an investigation of the influence of these choices).
However, as the paper studies an adaptation of the acquisition criterion, it is clearer to fix these parameters standard values of the literature. To this end, the function μ is taken as constant, and the function C is chosen in the class of tensorized Matérn kernels with smoothing parameter = 5/2 (see Santner et al. (2003) for alternative classes of functions):
[
x/y1 e
p(x) = B, C(x, x0) == ==
x, E X.
5

---

Published in Transactions Machine Learning Research (04/2024)
Algorithm Standard BO algorithm.
Choose No. budget, Y C)
Initialization
Draw at random No points a+1), ..... x (No) in X
Compute v(x("),, 1 & n & No, estimate the parameters on which j and C' depend Define YN,- == (x) = y(x("),.... ((x(N)) = y(x(No));
Set h == 0
Iteration
while & < budget do
Search for x (No := arg max EINo+
Evaluate at w((o+++11) (and potentially adjust the expressions of μ and C)
Define No+++11 := (x) v( ., (x ((z(Notk)))
Updale & - k +1
end
Return min1 gi No budget ((x(1)).
x(u)= = (1 vou zu²) exp (-v3u). u 2
The Matérn 5/2 kernel is selected here because it is well-performing and common in the literature (L Riche & Picheny (2021)). Furthermore,
will introduce acquisition criterion that needs the fourth order derivatives of the kernel (to have information about the curvatures of the trajectories). The Matérn 5/2 kernel is precisely four times differentiable, yielding trajectories that are two times con- tinuously differentiable. The hyperparameter vector 0 == (3, a, fi....,ea la) will either be considered known (via the definition of test functions to be minimized in the form of a particular realization of Gaussian pro- cess of chosen parameters), or estimated by its maximum likelihood estimator (see Williams & Rasmussen (2000) for further details). As focus costly functions,
will set the maximal budget between 10 and 20 times the dimension d of the problem, while No will be chosen small (most of the time will have N₂ = 3). The initial DoE will always be random space-filling Latin Hypercube Sample (LHS) Damblin et al. (2013) Perrin & Cannamela (2017). For objective numerical comparisons, the maximiza- tion of the acquisition criteria, whether it is the EI in Equation 7 or one of the new criteria of Section B] is always carried out in the same way. At each iteration, the acquisition criterion is first evaluated at a very large number of points randomly chosen in (typically of the order of 10(+1). The Pelder-Mead algorithm Nelder & Mead (1965) then maximizes the acquisition criterion starting from the 10 most promising points among the random points.
3 Extending the Expected Improvement with derivatives
We now show how exlend the Expected Improvement acquisition criterion so that accounts for gradient and Hessian information. The principles underlying the calculations are that GP derivatives GPs, and that local optima away from the bounds coincide with canceling derivatives and positive definite Hessians. These principles have already been used in the context of BO in Hernández- et al (2014) for approximating the entropy of local optima. An independent and differing version, adapted to EI, described hereafter.
3.1 Reminders on Gaussian process derivation
The acquisition criteria reviewed in the Introduction, in particular the El, are only based on the distribution of YN (x) and do not include information related to higher derivatives. Yet, when the functions x - u(x) and (x, x²) - C(x, x²) are sufficiently regular, the statistical properties of the derivatives of Y can be deduced by simple derivations of μ and C. Indced, as the Gaussian distribution is stable by linear
6

---

Published in Transactions Machine Learning Research (04/2024)
operations, for any linear operator such that Ly a function from Id to Rdd,, CY is also a Gaussian process, with:
E [I Lu(x), Cov(CY(z)) = ='(x,x'')
(10)
Here, the notations [C(x, x') and C(., x') indicate that operator L is applied as function of and x' respectively, so that Cov(CY (x))) is a (de de)-dimessional matrix. In particular, for de = + + 2, if we choose
AY
OY
a²Y
O²Y Our
L:Y - LY Y,
0m
daa"
dx
da,day
we obtain the joint distribution of Y and its first and second order derivatives. For each twice-diffeeenntaalle function 2, we introduce the following notations,
8
d2i
Ōz ==
822
D² == diag((?2)
(11)
Ze
and denote by M (d) the set of (d d)-dimensional positive definite matrices.
3.2 An acquisition criterion accounting for the derivatives
For any in X, it is well known that if (=(x)) = 0 and 0²x(x) E Md),,, æ local minimum of z. As the input space X is bounded, the reciprocal however not true, since local minimum be found at the boundary of X with non-zero gradient and/or @ 0²2(x) $ (d)... The case when the optima are the bounds will be discussed in Section b.1 For now we focus on the interior of X, to integrate as prior knowledge that the gradient will be zero and the matrix of curvatures posilive definite at the local minima
of y, the El criterion defined by Eq. (5) can be replaced by:
(eriv-EIx(a)) := E BBdde
max ymin Umin
(x)))
(12)
where for any > 0, ((s) {U E Rd, | < <) matrix such that
the d-dimensional hypersphere of radius E, Ry (x) is a
(o(e))CoV(QW((()) Id.
(13)
and for any event a, 1a is equal to 1 if is true and to 0 otherwise. OY has covariance matrix (made of the second derivatives of (x a²)) with correlations and ils density has ellipsoidal level sets. The normalized vector (2) is uncorrelated, its covariance is the identity Ia, and its density has spherical level sets that can be compared to the sphere B(e). The dependency of this scaling on æ disappears for E small (see Appendix A.. Equivalently, we can write the criterion deriv-EIy as
deriv-Ely (x) E L (aw.ddeeerreeo))eereeee.e. (a) max(0, Vmin Yx())))
(14)
with ((x,)) e Rd, 3² RN(a)) < a a d-dimensional ellipsoid. In connection with theoret-
ical decision under uncertainty Zilinskas & Calvin (2019), (eriv-Ely( is the expectation of a utility of
the function model (the GP trajectories) at æ. Here, the utility is defined as the progress of the stochastic model below the best observation knowing that the function model has a minimum at x i.e., it has null first order derivatives and positive curvatures. The key idea of deriv-EI is account only for minima of
7

---

Published in Transactions Machine Learning Research (04/2024)
the possible functions. On the contrary, El accounts for any value of the possible functions which below the best observation, which is less consistent with the goal of minimization. Because characterizes the behavior of the minima of the GP realizations, deriv-El can be seen criterion between El and informa- tion theoretic criteria based on the expected reduction in entropy of the optima Hernéndez--Loatt et. al. (2014).
By considering deriv-EIy rather than Ely as a new acquisition criterion in Algorithm II we expect to improve exploitation capabilities, without degrading its exploration capabilities too much. Like EIN; deriv-Ely needs only evaluations of the true function, y( y(x(N)) through Yv, dYn and 22Y It does not need derivatives of the true function, y. Only the GP is differentiated. However, this acquisition criterion can no longer be calculated simply, and in the general case it will require the use of sampling techniques for its evaluation, which may complicate its use. Nevertheless, if we choose small, if we neglect the off-diagonal terms of the Hessian (as it was already proposed in -eeeeenneze-ooooeee assuming a well-chosen conditional independence of its diagonal terms, we obtain the following relaxed acquisition criterion (see Appendix A for detailed derivation):
et al. (2014)) while
deriv-EIy (x)
LinelyMiny (æ) x cond-EIy( (x),
(15)
S i
d
TikelyMiny(x)) :- Ed x exp 2
x
7
(16)
cond-FIx(x) := ((2min - a) (=min) + ((2min))
(17)
where is a constant that does not depend on æ and sd, and where the following notations have been introduced to simplify the expressions:
@Yv(#) N (m,, D2YN ())YY)))
(18)
22
m
P1.
Pi,d
=0 N
m1
P1,1
81
(19)
Pa-1a
ma
Pd.1
Pd
3d
Zmin
ymin
m
PI 88;
73
m4
T;
v
(20)
8;
The precise choice of has thus no impact.
3.3 Comments on the proposed acquisition criterion
Analysis of the terms in deriv-EI
Comparing the criteria Ely (æ) and (eriv-Ely(æ), first notice the presence of the function (ik)lyMiny,r)), whose role is to concentrate the search of the new point to be evaluated around the points
& that are likely to lead to a zero gradient of (small values of in T S m)) while favouring the areas of positive second derivatives (high values of (fr.//I- - r for all i). The second function æ - cond-EIy (x) estimates the expected improvement assuming that the function has a minimum æ, and looks partic- ularly like the expression given by Eq. The more the sccond derivatives of YN will be positive in
8

---

Published in Transactions on Machine Learning Research (04/2024)
probability, which translates into large values of Ti, the more this similarity will be important because, in this case, the constant tends towards In addition, as the statistical properties of Yv (x), vYx(2) and (²) are known explicitly, is important notice that the evaluation cost of deriv-EIx (œ) is of the same order of magnitude as that of the classical Ely Importantly, there is no need to use sampling methods to estimate it.
In addition, if is chosen to be constant and C' is a stationary covariance kernel (which remains the most common configuration in BO), then eY (x) is statistically independent of (()) and (²Y(x)) for any x X. In particular, if we focus on the first iteration of the BO procedure (N 0), and put aside the constraint on the Hessian, it can be noted that for any E > 0 and any X,
E [1 Ro ( max(O, Umin Y(=))) Pc Elg(a)),
(21)
where P. (x) B(s)) is constant independent of as the statistical properties of (a do not depend (stationarity). In that case, Elo is very close to deriv-Elo (up to the influence of the second derivatives), and maximizing either of these criteria should give close results. Then, the more the process Y is conditioned by observations of y, the more Y, σY and oay are correlated, and the more chances there are for deriv-Ely (a) and EIy(x) to propose different points. After many observations, anticipated that the interesting areas from the EI point of view will have low gradients, so that the two criteria should again propose close new evaluation points. The a priori interest of the deriv-EIy (ar) criterion thus lies in intermediate values of N, in the exploration of the various local minima of y, and the search for the smallest zone of X likely to contain the global minimum of y.
At last, when the dimension d increases, one of the classical difficulties of BQ based on EIv is to favor exploration over exploitation, by placing a very large number of points on the edges of the domain, which effectively represent the majority of the volume of X when d is large Siivola et al (2018). This effect should be limited by substituting deriv-Ely for EIN, ie., by requiring that each partial derivative of YN be close to and that each main curvature be positive through the factor LikelyMin(z)), which becomes more restrictive as d increases.
A more exploratory deriv-EI
In return, by trying to quickly visit potential high-performance local minima, it is possible that the deriv-EIy criterion explores fewer regions of X than EIN, which could be penalizing for the minimiza- tion of functions with multiple local minima. If this were the case (this kind of phenomenon was not observed on the test cases studied in Section A several techniques could be proposed to rebalance the explorattooneeploitttiii ratio. For instance, the control of the exploration-exxlloittionn balance by chang- ing target values has been studied in Jones (2001) for the probability of improvement and likelihood crilerion. Such a shift in larget around Umin was included in the El criterion in Berk el all (2019): Lizolle (2008). Another way of reinforcing exploration with respect to exploitation consists in maximizing the expected improvement at a cortain power p > 1. Indeed, by taking p greater than 1, we further encour- age low-probability high improvements compared to more probable small improvements. This idea was pursued in Schonlau al. (1998) where expressions for the generalized EIy criterion with p > 2 can be found. The new El with derivatives can also benefit from elevating the improvement at given power. It becomes,
deriv-EI N = E Baaaai//@eEeaeeeeeeeeaeaaeeaeea
max(0, Ymii - (x()))
For (see Appendix A for more details) and using the same notations as in Section 3.2 such a criterion
can again be approximated under an analytical form close to the one of Eq. (5
deriv-Fl (2) (x) x LikelyMin(2z) x cond-FT(()(z))
(22)
(0)d-E[(2(x:) := ((1 .2
22 | ((zmiu) + (#min 2a))(2umin)
(23)
9

---

Published in Transactions on Machine Learning Research (04/2024)
Adaptation to noisy outputs
The criteria EIN and deriv-EIy are introduced in a context where the observations of Y are assumed noise-free. If it turns out that these outputs are in fact noisy, and if this noise can be modeled as a centered Gaussian vector with covariance matrix Σ (which can be assumed to be diagonal or not), only few adjustments are needed lo calculate these two criteria (and these adjustments are the same in both cases). First, in order to integrate the noisy nature of the observations in the conditioning formulas, the distribution of YN for any value of N is obtained by replacing C(X,X)- by (C(X, Σ)- in equations and (3 Then, as the observations are noisy, the notion of a current minimum value makes no longer sense, and we need to adapt the value of ymin in the proposed expressions accordingly. This value can, for example, be chosen as the value minimizing over X the predictive mean æ - E [Yx as it is done in the knowledge gradient approaches Frazier & Powell (2007). To limit the computational cost associated with the choice of Umin; it is also common practice to minimize the predictive mean only at the observed imput points. Once these two adjustments have been made, the criteria EIN and deriv-EIN can be applied to noisy observations.
4 Numerical experiments
In this Section, we first illustrate the way the proposed criterion works, and the differences it implics with the classical ET criterion. In particular, will be seen that iterates stemming from the maximization of deriv-EIy are more concentrated inside the scarch domaín than EIy iterates. Then, by considering functions with minima inside the scarch domain, show that deriv-EIx allows faster average convergence than EIy does. This is particularly visible with highly multimodal functions. In the experiments, deriv-Ely, is calculated through the approximation of Eq. 15
4.1 Analysis of the deriv-Ely criterion in dimension 1 and 2
Test functions and experimental protocol
We analyze the behavior of the deriv-Ely criterion through the study of an oscillating function in dimension d 1, noted y!, and of a modified Branin function in dimension d 2, noted y2D (sce Figure 2 for a graphical representation of these functions, and Appendix B for their defiititons.. In order to focus exclusively on the effects of the acquisition criterion, we fix the hyperparameters (length scales, variance, trend parameters) of the Gaussian predictor to their maximum likelihood estimate for large number of points. It has been observed that a good optimization of the acquisition criterion is a condition for BO to be efficient Le Riche & Picheny (2021). For this reason, the maximization of the acquisition criteria is performed by an exhaustive search on a fine grid, which is possible in such low dimensions. In higher dimension, a careful choice of the initial points of the acquisition function maximization is required Zhao al. (204)).
Visualizing the terms making the new acquisition criterion
We illustrate the roles of the LikelyMiny and cond-EIx functions (which are defined in Section 3.2). by evaluating y ID at No points and ya at N₂ 12 points randomly chosen in X. The evolutions of LikelyMin and cond-EIy associated to these evaluations are given in Figures D and b. As expected, the function Liny is large at the points the most likely to correspond to local minima, while the function cond-EIx highlights the areas the most likely to lead to GP trajectories that have a null gradient while having values lower than the current minimum. For these particular examples, the product of the two functions, which yields the deriv-EIy criterion, favors new points inside the input domain, when the EI, criterion encourages to evaluate y ID (resp. y²D) on an edge of X. We also notice that by concentrating the search at areas of low gradient for I or y22 we limit the significant values of deriv-EIn to sub-regions of x that are smaller than what Ely would give.
Performance of deriv Ely over one step
The performance of the deriv-EIN criterion now analyzed in terms of minimization of ID and y2D
10

---

Published in Transactions Machine Learning Research (04/2024)
(a) LikelyMinn (-) and cond-EIx (- -) for yD
(b) LikelyMing (-) and cond-El (- --- for y
19
(c) E and MIr (--) for y
(d) EIx and deriv-EIx (---) for yaD
Figure 2: The function y "D is shown in black thick solid lines in plots (a) and (c) where the grey areas correspond to 95% confidence intervals of the Gaussian predictor. Tdentically, the black solid lines in plots (b) and (d) are the contours of function y2D, In each plot, the global minimum is indicated by cyan triangle, while the green dots show the points where the function has been evaluated. Plots (a) and (b) show, for the two considered functions, the evolution of -- LikelyMin (x) in orange solid line, and of x ~ cond-Ely (x: in magenta dotted line. Plots (c) and (d) compare the evolution of - EIn (æ) (in blue solid line) to that æ deriv-Ely' (æ) (in red dotted line). For ease of reading, the functions Likely cond-EIn, EIN, and deriv-EI, are normalized in such way that their maximum value is fixed 1.
11

---

and each repetition of the experiment < i & 500, the function اار is evaluated at each point of 20) and Algorithm [ presented in Section 2 is run twice, taking as acquisition criterion deriv-El first, then the classical critcrion EI. At each iteration & > 1 of the algorithm, we note yk,4 current minima of I The empirical estimates of the median and the mean of these current minima shown Figure 4 The interest of the deriv-El acquisition criterion is again underlined by these results. Indeed, for all iterations k, the median and the mean of the current minima associated with the deriv- El criterion are lower than those of the current minima associated with the ET criterion. Again, the ad advantage of Eeriv-El over El takes place in the middle of the iterations k. Note that the median is well below the mean for the minimization of y 'P It comes from the fact that, for both EI and deriv-EI, some of the runs Published in Transactions on Machine Learning Research (04/2024)
We start with a single step. For each j E {1,2), and each k e 2d - 1, we generate 500 space-filling LHS made of & points in x Perrin & Cannamela (2017), which are written x()) J==1" For each 1 & i & 500, we
then construct Gaussian predictor of nل based on its evaluations at each point in x(6) RR, and we denote (). the points of X maximizing the criteria deriv-Elg and Elg, respectively. Let
deriivv and Tia
by kg. derivve be the smallest value of yID that we obtain:
EL and Vk.3 U,,
min ULe y (x),
deriiv-
min (z))
(24)
me x o
x o deriv-
By construction, the lower these values are, the better the acquisition criteria should be. In this prospect, for j E {1,2], Figure 3 compares the evolution of the 25%, 50% and 75% empirical quantiles of Dk. a()).
and Vk. de),vvei as a function of k. As announced in Section 3.3 the interest of the proposed criterion lies in the intermediate (about [5, 18] x d) values of For too low values, as the Gaussian predictor and its first-order derivatives are not very correlated, the criteria deriv-EIn and EIy are very close, and lead to similar results in terms of minimization of the objective function. For k large, the Gaussian predictor approaches the objective function with little uncertainty, and the criteria deriv-Ely and EIy are equally capable of identifying the global minimum. For intermediate values of k this phenomenon can be clearly seen in the evolution of the values of ydD (right plot). Because a one-dimensional space is rapidly explored, the advantage of deriv-FIy over EIN is less clear in the evolution of the values of yd (left plot). There, the two criteria give almost the same results, with deriv-Ely allowing only slight improvements.
Performance of deriv-Ely over many steps
In the above numerical experiments, one step was studied and the new evaluation points were independent of each other. Getting closer to a BO algorithm, we now quantify the effect of the acquisition criteria when defining a sequence of points where yاD is evaluated. To this end, for j E {1,2), we generate 500
(2) {x33 J For each j E (1,2)
500
new space-filling LHS in X composed of 3 points each, which are written
3,3
()),EI the obtained deriv-L and VE.s
have taken a significant number of iterations to identify the area of the global minimum.
4.2 Performance analysis in dimensions 2, 3 and 5
Test functions construction
The EI and deriv-El acquisition criteria are now compared on a larger set of test functions. To define this set of functions, we elaborate on the idea of using GPs/Hennig & Schuler (2012) which are by construction compatible with the working assumptions. We start by noting 2,(0) the Gaussian process defined on x x= [0, 1 such that for any x' E X and any 0 > 0
2 c x
E 7 (x)
000 (x')) 0, ( (k)
11 1=1
(25)
12

---

Published in Transactions Machine Learning Research (04/2024)
FI
deriv-EI
Quautiks mloine
Quautiles on the cureeat mitiim
Iteration &
Ieration d
(a) Minimization of yD
(b) Minimization of 2D
Figure 3: Influence of the acquisition criterion deriv-EIx and EIN when minimizing y1D and y2D For & b 2d 1, the lower and upper parts of the black rectangles correspond to the 25% and 75% quantiles of kk.
Ei while the black circles show the median value. Similarly, the lower and upper parts of the red
de).vviiVEE while the red triangles show the median rectangles correspond to the 25% and 75% quantiles of kk value.
deriv-EI
Mean msedian current minina
Mean and uordia of the curzeut minina
Iteration
Iseaaton &
(a) Minimization of y 1D
(b) Minimization y
Figure 4: Influence of the acquisition criterion deriv-EIy and EIN when minimizing nd and y For
k 1, the filled black circles (resp. the filled red triangles) represent the empirical median of { ()),1 500 (resp. of derivv- 500
yk Jin
J. Jim1 and the empty black circles (resp. empty red triangles) indicate the empirical
means.
13

---

Published in Transactions on Machine Learning Research (04/2024)
where k is the Matérn-5/2 covariance function of Equation (9), which is such that 2/00) is twice differentiable in the mean-square sense. Notice the normalization of the length scales in v d/2, allowing to define Gaussian processes in any dimension d with close dependence structures. This normalization can also be understood by seeing that distances (belween the two farthest points, or the expected distance of two points randomly drawn in X) grow in vd, therefore the length scales have to grow in vd. We consider as test function class the set F&d) of realizations of Zi² that admits global minimum strictly inside X (i.e., at a point of zero partial derivatives). The following numerical tests then focus on two particular values of 0: 0 = 0.2 will characterize strongly oscillating functions admitting a large number of local minima, while 9 = 0.5 will refer to more regular functions presenting a smaller number of local minima. For 0 E {0.2, 0.5)
and de E (2, 5), we generate 100 functions from F(() in a random and independent way. These functions are noted| 191 [d) 100 1=1 (see Appendix C for a detailed description of their construction). We finally subtract
from each function its minimum value so that
min
DEX 81 (z) = O,
(26)
and we proceed to the same shifting on the Y process. Figure b shows four examples of such functions belonging to Fl2 and F in the case d = 2.
Experimental protocol
The global minimum of these functions is then searched twice with Algorithm M by, first, taking deriv- EI and, then, EI as the acquisition criterion. The total number of calls to the objective function of each optimization run is equal to budget = 100. The two types of searches are initialized with the evaluation of Vi.8 at the same space-filiing LHS of dimension No 3 (a different design is generated for each function minimization). The size of the design is small and does not depend on d. As observed in Le Riche & Picheny (2021); Hutter et al. (2013), small random designs at the beginning of BO searches are more efficient. Moreover, the effect of the acquisition criterion is more visible for small initial random designs. The growth of the length scales in vd (Equation (25) guarantees that the correlation between the Nu points is the same, independently of d. In order to investigate the influence of the acquisition critcrion only on the optimization but not on the learning of the GP, the properties of the Gaussian process Y used to guide the search are chosen equal to those of 2 The maximization the acquisition criteria performed in two steps: each acquisition criterion is first evaluated in 10 points randomly chosen in X, and 10 Nelder-Meat algorithms starting from the 10 most promising points among the random points are then launched in parallel to identify the new point at which to evaluate the objective function.
(d)
Two quantities of interest are then extracted from these Bayesian optimizations. For each 1 & k & budget, each de (2,,,5), and each f E (0.2, 0.53, we first note Ve d(dl)deriv-E1(k)) (resp. 8% ((()) the empirical mean of the current minimum (mean best-so-far performance) obtained at the 4th iteration on all the tested functions when taking deriv-EI (resp. EI) as the acquisition criterion. Second, we define [(), 8) (resp. (0),EI s)), the mean time-to-targee best-so-far observation to be lower than a threshold s > 0 when using deriv-EI (resp. EI). Note that for both quantities of interest, the average is done on the different fest-iunc,ons,, which have the same kind of variations and the same minimum equal to O, which makes them comparable although potentially very different.
that is the average number of iterations necessary for the
Optimization results
The evolution of these quantities of interest are shown in Figure 0 for 0 = 0.2 and Figure 7 for 0 = 0.5. In all of these figures, a substantial gain is brought by the deriv-El criterion with respect to the EI criterion. The gain is visible both in terms of the mean best-so-far far objective function (plots (a) to (c)) and the mean time-to-target (plots (d) to (f)). We notice, as we had hoped in Section 33 that the observed improvements
14

---

Published in Transactions Machine Learning Research (04/2024)
19
(a) V1.022
V2.0.2
(2)
(c) V3.0.2
(d) 14.0.2
31
(e) 91.0.5
(4) (f)
12.0.5
(2) (g) V3,0.5
(4) (h) 14005
Figure 5: Representation of 4 particular elements of F(2) and Fo.s the set of test functions to be minimized. In each function instance, the global minimum location indicated with a cyan triangle.
0.2
brought by the deriv-El criterion are greater as the dimension of the input space increases. As expected, we also observe that choosing deriv-EI rather than EI is more interest for more multimodal functions, iee, when the length scale is small. Indeed, is these configurations with a large number of local minima that adding information about null first order derivatives and positive definite Hessian matricos useful.
Remark For all the test functions studied in this section, the condition numbers of the covariance matrices of the observation points were all between 10 and 2 x 105.
Extensions and conclusions
5.1 Management of minima on bounds
The article has assumndd until now that the minimumn is inside the search space If this not the case, orienting the search towards areas with zero gradient can actually be counterproduetive, as the global minimum will typically be associated with nonzero gradient. Denoting by dx the boundary of X, first possibility to cireumvent this problem is to penalize the objective function that optima on the boundary are pushed inside the domain but arbitrarily close to the boundary, and therefore are associated to a null gradient and positive definite Hessian. This is the idea of the barrier functions of the interior point methods Wright & Nocodal (2006). With barrier functions, the objective x - y(x) replaced by æ + (()) + Ae(x), where À is a positive constant and is a continuously twice-differentiaall positive function such that c(x:) would be close zero when is far from the boundaries of X, and would take potentially infinite values when 2%. For instance, % [0, 1J, the function c can be chosen as:
<xx) min(x, z) or c(x) = log(min(z. 1 1-x)) x))
0
(27)
The larger À is, the further from 2% the global minimum of y + Ac is, whether the global minimum on O or not. And by making À progressively tend towards 0, we make this global minimum, which
15

---

Published in Transactions Machine Learning Research (04/2024)
E1 deriv-El
Mean of the current minimann
Mean of the miniman
Monn of the current miniman
2.6
3
8
Iteration
Iseration
Iteration i (c) d 5, 0.2
(a) = 2,0 0.2
(b) d 3, 0.2
Menn mumber of function cvaluaionn
Moan number of finction csaluations
8
Mean function
8
88
0.005
0.020
0.100
0.500
0.005
0.020
0.100
0.300
2.000
0.006
0.020
0.100
0.500
2,000
Threshold
Threshold
Threshold
(d) = 2,0 0.2
(e) d 3,0
(f) d 5,0 = 0.2
(k) (in black solid line -) and k - J ((d),deriv-B((A (k) (in red dotted line ---) for strongly multimodal functions (0 = 0.2) and de (2, Plots (d), (e), and (l) give the mean time-to-targed RL).EL (0, s) (in black solid line) and
Figure 6: Plots (a), (b), and (c) show the mean best performance, k - EI
8 gi)d s) (in red dotted line).
16

---

Published in Transactions Machine Learning Research (04/2024)
deriv-EI
Mean of the curront miluhomm
Mesan the minimon
Moan of the current minimum
08
6.8
25
Iteration
Iveration
Iteration A (c) 5.6
(a) 2, 0.5
(b) d 3, 0.5
Meaa aumbler of funuttio ecalumiions
Menin mumber Junccion evalustious
Moan number of function cviluctions
9
8
0.005
0,020
0.050
0.200
0.005
0.080
0.100
0.500
0,006
0.000
0.100
0.500
Threshokd
Threshold
Threshold
(d) - 2,0 0.5
(c) d 3,0 0.5
(f) d 5, 0.5
Figure 7: Plots (a), (b), and (c) compare the mean best-so-far performances for the two acquisition criteria, à De (ld) (k) (in black solid line -) and - Dg der. (k) (in red dotted line ---), for moderatly multimodal functions (0 0.5) and and d (2,3, 5). Plots (d), (e), and (f) compare the mean ame-to-target, $ - (i, s) (in black solid line) and - deriv- (8, s) (in red dotted line).
17

---

Published in Transactions on Machine Learning Research (04/2024)
will be well associated to a zero gradient of y + dc, tend to the global minimum of y. Such an approach is well-studied for convex optimization problems, with bounds linking the choice of À to the degradation in the optimal value of the objective function Wright & Nocedal (2006). In general however, the effect of the A decay law is difficult to understand. And the penalty adds a steep function increase at the edge of X that GPR-based melamodels will have difficulty to learn.
Alternatively, we propose to evaluate a priori the likelihood, noted εx. that the minimum of y lies on the boundary of X.
The EN likelihood can be evaluated through a three-step procedure. Starting from the GP-based surrogate model of y, noted YN, the first step would be to look for positions that may correspond to local minima of y, by rumning in parallel M >> 1 regularized Newton descent algorithms on the trajectories of YN- These minimizations would be in that case initialized at randomly chosen points T e X, 1 & m & M, and we would denote by xx the obtained minimum when starting from ag Then, we generate Q > 1 random samples of the Gaussian random vector
(xxx)))), which we denote by
(Y (1)) .... (x 1E9EQ.
(28)
The indicator iN is finally estimated by counting how often the minima of the draws are on the bounds,
1 Q iN = 0 -
x+ m E arg min YN (a%).
(29)
Q q=1
# E0X
IsmEM
Depending on the value of εN, the method described in the rest of the article can be complemented in one of the two following fashions. If EN is too large, the traditional Ely acquisition criterion should be used instcad of deriv-EIy. Alternatively, yn can be calculated specifically for each bound and if it is likely that some specific bounds are hit, then the corresponding variables can be set to these bounds, the BO iteration being carried out with the deriv-EIy criterion in the lower dimensional space.
Nonetheless, the objective of this work was to come up with an acquisition criterion applicable when the minimum of y is not on the boundary of X. We leave the continuation of the above analysis, based on the likelihood to have the optimum at a bound, as a perspective to this work.
5.2 Summaay and further perspectives
Tn the context of Bayesian optimization, this work proposes a novel acquisition criterion allowing to integrate as additional a priorí the fact that interior minima are associated to zero first order derivatives and positive definite Hessians. With this addition, a classical acquisition criterion such as the expected improvement takes on feature of information theoretic criteria by characterizing the distribution of potential optima when the plain expected improvement accounts for all improving values of the function model. The new expected improvement with derivatives, called deriv-EI, does not need the derivatives of the true function. A computationally efficient approximation to deriv-EI is proposed in the article.
has been observed through several test cases that the new criterion allows significant gains in terms of function minimization at intermediate budgets function evaluations. This benefit is larger when the function to minimize presents several local minima or the dimension is high, since in these cases the classical expected improvement is too exploratory in particular in areas near the bounds Siivola et al. (0118).
All the consequences of the proposed acquisition criterion could not be investigated in this paper. In order to simplify the interpretation of the results, all the comparisons between the classical EI and the deriv-E) El criteria have been carried out in ideal configurations in the sense that the test functions are realizations of the Gaussian process guiding the minimization. The hyperparameters of the GP characterizing its
18

---

Published in Transactions on Machine Learning Research (04/2024)
statistical properties are always known by construction of the test functions. Therefore, it will interesting to study the sensitivity of Bayesian optimization with deriv-EI the iterative estimation of the GP hyperparameters, as it happens in practice. In the same manner, only problems in moderate dimensions are implemented (d < 5), as it seems important to restrict ourselves to cases for which the maximization of the El and deriv-El criteria can be sulficiently well solved. During the analysis of configurations in higher dimensions (d 3 10), the maximization of these criteria becomes a problem in itself and the performances of the EI and deriv-EI criteria turned out to be too dependent on our ability to correctly maximize them. Working at the definition of efficient methods to maximize the deriv-EI criterion would therefore be an appropriate continuation to this work.
Finally, this work has focused exclusively on the El acquisition criterion because it is a standard in BO, but other acquisition criteria should also benefit from the predictor's derivatives. For example, the El criterion of Equation 122 can be adapted to the related probability of improvement Kushner (1964): instead of we could maximize
maximizing - ((Y < Umin) = E [Iyw(e)spmmn)
x E 1 (t)ma x www.eeeereteeeeeeeeeeeeeeeeeee
(30)
In the same manner, the well-known upper confidence bound acquisition criterion Srinivas el al. (2009)
could be adapted by adding penalty term on the derivatives, which would make it possible to explicitly
play on the exploitation vs. exploration tradeoff in the same way as the standard deviation. The new point to be evaluated could for instance be sought as a solution to the following problem:
(N+1) arg argmin (2: (1) A2 [[)Yy
(31)
xex
with A and A2 two positive constants.
Acknowledgments
This action benefited from the support of the Chair Stress Test, Risk Management and Finan- cial Steering, led by the French Ecole polytechnique and its foundation and sponsored by BNP Paribas. This research also benefited from the consortium Applied Mathematics CIROQUO (https://doi. org/10. argg/1..281//eeee 5281/zenodo 6581217) gathering partners in technological and academia in the development of advanced methods for Computer Experiments.
References
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper-paramete parameter opti- mization. Advances in neural information processing systems, 24, 2011.
Julian Berk, Vu Nguyen, Sunil Gupta, Santu Rana, and Svetha Venkatesh. Exploration enhanced expected improvement for Bayesian optimization. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10 14, 2018, Proceedings, Part II 18, pp. 621-637. Springer, 2019.
Mohamed Amine Bouhlel, Nathalie Bartoli, Abdelkader Otsmane, and Joseph Morlier. Improving kriging surrogates of high-dimensional design models by partial least squares dimension reduction. Structural and Multidisciplinary Oplimization, 53:935-9522 -52, 2016.
G. Damblin, M. Couplet, and B. Iooss. Numerical studies of space filling designs: optimization of latin hypercube samples and subprojection properties. Journal of Simulation, 7:276 289,, 2013.
Youssef Diouane, Victor Picheny, Rodolphe Le Riche, and Alexandre Scotto Di Perrotolo. TREGO: trust-region framework for efficient global optimization. Journal of Global Optimization, pp. 1-23, 2022.
K.T. Fang, R. Lú, and A. Sudjianto. Design and modeling for computer experiments. Chapman Hall, Computer Science and Data Analysis Series, London, 2006.
19

---

Published in Transactions on Machine Learning Research (04/2024)
Alexander IJ Forrester, András Sóbester, and Andy J Keane. Multi-fidelity optimization via surrogate modelling, Proceedings of the royal society a: mathematical, physical and engineering sciences, 463 (2088):3251 3269, 2007.
Peter Frazier and Warren Powell. The knowledge gradient policy for offlin learning with independent normal rewards, In 2007 IEEE International Symposium on Approzimate Dynamic Programming and Reinforcement Learning, pp. 143 150. IEEE, 2007.
Peter Frazier. A tutorial on Bayesian optimization. aix preprint arXiv: 1800002811,1, 2018.
Roman Garnett. Bagesian optimization. Cambridge University Press, 2023.
David Gaudrie, Rodolphe Le Riche, Victor Picheny, Benoit Enaux, and Vincent Herbert. Modeling and optimization with Gaussian processes in reduced eigenbases. Structural and Multadisciplinary Oplimiza- tion, 61:2343 22018,23612222 2020.
Robert B Gramacy. Surrogates: Gaussion process modeling, design, and optimization for the applied sciences. Chapman and Hall/CRC, 2020.
Philipp Hennig and Christian Schuler. Entropy search for information-efficient global optimization. Journal of Machine Learning Research, 13(,), 2012.
José Miguel Hernández-Looann Matthew W Hoffman, and Zoubin Ghahramani. Predictive entropy search for efficient global optimization of black-box functions. Advances in neural information processing sys- lems, 27, 2014.
Frank Hutter, Holger Hoos, and Kevin Leyton-Brown. An evaluation of sequential model-based opti- mization for expensive blackbox functions. In Proceedings of the 15th annual conference companion on Genetic and evolutionary computation, pp. 1209 1216, 2013.
Donald Jones, Matthias Schonlau, and William Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13:455-492, 12 1998. doi:
Donald R Jones, A taxonomy of global optimization methods based on response surfaces. Journal of Global Oplimization, 21:345 383, 2001.
Samuel Kim, Peter Y Lu, Charlotte Loh, Jamic Smith, Jasper Snock, and Marin Soljacic. Deep learning for bayesian optimization of scientific problems with high-dimensional structure. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL
Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, and Frank Hutter. Fast bayesian optimization of machine learning hyperparameters on large datasets. In Artificial intelligence and statistics, pp. 528 536. PMLR, 2017.
Harold J Kushner. A versatile stochastic model of function of unknown and time varying form. Journal of Mathemalical Analysis and Applications, 5(1):150- 167, 1962.
Harold J Kushmer. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. J. Basic Engineering, 86:97 06:97-106, 1964.
Rémi Lam, Matthias Poloczek, Peter Frazier, and Karen E Willcox. Advances in Bayesian optimization with applications in acrospace engineering. In 2018 ATAA Non-Deterministic Approuches Conference,
pp. 1656, 2018.
Lue Laurent, Rodolphe Le Riche, Bruno Soulier, and Pierre-Alain Boucard. An overview of gradient- enhanced metamodels with applications. Archives of Computational Methods in Engineering, 26(1): 61-106, 2019.
Rodolphe Le Riche and Victor Picheny. Revisiting Bayesian optimization the light of the COCO benchmark. Structural and Multidisciplinary Optimization, 64(5):3063 3087, 2021.
20

---

Published in Transactions on Machine Learning Research (04/2024)
Daniel James Lizotte. Practical Bayesion optimization. PhD thesis, University of Alberta, 2008.
Jonas Močkus. On Bayesian methods of search for extremum. Automatics and Computers, 3:53 62, 1972.
Jonas Močkus. Bayesian approach to global optimization: theory and applications. Springer Science & Business Media, 2012.
John Nelder and Roger Mead. simplex method for function minimization. The computer journal, (4):308 313, 1965.
G. Perrin and C. Cannamela. A repulsion-basee based method for the definition and the enrichment of optimized space filling designs in constrained input spaces. Journal de la Sociélé Prançuise de Stalistique, 158(1): 37-67, 2017.
Victor Picheny, Pierre Casadebaig, Ronan Trépos, Robert Faivre, David Da Silva, Patrick Vincourt, and Evelyne Costes. Using numerical plant models and phenotypic correlation space to design achievable idcotypes. Plant, Cell & Environment, 40(9):1926-1999, 2017.
C. E. Rasmussen. Gaussian processes to speed up hybrid Monte Carlo for expensive Bayesian integrals. Bayesian Statistics, 7:651-659, 2003.
V. R. Saltenis. One method of multiextremum optimization. Automatika i Vychislitel'naya Tekhnika (Automatic Control and Computer Sciences), 3(3):33-38, 1971.
T. J. Santner, B.J. Williams, and W.I. Notz. The design and analysis of computer experiments. Springer, New York, 2003.
Matthias Schonlau. Computer experiments and global optimization. PhD thesis, University of Waterloo, 1997.
Matthias Schonlau, William J Welch, and Donald R Jones. Global versus local search in constrained optimization of computer models. Lecture notes-monograph series, pp. 11 11-25, 1998.
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the JEFF, 104(1):148--17 175, 2015.
Eero Siivola, Aki Vehtari, Jarno Vanhatalo, Javier González, and Michael Riis Andersen. Correcting boundary wer-exploration deficiencies in Bayesian optimization with virtual derivative sign observations. In 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1-6. IEEE, 2018
J. Snock, H. Larochelle, and R. P. Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in neural information processing systems, pp. 2951 2951-2959, 2012.
András Sobester, Alexander Forrester, and Andy Keane. Engineering design via surrogate modelling: practical guide. John Wiley & Sons, 2008.
Niranjan Srinivas, Andreas Kranse, Sham M Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint ar axi::0912.9900, 2009.
Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Malthias Seeger. Gaussian process op- timization in the bandit setting: No regret and experimental design. In JCML, 2010. URL
pdf
Ryan Turner, David Eriksson, Michael McCourt, Juha Kiili, Eero Laaksonen, Zhen Xu, and Isabelle Guyon. Bayesian optimization superior to random search for machine learning hyperparameter tuning: Analysis of the black- box optimization challenge 2020. In NewrIPS 2020 Competition and Demonstration Track, pp. 3-26. PMLRR, 2021.
Julien Villemonteix, Emmanuel Vazquez, and Eric Walter. An informational approach to the global optimization of expensive-to-evvaaat functions. Journal of Global Optimization, 44:509-534, 2009.
21

---

Published in Transactions Machine Learning Research (04/2024)
Yijia Wang, Matthias Poloczek, and Daniel R. Jiang. Dynamic subgoal-based exploration via bayesian optimization. Transactions on Machine Learning Reseorch, 2023. ISSN 2835-8856. URL https://openrevier wtt.://rwwww..toooooooooooooaaaa
C.K. Williams and C.E. Rasmussen. Gaussian processes for machine learning, volume 2 (3). MIT Press, Boston, 2006.
Stephen J Wright and Jorge Nocedal. Numerical optimization. Springer, 2006.
Jía Wu, Xiu-Yun Chen, Hao Zhang, Li-Dong Xiong, Hang Lei, and Si-Hao Deng. Hyperparameter opti- mization for machine learning models based Bayesian optimization. Journal of Electronic Science and Technology, 17(1):26-40, 2019. ISSN 1674-862X. doi: http:///oooor///01119////S11111
80904120. URL httpss//ww..cciencciieetttt http:///eeeeetetcecoocc/c//////e htt.a//cwcc.cc.ooo......oooo/ooa
Jian Wu, Matthias Poloczek, Andrew G Wilson, and Peter Frazier. Bayesian optimization with gradicnts. Tn I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL
Jiayu Zhao, Renyu Yang, Shenghao Qiu, and Zheng Wang, Unleashing the potential of acquisition functions in high-dimensional bayesian oplimization. Transactions on Machine Learning Research, 2024. ISSN
2835-8856. URL
Antanas Zilinskas and James Calvin. Bi-objective decision making in global optimization based on statis- tical models. Journal of Global Optimization, 74:599-600 6099,000,0000 2019.
A Approximation of the El with derivatives information
We start by recalling the notations used in Equations covariance matrices of random vectors N and (Yx(x)) Y (x)
19 and
For each x E X, the means and NY (x) = 0 are written
@Yv(œ) N (m,
s²
P1.1
Pi,
=0 ~ N
mi
P1,1
81
Pd-1.d
Mad
Pa.
Pd.d-1
Sa
where the notation DaYN= =
refers to the diagonal terms of the matrix O²YN-
Using the notations and the general expressions introduced in Section B.T
the expressions m and s
come from the conditioning of the (d + 1)-dimensional Gaussian vector (xx, (æ)), whose mean (xx)()), and whose covariance matrix is
CN x) dC (a. [
(2x(2,x))
(x,
Applying the conditioning formula yields,
m = (\v 0] (x( (c) T [ (u)0)))) s2 = VarlY (x) 0) = Cn (z. x) - 20x (x. x) x)
The expressions for the mean and covariance matrix of the Gaussian vector (x D2Y DX = 0, i.c. the symbols m, 3, and Pi,j Equation 19 are obtained following the same conditioning principle, but
22

---

Published in Transactions Machine Learning Research (04/2024)
applied to the Gaussian vector (Yv( (at,, (x)())), whose mean (ax(2)) Da JAN (ow((w)).. (x))eup @x)(())) and (z))
whose covariance matrix is made of properly ordered terms such as Cov
2 das
aYv day (x:)
_CCx (x, x)
Cov Y dz
(x);
Y (z))
CCc
(z, x), .....
dzy
The conditioning can be applied one more time observation of (x)), leading
0) account for an
(D (DYYN(E))(YYN(a))) = (x()) Niñ PIL(y pillymm)/8 -pie/
(32)
For p E (1,2), let us first assume that the off-diagonal terms of 02 can be neglected. In that case, ensuring that YY is in MM (d) comes down to ensuring that its diagonal terms are positive, i.c. ensuring that D2Y is [0, + [o and we can write (using the former notations):
deriv-FIx = I yy= (SES(ee)) - Svent (ymin - yy) y. 8)
CUmin
(33)
Vinio
(eE((n))) +0f0 (Umin v)" (ee dyddddj
As it is necessary for matrix to have positive terms on its diagonal to be positive definite, this approxi- mation an overestimation of the number of trajectories that are actually positive definite. In addition, if e, the size of the ellipsoid centered at to which (x) belongs, is sulficiently small, it is possible to approximate fay (i) by (ay for any ÿ in E(x, E), which leads to:
deriv-FIy(x))
x Voll(((z;)) (0) fyy S
Vmiia
(34)
+00 (ymin - 8)
(ly.-ooyyyyydyd dyddj
where VoI(E(xr,)) E)) the volume of E(x,e).
We further assume that for any e R, the components of D2YN (x) conditioned by the event (ōYw(x) O, YN (x: = y) are statistically independent. In other words, trajectory which passes through (x, y) and which is flat assumed to have independent curvatures. In this case, the density of (?) 0, NN (x) = y) is a product of univariate densities. This leads to,
Mmiin
(ymin - 4) - (ew. (l
(ymin - -//haaaatseee-////x//x/xxxxxxxx0 (r)ddd
y=
+000
Mmin
(ymin
4
(y (y-m)²
exp (-1.
(0: (ii-(mi) (m, + P1: (v m)
m)
djj
dy
1=-
(2x²)
exp
232
7.=0
23 / (s2
3i
P11
(35)
we now perform the following variable changes: == (y - m)/s, 2min = (ymin m)/s, r4 == Pie/ (1t((38)) and R; m 8,, it comes:
23

---

Published in Transactions Machine Learning Research (04/2024)
Ymnn
(3/mn 1)
y - m)²
d
(i/
(m: + P1:(y) m) //
di
dy
(2T)) 1
exp
2x²
exp
28 pir\ s2 87
8
0:
8
88,
(2min 2)
2²
(2) r22)2
(2 2+1
exp
2
exp
2(1 - (?))
V1 1,
dz
x
(2min =)
22
d
Fs + r;2
exp
d2.
v. 2T
(36)
Recalling that + and 6 are respectively the CDF and the PDF of the standard Gaussian variables, the former expression can be further simplified by introducing the following first order Taylor expansion of the function 4
122 1s +
7i
T2z
7
2 ①
(37)
and by truncating to the first polynomial orders, so that:
Smin
(2min
2)
d
T: +
7i2
cxp
dz
2T
2
i-1
7
Emin
(zmin
2)
exp
d
$
7i
Ti2
7
dz
(38)
5
2
i=1
V
r
r
r
Emmin
(amin
2)P(1 + za)
exp
dz
2T
2
i=1
where
V--?
(39)
1-1
V1-r?
It finally comes
deriv-Ely(x) ~ LikelyMin(z)) x cond-EI EI())()).
(40)
with
#
LikelyMin(x)= Vol(E(z,=)))errvla))
00)
i-1
(41)
m T
S
#
ved exp
2
11
i-1
where is constant independent of x and E, and with
cond-ET(P((a)) : ((zmin a))(mm)))) ((Emin)) if p =
(42)
((1 + 22 - 2a ) (2 + (Zmin 2a) if = 2.
24

---

Published in Transactions Machine Learning Research (04/2024)
The expression when the constraint on the second order derivatives is not considered can be recovered by making ñt tend to infinity, In this case, tends to 0 and IT + tends to 1.
In summary, the approximation provided by Eq. 140 is based on the following four assumptions:
the off-diagonal terms of g²YN can be neglected,
the value of is sufficiently small for (a) (i) to be approximated by (=) for any y in Ex,
the components of D2Yx(=)) conditioned by the event (oYx(x)) 0, YN(x)) y) are statistically independent,
the first order Taylor expansion of the function provided in Eq. KY holds.
Numerical assessment of the proposed approximation
In order mumerically evaluate the quality of the approximation of deriv-EIN, implement second Monte-Carlo based approximation:
(er ~ d((x) = Vol(E(z, (0)
1 M
Σ
1y Edmmn 18 mmMM+ (d) (Buni Ym),
(43)
m
where { Ymm Y m m-1
MM gathers M independent realizations of ( (z)))oYY 0. The term
()) is explicitly calculatod the same way in both approximations.
Focusing the test functions listed in Section 4.2 in dimensions d 3,, 5), and limiting ourselves to N E {2d, 5d, 10d), we uniformly sample 1000 points II...., E1000 in [0, 2 We compute the coefficient M
of determination R² between the values of [LikelyMinaz;) x cond-Er(e) (z;) j-1
(the proposed and fast
approximation of deriv-Ely) and the values of {á(z;)) M (the time consuming Monte-Carde approxima-
tion of deriv-Ely). The mean value and standard deviation of these coefficients of determination obtained when repeating 10 times the whole process are finally summarized in Table [ In each case, observe high coefficients of determination, which justifies the use of the fast approximation.
B Expression of the analytical test functions
The functions nD and 2D which were considered in Section 4.1 have the following expressions (notice the offset made such that the minimum of the functions is 0):
[0,1]
R
30
ID
x
cos(6TT + 0.4) + (x 10--00 0.5)2
1D (x) yo (x))
min
yo a8 (2). xe [0. 1].
=E(0,1]
[0, 1
R
2D yu
(x1, (2)))2 - 10 + + (1522-
3(152, (4
5(152, -5) - 6) + 10 cos 15x, 5) (1 8
25

---

Published in Transactions Machine Learning Research (04/2024)
d
0
N
mean of R²
standard deviation of R
2
0.2
4
0.94
0.04
2
0.5
4
0.96
0.03
0.2
10
0.94
0.02
0.5
10
0.95
0.02
0.2
20
0.95
0.02
0.5
20
0.98
0.02
0.2
0.96
0.02
0.5
0.96
0.06
0.2
15
0.95
0.01
0.5
15
0.98
0.02
0.2
30
0.96
0.02
0.5
30
0.98
0.01
0.2
10
0.93
0.04
0.5
10
0.97
0.03
0.2
25
0.92
0.02
0.5
25
0.96
0.03
5
0.2
50
0.94
0.01
0.5
50
0.95
0.06
Table 1: Coefficients of determination R² Monte-Carlo approximation.
between the proposed, fast, approximation of deriv-EIy and
2x) = 10 (x) min 0=
y0 (z), E [0.
C Generation of test functions as realizations of a Gaussian process
We describe here how deterministic functions are defined as particular realizations of a centered Gaussian process indexed by e X, where a compact subset of Rd and the covariance function of denoted by C. The process consists of the following steps.
1. We start by generating a design of experiments X' := {æ1,... .,N) of large size N covering as much as possible X.
2. We then project taining the values of Z realized :
this design of experiments, and we note (Zi,..., the vector con-
3. By Gaussian conditioning, the function a - r(x) can thus be seen as the continnous extension realization of Z whose projection X' is equal to z, where for all x X,
r(x)= (
((x
C(#1.21)
((11
C(x,
C(æN,
C(NNNNN)
It is clear that this lype of construction strongly depends on the dimension of X. Indeed, the larger the size of X, the closer the constructed function will look like a particular realization of Z. In addition, the larger the input space dimension, d, the more points will be needed X' for the contimnous extension to be relevant. For the examples treated in Section 4.2 is defined as the concatenation of a two-level factorial design (in order to cover all the vertices of X [0, 1]d) and space filling LHS design of size 100 d.
26


Reviews:
[{"review_id": "BOeIffhR8V", "reviewer": "Reviewer_Hpa1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, this is a good and solid paper I think, if some of the concerns are addressed.\n\nStrength: \n1. The idea is very simple yet very general and practically useful.\n2. The paper is overall very well written and the idea is clear.\n3. It talks about some limitations too (the case the minima are on the edge etc.)\n\nWeakness:\n1. The major weakness is the lack of detailed step-by-step explanations on the derivation of approximation (especially Appendix A)\n2. Why focusing on certain kernels, EI etc. are not very well motivated\n\nMinor weakness:\n1. Some typos\n2. Clarity of figures", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes the use of derivative & Hessian estimate of the target function in its acquisition criterion for BO.  It provides an approximation of such criterion, and show that it indeed helps finding better estimate quicker than the one without those information on toy examples.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please add more detailed step-by-step explanations on Appendix A.  Especially, although I might be missing something here, please define $m$, $\\dot{x}$, etc. etc. and please elaborate the approximate equality on (34) and (35) more so that I can easily verify the correctness.\n\n2. Please motivate the focus on the class of kernels used in this work and on EI; also please confirm that the experimental results are not very specific but that it should be the case this advantage is seen in wider domains of problems.\n\n3. Figure 1 is a bit hard to parse; please provide a bit more detailed explanation \n\nMinor points:\n1. In abstract, it says \"dimension 2, 3, and 5\" and it is a bit confusing why these specific numbers are important; otherwise, should say \"low dimension problems\" or so.\n2. a d-dimensional real space -> the?\n3. 4th row introduction:  the parenthesis seems not to be closed\n4. What do you mean by \"BO is difficult to beat\"?  it sounds a bit strange\n5. Please confirm that you use $C^2$ as twice differentiable function class, not twice continuously differentiable.\n6. What is \"MatÃľrn kernels\"?  Matern?\n7. in page 6  \"A\" -> \"Appendix A\"\n8. in 12 what do you mean by \"These functions are call\"?\n9. in 12   \"Section ??\" please fix this\n10. in page 13  \"will typicallybe\" -> \"will typically be\"\nfix other minor typos as well.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nlIlMOYYZ4", "reviewer": "Reviewer_QSQ4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The paper is well organized and written. As someone being familiar with the topic of BO but being rather an \"end-user\" of the methods, I had no issues following the paper.\n* I enjoyed the open discussion about the assumptions and limitations of the proposed method as well as the emphasis on it realizing a different tradeoff between exploration and exploitation than EI.\n* From a technical side, I enjoyed the use of the joint distribution over function values and first-and second-order derivatives.\n\nWeaknesses:\n* The proposed approach's motivation mostly stems from intuition, with no formal or statistical reason for why incorporating derivatives of GPs should improve performance (I'll detail that below).\n* The paper also lacks additional evaluations on other functions (detailed below) and an extended discussion of how and when the derived approximation to the deriv-EI objective fails.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a variant of the well-known expected improvement (EI) acquisition function for (Gaussian-process-based) Bayesian Optimization. The proposed acquisition function restricts the original EI acquisition function only to consider function values of the GP posterior near local minima, i.e., having an almost vanishing gradient and a positive-definite Hessian matrix. The authors then introduce an approximation of said objective that avoids the necessity of Monte-Carlo approximations for evaluating the acquisition function. Subsequently, they compare the resulting BO method to the EI baseline, showing accelerated minimization of the chosen evaluation functions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Minor: I would welcome legends in all Figures such that a reader does not need to search for the meaning of a line in the text caption.\n* Minor: There is a broken reference on Page 12\n* Minor: The statement that deriv-EI leads to lower, better values of $y^{1D}$ in Figure 2 for intermediate values of k seems overstated given the plot in Figure 2. Also why is Figure 2(a) in log-space compared to the others?\n* Minor: Could the authors clarify why they normalize the gradient with the square root of the covariance in Eq (12)?\n\n* The paper would benefit from a more extended discussion on why only looking at the minima of the \"imagined\" functions should improve optimization behavior. Looking at Figure 1(a), the number/location of the GP minima does not need to correlate with the minima of the true function and consequently, focusing only on the local minima of the GP may improve or degrade performance.\n* I would like to see more evaluations of the approximation of the deriv-EI objective. Table 1 shows that the objective works reasonably well for a low number of function samples but does not show how it evolves with an increasing number of samples. Given that, e.g., Figure 2(d) shows high levels of improvement of deriv-EI over EI for around N=13 data points on the corresponding two-dimensional function, it would be interesting to see how good the approximation is for this number of data points. I recommend that the authors add a hindsight analysis of the conducted experiments by comparing the approximated deriv-EI values with MC approximations for the generated optimization traces, thereby showing the approximation quality deriv-EI for the conducted optimizations. Additionally, it would be insightful to have an \"adversarial scenario\", in which the approximation clearly fails. I imagine that this happens with strong off-diagonal hessian terms, given that these are ignored in the approximation of deriv-EI.\n* Finally, I believe that adding the following experiments to Section 5 would strengthen the overall paper:\n    * An evaluation on functions for which the maxima are on the border of the search space. Although the authors clearly state that the method is not well-suited for these scenarios, it would be interesting to see whether and how much worse it is compared to EI.\n    * An evaluation on a real-world task, e.g., the [(surrogate) SVM task](https://github.com/automl/HPOBench/blob/master/hpobench/benchmarks/surrogates/svm_benchmark.py) used in [1]\n\n\n[1] Falkner, S., Klein, A., & Hutter, F. (2018, July). BOHB: Robust and efficient hyperparameter optimization at scale. In International conference on machine learning (pp. 1437-1446). PMLR.", "broader_impact_concerns": "No broader impact concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7CmZIKbQ5Q", "reviewer": "Reviewer_BQ67", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths:\n1. The central claim of the paper is met: deriv-EI appears to empirically outperform EI on the provided test functions.\n2. The idea is interesting and makes sense. There seems to be potential for this technique to be more generally applicable to other acquisitions beyond EI.\n3. The exposition is clear.\n\n### Weaknesses:\n1. For an algorithm with no theoretical guarantees, the empirical evaluation is somewhat narrow. All the test functions are synthetic and of low dimension. The empirical evaluation would be stronger if it included real world experiments, for example, hyperparameter tuning.\n2. While EI was originally proposed as for the noiseless evaluations setting, the modern BO research landscape almost always considers the case of noisy evaluations in which the observed values $y = f(x) + \\epsilon$ where $\\epsilon$ is a noise random variable. The proposed algorithm's practical usefulness is less proven when the experiments are run with noiseless observations.\n3. deriv-EI assumes that the global minima is within the interior of the search space, as opposed to on the boundary, an assumption that EI does not make and has to be handled as discussed in Sec 5.1. Furthermore, deriv-EI depends on an additional hyperparameter $\\epsilon$ which controls the size of the ellipsoid under which a gradient is taken to be sufficiently small. These additional complications provide a less compelling case for deriv-EI to be a drop-in replacement for EI. Can the authors explain the statement \"The precise choice of ε has thus no impact\"? Surely if we take $\\epsilon$ to be very large, the gradient condition disappears?\n4. A more complete discussion on related works also using GP derivatives would be helpful, e.g., explaining if this paper is related to \"Bayesian Optimization with Gradients\" (Wu et al., 2017) and other related papers.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes deriv-EI, a BO acquisition function that is an extension of expected improvement (EI). deriv-EI selects the next input to sample based not only on the improvement score, but on the likelihood that an input is a local minima, which is the case if the sample path gradient is 0 and the Hessian is positive semidefinite. This is possible because the derivatives of a GP are also GPs. deriv-EI is a tractable approximation to these criteria. deriv-EI is shown to outperform EI on synthetic test functions of dimension 1, 2, 3 and 5.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### Would strengthen work:\n1. Include real world experiments.\n2. Run experiments with noise.\n3. A more complete discussion on related works also using GP derivatives.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All the reviewers found that the claims were justified by evidence provided.  The ratings provided by reviewers, however, were somewhat mixed with one Accept, one Leaning Accept and one Leaning Reject.  Multiple reviewers leaned towards reject before the discussion period.  One of them seems to have been satisfied by the author response.  The other remaining reviewer cited the lack of response as a reason for their Leaning Reject rating, though they noted that they wouldn't be strongly opposed to acceptance.  The authors have since responded, but after the reviewer posted their decision (and they haven't checked the response since - nor are they required to given the timeline).  It seems that the authors addressed most of the reviewers' concerns within the response period and have made substantive changes to their manuscript.  Having checked the authors' response, it seems that they have satisfied the concerns of the remaining negative reviewer.  I appreciate that the authors updated the manuscript and highlighted the changes since the submission.  Therefore, and given the two accepts, I am recommending accept with minor revisions.  The minor revisions include the proposed changes from the rebuttal. Most of these are already worked into the manuscript, but obviously they should no longer be highlighted for the camera ready version.  Given the one reviewers' concerns, I would like to verify that these changes have made it into the camera ready.
Decision: Accept (poster)