Generalizing Bayesian Optimization with Decision-theoretic Entropies
Authors: Willie Neiswanger
Lantao Yu
Shengjia Zhao
Chenlin Meng
Stefano Ermon
Venue: NeurIPS 2022 Accept (2022)
Keywords: Bayesian optimization
entropy search
Bayesian optimal experimental design
knowledge gradient

Abstract: Bayesian optimization (BO) is a popular method for efficiently inferring optima of an expensive black-box function via a sequence of queries. Existing information-theoretic BO procedures aim to make queries that most reduce the uncertainty about optima, where the uncertainty is captured by Shannon entropy. However, an optimal measure of uncertainty would, ideally, factor in how we intend to use the inferred quantity in some downstream procedure. In this paper, we instead consider a generalization of Shannon entropy from work in statistical decision theory (DeGroot 1962, Rao 1984), which contains a broad class of uncertainty measures parameterized by a problem-specific loss function corresponding to a downstream task. We first show that special cases of this entropy lead to popular acquisition functions used in BO procedures such as knowledge gradient, expected improvement, and entropy search. We then show how alternative choices for the loss yield a flexible family of acquisition functions that can be customized for use in novel optimization settings. Additionally, we develop gradient-based methods to efficiently optimize our proposed family of acquisition functions, and demonstrate strong empirical performance on a diverse set of sequential decision making tasks, including variants of top-$k$ optimization, multi-level set estimation, and sequence search.

Generalizing Bayesian Optimization with Decision-theoretic Entropies
Willie Neiswanger', Lantao Yu", Shengjia Zhao, Chenlin Meng, Stefano Ermon Computer Science Department, Stanford University
Stanford, CA 94305
(neiswanger, hww.waaneerrlttayyy... sjzhao, chen inn ermon] @cs. stanford.
Abstract
Bayesian optimization (BO) is popular method for efficiently inferring optima of an expensive black-box function via sequence of queries. Existing information- theoretic BO procedures aim to make queries that most reduce the uncertainty about optima, where the uncertainty is captured by Shannon entropy. However, optimal measure of uncertainty would, ideally, factor in how intend the inferred quantity in some downstream procedure. In this paper, we instead consider a generalization of Shannon entropy from work in statistical decision theory [13, 39], which contains broad class of uncertainty measures parameterized by problem-specific loss function corresponding to downstream task. We first show that special cases of this entropy lead to popular acquisition functions used in BO procedures such knowledge gradient, expected improvement, and entropy search. We then show how alternative choices for the loss yield flexible family of acquisition functions that can be customized for use in novel optimization settings. Additionally, we develop gradient- methods to efficiently optimize our proposed family of acquisition functions, and demonstrate strong empirical performance on diverse set of sequential decision making tasks, including variants top-k optimization, multi-level set estimation, and sequence search".
Introduction
Bayesian optimization (BO) is a popular method for efficient global optimization of an expensive black-box function, which leverages a probabilistic model to judiciously choose sequence of function queries. In BO, there a few key paradigms that motivate existing methodologies. One paradigm is decision-theoretic BO, which includes methods such as knowledge gradient [16] and expected improvement [33, 25]. At each iteration of BO, these methods aim to make query that maximally increases the expected value, under the posterior, of final estimate the optima (some- times referred reduction and includes information based BO methods such as the family of entropy search methods [22, 24, 47, 35]. At each iteration of BO, these methods aim make query that most reduces the uncertainty, under the posterior, about a quantity of interest (such as the location of the optima).
terminal action). Another common paradigm is based maximal uncertainty
In the incertainty-reduction paradigm, the information-bant Shannon entropy as the measure of uncertainty. While Shannon entropy one measure of uncertainty that could aim to reduce at each iteration of BO, it is not the only measure, and it is not necessarily the most ideal measure for every optimization task. For instance, an optimal uncertainty function would, ideally, factor in how we intend to use the final uncertainty about an inferred quantity in some downstream procedure.
methods have predominantly used
"The first two authors contributed equally this work. For additional details, see the project website: htttp:://willeet github.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

In this paper, we develop framework that aims to first unify and then extend these two paradigms. Specifically, we adopt a generalized definition of entropy from past work in Bayesian decision theory [13, 39, 21], which proposes a family of decision-theoreticc entropies parameterized by problem-specific loss function and action set. This family includes Shannon entropy as a special case. Using this generalized entropy, we can view information-based BO methods as instances of decision- theoretic BO, with a terminal action chosen from a different type of action set. Similarly, this frame- work also includes as special cases the decision-theoretic methods such as expected improvement and knowledge gradient, which yields an uncertainty- reducctin view of these methods. Beyond this uni- fied view, our framework can be easily adapted to novel problem settings by choosing appropriate loss and action set tailored to a given downstream use case. This allows for handling new optimization scenarios that have not previously been studied and where no BO procedure currently exists.
As an example, there are many real-world problems where we want to estimate a set of optimal points, rather than a single global optimum. Use cases include when we wish to find set of highest-valus points subject to some constraints on the similarity between these points (e.g. to produce diverse set of candidates in drug or materials design [38, 44, 45]), or points which satisfy some sequential relation (e.g. to construct library of molecules that attains a sequence of desired measurements [15]). Further, we may wish to estimate other properties of a black-box function, such certain curves, surfaces, or subsets of the domain [54, 28, 40J. Due to the vast number of possibilities, most custom problem settings have not been explicitly studied in the literature. A key advantage of our framework is that it provides a way to approach these problems where no suitable methods have been developed.
Additionally, since we define this family of generalized entropies in standardized way, we can develop a common acquisition optimization procedure, which applies generically to many members of this family (where each member is induced by specific loss function and action set). In particular, we develop a fully differentiable acquisition optimization method inspired by recent work on one-shot knowledge gradient procedures [4]. This yields effective and computationally efficient algorithm for many optimization and sequential decision making tasks, as long as the problem-specine loss function is differentiable. In summary, our main contributions are the following:
We propose an acquisition function based on a family of Drcision-theoreti entropies parame- terized by loss function i' and action set A. Under certain choices of l and A. we can view multiple BO acquisition functions in a single -hceorinttheoreticc perspective, which sheds light on the settings for which each is best suited.
By selecting a suitable é and A, we can produce problem-specifte acquisition function, which is tailored to a given downstream use case. This yields a customizable BO method that can be applied to new optimization problems and other sequential decision making tasks, where no applicable methods currently exist.
We develop an acquisition optimization procedure that applies generically to many instance of our framework. This procedure is computationally efficient, using a gradient-based approach.
We demonstrate that our method shows strong empirical performance on a diverse set of tasks including top-k optimization with diversity, multi-level set estimation, and sequence search.
2 Setup
f C IR denote an expensive black-box function that maps from an input search space X' to an output space V, and f F. We assume that we can evaluate at an input E X, and will observe a noisy function value y = f(x) + e, where e ~ (N,
We also assume that our uncertainty about f is captured by probabilistic model with prior distribution p(f), which reflects our prior beliefs about f. Given a dataset of observed function evaluations D. = [(; Ve., |= our model gives a posterior distribution over F, denoted by pDf(D:).
Suppose that, after a given BO procedure is complete, we intend to choose a terminal action from some set of actions A, and then incur a loss based on both this action a and the function f. We denote this loss as e: F x A- + R. As one example, after the BO procedure, suppose we make a single guess for the function maximizer, and then incur loss based on the value of the function this guess. In this case, the action set is A = X and the loss is {(f a) f(a).

---

3 Decision-theoretic Entropy Search
In this section, we first describe family of decision-theoretic entropies from past work in Bayesian decision theory [13, 39, 21J, which are parameterized by problem-specific action set A and loss function This family includes Shannon entropy as a special case. We denote this family using the symbol He, and refer to it as the -ettoony
Definition 3.1. (Hr.A-entropy of f). Given prior distribution p(f) on functions, and dataset D of observed function evaluations, the posterior Hz, with loss l and action set A is defined as
He,A [f] D) inf PUIDD [((J,
(1)
aaA
Intuitively, after expending our budget of function queries, suppose that we must make terminal action a" E AA, where this action incurs loss a(f defined by the loss & and function f. Given posterior p(f(D) that describes our belief about f after observing D, we take the terminal action a" to be the Bayes action, i.e. the action that minimizes the posterior expected loss, then be viewed as the posterior expected
a" arg an Ep((DD) [((f.a), The Hentooyy loss of the Bayes action. We next describe how this generalizes Shannon entropy, and why it is a reasonable definition for uncertainty measure.
Example: Shannon entropy Let P(F) denote a set of probability distributions on function space F, which we assume contains the posterior distribution p( ID) P(F). Suppose. for the He.A-ennropy that we let the action set A = P(F), and loss function {(f, a) = - llo a( (f), for E P(F). Unlike the previous examples, note that the action set is now a set of distributions.
Then, the Bayes action will be a" = pl. D) (this can be shown by writing out the definition of the Bayes action as cross entropy, see Appendix A. and thus
He.A(f | D) = EPUID) [- log a" (f) = HIf D).
(2)
where HJS = - D) log | D) is the Shannon differential entropy. Thus, the Hetropy using the above Al equal the Shannon differential entropy.
Note that we have focused here on the Shannon entropy of the posterior over functions p(f D). In Section 4 we show how this example be extended to the Shannon entropy of the posterior over properties of f, such as the location (or values) of optima, which will provide a direct equivalence to entropy search methods in BO.
Why is this a reasonable measure of uncertainty? The He.t-entropy has been interpreted as measurement of uncertainty in the literature because it satisfies few intuitive properties. First, similar to Shannon differential entropy, the Hr.---ntropy is a concave uncertainty measure [13, 21J. Intuitively, if we have two distributions p1 and p2, and flip coin to sample from Pi or p2, then we should have less uncertainty if we were told the outcome of the coin flip than if we weren't. In other words, the average uncertainty of P1 and P2 (i.e. coin flip outcome known) should be less than the uncertainty of 0.5p1 + 0. 0.5pp (coin flip outcome unknown). Since Hr.A is concave, it has this property. As consequence- also similar Shannon differential entropy- the -entrooy of the posterior is less than the He,4 -eetroyy of the prior, in expectation. Intuitively, whenever we make additional observations (i.e. gain more information). the posterior entropy is expected to decrease.
Acquisition function We propose family of acquisition functions for BO based on the He,A- entropy, which are similar structure to coarmatiiontheoreed acquisition functions in the entropy search family. Like these, our acquisition function selects the query E X' that maximally reduces the uncertainty, as characterized by the -entooy in expectation. We refer to this quantity as the expected H iinfoornrraain gain (EHIG).
Definition 3.2. (Expected H -informattoi gain). Given a prior p(f) on functions and a dataset of observed function evaluations Dr, the expected -infoorooootioonn gain (EHIG), with loss é and action set A, is defined as
(H; Hr [f D.] Ep(LIP.) [H [f D. U ((z, va)lll
(3)
There are multiple benefits to developing this acquisition function. Though similar in form to entropy search acquisition functions, the EHIG yields (based on the definition of He,r) the one-step Bayes
3

---

optimal query for the associated decision problem specified by the given loss l and action set A. We prove in Section 4 that the EHIG casts both uncertainty uncertainty-reduccioo reduction and functions under common umbrella, using different choices of l and A; this standardization provides guidance on which acquisition function optimal for a given use case, based on details of the associated terminal action. More interestingly, in Section 5 we show how the EHIG allows us to derive problem-specific acquisition functions tailored to novel optimization and sequential decision making tasks. And importantly, since we frame acquisition optimization of this family in common way-as a bilevel optimization problem over the sample space and action space-we we can develop a single acquisition optimization method that can generically apply to many custom tasks (Section 6).
acquisition
In Algorithm 1, we present He.A-ENTROPE SEARCH, our full Bayesian optimization procedure using the EHIG acquisition function. This procedure takes as input loss f. action set A. and prior model p(f). At each iteration, the procedure optimizes EHIG, (a;; e, A)) to select design 3y E X to query, and then evaluates the black-box function on this design to observe an outcome Ya, f(xa) + 6. In Section we describe methods for optimizing the EHIG acquisition function via gradient-based procedures, which provide computationally efficient algorithm for many A and l.
Algorithm 1 -.NTROP SEARCH
Input: initial dataset D1, prior p([), action set A, loss
1: for = 1,..., T ....,TT do
2: It - arg m (2; (,A)
D Optimize the EHIG acquisition function
3: Yx, f(xx) +e
> Evaluate the function f at æe D. Update the dataset
4: D.+1 + (Ue(
Output: distribution p(ʃ DT+1)
A Unified View of Information-based and Deeision-theoretic Acquisitions
In this section, aim to show how acquisition functions commonly used in BO are special cases of the proposed EHIG family, for particular choices of and A. This will allow us to view each acquisition function (including information-based ones) from the perspective of common decision problem: after the BO procedure is complete, we choose a terminal action from action set A and then incur loss defined by f. Each acquisition function can be viewed as reducing the posterior uncertainty over f in a way that yields a terminal action with lowest expected loss.
This unified view provides two main benefits. First, it sheds light on the particular scenarios in which one of the existing acquisition functions is optimal over the others (which we focus on in this section). Second, it shows how using the EHIG with other choices for E and A provides new acquisition functions for a broader set of optimization scenarios and related tasks (which is the focus of Section 5).
Information-basee acquisition functions We state the family of entropy search acquisitions func- tion a general way that includes the entropy search (ES) [22], predictive entropy search (PES) [24], and max-value entropy search (MES) [47] algorithms. Let 0, E θ denote property of I we would like to infer. For example, we could set O = arg max,ex f(xr) = x² E X, i.e. the location of the global maximizer of f. 0, = max f(x) E R, i.c. the maximum value achieved by f in X. This family of entropy search acquisition function can then be written as:
(s. H [0, Ep) [H [0,|D, U (xx)))))]-
We can view this acquisition function as a special case of the EHIG in the following way. Suppose. after the BO procedure is complete, we choose a distribution q from a set of distributions P(0) and then incur loss equal to the negative log- likelihoodd of q for the true value of 0,. this case, we view the action set as A = P()) and the loss function as {(f, a) = log a(Oy), where a E A. To visualize this, in the case where f x=, see Figure (left). which shows the terminal action (gold density function) and corresponding loss (horizontal dashed line).
Under this choice, the Hentrony of will be equal to the Shannon entropy of Of, and thus the EHIG, will be equal to ES,, We formalize this in the following proposition.
Proposition 1. If we choose A = P()) and &(f. q) - log g00)), then the EHIG is equivalent to the entropy search acquisition function, i.e. EHIG, (a; e, A) (S,()).

---

Entropy Search
Knowledgee Gradient
Expected Imprrvement
k Guroes
Wel
X
Action P(X)
Action Irea: ([.,0)= -f()).
Action A (xi) Lrees ([..o) (()),
Loes (i log, ((("),
Love [[5,n] = (mn) f(oo))
Figure 1: Example acquisition functions, write the associated action set A and loss function below the plot. In each plot, the true function is a solid black line, the posterior mean is a red dashed line, the observed data are black dots, and the Bayes action is shown in gold, See Section 4 for further discussion.
their corresponding Bayes actions a' visualized. For each, we
Proof of Proposition 1. See Appendix A.I.
prcision-theoretic acquisition functions We next describe how the EHIG generalizes decision- theoretic acquisition functions such as knowledge gradient (KG) and expected improvement (EI), Since these acquisition functions are often motivated from a perspective of terminal decision, is straightforward to show how they are special case of the EHIG. However, the choice of A and & here is insightful to review before extending EHIG other scenarios.
First, the KG acquisition function can be written
(1) = Ep(ua (D:) [Mi+1 (30, Mn.]] -i::
where 14 = su (R(IID)) [f(x)) ]] is the max value of the posterior mean of f given data Dz, and (x, M²) = suparex (.p()).. [f('')] is the max value the posterior mean, given both data D, and observation (..))). Second, the acquisition function can be written
(1((2) = Ep(p= (P)) [max(yy= - -]]
where ft max (ax))(=x))) for 24 D, and fxx,) is the posterior expected value of f at z;- This definition is equal to the standard formulation of El in the noiseless setting (i.e. when V/- = f(x) for
queried æ)) and the plug-in formulation of EI in the noisy setting, when yı- f(x) e [37, 6).
To view these decision-theoretic acquisition functions as special cases the EHIG, suppose that after BO is complete, we aim to make single guess x" A for the maximizer of f. and then incur a loss equal to the value of the function at x=, i.c. {(f, case the Bayes action a* is the maximizer of the posterior mean, and for EI, we let A = 1- in which case a" is the best queried point. We visualize these as gold vertical lines in Figure 1 (center panels). In Appendix A, we prove this equivalence with EHIG, for KG, and El,. Additionally, in Appendix A.4 we discuss connections to the probability of improvement (PI) acquisition function.
- f(x*). ). For KG, we let A = X. in which
We thus see two key differences here, in comparison with information-basee BO: (i) the terminal action a" is a point estimate of the optimizer x² rather than distribution over X' and (ii) the loss does not depend on the particular value of the true optima x* (nor on how accurately a" provides an estimate of x²) but rather only depends on the function value of the terminal action, fla").
5 A Framework to Derive New Acquisition Functions for Custom Tasks
There are many real-world problems that go beyond simple black-box optimization, which have not been explicitly studied in the literature, and for which there does not exist a suitable acquisition function. For these use cases, we can define an action set and loss based on details of the problem, and use the EHIG to provide a corresponding problem-specific acquisiton function. As examples of this, below we apply the EHIG number of relevant problems where, as far as we are aware, no corresponding acquisition function has been developed by prior work.
Illustrative example: k-guesses As a simple example to illustrate our framework, suppose, after optimization is complete, we are allowed to make batch of k guesses for the function maximizer z", and then recieve a final reward based on the best guess. This setup appears in cases where, after BO is complete, we can make butch of final designs (e.g. synthesize a final set of materials [44] or train a final set of models [49, 41]), and only care about the single best design of the batch. We can thus view the action set A x and loss as l(f, a) max (a.)),..... f(a)))
5

---

Figure 1 (far right) provides visualization of this for k = 2. In this scenario, one of the Bayes actions (az) is near the maximizer of the posterior mean (similar to KG). while the other (aī) is separated from the first. Intuitively, to minimize the expected loss, the second guess should have both a high posterior mean, and also low correlation to the first guess-the first guess has a better chance of low loss, but in cases where it fails, we want the second guess to not fail (i.e. not match the first guess), while also achieving a low loss. In practice, this yields an EHIG acquisition function that has similar but distinct exploration strategy from KG. It spends a small portion of the budget on queries that give some information about not just the first guess, but the other k-1 guesses as well.
Top-k optimization with diversity Instead of a single optimal point in X. there are applications where we wish to estimate set of top-k optima, i.. the subset of X' of size k that has the highest sum of values under f. Examples of this can be found in materials discovery [30, 44), sensor networks [2, 18), and medicine [_ Note that the goal of this problem is distinct from the k-guesses example described above. When X' is continuous, to avoid redundant solutions, we may wish to carry out the task of top-k optimization with diversity, which aims to find the top-k optima such that d(x,, c, j E {l....,k} kj. for a problem-specific distance d. As one example of our EHIG framework, suppose that we choose A = X (where a = (a1....,aN, E A denotes a set of top-k points) and incur the loss
(f.o)= E
E ISieish
Mai,ay)
(4)
Note that we can select the distance function here to match the mroblem-specinne constraint. Intuitively speaking, this choice of (4) yields an EHIG acquisition function that makes a sequence of queries which rotate focus between multiple diverse optimal points in the domain.
Multi-level set estimation The goal of level set estimation (LSE) to estimate a subset of the
design space N. where function values are larger than given threshold S. = { E X' f(x) > c}. This task appears a number of applications, including catalyst design [54], interactive learning [5], and environmental monitoring [40]. While many prior works have studied standard LSE, here we consider the task of multi-level set estimation (MLSE), where we are given m thresholds satisfying
C1 ... < Cm and want to estimate m + 1 sets: S: = far E X G < f(x) < C++1 for m) (where cp := -o and 1 := +oo). This is useful in the above applications when we have more than one threshold of interest-for example, public health policy makers must estimate regions where disease prevalence exceeds 1 1%, 2%, etc., for graded reopening decisions [36, 52].
i
As one approach to MLSE using the EHIG, we focus on settings with a discrete set of design points No X. Xo| = [19, 26]. Suppose, after querying is complete. we must choose set of values E A [0. 1 (one for each x E Xo and E
which represent level set identity variables. Suppose we then incur a loss with the following form, that depends on these identity variables a, as well as on a flexible relation r r(f(z), between function values f (I) and thresholds C, i.e.
172
{((a(z))
Σ
Σ ( =E
(5)
For instance, if r() f(x) C.. then the optimal a.(x) should specify the c--suer level set for each i E (1...., .m]. i.e. (x(x)) = 1 for each x E An with f(xx) > e; and a;(x) = 0 otherwise. This example loss yields an acquisition function that, empirically, focuses samples around the boundaries of multiple level sets of a black-box function simultaneously.
Sequence search We define sequence search as the task of estimating a sequence of inputs Tm E x with outputs values matching set of problem-specifie criteria. For exam- ple, we may wish to estimate a sequence of inputs corresponding to predefined set of function values (yi* ...., ). This finds applications in materials science, such as in the task of synthesizing a nanoparticle library [15] (i.e. finding set of input conditions that yield a set of nanoparticles of different pre-defined sizes). As another example, in the context of public health, we may be interested in set of locations where vaccination rates equal some pre-specified values (e.g. (20%.... ... 80%)) when making decisions involving vaccine allocations, as we describe in Section 7, As an example of these applications in our EHIG framework, we might have the action set A = x and loss

---

- (()(am)) -ym! km))
6
f=1
These examples all aim to show that the EHIG can be used to define a problem-specine acquisition function, which be tailored to the details of particular case. As result, when used in Algorithm 1, we gain a customizable optimization framework that can be applied to a variety of novel problem settings with special percpae losses.
6 Gradient-based Acquisition Optimization
At each iteration of -E.A-EETROPP SEARCH (Algorithm 1), we optimize the acquisition function to select the next query It = arg m EHIG, (z; A.). Classically, zeroth order optimization routin- ues have been used for acquisition optimization BO. However, recent work has developed gradient- based methods for optimizing certain acquisition functions [48, 4], which can allow for efficient acqui- sition optimization over X' We work on similar methodology here-namely, we develop a gradient- based acquisition optimization procedure for appropriate settings (i.e. assuming continuous X' and A. and certain conditions on E). We can, for example, apply this gradient-based optimization to each of the acquisition functions described in Section 5, for which we show experimental results in Section 7.
Similar to related work [48, 4], we give the following derivation with a focus on Gaussian process (GP) models, though the methodology can be extended to other models in which we can apply the reparameterization procedure described below to differentiate through posterior model parameters.
Differentiable loss function We first describe a few assumptions that must be satisfied to carry out the gradient-based optimization procedure. Denote the posterior expected loss given 'D by L(D, a) = (PDD))) [)] We assume that this loss function depends only on the function value of f at finite number of points, i.e. there exists functions yı (a), .. FFx(a) (a), and a function f': RRR A R, for K E N, such that
((f.a) = E' (()())) ... f(K (a)).
This requirement is satisfied by the loss functions in Section 5. For brevity, denote the sequence (a),.... Fx (a) by F1:k(0) (a) and (a)(, ((a])))... [ by f(F (a)). We assume that the func- tions F& and E' are differentiable with respect all arguments. Given dataset D and GP prior, the posterior distribution of fI (a)) is also Gaussian. In particular, there exist functions
F1.K (a) X D + R² and U F1:K((a X D + K
such that fFE (a)) ) = m(Fi (a); D) + U(FIK (a); D)e where e is drawn from a K-dimensional standard normal distribution. We can combine the above results get
T(DD,0) a) = E, [e' (u((F:: (a); D) + (F ());))e, a)] key property is that we can compute unbiased gradients of this with respect both D and a, VL(D,a) E. V (a) D) + (a): ()): a)].
Differentiable acquisition function For given input predictive distribution of our model. Note that there exists a deterministic function π(I, D, A) such that y(æ, D) = y(.,, D, A), where À is drawn from standard normal distribution. Hence, if & satisfies Eq. (7), then we can optimize EHIG, with gradient descent. In particular, we can write
X, let y(2, D) denote the posterior
inf =EX
(Z; A) = inf inf Ex.. (A(æ, a(A) + 0(z,, a(A))e,
(8)
=EX a:A=AA
where in Eq. (8). avoid clutter, use the shorthand f(zz a()) µl[1: (a(A)); ( DUg(Z, A)). and (z, a(>)) =- (F U J(x D))).. Importantly, we can compute the unbiased gradi- Üa, a()))e, a()))] In practice, we can also take gradients of a Monte Carlo estimate of Eq. (8) [4], by fixing samples of X. throughout the optimization. Specifically, we can sample Ay. AM and f1, CN and approximate Eq. (8) via
ent of the quantity (f aA))
inf FH;G, (, FEX
inf inf TEX 01....M
Σ C'ji,, am) + E (a,, am 6nn am)) 172
NM
where we use am = a(Am) ) for brevity. Under the assumptions above, we can compute the unbiased gradient of this quantity, and by using systems such as GPyTorch [17] and BoTorch [4] we can compute this gradient efficiently via automatic differentiation.
7

---

Alyine
Alpine 5, R
Vactination. A =
17.5
14.0
Loee
04.4
163
14.0
Tandea Sund.
140
Soudl
Inmastion
Iterstioo
Random South
Uncretalaty Sumpling
Kuowdeege Grallient
Ri Entroey Search
Figure 2: Top-k optimization with diversity. Top row: Plots of negative loss [f, versus iteration for all methods, on the 3 and Voccination functions, where error bars represent one standard error. Bottom row: Visualization of methods in two dimensions, showing the set of ground-trute top-5 diverse design points (blue squares). queries D, taken (black dots), acquisition function optimizer (pink dot), and the estimated set top-5 diverse design points (gold stars).
7 Experiments
We evaluate our proposed method on the example tasks described in Section 5: top-k optimization with diversity, multi-level estimation, and sequence search. For these applications, we show comparisons against a set of baselines on real and synthetic black-box functions.
Comparison methods. In experiments, compare the following set of acquisition strategies:
ENTROPY SEARCH (HES). We follow Algorithm 1, using the loss and action set for each task as described in Section 5, and the gradient-based procedure outlined in Section 6.
RANDOM SEARCH (RS). At each iteration, draw a sample I, uniformly at random from X. UNCERTAINTY SAMPLING (US). At each iteration, we select the point that maximizes the posterior predictive variance, i.e. It arg 2 Varip(y, [D)]]-
KNOWLEDDGE GRADIENT (KG). We show KG as representative method for standard BO. KG allows us to carry out similar gradient-based procedure as HES.
PROBABILITTY OF MISCIASSIIFICTTION (POM). This is common acquisition function for level set estimation [7]. We predict whether point is above a threshold, represented by a binary variable and select the design with maximal label uncertainty
arg min max p(=[xr).
Note that we are restricted to comparing against relatively general-purpose more-specific acquisition functions have not previously been developed for the tasks below.
purpose baseline methods, as
Top-k Optimization with Diversity In our first task, the goal is to find set of k diverse elements in X, each with high value f. To assess each method, at each iteration we record the negative loss -((ff a²) using Eq. (4) the negative top-k with diversity loss of the Bayes action a* arg inf (1DD)) [E(J, a)] on the true function f-using the set of queries Dr produced by the given method. Intuitively, if method makes a set of queries that yield good estimate of diverse top-k elements, it will score a high value under this metric.
In Figure 2 (bottom row) we visualize results on the multimodal Alpine-d benchmark function (see appendix for details). Here, we can see that HES concentrates queries over five local optima of this function, while KG allocates majority of samples on only the highest peak, and both US and RS distribute their queries over the full domain X. In Figure 2 (top row), we compare performance by plotting the negative loss versus iteration on two higher dimensional examples. We also compare each method on the Vaccination function (provided by [53]). which returns the vaccination rate for locations in the continential United States, given an input (latitude, longitude). The goal of this task to efficiently find a set of five diverse locations with highest vaccination rates. We show results in Figure (top row, right), and see similar advantage of HES over comparison methods.

---

Pranserinis Night Light
Alpcne 2
Moltiille
110
10
1.5
Rande Soud
1.0
Unentsis Soapling
Kambedg Groden
hastie
Domaion
Fandoo: Seath
Uncertaing Sumniing
Knowledge Gralient
H, Pottooyy Soutch
Figure 3: Multi-level set estimation. Top row: Plots of accuracy versus iteration for all methods, where error bars represent one standard error. Bottom row: Visualization of methods on the Multihills function, showing the ground-truth level set boundaries (red and blue dashed lines) and queries D; taken (black dots).
Multi-level Set Estimation In our second task, the goal is to carry out multi-level set estima- tion. Here, we can assess each method using more conventional metric: we produce an estimate of the level for every a particular method), and then record the accuracy of this estimate averaged across all level set thresholds. Intuitively, method will achieve higher accuracy if it chooses queries that yield better estimate of the function near the threshold boundaries of the level sets. In Figure (bottom row), visualize results for a two-level set task the Multihills function, defined ture density (details given in appendix). We see that HES concentrates queries along both of the boundaries, which are drawn blue and red dashed lines. In the lop row, compare the per- formance of all methods, showing the accuracy vs. iteration. Here, the Pennsylvania Night Light function [1] released by NASA (additional details in the appendix), returns the relative level of light at a location in Pennsylvania, as queried by a satellite image. The goal of this experiment to determine the portion of land at which night light is above a specified threshold value. In Appendix B, we show additional experiments, including a visualization of results on this function.
No. using the model's posterior mean (given the queries selected by
mix-
Meriaaioo
Sequence Search In our third task, the goal is to find a sequence of elements whose value under the black-box function matches a set of 1823
Fox Bey Seeth Dicomaiity San
pre-specified function values (v1 ..... yn) To assess each method, at each iteration we record the negative loss -((f, aa),,aa) from Eq. (6)-i.e. the negative sequence search loss of the Bayes action a² using the set of queries D. produced by
Do
Figure 4: Sequence search. Left: Negative loss versus iteration, where error bars represent one stan- dard error. Right: Visualization of the Vaccination function, along with the queries D. taken by HES (black dots), and the estimated sequence (x/ ...,,..,,3" x.) (red diamonds), such that (f(=)).... (30%, 40%, 50%, 60%, 70%).
arg infae. (IDD) [[(
the given method. Intuitively, if a method makes a set of queries that yield good esti- mate of a sequence of (x ..... am such that (f(x))) f(x R (V ..... ) it will score a high value on this metric.
In Figure 4 (right) we visualize results on the Vaccination function (described above). Here, our goal is find sequence of five (latitude, longitude) coordinates with vaccination rates equal to (vi ..... yom ) = (30%, 40%, 50%, 60%, 70%). Estimates of locations that match these function values can be useful when making policy decisions involving a vaccine response or allocation. In this case, we see that HES concentrates queries along route from the relatively highly vaccinated region the East to the relatively lowly vaceinated region in the North. The left plots in Figure provides a quantitive comparison of methods on the Vaccination function (results on additional functions are shown in the appendix), plotting the negative loss vs. iteration. These again show the benefits of query selection performed by HES relative to the comparison strategies.

---

8 Conclusion
In this paper, we take a decision making perspective on information- based acquisition functions: after querying is complete, we assume that we must make some decision a" and then incur a loss ((f, a²). Our goal is thus to make a sequence of queries that reduce the uncertainty of the posterior distribution p(ʃ D2) in way to best help make this decision with low loss. Using -enrroyy [1], we can define an EHIG acquisition function which carries this out directly: selects a point that is expected to maximally reduce the posterior expected loss of the Bayes action a'. We incorporate this acquisition function into a procedure called - SEARCH, and show, in many cases, that can perform efficient gradient-based optimization of this acquisition function.
There are multiple interesting avenues for future work. First, we hope to develop acquisition optimization methods for additional settings, such as for non-continuous action sets A or design spaces X [12], and for functions with multidimensional outputs [23, 11]. One interesting avenue is hybrid optimization settings, where we can only take gradient steps with respect to either the design or action variables. Another potential direction is to incorporate cost-aware Bayesian optimization techniques into the EHIG framework [29, 50, 3]. We also wish to study how the proposed EHIG framework could be applied in practice to solve various problems in the sciences, including experimental physics [14, 32, 8]. drug discovery [43, 27, 20], and materials design [31, 46]. Finally, we wish to study in further detail how the EHIG acquisition function could be implemented for Bayesian decision making with other probabilistic models beyond Gaussian processes [42, 10, 9, 34].
Acknowledgments
We thank the anonymous reviewers, members of the Stanford SAIL community, and members of the CMU Auton Lab for helpful feedback on this paper. This work was supported by NSF (#1 #1651165). AFOSR (FA95501910024), ARO (W91NNF-21-1-0125)) CZ Biohub, and Sloan Fellowship.
References
[1] Nasa earth observatory: Earth at night. https://earthhobsrrvattot waw....nnaaagg.................. gov/ /etsureeeNihtttiiiee
[2] Ali Abbasi, Ahmad Khonsari, and Navid Farri. Mote: networks. In 2008 IEEE Symposium Computers and Communications, pages 957-962. IEEE, 2008.
efficient monitoring of top-k set in sensor
[3] Raul Astudillo, Daniel Jiang, Maximilian Balandat, Eytan Bakshy, and Peter Frazier. Multi-step budgeted bayesian optimization with unknown evaluation costs. Advances in Neural Information Processing Systems, 34:20197-20209, 2021.
[4] Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Benjamin Letham, An- drew Gordon Wilson, and Eytan Bakshy. Botorch: A framework for efficient monte-carlo bayesian optimization. Advances in Neural Information Processing Systems (NeurIPS), 2020.
[5] Benedikt Boecking, Willie Neiswanger, Eric Xing, and Artur Dubrawski. Interactive weak supervision: Learning useful heuristics for data labeling. arXiv preprint arXlv.2012.06046. 2020.
[6] Eric Brochu, Vlad M Cora, and Nando De Freitas. A tutorial on bayesian optimization of expensive cost functions, with application active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv: /012:2/012.2599 2010.
[7] Brent Bryan, Jeff Schneider, Robert Nichol, Christopher J Miller, Christopher R Genovese, and Larry Wasserman. Active learning for identifying function threshold boundaries. In NIPS, pages 163-170. Citeseer, 2005.
[8] Ian Char, Youngseog Chung, Willie Neiswanger, Kirthevasan Kandasamy, Andrew O Nelson, Mark Boyer, Egemen Kolemen, and Jeff Schneider. Offline contextual bayesian optimization. Advances in Neural Information Processing Systems, 32, 2019.
[9] Youngseog Chung, Ian Char, Han Guo, Jeff Schneider, and Willie Neiswanger. Uncertainty tool- box: an open-source library for assessing, visualizing, and improving uncertainty quantification. arXiv preprint arXiv:2109.100 10254, 2021.
10

---

[10] Youngseog Chung, Willie Neiswanger, Ian Char, and Jeff Schneider. Beyond pinball loss: Quantile methods for calibrated uncertainty quantification. Advances in Neural Information Processing Systems, 34:10971-10984 2021.
[11] Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Multi-objective bayesian optimization over high-dimensional search spaces. In Uncertainty in Artificial Intelli- gence, pages 507-517. PMLR, 2022.
[12] Samuel Daulton, Xingchen Wan, David Eriksson, Maximilian Balandat, Michael A Osbore,, and Eytan Bakshy. Bayesian optimization over discrete and mixed spaces via probabilistic reparameterizzation. In ICML2022 Workshop on Adaptive Experimental Design and Active Learning in the Real World, 2022.
[13] M H DeGroot. Uncertainty, information, and sequential experiments. Ann. Math. Stat., 33(2):404 419, 1962.
[14] Joseph Duris, Dylan Kennedy, Adi Hanuka, Jane Shtalenkova, Auralee Edelen, P Baxevanis, Adam Egger, T Cope, M McIntire, S Ermon, et al. Bayesian optimization of a free-electron laser. Physical review letters, 124(12):124801 2020.
[15] Anthony Y Fong, Lenson Pellouchoud, Malcolm Davidson, Richard C Walroth, Carena Church, Ekaterina Tcareva, Liheng Wu, Kyle Peterson, Bryce Meredig, and Christopher J Tassone. Utilization of machine learning to accelerate colloidal synthesis and discovery. The Journal of Chemical Physics, 154(22):224201, 2021.
[16] Peter Frazier, Warren Powell, and Savas Dayanik. The seowledge-gradiene policy for correlated normal beliefs. INFORMS journal Computing, 21(4):599-613, 2009.
[17] Jacob R Gardner, Geoff Pleiss, David Bindel, Kilian Q Weinberger, and Andrew Gordon Wilson. Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. arXiv preprint
2018.
[18] Roman Garnett, Michael A Osbore,, and Stephen Roberts. set selection. In Proceedings of the 9th ACM/IEEE international conference information processing in sensor networks, pages 209-219, 2010.
Bayesian optimization for sensor
[19] Alkis Gotovos, Nathalie Casati, Gregory Hitz, and Andreas Krause, Active learning for level set estimation. In Proceedings of the Twenty-Third international joint conference Artificial Intelligence, pages 1344-1350, 2013.
[20] Ryan-Rhys Griffiths, Leo Klarner, Henry B Moss, Aditya Ravuri, Sang Truong, Bojana Rankovic, Yuanqi Du, Arian Jamasb, Julius Schwartz, Austin Tripp, et al. Gauche: A li- brary for gaussian processes and bayesian optimisation in chemistry. 2022.
[21] Peter D Grünwald and A Philip Dawid. Game theory, maximum entropy, minimum discrepancy and robust bayesian decision theory. Ann. Stat., 32(4): 1367-1433, August 2004.
[22] Philipp Hennig and Christian J Schuler. Entropy search for Cerimaatoo-Efficienn global optimization. J. Mach. Learn. Res., 139577:189--1837, 2012.
[23] Daniel HernándezzLobato, Jose Hernandez-Lobato, Amar Shah, and Ryan Adams. Predictive entropy search for multi-objective bayesian optimization. International conference on machine learning, pages 1492-1501. PMLR, 2016.
[24] José Miguel Sermández-Lobateee Matthew W Hoffmann and Zoubin Ghahramani. Predictive entropy search for efficient global optimization of black-box functions. In NIPS, 2014.
[25] Donald R Jones, Matthias Schonlau, and William Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4):455-492, 1998.
[26] Kirthevasan Kandasamy, Willie Neiswanger, Reed Zhang, Akshay Krishnamurthy, Jeff Schnei- der, and Barnabas Poczos. Myopic posterior sampling for adaptive goal oriented design of experiments. In International Conference on Machine Learning, pages 3222-3232. PMLR, 2019.
[27] Parnian Kassraie, Andreas Krause, and Ilija Bogunovic. Graph neural network bandits. arXiv preprint 2oXiv:2207.06456, 2022.
[28] Mina Konakovic Lukovic, Yunsheng Tian, and Wojciech Matusik. Diversity-guides multi- objective bayesian optimization with batch evaluations. Advances in Neural Information Processing Systems, 33:17708-17720, 2020.

---

[29] Eric Hans Lee, Valerio Perrone, Cedric Archambeau, and Matthias Seeger. Cost-aware bayesian optimization. arXiv preprint arXiv:2003. 10870, 2020.
[30] Yue Liu, Tianlu Zhao, Wangwei Ju, and Siqi Shi. Materials discovery and design using machine learning. Journal of Materiomics, 3(3):159-177, 2017.
[31] Turab Lookman, Prasanna V Balachandran, Dezhen Xue, and Ruihao Yuan. Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design. npj Computational Materials, 5(1):1-17, 2019.
[32] Sara A Miskovich, Willie Neiswanger, William Colocho, Claudio Emma, Jacqueline Gar- rahan, Timothy Maxwell, Christopher Mayes, Stefano Ermon, Auralee Edelen, and Daniel Ratner. Bayesian algorithm execution for tuning particle accelerator emittance with partial measurements. arXiv preprint arXiv:2209.04550 2022.
[33] Jonas Močkus. On bayesian methods for seeking the extremum. In Optimization techniques IFIP technical conference, pages 400-404. Springer, 1975.
[34] Willie Neiswanger, Kirthevasan Kandasamy, Barnabas Poczos, Jeff Schneider, and Eric Xing. Probo: Versatile bayesian optimization using any probabilistic programming language. arXiv preprint arXiv: 11515 2019.
[35] Willie Neiswanger, Ke Alexander Wang, and Stefano Ermon. Bayesian algorithm execu- tion: Estimating computable properties of black-box functions using mutual information. In International Conference on Machine Learning, pages 8005-8015, PMLR, 2021.
[36] Eric J Oh, Alyssa Mikytuck, Vicki Lancaster, Joshua Goldstein, and Sallie Keller. Design and estimation for the population prevalence of infectious diseases. medRxiv, 2021.
[37] Victor Picheny, Tobias Wagner, and David Ginsbourger. A benchmark of kriging-based infill criteria for noisy optimization. Structural and Multidisciplinary Optimization, 48(3)6607-626, 2013.
[38] Edward O Pyzer-Knapp. Bayesian optimization for accelerated drug discovery. IBM Journal of Research and Development, 62(6):2-1, 2018.
[39] C Radhakrishna Rao. Convexity properties of entropy functions and analysis of diversity. Lecture Notes-Monograph Series, pages 68-77, 1984.
[40] Aarti Singh. Nonparametric Set Estimation Problems in Statistical Inference and Learning. PhD thesis, University of Wisconsin-Madison 2008
[41] Jasper Snock. Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine
learning algorithms. Advances in neural information processing systems, 25, 2012.
[42] Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable bayesian optimization using deep neural networks. In International conference on machine learning, pages 2171-2180. PMLR, 2015.
[43] Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson. Accelerating bayesian optimization for biological sequence design with denoising autoencoders. arXiv preprint arXiv:2203. 12742, 2022.
[44] Kei Terayama, Masato Sumita, Ryo Tamura, and Koji Tsuda. Black-box optimization for automated discovery. Accounts of Chemical Research, (446)):334-1344 2021.
[45] Kevin Tran, Willie Neiswanger, Kirby Broderick, Eric Xing, Jeff Schneider, and Zachary W Ulissi. Computational catalyst discovery: Active classification through myopic multiscale sampling. The Journal of Chemical Physics, 154(12):124115 2021.
[46] Kevin Tran, Willie Neiswanger, Junwoong Yoon, Qingyang Zhang, Eric Xing, and Zachary W Ulissi. Methods for comparing uncertainty quantifications for material property predictions. Machine Learning: Science and Technology, 12):0220006, 2020.
[47] Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient bayesian optimization. In Proceedings of the 34th International Conference on Machine Learming- Learning-Volume Volume 70, pages 3627-3635, 2017.
[48] James T Wilson, Frank Hutter, and Mare Peter Deisenroth. Maximizing acquisition functions for bayesian optimization. arXiv preprint arXiv:1805.10196 2018.
12

---

[49] Jian Wu and Peter Frazier. The parallel knowledge gradient method for batch bayesian opti- mization. Advances in neural information processing systems, 29, 2016.
[50] Yuxin Xiao, Eric P Xing, and Willie Neiswanger. Amortized auto-tuning: Cost-efficient transfer optimization for hyperparameter recommendation. arXiv preprint urXiv:2106.09) 179, 2021.
[51] Pengtao Xie. diversity-promoiingg and large-scale machine learning for healthcare. PhD thesis, University of Pittsburgh Medical Center, 2018.
[52] Constantin T Yiannoutsos, Paul K Halverson, and Nir Menachemi. Bayesian estimation of sars-cov-2 prevalence in indiana by random testing. Proceedings of the National Academy of Sciences, 1 118(5), 2021.
[53] Yuan Yuan, Eaman Jahani, Shengjia Zhao, Yong- Yeo Ahn, and Alex Sandy Pentland. Mobility network reveals the impact of geographic vaccination heterogeneity on covid-19. medRxiv, 2021.
[54] Miao Zhong, Kevin Tran, Yimeng Min, Chuanhao Wang, Ziyun Wang, Cao-Thang Dinh, Phil De Luna, Zonggian Yu, Armin Sedighian Rasouli, Peter Brodersen, al. Accelerated discovery of co electrocatalysts using active machine learning. Nature, (8177777517800) 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section and appendix Section A.
(b) Did you include complete proofs of all theoretical results? [Yes] See appendix
Section A.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] All code and instructions are included in supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] All training details are specified in the paper and included code.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] See Section
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See appendix B.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] All existing assets were cited.
(b) Did you mention the license of the assets? [Yes] Licences of all assets are properly attributed.
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes] New assets are included in the supplementary material.
(d) Did you discuss whether and how consent was obtained from people whose data you're asing/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
13

---

(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "uWIYyuuDO81", "reviewer": "Reviewer_YGMD", "summary": "This work introduces a family of acquisition functions (AFs) for Bayesian Optimization (BO) based on the decision-theoretic entropies, $H_{l,A}$-$\\textit{entropy}$, which is a generalized version of Shannon entropy. The AFs can be tailed to select queries that maximize the reduction of uncertainty in $H_{l,A}$-$\\textit{entropy}$, which is defined as $\\textit{expected}$ $H_{l,A}$-$\\textit{information gain}$ (EHIG). The EHIG is a general form of AFs and can be reduced to information-based AFs or decision-theoretic AFs by choosing the parameters, $l$ and $A$. A framework is provided for the applications of EHIG on several categories of problems. Moreover, a gradient-based acquisition optimization method is proposed. Finally, evaluations of the method are made on examples datasets.", "questions": "1. Does the Random Search (RS) method optimize any function? In top row of Fig.2 the negative loss of RS slowly increases comparing to HES, but from the description of RS (line 305) the samples are drawn randomly from the full domain, therefore, it is hard to understand why the negative loss increases. Is there an explanation?\n\n2. In Sequence Search task only the route of HES is shown and discussed. Bases on the left of the Fig. 4 at least the Uncertainty Sampling has comparable performance on the negative loss to HES. It would be better if all routes can be shown on the right of Fig. 4 for a thorough and clearer comparison. \n\n3. Some of the plots are too small to see the details and axes, eg. bottom rows of Fig. 2 and Fig. 3.", "limitations": "There is no societal impact from this work. The authors should state and summarize the limitations clearly to the readers.\n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. This paper is well-organized and easy to read. \n2. This article proposes a brand new general AF which unifies two branches, information-based and decision-theoretic AFs, for BO. Following the framework to carefully construct the the EHIG AF for each task and using the proposed gradient-based optimization method, the proposed $H_{l,A}$-$\\textit{Entropy Search}$ (HES) procedure has advantage over other baseline methods.\n\nWeaknesses:\n1. The experiments are carried out on tasks which no AF has been developed in other work. However, to make this paper more significant, there should be examples that compare the result of the proposed HES method to other traditional BO method with AF on well-studied tasks. In this way the readers can have a better idea on the performance of HES.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "QpIUsZst4ad", "reviewer": "Reviewer_ryq5", "summary": "The authors derive a generic acquisition function design strategy (EHIG) based upon a user-specified loss functions and action set applicable to a variety of custom tasks involving Bayesian optimization, viz., Top-k optimization with diversity, Multi-level set estimation and Sequence search.\nEHIG includes as special cases both information-based acquisition schemes such as entropy search (ES) and decision theoretic acquisition schemes such as knowledge gradient (KG) and expected improvement (EI).\nEmpirical results on multiple datasets indicate the superiority of EHIG on custom tasks considered.", "questions": "1. What distance function was considered for the diversity term in the experiments in Sec. 7 ? Was it Euclidean distance ? Did a parameter control the strength of the diversity penalty in eq. (4) ?\n\n2. Did the authors only consider two levels (Multihills) and one level (Pennsylvania Night Light) for the multi-level set estimation experiments ?\nOr were larger number of levels also considered ?\n", "limitations": "The authors discuss avenues for future work addressing implicit limitations in Sec 8.\n\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. The approach is clearly presented and well-motivated.\n\n2. The proofs in the appendix seem to convincingly demonstrate how EHIG yields multiple well-known special cases such as ES, KG and EI.\n\n3. Experimental validation is performed on a wide range of datasets.\n\nWeaknesses\n\n1. In the experimental comparison, only EHIG is tuned to the custom task and competing approaches are tuned to the conventional task of black box optimization (POM may be an exception in that it may be considered tuned for single level set estimation.)\nSo it appears to be a foregone conclusion that EHIG would outperform competitors.\nIt is unclear if any Bayesian Optimization approaches already exist for the custom tasks.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "H4bFFn1fhLa", "reviewer": "Reviewer_pn52", "summary": "The paper generalises Shannon entropy-based acquisition functions (AF) to a broader class of uncertainty measures. Doing so allows the authors to frame several popular AFs as special cases of the proposed entropy. The proposed AF is also able to provide customised solutions for a number of modified versions of BO setting. The authors study efficient optimization of the AF under certain smoothness conditions. Experimental evaluation compares the proposed method with several baselines.", "questions": "A few questions in addition to the concerns raised in the previous section:\n\nLine 115: How does the coin example help to motivate that this is a reasonable measure of uncertainty?\n\nExperiments section: How many trials did the authors average results over for the experiments?\n\nHow realistic is Eq 7 in general?\n\nHave the authors considered whether probability of improvement (the simplest BO AF one could imagine) can be obtained under the proposed framework?\n", "limitations": "-", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Post-rebuttal:\nI would like to thank the authors for answering my questions. Most of my concerns are addressed, so I will update my score to 6.\n__________\n\nThe paper is well written and appears to be technically sound. The idea of unifying the large zoo of BO acquisition functions under the same umbrella is neat (even though it was previously mentioned in the literature). It is particularly interesting to see both information-based and decision-theoretic AFs to be reached as special cases of proposed entropy.\n\nThe authors mention a diverse set of AFs for custom tasks. However, some of these applications seem somewhat artificially crafted. While the authors do provide a number of references, it’s not always clear on how such problems are derived from these references (e.g., line 221, reference 30 leads to a whole PhD thesis). Furthermore, with such limited description it's not always obvious why these tasks still require BO as an expensive black box optimization, and not some classic optimization techniques. It would be helpful if the authors could shed some more light on the motivation behind these custom tasks.\n\nThe experiments are one of the weaker points of the paper. While they do cover a diverse set of tasks, the set of methods is surprisingly small and lacks some “must-have” classic BO baselines, such as PI or GP-UCB. Even more surprisingly, EI, which is mentioned in the main paper as a special case of proposed entropy, is missing in the experimental section.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept