Joint Entropy Search for Multi-Objective Bayesian Optimization
Authors: Ben Tu
Axel Gandy
Nikolas Kantas
Behrang Shafei
Venue: NeurIPS 2022 Accept (2022)
Keywords: Multi-objective Optimization
Bayesian Optimization
Information-theoretic
Gaussian Processes

Abstract: Many real-world problems can be phrased as a multi-objective optimization problem, where the goal is to identify the best set of compromises between the competing objectives. Multi-objective Bayesian optimization (BO) is a sample efficient strategy that can be deployed to solve these vector-valued optimization problems where access is limited to a number of noisy objective function evaluations. In this paper, we propose a novel information-theoretic acquisition function for BO called Joint Entropy Search (JES), which considers the joint information gain for the optimal set of inputs and outputs. We present several analytical approximations to the JES acquisition function and also introduce an extension to the batch setting. We showcase the effectiveness of this new approach on a range of synthetic and real-world problems in terms of the hypervolume and its weighted variants.

Joint Entropy Search for Multi-Objective Bayesian Optimization
Ben Tu
Axel Gandy!
Nikolas Kantas
Behrang Shafei
"Imperial College London
|BASF SE
ben. won.tt1@@mmmeriall.co ac. uk
Abstract
Many real-world problems be phrased multi-objective optimization prob- lem, where the goal is to identify the best set of compromises between the compet- ing objectives. Multi-objective Bayesian optimization (BO) is sample efficient strategy that can be deployed to solve these vector- optimization problems where access is limited to number of noisy objective function evaluations. In this paper, we propose a novel information-theoreiic acquisition function for BO called Joint Entropy Search (JES), which considers the joint information gain for the optimal set of imputs and outputs. We present several analytical approximations the JES acquisition function and also introduce an extension the batch setting. We showcase the effectiveness of this new approach on a range of synthetic and real-world problems in terms of the hypervolume and its weighted variants.
Introduction
Bayesian optimization (BO) has demonstrated lot of success in solving black-box optimization problems in various domains such as machine learning [76, 77, 91], chemistry [27, 35], robotics [7, 14] and clinical trials [63, 79]. The procedure works by maintaining probabilistic model of the observed data in order to guide the optimization procedure into regions of interest. Specifically, at each iteration the black-box function evaluated at one more input locations that maximizes an acquisition function on the model. Implicitly, this function strikes a balance between exploring new areas and exploiting areas that have been shown be promising. In this work, we consider the more general problem, where the black-box function of interest is vector-valued. This increases the difficulty of the problem because there are now many directions in which the objectives can be improved, in contrast to the single-objective setting where there is only one. Informally, the end goal of objeetvb optimization is to identify a collection of points that describe the best trade-offs between the different objectives.
There are several ways to define acquisition function for multi-objective BO. A popular strategy random scalarization [51, 64], which works by transforming the multi-objective problem into a distribution of single-objective problems. These approaches are appealing because they enable the use of standard single-objective acquisition functions. weakness this approach that relies on random sampling encourage exploration and therefore the performance of this method might suffer early on when the scale of the objectives is unknown or when either the imput space or the objective space is nigh-dimessional [21, 64]. Another popular class of multi-objective acquisition functions are improvement-basee
These strategies focus on improving a performance metric over sets, for example the hypervolume indicator [18, 19, 26, 93] or the R2 indicator [24]. The main drawback of these approaches that the performance these methods can be biased towards a single performance metric, which be inadequate assess the multi-objective aspects of the problem [98]. There are also many other multi-objective acquisition functions discussed in the litera-
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

ture, which mainly differ by how they navigate the exploration-exploittiii trade-off [8, 92,2 68, 69].
Instead of relying on scalarizations or an improvement-based criterion, this paper considers the perspective where the goal of interest to improve the posterior distribution over the optimal points. We propose a novel information-theoreti acquisition function called the Joint Entropy Search (JES), which assesses how informative an observation will be in learning more about the joint distribution of the optimal inputs and outputs. This acquisition function combines the advantages of existing information-theoret
methods, which focus solely on improving the posterior of either the optimal inputs [31, 33, 39] or the optimal outputs [4, 6, 80]. We connect JES with the existing information -heeorti acquisition functions by showing that it is an upper bound to these utilities.
After acceptance of this work, we learnt of a parallel line of inquiry by Hvarfner et al. [46], who independently came up with the same JES acquisition function (3). Their work focussed on the single-objective setting and the approximation scheme they devised is subtly different to the ones we present. We see work being complementary to theirs because we both demonstrate the effectiveness of this new acquisition function in different settings.
Contributions and organization. In Section 2, we set up the problem and introduce the novel JES acquisition function. In Section 3, we present a catalogue of conditional entropy estimates to approximate this utility and present simple extension to the batch setting. These approximations analytically tractable and differentiable, which means that we can take advantage of gradient-base optimization. The main results that we developed here can be viewed as direct extensions to the recent work in the Bayesian optimization literature by Suzuki et al. [80] and Moss et al. [59]. In Section 4, present discussion the hypervolume indicator and explain how it can be misleading performance criterion because it is sensitive to the scale of the objectives. We show that inffommation-thooreti approaches are naturally invariant reparameterization of the objectives, which make them well-suited for multi-objective black-box optimization. For a more complete picture of performance, we propose novel weighted hypervolume strategy (Appendix K), which allows to assess the performance of a multi-objective algorithm over different parts of the objective space. In Section 5, we demonstrate the effectiveness of JES on some synthetic and real-life multi-objective problems. Finally in Section 6, we provide some concluding remarks. Additional results and proof's are presented in the Appendix.
2 Preliminaries
We consider the problem of maximizing a vector-valued function f X + RM over a bounded space of inputs X C R.. To define the maximum maxxex fx), we appeal to the Pareto partial ordering in M For the rest of this paper, will denote vectors by y (1(1), ,..,,, (MM) RM..
the non-negative real numbers by Rz0 and diagonal matrices by diag(-).
Pareto domination. We say a vector y RM weakly Pareto dominates another vector y' e RM if it performs just as well in all objectives if not better: 2 y < y-y's E IR-M Additionally, if the vectors are not equivalent, y * y', then we say strict Pareto domination holds: y + y y-y Ro \ {0m}. ). where O₂ is the M -i-imennsional zero vector. This binary relation can be M be sets, if the set B lies in the weakly dominated region of A, namely B C (A) = A RR : < a). then we say A weakly dominates B, denoted by A > B. In addition, if it also holds that the dominated regions are not equal, (A) # (B), we say strict Pareto domination holds, denoted by A > B.
further extended to define domination among sets. Let A, C R
Multi-objective optimization. The goal of multi-objective optimization is to identify the Pareto optimal set of inputs X* arg maxxex f(x) C X. The Pareto set is defined as the set of inputs whose objective vectors are not strictly Pareto dominated by another: x" E X" > x" E X and #x E X such that f(x) f(x"). The image of the Pareto set in the objective space Y f(x) = maxxex f(x) is called the Pareto front. For convenience of notation, we will denote the set of Pareto optimal input-output pairs by (X*, Y+).
Bayesian Optimization is a sample efficient global optimization strategy, which relies on a proba- bilistic model in order decide which points to query. In Appendix A.I, we present the pscudo-code

---

Pesecior Mackk.
Doliingg AAAA DLA
Orstioare Soiisssss Bo
Decombtrr
Samcin
Peeemer cntrape pypto. dil
Bale Mow. M.ra
x RbUs 0 a
Figure 1: Comparison of the samples (top) and change in entropy (bottom) for the posterior and conditional distributions. The red line in the posterior plots denotes the reference sample that is used to obtain the maximizer and maximum y², whilst the shaded blue region the 95% credible interval of the posterior p (x)( how correlated posterior probability that the objective surpasses y". Conditioning on (x", .y") leads to a drop in entropy based on both the input correlation with x² and the posterior probability of exceeding y".
Jr. |r, D.). Conditioning on x reduces the entropy for all inputs according to with x² Conditioning on y" reduces the entropy for all inputs according to the
for the standard BO procedure- for more details [13, 29, 75]. In this work, will use indepen- dent Gaussian process priors [71] on each objective, (m) ~ GP( _x(m)) where y ((m)) X R the mean function and ((m) X x X - IR is the covariance function for objective m. The observa- tions location where c ~ [O0 di (x))) denotes the observation noise with variance o (xx) E RRM. M evaluations, we have a data set D., = {(x,, )t) The posterior model p( D,) is collection
X will be assumed to be corrupted with additive Gaussian noise, y == fxx) c,
20" After n
of independent Gaussian processes [ GP( >.")). The explicit expressions for the mean and covariance are presented in Appendix A.2. The main focus of this work is on designing the arg max (x)
acquisition function, a I. which used select the inputs: X,+1
Information-theoret acquisition functions focus on maximizing the gain in information from the next observation and function of the probabilistic model. Initial work in BO focussed picking points to learn more about the distribution of the maximizer pD,) Specifically, the goal of interest was to maximize the mutual information between the observation y and the Pareto set X* conditional on the current data set D,:
(1) where [[)])]] = - p(x) logp(x)dd represents the differential entropy. This acquisition function commonly referred to as predictive entropy search (PES) [39, 40, 74). but was formerly' known entropy search (ES) [38, 84]. Despite the importance of obtaining more information about the maximizer, the PES acquisition function is heavily dependent on the approximation of p(y x. Dn, X*), which is both computationally difficult to implement and optimize. This motivated researchers to consider a simpler scheme that focusses on learning more about the distribution the maximum p ( [D,). The resulting acquisition function is known as the max-value entropy search (MES) [4, 44, 80, 86]:
PES PES (xl D,)) MI(y; X" x, D..) H[p(y|x, D.)) Ex.x [py())x (H.)x D,,XX'II
x
MES MES (xl MI(y; Y'|x, D.) H Hp(yy|x xx D,)] Ep(r D.)[H(p(s (x, Dn, Y"]]].
Unlike PES, the conditional probability p(y xx Dn, Y+) arising MES be approximated and optimized more easily because some approximations lead to closed-form expressions. Despite the favourable properties of MES, the primary goal of interest X and not necessarily the value of the maximum Y*. To combine the advantages of both of these approaches, we propose the joint entropy search acquisition function, which focusses on learning more about the joint distribution of the optimal points pXX",, Dx)):
identify the location of the maximizer
a JES D.) MI(y; (X*, Y [x, D.))
H [ply| x, D..)] - Ep(cx (x( (D.))
" [H[p(y|x D.. (X", Y*))]]-
'The difference in the naming convention stems solely from the approximation strategy used to estimate the mutual information. At a high level, ES applies expectation propagation [58] to estimate p(X') (X*|D. DD.. U {)) whilst PES applies expectation propagation estimate p(y x, Dn,

---

The JES acquisition function inherits the advantages of the PES and MES acquisition functions because it considers the knowledge learnt about the optimal points and is also simple to implement- more details in the next section. The following proposition shows that we can also interpret JES as an upper bound to both the PES and MES acquisition function.
Proposition 1. The JES is an upper bound to any convex combination of the PES and MES acquisition functions: LES (x) 2 3a PES (x) + (1 - 3)a MES (x D,), , for any 3 E [0, 1].
In Figure I, we illustrate the subtle differences between the different information- information-theoretiee acquisition functions. More specifically, we visualise the difference between the conditional distributions arising in each acquisition function for singlo-objective problem using sample of the optimal points.
Remark. In the BO literature it is common to distinguish between single-objective and multi- objective acquisition functions by appending 'MO' to the end of the acronym. For notational simplicity, we opt against this convention in this paper. In Appendix C, we emphasize the main differences that arise when computing the information-theorrtie algorithms in both settings.
3 Approximating JES
In this section, we present several approximations to the JES acquisition function (3) and a simple extension to the batch setting. The first term in the JES criterion (3) is the entropy of multivariate normal distribution:
M
M 2
H[p(y|x, Dn.) = log(2 +
Σ m=1
log( (m)) (x, x) + o (m) (x)).
(4)
2
The second term is an intractable expectation which is approximated by drawing Monte Carlo sam- ples from p((X,, Y*)| Dx)). The conditional entropy H [p(y|x, Dn, (X*, Y*)] is also an intractable
quantity which has to be estimated. The overall approximation of (3) will take the form
S (x) = H|ply|x, D.)) 5 Σ M(X), WD)xx,
(5)
where h denotes the conditional entropy estimate and (X., p((",, (X",,*)))) Dx) are the Monte Carlo samples. The distribution p(y|x, xx D,., (X)) is very challenging to work with because it enforces the global optimality condition that the function lies below the Pareto front f(X) - Y". Instead of enforcing global optimality, we make the common simplifying assumption as in [59, 80, 86] and only enforce the optimality condition the considered location: f(x) - Y By applying Bayes' theorem, the resulting density of interest becomes
ply|x, Dan, fxx 3 Y*) p([(x) 3 W x, Du+t)
p(y| Dns):
(6)
p(f(x) - 4 x, Drn))
where we have denoted the augmented data sets by Dn- = DA Y") and Do+ = Drs Uf(x }. We will refer to the quantity (x] Y*) as the cumulative distribution function (CDF). The following lemma shows that this CDF can be computed analytically when the set 4 c RM is discrete. This a standard result [16, 50, 68, 80] which can be derived by first partitioning the region of integration, D2 and then summing up the individual contributions see Figure for visual. This partition can be [M/2] computed using an incremental approach (Algorithm 1 of [55]), which has a cost of | + ).. In the single-objective setting, the maximum is single point y" E R and the box- box-decomposition is simply the interval ID ()v(")))) (-x,,y) y").
Uy+ {z E TR 2 I y"). into collection of hyperrectangle subsets
Lemma Lei Y C R be a finite set and a ~ (iag())) be an M-dimennsonnl mul- tivariate normal with mean a E R and variances b E RR Let (x*) U Bi =
U/= IM (1)m) (mi) be the box decomposition of the dominated space, then
J +
(m)
(m))
((m) (1))
p(z
Σ
②b
[blm]
Vblm)
j=1

---

B
B
B3
B-
Ba
Bi
Figure 2: Box decompositions for two-dimensional and three-dimensional Pareto front.
In Algorithm 1, we present the pseudo-code for the estimation of the JES acquisition function at a single candidate input. Several variables that calculated within the algorithm independent of the input (coloured in blue). For computational efficiency, we only compute these variables once and then save them memory for later use.
Algorithm 1: Joint Entropy Search (JES).
Input:
A candidate x; the data set D,--
// Cached variables are coloured in blue. Compute the initial entropy hg H[p(y/x, D,)].
2 for s ,,..., S do
Sample path f. p()
Compute the Pareto optimal points X; = arg max xx fo fx(x) and Y; = fx())
Compute the box decomposition Dy(())) U Bj-
Compute the conditional Dr, U (X;, )))).
Compute the estimate h2 h(( Y;)x;, xx, Dn).
end
return à JES [x) Dr)) = ho Ei=1le N1.
3.1
Estimating the conditional entropy
The entropy of (6) can be written M-dimensional expectation the multivariate normal distribution p(y|x, D..)) = =N(y; Mny (x), En. (x, x)):
H Hpyy)x Drs: f(x) = Y*]]
p(f(()) 7d x ) pff(x) N 1x, Dr
log p(f(x) 3 V D p(f(x) = Y+ x. Dn=)
p(y|x, pre)
(8)
-Epryyi,
To simplify the notation, define the m-th standardized value by
T' (2) = (z pmu (x))// (m)) (x,x)
(9)
for any scalar z E IR. Using this function together with Lemma 1, we denote the cumulative distribution (x 4 |x, Dr.)) by W = Σ W, Ej=1 I W where
j.m 1 (T (1) (m)) (17m(1)) (1))))
(10)
are the differences appearing in (7). Moreover, we denote the differences of the first derivative of W. jm and the negative of the second derivative (with respect 7m) by
Gj.m = (17mm (u)r)))) - (((m)(((())))) (m) j.m (m, ( ( (m) (m))) - 7m (m)) )o( (1(7(((1)))) (((m)))
(11)
(12)
where ò is the probability density function of standard normal distribution. In the setting where the observation noise zero, the conditional distribution a truncated multivariate normal, which known to have analytical equation formula for the entropy (Theorem 3.1. in [80]). In Appendix E,
5

---

we construct an ad hoc extension to this expression when the observation noise is non-zero.
In the noisy setting, the distribution of interest is type of multivariate skew normal distri- bution, which known to not have an analytical form for the entropy [1]. As a result, we propose two approximation strategies to estimate this entropy. The first strategy is to approximate the integral using Monte Carlo. The details of the Monte Carlo estimate h JES HES-MC described in Appendix F. The second strategy is to directly approximate the distribution with one that exhibits an analytical entropy. We consider the most obvious choice, which is a multivariate normal distribution with the same first two moments. The same strategy was proposed in [59] for the single-objective multi- multi-fidelity MES acquisition function. By a standard result (Chapter 12 of [17]), the entropy of this approximating distribution is actually N Y*)]= < MM log((Te) +
upper bound for the entropy of interest: result shows that the these central moments 2 log det Var(y|x x, Dne, f(xx N )*). The following be computed analytically.
H [p(y|| :
Proposition 2. Under the modelling set-up outlined in Section 2, for an input x E % the first and second central moment of p(y) x, Dres (xx 3 V*) are
((m) (X, x)
Gjmm
B Bly(m)) x Dno. f(x) 1 Y9 1/ (x)
W
W
W
j.m
and
(m)))) (mm0)) x, Donss fx) 3 )x)
Cov
Inm (xx,)) /xx/21(xxx))
(xxx) (x,,x)
=i Wj
Gam Wy.m
Gym'
1
Ei Wy
Gymm!
+ m';
W
Wyrm
(11r))) (x, x) + (m))
(x,xx)
W
Ej=1
Wj
Vim Wym
+ w
(Σ/=1
W
W
Gjm
m
m'.
As an upper bound on the conditional entropy leads to lower bound on the mutual information, will refer to the resulting conditional entropy estimate as the JES-LB estimate:
M
IES-LB ((X', x'); D,) log(2ne) + log det Var(y)x xx f()) Y*).
(13)
2
2
We could obtain a further lower bound by ignoring the off-diagonal terms in the covariance matrix.
We dub the resulting approximation as the JES-LB2 entropy estimate:
M
JES LB2 (X* Y'):x, D.)
A log(2 +
1
Σ m=
log (ar(y(m) xx, Dre, f(x) = Y*).
(14)
2
Figure 3 presents an illustration of the different density approximations that are used within the various conditional entropy estimates. An important remark is that all the conditional entropy estimates that we have developed here can also be applied to estimate MES. The only difference in the MES algorithm is that we no longer apply the conditioning step (line of Algorithm 1) because are interested in estimating H |ply| |x, Da, f(x) < Y*) as opposed to (8). Consequently, the MES acquisition function is cheaper to evaluate because the cost of evaluating the posterior variance
single input is O(n²). whereas JES incurs cost of O(n + 1)²))) more details are presented in the cost analysis in Appendix H.
3.2 Batch evaluations
..... Evaluating the JES acquisition functions for batch of points [(1: = (x......, x(9)) E X is expensive because the entropy estimates now depends on the g-dimensional normal CDF and its derivatives. To circumvent this issue, follow the example of [59] and propose a suboptimal batch approach by upper bounding the expensive joint conditional entropy term by the sum of the individual
entropies: H [pd [x() Da.,, f(x) < Y']] < Σ 1 [ Dne, f(x) I Y")]. The
resulting q-batch lower bound JES estimate is given by
S
GELB-EES (X [2:0l D..) (:gl (x, Dn)]
Σ
M((:), x!): Dw):
(15)
5

---

Sbow Meltve'nn Nomel
Inmaated Malivarale Nomel
Miltweruu Nomol (Full Covationn)
Maliondai Nomaal (Dhapceel Conatieed
(8)
(b)
(c)
(d)
Figure 3: Comparison of the density approximations to the skew multivariate normal distribution ply Dus: fx) = Y* shown in (a) for single input E X and sample Pareto front Y". A zero noise assumption leads to the truncated multivariate normal approximation shown (b), whilst a
moment matching approach leads to the multivariate normal approximations in (c) and (d).
where h is the conditional entropy estimate and
M H Hpp(yDal x D.)] log (2gg(2))+ +
1 Σ log det (E((r))
(x (x(1:s), (=2=))) d (m)) (x (16)
is the initial entropy. This acquisition function is defined over gDDdmimeniinnal space, which becomes more difficult to optimize as increases. Alternatively, we can maximize this function greedily by sequentially selecting the best input conditioned the previously chosen points. This greedy procedure turns out to be an effective strategy when the acquisition function is submodular [88]. In Appendix G, we show that this batch acquisition function is indeed submodular.
4 Performance criteria
In multi-objective optimization, the most common way to measure performance is by comparing the approximate Pareto set X* against the optimal Pareto set X* in the objective space: dXx(),) (X ((X))
where d: 2 x 2 - IR is function that measures the discrepancy between the sets of objective vectors. Existing work in multi-objective BO mainly focusses on the hypervolume (HV) discrepancy. dav (A, B) = [Unv (A) - Unv (B)). where the HV indicator, Unv = Spar 1r N z = Aldz, is
defined as the volume between reference point E RM and a set A C R The general guidance to set reference point to slightly worse than the nadir, which the vector consisting of the worst possible points, minxex ((m) (x), for objectives m 1, I,..., see [47] for more details.
attractive feature of the HV indicator is that it Pareto complete (or compliant) in sense that better set will lead assume the sets A and B are finite. The reverse implication known as Pareto compatibility does not hold for the HV indicator [98]. In other words, the HV can be used to discriminate between sets where one dominates another, but it cannot be relied upon when the sets are incomparable. Not all incomparable sets are treated equally by the HV indicator [96]. For instance Figure 4a shows example where the HV indicator places more emphasis on the end points the Pareto front. On other hand, if we apply monotonically increasing transformation Im R - R to each objective, the Pareto set will not change, whereas the HV comparison will (Figure 4b). Implicitly, the HV indicator assumes that linear change in one objective is equivalent to linear change in another.
larger HV [98]: > B
(Hv(A) > UHv (B), if
This assumption might not necessarily reflect the decision maker's 's outlook and this is something that typically overlooked when designing and benchmarking multi-objective optimization algorithms. The following result shows that coormation-thoorede acquisitions function are in fact agnostic to the choice of parameterization.
acquisition functions o PES MES and o JES are invariant to reparameterization of the objective space that are consistent with the Pareto ordering relations. For example, a JES (x) ) MI(y; (X*, ""') = MI(g(y); ); (X*, (x",g(""" g(Y"")) where the 9m IR + strictly monotonically increasing function acting only on the m-th objective.
Proposition 3. The information-theoree
To benchmark the algorithms, we use both the standard HV discrepancy (Section 5) and the HV discrepancy under different parameterizations (Appendix L). To casily obtain family of parameteri-
7

---

Oliective space
Oftective space
Transfomed dbjective space Pwe 10
Traesformed objoctive space (Be =0.92)
Cier 0.669
Gvv 0464
Keitone pazt Sumple Parck frex
Dondrand rijlaa
yd
(b)
Figure 4: Comparison of the HV of two sample Pareto fronts in (a) the standard objective space y = (y(") y and (b) the transformed objective space g(y) = (g (v()), 92 described in
Appendix K. The HV indicator prefers different set depending the choice of parameterization.
zations, we devise a novel weighting approach in Appendix K. which exploits the fact that the HV indicator can be written as an expectation over uniform distribution on the (M - -dddmmessoon hypercube [23, 94]. We observe that it is possible to assess the performance at different locations of the objective space by using alternate distributions over the hypercube. We call the resulting metric the generalized hypervolume (GHV). In our experiments, we found that the performance of each algorithm changed with regards to the choice of parameterization, but the JES approaches tended to be one of the strongest performers throughout.
5 Experiments
We empirically evaluate the JES acquisition function range of synthetic and real-world benchmark problems. We compare this approach with some popular acquisition functions in multi-objective
BO: TSEMO [12], ParEGO [51]. NParEGO [19], EHVI [18], NEHVI [19], PES [31, 33] and MES-0 [80]. We have also included the MES-LB, MES-LB2 and MES-MC acquisition functions, which can be easily derived from the conditional entropy estimates that we developed here. All algorithms are based on the open source Python library BoTorch [3], which uses features from GPy Torch [30] for Gaussian process regression and Py Torch [66] for automatic differentiation. All experiments are repeated using 100 different initial seeds and we generate the Pareto set recommendation X* of 50 points by maximizing the posterior mean using multi-objective solver (NSGA2 [22] from the Pymoo library [10]). The complete details of the experiments are outlined in Appendix I., whilst the code is available at https://gihubb.co//bmmluu
5.1 Benchmarks
Synthetic benchmark. We consider the ZDT2 [22] benchmark with D = 6 inputs and M = 2 objectives. We corrupt the observations with additive Gaussian noise with zero- zero-mean and standard deviation set to approximately 10% of the objective ranges.
Chemical reaction. This benchmark considers a nucleophilic aromatic substitution reaction (SnAr) between 2.4-dittoroniirobenzeeee and pyrrolidine in ethanol to produce mixture of desired product and two side-products [45]. The design space comprises of D 4 components relating to the initial conditions. The goal is to optimize M = 2 objectives, namely the space time yield and the environmental impact. We apply logarithm transform the objectives and contaminate the observations with additive Gaussian noise with 3% of the resulting objective ranges in order emulate potential real-world scenario.
and standard deviation set to approximately
Pharmaceutical manufacturing. This problem is concerned with optimizing the Penicillin pro- duction process outlined in [56]. The design space is made up of D = elements that control the initial condition of the reactions. The goal is to optimize M 3 objectives, which relates to the yield, the amount of carbon dioxide released and the time to ferment. We include additive zero-mean Gaussian noise with a standard deviation set approximately 1% of the objective ranges.

---

2012 (boet, Mes. Note-loR (-1)
Saft (Dmm) Mod, Neize gel)
Panallia (Dw)) MrS, Natzei ge1)
Masine Dmm, Med Novg 55
355
4.0
360
12.41
273
265
25
Pesare
Soholl
TSEMO
NPaBBOO
NEHVI
PES
MES-L8
IES-L8
(Dwo, Mr2 Nate- (05)
(Dmil
Pezicilla Nelew!
(DoG,
2250
2271
Bachvbc
Sak3 Aoc
Sobol
TSEMO
NPuEGO
NERVI
PES
LB-MES-L3
LBJES-LB
Figure 5: A comparison of the mean logarithm HV discrepancy with two standard errors over one hundred runs for the four benchmark problems on subset of the algorithms. We present the sequential and batch results on the top and bottom, respectively.
Marine design. This problem considers optimizing a family of bulk carriers subject to the con- straints imposed for ships travelling through the Panama Canal [65, 73]. The design space is made up of = 6 variables that determine the architecture of the carriers. The goal of this problem is to maximize the annual cargo, whilst minimizing the transportation cost and the ship weight subject to some design constraints. We consider the reformulation in [83], which converts the constraints into another objective. For this reformulated M objective problem, we corrupt the observations with additive zero-mean Gaussian noise with standard deviation set to approximately 0.5% of the objective ranges.
5.2 Results and discussion
We present the log HV discrepancy results for both the sequential and batch experiments in Figure 5. The JES approach is consistently one of the stronger performing algorithms for these set of experi- ments. A similar conclusion is reached when we consider the weighted variant of the hypervolume in Appendix L.8.
Conditional entropy estimates. We compared the performance of the different conditional entropy estimates for both the JES and MES acquisition function in Appendix L.6. We observed that the majority of cases all the estimates exhibit similar performance. As result, recommend using the cheapest approximation, which is usually the lower bound estimates, judging from the wall times presented in Appendix L.9.
The wall times in Appendix L.9 indicate that the cost of acquiring a new point with JES is comparable with NEHVI, slightly more expensive than MES, but cheaper than PES. We note that the wall times for all methods can be improved by taking advantage of parallelization. In particular, for entropy based methods we used a gradient-free optimizer sequentially optimize the multi-objective samples (line 4 in Algorithm 1), whereas in practice we should of ideally solved these problems in parallel using a gradient-based optimizer such as [57].
Acquisition wall times.
Querying high performing points. In certain domains it might be useful to directly query high- performing points because the final decision will be restricted only the sampled locations Xy- In Appendix L.5, we investigated the performance when such restriction was made. In this setting, we observed that the infformation-theoretic approaches were occasionally outperformed by the improvement and scalarization based acquisition functions, which picked points more greedily. We observed that the entropy based approaches had tendency to pick points that are more informative for the posterior over the optimal points opposed to directly selecting point that is known to

---

2D12 (de6, Med, Noie=10K, qe1)
Regien
Regioe
Rozien
Sobel
TSEMO
NPr8GO
NELVI
PES
MES-L8
JES-LB
Sobol
TSEMO
NParEGGO
NEEVI
PES
MES-LB
JES-LB
jogeduv) = 3.14
logidry> 191
jogtdgy)) 232
logidry> 238
jogtduy 233
Tog = 163
jogidgy ( L14
Pareto front:
Mean voctors: μ(R')
Objective vectors: yt8')
Figure 6: An example of the generalized hypervolume the ZDT2 benchmark. On the top plot present the mean logarithm GHV discrepancy results for three different regions. On the bottom plot, we present the Pareto front and its approximation at the final time for the run which achieved the top 20th percentile on the standard hypervolume.
perform well. To address this setting, we recommend combining information theoretic acquisition functions with an epsilon greedy approach, where points are occasionally picked according greedy strategy such as maximizing a function the posterior mean.
Assessing local performance. Using the generalized hypervolume, we can target different parts of the objective space in order to get a much better picture of performance. We demonstrate this on simple bi-objective example in Figure 6, where we assess that quality the approximations at three different regions of the objective space. We observe that all of the BO algorithms were quickly able to identify the right section of the Pareto front. Evidently, the main source difficulty for this problem arises from approximating the points in the left section of the Pareto front, which favours the second objective. This observation would not be apparent we focussed solely on the standard HV.
General guidance. The ideal acquisition functions is problem dependent and strongly depends the decision maker's plans and goals. In completely black-box setting, where there is no immediate preferences, Proposition 3 and the empirical results motivates the usage of information -theoretccc acquisition functions, which treats all points on the Pareto front as equally desirable a priori.
6 Conclusion
We introduced JES, a novel information-theorede acquisition function for multi-objective BO. To approximate this acquisition function, we presented several approximations the conditional entropy and also simple extension for the batch setting. Experimental results suggest that JES is very competitive with existing acquisition functions terms of the HV diserepancy and its weighted variants. The main limitation of the JES acquisition function is that it relies on routines such as box decompositions and mujti-objective optimization of function samples, which can be expensive to execute for very large problems. Future work could focus on improving the scalability of these information - methods and extending it to more general settings, which include constrained, decoupled and nulti-fidelity optimization see Appendix M for more details.
Acknowledgments and Disclosure of Funding
BT was supported by the EPSRC StatML CDT programme EP/S023151/1 and BASF SE, Lud- wigshafen am Rhein. NK was partially funded by JPMorgan Chase & Co. under J.P. Morgan A.I. Faculty Research Awards 2021.
10

---

References
[1] Reinaldo B. Arellano-Vall Javier E. Contreras Contreras-Reyes, and Mare G. Genton. Shannon Entropy and Mutual Information for Multivariate Skew-Elliptiaal Distributions, Scandinavian Journal of Statistics, 2013. Cited on page 6.
[2] Adelchi Azzalini. A Class of Distributions Which Includes the Normal Ones. Scandinavian Journal of Statistics, 1985. Cited on page 22.
[3] Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. In Advances in Neural Information Processing Systems. 2020. Cited on pages 8, 33, and 36.
[4] Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Max-value Entropy Search for Multi- Objective Bayesian Optimization. In Advances Neural Information Processing Systems, 2019. Cited on pages 2, 3, 19, 20, and 37.
[5] Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Multi-Fidelity Multi- Objective Bayesian Optimization: Output Space Entropy Search Approach. Proceedings of the AAAI Conference on Artificial Intelligence, 2020. Cited on page 49.
[6] Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Output Space Entropy Search Framework Multi Mojti-Obeectiveeee Bayesian Optimization. Journal of Artificial Intelligence Research, 2021. Cited on pages 2 and 49.
Felix Berkenkamp, Andreas Krause, and Angela Schoellig. Bayesian optimization with safety Con- straints: Safe and automatic parameter tuning robotics. Machine Learning, 2021. Cited on page 1.
[8] Mickael Binois, Victor Picheny, Patrick Taillandier, and Abderrahmane Habbal. The Kalai Smoroddkoyyy solution for many-objective Bayesian optimization. Journal of Machine Learning Research, 2020. Cited on page 2.
[9] Mickael Binois, Abderrahmanc Habbal, and Victor Picheny. A game thcoretic perspective Bayesian multi-objective optimization. arXiv:2104. 2021. Cited on page
[10] Julian Blank and Kalyanmoy Deb. Pymoo: Multi-Objective Optimization in Python. IEEE Access, 2020.
Cited pages and 33.
[11] Salomon Bochner, Monotonic Functions, Stieltjes Integrals, Harmonic Analysis, Morris Tenenbaum, and Harry Pollard. Lectures Fourier Integrals. Princeton University Press, 1959. Cited page 18.
[12] Eric Bradford, Artur M. Schweidtmann, and Alexei Lapkin. Efficient multiobjective optimization employ- ing Gaussian processes, spectral sampling and genetic algorithm. Journal of Global Optimization, 2018. Cited pages 8, 18, and 35.
[13] Eric Brochu, Vlad M. Cora, and Nando de Freitas. A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning. arXiv:1012.2599, 2010. Cited on pages and 18.
[14] Roberto Calandra, André Seyfarth, Jan Peters, and Marc Peter Deisenroth. Bayesian optimization for lcarning gaits under uncertainty. Annals of Mathematics and Arrificial Intelligence, 2016. Cited page 1.
[15] Emile Contal, David Buffoni, Alexandre Robicquet, and Nicolas Vayatis. Parallel Gaussian Process Optimization with Upper Confidence Bound and Pure Exploration. In Machine Learning and Knowledge Discovery in Databases, Lecture Notes in Computer Science, 2013. Cited on page 30.
[16] Ivo Couckuyt, Dirk Deschrijver, and Tom Dhaene. Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization. Journal of Global Optimization, 2014. Cited on page 4.
[17] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley-Interscience, 2nd edition edition, 2006. Cited pages 6, 19, and 21.
[18] Samuel Daulton, Maximilian Balandat, and Fytan Bakshy. Differentiable Expected Hypervolume Improve- ment for Parallel Multi-Ohjective Bayesian Optimization. In Advances in Neural Information Processing Systems. 2020. Cited pages 1, 8, 34, and 35.
[19] Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement. In Advances in Neural Information Processing Systems, 2021. Cited pages 1, 8, and 35.

---

[20] Samuel Daulton. Sait Cakmak, Maximilian Balandat, Michael A. Osborne, Enlu Zhou, and Eytan Bakshy. Robust Multi-Objectiin Bayesian Optimization Under Input Noise. In International Conference on Machine Learning. 2022. Cited on page 36.
[21] Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy, Multi-Ohjective Bayesian Optimization over High--Dimensionl Search Spaces. In Uncertainty in Artificial Intelligence, 2022. Cited on page I.
[22] Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and T. Mcyarivan. Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 2002. Cited on pages 8, 28, and 34.
[23] Jingda Deng and Qingfu Zhang. Approximating Hypervolume and Hypervolume Contributions Using Polar Coordinate. IEEE Transactions Evolutionary Computation, 2019. Cited on pages 8 and 32,
[24] André Deutz, Michael Emmerich, and Kaifeng Yang. The Expected R2-Indicator Improvement for Multi-ohjective Bayesian Optimization. In Evolutionary Multi-Criterion Optimizationn Lecture Notes in Computer Science, 2019. Cited on page I.
[25] Kun Dong, David Briksson, Hannes Nickisch, David Bindel, and Andrew G Wilson. Scalable Log Determinants for Gaussian Process Kernel Learning. In Advances in Neural Information Processing Systems. 2017. Cited on page 28.
[26] Michael T. M. Emmerich, Kyriakos C. Giannakoglou, and Boris Naujoks. Single- and Multiobjective Evolutionary Optimization Assisted by Gaussian Random Field Metamodels. IEEE Transactions on Evolutionary Computation, 2006. Cited on page 1.
[27] Kobi C. Felton, Jan G. Rittig, and Alexei A. Lapkin. Summit: Benchmarking Machine Learning Methods for Reaction Optimisation. Chemistry Methods, 2021. Cited pages I and 36.
[28] Daniel Femnnndez-Sánchez, Eduardo C. Garrido-Merchán, and Daniel Hernández-LLobto. Improved Max-value Entropy Scarch for Multi-objective Bayesian Optimization with Constraints. arXiv:201 150, 2021. Cited on page 49.
[29] Peter Frazier. A Tutorial on Bayesian Optimization. arXiv:1807, 028111 2018. Cited on pages 3 and 18.
[30] Jacob Gardner, Geoff Pleiss, Kilian Weinberger. David Bindel, and Andrew G Wilson. GPyTorchi Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration. Advances in Neural Information Processing Systems. 2018. Cited on pages 8, 28, and 33.
[31] Eduardo C. Marrido-Merchal and Daniel Hermández-LLoato. Predictive Entropy Search for Multi-ohjective Bayesian Optimization with Constraints. Neurocompuring 2019. Cited on pages 2, 8, 20, 28, 29, 34, 35, 37, and 49.
[32] Eduardo C. Garrido-Merehán and Daniel Hernández-Lobato. Dealing with categorical and integer-valuee variables in Bayesian Optimization with Gaussian processes. Neurocomputing, 2020. Cited on page 49,
[33] Eduardo C. Garrido-Merchán and Daniel Hernández-Lobato, Parallel Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints. arXiv:2004.0000, 2021. Cited on pages 8, 29, 35, and 49.
[34] Michael A. Gelbart, Jasper Snoek, and Ryan P. Adams. Bayesian Optimization with Unknown Constraints. In Uncertainty in Artificial Intelligence, 2014. Cited on page 49.
[35] Rafael Gómez-Bonee Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernndez- Benjamín Sánchez- Lengeling, Dennis Sheberla, Jorge Aguilera- Iparrauurre,, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Gun Guzik. Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules. ACS Centrul Science, 2018. Cited on page 1.
[36] Robert B. Gramacy, Annie Sauer, and Nathan Wycoff. Triangulation candidates for Bayesian optimization. arXiv:2112.07457 2021. Cited page 18.
[37] Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer. Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchann Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 2020. Cited page 33.
12

---

[38] Philipp Hennig and Christian J. Schuler. Entropy Search for Information-Efficiin Global Optimization. Journal of Machine Learning Research, 2012. Cited page 3.
[39] Daniel Hernandez-Lobate Jose Srrnandez-LLbbttee Amar Shah, and Ryan Adams. Predictive Entropy Search for Multi-bjjective Bayesian Optimization. In International Conference on Machine Learning. 2016. Cited on pages 3, 35, and 49.
[40] José Miguel Sermández-Lobanee Matthew W Hoffman, and Zoubin Ghahramani. Predictive Entropy Search Efficient Global Optimization of Black-box Functions. In Advances in Neural Information Processing Systems. 2014, Cited on pages 3, 18, 30, and 35.
[41] Jose Miguel Hernadeez-Loaan Michael Gelbart, Matthew Hoffman, Ryan Adams, and Zoubin Ghahra- mani. Predictive Entropy Search for Bayesian Optimization with Unknown Constraints. In International Conference on Machine Learning. 2015. Cited on page 49.
[42] José Miguel Hernández-Lohaton Michael A. Gelhart, Ryan P. Adams, Matthew W. Hoffman, and Zoubin Ghahramani. A General Framework for Constrained Bayesian Optimization using Information- Search. Journal of Machine Learning Research, 2016. Cited page 49.
[43] Fred J. Hickemell, Christiane Lemieux, and Art B. Owen. Control Variates for Quasi-Monte Carlo. Statistical Science, 2005. Cited on page 26.
[44] Matthew W Hoffman and Zoubin Ghahramani. Output-Space Predictive Entropy Search for Flexible Global Optimization. In NIPS Workshop on Bayesian Optimization, 2015, Cited on page 3.
[45] Christopher A. Hone, Nicholas Holmes, Geoffrey R. Akien, Richard A. Bourne, and Frans L.. Muller. Rapid multistep kinetic model generation from transient flow data, Reaction Chemistry & Engineering. 2017. Cited on pages 8 and 36.
[46] Carl Hvarfner, Frank Hutter, and Luigi Nardi. Joint Entropy Search For Maximally-Informed Bayesian Optimization. (eXii:2206604777 2022. Cited page 2.
[47] Hisao Ishibuchi. Ryo Imada, Yu Setoguchi, and Yusuke Nojima. How to Specify Reference Point in Hypervolume Calculation for Fair Performance Comparison. Evolutionary Computation, 2018. Cited on page 7.
[48] Kirthevasan Kandasamy, Gautam Dasarathy, Jeff Schneider, and Barnabás Póczos. Multi-fidelit Bayesian Optimisation with Continuous Approximations. In International Conference Machine Learning. 2017.
Cited on page 49.
[49] Tarun Kathuria, Amit Deshpande, and Pushmeet Kohli. Batched Gaussian Process Bandit Optimization Determinantal Point Processes. Advances in Neural Information Processing Systems. 2016. Cited on page 30.
[50] Andy J. Keane. Statistical Improvement Criteria for Use in Multiobjective Design Optimization. AIAA Journal, 2012. Cited on page 4.
[51] Joshua Knowles. ParEGO: A Hybrid Algorithm With On-I Line Landscape Approximation for Expensive Multiobjective Optimization Prohlems. IEEE Transactions on Evolutionary Computation, 2006. Cited on pages I, 8. and 35.
[52] Mina Konakovic Lukovic, Yunsheng Tian, and Wojeiech Matusik. Diversity-Guided Multi-Objective Bayesian Optimization With Batch Evaluations. In Advances in Neural Information Processing Systems. 2020. Cited on page 2.
[53] Andreas Krause and Daniel Golovin. Submodular Function Maximization. In Tractability, pages 71-104. Cambridge University Press, Cambridge, 2013. Cited on page 27.
[54] Alex Kulesza and Ben Taskar. Determinantal point processes for machine learning. Foundations and Trends@ Machine Learning, 2012. Cited pages 27 and 30.
[55] Renaud Lacour, Kathrin Klamroth, and Carlos M. Fonseca. A box decomposition algorithm to compute the hypervolume indicator. Computers Operations Research, 2017. Cited on pages 4, 28, and 34.
[56] Qiaohao Liang and Lipeng Lai. Scalable Bayesian Optimization Accelerates Process Optimization of Penicillin Production. In NeurlPS 202/ Al for Science Workshop, 2021. Cited pages 8 and 36.
[57] Xingchao Liu, Xin Tong, and Qiang Liu. Profiling pareto front with multi- objei--beeecive stein variational gradient descent. In Advances in Neural Information Processing Systems. 2021. Cited on page 9.
13

---

[58] Thomas P. Minka. Expectation Propagation for Approximate Bayesian Inference. In Uncertainty in Artificial Intelligence, 2001. Cited on pages 3 and 28.
[59] Henry B. Moss, David S. Leslie, Javier Gonzalez, and Paul Rayson. GIBBON: General-purpose Information-Basse Bayesian Optimisation. Journal of Machine Learning Research, 2021. Cited on pages 2, 4. 6, 19, 27, and 49.
[60] Henry B. Moss, David S. Leslie, and Paul Rayson. MUMBO: MUlti-task Max- Value Bayesian Optimiza- tion. In Machine Learning and Knowledge Discovery in Databuses, Lecture Notes in Computer Science, 2021. Cited on pages 19 and 49.
[61] Willie Neiswanger, Ke Alexander Wang, and Stefano Ermon. Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information, In International Conference on Machine Learning. 2021. Cited on page 19.
[62] Dang Nguyen, Sunil Gupta, Santu Rana, Alistair Shilton, and Svetha Venkatesh. Bayesian Optimization for Categorical and Category-Speciff Continuous Inputs. Proceedings of the AAAI Conference on Artificial Intelligence, 2020. Cited on page 49.
[63] Majid Nour, Zafer Cömert, and Kemal Polat. A Novel Medical Diagnosis model for COVID-19 infection detection based on Deep Features and Bayesian Optimization. Applied Soft Computing. 2020. Cited on page 1.
[64] Biswajit Paria, Kirthevasan Kandasamy, and Barnabás Póczos. A Flexible Framework for Multi-O Aulti-Ohjective Bayesian Optimization using Random Scalarizations. In Uncertainty in Artificial Intelligence. 2020. Cited page 1.
[65] Michael G. Parsons and Randall I.. Scott. Formulation of Multicriterion Design Optimization Problems for Solution With Scalar Numerical Optimization Methods, Journal of Ship Research, 2004. Cited on pages 9, 36, and 37.
[66] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary VeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems. 2019, Cited on pages 8, 33, and 34.
[67] Valerio Perrone, laroslav Shcherbatyi, Rodolphe Jenatton, Cedric Archambeau, and Matthias Seeger. Constrained Bayesian Optmization with Max- Value Entropy Search. NeurIPS Workshop on Meta- Meta-Learning 2019. Cited on page 49.
[68] Victor Picheny. Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reduction. Statistics and Computing, 2015. Cited on pages and 4.
[69] Victor Picheny, Mickael Binois, and Abderrahmane Habbal. A Bayesian optimization approach to find Nash equilibria. Journal of Global Optimization, 2019. Cited on page 2.
[70] Ali Rahimi and Benjamin Recht. Random Features for Large-Scale Kernel Machines. In Advances in Neural Information Processing Systems. 2008 Cited on page 18.
[71] Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. Adaptive Computation and Machine learming MIT Press, 2006. Cited on pages and 18.
[72] Binxin Ru, Ahsan Alvi, Vu Nguyen, Michael A. Osborne, and Stephen Roberts. Bayesian Optimisation over Multiple Continuous and Categorical Inputs. In International Conference on Machine Learning. 2020. Cited on page 49.
[73] Pratyush Sen and Jian-Bo Yang. Multiple Criteria Decision Support in Engineering Design. Springer London, 1998. Cited on pages 9, 36, and 37.
[74] Amar Shah and Zoubin Ghahramani. Parallel predictive entropy search for batch global optimization of expensive objective functions, In Advances in Neural Information Processing Systems. 2015. Cited on page 3.
[75] Bobak Shahriari, Kevin Swersky, Ziyu Wang. Ryan P. Adams, and Nando de Freitas. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proceedings of the IEEE, 2016. Cited on pages 3 and 18.
14

---

[76] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical Bayesian Optimization of Machine Learning Algorithmss. In Advances in Neural Information Processing Systems. 2012. Cited on page 1.
[77] Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable Bayesian Optimization Using Deep Neural Networks. In International Conference on Machine Learning. 2015. Cited on page
[78] Jialin Song, Yuxin Chen, and Yisong Yue. A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes. In International Conference on Artificial Intelligence and Statistics. 2019. Cited on page 49.
[79] Yanan Sui, Vincent Zhuang, Jocl Burdick, and Yisong Yue. Stagewisc Safe Bayesian Optimization with Gaussian Processes. International Conference on Machine Learning. 2018. Cited on page
[80] Shinya Suzuki, Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Multi- objective Bayesian Optimization using Parcto-frontier Entropy. In International Conference on Machine Learning. 2020. Cited on pages 3, 4, 5, 8, 19, 20, 25, 35, 37, and 49.
[81] Shion Takeno, Hitoshi Fukuoka, Yuhki Tsukada, Toshiyuki Koyama, Motoki Shiga, Ichiro Takeuchi, and Masayuki Karasuyama. Multi-fidelit Bayesian Optimization with Max-value Entropy Search and its Parallelization. In International Conference on Machine Learning, 37. 2020. Cited on pages 19 and 49,
[82] Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound. arXiv:2102.00788, 2021. Cited on page 49.
[83] Ryoji Tanabe and Hisao Ishibuchi. An Easy-to-use Real-world Multi-objective Optimization Problem Suite. Applied Soft Computing, 2020. Cited on pages 9, 36, and 37.
[84] Julien Villemonteix, Emmanuel Vazquez, and Fric Walter. An informational approach the global optimization expensive-to-cvaluat functions. Journal of Global Optimizaaiin, 2008 Cited on page 3.
[85] Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weekesser, Jonathan Bright, Stéfan J. van Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C. J. Carey, ilhan Polat, Yu Feng. Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, F. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, and Paul van Mulbregt. SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 2020. Cited on pages 33 and 34.
[86] Zi Wang and Stefanie Jegelka. Max-value Entropy Search for Efficient Bayesian Optimization. In International Conference on Machine Learning. 2017. Cited on pages 3, 4, 18, 20, 35, and 37.
[87] Zi Wang. Chengtao Li, Stefanie Jegelka, and Pushmeet Kohli. Batched High-dimensional Bayesian Optimization via Structural Kernel Learning. International Conference on Machine Learning. 2017. Cited on page 30.
[88] James Wilson, Frank Hutter, and Marc Deisenroth. Maximizing acquisition functions for Bayesian optimization. Advances in Neural Information Processing Systems. 2018 Cited on pages 7, 26, and 28.
[89] James Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Mare Deisenroth. Efticiently Sampling Functions from Gaussian Process Posteriors. In International Conference on Machine Learning. 2020. Cited on pages 18, 19, and 27.
[90] James T. Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Mare Peter Deisenroth. Pathwise Conditioning of Gaussian Processes. Journal of Machine Learning Research, 2021. Cited on pages 18 and 19.
[91] Wu. Xiu-Yun Chen, Hao Zhang, Li-Dong Xiong, Hang Lei, and Si-Hao Deng. Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization. Journal of Electronic Science and Technology, 2019. Cited on page 1.
[92] Jian Wu, Saul Palmmrin,,, Peter Frazier, and Andrew Gordon Wilson. Practical Multi-iidelity Bayesian Optimization for Hyperparameter Tuning, In Uncertainty in Artificiol Intelligence. 2020. Cited on page 49.
[93] Kaifeng Yang. Michael Emmerich, André Deutz, and Thomas Bäck. Efficient computation of expected hypervolume improvement using box decomposition algorithms. Journal of Global Optimization, 2019. Cited page I.
15

---

[94] Richard Zhang and Daniel Golovin. Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimization. In International Conference on Machine Learning. 2020. Cited pages and
[95] Yehong Zhang. Trong Nghia Hoang, Bryan Kian Hsiang Low, and Mohan Kankanhalli, Information- Based Multi- Fideliy Bayesian Optimization. NIPS Workshop on Bayesian Optimization, 2017. Cited on page 49,
[96] Eckart Zitzler and Lothar Thiele. Multiobjective Optimization Using Evolutionary Algorithms Comparative Case Study, Parallel Problem Solving from Nature, Lecture Notes Computer Science, 1998. Cited on page 7.
[97] Eckart Zitzler, Kalyanmoy Deb, Lothar Thiele. Comparison of Multiobjective Evolutionary Algorithms: Empirical Results. Evolutionary Computation. 2000. Cited on page 36.
[98] Eckart Zitzler, Lothar Thiele, Marco Laumanns, Carlos Fonseca, and Viviane Grunert Fonseca. Performance Assessment of Multiobjective Optimizers: An Analysis and Review. IEEE Transactions on Evolutionary Computation, 2003. Cited on pages and 7.
[99] Eckart Zitzler, Dimo Brockhoff, and Lothar Thiele. The Hypervolume Indicator Revisited: On the Design of preto-complian Indicators Via Weighted Integration. In Evolutionary Lecture Notes in Computer Science, 2007. Cited page 33.
Optimization,
16

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] The proofs for the main results are in the appendix.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material as URL)? [Yes] The code is available at
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] The experimental set-up described in Appendix L.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] The computing resources are described in the caption of the wall times plots e.g. Appendix L.9.
4. If you using existing assets (e.g., code, data, models) or curating/releasing
assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [No] We only cited the original code and corresponding manuscript in Appendix L.
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
17


Reviews:
[{"review_id": "nlwUP-QFiSv", "reviewer": "Reviewer_XDbZ", "summary": "In this paper, a Bayesian optimization algorithm using Joint Entropy Search (JES), a new entropy search type acquisition function, is proposed to efficiently estimate the Pareto solution set for multi-objective optimization problems. JES is defined as the mutual information between the pair of optimal solution set and the optimal value set,  (X^*, Y^*) and the new query point (x, y). While PES and MES, the main conventional entropy search type acquisition functions, focus only on the marginal distribution of the optimal solution and optimal value, respectively, JES determines the next acquisition point by considering their joint distribution. \n\nJES can be expressed as the difference of two differential entropies as in other entropy search type acquisition functions. For its optimization, the authors proposed a method to approximate the expected value of entropy for the pair (X^*, Y^*) of the optimal solution set and the optimal value set that appears in the second term of the JES acquisition function. Specifically, when there is no observation noise, entropy can be calculated analytically since the argument distribution is a multivariate truncated normal distribution, and when there are observation error, the entropy of the multivariate normal distribution that bounds it from above can be used as an approximation. Specifically, when there is no observation noise, entropy can be calculated analytically since the argument distribution is a multivariate truncated normal distribution. When there are observation errors, a multivariate normal distribution whose 1st and 2nd moments are equivalent to the argument distribution can be used as an approximation. This is because the entropy of latter distribution is bounded above by the entropy of former one.\n\nThe proposed method, the JES acquisition function, is also extended to the batch optimization setting. Specifically, the authors proposed a method in which the entropy of the joint distribution for the batch points is bounded from above by the sum of entropies for each point, and the latter is used to select each point independently.\n\nIn the experiments, the proposed method is compared with existing multiobjective Bayesian optimization methods for different optimization problems with 2-, 3-, and 4-objectives, using the difference of hypervolumes as an evaluation criteria.", "questions": "- As pointed out in Weakness, the assumption that Y^* is a finite set, as placed in Lemma 1, seems to be a strong one; it seems to be the key lemma for the approximate computation of the JES acquisition function, but is it not an issue?\n\n- As pointed out in weakness, since the proposed method decomposes all computations into point-by-point computations, I think that it needs to be carefully evaluated in terms of computational cost compared to existing methods. Have you done any comparison of actual computational costs?", "limitations": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n- The proposed new entropy-search type acquisition function, JES, considers the joint distribution of optimal position x^* and optimal value y^*, and is expected to provide more information about the optimal solution than PES, which considers only the former, or MES, which considers only the latter. \n\n- Different computation methods have been proposed for the entropy of the second term in JES, with and without noise. Both are derived in an analytic form, and the advantage is that costly numerical computations can be avoided.\n\n- As a method for obtaining batch points, the joint conditional distribution for a batch is upper bounded by the sum of the conditional distributions for each point, avoiding the intractable computations of the former. With respect to the goodness of this upper bound, the authors show that by showing the submodularity of the acquisition function, an iterative method of sequential acquisition at each point achieves a regret upper bound of e^-1.\n\n- While the hypervolume of the Pareto set (e.g., its expected reduction), which is often used as an acquisition function in multi-objective optimization, is not invariant with respect to monotonic transformations of the output, the proposed method, JES, is shown to be invariant to the same transformation.\n\nWeakness\n\n- The assumption is made in Lemma 1 that the true Pareto optimal value set Y^* is a finite set, which appears to be a strong condition given that in general there are an infinite number of Pareto solutions. \n\n- The computations of the conditional distribution for the Pareto set and the computations for acquiring batch points are each broken down into a computations for each point, which is expected to become a computationally expensive method when the number of elements in the Pareto solution set increases as iteration progresses.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "QzV_YTyqiHV", "reviewer": "Reviewer_LjBd", "summary": "This paper proposes a novel information-theoretic acquisition function for BO called Joint Entropy Search (JES), which considers the joint information gain for the optimal set of inputs and outputs. The authors present several analytical approximations to the JES acquisition function and also introduce an extension to the batch setting. The authors show the effectiveness of this new approach on a range of synthetic and real-world problems in terms of the hypervolume and its weighted variants", "questions": "No", "limitations": " the authors adequately addressed the limitations of their work. i.e., the computational complexity of JES.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength 1, it identifies the problem of predictive entropy search (PES) method, i.e., the acquisition function is heavily dependent on the approximation of p(y|x, Dn, X∗),  which is difficult to implement and optimize. Besides, it also shows that the max-value entropy search (MES) method, though comes with closed-form expressions, does not directly dealing with the maximum Y∗. JES combines the advantages of both methods.\n\nStrength 2, it gives several theoretic property of the JES function, with in-depth study.\n\nStrength 3, the experiments are very detail, and have a good overall design. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5jCPk6pD-3n", "reviewer": "Reviewer_WzLK", "summary": "The paper proposes a new information-based acquisition function for multi-objective Bayesian optimization that combines the approaches from the known PES and MES acquisition functions. The authors derive various approximations to efficiently compute this acquisition function and provide a comparison of their performance characteristics against a number of baselines.\n", "questions": "- l34: \" The main drawback of these approaches is that the performance of these methods are biased towards a single scalar utility criterion, which can be inadequate to assess the multi-objective aspects of the problem\". I don't really understand this point; after all, the scalar utility is defined specifically in order to capture the multi-objective aspects. In a sense your formulation is no different in that ultimately you optimize a scalar function, the information gain. Can you please clarify?\n- l38: This sentence makes it seem as if other papers do not consider a \"probabilistic perspective\", which of course they do (this is BO after all).\n- l156: what does \"cancelling out\" mean here? JES is computed on the covariance using the augmented set of observations so there shouldn't be any cancelling in the strict sense. Please clarify.\n\nNits / Typos:\n- l28/29: it enables -> they enable\n- l47: set-up -> set up\n- l89: \"and a property of the probabilistic model.\" <- not sure what is meant by this, please clarify\n", "limitations": "Yes. ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths:\n- The paper is well written, in particular Section 2 provides a nice buildup of background to understand JES in relation to (P)ES and MES.\n- The motivation for studying the new acquisition function and the theoretical contributions (in terms of JES and the various approximations) are solid; I view them primarily as well crafted technical work on top of a large body of existing literature in the field rather than a surprising or particularly novel set of contributions (which is not a bad thing).\n- The main strength of the paper beyond this solid execution is that it provides a comprehensive comparison and ablation studies of various state-of-the art acquisition functions.\n- The authors also provide a full implementation of the methods in a popular and maintained python library for Bayesian Optimization. This in itself is a significant contribution to the community, which can now easily use not just JES but also the other implemented baselines (without trying to get an unmaintained library such as spearmint running in a modern software environment...).\n- Pointing out the potential issues with HV w.r.t. rescaling objectives is valuable, and the fact that JES is invariant to that is nice and useful (though not surprising as one would expect that from an information-based acquisition function).\n- The supplementary material is very comprehensive and goes into lots of detail. This is very useful to the reader. For instance, I found the illustrations of the acquisition function in the contour plots instructive.\n\n# Weaknesses:\n- One shortcoming of the work is this that there aren't any variance estimates of the performance of the different algorithms in the empirical evaluation. The checklist states that \"We do not include the error bars directly on the plots because it made it difficult to visually compare between the different algorithms.\", but as a reader I would have hoped to see a more thorough analysis of this in the supplementary material. The quantile versions of the plots are useful to some extent, and by comparing the performance of the suggested methods across the detailed results of the benchmarks the reader can get a sense that the proposed algorithms may indeed improve over existing methods. However, this is a lot of work to require from the reader, and I would ask the authors to add a more comprehensive discussion to help practitioners understand in which the proposed methods are most useful and computationally feasible.\n- The information gain estimates rely crucially on the assumption that the different objective functions are modeled by independent GPs. If the objectives are correlated, then this assumption could result in poor performance. Other approaches, such as Monte-Carlo based EHVI methods are not subject to these limitations since their methodology applies in the same fashion to multi-output GPs. In fact, as they are based on MC sampling from the model's posterior, such methods are also compatible with other probabilistic models, which is not the case for the approach proposed in this paper.\n- Overall, I found the experimental section of the paper in the main text too brief to be able to get a good sense for the empirical performance of the JES approach in comparison to the baselines (both in terms of sample efficiency and computational footprint). I would suggest the authors move some of the mathy and notation-heavy details of Section 3  (e.g. Proposition 3) to the appendix, potentially streamline the exposition in other parts of the paper, and include additional results / discussion of the empirical performance in the main text.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4x35Ky8_7eI", "reviewer": "Reviewer_ucDe", "summary": "The paper proposes a new information gain acquisition function for multi-objective Bayesian optimization. The method selects the input that maximizes the joint information gain about both input and output space. The paper provides several approximations of the acquisition function using lower bound and assumptions about the observed variance. The method is extended to batch selection accompanied by a submodularity study. The paper provides an analysis of the sensitivity of the hypervolume performance metric to functions transformations. ", "questions": "Approximations\n+ How is the 0 variance assumption made, and what are the consequences of this on the selections made by the acquisition function? Canceling the effect of the predictive variances can lead to more exploitation in the acquisition function. It seems to be a little ad hoc just to add an additional term to cancel the variance effect. The transition is not explained well or justified. \n\nWriting and presentation: suggestions for readability enhancement: \n+ Several equations and inequalities are written in the text without any numbering and then used in the derivation without a referral. \n+ I suggest adding all the equalities/inequalities and definitions of variables used in separately numbered equations. I am aware of the space constraint, but several definitions can fit in one line, and white space reduction can also be used. \n+ Adding the propositions/lemmas written in the main paper near their corresponding proofs in the appendix would make it easier to follow the derivations. The reader won’t need to jump back and forth between the main paper and the appendix. \n+ The connections and substitutions used in the derivations should be detailed, at least in the appendix. \n+ f(x) <= Y* is a notation used repeatedly in the paper. However, the relation between two Pareto fronts is not always quantified as higher or equal and would depend on the tradeoff between several functions. A more rigorous notation should be used.  \n+ The confusion between SO and MO surfaces in the names used for algorithms and in the notation used in the paper. For example, in algorithm 1, line 3 can be confused as a single objective sampling or sampling from a single multi-task GP while each of the objectives is sampled separately. \n\nExperiments: \n+ Only batch sizes of 1 and 8 are used. Typically batch BO papers include extensive experiments on different batch sizes\n+ More synthetic experiments can be useful to show the behavior of the algorithm when the number of functions and number of dimensions is varied \n+ The noise choice for each experiment is different. How is this choice made? \n+ Figure 13 does not easily provide the needed information. It would be more informative to plot all the algorithms using the same number of samples in the same figure and have different figures for different numbers of samples \n+ Some relevant and important baselines were not cited or compared to:  Diversity-Guided Multi-Objective Bayesian Optimization With Batch Evaluations, Neurips 2020\n+ The standard errors are not plotted. The use of quantiles is unconventional and do not substitute or provide the same interpretation as the error. A light-colored error plot can still be readable.\n+  The performance improvement is minor, and the baselines sometimes work better. \n+ In the Penicillin experiment, the hypervolume progress seems unstable for all algorithms; what is causing the fluctuation, and how is this interpreted? \n+ Figure 1 provides an analysis based on single objective optimization, which does not necessarily hold in the multi-objective case. This figure is not discussed or analyzed anywhere in the text. The same issue applies to Figures 6, 7, and 10. I suggest removing these studies. They are not relevant, and they add to the confusion between SO and MO.\n+ The discussion on the sensitivity of the hypervolume as a metric is insightful. However, it is important to note that it is a common practice to fit the Gaussian processes to normalized data, and therefore the scale issues between the objectives become less problematic. Additionally, several other distance-based metrics overcome some of the weaknesses of HV when comparing equivalent Pareto fronts. Therefore, evaluating the performance using more than one metric is encouraged.\n\nTypos:\n+ Typo in line 186 “because the cost of evaluating”\n+ The structure of the sentence in line 176,177 is incorrect\n", "limitations": "The paper discussed some of the work's limitations.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The paper is technically rich and provides an extensive study of the problem space with several approximations and derivations\n+ The idea of the use of joint information gain is novel and interesting. \n+ The extension to batch setting seems naïve however, the discussion of submodularity of the acquisition function is interesting \n+ The paper provides a thorough cost analysis in terms of analytical complexity and wall clock time of optimization \n\nWeaknesses:\n+ Some design choices made in the approximation are not justified and seem ad hoc, including the cancellation of variance effect in the first approximation and choice of noise in experiments. \n+ Many parts of the derivation in the main paper and the appendix are inspired by [50] and [68]. However, this does not seem to be properly acknowledged in the paper. \n+ The paper has a significant presentation issue. In general, the derivations are very hard to follow, both in the main paper and the appendix. This is not due to the extensive mathematical formulation of the acquisition function but rather to the lack of smoothness in the writing and the assumption that the reader can easily follow all the equations and substitutions. I provide more details in the questions box.   \n+ The experimental setup has several weaknesses that I discuss more in detail in the questions and comments box\n+ The writing of the paper assumes the reader is familiar with previous work on Information-theoretic acquisition functions, which is not necessarily true. Several steps of the algorithm are not described thoroughly and have confusing notations.\n+ The paper uses “incorrect” acronyms to refer to previous work. This is problematic because the acronyms used in this paper are usually used to refer to different algorithms in SOO. For example, MES and PES are usually used to refer to a single objective acquisition function. In MO, they are referred to as PESMO and MESMO. The EHVI is usually used to refer to the sequential EHI method. The method discussed in this paper is the recently developed batch approach qEHVI and qNEHVI. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2ggTWRWEulj", "reviewer": "Reviewer_Z8pJ", "summary": "A new acquisition function for Bayesian Optimization (JES) is proposed in this paper. It considers the joint information gain for the optimal set of X and Y.  Several analytical approximations to the acquisition function are also introduced.", "questions": "- It is not quite clear from the paper why this new method is superior to the existing methods (for example PES and MES as mentioned in the paper) with respect to its cost-performance trade-off. The amount of work involved in using this new acquisition function may be large, especially compared to MES. \n- Maybe it is good to identify certain specific settings or scenarios where this joint information gain acquisition function has an advantage over both PES and MES. \n- For motivation and illustration purposes, maybe in the introduction, some small examples on a small instance can be worked out where this new method can be easily compared with PES and MES. In this way, this method can be better motivated.\n- The approximation methods can achieve different accuracy levels using a different number of samples, if possible, some discussion or guidance on that will be helpful in that section.", "limitations": "Not applicable.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: A new information-theoretic acquisition function for Bayesian Optimization is presented, and approximation methods are also discussed and analyzed in this paper.\n\nWeakness: It is not quite clear why this new method is superior to the existing methods (for example PES and MES as mentioned in the paper) with respect to its cost-performance trade-off.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "009daQaPBk", "reviewer": "Reviewer_QtD3", "summary": "This work proposes a novel acquisition function JES for multi-objective Bayesian optimization based on the joint entropy of the optimal inputs and outputs. Previously works have studied input and output entropy independently. The work proposes novel entropy approximations—particularly one based on moment-matching, which has been previously used in the single objective setting but not the multi-objective setting. The work extends the proposed JES acquisition function to the batch setting using an additional approximation. In addition, this work analyzes potential for negative pathologies when using the hypervolume indicator for evaluation and proposes an alternative weighted hypervolume evaluation criterion.", "questions": "* How well does the constrained version work empirically?\n* How robust is the performance to the noise level?  \n* The JES approach seems quite incremental. \n  * The input entropy and output entropy are well-studied\n  * The moment-matching approximation is well-studied in the single objective case [50]\n  * The box decomposition-based entropy computation is proposed in [68]\n  * Lemma 1 is not novel. It comes directly from [68]  and is used in [68] which should be made more prominent\n  * The whole JES computation algorithm is an incarnation of BAX with a different acquisition function than PES and a different approximation method.\n* JES does not support modeling correlated outcomes---independent GPs models are assumed to be used. Why then distinguish equations (10) and (11)? Also, Why does Figure 3 show correlation between outcomes? Aren’t the covariances matrices across outcomes for a single point always diagonal under the assumption of the independent GP models?\n* The batch extension follows [50] and is “suboptimal” (L193) and approximates the joint JES across a batch of points with the sum of the independent entropies of each point. Why would the best batch of `q` points not be the same point `q` times using this acquisition function?\n  * Also, sequential greedy batch selection is common practice. While the submodularity proof is neat, it applies to 1) approximating true batch JES with sequential greedy or 2) approximating the coarse approximation (sum of independent entropy) of batch JES with sequential greedy. It makes no guarantees about how well the coarse approximation (12) approximates batch JES (or some approximate version of JES without the batch independent entropy approximation (12)). This error seems like it could be quite large.\n  * How does JES scale with increasing batch size both in terms of wall time and in terms of optimization performance? It is hard to draw conclusions from q=1 and q=8 alone.\n* Why does this paper focus on multi-objective JES and not a single objective JES?\n* Being invariant to component-wise monotonic transformations of the objectives is a nice property of entropy-based methods (prop 4). But this is not specific to JES, and therefore seems a bit odd to include.\n* Page 7 footnote: “The reference point is typically chosen to be the nadir…”. Typically the reference point is typically chosen to be *slightly worse* than the nadir of the *Pareto frontier*, *not the entire objective space* as stated in the footnote. Using the nadir of entire objective space (as described) would often be a poor choice of reference point because it would be very far from the pareto frontier. Was this strategy used for evaluation purposes (hypervolume computation) in the experiments? Also using the nadir of the Pareto frontier directly would be a poor choice because it gives zero hypervolume to the end points, which is why the reference point is typically chosen to be slightly worse than the nadir (see Ishibuchi et al., 2011).\n  * L954/983: using a nadir over all observed points as the reference point is also is likely a poor choice for solving the MOOP problem for recommendation and for use in NEHVI and EHVI. A point that is slightly worse than the nadir point over the pareto frontier across the observed designs is likely a much better choice because it 1) will ensure that it is not very far from the current pareto frontier, which could be the case with using the nadir across all observed points and 2) gives non-zero hypervolume to the endpoints of the pareto frontier and beyond the endpoints, which is likely very important for discovering designs outside of the currently observed Pareto frontier with EHVI/NEHVI. \n  * It is very hard to distinguish the trend lines in Figure 5. I would recommend focusing on one JES method. This would also help the reader understand which JES method is best and should be used. Currently, this is very hard to parse. The colors for TSEMO and LB-JES-0 are very similar.\n  * Also, can you provide error bars please? Currently, it is not clear which differences in performance are statistically significant\n* How does the performance look on noiseless problems?\n* L288: Regarding being able to specify a preferred region (via the reference point) with hypervolume-based methods, this seems like a positive. It seems like a major drawback of entropy-based methods to only be able to identify the entire Pareto frontier and not just an area of interest.\n* For problems such as the ZDT family, the optimal designs lie in a small region of design space. This means that the heuristic for selecting the initial points for gradient-based acquisition optimization is particularly important for improvement-based acquisition functions since they can be 0 over a large portion of the design space. In L903, the paper says that initial points were not sampled near the Pareto frontier, which would have led to faster convergence for entropy-methods. I think it would be even more important for improvement-based methods. Can you provide empirical results were the initialization heuristic does sample points near the current best (pareto optimal) for acquisition optimization?\n\nMinor comments\n\nL24: “trade-off” -> “trade-offs”\n\nL31: “might suffer when the scale of the objectives is unknown apriori”. A common approach is simply to rescale the objectives to the unit cube based on the observed outcomes\n\nL32: “or when the input space is high-dimensional”. Wouldn’t high-dimensional output spaces be more problematic for random scalarizations? \n\nL50: “analytically available, which enables the use of gradient-based optimization”. It is not clear here why analytic availability implies differentiable\n\nL73: should be \\subseteq instead of \\subset\n\nL196: These acronyms are very confusing\n\nL298: it would be good to clarify that “multi-objective optimization” refers to the fact that computing JES requires solving a set of multi-objective optimization problems of optimizing GP function samples.\n", "limitations": "Some limitations are discussed in the last section, but further discussion around when to use which method would be useful. Additionally, characterizing drawbacks of the approach and sensitivity of the approach and entropy-based methods more generally to model-specification would be useful. Negative societal impacts are not discussed.", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Although this paper is largely well-written and provides precise detail on the empirical setup and evaluation, the contributions are limited. Multi-objective BO is an important but well-studied space. Hence, the work can, inherently, at best be incremental. Furthermore, entropy-based methods have been studied in multi-objective BO. The contributions of a novel joint input/output entropy acquisition function and a new multi-objective entropy approximation show improvement over previous ES methods, but these are small contributions that leverage well-studied ideas from the literature. Although JES approximations show good empirical performance, the evaluation is limited to evaluating the out-of-sample model predicted Pareto frontier. This evaluation technique favors and is well-suited to information theoretic methods, but as the paper states often time decision-makers would prefer to launch a previously evaluated design (regardless of whether this is observation noise). Therefore, it is not clear that JES approximations outperform alternative approaches such as DGEMO and NEHVI when the decision maker is only interested in choosing a final design at implementation time from the set of previously evaluated designs. Evaluation of the performance using this in-sample Pareto frontier (under the noiseless objectives) would help provide more clarity. Furthermore, the absense of error bars in the empirical evaluation makes it impossible to determine which differences in performance are statistically significant. Moreover, the paper does not propose one single JES approximation to use. There are numerous JES methods and many confusing acronyms, and the empirical evaluation does not obviously indicate a single best JES method (even based on the mean performance, without accounting for uncertainty). In addition, JES is computationally intensive and is slower than even EHVI-based methods---which are known to be computationally demanding.\n\nThe additional contributions of an analysis of the HV indicator and the proposed weighted HV metric are a bit orthogonal to the new JES method and feel out of place with the rest of the paper, which focuses on JES. It seems like the weighted HV metric should be de-emphasized in the main text (specifically, the contributions), since it is not actually used in the main text and it is unclear what distribution is appropriate for sampling scalarization weights. The weighted HV and HV analysis are not JES specific, but is a general comment on multi-objective optimization that is out-of-place with the JES-focus of the paper.\n\nIt is worth noting that code is provided and well-documented. This is particularly notable because PES is known to be difficult to implement and most BO papers leverage an old Spearmint implementation with now unsupported dependencies. A high-quality PES implementation is of value to the community.\n\nOn a whole, a) the contributions are scattered (results on JES, results on ES methods handling monotonic transformations, an analysis of using HV for evaluating multi-objective optimization algorithms), b) the JES-specific contributions are incremental (a new joint entropy and a new application of an approximation techniques from the single objective setting), and c) the empirical evaluation is difficult to interpret (which JES method is best/should be used, what does performance look like if only in-sample---rather than out-of-sample---designs can be used in the Pareto frontier) and lacks measures of statistical significance.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept