Joint Entropy Search for Multi-Objective Bayesian Optimization
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Many real-world problems can be phrased as a multi-objective optimization problem, where the goal is to identify the best set of compromises between the competing objectives.
Multi-objective Bayesian optimization (BO) is a sample efﬁcientstrategy that can be deployed to solve these vector-valued optimization problemswhere access is limited to a number of noisy objective function evaluations.
Inthis paper, we propose a novel information-theoretic acquisition function for BOcalled Joint Entropy Search (JES), which considers the joint information gain forthe optimal set of inputs and outputs.
We present several analytical approximationsto the JES acquisition function and also introduce an extension to the batch setting.
We showcase the effectiveness of this new approach on a range of synthetic andreal-world problems in terms of the hypervolume and its weighted variants
Introduction
Bayesian optimization (BO) has demonstrated a lot of success in solving black-box optimizationproblems in various domains such as machine learning [73, 74, 87], chemistry [26, 34], robotics[7, 14] and clinical trials [60, 76].
The procedure works by maintaining a probabilistic model of theobserved data in order to guide the optimization procedure into regions of interest.
Speciﬁcally, ateach iteration the black-box function is evaluated at one or more input locations that maximizesan acquisition function on the model.
Implicitly, this function strikes a balance between exploringnew areas and exploiting areas that have been shown to be promising.
In this work, we consider themore general problem, where the black-box function of interest is vector-valued.
This increasesthe difﬁculty of the problem because there are now many directions in which the objectives can beimproved, in contrast to the single-objective setting where there is only one.
Informally, the end goalof multi-objective optimization is to identify a collection of points that describe the best trade-offsbetween the different {+objectives.
+}
There are several ways to deﬁne an acquisition function for multi-objective BO.
A popularstrategy is random scalarization [48, 61], which works by transforming the multi-objective probleminto a distribution of single-objective problems.
These approaches are appealing because they enablethe use of standard single-objective acquisition functions.
A weakness of this approach is that itrelies on random sampling to encourage exploration and therefore the performance of this method might suffer early on when the scale of the objectives is unknown a priori or when either the inputspace or the objective space is high-dimensional [19, 61].
Another popular class of multi-objectiveacquisition functions are improvement-based.
These strategies focus on improving a performancemetric over sets, for example the hypervolume indicator [17, 18, 25, 89] or the R2 indicator [23].
Themain drawback of these approaches is that the performance of these methods can be biased towardsa single performance metric, which can be inadequate to assess the multi-objective aspects of theproblem [94].
There are also many other multi-objective acquisition functions discussed in the litera- ture, which mainly differ by how they navigate the exploration-exploitation trade-off [8, 9, 49, 65, 66].39 Instead of relying on scalarizations or an improvement-based criterion, this paper considersthe perspective where the goal of interest is to improve the posterior distribution at the optimal points.
We propose a novel information-theoretic acquisition function called the Joint Entropy Search (JES),which assesses how informative an observation will be in learning more about the joint distributionof the optimal inputs and outputs.
This acquisition function combines the advantages of existinginformation-theoretic acquisition functions, which focus solely on improving the posterior of eitherthe optimal inputs [30, 32, 38] or the optimal outputs [4, 6, 77].
We connect JES with the existinginformation-theoretic acquisition functions by proving that it is an upper bound to these utilities.
Contributions and organization.
In Section 2, we set up the problem and introduce the novel JES acquisition function.
In Section 3, we present a catalogue of conditional entropy estimates toapproximate this utility and present a simple extension to the batch setting.
These approximations areanalytically tractable and differentiable, which means that we can take advantage of gradient-basedoptimization.
These results that we developed are inspired by recent work in the single-objective
Bayesian optimization literature presented in [56, 77].
In Section 4, we present a discussion on thehypervolume indicator and explain how it can be a misleading performance criterion because it issensitive to the scale of the objectives.
We prove that information-theoretic approaches are naturallyinvariant to reparameterization of the objectives, which make them well-suited for multi-objectiveblack-box optimization.
For a more complete picture of performance, we propose a novel weightedhypervolume strategy (Appendix I), which allows us to assess the performance of a multi-objectivealgorithm over different parts of the objective space.
In Section 5, we demonstrate the effectivenessof JES on some synthetic and real-life multi-objective problems.
Finally in Section 6, we providesome concluding remarks.
Additional results and proofs are presented in the Appendix.
Preliminaries
We consider the problem of maximizing a vector-valued function f : X → R M over a boundedspace of inputs X ⊂ R D .
To deﬁne the maximum max x ∈ X f ( x ) , we appeal to the Pareto partialordering in R M .
For the rest of this paper, we will denote vectors by y =
( y (1) , . . .
, y ( M ) )
∈ R M ,the non-negative real numbers by R ≥ 0 and diagonal matrices by diag ( · ) .
Pareto domination.
We say a vector y ∈ R M weakly Pareto dominates another vector y (cid:48)
∈
R Mif it performs just as well in all objectives if not better: y (cid:23) y (cid:48)
⇐⇒ y − y (cid:
) ∈ R M ≥ 0 .
Additionally,if the vectors are not equivalent, y (cid:54) =
y (cid:48) , then we say strict Pareto domination holds:
y (cid:31) y (cid:48)
⇐⇒y − y (cid:48)
∈ R M ≥ 0 \ { 0 M } , where 0 M is the M -dimensional zero vector.
This binary relation can befurther extended to deﬁne domination among sets.
Let A, B ⊂ R M be sets, if the set B lies in theweakly dominated region of A , namely B ⊂ D (cid:22) ( A ) = ∪ a ∈ A { y ∈ R M : y (cid:22) a } , then we say Aweakly dominates B , denoted by A (cid:23) B .
In addition, if it also holds that the dominated regions arenot equal, D (cid:22) ( A ) (cid:54) =
D (cid:22) ( B ) , we say strict Pareto domination holds, denoted by A (cid:31) B .
Multi-objective optimization.
The goal of multi-objective optimization is to identify the Paretooptimal set of inputs X ∗ =
arg max x ∈ X f ( x ) ⊆ X .
The Pareto set is deﬁned as the set of inputswhose objective vectors are not strictly Pareto dominated by another: x ∗ ∈
X ∗ ⇐⇒ x ∗ ∈ 77 X
and (cid:64) x ∈ X such that f ( x ) (cid:31) f
( x ∗ ) .
The image of the Pareto set in the objective space Y ∗ =
f ( X ∗ )
= max x ∈ X f ( x ) is called the Pareto front.
For convenience of notation, we will denote theset of Pareto optimal input-output pairs by ( X ∗ , Y ∗ ) .
Bayesian Optimization is a sample efﬁcient global optimization strategy, which relies on a probabilistic model in order to decide which points to query.
In Appendix A.1, we present the pseudo-codefor BO procedure—for more details see [13, 28, 72].
In this work, we will use independent Gaussianprocess priors [68] on each objective, f ( m ) ∼ GP ( µ ( m ) 0 , Σ ( m ) 0 ) , where µ ( m ) :
X → R is the meanfunction and Σ ( m ) :
X × X
→ R is the covariance function for objective m .
The observationsat location x ∈ X will be assumed to be corrupted with additive Gaussian noise, y =
f ( x ) + (cid:15) ,where (cid:15) ∼ N (0 , diag ( σ ( x )) denotes the observation noise with variance σ ( x ) ∈ R M ≥ 0 .
After nevaluations, we have a data set D n = { ( x t , y t ) } t =1 ,...,n .
The posterior model p ( f | D n ) is a collection
Information-theoretic acquisition functions focus on maximizing the gain in information fromthe next observation and a function of the probabilistic model.
Initial work in BO focussed on pickingpoints to learn more about the distribution of the maximizer p ( X ∗ | D n ) .
Speciﬁcally, the goal ofinterest was to maximize the mutual information between the observation y and the Pareto set X ∗conditional on the current data set D n :
[Equation]
where H [ p ( x )]
= − (cid:82) p ( x ) log p ( x ) d x represents the differential entropy.
This acquisition functionis commonly referred to as predictive entropy search (PES) [38, 39, 71], but it was formerly 1known as entropy search (ES) [37, 81].
Despite the importance of obtaining more informationabout the maximizer, the PES acquisition function is heavily dependent on the approximation ofp ( y | x , D n , X ∗ ) , which is both computationally difﬁcult to implement and optimize.
This motivatedresearchers to consider a simpler scheme that focusses on learning more about the distribution of themaximum p ( Y ∗ | D n ) .
The resulting acquisition function is known as the max-value entropy search(MES)
[Equation]
Unlike PES, the conditional probability p ( y | x , D n , Y ∗ ) arising in MES can be approximated andoptimized more easily because some approximations lead to closed-form expressions.
Despite thefavourable properties of MES, the primary goal of interest is to identify the location of the maximizer 107 X ∗ and not necessarily the value of the maximum Y ∗ .
To combine the advantages of both of theseapproaches, we propose the joint entropy search acquisition function, which focusses on learningmore about the joint distribution of the optimal points p (( X ∗ , Y ∗ ) | D n ) :
[Equation]
The JES acquisition function inherits the advantages of the PES and MES acquisition functionsbecause it considers the knowledge learnt about the optimal points and is also simple to implement—more details in the next section.
The following proposition shows that we can also interpret JES as anupper bound to both the PES and MES acquisition function.
Proposition 1. The JES is an upper bound to any convex combination of the PES and MES acquisitionfunctions
[Equation]
any
[Equation]
Remark. In the BO literature it is common to distinguish between single-objective and multiobjective acquisition functions by adding ‘MO’ to the end of the acronym.
For notational simplicity,we opt against this convention in this paper.
Approximating JES
In this section, we present several approximations to the JES acquisition function (3) and a simpleextension to the batch setting.
The ﬁrst term in the JES criterion (3) is the entropy of a multivariatenormal distribution:
[Equation]
The second term is an intractable expectation which is approximated by drawing Monte Carlo samples from p (( X ∗ , Y ∗ )
| D n ) .
The conditional entropy H [ p ( y | x , D n , ( X ∗ , Y ∗ ))] is also an intractablequantity which has to be estimated.
The overall approximation of (3) will take the form
[Equation]
where h denotes the conditional entropy estimate and ( X ∗ s , Y ∗ s ) ∼ p (( X ∗ , Y ∗ )
| D n ) are the Monte
Carlo samples.
The distribution p ( y | x , D n , ( X ∗ , Y ∗ )) is very challenging to work with because itenforces the global optimality condition that the function lies below the Pareto front f ( X ) (cid:22) Y ∗ .
Instead of enforcing global optimality, we make the common simplifying assumption as in [56, 77, 83]and only enforce the optimality condition at the considered location: f ( x ) (cid:22) Y ∗ .
By applying Bayes’theorem, the resulting density of interest becomes
[Equation]
where we have denoted the augmented data sets by D n ∗
=
D n ∪ ( X ∗ , Y ∗ ) and D n + = D n ∗ ∪{ ( x , y ) } .
We will refer to the quantity p ( f ( x ) (cid:22) Y ∗ ) as the cumulative distribution function (CDF).
Thefollowing lemma shows that this CDF can be computed analytically when the set Y ∗ ⊂ R M isdiscrete.
This is a standard result [15, 47, 65, 77] which can be derived by ﬁrst partitioning the regionof integration, D (cid:22) ( Y ∗ ) = ∪ y
∗ ∈
Y ∗ { z ∈ R M : z (cid:22) y ∗ } , into a collection of hyperrectangle subsetsand then summing up the individual contributions—see Figure 2 for a visual.
This partition can becomputed using an incremental approach (Algorithm 1 of [52]), which has a cost of O ( | Y ∗ | (cid:98) M/ 2 (cid:99) +1 ) .
Lemma 1. Let Y ∗ ⊂ R M be a ﬁnite set and z ∼ N ( a , diag ( b )) be an M -dimensional multivariate normal with mean a ∈ R M and variances b ∈ R M ≥ 0 .
Let D (cid:22) ( Y ∗ ) =
(cid:83)
Jj =
1 B j =(cid:83)
Jj =1 (cid:81)
Mm =1 ( l ( m ) j , u ( m ) j ] be the box decomposition of the dominated space, thenp (
[Equation]
(cid:22)
[Equation]
= J (cid:88) j = M (cid:89) m = (cid:34) Φ (cid:32)
[Equation]
j − [Equation] m ) √ b ( m ) (cid:33) − Φ (cid:32) [Figure]
[Figure]
3.1 Estimating the conditional entropy
The entropy of (6) can be written as an M -dimensional expectation over the multivariate normaldistribution p ( y | x , D n ∗ ) =
N ( y ; µ n ∗ ( x ) [Equation] Σ n ∗ ( x , x )) : [Equation]
[Equation]
To simplify the notation, we deﬁne the m -th standardized value by γ m ( z ) =
( z −µ ( m ) n ∗ ( x )) / (cid:113) Σ ( m ) n ∗ ( x , x ) for any scalar z ∈ R .
Using this function together with Lemma 1, wedenote the cumulative distribution p ( f ( x ) (cid:22) Y ∗ | x , D n ∗ ) by W = (cid:80)
Jj =1 W j = (cid:80)
Jj =1 (cid:81)
Mm =1 W j,m ,where W j,m = Φ( γ m ( u ( m ) j )) − Φ( γ m ( l ( m ) j )) are the differences appearing in (7).
Moreover, we denote the differences of the ﬁrst derivative of W j,m (with respect to γ m )by G j,m = φ ( γ m ( u ( m ) j )) − φ ( γ m ( l ( m ) j )) and the negative of the second derivative by
V j,m = γ m ( u ( m ) j ) φ ( γ m ( u ( m ) j )) − γ m ( l ( m ) j ) φ ( γ m ( l ( m ) j )) , where φ is the probability densityfunction of a standard normal distribution.
In the setting where the observation noise is zero, theconditional distribution is a truncated multivariate normal, which is known to have an analyticalequation formula for the entropy (Theorem 3.1.
in [77]).
In Appendix D, we construct an ad hocextension to this expression when the observation [-variance, σ ( x ) = 
 M , then-]noise is non-zero.158 In the noisy setting, the distribution of interest is a type of multivariate skew normal distribution, which is known to not have an analytical form for the entropy [1].
As a result, we proposetwo approximation strategies to estimate this entropy.
The ﬁrst strategy is to approximate theintegral using Monte Carlo.
The details of the Monte Carlo estimate h JES-MC is described in Appendix E.
The second strategy is to directly approximate the distribution with one that exhibitsan analytical entropy.
We consider the most obvious choice, which is a multivariate normaldistribution with the same ﬁrst two moments.
The same strategy was proposed in [56] for thesingle-objective multi-ﬁdelity MES acquisition function.
By a standard result (Chapter 12 of [16]),the entropy of this approximating distribution is actually an upper bound for the entropy of interest:
H [ p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ )] ≤ M 2 log(2 πe ) + 12 log det V ar ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) .
The followingresult shows that the these central moments can be computed analytically.
Proposition 2. Under the modelling set-up outlined in Section 2, for an input x ∈ X the ﬁrst andsecond central moment of p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) are
[Equation]
and Cov
[Equation]
, y
[Equation]
y (1) y ( 2 )
Skew Multivariate Normal 0 .
( (1) n * ( x ), (2) n * ( x ))
Pareto front 0.25 0.
{+(a) y (1) y ( 2 ) Truncated Multivariate Normal 0 .0 .0 .
 (b) y (1) y ( 2 ) Multivariate Normal (Full Covariance) 0 .0 .0 .0 .1 . 0 1 .1 .+}
(c) y (1) y ( 2 )
Multivariate Normal (Diagonal Covariance) 0 .0 .0 .
(d)
=
[Equation] [Figure] [Equation] As an upper bound on the conditional entropy leads to a lower bound on the mutual information, wewill refer to the resulting conditional entropy estimate as the JES-LB estimate:
[Equation]
[Equation]
We could obtain a further lower bound by ignoring the off-diagonal terms in the covariance matrix.
We dub the resulting approximation as the JES-LB2 entropy estimate:
[Equation]
[Equation] Empirically, we observe that the lower bound JES approximations are not necessarily tight for allinputs x ∈ X .
Nevertheless, the maximizer of the approximate acquisition function is often close tothe maximizer of the exact JES—see Appendix H for a visualization on a single-objective problem.
Remark.
All of the conditional entropy estimates introduced here can be applied to the MESacquisition function.
The only difference is that we no longer apply the conditioning step (line 6 of
Algorithm 1) because we are interested in estimating H [ p ( y | x , D n , f ( x ) (cid:22) Y ∗ )] as opposed to (8).
Consequently, the MES acquisition function is cheaper to evaluate because the cost of evaluating theposterior variance at a single input is O ( n 2 ) , whereas JES incurs a cost of O (( n + | Y ∗ | ) 2 ) —moredetails are presented in the cost analysis in Appendix G.
3.2 Batch evaluations
Evaluating the JES acquisition functions for a batch of points x [1: q ] =
( x [1] , . . .
, x [ q ] ) ∈
X q isexpensive because the entropy estimates now depends on the q -dimensional normal CDF and itsderivatives.
To circumvent this issue, we follow the example of [56] and propose a suboptimal batchapproach by upper bounding the expensive joint conditional entropy term by the sum of the individualentropies: H [ p ( [Equation] [Equation] [Equation]
[Equation] Y [Equation] )] ≤ (cid:80) qi
=1 H [ p ( y [ i ] | x [ i ] , D n ∗ , f ( X ) (cid:22) Y ∗ )] .
Theresulting q -batch lower bound JES estimate is given by
[Equation]
11) where h is the conditional entropy estimate and
[Equation]
y (1) y ( 2 )
U HV =0. [Figure]
y (1) y ( 2 )
[-Truncated Multivariate Normal 0 .0 .0 .0 .0 .1 . 
-]{+U HV =0.
+}
Objective space (a) g 1 ( y (1) ) g 2 ( y ( 2 ) )
{+U HV =0.
 Transformed objective space g 1 ( y (1) )+}
g 2 ( y ( 2 ) )
U HV = 0.822 Transformed objective space (b) is the initial entropy.
This acquisition function is deﬁned over a qD -dimensional space, whichbecomes more difﬁcult to optimize as q increases.
Alternatively, we can maximize this functiongreedily by sequentially selecting the best input conditioned on the previously chosen points.
Thisgreedy procedure satisﬁes an e − 1 regret bound when the acquisition function is submodular [84].
In Appendix F, we show that this batch acquisition function is indeed submodular.
Performance criteria
In multi-objective optimization, the most common way to measure performance is by comparing theapproximate Pareto set ˆ X ∗ against the optimal Pareto set X ∗ in the objective space: d ( f (ˆ X ∗ ) , f ( X ∗ ))where d : 2 R M × 2 R M → R is a function that measures the discrepancy between the sets of objectivevectors.
Existing work in multi-objective BO mainly focusses on the hypervolume (HV) discrepancy,d HV ( A, B ) =
| U HV ( A ) − U HV ( B ) | , where the HV indicator, U HV ( A ) =
(cid:82) R M I [ r (cid:22) z
(cid:22) A ] d z , isdeﬁned as the volume between a reference point r ∈ R M and a set
A ⊂ R M .
The general guidanceis to set reference point to be slightly worse than the nadir, which is the vector consisting of the worstpossible points, min x ∈ X f ( m ) ( x ) , for objectives m = 1 , . .
, M —see [45] for more details.209 An attractive feature of the HV indicator is that it is Pareto complete (or compliant) in thesense that a better set will lead to a larger HV [94]: A (cid:31) B =
⇒ U HV
( A ) >
U HV ( B ) .
Thereverse implication known as Pareto compatability does not hold for the HV indicator [94].
In otherwords, the HV can be used to discriminate between sets where one dominates another, but it cannotbe relied upon when the sets are incomparable.
Not all incomparable sets are treated equally bythe HV indicator [92].
For instance Figure 4a shows an example where the HV indicator placesmore emphasis on the end points of the Pareto front.
On other hand, if we apply a monotonicallyincreasing transformation g m : R → R to each objective, the Pareto set will not change, whereas the HV comparison will (Figure 4b).
Implicitly, the HV indicator assumes that a linear change in oneobjective is equivalent to a linear change in another.
This assumption might not necessarily reﬂectthe decision maker’s outlook and this is something that is typically overlooked when designingand benchmarking multi-objective optimization algorithms.
The following result shows thatinformation-theoretic acquisitions function are in fact agnostic to the choice of parametrization.
Proposition 3. The information-theoretic acquisition functions α PES , α MES and α JES are invariant toreparametrizations of the objective space that are consistent with the Pareto ordering relations.
Forexample, α JES ( x | D n ) =
MI ( y ; ( X ∗ , Y ∗ )
| D n )
=
MI ( g ( y ); ( X ∗ , g ( Y ∗ ))
| D n ) , where the g m : R → 225 R is a strictly monotonically increasing function acting only on the m -th objective.
To benchmark the algorithms, we use both the standard HV discrepancy (Section 5) and the HVdiscrepancy under different parameterizations (Appendix J).
To easily obtain a family of parameterizations, we devise a novel weighting approach in Appendix I, which exploits the fact that the HVindicator can be written as an expectation over a uniform distribution on the ( M − 1) -dimensionalhypercube
We observe that it is possible to assess the performance at different locations ofthe objective space by using alternate distributions over hypercube.
[-U HV =0.
-]
[-U HV =0.
-]
In our experiments, we found that the performance of each algorithm changed with regards to the choice of parametrization, but the
JES approaches tended to be the strongest performers throughout.
Experiments
We empirically evaluate the JES acquisition function on a range of synthetic and real-world benchmarkproblems.
We compare this approach with some popular acquisition functions in multi-objective
BO: TSEMO [12], ParEGO [48], NParEGO [18], EHVI [17], NEHVI [18], PES [30, 32] and MES-0[77].
We have also included the MES-LB, MES-LB2 and MES-MC acquisition functions, whichcan be easily derived from the conditional entropy estimates that we developed here.
All algorithmsare based on the open source Python library BoTorch [3], which uses features from GPyTorch [29]for Gaussian process regression and PyTorch [63] for automatic differentiation.
All experimentsare repeated using 100 different initial seeds and we generate the Pareto set recommendation ˆ X ∗ of50 points by maximizing the posterior mean using a multi-objective solver (NSGA2 [21] from the
Pymoo library [10]).
The complete details of the experiments are outlined in Appendix J, whilst thecode is available at https://github.com/.../JES .
5.1 Benchmarks
Synthetic benchmark.
We consider the ZDT2 [21] benchmark with D = 6 inputs and M = 2objectives.
We corrupt the observations with additive Gaussian noise with zero-mean and standarddeviation set to approximately 10% of the objective ranges.
Chemical reaction.
This benchmark considers a nucleophilic aromatic substitution reaction (SnAr)between 2,4-diﬂuoronitrobenzene and pyrrolidine in ethanol to produce a mixture of a desired productand two side-products [44].
The design space comprises of D = 4 components relating to theinitial conditions.
The goal is to optimize M = 2 objectives, namely the space time yield andthe environmental impact.
We apply a logarithm transform to the objectives and contaminate theobservations with additive Gaussian noise with zero-mean and standard deviation set to approximately3% of the resulting objective ranges in order to emulate a potential real-world scenario.
Pharmaceutical manufacturing.
This problem is concerned with optimizing the Penicillin production process outlined in [53].
The design space is made up of D = 7 elements that control theinitial condition of the reactions.
The goal is to optimize M = 3 objectives, which relates to theyield, the amount of carbon dioxide released and the time to ferment.
We include additive zero-mean
Gaussian noise with a standard deviation set to approximately 1% of the objective ranges.
Marine design.
This problem considers optimizing a family of bulk carriers subject to the constraints imposed for ships travelling through the Panama Canal [62, 70].
The design space is madeup of D = 6 variables that determine the architecture of the carriers.
The goal of this problem is tomaximize the annual cargo, whilst minimizing the transportation cost and the ship weight subjectto some design constraints.
We consider the reformulation in [80], which converts the constraintsinto another objective.
[-Penicillin (D=7, M=3, Noise=1%, q=8) 
-]
For this reformulated M = 4 objective problem, we corrupt the observationswith additive zero-mean Gaussian noise with standard deviation set to approximately 0 .
5% of theobjective ranges.
5.2 Results and discussion
We present the log HV discrepancy results for both the sequential and batch experiments in Figure 5.
The JES approach is consistently one of the stronger performing algorithms for these set of experiments. A similar conclusion is reached when we consider the weighted variant of the hypervolume in Appendix J.5.
Conditional entropy estimates. We compared the performance of the different conditional entropyestimates for both the JES and MES acquisition function in Appendix J. We observed that in themajority of cases all the estimates exhibit similar performance. As a result, we recommend using thecheapest approximation, which is usually the lower bound estimates, judging from the wall timespresented in Appendix J.7.
[Figure]
Acquisition wall times. The wall times in Appendix J.7 indicate that the cost of acquiring a newpoint with JES is comparable with NEHVI, slightly more expensive than MES, but cheaper than
PES.
We note that theses wall times can however be improved for the entropy based methods bytaking advantage of parallelization.
For example, we used a gradient-free optimizer to sequentiallyoptimize the multi-objective samples (line 4 in Algorithm 1), whereas in practice we could solvethese problems in parallel using a gradient-based optimizer such as [54]. Querying high performing points.
In certain domains it might be useful to directly query highperforming points because the ﬁnal decision will be restricted to only the sampled locations X N . In Appendix J.4, we investigated the performance when such a restriction was made.
In this setting,we observed that the information-theoretic approaches were occasionally outperformed by theimprovement and scalarization based acquisition functions, which picked points more greedily. Weobserved that the entropy based approaches had a tendency to pick points that are more informativefor posterior over the optimal points as opposed to directly selecting a point that is known to performwell. To address this setting, we recommend combining information-theoretic acquisition functionswith an epsilon greedy approach, where points are occasionally picked according to a greedy strategysuch as maximizing a function of the posterior mean. General guidance.
The ideal acquisition functions is problem dependent and strongly depends onthe decision maker’s plans and goals.
In a completely black-box setting, where there is no immediatepreferences, Proposition 3 and the empirical results motivates the usage of information-theoreticacquisition functions, which treats all points on the Pareto front as equally desirable a priori.
Conclusion
We introduced JES, a novel information-theoretic acquisition function for multi-objective BO.
Toapproximate this acquisition function, we presented several approximations to the conditional entropyand also a simple extension for the batch setting.
Experimental results suggest that JES is verycompetitive with existing acquisition functions in terms of the HV discrepancy and its weightedvariants.
The main limitation of the JES acquisition function is that it relies on routines such asbox decompositions and multi-objective optimization of function samples, which can be expensiveto execute for very large problems.
Future work could focus on improving the scalability of theseinformation-theoretic methods and extending it to more general settings, which include constrained,decoupled and multi-ﬁdelity optimization—see Appendix K for more details.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] The proofs for the
main results are in the appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] The code is in
the supplementary material and will be made public in due time.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] The experimental set-up is described in Appendix J.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] We have now included error bars.
(d) Did you include the total amount of compute and the type of resources used (e.g.,
type of GPUs, internal cluster, or cloud provider)?
[Yes] The computing resources are
described in the caption of the wall times plots e.g. Appendix J.7.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[No] We only cited the original code and
corresponding manuscript in Appendix J.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
The code is in the supplementary material and will be made public in due time.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
Appendix to:Joint Entropy Search for
Multi -Objective Bayesian Optimization
A BasicsA.1 Bayesian optimization pseudocode
Bayesian optimization consists of two steps that are iterated until the budget of function evaluations
N is exhausted.
The ﬁrst step is modelling, this is where posterior of the probabilistic model p ( f | D n )is computed.
The second step is acquisition, this is where a utility function is optimized in order todetermine the next location to query.
The pseudo-code for this procedure is presented in Algorithm 2.
For a more thorough overview on Bayesian optimization consult the references [13, 28, 72].
Input : A black-box function f .
1 Initialize the probabilistic model p ( f ) .
2 for n = 0 , . . .
, N − 1 do
Optimize for the next point x n +1 = arg max x ∈ X α
( x | D n ) .
Evaluate the function y n +1 = f ( x n +1 ) + (cid:15) .
Augment the data set D n +1 =
D n ∪ { ( x n +1 , y n +1 ) } .
Compute the posterior p ( f | D n ) .
7 end 8 return D N and p ( f | D N ) .
A.2 Posterior Gaussian process
Under the independent Gaussian observation model described in Section 2, the posterior p ( f ( X ) | D n )evaluated at a vector X ⊂ X , is a collection of Gaussian processes [68] with meanµ ( m ) n ( X ) = [-µ ( m ) 
-][Equation] [Equation]
X [-) + Σ ( m ) 
-][Equation]
(cid:16) [Equation] m [-) 
-][Equation]
0 ( X n , X n ) + diag [Equation] m [Equation]
{+X n )) (cid:17) [Equation] ( Y ( m ) n − µ ( m ) 
 ( X n ))+} (13) and covarianceΣ ( m ) n ( X, X ) =
[-Σ ( m ) 
-][Equation]
( X, X
[Equation] [Equation]
where we denote the collection of sampled locations by X n = { x t } t =1 ,...,n and observations
Y ( m ) n = { y ( m ) t } t =1 ,...,n for objectives m = 1 , . . .
, M .
A.3 Sampling from a Gaussian process
Exact sampling of a one-dimensional Gaussian process over a ﬁnite set, X ⊂ X , scales cubicallywith the number of points | X | .
This cubic cost arises form the inversion of the covariance matrix.
As we want to sample global maximizers and maximums ( X ∗ , Y ∗ ) , exact sampling over the wholespace X is not computational feasible.
We could restrict the sampling to a discrete subset of X , butthis would require designing a principled strategy to select a reasonable subset, for example wecould try triangulating between existing points [35].
To avoid this difﬁculty, we follow the exampleof previous work [39, 83], which considers generating approximate samples via random Fourierfeatures [67, 85, 86].
The strategy centres around approximating the covariance kernel using a featurerepresentation Σ ( m )
( x , x (cid:48) )
≈ ϕ ( x ) T ϕ
( x (cid:48) ) .
For a stationary covariance kernel Σ ( m ) 0 , the Fouriertransform of the kernel is a non-negative measure that can be normalized to obtain a probabilitydistribution p ( θ ) —this result follows from Bochner’s Theorem [11] .
For the exponential and Maternkernels, this probability distribution is known in closed-form [12, 67].
Assuming Σ ( m )is stationary, an approximate sample of the Gaussian process prior GP ( µ ( m ) 0 , Σ ( m ) 0 ) can be written as a Bayesianlinear model [Equation]
[Equation]
where ϕ i ( x ) = (cid:112) 2 /L
cos( θ Ti x + τ i ) are the Fourier features depending on the random variablesτ i ∼ U (0 , 2 π )
and θ i ∼ p
( θ ) , whilst ω i ∼ N (0 , 1) are the random weights for i = 1 , . . .
, L .
Theposterior samples of GP ( µ ( m ) n , Σ ( m ) n ) can be obtained by adding an additional pathwise update via
Matherons’ rule [85, 86]:
[Equation]
where κ t =
(Σ ( m )
( X n , X n ) + diag ( σ ( m ) ( X n )))
− 1 ( Y n − f ( m )( X n )) .
For a more comprehensiveoverview of sampling from a Gaussian process refer to [86].
B Related workB.1 Conditional entropy estimates
In this work we devised a number of conditional entropy estimates for the JES acquisition function.
These estimates can also be used by the MES acquisition function.
In this brief section, we elaborate on the similarities of our estimates with the existing work relating to the MES acquisition [-function.
-]function.655 Firstly, the noiseless entropy estimate (27) was derived recently [77] to extend the noiseless MES acquisition function to the multi-objective setting.
This extension was called the
Pareto frontier entropy search, whilst we refer to this acquisition function as MES-0 for clarity.
An earlier attempt to extend the MES to noiseless setting resulted in the MESMO [4]acquisition function.
MESMO crudely approximates the dominated space by a single box 660 D (cid:22) ( Y ∗ )
≈ B = ( −∞ ,
max y ∈ Y ∗ y (1) ] × · · · × ( −∞ , max y ∈ Y ∗ y ( M ) ] .
Empirically, this approximation leads to a deterioration in performance [77].663 The closest work to the lower bound entropy estimate is the GIBBON [56] acquisitionfunction.
The GIBBON acquisition function was derived as a lower bound to the multi-ﬁdelity
MES acquisition function for a single objective problem.
The MES-LB2 can be interpreted as themulti-objective extension of GIBBON for the single ﬁdelity [-setting.
-]setting.668 The Monte Carlo entropy estimate has been used before for the multi-ﬁdelity MES acquisition in the single-objective setting [57, 78].
The MES-MC estimate can be interpreted as themulti-objective extension of these approaches for the single ﬁdelity setting. B.2 BAX
The JES acquisition function bares some similarity with a special case of the Bayesian Algorithm Execution (BAX) algorithm proposed in [58]. Speciﬁcally, if we set O A ( f ) =
( X ∗ , Y ∗ ) in [58], thenthe corresponding BAX acquisition function would reduce to
[Equation]
In the JES acquisition function, we condition on the augmented data set D n ∗ =
D n ∪ ( X ∗ , Y ∗ ) andthe optimality condition f
( X ) (cid:22) Y ∗ for the density arising in the conditional entropy term.
Whereasin BAX, we only condition on the augmented data set.
As conditioning never the increases theentropy (Theorem 2.6.5 of [16]), we have that BAX acquisition function is a lower bound to the JES.
C Proof of results
Before we begin the proofs, we will restate some important notation.
Let p ( y | x , D n ∗ ) =
N ( y ; µ n ∗ ( x ) , Σ n ∗ ( x , x )) denote the probability density at a point x ∈ X conditional on the data set
D n ∗ =
D n ∪ ( X ∗ , Y ∗ ) .
Then we denote the m -th standardized function by
[Equation]
x
[Equation]
[Equation]
for m = 1 , . . .
, M .
For the discrete set of points Y ∗ ⊂ R M , we decompose the dominated regioninto J boxes:
[Equation]
[Equation]
where l j = ( l (1) j , . . .
, l ( M ) j ) are lower bounds and u j = ( u (1)1 , . .
, u ( M ) J ) are the upper bounds forboxes j = 1 , . . .
, J .
Using the box decomposition, we deﬁne
[Equation]
for boxes j = 1 , . . .
, J and objectives m = 1 , . . .
, M , where Φ is the cumulative distribution functionof a standard normal distribution.
The ﬁrst derivative of W j,m (with respect to γ m ) is denoted by [Equation] 21) and the negative of the second derivative by [Equation]
[Equation] for boxes j = 1 , . . . , J and objectives m = 1 , . . . , M , where φ is the probability density function ofa standard normal distribution.
Proposition 1. The JES is an upper bound to any convex combination of the PES and MES acquisitionfunctions: α JES ( x | D n ) ≥ βα PES ( x | D n ) + (1 − β ) α MES ( x | D n ) , for any β ∈ [0 , 1] .
Proof. The upper bound property follows from the standard result that conditioning onmore variables will never increase the entropy (Theorem 2.6.5 of [16]): H ( A | B ) ≤ H ( A ) for random variables A and B .
Using this result, H [ p ( y | x , D n , ( X ∗ , Y ∗ ))]
≤max( H [ p ( y | x , D n , X ∗ )] , H [ p ( y | x , D n , Y ∗ )]) . Plugging this inequality into (3), we obtain theα JES ( x | D n ) ≥ max( α PES ( x | D n ) , α MES ( x | D n )) , which implies the result.(cid:4)
C.2 Proof of Lemma 1
Lemma 1.
Let Y ∗ ⊂ R M be a ﬁnite set and z ∼ N ( a , diag ( b )) be an M -dimensional multivariate normal with mean a ∈ R M and variances b ∈ R M ≥ 0 .
Let D (cid:22) ( Y ∗ ) =
(cid:83)
Jj =
1 B j =(cid:83) J j =1 (cid:81)
Mm =1 ( l ( m ) j , u ( m ) j ] be the box decomposition of the dominated space, thenp ( z (cid:22) Y ∗ ) =
J (cid:88) j
=
M (cid:89) m = (cid:34) Φ (cid:32) u ( m ) j − a ( m ) √ b ( m ) (cid:33) − Φ (cid:32) l ( m ) j − a ( m ) √ b ( m ) (cid:33)(cid:35)
[Equation]
Proof.
Under the assumptions of Lemma 1, the following series of equations holds:
[Equation]
(1) This step follows from the box decomposition of D (cid:22) ( Y ∗ ) and the independence between thecomponents of z . This step follows from the deﬁnition of the CDF.
(cid:4)
C.3 Proof of Proposition 2
Proposition 2.
Under the modelling set-up outlined in Section 2, for an input x ∈ X the ﬁrst andsecond central moment of p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) are
[Equation]
and
[Equation]
Proof. In the noisy setting, the density of interest is
[Equation]
From Lemma 1,
[Equation]
To obtain a tractable expression for the cumulative distribution p ( f ( x ) (cid:22) Y ∗ | x , D n + ) , weﬁrst compute the conditional distribution p ( y | x , D n + ) .
To that end, let p ( y | x , D n ∗ ) =
N ( y ; µ n ∗ ( x ) , Σ n ∗ ( x , x )) , then by standard Gaussian conditioning p ( y | x , D n + ) =
N ( y ; µ n + ( x )where
[Equation]
and
[Equation]
with ρ m :=
ρ m ( x ) = (cid:113) Σ ( m ) n ∗ ( x , x ) / (cid:113) Σ ( m ) n ∗ ( x , x ) + σ ( m ) ( x ) denoting correlation between theobservation y ( m ) and the objective value f ( m ) ( x ) , for objectives m = 1 , . .
, M .
Using the boxdecomposition, the posterior CDF is equal top
[Equation]
To simplify the notation we perform a change of variable:
[Equation]
where ¯ y ( m ) = ( y ( m ) − µ ( m ) n ∗ ( x )) /
(cid:113) Σ ( m ) n ∗ ( x , x ) + σ ( m ) ( x ) is the standardized observation forobjectives m = 1 , . . .
, M .
To derive the moments of p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) , we ﬁrst obtain themoment generating function of the standardized observation ¯y .
[Equation]
By Lemma 2 in [2], the expectation of the normal CDF is given by (cid:82) R Φ( az + b ) φ ( z ) dz =
Φ( b/ √ 1 + a 2 ) for any constants a, b ∈ R .
Using this result,(cid:90) R
[Equation]
The ﬁrst moment p (¯ y ( m ) | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) can be obtained by evaluating the ﬁrst derivative of
M ¯y ( t ) with respect to t ( m ) at t = 0 M . E
[Equation]
where for shorthand we deﬁned the intermediate variables β uj,m =
Φ( γ m ( u ( m ) j ) − ρ m t ( m ) ) , β lj,m =Φ( γ m ( l ( m ) j ) − ρ m t ( m ) ) , ζ uj,m = φ ( γ m ( u ( m ) j ) − ρ m t ( m ) ) and ζ lj,m = φ ( γ m ( l ( m ) j ) − ρ m t ( m ) ) forj = 1 , . .
, J and m = 1 , . . , M . Differentiating a second time, we can obtain the second moments. For m (cid:54) = m (cid:48) ,
[Equation]
and for m = m (cid:48) ,
[Equation]
[Equation]
where for shorthand we deﬁned the intermediate variables τ uj,m = γ m ( u ( m ) j ) − ρ m t ( m ) and τ lj,m =
γ m ( l ( m ) j ) − ρ m t ( m ) for j = 1 , . . .
, J and m = 1 , . .
, M .
The moments of y ( m ) can be now derivedby reversing the initial linear transformation: y ( m ) =
¯ y ( m ) (cid:113) Σ ( m ) n ∗ ( x , x ) + σ ( m ) ( x ) + µ ( m ) n ∗ ( x ) .
Theﬁrst moment is
[Equation]
The second moment:
[Equation]
This implies
[Equation]
) (cid:12)(cid:12)(cid:12) x , D n ∗ , f ( x ) (cid:22) Y ∗ (cid:105) =
(cid:113) Σ ( m ) n ∗ ( x , x ) + σ ( m ) ( x ) (cid:113) Σ ( m (cid:48) )
n ∗ ( x , x ) + σ ( m (cid:48) )
( x ) E [¯ y ( m )
¯ y ( m (cid:48) )
| x , D n ∗ , f ( x ) (cid:22) Y ∗ ] + E [¯ y ( m ) | x , D n ∗ , f ( x ) (cid:22) Y ∗ ] µ ( m (cid:48) )
n ∗ ( x ) + E [¯ y ( m (cid:48) )
| x , D n ∗ , f ( x ) (cid:22) Y ∗ ] µ ( m ) n ∗ ( x ) − µ ( m ) n ∗ ( x ) µ ( m (cid:48) )
n ∗ ( x ) .
Substituting in the expressions before, we have that for m (cid:54) = m (cid:48) ,
E (cid:104) y ( m ) y ( m (cid:48) ) (cid:12)(cid:12)(cid:12) x , D n ∗ , f ( x ) (cid:22) Y ∗ (cid:105) = µ ( m ) n ∗ ( x ) µ ( m (cid:48) )
n ∗ ( x ) + (cid:113) Σ ( m ) n ∗ ( x , x ) (cid:113) Σ ( m (cid:48) ) n ∗ ( x , x ) W × J (cid:88) j
= W j   G j,m W j,m
G j,m (cid:48)
W j,m (cid:48) − µ ( m (cid:48) ) n
[Equation] ( x ) (cid:113) Σ ( m (cid:48) )
n ∗ ( x , x )
G j,m
W j,m − µ ( m ) n ∗ ( x ) (cid:113) Σ ( m ) n ∗ [Equation] x , x )
G j,m (cid:48)
W j,m (cid:48) 
 and for m = m (cid:48)
[Equation]
By some additional algebraic manipulation, the covariance for m (cid:54) = m (cid:48) is given
[Equation]
j (cid:48) [Equation] [Equation] j (cid:48)
[Equation]
  and the variance is
[Equation]
C.4 Proof of Proposition
Proposition 3.
The information-theoretic acquisition functions α PES , α MES and α JES are invariant toreparametrizations of the objective space that are consistent with the Pareto ordering relations. Forexample, α JES ( x | D n ) = MI ( y ; ( X ∗ , Y ∗ ) | D n )
=
MI ( g ( y ); ( X ∗ , g ( Y ∗ ))
| D n ) , where the g m : R → 751 R is a strictly monotonically increasing function acting only on the m -th objective.
Proof.
We will prove the general statement that the mutual information is invariant under smoothbijective transformations.
Consider two random variables X and Y then following equations hold: [Equation]
[Equation]
where X (cid:48)
=
g x ( X ) and Y (cid:48) =
g y ( Y ) represents the transformed variables under some suitablydeﬁned smooth invertible functions g x and g y .
The expressions J x and J y correspond to the Jacobianof the transformation, which are non-zero because the transformation are assumed to be invertible.
The result follows by restricting the class of bijective functions to ones where the Pareto set remainsunchanged.
For example, the class of monotonic increasing functions in each objective ensuresthis property holds: g : R M
→ R M such that g ( y )
= ( g 1 ( y (1) ) , . . .
g M ( y ( M ) )) with g m beingmonotonically increasing.(cid:4)
D Noiseless entropy estimate
In this section, we present the conditional entropy estimate for the zero observation variance settingand an ad hoc extension for noisy setting.
Firstly, if we assume the observation variance is zero, theconditional distribution of interest is a truncated multivariate normal:
[Equation]
which is known to have the following analytical equation for the entropy:
Proposition 4.
(Theorem 3.1.
in [77])
{+Under the modelling set-up outlined in Section 2, if x ∈ X isan input with zero observation variance, σ ( x ) = 
+}
M , then the entropy of the truncated multivariatenormal distribution p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) is given by
[Equation]
Proof.
See [77] for the proof of this result.
{+[Equation] Assuming σ ( x ) = 
 M , we have [Equation]+}
The ﬁrst term is equal to expectation of the negative log-probability of the noiseless observation lyingbelow Pareto front:
[Equation]
This term accounts for the exploitation because it puts more emphasis on the points that are likely tolie above the Pareto front.
The remaining terms account for the exploration by placing more emphasison reducing the uncertainty at input location.
To still take advantage of the result in Proposition 4 forthe noisy observation setting, we propose an ad hoc extension that adjusts the exploration term toinclude the effects of the observation noise.
Speciﬁcally, we propose a modiﬁcation which replacesthe difference in log variances in the exploration term,
[Equation]
x ))] with the differences in log variances plus observation noise,
[Equation]
where σ ( m ) ( x ))( x ) is the observation variance for objectives m = 1 , . .
, M at x ∈ X .
Using thisadjustment, we deﬁne the resulting conditional entropy estimate by
[Equation]
Empirically, we observe that the performance of this conditional entropy estimate is in-line with theother conditional entropy approximations.
E Monte Carlo entropy estimate
In this section, we consider estimating the entropy of p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) via Monte Carlo.
Asa reminder, the entropy of interest can be written as an expectation over p ( y | x , D n ∗ ) :
[Equation]
The CDF in the denominator is p ( f ( x ) (cid:22) Y ∗ | x , D n ∗ )
=
W from (24), whilst the CDF in the numerator is
[Equation]
from (25).
By sampling y i ∼ p ( y | x , D n ∗ ) for i = 1 , . . .
, I , we can approximate the entropy withthe following Monte Carlo average:
[Equation]
where the Monte Carlo entropy estimate is given by
[Equation]
Instead of generating new samples for each call of the acquisition function, we follow the generalwisdom in BO [84] and apply the reparametrization trick on the sampling distribution: y i = µ n ∗ ( x )+
C n ∗ ( x ) z i , where C n ∗ ( x ) ∈ R M × M is the Cholesky factor of Σ n ∗ ( x , x ) and z i ∼ N (0 , I M ) arethe base samples that only need to be initialized once.
The variance of this estimate could potentiallybe reduced by including control variates.
For example,
[Equation]
where the expectations are known,
[Equation]
and β 1 , β 2 ∈ R are the regression coefﬁcients.
We did not assess the effect adding control variates toour Monte Carlo estimate because the standard quasi-Monte Carlo scheme with the reparametrizationtrick worked reasonably well out of the box.
One word of caution [42]: combining both controlvariates and quasi-Monte Carlo could lead to an increase in variance if the coefﬁcients are naivelyestimated.
F Submodularity
A set function g : 2 V → R is submodular if for every A, B ⊂ V , g ( A ∩ B )+ g ( A ∪ B ) ≤ g ( A )+ g ( B ) ,where V is the set of interest—for further details on submodularity refer to [50].
The followingproposition states that the lower bound batch acquisition function and its approximation for the JESand MES are submodular functions deﬁned over subsets of the input space.
Proposition 5. The lower bound batch acquisition function α qLB-JES ( X | D n ) and its approximationsˆ α qLB-JES ( X | D n ) are submodular functions deﬁned over subsets of X .
Proof.
Let X
= { x i } i =1 ,...,q ⊂ X denote a set of inputs.
If K X ∈ R q × q is a positive semi-deﬁnitematrix such that K i,j depends only on the inputs x i ∈ X and x j ∈ X , then the log-determinantfunction log det K X deﬁned over sets X is submodular [51].
Similar to [56], we will show that thislower bound batch acquisition function can be written as a sum of these log-determinants.
[Equation]
[Equation]
[Equation]
where
[Equation]
For notational convenience, let A ( m ) =
Σ ( m ) n ( x [1: q ] , x [1: q ] ) + diag ( σ ( m ) ( x [1: q ] )) for m = 1 , . .
, Mand B = diag (cid:16) e ζ ( x [1: q ] ) /M
(cid:17) .
Combining the terms, we obtain
[Equation]
m = (cid:16) log det
[Equation]
+
[Equation] M [Equation] m =
[Equation]
log det K ( [Equation] ) , where K ( m ) = BA ( m ) B with K ( m ) i,j = e ζ ( x [ i ] ) /M
e ζ ( x [ j ] ) /M
A ( m ) i,j for i, j = 1 , . . .
, q and m =1 , . .
, M .
Each summand, log det K ( m ) , is a submodular function [51].
As a sum of submodularfunctions is submodular, we conclude using that the lower bound batch acquisition function issubmodular.
For the approximate batch acquisition function, the expectation in (31) is replaced bythe corresponding Monte Carlo estimate.
The submodularity derivation above continues to hold whenusing the conditional entropy approximations.
[Equation]
G Cost analysis
In this section we consider the costs involved in evaluating the JES acquisition function according to
Algorithm 1.
We also include some discussion of the cost of the other information-theoretic criterion,namely MES and PES.
Sampling cost.
The cost of approximate sampling from a single Gaussian processes p ( f ( m ) | D n )using the random Fourier features (described brieﬂy in Appendix A.3) is O (min( n, L ) 3 ) , where L isthe number of Fourier features—for more details refer to [85].
An evaluation of a sample at a set ofinputs X ⊂ X has a linear cost depending on | X | [85].
Multi-objective optimization cost.
Given a sample f s , we optimize for p Pareto optimal points( X ∗ s , Y ∗ s ) using the popular genetic algorithm known as NSGA2 [21].
The cost of this algorithm is
O ( MN 2 pop N gen N off ) , where N pop is the size of the population, N gen is the number of generations and
N off is the number of offspring[21].
At a high level, NSGA2 works by evaluating the function at
N pop locations at time t = 1 and then moves on to evaluate N off candidates for the rest of the timet = 2 , . .
N gen .
The location of the offspring evaluations are determined by a random heuristicmotivated by the mechanisms involved in the theory of evolution: crossover, mutation, elitism anddiversity.
Box decomposition.
The cost of performing a single the box decomposition based on the incremental algorithm in [52] is O ( p (cid:98) M/ 2 (cid:99) +1 ) , where p is the number of Pareto optimal points.
Conditioning cost.
Conditioning on an additional p data-points requires updating the Choleskydecomposition of the input covariance matrix.
The cost of a single update of this kind relies on atriangular solve, which has a quadratic complexity
O ( M ( n + p ) 2 ) .
Initial entropy evaluation cost.
The initial entropy (4) can be computed directly from the posterior covariance.
The cost of instantiating the caches of the Gaussian process covariance, namely(Σ ( m ) 0 ( X n , X n ) + diag ( σ ( m ) ( X n )) − 1 is O ( n 3 ) .
The cost of evaluating the posterior covariance(14) at point x ∈ X is O ( n 2 ) .
In the batch case, we also have to compute a log-determinant of theq × q covariance matrix, which has a cost of O ( q 3 ) .
We use the implementation in the GPyTorch[29], which computes the approximate log-determinant, which only uses a linear cost of O ( q ) —thisapproximation approach is outlined in [24].
Conditional entropy evaluation cost.
Assuming the posterior model and conditioned model areinstantiated, we will consider the operations involved when evaluating the conditional entropyestimates.
The cost of evaluating the probability density function and CDF of a univariate normaldistribution at a point x ∈ X are both assumed to be O (1) .
As a result, the dominant cost ofevaluating the h JES- 0 (27) and h JES-LB 2 (10) comes from the evaluation of the variance.
For h JES-LB(9), we additionally have to populate an M × M matrix and compute its log-determinant.
For h JES-MC(28), we need to initialize a set of I base samples for the reparametrization trick [84] described in
Appendix E—this has a linear one time cost of O ( MI ) .
Assuming the Monte Carlo samples aregenerated, the rest of the operations depend linear on I .
Expectation propagation.
To approximate the density p ( y | x , D n , X ∗ ) in the PES acquisitionfunction (1), the authors of [30] consider using expectation propagation [55].
The dominant cost ofthis is from matrix inversions of the covariance matrix, which scales cubically with the number of datapoints in consideration.
In the initialization phase, we prepare the expectation propagation caches,which require inverting an ( n + p ) × ( n + p ) matrix, whilst during testing we invert a ( q + p ) ×
( q + p )matrix, where q is the batch size.
Total cost.
The total cost is the sum of the initialization cost and the query cost.
Using the variablesdeﬁned in Table 1 and Table 2, we can write down the initialization and query cost of JES, MES andPES.
For the calculations, we assume that we have already queried n points and we want to computethe acquisition function at a batch of q points using S Monte Carlo samples of the Pareto set and/orfront comprised of p points.•
JES.
The initialization cost is C init + SC sample ( L )+ SC moo ( N pop , N gen , N off )
+ SC bd ( p )+ SC cond ( p ) .There is an additional initial cost of SC base-sample ( I ) for the Monte Carlo conditional entropyestimate.
The query cost is C init-h ( q ) + SqC h ( p ) , where C h ∈ { C h-0 , C h-lb , C h-lb2 , C h-mc } is theconditional entropy estimation strategy.
Keeping the other parameters ﬁxed, the dominant querycost is Mq 2 when using the approximate log-determinant and Mq 3 when using the standardlog-determinant for large q .• MES.
We consider the MES algorithm obtained by excluding the conditioning step described in Algorithm 1.
The initialization cost of this approach is C init ( q ) + SC sample ( L )
+SC moo ( N pop , N gen , N off )
+ SC bd ( p ) .
The query cost is C init-h ( q ) + SqC h (0) , where C h ∈{ C h-0 , C h-lb , C h-lb2 , C h-mc } is the conditional entropy estimation strategy.
Keeping the other parameters ﬁxed, the dominant query cost is Mq 2 when using the approximate log-determinant and Mq when using the standard log-determinant for large q . • PES.
We consider the expectation propagation approach described in [30] to approximate the PESacquisition.
The batch extension is derived in a follow-up work [32].
The initialization cost of this approach is C init ( q ) + SC sample ( L )
+ SC moo ( N pop , N gen , N off )
+ SC
ep0 ( p ) .
The query cost isC init-h ( q ) + SC ep ( q, p ) .
Keeping the other parameters ﬁxed, the dominant query cost is SMq 3 for large q .
Naturally the initialization phase can be executed in parallel because we are running S independentoperations.
For our experiments, we did not take advantage of this advantageous property when performing the sampling multi- objective
optimization and box decomposition [Table]
Reference
Cost
Initializing the posterior p ( f | D n )
C init
Mn
Log-determinant of a K × K matrix
C logdet ( K )
K
Approximate log-determinant of a K × K matrix
C alogdet ( K )
K
Generating I base samples from N (0 , diag ( 1 M ))
C base-sample ( I )
MI
Approximate sampling of p ( f | D n )
C sample ( L )
M min( n, L )
Multi-objective optimization of f s
C moo ( N pop , N gen , N off )
MN 2 pop N gen N off
Box decomposition of Y ∗ s with p points
C bd ( p ) p (cid:98) M/ [Table] (cid:99)
[Table] Conditioning on the Pareto optimal point p ( f | D n ∗ )
C cond ( p )
M ( n + p )
Initialization of expectation propagation caches
C ep0 ( p )
M ( n + p )
[Table]
C
[Table]
( | X | , n )
M ( | X | n 2 + | X | 2 )
[Table]
C
[Table]
C [Table]
( q, n ) + MC [Table] [Table]
Conditional entropy estimate h JESC h-0 ( p )
C cov (1 , n + p )
Conditional entropy estimate h JES-LB
C h-lb ( p )
C cov (1 , n + p ) + M 2 + C logdet ( M )
Conditional entropy estimate h JES-LB
C h-lb2 ( p )
C cov (1 , n + p )
Conditional entropy estimate h JES-MC
C h-mc ( p ) C cov (1 , n + p ) + MI
Expectation propagation
C ep ( q, p )
M ( q + p )
H Contour plots
In this section we present some contour plots to illustrate visually the differences that emergesbetween the different approximation strategies for the information-theoretic acquisition functionson a single objective problem.
The approximation for PES, MES and JES are presented in Figure 7,
Figure 8 and Figure 9, respectively.
To obtain the ground truth for the acquisition function we runthe rejection sampling scheme discussed in [39].
As a comparison we also include the contours of some popular single objective acquisition functions.
Visually the estimates all to effective the PES
[Table]
1.0 x 1 0.
x
Rastrigin 0.
1.0 x 1 0.
x0 .0 .0 .0 .0 . [Table] [Table] [Table]
[Table]
1.0 x 1 0.
x
Gaussian Process Entropy 1 .1 .1 .1 .1 .1 .1 .1 .1 .2 .2 .2 .2 .2 .2 .2 .2 .
1.0 x 1 0.
x
PES ( x | D n ) 0 .0 .0 .0 .0 .0 .0 .0 .
1.0 x 1 0.
x
PES ( x | D n ) 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .
1.0 x 1 0.
x
TS ( x | D n ) 0 .
entropy search acquisition and its approximation obtained via expectation propagation.
For reference we also include a random Thompson sample.
The location of the maximizer is highlighted using a pair of dotted black lines.
1.0 x 1 0.
x
MES ( x | D n ) 0 .0 .0 .0 . 00 .0 .0 .0 .0 .
1.0 x 1 0.
x
MES 0 ( x | D n ) 0 .
1.0 x 1 0.
x
MES LB ( x | D n ) 0 .0 .0 .0 .0 .0 .0 .0 .0.
1.0 x 1 0.
x
MES MC ( x | D n ) 0 .0 .0 .0 .
1.0 x 1 0.
x0 .0 .0 .
JES ( x | D n ) 0.
1.0 x 1 0.
x
JES 0 ( x | D n ) 0 .0 .0 .0 .
1.0 x 1 0.
x
JES LB ( x | D n ) 0 .0 .0 .0 .0 .0 .
1.0 x 1 0.
x
JES MC ( x | D n ) 0 .0 .0 .
1.0 x 1 0.
x
EI ( x | D n ) 0 .0 .
1.0 x 1 0.
x
PoI ( x | D n ) 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0.
1.0 x 1 0.
x
UCB ( x | D n ) 0 . 72 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .
1.0 x 1 0.
x
KG ( x | D n ) 0 .0 .0 .0 .
I Hypervolume indicator
The hypervolume indicator is deﬁned as the area of the dominated region between a reference pointr ∈ R M and the set of interest A ⊂ R M :
[Equation]
As discussed in the main text (Section 4), the HV indicator is sensitive to the parametrization of theobjective space.
In order to build a more complete picture about the performance of a multi-objectiveoptimization algorithm, we consider tracking the HV under a number of different parameterization.
Instead of deriving a family of transformation function, we consider a more general approach basedon a different formulation of the HV indicator described in [22, 90].
In particular, the HV indicatorcan be written as an expectation over a probability distribution by performing a change of variablesinto spherical polar coordinates:
[Equation]
where Γ( · ) is the Gamma function and the scalarization function s λ : R M → R is deﬁned as
[Equation]
For the standard HV, the inverse weight distribution p ( λ ) is a uniform distribution over the surface ofthe M -dimensional unit sphere in the non-negative orthant S M − 1 + [Equation] [Equation] [Equation]
Mm [-=
-]= [Equation]
λ [Equation] m ) )
2 =1 } .
The inverse weight distribution p ( λ ) controls the radial contribution for each point of A towardsthe HV.
Existing work mainly considers a uniform distribution over the weights in order to computethe standard HV.
We make the novel observation that we can assess the quality of the Pareto front indifferent regions of the objective space by varying this weight distribution.
We call the resultingutility function the generalized hypervolume (GHV) indicator, denoted by U GHV {+.
+}
This GHV satisﬁes a weaker version of the Pareto complete property: A (cid:23) B = ⇒ U GHV ( A ) ≥
U GHV ( B ) .
This property can be proved from the fact that the scalarization function s λ ( a ) is amonotonic increasing function in a ∈ A , which implies that the Pareto order is maintained.
Thisperformance criteria satisﬁes the strict Pareto complete property when all of the weights have non-zerodensity p ( λ ) > 0 .
Intuitively, if a set of weights had zero density, then an improvement along thesedirection will not always result in a larger GHV.
This means that we will not always be able to distinguish between a set which strictly dominates another, which is essential for strict Pareto completeness.932 To generate different distributions p ( λ ) , we exploit the fact that the set S M − 1 + is isomorphic to the ( M − 1) -dimensional unit hypercube [0 , 1] M − 1 .
In particular, any point w ∈ [0 , 1] M − 1can be mapped onto the sphere to a point λ ( w )
∈ S M − 1 + with
[Equation]
[Figure] [Figure] [Equation] Using this result, we can use any distribution with a ﬁnite support to generate samples on S M − 1 + .
For example, we use M − 1 independent Beta distribution Beta ( a m , b m ) to generate samples fromw ∈ [0 , 1] M − 1 .
In Figure 12, we present the radial contributions to the GHV for different weightdistributions p ( w ) .
Remark.
We are not the ﬁrst to observe the importance of weighting the HV towards regionsof interest.
For example, an early paper [95] proposed another a weighted hypervolume indicator(WHV), U WHV ( A ) =
(cid:82) R M w ( z ) I
[ r (cid:22) z (cid:22) A ] d z , which introduces a weight function w : R M → Rdirectly into the integral.
This approach is not as ﬂexible as the one presented here because it reliedon designing hand-crafted weight functions and an involved symbolic integration procedure.
J Experiments
J.1 Implementation details
All of the numerical results was implemented on Python 3.8 using open-source libraries: BoTorch(0.5.1) [3], GPyTorch (1.6.0) [29], NumPy (1.21.2), Pymoo (0.5.0) [10] [36], PyTorch (1.9.0)
[63]and SciPy (1.7.3) [82].
All computations were performed on a computing cluster, where we restrictedthe computation to a single CPU core of an AMD EPYC 7742 64-Core Processor @ 2.25GHz.
Thecode is available at https://github.com/.../JES .
Gaussian process model.
For all the problems, we normalized the inputs and standardized theobservations before performing Gaussian process regression.
We assume an independent modelfor each objective with a constant mean and Matérn 5/2 ARD kernel.
Additionally, we assumea zero-mean Gaussian observation noise with a homogeneous variance σ ( x ) = σ ∈ R M .
Forconvenience, we rely on the default model in BoTorch, which additionally places Gamma priors forthe kernel hyperparameters and observation variance.
At each iteration of BO, we update the pointestimates for the model hyperparameters and observation variance by maximizing the exact marginallog-likelihood using SciPy’s default gradient optimizer.
Optimizing the acquisition function.
To determine the next point, we optimize the acquisitionfunction using multi-start L-BFGS-B [82].
We use the exact gradient inferred using automaticdifferentiation [63] for all algorithms except for the PES.
The automatic gradients inferred for the
PES acquisition function occasionally failed due to the differentiability issues arising from thedamping procedure within the expectation propagation update [30].
As a result, we used the default2-point ﬁnite difference method in SciPy to approximate the gradients of the PES acquisition function.
To initialize the multi-start gradient optimizer, we evaluated the acquisition function on a space-ﬁllingdesign of 1000 D random points.
The starting points were then chosen as the best performing pointsfrom the initial check.
The number of starting points that we used differed for each algorithm basedon the overall expenses.
For the sequential experiments, we used 20 starting points for PES, 5 Dstarting points for the MES and JES, and 10 D starting points for the rest.
For the batch experiments,we used 20 starting points for PES and 5 D starting points for the rest.
For the information-theoreticacquisition functions, we could have initialized the starts near the sampled Pareto optimal points inorder to promote faster convergence.
We abstained from doing this because we wanted to give a faircomparison between the existing approaches.
Speciﬁcally, we wanted to see how well informationbased acquisition function worked out of the box without any optimization compared to the existingoptimized approaches.
Sampling the Pareto points.
To sample the Pareto optimal inputs and outputs, we ﬁrst sample
Gaussian process paths.
We achieved this by using an approximate sampling strategy based onrandom Fourier feaures (Appendix A.3).
We used the implementation in BoTorch and set the numberof features to L = 500 .
To solve for the Pareto set and front, we used the multi-objective solver
NSGA2 [21], which was implemented in Pymoo.
We set the population size to be N pop = 100 , thenumber of generations to be N gen = 500 and the number of offspring to be N off = 10 .
In general,the multi-objective solver outputs an approximate set of Pareto optimal points with a size less thanor equal to N pop .
To truncate this set into a size p , we used a HV truncation strategy.
In particular,we greedily selected points based on their contribution to the sample HV generated by the samplef s ( X n ) —the reference point is set to the current estimate of the nadir minus some error ˆ r n − 0 .
1 | ˆ r n | ,where ˆ [Equation]
This reﬁnement strategy can be implemented quickly using theexpected HV improvement strategy discussed in [17].
We implemented this sampling and optimizingprocedure in sequence, but naturally this can be executed in parallel because we are performing Sindependent computations.
As motivated by the wall times in Appendix J.7, implementing this stepin parallel could be very beneﬁcial computationally.
Box decompositions.
We used the BoTorch implementation of the box decomposition strategydiscussed in Algorithm 1 of [52].
The box decompositions were performed in sequence instead ofbeing executed in parallel.
From the wall times presented in Appendix J.7, the time required tocompute the box decompositions becomes more demanding as the number of objectives increases.
Conditioning.
We used the fantasizing feature in BoTorch to condition the current posterior ona collection of S independent Pareto samples of size p .
We treated the Pareto samples as noisypseudo-observations in the conditioning.
Acquisition functions.
The benchmark comparison considers a range of popular acquisition functions that have all been implemented in BoTorch.
We implemented our own version of the multiobjective PES, MES and JES.
We now elaborate on the implementation details for each acquisitionfunctions in the benchmark experiments.• TSEMO.
The Thomson sampling algorithm (TSEMO) [12] is a random acquisition function,which selects the point that maximizes the HV improvement according to a single sample of thePareto front.
Unlike the original paper [12], we select the point that improves the HV of the samplefrontier f s ( X n ) as opposed to the observation frontier generated by Y n .
We ﬁnd this adjustment tobe sensible when there is observation noise.
We modiﬁed the greedy batch extension in the sameway.• ParEGO/ NParEGO.
The random scalarization strategy (ParEGO) [48] and its noisy counterpart(NParEGO) [18] are one of the most popular strategies for multi-objective BO.
At each iteration, arandom scalarization of the objectives is obtained by randomly drawing a weight w ∈ R M fromthe ( M − 1) -simplex.
To target all the Pareto optimal points, we use the augmented Chebyshevscalarization: min m =1 ,...,M w ( m ) ˜ f
( m ) + 0 .
01 (cid:80) m
=1 w ( m ) ˜
f
( m ) .
Here we have denoted ˜ f
∝ f asthe objective function, which has been approximately normalized to [0 , 1] M using the observations Y n .
For the single objective problem, a Monte Carlo estimate of the expected improvement andthe noisy expected improvement for the ParEGO and NParEGO algorithm are used respectively.
The number of base samples for the Monte Carlo estimates is set to 128 .
For the batch setting, wesampled q different weights and optimized the acquisition function sequentially. • EHVI/ NEHVI.
The expected hypervolume improvement (EHVI) [17] and its noisy counterpart(NEHVI) [18] are an improvement-based acquisition function for the HV indicator.
The numberof base samples for the Monte Carlo estimates is set to 128 .
We set the reference point of theHV indicator to be equal to the observed nadir minus some error, ˆ r n − 0 .
1 | ˆ r n | , where ˆ r ( m ) n =min t =1 ,...,n y ( m ) t .
This dynamic reference point strategy was recommended in the supplementarymaterial of [17].
For the batch setting, we considered a sequentially greedy optimization strategy[17, 18].• PES.
The predictive entropy search (PES) [30, 32, 38, 39] acquisition function is approximatedusing expectation propagation.
We implemented this algorithm from scratch in BoTorch underthe guidance of the supplementary material in [30].
We used S = 10 Monte Carlo samples andp = 10 number of Pareto optimal inputs.
We set the jitter for the matrix inversion to be 0 .
001 andthe convergence threshold for the initialization stage to be 5% relative change in the mean andcovariance.
If the expectation propagation failed to converge, we outputted a random vector fromthe already sampled Pareto sets as the ofﬁcial recommendation.
For the batch setting, we extendedthe approach outlined in [30].
This extension appears to be equivalent to the approach described in[32].
We optimized the resulting acquisition function using a joint optimization approach.• MES.
The maximum value entropy search (MES) [77, 83] can be approximated using all theconditional entropy estimates we devised in this paper.
We used S = 10 Monte Carlo samplesand p
= 10 number of Pareto optimal outputs.
For the Monte Carlo estimate MES-MC we set thenumber of base samples to 128.
For the batch setting, we considered a greedy optimization strategy[17, 18].
For the batch setting, we consider the lower bound described by (11).
We optimized theresulting acquisition function using a greedy optimization approach • JES.
The joint entropy search (JES) [77, 83] can be approximated using all the conditional entropyestimates we devised in this paper.
We used S =
Monte Carlo samples and p = 10 number ofPareto optimal points.
For the Monte Carlo estimate JES-MC we set the number of base samplesto 128.
For the batch setting, we consider the lower bound described by (11).
We optimized theresulting acquisition function using a greedy optimization approach
Optimizing for the recommendation.
To obtain the recommendation set, ˆ X ∗ n , we used the NSGA2multi-objective solver to optimize the posterior mean µ n .
Using the Pymoo implementation, we setthe population size to be N pop = 500 , the number of generations to be N gen = 500 and the numberof offspring to be N off = 10 .
We select the p = 50 points that greedily maximizes the HV generatedby the mean objectives.
The reference point for the HV truncation was set to the current estimateof the nadir minus some error:
ˆ r n − 0 .
1 | ˆ r n | , where ˆ r ( m ) n = min t =1 ,...,n y ( m ) t .
Before applying thetruncation, we ﬁrst augmented the Pareto set found at this iteration with the previous recommendationset ˆ X ∗ n − 1 in order to guarantee that some promising solutions weren’t missed due to randomness ofthe solver.
Optimizing for the Pareto set.
To obtain the baseline Pareto set, X ∗ , we used the Pymoo’s NSGA2with a population size to be N pop = 1000 , the number of generations to be N gen = 5000 and thenumber of offspring to be N off = 10 .
J.2 Benchmark problems
We initialized each test problem with with 2( D + 1) training points using a random space ﬁllingdesign.
The details of the individual benchmark problems is presented below.
[Equation]
The ZDT2 benchmark [93] is a bi-objective minimization problem min x ∈ X f ( x ) , over the D -dimensional hypercube X
=
[0 , 1] D , where
[Equation]
with g ( x ) = 1 − 9 D − 1 (cid:80) Di =2 x ( i ) .
For the experiments, we considered the maximization problem bynegating the objective: max x ∈ X ( − f ( x )) .
The standard deviation of the Gaussian observation noise is set to σ 1 / 2 = (0 . 1 , 0 .
8) , which is estimated to be around 10% the range of the objectives from1000 function evaluations.
For the HV indicators we use the reference point of r = ( − 11 , − 11) .
SnAr (D=4, M=2, Noise=3%).
The SnAr benchmark [26] is a bi-objective minimization problemmin x ∈ X f ( x ) , which is deﬁned over a four-dimensional space, X =
[0 . 5 , 2 . 0] × [1 . 0 , 5 . 0] × [0 . 1 , 0 . 5] ×[30 , 120] , where
[Equation]
with STY and E-Factor denoting the space-time yield and environmental factor, respectively.
Thefunctions STY and E-Factor depend on the solution of an ordinary differential equation governedby a kinetic model, where the rate constants have been estimated by empirical tests [44].
Unlike theoriginal paper [26], we have taken the logarithm of the output to accommodate for the additive noisein the objective space.
For the experiments, we considered the maximization problem by negatingthe objective: max x ∈ X ( − f ( x )) .
The standard deviation of the Gaussian observation noise is setto σ 1 / 2 = (0 .
12 , 0 . 08) , which is estimated to be around 3% the range of the objectives from 1000function evaluations.
For the HV indicators we use the reference point of r = (5 . 5 , − 5) .
Penicillin (D=7, M=3, Noise=1%).
The Penicillin benchmark [53] is a three objective minimization problem min x ∈ X f ( x ) , which is deﬁned over a seven-dimensional space, X = [60 , 120] ×[0 . 05 , 18] ×
[293 , 303] × [0 . 05 , 18] × [0 . 01 , 0 .
5] × [500 , 700] × [5 , 6 . 5] , where
[Equation]
with PenicillinConcentration , CO2Concentration and TimeToFerment denoting the concentration of the desirable product, the concentration of the subproduct and the time to ferment,respectively.
The equations describing the pharmaceutical simulation are given in the original reference [53].
We used the open-source implementation available in BoTorch [3, 20].
For the experiments,we considered the maximization problem by negating the objective: max x ∈ X ( − f ( x )) .
The standarddeviation of the Gaussian observation noise is set to σ 1 / 2 = (0 . 14 , 0 . 8 , 3 .
8) , which is estimated tobe around 1% the range of the objectives from 1000 function evaluations.
For the HV indicators weuse the reference point of r = ( − 1 . 85 , − 86 .
93 , − 514 .
Marine Design (D=6, M=4, Noise=0.5%).
This marine design benchmark [62, 70, 80] is a fourobjective minimization problem min x ∈ X f ( x ) , which is deﬁned over a six-dimensional space, X =[150 , 274 . 32] × [20 , 32 . 31] × [13 , 25] × [10 , 11 .
71] × [14 , 18] × [0 . 63 , 0 .
75] , where
[Equation]
with TransportationCost , Weight , AnnualCargo and SumOfConstraints denoting the transportation cost, the ship weight, the annual cargo transport capacity and the sum of the constraintviolations, respectively.
The formal equations describing the functions are presented in the original references [62, 70, 80].
For the experiments, we considered the maximization problem bynegating the objective: max x ∈ X ( − f ( x )) .
The standard deviation of the Gaussian observationnoise is set to σ 1 / 2 = (10 , 77 , 132 , 0 . 07) , which is estimated to be around 0 .
5% the range of theobjectives from 1000 function evaluations.
For the HV indicators we use the reference point ofr
= (250 , − 20000 , − 25000 , − 15 . 0) .
J.3 Sensitivity analysis
In the section we empirically analyse how sensitive the approximations to the information-theoreticacquisition functions are with different choices of Monte Carlo samples S and number of Pareto optimal points p .
For the experiments in the main section, we set S = 10 and p = 10 for allinformation-theoretic acquisition functions.
This selection is comparable to the existing literature[4, 30, 77, 83].
For the sake of brevity we present results only for one benchmark problem: ZDT2(D=2, M=2, Noise=10%, {+q=1).
+} To test the sensitivity with regards to the number of Monte Carlo samples S , we ﬁx p = 10 and ranthe benchmark problem 100 times with S ∈ { 1 , 5 , 10 , 25 , 50 } .
To test the sensitivity with regards tothe number of Pareto optimal points p , we ﬁx S = 10 and ran the benchmark problem 100 times withp ∈ { 1 , 5 , 10 , 25 , 50 } .
We report the mean log HV discrepancy with two standard errors over theruns in Figure 13 and Figure 14.
We report the wall times in Figure 15.1113 Performance-wise there does not appear to be much variation for acquisition function when
S > 1 and p > 1 .
Naturally the wall times increase with the number of Monte Carlo samples Sbecause the sampling and optimization of the Gaussian process paths are done in sequence (this stepcould be parallelized in practice).
Overall, for this example problem, there does not appear to bemuch beneﬁt in using a larger number of Monte Carlo samples S or Pareto optimal points p .
[Figure]
[Figure]
[Figure]
In this section we plot the results of the standard hypervolume computed using different stategies toselect the approximate Pareto set ˆ X ∗ . Maximizing the posterior mean over the whole input space: ˆ X ∗ = arg max x ∈ X µ N ( x ) .2.
Maximizing the posterior mean over the sampled points: ˆ X ∗ = arg max x ∈ X N µ N ( x ) .3. Using the complete data set: ˆ X ∗ = X N . This strategy is questionable when there is noisepresent in the data, but we report it for completeness.
We present the results of the logarithm standard HV discrepancy over 100 runs for the sequentialand batch problem in the following subsections under these three strategies. Our main ﬁnding isthat the conditional entropy estimates perform similarly across the board in terms of the standardhypervolume. As a result, we advocate the use of the cheapest estimate, which are typically the lowerbound estimates.
Another ﬁnding we made was that the entropy based methods have a tendency notto directly query the best performing points but instead opt for more informative points which havehigher uncertainty. As a result of this behaviour, information-theoretic algorithms tend to performwell in strategy 1 and less so on strategy 2 and 3. If directly querying high-performing points isimportant, it might be advantageous to use an epsilon greedy strategy, where points are occasionallypicked greedily according to the posterior mean. [Figure]
In this section, we set the approximate Pareto set to be ˆ X ∗ = arg max x ∈ X µ N ( x ) . [Figure]
ZDT [Figure] D
[Figure] [Figure] M=2, Noise=10%, q=1) MES
[Figure]
MES-LB
MES
[Figure]
Each row focusses on comparing a different subset of algorithms.
[Figure] J.4.2 Strategy In this section
[Figure]
we set the approximate Pareto set to be ˆ X ∗ =
arg max x ∈ X N µ N ( x ) . [Figure] [Figure] J.4.3 Strategy
In this section
[Figure]
we set the approximate Pareto set to be ˆ X ∗ =
X N .
[Figure] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] J.5 Generalized hypervolume
[Figure]
In this section we produce proﬁle plots of the generalized hypervolume at the ﬁnal time instance. Weonly consider the median performance from the multiple runs when setting the approximation set tobe the maximum of the ﬁnal posterior mean: ˆ X ∗ = arg max x ∈ X µ N ( x )
[Figure]
J.5.
[Figure] [Figure]
[Figure]
w )
[Figure]
[Figure]
[Figure]
(D=2, M=2, Noise=10%, q=1)
[Figure] [Figure]
[Figure]
J
[Figure]
5.3 SnAr (D=4, M=2, Noise=3%, q=1)
[Figure]
( d GHV )
[Figure]
(D=
[Figure]
M=2, Noise=3%, q=1) 3.
map comparison of the ﬁnal median logarithm GHV discrepancy.
The weight distributions are described in Figure 22.
J.5.4 Penicillin (D=7, M=3, Noise=1%, q=1)
[Figure]
NEHVIEHVI
NParEGOParEGOTSEMOSobol
[Figure]
q=
[Figure] [Figure] [Figure] D
[Figure]
NParEGOParEGOTSEMOSobol
[Figure] [Figure] ):ZDT2 (D=6, M=2, Noise=10%, q=8) [Figure]
M
[Figure]
q
[Figure]
Penicillin
[Figure] [Figure] [Figure]
J.6 Visualizations
[Figure]
In this section we present the plots of the best points identiﬁed by the posterior mean at the ﬁnal
[Figure]
time for the different algorithmsm. Out of the multiple runs, we select the run that attains the lower
[Figure]
quartile of the log HV discrepancy. [Figure] [Figure] 1 ZDT2(D=2, M=2,
[Figure] [Figure]
q=1)
[Figure] [Figure]
SnAr (D=
[Figure]
M=2, Noise=3% [Figure] q [Figure]
[Figure]
log( d HV )
= -2.
EHVI log( d HV )
= -2.
NEHVI log( d HV )
= -2.
PES log( d HV )
= -2.
MESlog( d HV ) = -2. MES-LB log( d HV )
= -2.
MES-LB log( d HV ) = -2. MES-MC log( d HV )
= -2. JESlog( d HV ) = -2.
JES-LB log( d HV )
= -2.
[-[Figure] =2, Noise=3%, q=8) 
-]JES-LB log( d HV ) = -2.
JES-MC
J.6.4 Marine (D=6, M=4, Noise=0.5%, q=1) J.
.5 ZDT
(D=2, M=2, Noise=10%, q=8)
[Figure]
J.6.6 SnAr (D=4, M=2, Noise=3%, q=
[Figure]
LB-MESlog( d HV ) = -2. LB-MES-LB log( d HV ) = -2.
LB-MES-LB log( d HV ) = -2. LB-MES-MC log( d HV ) = -2.
LB-JESlog( d HV ) = -1. LB-JES-LB log( d HV )
= -2.
LB-JES-LB log( d HV )
= -2.
LB-JES-MC
J.6.7 Penicillin (D=7, M=3, Noise=1%, q=8)
J.6.8 Marine (D=6, M=4, Noise=0.5%, q=8)
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Figure]
K Extensions
The discussion so far focussed on the problem of unconstrained multi-objective optimization where evaluations of the vector-valued function are executed individually or in batch.
Not all multi-objective optimization fall into this category, so we propose some additional extensions to handle more generalproblems.
Constrained optimization.
In the literature there are several examples of how to generalizeinformation-theoretic acquisition functions to constrained optimization problems [6, 27, 30, 32, 40,41, 64, 79].
We propose a simple extension for the JES acquisition function, similar to [79], to handleinequality constraints.
In particular, suppose we are interested in maximizing an M -dimensionalblack-box function f (1: M ) ( x ) subject to K black-box inequality constraints f
( M + k ) ( x ) ≤ 0 fork = 1 , . .
, K .
We can model the constraints f
( M + k : M + K ) ( x ) as additional independent objectivesusing the same observation model described in Section 2.
The only difference between the constrainedsetting and the unconstrained setting is the region of integration for the CDF.
In the constrainedsetting, the sampled Pareto front dominates only the vectors satisfying the constraint.
Hence theconstrained CDF is now of the form p ( z ∈ D K (cid:22) ( Y ∗ ))
where
[Equation] { z [Equation] K
[Equation]
z
[Equation]
(cid:22) [Equation] z
[Equation]
M +
[Equation]
M + K
[Equation]
(cid:22) 0 K ) or ( z
[Equation]
M +
[Equation]
M + K
[Equation]
{+(cid:31) 
+}
K
[Equation]
This region can be decomposed into boxes in the almost same way as before.
The only differenceis that we now have an additional box arising from region where the constraint is not satisﬁed.
Ingeneral, the JES (and MES) acquisition function estimates described here can handle any type ofblack-box constraint as long as we are able decompose the feasible objective region into boxes.
Forexample, interval constraints of the form f
( M + k ) ( x ) ∈ [ a, b ] , can also be readily handled in thisframework.
Decoupled evaluations.
Evaluating all objectives at each iteration can be costly and perhapsunnecessary for practical problems.
To address this problem, researchers in BO have consideredthe use of decoupled [33] acquisition functions α M , which considers the quality of querying asubset of objectives { f ( m ) : m ∈ M} .
To the best of our knowledge, all of the existing decoupledacquisition functions in multi-objective BO are based on information-theoretic acquisition functions[30, 38, 41, 77].
The novel JES-LB2 and MES-LB2 described in this paper possesses this decouplingproperty because it can be decomposed into a sum of acquisition functions for each objective:α JES-LB2 ( x | D n ) =
(cid:80)
Mm =
α m ( x | D n ) .
By Theorem 4.1 of [77], the JES-0 and MES-0 can also begeneralized to the decoupled setting via a marginalization argument.
Upon reviewing the experimentalresults of the cited papers, we see that decoupled evaluations typically provide only a marginalimprovement over the non-decoupled strategies.
This is possibly down to heuristic choice to searchthe space of subsets.
A more principled search method based on some emerging ideas about how tooptimize over categorical inputs [31, 59, 69] might be useful to obtain further improvements.
Multi-ﬁdelity Bayesian optimization.
For practical optimization problems of interest, it is occasionally possible to evaluate approximations of the true objectives that are much cheaper.
Thisadditional degree of freedom has been exploited before in literature under the name of multi-ﬁdelity
Bayesian optimization [5, 6, 46, 56, 57, 75, 78, 88, 91].
These strategies have demonstrated somebeneﬁt when optimizing with cost constraints on the function evaluations.
It is possible to adapt
JES to the multi-ﬁdelity by combining the ideas introduced here with the ideas before in the papersreferred to above such as using cost weights and conditioning arguments on lower ﬁdelities.
Themain obstacle to extending this work to the multi-ﬁdelity setting will likely arise from some lengthyalgebraic exercises relating to the conditional entropy and box decompositions.