Joint Entropy Search for Multi-Objective Bayesian Optimization
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Many real-world problems can be phrased as a multi-objective optimization problem, where the goal is to identify the best set of compromises between the competing objectives.
Multi-objective Bayesian optimization (BO) is a sample efﬁcientstrategy that can be deployed to solve these vector-valued optimization problemswhere access is limited to a number of noisy objective function evaluations.
Inthis paper, we propose a novel information-theoretic acquisition function for BOcalled Joint Entropy Search (JES), which considers the joint information gain forthe optimal set of inputs and outputs.
We present several analytical approximationsto the JES acquisition function and also introduce an extension to the batch setting.
We showcase the effectiveness of this new approach on a range of synthetic andreal-world problems in terms of the hypervolume and its weighted variants
Introduction
Bayesian optimization (BO) has demonstrated a lot of success in solving black-box optimizationproblems in various domains such as machine learning [64, 65, 78], chemistry [23, 31], robotics[7, 12] and clinical trials [53, 67].
The procedure works by maintaining a probabilistic model of theobserved data in order to guide the optimization procedure into regions of interest.
Speciﬁcally, ateach iteration the black-box function is evaluated at one or more input locations that maximizesan acquisition function on the model.
Implicitly, this function strikes a balance between exploringnew areas and exploiting areas that have been shown to be promising.
In this work, we consider themore general problem, where the black-box function of interest is vector-valued.
This increasesthe difﬁculty of the problem because there are now many directions in which the objectives can beimproved, in contrast to the single-objective setting where there is only one.
Informally, the end goalof multi-objective optimization is to identify a collection of points that describe the best trade-offbetween the different [-objectives.
-]objectives.
A popularstrategy is random scalarization [43, 54], which works by transforming the multi-objective probleminto a distribution of single-objective problems.
These approaches are appealing because itenables the use of standard single-objective acquisition functions.
A weakness of this approachis that it relies on random sampling to encourage exploration and therefore the performance ofthis method might suffer when the scale of the objectives is unknown a priori or when the inputspace is high-dimensional [16].
Another popular class of multi-objective acquisition functions areimprovement-based.
These strategies focus on improving a utility function over sets, for examplethe hypervolume indicator [14, 15, 22, 80] or the R2 indicator [20].
The main drawback of theseapproaches is that the performance of these methods are biased towards a single scalar utilitycriterion, which can be inadequate to assess the multi-objective aspects of the problem [83].
Instead of relying on scalarizations or an improvement-based criterion, this paper considersa probabilistic perspective where the goal of interest is to improve the posterior distribution at theoptimal points.
We propose a novel information-theoretic acquisition function called the Joint Entropy Search (JES), which assesses how informative an observation will be in learning moreabout the joint distribution of the optimal inputs and outputs.
This acquisition function combines theadvantages of existing information-theoretic acquisition functions, which focus solely on improvingthe posterior of either the optimal inputs [27, 29, 35] or the optimal outputs [4, 6, 68].
We connect JES with the existing information-theoretic acquisition functions by proving that it is an upper boundto these utilities.
Contributions and organization.
In Section 2, we set-up the problem and introduce the novel JESacquisition function.
In Section 3, we present a catalogue of conditional entropy estimates to approximate this utility and present a simple extension to the batch setting.
All of these approximations areanalytically available, which enables the use of gradient-based optimization.
In Section 4, we presenta discussion on the hypervolume indicator and explain how it can be a misleading performancecriterion because it is sensitive to the scale of the objectives.
We prove that information-theoretic approaches are naturally invariant to reparameterization of the objectives, which make them well-suitedfor multi-objective black-box optimization.
For a more complete picture of performance, we proposea novel weighted hypervolume strategy (Appendix H), which allows us to assess the performance of amulti-objective algorithm over different parts of the objective space.
In Section 5, we demonstrate theeffectiveness of JES on some synthetic and real-life multi-objective problems.
Finally in Section 6,we provide some concluding remarks.
Additional results and proofs are presented in the Appendix.
Preliminaries
We consider the problem of maximizing a vector-valued function f : X → R M over a boundedspace of inputs X ⊂ R D .
To deﬁne the maximum max x ∈ X f ( x ) , we appeal to the Pareto partialordering in R M .
For the rest of this paper, we will denote vectors by y =
( y (1) , . . .
, y ( M ) )
∈ R M ,the non-negative real numbers by R ≥ 0 and diagonal matrices by diag ( · ) .
Pareto domination.
We say a vector y ∈ R M weakly Pareto dominates another vector y (cid:48)
∈
R Mif it performs just as well in all objectives if not better: y (cid:23) y (cid:48)
⇐⇒ y − y (cid:
) ∈ R M ≥ 0 .
Additionally,if the vectors are not equivalent, y (cid:54) =
y (cid:48) , then we say strict Pareto domination holds:
y (cid:31) y (cid:48)
⇐⇒y − y (cid:48)
∈ R M ≥ 0 \ { 0 M } , where 0 M is the M -dimensional zero vector.
This binary relation can befurther extended to deﬁne domination among sets.
Let A, B ⊂ R M be sets, if the set B lies in theweakly dominated region of A , namely B ⊂ D (cid:22) ( A ) = ∪ a ∈ A { y ∈ R M : y (cid:22) a } , then we say Aweakly dominates B , denoted by A (cid:23) B .
In addition, if it also holds that the dominated regions arenot equal, D (cid:22) ( A ) (cid:54) =
D (cid:22) ( B ) , we say strict Pareto domination holds, denoted by A (cid:31) B .
Multi-objective optimization.
The goal of multi-objective optimization is to identify the Paretooptimal set of inputs X ∗ =
arg max x ∈ X f ( x ) ⊂ X .
The Pareto set is deﬁned as the set of inputswhose objective vectors are not strictly Pareto dominated by another: x ∗ ∈
X ∗ ⇐⇒ x ∗ ∈ 74 X
and (cid:64) x ∈ X such that f ( x ) (cid:31) f
( x ∗ ) .
The image of the Pareto set in the objective space Y ∗ =
f ( X ∗ )
= max x ∈ X f ( x ) is called the Pareto front.
For convenience of notation, we will denote theset of Pareto optimal input-output pairs by ( X ∗ , Y ∗ ) .
Bayesian Optimization is a sample efﬁcient global optimization strategy, which relies on a probabilistic model in order to decide which points to query.
In Appendix A.1, we present the pseudo-codefor BO procedure—for more details see [11, 25, 63].
In this work, we will use independent Gaussianprocess priors [59] on each objective, f ( m ) ∼ GP ( µ ( m ) 0 , Σ ( m ) 0 ) , where µ ( m ) :
X → R is the meanfunction and Σ ( m ) :
X × X
→ R is the covariance function for objective m .
The observationsat location x ∈ X will be assumed to be corrupted with additive Gaussian noise, y =
f ( x ) + (cid:15) ,where (cid:15) ∼ N (0 , diag ( σ ( x )) denotes the observation noise with variance σ ( x ) ∈ R M ≥ 0 .
After nevaluations, we have a data set D n = { ( x t , y t ) } t =1 ,...,n .
The posterior model p ( f | D n ) is a collectionof independent Gaussian processes f ( m ) | D n ∼ GP ( µ ( m ) n , Σ ( m ) n ) . The explicit expressions for themean and covariance are presented in Appendix A.2. The main focus of this work is on designing theacquisition function, α : X → R , which is used to select the inputs: x n +1 = arg max x ∈ X
α ( x | D n ) .
Information-theoretic acquisition functions focus on maximizing the gain in information from the next observation and a property of the probabilistic model.
Initial work in BO focussed on pickingpoints to learn more about the distribution of the maximizer p ( X ∗ | D n ) .
Speciﬁcally, the goal ofinterest was to maximize the mutual information between the observation y and the Pareto set X ∗conditional on the current data set D n :
[Equation]
where H [ p ( x )]
= − (cid:82) p ( x ) log p ( x ) d x represents the differential entropy.
This acquisition functionis commonly referred to as predictive entropy search (PES) [35, 36, 62], but it was formerly 1known as entropy search (ES) [34, 72].
Despite the importance of obtaining more informationabout the maximizer, the PES acquisition function is heavily dependent on the approximation ofp ( y | x , D n , X ∗ ) , which is both computationally difﬁcult to implement and optimize.
This motivatedresearchers to consider a simpler scheme that focusses on learning more about the distribution of themaximum p ( Y ∗ | D n ) .
The resulting acquisition function is known as the max-value entropy search(MES)
[Equation]
[Equation] Unlike PES, the conditional probability p ( y | x , D n , Y ∗ ) arising in MES can be approximated andoptimized more easily because some approximations lead to closed-form expressions.
Despite thefavourable properties of MES, the primary goal of interest is to identify the location of the maximizer 104 X ∗ and not necessarily the value of the maximum Y ∗ .
To combine the advantages of both of theseapproaches, we propose the joint entropy search acquisition function, which focusses on learningmore about the joint distribution of the optimal points p (( X ∗ , Y ∗ ) | D n ) :
[Equation]
The JES acquisition function inherits the advantages of the PES and MES acquisition functionsbecause it considers the knowledge learnt about the optimal points and is also simple to implement—more details in the next section.
The following proposition shows that we can also interpret JES as anupper bound to both the PES and MES acquisition function.
Proposition 1 The JES is an upper bound to any convex combination of the PES and MES acquisitionfunctions: α JES ( x | D n ) ≥
[Equation]
|
[Equation]
+
[Equation] − β ) α MES ( x | D n ) , for any β ∈ [0 , 1] .
Approximating JES
In this section, we present several approximations to the JES acquisition function (3) and a simpleextension to the batch setting.
The ﬁrst term in the JES criterion (3) is the entropy of a multivariatenormal distribution:
[Equation]
The second term is an intractable expectation which is approximated by drawing Monte Carlo samples from p (( X ∗ , Y ∗ )
| D n ) .
The conditional entropy H [ p ( y | x , D n , ( X ∗ , Y ∗ ))] is also an intractablequantity which has to be estimated.
The overall approximation of (3) will take the form
[Equation]
where h denotes the conditional entropy estimate and ( X ∗ s , Y ∗ s ) ∼ p (( X ∗ , Y ∗ )
| D n ) are the Monte
Carlo samples.
The distribution p ( y | x , D n , ( X ∗ , Y ∗ )) is very challenging to work with because itenforces the global optimality condition that the function lies below the Pareto front f ( X ) (cid:22) Y ∗ .
Instead of enforcing global optimality, we make the common simplifying assumption as in [50, 68, 74]and only enforce the optimality condition at the considered location: f ( x ) (cid:22) Y ∗ .
By applying Bayes’theorem, the resulting density of interest becomes
[Equation]
where we have denoted the augmented data sets by D n ∗
=
D n ∪ ( X ∗ , Y ∗ ) and D n + = D n ∗ ∪{ ( x , y ) } .
We will refer to the quantity p ( f ( x ) (cid:22) Y ∗ ) as the cumulative distribution function (CDF).
Thefollowing lemma shows that this CDF can be computed analytically when the set Y ∗ ⊂ R M isdiscrete.
This result can be derived by ﬁrst partitioning the region of integration, D (cid:22) ( Y ∗ ) =
∪ y ∗
∈ Y ∗ { z ∈ R M : z (cid:22) y ∗ } , into a collection of hyperrectangle subsets and then summing upthe individual contributions—see Figure 2 for a visual.
This partition can be computed using anincremental approach of (Algorithm 1 of [46]), which has a cost of O ( | Y ∗ | (cid:98) M/ 2 (cid:99) +1 ) .
Lemma 1 Let Y ∗ ⊂ R M be a ﬁnite set and z ∼ N ( a , diag ( b )) be an M -dimensional multivariate normal with mean a ∈ R M and variances b ∈ R M ≥ 0 .
Let D (cid:22) ( Y ∗ ) =
(cid:83)
Jj =
1 B j =(cid:83)
Jj =1 (cid:81)
Mm =1 ( l ( m ) j , u ( m ) j ] be the box decomposition of the dominated space, then
[Equation]
Input
[Figure]
B 2 B
[Figure]
3.1 Estimating the conditional entropy
The entropy of (6) can be written as an M -dimensional expectation over the multivariate normaldistribution p ( y | x , D n ∗ ) =
N ( y ; µ n ∗ ( x ) , Σ n ∗ ( x , x )) :
[Equation]
To simplify the notation, we deﬁne the m -th standardized value by γ m ( z ) =
( z −µ ( m ) n ∗ ( x )) / (cid:113) Σ ( m ) n ∗ ( x , x ) for any scalar z ∈ R .
Using this function together with Lemma 1, wedenote the cumulative distribution p ( f ( x ) (cid:22) Y ∗ | x , D n ∗ ) by W = (cid:80)
Jj =1 W j = (cid:80)
Jj =1 (cid:81)
Mm =1 W j,m ,where W j,m = Φ( γ m ( u ( m ) j )) − Φ( γ m ( l ( m ) j )) are the differences appearing in (7).
Moreover, wedenote the differences of the ﬁrst derivative of W j,m (with respect to γ m ) by G j,m = φ ( γ m ( u ( m ) j )) −φ ( γ m ( l ( m ) j )) and the negative of the second derivative by V j,m = γ m ( u ( m ) j ) φ ( γ m ( u ( m ) j ))
−γ m ( l ( m ) j ) φ ( γ m ( l ( m ) j )) , where φ is the probability density function of a standard normal distribution.
In the setting where the observation noise is zero, the conditional distribution is a truncatedmultivariate normal, which is known to have the following analytical equation for the entropy: Proposition 2 (Theorem 3.1.
in [68])
Under the modelling set-up outlined in Section 2, if x ∈ X isan input with zero observation {+variance, σ ( x ) = 
 M , then+} the entropy of the truncated multivariatenormal distribution p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) is given by [Equation] Consequently, we deﬁne the JES-0 entropy estimate by the equation
[Equation]
Notice that we have included an additional observation noise term σ ( m ) ( x ) into the logarithm ofthe variance.
This term will cancel out with the term in the initial entropy (4) in both the noiselessand noisy setting.
Intuitively, this is the implicit noise correction when applying Proposition 2 insettings where the observation noise is non-zero.
We will see in Section 5 that this correction is quiteeffective in practice.160 In the noisy setting, the distribution of interest is a type of multivariate skew normal distribution, which is known to not have an analytical form for the entropy [1].
As a result, we proposetwo approximation strategies to estimate this entropy.
The ﬁrst strategy is to approximate theintegral using Monte Carlo.
The details of the Monte Carlo estimate h JES-MC (( X ∗ , Y ∗ ); x , D n ) is described in Appendix D.
The second strategy is to directly approximate the distribution with onethat exhibits an analytical entropy.
We consider the most obvious choice, which is a multivariatenormal distribution with the same ﬁrst two moments.
The same strategy was proposed in [50] for thesingle-objective multi-ﬁdelity MES acquisition function.
By a standard result (Chapter 12 of [13]),the entropy of this approximating distribution is actually an upper bound for the entropy of interest: H [ p [Equation] y |
[Equation] , D n ∗ , f ( x ) (cid:22) Y [Equation] [Equation]
log
[Equation]
[Equation]
+
[Equation]
V ar ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) .
The followingresult shows that the these central moments can be computed analytically.
Proposition 3 Under the modelling set-up outlined in Section 2, for an input x ∈ X the ﬁrst andsecond central moment of p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) are
[Equation]
W
[Equation]
and
[Equation]
[-(a) y (1) y ( 2 ) Truncated Multivariate Normal 0 .0 .0 .
 (b) y (1) y ( 2 ) Multivariate Normal (Full Covariance) 0 .0 .0 .0 .1 . 0 1 .1 .-]As an upper bound on the conditional entropy leads to a lower bound on the mutual information.
Wedeﬁne the JES-LB entropy estimate as follows:
[Equation]
We could obtain a further lower bound by ignoring the off-diagonal terms in the covariance matrix.
We dub the resulting approximation as the JES-LB2 entropy estimate:
[Equation]
Empirically, we observe that the lower bound JES approximations are not necessarily tight for allinputs x ∈ X .
Nevertheless, the maximizer of the approximate acquisition function is often close tothe maximizer of the exact JES—see Appendix G for a visualization of a single-objective example.
Remark.
All of the conditional entropy estimates introduced here can be applied to the MESacquisition function.
The only difference is that we no longer apply the conditioning step (line 6 of
Algorithm 1) because we are interested in estimating H [ p ( y | x , D n , f ( x ) (cid:22) Y ∗ )] as opposed to (8).
Consequently, the MES acquisition function is cheaper to evaluate because the cost of a evaluatingthe posterior variance at a single input is O ( n 2 ) , whereas JES incurs a cost of O (( n + | Y ∗ | ) 2 ) —moredetails are presented in the cost analysis in Appendix F.
3.2 Batch evaluations
Evaluating the JES acquisition functions for a batch of points x [1: q ] =
( x [1] , . . .
, x [ q ] ) ∈
X q isexpensive because the entropy estimates now depends on the q -dimensional normal CDF and itsderivatives.
To circumvent this issue, we follow the example of [50] and propose a suboptimal batchapproach by upper bounding the expensive joint conditional entropy term by the sum of the individualentropies:
H [ p ( y [1: q ] | x [1: q ] , D n ∗ , f ( X ) (cid:22) Y ∗ )] ≤ (cid:80) qi
=1 H [ p ( y [ i ] | x [ i ] , D n ∗ , f ( X ) (cid:22) Y ∗ )] .
Theresulting q -batch lower bound JES estimate is given by
[Equation]
where h ∈ { h JES- 0 , h JES-LB , h JES-LB2 , h JES-MC } are the conditional entropy estimates and
H [ p ( y [1: q ] | x , D n )] = M 2 log(2 πe ) + 12 (cid:80) Mm =1 log det(Σ ( m ) n ( x [1: q ] , x [1: q ] ) + diag ( σ ( m ) ( x [1: q ] ))) is y (1) y ( 2 ) Skew Multivariate Normal 0 .0 .0 .0 .0 .
( (1) n * ( x ), (2) n * ( x ))
Pareto front 0.25 0.
(a) y (1) y ( 2 )
[-U HV =0.
-]{+Truncated Multivariate Normal 0 .0 .0 .0 .0 .1 . 
+}
(b) y (1) y ( 2 )
[-U HV =0.
 Transformed objective space g 1 ( y (1) )-]Multivariate Normal (Full Covariance) 0 .
(c) y (1) y ( 2 )
Multivariate Normal (Diagonal Covariance) 0 .0 . (d) the initial entropy.
This acquisition function is deﬁned over a qD -dimensional space, which becomesmore difﬁcult to optimize as q increases.
Alternatively, we can maximize this function greedily bysequentially selecting the best input conditioned on the previously chosen points.
This greedy procedure satisﬁes an e − 1 regret bound when the acquisition function is submodular [75].
In Appendix E,we show that this batch acquisition function is indeed submodular.
Performance criteria
In multi-objective optimization, the most common way to measure performance is by comparing theapproximate Pareto set ˆ X ∗ against the optimal Pareto set X ∗ in the objective space: d ( f (ˆ X ∗ ) , f ( X ∗ ))where d : 2 R M × 2 R M → R is a function that measures the discrepancy between the sets of objectivevectors.
Existing work in multi-objective BO mainly focusses on the hypervolume (HV) discrepancy,d HV ( A, B ) =
| U HV ( A ) − U HV ( B ) | , where the HV indicator, U HV ( A ) =
(cid:82) R M I [ r (cid:22) z
(cid:22) A ] d z , isdeﬁned as the volume between a reference point 2 r ∈ R M and a set
A ⊂ R M .
An attractive featureof the HV indicator is that it is Pareto complete (or compliant) in the sense that a better set willlead to a larger HV [83]: A (cid:31) B =
⇒ U HV
( A ) >
U HV ( B ) .
The reverse implication known as Pareto compatability does not hold for the HV indicator [83].
In other words, the HV can be usedto discriminate between sets where one dominates another, but it cannot be relied upon when thesets are incomparable.
Not all incomparable sets are treated equally by the HV indicator [84].
Forinstance Figure 4a shows an example where the HV indicator places more emphasis on the endpoints of the Pareto front.
On other hand, if we apply a monotonically increasing transformationg m : R → R to each objective, the Pareto set will not change, whereas the HV comparison will(Figure 4b).
Implicitly, the HV indicator assumes that a linear change in one objective is equivalent toa linear change in another.
This assumption might not necessarily reﬂect the decision maker’s outlookand this is something that is typically overlooked when designing and benchmarking multi-objectiveoptimization algorithms.
The following result shows that information-theoretic acquisitions functionare in fact agnostic to the choice of parametrization.
Proposition 4 The information-theoretic acquisition functions α PES , α MES and α JES are invariant toreparametrizations of the objective space that are consistent with the Pareto ordering relations.
Forexample, α JES ( x | D n ) =
MI ( y ; ( X ∗ , Y ∗ )
| D n )
=
MI ( g ( y ); ( X ∗ , g ( Y ∗ ))
| D n ) , where the g m : R → 225 R is a strictly monotonically increasing function acting only on the m -th objective.
To benchmark the algorithms, we use both the standard HV discrepancy (Section 5) and the HVdiscrepancy under different parameterizations (Appendix I).
To easily obtain a family of parameterizations, we devise a novel weighting approach in Appendix H, which exploits the fact that the HVindicator can be written as an expectation over a uniform distribution on the ( M − 1) -dimensionalhypercube
We observe that it is possible to assess the performance at different locations ofthe objective space by using alternate distributions over hypercube.
In our experiments, we found y (1) y ( 2 ) U HV =0. [Figure] y (1) y ( 2 )
{+U HV =0.
+}
Objective space (a) g 1 ( y (1) ) g 2 ( y ( 2 ) )
{+U HV =0.
+}
Transformed objective space g 1 ( y (1) )
g 2 ( y ( 2 ) )
U HV =
0.822 Transformed objective space (b) that the performance of each algorithm changed with regards to the choice of parametrization, but the
JES approaches tended to be the strongest performers throughout.
Experiments
We empirically evaluate the JES acquisition function on a range of synthetic and real-world benchmarkproblems.
We compare this approach with some popular acquisition functions in multi-objective
BO: TSEMO [10], ParEGO [43], NParEGO [15], EHVI [14], NEHVI [15], PES [27, 29] and MES-0[68].
We have also included the MES-LB, MES-LB2 and MES-MC acquisition functions, whichcan be easily derived from the conditional entropy estimates that we developed here.
All algorithmsare based on the open source Python library BoTorch [3], which uses features from GPyTorch [26]for Gaussian process regression and PyTorch [56] for automatic differentiation.
All experimentsare repeated using 100 different initial seeds and we generate the Pareto set recommendation ˆ X ∗ of50 points by maximizing the posterior mean using a multi-objective solver (NSGA2 [18] from the
Pymoo library [8]).
The complete details of the experiments are outlined in Appendix I, whilst thecode is available at https://github.com/.../JES .
5.1 Benchmarks
Synthetic benchmark.
We consider the ZDT2 [18] benchmark with D = 6 inputs and M = 2objectives.
We corrupt the observations with additive Gaussian noise with zero-mean and standarddeviation set to approximately 10% of the objective ranges.
Chemical reaction.
This benchmark considers a nucleophilic aromatic substitution reaction (SnAr)between 2,4-diﬂuoronitrobenzene and pyrrolidine in ethanol to produce a mixture of a desired productand two side-products [41].
The design space comprises of D = 4 components relating to theinitial conditions.
The goal is to optimize M = 2 objectives, namely the space time yield andthe environmental impact.
We apply a logarithm transform to the objectives and contaminate theobservations with additive Gaussian noise with zero-mean and standard deviation set to approximately3% of the resulting objective ranges in order to emulate a potential real-world scenario.
Pharmaceutical manufacturing.
This problem is concerned with optimizing the Penicillin production process outlined in [47].
The design space is made up of D = 7 elements that control theinitial condition of the reactions.
The goal is to optimize M = 3 objectives, which relates to theyield, the amount of carbon dioxide released and the time to ferment.
We include additive zero-mean
Gaussian noise with a standard deviation set to approximately 1% of the objective ranges.
Marine design.
This problem considers optimizing a family of bulk carriers subject to the constraints imposed for ships travelling through the Panama Canal [55, 61].
The design space is madeup of D = 6 variables that determine the architecture of the carriers.
The goal of this problem is tomaximize the annual cargo, whilst minimizing the transportation cost and the ship weight subjectto some design constraints.
We consider the reformulation in [71], which converts the constraints [Table] 3 4 5 6 7 8 9 10 11 12 Iterations 2.00 1.
l o g ( d H V ) SnAr (D=4, M=2, Noise=3%, q=8) 0 {+1 2 3 4 5 6 7 8 9 10 11 12 Iterations 12. l o g ( d H V ) Penicillin (D=7, M=3, Noise=1%, q=8) 
+}
1 2 3 4 5 6 7 8 9 10 11 12 Iterations 28.
l o g ( d H V )
Marine (D=6, M=4, Noise=0.5%, q=8) into another objective.
For this reformulated M = 4 objective problem, we corrupt the observationswith additive zero-mean Gaussian noise with standard deviation set to approximately 0 .
5% of theobjective ranges.
5.2 Results and discussion
We present the mean log HV discrepancy for both the sequential and batch experiments in Figure 5.
Appendix I contains additional ﬁgures describing the variation of the HV (and weighted HV) amongthe experiments, the wall times and some visual snapshots of the Pareto front approximations. Overall we see that the JES acquisition function is superior to existing methods in terms of the HV(and weighted HV) criterion on these benchmark problems.
This gain in performance does come atthe cost of additional wall time, which could potentially be alleviated by performing some routines inparallel—further details in Appendix I.6. Among the different conditional entropy approximations,we observe that the novel lower bound approximations offers the best trade-off between the HVperformance and the wall time for both the JES and MES.281 The improvement and
scalarization
based acquisition functions are faster to optimize than
JES
but they performed marginally worse in these sets of experiments since they tend to be moregreedy.
Speciﬁcally, these algorithms focus of querying high-performing points as opposed to thepoints that are the most informative for the posterior.
As discussed in [15], this feature might bedesirable for certain domains where the end goal is to make a decision from the ﬁnal data set D N asopposed to using the ﬁnal model p ( f | D N ) .
Nevertheless, improvement-based acquisition functionsinherently place a preference on certain parts of the Pareto front and therefore some additionalthought has to be taken before applying these methods.
Proposition 4 and the empirical resultsmotivates the usage of information-theoretic acquisition functions in the truly black-box settingwhere all points on the Pareto front are equally desirable a priori.
Conclusion
We introduced JES, a novel information-theoretic acquisition function for multi-objective BO.
Toapproximate this acquisition function, we presented several approximations to the conditional entropyand also a simple extension for the batch setting.
Experimental results suggest that JES is verycompetitive with existing acquisition functions in terms of the HV discrepancy and its weightedvariants.
The main limitation of the JES acquisition function is that it relies on routines such asbox decompositions and multi-objective optimization, which can be expensive to execute for verylarge problems.
Future work could focus on improving the scalability of these information-theoreticmethods and extending it to more general settings—see Appendix J.
Checklist
For all authors
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and
scope?
[Yes
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] The proofs for the
main results are in the appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] The code is in
the supplementary material and will be made public in due time.
(b) Did you specify all the training details (e g., data splits, hyperparameters,
how they
were chosen)?
[Yes] The experimental set-up is described in Appendix I.
(c) Did you report error bars (e g , with respect to the random seed after running ex-
periments multiple times)?
[No] We do not include the error bars directly on the
plots because it made it difﬁcult to visually compare between the different algorithms.
Alternatively, we opted to plot the different quantiles side by side e.g. Figure 16.
(d) Did you include the total amount of compute and the type of resources used (e.g.,
type of GPUs, internal cluster, or cloud provider)?
[Yes] The computing resources are
described in the caption of the wall times plots e.g. Appendix I.6.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[No] We only cited the original code and
corresponding manuscript in Appendix I.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
The code is in the supplementary material and will be made public in due time
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
Appendix
to Joint Entropy Search for
Multi-Objective Bayesian Optimization
A BasicsA.1 Bayesian optimization pseudocode
Bayesian optimization consists of two steps that are iterated until the budget of function evaluations
N is exhausted.
The ﬁrst step is modelling, this is where posterior of the probabilistic model p ( f | D n )is computed.
The second step is acquisition, this is where a utility function is optimized in order todetermine the next location to query.
The pseudo-code for this procedure is presented in Algorithm 2.
For a more thorough overview on Bayesian optimization consult the references [11, 25, 63].
Input : A black-box function f .
1 Initialize the probabilistic model p ( f ) .
2 for n = 0 , . . .
, N − 1 do
Optimize for the next point x n +1 = arg max x ∈ X α
( x | D n ) .
Evaluate the function y n +1 = f ( x n +1 ) + (cid:15) .
Augment the data set D n +1 =
D n ∪ { ( x n +1 , y n +1 ) } .
Compute the posterior p ( f | D n ) .
7 end 8 return D N and p ( f | D N ) .
A.2 Posterior Gaussian process
Under the independent Gaussian observation model described in Section 2, the posterior p ( f ( X ) | D n )evaluated at a vector X ⊂ X , is a collection of Gaussian processes [59] with meanµ ( m ) n ( X ) = {+µ ( m ) 
+}
( X {+) + Σ ( m ) 
+}
( X, X n ) (cid:16) Σ ( m {+) 
+}
( X n , X n ) + diag ( σ ( m ) ( X n )) (cid:17) − 1 ( Y ( m ) n − µ ( m ) 0 ( X n ))
[-X n )) (cid:17) [Equation] ( Y ( m ) n − µ ( m ) 
 ( X n ))-] (13) and covarianceΣ ( m ) n ( X, X ) =
{+Σ ( m ) 
+}
( X, X
[Equation] [Equation]
where we denote the collection of sampled locations by X n = { x t } t =1 ,...,n and observations
Y ( m ) n = { y ( m ) t } t =1 ,...,n for objectives m = 1 , . . .
, M .
A.3 Sampling from a Gaussian process
Exact sampling of a one-dimensional Gaussian process over a ﬁnite set, X ⊂ X , scales cubicallywith the number of points | X | .
This cubic cost arises form the inversion of the covariance matrix.
As we want to sample global maximizers and maximums ( X ∗ , Y ∗ ) , exact sampling over the wholespace X is not computational feasible.
We could restrict the sampling to a discrete subset of X , butthis would require designing a principled strategy to select a reasonable subset, for example wecould try triangulating between existing points [32].
To avoid this difﬁculty, we follow the exampleof previous work [36, 74], which considers generating approximate samples via random Fourierfeatures [58, 76, 77].
The strategy centres around approximating the covariance kernel using a featurerepresentation Σ ( m )
( x , x (cid:48) )
≈ ϕ ( x ) T ϕ
( x (cid:48) ) .
For a stationary covariance kernel Σ ( m ) 0 , the Fouriertransform of the kernel is a non-negative measure that can be normalized to obtain a probabilitydistribution p ( θ ) —this result follows from Bochner’s Theorem [9] .
For the exponential and Maternkernels, this probability distribution is known in closed-form [10, 58].
Assuming Σ ( m )is stationary, an approximate sample of the Gaussian process prior GP ( µ ( m ) 0 , Σ ( m ) 0 ) can be written as a Bayesianlinear model
[Equation]
where ϕ i ( x ) = (cid:112) 2 /L
cos( θ Ti x + τ i ) are the Fourier features depending on the random variablesτ i ∼ U (0 , 2 π )
and θ i ∼ p
( θ ) , whilst ω i ∼ N (0 , 1) are the random weights for i = 1 , . . .
, L .
Theposterior samples of GP ( µ ( m ) n , Σ ( m ) n ) can be obtained by adding an additional pathwise update via
Matherons’ rule [76, 77]:
[Equation]
where κ t =
(Σ ( m )
( X n , X n ) + diag ( σ ( m ) ( X n )))
− 1 ( Y n − f ( m )( X n )) .
For a more comprehensiveoverview of sampling from a Gaussian process refer to [77].
B Related work
In this work we devised a number of conditional entropy estimates for the JES acquisition function.
These estimates can also be used by the MES acquisition function.
In this brief section, we elaborate on the similarities of our estimates with the existing work relating to the MES acquisition {+function.
+} Firstly, the noiseless entropy estimate (9) was derived recently [68] to extend the noiseless MES acquisition function to the multi-objective setting.
This extension was called the
Pareto frontier entropy search, whilst we refer to this acquisition function as MES-0 for clarity.
An earlier attempt to extend the MES to noiseless setting resulted in the MESMO [4]acquisition function.
MESMO crudely approximates the dominated space by a single box 637 D (cid:22) ( Y ∗ )
≈ B = ( −∞ ,
max y ∈ Y ∗ y (1) ] × · · · × ( −∞ , max y ∈ Y ∗ y ( M ) ] .
Empirically, this approximation leads to a deterioration in performance [68].640 The closest work to the lower bound entropy estimate is the GIBBON [50] acquisitionfunction.
The GIBBON acquisition function was derived as a lower bound to the multi-ﬁdelity
MES acquisition function for a single objective problem.
The MES-LB2 can be interpreted as themulti-objective extension of GIBBON for the single ﬁdelity {+setting.
+}
The Monte Carlo entropy estimate has been used before for the multi-ﬁdelity MES acquisition in the single-objective setting [51, 69].
The MES-MC estimate can be interpreted as themulti-objective extension of these approaches for the single ﬁdelity setting.
C Proof of resultsC.1 Proof of Proposition 1
The upper bound property follows from the standard result that conditioning on more variables willnever increase the entropy (Theorem 2.6.5 of [13]): H ( A | B ) ≤ H ( A ) for random variables A and
B .
Using this result, H [ p ( y | x , D n , ( X ∗ , Y ∗ ))] ≤ max( H [ p ( y | x , D n , X ∗ )] , H [ p ( y | x , D n , Y ∗ )]) .
Plugging this inequality into (3), we obtain the α JES ( x | D n ) ≥ max( α PES ( x | D n ) , α MES ( x | D n )) ,which implies the result.(cid:4)
C.2 Proof of Lemma 1
Under the assumptions of Lemma 1, the following series of equations holds:
[Equation]
The third equality follows from the box decomposition of D (cid:22) ( Y ∗ ) and the independence betweenthe components of z , whilst the last equality follows from the deﬁnition of the CDF.(cid:4)
C.3 Proof of Proposition
In the noiseless setting, the density of interest is [Equation] The corresponding entropy is
[Equation] where the CDF is given by [Equation] and the integral is given by
[Equation] m = log( p ( y ( m ) | x , D n ∗ )) d y = −
J (cid:88) j
=
M (cid:88) m = (cid:90)
B j
M (cid:89) m (cid:48)
=
p ( y ( m (cid:48) )
| x , D n ∗ ) log( p ( y ( m ) | x , D n ∗ ))
d y = −
J (cid:88) j
=
M (cid:88) m = (cid:90) B j p ( y ( m ) | x , D n ∗ ) log( p ( y ( m ) | x , D n ∗ )) (cid:89) m (cid:48) (cid:54) =
m p ( y ( m (cid:48) )
| x , D n ∗ )
d y = −
J (cid:88) j
=
M (cid:88) m =
W j
W j,m (cid:90) u ( m ) j l ( m ) j p ( y ( m ) | x , D n ∗ ) log( p ( y ( m ) | x , D n ∗ )) dy ( m )
[Equation]
The ﬁrst equality follows from the box decomposition and the independence between the objectives,p ( y | x , D n ∗ ) = (cid:81)
Mm =1 p ( y ( m ) | x , D n ∗ ) . The fourth equality follows from the deﬁnition that W j,m (cid:48) =(cid:82) u ( m (cid:48) ) j l
[Equation]
m
[Equation] j p ( y ( m (cid:48) )
| x , D n ∗ ) dy ( m (cid:48) ) and W j = (cid:81) Mm =1 W j,m .
The sixth equality comes from noticingthat p ( y ( m ) | x , D n ∗ ) /W j,m is a truncated normal distribution on [ l ( m ) j , u ( m ) j ] whose entropy can becomputed analytically (Section 4.26 of [48]).
Combining all the expression above gives the ﬁnalresult.(cid:4)
C.4 Proof of Proposition 3
In the noisy setting, the density of interest is
[Equation]
where p ( f ( x ) (cid:22) Y ∗ | x , D n ∗ )
= W from (18).
To obtain a tractable expression for the cumulativedistribution p ( f ( x ) (cid:22) Y ∗ | x , D n + ) , we ﬁrst compute the conditional distribution p ( y | x , D n + ) .
Tothat end, let p ( y | x , D n ∗ ) =
N ( y ; µ n ∗ ( x ) , Σ n ∗ ( x , x )) , then by standard Gaussian conditioningp ( y | x , D n + ) =
N ( y ; µ n + ( x ) where
[Equation]
and
[Equation]
[Equation]
x , x
[Equation]
x , x ) = Σ [Equation] x , x ) [Equation]
2 m [Equation] x )) with ρ m := ρ m ( x ) = (cid:113) Σ ( m ) n ∗ ( x , x ) / (cid:113) Σ ( m ) n ∗ ( x , x ) + σ ( m ) ( x ) denoting correlation between theobservation y ( m ) and the objective value f ( m ) ( x ) , for objectives m = 1 , . .
, M .
Using the boxdecomposition, the posterior CDF is equal top
[Equation]
To simplify the notation we perform a change of variable:
[Equation]
where γ m ( z ) = ( z − µ ( m ) n ∗ ( x )) / (cid:113) Σ ( m ) n ∗ ( x , x ) is the m -th standardized value and ¯ y ( m ) = ( y ( m ) −µ ( m ) n ∗ ( x )) /
(cid:113) Σ ( m ) n ∗ ( x , x ) + σ ( m ) ( x ) is the standardized observation for m = 1 , . . .
, M .
To derivethe moments of p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) , we ﬁrst obtain the moment generating function of thestandardized observation ¯y .
[Equation]
By Lemma 2 in [2], the expectation of the normal CDF is given by (cid:82) R Φ( az + b ) φ ( z ) dz =
Φ( b/ √ 1 + a 2 ) for any constants a, b ∈ R .
Using this result,
[Equation]
The ﬁrst moment p (¯ y ( m ) | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) can be obtained by evaluating the ﬁrst derivative of
M ¯y ( t ) with respect to t ( m ) at t = 0 M .
[Equation]
where for shorthand we deﬁned the intermediate variables β uj,m =
Φ( γ m ( u ( m ) j ) − ρ m t ( m ) ) , β lj,m =Φ( γ m ( l ( m ) j
[Equation]
[Equation]
and for m = m (cid:48) ,
[Equation]
where for shorthand we deﬁned the intermediate variables τ uj,m = γ m
[Equation]
and τ lj,m =
γ m ( l ( m ) j ) − ρ m t ( m ) for j = 1 , . . .
, J and m = 1 , . .
, M .
The moments of y ( m ) can be now derivedby reversing the initial linear transformation: y ( m ) =
¯ y ( m ) (cid:113) Σ ( m ) n ∗ ( x , x ) + σ ( m ) ( x ) + µ ( m ) n ∗ ( x ) .
Theﬁrst moment is
[Equation]
The second moment:
[Equation]
This implies
[Equation]
Substituting in the expressions before, we have that for m (cid:54) = m (cid:48) ,
[Equation]
W
[Equation]
G j,m (cid:48)
W j,m (cid:48) − µ ( m (cid:48) )
n ∗ ( x ) (cid:113) Σ ( m (cid:48) )
n ∗ ( x , x )
G j,m
W j,m − µ ( m ) n ∗ ( x ) (cid:113) Σ ( m ) n ∗ ( x , x )
G j,m (cid:48)
W j,m (cid:48) 
 and for m = m (cid:48)
[Equation]
By some additional algebraic manipulation
[Equation]
the covariance for m (cid:54) = m (cid:48) is given
[Equation]
 and the variance is
V
[Equation]
C.5 Proof of Proposition 4
We will prove the general statement that the mutual information is invariant under smooth bijectivetransformations.
Consider two random variables X and Y then following equations hold:
[Equation]
where X (cid:48)
=
g x ( X ) and Y (cid:48) =
g y ( Y ) represents the transformed variables under some suitablydeﬁned smooth invertible functions g x and g y .
The expressions J x and J y correspond to the Jacobianof the transformation, which are non-zero because the transformation are assumed to be invertible.
The result follows by restricting the class of bijective functions to ones where the Pareto set remainsunchanged.
For example, the class of monotonic increasing functions in each objective ensuresthis property holds: g : R M
→ R M such that g ( y )
= ( g 1 ( y (1) ) , . . .
g M ( y ( M ) )) with g m beingmonotonically increasing.
[-Under the modelling set-up outlined in Section 2, if x ∈ X isan input with zero observation variance, σ ( x ) = 
-]
[-Assuming σ ( x ) = 
-]
[Equation]
D
Monte Carlo entropy estimate
In this section, we consider estimating the entropy of p ( y | x , D n ∗ , f ( x ) (cid:22) Y ∗ ) via Monte Carlo.
Asa reminder, the entropy of interest can be written as an expectation over p ( y | x , D n ∗ ) :
[Equation]
The CDF in the denominator is p ( f ( x ) (cid:22) Y ∗ | x , D n ∗ )
=
W from (18), whilst the CDF in thenumerator is
[Equation]
from (19).
By sampling y i ∼ p ( y | x , D n ∗ ) for i = 1 , . . .
, I , we can approximate the entropy withthe following Monte Carlo average:
[Equation]
where the Monte Carlo entropy estimate is given by
[Equation]
Instead of generating new samples for each call of the acquisition function, we follow the generalwisdom in BO [75] and apply the reparametrization trick on the sampling distribution: y i = µ n ∗ ( x )+
C n ∗ ( x ) z i , where C n ∗ ( x ) ∈ R M × M is the Cholesky factor of Σ n ∗ ( x , x ) and z i ∼ N (0 , I M ) arethe base samples that only need to be initialized once.
The variance of this estimate could potentiallybe reduced by including control variates.
For example,
[Equation]
where the expectations are known,
[Equation]
and β 1 , β 2 ∈ R are the regression coefﬁcients.
We did not assess the effect adding control variates toour Monte Carlo estimate because the standard quasi-Monte Carlo scheme with the reparametrizationtrick worked reasonably well out of the box.
One word of caution [39]: combining both controlvariates and quasi-Monte Carlo could lead to an increase in variance if the coefﬁcients are naivelyestimated.
E Submodularity
A set function g : 2 V → R is submodular if for every A, B ⊂ V , g ( A ∩ B )+ g ( A ∪ B ) ≤ g [Equation] A )+ g [Equation]
B ) ,where V is the set of interest—for further details on submodularity refer to [44].
The followingproposition states that the lower bound batch acquisition function and its approximation for the JESand MES are submodular functions deﬁned over subsets of the input space.
Proposition 5 The lower bound batch acquisition function α qLB-JES ( X | D n ) and its approximationsˆ α qLB-JES ( X | D n ) are submodular functions deﬁned over subsets of X .
Proof.
Let X
= { x i } i =1 ,...,q ⊂ X denote a set of inputs.
If K X ∈ R q × q is a positive semi-deﬁnitematrix such that K i,j depends only on the inputs x i ∈ X and x j ∈ X , then the log-determinantfunction log det K X deﬁned over sets X is submodular [45].
Similar to [50], we will show that thislower bound batch acquisition function can be written as a sum of these log-determinants. [Equation] ) n
[Equation]
x [1: q ] , x [1: q ] ) + [Equation] [Equation] x [1: q ] )))
+ log det (cid:16) diag (cid:16) e
[Equation]
( x [1: q ] ) /M (cid:17)(cid:17)(cid:17) where
[Equation]
For notational convenience, let A ( m ) =
Σ ( m ) n ( x [1: q ] , x [1: q ] ) + diag ( σ ( m ) ( x [1: q ] )) for m = 1 , . .
, Mand B = diag (cid:16) e ζ ( x [1: q ] ) /M
(cid:17) .
Combining the terms, we obtain
[Equation]
, where K ( m ) =
BA ( m ) B with K ( m ) i,j = e ζ ( x [ i ] ) /M
e ζ ( x [ j ] ) /M
A ( m ) i,j for i, j = 1 , . . .
, q and m =1 , . .
, M .
Each summand, log det K ( m ) , is a submodular function [45].
As a sum of submodularfunctions is submodular, we conclude using that the lower bound batch acquisition function issubmodular.
For the approximate batch acquisition function, the expectation in (23) is replaced bythe corresponding Monte Carlo estimate.
The submodularity derivation above continues to hold whenusing the conditional entropy approximations.
[Equation]
F Cost analysis
In this section we consider the costs involved in evaluating the JES acquisition function according to
Algorithm 1.
We also include some discussion of the cost of the other information-theoretic criterion,namely MES and PES.
Sampling cost.
The cost of approximate sampling from a single Gaussian processes p ( f ( m ) | D n )using the random Fourier features (described brieﬂy in Appendix A.3) is O (min( n, L ) 3 ) , where L isthe number of Fourier features—for more details refer to [76].
An evaluation of a sample at a set ofinputs X ⊂ X has a linear cost depending on | X | [76].
Multi-objective optimization cost.
Given a sample f s , we optimize for p Pareto optimal points( X ∗ s , Y ∗ s ) using the popular genetic algorithm known as NSGA2 [18].
The cost of this algorithm is
O ( MN 2 pop N gen N off ) , where N pop is the size of the population, N gen is the number of generations and
N off is the number of offspring[18].
At a high level, NSGA2 works by evaluating the function at
N pop locations at time t = 1 and then moves on to evaluate N off candidates for the rest of the timet = 2 , . .
N gen .
The location of the offspring evaluations are determined by a random heuristicmotivated by the mechanisms involved in the theory of evolution: crossover, mutation, elitism anddiversity.
Box decomposition.
The cost of performing a single the box decomposition based on the incremental algorithm in [46] is O ( p (cid:98) M/ 2 (cid:99) +1 ) , where p is the number of Pareto optimal points.
Conditioning cost.
Conditioning on an additional p data-points requires updating the Choleskydecomposition of the input covariance matrix.
The cost of a single update of this kind relies on atriangular solve, which has a quadratic complexity
O ( M ( n + p ) 2 ) .
Initial entropy evaluation cost.
The initial entropy (4) can be computed directly from the posterior covariance.
The cost of instantiating the caches of the Gaussian process covariance, namely(Σ ( m ) 0 ( X n , X n ) + diag ( σ ( m ) ( X n )) − 1 is O ( n 3 ) .
The cost of evaluating the posterior covariance(14) at point x ∈ X is O ( n 2 ) .
In the batch case, we also have to compute a log-determinant of theq × q covariance matrix, which has a cost of O ( q 3 ) .
We use the implementation in the GPyTorch[26], which computes the approximate log-determinant, which only uses a linear cost of O ( q ) —thisapproximation approach is outlined in [21].
Conditional entropy evaluation cost.
Assuming the posterior model and conditioned model areinstantiated, we will consider the operations involved when evaluating the conditional entropyestimates.
The cost of evaluating the probability density function and CDF of a univariate normaldistribution at a point x ∈ X are both assumed to be O (1) .
As a result, the dominant cost ofevaluating the h JES- 0 (9) and h JES-LB 2 (11) comes from the evaluation of the variance.
For h JES-LB(10) , we additionally have to populate an M × M matrix and compute its log-determinant.
For h JES-MC(20), we need to initialize a set of I base samples for the reparametrization trick [75] described in
Appendix D—this has a linear one time cost of O ( MI ) .
Assuming the Monte Carlo samples aregenerated, the rest of the operations depend linear on I .
Expectation propagation.
To approximate the density p ( y | x , D n , X ∗ ) in the PES acquisitionfunction (1), the authors of [27] consider using expectation propagation [49].
The dominant cost ofthis is from matrix inversions of the covariance matrix, which scales cubically with the number of datapoints in consideration.
In the initialization phase, we prepare the expectation propagation caches,which require inverting an ( n + p ) × ( n + p ) matrix, whilst during testing we invert a ( q + p ) ×
( q + p )matrix, where q is the batch size.
Total cost.
The total cost is the sum of the initialization cost and the query cost.
Using the variablesdeﬁned in Table 1 and Table 2, we can write down the initialization and query cost of JES, MES and PES.
For the calculations, we assume that we have already queried n points and we want to computethe acquisition function at a batch of q points using S Monte Carlo samples of the Pareto set and/orfront comprised of p points.
• JES.
The initialization cost is C init + SC sample ( L )+ SC moo ( N pop , N gen , N off )
+ SC bd ( p )+ SC cond ( p ) . There is an additional initial cost of SC base-sample ( I ) for the Monte Carlo conditional entropyestimate.
The query cost is C init-h ( q ) + SqC h ( p ) , where C h ∈ { C h-0 , C h-lb , C h-lb2 , C h-mc } is theconditional entropy estimation strategy.
Keeping the other parameters ﬁxed, the dominant querycost is Mq 2 when using the approximate log-determinant and Mq 3 when using the standardlog-determinant for large q • MES.
We consider the MES algorithm obtained by excluding the conditioning step described in Algorithm 1.
The initialization cost of this approach is C init ( q ) + SC sample ( L )
+SC moo ( N pop , N gen , N off )
+ SC bd ( p ) .
The query cost is C init-h ( q ) + SqC h (0) , where C h ∈{ C h-0 , C h-lb , C h-lb2 , C h-mc } is the conditional entropy estimation strategy.
Keeping the other parameters ﬁxed, the dominant query cost is Mq 2 when using the approximate log-determinant and Mq 3when using the standard log-determinant for large q .• PES.
We consider the expectation propagation approach described in [27] to approximate the PESacquisition.
The batch extension is derived in a follow-up work [29].
The initialization cost ofthis approach is C init ( q ) + SC sample ( L )
+ SC moo ( N pop , N gen , N off )
+ SC
ep0 ( p ) .
The query cost isC init-h ( q ) + SC ep ( q, p ) .
Keeping the other parameters ﬁxed, the dominant query cost is SMq 3 forlarge q .
Naturally, the initialization phase can be executed in parallel because we are running S independentoperations.
For our experiments, we did not take advantage of this advantageous property whenperforming the sampling, multi-objective optimization and box decomposition.
Operation
Reference
Cost
Initializing the posterior p ( f | D n )
C init
Mn
Log-determinant of a K × K matrix
C logdet ( K )
K
Approximate log-determinant of a K × K matrix
C alogdet ( K )
K
Generating I base samples from N (0 , diag ( 1 M ))
C base-sample ( I )
MI
Approximate sampling of p ( f | D n )
C sample ( L )
M min( n, L )
Multi-objective optimization of f s
C moo ( N pop , N gen , N off )
MN 2 pop N gen N off
Box decomposition of Y ∗ s with p points
C bd ( p ) p (cid:98) M/ 2 (cid:99) +
Conditioning on the Pareto optimal point p ( f | D n ∗ )
C cond ( p )
M ( n + p )
Initialization of expectation propagation caches
[Table]
[Table]
[Table]
Posterior covariance Σ n ( X, X )
C cov ( | X | , n )
M ( | X | n 2 + | X | 2 )
Initial entropy H [ p ( y [1: q ] | x [1: q ] , D n )]
C init-h ( q )
C cov ( q, n ) + MC logdet ( q )
Conditional entropy estimate h JESC h-0 ( p )
C cov (1 , n + p )
Conditional entropy estimate h JES-LB
C h-lb ( p )
C cov (1 , n + p ) + M 2 + C logdet ( M )
Conditional entropy estimate h JES-LB
C h-lb2 ( p )
C cov ( [Equation]
Conditional entropy estimate h JES-MC
[Equation]
Expectation propagation
[Equation]
G Contour plots
In this section we present some contour plots to illustrate visually the differences that emergesbetween the different approximation strategies for the information-theoretic acquisition functionson a single objective problem.
The approximation for PES, MES and JES are presented in Figure 7,
Figure 8 and Figure 9, respectively.
To obtain the ground truth for the acquisition function we runthe rejection sampling scheme discussed in [36].
As a comparison we also include the contoursof some popular single objective acquisition functions.
Visually the estimates all appear to bereasonably effective for this set-up.
Interestingly, the PES and MES prefer querying around the lowermode, whilst JES prefers the upper mode which perhaps offers a better trade-off between the gain ininformation about both the optimal input and output.
1.0 x 1 0.
x
Rastrigin 0.
1.0 x 1 0.
x0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .
Gaussian Process Mean 0.
1.0 x 1 0.
x
Gaussian Process Entropy 1 .
1.0 x 1 0.
x
PES ( x | D n ) 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0.
1.0 x 1 0.
x
PES ( x | D n ) 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .
1.0 x 1 0.
x
TS ( x | D n ) 0 .
1.0 x 1 0.
x
MES ( x | D n ) 0 .
1.0 x 1 0.
x
MES 0 ( x | D n ) 0 .0 .0 .0 .
1.0 x 1 0.
x
MES LB ( x | D n ) 0 .
1.0 x 1 0.
x
MES MC ( x | D n ) 0 .
1.0 x 1 0.
x0 .0 .0 .
JES ( x | D n ) 0.
1.0 x 1 0.
x
JES 0 ( x | D n ) 0 .0 .0 .
1.0 x 1 0.
x
JES LB ( x | D n ) 0 .0 .0 .
1.0 x 1 0.
x
JES MC ( x | D n ) 0 .0 .0 .0 .0 .
1.0 x 1 0.
x
EI ( x | D n ) 0 .0 .
1.0 x 1 0.
x
PoI ( x | D n ) 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0.
1.0 x 1 0.
x
UCB ( x | D n ) 0 .0 .0 .0 .0 .
1.0 x 1 0.
x
KG ( x | D n ) 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .
H Hypervolume indicator
[Figure]
The hypervolume indicator is deﬁned as the area of the dominated region between a reference pointr ∈ R M and the set of interest A ⊂ R M :
[Equation]
As discussed in the main text (Section 4), the HV indicator is sensitive to the parametrization of theobjective space.
In order to build a more complete picture about the performance of a multi-objectiveoptimization algorithm, we consider tracking the HV under a number of different parameterization.
Instead of deriving a family of transformation function, we consider a more general approach basedon a different formulation of the HV indicator described in [19, 81].
In particular, the HV indicatorcan be written as an expectation over a probability distribution by performing a change of variablesinto spherical polar coordinates:
[Equation]
[Equation]
where Γ( · ) is the Gamma function and the scalarization function s λ : R M → R is deﬁned as
[Equation]
For the standard HV, the inverse weight distribution p ( λ ) is a uniform distribution over the surface ofthe M -dimensional unit sphere in the non-negative orthant S M − 1 + = { λ ∈ R M ≥ 0 :
(cid:80) Mm {+=
+}
( λ ( m ) )
2 =1 } .
The inverse weight distribution p ( λ ) controls the radial contribution for each point of A towardsthe HV.
Existing work mainly considers a uniform distribution over the weights in order to compute [Figure] the standard HV.
We make the novel observation that we can assess the quality of the Pareto front indifferent regions of the objective space by varying this weight distribution.
We call the resultingutility function the generalized hypervolume (GHV) indicator, denoted by U GHV [-.
-].
[Figure]
This GHV satisﬁes a weaker version of the Pareto complete property: A (cid:23) B = ⇒ U GHV ( A ) ≥
[Figure]
U GHV ( B ) .
This property can be proved from the fact that the scalarization function s λ ( a ) is a [Figure] [Figure] space
[Figure]
Objective space
[Figure]
monotonic increasing function in a ∈ A , which implies that the Pareto order is maintained.
Thisperformance criteria satisﬁes the strict Pareto complete property when all of the weights have non-zerodensity p ( λ ) > 0 .
Intuitively, if a set of weights had zero density, then an improvement along thesedirection will not always result in a larger GHV.
This means that we will not always be able to distinguish between a set which strictly dominates another, which is essential for strict Pareto completeness.864 To generate different distributions p ( λ ) , we exploit the fact that the set S M − 1 + is isomorphic to the ( M − 1) -dimensional unit hypercube [0 , 1] M − 1 .
In particular, any point w ∈ [0 , 1] M − 1can be mapped onto the sphere to a point λ ( w )
∈ S M − 1 + with
[Equation]
Using this result, we can use any distribution with a ﬁnite support to generate samples on S M − 1 + .
For example, we use M − 1 independent Beta distribution Beta ( a m , b m ) to generate samples fromw ∈ [0 , 1] M − 1 .
In Figure 12, we present the radial contributions to the GHV for different weightdistributions p ( w ) .
Remark.
We are not the ﬁrst to observe the importance of weighting the HV towards regionsof interest.
For example, an early paper [86] proposed another a weighted hypervolume indicator(WHV), U WHV ( A ) =
(cid:82) R M w ( z ) I
[ r (cid:22) z (cid:22) A ] d z , which introduces a weight function w : R M → Rdirectly into the integral.
This approach is not as ﬂexible as the one presented here because it reliedon designing hand-crafted weight functions and an involved symbolic integration procedure.
I ExperimentsI.1 Implementation details
All of the numerical results was implemented on Python 3.8 using open-source libraries: BoTorch(0.5.1) [3], GPyTorch (1.6.0) [26], NumPy (1.21.2), Pymoo (0.5.0) [8] [33], PyTorch (1.9.0) [56] and
SciPy (1.7.3) [73].
All computations were performed on a computing cluster, where we restricted thecomputation to a single CPU core of an AMD EPYC 7742 64-Core Processor @ 2.25GHz.
The codeis available at https://github.com/.../JES .
Gaussian process model.
For all the problems, we normalized the inputs and standardized theobservations before performing Gaussian process regression.
We assume an independent model for each objective with a constant mean and Matérn 5/2 ARD kernel.
Additionally, we assumea zero-mean Gaussian observation noise with a homogeneous variance σ ( x ) = σ ∈ R M .
Forconvenience, we rely on the default model in BoTorch, which additionally places Gamma priors forthe kernel hyperparameters and observation variance.
At each iteration of BO, we update the pointestimates for the model hyperparameters and observation variance by maximizing the exact marginallog-likelihood using SciPy’s default gradient optimizer.
Optimizing the acquisition function.
To determine the next point, we optimize the acquisitionfunction using multi-start L-BFGS-B [73].
We use the exact gradient inferred using automaticdifferentiation [56] for all algorithms except for the PES.
The automatic gradients inferred for the
PES acquisition function occasionally failed due to the differentiability issues arising from thedamping procedure within the expectation propagation update [27].
As a result, we used the default2-point ﬁnite difference method in SciPy to approximate the gradients of the PES acquisition function.
To initialize the multi-start gradient optimizer, we evaluated the acquisition function on a space-ﬁllingdesign of 1000 D random points.
The starting points were then chosen as the best performing pointsfrom the initial check.
The number of starting points that we used differed for each algorithm basedon the overall expenses.
For the sequential experiments, we used 20 starting points for PES, 5 Dstarting points for the MES and JES, and 10 D starting points for the rest.
For the batch experiments,we used 20 starting points for PES and 5 D starting points for the rest.
For the information-theoreticacquisition functions, we could have initialized the starts near the sampled Pareto optimal points inorder to promote faster convergence.
We abstained from doing this because we wanted to give a faircomparison between the existing approaches.
Speciﬁcally, we wanted to see how well informationbased acquisition function worked out of the box without any optimization compared to the existingoptimized approaches.
Sampling the Pareto points.
To sample the Pareto optimal inputs and outputs, we ﬁrst sample
Gaussian process paths.
We achieved this by using an approximate sampling strategy based onrandom Fourier feaures (Appendix A.3).
We used the implementation in BoTorch and set the numberof features to L = 500 .
To solve for the Pareto set and front, we used the multi-objective solver
NSGA2 [18], which was implemented in Pymoo.
We set the population size to be N pop = 100 , thenumber of generations to be N gen = 500 and the number of offspring to be N off = 10 .
In general,the multi-objective solver outputs an approximate set of Pareto optimal points with a size less thanor equal to N pop .
To truncate this set into a size p , we used a HV truncation strategy.
In particular,we greedily selected points based on their contribution to the sample HV generated by the samplef s ( X n ) —the reference point is set to the current estimate of the nadir minus some error ˆ r n − 0 .
1 | ˆ r n | ,where ˆ r ( m ) n = min t =1 ,...,n y ( m ) t .
This reﬁnement strategy can be implemented quickly using theexpected HV improvement strategy discussed in [14].
We implemented this sampling and optimizingprocedure in sequence, but naturally this can be executed in parallel because we are performing Sindependent computations.
As motivated by the wall times in Appendix I.6, implementing this stepin parallel could be very beneﬁcial computationally.
Box decompositions.
We used the BoTorch implementation of the box decomposition strategydiscussed in Algorithm 1 of [46].
The box decompositions were performed in sequence insteadof being executed in parallel.
From the wall times presented in Appendix I.6, the time required tocompute the box decompositions becomes more demanding as the number of objectives increases.
Conditioning.
We used the fantasizing feature in BoTorch to condition the current posterior ona collection of S independent Pareto samples of size p .
We treated the Pareto samples as noisypseudo-observations in the conditioning.
Acquisition functions.
The benchmark comparison considers a range of popular acquisition functions that have all been implemented in BoTorch.
We implemented our own version of the multiobjective PES, MES and JES.
We now elaborate on the implementation details for each acquisitionfunctions in the benchmark experiments.• TSEMO.
The Thomson sampling algorithm (TSEMO) [10] is a random acquisition function,which selects the point that maximizes the HV improvement according to a single sample of thePareto front.
Unlike the original paper [10], we select the point that improves the HV of the samplefrontier f s ( X n ) as opposed to the observation frontier generated by Y n .
We ﬁnd this adjustment to be sensible when there is observation noise.
We modiﬁed the greedy batch extension in the sameway.• ParEGO/ NParEGO.
The random scalarization strategy (ParEGO) [43] and its noisy counterpart(NParEGO) [15] are one of the most popular strategies for multi-objective BO.
At each iteration, arandom scalarization of the objectives is obtained by randomly drawing a weight w ∈ R M fromthe ( M − 1) -simplex.
To target all the Pareto optimal points, we use the augmented Chebyshevscalarization: min m =1 ,...,M w ( m ) ˜ f
( m ) + 0 .
01 (cid:80) m
=1 w ( m ) ˜
f
( m ) .
Here we have denoted ˜ f
∝ f asthe objective function, which has been approximately normalized to [0 , 1] M using the observationsY n .
For the single objective problem, a Monte Carlo estimate of the expected improvement andthe noisy expected improvement for the ParEGO and NParEGO algorithm are used respectively.
The number of base samples for the Monte Carlo estimates is set to 128 .
For the batch setting, wesampled q different weights and optimized the acquisition function sequentially.• EHVI/ NEHVI.
The expected hypervolume improvement (EHVI) [14] and its noisy counterpart(NEHVI) [15] are an improvement-based acquisition function for the HV indicator.
The numberof base samples for the Monte Carlo estimates is set to 128 .
We set the reference point of theHV indicator to be equal to the observed nadir minus some error, ˆ r n − 0 .
1 | ˆ r n | , where ˆ r ( m ) n =min t =1 ,...,n y ( m ) t .
This dynamic reference point strategy was recommended in the supplementarymaterial of [14].
For the batch setting, we considered a sequentially greedy optimization strategy[14, 15].• PES.
The predictive entropy search (PES) [27, 29, 35, 36] acquisition function is approximatedusing expectation propagation.
We implemented this algorithm from scratch in BoTorch underthe guidance of the supplementary material in [27].
We used S = 10 Monte Carlo samples andp = 10 number of Pareto optimal inputs.
We set the jitter for the matrix inversion to be 0 .
001 andthe convergence threshold for the initialization stage to be 5% relative change in the mean andcovariance.
If the expectation propagation failed to converge, we outputted a random vector fromthe already sampled Pareto sets as the ofﬁcial recommendation.
For the batch setting, we extendedthe approach outlined in [27].
This extension appears to be equivalent to the approach described in[29].
We optimized the resulting acquisition function using a joint optimization approach.• MES.
The maximum value entropy search (MES) [68, 74] can be approximated using all theconditional entropy estimates we devised in this paper.
We used S = 10 Monte Carlo samplesand p
= 10 number of Pareto optimal outputs.
For the Monte Carlo estimate MES-MC we set thenumber of base samples to 128.
For the batch setting, we considered a greedy optimization strategy[14, 15].
For the batch setting, we consider the lower bound described by (12).
We optimized theresulting acquisition function using a greedy optimization approach.• JES.
The joint entropy search (JES) [68, 74] can be approximated using all the conditional entropyestimates we devised in this paper.
We used S =
Monte Carlo samples and p = 10 number ofPareto optimal points.
For the Monte Carlo estimate JES-MC we set the number of base samplesto 128.
For the batch setting, we consider the lower bound described by (12).
We optimized theresulting acquisition function using a greedy optimization approach.
Optimizing for the recommendation.
To obtain the recommendation set, ˆ X ∗ n , we used the NSGA2multi-objective solver to optimize the posterior mean µ n .
Using the Pymoo implementation, we setthe population size to be N pop = 500 , the number of generations to be N gen = 500 and the numberof offspring to be N off =
We select the p = 50 points that greedily maximizes the HV generatedby the mean objectives.
The reference point for the HV truncation was set to the current estimateof the nadir minus some error:
ˆ r n − 0 .
1 | ˆ r n | , where ˆ r ( m ) n = min t =1 ,...,n y ( m ) t .
Before applying thetruncation, we ﬁrst augmented the Pareto set found at this iteration with the previous recommendationset ˆ X ∗ n − 1 in order to guarantee that some promising solutions weren’t missed due to randomness ofthe solver.
Optimizing for the Pareto set.
To obtain the baseline Pareto set, X ∗ , we used the Pymoo’s NSGA2with a population size to be N pop = 1000 , the number of generations to be N gen = 5000 and thenumber of offspring to be N off = 10 .
I.2 Benchmark problems
We initialized each test problem with with 2( D + 1) training points using a random space ﬁllingdesign.
The details of the individual benchmark problems is presented below.
ZDT2 (D=2, M=2, Noise=10%).
The ZDT2 benchmark [85] is a bi-objective minimization problem min x ∈ X f ( x ) , over the D -dimensional hypercube X
=
[0 , 1] D , where
[Equation]
[Equation]
x
[Equation]
with g ( x ) = 1 − 9 D − 1 (cid:80) Di =2 x ( i ) .
For the experiments, we considered the maximization problem bynegating the objective: max x ∈ X ( − f ( x )) .
The standard deviation of the Gaussian observation noiseis set to σ 1 / 2 = (0 . 1 , 0 .
8) , which is estimated to be around 10% the range of the objectives from1000 function evaluations.
For the HV indicators we use the reference point of r = ( − 11 , − 11) .
SnAr (D=4, M=2, Noise=3%).
The SnAr benchmark [23] is a bi-objective minimization problemmin x ∈ X f ( x ) , which is deﬁned over a four-dimensional space, X =
[0 . 5 , 2 . 0] × [1 . 0 , 5 . 0] × [0 . 1 , 0 . 5] ×[30 , 120] , where
[Equation]
with STY and E-Factor denoting the space-time yield and environmental factor, respectively.
Thefunctions STY and E-Factor depend on the solution of an ordinary differential equation governedby a kinetic model, where the rate constants have been estimated by empirical tests [41].
Unlike theoriginal paper [23], we have taken the logarithm of the output to accommodate for the additive noisein the objective space.
For the experiments, we considered the maximization problem by negatingthe objective: max x ∈ X ( − f ( x )) .
The standard deviation of the Gaussian observation noise is setto σ 1 / 2 = (0 .
12 , 0 . 08) , which is estimated to be around 3% the range of the objectives from 1000function evaluations.
For the HV indicators we use the reference point of r = (5 . 5 , − 5) .
Penicillin (D=7, M=3, Noise=1%).
The Penicillin benchmark [47] is a three objective minimization problem min x ∈ X f ( x ) , which is deﬁned over a seven-dimensional space, X =
[60 , 120] ×[0 . 05 , 18] × [293 , 303] × 05 , 18] ×
[0 . 01 , 0 . 5] × [500 , 700] × [5 , 6 . 5] , where
[Equation]
with PenicillinConcentration , CO2Concentration and TimeToFerment denoting the concentration of the desirable product, the concentration of the subproduct and the time to ferment,respectively.
The equations describing the pharmaceutical simulation are given in the original reference [47].
We used the open-source implementation available in BoTorch [3, 17].
For the experiments,we considered the maximization problem by negating the objective: max x ∈ X ( − f ( x )) .
The standarddeviation of the Gaussian observation noise is set to σ 1 / 2 = (0 . 14 , 0 . 8 , 3 .
8) , which is estimated tobe around 1% the range of the objectives from 1000 function evaluations.
For the HV indicators weuse the reference point of r = ( − 1 . 85 , − 86 .
93 , − 514 .
Marine Design (D=6, M=4, Noise=0.5%).
This marine design benchmark [55, 61, 71] is a fourobjective minimization problem min x ∈ X f ( x ) , which is deﬁned over a six-dimensional space, X =[150 , 274 . 32] × [20 , 32 . 31] × [13 , 25] × [10 , 11 .
71] × [14 , 18] × [0 . 63 , 0 .
75] , where
[Equation]
[Equation]
with TransportationCost , Weight , AnnualCargo and SumOfConstraints denoting the transportation cost, the ship weight, the annual cargo transport capacity and the sum of the constraintviolations, respectively.
The formal equations describing the functions are presented in the original references [55, 61, 71].
For the experiments, we considered the maximization problem bynegating the objective: max x ∈ X ( − f ( x )) .
The standard deviation of the Gaussian observationnoise is set to σ 1 / 2 = (10 , 77 , 132 , 0 . 07) , which is estimated to be around 0 .
5% the range of theobjectives from 1000 function evaluations.
For the HV indicators we use the reference point ofr
= (250 , − 20000 , − 25000 , − 15 . 0) .
I.3 Sensitivity analysis
In the section we empirically analyse how sensitive the approximations to the information-theoreticacquisition functions are with different choices of Monte Carlo samples S and number of Paretooptimal points p .
For the experiments in the main section, we set S = 10 and p = 10 for allinformation-theoretic acquisition functions.
This selection is comparable to the existing literature[4, 27, 68, 74].
For the sake of brevity we present results only for one benchmark problem: ZDT2(D=2, M=2, Noise=10%, [-q=1).
-]q=1).1039 To test the sensitivity with regards to the number of Monte Carlo samples S , we ﬁx p = 10 and ranthe benchmark problem 100 times with S ∈ { 1 , 5 , 10 , 25 , 50 } .
To test the sensitivity with regards tothe number of Pareto optimal points p , we ﬁx S = 10 and ran the benchmark problem 100 timeswith p ∈ { 1 , 5 , 10 , 25 , 50 } .
We report the median log HV discrepancy over the runs in Figure 13 and Figure 14.
We report the wall times in Figure 15. For the log HV plots, we did not include error bars [Figure] because it is very hard to visually discriminate between the results.
We analysed the upper and lower [Figure]
quartile plots and they showed a similar ranking to the plots displayed below. Some experiments [Figure] could not be performed because of the unwieldy space and time requirements.
[Figure]
Performance-wise there does not appear to be much variation for acquisition function when [Figure] S > 1 and p > 1 . Naturally the wall times increase with the number of Monte Carlo samples S
[Figure]
because the sampling and optimization of the Gaussian process paths are done in sequence (this stepcould be parallelized in practice). Overall, for this example problem, there does not appear to be
[Figure]
much beneﬁt in using a larger number of Monte Carlo samples S or Pareto optimal points p .
[Figure]
[Figure] l o g ( d H V ) PES 100 Iterations 2.
l o g ( d H V ) JESp=1p=5p= p=25p=      100 Iterations 2.
l o g ( d H V ) JES-LB     100 Iterations 2.
l o g ( d H V )
JES-LB
[Figure]
l o g ( d H V ) JES
[Figure]
MC
[Figure]
[Figure] [Figure] [Figure] [Figure]
[Figure]
l
[Figure]
g ( [Figure] H V [Figure]
[Figure] H V [Figure] [Figure]
[Figure]
H
[Figure]
MC
[Figure]
optimal samples p .
[Figure]
I.4 Sequential experiments (q=1)
For the sequential experiments, we have produced many different plots to compare the performanceof the various algorithm.
For all the experiments, we have plotted the log HV discrepancy at the meanand the quartiles.
We also include the log GHV discrepancy plots, which considers the HV underdifferent weight distributions as described in Appendix H. We also include plots of the Pareto frontapproximation in the objective space.
I.4.1 ZDT2 (D=2, M=2, Noise=10%, q=1)
From the plots below, we see that all the BO algorithms are all capable of targetting the points attop right of the Pareto front.
Notably, these points contributes the most towards the standard HVindicator.
The Thompson sampling and information-theoretic driven algorithms are able to a morediverse range of points on the Pareto front for this test problem.
[Figure]
[Figure]
G
[Figure]
V
[Figure]
[Figure]
[Figure]
D=
[Figure]
M=
[Figure]
, Noise=10%
[Figure]
q=
[Figure]
[Figure]
I.4.2 SnAr (D=4, M=2, Noise=3%
[Equation]
[Figure]
Almost all of the BO algorithms performed similarly on this benchmark—the only weak performerwas PES.
Despite achieving small HV discrepancy, Figure 24 illustrates that many of the BOapproaches struggle to identify the shape of the true Pareto front.
When looking at the variousparameterization in Figure 23, we see that the lower bound estimates MES-LB and JES-LB performedslightly better than the rest, especially in the left region of the Pareto front, where the second objectiveis preferred more than the ﬁrst.
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Figure]
[Figure]
[Figure]
I.4.3 Penicillin (D=7, M=3, Noise=1%, q=1)
The BO algorithms all performed similarly on this benchmark problems.
The HV plots gives anindication to how difﬁcult the underlying objective function is to model using a small number ofobservations.
The best performers in terms of the ﬁnal standard HV discrepancy were TSEMO,
NParEGO, NEHVI and JES.
From examining Figure 28, it is clear that the HV is dominated by thethird objective, which has a considerably larger range than the other objectives.
Figure 26 shows that
PES actually outperforms the other algorithms in identifying many parts of the Pareto front that donot contribute that much to the standard HV calculation.
This example again illustrates the difﬁcultyin assessing multi-objective algorithms using only a single criterion.
[Figure]
[Figure]
[Figure]
[Figure] log( d GHV ):Penicillin (D=7, M=3, Noise=1%, q=1) I.4.4 Marine (D=6, M=4, Noise=0.5%, q=1) For this benchmark, the lower bound JES approximation is the best in terms of the standard HV.
The noiseless estimates MES-0 and JES-0 perform much worse than the other conditional entropyapproximations.
We omitted the experiments for the MES-MC and JES-MC algorithms because theywere too expensive to compute.
From viewing the Pareto front plots Figure 33, it is clear that this thenoiseless approximations are more greedy than their counterparts.
[Figure]
[Figure]
[Figure]
[Figure]
I.5 Batch experiments (q=8)
[Figure]
In the previous section we considered the sequential setting where only q = 1 objective function
[Figure]
evaluation was allowed to take place at each time.
In this section, we consider the batch setting, [Figure] where we allow q = 8 function evaluations to take place at each time [Figure]
[Figure] [Figure] [Figure] [Figure]
log [Figure] d HV )
[Figure]
EHVI log( d HV )
[Figure]
NEHVI log( d HV )
[Figure]
PES log( d HV )
[Figure]
LB [Figure] MES
[Figure]
[Figure]
I.5.2 SnAr (D=4, M=2, Noise=3%, q=8)
[Figure]
[Figure]
{+=2, Noise=3%, q=8) 
 1 2 3 4 5 6 7 8 9 10 11 12 Iterations 1.+}
P 12 l o g ( d G H V ) SnAr (D=4, M=2, Noise=3%, q=8) 0 1 2 3 4 5 6 7 8 9 10 11 12 Iterations 2.
P 16 l o g ( d G H V )
SnAr (D=4, M=2, Noise=3%, q=8)
[Figure]
[Figure]
of model posterior means
I.5.3 Penicillin (D=7, M=3, Noise=1%, q=8)
We omitted the experiments for the qLB-MES-MC and qLB-JES-MC algorithms because they weretoo expensive to compute.
[Figure]
logarithm HV discrepancy, respectively.
[Figure]
[Figure]
I.5.4 Marine (D=6, M=4, Noise=0.5%, q=8)
We omitted the experiments for the qEVHI, qLB-MES-MC and qLB-JES-MC algorithms becausethey were too expensive to compute
[Figure]
[Figure]
[Figure]
I.6 Wall times
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
J Extensions
The discussion so far focussed on the problem of unconstrained multi-objective optimization whereevaluations of the vector-valued function are executed individually or in batch.
Not all multi-objective optimization fall into this category, so we propose some additional extensions to handle more generalproblems.
Constrained optimization.
In the literature there are several examples of how to generalizeinformation-theoretic acquisition functions to constrained optimization problems [6, 24, 27, 29, 37,38, 57, 70].
We propose a simple extension for the JES acquisition function, similar to [70], to handleinequality constraints.
In particular, suppose we are interested in maximizing an M -dimensionalblack-box function f (1: M ) ( x ) subject to K black-box inequality constraints f
( M + k ) ( x ) ≤ 0 fork = 1 , . .
, K .
We can model the constraints f
( M + k : M + K ) ( x ) as additional independent objectivesusing the same observation model described in Section 2.
The only difference between the constrainedsetting and the unconstrained setting is the region of integration for the CDF.
In the constrainedsetting, the sampled Pareto front dominates only the vectors satisfying the constraint.
Hence theconstrained CDF is now of the form p ( z ∈ D K (cid:22) ( Y ∗ ))
where
D K
[Equation]
( Y
[Equation]
) = { z ∈ R M + K : ( z (1: M ) (cid:22) Y ∗ and z ( M +1: M + K ) (cid:22) 0 K ) or ( z ( M +1: M + K ) (cid:31) 0 K ) }
[Equation]
[Equation]
M
[Equation]
(cid:31) (
[Equation]
[Equation]
) } =
[Equation]
[-(cid:31) 
-]
[Equation]
0 K )) .
This region can be decomposed into boxes in the almost same way as before.
The only differenceis that we now have an additional box arising from region where the constraint is not satisﬁed.
Ingeneral, the JES (and MES) acquisition function estimates described here can handle any type ofblack-box constraint as long as we are able decompose the feasible objective region into boxes.
Forexample, interval constraints of the form f
( M + k ) ( x ) ∈ [ a, b ] , can also be readily handled in thisframework.
Decoupled evaluations.
Evaluating all objectives at each iteration can be costly and perhapsunnecessary for practical problems.
To address this problem, researchers in BO have consideredthe use of decoupled [30] acquisition functions α M , which considers the quality of querying asubset of objectives { f ( m ) : m ∈ M} .
To the best of our knowledge, all of the existing decoupledacquisition functions in multi-objective BO are based on information-theoretic acquisition functions[27, 35, 38, 68].
The novel JES-LB2 and MES-LB2 described in this paper possesses this decouplingproperty because it can be decomposed into a sum of acquisition functions for each objective:α JES-LB2 ( x | D n ) =
(cid:80)
Mm =
α m ( x | D n ) .
By Theorem 4.1 of [68], the JES-0 and MES-0 can also begeneralized to the decoupled setting via a marginalization argument.
Upon reviewing the experimentalresults of the cited papers, we see that decoupled evaluations typically provide only a marginalimprovement over the non-decoupled strategies.
This is possibly down to heuristic choice to searchthe space of subsets.
A more principled search method based on some emerging ideas about how tooptimize over categorical inputs [28, 52, 60] might be useful to obtain further improvements.
Multi-ﬁdelity Bayesian optimization.
For practical optimization problems of interest, it is occasionally possible to evaluate approximations of the true objectives that are much cheaper.
Thisadditional degree of freedom has been exploited before in literature under the name of multi-ﬁdelity
Bayesian optimization [5, 6, 42, 50, 51, 66, 69, 79, 82].
These strategies have demonstrated somebeneﬁt when optimizing with cost constraints on the function evaluations.
It is possible to adapt
JES to the multi-ﬁdelity by combining the ideas introduced here with the ideas before in the papersreferred to above such as using cost weights and conditioning arguments on lower ﬁdelities.
Themain obstacle to extending this work to the multi-ﬁdelity setting will likely arise from some lengthyalgebraic exercises relating to the conditional entropy and box decompositions.