Joint Entropy Search For Maximally-Informed Bayesian Optimization
Anonymous Author(s)
Abstract
Information-theoretic Bayesian optimization techniques have become popular foroptimizing expensive-to-evaluate black-box functions due to their non-myopic qualities.
Entropy Search and Predictive Entropy Search both consider the entropy overthe optimum in the input space, while the recent Max-value Entropy Search considers the entropy over the optimal value in the output space.
We propose Joint Entropy
Search (JES), a novel information-theoretic acquisition function that considers anentirely new quantity, namely the entropy over the joint optimal probability densityover both input and output space.
To incorporate this information, we consider thereduction in entropy from conditioning on fantasized optimal input/output pairs.
The resulting approach primarily relies on standard GP machinery and removescomplex approximations typically associated with information-theoretic methods.
With minimal computational overhead, JES shows superior decision-making, andyields state-of-the-art performance for information-theoretic approaches across awide suite of tasks.
As a light-weight approach with superior results, JES providesa new go-to acquisition function for Bayesian optimization.
Introduction
The optimization of expensive black-box functions is a prominent task, arising across a wide rangeof applications.
Bayesian optimization (BO) [24, 33] is a sample-efficient approach and has beensuccessfully applied to various problems, including machine learning hyperparameter optimization [2,19 , 31, 35], robotics [3, 6, 23], hardware design [10, 25] and tuning reinforcement learning agents like
AlphaGo [7].
In BO, a probabilistic surrogate model is used for modeling the (unknown) objective.
The selection policy employed by the BO algorithm is dictated by an acquisition function, whichdraws on the uncertainty of the surrogate to guide the selection of the next query.
The choice of acquisition function is significant for the success of the BO algorithm.
A popular lineof acquisition functions takes an information-theoretic angle and considers the expected informationgain regarding the location of the optimum that is obtained from an upcoming query.
Entropy Search(ES)
[14], Predictive Entropy Search ( PES ) [15] and the earlier work of IAGO [42] select queriesby maximizing this quantity.
While ES and PES are efficient in the number of queries to optimizethe objective, they both require significant computational effort and complex approximations of theexpected information gain, which impacts their performance and practical use [15, 43].
To addressthis computational expense, efficient parameterizations of the objective have been proposed [29].
However, this solution imposes constraints in the ability to accurately model the objective.
A related information-theoretic family of approaches considers the information gain on the optimalobjective value [17, 43].
Max-value Entropy Search ( MES ) [43] was the first information-theoreticapproach to have a proven convergence rate, albeit only in a noiseless problem setting.
Moreover,its consideration of a one-dimensional density over the output space as opposed to a D -dimensional input space and a reduction in intricate approximations yielded a computationally efficient alternativeto the ES / PES line of approaches.
Despite its empirical success, some crucial shortcomings of MEShave been highlighted in recent works [26, 39].
Most importantly, it does not differentiate betweenthe (unobserved) maximal objective value f ∗ and the observed noisy maximum y max .
As such, itsassumption on the posterior distribution of the output p ( y |D , x ) does not hold in any setting wherenoise is present [26].
We propose an approach which merges the ES / PES and MES lines of work, and provides an allencompassing perspective on information gain regarding optimality.
We introduce Joint Entropy
Search (JES), a novel acquisition function which has the following advantages over existinginfomation-theoretic approaches:
It utilizes two sources of information, by considering the entropy over both the optimumand optimal value;2.
It retains low computational and conceptual complexity by utilizing the full optimal obser-
vation, allowing it to rely on standard GP machinery instead of complex approximations;and3.
It intrinsically supports noisy objective functions, making it particularly efficient for these types of tasks.
The code for reproducing the experiments is available athttps://github.com/jointentropysearch/JointEntropySearch .
Background and related work
Bayesian optimization.
We consider the problem of optimizing a black-box function f across a setof feasible inputs X ⊂ R d :
[Equation]
We assume that f ( x ) is expensive to evaluate, and can potentially only be observed through anoise-corrupted estimate, y , where
y = f ( x ) + ϵ, ϵ ∼ N (0 , σ 2 ϵ ) for some noise level σ 2 ϵ .
In thissetting, we wish to maximize f in an efficient manner, typically while adhering to a budget whichsets a cap on the number of points that can be evaluated.
BO aims to globally maximize f by aninitial design and thereafter sequentially choosing new points x n for some iteration n , creating thedata D n = D n − 1 ∪ { ( x n , y n ) } .
After each new observation, BO constructs a probabilistic surrogatemodel p ( f |D n ) and uses that surrogate to build an acquisition function α ( x , D n ) .
The combinationof surrogate model and acquisition function encodes the strategy for selecting the next point x n +1 .
After the full budget of N iterations is exhausted, a best configuration x ∗ N is returned as either thearg max of the observed values, or the optimum as predicted by the surrogate model.
Gaussian processes.
When constructing the surrogate, the most common choice is the Gaussianprocess (GP) [28].
Formally, a GP is an infinite collection of random variables, such that every finitesubset of those variables follows a multivariate Gaussian distribution.
The GP utilizes a covariance(or kernel) function k , which encodes a prior belief for the smoothness of f , and determines howprevious observations influence prediction.
Typical choices for the kernel are the squared exponentialand Matérn-5/2 [22] kernel.
Given the observations D n at iteration n , the density p ( f |D n ) over theobjective is characterized by the posterior mean m n and variance s n of the GP:
[Equation]
where ( K n ) ij = k
( x i , x j ) , k n ( x ) =
[ k ( x , x 1 ) , . .
k ( x , x n )]
⊤ and σ 2 ϵ is the noise variance.
Alternative surrogate models include random forests [18] and Bayesian neural networks [36, 37].
Acquisition functions.
The acquisition function acts on the surrogate model to quantify theattractiveness of a point in the search space.
Acquisition functions employ a trade-off betweenexploration and exploitation, typically using a greedy heuristic to do so.
Simple, computationallycheap heuristics are Expected Improvement( EI ) [5, 20].
For a noiseless function, EI selects the nextpoint x n +1 as
[Equation]
where Z = ( y ∗ n − m n ( x ))
/s n ( x ) .
Other acquisition functions which use similar heuristics are the
Upper Confidence Bound ( UCB ) [38], and Probability of Improvement(PI) [21].
A more sophisticatedapproach related to EI is the Knowledge Gradient [11].
Information-theoretic acquisition functions.
Information-theoretic acquisition functions [14, 15,30, 43] and their various adaptations [1, 16, 32] seek to maximize the expected information gain I from observing a subsequent query ( x , y ) regarding the optimum, x ∗ .
This equates to reducingthe uncertainty of the density over the optimum, p ( x ∗ |D )
=
P ( x = arg max x ′ ∈X f ( x ′ ) |D ) , usingthe information obtained through ( x , y ) .
By quantifying uncertainty through differential entropy H,design points are selected based on the expected reduction in this quantity over p ( x ∗ |D ) .
Formally,this is expressed as the difference between the current entropy over p ( x ∗ |D ) , and the expected entropyof that density after observing the next query:
[Equation]
By utilizing the symmetric property of the mutual information, one can arrive at an equivalentexpression, where the entropy is computed with regard to the density over the output y ,
[Equation]
Eq. 4 is the original formulation used in ES [14] and Eq.
5 is the formulation introduced with PES [15].
Both formulations require a series of approximations and expensive computational steps to computethe entropy in the second term.
For PES specifically, with n data points of dimension d , the secondterm is estimated through Monte Carlo (MC) methods by computing Cholesky decompositions ofsize O ( n + d 2 / 2) 3 , and approximating the Hessian at the optimum for each MC sample.
MES [43] avoids this computational hurdle by considering the information gain I (( x , y ); y ∗ ) regardingthe optimal value y ∗ .
As such, it computes the entropy reduction for a one-dimensional density:
[Equation]
Here, it is assumed that the posterior predictive distribution p ( y |D , x , y ∗ ) is a truncated Gaussiandistribution, for which the entropy can be computed in closed form.
However, p ( y |D , x , y ∗ ) takesthis form only in a strictly noiseless setting [26, 39], where it holds true that f ∗ =
y ∗ , i.e., whenthe maximal observation and the optimal value of the objective function are the same.
For noisyapplications, this assumption leads to an overestimation of the entropy reduction [26].
Joint Entropy Search
We now present Joint Entropy Search ( JES ), a novel information-theoretic approach for Bayesianoptimization.
As for other information-theoretic acquisition functions, JES considers a mutualinformation quantity.
However, unlike its predecessors, JES adds an additional piece of information:compared to ES / PES , it adds the density over the noiseless optimal value f ∗ , and compared to MES ,it adds the density over x ∗ .
It utilizes a novel two-step reduction in the predictive entropy fromconditioning on sampled optima and their associated values.
Throughout the section, we will refer toa sampled optimum and its associated value, ( x ∗ , f ∗ ) , as an optimal pair .
3.1 Joint density over the
optimum
and
optimal
value:
pictorial
JES considers the joint probability density p ( x ∗ , f ∗ ) over both the optimum x ∗ and the true, noiselessoptimal value f ∗ .
Fig.
1 visualizes the densities p ( x ∗ ) and p ( f ∗ ) , considered by ES / PES and MES ,respectively, and the joint density p ( x ∗ , f ∗ ) , considered by JES . As highlighted by the vertical dashedlines for the point selection of each strategy (bottom), PES chooses strictly to reduce the uncertaintyover x ∗ , and as such, considers a region where the uncertainty over the optimal value is low. However,it can effectively determine that the right side of the local optimum is more promising to querynext. MES seeks to reduce the tail of the probability density over f ∗ (right), which in this case leadsto an exploratory query.
JES ’ joint probability density over optimum and optimal value capturesuncertainties over both “where” and “how large” the optimum will be. As such, it selects a pointwhich is uncertain under both measures. For the selected query in Fig. 1, JES will learn substantiallyabout both x ∗ and f ∗ by querying it, whereas PES and MES learn only about one of them.
[Figure] [Figure]
3.2 The Joint Entropy Search acquisition function
[Figure] We consider the mutual information between the random variables ( x ∗ , f ∗ ) and a future query ( x , y ) :
[Equation]
Eq. 8 is similar to Eq.
but with the addition of x ∗ and the replacement of y ∗ with f ∗ in theconditioning of the second term.
The expectation is computed with respect to a D + 1 -dimensionaljoint probability density over x ∗ and f ∗ .
In Eq.
9, we make it explicit that the conditional densityinside the expectation is obtained after (1) conditioning the GP on the previous data D , plus oneadditional noiseless optimal pair ( x ∗ , f ∗ ) , and (2) knowing that the noiseless optimal value is infact f ∗ .
By utilizing the complete observation ( x ∗ , f ∗ ) , and of just x ∗ or f ∗ , we can treat it likeany other (noiseless) observation.
As such, we quantify much of the entropy reduction by utilizingstandard GP functionality.
For (2), we gain knowledge of the maximum of the noiseless objective,globally.
Following [26, 43], the resulting effect is to truncate the GP’s posterior over f , upperbounding it to f ∗ .
The expectation in Eq.
9 is approximated through MC by sampling L optimalpairs { ( x ∗ ℓ , f ∗ ℓ ) } Lℓ =1 from p ( x ∗ , f ∗ ) using an approximate version of Thompson Sampling ( TS )
[40].as explained in Sec. 3.3.
In Fig.
2, the resulting posterior distribution of the two-step conditioning isshown in greater detail.
As pointed out in [26, 39], after conditioning on f ∗ , the posterior predictivedensity over y is a sum of a truncated Gaussian distribution over f and the Gaussian noise ϵ .
As such,we can loosely relate the entropy reduction to two separate variance reduction steps over p ( y |D , x ) :a conditioning term and a truncation term.
The former primarily reduces the entropy over x ∗ , whilethe latter mainly reduces the entropy over f ∗ .
Moreover, the former is easily computed exactly, andthe latter, which is generally a smaller term, requires approximation, as shown in Sec. 3.4.
[Figure]
Fig.
3 shows the difference in log variance overp ( y |D , x ) resulting from conditioning (in blue)and truncation (in orange) for the scenario in Fig.
The overall reduction is largest close tothe point of conditioning, and the truncation termmainly contributes at uncertain regions far awayfrom the conditioned point.
Moreover, the magnitude of the conditioning term will rely on theprior variance at the conditioned point, as a largerprior variance will lead to a larger reduction inentropy from conditioning.
As we average overoptimal pairs, many such entropy reduction termsaccumulate.
[Figure]
3.3 Incorporating optimal pairs
To obtain samples ( x ∗ , f ∗ ) , we utilize an approximate variant of TS [40], originally proposed in
PES [15].
We utilize Bochner’s theorem [4], which, for any stationary kernel k , asserts the existenceof its Fourier dual s ( w ) .
By normalizing s ( w ) , we obtain the spectral density p ( w ) =
s ( w ) /α ,where α is a normalization constant.
We can then write the kernel as an expectation,
[Equation]
⊺ x ′ + b )] , (10) where b ∼ U ( 0 , 2 π I ) .
Following Rahimi and Recht [27], we sample b and w to obtain an unbiasedestimate of the kernel k .
From this approximation, approximate sample paths can be drawn as aweighted sum of basis functions.
This form allows for fast and dense querying of the sample paths– the arg max and max of which is an approximate draw from p ( x ∗ , f ∗ ) .
In PES , each sample x ∗ ℓalong with its inverted Hessian is required for computing the acquisition function.
To obtain the
Hessian, each sample needs to be thoroughly optimized through gradient-based optimization.
JES onthe other hand, only requires ( x ∗ , f ∗ ) .
As such, it can rely on cheap, approximate optimization ofthese samples, e.g., by densely querying sample points on a non-uniform grid.
After obtaining a set of optimal pairs { ( x ∗ ℓ , y ∗ ℓ ) } Lℓ =1 , JES computes the conditional entropy quantityover the output y .
Concretely, we generate L GPs, each modeling a posterior density { p ( y |D ∪( x ∗ ℓ , f ∗ ℓ ) , x ) } Lℓ =1 conditioned on an optimal pair and previously observed data D .
Since each optimalpair is drawn from the current GP hyperparameter set, we know that the current hyperparameter set isthe correct one even after adding the optimal pair to the data.
By consequence, JES can compute theupdated inverse Gram matrix, ( K + σ 2 ϵ ) − 1 , through a rank-1 update, instead of solving a linear systemof equations.
Utilizing the Sherman–Morrison formula [34], we obtain updated Gram matrices in
O ( n 2 ) for each sample, as opposed to O ( n 3 ) for solving the linear system of equations.
3.4 Approximating the truncated entropy
As highlighted in the right panel of Fig. 2, conditioning on f ∗ yields a truncated normal distributionp ( f |D , f ∗ ) , but the entropy is computed with regard to the density over noisy observations, y = f
+ ϵ ,which leads to an intractable entropy.
We approximate this quantity through moment matchingof the truncated Gaussian distribution over f .
Consequently, we obtain two Gaussian densitiesˆ p ( f |D , f ∗ ) ∼ N ( m f | f ∗ , σ 2 f | f ∗ ) and p ( ϵ ) ∼ N (0 , σ 2 ϵ ) , where m f | f ∗ and σ 2 f | f ∗ are the mean andvariance of the truncated Gaussian posterior p ( f |D , f ∗ ) .
Due to independence between f and ϵ andthe linearity of Gaussian distributions, we can then compute the entropy of the approximate densityˆ p y[-|D∪ ( x ∗ , f ∗ ) , x , f ∗ )] = log(
 π ( σ 2 ϵ + σ 2 f | f ∗ )) .-]
exactly as H [ˆ p ( y |D ∪ ( x ∗ , f ∗ ) , x {+, f ∗ )] = log(
+}
π ( σ 2 ϵ + σ 2 f | f ∗ )) .
The quality of this approximationis studied in greater detail in Appendix E.
3.5 Greedy selection to guard against model misspecification
As with all information-theoretic approaches, JES aims to reduce the uncertainty over the location ofthe optimum.
With this strategy, the incentive to query the perceived optimum is often lower than forheuristic approaches, such as EI .
In cases where the surrogate model is misspecified, informationtheoretic approaches risk reducing the entropy based on a faulty belief of the optimum, which candrastically impact their performance.
As a remedy, we utilize an inverse- γ -greedy approach: withprobability γ , JES will query the arg max of the posterior mean to confirm its belief of the locationof the optimum.
If the model is misspecified, these greedy steps enable the algorithm to reconsiderits beliefs, rather than continuing to act based on faulty ones.
In Appendix F, we show how thisapproach can substantially improve performance in cases of surrogate model misspecification, whilehaving negligible impact on performance in the worst case.
3.6 Putting it all together: The JES algorithm
For a sampled set of size L , containing optimal pairs { ( x ∗ ℓ , y ∗ ℓ ) } Lℓ =1 and GPs with mean and covariance functions { m ℓn ( x ) , s ℓn ( x ) } Lℓ =1 , the expression for the JES acquisition function is
[Equation] [Equation]
(12) where σ 2 f | f ∗ ( x ; D , x ∗ ℓ , f ∗ ℓ ) =
σ 2 T ( f ∗ ; m ℓn ( x ) , s ℓn ( x )) and σ 2 T ( α ; µ, σ 2 ) is the variance of an uppertruncated Gaussian distribution with parameters ( µ, σ 2 ) , truncated at α .
The first term in 12 is simplythe entropy of a Gaussian that can be computed in closed form.
The second term contains both theconditioning term, which is exact, and the truncation, which is approximated as described in Sec. 3.4.
In Algorithm 1, we outline the pseudocode for JES in its entirety. Algorithm
Experimental evaluation
Benchmarks.
We now evaluate JES on a suite of diverse tasks.
We consider three different typesof benchmarks: samples drawn from a GP prior, commonly used synthetic test functions [15], and a collection of classification tasks on tabular data using an MLP, provided through HPOBench [9].
For the GP prior tasks, the hyperparameters are known for all methods to evaluate the effect of theacquisition function in isolation.
Consequently, we do not use the inverse greedy approach from
[Figure]
[Figure] Section 3.5 in this case (i [Figure] e., we set γ = 0 in Algorithm 1).
For the synthetic and MLP tasks, wemarginalize over the GP hyperparameters
[Figure]
and set γ = 0 . 1 .
The hyperparameters of the GP priorexperiments can be found in Appendix B, and ablation studies on γ in Appendix C.
Evaluation criteria.
We use two types of evaluation criteria as in [43]: simple regret and inferenceregret .
The simple regret r n = max x ∈X f ( x ) − max t ∈ [1 ,n ] f ( x t ) measures the value of the bestqueried point so far.
After a query, we may infer an arg max of the function, which is chosen as x ∗ n =arg max x ∈X m n ( x ) [14, 15, 43].
We denote the inference regret as r n = max x ∈X f ( x ) − f ( x ∗ n ) .
Since information-theoretic approaches do not necessarily seek to query the optimum, but only toknow its location, inference regret characterizes how satisfying our belief of the arg max is.
Notably,this metric is non-monotonic, meaning that the best guess can worsen with time.
We use this metric inthe ideal model benchmarking setting, when we sample tasks from a GP with known hyperparameters.
We use simple regret for the synthetic test functions, as it constitutes a metric that is more robust tosurrogate model misspecification.
Inference regret for these tasks can be found in Appendix G. Forthe HPOBench tasks, inference regret is unobtainable.
The experimental setup.
We compare against other state-of-the-art information-theoretic approaches: PES [15] and MES [43], as well as EI [20].
The acquisition functions are all run in the sameframework written in MATLAB, created for the original PES implementation by Hernández-Lobatoet al.
All synthetic experiments were run for 50 D iterations.
In the main paper, we fix thenumber of MC samples for MES, PES and JES to 100 each.
In Appendix D we assess the sensitivityof JES to this number and quantify the computational expense.
In Appendix B, we provide all detailson our experimental setup.
4.1 GP prior samples
We consider samples from a GP prior for four different dimensionalities: 2D, 4D, 6D, and 12D, witha noise standard deviation of 0 .
1 for a range of outputs spanning roughly [ − 10 , 10] .
These tasksconstitute an optimal setting for each algorithm, as the surrogate perfectly models the task at hand.
JES demonstrates empirically the value of the additional source of information, finishing at least half
[Figure]
Hartmann
[Figure]
Fig.
4 compares JES (top left) against PES , MES and EI in terms of point selection for one repetition ona two-dimensional sample task, where all runs are initialized with D +1 identical random samples.
Weobserve that JES succeeds in finding all attractive regions of the search space, and queries the regionaround the optimum densely, which is sensible in a noisy setting.
We further notice that EI (bottomright) fails to query the two circled local optima.
PES (bottom left) also ignores two local optima to various degrees, and tends to circle the (perceived) optimum densely, which is expensive in terms of number of evaluations.
We believe this showcases a shortcoming of only considering the density over theoptimum: PES circles the optimum, but does not query its value.
Lastly, MES (top right) successfullyqueries all attractive regions of the space, but also samples regions that are evidently poor the mostdensely out of the four approaches, despite information given by earlier (brighter) samples.
Since JESconsiders the information conveyed by both MES and PES , it successfully excludes the apparent suboptimal regions of the space, finds all relevant optima, and queries these optima in a desirable manner.
[Figure]
We additionally evaluate the performance of allapproaches on GP sample tasks that have a substantial amount of noise - its standard deviationroughly accounting for 10% of the total outputrange.
We run these tasks for a larger numberof iterations, 125D, to display the stagnationof some approaches.
While MES and PES slowdown approximately at the halfway point forboth tasks, JES steadily improves for the entirelength of the run.
This robustness to large noisemagnitudes highlights the importance of intrinsically handling noisy objectives in JES .
4.2 Synthetic test functions
Next, in Fig.
7, we study the performance of JES on three optimization test functions: Branin (2D),
Hartmann (3D) and Hartmann (6D).
For these tasks, we follow convention [15, 29] and marginalizeover GP hyperparameters.
On Branin, JES starts out slightly slower than MES but reaches the sameperformance in 100 iterations; and on the two Hartmann functions, JES performs amongst the best inthe beginning and clearly best in the end.
We note that PES experienced numerical issues on Branin,and as such, we acknowledge that its performance should be better than what is reported.
4.3 MLP tasks
Lastly, we evaluate the performance of JES on the tuning an MLP model’s 4 hyperparameters
for20 D iterations on three datasets.
These tasks are part of the OpenML 1 library of tasks, and the
HPO benchmark is provided through the HPOBench [9] suite.
The tasks, which are classificationproblems with tabular data, regard classifying: (1) vehicles into one of four different categories, (2)blood donors for a blood transfusion service center, and (3) anonymous credit card applications.
[Figure]
i f i c a t i o n
[Figure]
cc u r a c y
Vehicle Silhouette
[Figure]
We measure the best observed classification accuracy.
Notably, these tasks have a large amountof noise, which causes the performance to fluctuate substantially between repetitions.
We observethat
JES performs almost identically to MES and EI on vehicle silhouette and credit application, andsubstantially better on blood transfusion.
PES lags behind significantly on all tasks.
Conclusions
We have presented Joint Entropy Search, an information-theoretic acquisition function that considersan entirely new quantity, namely the joint density over the optimum and optimal value.
By utilizingthe entropy reduction from fantasized optimal observations, JES obtains a simple form for the entropyreduction regarding the joint distribution.
As such, the additional information considered comeswith minimal computational overhead, avoids restrictive assumptions on the objective, and yieldsstate-of-the-art performance along with superior decision-making.
We believe JES to be a new go-toacquisition function for BO, and to establish a new standard for subsequent information-theoretictechniques.
Limitations and Future Work
The main contribution of this paper is to provide a novel information-theoretic acquisition functionwhich, given a sufficiently accurate model, yields impressive results.
However, the non-myopic,speculative nature of information-theoretic approaches lend them to be susceptible to model misspecification, such as a poor choice of GP kernel or GP hyperparameters.
In our view, information-theoreticapproaches are possibly more susceptible to this issue than their myopic counterparts ( EI , UCB , TS ).
While we propose a remedy to stabilize and improve the acquisition function under model misspecification with the inverse γ -greedy approach, this technique only serves to discover misspecificationand adjust accordingly, but not to inherently fix the misspecification.
We believe misspecification canonly be remedied by altering the surrogate model.
It is thus very promising to combine advancedmodelling techniques with information-theoretic acquisition functions, as already done with theadditive GP approach utilized in conjunction with MES by Wang and Jegelka [43]; further promisingadditions would be to tackle heterogeneous noise and input warping as done by HEBO [8].
We also note that, since JES computes the entropy reduction from conditioning on the optimal pair,it relies on some level of noise in the objective.
A surrogate model with zero noise will result inan infinite information gain for every optimal pair, which (by utilizing some random tie-breakingstrategy) would make JES equivalent to TS.
However, if JES is to be used in a completely noiselesssetting, we argue that a small noise term should be added as a remedy.
As this is done by defaultin many prominent GP frameworks [12, 13, 41], we do not view this as a major limitation of ourapproach.
Nevertheless, improving upon this strategy would be interesting in future work.
For future work, we also envision work on the adaptation of JES to various different domains, suchas multi-fidelity [44] and multi-objective optimization [1], as well as the integration of user priorknowledge over the location of the optimum [19] to accelerate optimization.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] Our abstract and introduction accurately reflect ourpaper.(b) Did you describe the limitations of your work?
[Yes] See Section (c) Did you discuss any potential negative societal impacts of your work? [Yes] SeeSection A
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes] We discuss the ethics guidelines in Section A
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
As our methodis proposed as an heuristic, we do not include theoretical results.
(b) Did you include complete proofs of all theoretical results?
[N/A]
As our method isproposed as an heuristic, we do not include theoretical results.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] The URL to therepository where all experiments were run is included in Section
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] All experimental details are reported in Appendix B
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] All results have error bars include.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] See Section A
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] All the repositories
that we have used or built on have been cited.
(b) Did you mention the license of the assets? [Yes] The licenses of the assets are mentioned
in Appendix B
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
e include new surrogate benchmarks and frameworks in the supplementary material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] We did not use or release any datasets with personal data.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
The data we are using does not contain personal
information or offensive content.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
We did not run experiments with human subjects.
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
We did not run experiments with humansubjects.
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
We did not run experiments with humansubjects.