Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization
Abstract
Optimizing expensive-to-evaluate black-box functions of discrete (and potentiallycontinuous) design parameters is a ubiquitous problem in scientiﬁc and engineeringapplications.
Bayesian optimization (BO) is a popular sample-efﬁcient method thatselects promising designs to evaluate by optimizing an acquisition function (AF)over some domain based on a probabilistic surrogate model.
However, maximizing the AF over mixed or high-cardinality discrete search spaces is challengingas we cannot use standard gradient-based methods or evaluate the AF at everypoint in the search space.
To address this issue, we propose using probabilisticreparameterization (PR).
Instead of directly optimizing the AF over the searchspace containing discrete parameters, we instead maximize the expectation of the
AF over a probability distribution deﬁned by continuous parameters.
We prove thatunder suitable reparameterizations, the BO policy that maximizes the probabilisticobjective is the same as that which maximizes the AF, and therefore, PR enjoys thesame regret bounds as the underlying AF.
Moreover, our approach admits provablyconverges to a stationary point of the probabilistic objective under gradient ascentusing scalable, unbiased estimators of both the probabilistic objective and its gradient, and therefore, as the numbers of starting points and gradient steps increaseour approach will recover of a maximizer of the AF (an often neglected requisitefor commonly-used BO regret bounds).
We validate our approach empiricallyand demonstrate state-of-the-art optimization performance on many real-worldapplications.
PR is complementary to (and beneﬁts) recent work and naturallygeneralizes to settings with multiple objectives and black-box constraints.
Introduction
Many scientiﬁc and engineering problems involve tuning discrete and/or continuous parameters tooptimize an objective function.
Often, the objective function is “black-box”, meaning it has no knownclosed-form expression.
For example, optimizing the design of an electrospun oil sorbent—a materialthat can be used to absorb oil in the case of a marine oil spill to mitigate ecological harm—caninvolve tuning both discrete ordinal experimental conditions and continuous parameters controllingthe composition of the material to maximize the oil absorption capacity, mechanical strength, andwater contact angle [57].
For another example, optimizing the structural design of a welded beamcan involve tuning categorical parameters such as the type of metal, binary parameters such as thewelding type, and discrete ordinal parameters such as the dimensions of the different components ofthe beam–resulting in a search space with over 370 million possible designs [53].
We consider thescenario where querying the objective function is expensive, in which case sample-efﬁciency is crucial.
In the case of designing the oil sorbent, evaluating the objective function requires manufacturing thematerial and measuring its properties in a laboratory, requiring signiﬁcant time and resources.
Bayesian optimization (BO) is a popular technique for sample-efﬁcient black-box optimization, due toits proven performance guarantees in many settings [5, 51] and its strong empirical performance [23].
BO leverages a probabilistic surrogate model of the unknown objective(s) and an acquisition function(AF) that provides utility values for evaluating a new design to balance exploration and exploitation.
Typically, the maximizer of the AF is selected as the next design to evaluate.
However, maximizingthe AF over mixed search spaces (i.e., those consisting of discrete and continuous parameters) orlarge discrete search spaces is challenging 1 because continuous (or gradient-based) optimizationroutines cannot be directly applied.
Theoretical performance guarantees of BO policies require thatthe maximizer of the acquisition function is found and selected as the next design to evaluate onthe black-box objective function [51].
When the maximizer is not found, regret properties are notguaranteed, and the performance of the BO policy may degrade.
To tackle these challenges, we propose a technique for improving AF optimization using a probabilistic reparameterization (PR) of the discrete parameters.
Our main contributions are:
We use PR to optimize a probabilistic objective (PO): the expectation of the AF over aninduced distribution over the discrete parameters.
We prove that there is an equivalence between the maximizers of the acquisition function
and the the maximizers of the PO and hence, the policy that chooses designs that are best
with respect to the PO enjoys the same performance guarantees as the standard BO policy.
We derive scalable, unbiased Monte Carlo estimators of the PO and its gradient with respect
to the parameters of the induced distribution.
We show that stochastic gradient ascent using
our gradient estimator is guaranteed to converge to a stationary point on the PO surface
and will recover a global maximum of the underlying AF as the number of starting points
and gradient steps increase.
This is important because many BO regret bounds require
maximizing the AF [51].
Although the AF is often non-convex and maximization is hard,
empirically, with a modest number of starting points, PR leads to better AF optimization
than alternative methods.4.
We show that PR yields state-of-the-art optimization performance on a wide variety ofreal-world design problems with discrete and mixed search spaces.
Importantly, PR is complementary to many existing approaches such as popular multi-objective, constrained, andtrust region-based approaches; in particular, PR is agnostic to the underlying probabilisticmodel over discrete parameters—which is not the case for many alternative methods.
Preliminaries
Bayesian Optimization We consider the problem of optimizing a black-box function f : X ×Z → 69 R over a compact search space X
× Z , where X = X (1) × · · · × X ( d ) is the domain of the d ≥ 0continuous parameters ( x ( i )
∈ X ( i ) for i
= 1 , ..., d ) and Z = Z (1) × · · · × Z ( d z ) is the domain ofthe d z ≥ 1 discrete parameters ( z ( i
) ∈
Z ( i ) for i
= 1 , ..., d z ).
When the black-box function is expensive-to-evaluate, Bayesian optimization (BO) is popular owingto its sample efﬁciency.
BO leverages (i) a probabilistic surrogate model—typically a Gaussianprocess (GP) [45]—ﬁt to a data set D n = { x i , z i , y i } ni =1 of designs and corresponding (potentiallynoisy) observations
y i
=
f
( x i , z i ) + (cid:15) i , (cid:15) i ∼ N (0 , σ 2 ) , and (ii) an acquisition function α ( x , z ) thatuses the surrogate model’s posterior distribution to quantify the value of evaluating a new design.
Common AFs include expected improvement (EI) [32] and upper conﬁdence bound (UCB) [51]—thelatter of which enjoys no-regret guarantees in certain settings.
The next design to evaluate is chosen bymaximizing the AF α ( x , z ) over X × Z .
Although the black-box objective f is expensive-to-evaluate,the AF is relatively cheap-to-query, and therefore, can be optimized numerically (gradient-basedoptimization routines are often used to maximize the AF over continuous domains).
Throughout this paper, we use a mixed search space
X ×
Z in our derivations, theorems, and proofs, without loss of generality with respect to the case of a purely discrete search space.
If d = 0 , then the objective function f : Z → R is deﬁned over the discrete space Z and the continuous parameters in this exposition can simply be ignored.
Discrete Parameters Typically, BO assumes that the inputs are continuous.
However, discreteparameters such as binary parameters, discrete ordinal parameters, and non-ordered categoricalparameters, which are summarized in Table 1, are ubiquitous in many applications.
In the presence ofsuch parameters, optimizing the AF is more difﬁcult, as standard gradient-based approaches cannotbe directly applied.
Recent works have proposed various approaches including multi-armed bandits[39, 47] and local search [40] for discrete domains and interleaved discrete/continuous optimizationprocedures for mixed domains [15, 55].
A simple and widely-used approach across many popular
BO packages [1, 52] is to one-hot encode the categorical parameters, apply a continuous relaxationwhen solving the optimization, and discretize (round) the resulting continuous candidates.
Examplesof continuous relaxations and discretization functions are listed in Table 1.
[Table] [Table]
Although using a continuous relaxation allows for efﬁcient optimization using standard optimizationroutines in an alternate continuous domain Z (cid:48)
⊂ R m , the AF value for an infeasible continuousvalue (i.e., z (cid:48) / ∈ Z ) does not account for the discretization that must occur before the black-boxfunction is evaluated.
Moreover, the acquisition value for an infeasible continuous value can belarger than the AF value after discretization.
For an illustration of this, see Fig.
1 (middle/right).
In the worst case, BO will repeatedly select the same infeasible continuous design due to its high
AF value, but discretization will result in a design that has already been evaluated and has zero AFvalue.
To mitigate this degenerate behavior and avoid the over-estimation issue, Garrido-Merchánand Hernández-Lobato [25] propose discretizing z (cid:48) before evaluating the AF, but the AF is thennon-differentiable with respect to the z (cid:48) .
While this improves performance on small search spaces, the
AF and surrogate have large ﬂat regions after discretizing z (cid:48) , which makes it difﬁcult to optimize the
AF.
The authors of [25] propose to approximate the gradients using ﬁnite differences, but, empirically,we ﬁnd that this approach to be sub-optimal for optimizing the AF.
Probabilistic Reparameterization
We propose an alternative approach based on probabilistic reparameterization, a relaxation of theoriginal optimization problem involving discrete parameters.
Rather than directly optimizing the AFvia a continuous relaxation z (cid:48) of the design z , we instead reparameterize the optimization problemby introducing a discrete probability distribution p ( Z | θ ) over a random variable Z with supportexclusively over Z .
This distribution is parameterized by a vector of continuous parameters θ .
[Table]
[Equation]
ATEGORICAL
[Equation]
We use z to denote the vector ( z (1) , ..., z ( d z ) ) , where each element is a different (possibly vectorvalued) discrete parameter.
Given this reparameterization, the probabilistic objective (PO) is deﬁnedas:
[Equation]
PR enables optimizing θ and x over a continuous space to maximize the PO instead of optimizingx , z to maximize α directly over the mixed search space
X × Z .
As we will show later, maximizingthe PO allows us to recover a maximizer of α over the space X × Z .
Choosing p ( Z | θ ) to be adiscrete distribution over Z means the realizations of Z are feasible values in Z .
Hence, the AF isonly evaluated for feasible discrete designs.
Since p ( Z | θ ) is a discrete probability distribution, we can express E Z ∼ p
( Z | θ )
[ α ( x , Z )] as a linear combination where each discrete design is weighted byits probability mass:
[Figure]
[Equation]
Example distributions for binary, ordinal, and categorical parameters are provided in Table 2.
Although ordinal parameters could use the same categorical distributions as the non-ordered categoricalparameters, we opt for the provided proposal distribution since it uses a scalar θ , rather than a
C -element vector.
Using an independent random variable Z ( i ) ∼ p ( Z ( i ) | θ ( i ) ) for each parameter z ( i )for i = 1 , ..., d z means that the probabilistic objective can be expressed as
[Equation]
[Figure]
[Equation]
[Figure]
3.1 Analytic Gradients
One important beneﬁt of PR is that the PO in (1) is differentiable with respect to θ (and x , if thegradient of α with respect to x exists), whereas α ( x , z ) is not differentiable with respect to z .
Thegradients of the PO with respect to θ and x can be obtained by differentiating Equation 2:
[Equation] [Equation]
This enables optimizing the PO (line 4 of Algorithm 1) efﬁciently and effectively using gradient-basedmethods over discrete or mixed search spaces.
3.2 Theoretical Properties
In this section, we derive theoretical properties of PR
Theorem 1 (Consistent Maximizers) .
Suppose that α is continuous in x for every z ∈ Z .
Let H ∗be the maximizers of α ( x , z ) : H ∗ = { ( x , z ) ∈ arg max ( x , z )
∈X×Z α ( x , z ) } .
Let J ∗ ⊆
X × Θ bethe maximizers of E Z ∼ p
( Z | θ )
[ α ( x , Z )] : J ∗ = { ( x , θ ) ∈ arg max ( x , θ ) ∈X×
Θ E Z ∼ p
( Z | θ )
[ α ( x , Z )] } ,where Θ is the domain of θ .
Let ˆ H ∗ ⊆
X × Z be deﬁned as: ˆ H ∗ =
{ ( x , ˜ z ) : ( x , θ ) ∈ J ∗ , ˜ z ∼p ( Z | θ ) } .
Then, ˆ H ∗ =
H ∗ .
Algorithm 1 outlines BO with probabilistic reparameterization.
Importantly, Theorem 1 states thatsampling from the distribution parameterized by a maximizer of the PO yields a maximizer of α , andtherefore, Algorithm 1 enjoys the performance guarantees of α ( · ) .
Corollary 1 (Regret Bounds) .
Let α ( x , z ) be an acquisition function with bounded regret over asearch space X × Z .
If the conditions for the regret bounds of α ( x , z ) are satisﬁed, then Algorithm 1with
α enjoys the same regret bound.
1: Input: black-box objective f : X × Z → R 2: Initialize D 0 ← ∅ , GP 0 ← GP ( 0 , k ) 3: for n = 1 to N iterations do 4: ( x n , θ n ) ← arg max ( x , θ ) ∈X×
Θ E Z ∼ p
( Z | θ )
[ α ( x , Z )] 5:
Sample z n ∼ p
( Z | θ n ) 6:
Evaluate f ( x n , z n ) 7:
D n ← D n − 1 ∪ { ( x n , z n , f ( x n , z n )) } 8:
Update posterior GP n given D n 9: end for
Examples of such AFs are upperconﬁdence bound (UCB) [51] or
Thompson sampling (TS) [48] forsingle objective optimization, and
UCB or TS with Chebyshev [42] orhypervolume [64] scalarizations inthe multi-objective setting.
Although the BO policy selects amaximizer of α is equivalent to the
BO policy in Algorithm 1, maximizing the AF over mixed or highdimensional discrete search spacesis challenging because commonlyused gradient-based methods cannotdirectly be applied.
The key advantage of our approach is that maximizers of the AF can be identiﬁedefﬁciently and effectively by optimizing the PO using gradient information instead of directly optimizing the AF.
We ﬁnd that optimizing PR yields signiﬁcantly better results than directly optimizingα or other common relaxations as shown in Figure 1(Left), where we compare AF optimizationmethods on the chemical synthesis problem [50] (see Appendix C for details).
The violin plots showthe distribution of ﬁnal AF values and the mean.
“Cont.
Relax.”
denotes optimizing a continuousrelaxation of the categoricals with exact gradients.
“Exact Round” refers to optimizing a continuousrelaxation with approximate gradients (via ﬁnite difference), but discretizes the relaxation beforeevaluating the surrogate [25].
"Interleaved" alternates between one step of local search on the discreteparameters and one step of gradient ascent on the continuous parameters (used in C ASMOPOLITAN[55]).
All methods except interleaved use L-BFGS-B and the expected improvement AF [32].
Practical Monte Carlo Estimators
4.1 Unbiased estimators of the Probabilistic Reparameterization and its Gradient
As the number of discrete conﬁgurations ( |Z| ) increases, the PO and its gradient may becomecomputationally expensive to evaluate analytically because both require a summation of |Z| terms.
Therefore, we propose to estimate the PO and its gradient using Monte Carlo (MC) sampling.
The
MC estimator of the PO is given by
[Equation]
where ˜ z 1 , ...,
˜ z N are samples from p ( Z | θ ) .
This estimator is unbiased and can be computed for alarge number of samples by evaluating the AF independently (or in chunks) for each input ( x , ˜ z n ) .
MC can also be used to estimate the gradient of the PO with respect to θ .
We opt for using ascore function gradient estimator [34] (also known as REINFORCE [60]) and the likelihood ratioestimator [26]) because it is simple, scalable, and can be computed simply using the acquisition values{ α ( x ,
˜ z i ) } Ni =1 that are already required for the MC estimator of the PO.
Many alternative lowervariance estimators (e.g. Yin et al.
[62], Yin et al.
[63]) would require many additional AF evaluations(see Mohamed et al.
[38] for a review of MC gradient estimation).
The score function is the gradient of the log probability with respect to the parameters of the distribution: ∇ θ log p ( Z | θ ) = ∇ θ p ( Z | θ ) p
( Z | θ ) .
Using this score function, we can express the analytic gradient as
[Equation]
The unbiased MC estimator of the gradient of the PO with respect to θ is given by
[Equation]
Since the score function gradient is only deﬁned when p ( z | θ ) > 0 , we reparameterize θ to ensure p ( z | θ ) > 0 for all z and θ by using the softmax transformations provided in Table 3, which are commonly used for computational convenience and stabilityin probablistic reparameterization [62, 63].
Although θ can only be 0 or 1 when τ →0 , for any (cid:15) > 0 , there exists some τ such that the approximation error is less than (cid:15) .
P ARAMETER T YPE
[Equation]
RANSFORMATION
[Equation]
INARY
[Equation]
RDINAL
[Equation]
C ATEGORICAL
[Equation]
Moreover, even though p ( z | θ ) > 0 ,when p ( z | θ ) is small, a small number N of MC samples are unlikely toproduce any samples where ˜ z
= z .
Instead of optimizing θ directly, weinstead optimize φ .
Since the transformations g ( · ) are differentiable withrespect to φ , the gradient (and MCgradient estimator) of the PO with respect to φ are easily obtained using the gradient of the PO withrespect to θ and a simple application the chain rule (multiplying by ∇ φ θ ).
4.2 Convergence Guarantee using Stochastic Gradient Ascent
Since the score function gradient estimator is unbiased, we can leverage previous work on convergencein probability under stochastic gradient ascent [46] to arrive at our main convergence result foracquisition optimization.
Theorem 2 (Convergence Guarantee) .
Let α : X × Z
→ R be differentiable in x for every z ∈ Z .
Let (ˆ x t,m , ˆ θ t,m ) be the best solution after running stochastic gradient ascent for t time steps on theprobabilistic objective
E
Z ∼ p
( Z | θ )
[ α ( x , Z )] from m starting points with its unbiased MC estimatorsproposed above.
Let { a t } ∞ t =1 be a sequence of positive step sizes such that 0 < (cid:80) ∞
t
a 2 t =
A < ∞ ,where a t is the step size used in stochastic gradient ascent at time step t .
Let ˆ z t,m ∼ p
( Z | ˆ θ t,m ) .
Thenas t → ∞ and m → ∞ , (ˆ x t,m , ˆ z t,m )
→ ( x ∗ , z ∗ ) ∈ arg max ( x , z ) ∈X×Z α ( x , z ) in probability.
The signiﬁcance of Theorem 2 is that optimizing the PO is guaranteed to converge in probability toa global maximizer of the AF, meaning that optimizing the PO guarantees that resulting candidatedesign has maximal AF value.
The implication is that the intended BO policy is followed andthe underlying regret bounds of the AF are recovered (provided that the other conditions of theregret bound are met).
Although global convergence is only guaranteed as m → ∞ , we observe in
Figure 1(left) that PR yields stable, convergent acquisition optimization with only m = 20 startingpoints and outperforms alternative optimization approaches.
4.3 Deterministic Optimization via Sample Average Approximation
Although multi-start stochastic ascent is provably convergent, we opt instead to use lower variancedeterministic MC estimators using common random numbers.
In order to maintain valid samples˜ z i ∼ p
( Z | θ ) for i = 1 , ..., N as θ changes over the course of the acquisition optimization, wereparameterize Z as a deterministic function h ( · , · ) that operates component-wise on θ and therandom variable U =
( u (1) , ..., u ( d z ) ) , u ( i ) ∼ Uniform (0 , 1) :
Z = h ( θ , U ) .
Descriptions of h ( · , · )for provided for each parameter type in Appendix C.4.
Using a ﬁxed a set of base samples { ˜ u i } Ni =1 ,the sample average approximation estimators are obtained by substituting z i
= h ( θ , ˜ u
i ) into (6) and(7).
Although biased, sample average approximation estimators are deterministic, enabling the use ofsecond-order numerical methods for efﬁcient acquisition optimization [1].
We reduce the variancefurther by leveraging quasi-MC sampling [41] instead of i.i.d.
sampling.
Related Work
Many methods for BO over discrete and mixed search spaces have been proposed.
Previous workhas largely focused on improving BO by either (i) improving the surrogate models or (ii) improvingtechniques for optimizing AFs.
Improving models : Historically, methods leveraging tree-based surrogate models, e.g., SMAC[30] and TPE [4], have been popular for optimizing discrete or mixed search spaces.
BOCSencodes categorical parameters as binary variables and uses Bayesian linear regression with pairwiseinteractions [2].
COMBO uses a diffusion kernel on the graph built from a graph Cartesian productof discrete parameters [40].
M ER CBO similarly exploits the combinatorial graph, but with Mercerfeatures and Thompson sampling [16].
H Y BO extends the diffusion kernels to mixed continuousdiscrete spaces [15].
However, these methods scale poorly with respect to the number of data pointsand parameters.
Moreover, of the methods listed above, only H Y BO supports continuous parameterswithout discretization.
H Y BO enjoys a universal approximation property, but relies on summingover all possible orders of interactions between base kernels for each parameter which results inexponential complexity with respect to the number of parameters and limits its applicability tolow-dimensional problems.
G RYFFIN [28] uses kernel density estimation, but is limited to categoricalsearch spaces.
M I V A BO uses a linear combination of basis functions (e.g. pseudo Boolean features[7] for discrete parameters) with interaction terms[13].
MVRSM [6] uses ReLU-based surrogatesfor computational efﬁciency, but is limited by the expressiveness of these models.
Optimizing acquisition functions : As discussed previously, Garrido-Merchán and HernándezLobato [25] propose using continuous relaxation and discretize the inputs before evaluating the AF.
However, the resulting AF after discretization is piece-wise-ﬂat along slices of the discrete parametersand is difﬁcult to optimize.
C O C
A BO [47] sample discrete parameters using a multi-armed banditand optimize the continuous parameters conditioned on the sampled discrete parameters.
However,
C O C A
BO’s performance degrades as number of discrete conﬁgurations increases.
C ASMOPOLITAN[55] uses local trust regions combined with an interleaved AF optimization strategy that alternatesbetween local search steps on the discrete parameters and gradient ascent for the continuous parameters.
Furthermore, both C O C A BO and C ASMOPOLITAN do not inherently exploit ordinal structure.
Moreover, the computational issues of many existing approaches make it difﬁcult to apply them tomulti-objective and constrained optimization.
Probabilistic reparameterization : PR has been considered for optimizing discrete parameters inother domains such as reinforcement learning [60] and sparse regression [63].
However, PR has notbeen leveraged for BO (although a different reparameterization is commonly used in MC AFs [61]).
Alternative methods for propagating gradients : Alternative methods for propagating gradientsthrough discrete structures have been considered in the deep learning community (among others).
One approach is to use approximate discrete Concrete distributions [31, 36], which admit samplepath gradients.
However, samples from Concrete distributions are not discrete and signiﬁcantapproximation error can result in pathologies similar to evaluating the AF using continuous relaxation.
Moreover, approximately discrete samples prohibit using surrogate models that require discrete inputs,e.g., GPs with Hamming distance kernels [47].
Another approach for gradient propagation in thedeep learning community is to use straight-through gradient estimators (STE) [3], where the gradientof the discretization function with respect to its input is estimated using an identity function.
Thisapproach works well empirically, these estimators are not well-grounded theoretically.
Nevertheless,we discuss and evaluate using STE for AF optimization in Appendix G.
Experiments
In this section, we provide an empirical evaluation of PR using SAA estimators on a suite of syntheticproblems and real world applications.
We use N = 1 , 024 MC samples in our experiments anddemonstrate that PR is robust with respect to the number of MC samples (and compare againstanalytic PR, where computationally feasible) in Appendix
F.
We compare PR against two alternativeacquisition optimization strategies: using a continuous relaxation (C ONT .
R ELAX .)
and using exactdiscretization and approximate gradients (E XACT R OUND )
In addition, we compare againsttwo state-of-the-art methods for discrete/mixed BO: a modiﬁed version of C ASMOPOLITAN [55] thatadditionally supports ordinal variables introduced in Wan et al.
[56] and H Y BO [15], both of whichare shown to outperform the other related works discussed in Section 5.
In addition, we showcasehow PR is complementary to existing methods such as trust region methods [22].
We demonstrate this by using PR with a trust region for the continuous and discrete ordinal parameters and optimize
PR within this trust region.
In Appendix G, we provide comparison of TR methods with alterativeoptimizers and ﬁnd that PR is the best optimizer for TR on 6 of the 7 benchmark problems.
See
Appendix C for additional discussion of PR + TR.
For PR, E XACT R OUND , and PR + TR we use thesum of a product kernel and a sum kernel of a categorical kernel [47] for the categorical parametersand Matérn-5/2 kernel for all other parameters.
C ONT .
R ELAX .
, E XACT R OUND , PR, and PR + TR use expected improvement [24, 32] for singleobjective (constrained problems) and expected hypervolume improvement [20] for the multi-objectiveoil sorbent problem (where exact gradients with respect to continuous parameters are computedusing auto-differentiation [8]).
We report the mean for each method ± 2 standard errors across 20replications.
Performance is evaluated in terms of regret (feasible regret for constrained problemsand hypervolume regret for multi-objective problems).
C ASMOPOLITAN and H Y BO are not run on
Welded Beam and Oil Sorbent as they do not support constrained and multi-objective optimization.
We also leave the multi-objective extension of PR+TR to future work because it would add additionalcomplexity [11].
For H Y BO, we only run 60 BO iterations on SVM due to the large wall time (see
Figure 4) and only report partial results on Cellular Network due to a singular covariance matrixerror.
See Appendix C for details on the experiment setup, regret metrics, benchmark problems, andmethodological details.
We leverage existing open source implementations of C ASMOPOLITAN and
H Y BO (see Appendix C for links), and we will open source our implementations of all of othermethods upon publication.
6.1 Synthetic Problems
We evaluate all methods on 3 synthetic problems: Ackley is a 13-dimensional function with 10binary and 3 continuous parameters (a modiﬁed version of the problem in Bliek et al.
Mixed
Int F1 is a 16-dimensional variant of the F1 function from Tušar et al.
[54] with 2 binary, 6 discreteordinal parameters, and 8 continuous parameters.
The discrete ordinal parameters have followingcardinalities: 2 parameters with 3 values, 2 with 5 values, and 2 with 7 values.
Rosenbrock is a10-dimensional Rosenbrock function with 6 discrete ordinal parameters with 4 values each and 4continuous parameters.
6.2 Real World Problems
We consider 5 real world applications including a problem with 5 black-box outcome constraints anda 3-objective problem (see Appendix D for details on constrained and multi-objective BO).
Welded Beam Optimizing the design of a welded steel beam is a classical engineering optimization.
In this problem, the goal is to minimize manufacturing cost subject to 5 black-box constraints onstructural properties of the beam (including shear stress, bending stress, and buckling load) my tuning6 parameters: the welding conﬁguration (binary), the metal material type (categorical with 4 options),and 4 ordinal parameters controlling the dimensions of the beam [53].
SVM Feature Selection
This problem involves jointly performing feature selection and hyperparameter optimization for a Support Vector Machine (SVM) trained on the CTSlice UCI data set [18, 35].
This design space for this problem involves 50 binary parameters controlling whether a particularfeature is included or not, and 3 continuous hyperparameters of the SVM.
Cellular Network Optimization
In this 30-dimensional problem, the goal is to tune the tilt (ordinalwith 6 values) and transmission power (continuous) for a set of 15 antennas [49] to maximize acoverage quality metric that is a function of signal power and interference [37] over a geographicregion of interest.
We use the simulator from Dreifuerst et al.
Direct Arylation Chemical Synthesis Palladium-catalysed direct arylation has generating signiﬁcantinterest in pharamceutical development [12].
In this problem, the goal is maximize yield for a directarylation chemical reaction by tuning the 3 categorical parameters corresponding to the choiceof solvent, base, and ligand, as well 2 continuous parameters controlling the temperature andconcentration.
We ﬁt a surrogate model to the direct arylation dataset from Shields et al.
[50] in orderto facilitate continuous optimization of temperature and concentration.
Electrospun Oil Sorbent Marine oil spills can cause ecological catastrophe.
One avenue for mitigating environmental harm is to design and deploy absorbent materials to capture the spilled oil.
In this problem, we tune 5 ordinal parameters (3 parameters with 5 values and 2 with 4 values) and 2continuous parameters controlling the composition and manufacturing conditions for an electrospunoil sorbent material to maximize 3 competing objectives: the oil absorbing capacity, the mechanicalstrength, and the water contact angle [57].
[Figure]
6.3 Results
We ﬁnd PR consistently delivers strong empirical performance as shown in Figure 2.
On all benchmark problems, PR (or PR + TR ) outperforms all baseline methods (except for Mixed Int F1, where
H Y BO performs comparably) .
Figure 4 in Section E shows the wall time for candidate generationover the number of BO iterations.
Although PR is computationally intensive, the computation isembarrassingly parallel and therefore exploiting GPU acceleration yields competitive wall times.
Importantly, PR’s wall time scales well with the number of observations and design parameters,unlike H Y BO which scales poorly with both.
However, the complexity of PR is multiplicative in thenumber of GPs being used (e.g. outcomes being modeled), assuming they are evaluated sequentially.
Hence, in multi-objective or constrained settings, PR incurs a high cost in terms of wall time.
However, empirically PR achieves better optimization performance on constrained and multi-objectiveproblems relative to C ONT .
R ELAX .
and E XACT R OUND .
We note that C ASMOPOLITAN does notsupport multi-objective BO or constrained BO, and although H Y BO could be used in those settings,it would be impractically slow because 1) its wall time would scale linearly with the number ofmodeled outcomes (using independent GPs) and 2) its diffusion kernel is non-differentiable, whichwould make optimizing hypervolume-based AFs slow [8, 9].
Discussion
The performance and regret properties of BO depend critically on adequately maximizing the AF.
For problems with discrete features, exhaustively trying all possible combinations of discrete valuesquickly becomes infeasible as the number of combinations grows.
Alternatives such as trying asubset of the possible combinations or resorting to continuous relaxations often leads to a failure toeffectively optimizing the AF which may result in sub-optimal BO performance.
As an alternative,we propose using PR which allows us to better optimize the AF and we showed that PR showsstrong performance on a large number of real-world problems.
Our approach is complementary toother approaches and can easily be combined with other approaches such as trust region methodsand specialized kernels for discrete parameters.
One limitation of PR is that it requires introducescomputationally-demanding integration.
However, given that the computation in PR is embarrassinglyparallel, it motivates for future research on optimizing AFs on distributed hardware.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See
Appendix A.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e
g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix C.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes] See Appendix C.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
Appendix to:
Bayesian Optimization over Discrete and Mixed
Spaces via Probabilistic Reparameterization
A Potential Societal Impacts
Our work advances Bayesian optimization, a generic class of methods for optimization of expensive,difﬁcult-to-optimize black-box problems.
With this paper in particular, we improve the performanceof Bayesian optimization on problems with mixed types of inputs.
Given the ubiquity of suchproblems in many practical applications, we believe that our method could lead to positive broaderimpacts by solving these problems better and more efﬁciently while reducing the costs incurred forsolving them.
Concrete and high-stake examples where our method could be potentially applied(some of which have been already demonstrated by the benchmark problems considered in the paper)include but are not limited to applications in communications, chemical synthesis, drug discovery,engineering optimization, tuning of recommender systems, and automation of machine learningsystems.
On the ﬂip side, while the method proposed is ethically neutral, there is potential of misusegiven that the exact objective of optimization is ultimately decided by the end users; we believe thatpractitioners and researchers should be aware of such possibility and aim to mitigate any potentialnegative impacts to the furthest extent.
B Theoretical Results and ProofsB.1 Results
Let P ( i ) Z :=
P ( Z ( i ) ) denote the set of probability measures on Z ( i ) for each i
= 1 , ..., d z , and let
P Z := (cid:81)
d
z i =1 P ( i ) Z .
For any α : X × Z → R , deﬁne ˜
α : X × P → R as˜ α ( x , p ) =
(cid:90)
Z α ( x , z ) dp ( z ) = (cid:88) z ∈Z α ( x , z ) p ( { z } ) .
Let Θ be a compact metric space, and consider the set of functionals Φ = { ϕ s.t.
ϕ : Θ → P Z } .
Letˆ α ( x , θ ) := ˜
α ( x ,
[Equation]
Since Z is ﬁnite, each element of ϕ ∈ Φ can be expressed as a mapping from Θ to R |Z| .
Namely,each ϕ ( θ ) corresponds to a vector with |Z| elements containing the probability mass for eachelement of Z under p ϕ ( θ ) .
Thus ( P Z , (cid:107) · (cid:107) ) is a metric space under any norm (cid:107) · (cid:107) on R | Z | .
Let α ∗ := max ( x , z ) ∈ ( X×Z )
α ( x , z ) and let H ∗ := arg max ( x , z ) ∈ ( X×Z ) α
( x , z ) denote the set ofmaximizers of α .
Lemma 1.
Suppose α is continuous in x for every z ∈ Z and that ϕ : Θ (cid:55)→ ( P Z , (cid:107)·(cid:107) ) is continuouswith ϕ (Θ)
= P Z .
Let J ∗ :
= arg max ( x , θ ) ∈X×
Θ ˆ α ( x , θ ) .
Then for any ( x ∗ , θ ∗ ) ∈
J ∗ , it holds that( x ∗ , z ) ∈
H ∗ for all z ∈ supp p ϕ
( θ ∗ ) .
Proof.
First, note that ˆ α : X × Θ → R is continuous (using that ϕ is continuous and α is bounded).
Since both X and Θ are compact ˆ α attains its maximum, i.e., J ∗ exists.
Let ( x ∗ , θ ∗ ) ∈
J ∗ .
Clearly,there exists z ∗ ∈ arg max z ∈Z α ( x ∗ , z ) such that α ( x ∗ , z ∗ ) =
α ∗ .
Suppose there exists z (cid:48) ∈supp p ϕ ( θ ∗ ) such that ( x ∗ , z (cid:48) ) / ∈ H ∗ .
Then α ( x ∗ , z (cid:48) ) <
α ∗ and, since Z is ﬁnite, p ϕ ( θ ∗ ) ( { z (cid:48) } ) > 0 .
Consider the probability measure p (cid:48) given byp
[Equation] [Equation]
=
[Equation]
if z
[Equation]
otherwise
Then
[Equation]
Now p (cid:48) ∈
P Z , and so p (cid:48) =
ϕ ( θ (cid:48) ) for some θ (cid:48) ∈ Θ .
But then ˆ α ( x ∗ , θ (cid:48) )
>
ˆ α ( x ∗ , θ ∗ ) .
This is acontradiction.
Corollary 2.
Suppose the optimizer of g is unique, i.e., that H ∗ = { ( x ∗ , z ∗ ) } is a singleton.
Thenthe optimizer of ˆ α is also unique and J ∗ =
{ ( x ∗ , θ ∗ ) } , with p ϕ ( θ ∗ ) ( { z ∗ } ) = 1 .
Corollary 3.
Consider the following mappings:
• Binary: ϕ : [0 , 1] → P { 0 , 1 } with p ϕ ( θ ) ( { 1 } )
= θ and p
ϕ ( θ ) ( { 0 } ) =
1 − θ .
• Ordinal:
ϕ : [0 , C − 1]
→ P { 0 , 1 ,...,C } with p ϕ ( θ ) ( { i } )
= (1 − | i − θ | ) 1 {| i − θ | ≤ 1 } fori = 1 , . . .
, C .
• Categorical:
ϕ : [0 , 1] C → P { 0 , 1 ,...,C } with p ϕ ( θ ) ( { i } )
=
θ i (cid:80) Ci =1 θ i .
These mappings satisfy the conditions for Lemma 1.
In the setting with multiple discrete parameterswhere the above mappings are applied in component-wise fashion for each discrete parameter, thecomponent-wise mappings also satisfy the conditions for Lemma 1.
Clearly, the mappings given in Corollary 3 are continuous functions of θ .
In the setting with multiplediscrete parameters , the component-wise function is also continuous with respect to the distributionparameters for each discrete parameter.
Hence, the mappings satisfy the conditions for Lemma 1.
Lemma 2.
If ( x ∗ , z ∗ ) ∈ H ∗ = arg max ( x , z ) ∈X×Z
α ( x , z ) , then
[Equation]
Proof.
For any z ∗ , let θ ∗ be the parameters such that p ( z ∗ | θ ∗ )
= 1 (i.e. a point mass on z ∗ ).
From
Equation (2),
[Equation]
Claim:
[Equation]
Suppose there exists θ (cid:48)
such that E Z ∼ p
( Z | θ (cid:48) )
[ α ( x ∗ , Z )] > E Z ∼ p
( Z | θ ∗ )
[ α ( x ∗ , Z )] .
Since ( x ∗ , z ∗ ) ∈
H ∗ , α ( x ∗ , z ∗ ) = max ( x , z )
∈X×Z
α ( x , z ) .
Hence, there is no convex combination of values of αthat is greater than α ( x ∗ , z ∗ ) .
This is a contradiction.
Theorem 1 (Consistent Maximizers) .
Suppose that α is continuous in x for every z ∈ Z .
Let H ∗be the maximizers of α ( x , z ) : H ∗ = { ( x , z ) ∈ arg max ( x , z )
∈X×Z α ( x , z ) } .
Let J ∗ ⊆
X × Θ bethe maximizers of E Z ∼ p
( Z | θ )
[ α ( x , Z )] : J ∗ = { ( x , θ ) ∈ arg max ( x , θ ) ∈X×
Θ E Z ∼ p
( Z | θ )
[ α ( x , Z )] } ,where Θ is the domain of θ .
Let ˆ H ∗ ⊆
X × Z be deﬁned as: ˆ H ∗ =
{ ( x , ˜ z ) : ( x , θ ) ∈ J ∗ , ˜ z ∼p ( Z | θ ) } .
Then, ˆ H ∗ =
H ∗ .
Proof.
From Lemma 1, we have that for any ( x ∗ , θ ∗ ) ∈
J ∗ , it holds that ( x ∗ , z ) ∈
H ∗ for allz ∈ supp p
ϕ
θ ∗ ) .
Hence, ˆ H ∗ ⊆
H ∗ .
Now, let ( x ∗ , z ∗ ) ∈
H ∗ .
Let θ ∗ ∈
Θ such that p ( z ∗ | θ ∗ ) = 1 .
From the proof of
Lemma 2, we have that E
Z ∼ p
( Z | θ ∗ )
[ α ( x ∗ , Z )]
= α ( x ∗ , z ∗ ) .
As in the proof of Lemma 2, thereis no convex combination of values of α greater than α ( x ∗ , z ∗ ) .
So E Z ∼ p
( Z | θ ∗ )
[ α ( x ∗ , Z )]
=max ( x , θ ) ∈X×
Θ E Z ∼ p
( Z | θ )
[ α ( x , Z )] , and therefore, x ∗ , θ ∗ ∈ J ∗ .
Hence ( x ∗ , z ∗ ) ∈
ˆ H ∗ .
So
H ∗ ⊆
ˆ H ∗ , and hence, ˆ H ∗ =
H ∗ .
Lemma 3.
Suppose that α : ( x , z ) (cid:55)→ R is differentiable with respect to x for all z ∈ Z , and thatthe mapping ϕ : θ (cid:55)→ P Z is such that p ϕ ( θ ) ( { z } ) is differentiable with respect to θ for all z ∈ Z .
Then the probabilistic objective E Z ∼ p
( Z | θ )
[ α ( x , Z )] is differentiable with respect to ( x , θ ) .
Proof.
For any z ∈ Z , the function p ( z , θ ) α
( x , z ) =
p ϕ ( θ ) ( { z } ) α
( x , z ) is the product of twodifferentiable functions, hence differentiable.
Therefore the probabilistic objective is a (ﬁnite) linearcombination of differentiable functions, hence differentiable.
Theorem 2 (Convergence Guarantee) .
Let α : X × Z
→ R be differentiable in x for every z ∈ Z .
Let (ˆ x t,m , ˆ θ t,m ) be the best solution after running stochastic gradient ascent for t time steps on theprobabilistic objective
E
Z ∼ p
( Z | θ )
[ α ( x , Z )] from m starting points with its unbiased MC estimatorsproposed above.
Let { a t } ∞ t =1 be a sequence of positive step sizes such that 0 < (cid:80) ∞
t
a 2 t =
A < ∞ ,where a t is the step size used in stochastic gradient ascent at time step t .
Let ˆ z t,m ∼ p
( Z | ˆ θ t,m ) .
Thenas t → ∞ and m → ∞ , (ˆ x t,m , ˆ z t,m )
→ ( x ∗ , z ∗ ) ∈ arg max ( x , z ) ∈X×Z α ( x , z ) in probability.
Proof.
The binary and categorical mappings in Corollary 3 are differentiable in θ (the ordinal mappingis differentiable almost everywhere 4 ).
Since the acquisition function α : X × Z → R is differentiablein x for every z ∈ Z , this means that the PO is differentiable.
Using the prescribed sequence ofstep sizes, optimizing the PO using stochastic gradient ascent will converge almost surely to a localmaximum after a sufﬁcient number of steps [46].
As we increase the number of randomly distributedstarting points, the probability of not ﬁnding the global maximum of the PO will converge to zero[58].
From Theorem 1, the PO and the AF have the same set of maximizers.
Hence, convergence inprobability to a global maximizer of the PO means convergence in probability to a global maximizerof the AF.
C Experiment Details
For each BO optimization replicate, we use N init =
min(20 , 2 ∗ d eff ) points from a scrambledsobol sequence, where d eff is the “effective dimensionality” after one-hot encoding categoricalparameters.
Unless otherwise noted, all experiments use 20 replications and conﬁdence intervalsrepresent 2 standard errors of the mean.
The same initial points are used for all methods for thatreplicate and different initial points are used for each replicate.
For each method we report thelog 10 regret.
Since the optimal value is unknown for many problems, we set the optimal valueto be max(1 .
001 · f ∗ , f ∗ + 0 . 1) where f ∗ is the best observed value across all methods and allreplications.
For constrained optimization f ∗ is the best feasible observed value and for multiobjective optimization f ∗ is the maximum hypervolume across all methods and replications.
Intotal, the experiments in the main text (excluding HyBO and Casmopolitan) ran for an equivalent of2,009.82 hours on a single Tesla V100-SXM2-16GB GPU.
The baseline experiments (HyBO and
Casmopolitan) ran for an equivalent of 745.10 hours on a single Intel Xeon Gold 6252N CPU.
C.1 Additional Problem Details
In this section, we describe the details of each synthetic problem considered in the experiments (thedetails of the remaining real-world problems are already described in Section 6.2).
Ackley.
We use an adapted version of the 13-dimensional Ackley function modiﬁed from Blieket al.
The function is given by:
[Equation]
where in this case a = 20 , n = 0 . 2 , c = 2 π and d = 13 and x ∈ [ − 1 , 1] 13 .
We discretize the ﬁrst 10dimensions to be binary with the choice {− 1 , 1 } , and the ﬁnal 3 dimensions are unmodiﬁed with theoriginal bounds.
Mixed Int F1.
Mixed Int F1 is a partially discretized version of the 16-dimensional Sphere optimization problem [27], given by:
[Equation]
where f opt is sampled from a Cauchy distribution with median = 0 and roughly 50% of the valuesbetween − 100 and 100 .
The sampled f opt is then clamped to be between [ − 1000 , 1000] and roundedto the nearest integer.
x opt is sampled uniformly in [ − 4 , 4] d , and in this case d = 16 .
We discretizethe ﬁrst 8 dimensions as follows: the ﬁrst 2 dimensions are binary with 2 choices {− 5 , 5 } ; the next 2dimensions are ordinal with 3 choices {− 5 , 0 , 5 } ; the next 2 dimensions are ordinal with 5 choices{− 5 , − 2 . 5 , 0 , 2 . 5 , 5 } ; the ﬁnal 2 dimensions are ordinal with 7 choices {− 5 , − 103 , − 53 , 0 , 53 , 103 , 5 } .
The remaining 8 dimensions are continuous with bounds [ − 5 , 5] 8 .
Rosenbrock.
We use an adapted version of the Rosenbrock function, given by:
[Equation]
where in this case d = 10 .
The ﬁrst 6 dimensions are discretized to be ordinal variables, with 4possible values each x i ∈ {− 5 , 0 , 5 , 10 }∀ i ∈
The ﬁnal 4 dimensions are continuous withbounds [ − 5 , 10] 4 .
Chemical Reaction (Direct Arylation Chemical Synthesis).
For this problem, we ﬁt a GP surrogate (with the same kernel used by the BO methods) to the dataset from Shields et al.
[50] (available at https://github.com/b-shields/edbo/tree/master/experiments/data/direct_arylation under the MIT license) in order to facilitate continuous optimization of temperature andconcentration.
The surrogate is included with our source code.
Oil Sorbent.
We set the reference point for this problem to be [ − 125 . 3865 , − 57 .
8292 , 43 . 2665] ,which we choose using a commonly used heuristic to scale the nadir point (component-wise worstobjective values across the Pareto frontier) [59].
C.2 Method details
PR, C ONT .
R ELAX .
, E XACT R OUND , PR + TR, and E XACT R OUND + STE .
We implementedall of these methods using BoTorch [1], which is available under the MIT license at https://github.com/pytorch/botorch .
All AFs are optimized via L-BFGS-B with 20 random restarts,run for a maximum of 200 iterations.
We follow the default initialization heuristic in BoTorch [1],which initializes the optimizer by evaluating the acquisition function at a large number of startingpoints (here, 1024, chosen from a scrambled Sobol sequence), and selecting (20) points using
Botlzmann sampling [19] of the 1024 initial points, according to their acquisition function utilities.
Combining PR with trust regions : When combining PR with the trust regions used in T U RBOwe only use a trust region over the continuous parameters and discrete ordinals with at least 3values.
While methods like C ASMOPOLITAN uses a Hamming distance for the trust regions over thecategorical parameters, we choose not to do so as there is no natural way of efﬁciently optimizing PRusing gradient-based methods.
Finally, we do not use a trust region over the Boolean parameters as thetrust region will quickly shrink to only include one possible value.
We use the same hyperparametersas T U RBO [22] for unconstrained problems and SCBO [21] in the presence of outcome constraints,including default trust region update settings.
Casmopolitan : We use the ofﬁcial implementation of C ASMOPOLITAN —which is availableat https://github.com/xingchenwan/Casmopolitan under the MIT licence—but modify itwhere appropriate to additionally handle the ordinal variables.
Speciﬁcally, the ordinal variablesare treated as continuous when computing the kernel.
However, during interleaved search, ordinalvariables are searched via local search similar to the categorical variables.
We use a set of C ASMOPOLITAN hyperparameters (i.e. success/failure sensitivity, initial trust region sizes and expansionfactor) recommended by the authors.
HyBO :
We use the ofﬁcial implementation of H Y BO at https://github.com/aryandeshwal/
HyBO , which is licensed by the University of Amsterdam.
We use the default hyperparameters recommended by the authors in all the experiments, and we use the full HyBO method with marginalizationtreatment of the hyperparameters as it has been shown to perform stronger empirically [14].
C.3 Gaussian process regression
When there are no categorical variables we use k ordinal which is a product of an isotropic Matern5/2 kernel for the binary parameters and a Matern-5/2 kernel with ARD for the remaining ordinalparameters.
In the presence of categorical parameters, this kernel is combined with a categoricalkernel [47] k cat as k cat × k ordinal + k cat + k ordinal .
We use a constant mean function.
The GPhyperparameters are ﬁtted using L-BFGS-B by optimizing the log-marginal likelihood.
The rangesfor the ordinal parameters are rescaled to [0 , 1] and the outcomes are standardized before ﬁtting the
GP.
C.4 Sample Average Approximation
As discussed in Section 4.3, we instead of directly optimizing the stochastic MC estimators, weinstead use sample average approximation.
The idea is to use common random numbers (i.e. a ﬁxedset of samples) to reduce variance when comparing a stochastic function at different inputs by usingthe same random numbers.
This leads to biased deterministic estimators that are lower-variance thantheir stochastic counterparts where random numbers are re-sampled at each step.
Such techniqueshave been used in the context of BO in settings such as efﬁciently optimizing MC acquisition functions[1] and for optimizing risk measures of acquisition functions under random inputs [10].
Sampling a ﬁxed set of points ˜
z 1 , ...,
˜ z N ∼ p
( Z | θ ) would be a poor choice because p ( Z | θ ) canvary widely during AF optimization as θ changes.
Therefore, instead sample from p ( Z | θ ) usingreparameterizations provided in Table 4.
Speciﬁcally, we reparameterize Z as a deterministic functionh ( · , · ) that operates component-wise on θ and the random variable U
=
( u (1) , ..., u ( d z ) ) , u ( i ) ∼
Uniform (0 , 1) :
Z = h ( θ , U ) .
That is, each random variable Z ( i ) has a corresponding independentbase
random variable
U
( i ) such that Z ( i ) =
h ( θ ( i ) , U ( i ) ) .
Using a ﬁxed a set of base samples{ ˜ u i } Ni =1 , the samples of Z can be be computed as z i
= h ( θ , ˜ u i ) .
We note that even with ﬁxed basesamples, the samples { z i } Ni =1 depends on θ , and hence, by using common base uniform samples, weobtain a deterministic estimator where the values of the samples ˜
z 1 , ...,
˜ z N can still vary with θ .
[Table]
D Constrained and Multi-Objective Bayesian Optimization
In many practical problems, the black-box objective must be maximized subject to V > 0 black-boxoutcome constraints f ( v ) c
( x , z ) ≥ 0 for v = 1 , ..., V .
See Gardner et al.
[24] for a more in depthreview of black-box optimization with black-box constraints and BO techniques for this class ofproblems.
In the multi-objective setting, the goal is to maximize (without loss of generality) a set of M objectivesf (1) , ..., f ( M ) .
Typically there is no single best solution, and hence the goal is to learn the Paretofrontier (i.e. the set of optimal trade-offs between objectives).
In the multi-objective setting, thehypervolume indicator is a common metric for evaluating the quality of a Pareto frontier.
See [20] fora review of multi-objective optimization.
E Wall times
[Figure]
In this section, we reported the AF optimization wall times for the experiments in the main text.
[Figure]
an Intel Xeon Gold 6252N CPU.
[Figure]
Rosenbrock
[Figure]
Ackley
[Figure]
Welded Beam (5 constraints)
[Figure]
F Probabilistic Reparameterization Sensitivity Analyses
[Figure]
F.1 Monte Carlo samples
[Figure]
The main text considers 1024 MC for PR.
We consider 128, 256, and 512 samples, in additionto the default of 1024.
For problems with discrete spaces that are enumerable, we also consideranalytic PR.
We do not ﬁnd statistically signiﬁcant differences between the ﬁnal regret of any ofthese conﬁgurations (Figure 5).
Run time is linear with respect to MC samples, and so substantialcompute savings are possible when fewer MC samples are used (Figure 6).
We observe comparableperformance between PR with 1,024 MC samples and as few as 128 MC samples.
With 64 or fewer
MC samples, we observe performance degradation with respect to log regret in Figure 7, althoughwall time is considerably faster for fewer 64 or less MC samples as shown in Figure 8.
[Figure]
PR (
[Figure] [Figure]
We observe that wall time scales linearly with the number of MC samples, which is expected since we compute PR in N 32 chunks to avoid overﬂowing memory
[Figure]
r
[Figure]
t
[Figure] [Figure]
Since PR performs well with 128 MC samples
[Figure]
we report the main results from the paper using 128samples with PR and PR + TR in Figures 9 and 10.
We observe comparable performance to PR and
PR + TR with 1,024 MC samples and much faster wall times—which are competitive with othermethods
[Figure] [Figure]
[Figure]
PR and PR + TR using 128 MC samples.
C ONT .
R ELAX .
, E XACT R OUND , PR, and PR + TR are run on a single
Tesla V100-SXM2-16GB GPU and other methods are run on an Intel Xeon Gold 6252N CPU.
F.2 Effect of τ in Transformation
Throughout the main text, we use τ = 0 . 1 , which we selected based on the observation that it providesa reasonable balance between retaining non-zero gradients of g ( φ ) with respect to φ and allowing θto become close to 0 or 1.
This observation is obvious from Figure 11.
We further examine the effects of this hyperparameter by testing values of τ across four orders ofmagnitude from 10 0 to 10 − 3 in the following sensitivity analysis.
We ﬁnd that our default choice of0.1 tends to perform well across all benchmark problems considered in this work.
As τ → 0 , the θ can take more extreme values, but the gradient of the transformation with respect toφ also moves closer to zero.
For larger values of τ , the gradient of the transformation with respectto φ is larger, but θ has a more limited domain with less extreme values.
We ﬁnd that τ = 0 .
1 is arobust setting across all experiments.
[Figure] [Figure]
G Alternative methods
G.1 Straight-through gradient estimators
An alternative approach to using approximating the gradients under exact rounding using ﬁnitedifferences is to approximate the gradients using straight-through gradient estimation (STE) [3].
Theidea of STE is to approximate the gradient of a function with the identity function.
In our setting, thegradient of the discretization function with respect to its input is estimated using an identity function.
Using this estimator enables gradient-based AF optimization, even though the true gradient of thediscretization function is zero everywhere that it is deﬁned.
Although STEs have been shown towork well empirically, these estimators are not well-grounded theoretically.
Their robustness andpotential pitfalls in the context of AF optimization have not been well studied.
Below, we evaluate theaforementioned E XACT R OUND + STE approach and show that it offers competitive optimizationperformance (Figure 13) with fast wall times (Figure 14), but does not quite match the optimizationperformance of PR on several benchmark problems.
[Figure] [Figure]
G. TR methods with alternative optimizers
In this section, we consider alternative methods to PR for optimizing AFs using within trust regions.
The results in Figure 15 show that PR is a consistent best optimizer using TRs, but that STEs workquite well with TRs in many scenarios.
[Figure] [Figure]
H Alternative categorical kernels
In this section
[Figure]
we demonstrate that PR can be used with arbitrary kernels over the categoricalparameters including those that require discrete inputs (which C ONT .
R ELAX . is incompatible with).
Speciﬁcally for the categorical parameters, we compare using a Matérn-5/2 Kernel with one-hotencoded categoricals, a Categorical kernel (default)
[Figure]
and a latent embedding kernel [65].
For the latentembedding kernel, we follow Pelamatti et al.
[43] and use a 1-d latent embedding for categoricalparameters where the cardinality is less than or equal to 3 and a 2-d embedding for categoricalparameters where the cardinality is greater than 3.
For each latent embedding, we use an isotropic
Matérn-5/2 kernel and use product kernel across the kernels for the categorical, binary, ordinal, andcontinuous parameters.
The results presented in Figure 18 show that latent embedding and categoricalkernels have much lower variance across replications on the chemical reaction problem.
[Figure]
Welded Beam (5 constraints)
[Figure]
Chemical Reaction
I Alternative Acquisition Functions
In this section
[Figure]
we compare PR with expected improvement (EI) against PR with upper conﬁdencebound (UCB).
For UCB, we set the hyperparameter β in each iteration using the method in Kandasamyet al.
Although UCB comes enjoys bounded regret [51]
[Figure]
we ﬁnd empirically that EI works betteron most problems.
[Figure]
J Additional Results on Optimizing Acquisition Functions
In this section, we provide additional results on various approaches for optimizing acquisitionfunctions using the same evaluation procedure as in the main text.
We use 50 replications.
[Figure]
K Stochastic vs Deterministic Optimization
In this section, we compare optimizing PR with stochastic and deterministic optimization methods.
For stochastic methods, we use stochastic gradient ascent (SGA) with various initial learning ratesbetween 10 − 1 and 10 − 3 .
The learning rate is multiplied by 0.1 every 30 steps following the approachin Huang et al.
For SGA, the MC estimators of PR and its gradient resample the base samplesat each evaluation.
For deterministic optimization, base samples are kept ﬁxed.
SGA is run for 200iterations and L-BFGS-B is run for a maximum of 200 iterations.
In Figure 20, we observe that
SGA works well on some problems with some learning rates, but SGA is sensitive to the choice oflearning rate.
Furthermore, we observe that the best learning rate varies from problem to problem.
Deterministic optimization consistently performs better than stochastic optimization and has lowervariance across replications.
[Figure]
[Figure]
L Discussion of related work
Although the reparameterization trick used by Wilson et al.
[61] is in a similar vein to PR, Wilsonet al.
[61] reparameterizes an existing multivariate normal random variable in terms of standardnormal random variables and then proposes to use sample-path gradient estimators.
In contrast, ourapproach introduces a new probabilistic formulation using discrete probability distributions and useslikelihood-ratio-based gradient estimators since sample-path gradients cannot be computed throughdiscrete sampling.
M Comparison with an Evolutionary Algorithm
In Figures 21 and 21, we compare against the evolutionary algorithm PortfolioDiscreteOnePlusOne,which is the recommended algorithm for discrete and mixed search spaces in the Nevergrad package[44].
We ﬁnd that PR signiﬁcant outperforms this baseline with respect to log regret, but is slowerthan the evolutionary algorithm with respect to wall time.
[Figure] [Figure]
N Comparison with Enumeration
When computationally feasible, the gold standard for acquisition optimization over discrete andmixed search spaces is to enumerate the discrete options and optimize any continuous parameters foreach discrete conﬁguration (or simply evaluated each discrete conﬁguration for fully discrete spaces).
In Figures 23 and 24 we compare PR (with 1024 MC samples) against enumeration and show that
PR achieves log regret performance that is comparable to the gold standard of enumeration and doesso in less wall time.
[Figure] [Figure]
O Acquisition Function Optimization at a Given Wall Time Budget
In Figure 25, we provide additional starting points (64 points, rather than 20) to other non-PR methodsin order to provide them with additional wall-time budget.
We ﬁnd that using PR with 64 MC samples,
PR provides rapid convergence compared to other baselines and therefore is a good optimizationroutine for any wall time budget.
[Figure]
Cont
[Figure]
Relax
[Figure]
PR
P Evaluation of Approximation Error in MC Sampling
We examine the MC approximation error relative to analytic PR on the chemical reaction and ackley problems.
The results in Figure 26 show the mean absolute percentage error (MAPE)
[Figure]
[Equation]
[Figure]
[Equation]
evaluated over a random set of | X discrete | = | Θ discrete | 10 , 000 points from X × × (the sampled setsare denoted X discrete , Θ discrete ).
We observe a rapid reduction in MAPE as we increase the number of samples.
With 1024 samples, MAPE is 0.055% (+/- 0.0002 %) over 20 replications (different MCsamples in PR) on the chemical reaction problem and MAPE is 0.018% (+/- 0.0003 %) on the ackleyproblem.
With 128 samples, MAPE is 0.282% (+/- 0.0029 %) over 20 replications (different MC samples in
PR) on the chemical reaction problem and MAPE is 0.052% (+/- 0.0021 %) on the ackley problem.
[Figure]
M e
[Figure]
n A b s o l u t e
P e r c e n t
E rr o r
Chemical Reaction
Number of MC Samples 0.
Ackley