P RIOR - GUIDED B AYESIAN O PTIMIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
While Bayesian Optimization (BO) is a very popular method for optimizing expensive black-box functions, it fails to leverage the experience of domain experts.
This causes BO to waste function evaluations on bad design choices (e.g., machine learning hyperparameters) that the expert already knows to work poorly.
To address this issue, we introduce Prior-guided Bayesian Optimization (PrBO).
PrBO allows users to inject their knowledge into the optimization process in the form of priors about which parts of the input space will yield the best performance, rather than BO’s standard priors over functions (which are much less intuitive for users).
PrBO then combines these priors with BO’s standard probabilistic model to form a pseudo-posterior used to select which points to evaluate next.
We show that PrBO is around 12x faster than state-of-the-art methods without user priors and 10,000x faster than random search on a common suite of benchmarks, and achieves a new state-of-the-art performance on a real-world hardware design application.
We also show that PrBO converges faster even if the user priors are not entirely accurate and that it robustly recovers from misleading priors.
I NTRODUCTION
Bayesian Optimization (BO) is a data-efﬁcient method for the joint optimization of design choices that gained great popularity in recent years.
It is impacting a wide range of areas, including hyperparameter optimization (Snoek et al., 2012;
Falkner et al., 2018), AutoML (Feurer et al., 2015a; Hutter et al., 2018), robotics (Calandra et al., 2016), computer vision (Nardi et al., 2017; Bodin et al., 2016), environmental monitoring (Marchant & Ramos, 2012), combinatorial optimization (Hutter et al., 2011), experimental design (Azimi et al., 2012), RL (Brochu et al., 2010), Computer Go (Chen et al., 2018), hardware design (Koeplinger et al., 2018; Nardi et al., 2019) and many others.
It promises greater automation so as to increase both product quality and human productivity.
As a result, BO is also established in many large tech companies, e.g., with Google Vizier (Golovin et al., 2017) and
Facebook BoTorch (Balandat et al., 2019).
Nevertheless domain experts often have substantial prior knowledge that standard BO cannot incorporate.
Users can incorporate prior knowledge by narrowing the search space; however, this type of hard prior can lead to poor performance by missing important regions.
BO also supports a prior over functions p ( f ) , e.g., via a kernel function.
However, this is not the prior experts have: users often know which ranges of hyperparameters tend to work best, and are able to specify a probability distribution p best ( x ) to quantify these priors.
E.g., many users of the Adam optimizer (Kingma &
Ba, 2015) know that its best learning rate is often in the vicinity of 1e-3 (give or take one order of magnitude), yet one may not know the accuracy one may achieve in a new application.
Similarly,
Navruzyan et al.
(2019) derived neural network hyperparameter priors for image datasets based on their experience with ﬁve datasets.
In these cases, users know potentially good values for a new application, but cannot be certain about them.
As a result, many competent users instead revert to manual search, which can fully incorporate their prior knowledge.
A recent survey showed that most NeurIPS 2019 and ICLR 2020 papers that reported having tuned hyperparameters used manual search, with only a very small fraction using
BO (Bouthillier & Varoquaux, 2020).
In order for BO to be adopted widely, and help facilitate faster progress in the ML community by tuning hyperparameters faster and better, it is therefore crucial to devise a method that fully incorporates expert knowledge into BO.
In this paper, we introduce
Prior-guided Bayesian Optimization (PrBO), a novel BO variant that combines user prior knowledge with a probabilistic model of the observations made.
Our technical contributions with PrBO are:
For the ﬁrst time, user prior knowledge can be combined with standard BO probabilistic models, such as Gaussian Processes (GPs), Random Forests (RFs), and Bayesian Neural Networks.
Unlike previous approaches, PrBO is ﬂexible w.r.t.
how the prior is deﬁned, allowing previously impossible-to-inject (e
g. decay and exponential) priors.
PrBO gives more importance to the model as iterations progress, gradually forgetting the prior and ensuring robustness against misleading priors.
We demonstrate the effectiveness of PrBO on a comprehensive set of real-world applications and synthetic benchmarks, showing that accurate prior knowledge helps PrBO to achieve similar performance to current state-of-the-art on average 12 .
12 × faster on synthetic benchmarks and 1 . 49 × faster on a real-world application.
PrBO also achieves equal or better ﬁnal performance in all but one of the benchmarks tested.
B ACKGROUND 2. B AYESIAN O PTIMIZATION
Bayesian Optimization (BO) is an approach for optimizing an unknown function f : X → R that is expensive to evaluate over an input space X .
In this paper, we aim to minimize f , i.e., ﬁnd x ∗ ∈ arg min x ∈X f ( x ) .
BO approximates x ∗ with an optimal sequence of evaluations x 1 , x 2 , . . .
∈ X , with each new x n +1 depending on the previous function values y 1 , y 2 , . . .
, y n at x 1 , . .
, x n .
BO achieves this by building a posterior on f based on the set of evaluated points.
At each BO iteration, a new point is selected and evaluated based on the posterior, and the posterior is updated to include the new point ( x n +1 , y n +1 ) .
The points explored by BO are dictated by the acquisition function, which attributes a value to each x ∈ X by balancing the predicted value and uncertainty of the prediction for each x .
In this work, as the acquisition function we choose Expected Improvement (EI) (Mockus et al., 1978), which quantiﬁes the expected improvement over the best function value found so far: EI f inc ( x ) :
= (cid:82) inf − inf max( f inc − y, 0) p
( y | x ) dy , where f inc is the incumbent function value, i.e., the best objective function value found so far, and p ( y | x ) is a probabilistic model, e.g., a GP.
Alternatives to EI would be Probability of Improvement (PI) (Jones, 2001), upper-conﬁdence bounds (UCB) (Srinivas et al., 2010), and entropy-based methods (e.g. Hernández-Lobato et al.
2. T REE - STRUCTURED P ARZEN E STIMATOR
Whereas the standard probabilistic model in BO directly models p ( y | x ) , the Tree-structured Parzen
Estimator (TPE) approach of Bergstra et al.
(2011) models p ( x | y ) and p ( y ) instead.
This is done by constructing two parametric densities, g ( x ) and l ( x ) , which are computed using the observations with function value above and below a given threshold, respectively.
The separating threshold y ∗ is deﬁned as a quantile of the observed function values.
TPE uses the densities g ( x ) and l ( x ) to deﬁne p ( x | y ) as:
[Equation]
where I ( y < y ∗ ) is 1 when y < y ∗ and 0 otherwise.
The parametrization of the generative model p ( x , y ) =
p ( x | y ) p ( y ) facilitates the computation of EI as it leads to EI y ∗ ( x ) ∝ l ( x ) /g ( x ) and, thus, arg max x ∈X EI y ∗ ( x ) =
arg max x ∈X l ( x ) /g ( x ) .
B AYESIAN O PTIMIZATION WITH P RIORS
We propose a BO approach dubbed PrBO that allows ﬁeld experts to inject user prior knowledge into the optimization in the form of priors.
PrBO combines this user-deﬁned prior with a probabilistic model that captures the likelihood of the observed data ( x i , y i ) ni
=1 .
PrBO is independent from the probabilistic model being used, i.e., it can be freely combined with, e.g., GPs, RFs, or Bayesian
Neural Networks.
3. P RIORS
PrBO allows users to inject prior knowledge into BO.
This is done via a prior distribution that informs where in the input space
X we expect to ﬁnd good f ( x ) values.
A point is considered “good” if it leads to low function values.
We denote the prior distribution P g ( x ) , where g denotes that this is a prior on good points and x ∈ X is a given point.
Similarly, we deﬁne a prior on where in the input space we expect to have “bad” points.
Although we could have a separate user-deﬁned probability distribution P b ( x ) , we aimed to keep the load on users low and thus, for simplicity, only require the deﬁnition of P g ( x ) and compute P b ( x ) = 1 − P g ( x ) .
1 P g ( x ) is normalized to [0 , 1] by min-max scaling before computing P b ( x ) .
In practice, x contains several dimensions but it is difﬁcult for domain experts to provide a multivariate distribution P g ( x ) .
Users can easily specify, e.g., draw, a univariate or bivariate probability distribution for continuous dimensions or provide a list of probabilities for discrete dimensions.
In
PrBO, users are free to deﬁne a complex multivariate distribution, but we expect the standard use case to be that users only want to specify univariate distributions, implicitly assuming a prior that factors as P g ( x )
= (cid:81)
Di =1 P g ( x i ) , where D is the number of dimensions in X and x
i is the i-th input dimension of X .
We show examples of continuous and discrete priors in Appendices B and F, respectively.
We use factorized priors in our experiments to mimic what we expect most users will provide.
In Appendix G, we show that these factorized priors lead to similar performance compared to multivariate priors.
3. M ODEL
Whereas the standard probabilistic model in BO, e.g., a GP, quantiﬁes p ( y | x ) directly, that model is hard to combine with the user-deﬁned prior P g ( x ) .
We therefore now introduce a method to translate the standard probabilistic model p ( y | x ) into a model that is easier to combine with this prior.
Similar to the TPE work described in Sec. 2.2, our generative model combines p ( x | y ) and p ( y ) instead of directly modeling p ( y | x ) .
The computation we perform for this translation is to quantify the probability that a given input x is “good” under our standard probabilistic model p ( y | x ) .
As in TPE, we deﬁne conﬁgurations as “good” if their observed y -value is below a certain quantile γ of the observed function values (so that p ( y < f γ ) = γ ).
We in addition exploit the fact that our standard probabilistic model p ( y | x ) has a
Gaussian form, and under this Gaussian prediction we can compute the probability M g ( x ) of the function value lying below a certain quantile using the standard closed-form formula for PI (Kushner, 1964):
[Equation]
where µ x and σ x are the mean and standard deviation of the probabilistic model at x , and Φ is the standard normal CDF, see Figure 1.
Likewise, we compute a probability M b ( x ) of x being bad.
3. P SEUDO - POSTERIOR
PrBO combines the prior in Section (3.1) and the model in Eq.
(2) into a pseudo-posterior on “good” points.
This pseudo-posterior represents the updated beliefs on where we can ﬁnd good points, based on the prior and data that has been observed.
The pseudo-posterior is computed as the product of the prior and the model:
[Equation]
where t is the current optimization iteration, β is an optimization hyperparameter, M g ( x ) is deﬁned in Eq.
(2), and P g ( x ) is the prior deﬁned in Sec 3.1, rescaled to [0, 1].
We note that this pseudoposterior is not normalized, but this sufﬁces for PrBO to determine the next x t as the normalization constant cancels out (c.f. Section 3.4).
Since g ( x ) is not normalized and we include the exponent t/β in Eq.
3, we refer to g ( x ) as a pseudo-posterior, to emphasize that it is not a standard posterior probability distribution.
The t/β fraction in Eq.
(3) controls how much weight is given to the model.
As the optimization progresses, more weight is given to the model over the prior.
Intuitively, we put more emphasis on the model as it observes more data and becomes more accurate.
We do this under the assumption that the model will eventually be better than the user at predicting where to ﬁnd good points.
This also allows to recover from misleading priors as we show in Section 4.2; similar to, and inspired by
Bayesian models, the data ultimately washes out the prior.
The β hyperparameter deﬁnes the balance between prior and model, with higher β values giving more importance to the prior and requiring more data to overrule it.
We note that, computing Equation (3) directly can lead to numerical issues.
Namely, the pseudoposterior can reach extremely low values if the prior and model probabilities are low, especially as the t/β exponent grows.
To prevent this, in practice, PrBO uses the logarithm of the pseudo-posterior instead: log( g ( x ))
∝ log( P g ( x )) + tβ · log( M g ( x )) .
Once again, we also deﬁne an analogous pseudo-posterior distribution on bad x : b ( x ) ∝
P b ( x ) M b ( x ) tβ .
We then use these quantities to deﬁne a density model p ( x | y ) as follows:
[Equation]
3. A CQUISITION F UNCTION
We adopt the EI formulation used in (Bergstra et al., 2011) by replacing their Adaptive Parzen
Estimators with our computation of the pseudo-posterior in Eq.
Namely, we compute EI as:
[Equation]
The full derivation of Eq.
(5) is shown in Appendix C. Eq.
(5) shows that to maximize improvement we would like points x with high probability under g ( x ) and low probability under b ( x ) , i.e., minimizing the ratio b ( x ) /g
( x ) .
We note that the point that minimizes the ratio for our unnormalized pseudo-posteriors will be the same that minimizes the ratio for the normalized pseudo-posterior and, thus, the computation of the normalized pseudo-posteriors is unnecessary.
The dynamics of the PrBO algorithm can be understood in terms of the following proposition (proof in Appendix C):
Proposition 1 Given f γ , P g ( x ) , P b ( x ) , M g ( x ) , M b ( x ) , g ( x ) , b ( x ) , p ( x | y ) , and β as above, then
[Equation]
where EI f γ is the Expected Improvement acquisition function as deﬁned in Eq.
5 and M g ( x ) is as deﬁned in Equation (2).
In early BO iterations the prior will have a predominant role, but in later BO iterations the model will grow more important, and as Proposition 1 shows, if PrBO is run long enough the prior washes out and PrBO only trusts the probabilistic model informed by the data.
3. P UTTING I T A LL T OGETHER
Algorithm 1 shows the PrBO algorithm, based on the components deﬁned in the previous sections.
In
Line 3, PrBO starts with a design of experiments (DoE) phase, where it randomly samples a number of points from the user-deﬁned prior P g ( x ) .
After initialization, the BO loop starts at Line 4.
In each loop iteration, PrBO ﬁts the probabilistic model on the previously evaluated points (lines 5 and 6) and computes the pseudo-posteriors g ( x ) and b ( x ) (lines 7 and 8 respectively).
The EI acquisition function is computed next, using the pseudo-posteriors, and the point that maximizes EI is selected as the next point to evaluate at line 9.
The black-box function evaluation is performed at Line 10.
This
BO loop is repeated for a pre-deﬁned number of iterations, according to the user-deﬁned budget B .
In Appendix A, we show a visualization of PrBO on a 1-dimensional function.
E XPERIMENTS
We implement both GPs and RFs as predictive models and use GPs in our experiments unless stated otherwise.
We set the model weight β
= 10 and the model quantile to γ = 0 .
05 , see our sensitivity hyperparameter study in Appendices L and M. Before starting the main BO loop in PrBO, we randomly sample D + 1 points from the prior consistently on all benchmarks.
In Appendix J, we compare PrBO to other baselines with the same prior initialization and show that it leads to similar results.
We optimize our EI acquisition function using a multi-start local search, similar to
SMAC (Hutter et al., 2011).
We start with four synthetic benchmarks: Branin, SVM, FC-Net, and
XGBoost, which are 2, 2, 6, and 8 dimensional, respectively.
The last three are part of the Profet benchmarks (Klein et al., 2019), generated by a generative model built using performance data on
OpenML or UCI datasets.
See Appendix D for more details on the experimental setup.
Due to space constraints, we defer the (qualitatively similar) results for the SVM benchmark to Appendix E.
4. P RIOR S ELECTION
In this section we study the effect of choosing a prior.
A suitable property of the prior is that, by selecting a tighter prior around an optimum, we would expect sampling from the prior to have an increased performance.
To the limit, if the prior is composed by only one point which is one of the global optima, then the ﬁrst sample (and all of them) from the prior will hit the optimum.
To have a sanity check of this property, we build an artiﬁcial prior in a controlled way.
We rely on an automated computation of the prior by computing a univariate Kernel Density Estimation (KDE) using a Gaussian kernel on the synthetic benchmarks introduced above.
We note that the goal of these synthetic priors is to have an unbiased prior for our experiments, whereas manual priors would be biased by our own expertise of these benchmarks.
In practice, users will manually deﬁne these priors without needing additional experiments.
Appendix G shows that using univariate KDEs for the prior lead to similar performance compared to multivariate KDEs.
We experiment with an array of varying quality priors.
We select a constant 10 D points in each prior and vary the size of the random sample dataset so that we can make the priors more sharply peaked around the optima in a controlled environment.
We use the best performing 10 D samples to create the prior from a uniform random sample dataset size of 10 D 100 x ; we refer to this prior as x % in
Figure 3.
As an example the XGBoost benchmark has d = 8 , so, 100% means we sample 80 points and use all 80 to create the prior, 10% means we sample 800 points and use the best performingto create the prior, 1% means we sample 8 , 000 and use the best 80 to create the prior, and so on.
[Figure]
Figure 3 shows the performance of purely sampling from the prior and running PrBO, respectively, after 10 D function evaluations with different priors.
A bigger random sample dataset and a smaller percentage leads to a tighter prior around the optimum, making the argument for a stronger prior.
This is conﬁrmed by Figure 3, where a sharply peaked prior (right side of the ﬁgure) leads to a better performance in both scenarios.
In addition we observe that in contrast to sampling from the prior,
PrBO achieves a smaller regret by being able to evolve from the initial prior and making independent steps towards better values of the objective function.
More extensive experiments with a similar trend, including the rest of the benchmarks, are in Appendix K.
4. P RIOR F ORGETTING
In this section, we show that PrBO can recover from a misleading prior, thanks to our predictive model and the t/β parameter in the pseudo-posterior computation Eq. 3.
As BO progresses, the predictive model becomes more accurate and receives more weight, guiding optimization away from the wrong prior and towards better values of the objective function.
Figure 2 shows PrBO on the 1D
Branin function with an exponential prior.
Columns (b), (c), and (d) show PrBO after D + 1 =initial samples and 0 , 10 , 20 BO iterations, respectively.
After initialization, as shown in Column (b), the pseudo-posterior is nearly identical to the exponential prior and guides PrBO towards the region of the space on the right, which is towards the local optimum.
This happens until the predictive model becomes certain there will be no more improvement from sampling that region (Columns (c) and (d)).
After that, the predictive model guides the pseudo-posterior towards exploring regions with high uncertainty.
Once the global minimum region is found, the pseudo-posterior starts balancing exploiting the global minimum and exploring regions with high uncertainty, as shown in 2d (bottom).
Notably, the pseudo-posterior after x > 4 falls to 0 in 2d (top), as the predictive model is certain there will be no improvement from sampling the region of the local optimum.
We provide additional examples of forgetting in Appendix B, and a comparison of PrBO with misleading priors, no prior, and correct priors in Appendix H.
4. C OMPARISON A GAINST S TRONG B ASELINES
Figure 4 compares PrBO to other optimizers using the log simple regret on ﬁve runs (mean and std error reported) on the synthetic benchmarks.
We compare the results of PrBO with and without priors (both weak and strong) to 10 ,
× random search (RS, i.e., for each BO sample we draw 10 ,uniform random samples), sampling from the strong prior only, and Spearmint (Snoek et al., 2012) which is a well-adopted BO approach using GPs and the EI acquisition function.
[Figure] [Figure]
PrBO prior beats 10 , 000 × RS and PrBO weak prior on all benchmarks.
It also outperforms the performance of sampling from the prior; this is expected because prior sampling cannot recover from a non-ideal prior.
The two methods are identical up to the initialization phase because they both sample from the same prior in that phase.
PrBO Prior is more sample efﬁcient and ﬁnds better or equal results than Spearmint on three out of the four benchmarks.
On the fourth benchmark, XGBoost, PrBO leads the performance until BO iterations, where Spearmint catches up and achieves slightly better results in the end (function values of 8 .
986 for Spearmint vs. 9 .
026 for PrBO after 300 iterations).
Importantly, in all our experiments, PrBO with a good prior consistently shows tremendous speedups in the early phases of the optimization process, typically only requiring on average 8.25 iterations to reach the performance that Spearmint reaches after 100 iterations ( 12 . 12 × faster).
Thus, in comparison to Spearmint and other traditional BO approaches, PrBO makes use of the best of both worlds, leveraging prior knowledge and efﬁcient optimization based on BO.
4. T HE S PATIAL U SE - CASE
We next apply PrBO to the Spatial (Koeplinger et al., 2018) real-world application.
Spatial is a programming language and corresponding compiler for the design of application accelerators, i.e.,
FPGAs.
We apply PrBO to three Spatial benchmarks, namely, 7D shallow and deep CNNs, and a 10D molecular dynamics grid application.
We compare the performance of PrBO to RS, manual optimization, and HyperMapper (Nardi et al., 2019), the current state-of-the-art BO solution for
Spatial .
For a fair comparison, both PrBO and HyperMapper use RFs as predictive probabilistic model.
The manual optimization and the prior for PrBO were provided by an unbiased Spatial developer, who is not an author of this paper.
The priors were provided once and kept unchanged for the whole project.
More details on the setup, including the priors used, are presented in Appendix F.
Figure 5 shows the log regret on the Spatial benchmarks.
PrBO vastly outperforms RS in all benchmarks; notably, RS does not improve over the default conﬁguration in MD Grid.
PrBO is also able to leverage the expert’s prior and outperform the expert’s conﬁguration in all benchmarks ( 2 . 68 × , 1 . 06 × , and 10 . 4 × speedup for shallow CNN, deep CNN, and MD Grid, respectively).
Compared to
HyperMapper, PrBO achieves better performance in the MD Grid benchmark ( 1 . 28 × speedup).
For context, this is a signiﬁcant improvement in the FPGA ﬁeld, where a 10% improvement could qualify for acceptance in a top-tier conference.
In the CNN benchmarks, PrBO converges to the minima regions faster than HyperMapper ( 1 . 58 × and 1 . 4 × faster for shallow and deep respectively).
Thus,
PrBO leverages the best of both worlds (the expert’s prior knowledge and BO) to provide a new state of the art for Spatial .
R ELATED W ORK
TPE by Bergstra et al.
(2011), the default optimizer in the popular HyperOpt package (Bergstra et al., 2013), supports limited hand-designed priors in the form of normal or log-normal distributions.
We make three technical contributions that make PrBO more ﬂexible than TPE.
First, we generalize over the TPE approach by allowing more ﬂexible priors; second, our approach is model-agnostic (i.e.,
[Figure]
PrBO is not limited to the TPE model; we use GPs and RFs in our experiments); and third, PrBO is inspired by Bayesian models that give more importance to the data as iterations progress.
We also show that PrBO outperforms HyperOpt’s TPE in Appendix I.
In parallel work, Li et al.
(2020) allow users to specify priors via a probability distribution, similarly to our approach.
Their two-level approach ﬁrst samples a number of conﬁgurations by maximizing
Thompson samples from a GP pseudo-posterior and then chooses the conﬁguration that has the highest prior as the next to evaluate.
In contrast, our method leverages the information from the prior more directly and ensures that the prior gets washed out as we collect more data, enabling PrBO to overcome misspeciﬁed priors.
Similarly, black-box optimization tools, such as SMAC (Hutter et al., 2011) or iRace (López-Ibáñez et al., 2016) also support simple hand-designed priors, e.g. log-transformations.
However, these are not properly reﬂected in the predictive models and both cannot explicitly recover from bad priors.
The same applies to the works of Oh et al.
and Siivola et al.
Our work also relates to meta-learning for BO (Vanschoren, 2019), where BO is applied to many similar optimization problems in a sequence such that knowledge about the general problem structure can be exploited in future optimization problems.
In contrast to meta-learning, PrBO is the ﬁrst method that allows human experts to explicitly specify their priors.
Furthermore, PrBO does not depend on any meta-features (Feurer et al., 2015b) and only incorporates a single prior instead of many priors from different experiments (Lindauer & Hutter, 2018).
C ONCLUSIONS AND F UTURE W ORK
We have proposed a novel BO variant, PrBO, that allows users to inject their expert knowledge into the optimization in the form of priors about which parts of the input space will yield the best performance.
These are different than standard priors over functions which are much less intuitive for users.
BO failed so far to leverage the experience of domain experts, not only causing inefﬁciency but also getting users away from applying BO approaches because they could not exploit their years of knowledge in optimizing their black-box functions.
PrBO addresses this issue and will therefore facilitate the adoption of BO.
We showed that PrBO is 12.12x more sample efﬁcient than state-of-the-art methods, and 10,000 × faster than random search, on a common suite of benchmarks and achieves a new state-of-the-art performance on a real-world hardware design application.
We also showed that PrBO converges faster and that it robustly recovers from misleading priors.
In future work, we will study how our approach can be used to leverage prior knowledge from meta-learning.
Bringing these two worlds together will likely boost the performance of BO even further.