Prior-guided Bayesian Optimization

1 INTRODUCTION . Bayesian Optimization ( BO ) is a data-efficient method for the joint optimization of design choices that gained great popularity in recent years . It is impacting a wide range of areas , including hyperparameter optimization ( Snoek et al. , 2012 ; Falkner et al. , 2018 ) , AutoML ( Feurer et al. , 2015a ; Hutter et al. , 2018 ) , robotics ( Calandra et al. , 2016 ) , computer vision ( Nardi et al. , 2017 ; Bodin et al. , 2016 ) , environmental monitoring ( Marchant & Ramos , 2012 ) , combinatorial optimization ( Hutter et al. , 2011 ) , experimental design ( Azimi et al. , 2012 ) , RL ( Brochu et al. , 2010 ) , Computer Go ( Chen et al. , 2018 ) , hardware design ( Koeplinger et al. , 2018 ; Nardi et al. , 2019 ) and many others . It promises greater automation so as to increase both product quality and human productivity . As a result , BO is also established in many large tech companies , e.g. , with Google Vizier ( Golovin et al. , 2017 ) and Facebook BoTorch ( Balandat et al. , 2019 ) . Nevertheless domain experts often have substantial prior knowledge that standard BO can not incorporate . Users can incorporate prior knowledge by narrowing the search space ; however , this type of hard prior can lead to poor performance by missing important regions . BO also supports a prior over functions p ( f ) , e.g. , via a kernel function . However , this is not the prior experts have : users often know which ranges of hyperparameters tend to work best , and are able to specify a probability distribution pbest ( x ) to quantify these priors . E.g. , many users of the Adam optimizer ( Kingma & Ba , 2015 ) know that its best learning rate is often in the vicinity of 1e-3 ( give or take one order of magnitude ) , yet one may not know the accuracy one may achieve in a new application . Similarly , Navruzyan et al . ( 2019 ) derived neural network hyperparameter priors for image datasets based on their experience with five datasets . In these cases , users know potentially good values for a new application , but can not be certain about them . As a result , many competent users instead revert to manual search , which can fully incorporate their prior knowledge . A recent survey showed that most NeurIPS 2019 and ICLR 2020 papers that reported having tuned hyperparameters used manual search , with only a very small fraction using BO ( Bouthillier & Varoquaux , 2020 ) . In order for BO to be adopted widely , and help facilitate faster progress in the ML community by tuning hyperparameters faster and better , it is therefore crucial to devise a method that fully incorporates expert knowledge into BO . In this paper , we introduce Prior-guided Bayesian Optimization ( PrBO ) , a novel BO variant that combines user prior knowledge with a probabilistic model of the observations made . Our technical contributions with PrBO are : 1 . PrBO bridges the TPE methodology and standard BO probabilistic models , such as GPs , RFs or Bayesian NNs , instead of Tree Parzen Estimators only . 2 . PrBO is flexible w.r.t . how the prior is defined , allowing previously hard-to-inject ( e.g . exponential ) priors . 3 . PrBO gives more importance to the model as iterations progress , gradually forgetting the prior and ensuring robustness against misleading priors . We demonstrate the effectiveness of PrBO on a comprehensive set of real-world applications and synthetic benchmarks , showing that accurate prior knowledge helps PrBO to achieve similar performance to current state-of-the-art on average 12.12× faster on synthetic benchmarks and 1.49× faster on a real-world application . PrBO also achieves equal or better final performance in all but one of the benchmarks tested . 2 BACKGROUND . 2.1 BAYESIAN OPTIMIZATION . Bayesian Optimization ( BO ) is an approach for optimizing an unknown function f : X → R that is expensive to evaluate over an input space X . In this paper , we aim to minimize f , i.e. , find x∗ ∈ arg minx∈X f ( x ) . BO approximates x ∗ with an optimal sequence of evaluations x1 , x2 , . . . ∈ X , with each new xn+1 depending on the previous function values y1 , y2 , . . . , yn at x1 , . . . , xn . BO achieves this by building a posterior on f based on the set of evaluated points . At each BO iteration , a new point is selected and evaluated based on the posterior , and the posterior is updated to include the new point ( xn+1 , yn+1 ) . The points explored by BO are dictated by the acquisition function , which attributes a value to each x ∈ X by balancing the predicted value and uncertainty of the prediction for each x . In this work , as the acquisition function we choose Expected Improvement ( EI ) ( Mockus et al. , 1978 ) , which quantifies the expected improvement over the best function value found so far : EIfinc ( x ) : =∫ inf − inf max ( finc − y , 0 ) p ( y|x ) dy , where finc is the incumbent function value , i.e. , the best objective function value found so far , and p ( y|x ) is a probabilistic model , e.g. , a GP . Alternatives to EI would be Probability of Improvement ( PI ) ( Jones , 2001 ) , upper-confidence bounds ( UCB ) ( Srinivas et al. , 2010 ) , and entropy-based methods ( e.g . Hernández-Lobato et al . ( 2014 ) ) . 2.2 TREE-STRUCTURED PARZEN ESTIMATOR . Whereas the standard probabilistic model in BO directly models p ( y|x ) , the Tree-structured Parzen Estimator ( TPE ) approach of Bergstra et al . ( 2011 ) models p ( x|y ) and p ( y ) instead1 . This is done by constructing two parametric densities , g ( x ) and l ( x ) , which are computed using the observations with function value above and below a given threshold , respectively . The separating threshold y∗ is defined as a quantile of the observed function values . TPE uses the densities g ( x ) and l ( x ) to define p ( x|y ) as : p ( x|y ) = l ( x ) I ( y < y∗ ) + g ( x ) ( 1− I ( y < y∗ ) ) , ( 1 ) where I ( y < y∗ ) is 1 when y < y∗ and 0 otherwise . The parametrization of the generative model p ( x , y ) = p ( x|y ) p ( y ) facilitates the computation of EI as it leads to EIy∗ ( x ) ∝ l ( x ) /g ( x ) and , thus , arg maxx∈X EIy∗ ( x ) = arg maxx∈X l ( x ) /g ( x ) . 3 BAYESIAN OPTIMIZATION WITH PRIORS . We propose a BO approach dubbed PrBO that allows field experts to inject user prior knowledge into the optimization in the form of priors . PrBO combines this user-defined prior with a probabilistic 1Note that , technically , the model does not parameterize p ( y ) , since it is computed based on the observed data points , which are heavily biased towards low values due to the optimization process . Instead , it parameterizes a dynamically changing p ( yi ) ti=1 , which helps to constantly challenge the model to yield better observations . model that captures the likelihood of the observed data ( xi , yi ) ti=1 . PrBO is independent from the probabilistic model being used , i.e. , it can be freely combined with , e.g. , GPs , RFs , or Bayesian Neural Networks . 3.1 PRIORS . PrBO allows users to inject prior knowledge into BO . This is done via a prior distribution that informs where in the input space X we expect to find good f ( x ) values . A point is considered “ good ” if it leads to low function values . We denote the prior distribution Pg ( x ) , where g denotes that this is a prior on good points and x ∈ X is a given point . Similarly , we define a prior on where in the input space we expect to have “ bad ” points . Although we could have a separate user-defined probability distribution Pb ( x ) , we aimed to keep the load on users low and thus , for simplicity , only require the definition of Pg ( x ) and compute Pb ( x ) = 1 − Pg ( x ) .2 Pg ( x ) is normalized to [ 0 , 1 ] by min-max scaling before computing Pb ( x ) . In practice , x contains several dimensions but it is difficult for domain experts to provide a multivariate distribution Pg ( x ) . Users can easily specify , e.g. , draw , a univariate or bivariate probability distribution for continuous dimensions or provide a list of probabilities for discrete dimensions . In PrBO , users are free to define a complex multivariate distribution , but we expect the standard use case to be that users only want to specify univariate distributions , implicitly assuming a prior that factors as Pg ( x ) = ∏D i=1 Pg ( xi ) , where D is the number of dimensions in X and xi is the i-th input dimension of X . We show examples of continuous and discrete priors in Appendices B and F , respectively . We use factorized priors in our experiments to mimic what we expect most users will provide . In Appendix G , we show that these factorized priors lead to similar performance compared to multivariate priors . 3.2 MODEL . Whereas the standard probabilistic model in BO , e.g. , a GP , quantifies p ( y|x ) directly , that model is hard to combine with the user-defined prior Pg ( x ) . We therefore now introduce a method to translate the standard probabilistic model p ( y|x ) into a model that is easier to combine with this prior . Similar to the TPE work described in Sec . 2.2 , our generative model combines p ( x|y ) and p ( y ) instead of directly modeling p ( y|x ) . The computation we perform for this translation is to quantify the probability that a given input x is “ good ” under our standard probabilistic model p ( y|x ) . As in TPE , we define configurations as “ good ” if their observed y-value is below a certain quantile γ of the observed function values ( so that p ( y < fγ ) = γ ) . We in addition exploit the fact that our standard probabilistic model p ( y|x ) has a Gaussian form , and under this Gaussian prediction we can compute the probabilityMg ( x ) of the function value lying below a certain quantile using the standard closed-form formula for PI ( Kushner , 1964 ) : Mg ( x ) = p ( f ( x ) < fγ |x , ( xi , yi ) ti=1 ) = Φ ( fγ − µx σx ) , ( 2 ) where ( xi , yi ) ti=1 are the evaluated configurations and their function value , µx and σx are the mean and standard deviation of the probabilistic model at x , and Φ is the standard normal CDF , see Figure 1 . Note that there are two probabilistic models here : • The standard probabilistic model of BO , with a prior over functions p ( f ) , updated by data ( xi , yi ) t i=1 to yield a posterior over functions p ( f | ( xi , yi ) ti=1 ) , allowing us to quantify the probabilityMg ( x ) = p ( f ( x ) < fγ |x , ( xi , yi ) ti=1 ) in Equation 2 • The TPE-like generative model that combines p ( y ) and p ( x|y ) instead of directly modelling p ( y|x ) . Equation 2 bridges these two models by using the probability of improvement from BO ’ s standard model as the probabilityMg ( x ) in TPE ’ s model . Ultimately , this is a heuristic since there is no formal 2We note that for continuous spaces , this Pb ( x ) is not a probability distribution , and therefore only a pseudoprior , as it does not integrate to 1 . For discrete spaces , we normalize Pb ( x ) so that it sums to 1 and therefore is a proper probability distribution and prior . Figure 1 : Our model is composed by a probabilistic model and the probability of improving over the threshold fγ , i.e. , right tail of the Gaussian . The black curve is the probabilistic model ’ s mean and the shaded area is the model ’ s variance . Algorithm 1 PrBO Algorithm . D keeps track of all function evaluations so far : ( xi , yi ) ti=1 . 1 : Input : Input space X , user-defined prior distributions Pg ( x ) and Pb ( x ) , quantile γ and BO budget B . 2 : Output : Optimized point xinc . 3 : D ← Initialize ( X ) 4 : for t = 1 to B do 5 : Mg ( x ) ← fit_model_good ( D ) 6 : Mb ( x ) ← fit_model_bad ( D ) 7 : g ( x ) ← Pg ( x ) · Mg ( x ) t β 8 : b ( x ) ← Pb ( x ) · Mb ( x ) t β 9 : xt ∈ arg maxx∈X EIfγ ( x ) 10 : yt ← f ( xt ) 11 : D = D ∪ ( xt , yt ) 12 : end for 13 : xinc ← ComputeBest ( D ) 14 : return xinc connection between the two probabilistic models . However , we believe that the use of BO ’ s familiar , theoretically sound framework of probabilistic modelling of p ( y|x ) , followed by the computation of the familiar PI formula is a very intuitive choice for obtaining the probability of an input achieving at least a given performance threshold – exactly the term we need for TPE ’ sMg ( x ) . Similarly , we also define a probabilityMb ( x ) of x being bad asMb ( x ) = 1−Mg ( x ) .

Summary: This paper incorporates a prior distribution given by experts into Bayesian optimization (BO), to leverage useful human knowledge to accelerate BO. The algorithm uses an intuitive approach to combine the prior with the probabilistic surrogate model of BO to derive a pseudo-posterior, which naturally leads the EI acquisition function. As the BO progresses, the prior information is gradually overwhelmed by the observed data, which ensures the asymptotically correct behaviour.