Prior-guided Bayesian Optimization

1 INTRODUCTION . Bayesian Optimization ( BO ) is a data-efficient method for the joint optimization of design choices that gained great popularity in recent years . It is impacting a wide range of areas , including hyperparameter optimization ( Snoek et al. , 2012 ; Falkner et al. , 2018 ) , AutoML ( Feurer et al. , 2015a ; Hutter et al. , 2018 ) , robotics ( Calandra et al. , 2016 ) , computer vision ( Nardi et al. , 2017 ; Bodin et al. , 2016 ) , environmental monitoring ( Marchant & Ramos , 2012 ) , combinatorial optimization ( Hutter et al. , 2011 ) , experimental design ( Azimi et al. , 2012 ) , RL ( Brochu et al. , 2010 ) , Computer Go ( Chen et al. , 2018 ) , hardware design ( Koeplinger et al. , 2018 ; Nardi et al. , 2019 ) and many others . It promises greater automation so as to increase both product quality and human productivity . As a result , BO is also established in many large tech companies , e.g. , with Google Vizier ( Golovin et al. , 2017 ) and Facebook BoTorch ( Balandat et al. , 2019 ) . Nevertheless domain experts often have substantial prior knowledge that standard BO can not incorporate . Users can incorporate prior knowledge by narrowing the search space ; however , this type of hard prior can lead to poor performance by missing important regions . BO also supports a prior over functions p ( f ) , e.g. , via a kernel function . However , this is not the prior experts have : users often know which ranges of hyperparameters tend to work best , and are able to specify a probability distribution pbest ( x ) to quantify these priors . E.g. , many users of the Adam optimizer ( Kingma & Ba , 2015 ) know that its best learning rate is often in the vicinity of 1e-3 ( give or take one order of magnitude ) , yet one may not know the accuracy one may achieve in a new application . Similarly , Navruzyan et al . ( 2019 ) derived neural network hyperparameter priors for image datasets based on their experience with five datasets . In these cases , users know potentially good values for a new application , but can not be certain about them . As a result , many competent users instead revert to manual search , which can fully incorporate their prior knowledge . A recent survey showed that most NeurIPS 2019 and ICLR 2020 papers that reported having tuned hyperparameters used manual search , with only a very small fraction using BO ( Bouthillier & Varoquaux , 2020 ) . In order for BO to be adopted widely , and help facilitate faster progress in the ML community by tuning hyperparameters faster and better , it is therefore crucial to devise a method that fully incorporates expert knowledge into BO . In this paper , we introduce Prior-guided Bayesian Optimization ( PrBO ) , a novel BO variant that combines user prior knowledge with a probabilistic model of the observations made . Our technical contributions with PrBO are : 1 . PrBO bridges the TPE methodology and standard BO probabilistic models , such as GPs , RFs or Bayesian NNs , instead of Tree Parzen Estimators only . 2 . PrBO is flexible w.r.t . how the prior is defined , allowing previously hard-to-inject ( e.g . exponential ) priors . 3 . PrBO gives more importance to the model as iterations progress , gradually forgetting the prior and ensuring robustness against misleading priors . We demonstrate the effectiveness of PrBO on a comprehensive set of real-world applications and synthetic benchmarks , showing that accurate prior knowledge helps PrBO to achieve similar performance to current state-of-the-art on average 12.12× faster on synthetic benchmarks and 1.49× faster on a real-world application . PrBO also achieves equal or better final performance in all but one of the benchmarks tested . 2 BACKGROUND . 2.1 BAYESIAN OPTIMIZATION . Bayesian Optimization ( BO ) is an approach for optimizing an unknown function f : X → R that is expensive to evaluate over an input space X . In this paper , we aim to minimize f , i.e. , find x∗ ∈ arg minx∈X f ( x ) . BO approximates x ∗ with an optimal sequence of evaluations x1 , x2 , . . . ∈ X , with each new xn+1 depending on the previous function values y1 , y2 , . . . , yn at x1 , . . . , xn . BO achieves this by building a posterior on f based on the set of evaluated points . At each BO iteration , a new point is selected and evaluated based on the posterior , and the posterior is updated to include the new point ( xn+1 , yn+1 ) . The points explored by BO are dictated by the acquisition function , which attributes a value to each x ∈ X by balancing the predicted value and uncertainty of the prediction for each x . In this work , as the acquisition function we choose Expected Improvement ( EI ) ( Mockus et al. , 1978 ) , which quantifies the expected improvement over the best function value found so far : EIfinc ( x ) : =∫ inf − inf max ( finc − y , 0 ) p ( y|x ) dy , where finc is the incumbent function value , i.e. , the best objective function value found so far , and p ( y|x ) is a probabilistic model , e.g. , a GP . Alternatives to EI would be Probability of Improvement ( PI ) ( Jones , 2001 ) , upper-confidence bounds ( UCB ) ( Srinivas et al. , 2010 ) , and entropy-based methods ( e.g . Hernández-Lobato et al . ( 2014 ) ) . 2.2 TREE-STRUCTURED PARZEN ESTIMATOR . Whereas the standard probabilistic model in BO directly models p ( y|x ) , the Tree-structured Parzen Estimator ( TPE ) approach of Bergstra et al . ( 2011 ) models p ( x|y ) and p ( y ) instead1 . This is done by constructing two parametric densities , g ( x ) and l ( x ) , which are computed using the observations with function value above and below a given threshold , respectively . The separating threshold y∗ is defined as a quantile of the observed function values . TPE uses the densities g ( x ) and l ( x ) to define p ( x|y ) as : p ( x|y ) = l ( x ) I ( y < y∗ ) + g ( x ) ( 1− I ( y < y∗ ) ) , ( 1 ) where I ( y < y∗ ) is 1 when y < y∗ and 0 otherwise . The parametrization of the generative model p ( x , y ) = p ( x|y ) p ( y ) facilitates the computation of EI as it leads to EIy∗ ( x ) ∝ l ( x ) /g ( x ) and , thus , arg maxx∈X EIy∗ ( x ) = arg maxx∈X l ( x ) /g ( x ) . 3 BAYESIAN OPTIMIZATION WITH PRIORS . We propose a BO approach dubbed PrBO that allows field experts to inject user prior knowledge into the optimization in the form of priors . PrBO combines this user-defined prior with a probabilistic 1Note that , technically , the model does not parameterize p ( y ) , since it is computed based on the observed data points , which are heavily biased towards low values due to the optimization process . Instead , it parameterizes a dynamically changing p ( yi ) ti=1 , which helps to constantly challenge the model to yield better observations . model that captures the likelihood of the observed data ( xi , yi ) ti=1 . PrBO is independent from the probabilistic model being used , i.e. , it can be freely combined with , e.g. , GPs , RFs , or Bayesian Neural Networks . 3.1 PRIORS . PrBO allows users to inject prior knowledge into BO . This is done via a prior distribution that informs where in the input space X we expect to find good f ( x ) values . A point is considered “ good ” if it leads to low function values . We denote the prior distribution Pg ( x ) , where g denotes that this is a prior on good points and x ∈ X is a given point . Similarly , we define a prior on where in the input space we expect to have “ bad ” points . Although we could have a separate user-defined probability distribution Pb ( x ) , we aimed to keep the load on users low and thus , for simplicity , only require the definition of Pg ( x ) and compute Pb ( x ) = 1 − Pg ( x ) .2 Pg ( x ) is normalized to [ 0 , 1 ] by min-max scaling before computing Pb ( x ) . In practice , x contains several dimensions but it is difficult for domain experts to provide a multivariate distribution Pg ( x ) . Users can easily specify , e.g. , draw , a univariate or bivariate probability distribution for continuous dimensions or provide a list of probabilities for discrete dimensions . In PrBO , users are free to define a complex multivariate distribution , but we expect the standard use case to be that users only want to specify univariate distributions , implicitly assuming a prior that factors as Pg ( x ) = ∏D i=1 Pg ( xi ) , where D is the number of dimensions in X and xi is the i-th input dimension of X . We show examples of continuous and discrete priors in Appendices B and F , respectively . We use factorized priors in our experiments to mimic what we expect most users will provide . In Appendix G , we show that these factorized priors lead to similar performance compared to multivariate priors . 3.2 MODEL . Whereas the standard probabilistic model in BO , e.g. , a GP , quantifies p ( y|x ) directly , that model is hard to combine with the user-defined prior Pg ( x ) . We therefore now introduce a method to translate the standard probabilistic model p ( y|x ) into a model that is easier to combine with this prior . Similar to the TPE work described in Sec . 2.2 , our generative model combines p ( x|y ) and p ( y ) instead of directly modeling p ( y|x ) . The computation we perform for this translation is to quantify the probability that a given input x is “ good ” under our standard probabilistic model p ( y|x ) . As in TPE , we define configurations as “ good ” if their observed y-value is below a certain quantile γ of the observed function values ( so that p ( y < fγ ) = γ ) . We in addition exploit the fact that our standard probabilistic model p ( y|x ) has a Gaussian form , and under this Gaussian prediction we can compute the probabilityMg ( x ) of the function value lying below a certain quantile using the standard closed-form formula for PI ( Kushner , 1964 ) : Mg ( x ) = p ( f ( x ) < fγ |x , ( xi , yi ) ti=1 ) = Φ ( fγ − µx σx ) , ( 2 ) where ( xi , yi ) ti=1 are the evaluated configurations and their function value , µx and σx are the mean and standard deviation of the probabilistic model at x , and Φ is the standard normal CDF , see Figure 1 . Note that there are two probabilistic models here : • The standard probabilistic model of BO , with a prior over functions p ( f ) , updated by data ( xi , yi ) t i=1 to yield a posterior over functions p ( f | ( xi , yi ) ti=1 ) , allowing us to quantify the probabilityMg ( x ) = p ( f ( x ) < fγ |x , ( xi , yi ) ti=1 ) in Equation 2 • The TPE-like generative model that combines p ( y ) and p ( x|y ) instead of directly modelling p ( y|x ) . Equation 2 bridges these two models by using the probability of improvement from BO ’ s standard model as the probabilityMg ( x ) in TPE ’ s model . Ultimately , this is a heuristic since there is no formal 2We note that for continuous spaces , this Pb ( x ) is not a probability distribution , and therefore only a pseudoprior , as it does not integrate to 1 . For discrete spaces , we normalize Pb ( x ) so that it sums to 1 and therefore is a proper probability distribution and prior . Figure 1 : Our model is composed by a probabilistic model and the probability of improving over the threshold fγ , i.e. , right tail of the Gaussian . The black curve is the probabilistic model ’ s mean and the shaded area is the model ’ s variance . Algorithm 1 PrBO Algorithm . D keeps track of all function evaluations so far : ( xi , yi ) ti=1 . 1 : Input : Input space X , user-defined prior distributions Pg ( x ) and Pb ( x ) , quantile γ and BO budget B . 2 : Output : Optimized point xinc . 3 : D ← Initialize ( X ) 4 : for t = 1 to B do 5 : Mg ( x ) ← fit_model_good ( D ) 6 : Mb ( x ) ← fit_model_bad ( D ) 7 : g ( x ) ← Pg ( x ) · Mg ( x ) t β 8 : b ( x ) ← Pb ( x ) · Mb ( x ) t β 9 : xt ∈ arg maxx∈X EIfγ ( x ) 10 : yt ← f ( xt ) 11 : D = D ∪ ( xt , yt ) 12 : end for 13 : xinc ← ComputeBest ( D ) 14 : return xinc connection between the two probabilistic models . However , we believe that the use of BO ’ s familiar , theoretically sound framework of probabilistic modelling of p ( y|x ) , followed by the computation of the familiar PI formula is a very intuitive choice for obtaining the probability of an input achieving at least a given performance threshold – exactly the term we need for TPE ’ sMg ( x ) . Similarly , we also define a probabilityMb ( x ) of x being bad asMb ( x ) = 1−Mg ( x ) .

Summary: The goal of this paper is to enable the introduction of prior expert knowledge in Bayesian optimization. This is performed by defining a prior distribution for the optimal value, which is included in the pseudo-posterior used to select new points by Expected Improvement. The number of iterations it takes to overcome a potentially wrong prior information is controlled by a parameter, whose sensitivity is studied. Extensive experiments are conducted on toy examples as well are more realistic hyper-parameter test cases.