$\pi$BO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization

1 INTRODUCTION . The optimization of expensive black-box functions is a prominent task , arising across a wide range of applications . Bayesian optimization ( BO ) is a sample-efficient approach to cope with this task , and has been successfully applied to various problem settings , including hyperparameter optimization ( HPO ) ( Snoek et al. , 2012 ) , neural architecture search ( NAS ) ( Ru et al. , 2021 ) , joint NAS and HPO ( Zimmer et al. , 2021 ) , algorithm configuration ( Hutter et al. , 2011 ) , hardware design ( Nardi et al. , 2019 ) , robotics Calandra et al . ( 2014 ) , and the game of Go ( Chen et al. , 2018 ) . Despite the demonstrated effectiveness of BO for HPO ( Bergstra et al. , 2011 ; Turner et al. , 2021 ) , its adoption among practitioners remains limited . In a survey covering NeurIPS 2019 and ICLR 2020 ( Bouthillier & Varoquaux , 2020 ) , manual search was shown to be the most prevalent tuning method , with BO accounting for less than 7 % of all tuning efforts . As the understanding of hyperparameter settings in deep learning ( DL ) models increase ( Smith , 2018 ) , so too does the tuning proficiency of practitioners ( Anand et al. , 2020 ) . As previously displayed ( Smith , 2018 ; Anand et al. , 2020 ; Souza et al. , 2021 ; Wang et al. , 2019 ) , this knowledge manifests in choosing single configurations or regions of hyperparameters that presumably yield good results , demonstrating a belief over the location of the optimum . BO ’ s deficit to properly incorporate said beliefs is a reason why practitioners prefer manual search to BO ( Wang et al. , 2019 ) , despite its documented shortcomings ( Bergstra & Bengio , 2012 ) . To improve the usefulness of automated HPO approaches for ML practictioners , the ability to incorporate such knowledge is pivotal . Well-established BO frameworks ( Snoek et al. , 2012 ; Hutter et al. , 2011 ; The GPyOpt authors , 2016 ; Kandasamy et al. , 2020 ; Balandat et al. , 2020 ) support user input to a limited extent , such as by biasing the initial design , or by narrowing the search space ; however , this type of hard prior can lead to poor performance by missing important regions . BO also supports a prior over functions p ( f ) via the Gaussian Process kernel . However , this option for injecting knowledge is not aligned with the knowledge that experts possess : they often know which ranges of hyperparameter values tend to work best ( Perrone et al. , 2019 ; Smith , 2018 ; Wang et al. , 2019 ) , and are able to specify a probability distribution to quantify these priors . For example , many users of the Adam optimizer ( Kingma & Ba , 2015 ) know that its best learning rate is often in the vicinity of 1× 10−3 . In practice , DL experiments are typically conducted in a low-budget setting of less than 50 full model trainings ( Bouthillier & Varoquaux , 2020 ) . As such , practitioners want to exploit their knowledge efficiently without wasting early model trainings on configurations they expect to likely perform poorly . Unfortunately , this suits standard BO poorly , as BO requires a moderate number of function evaluations to learn about the response surface and make informed decisions that outperform random search . While there is a demand to increase knowledge injection possibilities to further the adoption of BO , the concept of encoding prior beliefs over the location of an optimum is still rather novel : while there are some initial works ( Ramachandran et al. , 2020 ; Li et al. , 2020 ; Souza et al. , 2021 ) , no approach exists so far that allows the integration of arbitrary priors and offers flexibility in the choice of acquisition function ; theory is also lacking . We close this gap by introducing a novel , remarkably simple , approach for injecting arbitrary prior beliefs into BO that is easy to implement , agnostic to the surrogate model used and converges at standard BO rates for any choice of prior . Our contributions After discussing our problem setting , related work , and background ( Section 2 ) , we make the following contributions : 1 . We introduce πBO , a novel generalization of myopic acquisition functions that accounts for user-specified prior distributions over possible optima , is demonstrably simple-to-implement , and can be easily combined with arbitrary surrogate models ( Section 3.1 & 3.2 ) ; 2 . We formally prove that πBO inherits the theoretical properties of the well-established Expected Improvement acquisition function ( Section 3.3 ) ; 3 . We demonstrate on a broad range of established benchmarks and in DL case studies that πBO can yield 12.5× time-to-accuracy speedup over vanilla BO ( Section 4 ) . 2 BACKGROUND AND RELATED WORK . 2.1 BLACK-BOX OPTIMIZATION . We consider the problem of optimizing a black-box function f across a set of feasible inputs X ⊂ Rd : x∗ ∈ arg min x∈X f ( x ) . ( 1 ) We assume that f ( x ) is expensive to evaluate , and can potentially only be observed through a noisy estimate , y . In this setting , we wish to minimize f in an efficient manner , typically adhering to a budget which sets a cap on the number of points that can be evaluated . Black-Box Optimization with Probabilistic User Beliefs In our work , we consider an augmented version of the optimization problem in Eq . ( 1 ) , where we have access to user beliefs in the form of a probability distribution on the location of the optimum . Formally , we define the problem of black-box optimization with probabilistic user beliefs as solving Eq . ( 1 ) , given a user-specified prior probability on the location of the optimum defined as π ( x ) = P ( f ( x ) = min x′∈X f ( x′ ) ) , ( 2 ) where regions that the user expects to likely to contain an optimum will have a high value . We note that , without loss of generality , we require π to be strictly positive on all of X , i.e. , any point in the search space might be an optimum . Since the user belief π ( x ) can be inaccurate or even misleading , optimizing Eq . ( 1 ) given ( 2 ) is a challenging problem . 2.2 BAYESIAN OPTIMIZATION . We outline Bayesian optimization ( Mockus et al. , 1978 ; Brochu et al. , 2010 ; Shahriari et al. , 2016b ) . Model BO aims to globally minimize f by an initial experimental design D0 = { ( xi , yi ) } Mi=1 and thereafter sequentially deciding on new points xn to form the data Dn = Dn−1 ∪ { ( xn , yn ) } for the n-th iteration with n ∈ { 1 . . . N } . After each new observation , BO constructs a probabilistic surrogate model of f and uses that surrogate to evaluate an acquisition function α ( x , Dn ) . The combination of surrogate model and acquisition function encodes the policy for selecting the next point xn+1 . When constructing the surrogate , the most common choice is Gaussian processes ( Rasmussen & Williams , 2006 ) , which model f as p ( f |Dn ) = GP ( m , k ) , with prior mean m ( which is typically 0 ) and positive semi-definite covariance kernel k. The posterior mean mn and the variance s2n are mn ( x ) = kn ( x ) > ( Kn + σ 2 nI ) y , s 2 n ( x ) = k ( x , x ) − kn ( x ) > ( Kn + σ2nI ) kn ( x ) , ( 3 ) where ( Kn ) ij = k ( xi , xj ) , kn ( x ) = [ k ( x , x1 ) , . . . , k ( x , xn ) ] > and σ2n is the estimation of the observation noise variance σ2 . Alternative surrogate models include Random forests ( Hutter et al. , 2011 ) and Bayesian neural networks ( Springenberg et al. , 2016 ) . Acquisition Functions To obtain new candidates to evaluate , BO employs a criterion , called an acquisition function , that encapsulates an explore-exploit trade-off . By maximizing this criterion at each iteration , one or more candidate point are obtained and added to observed data . Several acquisition functions are used in BO ; the most common of these is Expected Improvement ( EI ) ( Jones et al. , 1998 ) . For a noiseless function , EI selects the next point xn+1 , where f∗n is the minimal objective function value observed by iteration n , as xn+1 ∈ arg max x∈X E [ [ ( f∗n − f ( x ) ] + ] = arg max x∈X Zsn ( x ) Φ ( Z ) + sn ( x ) φ ( Z ) , ( 4 ) where Z = ( f∗n −mn ( x ) ) /sn ( x ) . Thus , EI provides a myopic strategy for determining promising points ; it also comes with convergence guarantees ( Bull , 2011 ) . Similar myopic acquisition functions are Upper Confidence Bound ( UCB ) ( Srinivas et al. , 2012 ) , Probability of Improvement ( PI ) ( Jones , 2001 ; Kushner , 1964 ) and Thompson Sampling ( TS ) ( Thompson , 1933 ) . A different class of acquisition functions is based on non-myopic criteria , such as Entropy Search ( Hennig & Schuler , 2012 ) , Predictive Entropy Search ( Hernández-Lobato et al. , 2014 ) and Max-value Entropy Search ( Wang & Jegelka , 2017 ) , which select points to minimize the uncertainty about the optimum , and the Knowledge Gradient ( Frazier et al. , 2008 ) , which aims to minimize the posterior mean of the surrogate at the subsequent iteration . Our work applies to all acquisition functions in the first class , and we leave its extension to those in the second class for future work . 2.3 RELATED WORK . There are two main categories of approaches that exploit prior knowledge in BO : approaches that use records of previous experiments , and approaches that incorporate assumptions on the black-box function provided either directly or indirectly by the user . As πBO exploits prior knowledge from users , we briefly discuss approaches which utilize previous experiments , and then comprehensively discuss the literature on exploiting expert knowledge . Learning from Previous Experiments Transfer learning for BO aims to automatically extract and use knowledge from prior executions of BO . These executions can come , for example , from learning and optimizing the hyperparameters of a machine learning algorithm on different datasets ( van Rijn & Hutter , 2018 ; Swersky et al. , 2013 ; Wistuba et al. , 2015 ; Perrone et al. , 2019 ; Feurer et al. , 2015 ; 2018 ) , or from optimizing the hyperparameters at different development stages ( Stoll et al. , 2020 ) . For a comprehensive overview of meta learning for hyperparameter optimization , please see the survey from Vanschoren ( 2018 ) . In contrast to these transfer learning approaches , πBO and the related work discussed below does not hinge on the existence of previous experiments , and can therefore be applied more generally . Incorporating Expert Priors over Function Structure BO can leverage structural priors on how the objective function is expected to behave . Traditionally , this is done via the surrogate model ’ s prior over functions , e.g. , the kernel of the GP . However , there are lines of work that explore additional structural priors for BO to leverage . For instance , both SMAC ( Hutter et al. , 2011 ) and iRace ( LópezIbáñez et al. , 2016 ) support structural priors in the form of log-transformations , Li et al . ( 2018 ) propose to use knowledge about the monotonicity of the objective function as a prior for BO , and Snoek et al . ( 2014 ) model non-stationary covariance between inputs by warping said inputs . Oh et al . ( 2018 ) and Siivola et al . ( 2018 ) both propose structural priors tailored to high-dimensional problems , addressing the issue of over-exploring the boundary described by Swersky ( 2017 ) . Oh et al . ( 2018 ) propose a cylindrical kernel that expands the center of the search space and shrinks the edges , while Siivola et al . ( 2018 ) propose adding derivative signs to the edges of the search space to steer BO towards the center . Lastly , Shahriari et al . ( 2016a ) propose a BO algorithm for unbounded search spaces which uses a regularizer to penalize points based on their distance to the center of the user-defined search space . All of these approaches incorporate prior information on specific properties of the function or search space , and are thus not always applicable . Moreover , they do not generally direct the search to desired regions of the search space , offering the user little control over the selection of points to evaluate . Incorporating Expert Priors over Function Optimum Few previous works have proposed to inject explicit prior distributions over the location of an optimum into BO . In these cases , users explicitly define a prior that encodes their beliefs on where the optimum is more likely to be located . Bergstra et al . ( 2011 ) suggest an approach that supports prior beliefs from a fixed set of distributions . However , this approach can not be combined with standard acquisition functions . BOPrO ( Souza et al. , 2021 ) employs a similar structure that combines the user-provided prior distribution with a data-driven model into a pseudo-posterior . From the pseudo-posterior , configurations are selected using the EI acquisition function , using the formulation in Bergstra et al . ( 2011 ) . While BOPrO is able to recover from misleading priors , its design restricts it to only use EI . Moreover , it does not provide the convergence guarantees of πBO . Li et al . ( 2020 ) propose to infer a posterior conditioned on both the observed data and the user prior through repeated Thompson sampling and maximization under the prior . This method displays robustness against misleading priors but lacks in empirical performance . Additionally , it is restricted to only one specific acquisition function . Ramachandran et al . ( 2020 ) use the probability integral transform to warp the search space , stretching high-probability regions and shrinking others . While the approach is model- and acquisition function agnostic , it requires invertible priors , and does not empirically display the ability to recover from misleading priors . In Section 4 , we demonstrate that πBO compares favorably for priors over the function optimum , and shows improved empirical performance . Additionally , we do a complete comparison of all approaches in Appendix C. In summary , πBO sets itself apart from the methods above by being simpler ( and thus easier to implement in different frameworks ) , flexible with regard to different acquisition functions and different surrogate models , the availability of theoretical guarantees , and , as we demonstrate in Section 4 , better empirical results .

Summary: ## Summary:  PiBO is a very straightforward paper that seeks to incorporate prior knowledge about the optimum to accelerate Bayesian optimization.   They achieve this by simply multiplying the acquisition function (Expected Improvement, in the paper) by the prior distribution, and then maximizing that. They then decay the prior in order to deal with mis-specified priors. The authors also provide some theory that this weighted acquisition function performs (for some particular definition of performance) no worse than EI times a constant that depends on the decay rate. It’s a reasonable idea; honestly I’m surprised no paper has tried this before.   Synthetic and real-world experiments indicate that: -With a well-specified prior, PiBO demonstrates superior performance over it’s basic EI counterpart.  -With a poorly-specified prior, PiBO performs poorly worse than it’s EI counterpart but is usually able to recover given enough iterations.