$\pi$BO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization

1 INTRODUCTION . The optimization of expensive black-box functions is a prominent task , arising across a wide range of applications . Bayesian optimization ( BO ) is a sample-efficient approach to cope with this task , and has been successfully applied to various problem settings , including hyperparameter optimization ( HPO ) ( Snoek et al. , 2012 ) , neural architecture search ( NAS ) ( Ru et al. , 2021 ) , joint NAS and HPO ( Zimmer et al. , 2021 ) , algorithm configuration ( Hutter et al. , 2011 ) , hardware design ( Nardi et al. , 2019 ) , robotics Calandra et al . ( 2014 ) , and the game of Go ( Chen et al. , 2018 ) . Despite the demonstrated effectiveness of BO for HPO ( Bergstra et al. , 2011 ; Turner et al. , 2021 ) , its adoption among practitioners remains limited . In a survey covering NeurIPS 2019 and ICLR 2020 ( Bouthillier & Varoquaux , 2020 ) , manual search was shown to be the most prevalent tuning method , with BO accounting for less than 7 % of all tuning efforts . As the understanding of hyperparameter settings in deep learning ( DL ) models increase ( Smith , 2018 ) , so too does the tuning proficiency of practitioners ( Anand et al. , 2020 ) . As previously displayed ( Smith , 2018 ; Anand et al. , 2020 ; Souza et al. , 2021 ; Wang et al. , 2019 ) , this knowledge manifests in choosing single configurations or regions of hyperparameters that presumably yield good results , demonstrating a belief over the location of the optimum . BO ’ s deficit to properly incorporate said beliefs is a reason why practitioners prefer manual search to BO ( Wang et al. , 2019 ) , despite its documented shortcomings ( Bergstra & Bengio , 2012 ) . To improve the usefulness of automated HPO approaches for ML practictioners , the ability to incorporate such knowledge is pivotal . Well-established BO frameworks ( Snoek et al. , 2012 ; Hutter et al. , 2011 ; The GPyOpt authors , 2016 ; Kandasamy et al. , 2020 ; Balandat et al. , 2020 ) support user input to a limited extent , such as by biasing the initial design , or by narrowing the search space ; however , this type of hard prior can lead to poor performance by missing important regions . BO also supports a prior over functions p ( f ) via the Gaussian Process kernel . However , this option for injecting knowledge is not aligned with the knowledge that experts possess : they often know which ranges of hyperparameter values tend to work best ( Perrone et al. , 2019 ; Smith , 2018 ; Wang et al. , 2019 ) , and are able to specify a probability distribution to quantify these priors . For example , many users of the Adam optimizer ( Kingma & Ba , 2015 ) know that its best learning rate is often in the vicinity of 1× 10−3 . In practice , DL experiments are typically conducted in a low-budget setting of less than 50 full model trainings ( Bouthillier & Varoquaux , 2020 ) . As such , practitioners want to exploit their knowledge efficiently without wasting early model trainings on configurations they expect to likely perform poorly . Unfortunately , this suits standard BO poorly , as BO requires a moderate number of function evaluations to learn about the response surface and make informed decisions that outperform random search . While there is a demand to increase knowledge injection possibilities to further the adoption of BO , the concept of encoding prior beliefs over the location of an optimum is still rather novel : while there are some initial works ( Ramachandran et al. , 2020 ; Li et al. , 2020 ; Souza et al. , 2021 ) , no approach exists so far that allows the integration of arbitrary priors and offers flexibility in the choice of acquisition function ; theory is also lacking . We close this gap by introducing a novel , remarkably simple , approach for injecting arbitrary prior beliefs into BO that is easy to implement , agnostic to the surrogate model used and converges at standard BO rates for any choice of prior . Our contributions After discussing our problem setting , related work , and background ( Section 2 ) , we make the following contributions : 1 . We introduce πBO , a novel generalization of myopic acquisition functions that accounts for user-specified prior distributions over possible optima , is demonstrably simple-to-implement , and can be easily combined with arbitrary surrogate models ( Section 3.1 & 3.2 ) ; 2 . We formally prove that πBO inherits the theoretical properties of the well-established Expected Improvement acquisition function ( Section 3.3 ) ; 3 . We demonstrate on a broad range of established benchmarks and in DL case studies that πBO can yield 12.5× time-to-accuracy speedup over vanilla BO ( Section 4 ) . 2 BACKGROUND AND RELATED WORK . 2.1 BLACK-BOX OPTIMIZATION . We consider the problem of optimizing a black-box function f across a set of feasible inputs X ⊂ Rd : x∗ ∈ arg min x∈X f ( x ) . ( 1 ) We assume that f ( x ) is expensive to evaluate , and can potentially only be observed through a noisy estimate , y . In this setting , we wish to minimize f in an efficient manner , typically adhering to a budget which sets a cap on the number of points that can be evaluated . Black-Box Optimization with Probabilistic User Beliefs In our work , we consider an augmented version of the optimization problem in Eq . ( 1 ) , where we have access to user beliefs in the form of a probability distribution on the location of the optimum . Formally , we define the problem of black-box optimization with probabilistic user beliefs as solving Eq . ( 1 ) , given a user-specified prior probability on the location of the optimum defined as π ( x ) = P ( f ( x ) = min x′∈X f ( x′ ) ) , ( 2 ) where regions that the user expects to likely to contain an optimum will have a high value . We note that , without loss of generality , we require π to be strictly positive on all of X , i.e. , any point in the search space might be an optimum . Since the user belief π ( x ) can be inaccurate or even misleading , optimizing Eq . ( 1 ) given ( 2 ) is a challenging problem . 2.2 BAYESIAN OPTIMIZATION . We outline Bayesian optimization ( Mockus et al. , 1978 ; Brochu et al. , 2010 ; Shahriari et al. , 2016b ) . Model BO aims to globally minimize f by an initial experimental design D0 = { ( xi , yi ) } Mi=1 and thereafter sequentially deciding on new points xn to form the data Dn = Dn−1 ∪ { ( xn , yn ) } for the n-th iteration with n ∈ { 1 . . . N } . After each new observation , BO constructs a probabilistic surrogate model of f and uses that surrogate to evaluate an acquisition function α ( x , Dn ) . The combination of surrogate model and acquisition function encodes the policy for selecting the next point xn+1 . When constructing the surrogate , the most common choice is Gaussian processes ( Rasmussen & Williams , 2006 ) , which model f as p ( f |Dn ) = GP ( m , k ) , with prior mean m ( which is typically 0 ) and positive semi-definite covariance kernel k. The posterior mean mn and the variance s2n are mn ( x ) = kn ( x ) > ( Kn + σ 2 nI ) y , s 2 n ( x ) = k ( x , x ) − kn ( x ) > ( Kn + σ2nI ) kn ( x ) , ( 3 ) where ( Kn ) ij = k ( xi , xj ) , kn ( x ) = [ k ( x , x1 ) , . . . , k ( x , xn ) ] > and σ2n is the estimation of the observation noise variance σ2 . Alternative surrogate models include Random forests ( Hutter et al. , 2011 ) and Bayesian neural networks ( Springenberg et al. , 2016 ) . Acquisition Functions To obtain new candidates to evaluate , BO employs a criterion , called an acquisition function , that encapsulates an explore-exploit trade-off . By maximizing this criterion at each iteration , one or more candidate point are obtained and added to observed data . Several acquisition functions are used in BO ; the most common of these is Expected Improvement ( EI ) ( Jones et al. , 1998 ) . For a noiseless function , EI selects the next point xn+1 , where f∗n is the minimal objective function value observed by iteration n , as xn+1 ∈ arg max x∈X E [ [ ( f∗n − f ( x ) ] + ] = arg max x∈X Zsn ( x ) Φ ( Z ) + sn ( x ) φ ( Z ) , ( 4 ) where Z = ( f∗n −mn ( x ) ) /sn ( x ) . Thus , EI provides a myopic strategy for determining promising points ; it also comes with convergence guarantees ( Bull , 2011 ) . Similar myopic acquisition functions are Upper Confidence Bound ( UCB ) ( Srinivas et al. , 2012 ) , Probability of Improvement ( PI ) ( Jones , 2001 ; Kushner , 1964 ) and Thompson Sampling ( TS ) ( Thompson , 1933 ) . A different class of acquisition functions is based on non-myopic criteria , such as Entropy Search ( Hennig & Schuler , 2012 ) , Predictive Entropy Search ( Hernández-Lobato et al. , 2014 ) and Max-value Entropy Search ( Wang & Jegelka , 2017 ) , which select points to minimize the uncertainty about the optimum , and the Knowledge Gradient ( Frazier et al. , 2008 ) , which aims to minimize the posterior mean of the surrogate at the subsequent iteration . Our work applies to all acquisition functions in the first class , and we leave its extension to those in the second class for future work . 2.3 RELATED WORK . There are two main categories of approaches that exploit prior knowledge in BO : approaches that use records of previous experiments , and approaches that incorporate assumptions on the black-box function provided either directly or indirectly by the user . As πBO exploits prior knowledge from users , we briefly discuss approaches which utilize previous experiments , and then comprehensively discuss the literature on exploiting expert knowledge . Learning from Previous Experiments Transfer learning for BO aims to automatically extract and use knowledge from prior executions of BO . These executions can come , for example , from learning and optimizing the hyperparameters of a machine learning algorithm on different datasets ( van Rijn & Hutter , 2018 ; Swersky et al. , 2013 ; Wistuba et al. , 2015 ; Perrone et al. , 2019 ; Feurer et al. , 2015 ; 2018 ) , or from optimizing the hyperparameters at different development stages ( Stoll et al. , 2020 ) . For a comprehensive overview of meta learning for hyperparameter optimization , please see the survey from Vanschoren ( 2018 ) . In contrast to these transfer learning approaches , πBO and the related work discussed below does not hinge on the existence of previous experiments , and can therefore be applied more generally . Incorporating Expert Priors over Function Structure BO can leverage structural priors on how the objective function is expected to behave . Traditionally , this is done via the surrogate model ’ s prior over functions , e.g. , the kernel of the GP . However , there are lines of work that explore additional structural priors for BO to leverage . For instance , both SMAC ( Hutter et al. , 2011 ) and iRace ( LópezIbáñez et al. , 2016 ) support structural priors in the form of log-transformations , Li et al . ( 2018 ) propose to use knowledge about the monotonicity of the objective function as a prior for BO , and Snoek et al . ( 2014 ) model non-stationary covariance between inputs by warping said inputs . Oh et al . ( 2018 ) and Siivola et al . ( 2018 ) both propose structural priors tailored to high-dimensional problems , addressing the issue of over-exploring the boundary described by Swersky ( 2017 ) . Oh et al . ( 2018 ) propose a cylindrical kernel that expands the center of the search space and shrinks the edges , while Siivola et al . ( 2018 ) propose adding derivative signs to the edges of the search space to steer BO towards the center . Lastly , Shahriari et al . ( 2016a ) propose a BO algorithm for unbounded search spaces which uses a regularizer to penalize points based on their distance to the center of the user-defined search space . All of these approaches incorporate prior information on specific properties of the function or search space , and are thus not always applicable . Moreover , they do not generally direct the search to desired regions of the search space , offering the user little control over the selection of points to evaluate . Incorporating Expert Priors over Function Optimum Few previous works have proposed to inject explicit prior distributions over the location of an optimum into BO . In these cases , users explicitly define a prior that encodes their beliefs on where the optimum is more likely to be located . Bergstra et al . ( 2011 ) suggest an approach that supports prior beliefs from a fixed set of distributions . However , this approach can not be combined with standard acquisition functions . BOPrO ( Souza et al. , 2021 ) employs a similar structure that combines the user-provided prior distribution with a data-driven model into a pseudo-posterior . From the pseudo-posterior , configurations are selected using the EI acquisition function , using the formulation in Bergstra et al . ( 2011 ) . While BOPrO is able to recover from misleading priors , its design restricts it to only use EI . Moreover , it does not provide the convergence guarantees of πBO . Li et al . ( 2020 ) propose to infer a posterior conditioned on both the observed data and the user prior through repeated Thompson sampling and maximization under the prior . This method displays robustness against misleading priors but lacks in empirical performance . Additionally , it is restricted to only one specific acquisition function . Ramachandran et al . ( 2020 ) use the probability integral transform to warp the search space , stretching high-probability regions and shrinking others . While the approach is model- and acquisition function agnostic , it requires invertible priors , and does not empirically display the ability to recover from misleading priors . In Section 4 , we demonstrate that πBO compares favorably for priors over the function optimum , and shows improved empirical performance . Additionally , we do a complete comparison of all approaches in Appendix C. In summary , πBO sets itself apart from the methods above by being simpler ( and thus easier to implement in different frameworks ) , flexible with regard to different acquisition functions and different surrogate models , the availability of theoretical guarantees , and , as we demonstrate in Section 4 , better empirical results .

Summary: In this article the authors propose to incorporate domain knowledge on where good configurations are located in Bayesian optimization (BO). They do so by weighting the acquisition function by a prior that decays and reverts to uniform as iterations go. The asymptotic rate of convergence is shown to be of the same order as the non-prior version. Empirical tests are provided on various synthetic and  hyperparameter tuning tasks, with diverse baseline methods and ablation studies.