Constrained Discrete Black-Box Optimization using Mixed-Integer Programming

1 INTRODUCTION . The problem of optimizing an expensive black-box function f : Ω 7→ R over a discrete , constrained domain arises in numerous application domains , e.g . neural architecture search ( Zoph & Le , 2017 ) , program synthesis ( Summers , 1977 ; Biermann , 1978 ) , small-molecule design ( Elton et al. , 2019 ) , and protein design ( Yang et al. , 2019 ) . In such resource-constrained settings , it is desirable to develop algorithms that exploit known combinatorial structure in Ω to search the space more efficiently . Model-based Black-box Optimization ( MBO ) , a popular paradigm that includes Bayesian Optimization as a special case , iteratively refines a function approximator f̂ ( x ) ≈ f ( x ) and selects new points to query by optimizing an acquisition function a ( x ) derived from a point estimate or posterior distribution over f̂ ( Section 2.1 ) . This inner-loop optimization problem is assumed to be easier than the original , since , for example , a ( x ) is less expensive to query than f ( x ) or provides “ white-box ” properties such as gradients . There is a vast literature addressing the challenges of applying MBO in practice . We focus on two of these : first , optimizing a ( x ) may itself be a computationally-difficult optimization problem ; second , in many applications , practitioners are confronted by additional constraints on x . For example , in neural architecture search , x might represent a computation graph that must be both connected and acyclic . Due to the difficulty in optimizing the acquisition function over a combinatorial domain , most approaches resort to heuristic inner-loop solvers , which often need to be specialized to the problem at hand to ensure feasibility , e.g . evolutionary solvers with custom mutation operators . To address the challenge of inner-loop optimization , we introduce a new MBO framework , NN+MILP , that exactly solves the acquisition problem using mixed-integer linear programming ( MILP ) . Crucially , by framing the inner-loop optimization as an MILP , our approach can flexibly incorporate a wide variety of logical , combinatorial , and polyhedral constraints , which need only be provided in a declarative sense . Using MILP in the inner loop does restrict the functional form of f̂ ( or the acquisition function based on it ) , but it supports any piecewise linear function . In particular , we employ the class of neural network ( NN ) approximators with ReLU activation functions due to their scalability and accuracy in practice , and because we can draw on recent work improving the performance of MILP for optimizing such NNs with respect to their inputs ( Anderson et al. , 2020 ) . For us , MILP is practical to use in the inner loop because the dimensionality of typical black-box optimization problems is orders of magnitude smaller than those usually considered by MILP solvers . Our contributions are as follows : • We introduce NN+MILP , an MBO framework for discrete black-box problems with NN surro- gates and exact optimality guarantees for solving the acquisition problem . • We observe in our experiments that the runtime of MILP is practical for use with black-box prob- lems of real-world scale , often solving the inner acquisition problem in seconds using standard packages and hardware . Moreover , MILP provides a simple declarative language for problemspecific constraints . • We show that NN+MILP matches or surpasses the performance of strong MBO baselines based on problem-specific evolutionary algorithms on a wide range of synthetic and real-world discrete black-box problems . • We use the NAS-Bench-101 neural architecture search benchmark as a case study , presenting a novel MILP formulation of its graph-structured domain . 2 BACKGROUND AND RELATED WORK . 2.1 MODEL-BASED BLACK-BOX OPTIMIZATION . Model-based Black-box Optimization ( MBO ) is a broad family of methods that includes Bayesian optimization as a special case ( Mockus et al. , 1978 ; Jones et al. , 1998 ; Hutter et al. , 2011 ; Snoek et al. , 2012 ; Shahriari et al. , 2015 ) . As depicted in Algorithm 1 , the method proposes xt at iteration t using three steps . First , the user performs inference over a surrogate model f̂ to approximate f using the data previously collected from the black-box function . Here , fit ( ) may return a point estimate for f̂ , a posterior distribution over f̂ , or a posterior predictive distribution . Next , an acquisition function a ( x ) based on f̂ ( x ) is posed that quantifies the quality of new points to query . Finally , xt is selected as the best point found by solving the acquisition problem , where an inner-loop solver ( approximately ) optimizes a ( x ) . The acquisition problem is typically designed such that it is more approachable than directly solving the original problem . For example , a ( x ) may be orders of magnitude less expensive to evaluate or have a tractable functional form . Practitioners can encode prior knowledge about the structure of f via a choice of inductive bias for f̂ , e.g . a suitable Gaussian Process kernel or neural-network architecture . Bayesian optimization performs Bayesian inference over f̂ and employs an acquisition function that accounts for uncertainty in f̂ . Doing so provides principled mechanisms for balancing exploration and exploitation ( Mockus et al. , 1978 ; Srinivas et al. , 2010 ) and is particularly important in early rounds of optimization when models are fit on limited data . We refer to our method as an instance of MBO , not Bayesian optimization , because it does not assume formal Bayesian inference for f̂ . Gaussian processes ( GPs ) are often used for f̂ in Bayesian optimization , since they provide closedform posterior inference , naturally adjust their expressivity as the dataset grows , and users can inject domain knowledge via a choice of kernel ( Rasmussen & Williams , 2006 ; Oh et al. , 2019 ) . On the other hand , neural networks provide a practical alternative ( Snoek et al. , 2015 ; Hernández-Lobato et al. , 2017 ) , since they often scale more gracefully , either computationally or statistically , to large datasets or high-dimensional domains . 2.2 SOLVING THE DISCRETE MBO ACQUISITION PROBLEM . In general , the inner-loop problem is itself a non-trivial global optimization problem . Prior work on discrete MBO has mainly employed local search solvers , such as evolutionary search , with limited guarantees ( Hutter et al. , 2011 ; Müller , 2016 ; Oh et al. , 2019 ; Kandasamy et al. , 2020 ) . A key advantage of such solvers is that they treat a ( x ) as a black box , which provides practitioners with freedom when designing application-specific surrogate models . On the other hand , particular choices of surrogate model and acquisition function lead to acquisition problems that can be ( approximately ) solved using specialized combinatorial solvers ( Baptista & Poloczek , 2018 ; Deshwal et al. , 2020 ) , mixed-integer nonlinear programming ( MINLP ) solvers ( Costa & Nannicini , 2018 ; Kim & Boukouvala , 2020 ) , or continuous optimization solvers ( Bliek et al. , 2021 ) . Algorithm 1 MBO Input : hypothesis class F , budget N , initial dataset Dn = { xi , f ( xi ) } ni=1 , optimization domain Ω for t = n+ 1 to t = N do P ( f̂t ) ← fit ( F , Dt−1 ) a ( x ) ← get acquis func ( P ( f̂t ) ) xt ← inner solver ( a ( x ) , Ω ) Dt ← Dt−1 ∪ { xt , f ( xt ) } end for return arg max ( xt , yt ) ∈DN yt Algorithm 2 NN+MILP Input : hypothesis classF , budgetN , initial dataset Dn = { xi , f ( xi ) } ni=1 , MILP domain formulation MΩ for t = n+ 1 to t = N do f̂t ← fit ( F , Dt−1 ) ( 3.2 ) Mt ← build milp ( f̂t , MΩ , Dt−1 ) ( 3.3 ) xt ← optimize ( Mt ) ( generic MILP solver ) Dt ← Dt−1 ∪ { xt , f ( xt ) } end for return arg max ( xt , yt ) ∈DN yt Therefore , practitioners must decide between either introducing difficult-to-analyze approximations due to inexact heuristic solvers or using tractable surrogate models that may be mis-specified for the application domain . This serves as a key motivation for our work : we seek to enable practitioners to employ broad families of surrogate models and exactly solve the acquisition problem with reasonable computational overhead in practice . 2.3 CONSTRAINED MBO . In many applications , x is subject to non-trivial structural constraints . Prior work has largely focused on the case where determining whether x is feasible requires evaluating an expensive , perhaps noisy , black-box function h ( x ) with cost comparable to f ( x ) ( Schonlau et al. , 1998 ; Gelbart et al. , 2014 ; Hernández-Lobato et al. , 2016 ; Ariafar et al. , 2019 ; Letham et al. , 2019 ) . Here , standard acquisition functions can be extended to account for an additional classifier ĥ ( x ) trained to predict h ( x ) . Problems with inexpensive white-box h ( x ) can be tackled using these approaches for black-box constraints , but doing so may lead to slower optimization and may query f ( x ) at invalid x , which can be unsafe when performing physical experiments ( Berkenkamp et al. , 2016 ) . Instead , the inner-loop solver can be modified directly to guarantee feasibility , e.g. , by using rejection sampling ( Shi et al. , 2020 ; Kandasamy et al. , 2020 ) . If using local search algorithms , the solver would need to be customized for each family of constraints , a task usually left to the user . Prior work employing MINLP solvers addresses white-box constraints either by adding a penalty for constraint violation ( Costa & Nannicini , 2018 ) or in small-scale settings ( Kim & Boukouvala , 2020 ) . 2.4 MIXED INTEGER LINEAR PROGRAMMING . Mixed Integer Linear Programming ( MILP ) seeks to maximize a linear function over a set of decision variables , some of which may be integral , subject to linear inequality constraints . Decades of development have allowed MILP to have a significant impact in a wide range of applications due to its better-than-expected computational performance ( Jünger et al. , 2010 ) . Indeed , while MILP problems are computationally hard ( NP-complete ) , they are routinely solved ( to global or near-global optimality ) in production environments thanks to state-of-the-art solvers that nearly double their machine-independent performance every year ( Achterberg & Wunderling , 2013 ; Bixby , 2012 ) . A notable aspect of MILP is that it provides a simple yet extremely versatile declarative language for white-box constraints . It is well known that linear inequalities over integer variables can be used to easily build pure-integer formulations for logical constraints and combinatorial optimization problems ( Williams , 2013 ; Schrijver , 2003 ; Wolsey & Nemhauser , 1999 ) . In addition , using both integer and continuous variables leads to mixed-integer formulations that can combine polyhedral and logical constraints ( Jeroslow , 1989 ; Pochet & Wolsey , 2006 ; Vielma , 2015 ) . Particularly interesting to our proposed approach are MILP formulations for piecewise-linear functions ( Huchette & Vielma , 2019 ; Vielma et al. , 2010 ) . Specifically , our work leverages MILP formulations for trained neural networks with piecewise-linear activation functions such as ReLUs ( Anderson et al. , 2020 ) . Optimizing over trained ReLU networks with MILP has been done in contexts such as neural network verification ( Cheng et al. , 2017 ; Lomuscio & Maganti , 2017 ; Tjeng et al. , 2019 ) , reinforcement learning ( Ryu et al. , 2020 ; Delarue et al. , 2020 ) , and analysis and exact com- pression of neural networks ( Serra et al. , 2018 ; 2021 ) . In particular , MILP has also been used to optimize ReLU network surrogates of simulation-based constraints ( Grimstad & Andersson , 2019 ) , although their approach optimizes a single surrogate model once , unlike in ours .

Summary: This paper solves a constrained discrete black-box optimization problem that employs a surrogate model in modeling an unknown objective function. Unlike the formulation of standard Bayesian optimization, it constructs a surrogate model using a piecewise linear neural network. Under the assumption that a randomly-initialized neural network is able to produce an uncertainty for exploration (against exploitation), the proposed method optimizes a surrogate model directly with mixed-integer linear programming. It follows a spirit of Thompson sampling. Finally the authors conduct their method on several experimental circumstances and show the validity of their method.