Package: pwasopt
Global and Preference-based Optimization with Mixed Variables using Piecewise Affine Surrogates (PWAS/PWASp)ContentsPackage descriptionInstallationBasic usageContributorsCiting PWASLicensePackage descriptionWe propose a novel surrogate-based global optimization algorithm, called PWAS, based on constructing apiecewiseaffinesurrogate of the objective function over feasible samples. We introduce two types of exploration functions to efficiently search the feasible domain via mixed integer linear programming (MILP) solvers. We also provide a preference-based version of the algorithm, called PWASp, which can be used when only pairwise comparisons between samples can be acquired while the objective function
remains unquantified. For more details on the method, please read our paperGlobal and Preference-based Optimization with Mixed Variables using Piecewise Affine Surrogates.[1] M. Zhu and A. Bemporad, "Global and preference-based optimization with mixed variables using piecewise aﬃne surrogates,"Submitted for publication, 2023. [bib entry]Installationpip install pwasoptDependencies:python 3numpyscipypulpsklearnpyparcpyDOEpycddlibExternal dependencies:MILP solver:PWAS/PWASp useGUROBIas the default solver to solve the MILP problem of acquisition optimization,
which is found to be the most robust during benchmark testing. Alternatively, we also includeGLPK, which may introduce
errors occasionally depending on the test problem and initial samples. User can also switch to another MILP solver by editing the
relevant codes inacquisition.pyandsample.py. Check the compatability of the MILP solver withpulp(the LP modeler)
at theproject webpage.GUROBI:academic licensesconfigure GUROBI with pythonGLPK:project webpagestep-by-step explanation for the installation on Stack OverflowBasic usageExamplesExamples of benchmark testing using PWAS/PWASp can be found in theexamplesfolder:mixed_variable_benchmarks.py: benchmark testing on constrained/unconstrained mixed-variable problemsTest results are reported in thepaperNote: to test benchmarkNAS-CIFAR10download the data from itsGitHub repositoryindicate thedata_pathinmixed_variable_benchmarks.pysince the dataset is compiled withTensorFlowversion 1.x,python version < 3.8is  required (withTensorFlow< 2.x)other_benchmarks.py: various NLP, MIP, INLP, MIP Benchmarks tested with PWAS/PWASpTest results are reported intest_results_on_other_benchmarks.pdfunder theexamplesfolderHere, we show a detailed example using PWAS/PWASp to optimize the parameters of thexgboostalgorithmforMNISTclassificationtask.Problem discriptionObjective:
Maximize the classification accuracy on test data.
Note PWAS/PWASp optimizes the problem usingminimization, and therefore we minimize the negative of classification accuracy.Optimization variables:
$n_c = 4$ (number of continuous variables), $n_{\rm int} = 1$ (number of integer variables,ordinal),
and $n_d = 3$ (number of categorical variables,non-ordinal) with $n_{i} = 2$, for $i = 1, 2, 3$.
Each categorical variable ($n_{di}$) can be either 0 or 1.
The bounds are $\ell_x = [10^{-6}\ 10^{-6}\ 0.001\ 10^{-6}]'$, $u_x = [1\  10\  1\  5]'$; $\ell_y = 1$, $u_y = 10$.Notes:
The 0.7/0.3 stratified train/test split ratio is applied.
Thexgboostpackage is used onMNISTclassification.
The optimization variables in this problem are the parameters of thexgboostalgorithm.
Specifically, the continuous variables $x_1$, $x_2$, $x_3$, and $x_4$ refer to the following parameters inxgboost,
respectively:learning_rate,min_split_loss,subsample, andreg_lambda.
The integer variable $y$ stands for themax_depth. As for the categorical variables, $n_{d1}$ indicates the booster type inxgboostwhere $n_{d1} = {0, 1}$ corresponding to {gbtree,dart}. $n_{d2}$ represents thegrow_policy,
where $n_{d2} = {0, 1}$ corresponding to {depthwise,lossguide}.
$n_{d3}$ refers to theobjective, where $n_{d3} = {0, 1}$ corresponding to {multi:softmax,multi:softprob}.Problem specification in Pythonimportxgboostfromsklearn.datasetsimportload_digitsfromsklearn.model_selectionimporttrain_test_splitfromsklearnimportmetricsimportnumpyasnp# info of optimization variablesnc=4# number of continous variablesnint=1# number of integer variables, ordinalnd=3# number of categorical variables, non-ordinalX_d=[2,2,2]# possible number of classes for each categorical variableslb_cont_int=np.array([1e-6,1e-6,0.001,1e-6,1])# lower bounds for continuous and integer variablesub_cont_int=np.array([1,10,0.99999,5,10])# upper bounds for continuous and integer variableslb_binary=np.zeros((nd))# lower bounds for one-hot encoded categorical variables, note the dimension is same as nd, it will be updated within the codeub_binary=np.array([1,1,1])# upper bounds for one-hot encoded categorical variableslb=np.hstack((lb_cont_int,lb_binary))# combined lower and upper bounds for the optimization variablesub=np.hstack((ub_cont_int,ub_binary))# load dataset# example code: https://github.com/imrekovacs/XGBoost/blob/master/XGBoost%20MNIST%20digits%20classification.ipynbmnist=load_digits()X,y=mnist.data,mnist.targetX_train,X_test,y_train,y_test=train_test_split(X,y,train_size=0.7,test_size=0.3,stratify=y,random_state=1)# random_state used for reproducibilitydtrain=xgboost.DMatrix(X_train,label=y_train)dtest=xgboost.DMatrix(X_test,label=y_test)# define the objective function, x collects all the optimization variables, ordered as [continuous, integer, categorical]deffun(x):xc=x[:nc]# continuous variablesxint=x[nc:nc+nint]# integer variablesxd=x[nc+nint:]# categorical variablesifxd[0]==0:mnist_booster='gbtree'else:mnist_booster='dart'ifxd[1]==0:mnist_grow_policy='depthwise'else:mnist_grow_policy='lossguide'ifxd[2]==0:mnist_obj='multi:softmax'else:mnist_obj='multi:softprob'param={'booster':mnist_booster,'grow_policy':mnist_grow_policy,'objective':mnist_obj,'learning_rate':xc[0],'min_split_loss':xc[1],'subsample':xc[2],'reg_lambda':xc[3],'max_depth':int(round(xint[0])),'num_class':10# the number of classes that exist in this datset}bstmodel=xgboost.train(param,dtrain)y_pred=bstmodel.predict(dtest)# somehow predict gives probability of each class instead of which class it belongs in...try:acc=metrics.accuracy_score(y_test,y_pred)except:y_pred=np.argmax(y_pred,axis=1)acc=metrics.accuracy_score(y_test,y_pred)return-acc# maximize the accuracy, minimze the -acc# Specify the number of maximum number of evaluations (including initial sammples) and initial samplesmaxevals=100n_initil=20# default setting for the benchmarksisLin_eqConstrained=False# specify whether linear equality constraints are presentisLin_ineqConstrained=False# specify whether linear inequality constraints are presentAeq=np.array([])# linear equality constraintsbeq=np.array([])Aineq=np.array([])# linear inequality constraintsbineq=np.array([])Solve use PWASOne can solve the optimization problem either by explicitly passing the function handlefunto PWAS, or by passing
the evaluation offunstep-by step.Solve by explicitly passing the function handlefrompwasopt.main_pwasimportPWASkey=0np.random.seed(key)# rng default for reproducibilitydelta_E=0.05# trade-off hyperparameter in acquisition function between exploitation of surrogate and exploration of exploration functionacq_stage='multi-stage'# can specify whether to solve the acquisition step in one or multiple stages (as noted in Section 3.4 in the paper [1]. Default: `multi-stage`feasible_sampling=True# can specify whether infeasible samples are allowed. Default TrueK_init=20# number of initial PWA partitions# initialize the PWAS solveroptimizer1=PWAS(fun,lb,ub,delta_E,nc,nint,nd,X_d,nsamp,maxevals,# pass fun to PWASfeasible_sampling=feasible_sampling,isLin_eqConstrained=isLin_eqConstrained,Aeq=Aeq,beq=beq,isLin_ineqConstrained=isLin_ineqConstrained,Aineq=Aineq,bineq=bineq,K=K_init,categorical=False,acq_stage=acq_stage)xopt1,fopt1=optimizer1.solve()X1=np.array(optimizer1.X)fbest_seq1=optimizer1.fbest_seqSolve by passing the function evaluation step-by stepoptimizer2=PWAS(fun,lb,ub,delta_E,nc,nint,nd,X_d,nsamp,maxevals,# here, fun is a placeholder passed to PWAS, not usedfeasible_sampling=feasible_sampling,isLin_eqConstrained=isLin_eqConstrained,Aeq=Aeq,beq=beq,isLin_ineqConstrained=isLin_ineqConstrained,Aineq=Aineq,bineq=bineq,K=K_init,categorical=False,acq_stage=acq_stage)x2=optimizer2.initialize()forkinrange(maxevals):f=fun(x2)# evaluate funx2=optimizer2.update(f)# feed function evaluation step by step to PWASX2=np.array(optimizer2.X[:-1])# it is because in prob.update, it will calculate the next point to query (the last x2 is calculated at max_evals +1)xopt2=optimizer2.xbestfopt2=optimizer2.fbestX2=np.array(optimizer2.X)fbest_seq2=optimizer2.fbest_seqBelow we show the best valuesfbest_seq1found by PWAS.Solve use PWASpWhen solve with PWASp, instead of using the function evaluations, we solve a preference-based optimization problem
with preference function $\pi(x_1,x_2)$, $x_1,x_2\in\mathbb{R}^n$
within the finite boundslb$\leq x\leq$ub(see Section 4 of[1]).Similarly to PWAS, one can solve the optimization problem either by
explicitly passing the preference indicator/synthetic preference function to PWASp, or by passing
the expressed preferencepref_evalstep-by step.Note: for this example, we usefunas asynthetic decision maker(synthetic_dm = True) to express preferences. However, the explicit evaluation offunis unknow to PWASp.When solve byexplicitlypassing the preference indicator:Ifsynthetic_dm = True, we have included two preference indicator functionspref_fun.pyandpref_fun1.pyto provide preferences based on function evaluations and constraint satisfaction.Ifsynthetic_dm = False, one need to pass afunsuch that given two decision vectors,
output -1, 0, or 1 depending on the expressed preferences.Solve by explicitly passing the preference indicatorfrompwasopt.main_pwaspimportPWASpkey=0np.random.seed(key)# rng default for reproducibilitydelta_E=1# trade-off hyperparameter in acquisition function between exploitation of surrogate and exploration of exploration functionoptimizer1=PWASp(fun,lb,ub,delta_E,nc,nint,nd,X_d,nsamp,maxevals,feasible_sampling=feasible_sampling,isLin_eqConstrained=isLin_eqConstrained,Aeq=Aeq,beq=beq,isLin_ineqConstrained=isLin_ineqConstrained,Aineq=Aineq,bineq=bineq,K=K_init,categorical=False,acq_stage=acq_stage,synthetic_dm=True)xopt1=optimizer1.solve()X1=np.array(optimizer1.X)fbest_seq1=list(map(fun,X1[optimizer1.ibest_seq]))# for synthetic problems, we can obtain the function evaluation for assessment of the solverfbest1=min(fbest_seq1)Solve by passing the expressed preference step-by stepfrompwasopt.pref_fun1importPWASp_fun1# import the preference indicator functionfrompwasopt.pref_funimportPWASp_funoptimizer2=PWASp(fun,lb,ub,delta_E,nc,nint,nd,X_d,nsamp,maxevals,feasible_sampling=feasible_sampling,# fun is a placeholder hereisLin_eqConstrained=isLin_eqConstrained,Aeq=Aeq,beq=beq,isLin_ineqConstrained=isLin_ineqConstrained,Aineq=Aineq,bineq=bineq,K=K_init,categorical=False,acq_stage=acq_stage,synthetic_dm=True)comparetol=1e-4ifisLin_ineqConstrainedorisLin_eqConstrained:pref_fun=PWASp_fun1(fun,comparetol,optimizer2.prob.Aeq,optimizer2.prob.beq,optimizer2.prob.Aineq,optimizer2.prob.bineq)# preference function objectelse:pref_fun=PWASp_fun(fun,comparetol)pref=lambdax,y,x_encoded,y_encoded:pref_fun.eval(x,y,x_encoded,y_encoded)pref_fun.clear()xbest2,x2,xsbest2,xs2=optimizer2.initialize()# get first two random samplesforkinrange(maxevals-1):pref_eval=pref(x2,xbest2,xs2,xsbest2)# evaluate preferencex2=optimizer2.update(pref_eval)xbest2=optimizer2.xbestX2=np.array(optimizer2.X[:-1])xopt2=xbest2fbest_seq2=list(map(fun,X2[optimizer2.ibest_seq]))fbest2=min(fbest_seq2)Below we show the best valuesfbest_seq1found by PWASp. Note that function evaluations here are shown solely for demonstration purposes, which are unknown to PWASp during the solution process.Include constraintsWe note below how to include constraints if present.Note:  current package only supportslinearequality/inequality constraints.Aeq: np array, dimension: (# of linear eq. const by n_encoded), where n_encoded is the length of the optimization variableAFTERbeingencoded, the coefficient matrix for the linear equality constraintsbeq: np array, dimension: (n_encode by 1), the RHS of the linear eq. constraintsAineq: np array, dimension: (# of linear ineq. const by n_encoded), the coefficient matrix for the linear inequality constraintsAFTERbeingencodedthe coefficient matrix for the linear equality constraintsbineq: np array, dimension: (n_encode by 1) the RHS of the linear ineq. constraintsMake sure to update theAeqandAineqwith the one-hot encoded categorical variables, if present.# if there is equality constraintsisLin_eqConstrained=True#(Aeq x  = beq)# specify the constraint matrix and right-hand-side vectorifisLin_eqConstrained:# an exampleAeq=np.array([1.6295,1])beq=np.array([3.0786])# if there is inequality constraintsisLin_ineqConstrained=True# (Aineq x <= bineq)# specify the constraint matrix and right-hand-side vectorifisLin_ineqConstrained:# an exampleAineq=np.array([[1.6295,1],[0.5,3.875],[-4.3023,-4],[-2,1],[0.5,-1]])bineq=np.array([[3.0786],[3.324],[-1.4909],[0.5],[0.5]])ContributorsThis package was coded by Mengjia Zhu with supervision from Prof. Alberto Bemporad.This software is distributed without any warranty. Please cite the paper below if you use this software.Citing PWAS/PWASp@article{ZB23,
    author={M. Zhu, A. Bemporad},
    title={Global and Preference-based Optimization with Mixed Variables using Piecewise Affine Surrogates},
    journal={arXiv preprint arXiv:2302.04686},
    year=2023
}LicenseApache 2.0(C) 2021-2023 M. Zhu, A. Bemporad