Package: greattunes
Easy-to-use Bayesian optimization librarymade available for either closed-loop or user-driven (manual) optimization of either
known or unknown objective functions. Drawing onPyTorch(GPyTorch),BOTorchand with proprietary extensions.A short primer on Bayesian optimization is provided inthis section.FeaturesHandlescontinuous,integerandcategoricalcovariates.Optimization of eitherknownorunknownfunctions. The allows for optimization of e.g. real-world experiments
without specifically requiring a model of the system be defined a priori.Simple interface with focus on ease of use: only few lines of code required for full Bayesian optimization.Erroneous observations of either covariates or response can be overwritten during optimization.Well-documented code with detailed end-to-end examples of use, seeexamples.Optimization can start from scratch or repurpose existing data.Design decisionsMultivariate covariates, univariate system response:It is assumed that input covariates (the independent
variables) can be either multivariate or univariate, while the system response (the dependent variable) is only
univariate.Optimizing across continuous, integer and categorical covariates:Problems can depend on any of these types of
variables, in any combination. Special attention is given to implementation of integer and categorical variables
which are handled via the method of Garrido-Merchán and Hernandéz-Lobato (E.C. Garrido-Merchán and D. Hernandéz-Lobato, Neurocomputing, seeReferences).System-generated or manual input:Observations of covariates and responses during optimization can be provided
both programmatically or manually via prompt input.Optimizes known and unknown response functions:Both cases where the response function can be formulated
mathematically and cases where the response can only be measured (e.g. a real-life experiment) can be
optimized.Observed covariates can vary from the proposed covariates:The optimization routine at each iteration proposes
new covariate data points to investigate, but there is no requirement that this is also the observed data point.
At each iteration step, proposed covariates, observed covariates and observed response are 3 separate entities. That
that noisy or unexpected measurement points will be fully useful (no introduce any errors), even if they vary a lot
from the proposed covariate data points.Data stored in class instance:Data forproposed covariate data points,observed covariatesandobserved
responsesis stored in the instantiated class object.Data format and type validation:Input data is validated at each iteration.Observations of covariates and response can be overridden during execution:If an observation of either covariates
or response seems incorrect, the framework allows overriding the previous observation.Consistency in number of covariates and observations:It is assumed that there is consistency in the number of
observations of covariates and responses: at each step a new covariate data point is proposed, before observations
of covariates and responsefor this iterationare reported (specifically the number of proposed data points cannot
exceed the number of observed covariates by more than 1, and the number of observed covariates also cannot exceed the
number of observed responses by more than 1). If additional data is provided for either observed covariates or
observed response, this will override the last provided data.InstallationViapipThe library is available onhttps://pypi.org/, so to install simply runpipinstallgreattunesFrom sourceYou can also download the library source code and install it from there.First installtorchdependenciesInstallingtorchdependencies is not always a requirement.Unfortunately, in some casestorch-libraries have to
be installed outside normal bulkpip install -r requirements.txt.First try to install directly via steps 1-3 inInstall librarybelow, and only installtorchlibraries manually if direct installation fails.To find the right installation command fortorch, usethis linkto determine the details and add as a separate command in thegithubactions yaml. As an example, the following is the
install command on my local system (anUbuntu-based system withpipand withoutCUDAaccess)pipinstalltorch==1.6.0+cputorchvision==0.7.0+cpu-fhttps://download.pytorch.org/whl/torch_stable.htmlInstall libraryCurrently the code is not available on any repo servers except the private GitHub account. The best way to install the
code (after addingtorchandtorchvision) is follow this series of steps.Upgrade local versions of packaging librariespipinstall--upgradesetuptoolswheelClone this repoDo local installationpython-mpipinstallhttps://github.com/svedel/greattunes/Step 3 will install by runninggreattunes/setup.pylocally and installing. This step can also be broken into two,
which might improve debuggingpython3https://github.com/svedel/greattunes/setup.pybdist_wheelpython-mpipinstallhttps://github.com/svedel/greattunes/dist/greattunes-<version>-py3-none-any.whlwhere<version>is the latest version in normalpythonformat ofMAJOR.MINOR[.MICRO](check/dist-folder to see which one to pick).Using the frameworkAll capabilities of the framework are described below.For readers wanting to skip directly to working with the framework, a number of examples of how to use the framework end-to-end are included as Jupyter notebooks inexamples.Solving a problemSolving an optimization problem consists of two steps in this framework:Define the input variables (covariates), the surrogate model type and the acquisition function. Also define the
response function if this is knownOptimize based on closed-loop or iterative interfaceHere's a simple illustration of how to do this for a known functionf.Step 1: Define the problemThe critical things to define in this step areThe number of covariates. Upper and lower limits must be provided for each covariate to constraint the search space,
and initial guess for each to be provided as well. Works for both univariate and multivariate covariate structures.The type of surrogate model. The model will be fitted at each step of the optimization.The type of acquisitition function. This will also be fitted at each step of the optimization.# import libraryfromgreattunesimportTuneSession# === Step 1: define the input ===# specify covariate. For each covariate of the model provide best guess of starting point together with upper and lower# limitx_start=0.5# initial guessx_min=0# lower limitx_max=1# upper limitcovars=[(x_start,x_min,x_max)]# initialize the classcls=TuneSession(covars=covars,model="SingleTaskGP",acq_func="ExpectedImprovement")Step 2: Solve the problemIn order to optimize, we must first describewhichfunction we want to do this for. The framework works both when this
function can be formulated mathematically and when it can only be sampled (e.g. through examples) but cannot be
formulated. For an illustrate of the latter see Example 2 underexamples.Here we will work with a known objective function to optimize# === Step 2: solve the problem ===# univariate function to optimizeimportnumpyasnpdeff(x):return-(6*x-2)**2*np.sin(12*x-4)Beware that the number of covariates (including their range) specified bycovarsunder Step 1 must comply with the
functional dependence of the objective function (xin the case above).We are now ready to solve the problem. We will run formax_iter=20 iterations.# run the auto-methodcls.auto(response_samp_func=f,max_iter=max_iter)Had we worked with an objective functionfwhich could not be formulated explicitly, the right entrypoint would have
been to use the.ask-.tellmethods instead of.auto.Key attributes and methodsUser-facing attributes/methods: easily-accessible covariates and responseThe following key attributes are stored for each optimization as part of the instantiated class. These primary
data structures for users are stored inpandasdataframes in pretty format.current_bestandbest_predictedare
methods which print their output to the prompt.Attribute/methodCommentsx_dataAllobservedcovariates with dimensions, one row per observation. If no names have been added to the covariates they will take the naems "covar0", "covar1", ... . Dimensionsnum_observationsXnum_covariates.y_dataAllobservedresponses corresponding to the covariate points (rows) inx_data. Dimensionsnum_observationsX 1.best_responseBestobservedresponse value during optimization run, including current iteration. Dimensionsnum_observationsX 1.covars_best_responseObservedcovariates for best response value during optimization run, i.e. each row incovars_best_responsegenerated the same row inbest_response. Dimensionsnum_observationsXnum_covariates.current_best()Returns the bestobservedresponse of the objective up to the current iteration.best_predicted()Bestpredictedresponse from the surrogate model. Calculates for the mean model as well as for the lower confidence region (e.g. mean minus one standard deviation) of the full model. For both cases also returns the covariates resulting in the maximum.Backend attributesIn the backend the framework makes use of different data structures based on thetensorstructure fromtorchwhich
also handles one-hot encoding of categorical variables. The key backend attributes are listed in the table below.AttributeCommentstrain_XAllobservedcovariates with dimensionsnum_observationsXnum_covariates. Backend equivalent tox_data.proposed_XAllproposedcovariate datapoints to investigate, with dimensionsnum_observationsXnum_covariates.train_YAllobservedresponses corresponding to the covariate points intrain_X. Dimensionsnum_observationsX 1. Backend equivalent toy_data.best_response_valueBestobservedresponse value during optimization run, including current iteration. Dimensionsnum_observationsX 1. Backend equivalent tobest_response.covars_best_response_valueObservedcovariates for best response value during optimization run, i.e. each row incovars_best_response_valuegenerated the same row inbest_response_value. Dimensionsnum_observationsXnum_covariates. Backend equivalent tocovars_best_response.Covariates: the free parameters which are adjusted by the framework during optimizationThe user must detail which covariates the framework can adjust in order to optimize (maximize/minimize) the
response. This is a mandatory part of class initialization and set viacovarsinput variable; without any knowledge
of the covariates, the framework cannot proceed to optimization. Here's an example for a problem with two covariatescovars=[(0.5,0,1),(2,1,4)]# each tuple defines one covariate; the tuple entries are (initial guess, min, max)# initialize the classcls=TuneSession(covars=covars,...)This is also illustrated for a single-variable situation inStep 1: Define the problemabove.Supported types: Handling continuous, integer and categorical covariatesThe following three types of covariates are supported.Continuous: Variables which can take any numerical value, i.e. can take values which include decimals. The data
type of a continuous variable will be amongfloattypes. Typical examples of continuous covariates will be weights
in a model and time thresholds (imagine a case where total runtime was a parameter).Integer: Variables which can only take integer values; the data types of these variables will be amonginttypes.
Special consideration must be taken during optimization because these variables only can update in discrete steps,
resulting in step changes of the response. Examples of integer covariates include number of layers in a neural network
and number of eggs in a recipe.Categorical: Variables that can take different discrete values, which, contrary to integers do not even have any
internal relation in terms of size. An example is a variable which can take the values {green,blue,red} where
there clearly is no direct numerical relationship between the potential values; in contrast, a numerical relationship
does exist for integer variables (e.g. 5 is bigger than 2). In addition to the color example above, another example of
a categorical variable can be one which determines the make of a car (e.g. take valuesvolvo,lincoln,fiatetc)The framework follows the method of Garrido-Merchán and Hernandéz-Lobato (seeReferences) to integrate
the different types of covariates and bring them to a form that is consistent with using continuous Gaussian
processes to drive the optimization. Briefly, the method relies on adding a transformation of variables in the
correlation (kernel) function of the Gaussian processes with the following properties: integer covariates are rounded to
nearest integer and categorical variables are one-hot encoded and only the one with highest numerical value is carried
forward in each round by adjusting the value of its associated one-hot encoded variable to 1 and setting all other
one-hot encoded variables to 0.Two approaches to defining covariates in framework: working with named covariates and setting data typesTwo ways are offered to provide covariate details to the framework: the simple way which assigns names to covariates
and infers their data types from the provided data incovars(used so far), and an elaborate way which allows for
naming covariates and gives more control to specify data types. In either case, the information is given to the
framework via thecovarsinput variable.Simple approach: faster, but no control over covariate names and data typesEach covariate is defined by a tuple, and the order of the tuples defines the order of the covariates. The same order
must be used later if covariates are manually reported via the.tell-method.Covariate data typesCovariate data type is critical because it impacts how to handle the covariate during the optimization. In this simple
approach, data types are inferred from the provided data incovarsas indicated by the table below.Data typeHow reportExampleCommentsInteger(<initial_guess>,<parameter_minimum>,<parameter_maximum>)(2, 0, 5)All tuple entries must be of data typeintfor covariate to be taken as integerContinuous(<initial_guess>,<parameter_minimum>,<parameter_maximum>)(2.0, -1.2, 2.5)Only one tuple entry has to be afloatfor the covariate to be set to continuousCategorical(<initial_guess>,<option_1>,<option_2>, ...)(volvo, fiat, aston martin, ford, toyota)Covariate is taken as categorical if any entry has data typestr. There must be at least one other option than<initial_guess>, but otherwise no limit to the number of entries.Here's an example of how to use the simple approach to define thecovars-variable to communicate covariates of
different data types. Thiscovarscould be used to initialize a class instantiationcovars=[(1,0,2),# will be taken as INTEGER (type: int)(1.0,0.0,2.0),# will be taken as CONTINUOUS (type: float)(1,0,2.0),# will be taken as CONTINUOUS (type: float)("red","green","blue","yellow"),# will be taken as CATEGORICAL (type: str)("volvo","chevrolet","ford"),# will be taken as CATEGORICAL (type: str)("sunny","cloudy"),# will be taken as CATEGORICAL (type: str)]Covariate namesCovariates are assigned names behind the scenes of the typecovar1,covar2etc. with numbers added in the order in
which the variable is processed from thecovarslist of tuples during class initialization (beware that this order may
not be preserved). Covariate names are visible as the column names in thex_dataattribute.Elaborate approach: allows for specifying names and data types of covariatesThis approach requires a bit more details to be provided, but also offers much more flexibility.In this approach, all covariates are defined in a dictionary which is fed via thecovarsparameter, and each covariate is defined by their own dictionary
nested within the outer dictionary specifying all covariates. An example, which will be elaborated further in the
following, is given below for 3 covariates to make this concretecovars={'variable1':# type: integer{'guess':1,'min':0,'max':2,'type':int,},'variable2':# type: continuous (float){'guess':12.2,'min':-3.4,'max':30.8,'type':float,},'variable3':# type: categorical (str){'guess':'red','options':{'red','blue','green'},'type':str,}}Each nested dictionary gives the details of an individual covariate, and the name of these nested dictionaries are used
to name the covariate.Covariate names: Anything that's permissable as apythonstring is a valid covariate name. These names are used
throughout the framework (will be inherited intox_data).Specifying data type: The variabletypeindicates the type of the covariate. The framework uses the following typesint: integer covariatefloat: continuous covariatestr: categorical covariate
Beware that the data type (and not a string) is used to define the type (i.e. use e.g.strnot'str'to indicate a
categorical variable).Required information for each covariate: Requirements vary with the covariate data type. The following is required
for each type of covariateInteger ('type': int): Required fields areguess,minandmax(all single entries of type:int), as well astype(must beintto specify categorical).Continuous ('type': float): Required fields areguess,minandmax(all single entries of typesintorfloat), as well astype(must befloatto specify categorical).Categorical ('type': str): Required fields areguess(a single entry, type:str),options(dictionary ofstr, one
for each option the covariate can take. Must also include the element inguess) andtype(must bestrto specify
categorical).The example above shows 3 covariates but the framework can handle any number of covariates. Simply adjust the number of
nested dictionaries to meet the need (and use appropriate naming and covariate specification for your application).Multivariate covariatesMultivariate covariates are set via the (mandatory)covarsparameter during class initialization. Each covariate is
given as a 3-tuple of parameters (<initial_guess>,<parameter_minimum>,<parameter_maximum>) (the order matters!), withcovarsbeing a
list of these tuples. As an example, for a cases with 3 covariates, thecovarsparameter would becovars=[(1,0,4.4),(5.2,1.5,7.0),(4,2.2,5.1)]The order of the covariates matters since framework does not work with named covariates. Hence, the parameter defined
by the first tuple incovarswill always have to be reported as the first covariate when iterating during
optimization, the second covariate will be initialized by the second tuple incovarsetc.Observations of multivariate covariates are specified as columns in thetrain_Xattribute (format:torch.tensor),
with observations added as rows. As an example, the initial guess for the three covariates defined bycovarsabove
would betrain_X=torch.tensor([[1,5.2,4]],dtype=torch.double)Initialization optionsStarting with historical dataIf historical data for pairs of covariates and response is available for your system, this can be added during
initialization. In this case the optimization framework will have a better starting position and will likely converge
more quickly.Historical data is added during class initialization. The number of observations (rows) of covariates and response must
match. Historical training data is added during class instantiation via argumentstrain_X=<>andtrain_Y=<>as
illustrated below for the following casesMultiple observations of multivariate systemSingle observation of univariate systemSingle observation of multivariate system# importimporttorchfromgreattunesimportTuneSession### ------ Case 1 - multiple observations (multivariate) ------ #### set range of datacovars=[(1,0,4.4),(5.2,1.5,7.0),(4,2.2,5.1)]# define initial dataX=torch.tensor([[1,2,3],[3,4.4,5]],dtype=torch.double)Y=torch.tensor([[33],[37.8]],dtype=torch.double)# initialize classcls=TuneSession(covars=covars,train_X=X,train_Y=Y)### ------ Case 2 - single observation (univariate) ------ #### set range of datacovars=[(1,0,4.4)]# define initial dataX=torch.tensor([[1]],dtype=torch.double)Y=torch.tensor([[33]],dtype=torch.double)# initialize classcls=TuneSession(covars=covars,train_X=X,train_Y=Y)### ------ Case 3 - single observation (multivariate) ------ #### set range of datacovars=[(1,0,4.4),(5.2,1.5,7.0),(4,2.2,5.1)]# define initial dataX=torch.tensor([[1,2,3]],dtype=torch.double)Y=torch.tensor([[33]],dtype=torch.double)# initialize classcls=TuneSession(covars=covars,train_X=X,train_Y=Y)Random initializationStarting from a few randomly sampled datapoints typically increases the convergence of the optimization because it
makes it less likely that the algorithm locks onto a local maximum without consideration for an unknown global one.
Furthermore, in the absence of historical data, random sampling is the best option is to start.Random initialization is enabled via the parameterrandom_startduring initialization and can be applied both in case
historical data has been added or not (default israndom_start = True).# importimporttorchfromgreattunesimportTuneSession### ------ Case 1 - No historical data ------ #### set range of datacovars=[(1,0,4.4),(5.2,1.5,7.0),(4,2.2,5.1)]# define initial dataX=torch.tensor([[1,2,3],[3,4.4,5]],dtype=torch.double)Y=torch.tensor([[33],[37.8]],dtype=torch.double)# initialize classcls=TuneSession(covars=covars,random_start=True)### ------ Case 2 - With historical data ------ #### set range of datacovars=[(1,0,4.4),(5.2,1.5,7.0),(4,2.2,5.1)]# define initial dataX=torch.tensor([[1,2,3],[3,4.4,5]],dtype=torch.double)Y=torch.tensor([[33],[37.8]],dtype=torch.double)# initialize classcls=TuneSession(covars=covars,train_X=X,train_Y=Y,random_start=True)Parameters for random startNumber of random datapoints:The number of random datapoints to be sampled is set via the kwargnum_initial_randomduring initialization. This defaults to the closest integer to $\sqrt{d}$ for a problem with $d$ covariates unless a value is provided.Sampling method:Two sampling methods are available:random: Fully random sampling within the whole hypercube specified bycovars.latin_hcs:Latin hypercube samplingwithin the hypercube specified bycovars.
The sampling method is determined by the kwargrandom_sampling_methodduring class initialization.Improved convergence: adding randomly sampled points during optimizationJust like random initialization helps with convergence, best practice also prescribes adding randomly sampled points
during the optimization run.This is easily done within this framework. The parameterrandom_step_cadencedetermines the cadence between randomly
sampled datapoints (in between points sampled via Bayesian optimization).Kernels for Gaussian process surrogate modelThe following kernels for Gaussian process surrogate model are implemented. Model type and listed parameters are
provided as input to class initialization, i.e. during initialization ofTuneSessionModel nameParametersComments"SingleTaskGP"N/AA single-task exact kernel for Gaussian process regression. Follow this link formore details."FixedNoiseGP"train_YvarA single-task exact kernel for Gaussian process regression assuming a fixed noise level. Follow this link formore details."HeteroskedasticSingleTaskGP"train_YvarA single-task exact kernel for Gaussian process regression using a heteroskedastic noise model. Follow this link formore details."SimpleCustomMaternGP"nuA custom Matérn kernel with parameternu(a float). For more details on Matérn kernels seewiki page, and see the source code for the model ingreattunes\custom_models.Acquisition functionsThese acquisition functions are currently available. Parameters (if any) are provided during initialization of theTuneSessionclass instance.Acquisition function nameParameterComments"ExpectedImprovement"N/AExpected improvement acquisition function. This is the default forgreattunes. For more detailssee hereor Section 2in this paper."NoisyExpectedImprovement"num_fantasies(default: 20)Expected improvement acquisition averaged overnum_fantasiesrealizations of a single but noisy model. Requires that the Gaussian process model is of the typeFixedNoiseGP. For more detailssee here."qExpectedImprovement"sampler(default:botorch.sampling.SobolQMCNormalSampler)Monte Carlo-based expected improvement function. For more detailssee here."qNoisyExpectedImprovement"sampler(default:botorch.sampling.SobolQMCNormalSampler)Monte Carlo-based noisy expected improvement function. For more detailssee here."PosteriorMean"N/APosterior mean. Requires the surrogate (Gaussian process) model to have a mean property (all implemented models do). For more detailssee here."ProbabilityOfImprovement"N/AProbability of improvement over the current best observed value, computed using the analytic formula under a Normal posterior distribution. Requires the outcome to be Gaussian. For more detailssee here."qProbabilityOfImprovement"sampler(default:botorch.sampling.SobolQMCNormalSampler)Monte Carlo based probability of improvement method. For more detailssee here."qSimpleRegret"sampler(default:botorch.sampling.SobolQMCNormalSampler)Monte Carlo method for simple regret. For more detailssee here."UpperConfidenceBound"beta(default: 0.2)Analytic upper confidence bound that comprises of the posterior mean plus an additional term: the posterior standard deviation weighted by a trade-off parameter,beta. For more detailssee here."qUpperConfidenceBound"beta(default: 0.2),sampler(default:botorch.sampling.SobolQMCNormalSampler)Monte carlo based Upper Confidence Bound method. For more detailssee hereorhere."qKnowledgeGradient"num_fantasies(default: 20)Computes the Knowledge Gradient using realizations ("fantasies") for the outer expectation and either the model posterior mean or MC-sampling for the inner expectation. For a fixed number of realizations ("fantasies"), optimizes in a “one-shot” fashion. For more detailssee hereorhere."qMaxValueEntropy"N/AUses max-value entropy search. This acquisition function computes the mutual information of max values and a candidate point. For more detailssee hereorhere."qMultiFidelityMaxValueEntropy"N/AMulti-fidelity max-value entropy search. For more detailssee hereorhere.Closed-loop: the.automethodClosed-loop optimization refers to situations where the function is known and therefore can iterate itself to
optimality. These are addressed via the.automethod, which takes a function handleresponse_samp_funcas well as a
maximum number of iterationsmax_iteras input parameters. See theexample aboveas
illustration of how to use the method.Stopping based on relative improvement in best observed response:rel_tolandrel_tol_stepsThe optimization can be stopped beforemax_itersteps have been taken by specifying the limit on the relative
improvement in best observed response value (best_response_value). This is invoked by providing the parameterrel_tolto the.automethod.# some function to optimizedeff(x):...# parametersmax_iter=100rel_tol=1e-10# run the auto-methodcls.auto(response_samp_func=f,max_iter=max_iter,rel_tol=rel_tol)In most cases the best results are found by requiring therel_tollimit to be satisfied for multiple consecutive
iterations. This can be achieved by also providing the number of consecutive steps requiredrel_tol_steps. Ifrel_tol_stepsis not provided, the limit on relative improvement only needs to be reached once for convergence.# some function to optimizedeff(x):...# parametersmax_iter=100rel_tol=1e-10rel_tol_steps=5# run the auto-methodcls.auto(response_samp_func=f,max_iter=max_iter,rel_tol=rel_tol,rel_tol_steps=rel_tol_steps)Best practises on usingrel_tolandrel_tol_stepsare provided in Example 5 inexamples.Iterative: the.askand.tellmethodsThe true value of Bayesian optimization is its ability to optimize problems which cannot be formulated mathematically.
The mathematical method can work as long as a response can be generated, and in fact makes no assumptions on the
nature of the problem (except that a maximum is present). Thus, whether the response is generated as a measurement from
an experiment, the feedback from users or the output of a defined mathematical function does not matter; all can be
optimized via the framework.Optimization of unknown functions is handled by the methods.askand.tell..askprovides a best guess of the next covariate data point to sample, given the history of previously sampled points for the
problem (that is,.askprovides the output of the acquisition function).tellis the method to report the observed covariate data point and the associated response
One call to.askfollowed by a call to.tellperforms one iteration of.autofrom the point of view of the
Bayesian optimization; the difference is only in how to interface with it. Examples 2 and 3 inexamplesshows how to use.ask-.tellto solve problems end-to-end.To solve a problem, apply these problems iteratively: in each iteration start by calling.ask, then use the proposed
new data point to sample the system response and provide both this value and the actually sampled covariate values (can
be different from proposed values) back via.tell.# in below, "cc" is an instantiated version of TuneSession class (identical initialization as when using .auto method)max_iter=20foriinrange(max_iter):# generate candidatecls.ask()# new candidate is last row in cc.proposed_X# sample response (beware results must be formulated as torch tensors)observed_covars=<frommeasurementorfromcc.proposed_X>observed_response=<frommeasurementorfromspecifiedobjectivefunction># report responsecls.tell(covars=observed_covars,response=observed_response)Providing input via promptObservations of covariates and response can be provided manually to.tell. To do so, simply call.tellwithout any
arguments at each iteration (all book keeping will be handled on backend)# in below, "cc" is an instantiated version of TuneSession class (identical initialization as when using .auto method)max_iter=20foriinrange(max_iter):# generate candidatecls.ask()# new candidate is last row in cc.proposed_X# report responsecls.tell()In this case, the user will be prompted to provide input manually. There will be 3 attempts to provide covariates
(another 3 for response), and the method will stop if not successful within these attempts. Provided input data will be
validated for number of variables and data type as part of these cycles.Any ofcovarsandresponsenot provided as (named) parameter to.tellthe user will be requested to provide via
manual input in prompt. It is thus possible to get e.g. covariates automatically but manually read off response values
from an instrument.Overriding reported values of covariates or responseObserved covariates and observed responses are sometimes off. To override the latest datapoint for either, simply
provide it again in the same iteration. This will automatically override the latest reported value# in below, "cc" is an instantiated version of TuneSession class (identical initialization as when using .auto method)# further assumes that at least on full iteration has been taken# define a responsedeff(x):...# generate candidatecls.ask()# new candidate is last row in cc.proposed_X# first resultobserved_results=torch.tensor([[it.item()foritincc.proposed_X[-1]]],dtype=torch.double)observed_response=torch.tensor([[f(cc.proposed_X[-1]).item()]],dtype=torch.double)# report first responsecls.tell(covars=observed_results,response=observed_response)# second resultobserved_response_second=observed_response+1# update responsecls.tell(covars=observed_results,response=observed_response_second)Plotting and results presentationSome standard plots and standard methods for presenting the results have been included.Pre-defined plotsplot_1d_latest(): plots the latest retrained surrogate model (mean and variance), including all sampled data points.plot_convergence(): plots the relative error between consecutive iterations.plot_best_objective(): plots the best recorded value of the objective function as a function of the number of iterations.Result summariesThese methods print their results to the prompt.current_best(): returns the largestobservedresponse value (observed in either previous or current iteration). Also returns the corresponding values of the covariates.best_predicted(): returns the largest responsepredictedfrom the surrogate model trained on all available data. Two values are returned: the largest mean and the largest of the lower confidence region, i.e. the largest value of the mean minus the first standard deviation (note: heteroskedacticity is allowed, so the standard deviation will vary across different covariates). Also returns the corresponding covariate values. Uses theNelder-Mead method, a multivariate equivalent to bisection, to find maximum value of surrogate model.ContributingWe are happy if you would like to invest time in this project! Details are given inCONTRIBUTING.mdon how to get started.ExamplesA number of examples showing how to use the framework injupyternotebooks is available in theexamplesfolder. This includes both closed-loop and iterative usages, as well as a few real-world examples (latter to come!)ReferencesE.C. Garrido-Merchán and D. Hernandéz-Lobato: Dealing with categorical and integer-valued variables in Bayesian
Optimization with Gaussian processes, Neurocomputing vol. 380, 7 March 2020, pp. 20-35,ArXiv preprintA primer on Bayesian optimizationA number of good resources are available for Bayesian optimization, so below follows only a short primer. Interested
readers are referred to the references listed below for more information.Basics of Bayesian optimizationBriefly and heuristically, Bayesian optimization works as follows.Define aobjective function. The goal of the optimization is to maximize this function.Define asurrogate model. This is an approximation of the actual functional dependencies underlying the objective
function. Because Bayesian optimization builds its own model there is no requirement that the objective function can be
written as a mathematical expression.Define anacquisition function. This function is applied to the surrogate model to identify the next datapoint to
sample (as such, the acquisition function is actually a functional)Iterate:Use the acquisition function to identify the next data point to sample.Observe the response of the objective function at the proposed pointBased on all observed covariates and responses of the objective function, update the surrogate model via Bayes
theorem and repeat.Surrogate modelsA typical choice of surrogate model class is theGaussian process,
but this is not a strict requirement. Examples exist in which both random forest and various types of neural networks
have been used.Formally, Bayesian optimization considers the function to be optimized as unknown and instead places a Bayesian prior
distribution over it. This is the initial surrogate model. Upon observing the response, the prior model is updated to
obtain the posterior distribution of functions.The benefit of Gaussian process models is their explicit modeling of the uncertainty and ease of obtaining the posterior.Acquisition functionsAcquisition functions (functionals) propose the best point to sample for a particular problem, given the
prior distribution of the surrogate model.A number of different functions exist, with some typical ones provided in Peter Frazier'sTutorial on Bayesian Optimization. They typically balance exploration and
exploitation in different ways.ReferencesA list of Bayesian optimization references for later useWikipedia entry on Bayesian optimizationBoTorchintroduction to Bayesian optimizationborealis.aibayesopt, SigOpt pageTowards Data ScienceGaussian processes for dummiesPeter Frazier, Cornell, Bayesian Optimization expertTutorial on Bayesian OptimizationBayesian Optimization, Martin Krasser's blogBayesian Optimization with inequality constraintsBayesian deep learning