Package: nemo-bo
Nomadic Exploratory Multi-objective Optimisation (NEMO)Meet NEMO - our ‘Nomadic Explorer’. NEMO is quite the connoisseur when it comes to machine learning optimisation - only the best model types and model parameters will suffice. In the case of a single dataset, NEMO will scour the lands for the optimal model type and parameters to fit a given dataset. A range of outputs will then be generated for you to assess, interpret and utilise your newly created model.If you decide to take your analyses a step further into the realms of Multi-objective Bayesian optimisation, then our Nomadic Explorer will tirelessly search for the best model type and parameters at each iteration of the optimisation. At each stage, the optimal set of conditions will be provided to aid your pursuit of the elusive multi-dimensional pareto front.NEMO is prepared for the journey with a cavernous bag of tools. However, if your aspirations are more exotic, then NEMO supports the inclusion of custom models, samplers and functions.Check out the examples to see NEMO in action and get started with your own ML workflows.What is NEMO?NEMO is a package designed for Bayesian optimisation of one or multiple objectives simultaneously, with a focus on applying to chemical processes.InstallationTo install NEMO via pip:pip install nemo-boHow does NEMO work?Firstly, the parameters (variables) and targets (objectives) of a chemical process are provided to the algorithm. After
providing NEMO with a dataset from prior experiments, it will then identify the relationship between the parameters and targets
and then suggest the ideal parameters to use for the optimisation iteration.In comparison to other open-source optimisation libraries, NEMO will automatically optimise the hyperparameters for
various machine learning models and select the one with the best predictive accuracy for a given objective. This
ensures that the model is continuously optimised over the course of an optimisation campaign. Furthermore, NEMO
natively supports objectives that can be calculated if the exact relationship between the parameters and the target
(e.g. materials cost) is known.What features are in NEMO?Although NEMO includes many machine learning models, acquisition functions, constraints, and sample generators, the
base classes for these are all included and can be utilised as a template for adding your own custom solutions.The features natively found in NEMO are the following:Resuming an optimisationEvery iteration, the progress of optimisation runs and ML model information are also saved at two points:Firstly, when candidates have been suggestedAnd secondly, after the new results have been inputted, at the end of the iterationThis allows users to resume optimisation runs from two convenient positionsMachine learning models availableGaussian processes (GPs) using theBoTorchlibraryVarious neural networks from theDeeply Uncertain code repository:Bayesian neural networksConcrete dropoutDeep ensemblesVarious decision-tree based models:XGBoost DistributionNGBoostRandom Forest using theforest-confidence-intervalVariable types availableContinuous variables (ContinuousVariable)Categorical variables with discrete variables (CategoricalVariableDiscreteValues)Categorical variables with descriptors (CategoricalVariableWithDescriptors)Categorical variables without any description (e.g. one-hot encoding) is not currently supportedObjective types availableObjectives modelled using machine learning models (RegressionObjective)Calculated objectives using a user-provided function (CalculableObjective)Classification objectives are not currently supportedUser-selectable acquisition functions availableExpected improvement based methods (ExpectedImprovement)A modifed single-objective expected improvement algorithm that is better at exploration than the standard analytical methodA modifed multi-objective expected hypervolume improvement algorithm that is better at exploration than the standard analytical methodqNEIandqNEHVIBoTorch methods (only compatible with GP models)A Unified evolutionary optimization algorithmU-NSGA-IIIbased method that derives uncertainty in the inference by sampling from a distribution (NSGAImprovement)A fully explorative method that identifies the candidates that have the highest uncertainty in the objective predictions (HighestUncertainty)Input constraints availableLinear equality and inequality constraints(LinearConstraint)Basic non-linear equality and inequality constraints that incorporates an exponent for each input variable (NonLinearPowerConstraint)Equality and inequality constraints that allows the user to pass a function to calculate the left-hand-side of the constraint (FunctionalConstraint)Stoichiometry constraints that forces the ratio between two input variable to be equal to or greater than a specified value (StoichiometricConstraint)A constraint type to limit the number of active variables (MaxActiveFeaturesConstraint)A constraint type that prevents certain categorical constraints from being selected simulatenously (CategoricalConstraint)Benchmarking functionality availableBenchmark functions are typically used to simulate the outcomes of experiments in a closed-loop manner, and therefore
the user is not promoted to input the actual output values of suggested candidates. Therefore, they can be helpful to
evaluate the quality of an optimisation (inferred from the effectiveness of the utilised model(s) and/or acquisition
function to identify the optimum)Machine learning model based on a provided dataset (ModelBenchmark)Single objective synthetic functions (SingleObjectiveSyntheticBenchmark)Multi-objective synthetic functions (MultiObjectiveSyntheticBenchmark)Sample generators availableMethods for generating a samples of parameter values during an optimisation. These can be used independently outside of an optimisation too by calling thegenerate_samplesfunctionLatin hypercube sampling (with a mixed-integer implementation for efficient sampling of categorical variables) (LatinHyperCubeSampling)Sobol sampling (SobolSampling)Polytope sampling (PolytopeSampling)Random sampling (RandomSampling)Pool-based sampling using a user-defined set of data points. Typically used as an alternative to a machine learning model benchmark function (PoolBased)Other utilities/functions availableIncluded template for provided the dataset with automated extractionScatter and bar chart plotting functionality for displaying model quality and optimisation progressGetting startedThe following code demonstrates how to set-up a simple bayesian optimisation using a user-provided dataset containing four continuous variables (X) and two objectives (Y):# Import the variable, objectives, sampler, acquisition function, and the optimisation classesimportnumpyasnpfromnemo_bo.opt.variablesimportContinuousVariable,VariablesListfromnemo_bo.opt.objectivesimportRegressionObjective,ObjectivesListfromnemo_bo.acquisition_functions.expected_improvement.expected_improvementimportExpectedImprovementfromnemo_bo.opt.samplersimportLatinHyperCubeSamplingfromnemo_bo.opt.optimisationimportOptimisation# Create the variable objectsvar1=ContinuousVariable(name="variable1",lower_bound=1.0,upper_bound=10.0)var2=ContinuousVariable(name="variable2",lower_bound=0.02,upper_bound=0.2)var3=ContinuousVariable(name="variable3",lower_bound=30.0,upper_bound=70.0)var4=ContinuousVariable(name="variable4",lower_bound=5.0,upper_bound=15.0)var_list=VariablesList([var1,var2,var3,var4])# Create the objective objectsobj1=RegressionObjective(name="objective1",# obj_max_bool when True defines the objective is to be maximisedobj_max_bool=True,lower_bound=0.0,upper_bound=100.0,predictor_type=["gp","xgb"],)obj2=RegressionObjective(name="objective2",obj_max_bool=False,# obj_max_bool when False defines the objective is to be minimisedlower_bound=0.01,upper_bound=0.15,predictor_type=["gp","xgb"],)obj_list=ObjectivesList([obj1,obj2])# Instantiate the samplersampler=LatinHyperCubeSampling()# Instantiate the acquisition functionacq_func=ExpectedImprovement(num_candidates=4)# num_candidates defines how many sets of parameters to return at each optimisation iteration# Set up the optimisation instance# opt_name is used to store the optimisation information in a sub-folder with this nameoptimisation=Optimisation(var_list,obj_list,acq_func,sampler=sampler,opt_name="README optimisation")# Start the optimisation using the convenient run function that will run for the specified number of iterations# X and Y arrays represent an initial user-provided datasetX=np.array([[6.82,0.16,34,6.2],[6.15,0.08,47,8.5],[4.92,0.05,32,11.1],[9.24,0.15,41,12.1],[1.07,0.12,67,8.2],[5.66,0.09,53,12.7],[8.08,0.19,54,5.4],[1.87,0.11,68,9.2],[4.08,0.13,58,10.4],[4.38,0.18,36,14.6],])Y=np.array([[33.31,0.12],[41.89,0.10],[36.87,0.09],[46.32,0.13],[0.00,0.09],[36.52,0.10],[45.77,0.14],[0.00,0.09],[30.95,0.11],[34.89,0.12],])optimisation_data=optimisation.run(X,Y,number_of_iterations=10)# During the optimisation, after candidates have been suggested, the user will be prompted to input the actual output# values into the python console. At this point, the model information, optimisation progress, and candidates have been# saved and the user can either choose to leave the python console open whilst they obtain the results, or they can# stop the python process, and then resume the optimisation and input the values at a more convenient time later# After the actual output values have been inputted, the optimisation run will be saved again, and then the next# iteration starts automaticallyMore tutorialsWe encourage you to look through the tutorials written in thetutorialsfolder to see how to use some other
NEMO functionsHow to select specific machine learning models types for the objectivesSetting up a single objective optimisationHow to use calculable objectivesHow to define transformers for variables and objectivesHow to define categorical variables with descriptorsUtilising the machine learning model fitting in NEMO without Bayesian optimisationHow to create a closed-loop optimisation using a machine learning model as the benchmark functionHow to create a closed-loop optimisation using a multiobjective synthetic function as the benchmark functionHow to create a closed-loop optimisation using a single objective synthetic function as the benchmark functionHow to create a closed-loop optimisation using a pool-based sampler as the benchmarkSetting up an optimisation with input constraintsGenerating samples without needing to perform an optimisationHow to set up a manual optimisationHow to resume an optimisation runHow to use the BoTorch (quasi-) Monte-Carlo based acquisition functions in NEMOHow to set up an optimisation that uses U-NSGA-III as the acquisition functionUsing the input template excel file template to import the variables and objectives dataHow to set up an optimisation that uses the highest uncertainty acquisition functionWhat to do if you find any issues?Leave a message in the issues section and we will get back to you as soon as we can.AcknowledgementsMuch of the functionality in NEMO is built on top of the work by the authors of the features we incorporate. We are grateful
to them for continuously supporting their libraries and establishing their platforms for optimisation work. We reference
the works throughout the .py files.