Package: optml
This package offers implementations of several black-box optimisation methods to tune
hyperparameters of machine learning models. Its purpose is to enable data scientists to use
optimization techniques for rapid protyping. Simply import OptML and supply it with a model and the
parameters to optimize.OptML offers a unified interface for models built with Scikit-Learn, Keras, XGBoost (and hopefully
soon Statsmodels).PrerequisitesThis package requires scikit-learn with version 0.19.0 or higher. If Scikit-Learn is not yet
installed runpip installscikit-learn==0.19.0. If you want to make use of theHyperoptOptimizerthen you also need to installhyperopt(e.g. bypip install hyperopt).In order to run withKerasandXGBoostmodels these libraries have to be install as well, of
course.InstallationIf Scikit-Learn is version 0.19 or higher simply install optml usingpip install optmland
you’re ready to go.UsageSpecify your ML model and the parameters you want to optimize over. For the parameters you have to
choose the type (such as integer, categorical, boolean, etc.) and the range of values it can take.model=SomeMLModel()params=[Parameter(name='param1',param_type='continuous',lower=0.1,upper=5),Parameter(name='param2',param_type='integer',lower=1,upper=5),Parameter(name='param3',param_type='categorical',possible_values=['val1','val2','val3'])]Then define the evaluation function. This can be anything from RMSE to crossentropy to custom
functions. The first argument of the evaluation function is the array of true labels and the second
argument is an array with model predictions.defclf_score(y_true,y_pred):returnnp.sum(y_true==y_pred)/float(len(y_true))Import and initialize an optimizer and optimize the model for some training data.fromoptml.bayesian_optimizerimportBayesianOptimizerbayesOpt=BayesianOptimizer(model=model,hyperparams=params,eval_func=clf_score)bayes_best_params,bayes_best_model=bayesOpt.fit(X_train=X_train,y_train=y_train,n_iters=50)FeaturesAt the moment this library includes: * Random Search * Parallelized Gridsearch * A simple Genetic
Algorithm * Bayesian Optimisation (also supporting categorical parameters) * Hyperopt (usinghyperopt)How to Choose an OptimizerOptML implements several optimization methods to address a range of requirements that can arise in
data science problems. One of the main concerns is the effort required to evaluate a model for a set
of parameters: If a model takes a long time to train we should choose an optimizer that maximises
the potential improvement with every new set of parameters. In this case Bayesian Optimization and
Hyperopt are more applicable. If a model is cheap to train then we can seek to parallelise the
evaluations.Also consider the number of parameters and their ranges. Clearly, it is more difficult to optimize
over a large search space. It is advised to only include parameters in the optimization if they are
expected to improve the final model.Please also note that all of OptML’s optimizers require parameters to be bounded.number of
evaluationsworks with large
search spacecan use
training in
parallelhandles categorical
parametersstochastic
optimisationGridsearchhighnoyesyesnoRandom
SearchhighyesyesyesyesGenetic
Algorithmhighyesnot
implementedyesyesBayesian
Optimizerlowyesnot
implementedyesyesHyperoptlowyesyesyesyesTODOsalgorithms:implement more options for genetic algorithmsmeta heuristics/swarm optimisationfunctionalityearly stopping if there is no significant improvement after x iterationsusabilitybetter documenationAuthorJohannes Petrat-Initial Release-johannespetratLicenseThis project is licensed under the MIT License - see theLICENSE.mdfile for
details