Package: optiseek
An open source collection of single-objective optimization algorithms for multi-dimensional functions.The purpose of this library is to give users access to a variety of versatile black-box optimization algorithms with extreme ease of use and interoperability.
The parameters of each of the algorithms can be tuned by the users and there is a high level of parameter uniformity between algorithms.Benefits of using Optiseek include:support for float, integer, categorical, and boolean inputs for objective functionscompatibility with black-box objective functions (requires no information on gradients or form of function)the algorithms are simple compared to alternatives (e.g. Bayesian optimization) with faster runtime on basic objective functionscompetitive convergence for computionally expensive objective functions in terms of number of function evaluationsseamless integration into ML pipelines for hyper-parameter tuningaccess to a variety of stopping criteria, suitable for both computationally expensive and cheap objective functionscarefully chosen default parameters for algorithms, with ability for user-defined fine tuningDocumentationFor full documentation, visit thegithub pages site.InstallationpipinstalloptiseekUsageBasic Implementationoptiseekprovides access to numerous optimization algorithms that require minimal effort from the user. An example using the well-known particle swarm optimization algorithm can be as simple as this:fromoptiseek.variablesimportvar_floatfromoptiseek.metaheuristicsimportparticle_swarm_optimizerfromoptiseek.testfunctionsimportbooth# define a list of variables and their domains for the objective functionvar_list=[var_float('x1',[-10,10]),var_float('x2',[-10,10])]# create an instance of the algorithm to optimize the booth test function and set its parametersalg=particle_swarm_optimizer(booth,var_list)# define stopping criteria and optimizealg.optimize(find_minimum=True,max_iter=10)# show the results!print(f'best_value ={alg.best_value:.5f}')print(f'best_position ={alg.best_position}')print(f'n_iter ={alg.completed_iter}')best_value = 0.00217
best_position = {'x1': 0.9921537320723116, 'x2': 3.0265668104168326}
n_iter = 10This is a fairly basic example implementation without much thought put into parameter selection. Of course, the user is free to tune the parameters of the algorithm any way they would like.Tuning ML HyperparametersThe steps to tuning hyperparameters for an ML algorithm withoptiseekare straighforward:Prepare the training dataCreate a function that performs cross-validation with the hyperparameters are arguments and error metric as outputDefine the search spacePass this to an algorithm inoptiseekand optimize# importsfromoptiseek.variablesimport*fromoptiseek.metaheuristicsimportparticle_swarm_optimizerfromsklearn.datasetsimportfetch_california_housingfromsklearn.model_selectionimportcross_validatefromlightgbmimportLGBMRegressor# import and prepare the datacalifornia_housing=fetch_california_housing(as_frame=True)X=california_housing.datay=california_housing.target# set up cross-validation of the model as the objective functiondefobjective_function(learning_rate,num_leaves,max_depth,min_child_weight,min_child_samples,subsample,colsample_bytree,reg_alpha):# assign the parametersparams={'n_estimators':300,'learning_rate':learning_rate,'num_leaves':num_leaves,'max_depth':max_depth,'min_child_weight':min_child_weight,'min_child_samples':min_child_samples,'subsample':subsample,'colsample_bytree':colsample_bytree,'reg_alpha':reg_alpha,'verbose':-1}# create the modelmodel=LGBMRegressor(**params)# cross validate and return average validation MRSEcv_results=cross_validate(model,X,y,scoring='neg_root_mean_squared_error',cv=5)cv_score=-np.mean(cv_results['test_score'])returncv_score# define the search spacevar_list=[var_float('learning_rate',[0.001,0.3]),var_int('num_leaves',[20,3000]),var_int('max_depth',[3,12]),var_float('min_child_weight',[0.0005,0.1]),var_int('min_child_samples',[5,50]),var_float('subsample',[0.5,1]),var_float('colsample_bytree',[0.5,1]),var_float('reg_alpha',[0.001,0.1])]# instantiate an optimization algorithm with the function and search domainalg=particle_swarm_optimizer(objective_function,var_list,results_filename='cv_results.csv')# set stopping criteria and optimizealg.optimize(find_minimum=True,max_function_evals=300)# show the results!print(f'best_value ={alg.best_value:.5f}')print(f'best_position ={alg.best_position}')best_value = 0.60881
best_position = {'learning_rate': 0.24843279626513076, 'num_leaves': 3000, 'max_depth': 3, 'min_child_weight': 0.06303795879741575, 'min_child_samples': 27, 'subsample': 0.5, 'colsample_bytree': 0.9620615099733333, 'reg_alpha': 0.022922559999999998}Licenseoptiseekwas created by Alex Dundore. It is licensed under the terms of the MIT license.Credits and Dependenciesoptiseekis powered bynumpy.