Package: hpbandster-sklearn
hpbandster-sklearnhpbandster-sklearnis a Python library providing ascikit-learnwrapper -HpBandSterSearchCV- forHpBandSter, a hyper parameter tuning library.MotivationHpBandSterimplements several cutting-edge hyper parameter algorithms, including HyperBand and BOHB. They often outperform standard Random Search, finding best parameter combinations in less time.HpBandSteris powerful and configurable, but its usage is often unintuitive for beginners and necessitating a large amount of boilerplate code. In order to solve that issue,HpBandSterSearchCVwas created as a drop-in replacement forscikit-learnhyper parameter searchers, following its well-known and popular API, making it possible to tunescikit-learnAPI estimators with minimal setup.HpBandSterSearchCVAPI has been based onscikit-learn'sHalvingRandomSearchCV, implementing nearly all of the parameters it does.Installationpip install hpbandster-sklearnUsageUse it like any otherscikit-learnhyper parameter searcher:importnumpyasnpfromsklearn.datasetsimportload_irisfromsklearn.ensembleimportRandomForestClassifierfromsklearn.utils.validationimportcheck_is_fittedfromhpbandster_sklearnimportHpBandSterSearchCVX,y=load_iris(return_X_y=True)clf=RandomForestClassifier(random_state=0)np.random.seed(0)param_distributions={"max_depth":[2,3,4],"min_samples_split":list(range(2,12))}search=HpBandSterSearchCV(clf,param_distributions,random_state=0,n_jobs=1,n_iter=10,verbose=1).fit(X,y)search.best_params_You can also useConfigSpace.ConfigurationSpaceobjects instead of dicts (in fact, it is recommended)!importnumpyasnpfromsklearn.datasetsimportload_irisfromsklearn.ensembleimportRandomForestClassifierfromsklearn.utils.validationimportcheck_is_fittedfromhpbandster_sklearnimportHpBandSterSearchCVimportConfigSpaceasCSimportConfigSpace.hyperparametersasCSHX,y=load_iris(return_X_y=True)clf=RandomForestClassifier(random_state=0)np.random.seed(0)param_distributions=CS.ConfigurationSpace(seed=42)param_distributions.add_hyperparameter(CSH.UniformIntegerHyperparameter("min_samples_split",2,11))param_distributions.add_hyperparameter(CSH.UniformIntegerHyperparameter("max_depth",2,4))search=HpBandSterSearchCV(clf,param_distributions,random_state=0,n_jobs=1,n_iter=10,verbose=1).fit(X,y)search.best_params_Please refer to thedocumentation of this library, as well as to the documentation ofHpBandSterandConfigSpacefor more information.Pipelines andTransformedTargetRegressorare also supported. Make sure to prefix the hyper parameter and resource names accordingly should you use either (or both) - for example,final_estimator__n_estimators.n_samplesis not to be prefixed.Early stoppingAs almost every search algorithm inHpBandSterleverages early stopping (mostly through Successive Halving), the user can configure the resource and budget to be used through the arguments ofHpBandSterSearchCVobject.search=HpBandSterSearchCV(clf,param_distributions,resource_name='n_samples',# can be either 'n_samples' or a string corresponding to an estimator attribute, eg. 'n_estimators' for an ensembleresource_type=float,# if specified, the resource value will be cast to that type before being passed to the estimator, otherwise it will be derived automaticallymin_budget=0.2,max_budget=1,)search=HpBandSterSearchCV(clf,param_distributions,resource_name='n_estimators',# can be either 'n_samples' or a string corresponding to an estimator attribute, eg. 'n_estimators' for an ensembleresource_type=int,# if specified, the resource value will be cast to that type before being passed to the estimator, otherwise it will be derived automaticallymin_budget=20,max_budget=200,)By default, the object will try to automatically determine the best resource, by checking the following in order:'n_estimators', if the model has that attribute and thewarm_startattribute'max_iter', if the model has that attribute and thewarm_startattribute'n_samples'- if the model doesn't supportwarm_start, the dataset samples will be used as the resource instead, meaing the model will be iteratively fitted on a bigger and bigger portion of the dataset.Furthermore, special support has been added forLightGBM,XGBoostandCatBoostscikit-learnestimators.Documentationhttps://hpbandster-sklearn.readthedocs.io/en/latest/ReferencesHpBandSter-https://github.com/automl/HpBandSterConfigSpace-https://github.com/automl/ConfigSpacescikit-learn-http://scikit-learn.org/AuthorAntoni Baum (Yard1)LicenseMIT