Package: hgboost
hgboost - Hyperoptimized Gradient Boostinghgboostis short forHyperoptimized Gradient Boostingand is a python package for hyperparameter optimization forxgboost,catboostandlightboostusing cross-validation, and evaluating the results on an independent validation set.hgboostcan be applied for classification and regression tasks.hgboostis fun because:* 1. Hyperoptimization of the Parameter-space using bayesian approach.
* 2. Determines the best scoring model(s) using k-fold cross validation.
* 3. Evaluates best model on independent evaluation set.
* 4. Fit model on entire input-data using the best model.
* 5. Works for classification and regression
* 6. Creating a super-hyperoptimized model by an ensemble of all individual optimized models.
* 7. Return model, space and test/evaluation results.
* 8. Makes insightful plots.⭐️ Star this repo if you like it ⭐️BlogsMedium Blog 1:The Best Boosting Model using Bayesian Hyperparameter Tuning but without Overfitting.Medium Blog 2:Create Explainable Gradient Boosting Classification models using Bayesian Hyperparameter Optimization.Documentation pagesOn thedocumentation pagesyou can find detailed information about the working of thehgboostwith many examples.Colab NotebooksRegression exampleClassification exampleSchematic overview of hgboostInstallation Environmentcondacreate-nenv_hgboostpython=3.8condaactivateenv_hgboostInstall from pypipipinstallhgboost
pipinstall-Uhgboost# Force updateImport hgboost packageimporthgboostashgboostExamplesExample: Fit catboost by hyperoptimization and cross-validationExample: Fit lightboost by hyperoptimization and cross-validationExample: Fit xgboost by hyperoptimization and cross-validationExample: Plot searched parameter spaceExample: plot summaryExample: Tree plotExample: Plot the validation resultsExample: Plot the cross-validation resultsExample: use the learned model to make new predictionsExample: Create ensemble model for ClassificationExample: Create ensemble model for RegressionClassification example for xgboost, catboost and lightboost:# Load libraryfromhgboostimporthgboost# Initializationhgb=hgboost(max_eval=10,threshold=0.5,cv=5,test_size=0.2,val_size=0.2,top_cv_evals=10,random_state=42)# Fit xgboost by hyperoptimization and cross-validationresults=hgb.xgboost(X,y,pos_label='survived')# [hgboost] >Start hgboost classification..# [hgboost] >Collecting xgb_clf parameters.# [hgboost] >Number of variables in search space is [11], loss function: [auc].# [hgboost] >method: xgb_clf# [hgboost] >eval_metric: auc# [hgboost] >greater_is_better: True# [hgboost] >pos_label: True# [hgboost] >Total dataset: (891, 204)# [hgboost] >Hyperparameter optimization..#  100% |----| 500/500 [04:39<05:21,  1.33s/trial, best loss: -0.8800619834710744]# [hgboost] >Best performing [xgb_clf] model: auc=0.881198# [hgboost] >5-fold cross validation for the top 10 scoring models, Total nr. tests: 50# 100%|██████████| 10/10 [00:42<00:00,  4.27s/it]# [hgboost] >Evalute best [xgb_clf] model on independent validation dataset (179 samples, 20.00%).# [hgboost] >[auc] on independent validation dataset: -0.832# [hgboost] >Retrain [xgb_clf] on the entire dataset with the optimal parameters settings.# Plot the ensemble classification validation resultshgb.plot_validation()References* http://hyperopt.github.io/hyperopt/
* https://github.com/dmlc/xgboost
* https://github.com/microsoft/LightGBM
* https://github.com/catboost/catboostMaintainersErdogan Taskesen, github:erdogantContributeContributions are welcome.LicenceSeeLICENSEfor details.CoffeeIf you wish to buy me aCoffeefor this work, it is very appreciated :)