Package: shap-hypetune
shap-hypetuneA python package for simultaneous Hyperparameters Tuning and Features Selection for Gradient Boosting Models.OverviewHyperparameters tuning and features selection are two common steps in every machine learning pipeline. Most of the time they are computed separately and independently. This may result in suboptimal performances and in a more time expensive process.shap-hypetune aims to combine hyperparameters tuning and features selection in a single pipeline optimizing the optimal number of features while searching for the optimal parameters configuration. Hyperparameters Tuning or Features Selection can also be carried out as standalone operations.shap-hypetune main features:designed for gradient boosting models, as LGBModel or XGBModel;developed to be integrable with the scikit-learn ecosystem;effective in both classification or regression tasks;customizable training process, supporting early-stopping and all the other fitting options available in the standard algorithms api;ranking feature selection algorithms: Recursive Feature Elimination (RFE); Recursive Feature Addition (RFA); or Boruta;classical boosting based feature importances or SHAP feature importances (the later can be computed also on the eval_set);apply grid-search, random-search, or bayesian-search (from hyperopt);parallelized computations with joblib.Installationpipinstall--upgradeshap-hypetunelightgbm, xgboost are not needed requirements. The module depends only on NumPy, shap, scikit-learn and hyperopt. Python 3.6 or above is supported.MediaSHAP for Feature Selection and HyperParameter TuningBoruta and SHAP for better Feature SelectionRecursive Feature Selection: Addition or Elimination?Boruta SHAP for Temporal Feature SelectionUsagefromshaphypetuneimportBoostSearch,BoostRFE,BoostRFA,BoostBorutaHyperparameters TuningBoostSearch(estimator,# LGBModel or XGBModelparam_grid=None,# parameters to be optimizedgreater_is_better=False,# minimize or maximize the monitored scoren_iter=None,# number of sampled parameter configurationssampling_seed=None,# the seed used for parameter samplingverbose=1,# verbosity moden_jobs=None# number of jobs to run in parallel)Feature Selection (RFE)BoostRFE(estimator,# LGBModel or XGBModelmin_features_to_select=None,# the minimum number of features to be selectedstep=1,# number of features to remove at each iterationparam_grid=None,# parameters to be optimizedgreater_is_better=False,# minimize or maximize the monitored scoreimportance_type='feature_importances',# which importance measure to use: default or shaptrain_importance=True,# where to compute the shap feature importancen_iter=None,# number of sampled parameter configurationssampling_seed=None,# the seed used for parameter samplingverbose=1,# verbosity moden_jobs=None# number of jobs to run in parallel)Feature Selection (BORUTA)BoostBoruta(estimator,# LGBModel or XGBModelperc=100,# threshold used to compare shadow and real featuresalpha=0.05,# p-value levels for feature rejectionmax_iter=100,# maximum Boruta iterations to performearly_stopping_boruta_rounds=None,# maximum iterations without confirming a featureparam_grid=None,# parameters to be optimizedgreater_is_better=False,# minimize or maximize the monitored scoreimportance_type='feature_importances',# which importance measure to use: default or shaptrain_importance=True,# where to compute the shap feature importancen_iter=None,# number of sampled parameter configurationssampling_seed=None,# the seed used for parameter samplingverbose=1,# verbosity moden_jobs=None# number of jobs to run in parallel)Feature Selection (RFA)BoostRFA(estimator,# LGBModel or XGBModelmin_features_to_select=None,# the minimum number of features to be selectedstep=1,# number of features to remove at each iterationparam_grid=None,# parameters to be optimizedgreater_is_better=False,# minimize or maximize the monitored scoreimportance_type='feature_importances',# which importance measure to use: default or shaptrain_importance=True,# where to compute the shap feature importancen_iter=None,# number of sampled parameter configurationssampling_seed=None,# the seed used for parameter samplingverbose=1,# verbosity moden_jobs=None# number of jobs to run in parallel)Full examples in thenotebooks folder.