Package: gecs
(100)gecsBayesian hyperparameter tuning for LGBMClassifier, LGBMRegressor, CatBoostClassifier and CatBoostRegressor with a scikit-learn APITable of ContentsProject OverviewIntroductionInstallationUsageExampleContributingLicenseProject Overviewgecsis a tool to help automate the process of hyperparameter tuning for boosting classifiers and regressors, which can potentially save significant time and computational resources in model building and optimization processes. TheGECstands forGoodEnoughClassifier, which allows you to focus on other tasks such as feature engineering. If you deploy 100 of them, you get 100GECs.IntroductionThe primary class in this package isLightGEC, which is derived fromLGBMClassifier. Like its parent, LightGEC can be used to build and train gradient boosting models, but with the added feature ofautomated bayesian hyperparameter optimization. It can be imported fromgecs.lightgecand then used in place ofLGBMClassifier, with the same API.By default,LightGECoptimizesnum_leaves,boosting_type,learning_rate,reg_alpha,reg_lambda,min_child_samples,min_child_weight,colsample_bytree,subsample_freq,subsampleand optionallyn_estimators. Which hyperparameters to tune is fully customizable.InstallationThe installation requirescmake, which can be installed usingapton linux orbrewon mac. Then you can install (100)gecs using pip.pip install gecsUsageTheLightGECclass provides the same API to the user as theLGBMClassifierclass oflightgbm, and additionally:the two additional parameters to the fit method:n_iter: Defines the number of hyperparameter combinations that the model should try. More iterations could lead to better model performance, but at the expense of computational resourcesfixed_hyperparameters: Allows the user to specify hyperparameters that the GEC should not optimize. By default, onlyn_estimatorsis fixed. Any of the LGBMClassifier init arguments can be fixed, and so cansubsample_freqandsubsample, but only jointly. This is done by passing the valuebagging.the methodsserializeanddeserialize, which stores theLightGECstate for the hyperparameter optimization process,but not the fittedLGBMClassifierparameters, to a json file. To store the boosted tree model itself, you have to provide your own serialization or usepicklethe methodsfreezeandunfreezethat turn theLightGECfunctionally into aLGBMClassifierand backExampleThe default use ofLightGECwould look like this:from sklearn.datasets import load_iris
from gecs.lightgec import LightGEC # LGBMClassifier with hyperparameter optimization
from gecs.lightger import LightGER # LGBMRegressor with hyperparameter optimization
from gecs.catgec import CatGEC # CatBoostClassifier with hyperparameter optimization
from gecs.catger import CatGER # CatBoostRegressor with hyperparameter optimization


X, y = load_iris(return_X_y=True)


# fit and infer GEC
gec = LightGEC()
gec.fit(X, y)
yhat = gec.predict(X)


# manage GEC state
path = "./gec.json"
gec.serialize(path) # stores gec data and settings, but not underlying LGBMClassifier attributes
gec2 = LightGEC.deserialize(path, X, y) # X and y are necessary to fit the underlying LGBMClassifier
gec.freeze() # freeze GEC so that it behaves like a LGBMClassifier
gec.unfreeze() # unfreeze to enable GEC hyperparameter optimisation


# benchmark against LGBMClassifier
from lightgbm import LGBMClassifier
from sklearn.model_selection import cross_val_score
import numpy as np

clf = LGBMClassifier()
lgbm_score = np.mean(cross_val_score(clf, X, y))

gec.freeze()
gec_score = np.mean(cross_val_score(gec, X, y))

print(f"{gec_score = }, {lgbm_score = }")
assert gec_score > lgbm_score, "GEC doesn't outperform LGBMClassifier"

#check what hyperparameter combinations were tried
gec.tried_hyperparameters()ContributingIf you want to contribute, please reach out and I'll design a process around it.LicenseMITContact InformationYou can find my contact information on my website:https://leonluithlen.eu