Package: xklearn
extrakit-learnMachine learnings components built to extend scikit-learn. All components use scikit'sobject APIto work interchangably with scikit components. It is mostly a collection of tools that have been useful forKagglecompetitions. extrakit-learn is in no way affiliated with scikit-learn in anyway, just inspired by it.Installationpip install xklearnComponentsCategoryEncoder- Like scikit's LabelEncoder but supports NaNs and unseen values.CountEncoder- Categorical feature engineering on a column based on value counts.TargetEncoder- Categorical feature engineering on a column based on target means.MultiColumnEncoder- Apply a column encoder to multiple columns.FoldEstimator- K-fold on scikit estimator wrapped into an estimator.FoldLightGBM- K-fold on LGBM wrapped into an estimator.FoldXGBoost- K-fold on XGBoost wrapped into an estimator.StackClassifier- Stack an ensemble of classifiers with a meta classifier.StackRegressor- Stack an ensemble of regressors with a meta regressor.compress_dataframe- Reduce memory of a Pandas dataframe.Hierachyxklearn
â”‚
â”œâ”€â”€ preprocessing
â”‚   â”œâ”€â”€ CategoryEncoder
â”‚   â”œâ”€â”€ CountEncoder
â”‚   â”œâ”€â”€ TargetEncoder      
â”‚   â””â”€â”€ MultiColumnEncoder
â”‚
â”œâ”€â”€ models
â”‚   â”œâ”€â”€ FoldEstimator
â”‚   â”œâ”€â”€ FoldLightGBM
|   â”œâ”€â”€ FoldXGBoost
|   â”œâ”€â”€ StackClassifier
|   â””â”€â”€ StackRegressor
|
â””â”€â”€ utilsExamplefrom xklearn.models import FoldEstimatorCategoryEncoderWraps scikit's LabelEncoder, allowing missing and unseen values to be handled.Argumentsunseen- Strategy for handling unseen values. See replacement strategies below for options.missing- Strategy for handling missing values. See replacement strategies below for options.Replacement strategies'encode'- Replace value with -1.'nan'- Replace value with np.nan.'error'- Raise ValueError.Examplefromxklearn.preprocessingimportCategoryEncoder...ce=CategoryEncoder(unseen='nan',missing='nan')X[:,0]=ce.fit_transform(X[:,0])CountEncoderReplaces categorical values with their respective value count during training. Classes with a count of one and previously unseen classes during prediction are encoded as either one or NaN.Argumentsunseen- Strategy for handling unseen values. See replacement strategies below for options.missing- Strategy for handling missing values. See replacement strategies below for options.Replacement strategies'one'- Replace value with 1.'nan'- Replace value with np.nan.'error'- Raise ValueError.Examplefromxklearn.preprocessingimportCountEncoder...ce=CountEncoder(unseen='one')X[:,0]=ce.fit_transform(X[:,0])TargetEncoderPerforms target mean encoding of categorical features with optional smoothing.Argumentssmoothing- Smoothing weight.unseen- Strategy for handling unseen values. See replacement strategies below for options.missing- Strategy for handling missing values. See replacement strategies below for options.Replacement strategies'global'- Replace value with global target mean.'nan'- Replace value with np.nan.'error'- Raise ValueError.Examplefromxklearn.preprocessingimportTargetEncoder...te=TargetEncoder(smoothing=10)X[:,0]=te.fit_transform(X[:,0],y)MultiColumnEncoderApplies a column encoder over multiple columns.Argumentsenc- Base encoder that will be applied to selected columnscolumns- Column selection, either bool-mask, indices or None (default=None).Examplefromxklearn.preprocessingimportCountEncoderfromxklearn.preprocessingimportMultiColumnEncoder...columns=[1,3,4]enc=CountEncoder()mce=MultiColumnEncoder(enc,columns)X=mce.fit_transform(X)FoldEstimatorK-fold wrapped into an estimator that performs cross validation over a selected folding method automatically when fit. Can optionally be used as a stacked ensemble of k estimators after fit.Argumentsest- Base estimator.fold- Folding cross validation object, i.e KFold and StratifedKfold.metric- Evaluation metric.refit_full- Flag indicting post fit behaviour. True will do a full refit on the full data, False will make it a stacked ensemble trained on the different folds.verbose- Flag for printing fold scores during fit.Examplefromxklearn.modelsimportFoldEstimator...base=RandomForestRegressor(n_estimators=10)fold=KFold(n_splits=5)est=FoldEstimator(base,fold=fold,metric=mean_squared_error,verbose=1)est.fit(X_train,y_train)est.predict(X_test)Output:Finished fold 1 with score: 200.8023
Finished fold 2 with score: 261.2365
Finished fold 3 with score: 169.2404
Finished fold 4 with score: 186.7915
Finished fold 5 with score: 205.0894
Finished with a total score of: 204.6813FoldLightGBMK-fold wrapped into an estimator that performs cross validation on a LGBM over a selected folding method automatically when fit. Can optionally be used as a stacked ensemble of k estimators after fit.Argumentslgbm- Base estimator.fold- Folding cross validation object, i.e KFold and StratifedKfold.metric- Evaluation metric.fit_params- Dictionary of parameter that should be fed to the fit method.refit_full- Flag indicting post fit behaviour. True will do a full refit on the full data, False will make it a stacked ensemble trained on the different folds.refit_params- Dictionary of parameter that should be fed to the refit if refit_full=False.verbose- Flag for printing fold scores during fit.Examplefromxklearn.modelsimportFoldLightGBM...base=LGBMClassifier(n_estimators=1000)fold=KFold(n_splits=5)fit_params={'eval_metric':'auc','early_stopping_rounds':50,'verbose':0}fold_lgbm=FoldLightGBM(base,fold=fold,metric=roc_auc_score,fit_params=fit_params,verbose=1)fold_lgbm.fit(X_train,y_train)fold_lgbm.predict(X_test)Output:Finished fold 1 with score: 0.9114
Finished fold 2 with score: 0.9265
Finished fold 3 with score: 0.9419
Finished fold 4 with score: 0.9189
Finished fold 5 with score: 0.9152
Finished with a total score of: 0.9225FoldXGBoostK-fold wrapped into an estimator that performs cross validation on a XGBoost over a selected folding method automatically when fit. Can optionally be used as a stacked ensemble of k estimators after fit.Argumentsxgb- Base estimator.fold- Folding cross validation object, i.e KFold and StratifedKfold.metric- Evaluation metric.fit_params- Dictionary of parameter that should be fed to the fit method.refit_full- Flag indicting post fit behaviour. True will do a full refit on the full data, False will make it a stacked ensemble trained on the different folds.refit_params- Dictionary of parameter that should be fed to the refit if refit_full=False.verbose- Flag for printing fold scores during fit.Examplefromxklearn.modelsimportFoldXGBoost...base=XGBRegressor(objective="reg:linear",random_state=42)fold=KFold(n_splits=5)fit_params={'eval_metric':'mse','early_stopping_rounds':5,'verbose':0}fold_xgb=FoldXGBoost(base,fold=fold,metric=mean_squared_error,fit_params=fit_params,verbose=1)fold_xgb.fit(X_train,y_train)fold_xgb.predict(X_test)Output:Finished fold 1 with score: 3212.8362
Finished fold 2 with score: 2179.7843
Finished fold 3 with score: 2707.8460
Finished fold 4 with score: 2988.6643
Finished fold 5 with score: 3281.4299
Finished with a total score of: 3274.9001StackClassifierEnsemble classifier that stacks an ensemble of classifiers by using their outputs as input features.Argumentsclfs- List of ensemble of classifiers.meta_clf- Meta classifier that stacks the predictions of the ensemble.keep_features- Flag to train the meta classifier on the original features too.refit- Flag to retrain the ensemble of classifiers during fit.Examplefromxklearn.modelsimportStackClassifier...meta_clf=RidgeClassifier()ensemble=[RandomForestClassifier(),KNeighborsClassifier(),SVC()]stack_clf=StackClassifier(clfs=ensemble,meta_clf=meta_clf,refit=True)stack_clf.fit(X_train,y_train)y_=stack_clf.predict(X_test)StackRegressorEnsemble regressor that stacks an ensemble of regressors by using their outputs as input features.Argumentsregs- List of ensemble of regressors.meta_reg- Meta regressor that stacks the predictions of the ensemble.drop_first: Drop first class probability to avoid multi-collinearity.keep_features- Flag to train the meta regressor on the original features too.refit- Flag to retrain the ensemble of regressors during fit.Examplefromxklearn.modelsimportStackRegressor...meta_reg=RidgeRegressor()ensemble=[RandomForestRegressor(),KNeighborsRegressor(),SVR()]stack_reg=StackRegressor(regs=ensemble,meta_reg=meta_reg,refit=True)stack_reg.fit(X_train,y_train)y_=stack_reg.predict(X_test)compress_dataframeReduce memory usage of a Pandas dataframe by finding columns that use larger variable types than unnecessary.Argumentsdf- Dataframe for memory reduction.verbose- Flag for printing result of memory reduction.Examplefromxklearn.utilsimportcompress_dataframe...train=compress_dataframe(train,verbose=1)Output:Dataframe memory decreased to 169.60 MB (64.6% reduction)