Package: skippa
SkippaSciKIt-learn Pre-processing Pipeline in PAndasRead more in theintroduction blog on towardsdatascienceWant to create a machine learning model using pandas & scikit-learn? This should make your life easier.Skippa helps you to easily create a pre-processing and modeling pipeline, based on scikit-learn transformers but preserving pandas dataframe format throughout all pre-processing. This makes it a lot easier to define a series of subsequent transformation steps, while referring to columns in your intermediate dataframe.So basically the same idea asscikit-pandas, but a different (and hopefully better) way to achieve it.pypiDocumentationInstallationpip install skippaOptional, if you want to use thegradio app functionality:pip install skippa[gradio]Basic usageImportSkippaclass andcolumnshelper functionimport numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression

from skippa import Skippa, columnsGet some datadf = pd.DataFrame({
    'q': [0, 0, 0],
    'date': ['2021-11-29', '2021-12-01', '2021-12-03'],
    'x': ['a', 'b', 'c'],
    'x2': ['m', 'n', 'm'],
    'y': [1, 16, 1000],
    'z': [0.4, None, 8.7]
})
y = np.array([0, 0, 1])Define your pipeline:pipe = (
    Skippa()
        .select(columns(['x', 'x2', 'y', 'z']))
        .cast(columns(['x', 'x2']), 'category')
        .impute(columns(dtype_include='number'), strategy='median')
        .impute(columns(dtype_include='category'), strategy='most_frequent')
        .scale(columns(dtype_include='number'), type='standard')
        .onehot(columns(['x', 'x2']))
        .model(LogisticRegression())
)and use it for fitting / predicting like this:pipe.fit(X=df, y=y)

predictions = pipe.predict_proba(df)If you want details on your model, use:model = pipe.get_model()
print(model.coef_)
print(model.intercept_)(de)serializationAnd of course you can save and load your model pipelines (for deployment).
N.B.dillis used for ser/de because joblib and pickle don't provide enough support.pipe.save('./models/my_skippa_model_pipeline.dill')

...

my_pipeline = Skippa.load_pipeline('./models/my_skippa_model_pipeline.dill')
predictions = my_pipeline.predict(df_new_data)See the./examplesdirectory for more examples:01-standard-pipeline.py02-preprocessing-only.py03a-gridsearch.py03b-hyperopt.py04-gradio-app.py05-PCA.pyTo DoSupport pandas assign for creating new columns based on existing columnsSupport cast / astype transformerSupport for .apply transformer: wrapper aroundpandas.DataFrame.applyCheck how GridSearch (or other param search) works with SkippaAdd a method to inspect a fitted pipeline/model by creating a Gradio app defining raw features input and model outputSupport PCA transformerFacilitate random seed in Skippa object that is dispatched to all downstream operationsfit-transform does lazy evaluation > cast to category and then selecting category columns doesn't work > each fit/transform should work on the expected output state of the previous transformer, rather than on the original dataframeInvestigate if Skippa can directly extend sklearn's Pipeline -> usinggetitemtrickUse sklearn's new dataframe output settingValidation of pipeline stepsInput validation in transformersTransformer for replacing values (pandas .replace)Support arbitrary transformer (if column-preserving)Eliminate the need to call columns explicitlyCreditsSkippa is powered byData Science Lab AmsterdamThis project structure is based on theaudreyr/cookiecutter-pypackageproject template.History0.1.16 (2023-08-17)Bugfix: missing _replace_none attribute for SimpleImputer with strategy='constant'0.1.15 (2022-11-18)Fix: when saving a pipeline, include dependencies in dill serialization.0.1.14 (2022-05-13)Bugfix in .assign: shouldn't have columnsBugfix in imputer: explicit missing_values arg leads to issuesUsed space-titanic data in examplesLogo added :)0.1.13 (2022-04-08)Bugfix in imputer: using strategy='constant' threw a TypeError when used on string columns0.1.12 (2022-02-07)Gradio & dependencies are not installed by default, but are declared an optional extra in setup0.1.11 (2022-01-13)Example added for hyperparameter tuning with Hyperopt0.1.10 (2021-12-28)Added support for PCA (including example)Gradio app support extended to regressionMinor cleanup and improvements0.1.9 (2021-12-24)Added support for automatic creation of Gradio app for model inspectionAdded example with Gradio app0.1.8 (2021-12-23)Removed print statement in SkippaSimpleImputerAdded unit tests0.1.7 (2021-12-20)Fixed issue that GridSearchCV (or hyperparam in general) did not work on Skippa pipelineExample added using GridSearch0.1.6 (2021-12-17)Docs, setup, readme updatesUpdated.apply()method so that is accepts a columns specifier0.1.5 (2021-12-13)Fixes for readthedocs0.1.4 (2021-12-13)Cleanup/fix in examples/full-pipeline.py0.1.3 (2021-12-10)Added.apply()transformer forpandas.DataFrame.apply()functionalityDocumentation and examples update0.1.2 (2021-11-28)Added.assign()transformer forpandas.DataFrame.assign()functionalityAdded.cast()transformer (with aliases.astype()&.as_type()) forpandas.DataFrame.astypefunctionality0.1.1 (2021-11-22)Fixes and documentation.0.1.0 (2021-11-19)First release on PyPI.