Building a Machine Learning Pipeline with Scikit-Learn | by Ezgi Gumusbas | Towards Data Science

Data science projects need iterative progress. For example, we clean and prepare data for modeling with transforming to the proper format, run the model, get results, improve the model/change the model and work on feature engineering, get new results, compare them with other results, etc. It is not easy and smart to do every step again and again. To solve this problem, we can use a pipeline to integrate steps of machine learning workflow.
Pipelines are super useful for transforming and training data quickly. Additionally, we can compare different models and tune hyperparameters by integrating grid search in our pipeline. In this article, I write about how to create pipelines in scikit-learn to show the magical world of them.
There are many ways to make a pipeline but I will show one of the easiest and smart versions of them in this blog.
To use the pipeline function of scikit-learn we have to import the Pipeline module.
from sklearn.pipeline import Pipeline
By using a list of (key, value) pairs, the pipeline is built. Here, the key is a string containing the name you want to give and the value is the estimator object. Very simple example code to show how to use;
estimators = [('reduce_dim', PCA()), ('clf',SVC())]pipe = Pipeline(estimators)
For more details, you can check the scikit-learn documentation.
‘make_pipeline’ is a utility function that is a shorthand for constructing pipelines. It takes a variable number of estimates and returns a pipeline by filling the names automatically.
I would like to explain the usage of pipelines in examples because I think, it is easier to figure out when we see the application of the modules in code. In real-life, data sets generally consist of numerical and categorical columns. We have to transform these columns with different techniques. When we scale numerical columns by using a scaler, we should encode categorical columns with an encoder. It is easy to do this transform for the first time, but generally, in data science projects we would like to try different scalers and encoders. To make this quickly and easily, we use pipelines.
For one of my projects, I predict the status of water-wells in Tanzania by using classification techniques. I used different scalers, encoders, and classification models with the pipeline. If you want to see the full Jupiter notebook with data and how to use the pipeline in the modeling process, you can find it here on my Github. A pipeline example from that project;
I only show how to import the pipeline module here. But of course, we need to import all libraries and modules which we plan to use such as pandas, NumPy, RobustScaler, category_encoders, train_test_split, etc.
from sklearn.pipeline import make_pipeline
df = pd.read_csv('clean_data.csv')
If your data has some meaningless features, null/wrong values, or if it needs any type of cleaning process, you can do it at this stage. Because the quality of data affects the quality of the model. In this blog, my aim is to show the pipeline process so I skip this part and use the cleaned version of my data.
cat_col = ['basin','region','extraction_type_group','management','payment','water_quality','quantity','source','waterpoint_type','decade','installer_cat','funder_cat']              num_col = ['gps_height','longitude','latitude','district_code','population','public_meeting','permit']
target='status_group'used_cols = [c for c in df.columns.tolist() if c not in [target]]X=df[used_cols]y=df[target]X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# making pipelinescaler = RobustScaler()encoder = ce.TargetEncoder(cols=cat_col)# putting numeric columns to scaler and categorical to encodernum_transformer = make_pipeline(scaler)cat_transformer = make_pipeline(encoder)# getting together our scaler and encoder with preprocessorpreprocessor = ColumnTransformer(      transformers=[('num', num_transformer, num_col),                    ('cat', cat_transformer, cat_col)])# choosing modelmodel_name = LogisticRegression(class_weight = 'balanced', solver = 'lbfgs', random_state=42)# giving all values to pipelinepipe = make_pipeline(preprocessor,model_name)pipe.fit(X_train, y_train)# make predictions on training sety_pred = pipe.predict(X_train)# make predictions on test sety_pred_test = pipe.predict(X_test)# to print the results in good wayprint("Accuracy:"); print("="*len("Accuracy:"))print(f"TRAIN: {accuracy_score(y_train, y_pred)}")print(f"TEST: {accuracy_score(y_test, y_pred_test)}")print("\nBalanced Accuracy:"); print("="*len("Balanced Accuracy:"))print(f"TRAIN: {balanced_accuracy_score(y_train, y_pred)}")print(f"TEST: {balanced_accuracy_score(y_test, y_pred_test)}")
In this example, it is seen that we can add features to our pipelines such as the preprocessor which contains scaler and encoder. We also can add binner or more functions to the pipeline for different problems. This helps to transform our data in a quick and simple way. In this example, the make_pipeline function automatically applied scaler, encoder, and our model to the pipe and we fit it very easily.
When we write a function and put our pipeline in this function and return results, it is super easy to change the model also. For this example, when we only change the model_name to try another classification model and run the cell which calls the corresponding function, it easily turns results within the pipeline. In short, we do not need to change our data sets for transforming the data. We can do every transform in the pipeline and keep our data set originally.
We can also use make_pipeline to getting together imputer and scaler for numerical columns;
# Imputing nulls and scaling for numeric columnsnum_imputer = SimpleImputer(strategy='median')scaler = RobustScaler()# Imputing nulls through the encoding for categorical columnsencoder = ce.TargetEncoder(cols=cat_cols, handle_missing="value")# Defining different transformers for numeric and categorical columnsnum_transformer = make_pipeline(num_imputer, scaler)cat_transformer = make_pipeline(encoder)# getting together our scaler and encoder with preprocessorpreprocessor = ColumnTransformer(      transformers=[('num', num_transformer, num_col),                    ('cat', cat_transformer, cat_col)])# choosing modelmodel_name = LogisticRegression(class_weight = 'balanced', solver = 'lbfgs', random_state=42)# giving all values to pipelinepipe = make_pipeline(preprocessor,model_name)
In this example, I also added imputer to my pipeline. As you see, it is very easy to add and change modules when we build a good pipeline without transforming our data manually.
The aim of the pipeline is to combine several steps that can be cross-validated together while setting different parameters. With gathering these steps, it helps us to add new parameters or make changes to our model easily. Also, it makes our code more readable and understandable. By building an understandable work-flow, it helps the reproducibility of the project. By using pipeline, we do not transform our data at the beginning of the process. Pipeline does needed transforms for us and keep the original data.
If you would like to deep into scikit-learn library documentation, there are some useful links here.
scikit-learn.org
scikit-learn.org
If you have any feedback or suggestions for this article, feel free to connect with me via LinkedIn.