Are you using Pipeline in Scikit-Learn? | by Ankit Goel | Towards Data Science

If you are doing Machine Learning, you would have come across pipelines as they help you to make a better machine learning workflow which is easy to understand and reproducible.
In case you are not aware of the pipelines you can refer awesome blogs from Rebecca Vickery “A Simple Guide to Scikit-learn Pipelines” and Saptashwa Bhattacharyya “A Simple Example of Pipeline in Machine Learning with Scikit-learn”.
I recently discovered that you can combine Pipeline with GridSearchCV to not only find best hyperparameters for your model but can also find the best transformers for your machine learning tasks like-
Scaler to scale your data.Impute strategy to fill missing values.The number of components in PCA you should use.
Scaler to scale your data.
Impute strategy to fill missing values.
The number of components in PCA you should use.
and many others. Let’s see how it can be done.
To best demonstrate, I am going to use the Titanic dataset from OpenML here to walkthrough on how you can create a data pipeline.
You can download the dataset using the following commands-
from sklearn.datasets import fetch_openml# Dataset details at- https://www.openml.org/d/40945X, y = fetch_openml("titanic", version=1, as_frame=True, return_X_y=True)
Also, have a look at my notebook which have more details about each operation, feel free to download and import it in your environment and play around-
github.com
I am going to use a subset of features for the demo purposes here. The dataset has both categorical and numerical features and we would be applying different operations on them. Here is the flow-
1. Define numerical and categorical features.
2. For numerical features, fill the missing values and then scale the data.
The numerical features in the dataset have few missing values and we are using SimpleImputer here to fill these values, other imputers can be used as well. Also, we should standardize the range of numerical features using Scaling as many machine learning models require features that follow the same scale. I have not defined the strategy for imputing and the scaler need to be used as the same can be searched using GridSearchCV.
3. For categorical features, fill the missing values and then apply one-hot encoding.
The ‘Embarked’ feature in our dataset has few missing values. The reason to create a different imputer for this as we have different strategies to fill categorical features. Also, as categorical features have only a few values, I am using OneHotEncoder which is commonly used to create different features for each value in categorical columns. The strategy of imputing is searched using GridSearchCV.
4. Combine numerical and categorical transformer using ColumnTransformer.
ColumnTransformer helps to define different transformers for different types of inputs and combine them into a single feature space after transformation. Here we are applying numerical transformer and categorical transformer created above for our numerical and categorical features.
5. Apply PCA to reduce dimensions.
Principle Component Analysis aka PCA is a linear dimensionality reduction algorithm that is used to reduce the number of features in the dataset by keeping the maximum variance. We might not need to apply PCA here as it is a small dataset but the idea is to show how you can search for number_of _components in PCA using GridSearchCV.
Above, we created a preprocessor pipeline with all the operations we would like to apply to our data. I am going to use Logistic Regression here as a classifier for our problem. In case you are interested to know how you can apply multiple models, Rebecca Vickery has mentioned a nice way of doing it in her blog.
Every Data Scientist would be aware of GridSearchCV which helps us to find the best hyperparameters of the model. One thing which is missed is that you can even use it to find out the best transformers with the help of Pipeline. In the preprocessor pipeline we created above, you might have noticed that we didn’t define-
The Scaler,Strategy for imputing missing values,The number of components in PCA
The Scaler,
Strategy for imputing missing values,
The number of components in PCA
as we can use GridSearchCV to find the optimal value for them. Here is how it can be done-
Above, I am trying to find the best Scaler out of 3 different scalers and Impute strategy for our numerical and categorical columns as well as components for PCA along with hyper-parameters for our model. Helpful, isn’t it?
Scikit-Learn 0.23.1 has added the functionality to visualize composite estimators which can be very helpful to cross-check the steps you applied. The below code can help you to visualize the data pipeline.
Here I created many different transformers on the data which can be computationally intensive. The pipeline provides a parameter ‘memory’ which can be used to cache the transformer if the parameters and input data are identical. We can set ‘memory’ as the directory path where to cache the transformers or it can be a joblib.Memory object. Below, I am using the current directory as the cache directory where all the cache objects for the transformer would be created.
cache_dir = "."cached_pipeline = Pipeline(estimators, memory=cache_dir)
I use Pipeline all the time as it is super easy to implement and helps us to get rid of many problems. I hope this would help you guys as well to improve your machine learning workflow. If you are just starting your data science journey or would like to learn about some cool python libraries for data science, then check out this- Start your Data Science journey today.
Stay Safe !!! Keep Learning !!!