Pipelines in Machine Learning
By: Iftekher Mamun
Tags: Automation, Pipeline, Automl, Data Science, Machine Learning

Pipelines in Machine Learning

Why pipelines are an important aspect of ML

One of the biggest topics in Machine Learning is Automation. AWS has an amazing automation machine learning system set up where you just point and click and all data are calculated for you. This is built on the system based on creating pipeline.

Pipelines are useful tools that can automate the process and speed up time spent on some aspects of machine learning. Pipelines are used daily by data engineers, scientists and analysts to help automate some of the more daily data cleaning and processing that does not require much output. While AWS uses a rather complex set of networks of pipelines, I just want to discuss some of the small advantages of setting up pipelines and the codes required for it to happen.

A great comparison to the pipeline would be gridsearch. Gridsearch is another great tool that you can use to help optimize the hyperparameters of machine learning. You give it some range or specifications and your algorithm will run and display the most optimum of the specification given. However, we can further automate gridsearch by including a few simple pipeline in it. There are two ways to use the pipeline. The simplest and most well known is to import it from sci-kit learn directly:

from sklearn.pipeline import Pipeline

Following that, you can initialize the pipeline and set the required specification that you want to look for:

pipe = Pipeline([('mms', MinMaxScaler()),

('pca', PCA(n_components=10)),

('tree', tree.DecisionTreeClassifier(random_state=123))])

So the initialization is the Pipeline() itself. Then within that, we set what we want to see. First, we want to automate the Min_Max_Scaler. Following that, we will run our features through a Principal Component Analysis- PCA- to help with feature reduction. The n_componenets just means how many total features you want. If you have fewer features then your n-components, then those extra features will come up as 0 in separate columns. Then we see that we are going to run those two on a decision tree, using a DecisionTreeClassifier. These simple lines have set up a pipeline that now you can on your own data. The dataset then will run through this pipeline and scale your data accordingly, then apply PCA and finally run it through the tree classifier. To fit it into your data, remember to first have your data be split using train, test, split. Then the following code:

pipe.fit(X_train, y_train)

That’s it. You have created a pipeline. I mentioned above that you can also implement pipeline into your gridsearch model as well. To do that, you have to do the following:

# Create the pipeline

pipe = Pipeline([('scl', MinMaxScaler()),

('pca', PCA(n_components=10)),

('svm', svm.SVC(random_state=123))])

In this example code, the machine learning algorithm will go through Support Vector Machine instead of Decision Tree. SVM is useful when you have unlabeled data and you want to use unsupervised learning to find the underlying data you are looking for.

# Create the grid parameter

grid = [{'svm__kernel': ['poly', 'sigmoid'],

'svm__C': [0.01, 1, 100],

'svm__degree0': [2,3,4,5],

'svm__gamma': [0.001, 0.01]}]

Now we create a basic grid parameter. Those are just different components/tools of the SVM algorithm.

#Input pipeline and grid into gridsearch

gridsearch = GridSearchCV(estimator=pipe,

param_grid=grid,

scoring='accuracy',

cv=3)

Finally, we fit our data to gridsearch this time, instead of the pipeline like above:

# Fit using grid search

gridsearch.fit(X_train, y_train)

That’s pretty much how most people do it conventionally. But I will also post the official sklearn documentation below so you can take a look at it yourself.