How to Efficiently Fine-Tune your Machine Learning Models
By: Khuyen Tran
Tags: Programming, Python, Data Science, Machine Learning, Pipeline

How to Efficiently Fine-Tune your Machine Learning Models

Motivation

Have you ever wanted to experiment with different parameters for your model but find it really time-consuming to do so? Let’s say you want to use support vector machines (SVM) to train and predict the model.

You debate what could be the best value for C so you keep experimenting with different values of C manually and hopefully get good results. That is time-consuming but still not so bad.

But what if you want to first use sklearn.feature_extraction.text.TfidfVectorizer to extract important texts, then use sklearn.feature_selection.SelectKBest to find the best features, and finally use sklearn.SVM.LinearSVC to train and predict the model? There would be so many parameters to tune in. Plus you need to find a way to record the results for comparison. So your notebook can look like a mess!

Is there a better way to select the parameters, shorten the code, as well as save the results for comparison? Yes, of course! Our method will be like a delicious cake with three simple ingredients:

Pipeline to combine different models

Grid search to find the best parameters for each model

MLflow to record the results

Have all the ingredients you need? Awesome! Now we are ready to put them together.

Photo by Drew Patrick Miller on Unsplash

Pipeline

What is pipeline? sklearn.pipeline.Pipeline allows us to combine a list of transforms and a final estimator into a chain.

For example, to predict whether a tweet is aggressive or not, after processing the text, we use tfidf_vectorizer, and SelectKBest as the transformers and svm.LinearSVC as a final estimator.

The codes above could be combined into a pipeline with a list of steps represented as tuples. In each tuple, the first parameter is the name of the class, and the second parameter is the class.

Much shorter and organized! We could easily switch any of the steps to another transformer or estimator. Make sure the classes in the pipeline are from sklearn. If they are not, you could easily create customized sklearn classes with def __init__, def fit(), and def transform() like this.

Find the example source code here

Grid Search CV

So we got the pipeline ready to transform and predict our data. But how about parameter tuning? We could easily find the best parameters for transformers and estimator in our pipeline with sklearn.model_selection.GridSearchCV.

Let’s say we want to find the best value of k for SelectKBest , the best value of C for svm.LinearSVC, best values of analyzer, ngram_range, and binary for TfidfVectorizer. We could give our grid several parameters to search for. The parameters are represented as lists nested inside a dictionary.

To set the parameters of a particular class, we use class_name__parameter = [para_1, para_2, para_3]. Make sure to have two underscores between class’s name and parameter.

grid_search.fit(X_train, y_train) creates several runs using different parameters with specified transformations, and estimator. The combination of parameters yielding the best result will be chosen for the transformation step.

We could also create other pipelines with different transformers and estimators and again use GridSearchCV to find the best parameters! But how could we save the results of all the runs we perform?

It is possible to observe the results by scrolling up and down the screen or taking photos of the results. But that is time-consuming, not taking into account that we might want to look at the results later for future projects. Is there a better way to track our results? That is when we need a tracking tool like MLflow

MLflow

MLflow is a tool for managing the end-to-end machine learning lifecycle including tracking experiments, packaging ML code in a reproducible form for sharing, and deployment. We will utilize this tool to keep the log of our results.

Installation of MLflow can be just simple as this

pip install mlflow

Let’s combine what we have done so far and MLflow tracking tool

In this code, we just add three elements to log the results:

mlflow.set_experiment('name_of_experiment') to create a new experiment

to create a new experiment mlflow.start_run() inside which the codes we want to run the experiment are in

inside which the codes we want to run the experiment are in mlflow.log_param() and mlflow.log_metric() to log the parameters and metrics

Save the file above as train.py. The last step is to run the file

python train.py

After the code finish running, we could find the log of MLflow by running

mlflow ui

Access to the link http://localhost:5000 and we should see this

As we can see from above, the metrics and the parameters of the experiments are logged! We could create several experiments and log the results of those experiments for easy and efficient comparison.

Conclusion

Congratulations! You have learned how to tune parameters for your machine learning models efficiently with Pipeline, GridSearchCV, and MLflow. You could find similar example codes for this article here. I encourage you to try out these methods with your existing machine learning projects. Just a little change in your coding approach can make a big difference in the long run. Experimenting and tracking the results efficiently will not only save you time but also make it much easier to find the best parameters and learn from your experiments.

I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.

Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these: