Train your Machine Learning Model 150x Faster with cuML
By: Khuyen Tran
Tags: Machine Learning, Sklearn, Nvidia, Data Science, Gpu

Train your Machine Learning Model 150x Faster with cuML

And how Triple your Graphic Card’s Memory Speeds the Training 10X Faster!

Motivation

Sklearn is a great library with a variety of machine learning models that you can use to train your data. But if your data is big, it might take you a long time to train your data, especially when you experiment with different hyperparameters to find the best model.

Is there a way that you can increase the speed of training your machine learning model by 150 times faster than using Sklearn with minimal change? Yes, you can do that with cuML.

Below is the chart that compares the time it takes to train the same model using Sklearn’s RandomForestClassifier and cuML’s RandomForestClassifier.

cuML is a suite of fast, GPU-accelerated machine learning algorithms designed for data science and analytical tasks. Its API is similar to Sklearn’s. This means you can use the same code you use to train Sklearn’s model to train cuML’s model.

In this article, I will compare the performance of these 2 libraries using different models. I will also show how increasing the memory of the graphic card can increase the speed by 10 times.

Install cuML

To install cuML, follow the instructions to install on Rapids’ page. Make sure to check the prerequisites before installing the library. You can either install all packages or just cuML. If you have limited space on your computer, I recommend installing cuDF and cuML.

Although in many cases, you don’t need to install cuDF to use cuML, cuDF is a nice complement for cuML since it is a GPU DataFrame.

Make sure to choose the options that fit your computer.

Create Data

Since cuML often works better than Sklearn when there is a lot of data, we will create a dataset with 40000 rows of data using sklearn.datasets.

from sklearn import datasets

X, y = datasets.make_classification(n_samples=40000)

Convert the datatypes to np.float32 because some cuML models require the input to be np.float32.

X = X.astype(np.float32)

y = y.astype(np.float32)

Support Vector Machine

We will create the function to train the model. Using this function will make it easier for us to compare different models.

def train_data(model, X=X, y=y):

clf = model

clf.fit(X, y)

We use %timeit , an iPython’s magic command to run each function 7 times and take the average of all experiments.

Average time of sklearn's SVC 48.56009825014287 s

Average time of cuml's SVC 19.611496431714304 s

Ratio between sklearn and cuml is 2.476103668030909

cuML’s SVC is 2.5 times faster than sklearn’s SVC!

Let’s visualize it through the plot. We create a function to plot the speed of the models. I use cutecharts because it makes the bar charts cuter.

Better Graphics Card

Since cuML’s models are faster than Sklearn’s when running on large data because they were trained using GPU, what will happen if we triple the memory of the GPU?

I use an Alienware M15 laptop with NVIDIA GeForce 2060 and a graphics card’s memory of 6.3 GB for the previous comparison.

Now I will use a Dell Precision 7740 with NVIDIA Quadro RTX 5000 and a graphics card’s memory of 17 GB to test the speed when the memory of the GPU increases.

Average time of sklearn's SVC 35.791008955999914 s

Average time of cuml's SVC 1.9953700327142931 s

Ratio between sklearn and cuml is 17.93702840535976

Wow! cuML’s SVM is 18 times faster than Sklearn’s SVM when it is trained on a machine with a graphics card’s memory of 17 GB! And it is 10 times faster than the speed of training on a laptop with a graphics card’s memory of 6.3 GB.

That is why it is better for us to have a good GPU in our machines if we are using GPU-accelerated libraries like cuML.

Random Forest Classifier

Average time of sklearn's RandomForestClassifier 29.824075075857113 s

Average time of cuml's RandomForestClassifier 0.49404465585715635 s

Ratio between sklearn and cuml is 60.3671646323408

cuML’s RandomForestClassifier is 60 times faster than Sklearn’s RandomForestClassifier! If it takes 30 seconds for you to train a Sklearn’s RandomForestClassifier, it takes less than half of a second to train a cuML’s RandomForestClassifier. How crazy is that?

Better Graphics Card

Average time of Sklearn's RandomForestClassifier 24.006061030143037 s

Average time of cuML's RandomForestClassifier 0.15141178591425808 s.

The ratio between Sklearn’s and cuML is 158.54816641379068

cuML’s RandomForestClassifier is 158 times faster than Sklearn’s RandomForestClassifier when trained on my Dell Precision 7740 laptop!

Nearest Neighbors Classifier

Average time of sklearn's KNeighborsClassifier 0.07836367340000508 s

Average time of cuml's KNeighborsClassifier 0.004251259535714585 s

Ratio between sklearn and cuml is 18.43304854518441

Note: 20m on the y-axis means 20ms.

cuML’s KNeighborsClassifier is 18 times faster than Sklearn’s KNeighborsClassifier.

Better Graphics Card

Average time of sklearn's KNeighborsClassifier 0.07511190322854547 s

Average time of cuml's KNeighborsClassifier 0.0015137992111426033 s

Ratio between sklearn and cuml is 49.618141346401956

cuML’s KNeighborsClassifier is 50 times faster than Sklearn’ KNeighborsClassifier when trained on my Dell Precision 7740 laptop.

Summarize

You can find the code for the rest of the comparisons here.

Here are the two tables that summarize the speed of different models between 2 libraries on:

Alienware M15 — GeForce 2060 and a graphic card’s memory of 6.3 GB

Dell Precision 7740 — Quadro RTX 5000 and a graphic card’s memory of 17 GB

Pretty impressive, isn’t it?

Conclusion

Congratulations! You have just learned how fast training different models on cuML is compared to Sklearn. If it takes a long time to train your model using Sklearn, I highly recommend giving cuML a try, considering there is literally no change in code at all compared to Sklearn’s API.

And of course, with a library that uses GPU to execute the code like cuML, the better graphic card you have, the faster the training will be on average.

Check out the doc of cuML for details on other machine learning models.

The source code for this article could be found here.

I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.

Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these: