Accelerate Kaggle Challenges Using Intel AI Analytics Toolkit
By: Kirill Petrov
Tags: Machine Learning, Data Analytics, Artificial Intelligence, Scikit Learn, Intel

Photo by Mathew Schwartz on Unsplash

Accelerate Kaggle Challenges Using Intel AI Analytics Toolkit

A few months ago, some of my colleagues showed how to improve machine learning performance using the optimized scikit-learn (SKL) in the Intel Distribution for Python (IDP), which is part of the Intel AI Analytics Toolkit:

Installing daal4py and changing a couple of lines of code significantly improved times for the scikit‑learn_bench benchmark. Internally, daal4py uses Intel oneAPI Data Analytics Library (oneDAL) to deliver best performance. Installation is a simple as:

$ conda install -c intel daal4py

Once installed, you can accelerate SKL applications in either of two ways. First, you can simply load the daal4py module from the Python command-line, i.e.:

$ python -m daal4py your_application.py

This is fine for testing and experimentation, but you can also patch scikit-learn inside your Python program before importing scikit-learn modules, i.e.:

import daal4py.sklearn

daal4py.sklearn.patch sklearn()

We applied this patch to the Jupyter notebooks from various Kaggle challenges to demonstrate the performance gains that are possible for real-world workloads (Table 1). The Intel-optimized SKL improves the performance of these notebooks by as much as 227x, with little or no code modification. Most of the notebooks experience double-digit speedups. Even the least impressive speedup is still twice as fast as the baseline performance.

Table 1. The Intel-optimized version of scikit-learn yields significant performance improvements for common machine learning algorithms.

kddcup99-knn-classification (KDD Cup 1999): This computer networking (CN) challenge asked competitors to build a predictive model to detect unauthorized network intrusions. This notebook used the k-nearest neighbors (KNN) algorithm.

p2-sklearn-svm-hyperparameter-optimization (Credit Card Default): This challenge asked competitors to predict the likelihood of credit card payment default. This notebook used the support vector clustering (SVC) algorithm.

digit-recognition-using-knn [Digit Recognizer (KNN)] and gridsearch-svc [Digit Recognizer (SVC)]: This is the familiar image classification (IC) problem of recognizing hand-drawn digits. These notebooks used the KNN, SVC, and principal component analysis (PCA) algorithms.

melanoma-svc-32x32 (Melanoma Identification): This IC challenge asked competitors to identify cancerous lesions in medical images. This notebook used the SVC algorithm.

xg-svm-tf-cooking (What’s Cooking?): Competitors were asked to predict cuisine category based on the list of ingredients in the dish. The notebook for this natural language processing (NLP) challenge used SVC and the XGBoost implementation of the gradient boosted decision trees algorithm.

transformer-svm-semantically-identical-tweets (Real or Not? Disaster Tweets): This NLP task challenges competitors to predict whether a Tweet is about a real disaster. This notebook used SVC.

random-forest-k-fold-cross-validation (Home Credit Default): This challenge asked competitors to predict whether an applicant would be able to repay a home loan. This notebook used the random forest algorithm.

Daal4py adds more accelerated scikit-learn algorithms to each release, learn what algorithms are optimized here. Don’t leave performance on the table when using SKL. Accelerate these workloads with Intel AI Analytics Toolkit to see the performance improvements for yourself.

Hardware and Software

Intel Xeon Gold 5218 @ 2.3 GHz (2nd generation Intel Xeon Scalable processors): 2 sockets, 16 cores per socket, HT:off, Turbo:off. OS: Red Hat Enterprise Linux 8.0, total memory of 193 GB (12 slots/16 GB/2933 MHz). Software: Scikit-learn 0.23.2, daal4py 2020.3, Numpy 1.18.5, and XGBoost 1.2.1.