Implementing different kernels of SVC Algorithm on the Iris Dataset
By: Mahnoor Javed
Tags: Machine Learning, Artificial Intelligence, Python, Sklearn, Data Science

Code

Here, we will go through the coding segment. The dataset used for this implementation is the iris dataset which can be imported from the sklearn library.

Importing Dataset and Relevant Libraries/Modules

Firstly, we will import the numpy and matplotlib for mathematical manipulation of data and plotting the graphs. Then we will import the svm classifier and the iris dataset from the sklearn library.

import numpy as np

import matplotlib.pyplot as plt

from sklearn import svm, datasets iris = datasets.load_iris()

The Iris Dataset (Image by Author)

The Iris Dataset consists of 150 samples, each having 4 features listed: sepal length, sepal width, petal length, and petal width.

Features in the Iris Dataset (Image by Author)

The data has 3 classes: setosa, versicolor and virginica.

Classes of Iris: Setosa, Versicolor and Virginica (Images from Wikipedia)

Features & Target Numpy Array

Next, we will take the first two features of the iris data, which are its sepal length and sepal width, both in cm.

Our X variable will contain the features and y variable will contain the target.

X = iris.data[:, :2]

y = iris.target

The Features & Target Numpy Arrays (Image by Auhtor)

Model Creation

Next, we will create models of the svm algorithms on the Iris dataset and use different kernels to observe the results.

We will keep the value of C = 1.0 for all the models.

C = 1.0 # SVM regularization parameter

models = (svm.SVC(kernel='linear', C=C),

svm.LinearSVC(C=C, max_iter=10000),

svm.SVC(kernel='rbf', gamma=0.7, C=C),

svm.SVC(kernel='poly', degree=3, gamma='auto', C=C))

models = (clf.fit(X, y) for clf in models)

We have created 4 models:

svm.SVC with linear kernel and value of C kept at 1.0 svm.LinearSVC (which uses the liblinear), with same value of C svm.SVC with rbf kernel svm.SVC with kernel = ‘poly’, degree = 3, gamma = ‘auto’ and default value of C

Make Meshgrid

Next, we will define a function to create a meshgrid to plot our 4 models.

def make_meshgrid(x, y, h=.02):

x_min, x_max = x.min() - 1, x.max() + 1

y_min, y_max = y.min() - 1, y.max() + 1

xx, yy = np.meshgrid(np.arange(x_min, x_max, h),

np.arange(y_min, y_max, h))

return xx, yy

Plotting Contours

This function will be called to plot the decision boundaries for the classifiers.

def plot_contours(ax, clf, xx, yy, **params):

Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])

Z = Z.reshape(xx.shape)

out = ax.contourf(xx, yy, Z, **params)

return out

Plotting

Next, we will define the titles of each plot and set up a 2x2 grid to plot our 4 classifiers plots.

Then, we will fit the model one by one on the Iris Dataset and plot the results:

# title for the plots

titles = ('SVC with linear kernel',

'LinearSVC (linear kernel)',

'SVC with RBF kernel',

'SVC with polynomial (degree 3) kernel') # Set-up 2x2 grid for plotting.

fig, sub = plt.subplots(2, 2)

plt.subplots_adjust(wspace=0.4, hspace=0.4) X0, X1 = X[:, 0], X[:, 1]

xx, yy = make_meshgrid(X0, X1) for clf, title, ax in zip(models, titles, sub.flatten()):

plot_contours(ax, clf, xx, yy,

cmap=plt.cm.coolwarm, alpha=0.8)

ax.scatter(X0, X1, c=y, cmap=plt.cm.coolwarm, s=20, edgecolors='k')

ax.set_xlim(xx.min(), xx.max())

ax.set_ylim(yy.min(), yy.max())

ax.set_xlabel('Sepal length')

ax.set_ylabel('Sepal width')

ax.set_xticks(())

ax.set_yticks(())

ax.set_title(title) plt.show()

The following is the plot:

Plot of SVC using various kernels (Image by Author)

The sepal length has been taken on the x-axis whereas the sepal width has been taken on the y-axis.