Support Vector Machine with Python
By: Nikhil Adithyan
Tags: Programming, Machine Learning, Data Science, Python, Algorithms

Data processing

Let’s first have a look at the data type of the variables in our cancer data. To do this we can use the ‘dtypes’ function provided by the Pandas package in python.

Python Implementation:

Output:

Image by Author

It looks like the ‘BareNuc’ column includes values that are not numerical. Using the ‘astype’ function and the ‘numeric’ we can convert object type variables into integer variables. Follow the code to convert the values in python.

Python Implementation:

Output:

Image by Author

Our next step is to define the independent variable and the dependent, and using that we will be splitting our data into a training set and testing set.

Feature Selection & Train Test Split

As I said before, we are going to define the X and Y variables. After defining the variables, it is highly recommended to convert them into arrays that can be helpful while building the model. Follow the code to define the variables in python.

Python Implementation:

Output:

Image by Author

Now we can split the data using our defined X and Y variables into a training set and testing set. To do this, we can use the ‘train_test_split’ function provided by scikit-learn in python.

Python Implementation:

Output:

Image by Author

Modeling & Prediction

The SVM algorithm offers a choice of kernel functions for performing its processing. Basically, mapping data into a higher dimensional space is called kernelling. The mathematical function used for the transformation is known as the kernel function, and can be of different types, such as:

Linear

Polynomial

Radial Basis Function (RBF)

Sigmoid

Each of these functions has its characteristics, its pros and cons, and its equation, but as there’s no easy way of knowing which function performs best with any given dataset, we usually choose different functions in turn and compare the results. In this article, we are going to use the Radial Basis Function (RBF) kernel to build and train our model. Let’s do it in python!

Python Implementation:

Output:

Image by Author

With our trained SVM model, we can pass on some test values into it to make some predictions. Follow the code to make predictions in Python.

Python Implementation:

Output:

Image by Author

Evaluation

Now that we have built, trained, and made some predictions using our SVM model. In order to check the accuracy of our model results, we can use the evaluation metrics functions provided by scikit-learn in python. In this article, we are going to use the ‘accuracy_score’ metric and the ‘confusion_matrix’ metric to evaluate our model. Let’s start with the ‘accuracy_score’ evaluation metric in python.

Python Implementation:

Output:

Image by Author

Our next evaluation metric is ‘confusion_matrix’. Instead of just printing the confusion matrix, it will make more sense when it is being graphed or plotted. Even though there are built-in functions to make a confusion matrix plot, it will be more understandable when it is plotted manually. Follow the code to produce a confusion matrix plot in python.

Python Implementation:

Output:

Image by Author

Final Thoughts!

In this article, we walked through the basics of SVM, its advantages, and its disadvantages. Followed by that, we have learned to program SVM models in python to deal with the cancer data and made some predictions too. Finally, we learned to evaluate our model using the evaluation metrics in python. The only thing we missed out is the math part. So, don’t forget to cover it before you move on to the next concept. Also, in this article, we have utilized only one kernel and there are other kernels too. So ensure that you practice with that too to solve problems. With that, we come to the end of this article and I’ve provided the source code for the built SVM models at the end of the article.

Happy Machine Learning!

Full code: