Implementing a Naive Bayes Classifier | by Tarun Gupta | Towards Data Science

In the context of Supervised Learning (Classification), Naive Bayes or rather Bayesian Learning acts as a gold standard for evaluating other learning algorithms along with acting as a powerful probabilistic modelling technique.
In this post, we are going to discuss the workings of Naive Bayes classifier implementationally with Python by applying it to a real world dataset.
The post is divided more broadly into the following parts:
Data Preprocessing
Training the model
Predicting the results
Checking the performance of the model
The above parts can be further divided as follows:
→ Data Preprocessing
Importing the librariesImporting the datasetSplitting the dataset into the training set and testing setFeature Scaling
Importing the libraries
Importing the dataset
Splitting the dataset into the training set and testing set
Feature Scaling
→ Training the model
Training the Naive Bayes model on the training set
Training the Naive Bayes model on the training set
→ Predicting the results
Predicting the test set results
Predicting the test set results
→ Checking the performance of the model
Making the Confusion Matrix
Making the Confusion Matrix
→ Visualisation
Visualising the Confusion Matrix
Visualising the Confusion Matrix
Before we begin to delve into the code itself, we need to talk about the dataset itself. For this implementation, we are going to use The 20 newsgroups text dataset. This dataset is publicly available for the purpose that people can learn and hone their Machine Learning skills.
We are going to use scikit-learn (sklearn) as the Machine Learning library and the repository for the dataset itself. This is what sklearn’s website tells you about the dataset.
The 20 newsgroups dataset comprises around 18000 newsgroups posts on 20 topics split in two subsets: one for training (or development) and the other one for testing (or for performance evaluation). The split between the train and test set is based upon a messages posted before and after a specific date.
Note: Since the raw data is natural language text, we can’t directly work with it. We need to transform the data into numbers before we can start working on it. There are multiple ways to do that, namely:
CountVectorizer
TfIdfVectorizer
HashingVectorizer
It depends on the designer of the solution and the problem itself to choose the Vectorizer. For the purpose of this post, I am going to use CountVectorizer. I have provided the link to sklearn’s documentation for all of the Vectorizers. You can certainly have a detailed look there and over the Internet as well.
Now, that we have talked about the dataset a bit, let’s start with the code itself one step at a time.
Importing the libraries
Importing the libraries
from sklearn.datasets import fetch_20newsgroupsfrom sklearn.feature_extraction.text import CountVectorizerfrom sklearn.naive_bayes import GaussianNBfrom sklearn.metrics import confusion_matrix, plot_confusion_matriximport matplotlib.pyplot as plt
The above-mentioned libraries are used/required (explained in order) as follows:
a) Importing the dataset from sklearn itself
b) Importing the CountVectorizer to convert raw natural language text to machine understandable numbers
c) Importing the Naive Bayes classifier, in this case we are using Gaussian Naive Bayes
d) Importing the confusion matrix methods to check the performance of the model and visualise it.
e) For visualisation of Confusion Matrix
2 & 3. Importing the dataset and Splitting the dataset into the Training set and Test set
data_train = fetch_20newsgroups(subset='train', categories=None,                                remove=('headers', 'footers',                                       'quotes'))data_test = fetch_20newsgroups(subset='test', categories=None,                               remove=('headers', 'footers',                                'quotes'))X_train = data_train.datay_train = data_train.targetX_test = data_test.datay_test = data_test.target
data_train contains the training set from the data itself. The parameters passed in fetch_20newsgroups can be understood as:
i) subset — to define training or test set
ii) categories — The dataset contains 20 categories or class labels. The subset of the dataset can be used by providing a list of categories in the parameter from the following:
from pprint import pprintpprint(list(newsgroups_train.target_names))
The output being:
['alt.atheism', 'comp.graphics', 'comp.os.ms-windows.misc', 'comp.sys.ibm.pc.hardware', 'comp.sys.mac.hardware', 'comp.windows.x', 'misc.forsale', 'rec.autos', 'rec.motorcycles', 'rec.sport.baseball', 'rec.sport.hockey', 'sci.crypt', 'sci.electronics', 'sci.med', 'sci.space', 'soc.religion.christian', 'talk.politics.guns', 'talk.politics.mideast', 'talk.politics.misc', 'talk.religion.misc']
iii) remove — The text in the dataset contains headers, footers and quotes but we want to apply the model on the body of the data.
The independent and dependent variables can be separated by calling .data and .target on the data_train and data_test to get bifurcated training and test data.
The values in the array represents the index of the above mentioned categories list.
4. Feature Scaling
vectorizer = CountVectorizer()X_train = vectorizer.fit_transform(X_train)X_test = vectorizer.transform(X_test)
Making an object of the CountVectorizer class followed by fitting the vectorizer object on both X_train and X_test data.
Training the Naive Bayes model on the training set
Training the Naive Bayes model on the training set
classifier = GaussianNB()classifier.fit(X_train.toarray(), y_train)
Making an object of the GaussianNB class followed by fitting the classifier object on X_train and y_train data. Here .toarray() with X_train is used to convert a sparse matrix to a dense matrix.
Predicting the test set results
Predicting the test set results
y_pred = classifier.predict(X_test.toarray())
Calling the .predict method on the classifier object and passing the X_test to predict the results of the trained model on previously unseen data. Here .toarray() with X_test is used to convert a sparse matrix to a dense matrix.
Making the Confusion Matrix
Making the Confusion Matrix
cm = confusion_matrix(y_test, y_pred)print(cm)
Visualising the Confusion Matrix
Visualising the Confusion Matrix
plot_confusion_matrix(classifier, X_test.toarray(), y_test, display_labels=['alt.atheism', 'comp.graphics', 'comp.os.ms-windows.misc', 'comp.sys.ibm.pc.hardware'], cmap=plt.cm.Blues)
For visualisation purpose I am limiting the number of categories to 4 so that it is properly visible.
Now, that the whole pipeline of a Machine Learning model is finished, I hope I was able to part some knowledge. This is a very basic Machine Learning pipeline but it is rather important in terms of building a foundation when you want to build better and more complex Machine Learning models. I hope to bring more dynamic and complex models in the future so stay tuned.
Here is a link to the complete jupyter notebook.
github.com
I am giving away a free eBook on Consistency. Get your free eBook here.
Thank you for reading.
If you enjoy reading stories like these and want to support me as a writer, consider signing up to become a Medium member. It’s $5 a month, giving you unlimited access to stories on Medium. If you sign up using my link, I’ll earn a small commission at no extra cost to you.
tarun-gupta.medium.com
You can read more of my posts here: