Naïve Bayes from Scratch using Python only— No Fancy Frameworks | by Aisha Javed | Towards Data Science

So in my previous blog post of Unfolding Naïve Bayes from Scratch! Take-1 🎬, I tried to decode the rocket science behind the working of The Naïve Bayes (NB) ML algorithm, and after going through it’s algorithmic insights, you too must have realized that it’s quite a painless algorithm. In this blog post, we will walk-through it’s complete step by step pythonic implementation ( using basic python only) and it will be quite evident that how easy it is to code NB from scratch and that NB is not that Naïve at classifying !
Since I always wanted to decipher ML for absolute beginners and as it is said that if you can’t explain it, you probably didn't understand it, so yeah this blog post too is especially intended for ML beginners.
As I just mentioned above, a complete walk-through of NB pythonic implementation
Milestone # 1: Data Preprocessing Function
Milestone # 2: Implementation of NaiveBayes Class — Defining Functions for Training & Testing
Milestone # 3: Training NB Model on Training Dataset
Milestone # 4: Testing Using Trained NB Model
Milestone # 5: Proving that the Code for NaiveBayes Class is Absolutely Generic!
Before we begin writing code for Naive Bayes in python, I assume you are familiar with:
Python ListsNumpy & just a tad bit of vectorized codeDictionariesRegex
Python Lists
Numpy & just a tad bit of vectorized code
Dictionaries
Regex
Let’s Begin the with the Pythonic Implementation !
Let’s begin with a few imports that we would need while implementing Naive Bayes
Milestone # 1 Achieved 👍
The Bonus Part : We will be writing a a fully generic code for the NB Classifier! No matter how many classes come into the training dataset and whatever text dataset is given — it will still be able to train a fully working model.
The code for NaiveBayes is just a little extensive — but we just to need to spend a maximum of 10–15 minutes to grasp it! After that, you will have a sound grip over all crucial concepts.
There are in total four functions defined in the NaiveBayes Class:
1. def addToBow(self,example,dict_index)2. def train(self,dataset,labels)3. def getExampleProb(self,test_example)4. def test(self,test_set)
And the code is divided into two major functions i.e train & test functions. Once you understand the statements defined inside these two functions, you will surely get to know what the code is actually doing and in what order the other two functions are being called.
1. Training function that trains NB Model :   def train(self,dataset,labels)2. Testing function that is used to predict class labels    for the given test examples :   def test(self,test_set)
The other two functions are defined to supplement these two major functions
1. BoW function that supplements training function    It is called by the train function.   It simply splits the given example using space as a tokenizer    and adds every tokenized word to its corresponding BoW :    def addToBow(self,example,dict_index)2. Probability function that supplements test function.    It is called by the test function.   It estimates probability of the given test example so that    it can be classified for a class label :   def getExampleProb(self,test_example)
You can view the above code in this Jupyter Notebook too
It is much much more easier to organize and reuse the code if we define a class of NB rather than use the traditional structured programming approach. That’s the reason of defining a NB class and all it’s relevant functions inside it.
We don’t just want to write code, rather we want to write beautiful, neat & clean, handy, and reusable code . Yes that’s right - we want to possess all the traits that a good data scientist could possibly have !
And guess what? Whenever we will be dealing with a text classification problem that we intend to solve using NB, we will simply instantiate its object and by using the same programming interface, we will be able to train a NB classifier. Plus, as its the general principle of Object Oriented Programming that we only define functions relevant to a class inside that class, so all functions that are not relevant to NB class will be defined separately
Milestone # 2 Achieved 👍 👍
And yes that’s it !Just a matter of four functions and we are all set to go for training our NB Model on any text dataset and with any number of class labels!
If you are curious to know what the training data actually looks like ..... It’s a newsgroups dataset consisting of newsgroups posts on 20 topics . It has 20 classes, but for the time being, we will train our NB model on just four categories — [‘alt.atheism’, ‘comp.graphics’, ‘sci.med’, ‘soc.religion.christian’] but the code works perfectly well for training against all 20 categories as well.
You might be wondering why the column of “Training Labels” is in numeric form rather than their original string textual form. It’s just that every string label has been mapped to it’s unique numeric Integer form. Even if this is unclear to you at the moment, just consider that a dataset has been provided and it has it’s labels in numeric form. Simple !
So before we start training a NB Model, let’s load this dataset...We will load a dataset from sklearn ( python’s ML Framework)— but we still coded NB from scratch!
Let’s actually begin Training!
Training is Completed !!!
Milestone # 3 Achieved 👍 👍 👍
So now that we have trained our NB Model — let’s move to testing!Loading the test set.....
Testing on above loaded test examples using our trained NB Model....
Wow! Pretty Good Accuracy of ~ 93% ✌️ See now you realize NB is not that Naïve !
Milestone # 4 Achieved 👍 👍 👍👍
As I mentioned in the beginning that the code we have written is generic, so let’s use the same code on a different dataset and with different class labels to prove it’s “genericity” !
The other text dataset consists of movie reviews and their sentiments & looks something like below:
Notice See how the same NaiveBayes code works like a charm on different datasets and yet with same programming interface !
Milestone # 5 Achieved 👍 👍 👍👍👍
So that’s all for this blog post and now you know it all!
Upcoming post will include :
Unfolding Naïve Bayes from Scratch! Take-3 🎬 Implementation of Naive Bayes using scikit-learn (Python’s Machine Learning Framework)
Until that Stay Tuned 📻 📻 📻
If you have any thoughts, comments, or questions, feel free to comment below or connect 📞 with me on LinkedIn