Classifying Text Reviews of Amazon Products Using Naive Bayes
By: Gaurav Thapa
Tags: Machine Learning, NLP, Naive Bayes, Data Science, Towards Data Science

Working of Naive Bayes

Let us roughly see how Naive Bayes works in text classification. Suppose we have a review saying “great product” in our corpus and we want to classify its Response. Naive Bayes first calculates the probabilities of the “Positive” and “Negative” responses.

Next, it calculates the probabilities of getting the word ‘great’ in both a “Positive” and in a “Negative” review. It does the same for the word “product”.

The calculations done so far can be summarized in the form of a frequency table.

Now, the Naive Bayes classifier calculates scores which help indicate whether the review is “Positive” or “Negative”. For the “Positive” label, the score is calculated as follows.

Similarly, for the “Negative” label.

These two scores are respectively proportional to the probabilities of the review being a “Positive” one or a “Negative” one, given the fact that the review says “great product”. Since the score for the “Positive” label is greater, Naive Bayes classifies the review as “Positive”.

Fitting an H2O Naive Bayes Model

Since Naive Bayes works best when all the independent variables are categorical, we shall convert the frequency of each term into a factor variable with labels “Yes” if frequency is more than 0, and “No” if it is equal to 0. Then we split the data into train and test datasets in the ratio of 8:2. Using simple prop.table() function, we can check if the distribution of the Response labels in the two datasets are equal and also get a baseline accuracy. Since roughly 77% of the data has Response as “Positive” in both the datasets, so even if we label all the data as “Positive”, we are sure to get an accuracy of 77%(baseline).

Fig 8. Getting a baseline accuracy.

Now we proceed to make an H2O Naive Bayes model from the training data. To account for any term which is present in the training data but might be missing in the testing data, we take the value of the hyperparameter laplace equal to 1. This ensures that we do not get the probability of any term from the testing data which is not present in the training data as 0. The said model gives us results with accuracy of almost 83%, which is better than the baseline.

Model Improvement: N-Gram Tokenizer

So far we have only considered the occurrence of single words in the text files while determining the Response type. In the two barplots shown in Fig. 6, we can see many words appearing in both the Positive and Negative responses. Thus, it is difficult to clearly interpret the word-to-response association by merely looking at the single words.

Tokenization is the process of splitting the text into a list of tokens, which can be anything: individual characters, words or even sentences. What an n-gram tokenizer does is split each text file into sequences containing n consecutive words. Adjacent words offer more meaning to their association with the corresponding Response type and therefore might lead to better results.

Fig 9. Creating a DTM using a 2-gram tokenizer.

Just like the aforementioned Fig. 6, we can make barplots showing the frequency of different two-word combinations occurring in the text files according to their Response type.