Naive Bayes Classifiers for Text Classification | by Pınar Ersoy | Towards Data Science

Text classification can be described as one of the major subdivisions of the Natural Language Processing field. The text-based datasets composed of unstructured formatted content. This nature causes them to extract meaningful pieces in a difficult way. To be able to cope with this task, it is critical to choose the corresponding technique. Serving this purpose, Naive Bayes Classifier approaches can be performed by considering the structure types of the algorithms.
The general formula of the Bayes Theorem can be simply highlighted as the following.
P (X|Y) = ( P(Y|X) * P(X) ) / P(Y)
By observing the formulation, X and Y can be accepted as events with the assumption that X is appearing while Y has already happened. With the help of this methodology, the inference of single occupancy of an event does not influence the other one can be raised which is also named as naive approach.
The upcoming sections of this article include three distinct methods as Multinomial, Bernoulli, and Gaussian Naive Bayes.
The Multinomial Naive Bayes can be accepted as the probabilistic approach to classifying documents in the case of acknowledging the frequency of a specified word in a text document.
The term “bag of words” [1] is widely used as the selected document to be processed under the context of Naive Bayes while depicting the document itself as a bag and each vocabulary in the texture as the items in the bag by permitting multiple occurrences.
To be able to properly classify, the existence of the word in the given text shall be known beforehand. This classifier achieves well on discrete types as the number of words found in a document. An example of the usage area of this approach can be the prediction of the matching category of the document with the help of the occurrences of the words allocated in the document.
The output of this algorithm produces a vector composed of integer frequency values of the word set.
P(M|N) = ( P(M) * P(N|M) ) / P(N)
The Bernoulli or “Multivariate Bernoulli” [2] Naive Bayes may be expressed as the statistical method that generates outputs on a boolean basis by exploiting the desired text’s existence. This classifier feeds from Bernoulli Distribution [3] which has a discrete nature.
P(M) = 1-p for M = 0,        p   for M = 1P(Mi|N)= P(i|N) * Mi + ( 1 - P(i|N) ) * (1 - Mi)
This kind of Naive Bayes classifier can be useful when an undesired word would like to be detected or a specific type of word would like to be tagged in a given document. Also, it differentiates from the multinomial method by generating binary output as 1–0, True-False, or Yes-No.
The concept of classifying text-formatted data has a wider tendency to work with categorical types. In addition to discrete data, Naive Bayes can be applied to continuous types, too.
The Gaussian Naive Bayes can be useful to be applied on real or continuous-valued datasets rather than categorical or discrete-valued features.
In the formula, the underlined Probability Density Function belongs to Gaussian by generating a calculation of the probability of any additional input word.
Probability Density Function (x, mean, stdDev) = (1 / (sqrt(2 * PI) * stdDev)) * exp(-((x-(mean^2))/(2*(stdDev^2))))
The reason why is that the corresponding classification is actualized by considering a Gaussian distribution [4].
For the usage area, the datasets that have words with corresponding continuous values as height or width can be proper for the Gaussian Naive Bayes algorithm to be performed on.
The Naive Bayes classifiers provide insightful outcomes in the fields of detecting sentiments and spam in text contexts [5]. Varying to the fields, the need for a classifier also differentiates, since the math behind the algorithm changes. In order to be able to accurately produce correct word classes and categories, it is highly essential to define the type of problem.
Thank you so much for reading!
Text Classification using Naive BayesMultivariate Bernoulli DistributionBernoulli DistributionGaussian Distribution FunctionNaive Bayes and Sentiment Classification
Text Classification using Naive Bayes
Multivariate Bernoulli Distribution
Bernoulli Distribution
Gaussian Distribution Function
Naive Bayes and Sentiment Classification