# Benchmarking of Multi-Class Algorithms for Classifying Documents Related to Stunting
## 3 Methodology
### Training Process
#### 3.4.2 Training Process Using Naive Bayes  
The naive Bayes model determines the class of data by summing up the frequency of combinations of given dataset values to calculate a set of probabilities [25]. In classifying a sample of data, the naive Bayes classifier assumes that the presence or absence of a particular feature of a class is independent of the presence or absence of other features [26]. The following is the equation of the naive Bayes classifier:  
\[P\big{(}c\Big{|}X\overset{\rightarrow}{x}\big{)}=\ \frac{P(\overset{ \rightarrow}{x}\big{|}c).\ P(c)}{P\big{(}\overset{\rightarrow}{x}\big{)}} \tag{9}\]  
where  
\(\overset{\rightarrow}{x}\): a document with unknown class, \(\overset{\rightarrow}{x}=(w_{1},w_{2},\ldots,w_{V})\)  
c: document hypothesis, is a specific class, \(c\in C\)  
C: set of classes  
V: size of vocabulary  
\(P\big{(}c\Big{|}\overset{\rightarrow}{x}\big{)}\): posterior probability, probability that a document \(\overset{\rightarrow}{x}\) belongs to the class \(c\)  
\(P(c)\): prior probability that any random document from the corpus belongs to class \(c\)  
\(P\big{(}\overset{\rightarrow}{x}\big{|}c\big{)}\): likelihood probability, probability that a randomly chosen document from documents in the class \(c\) is the document \(\overset{\rightarrow}{x}\)  
\(P\big{(}\overset{\rightarrow}{x}\big{)}\): evidence probability, probability that a randomly chosen document from the corpus is document \(\overset{\rightarrow}{x}\)  
There are two types of Naive Bayes which is implemented in this study, i.e., multinomial naive Bayes and Gaussian naive Bayes. The Gaussian naive Bayes was chosen for the TF-IDF feature because the values from TF-IDF are included in the continuous data category and there is a typical assumption that the continuous values associated with each class are distributed according to a normal (or Gaussian) distribution. Meanwhile, the selection of multinomial naive Bayes for the term occurrence and term presence features is because the vectors of these two features represent the frequency and appearance of words that can be associated as a multinomial distribution. The prior probability can be estimated by dividing number of documents belonging to class \(c\) by the total number of documents. As likelihood probability for multinomial naive Bayes can be calculated as follows:  
\[P\big{(}\overset{\rightarrow}{x}\big{|}c\big{)}=\ \prod_{i=1}^{V}P(w_{i}|c)= \prod_{i=1}^{V}\frac{count(w_{i},c)+1}{count(c)+V} \tag{10}\]  
\[P\big{(}\overset{\rightarrow}{x}\big{)}=\sum_{c\in C}(P\big{(}\overset{ \rightarrow}{x}\big{|}c\big{)}P(c)) \tag{11}\]where \(w_{i}\): the \(i\)-th word in a document  
\(i\): index of word, \(i=1,\ 2,\ \dots,\ V\)  
\(V\): size of vocabulary  
\(P(w_{i}c)\): the probability of the occurrence of the \(i\)-th word if the class is known to be c  
\(count(w_{i},c)\): the number of occurrences of term \(w_{i}\) in training documents from class \(c\), including multiple occurrences of a term in a document.  
\(count(c)\): the number of terms belonging to class \(c\)  
Furthermore, the difference between Gaussian naive Bayes and multinomial naive Bayes lies in the calculation of the likelihood value as follows:  
\[P\left(\overset{\rightarrow}{x}\middle|c\right)=\prod_{i=1}^{V}\mathcal{N} \left(w_{i}\middle|\mu_{c,i},\sigma_{c,i}^{2}\right)=\prod_{i=1}^{V}\frac{1}{ \sqrt{2\pi\sigma_{c,i}^{2}}}\textit{exp}\left(-\frac{(w_{i}-\mu_{c,i})^{2}}{2 \sigma_{c,i}^{2}}\right) \tag{12}\]  
where  
\(\mu_{c,i}\): the mean of the feature values of term \(w_{i}\) associated with class \(c\)  
\(\sigma_{c,i}^{2}\): the variance of the feature values of term \(w_{i}\) associated with class \(c\)