# Gender Classification Using Sentiment Analysis and Deep Learning in a Health Web Forum
## 4 Results  
### Traditional Machine Learning Results  
Initially, 70% of data, including gender information, were used as training data and 30% as test data. We experimented with naive Bayes (NB), support vector machine (SVM), and random forest (RF), which are representative algorithms proven to outperform other algorithms. The precision measure and the recall measure are calculated for each class. For the female class, the precision is calculated as TP/(TP + FP) and recall is calculated as TP/(TP + FN). For the male class, the precision and recall are calculated as FN/(FN + TN) and TN/(FP + TN), respectively. The positive class is defined as the class of interest. In this work, the female class and male class are treated equally, but the positive class is set as the female class. The accuracy, the true positive rate, and the true negative rate are calculated considering two classes. Accuracy indicates how well a binary classification test correctly identifies each class. False positive rate is calculated as FP/(FP + TN) and indicates the ratio of the wrongly classified females among the males. False negative rate is calculated as FN/(FN + TP). Using only sentiment features, overall, the female class has poor performance. Three algorithms did not recognize the female classes well and classified most of the users into the male class. This indicates that sentiment features such as emotion types and their density are not effective for classifying authors' gender on their own. To test the effects of sentiment features, we performed three experiments as follows. At first, we only built a sentiment feature set including the ratio of 10 types of emotional occurrences, objectivity, and emotional complexity. The accuracy of them are 58.33% (NB), 60.86% (SVM), and 58.66% (RF), respectively, as shown in Table 4. Compared to the male class, the female class has a poor performance. Over 50% of female users were classified into the male class. The second experiment was performed using senti-word features. Instead of calculating sentiment value from senti-word, we checked the senti-word occurrence. Among the 7954 words that appeared more than five times each, we selected only 1594 senti-words as features that are in the NRC dictionary. The accuracy increased a little bit with the SVM and RF algorithms, as shown in Table 5. The precision for the female class improved. The senti-words used to express the author's emotion were comparatively effective for gender classification. Among the three algorithms, SVM had the highest performance for the sentiment feature set and the senti-word feature set. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Real** & **Algorithm** & **NB** & \multicolumn{3}{c}{**SVM**} & \multicolumn{3}{c}{**RF**} \\ \cline{2-7}  & **Female** & **Male** & **Female** & **Male** & **Female** & **Male** \\ \hline Female & 174 (True positive: TP) & 200 (False negative: FN) & 81 & 42 & 402 & 413 \\ Male & 1679 (False positive: FP) & 2456 (True negative: TN) & 1723 & 2663 & 1451 & 2243 \\ \hline Precision & 9.39\% & & 92.47\% & 4.49\% & 98.45\% & 21.69\% & 84.45\% \\ Recall & 46.52\% & & 59.40\% & 65.85\% & 60.72\% & 49.33\% & 60.72\% \\ Accuracy & & 58.33\% &  
The final experiment was performed using the entire feature set, combining word features and sentiment features. When we assessed all 7954 words that occurred more than five times along with the sentiment features, the accuracy improved. However, the performance of SVM rather decreased. In this case, RF has the highest performance and the precision for the female class somewhat improved. However, the female class still has a poor performance as shown in Table 6.  
Based on the three experiments varying the feature set, we obtained two findings, one is the algorithm perspective and the other is the feature perspective. We can conclude that SVM underperforms other algorithms in the case of a few features. On the other hand, RF works well with more features. In addition, we can conclude that NB is not appropriate for text classification task.