# Meta-Learner for Amharic Sentiment Classification
## 4 Results and Discussions
### Discussions
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Metric**} & \multicolumn{2}{c}{**Exp I: CoS**} & **Exp II: CS** & **Exp III: WoS** & **Exp IV: WS** \\ \cline{3-6}  & & **Mean \(\pm\) SD** & **Mean \(\pm\) SD** & **Mean \(\pm\) SD** & **Mean \(\pm\) SD** \\ \hline SVM & Accuracy & 0.78 \(\pm\) 0.02 & **0.85**\(\pm\) 0.07 & 0.72 \(\pm\) 0.01 & 0.74 \(\pm\) 0.02 \\  & Recall & 0.70 \(\pm\) 0.02 & 0.85 \(\pm\) 0.07 & 0.64 \(\pm\) 0.01 & 0.74 \(\pm\) 0.02 \\  & Precision & 0.79 \(\pm\) 0.03 & **0.87**\(\pm\) 0.06 & 0.70 \(\pm\) 0.02 & 0.75 \(\pm\) 0.03 \\  & F1 & 0.71 \(\pm\) 0.02 & 0.85 \(\pm\) 0.07 & 0.64 \(\pm\) 0.02 & 0.73 \(\pm\) 0.02 \\  & AUC & 0.84 \(\pm\) 0.02 & **0.94**\(\pm\) 0.04 & 0.72 \(\pm\) 0.03 & 0.78 \(\pm\) 0.04 \\  & LogLoss - & - & 5.02 \(\pm\) 2.38 & 9.63 \(\pm\) 0.46 & 9.11 \(\pm\) 0.85 \\ \hline RF & Accuracy & 0.76 \(\pm\) 0.02 & 0.83 \(\pm\) 0.07 & 0.72 \(\pm\) 0.02 & 0.73 \(\pm\) 0.03 \\  & Recall & 0.69 \(\pm\) 0.02 & 0.83 \(\pm\) 0.07 & 0.65 \(\pm\) 0.02 & 0.73 \(\pm\) 0.03 \\  & Precision & 0.76 \(\pm\) 0.03 & 0.84 \(\pm\) 0.06 & 0.69 \(\pm\) 0.02 & 0.75 \(\pm\) 0.04 \\  & F1 & 0.70 \(\pm\) 0.03 & 0.83 \(\pm\) 0.07 & 0.65 \(\pm\) 0.02 & 0.73 \(\pm\) 0.03 \\  & AUC & 0.81 \(\pm\) 0.02 & 0.92 \(\pm\) 0.06 & 0.74 \(\pm\) 0.02 & 0.81 \(\pm\) 0.04 \\  & LogLoss - & - & 5.77 \(\pm\) 2.33 & 9.81 \(\pm\) 0.53 & 9.18 \(\pm\) 1.16 \\ \hline NB & Accuracy & 0.76 \(\pm\) 0.02 & 0.79 \(\pm\) 0.03 & 0.72 \(\pm\) 0.02 & 0.68 \(\pm\) 0.02 \\  & Recall & 0.69 \(\pm\) 0.02 & 0.79 \(\pm\) 0.03 & 0.64 \(\pm\) 0.02 & 0.68 \(\pm\) 0.02 \\  & Precision & 0.76 \(\pm\) 0.02 & 0.79 \(\pm\) 0.03 & 0.72 \(\pm\) 0.03 & 0.69 \(\pm\) 0.02 \\  & F1 & 0.70 \(\pm\) 0.02 & 0.79 \(\pm\) 0.03 & 0.64 \(\pm\) 0.02 & 0.68 \(\pm\) 0.02 \\  & AUC & 0.82 \(\pm\) 0.02 & 0.87 \(\pm\) 0.03 & 0.75 \(\pm\) 0.02 & 0.77 \(\pm\) 0.02 \\  & LogLoss - & - & 7.35 \(\pm\) 1.18 & 9.51 \(\pm\) 0.52 & 11.06 \(\pm\) 0.82 \\ \hline Stack & Accuracy & 0.79 \(\pm\) 0.02 & **0.86**\(\pm\) 0.06 & 0.73 \(\pm\) 0.01 & 0.74 \(\pm\) 0.03 \\  & Recall & 0.74 \(\pm\) 0.02 & 0.86 \(\pm\) 0.06 & 0.64 \(\pm\) 0.01 & 0.74 \(\pm\) 0.03 \\  & Precision & 0.77 \(\pm\) 0.02 & **0.87**\(\pm\) 0.05 & 0.71 \(\pm\) 0.03 & 0.75 \(\pm\) 0.04 \\  & F1 & 0.75 \(\pm\) 0.02 & 0.86 \(\pm\) 0.06 & 0.64 \(\pm\) 0.02 & 0.73 \(\pm\) 0.03 \\  & AUC & 0.84 \(\pm\) 0.02 & **0.94**\(\pm\) 0.04 & 0.76 \(\pm\) 0.02 & 0.82 \(\pm\) 0.03 \\  & LogLoss & - & **4.85**\(\pm\) 1.95 & 9.49 \(\pm\) 0.45 & 9.13 \(\pm\) 1.11 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of Performance of Ensemble Classifier over base learners using TF-IDF character (1,7) grams with (no) SMOTE and TF-IDF word uni-gram with (no) SMOTE and using CV of 5 folds relying on annotated user comments data (i.e., all four data sets). CoS = TF-IDF character (1,7) + NoSMOTE, CS=TF-IDF character (1,7) + SMOTE, WoS = TF-IDF word uni-gram + NoSMOTE, and WS = TF-IDF word uni-gram + SMOTE, Exp = Experiment. The numeric values which are formatted bold showing high performance values of the respective classifiers. in green and blue). As a result, the results revealed that character \(n\)-grams feature is most salient feature which helps the classifiers to discriminate the sentiment class categories of Amharic texts. (iii) Comparison of Classifiers: As depicted in Table 4, in each experiment, the stack classifier has better performance over the base learners. For example, experiment IV with TF-IDF uni-gram feature with application of SMOTE, stack learner has achieved better accuracy of 74%, recall of 74%, precision of 75%, F1 of 73%, and AUC of 82% over the base learners' performance with the same settings. However, SVM has a comparable performance of accuracy of 74%, recall of 74%, precision of 75%, F1 of 73%, and AUC of 78%. Besides, SVM has less log loss of 9.11 as compared to stack classifier's log loss 9.13 with SD of \(\pm\)1.11 which is a bit deviated as compared to the deviation of SVM (i.e., \(\pm\)0.85) in the same experimental settings.