# Detection of Students' Problems in Distance Education Using Topic Modeling and Machine Learning
## 4 Results
### Classifier Result  
From the category developing phase, we had a total of 5000 tweets annotated with five categories and 11 sub-categories. To train the classifiers, we used 70% for training (3500 tweets) and 30% for testing (1500 tweets). We implemented a one-vs-all classifier using OneVsRestClassifier with LinerSVC where the kernel parameter is linear. We selected LinerSVC because it uses with multi-class one-vs-all mode. Hyper parameter C is tuned using an exhaustive grad search that considers all the parameters combinations. The linear kernel and C values (1, 10, 100, 1000) are estimated, and the best value for C was 10. Tables 6 and 7 demonstrate the result of the evaluation measures for the first and second-layer classifiers. The result of classifying the tweets into categories are summarized  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Predicated Classes** & **Classes** & **EI** & **SI** & **TI** & **HI** & **AI** & **Others** \\ \hline \multirow{4}{*}{**Actual Classes**} & EI & TN & FP & TN & TN & TN \\  & SI & FN & TP & FN & FN & FN \\  & TI & TN & FP & TN & TN & TN \\  & HI & TN & FP & TN & TN & TN \\  & AI & TN & FP & TN & TN & TN \\  & Others & TN & FP & TN & TN & TN \\ \hline \hline \end{tabular}
\end{table}
Table 5: Confusion matrix for multi-class classification for social issues (SI) class vs. other classes: Education issues (EI), Social issues (SI), Technology issues (TI), Health issues (HI), and Attitudes and Ethical issues (HI).  
in Table 6, whereas Table 7 shows the result of classifying the tweets into more specific issues. From Table 6, we note that the best result is achieved by educational issues and then social issues with F1 of 0.715 and 0.664, respectively. On other hand, the worst result is achieved by attitudes and ethical issues with an F1 of 0.616. We notice that the variation of the performance is because of the differences in the training instances used in each of the above categories, which are 456 and 233 for educational and social issues and 62 for attitudes and ethical issues. From Table 7, we find that best results are achieved by the topics related to social issues, which is closely followed by topics related to educational and technology issues. The worst results are achieved by topics related to attitudes and ethical issues. The detection of sub-category topics is satisfactory, with an F1 of 72%.  
The results for the multi-layer multi-class classification model are summarized in Table 8. It is notable that the micro-averaged values register higher scores compared to macro-averaged due to the unbalanced class distributions. Thus, the more frequent the occurrence of the class, the more can be detected and classified accurately. Clearly, using the resulting category as a feature leads to improvement in the classification decision of the sub-category classifiers. The performance evaluation of the two-layer classifiers is shown in Figures 6 and 7; it can be seen that the precision, recall and F-measure of the sub-category classifier are around 72%, meanwhile, the category classifier is below 70%. If the classifier detects that a tweet belongs to \"others\", then none of the lower-level classifiers would be applied.