# Multi-Class Phased Prediction of Academic Performance and Dropout in Higher Education
## 5 Results and Discussion

Table 5: Global evaluation metrics for each learning model built with the information at the end of second semester (dataset S2). Values represent mean and standard deviation. \"Dropout\" classes are obtained by BRF. Therefore, if the goal is to obtain a most accurate prediction for such classes, Balanced Random Forest models is preferred. The lowest F1 scores in the entire experiment are obtained for the \"Dropout\" class at the end of the second semester. This is related to the small number of records labeled \"Dropout\" in the dataset S2 (only 398 in the whole dataset, of which 358 are used in the training set and 40 in the test set). The confusion matrices obtained for each dataset, and for each model, are presented in Figures 2-4. Given the small number of students labeled as \"Dropout\" remaining in dataset S2, as well as the corresponding poor scores, the question is raised whether it would not be better to model the problem for this dataset as a binary one. With this question in mind, an additional experiment was carried out with dataset S2, where registers labeled as \"Enrolled\" and \"Dropout\" were relabeled as \"Not Graduate\". The data processing, model training, and evaluation for the two-class classification task proceeded as previously described. The results are presented in Table 9. \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{3}{c}{**Dataset S0–F1 Score**} \\ \hline
**ML Model** & **Global** & **Graduate** & **Enrolled** & **Dropout** \\ \hline
**SMOTE + RF** & 0.649 \(\pm\) 0.007 & 0.762 \(\pm\) 0.008 & 0.493 \(\pm\) 0.011 & 0.577 \(\pm\) 0.009 \\
**SVMSMOTE + RF** & 0.649 \(\pm\) 0.016 & **0.762 \(\pm\) 0.022** & 0.454 \(\pm\) 0.029 & 0.608 \(\pm\) 0.026 \\
**BRF** & **0.650 \(\pm\) 0.025** & 0.722 \(\pm\) 0.028 & **0.511 \(\pm\) 0.040** & **0.638 \(\pm\) 0.075** \\
**EE** & 0.627 \(\pm\) 0.026 & 0.706 \(\pm\) 0.022 & 0.484 \(\pm\) 0.037 & 0.606 \(\pm\) 0.049 \\
**RB** & 0.606 \(\pm\) 0.020 & 0.690 \(\pm\) 0.024 & 0.433 \(\pm\) 0.029 & 0.599 \(\pm\) 0.029 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: F1 score evaluation for each class for each learning model built with information at the time of enrollment. Values represent mean and standard deviation. \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{3}{c}{**Dataset S1–F1 Score**} \\ \hline
**ML Model** & **Global** & **Graduate** & **Enrolled** & **Dropout** \\ \hline
**SMOTE + RF** & 0.709 \(\pm\) 0.005 & 0.804 \(\pm\) 0.005 & 0.560 \(\pm\) 0.010 & 0.660 \(\pm\) 0.008 \\
**SVMSMOTE + RF** & **0.745 \(\pm\) 0.014** & **0.847 \(\pm\) 0.010** & 0.555 \(\pm\) 0.028 & 0.718 \(\pm\) 0.030 \\
**BRF** & 0.744 \(\pm\) 0.015 & 0.828 \(\pm\) 0.016 & **0.582 \(\pm\) 0.025** & **0.728 \(\pm\) 0.026** \\
**EE** & 0.729 \(\pm\) 0.014 & 0.814 \(\pm\) 0.013 & 0.572 \(\pm\) 0.025 & 0.705 \(\pm\) 0.028 \\
**RB** & 0.702 \(\pm\) 0.017 & 0.791 \(\pm\) 0.018 & 0.524 \(\pm\) 0.029 & 0.693 \(\pm\) 0.030 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: F1 score evaluation for each class for each learning model built with information at the end of first semester. Values represent mean and standard deviation. \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{3}{c}{**Dataset S2–F1 Score**} \\ \hline
**ML Model** & **Global** & **Graduate** & **Enrolled** & **Dropout** \\ \hline
**SMOTE + RF** & **0.741 \(\pm\) 0.006** & 0.852 \(\pm\) 0.005 & **0.616 \(\pm\) 0.009** & 0.448 \(\pm\) 0.022 \\
**SVMSMOTE + RF** & **0.741 \(\pm\) 0.005** & **0.852 \(\pm\) 0.005** & 0.615 \(\pm\) 0.010 & 0.457 \(\pm\) 0.017 \\
**BRF** & 0.719 \(\pm\) 0.018 & 0.843 \(\pm\) 0.016 & 0.541\(\pm\) 0.034 & **0.493 \(\pm\) 0.055** \\
**EE** & 0.725 \(\pm\) 0.015 & 0.841 \(\pm\) 0.020 & 0.581\(\pm\) 0.031 & 0.455 \(\pm\) 0.058 \\
**RB** & 0.657 \(\pm\) 0.025 & 0.778 \(\pm\) 0.028 & 0.499 \(\pm\) 0.032 & 0.398 \(\pm\) 0.109 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: F1 score evaluation for each class for each learning model built with information at the end of second semester.