## 4 Results
### Research Questions
#### 4.1.1 RQ1. Which Are the Most Appropriate Datasets to Early Predict at-Risk Students?
At the same time, it is essential to point out that the best results were obtained from DB2 and DB5, which are variations of DB1 that do not consider the different types of presence (cognitive, social and teaching). 1.3 RQ3. Does the Use of Oversampling Techniques (SMOTE) Help the Models to Achieve Better Performances? To answer this question, we calculated the median of the performances of the models generated with original DBs (without the application of SMOTE). We compared them with the performances of the models generated with oversampled DBs. SMOTE was applied on the training set after splitting the data in training/testing sets. To check if there is any statistical difference between them, we apply the Mann-Whitney test again. Table 7 summarizes the results. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Semester** & **Combination 1** & **Combination 2** & _p-Value_ \\ \hline \multirow{4}{*}{2016-1} & DB12—AdaBoost & DB2—AdaBoost & 0.13032 \\ \cline{2-4}  & DB12—AdaBoost & DB5—AdaBoost & 0.19794 \\ \cline{2-4}  & DB12—AdaBoost & DB9—AdaBoost & 0.48212 \\ \cline{2-4}  & DB12—AdaBoost & DB12—MLP & 0.48224 \\ \hline \multirow{4}{*}{2016-2} & DB2—AdaBoost & DB5—AdaBoost & 0.32775 \\ \cline{2-4}  & DB2—AdaBoost & DB9—kNN & 0.50000 \\ \cline{2-4}  & DB2—AdaBoost & DB12—kNN & 0.50000 \\ \cline{2-4}  & DB2—AdaBoost & DB1—Random Forest & 0.42911 \\ \hline \multirow{4}{*}{2017-1} & DB2—AdaBoost & DB5—AdaBoost & 0.34448 \\ \cline{2-4}  & DB2—AdaBoost & DB6—kNN & 0.14441 \\ \cline{2-4}  & DB2—AdaBoost & DB8—kNN & 0.14441 \\ \cline{2-4}  & DB2—AdaBoost & DB10—Random Forest & 0.34525 \\ \hline \multirow{4}{*}{2017-2} & DB2—AdaBoost & DB5—AdaBoost & 0.41160 \\ \cline{2-4}  & DB2—AdaBoost & DB13—AdaBoost & 0.18712 \\ \cline{2-4}  & DB2—AdaBoost & DB2—Random Forest & 0.32884 \\ \cline{1-1} \cline{2-4}  & DB2—AdaBoost & DB4—Random Forest & 0.26740 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Application of the Mann-Whitney test on the five best results, comparing the best with the other four combinations. Results show that there is an improvement on the median of ROC values in 9 out of the 16 cases (only for DB5 - AdaBoost on 2016-2 semester the results got worse) but these differences are statistically significant in only 7 out of the 16 cases (\(p\)-value is smaller than 0.05). Figures 4-7 help to better visualize the results for the first 8 weeks of the four evaluated semesters (2016-1, 2016-2, 2017-1 and 2017-2 respectively). Figures 4 and 5 show similar results for the semesters 2016-1 and 2016-2, respectively, which are when SMOTE is not applied, the results are better. It happens for the two cases, in both figures. It can be seen in Figures 4 and 5 that the AdaBoost achieved the best prediction result with DB5. In 2017-1 (Figure 6), it can be seen that the use of SMOTE improved the prediction results since the AdaBoost--DB5 combination with SMOTE presented the best results for the eight week. However, in Week 8, we can see that the Random Forest--DB5 combination presented similar results. In Figure 7 we can see that the application of SMOTE brought the biggest difference if compared to the data without SMOTE. The Random Forest classifier (with DB2 and DB5) presented the best results and the application of SMOTE improved the results. However, in Week 8, results of Random Forest (without the SMOTE application) and AdaBoost--DB2/DB5 were pretty similar. From the results, one can say that the use of SMOTE helps on improving the performances of the models in only 43.75% of the cases considering all four semesters. Moreover, the use of SMOTE showed the best improvement on 2017-2, where the ROC value for Random Forest with DB2 and DB5 stayed above all the other combinations on all the weeks.