## 4 Results
### Research Questions
#### 4.1.1 RQ1. Which Are the Most Appropriate Datasets to Early Predict at-Risk Students?
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Semester** & **Combination** & \(p\)**-Value** & **Median (Normal)** & **Median (Oversample)** \\ \hline \multirow{4}{*}{2016-1} & DB2—AdaBoost & 0.03123 & 0.55128 & 0.59615 \\ \cline{2-5}  & DB5—AdaBoost & 0.07874 & 0.55128 & 0.63461 \\ \cline{2-5}  & DB2—Random Forest & 0.00603 & 0.42308 & 0.50641 \\ \cline{2-5}  & DB5—Random Forest & 0.01344 & 0.42949 & 0.47436 \\ \hline \multirow{4}{*}{2016-2} & DB2—AdaBoost & 0.00280 & 0.71795 & 0.51282 \\ \cline{2-5}  & DB5—AdaBoost & 0.00038 & 0.71795 & 0.42949 \\ \cline{2-5}  & DB2—Random Forest & 0.02070 & 0.62820 & 0.67949 \\ \cline{2-5}  & DB5—Random Forest & 0.36134 & 0.67308 & 0.63461 \\ \hline \multirow{4}{*}{2017-1} & DB2—AdaBoost & 0.06606 & 0.66013 & 0.54575 \\ \cline{2-5}  & DB5—AdaBoost & 0.01047 & 0.63072 & 0.51961 \\ \cline{2-5}  & DB2—Random Forest & 0.16419 & 0.54902 & 0.57516 \\ \cline{2-5}  & DB5—Random Forest & 0.00067 & 0.52287 & 0.63399 \\ \hline \multirow{4}{*}{2017-2} & DB2—AdaBoost & 0.35254 & 0.83974 & 0.75641 \\ \cline{2-5}  & DB5—AdaBoost & 0.35279 & 0.83654 & 0.75320 \\ \cline{1-1} \cline{2-5}  & DB2—Random Forest & 0.00016 & 0.67949 & 0.91987 \\ \cline{1-1} \cline{2-5}  & DB5—Random Forest & 0.00011 & 0.67628 & 0.91987 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison between the ROC values of the normal and oversampled data. Figure 4: ROC Curve: 2016-1. Figure 5: ROC Curve: 2016-2. Figure 6: ROC Curve: 2017-1. 1.4 RQ4. Does the Use of Data from Questionnaires Applied at the Beginning of the Course Help to Improve Models Performance? To answer this question, we used the same methodology of the previous question, initially not including the use of SMOTE and then applying the SMOTE. Table 8 presents the results. On the one hand, in Table 8 it is possible to see the improvement of the performances (median of ROC values) in only 3 out of 16 cases, where only 2 cases are statistically significant. Both in 2016-1, with DB2 and DB5 with AdaBoost. On the other hand, there are some cases where the inclusion of data from the questionnaire decreases the performances of the models, in which two of them have a statistically significant level. According to these results, we can say that using questionnaire data, without performing feature selection, does not help with the prediction of failing of the students. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Semester** & **Combination** & \(p\)**-Value** & **Median (Normal)** &
\begin{tabular}{c} **Median** \\ **(Questionnaire)** \\ \end{tabular} \\ \hline \multirow{4}{*}{2016-1} & DB2—AdaBoost & 0.02081 & 0.55128 & 0.67308 \\ \cline{2-5}  & DB5—AdaBoost & 0.03846 & 0.55128 & 0.62820 \\ \cline{2-5}  & DB2—Random Forest & 0.48202 & 0.42308 & 0.42308 \\ \cline{2-5}  & DB5—Random Forest & 0.19819 & 0.42949 & 0.38461 \\ \hline \multirow{4}{*}{2016-2} & DB2—AdaBoost & 0.44679 & 0.71795 & 0.67949 \\ \cline{2-5}  & DB5—AdaBoost & 0.05542 & 0.71795 & 0.64103 \\ \cline{2-5}  & DB2—Random Forest & 0.26698 & 0.62820 & 0.62820 \\ \cline{2-5}  & DB5—Random Forest & 0.16124 & 0.67308 & 0.66667 \\ \hline \multirow{4}{*}{2017-1} & DB2—AdaBoost & 0.00302 & 0.66013 & 0.49020 \\ \cline{2-5}  & DB5—AdaBoost & 0.01466 & 0.63072 & 0.49346 \\ \cline{2-5}  & DB2—Random Forest & 0.08310 & 0.54902 & 0.44118 \\ \cline{2-5}  & DB5—Random Forest & 0.41219 & 0.52287 & 0.49673 \\ \hline \multirow{4}{*}{2017-2} & DB2—AdaBoost & 0.39453 & 0.83974 & 0.79808 \\ \cline{2-5}  & DB5—AdaBoost & 0.29755 & 0.83654 & 0.75641 \\ \cline{2-5}  & DB2—Random Forest & 0.48230 & 0.67949 & 0.71154 \\ \cline{2-5}  & DB5—Random Forest & 0.50000 & 0.67628 & 0.67628 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Comparison between the ROC values for the normal data and including questionnaire answers. Figure 7: ROC Curve: 2017-2. We also analyzed the case where over-sampled data was compared with the DBs plus questionnaire data (also over-sampled). Table 9 shows the results. In Table 9, it is important to point out that the results got better in 8 out of 16 cases.