# Implementing AutoML in Educational Data Mining for Prediction Tasks
## 5 Results
### Predicting Pass/Fail Students  
At first, we conducted a series of experiments to identify the effectiveness of classic data mining algorithms to predict students that are likely to fail at early enough stages. We performed 6 classic machine-learning algorithms on the datasets of 3 courses, split into chronological sets (month A, month B, etc.). An exception is the \"Physical Chemistry I\" course, which did not have any logs on the first month, and as such, we did not conduct experiments during that period. Tables 8-10 present the effectiveness results, represented by the accuracy measure. We observe that Bagging and the Random Forest ensemble algorithms outperform the others in most cases in the first course, Naive Bayes, PART and IBk algorithms outperform the others in most cases in the second course, and SMO and Random Forest algorithms outperform the others in most cases in the third course. In total, SMO and Random Forest are noted to be among the best performers 9 times and Bagging 7 times. As such, we could not conclude one best performer for our given datasets. In addition, we used Auto WEKA to run automated machine learning experiments for the corresponding datasets. From the results, we identify that in most cases, the accuracy was significantly increased. Auto-WEKA was able to optimize the results up to 10% (Figure 4). The suggested classifiers and hyperparameters again vary across the datasets, including LMT, J48, PART, and JRip, to name \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **February 2018** & **March 2018** & **April 2018** & **May 2018** & **June 2018** & **July 2018** \\ \hline Naïve Bayes & 61.24 & 63.57 & 66.67 & 83.72 & 84.50 & 84.50 \\ \hline Random Forest & 78.29 & 85.27 * & 86.05 * & 90.70 * & 90.70 & 89.92 \\ \hline Bagging & 79.84 & 85.27 * & 84.50 & 88.37 & 88.37 & 88.37 \\ \hline PART & 72.09 & 76.74 & 75.97 & 86.04 & 86.82 & 86.82 \\ \hline SMO & 81.40 & 86.05 & 86.05 * & 89.92 & 91.47 * & 90.70 * \\ \hline IBk-5 NN & 75.19 & 85.27 * & 84.50 & 89.92 & 89.92 & 89.92 \\ \hline Auto-WEKA & 81.40 & 86.82 & 86.05 & 93.02 & 93.02 & 93.02 \\ LMT & JRip & LMT & LMT & LMT & LMT \\ \hline \hline \end{tabular} \(t\)-test: \(p\)-value = 0.0223, a = 0.05.  
\end{table} 
Table 10: Overall accuracy results regarding the “Analytical Chemistry Laboratory” course and Pass/Fail classification task. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **February 2018** & **March 2018** & **April 2018** & **May 2018** & **June 2018** & **July 2018** \\ \hline Naïve Bayes & - & 59.22 & 68.09 & 75.18 & 75.18 \\ \hline Random Forest & - & 62.06 & 78.01 * & 82.62 * & 81.56 * & 81.56 * \\ \hline Bagging & - & 62.4 & 74.47 & 81.21 & 81.21 & 81.20 \\ \hline PART & & 63.12 * & 73.76 & 72.70 & 75.89 & 74.11 \\ \hline SMO & - & 58.16 & 74.82 & 79.43 & 80.85 & 80.14 \\ \hline IBk-5 NN & - & 59.93 & 70.92 & 78.01 & 79.43 & 79.08 \\ \hline Auto-WEKA & - & 66.67 & 81.20 & 83.68 & 82.27 & 81.56 \\ Random Tree & LMT & LMT & REPTree & PART \\ \hline \hline \end{tabular} \(t\)-test: \(p\)-value = 0.0365, a = 0.05.  
\end{table} 
Table 8: Overall accuracy results regarding the “Physical Chemistry I” course and Pass/Fail classification task.