# Implementing AutoML in Educational Data Mining for Prediction Tasks
## 6 Discussion
Lastly, dropout classification tasks yielded the widest variety of features among its important compared with the other tasks. As previous studies have shown [17; 84; 85], there is no algorithm that is the best across all classification problems. A similar conclusion was reached by our experiments, as we could not result in one best performer in our educational data mining tasks illustrated in Section 5. We applied 6 well known supervised machine learning algorithms--Naive Bayes, Random Forest, Bagging, PART/MSRules, SMO/SMOreg and IBk--one representative of the six main learners' categories. Each time we compared the performance of the classic models with the calculated by Auto-WEKA model. Auto-WEKA was set to allow only ten tree or rule-based classifiers. The automated one was in most cases better. Significant differences calculated by \(t\)-test were marked. We did not conclude an overall winner classifier suggested by the Auto-WEKA tool either. However, when we grouped the proposed classifiers under the 6 learner's categories, it was clear that Auto-WEKA suggestions were mostly tree-based classifiers (Table 16). There are several advantages to using decision trees in classification and prediction applications. Decision trees models are easy to interpret and explain [14]. Compared to other algorithms, they require \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Pass/Fail**} & \multicolumn{2}{c}{**Regression**} & \multicolumn{2}{c}{**Dropout**} \\ \cline{2-7}  & \begin{tabular}{c} **Best of Classic** \\ **Methods** \\ \end{tabular} & \begin{tabular}{c} **Auto-WEKA** \\ \end{tabular} & \begin{tabular}{c} **Best of Classic** \\ **Methods** \\ \end{tabular} & \begin{tabular}{c} **Auto-WEKA** \\ \end{tabular} & \begin{tabular}{c} **Best of Classic** \\ **Methods** \\ \end{tabular} & \
\begin{tabular}{c} **Auto-WEKA** \\ \end{tabular} \\ \hline \hline bayes & 5\% & - & Not applicable &
\begin{tabular}{c} Not \\ applicable \\ \end{tabular} & 12\% & - \\ \hline functions & 27\% & - & 19\% & - & 4\% & - \\ \hline lazy & 9\% & - & 7\% & - & 8\% & - \\ \hline meta & 21\% & - & 37\% & - & 32\% & - \\ \hline rules & 9\% & 29\% & 4\% & 18\% & 12\% & 0\% \\ \hline tree & 27\% & 71\% & 33\% & 82\% & 32\% & 100\% \\ \hline  & functions/trees & trees & meta & trees & meta/trees & trees \\ \hline \hline \end{tabular}
\end{table} 
Table 16: Classic outperformers and Auto-WEKA suggestions grouped under 6 learners’ categories. Twelve out of 17 proposed Auto-WEKA classifiers (71%) were tree classifiers. Similarly, 14 out of 17 regressors (82%) and 11 out of 11 (100%) suggested dropout models also belong to the tree classifiers category. less effort for data preparation during pre-processing. A decision tree does not require normalization of data nor scaling of data. Finally, automated hyperparameter optimization familiarized us with a wide range of models and configurations that were practically applied to our settings [10]. It allowed us to test models with many variables that would be complicated to be tuned by hand. However, some limitations should also be noted. First, it was not clear what was the appropriate time limit that imposed on our (relatively small) datasets. We marked differences between the suggested models for the same datasets when Auto-WEKA was executed e.g., for 5 min and when it was executed for 1 or 2 h. In limited cases, as the time limit increased, the performance was not necessarily better. This behavior is probably due to the fact that the parameter space is too large to explore, and different randomizations (e.g., during the train/test splitting, or in the underline SMAC optimizer that is used) allow to explore a smaller or a larger part of it. Another possible explanation is the fact that datasets of such size are prone to overfitting and underfitting. In any case, the suggested model was generally beneficial over using default values. Secondly, autoML methods, by definition, take much longer to train. Such an effect could be afforded due to the reliability of the results.