## 4 Survey Results
### Predictive Models of Learning Outcomes
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Learning Type** & **Number of Studies** & **Studies** \\  & **(Percentage of Occurrence (\%))** & \\ \hline Statistical analysis & 28 (45.16\%) & [43; 44; 45; 50; 52; 54; 56; 57; 58; 64; 73; 75; 76; 81; 83; 90; 92; 97; 98; 99] \\ \hline Supervised machine learning & 25 (40.32\%) & [46; 51; 55; 59; 62; 65; 68; 70; 74; 78; 80; 84; 89; 95; 96; 101; 102; 103] \\ \hline Data mining & 5 (8.06\%) & [63; 77; 94; 100; 104] \\ \hline Supervised and unsupervised learning & 3 (4.83\%) & [66; 79; 93] \\ \hline Unsupervised machine learning & 1 (1.61\%) & [67] \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Distribution of student outcomes predictive models per learning type. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Learning Model** & **Number of Studies** & **Studies** \\  & **(Percentage of Occurrence (\%))** & \\ \hline Statistical models & 32 (51.61\%) & [43; 44; 45; 56; 57; 58; 64; 67; 69; 71; 72; 73; 76; 81; 83; 88; 92; 97; 98; 99] \\ \hline Neural networks & 9 (14.51\%) & [51; 68; 70; 74; 78; 86; 95; 96; 102] \\ \hline Tree-based models & 9 (14.51\%) & [55; 63; 65; 75; 77; 85; 101; 104] \\ \hline Bayesian-based models & 5 (8.06\%) & [61; 62; 79; 93; 94] \\ \hline Support Vector Machines & 2 (3.22\%) & [59; 60] \\ \hline Instance-based models & 1 (1.62\%) & [103] \\ \hline Other & 4 (6.45\%) & [48; 80; 84; 100] \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Distribution of intelligent predictive algorithms per category. squared error (RSE), and R-squared [66; 85]. However, 21 (33.87%) studies did not specify the metrics they used, so the quality of their predictions was inconclusive. When we counted the frequency of performance metrics utilized in the studies, we found that 28 (45.16%) intelligent models used 'accuracy' to measure the prediction quality, followed by the root mean square error (RMSE) (10 studies, 16.12%), ROC-AUC (8 studies, 12.90%), R square (8 studies, 12.90%), and mean absolute error (MAE) (7 studies, 11.29%). Table 9 summarizes the top and worst-performing prediction models of learning outcomes. Accordingly, the hybrid random forest [101] demonstrated the best classification accuracy, while the linear regression gave the worst predictions [88]. Figure 12 shows that 38 (61.29%) studies did not benchmark the performance of their intelligent models against any baseline competitors. Fifteen (24.19%) studies compared their models with one to three competitor models. The remaining studies (14.51%) made a performance comparison with four or more baseline classifiers. The most compared against techniques included the Decision Tree (9 times), K-Nearest Neighbor (9 times), Support Vector Machines (8 times), Naive Bayes (8 times), and Random Forest (6 times). Only 5 (8.06%) studies reported using multiple datasets to verify the performance of their predictive models to check the consistency and validity of the learning outcomes predictions [65; 80; 86; 96; 102]. The remaining studies (91.93%) used only one dataset. With respect to the software used to analyze the datasets, statistical tools (e.g., SPSS, R, and Mplus) appeared in 14 studies, followed by data mining tools (e.g., WEKA), and machine learning frameworks (e.g., Keras, TensorFlow, and Scikit-learn). Other tools used included numerical computation (e.g., Octave) and in-house developed software (6 studies). It is worth noting that 29 articles did not specify the software tools they used to develop their predictive models. Figure 12: Number of intelligent methods against which each predictive model is benchmarked.