# An Early Warning System to Detect At-Risk Students in Online Higher Education
## 4 Algorithms and Training Dataset Selection for the GAR Model
### Algorithm and Training Set Selection
```
Input:\(S_{TE}\): Semester test dataset Output:\(B_{CL}\): Best classification algorithm and training set per course and activity  Steps: //Train and test all semesters with respect to \(S_{TE}\) \(B_{CL}\leftarrow\emptyset\) For each semester \(S\in[20171,\ldots,S_{TE}]\)do \(B_{CL}\gets B_{CL}\cup\text{Best\_Classification\_Algorithm }(S,S_{TE})\) //Compare on same semester For each semester \(S\in[20171,\ldots,S_{TE}]\)do For each course \(C\in S\)do For each pair \((A_{i},A_{j})\in C\) st. \(i<j\)do \(\{S,M_{Ai}\}\gets B_{CL}(C,A_{i},S)\) \(\{S,M_{Ai}\}\gets B_{CL}(C,A_{j},S)\) If\(L(S,M_{Ai})>L(S,M_{Ai})\)then\(B_{CL}(C,A_{j},S)\leftarrow\{S,M_{Ai}\}\) //Compare inter semester For each course \(C\in S_{TE}\)do For each activity \(A\in C\)do For each pair \((S,S_{TE})\) st. \(S\in\{20171,\ldots,S_{TE-1}\}\) \(\{S,M_{S}\}\) \(\gets B_{CL}(C,A,S)\) \(\{S_{TE},M_{STE}\}\gets B_{CL}(C,A,S_{TE})\) If\(L(S,M_{S})>L(S_{TE},M_{STE})\)then\(B_{CL}(C,A,S_{TE})\leftarrow\{S,M_{S}\}\) Return\(B_{CL}\)
```  
**Algorithm 2** Pseudocode of Best_Classification_and_Training_Set. In order to analyze the performance of these two algorithms, the evaluation test has been performed only in the last semester 20182 for the different metrics and taking as a baseline the NB classifier results. The results are shown in Figure 3 and Table 1 in Appendix 0.A. The best algorithm selection slightly improves the different metrics since the best selection is only performed within the explored algorithms on an activity. In the case of the TPR, the LOESS regression compared to the NB improves from the range 44.60-86.04% to 47.88-88.43%. However, considerable improvement is obtained with the second algorithm, where the LOESS regression increases until 58.79-93.60%. This result will help to define a more significant threshold to consider a high-quality submodel, as we will see in the next section. Although the last finding proves that the second process helps to identify the best classifier and training set for each assessment activity, it is not clear which classifier has been mostly selected and from which origin semester. Table 3 summarizes this information with interesting insights. The table shows the percentage of classifiers and from which semester they have been selected. The process mostly selected the DT with more than half of the trained submodels. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **20171** & **20172** & **20181** & **20182** & **Total** \\ \hline Decision Tree & 10.42\% & 9.27\% & 12.10\% & 19.19\% & 50.98\% \\ K-Nearest Neighbors & 2.47\% & 3.43\% & 4.80\% & 5.19\% & 15.89\% \\ Naive Bayes & 4.66\% & 5.72\% & 5.81\% & 9.73\% & 25.92\% \\ Support Vector & 1.43\% & 1.54\% & 1.66\% & 2.58\% & 7.21\% \\ Total & 18.97\% & 19.97\% & 24.37\% & 36.69\% & 100.00\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Distribution of the classification algorithm selection for 2019 spring semester. The NB proposed in [3] is only selected the 25.92% of the total. Thus, we conclude that the NB is not the most appropriate classifier to train the models when more data are available. Finally, we can observe that the distribution among semesters is quite reasonable. The training dataset of the last available semester is the most selected one (36.69%), but for some submodels, the dataset from previous semesters with less training data helps to get more accurate models. Finally, we give an insight about the computational cost of this operation in absolute numbers for semester 20182. The number of training operations for each semester are 26,468 in 20182, 24,996 in 20181, 23,144 in 20172 and 24,184 in 20171. Note that, this number for a semester takes into account the training operation for the four classifiers for all courses and activities.