# An Early Warning System to Detect At-Risk Students in Online Higher Education
## 4 Algorithms and Training Dataset Selection for the GAR Model
### Algorithm and Training Set Selection  
As mentioned previously, we observed that the metrics do not always improve over semesters for specific courses, even though more training data are available. Different factors may impact on these results: 1) The behavior of students in some semesters may add noise to the model that produces worse results; or 2) some academic change such as new resources or changes in the difficulty or the design of the assessment activities may impact the grades of a specific semester. In order to deal with these issues, we propose a method to select the best training set and classification algorithm for each course and activity. This selection can be reduced to an optimization problem with the objective function \(L\) to be maximized  
\[L\left(S_{TR},\ M\right)=TPR\Big{(}D_{S_{TR},C,A},D_{S_{TE},C,A},\ M\Big{)}+ TNR\Big{(}D_{S_{TR},C,A},D_{S_{TE},C,A},M\Big{)} \tag{2}\]  
where \(S_{TR}\) is the semester to explore as training set, \(S_{TE}\) is the semester to perform the validation test, \(M\) is the classification algorithm used to perform the training, and \(D_{S,C,A}\) is a slice from the whole dataset \(D\) based on the semester \(S\), course \(C\) and activity \(A\). After different tests, we found that the best choice is to maximize the sum of the TPR and TNR. This function tends to select classifiers and training sets that achieve good results in both metrics, while maximizing one of the metrics tends to penalize the discarded one. The method has been split into two algorithms. The first algorithm selects the best classification algorithm, while the second one selects the best training set for each assessment activity of each course. The selection of the best algorithm (see Algorithm 1) searches for the best algorithm based on four available classification algorithms: NB, DT, KNN, and SVM. The process is quite simple but computationally expensive since four training processes are run for each course and assessment activity. However, there is a substantial benefit since each activity (i.e., each submodel for the GAR model associated with the course) has the best-selected algorithm. ```
Input:\(S_{TR}\): Semester training dataset, \(S_{TE}\): Semester test dataset Output:\(B_{CL}\): Best Classification Algorithm per course and activity Steps:  Initialize(\(B_{CL}\)) For each course \(C\in S_{TE}\)do For each activity \(A\in C\)do \(M_{CL}\leftarrow\) Select best classifier \(M\in\) {NB, KNN, DT, SVM} based on opt. function \(L(S_{TR},M)\) (Equation (2)) \(B_{CL}(C,A,S_{TR})\leftarrow\{S_{TR},M_{CL}\}\) Return\(B_{CL}\)
```  
**Algorithm 1** Pseudocode of Best_Classification_Algorithm. The second process is presented in Algorithm 2. In this case, the process goes further, and it explores for the submodel of the activity the surrounding submodels in terms of previous activities within the same course, and the same submodel within the same course in the previous semesters. We observed that sometimes an activity is not mandatory, or it has a low impact on the final grade of the course, or a semester adds noise to the model, and such a semester produces a lower accurate submodel. For example, it is commonly observed in our institution that the behavior of the students is different depending on the spring or fall semester. Thus, in such cases, we suggest selecting the best-known training set within the same course in previous activities or in the same activity among previous semesters. Note that the computational cost is even higher than the former since the evaluation test should be redone with the training set of the selected semester and the test set of last available semester. We are aware that this process will be unfeasible when the number of semesters increases. Thus, a window of training semesters should be defined in the future.