# Identifying At-Risk Students for Early Intervention--A Probabilistic Machine Learning Approach
## 4 Results
The evaluation of the models' performances in the five stages revealed slight discrepancies between the LR and PLR models. Specifically, the metrics of accuracy, F1-score, precision, and recall exhibited minor variations in these stages. The LR and PLR models demonstrated higher or lower performance in the various stages. These differences are illustrated in the form of bold values--the better-performing model being denoted by bold text. Notably, the LR model was found to outperform the PLR model in terms of F1-score in stages 2 and 5, and the PLR model demonstrated superior precision in stages 3 and 5. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Variables** & **Stage 1** & **Stage 2** & **Stage 3** & **Stage 4** & **Stage 5** \\ \hline TITLE & 0.0359 & 0.0349 & 0.0517 & 0.0500 & 0.0380 \\ QUAL CODE & 0.1153 & 0.1040 & 0.0347 & 0.0746 & 0.0598 \\ CLASS GROUP & 0.0980 & 0.0929 & 0.0402 & 0.0540 & 0.0564 \\ \# OF COURSES & 0.1768 & 0.1690 & 0.0412 & 0.0560 & 0.0723 \\ TIME ON SITE & 0.1945 & 0.2077 & 0.1577 & 0.1207 & 0.0896 \\ TIME ON COURSES & 0.1985 & 0.2061 & 0.1352 & 0.0991 & 0.0878 \\ TIME ON ACTIVITIES & 0.1809 & 0.1852 & 0.1716 & 0.0787 & 0.0975 \\ AS 1 (S1 \& \$2) * & & 0.3676 & 0.2107 & 0.1958 \\ TM 1 (S3) * & & & & 0.2562 & 0.1093 \\ TM 2 (S4) * & & & & & 0.1958 \\ CA 1 (S5) * & & & & & \\ \hline \hline \end{tabular} The variables with asterisks (*) are assessments predicted at each stage.  
\end{table} 
Table 3: Extra trees classifier: feature importance between variables with and without an asterisk. Moreover, a significant increase in model performance was observed from stages 3 to 5. The highest accuracy was achieved at stage 3, in the sixth week of the 2020 second-semester academic calendar. This finding responds to the study's third objective, which sought to determine the optimal week in which to predict at-risk students. The observed increases and decreases in the model's performance mean that more or less predictive power was obtained over time, thereby suggesting an optimal week for predicting at-risk students. A 94% HDI for the model's parameters was observed throughout the five stages, as shown in Table 5. The \((x_{1},x_{2})\) represents the 94% HDI values, where \(x_{1}\) is the lower HDI value and \(x_{2}\) is the upper HDI value, and \(\overline{x}\) is the mean value of \(x_{1}\) and \(x_{2}\). \(\left|x_{d}\right|\) is the absolute difference between \(x_{2}\) and \(x_{1}\). The model is more certain about a parameter if the \(\left|x_{d}\right|\) is close to 0. Throughout the five PML models, the uncertainty of TITLE and TIME ON COURSES increased after including student performance variables. As the semester progressed, the PML models demonstrated more certainty for assessment 1 (AS_1). Furthermore, the PML models demonstrated 43%, 80%, and 56% decreases in model uncertainty for TITLE, TIME ON COURSES, and AS_1, respectively, from stage 3 to stage 5. This is an implication of great model reliability from stage 3 to 5, as model predictions can be performed with greater certainty. To showcase the capabilities of a probabilistic machine learning (PML) model, a student's at-risk status was predicted for all five stages using a probabilistic logistic regression (PLR) model. The PLR model operates by making predictions through sampling from the posterior distribution. In details, 1000 samples used to make predictions with a 95% credible interval. The 95% credible interval indicates the range of values for which the PML model predicts the student's risk status with 95% certainty. The results of these predictions are presented in Table 6. The student was predicted as being at-risk throughout stages 1 to 4, and higher levels of certainty were reported for stages 3 and 4, where the differences between the upper and lower limits were minimal. The student was identified as not being at-risk at stage 5. The 61% probability exceeded the 50% threshold; however, this prediction still indicates that there is a possibility that the student could be at-risk.