# A Study on Dropout Prediction for University Students Using Machine Learning
## 4 Experimental Results
### Model Performance  
To check the performance of prediction models implemented with LR, DT, RF, SVM, DNN, and LightGBM, which were discussed in Section 3.3, experiments were conducted on the summary table with the 20,050 student records shown in Table 6. The experiments were performed on an HP server equipped with an Intel Xeon E5-2609 1.70 GHz CPU, MSI GeForce RTX-4090 GPU, and 32 GB of memory. TensorFlow 2.10.1 was used for DNN, and CUDA 11.8 was used for GPU acceleration.  
Table 8 shows the prediction performance of the six models, where each value in the table represents the average value of 5 runs obtained through the experiments. The average accuracy of the six models was high at 0.945, while their average F1-score was relatively low at 0.796. Among the models, the LightGBM model showed the highest prediction performance with an F1-score of 0.840.  
Figure 2 depicts the differences between the precision, recall, and F1-scores of the six models. We can see that the graph of the F1-score is more affected by recall than precision; the difference between the F1-score and recall values is smaller than the difference between the F1-score and precision values. The standard deviation of recalls is also larger than that of precisions, which is shown in Table 8.  
Figure 3 compares the average execution time of the six models. The LR, DT, RF, and LightGBM models showed relatively good performance within 3 s, while the SVM and DNN models have longer execution times. Especially in the case of the DNN model, it took over 27 s to run 30 epochs.  
Considering the prediction accuracy and execution time of the six models obtained through the experiments, the LightGBM model showed the best performance and is expected to be most suitable for the dropout prediction. Table 9 compares the performance of the proposed LightGBM model with the models presented in previous studies. As discussed in Section 2, the models of [8,9,29,30,32] were compared. In [8], an F1-score of 0.808 was obtained using an ensemble model of CatBoost and XGBoost. In [9], the  
Figure 3: Average execution time of the LR, DT, RF, SVM, DNN, and LightGBM models.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Algorithm** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline Linear Regression (LR) & 0.927 & 0.815 & 0.637 & 0.715 \\ Decision Tree (DT) & 0.947 & 0.856 & 0.755 & 0.802 \\ Random Forest (RF) & 0.953 & 0.883 & 0.777 & 0.827 \\ Support Vector Machine (SVM) & 0.942 & 0.825 & 0.753 & 0.787 \\ Deep Neural Network (DNN) & 0.947 & 0.843 & 0.776 & 0.808 \\ Light Gradient Boosting Machine (LightGBM) & 0.955 & 0.867 & 0.814 & 0.840 \\ \hline Mean & 0.945 & 0.848 & 0.752 & 0.796 \\ Standard Deviation & 0.010 & 0.026 & 0.061 & 0.044 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Prediction performance of the LR, DT, RF, SVM, and DNN models discussed in Section 3.3.  
Figure 2: Precision, recall, and F1-scores of the proposed prediction models.  
performance of the ridge regression model was the best, providing a precision of 0.739. In [29], the RF model provided the best performance with an F1-score of 0.810. In [30], the SVM model was the best, providing an F1-score of 0.804. In [32], the DNN model achieved an accuracy of 0.768. Compared to the five existing models, the proposed LightGBM model showed improved performance in all aspects with accuracy, precision, and F1-score of 0.955, 0.867, and 0.840, respectively.