## 3 Methodology
### Machine Learning Models Used  
This study use tree-based models, kernel-based models, and linear models for student dropout prediction, which belongs to a binary classification problem that the student will be dropped out or not.  
Tree-based models use if-then-else rules to solve problems. All tree-based models can be used for classification (predicting categorical values) as well as regression (predicting numerical values). Kernel-based models transform nonlinear problems into linear problems in feature space to solve the problem. Linear models can be generalized as functions that make predictions from linear combinations of features. In this paper, five commonly used classification models have been used:  
* Four tree-based models: Decision Tree [38], which draws the different solutions of the decision as branches of the tree and uses the branching and pruning method to find the optimal solution. Random Forest [18], which consists of a bootstrap aggregation method that combines the predictions of many trees. LightGBM [39], which uses histogram-based algorithms and bucket continuous feature (attribute) values into discrete bins. XGBoost [17], which provides a parallel tree boosting to solve problems quickly  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Year of** **Admission**} & **Number of** **Students in** & **Number of** **Student** & **Number of** **Student** & **Sum** & **Dropout Rate** \\  & **School** & **Graduates** & **Dropouts** & & **Rate** \\ \hline
2013 & 126 & 4259 & 738 & 5123 & 0.14 \\ \hline
2014 & 371 & 3914 & 794 & 5079 & 0.16 \\ \hline
2015 & 935 & 3086 & 680 & 4701 & 0.14 \\ \hline
2016 & 1923 & 2349 & 588 & 4860 & 0.12 \\ \hline
2017 & 2870 & 1290 & 519 & 4679 & 0.11 \\ \hline
2018 & 4033 & 336 & 432 & 4801 & 0.09 \\ \hline
2019 & 4178 & 280 & 336 & 4794 & 0.07 \\ \hline
2020 & 4614 & 0 & 259 & 4873 & 0.05 \\ \hline
2021 & 4814 & 0 & 51 & 4865 & 0.01 \\ \hline Sum & 23,948 & 29,099 & 6963 & 60,010 & 0.13 \\ \hline \hline \end{tabular}
\end{table}
Table 3: _Cont._One linear model: Logistic regression [40], which is a linear model for classification often used as a baseline model;  
One Kernel-based model: Support Vector Machine [41], which transforms a linearly inseparable problem in the original feature space into a linearly separable problem in a high-dimensional feature space.  
Since student dropouts can be classified as a binary classification problem in machine learning, this study used four performance metrics to evaluate our models: accuracy, precision, recall, and F1 Score based on the confusion matrix, as shown in Table 4.  
The accuracy is the percentage of the total sample that the model correctly predicted, defined as follows:  
\[\text{Accuracy}= \frac{\text{TP}+\text{TN}}{\text{TP}+\text{FP}+\text{FN}+\text{TN}} \tag{1}\]  
The recall (or true positive rate) measures the ability of the model to detect positive samples, defined as follows:  
\[\text{Recall}= \frac{\text{TP}}{\text{TP}+\text{FN}} \tag{2}\]  
The precision (or positive predictive value) is the ratio between the number of samples correctly classified as positive and the total number of samples classified as positive (correct or incorrect), defined as follows:  
\[\text{Precision}= \frac{\text{TP}}{\text{TP}+\text{FP}} \tag{3}\]  
The F1 score comprehensively evaluates the classification performance of the classifier with precision and recall, defined as follows:  
\[\text{F1 Score}= \frac{2\times\text{TP}}{2\times\text{TP}+\text{FP}+\text{FN}} \tag{4}\]