# A Machine Learning Framework for Student Retention Policy Development: A Case Study
## 3 Methodology
### Proposed Framework
#### 3.2.2 Machine Learning Models Used  
We tested eight of the most frequently employed machine learning algorithms during the dropout prediction phase of the proposed framework, allowing us to identify the highest-performing model for predicting at-risk students. Following the prediction stage, two additional machine learning algorithms were applied to visualize the students identified as at risk of dropout, supporting the policy-making phase. Below, we provide brief definitions of all 8 machine learning algorithms used for predicting at-risk students in this study:  
* Support Vector Classifier (SVC) is a model designed especially for binary classification problems. It is a statistical learning method that can handle both linear and nonlinear problems by constructing a line or hyperplane to separate the data into different classes. Kernel tricks such as radial basis function (RBF) are used to map the data into higher dimensional spaces to allow the classification of nonlinear data using hyperplanes [14; 20]. * K-Nearest Neighbors (KNN) is a lazy algorithm that determines the class of a sample by assigning a given sample to the class that is most frequently represented among its \"K\" nearest data points based on a similarity measure. This approach relies on the principle that similar data points tend to be close to each other in the feature space [21]. * Random Forest (RF) is an ensemble learning algorithm that enhances predictive performance by combining the outputs of multiple decision trees. Each tree in the Random Forest (RF) is a random forest. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline
**Feature** & **Description** & **Category** \\ \hline BirthYear & Birth year of the student & Demographic \\ \hline Gender & The biological sex of the student & Demographic \\ \hline GraduateDateHighSchool & Year and month of the high school graduation date & Demographic \\ \hline Nationality & Student’s nationality & Demographic \\ \hline RegistrationAge & Student’s age at registration year & Demographic \\ \hline Department & The department where the student is enrolled & Enrolment \\ \hline EntryType & The admission type (university entrance exam, international exams, etc.) of the student & Enrolment \\ \hline Faculty & The faculty where the student is enrolled & Enrolment \\ \hline RegistrationDate & Year and month of the registration date & Enrolment \\ \hline RegistrationSemester & Shows the intake semester (fall or spring) & Enrolment \\ \hline RegistrationYear & The academic year in which a student enrolls typically begins in September of one year and ends in June of the following year & Enrolment \\ \hline ScholarRate & The scholarship percentage of the student & Enrolment \\ \hline NotPass\_Course\_Semester1 & Number of courses the student did not pass at the end of the first semester & Academic performance \\ \hline NotPass\_Course\_Semester2 & Number of courses the student did not pass at the end of the second semester & Academic performance \\ \hline Pass\_Course\_Semester2 & Number of courses the student passed at the end of the second semester & Academic performance \\ \hline Pass\_Course\_Semester1 & Number of courses the student passed at the end of the first semester & Academic performance \\ \hline Repeat\_Course & Number of courses the student repeated & Academic performance \\ \hline \end{tabular}
\end{table} 
Table 2: Features extracted from the dataset. Forest is trained on a random subset of the data, and the final prediction is determined by aggregating the predictions of all individual trees. Ensemble methods aim to produce a more robust and superior model by combining several weak learners [20]. * The Categorical Boosting (CatBoost) algorithm model is effective in classification problems. CatBoost uses categorical variant-specific variations of gradient boosting algorithms to improve the handling and use of categorical variables. Categorical variables are variables that have non-numeric values that represent class labels in classification problems [22].