# A Machine Learning Framework for Student Retention Policy Development: A Case Study
## 2 Literature Review
implemented six different machine learning models--Logistic Regression, Decision Tree, Random Forest, Support Vector Machine, Deep Neural Network, and Light Gradient Boosting Machine (LightGBM)--to investigate dropout rates by students who were enrolled at Sahmyook University. The key focus of their research was that they addressed the challenge of data imbalance in dropout cases. To mitigate this issue, they experimented with various oversampling methods, including SMOTE, ADASYN, and Borderline-SMOTE. Among the tested models, the LightGBM model has been found to be the most effective one, which reported the highest F1-score of 84%. Likewise, [PERSON], [PERSON] conducted data balancing techniques for predicting student dropout by using the ML approach, which is a challenging problem in the education sector [16]. [PERSON] intended to employ a data balancing technique for his estimations due to the imbalances of student dropout data. He reported that the number of enrolled students is always more than the number of dropouts. In this study, 57,348 records from the Uwezo dataset and 13,430 records from the India dataset were used together. The solution to the data imbalance issue was studied using Random Over Sampling (ROS), Random Under Sampling (RUS), Synthetic Minority Over Sampling Technique (SMOTE), SMOTE with Edited Nearest Neighbor (SMOTE-ENN), and SMOTE with Tomek Links (SMOTE-TOMEK). Such methods were conducted to enhance the prediction Accuracy of the minority class while maintaining satisfactory overall classification performance. These techniques were evaluated in combination with three widely used classification models, such as Logistic Regression (LR), Random Forest (RF), and Multi-layer Perceptron (MLP), which are in conformity with our estimation methods. The results demonstrated that SMOTE-ENN achieved the highest classification performance and Logistic Regression correctly classified the highest number of dropout students. The application of these models enables the development of early intervention strategies to reduce school dropout rates by ensuring that students at risk are accurately identified. Similarly, [PERSON] et al. developed a model to predict student dropouts by integrating machine learning and artificial intelligence techniques into the sustainability of education [17]. The Support Vector Machine (SVM), Random Forest, Gradient Boosting, K-Nearest Neighbors (KNN), and Logistic Regression algorithms were used for estimating student dropouts. [PERSON] et al. collected data samples, which consisted of 500 students' academic records, attendances, demographics, socioeconomic status, and engagement factors of participation in extracurricular activities during the 2021 academic year. The NeuralNetwork model provided the highest dropout prediction, and the estimated results obtained are as follows: Precision 85%, Recall 80%, and F1-score 82%. The empirical results obtained by Neural Network are robust and the scores above 80% can be considered highly reliable. The research recently undertaken by [PERSON] et al. employed big data and applied a machine learning approach to identify key factors that are contributing to student dropout [18]; moreover, they developed a predictive model to enable education authorities to monitor students in real time and intervene when necessary. [PERSON] et al. conducted a data sample with 197 students' demographics and academic record information for the period 2006-2011 at a private university in Malaysia. The study carried out by [PERSON] et al. examined information about students' socioeconomic backgrounds and psychological factors. The Logistic Regression (LR) classification model was employed by [PERSON] et al. and the estimated results reported that they achieved an Accuracy rate of 85%, Precision of 83%, and Recall of 80%, as well as a high F1-score rate of 81%, which revealed that academic performance is the most important predictor of student attrition.