# A Machine Learning Framework for Student Retention Policy Development: A Case Study
## 5 Conclusions and Future Work  
Student retention in universities is an important educational challenge that affects all stakeholders from economic, quality, and psychological aspects. The proposed framework identifies the students at risk of dropping out at the end of the first year. Since a significant percentage of dropouts occur before the second year, our study employs the academic performance information of students in their first year. The data used for the system are accessible through any student registration system; specifically, it comprises the data used for generating student record sheets or transcripts. Despite the previous work, which employs data from a very large number of sources or utilizes detailed data pertaining to specific departments or classes, the proposed framework is self-contained in the registration or transcript generation system of any educational institution. This facilitates the broad application of the proposed solution to other educational institutions. In this case study, to implement the first workflow of the proposed framework, a comprehensive feature analysis followed by a comparison of the most frequently used machine learning models was performed on the dataset extracted from the registration system. The findings suggest that even though certain models like CatBoost and RF inherently perform well, other models such as SVC and MLP can achieve competitive performance with parameter tuning or feature engineering. Therefore, parameter optimization and optimal feature extraction may be employed to improve their performance for dropout prediction. The experimental results indicated that the CatBoost classifier is the best-performing classifier and it was employed in the proposed framework to predict the at-risk students in the context of the current case study. The second workflow in the proposed framework for student retention involves clustering the dropouts into categories to enable the design of tailored retention policies. To provide a human interpretable representation of the clustering, a decision tree with a maximum depth of three levels was used in the current work. The decision tree provides the policymakers with a verbal description of each of the clusters formed. This verbalization may then be used by the policymakers to interpret and comprehend the clusters formed and thus devise tailored policies for each cluster. The results of this study show that, even with limited information used for training, the system achieved a high predictive performance with an F1-Score of 82%. The proposed framework is adaptable for use in any HEI as it relies exclusively on the data used to generate student transcripts. The decision to employ limited data aligns with the goal of designing an accessible system that can be adopted by higher education institutions globally. Moreover, the results presented in this study show that even without behavioral or socioeconomic features, competitive prediction performance is possible. However, the framework could be enhanced by incorporating additional information from the registration database, such as students' grades in specific courses or course groups, as well as data from the institution's learning management systems, at both the dropout prediction and policy-making stages of the framework. It is anticipated that using more comprehensive data that covers, in addition to the mainly descriptive features employed in this study, predictive and diagnostic information, will further improve the predictive performance of the dropout prediction stage. Tailoring retention policies would also benefit from these additional data. For example, in the visualization or analysis stages, including information about the specific courses in which students struggle would enable policymakers to better identify the underlying causes of these challenges. Furthermore, the policy-making process could also be enhanced using various visualization tools and explainable AI techniques.