# A Machine Learning Framework for Student Retention Policy Development: A Case Study
## 3 Methodology
### Proposed Framework
#### 3.2.2 Machine Learning Models Used
* Multi-layer Perceptron (MLP) is a fully connected feedforward Artificial Neural Network class. MLP creates a model for classifying data using multi-layer neural networks [11]. * The Naive Bayes (NB) algorithm is based on [PERSON] theorem and the assumption of independence between features. NB is a probability-based model that is effective in classification problems [12]. * Logistic Regression (LR) is a statistical model used to explain or predict a dependent variable (usually a categorical or binary variable) by a set of independent variables. LR is used to estimate the probability distribution of the dependent variable [23]. * Classification and Regression Trees (CART) is a machine learning algorithm used for classification and regression tasks. The purpose of this algorithm is to give the piecewise constant estimator of a classifier or a regression function from the training sample of observations. This algorithm creates branches and leaves by splitting the dataset to create decision trees. It is based on binary tree-structured partitions and a penalized criterion that allows some \"good\" tree-structured estimators to be selected from a large collection of trees [24]. Table 3 presents a comparison of the ML classifiers employed in the current study. It can be seen that SVC, MLP, RF, CatBoost, and CARTs are computationally more demanding compared to KNNs, NB, and LR, which are simpler algorithms. Furthermore, the ensemble-based ML algorithms CatBoost, RF, and CARTs are known for their inherent capabilities to handle slight imbalances during their training process [25]. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**ML Algorithm** & **Pros** & **Cons** \\ \hline
**Support Vector Classifier (SVC)** & Powerful for high-dimensional data and works well with small to medium-sized datasets [26]. & Computationally expensive for large datasets, requiring alternative approaches like linear SVMs or deep learning [27]. \\ \hline
**K-Nearest Neighbors (KNN)** & Simple, effective for small datasets, and adaptable to various data distributions [28]. & Expensive and slow for large datasets, making it impractical in big data applications [29]. \\ \hline
**Logistic Regression (LR)** & Simple, interpretable, and efficient for linear problems [30]. & Struggles with complex, nonlinear relationships and high-dimensional data [31]. \\ \hline
**Gaussian Naïve Bayes (NB)** & Efficient, works well with small datasets, and is effective in high-dimensional spaces [32]. & The assumption of feature independence is often unrealistic, leading to poor performance when features are correlated [33]. \\ \hline
**Artificial Neural Network (MLP)** & Excels at capturing complex nonlinear relationships, making them ideal for tasks like image recognition and NLP [34]. & Require high computational power and are prone to overfitting without proper regularization [35]. \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of the classifiers used. A challenge of the machine learning frameworks is the interpretability of the results. Specifically for dropout prediction, it is not sufficient to predict the students who are likely to drop out. The framework must provide policymakers with tools and information they can use to devise intervention strategies or retention policies. To enable the decision-making and policy-generation process, the current study clusters the students into categories and then fits a decision tree to allow verbalization of the clusters. These clusters are used as visualization tools to support the policy generation process. These ML algorithms are briefly described below:  
* The clustering (K-means) algorithm applies a simple and straightforward approach to divide a particular dataset into a specified number of clusters [41]. The aim is to form the clusters so that inter-cluster similarities between the members of a cluster are maximized. At the same time, each cluster is expected to be dissimilar to all other clusters; that is, the intra-cluster similarities are minimized.