## 5 Proposed Feature Selection Method (A2 BCF)
Since our target population are the students in computing [1], we filtered the dataset to only include students from computing majors (CP = 11) to predict their success in their programs. * **Feature selection optimization using A2 BCF:** Feature selection is a primary task in Automated ML that helps the model achieve two main goals: better performance and reduced computational time. This step is the study's main contribution, an iterative ABC-based process to find the optimal set of features in a dataset. The proposed algorithm (A2 BCF) is an improved version of the Opt-ABC algorithm [32] used for optimizing the hyperparameter tuning of the ML algorithms [34; 35]. The output in this step is a subset of features selected by the algorithm. Figure 3: The overall structure of feature selection in this study. * **Data transformation:** This process converts data to the required format of our destination system. As shown in Figure 3, after preprocessing the original dataset, it is divided into splits of training and test sets (using k-fold cross-validation). Then, the A2 BCF optimization process starts. Depending on the selected subset of features by the algorithm, original train and test sets are transformed to new train and test sets (corresponding to the chosen features). Once the transformed training and test sets are ready, we leverage the ML algorithm on the train set. Then, the results are evaluated and stored using the testing set. This process continues until the stopping condition is met. Finally, the highest average accuracy (over three folds) and the optimal set of features are reported. * **Train the transformed data:** In this step, the desired ML algorithm is submitted, and the ML algorithm gets trained. Finally, the accuracy of the ML algorithm and final features are returned. ### Machine Learning Classifiers

A summary of the ML classifier used in this experiment is as follows:

1. Random Forest (RF) [36] is an ensemble method and a type of Decision Tree (DT) learner that operates by constructing many DTs in the training phase. This is the reason it is called 'forest'. The term 'random' is also used because the trees are built differently with random samples and random features to add diversity to the models and decrease the chance of overfitting [36; 37; 38]. Random forest initially uses the bagging method to combine the predictions from each tree and calculate the overall predictions. 2. Extreme Gradient Boosting (XGBoost) [39] is a developed version of Gradient Boosting that utilizes a gradient boosting framework as an ensemble. One of the focuses of XGBoost is the efficiency and speed of the model that supports parallelization. XGBoost also attempts to prevent overfitting using Ridge and Lasso regularization. XGBoost trains the model iteratively, correcting or fixing the newer models in each iteration [39]. 3. Support Vector Machine (SVM) is a supervised learning technique that generates input-output mapping functions. The mapping function for this study is a classification function in which nonlinear kernel functions are used to transform input to a high-dimensional feature space. In this feature space, the data become more separable when compared with the raw input. SVM works by finding the maximum-margin hyperplanes between positive and negative observations. Using mapping function, SVM transforms the nonseparable feature to linearly separable features [40; 41]. ### Setting

In order to evaluate the performance of the A2 BCF algorithm, the MIDFIELD dataset is used [42], and the goal is to predict students' success in computing majors (CP = 11). Students' attrition in computing fields is particularly problematic in computing. As mentioned in the previous sections, the ratio of the positive and negative classes in this dataset is 54:46. In other words, the percentage of negative values (attrition rate) is 46 percent and the percentage of positive values is 54 percent, which is in line with students' withdrawal from computing fields.