## 3 Research Methodology
### Framework of the Research Method  
Figure 1 shows the framework of the research method, which contains several steps:  
* **Step 1. ** We load the datasets, details of which, along with independent and dependent variables are described in Section 4.1. * **Step 2. ** We analyze the datasets to eliminate any metrics which contain empty values, and redundant metrics that are strongly correlated with each other. This step is performed using descriptive statistics and Spearman correlation. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**The Area of** **Related Work** & **Ref.** & **Advantages** & **Disadvantages** & **Contributions of This Study** \\ \hline \multirow{3}{*}{\begin{tabular}{c} **Change-proneness** \\ **metrics** \\ \end{tabular} } & \begin{tabular}{c} A limited number of \\ metrics have been used \\ by previous studies as \\ predictors of \\ change-proneness. \\ \end{tabular} & \begin{tabular}{c} Several metrics \\ have not been \\ investigated by \\ previous studies. \\ \end{tabular} & This study explores a total of 102 metrics. \\ \hline \multirow{3}{*}{\begin{tabular}{c} **Ensemble** \\ **prediction** \\ **models** \\ \end{tabular} } & \begin{tabular}{c} Ensemble models have \\ yielded improved \\ prediction accuracy over \\ individual models. \\ \end{tabular} & \begin{tabular}{c} Limited previous \\ studies used \\ ensemble models to \\ predict \\ change-proneness, \\ \end{tabular} & \begin{tabular}{c} This study investigates the \\ performance of RF and the \\ impact of the influence of \\ Mry parameter tuning \\ using the grid search. \\ \end{tabular} \\ \hline \multirow{3}{*}{\begin{tabular}{c} **Ensemble** \\ **feature** \\ **selection** \\ **techniques** \\ \end{tabular} } & \begin{tabular}{c} Using ensemble FS \\ techniques outperformed \\ individual FS techniques. \\ \end{tabular} & \begin{tabular}{c} No previous studies \\ used ensemble PS to \\ predict \\ change-proneness. \\ \end{tabular} & \begin{tabular}{c} This study performs \\ ensemble PS (i.e., relief and \\ Pearson's correlation \\ coefficient). \\ \end{tabular} \\ \hline \multirow{3}{*}{\begin{tabular}{c} **Ensemble** \\ **tembling** \\ **techniques** \\ \end{tabular} } & \begin{tabular}{c} Using ensemble sampling \\ techniques performed \\ sampling techniques. \\ \end{tabular} & \
\begin{tabular}{c} No previous studies \\ used ensemble \\ sampling to predict \\ change-proneness. \\ \end{tabular} &
\begin{tabular}{c} This study applies ensemble \\ sampling (i.e., SMOTE, \\ spread sub-sample and \\ randomize). \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of the related work. * **Step 4. ** We perform ensemble FS using the relief and Pearson's correlation coefficients to select the best ten metrics (independent variables) from each dataset. * **Step 5. ** We employ ensemble sampling techniques, namely, SMOTE, to perform oversampling by increasing the number of the minority class, and spread sub-sample to perform undersampling by decreasing the number of the majority class, and randomization to rearrange the instances randomly. The main reason for applying this randomization is to avoid overfitting in ten-fold cross-validation because SMOTE inserts the additional instances at the end of the dataset. * **Step 6. ** We conduct four different scenarios across seven change-proneness datasets (see Table 1) as follows: **First scenario** (i.e., Step 1, Step 2, and Step 3), **Second scenario** (i.e., Step 1, Step 2, Step 3, and Step 4), **Third scenario** (i.e., from Step 1, Step 2, Step 3, and Step 5 without Step 4) and **Fourth scenario** (i.e., Step 1, Step 2, Step 3, Step 4, and Step 5) (see Figure 2). * **Step 7. ** We divide the datasets into training and test sets using ten-fold cross-validation. Figure 1: Framework of the research method. * **Step 8. ** We construct the prediction models, which encompass three individual models (NB, SVM, and KNN) and one ensemble model (RF). * **Step 9.