## 5 Results and Analyses
### Prediction Models
#### 5.3.8 Discussion and Answers to Research Questions
This supports the findings in the previous question indicting that SVM and RF algorithms can deal with FS during their creation. Consequently, these algorithms respond to sampling techniques more than FS techniques. RQ3. What is the impact of applying both ensemble FS and sampling techniques on the performance of prediction models? Applying both ensemble FS and sampling techniques improved the performance of the prediction models, and RF achieved the best performance across all datasets except mcMMO. These findings corroborate the ideas of [PERSON] and [PERSON], who used the same datasets to predict refactoring and performed principal component analysis and SMOTE techniques to extract the best features and resolve the imbalanced data problem [8]. Their results indicated that the prediction accuracy with the SMOTE technique was better than that without SMOTE, and the prediction accuracy of all metrics was better than that with FS. A possible explanation for these results may be the lack of adequate datasets, and the rank of the best ten features selected is considered low (the average ranking of the best ten metrics ranges from 0.1 to 0.6), whereas the difference before and after applying sampling techniques is high (see Tables 10 and 11). RQ4. What is the performance of the individual model and do the ensemble models (i.e., RF) outperform individual models? KNN achieved the best prediction accuracy among most cases. RF outperformed other individual models and achieved the best result in terms of average AUC value (see Figure 3) across all scenarios. The results of the ANOVA test reveal that there were significant differences between RF and all individual models (see Figure 5). In addition, the results of the effect size were large (see Table 16). Therefore, applying ensemble sampling techniques on RF produced the highest accuracy to predict change-proneness. This finding is in accordance with previous studies, in which RF provided the best performance to predict change-proneness [4], software faults [60], and maintenance changes [61]. RQ5. What is the impact of the Mtry parameter tuning in RF?Mtry parameter tuning in RF using grid search method, along with RF, mlbench, and caret packages, improved prediction accuracy. This improvement increased in the original datasets (e.g., without FS or sampling) and decreased in the edited datasets (e.g., with FS and sampling in the fourth scenario). The findings of this RQ are consistent with those of [PERSON] et al., who stated that RF created using caret package in R was the best model among 179 models applied on 121 original datasets [62]. Furthermore, the Mtry parameter differed from various datasets, and this difference was behind the study showing that there are no suggestions to choose the specific number of the Mtry parameter [26]. In addition, this supports the use of automatic parameters tuning to simply and effectively improve performance [63; 64]. Based on these findings, it is recommended to tune the Mtry parameter automatically to save time and efforts and improve the results. In future work, a statistical test will be used to investigate the performance difference between RF without and with parameter tuning.