## 7 Conclusions and Future Work  
Ensemble FS and sampling techniques play a vital role for improving the prediction accuracy of machine learning models. However, the application of these techniques on software maintainability is limited. In this study, we apply three individual models, NB, SVM, and KNN, along with one ensemble model, RF, on seven publicly available datasets collected from open-source software systems. We evaluated and compared the effectiveness of ensemble FS and ensemble sampling techniques on the performance of prediction change-proneness. This study presents many insights, which are based on comprehensive experimentation and analyses:  
* The results obtained from this study provide empirical evidence of the positive impact of ensemble FS in improving the performance of some of the prediction models (KNN and NB). However, ensemble FS techniques had no clear effect on the overall performance of SVM and RF because these models have FS techniques as a part of the model's creation.
* This study also showed the good performance of applying ensemble sampling methods on all prediction models and that there was a clear improvement in SVM and RF.
* Across all scenarios, the ensemble model (RF) achieved the best performance in predicting change-proneness compared to other models, and there were significant differences between RF and all individual models. In addition, the effect size was large. A possible explanation of the good performance of RF in both high-dimensional and imbalanced datasets is that RF performs the concept of FS through the creation of several decision trees. It also involves many decision trees that perform well with the imbalanced dataset.
* The experimental results in this empirical study showed that the performance of the ensemble models for predicting change-proneness is significantly improved, and the effect size was large in all prediction models.  
Indeed, the results obtained from this study provide empirical evidence of the positive impact of ensemble FS and sampling techniques in improving the performance of the prediction models to predict change-proneness. We observed that the ensemble model (RF) achieved the best performance in predicting change-proneness compared to other models. Moreover, the proposed ensemble FS and sampling techniques improved the prediction accuracy of change-proneness. In future work, we will investigate the combination of different FS and sampling techniques to achieve more consistent results. Moreover, we will employ different prediction models on various software maintainability datasets, and the study can also be extended to recognize the most suitable metrics to indicate change-proneness.  
The following are available at [[https://www.mdpi.com/article/10.3390/app12105234/s1](https://www.mdpi.com/article/10.3390/app12105234/s1)]([https://www.mdpi.com/article/10.3390/app12105234/s1](https://www.mdpi.com/article/10.3390/app12105234/s1)), Supplementary A: Provides the descriptive static of the datasets. Supplementary B: Shows a correlation matrix to identify the dependence between variables. Supplementary C: Presents the datasets used in this study.  
Conceptualization, H.A.; methodology, H.A.; software, H.A.; validation, H.A.; formal analysis, H.A.; investigation, H.A.; resources, H.A.; data curation, H.A.; writing--original draft preparation, H.A.; writing--review and editing, H.A.; visualization, H.A.; supervision, M.R.; project administration, M.R.; funding acquisition, H.A. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the Deanship of Scientific Research at Princess Nourah hint Abdulrahman University, through the Pioneer Researcher Funding Program (Grant No#PR-1440-5).  
Not Applicable.  
The authors declare no conflict of interest.