# Improving Software Defect Prediction in Noisy Imbalanced Datasets
## 3 Materials and Methods
### Predictor Construction
Fine-tuning these parameters may improve their performance slightly. ### Evaluation

In our study, AUC was used to evaluate the prediction performance of the models. Other indicators such as MCC were also calculated in the experiment, but due to the limited length of the article, these indicators are not fully addressed in the paper. Readers who are interested in detailed results should contact the authors by email. ## 7 Discussion and Conclusions

In the past two decades, software defect prediction (SDP) has developed rapidly as an important way to improve software quality and reliability. However, class imbalance and noise are often present in SDP datasets. With the continuous improvement of SDP algorithms based on machine learning, improving the quality of software defect data offers a further significant effect to improve SDP performance. Oversampling such as SMOTE is one of the most well-known methods to balance the datasets, but it can introduce overfitting noise to datasets. Thus, our article attempts to solve the data imbalance in the SDP dataset and reduce the noise in it. In this paper, US-PONR is proposed as a dataset pre-processing method to improve SDP dataset quality. To alleviate imbalance in a dataset, US-PONR first uses undersampling to remove redundant samples due to version iterations, and then it uses oversampling to remove potential noise samples, relying on propensity score matching as a dimension of the feature vector. A total of 24 versions of software data from 11 projects were used in experiments to test the proposed method. In an SDP experiment conducted in different noise environments, US-PONR demonstrated its ability to improve the quality of SDP datasets. And in experiments on removing label noise samples, the new method demonstrated that it can effectively identify label noise samples and remove them. However, there are still a few limitations related to the datasets will need to be addressed. For our data, the experiments only used the CK, process, and network metrics extracted from PROMISE datasets. In the future, we hope to test our method against commercial datasets and more metrics that represent software defects. And artificial noise may be introduced in the generation of noise datasets using for experiments, so the ability of the method to identify and remove label noise samples needs further validation in real software projects. **Author Contributions:** Conceptualization, H.S. ; methodology, H.S. ; software, H.S. ; validation, H.S. and J.X. ; formal analysis, H.S. ; investigation, H.S. and J.X. ; resources, J.X. ; data curation, J.X. ; writing--original draft preparation, H.S. ; writing--review and editing, H.S. and J.L. ; visualization, H.S. ; supervision, J.A. ; project administration, J.A. ; funding acquisition, No. All authors have read and agreed to the published version of the manuscript. **Funding:** This research received no external funding. **Institutional Review Board Statement:** Not applicable. **Informed Consent Statement:** Not applicable. **Data Availability Statement:** The entire meta dataset can be found at [[https://github.com/buaaSof](https://github.com/buaaSof)]([https://github.com/buaaSof](https://github.com/buaaSof)) twareReliabilityGroup/US-PONR (accessed on 18 September 2023). **Conflicts of Interest:** The authors declare no conflict of interest. ## References

* (1) [PERSON]; [PERSON]; [PERSON] Be more familiar with our enemies and pave the way forward: A review of the roles bugs played in software failures. _J. Syst. Softw._**2017**, _133_, 68-94. [CrossRef]
* (2) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Recent catastrophic accidents: Investigating how software was responsible. In Proceedings of the SSIR 2010--4 th IEEE International Conference on Secure Software Integration and Reliability Improvement, Singapore, 9-11 June 2010; pp. 14-22. [CrossRef]
* (3) [PERSON]; [PERSON]; [PERSON] Benchmarking Machine Learning Techniques for Software Defect Detection. _Int. J. Softw. Eng. Appl._**2015**, \(6\), 11-23.