# Improving Software Defect Prediction in Noisy Imbalanced Datasets
## 3 Materials and Methods  
### Framework  
This section introduces the framework of US-PONR (or Undersampling, Propensity Score Matching-Based Oversampling, and Noise Reduction), which is illustrated in Figure 1. The US-PONR consists of three main steps: undersampling, PSM-based oversampling and noise reduction, and predictor construction. An overview of each step appears below, while a more in-depth discussion of each step follows in Sections 3.2-3.4.  
In our method, after obtaining the defect datasets, the CK [6], process [7], and network metrics [11,13] were selected to characterize the source code samples, which contain 80 metrics. Tables 1-3 shows some of the metrics used, while the entire meta dataset can be found at [[https://github.com/buaaSoftwareReliabilityGroup/US-PONR](https://github.com/buaaSoftwareReliabilityGroup/US-PONR)]([https://github.com/buaaSoftwareReliabilityGroup/US-PONR](https://github.com/buaaSoftwareReliabilityGroup/US-PONR)) (accessed on 18 September 2023). The first step in US-PONR is undersampling (US). After the source code is represented using metrics, the software data are undersampled to pre-adjust the degree of imbalance in the dataset, which alleviates the problem of class imbalance and overfitting of the dataset by reducing the number of repeated non-defective samples in the dataset. At this point in the process, the ratio of undersampling also needs to be determined. A more detailed account of undersampling in US-PONR is provided in Section 3.2.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Description** \\ \hline WMC & Weighted methods per class \\ DIT & Depth of inheritance tree \\ NOC & Number of children \\ CBO & Coupling between object classes \\ RFC & Response for a class \\ LCOM & Lack of cohesion in methods \\ LOC & Lines of code \\ \hline \hline \end{tabular}
\end{table}
Table 1: CK metrics.  
Figure 1: Framework of US-PONR.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Description** \\ \hline REVISIONS & Number of revisions of a file. \\ AUTHORS & Number of distinct authors that checked a file into the repository. \\ LOC\_ADDED & Sum over all revisions of the lines of code added to a file. \\ MAX\_LOC\_ADDED & Maximum number of lines of code added for all revisions. \\ AVE\_LOC\_ADDED & Average lines of code added per revision. \\ LOC\_DELETED & Sum over all revisions of the lines of code deleted from a file. \\ MAX\_LOC\_DELETED & Maximum number of lines of code deleted for all revisions. \\ AVE\_LOC\_DELETED & Average lines of code deleted per revision. \\ CODECHURN & Sum of (added lines of code—deleted lines of code) over all revisions. \\ MAX\_CODECHURN & Maximum CODECHURN for all revisions. \\ AVE\_CODECHURN & Average CODECHURN per revision. \\ \hline \hline \end{tabular}
\end{table}
Table 2: Process metrics.  
The second step in US-PONR is PSM-based oversampling and noise reduction (PONR). Both of these tasks are performed using the PSM technique to obtain the final dataset. PONR is performed on the undersampled dataset by calculating the propensity score of each sample in the dataset and then by obtaining the nearest neighbor sample set of each sample based on propensity score matching. According to the distribution of the PSM-based nearest neighbor sample set, oversample synthesis is performed to balance the dataset. After oversampling, PSM is performed again to judge whether a sample is noise and to exclude noise samples. Section 3.3 covers the specific calculation for the propensity score of each sample and the detailed steps of PONR.  
The third step in US-PONR is predictor construction. To obtain the defect predictor, an ensemble model aggregating multi-ML classifiers was trained by the cross-validation (CV) method, in order to select the most appropriate ML model for different datasets. Section 3.4 describes how to perform cross-validation and to aggregate the model in detail.