# Improving Software Defect Prediction in Noisy Imbalanced Datasets
## 3 Materials and Methods
### Predictor Construction
We marked each introduced label noise sample and then checked the residual noise ratio in the datasets after using each method. The residual noise ratio is the number of the remaining marked noise samples divided by the total number of samples after using the data processing method. The residual noise ratio in the dataset is used to verify the effectiveness of the noise reduction method: the smaller the proportion of remaining marked noise samples in the datasets, the more effective the noise-reduction method. The setting of dataset parameters in this experiment was the same as that used to test RQ2. Table 9 shows the proportion of residual marked label noise samples after using each method in 10%, 20%, and 30% noise environments. For example, in 10% noise level ant, the residual noise sample ratio recorded for US-PONR was 1.77%, which means that US-PONR identified and removed 8.23% of the noise samples. The table shows that in each dataset and noise level, the residual noise ratio was the lowest after using US-PONR (maximum: \(-22.82\%\) in 30% noise Xerces; minimum: \(-15.4\%\) in 30% noise level Velocity), which proves that the proposed method is effective in removing noise samples and improving the quality of datasets. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline  & **US-PONR** & **SOMO** & **MWMOTE** & \begin{tabular}{c} **SMOTE,** \\ **IF** \\ \end{tabular}  & \begin{tabular}{c} **SMOTE,** \\ **RS5B** \\ \end{tabular}  & \begin{tabular}{c} **SMOTE,** \\ **RSST\_2T** \\ \end{tabular}  & \begin{tabular}{c} **SMOTE** \\ **TonetLinks** \\ \end{tabular}  & \ \begin{tabular}{c} **SMOTE** \\ **SMOTE** \\ \end{tabular}  & **ADASNN** & **MSMOTE** & \begin{tabular}{c} **Gaussian,** \\ **SMOTE** \\ \end{tabular}  & **CCR** \\ \hline \multicolumn{1}{l}{**anti**} & & & & & & & & & & & & \\
10\% noise & 0.017 & 0.0995 & 0.0675 & 0.0670 & 0.0687 & 0.0662 & 0.0644 & 0.0000 & 0.0643 & 0.0644 & 0.0647 & 0.0643 & 0.0672 \\
20\% noise & 0.0155 & 0.1996 & 0.1479 & 0.1487 & 0.1579 & 0.1448 & 0.1417 & 0.1340 & 0.1415 & 0.1498 & 0.1413 & 0.1550 \\
30\% noise & 0.0026 & 0.2844 & 0.2416 & 0.2407 & 0.2509 & 0.2405 & 0.2355 & 0.2180 & 0.2355 & 0.2399 & 0.2347 & 0.2354 & 0.2193 \\  & \multicolumn{1}{l}{US-PONR} & **SOMO** & **MWMOTE** & **SMOTE** & **SMOTE** & **SMOTE** & **SMOTE** & **Selfometric** & **ADASNN** & **MSMOTE** & **CCR** \\ \multicolumn{1}{l}{**came1**} & & & & & & & & & & & & & \\
10\% noise & 0.0185 & 0.0996 & 0.0677 & 0.0650 & 0.0884 & 0.0685 & 0.0682 & 0.0597 & 0.0678 & 0.0682 & 0.0683 & 0.0675 & 0.0637 \\
20\% noise & 0. ## 6 Threats to Validity

### Datasets

In our SDP experiment, we tested US-PONR on 24 versions of software data from 11 projects in the PROMISE dataset, and our proposed method performed admirably. However, before being put into practice, US-PONR needs to be tested against actual commercial projects. Moving forward, we plan to apply this method to some commercial projects to verify its validity. Although our team members manually labeled the samples to simulate as much as possible the process of generating label noise in real data (see Section 4.2.2), our test methodology inevitably brought some artificial features to the dataset. In the future, US-PONR's effectiveness will also need to be verified in other noisy datasets. Finally, the CK, process, and network metrics were chosen to be used in the experiment as feature vectors for representing software defects. There are other metrics that can be used to characterize software defects, such as the Hausdorff metric [61]. Validation of the method against these metrics also needs to be carried out in future work. ### Model Hyperparameters

US-PONR and other methods for detecting software defects have difficulty controlling every variable and every model parameter, such as the n_neighbors k, the noise discrimination threshold t, or the weight of the number of samples generated. Therefore, in our experiment, we used the parameters recommended by their authors, and we performed no further tuning of them.