# Improving Software Defect Prediction in Noisy Imbalanced Datasets
## 3 Materials and Methods
### Predictor Construction
Before training different ML classifiers to obtain different machine learning models using \(\text{x}_{\text{train}}\), some classifiers need hyperparameter or threshold optimization to obtain the best performance. In this method, for RR and LAR, the RandomizedSearchCV algorithm is used to tune up. For KNN, the GridSearchCV algorithm is used to determine the best n-neighbors. For LiR, KR and other classifiers that need to optimize the threshold, set the initial threshold as the median of the prediction results, then use the dichotomy to iterate, \begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Description** \\ \hline LiR & Linear regression \\ RR & Ridge regression \\ LoR & Logistic regression \\ LDA & Linear discriminant analysis \\ QDA & Quadratic discriminant analysis \\ KR & Kernel ridge \\ SVC & C-Support vector classification \\ SGDC & Linear classifiers (SVM) \\ KNN & K-Nearest neighbors vote classifier \\ GNB & Gaussian naive Bayes \\ DT & Decision tree classifier \\ RF & Random forest classifier \\ ET & Extra trees classifier \\ AB & AdaBoost classifier \\ \hline \hline \end{tabular}
\end{table} 
Table 4: ML algorithms. and following this, select the threshold with the maximum score as the optimal threshold. The classifiers are trained to obtain ML models, then they are used to predict \(\text{x}_{\text{validate}}\), and the results are scored using AUC. AUC indicates the probability that a positive sample is greater than a negative sample when a trained classifier is used to predict a pair of data. The formula for AUC is

\[\text{AUC}=\frac{\sum_{\text{i}\in\text{positiocClass}}\text{rank}_{\text{i}}- \frac{\text{M}(1+\text{M})}{2}}{\text{M}\times\text{N}} \tag{12}\]

Repeat the above operation until each group of data is regarded as \(\text{x}_{\text{validate}}\) once. Calculate the average \(\text{AUC}\)\(\overline{\text{AUC}_{\text{max}}}\) of the scores of different machine learning models in \(\text{N}_{\text{k}}\) experiments and take the model algorithm with the largest \(\overline{\text{AUC}_{\text{max}}}\) and the model algorithms in \(\text{C}_{\text{CV}}\) within the difference \(\left|\text{AUC}-\overline{\text{AUC}_{\text{max}}}\right|\leqslant\text{T}_ {\text{dev}}\) as the final selection algorithms. Due to the different datasets, class imbalance, data distribution, and other characteristics of different software, in the process of constructing the predictor, the balance parameter settings of the undersampling process should be optimized. Here, a simple scheme is proposed in order to optimize \(\text{r}_{\text{us}}\) in the CV aggregation process described above. For a dataset with a class unbalanced ratio r, experiment with different \(\text{r}_{\text{us}}\) values with a step size between [1, r]. Take the \(\text{r}_{\text{us}}\) with the highest \(\overline{\text{AUC}_{\text{max}}}\) as \(\text{r}_{\text{us\_best}}\) shown in Algorithm 2. Then, take the parameter \(\text{r}_{\text{us\_best}}\) as the selected undersampling ratio to obtain \(\text{X}_{\text{final}}\) during the actual prediction. Train the selected aggregated ML model using \(\text{X}_{\text{final}}\) to obtain the defect prediction model. Measure metrics of the test set of the target software, and then implement prediction. Multiple LM models in the aggregated model were selected in the previous step, and the prediction results may not be the same, so the prediction with the highest AUC should be selected as the final prediction result. However, because it costs much less to test a software defect than to repair it, the prediction results should include as many defective modules as possible, so that more defects can be found when testing. In practice, the final defect prediction result should combine the defect modules predicted by each model.