# Improving Software Defect Prediction in Noisy Imbalanced Datasets
## 3 Materials and Methods
### Ponr  
After undersampling, a dataset with repeated sample removal is obtained. However, undersampling greatly reduces the sample size in the dataset. To keep the data volume sufficient, the dataset cannot be balanced only with undersampling. At present, the most popular dataset balancing method in machine learning is SMOTE and its derivative methods. But the synthetic data strategy of this method creates the problems of overfitting and noise amplification. In order to overcome overfitting and dataset noise, propensity score matching (PSM) is introduced to generate synthetic samples and reduce dataset noise. The theoretical framework of the PSM method is to solve the difference between the observed sampling data distribution and the overall distribution. The systematic deviation uses the propensity score to measure the difference in high-dimensional data in the feature space. PSM is used for nearest neighbor search, which generates synthetic samples and reduces dataset noise. PSM is also used to reduce the complexity and cost of the algorithm while effectively denoising the dataset and the synthesis of minority samples, oversampling the samples closest to the center of the minority class, and improving the quality of the software defect dataset. This subsection calculates the propensity score of each sample in \(X_{\text{us}}\) by solving the weight vector of \(X_{\text{us}}\) and adds it as an additional feature to \(X_{\text{us}}\) for providing a basis for the subsequent PSM-based noise removal and oversampling process. First, use the logistic function to characterize the distribution of data:  
\[f(x)=\frac{1}{1+e^{-x}} \tag{2}\]  
Next, define an m-dimensional (m is the dimension of feature) weight vector \(\beta\) to characterize the similarity of samples. Also, define a constant \(\beta_{0}\) and perform a minimum initialization, allowing each element of \(\beta_{0}\) be a random decimal value close to 0 to ensure that \(-x_{i}{}^{\text{T}}\beta+\beta_{0}\
eq 0\):  
Now, define the maximum log-likelihood probability of the dataset as  
\[\ln L=\sum_{i=1}^{n}\Bigl{(}y_{i}\ln\Bigl{(}f\Bigl{(}x_{i}^{\text{T}}\beta \Bigr{)}\Bigr{)}+(1-y_{i})\ln\Bigl{(}1-f\Bigl{(}x_{i}{}^{\text{T}}\beta\Bigr{)} \Bigr{)}\Bigr{)} \tag{3}\]  
Solve the \(\beta\) that maximizes the maximum log-likelihood probability of the dataset:  
\[\operatorname{argmax}_{\beta}\ln L=\sum_{i=1}^{n}\Bigl{(}y_{i}\ln\Bigl{(}f \Bigl{(}x_{i}^{\text{T}}\beta\Bigr{)}\Bigr{)}+(1-y_{i})\ln\Bigl{(}1-f\Bigl{(}x_ {i}{}^{\text{T}}\beta\Bigr{)}\Bigr{)}\Bigr{)} \tag{4}\]  
Substitute the obtained weight vector \(\beta\) and the feature vector of each sample for the logistic function to obtain the propensity score \(f_{\beta}(x_{i})\) of each sample:  
\[f_{\beta}(x_{i})=\frac{1}{1+e^{-x_{i}{}^{\text{T}}\beta+\beta_{0}}} \tag{5}\]  
Add the propensity score of each sample to \(X_{\text{us}}\) as the \((m+1)\) th dimensional feature of the sample to obtain the propensity score dataset \(X_{\beta}\). For each sample \(x_{i}\), traverse the other samples in \(X_{\beta}\) to calculate the Euclidean distance between each sample:  
\[d_{\bar{\eta}}=(\sum_{x=1}^{m+1}\Bigl{|}x_{i}^{\text{z}}-x_{j}^{\text{z}} \Bigr{|}_{2})^{1/2}\operatorname{iff}x_{i}\
eq x_{j}\text{ and }x_{i},x_{j}\in X_{\beta} \tag{6}\]  
where \(x_{i}^{\text{z}}\) is the z-th dimension feature of \(x_{i}\). Then, find the k samples whose Euclidean distance is closest to \(x_{i}\) to form the k nearest neighbor sample set \(X_{i}\) of \(x_{i}\):  
\[XK_{i}=\Bigl{\{}\Bigl{(}x_{ij},y_{ij}\Bigr{)}\Bigr{|}j\
eq i,j\in(1,k+1),\min \bigl{(}d_{\bar{\eta}}\bigr{)}\text{ in }X_{\beta}\Bigr{\}} \tag{7}\]  
Then, oversampling the data, the number of synthetic samples to be generated is  
\[N_{\text{new}}=r_{\text{os}}\times\Bigl{(}\operatorname{num}_{y_{\bar{\eta}} =0}-\operatorname{num}_{y_{\bar{\eta}}=1}\Bigr{)},(x_{i},y_{i})\text{in }X_{\text{us}} \tag{8}\]  
where parameter \(r_{\text{os}}\) is the weight of the number of samples generated and is set to 1 by default.