# Tsbagging: A Novel Cross-Project Software Defect Prediction Algorithm Based on Semisupervised Clustering

Authors: Shiqi Tang; Song Huang; ErHu Liu; YongMing Yao; KaiShun Wu; Haijin Ji
Journal: Scientific Programming
 (2022)

---

## Abstract

Software defect prediction (SDP) is an important technology which is widely applied to improve software quality and reduce development costs. It is difficult to train the SDP model when software to be test only has limited historical data. Cross-project defect prediction (CPDP) has been proposed to solve this problem by using source project data to train the defect prediction model. Most of CPDP methods build defect prediction models based on the similarity of feature space or data distance between different projects. However, when the target project has a small amount of label data, these methods usually do not consider this part of data information. Therefore, when the distribution between source project and target project is quite different, these methods are difficult to achieve good prediction performance. To solve this problem, this paper proposes a CPDP method based on a semisupervised clustering (namely, Tsbagging). Tsbagging has two stages; in the first stage, we cluster to the source project data based on the limited labeled data in the target project and assign different weights to these source project data according to the clustering results. In the second stage, we use bagging method to train the prediction model based on the weight assigned in the first stage. The experimental results show that the performance achieved by Tsbagging is better than other existing SDP methods.


---

## Body

## 1. Introduction

SDP is a very important technology in the software testing stage. It can quickly predict defects and provide guidance for allocating test resources and manpower in the early stage of software development [1, 2]. At present, most SDP methods use machine learning technology to build defect prediction models [3–5]. For example, Lessmann et al. [6] and Shepperd et al. [7] used traditional machine learning algorithms, such as decision tree, naive Bayes (NB), neural network, and support vector machine (SVM) to SDP, and achieved good results. Elish et al. [8] compared 8 machine learning methods on the NASA dataset, and the results showed that SVM was better than other algorithms. In addition, researchers also improved the traditional machine learning methods based on the characteristics of SDP. For example, Wang et al. [9] proposed a Compressed C4.5 Models (CCM) based on C4.5 decision tree by using Spearman rank correlation coefficient. Ji et al. [10] applied the Kolmogorov–Smirnov test on the datasets from the promise database and found that feature space of those datasets was not normally distributed. Therefore, they improved the NB and proposed an SDP method based on kernel density estimation. Moreover, Ji et al. [11] proposed a weighted naive Bayes method (WNB). WNB used the information diffusion model instead of the default probability density function of normal distribution to calculate the probability density of each feature. This method also achieved good performance. Li et al. [12] proposed CoForest and ACoForest. CoForest is a sampling method based on semisupervised learning. On the basis of random forest classification, this method can find optimal sampling example by random sampling. ACoForest further enhanced the prediction performance of CoForest base on active learning. Studies demonstrated that these two methods showed better performance than other traditional machine learning methods. Bejjanki et al. [13] proposed a new class imbalance reduction algorithm (CIR) for the class imbalance problem in defect prediction dataset. This method made the defect data and nondefect data in unbalanced dataset symmetrical by considering the distribution characteristics of the unbalanced dataset. Experiments showed that CIR has better defect prediction performance than traditional methods.Most of the above research studies have focused on within project defect prediction (WPDP). However, sometimes, few local training data can be available in the target software in which defects must be detected, because past local defective modules are expensive to be labeled for the module development in an unfamiliar domain [4, 14]. To solve this problem, the CPDP method used the source project data training model to predict defects in the target project. Zimmermann et al. [15] constructed 622 cross-project task combinations from 12 projects to evaluate CPDP model performance. The experimental results showed that traditional defect prediction methods have difficulty to achieving good CPDP performance. It is because that these methods assumed that the training and test data have similar distributions, whereas data from different projects have different distributions. Ma et al. [16] assigned different weights to the data in the source project by calculating the similarity between those data and the target project and then trained a weighted Bayes classifier (namely, TNB) according to these weights. Nam et al. [17] proposed the TCA+, TCA + used the transfer component analysis technology to find the potential feature space of different project data, when the potential space is determined, and it mapped the source project and target project to the space to eliminate the difference of feature distribution between different projects. Turhan et al. [18] proposed a CPDP method based on nearest neighbors (namely NN). NN constructed a training dataset by selecting nearest neighbors with target project data in source project and use the training dataset to train the defect prediction model. Kawata et al. [19] proposed a CPDP method based on DBSCAN [20], they use DBSCAN to find subclusters and then select subclusters which have at least one record of the target project data to build and train the dataset. Ryu et al. [21] proposed a CPDP method (VCB + SVM). Firstly, VCB + SVM calculates the similarity weight of each instance in the source project based on the dataset of the target project and then constructs the defect prediction model based on the support vector machine and boosting method. Chen et al. [22] proposed a CPDP method collective transfer defect prediction (CTDP) based on multisource transfer learning. Firstly, CTDP extended the source project dataset by using different normalization and TCA. Then, they built several base classifiers on the extended source project dataset. According to the contribution of each classifier to the target project, CTDP uses the particle swarm optimization algorithm to adaptively weight these base classifiers to construct an ensemble classifier. Finally, CTDP uses the ensemble classifier to predict defect. He et al. [23] proposed a CPDP method on multisource transfer learning (FSS + bagging). FSS + bagging has three steps. Firstly, FSS + bagging calculated the similarity between the candidate source project and target project and selected the first k most similar source project. Next, for each source project which be selected, FSS + bagging removed some unstable features to reduce the data distribution difference between source project and target project. Finally, FSS + bagging used bagging to get the final defect prediction results. To solve the class imbalance in CPDP, Limsettho et al. [24] proposed a new oversampling method CDE-SMOTE, which used CDE to estimate the class distribution of the target project and uses SMOTE to modify the class distribution of training data until it is similar to the distribution of target project. Although the above methods have achieved good performance in CPDP, most of them build CPDP models based on the similarity of feature space or data distance between source project and target project. However, when the target project has limited labeled data, those methods often do not take into account this part of label information. Turhan et al. [25] found that mixing a limit of target project data and source project data which are selected by the NN method to train classifier can significantly improve the performance of CPDP. However, this method also does not consider the limited label information when selecting the source project data. Therefore, when the data distribution between the target project and the source project was very different (especially, when concept shift occurs), those method are difficult to achieve good performance. Chen et al. [26] propose a two-stage transfer learning method, double transfer boosting (DTB), to solve this problem. In the first stage, DTB uses the data gravitation method proposed by Ma et al. [16] to assign weights to the source project data; in the second stage, DTB uses Tradaboost [27] to build classifier based on the weight assigned in the first stage. However, DTB does not consider the impact of concept shift when assigning weights, which affects the final defect prediction performance. This paper proposes a CPDP method based on semisupervised clustering (namely, Tsbagging). In Tsbagging, we firstly proposed a semisupervised clustering algorithm (namely, TSCluster) to clustering the source project data based on limited target project data. Next, we trained several base classifiers based on the clustering results and integrated these classifiers by bagging to predict defect. We compare the performance of Tsbagging with other SDP methods on 42 defect prediction datasets.The experimental results show that Tsbagging has better performance than other methods.


## 2. Methodology

### 2.1. Nearest Neighbor Filter

Since the CPDP method can use source project data to help train the defect prediction model when target project only has limited software historical data, it has always been the research focus in the field of SDP. However, an important problem in CPDP is how to select data similar to target project from source project. The NN (nearest neighbor filter) is a commonly used technology for selecting data from source project [18, 25]. The specific steps are as follows: firstly, for each data in the target project, NN select K-nearest neighbors of it from the source project based on the Euclidean distance. If there are n data in the target project, the number of finally selected nearest neighbors is k × n. Then, NN removes duplicate data in the selected nearest neighbor data to build the final train dataset. Finally, NN uses the final train dataset to train the SDP model. Moreover, Turhan found that the NN + WP (WP means within project) which is trained by the limited target data and source data selected by NN can achieve better performance than NN. However, both NN and NN + WP only select data in the source project based on the similarity between the data distances of different projects. When the data distribution between target project and source project is very different, especially in the case of concept shift, it is difficult for NN to accurately select the data similar to the target project from the source project.


### 2.2. Concept Shift

Concept shift is also called concept drift [28]. According to Moreno-Torres’s theory [28], the concept shift is defined as: PS(x) = PT(x), PS(y | x) and PT(y | x) are different. Where PS(y | x) and PT(y | x) represent the posteriori probabilities in the source project and the target project, respectively. PS(x) and PT(x) represent the marginal distribution of source and target projects, respectively. Moreno-Torres considers that nonstationary environments can lead to concept shift, and the relationship between the input and class variables has changes. Kanayake at al. [29] finds the concept shift can occur in SDP and influence the defect prediction quality. In CPDP, different development environments or application fields can lead to the concept shift. When concept shift occurs, due to PS(y | x) and PT(y | x) are different, even if the distance between source project data and target project data is close, their label maybe very different. Therefore, when the concept shift occurs, the NN method is difficult to accurately select the appropriate data from source project. As shown in Figure 1, all data are distributed in four areas A, B, C, and D, in which circular points represent nondefect data, triangular points represent defect data, red points represent target project data, and blue points represent source project data. As a result of concept shift, it is found that the target project’s label is different to its nearest neighbors in areas A and D. If NN is used to select data from the source project, the data with large difference from the distribution of the target project will be selected.Figure 1
Schematic diagram of concept shift.


### 2.3. Semisupervised Clustering

Semisupervised clustering can use additional information to achieve better clustering performance than unsupervised clustering. At present, there are mainly two types of semisupervised clustering methods. One is based on connected and unconnected constraints. The connected constraints means the data belong to the same cluster, and the unconnected constraints means the data do not belong to the same cluster. For example, Wagstaff et al. [30] proposed the constrained k-means algorithm. This algorithm assigns appropriate clusters to each data by checking whether the data violate the given connected and unconnected constraint pairs in the clustering process of k-means. The other type of the semisupervised clustering method is based on the given cluster label information. This method helps clustering by taking the given cluster label as a constraint seed. For example, Bsau et al. [31] proposed the constrained seed k-means method, and this method initializes the cluster center of k-means according to the given cluster label information and did not change the cluster membership relationship of seed samples in the clustering. In this paper, we use a limit of target project data as constraint seeds to cluster the source project data selected by the NN method and exclude the data with large differences with the target project data for achieving better prediction performance.


## 2.1. Nearest Neighbor Filter

Since the CPDP method can use source project data to help train the defect prediction model when target project only has limited software historical data, it has always been the research focus in the field of SDP. However, an important problem in CPDP is how to select data similar to target project from source project. The NN (nearest neighbor filter) is a commonly used technology for selecting data from source project [18, 25]. The specific steps are as follows: firstly, for each data in the target project, NN select K-nearest neighbors of it from the source project based on the Euclidean distance. If there are n data in the target project, the number of finally selected nearest neighbors is k × n. Then, NN removes duplicate data in the selected nearest neighbor data to build the final train dataset. Finally, NN uses the final train dataset to train the SDP model. Moreover, Turhan found that the NN + WP (WP means within project) which is trained by the limited target data and source data selected by NN can achieve better performance than NN. However, both NN and NN + WP only select data in the source project based on the similarity between the data distances of different projects. When the data distribution between target project and source project is very different, especially in the case of concept shift, it is difficult for NN to accurately select the data similar to the target project from the source project.


## 2.2. Concept Shift

Concept shift is also called concept drift [28]. According to Moreno-Torres’s theory [28], the concept shift is defined as: PS(x) = PT(x), PS(y | x) and PT(y | x) are different. Where PS(y | x) and PT(y | x) represent the posteriori probabilities in the source project and the target project, respectively. PS(x) and PT(x) represent the marginal distribution of source and target projects, respectively. Moreno-Torres considers that nonstationary environments can lead to concept shift, and the relationship between the input and class variables has changes. Kanayake at al. [29] finds the concept shift can occur in SDP and influence the defect prediction quality. In CPDP, different development environments or application fields can lead to the concept shift. When concept shift occurs, due to PS(y | x) and PT(y | x) are different, even if the distance between source project data and target project data is close, their label maybe very different. Therefore, when the concept shift occurs, the NN method is difficult to accurately select the appropriate data from source project. As shown in Figure 1, all data are distributed in four areas A, B, C, and D, in which circular points represent nondefect data, triangular points represent defect data, red points represent target project data, and blue points represent source project data. As a result of concept shift, it is found that the target project’s label is different to its nearest neighbors in areas A and D. If NN is used to select data from the source project, the data with large difference from the distribution of the target project will be selected.Figure 1
Schematic diagram of concept shift.


## 2.3. Semisupervised Clustering

Semisupervised clustering can use additional information to achieve better clustering performance than unsupervised clustering. At present, there are mainly two types of semisupervised clustering methods. One is based on connected and unconnected constraints. The connected constraints means the data belong to the same cluster, and the unconnected constraints means the data do not belong to the same cluster. For example, Wagstaff et al. [30] proposed the constrained k-means algorithm. This algorithm assigns appropriate clusters to each data by checking whether the data violate the given connected and unconnected constraint pairs in the clustering process of k-means. The other type of the semisupervised clustering method is based on the given cluster label information. This method helps clustering by taking the given cluster label as a constraint seed. For example, Bsau et al. [31] proposed the constrained seed k-means method, and this method initializes the cluster center of k-means according to the given cluster label information and did not change the cluster membership relationship of seed samples in the clustering. In this paper, we use a limit of target project data as constraint seeds to cluster the source project data selected by the NN method and exclude the data with large differences with the target project data for achieving better prediction performance.


## 3. Tsbagging

In this section, we propose a CPDP method, namely Tsbagging. Tsbagging mainly consists of clustering stage and ensemble stage. In the clustering stage, we propose a semisupervised method (TSCluster) to clustering the data of the source project and assign different weights to these data according to the clustering results. In the ensemble stage, based on the weights given in the clustering stage, we use bagging [32] to train several classifiers and integrate them to predict the defect. In the following sections, we describe Tsbagging in details.


### 3.1. Problem Description

In this paper, we define three sets:DS=xiS,…,yiS|i=1,2,…,Sn, DWP=xjWC,yjWC|j=1,2,…,Wn, and DTest=xtTest|t=1,2,…,Tn, where DS represents the source project dataset,DWP represents the labeled dataset in the target project (WP means within project), and DTest represents the dataset to be tested in the target project. Note that the data from Dtest and DWP belong to the same distribution, while DS belongs to different distribution from Dtest and DWC. xiS, xjWC, and xtTest are m-dimensional vectors. Each dimension in the vector represents a measured value of a metric to a program module. yiS and yjWC∈ {−1, 1}, −1 represents that a module is a nondefect proneness module, 1 means that a module is a defect proneness module, Sn, Wn, and Tn represent the data amount of DS, DWP, and DTest, respectively, and Wn ≪ Tn.


### 3.2. Overall Flow of Tsbagging

The overall process of Tsbagging is shown in Figure2. Tsbagging consists of clustering stage and ensemble stage. In the clustering stage, firstly, we use the NN filter to select the candidate dataset (namely, FCCData) from DS, and then we propose a two-step clustering method. In the first clustering, we use the k-means to cluster the data from different class in DWP, respectively, so that in the final clustering result, the data in each cluster belong to the same class, and this class label is regarded as the cluster label. For example, in Figure 2, the result of the first cluster contains six clusters. A red circle indicates that all the data in the cluster are defective data, and a green circle indicates that all the data in the cluster are nondefective data. In the second clustering, we use the semisupervised method based on the given clustering label information to cluster FCCData. In this clustering process, we take the clusters gathered by the first clustering as constraint seed, and we do not change the cluster membership of data from DWP in this cluster. For example, in Figure 2, there are still six clusters in the second result. After the second clustering, we assign weights to data in each cluster based on the clustering hypothesis to build the final training dataset (namely, CCTrainData). The clustering hypothesis is that if all data in the dataset obey the same distribution, data from same cluster are more likely to have the same label. Conversely, if the label of data from the same cluster is different, they do not obey the same distribution. So, the idea of assign weight as follows: in each cluster, if the label of a data from DS is the same as the cluster label, we believe that the data obey the clustering hypothesis; that is, the data may obey the data distribution of the target project, so we give the data a higher weight. If the label of the data DS is different from the cluster label, we believe that the data violate the clustering hypothesis. It means that the data may not obey the data distribution of the target project, so the data are given a lower weight. When all the data from DS are given weights, we use the bagging method to train the final classifier from CCTrainData to predict Dtest. We will describe the algorithm in detail in the following section.Figure 2
Overall flow of Tsbagging.


### 3.3. Clustering Stage

The clustering stage has two steps. In the first step, we use the NN method to select the candidate dataset (FCCData). In the second step, we use TSCluster proposed to clustering FCCData and assgin weights to the data in FCCData according to the clustering results. The flow of TSCluster is shown in algorithm 1.Algorithm 1: TSCluster.
Input: rep, pk,nk,FCCDataDWP, ϵOutput: CCTrainData//The first clustering to construct the initial cluster setInitClusterSet = FristCluser(pk, nk, Dwp);//The second clusteringfinalClusterSet = [];fori in range(1,InitClusterSet.length) dofinalClusterSet.add(InitClusterSet[i]);forq in range(1,rep) do //rep is the number of cluster iterationsnowClustersSet = [];fori in range(1,InitClusterSet.length) do //Initialize cluster seed according to InitClusterSetnowClustersSet.add(InitClusterSet[i]);for (xi, yi) ∈ FCCData doCalculatethe distance between (xiyi) and the mean uj of finalClusterSet[j](1 <j< InitClusterSet.length): dij  =  xi−uj2;Find the cluster closest to (xiyi)according to dij:ti=argminti∈1,2,…,InitclusterSet.lengthdij;nowClustersSetti.add((xi,yi));if the mean valueuj of finalClusterSet[j] is equal to the mean value now uj of nowClustersSet[j] then //Stopping conditionbreak;finalClusterset = nowClustersSet;//Set labelfinalClusterSet[j].Clusterlabel =yinit,where, (1 <j< InitClusterSet.length), (xinityinit) ∈ InitClusterSet[j];//Assign weightCCTrainData = [];forj in range(1,InitClusterSet.length) dofor (xiyi) ∈ finalClusterSet[j] doif (xiyi) ∈ FCCData thenwij  =  ϵ∗log((1/dis(xi,uj))+e);ifyi  = = finalClusterSet[j].Clusterlabel thenXi.weight = wij;elseXi.weight = 1/ wij;CCTrainData.add((xiyi));return CCTrainData;TSCluster has two clustering processes. In the first clustering, we cluster to the defect data and nondefect data inDWP, respectively, and construct the initial cluster seed of the second clustering processes according to the clustering results. Specifically, we first select defect dataset and nondefect dataset from DWP, repectively. Next, we use k-means method to clustering these datasets to build a defect cluster set and nondefect cluster set (where defect cluster set has pk clusters and nondefect cluster set has nk clusters). Then, this two cluster sets are merged to built the initial cluster seed (namely, InitClusterSet) for semisupervised clustering. The specific process is shown in algorithm 2.Algorithm 2: FristCluser.
Input: pk,nk,DWPOutput: InitClusterSetDefectDataset = SelectDefectData(DWP)//Selectdefect data;NonDefectDataset = SelectNonDefectData(DWP)//Select non defect data;ifpk > DefectDataset.size thenpk = DefectData.size;ifnk > NonDefectDataset.size thennk = NonDefectData.size;//Use kmeans to clustering the DefectDataset and NonDefectDatasetDefectClusterSet = Kmeans(pk,DefectData);NonDefectClusterSet = Kmeans(nk,NonDefectData);InitClusterSet = DefectClusterSet + NonDefectClusterSet;Remove the empty cluster in InitClusterSet;Return InitClusterSet:In the second clustering, based on the idea that is similar to constrained k-means [31], we use the InitClusterSet constructed by the first clustering as a constraint seed to clustering the FCCData dataset. Specifically, in each iteration of the clustering process, we use InitClusterSet to initialize the new cluster set and then calculate the distance between FCCData’s data and each cluster center; then, we assign data to the nearest cluster. It is worth noting that in this process, we do not change the cluster membership of the data from DWP. When all FCCData’s data are assigned, we update the centers of all clusters; if all cluster centers are not updated, the clustering is terminated and the final cluster set (finalClusterSet) is obtained.After two clustering, we assign different weights to the data in each cluster. The specific steps are as follows: firstly, for the thejth cluster in final cluster set (finalClusterSet[j]), we label a cluster label (namely, Clusterlabel), and set Clusterlabel is yinit, where xinityinit ∈ InitClusterSet[j]; then, according to the clustering assumption (i.e., data in the same cluster are more likely to have the same label), we assign higher weight to data which label is the same as Clusterlabel and assign lower weight to data which label is different from Clusterlabel. The specific calculation formula is as follows:(1)wij=ϵ∗log1disxi,uj+eyi=clusterlabel,1ϵ∗log1/disxi,uj+eyi≠clusterlabel,where (xi, yi) ∈ finalClusterSet[j] ∩ FCCData, uj is the cluster center of finalClusterSet[j], and dis(.) is the Euclidean distance between xi and uj, ϵ is an amplification factor, and in this paper, ϵ = 2. In equation (1), the weight of data with the same label as Clusterlabel is higher than the weight of data with different label to Clusterlabel. For data (xiyi), when yi  = Clusterlabel, it is closer to the cluster center, it is more likely obey the data distribution of target project; therefore, the weight of data is higher. On the other hand, when yi≠ Clusterlabel, it is closer to the cluster center, the data are less likely to obey the data distribution of target project; therefore, the weight of data is lower. After all data have been assigned weight, we add these data to a final training dataset (namely, CCTrainData), and then in the ensemble stage, we use CCTrainData to train the classifier.


### 3.4. Ensemble Stage

In the ensemble stage, we use the bagging method to train several classifiers (in this paper, we use NB as a classifier) and integrate these classifiers to predict the target data. Firstly, we sample data from CCTrainData to built several datasets according to the weight assigned in the clustering stage. It should be noted that a data with a higher weight mean a higher probability of it be sampled. Each dataset has CClen data. CClen is the amount of data in CCTrainData. Then, we combine these datasets withDWP to train the base classifiers, and add these base classifiers to a classifier set (Classifierpool). Finally, when all classifiers are trained, we use the plurality voting to ensemble all base classifiers in the Classifierpool to predict the data in the DTest. The specific ensemble method is as follows:(2)Hx=Cargmaxj∑i=1ClassifiersizeIhixtTest=Cj,where Cj is the predicted label, and its value is C0 = 1 or C1 = 1 Classifiersize is the amount of all base classifiers in the Classifierpool. When the classifier classifies x as cj, hi (x) is the prediction function of the ith classifier, hi (x) ∈ {−1, 1}, I(x)is an indicator when x is true, and I (x) = 1, otherwise I(x) = 0. The overall process of Tsbagging is shown in algorithm 3.Algorithm 3: Tsbagging.
Input:DS, DWP, DTest, ϵ,rep,pk,nk,ClassifiersizeClassifierpool = [];//Using NN to select the candidate datasetFCCData = NN (DTestDS);//Clustering stageCCTrainData = TSCluster(rep, pk,nk,FCCData,DWP);//Ensemble stagefori in range(1,classifiersize) doTrainDatai  =  DWP + resampleWithWeights (CCTrainData);Classifieri  = Train (TrainDatai)//train the classifier;Classifierpool.add (Classifieri);Use equation (2) to predict the defects of DTest.


## 3.1. Problem Description

In this paper, we define three sets:DS=xiS,…,yiS|i=1,2,…,Sn, DWP=xjWC,yjWC|j=1,2,…,Wn, and DTest=xtTest|t=1,2,…,Tn, where DS represents the source project dataset,DWP represents the labeled dataset in the target project (WP means within project), and DTest represents the dataset to be tested in the target project. Note that the data from Dtest and DWP belong to the same distribution, while DS belongs to different distribution from Dtest and DWC. xiS, xjWC, and xtTest are m-dimensional vectors. Each dimension in the vector represents a measured value of a metric to a program module. yiS and yjWC∈ {−1, 1}, −1 represents that a module is a nondefect proneness module, 1 means that a module is a defect proneness module, Sn, Wn, and Tn represent the data amount of DS, DWP, and DTest, respectively, and Wn ≪ Tn.


## 3.2. Overall Flow of Tsbagging

The overall process of Tsbagging is shown in Figure2. Tsbagging consists of clustering stage and ensemble stage. In the clustering stage, firstly, we use the NN filter to select the candidate dataset (namely, FCCData) from DS, and then we propose a two-step clustering method. In the first clustering, we use the k-means to cluster the data from different class in DWP, respectively, so that in the final clustering result, the data in each cluster belong to the same class, and this class label is regarded as the cluster label. For example, in Figure 2, the result of the first cluster contains six clusters. A red circle indicates that all the data in the cluster are defective data, and a green circle indicates that all the data in the cluster are nondefective data. In the second clustering, we use the semisupervised method based on the given clustering label information to cluster FCCData. In this clustering process, we take the clusters gathered by the first clustering as constraint seed, and we do not change the cluster membership of data from DWP in this cluster. For example, in Figure 2, there are still six clusters in the second result. After the second clustering, we assign weights to data in each cluster based on the clustering hypothesis to build the final training dataset (namely, CCTrainData). The clustering hypothesis is that if all data in the dataset obey the same distribution, data from same cluster are more likely to have the same label. Conversely, if the label of data from the same cluster is different, they do not obey the same distribution. So, the idea of assign weight as follows: in each cluster, if the label of a data from DS is the same as the cluster label, we believe that the data obey the clustering hypothesis; that is, the data may obey the data distribution of the target project, so we give the data a higher weight. If the label of the data DS is different from the cluster label, we believe that the data violate the clustering hypothesis. It means that the data may not obey the data distribution of the target project, so the data are given a lower weight. When all the data from DS are given weights, we use the bagging method to train the final classifier from CCTrainData to predict Dtest. We will describe the algorithm in detail in the following section.Figure 2
Overall flow of Tsbagging.


## 3.3. Clustering Stage

The clustering stage has two steps. In the first step, we use the NN method to select the candidate dataset (FCCData). In the second step, we use TSCluster proposed to clustering FCCData and assgin weights to the data in FCCData according to the clustering results. The flow of TSCluster is shown in algorithm 1.Algorithm 1: TSCluster.
Input: rep, pk,nk,FCCDataDWP, ϵOutput: CCTrainData//The first clustering to construct the initial cluster setInitClusterSet = FristCluser(pk, nk, Dwp);//The second clusteringfinalClusterSet = [];fori in range(1,InitClusterSet.length) dofinalClusterSet.add(InitClusterSet[i]);forq in range(1,rep) do //rep is the number of cluster iterationsnowClustersSet = [];fori in range(1,InitClusterSet.length) do //Initialize cluster seed according to InitClusterSetnowClustersSet.add(InitClusterSet[i]);for (xi, yi) ∈ FCCData doCalculatethe distance between (xiyi) and the mean uj of finalClusterSet[j](1 <j< InitClusterSet.length): dij  =  xi−uj2;Find the cluster closest to (xiyi)according to dij:ti=argminti∈1,2,…,InitclusterSet.lengthdij;nowClustersSetti.add((xi,yi));if the mean valueuj of finalClusterSet[j] is equal to the mean value now uj of nowClustersSet[j] then //Stopping conditionbreak;finalClusterset = nowClustersSet;//Set labelfinalClusterSet[j].Clusterlabel =yinit,where, (1 <j< InitClusterSet.length), (xinityinit) ∈ InitClusterSet[j];//Assign weightCCTrainData = [];forj in range(1,InitClusterSet.length) dofor (xiyi) ∈ finalClusterSet[j] doif (xiyi) ∈ FCCData thenwij  =  ϵ∗log((1/dis(xi,uj))+e);ifyi  = = finalClusterSet[j].Clusterlabel thenXi.weight = wij;elseXi.weight = 1/ wij;CCTrainData.add((xiyi));return CCTrainData;TSCluster has two clustering processes. In the first clustering, we cluster to the defect data and nondefect data inDWP, respectively, and construct the initial cluster seed of the second clustering processes according to the clustering results. Specifically, we first select defect dataset and nondefect dataset from DWP, repectively. Next, we use k-means method to clustering these datasets to build a defect cluster set and nondefect cluster set (where defect cluster set has pk clusters and nondefect cluster set has nk clusters). Then, this two cluster sets are merged to built the initial cluster seed (namely, InitClusterSet) for semisupervised clustering. The specific process is shown in algorithm 2.Algorithm 2: FristCluser.
Input: pk,nk,DWPOutput: InitClusterSetDefectDataset = SelectDefectData(DWP)//Selectdefect data;NonDefectDataset = SelectNonDefectData(DWP)//Select non defect data;ifpk > DefectDataset.size thenpk = DefectData.size;ifnk > NonDefectDataset.size thennk = NonDefectData.size;//Use kmeans to clustering the DefectDataset and NonDefectDatasetDefectClusterSet = Kmeans(pk,DefectData);NonDefectClusterSet = Kmeans(nk,NonDefectData);InitClusterSet = DefectClusterSet + NonDefectClusterSet;Remove the empty cluster in InitClusterSet;Return InitClusterSet:In the second clustering, based on the idea that is similar to constrained k-means [31], we use the InitClusterSet constructed by the first clustering as a constraint seed to clustering the FCCData dataset. Specifically, in each iteration of the clustering process, we use InitClusterSet to initialize the new cluster set and then calculate the distance between FCCData’s data and each cluster center; then, we assign data to the nearest cluster. It is worth noting that in this process, we do not change the cluster membership of the data from DWP. When all FCCData’s data are assigned, we update the centers of all clusters; if all cluster centers are not updated, the clustering is terminated and the final cluster set (finalClusterSet) is obtained.After two clustering, we assign different weights to the data in each cluster. The specific steps are as follows: firstly, for the thejth cluster in final cluster set (finalClusterSet[j]), we label a cluster label (namely, Clusterlabel), and set Clusterlabel is yinit, where xinityinit ∈ InitClusterSet[j]; then, according to the clustering assumption (i.e., data in the same cluster are more likely to have the same label), we assign higher weight to data which label is the same as Clusterlabel and assign lower weight to data which label is different from Clusterlabel. The specific calculation formula is as follows:(1)wij=ϵ∗log1disxi,uj+eyi=clusterlabel,1ϵ∗log1/disxi,uj+eyi≠clusterlabel,where (xi, yi) ∈ finalClusterSet[j] ∩ FCCData, uj is the cluster center of finalClusterSet[j], and dis(.) is the Euclidean distance between xi and uj, ϵ is an amplification factor, and in this paper, ϵ = 2. In equation (1), the weight of data with the same label as Clusterlabel is higher than the weight of data with different label to Clusterlabel. For data (xiyi), when yi  = Clusterlabel, it is closer to the cluster center, it is more likely obey the data distribution of target project; therefore, the weight of data is higher. On the other hand, when yi≠ Clusterlabel, it is closer to the cluster center, the data are less likely to obey the data distribution of target project; therefore, the weight of data is lower. After all data have been assigned weight, we add these data to a final training dataset (namely, CCTrainData), and then in the ensemble stage, we use CCTrainData to train the classifier.


## 3.4. Ensemble Stage

In the ensemble stage, we use the bagging method to train several classifiers (in this paper, we use NB as a classifier) and integrate these classifiers to predict the target data. Firstly, we sample data from CCTrainData to built several datasets according to the weight assigned in the clustering stage. It should be noted that a data with a higher weight mean a higher probability of it be sampled. Each dataset has CClen data. CClen is the amount of data in CCTrainData. Then, we combine these datasets withDWP to train the base classifiers, and add these base classifiers to a classifier set (Classifierpool). Finally, when all classifiers are trained, we use the plurality voting to ensemble all base classifiers in the Classifierpool to predict the data in the DTest. The specific ensemble method is as follows:(2)Hx=Cargmaxj∑i=1ClassifiersizeIhixtTest=Cj,where Cj is the predicted label, and its value is C0 = 1 or C1 = 1 Classifiersize is the amount of all base classifiers in the Classifierpool. When the classifier classifies x as cj, hi (x) is the prediction function of the ith classifier, hi (x) ∈ {−1, 1}, I(x)is an indicator when x is true, and I (x) = 1, otherwise I(x) = 0. The overall process of Tsbagging is shown in algorithm 3.Algorithm 3: Tsbagging.
Input:DS, DWP, DTest, ϵ,rep,pk,nk,ClassifiersizeClassifierpool = [];//Using NN to select the candidate datasetFCCData = NN (DTestDS);//Clustering stageCCTrainData = TSCluster(rep, pk,nk,FCCData,DWP);//Ensemble stagefori in range(1,classifiersize) doTrainDatai  =  DWP + resampleWithWeights (CCTrainData);Classifieri  = Train (TrainDatai)//train the classifier;Classifierpool.add (Classifieri);Use equation (2) to predict the defects of DTest.


## 4. Experiment

In this section, we describe three experiments we conducted to verify the performance of Tsbagging.


### 4.1. Datasets

In this experiment, we use seven projects in the public database promise [33] to build cross-project defect prediction datasets. The detailed description of these datasets is shown in Table 1. Each dataset in these datasets represents the test results of a program module, which contains 20 attributes and one label. The detailed description of these attributes is shown in Table 2 [34]. The label values are −1 and 1.1 represents defective module, and −1 represents nondefective module. Similar to reference [35–37], in each experiment, we select two different datasets from the seven datasets, and one of which is used as the target project and the other as the source project to build the CPDP experimental dataset (for example, we use poi-2.0 as the target project and synapse-1.2 as the source project); therefore, we have created 42 experimental datasets.Table 1
Details of the experiment dataset.
ProjectModuleDefect-prone moduleDefect-prone (%)Poi-2.03143712Synapse-1.22568634Ant-1.63519226Camel-1.260821636log4j-1.11093734Jedit-4.03067525Xerces-1.24407116Table 2
Attribute description.
AttributewmcWeighted methods per classditDepth of inheritance treenocNumber of childrenrfcResponse for a classlcomLack of cohesion in methodslcom3Normalized version of LCOMnpmNumber of public methodsLocLines of codedamData access metricmoaMeasure of aggregationmfaMeasure of function abstractioncamCohesion among methods of classicInheritance couplingamcAverage method complexityCaAfferent couplingsceEfferent couplingsmax_ccMaximum values of methods in the same classavg_ccMean values of methods in the same class of methods in a given classcbmCoupling between methodscboCoupling between object classes


### 4.2. Performance Measures

In this paper, we useF1-measure, MCC, g-measure, and balance [36, 38–40] to evaluate the performance of defect prediction. These performance measures are calculated from the confusion matrix shown in Table 3, where TP means true defect, FP means false defect, FN means false nondefect, and TN means true nondefect.Table 3
Obfuscation matrix.
ActualDefectNondefectPredictedDefectTPFPNondefectFNTNRecall is a measure of completeness, describing probabilities of true defective modules in comparison with the total number of defective modules:(3)recall=TPTP+FN.Precision is a measure of exactness, which defines the probabilities of the presence of modules that are truly defective from the number of modules predicted to be defective:(4)Precision=TPTP+FP.PF shows the proportion of all the modules without defects predicted to be defective:(5)PF=FPFP+TN.F1-measure is the harmonic average of recall and precision. An algorithm with a higher F1-measure value (namely, F1 value) implies a better performance as(6)F1=2∗recall∗Precisionrecall+Precision.Matthews correlation coefficient (MCC) can take into account the measurement of TP, FP, TN, and TN in a more comprehensive way. Its range of values is between [−1, 1], where 1 denotes a perfect prediction, and −1 indicates complete disagreement between actual and predicted values. The more the values are means the better the performance of the classifier will be:(7)MCC=TP∗TN−FN∗FPTP+FNTP+FPFN+TNFP+TN.G-measure [14] is the harmonic average of recall and (1–PF). An algorithm with a higher g-measure value implies a better performance as(8)g−measure=2∗Recall∗1−PFRecall+1−PF.The measure Balance is introduced by calculating the Euclidean distance from the real (recall, PF) point to (1, 0), since the point (recall = 1, PF = 0) is the optimal point where all defects are detected without missing:(9)Balance=1−0−PF2+1−Recall22.


### 4.3. Experimental Design

In this section, we design experiments to verify the performance of Tsbaging in the following four questions:RQ1. Can Tsbagging effectively overcome the impact of different data distributions between different projects, so as to obtain better prediction performance compared with traditional defect prediction methods?To research this question, we compare Tsbagging with other traditional SDP methods and calculate theirF1-measure, MCC, g-measure, and balance values. It is worth noting that the traditional SDP methods do not take into account the impact of data distribution differences between source project and target project when training prediction models. If the performance of Tsbagging is better than other traditional SDP methods, Tsbagging can effectively overcome the impact of different data distributions between different projects. Specifically, we compare Tsbagging with Naive Bayesian (NB) [18], NB + WP, and adaboost [41], where adaboost and NB only use DS to train prediction models, and Tsbagging, NB + WP use DS and DWP data to train prediction models, where the amount of DWP is 10%, 15%, and 25% of the target project data, respectively.RQ2.Can Tsbagging achieve better prediction performance than other CPDP methods?In order to research this question, we compare Tsbagging with other CPDP methods and calculate theirF1-measure, MCC, g-measure, and balance values. Specifically, we compare Tsbagging with NN, DBSCAN filter [19], VCB + SVM [21], NN + WP, and DTB, in which NN, DBSCAN filter, and VCB + SVM only use DS data to train prediction models , Tsbagging, DTB [26], and NN + WP [25] use DS and DWP data to train prediction models, where the amount of DWP is also 10%, 15%, and 25% of the target project data, respectively. It is worth noting that NN, DBSCAN filter, VCB + SVM, and NN + WP do not take into account the impact of concept shift, and DTB also do not take into account the impact of concept shift when it assigns weights. If the performance of Tsbagging is better than other CPDP methods, Tsbagging can effectively overcome the impact of concept shift.RQ3.Are the experimental results of RQ1 and RQ2 statistically significant?Similar to the literature [39, 40], this paper uses the hypothesis test to statistically analyze the experimental results of RQ1 and RQ2, so as to verify whether the conclusions are statistically significant. Specifically, we use the Wilcoxon rank sum test to judge whether the experimental results of Tsbagging have significant differences compad with other methods. In this section, experiments are carried out on 42 datasets and 10 times are repeated on each dataset. Therefore, this paper will make statistical analysis on these 420 data points. The confidence level used by the Wilcoxon rank sum test is 0.05. Moreover, we use the box plot to display the distribution of those 420 points in more detail. The box plot has five numerical aspects: minimum, lower quartile, median, upper quartile, and maximum, and it is often used in SDP experiments [42–44].RQ4.How do we determine the number of clusters (i.e., nk and pk) for Tsbagging?In order to study this problem, we will measure the prediction performance of Tsbagging under different nk and pk , then we find out the reasonable value range of nk and pk. Specifically, we calculateF1-measure, MCC, g-measure, and balance of Tsbagging under nk = pk = 3, 5, 7, and 10, respectively. Tsbgging also uses 10%, 15%, and 25% of the target project data, respectively.It should be noted that in each experiment, we randomly take 10%, 15%, and 25% of the data from target data asDWP and the remaining data as DTest. We repeat 10 times experiments on each dataset and take the average value of these 10 times experiments as the final result. In addition, as suggested by Turhan et al. [18], we replace all numeric values with a “log-filter”, i.e., N with ln(N). This spreads out skewed curves more evenly across the space from the minimum to maximum values. This “spreading” can significantly improve the effectiveness of data mining, as the distribution of log-filtered feature values fits better to the normal distribution assumption [47]. In this experiment, all methods are implemented by the famous machine learning framework Weka3.9 [45] and the running environment of experiment is Java 11 and Windows 10. The parameter settings of each method are shown in Table 4.Table 4
Parameter setting of each method.
AlgorithmParameterAdaboostThe number of iterations is 20, and the basic classifier is NB.NNThe number of K-nearest neighbors is 10, and NB is used as the basic classifier.DBSCAN filterThe number of minimum records was set to 10, and the distance is set to 10VCB + SVMThe number of iterations is 5Tsbaggingϵ  = 2. The number of clustering iterations is 100, pk = 5, nk = 5, classifiersize = 20, and the base classifier is NBDTBThe number of iterations is 20, and the base classifier is NB


## 4.1. Datasets

In this experiment, we use seven projects in the public database promise [33] to build cross-project defect prediction datasets. The detailed description of these datasets is shown in Table 1. Each dataset in these datasets represents the test results of a program module, which contains 20 attributes and one label. The detailed description of these attributes is shown in Table 2 [34]. The label values are −1 and 1.1 represents defective module, and −1 represents nondefective module. Similar to reference [35–37], in each experiment, we select two different datasets from the seven datasets, and one of which is used as the target project and the other as the source project to build the CPDP experimental dataset (for example, we use poi-2.0 as the target project and synapse-1.2 as the source project); therefore, we have created 42 experimental datasets.Table 1
Details of the experiment dataset.
ProjectModuleDefect-prone moduleDefect-prone (%)Poi-2.03143712Synapse-1.22568634Ant-1.63519226Camel-1.260821636log4j-1.11093734Jedit-4.03067525Xerces-1.24407116Table 2
Attribute description.
AttributewmcWeighted methods per classditDepth of inheritance treenocNumber of childrenrfcResponse for a classlcomLack of cohesion in methodslcom3Normalized version of LCOMnpmNumber of public methodsLocLines of codedamData access metricmoaMeasure of aggregationmfaMeasure of function abstractioncamCohesion among methods of classicInheritance couplingamcAverage method complexityCaAfferent couplingsceEfferent couplingsmax_ccMaximum values of methods in the same classavg_ccMean values of methods in the same class of methods in a given classcbmCoupling between methodscboCoupling between object classes


## 4.2. Performance Measures

In this paper, we useF1-measure, MCC, g-measure, and balance [36, 38–40] to evaluate the performance of defect prediction. These performance measures are calculated from the confusion matrix shown in Table 3, where TP means true defect, FP means false defect, FN means false nondefect, and TN means true nondefect.Table 3
Obfuscation matrix.
ActualDefectNondefectPredictedDefectTPFPNondefectFNTNRecall is a measure of completeness, describing probabilities of true defective modules in comparison with the total number of defective modules:(3)recall=TPTP+FN.Precision is a measure of exactness, which defines the probabilities of the presence of modules that are truly defective from the number of modules predicted to be defective:(4)Precision=TPTP+FP.PF shows the proportion of all the modules without defects predicted to be defective:(5)PF=FPFP+TN.F1-measure is the harmonic average of recall and precision. An algorithm with a higher F1-measure value (namely, F1 value) implies a better performance as(6)F1=2∗recall∗Precisionrecall+Precision.Matthews correlation coefficient (MCC) can take into account the measurement of TP, FP, TN, and TN in a more comprehensive way. Its range of values is between [−1, 1], where 1 denotes a perfect prediction, and −1 indicates complete disagreement between actual and predicted values. The more the values are means the better the performance of the classifier will be:(7)MCC=TP∗TN−FN∗FPTP+FNTP+FPFN+TNFP+TN.G-measure [14] is the harmonic average of recall and (1–PF). An algorithm with a higher g-measure value implies a better performance as(8)g−measure=2∗Recall∗1−PFRecall+1−PF.The measure Balance is introduced by calculating the Euclidean distance from the real (recall, PF) point to (1, 0), since the point (recall = 1, PF = 0) is the optimal point where all defects are detected without missing:(9)Balance=1−0−PF2+1−Recall22.


## 4.3. Experimental Design

In this section, we design experiments to verify the performance of Tsbaging in the following four questions:RQ1. Can Tsbagging effectively overcome the impact of different data distributions between different projects, so as to obtain better prediction performance compared with traditional defect prediction methods?To research this question, we compare Tsbagging with other traditional SDP methods and calculate theirF1-measure, MCC, g-measure, and balance values. It is worth noting that the traditional SDP methods do not take into account the impact of data distribution differences between source project and target project when training prediction models. If the performance of Tsbagging is better than other traditional SDP methods, Tsbagging can effectively overcome the impact of different data distributions between different projects. Specifically, we compare Tsbagging with Naive Bayesian (NB) [18], NB + WP, and adaboost [41], where adaboost and NB only use DS to train prediction models, and Tsbagging, NB + WP use DS and DWP data to train prediction models, where the amount of DWP is 10%, 15%, and 25% of the target project data, respectively.RQ2.Can Tsbagging achieve better prediction performance than other CPDP methods?In order to research this question, we compare Tsbagging with other CPDP methods and calculate theirF1-measure, MCC, g-measure, and balance values. Specifically, we compare Tsbagging with NN, DBSCAN filter [19], VCB + SVM [21], NN + WP, and DTB, in which NN, DBSCAN filter, and VCB + SVM only use DS data to train prediction models , Tsbagging, DTB [26], and NN + WP [25] use DS and DWP data to train prediction models, where the amount of DWP is also 10%, 15%, and 25% of the target project data, respectively. It is worth noting that NN, DBSCAN filter, VCB + SVM, and NN + WP do not take into account the impact of concept shift, and DTB also do not take into account the impact of concept shift when it assigns weights. If the performance of Tsbagging is better than other CPDP methods, Tsbagging can effectively overcome the impact of concept shift.RQ3.Are the experimental results of RQ1 and RQ2 statistically significant?Similar to the literature [39, 40], this paper uses the hypothesis test to statistically analyze the experimental results of RQ1 and RQ2, so as to verify whether the conclusions are statistically significant. Specifically, we use the Wilcoxon rank sum test to judge whether the experimental results of Tsbagging have significant differences compad with other methods. In this section, experiments are carried out on 42 datasets and 10 times are repeated on each dataset. Therefore, this paper will make statistical analysis on these 420 data points. The confidence level used by the Wilcoxon rank sum test is 0.05. Moreover, we use the box plot to display the distribution of those 420 points in more detail. The box plot has five numerical aspects: minimum, lower quartile, median, upper quartile, and maximum, and it is often used in SDP experiments [42–44].RQ4.How do we determine the number of clusters (i.e., nk and pk) for Tsbagging?In order to study this problem, we will measure the prediction performance of Tsbagging under different nk and pk , then we find out the reasonable value range of nk and pk. Specifically, we calculateF1-measure, MCC, g-measure, and balance of Tsbagging under nk = pk = 3, 5, 7, and 10, respectively. Tsbgging also uses 10%, 15%, and 25% of the target project data, respectively.It should be noted that in each experiment, we randomly take 10%, 15%, and 25% of the data from target data asDWP and the remaining data as DTest. We repeat 10 times experiments on each dataset and take the average value of these 10 times experiments as the final result. In addition, as suggested by Turhan et al. [18], we replace all numeric values with a “log-filter”, i.e., N with ln(N). This spreads out skewed curves more evenly across the space from the minimum to maximum values. This “spreading” can significantly improve the effectiveness of data mining, as the distribution of log-filtered feature values fits better to the normal distribution assumption [47]. In this experiment, all methods are implemented by the famous machine learning framework Weka3.9 [45] and the running environment of experiment is Java 11 and Windows 10. The parameter settings of each method are shown in Table 4.Table 4
Parameter setting of each method.
AlgorithmParameterAdaboostThe number of iterations is 20, and the basic classifier is NB.NNThe number of K-nearest neighbors is 10, and NB is used as the basic classifier.DBSCAN filterThe number of minimum records was set to 10, and the distance is set to 10VCB + SVMThe number of iterations is 5Tsbaggingϵ  = 2. The number of clustering iterations is 100, pk = 5, nk = 5, classifiersize = 20, and the base classifier is NBDTBThe number of iterations is 20, and the base classifier is NB


## 5. Experimental Results

In this section, we answer the four research questions proposed.


### 5.1. Results forRQ1

In order to answerRQ1, we calculated the prediction performance of Tsbagging and other SDP methods on different DWP. The experimental results are shown in Tables 5–7. From the experimental results, it can be seen that NB and Adaboost only use the source project data to train the prediction model, so they are difficult to achieve good prediction performance. NB + WP uses a limit of target project data during training. Therefore, the performance of NB + WP is better than NB and Adaboost. However, NB + WP still does not take into account the impact of distribution differences between different projects, so the performance of NB + WP is still lower than that of Tsbagging. Moreover, on the 10% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 18.5%, 52.7%, 24.5%, and 13.5%, respectively. On the 15% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 21.6%, 59.6%, 25.6%, and 14.1%, respectively. On the 25% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 24%, 71.4%, 26.7%, and 15.2%, respectively. In summary, the Tsbagging can use the TScluster method to find out the data which obey the distribution of the target project, so as to effectively overcome the impact caused by the difference of data distribution between projects and achieve better prediction performance.Table 5
Tsbagging compared with other SDP methods on 10% target data.
PerformanceNBAdaboostNB + WPTsbaggingF10.373310.373670.388910.4427MCC0.135510.141120.149560.20694g-measure0.439270.445490.450310.54691Balance0.501410.507040.508970.56982Table 6
Tsbagging compare with other SDP methods on 15% target data.
PerformanceNBAdaboostNB + WPTsbaggingF10.374370.374360.394130.45525MCC0.13790.143130.154460.22013g-measure0.440250.446110.452970.55298Balance0.502160.507610.511330.57317Table 7
Tsbagging compare with other SDP methods on 25% target data.
PerformanceNBAdaboostNB + WPTsbaggingF10.372250.372490.403130.46171MCC0.132950.138450.162390.22791g-measure0.43780.443670.460550.55494Balance0.500560.505850.516590.57654


### 5.2. Results forRQ2

In order to answerRQ2, we compare Tsbagging with other CPDP methods and calculate their F1, MCC, g-measure, and balance on different datasets. The experimental results are shown in Tables 8–10. The experimental results show that due to NN, DBSCAN filter, and VCB + SVM only use DS to train the prediction model, when the distribution of source project and target project is quite different, especially it occurs concept shift, NN, DBSCAN filter, and VCB + SVM are difficult to achieve good prediction performance, even DBSCAN filter and VCB + SVM have negative transfer [46](that is, the CPDP method’s prediction performance is lower than the traditional SDP methods). DTB and NN + WP use a limit of target project data to training the prediction model, so their performance are better than NN, DBSCAN filter, and VCB + SVM. However, NN + WP and DTB also do not consider the impact of concept shift when selecting DS data and assigning weights, so their F1, MCC, g-measure, and balance are lower than Tsbagging. Moreover, on the 10% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 23.2%, 60.6%, 26.4%, and 18.6%, respectively. On the 15% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 26.6%, 74.1%, 36.1%, and 19%, respectively. On the 25% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 28.6%, 84.4%, 38%, and 20.3%, respectively. In summary, Tsbagging can overcome the impact of concept shift using semisupervised clustering to achieve better performance than other CPDP methods.Table 8
Tsbagging compared with other CPDP methods on 10% target data.
PerformanceNNDBSCAN filterVCB + SVMNN + WPDTBTsbaggingF10.38710.368470.359150.401690.408490.4427MCC0.151570.131120.128870.162830.163520.20694g-measure0.4680.432370.402610.474390.500110.54691Balance0.521070.498510.480520.525640.53510.56982Table 9
Tsbagging compared with other CPDP methods on 15% target data.
PerformanceNNDBSCAN filterVCB + SVMNN + WPDTBTsbaggingF10.390690.368850.359660.410240.410260.45525MCC0.155240.132740.12640.173050.163930.22013g-measure0.471410.43260.406310.481990.492430.55298Balance0.522930.498860.481590.529790.529150.57317Table 10
Tsbagging compared with other CPDP methods on 25% target data.
PerformanceNNDBSCAN filterVCB + SVMNN + WPDTBTsbaggingF10.389720.367280.358780.42010.425110.46171MCC0.153340.128280.123550.182240.178220.22791g-measure0.474160.431040.401770.491390.504640.55494Balance0.524380.497750.479320.536690.537510.57654


### 5.3. Results forRQ3

In order to verify whether the experimental results ofRQ1 and RQ2 are statistically significant, this paper uses the Wilcoxon rank sum test and box plot to statistically analyze all experimental results. Tables 11−13 show the p values on different datasets. The experimental results show that the p values of the four performance measures on all datasets are lower than 0.05 compared with other defect prediction methods. Therefore, it shows that the prediction results of Tsbagging are statistically significantly different from those of other methods. On the other hand, the box plot of performance measures is shown on each dataset. From Figures 3–14, we can see that the median and lower quartile of MCC, g-measure, and balance are higher than all of other methods, and the median and lower quartile of F1 are higher than most of the other methods (the median values of DTB and Tsbagging are roughly the same level, but the lower quartile of Tsbagging is significantly higher than that of DTB). Therefore, it can be seen that the prediction performance of Tsbagging is better than other defect prediction methods. In conclusion, the experimental results of RQ1 and RQ2 are statistically significant.Table 11
Wilcoxon rank sum testp value on 10% target project data.
p valueNBAdaboostNB + WPNNDBSCAN filterVCB + SVMNN + WPDTBF1-measure2.74e–092.35e–091.19e–056.93e–065.50e–093.74e–170.00150.0078MCC3.30e–134.18e–121.18e–081.16e–071.91e–137.76e–151.51e–058.36e–05G-measure5.04e–177.48e–141.91e-148.65e–091.33e–172.24e–302.01e–083.96e–07Balance7.67e–182.69e–141.73e-142.69e–092.15e–183.15e–312.10e–081.72e–07Table 12
Wilcoxon rank sum testp value on 15% target project data.
p valueNBAdaboostNB + WPNNDBSCAN filterVCB + SVMNN + WPDTBF1-measure1.23e–111.06e–118.60e–071.21e–072.07e–119.09e–225.97e–048.08e–04MCC1.84e–163.70e–151.69e–109.13e–101.29e–164.35e–201.09e – 42.48e–06G-measure9.07e–181.60e–143.55e–153.78e–092.86e–183.01e–313.99e–086.56e–10Balance6.41e–192.46e–152.01e–156.11e–101.76e–191.86e–322.95e–081.30e–10Table 13
Wilcoxon rank sum testp value on 25% target project data.
p valueNBAdaboostNB + WPNNDBSCAN filterVCB + SVMNN + WPDTBF1-measure3.18e–152.30e–151.16e–061.83e–107.21e–151.25e–246.58e–040.0054MCC1.01e–211.94e–203.69e–115.8e–131.48e-212.02e–261.6e–053.46e–05G-measure1.98e–192.68e–162.62e–142.36e–098.94e–203.52e–338.41e–077.94e–09Balance1.60e–218.85e–181.55e–141.01e–107.53e–226.28e–366.31e–077.60e–10Figure 3
Box plot ofF1-measure on 10% target project data.Figure 4
Box plot of MCC on 10% target project data.Figure 5
Box plot ofg-measure on 10% target project data.Figure 6
Box plot of balance on 10% target project data.Figure 7
Box plot ofF1-measure on 15% target project data.Figure 8
Box plot of MCC on 15% target project data.Figure 9
Box plot ofg-measure on 15% target project data.Figure 10
Box plot of balance on 15% target project data.Figure 11
Box plot ofF1-measure on 25% target project data.Figure 12
Box plot of MCC on 25% target project data.Figure 13
Box plot ofg-measure on 25% target project data.Figure 14
Box plot of balance on 25% target project data.


### 5.4. Results forRQ4

In order to answerRQ4, we calculate F1, MCC, g-measure, and balance values of Tsbagging under different nk and pk, where the value range of nk and pk is 3, 5, 7, and 10. The experimental results are shown in Table 14 and Figure 15–17. The Y-axis and X-axis of Figures 15–17, represent a performance index and its corresponding value, respectively. From the experimental results, it can be seen that the prediction performance of Tsbagging does not change significantly in different nk and pk, and they are better than other methods. Therefore, it is reasonable to take 3, 5, 7, and 10 for nk and pk at the same time for Tsbagging.Table 14
The performance of Tsbagging on different nk and pk.
Performancenk = pk = 3nk = pk = 5nk = pk = 7nk = pk = 1010% target dataF10.440650.442700.440660.43836MCC0.201880.206940.202840.20313g-measure0.544670.546910.542500.54014Balance0.567110.569820.566040.5650715% target dataF10.454180.455250.454540.45598MCC0.219720.220130.221310.22120g-measure0.554010.552980.553350.55630Balance0.573710.573170.574550.5766125% target dataF10.465210.461710.464410.46497MCC0.232020.227910.232020.23190g-measure0.558630.554940.557950.55874Balance0.578580.576540.578420.57867Figure 15
Tsbagging’s performance under different nk and pk target project data on 10% target project data.Figure 16
Tsbagging’s performance under different nk and pk target project data on 15% target project data.Figure 17
Tsbagging’s performance under different nk and pk target project data on 25% target project data.


## 5.1. Results forRQ1

In order to answerRQ1, we calculated the prediction performance of Tsbagging and other SDP methods on different DWP. The experimental results are shown in Tables 5–7. From the experimental results, it can be seen that NB and Adaboost only use the source project data to train the prediction model, so they are difficult to achieve good prediction performance. NB + WP uses a limit of target project data during training. Therefore, the performance of NB + WP is better than NB and Adaboost. However, NB + WP still does not take into account the impact of distribution differences between different projects, so the performance of NB + WP is still lower than that of Tsbagging. Moreover, on the 10% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 18.5%, 52.7%, 24.5%, and 13.5%, respectively. On the 15% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 21.6%, 59.6%, 25.6%, and 14.1%, respectively. On the 25% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 24%, 71.4%, 26.7%, and 15.2%, respectively. In summary, the Tsbagging can use the TScluster method to find out the data which obey the distribution of the target project, so as to effectively overcome the impact caused by the difference of data distribution between projects and achieve better prediction performance.Table 5
Tsbagging compared with other SDP methods on 10% target data.
PerformanceNBAdaboostNB + WPTsbaggingF10.373310.373670.388910.4427MCC0.135510.141120.149560.20694g-measure0.439270.445490.450310.54691Balance0.501410.507040.508970.56982Table 6
Tsbagging compare with other SDP methods on 15% target data.
PerformanceNBAdaboostNB + WPTsbaggingF10.374370.374360.394130.45525MCC0.13790.143130.154460.22013g-measure0.440250.446110.452970.55298Balance0.502160.507610.511330.57317Table 7
Tsbagging compare with other SDP methods on 25% target data.
PerformanceNBAdaboostNB + WPTsbaggingF10.372250.372490.403130.46171MCC0.132950.138450.162390.22791g-measure0.43780.443670.460550.55494Balance0.500560.505850.516590.57654


## 5.2. Results forRQ2

In order to answerRQ2, we compare Tsbagging with other CPDP methods and calculate their F1, MCC, g-measure, and balance on different datasets. The experimental results are shown in Tables 8–10. The experimental results show that due to NN, DBSCAN filter, and VCB + SVM only use DS to train the prediction model, when the distribution of source project and target project is quite different, especially it occurs concept shift, NN, DBSCAN filter, and VCB + SVM are difficult to achieve good prediction performance, even DBSCAN filter and VCB + SVM have negative transfer [46](that is, the CPDP method’s prediction performance is lower than the traditional SDP methods). DTB and NN + WP use a limit of target project data to training the prediction model, so their performance are better than NN, DBSCAN filter, and VCB + SVM. However, NN + WP and DTB also do not consider the impact of concept shift when selecting DS data and assigning weights, so their F1, MCC, g-measure, and balance are lower than Tsbagging. Moreover, on the 10% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 23.2%, 60.6%, 26.4%, and 18.6%, respectively. On the 15% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 26.6%, 74.1%, 36.1%, and 19%, respectively. On the 25% target project dataset, the largest increases of F1, MCC, g-measure, and balance are 28.6%, 84.4%, 38%, and 20.3%, respectively. In summary, Tsbagging can overcome the impact of concept shift using semisupervised clustering to achieve better performance than other CPDP methods.Table 8
Tsbagging compared with other CPDP methods on 10% target data.
PerformanceNNDBSCAN filterVCB + SVMNN + WPDTBTsbaggingF10.38710.368470.359150.401690.408490.4427MCC0.151570.131120.128870.162830.163520.20694g-measure0.4680.432370.402610.474390.500110.54691Balance0.521070.498510.480520.525640.53510.56982Table 9
Tsbagging compared with other CPDP methods on 15% target data.
PerformanceNNDBSCAN filterVCB + SVMNN + WPDTBTsbaggingF10.390690.368850.359660.410240.410260.45525MCC0.155240.132740.12640.173050.163930.22013g-measure0.471410.43260.406310.481990.492430.55298Balance0.522930.498860.481590.529790.529150.57317Table 10
Tsbagging compared with other CPDP methods on 25% target data.
PerformanceNNDBSCAN filterVCB + SVMNN + WPDTBTsbaggingF10.389720.367280.358780.42010.425110.46171MCC0.153340.128280.123550.182240.178220.22791g-measure0.474160.431040.401770.491390.504640.55494Balance0.524380.497750.479320.536690.537510.57654


## 5.3. Results forRQ3

In order to verify whether the experimental results ofRQ1 and RQ2 are statistically significant, this paper uses the Wilcoxon rank sum test and box plot to statistically analyze all experimental results. Tables 11−13 show the p values on different datasets. The experimental results show that the p values of the four performance measures on all datasets are lower than 0.05 compared with other defect prediction methods. Therefore, it shows that the prediction results of Tsbagging are statistically significantly different from those of other methods. On the other hand, the box plot of performance measures is shown on each dataset. From Figures 3–14, we can see that the median and lower quartile of MCC, g-measure, and balance are higher than all of other methods, and the median and lower quartile of F1 are higher than most of the other methods (the median values of DTB and Tsbagging are roughly the same level, but the lower quartile of Tsbagging is significantly higher than that of DTB). Therefore, it can be seen that the prediction performance of Tsbagging is better than other defect prediction methods. In conclusion, the experimental results of RQ1 and RQ2 are statistically significant.Table 11
Wilcoxon rank sum testp value on 10% target project data.
p valueNBAdaboostNB + WPNNDBSCAN filterVCB + SVMNN + WPDTBF1-measure2.74e–092.35e–091.19e–056.93e–065.50e–093.74e–170.00150.0078MCC3.30e–134.18e–121.18e–081.16e–071.91e–137.76e–151.51e–058.36e–05G-measure5.04e–177.48e–141.91e-148.65e–091.33e–172.24e–302.01e–083.96e–07Balance7.67e–182.69e–141.73e-142.69e–092.15e–183.15e–312.10e–081.72e–07Table 12
Wilcoxon rank sum testp value on 15% target project data.
p valueNBAdaboostNB + WPNNDBSCAN filterVCB + SVMNN + WPDTBF1-measure1.23e–111.06e–118.60e–071.21e–072.07e–119.09e–225.97e–048.08e–04MCC1.84e–163.70e–151.69e–109.13e–101.29e–164.35e–201.09e – 42.48e–06G-measure9.07e–181.60e–143.55e–153.78e–092.86e–183.01e–313.99e–086.56e–10Balance6.41e–192.46e–152.01e–156.11e–101.76e–191.86e–322.95e–081.30e–10Table 13
Wilcoxon rank sum testp value on 25% target project data.
p valueNBAdaboostNB + WPNNDBSCAN filterVCB + SVMNN + WPDTBF1-measure3.18e–152.30e–151.16e–061.83e–107.21e–151.25e–246.58e–040.0054MCC1.01e–211.94e–203.69e–115.8e–131.48e-212.02e–261.6e–053.46e–05G-measure1.98e–192.68e–162.62e–142.36e–098.94e–203.52e–338.41e–077.94e–09Balance1.60e–218.85e–181.55e–141.01e–107.53e–226.28e–366.31e–077.60e–10Figure 3
Box plot ofF1-measure on 10% target project data.Figure 4
Box plot of MCC on 10% target project data.Figure 5
Box plot ofg-measure on 10% target project data.Figure 6
Box plot of balance on 10% target project data.Figure 7
Box plot ofF1-measure on 15% target project data.Figure 8
Box plot of MCC on 15% target project data.Figure 9
Box plot ofg-measure on 15% target project data.Figure 10
Box plot of balance on 15% target project data.Figure 11
Box plot ofF1-measure on 25% target project data.Figure 12
Box plot of MCC on 25% target project data.Figure 13
Box plot ofg-measure on 25% target project data.Figure 14
Box plot of balance on 25% target project data.


## 5.4. Results forRQ4

In order to answerRQ4, we calculate F1, MCC, g-measure, and balance values of Tsbagging under different nk and pk, where the value range of nk and pk is 3, 5, 7, and 10. The experimental results are shown in Table 14 and Figure 15–17. The Y-axis and X-axis of Figures 15–17, represent a performance index and its corresponding value, respectively. From the experimental results, it can be seen that the prediction performance of Tsbagging does not change significantly in different nk and pk, and they are better than other methods. Therefore, it is reasonable to take 3, 5, 7, and 10 for nk and pk at the same time for Tsbagging.Table 14
The performance of Tsbagging on different nk and pk.
Performancenk = pk = 3nk = pk = 5nk = pk = 7nk = pk = 1010% target dataF10.440650.442700.440660.43836MCC0.201880.206940.202840.20313g-measure0.544670.546910.542500.54014Balance0.567110.569820.566040.5650715% target dataF10.454180.455250.454540.45598MCC0.219720.220130.221310.22120g-measure0.554010.552980.553350.55630Balance0.573710.573170.574550.5766125% target dataF10.465210.461710.464410.46497MCC0.232020.227910.232020.23190g-measure0.558630.554940.557950.55874Balance0.578580.576540.578420.57867Figure 15
Tsbagging’s performance under different nk and pk target project data on 10% target project data.Figure 16
Tsbagging’s performance under different nk and pk target project data on 15% target project data.Figure 17
Tsbagging’s performance under different nk and pk target project data on 25% target project data.


## 6. Validity Threats

### 6.1. Internal Validity

We compared our method with other SDP methods.To avoid the potential faults as much as possible during the implementation process of the experiment, we implement these models based on the Java machine learning library Weka.


### 6.2. External Validity

External validity refers to the degree to generalize the research results to other situations. The most commonly used promise dataset in the cross-project defect prediction research is selected as the experimental data to ensure that the experimental results have certain representative significance. At the same time, the prediction performance is evaluated in terms of four well-known performance measures (i.e.,F1-measure, g-measure, balance, and MCC) to ensure the generality of the experiment.


### 6.3. Statistical Validity

In this paper, we use the Wilcoxon rank sum test to statistically analyze the experimental results. The Wilcoxon rank sum test is a nonparametric test. It has no requirements for the distribution of dataset, and it is more convenient to test whether there are differences between the two groups of distribution. The results of the Wilcoxon rank sum test show that thep value on all datasets is less than 0.05. Therefore, it can be said that our experiment is statistically significant. In addition, we use the box plot to analyze the experimental results. The box plot can show the overall distribution of the prediction results, and the results also show that Tsbagging achieves better defect prediction performance than others.


## 6.1. Internal Validity

We compared our method with other SDP methods.To avoid the potential faults as much as possible during the implementation process of the experiment, we implement these models based on the Java machine learning library Weka.


## 6.2. External Validity

External validity refers to the degree to generalize the research results to other situations. The most commonly used promise dataset in the cross-project defect prediction research is selected as the experimental data to ensure that the experimental results have certain representative significance. At the same time, the prediction performance is evaluated in terms of four well-known performance measures (i.e.,F1-measure, g-measure, balance, and MCC) to ensure the generality of the experiment.


## 6.3. Statistical Validity

In this paper, we use the Wilcoxon rank sum test to statistically analyze the experimental results. The Wilcoxon rank sum test is a nonparametric test. It has no requirements for the distribution of dataset, and it is more convenient to test whether there are differences between the two groups of distribution. The results of the Wilcoxon rank sum test show that thep value on all datasets is less than 0.05. Therefore, it can be said that our experiment is statistically significant. In addition, we use the box plot to analyze the experimental results. The box plot can show the overall distribution of the prediction results, and the results also show that Tsbagging achieves better defect prediction performance than others.


## 7. Conclusion and Future Work

This paper proposes a CPDP method Tsbagging based on semisupervised clustering. Tsbagging combines the semisupervised clustering method with the transfer learning method to overcome shortcomings of traditional SDP methods in CPDP. The experimental results (the detailed description of each experimental result is shown in Tables15–26) show that it has some advantages in performance compared with other defect prediction methods of Tsbagging. In the future work, we will try to integrate the information in multiple source projects for knowledge transfer based on the multisource transfer learning framework, so as to help the classifier achieve better performance in CPDP.



---

*Source: 6339684-2022-09-28.xml*