# Dissimilarity Space Based Multi-Source Cross-Project Defect Prediction
## 4 Experiments
### Analysis of Results
#### 4.3.2 Experiments on Multi-Source Data Sets and Dissimilarity Space  
In order to verify the impact of multi-source data and sample dissimilarity representation on the performance of predictive models, we compare the TrAdaBoost, Multi-source TrAdaBoost, dissimilarity space based single-source CPDP (DS-CPDP) method, and DM-CPDP method. All the comparisons base on F-measure, AUC, and cost-effectiveness to complete. Each experiment was repeated 10 times, and the average values of these experiments are obtained from the results. The experimental details of each method are as follows:  
TrAdaBoost: randomly selects 90% of the source domain data set and target set as the training set, and the remaining target set as the test set. The prediction model was built using TrAdaBoost. Multi-source TrAdaBoost: before using the TrAdaBoost algorithm to build a prediction model, the EMD method is used to select multiple data sets that are highly correlated with the target set, then these data sets are combined with the target set as a training set. Finally, the TrAdaBoost algorithm is used for modeling. DS-CPDP: randomly selects a data set from the source domain for dissimilarity transformation as the training set, and then uses KNN method to assign labels to unlabeled target, after that use the TrAdaBoost method to build the model. The experimental results are shown in Tables 7-9, bold numbers indicate optimal results on a dataset. In order to prove the importance of multi-source data, we verify it in the dissimilarity space and the feature space respectively. By comparing the results of DM-CPDP and DS-CPDP method, it can be found that DM-CPDP is superior to DS-CPDP in the three indexes of F-measure, AUC, and cost-effectiveness in the dissimilarity space. The average value of F-measure is higher than the DS-CPDP method by 6.4% and 3.2% in the two series of datasets respectively. The average value of AUC is higher than the DS-CPDP method by 7.0% and 5.4%. The average value of cost-effectiveness is lower than the DS-CPDP method by 1.1% and 0.4%. By comparing the results of TrAdaBoost and Multi-source TrAdaBoost, it can be found that the Multi-source TrAdaBoost method is better than the TrAdaBoost in the feature space, and the average value of F-measure is higher than the TrAdaBoost method by 2.1%, 8.9%. The average value of AUC is higher than the TrAdaBoost method by 2.3%, 7.6%. The average value of cost-effectiveness is lower than the TrAdaBoost method by 3.5%, 0.8%. These results prove that the CPDP method based on multi-source data can improve the performance of prediction models, whether in the dissimilarity space or in the feature space. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Target Set** & **TrdaBoost** & \begin{tabular}{c} **Multi-Source** \\ **TrdaBoost** \\ \end{tabular} & \
\begin{tabular}{c} **DS-CPDP** \\ \end{tabular} &
\begin{tabular}{c} **DM-CPDP** \\ \end{tabular} \\ \hline \multicolumn{5}{c}{NASA to NASA} \\ \hline CM1 & 0.419 & 0.525 & 0.564 & **0.594** \\ PC1 & 0.364 & 0.412 & 0.538 & **0.635** \\ PC2 & 0.259 & 0.160 & 0.295 & **0.368** \\ PC3 & 0.578 & 0.601 & 0.698 & **0.776** \\ PC4 & 0.239 & 0.237 & 0.316 & **0.468** \\ PC5 & 0.485 & 0.493 & 0.601 & **0.647** \\ KC1 & 0.534 & 0.556 & 0.788 & **0.832** \\ KC2 & 0.471 & 0.514 & 0.652 & **0.684** \\ KC3 & 0.268 & 0.315 & 0.400 & **0.450** \\ KC4 & 0.427 & 0.462 & 0.454 & **0.502** \\ MC1 & 0.620 & 0.629 & 0.687 & **0.755** \\ MC2 & 0.395 & 0.408 & 0.521 & **0.597** \\ MW1 & 0.169 & 0.201 & 0.385 & **0.393** \\ JM1 & 0.535 & 0.549 & 0.718 & **0.811** \\ \hline average & 0.412 & 0.433 & 0.544 & **0.608** \\ \hline \multicolumn{5}{c}{NASA to SOFTLAB} \\ \hline ar3 & 0.586 & 0.705 & 0.739 & **0.781** \\ ar4 & 0.577 & 0.691 & 0.724 & **0.753** \\ ar5 & 0.795 & 0.829 & 0.837 & **0.863** \\ \hline average & 0.653 & 0.742 & 0.767 & **0.799** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: F-measure value of different model construction methods. DS-CPDP: dissimilarity space based single-source CPDP.