# Transfer Convolutional Neural Network for Cross-Project Defect Prediction
## 4 Experimental Setup
### Compared Models  
To assess the performance of the TCNN model, we explored the following two research questions:  
**RQ1:** Can TCNN perform better than the classic CPDP methods that use only traditional handcrafted features? **RQ2:** How effective is the TCNN approach compared to state-of-the-art SDP methods adopting DL-learning features? Thus, we selected two categories of compared methods in this paper, including the models with only handcrafted or DL-generated features. In summary, we compared our TCNN with LR, NNFilter [13], TNB [14], TCA [20], TCA+ [10], DBN [9], DPDBN [8], CNN [8], and DPCNN [8]. In our experiments, we adopted LR as the base classifier for all models because LR has been widely used in the previous defect prediction studies [8; 10; 21]. We used the implementation of _LogisticRegression_ in _sklearn.linear_model_, and we adopted default parameters settings by _sklearn_. With regard to implementing DL-generated CPDP models, we followed the same code-parsing process \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Truly Positive** & **Truly Negative** \\ \hline Predictive Positive & TP & FP \\ \hline Predictive Negative & FN & TN \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Confusion matrix. to generate integer vectors for neural networks. Considering that the process of random oversampling involves randomness, we conducted each method 20 times and recorded the average result. The traditional CPDP models using only the 20 handcrafted features provided by PROMISE include the following methods:  
* **LR:** Traditional SDP method, which builds an LR classifier only using handcrafted features. * **NNFilter:** This method gathers similar instances together to construct a training set that is homogeneous with the target dataset [13]. * **DG:** The predict model is built based on the weighted training data whose weight is calculated by analogy with data gravitation [19]. * **TCA:** A classic transferable feature learning method [20]. We used the source code provided by its author. * **TCA+. ** A variant of TCA [10], which extends TCA with customized normalization rules. The state-of-the-art SDP models using DL-generated features include the following methods:  
* **DBN:** A standard DBN model to extract semantic features for SDP. Regarding the implementation of DBN, we adopted the same network architectures and parameters (i.e., 10 hidden layers and 100 nodes in each hidden layer) as in [9]. * **CNN:** An SDP method that extracts DL-generated features through standard CNN. When implementing CNN, referring to [8], we set the batch size as 32, the epoch number as 15, the embedding dimension as 30, the number of hidden nodes as 100, the number of filters as 10, and the filter length as 5. * **DPDBN:** A variant of DBN, which concatenates the DBN-learned features with the handcrafted features. * **DPCNN:** An SDP method that is an improved version of the CNN proposed by [8]. To clearly show the performance differences between methods, we adopted the Scott-Knott test [31] to compare the performance of different CPDP approaches. The Scott-Knott test divides the measurement means into statistically distinct groups by hierarchical clustering analysis. The two main limitations of the traditional Scott-Knott test are: (1) it assumes the data are in a normal distribution; (2) it may create groups that are trivially different from each other. To avoid the limitations of the Scott-Knott test, in this paper, we adopted its normality and effect size-aware variant, the Scott-Knott effect size difference (ESD) test [32; 33]. The Scott-Knott ESD test would (1) correct the normal distribution of the input dataset and (2) merge any two statistically different groups of negligible effects. A detailed description of the Scott-Knott ESD test can be found in [32].