# A Deep Ensemble Learning Method for Effort-Aware Just-In-Time Defect Prediction
## 4 Experimental Evaluation
### Discussion  
In practicality, there is no optimal approach that can meet all potential scenarios. Therefore, the unsupervised learning is a great way of learning when the labelled data is not applied, however the effort-aware JIT seems to be outperformed by the state-of-the-art supervised model. The model used by [PERSON] et al. [1] doesn't hold true for all the attributes as LA and LD had to be not taken into consideration. Our approach tries to take into account of the abstract features by using the deep ensemble technique and XGBoost and Random Forest, making it more robust and thus outperforms all the models available.  
[PERSON] [1] greatly validates the importance of having the unsupervised model, as well as their methodology of computing the reciprocal of the raw metric and excluding the LA and LD and then  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Model** & **Acc** & **P-score** & **F1-Score** & **Sensitivity** & **Specificity** & **Acc2** & **P-score2** & **F1-score2** & **Sensitivity2** & **Specificity2** \\ \hline LR & 0.6550 & 0.6714 & 0.6714 & 0.6368 & 0.6714 & 0.6303 & 0.4237 & 0.4911 & 0.7805 & 0.4237 \\
**SVM** & 0.6550 & 0.6551 & 0.6877 & 0.6547 & 0.6551 & 0.5833 & 0.3897 & 0.4768 & 0.7727 & 0.3897 \\
**linear** & **SVM** & 0.4725 & 0.4000 & 0.0186 & 0.4734 & 0.4000 & 0.6942 & 0.4000 & 0.0029 & 0.6946 & 0.4000 \\
**RBF** & 0.5775 & 0.5597 & 0.6943 & 0.6842 & 0.5597 & 0.6828 & 0.4726 & 0.3885 & 0.7398 & 0.4726 \\
**J48** & 0.6600 & 0.6813 & 0.6714 & 0.6377 & 0.6813 & 0.6823 & 0.4571 & 0.2893 & 0.7194 & 0.4571 \\
**RF** & 0.7100 & 0.7701 & 0.6979 & 0.6637 & 0.7701 & 0.6754 & 0.4519 & 0.3561 & 0.7308 & 0.4519 \\
**AdBoost** & 0.6900 & 0.7171 & 0.6633 & 0.6960 & 0.6633 & 0.6808 & 0.4760 & 0.4605 & 0.7628 & 0.4760 \\
**XGBoost** & 0.7000 & 0.7293 & 0.6984 & 0.6650 & 0.7293 & 0.6823 & 0.4571 & 0.2893 & 0.7194 & 0.4571 \\
**DNN** & 0.6275 & 0.6177 & 0.6823 & 0.6453 & 0.6177 & 0.5322 & 0.3289 & 0.4001 & 0.7156 & 0.3289 \\
**K-Means** & 0.4775 & 1.000 & 0.0094 & 0.4761 & 1.0 & 0.6962 & 0.5178 & 0.1463 & 0.7057 & 0.5178 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance of different supervised, unsupervised and deep learning methods.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Accuracy** & **P Score** & **F1 Score** & **Sensitivity** & **Specificity** \\ \hline Sample Dataset & 0.7739 & 0.7842 & 0.7546 & 0.7798 & 0.7842 \\ Different Dataset & 0.8185 & 0.7993 & 0.5860 & 0.8162 & 0.7993 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Performance on sample and different dataset.  
removing the highly correlated data, as this helped in ranking the values obtained in descending order [31]. However, whilst observing the correlation with the target value and availability of sufficient data, it became obvious to opt for the supervised model. The supervised model XGBoost outperformed all the models in all aspects with a highly sensitive score.  
Whilst using the model for cross validation cross project, we observed the same trend for the classification model performance. The accuracy and all other metrics performed well but XGBoost performed across all the models in both local and global scenarios.