## 5 Results and Analysis
### Feature Weightage Calculation and Feature Subset Acquisition  
As mentioned earlier, both ensemble-1 and ensemble-2 approaches require a robust feature scoring function. In this regard, three decision tree-based classifiers are used as the scoring functions to obtain a consolidated feature score. In the following section, we elaborate on the feature scores obtained from different functions along with the feature subsets selected. Please note that ensemble-1 combines individual weightage obtained from multiple scoring functions and afterward selects a threshold, whereas ensemble-2 is based on an eager approach in which threshold is applied to individual scoring functions and a set of feature subsets are obtained which are afterward combined into a consolidated feature set. * Decision Tree Score: Features are scored through the CART decision tree classifier. The blue line in Figure 6 shows the feature weightage (FW) in the decreasing order of their importance while the orange line denotes accumulated cost (Cscore). Both values are normalized. The point of intersection between FW and Cscore is found around feature number 5 as shown in Figure 6.
* Random Forest Score: The second scoring function is based on random forest. The blue line in Figure 7 shows the feature weightage (FW) while the orange line denotes accumulated cost (Cscore). Both the values are normalized. The point of intersection between FW and Cscore can be observed around feature number 9 as shown in Figure 7.
* Gradient Boosted Trees Score: The last scoring function is based on GBT. The blue line in Figure 8 shows the feature weightage (FW) while the orange line denotes accumulated cost (Cscore). Both the values are normalized. The point of intersection between FW and Cscore is around feature number 3 as shown in Figure 8.  
In this study, all the feature scoring functions are based on decision tree family models. Therefore, it would be interesting to investigate the correlation of generated ranking lists. We used the Kendall rank correlation coefficient [49] to compute the pair-wise correlation of lists produced by DT, RF, GBT as shown in Table 7. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Models** & **Accuracy** & **Precision** & **Recall** & **Specificity** & **F-Measure** & **AUC** & **Cost** \\ \hline NB & \(62.3\pm 2.0\%\) & \(62.3\pm 2.0\%\) & \(100.0\pm 0.0\%\) & \(0.0\pm 0.0\%\) & \(76.6\pm 1.5\%\) & \(0.908\pm 0.09\) & \\ LG & \(84.3\pm 6.6\%\) & \(83.2\pm 5.8\%\) & \(94.3\pm 9.3\%\) & \(66.9\pm 14.0\%\) & \(88.2\pm 5.6\%\) & \(0.952\pm 0.58\) & \\ DL & \(89.5\pm 2.3\%\) & \(85.4\pm 2.8\%\) & \(100.0\pm 0.0\%\) & \(72.8\pm 5.7\%\) & \(92.1\pm 1.6\%\) & \(1.0\pm 0.0\) & \\ DT & \(87.7\pm 4.8\%\) & \(97.3\pm 3.7\%\) & \(90.4\pm 10.5\%\) & \(95.6\pm 6.1\%\) & \(93.3\pm 4.5\%\) & \(0.966\pm 0.03\) & 475.36 \\ RF & \(89.5\pm 3.5\%\) & \(83.9\pm 3.6\%\) & \(100.0\pm 0.0\%\) & \(66.4\pm 11\%\) & \(91.2\pm 2.2\%\) & \(0.998\pm 0.004\) & \\ GBT & \(73.8\pm 5.8\%\) & \(86.2\pm 6.9\%\) & \(100.0\pm 0.0\%\) & \(71.4\pm 15.6\%\) & \(92.5\pm 4.0\%\) & \(1.0\pm 0.0\) & \\ SVM & \(92.2\pm 9.0\%\) & \(71.8\pm 8.1\%\) & \(97.2\pm 3.8\%\) & \(35.8\pm 20.7\%\) & \(82.4\pm 5.6\%\) & \(0.844\pm 0.12\) & \\ Average & \(82.75\pm 4.8\%\) & \(81.44\pm 4.7\%\) & \(97.41\pm 3.3\%\) & \(58.41\pm 10.4\%\) & \(88.07\pm 3.5\%\) & \(0.952\pm 0.04\) & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Evaluation results for baseline models. \begin{table}
\begin{tabular}{c c c} \hline \hline
**DT and RF** & **RF and GBT** & **GBT and DT** \\ \hline \(-0.17\) & \(-0.09\) & \(0.22\) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Kendall rank correlation coefficient for different scoring functions. Figure 8: Gradient boosted trees based feature scoring. Figure 6: Decision tree-based feature scoring. Figure 7: Random forest-based feature scoring. As it can be observed that the correlation between different lists is closer to zero. The correlation results of the ranked lists support the null hypothesis of mutual independence.