## 4 Results  
This section presents the results of the experiments supporting the research methodology which is described in Section 3.2. The intensive processing of the dataset asserts the high level of challenge in finding an accurate prediction model. Adding cost sensitivity to the classification algorithm increased dramatically the prediction accuracy of the positive class, i.e., the target class. At the same time, cost sensitivity led to relatively slight drop in the accuracy of the negative class. The effects of different cost factors on the classification performance are illustrated by _Geometric Mean_ in Figure 3, and Type I and Type II Errors in Figure 4.  
Taking into consideration the target class, i.e., bank clients who subscribed a term deposit. It is apparent that there is a positive correlation between the cost factor and the classification accuracy at _FN-Cost Factor_ in the range 1 to 200; nonetheless, an increase of _FN-Cost Factor_ over 200 degrades the performance of the prediction model. Interestingly, the classifiers do not attain any significant \begin{table}
\begin{tabular}{l l} \hline \hline
**Metric** & **Equation** \\ \hline Total Accuracy (_All Correctly Classified Instances_) & \(\frac{TP+TN}{TP+TN+FP+FN}\) \\ True Positive Rate (TPR), _alternatively Recall_ & \(\frac{TP}{TP+FN}\) \\ True Negative Rate (TNR), _alternatively Precision_ & \(\frac{TN}{TN+FP}\) \\ Type I Error [45] & \(1-Recall\_Yes=\frac{FN}{TP+FN}=\) _false negative rate_ as \(\alpha\) \\ Type II Error [45] & \(1-Recall\_No=\frac{FP}{TN+FP}=\) _false positive rate_ as \(\beta\) \\ Geometric Mean & \(\sqrt{TPR\times TNR}\) \\ Lift [46] & \(Lift=\frac{TPR}{Sample\_Size}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Evaluation metrics that are used in this research. Figure 3: The effect of Adjusting FN-Cost Factor on Geometric Mean. improvement in terms of geometric mean metric after the _FN-Cost Factor_ of 10. However, it is clear that Meta-Cost algorithm outperforms Cost-Sensitive Classifier at higher values of _FN-Cost Factor_. Error rates (i.e., Type I and Type II errors) reveal an interesting behavior of the cost-sensitive schemes. Meta-Cost scheme caused dramatic decrease in Type I error, while Cost-Sensitive Classifier resulted in a relatively slight increase of Type II Error. Since the target class, i.e., positive class, has more impact on the main research problem, Meta-Cost scheme of cost-sensitive analysis is proved to be a candidate solution in overcoming the issue of highly imbalanced bank marketing data. Figure 5 plots the Lift Curve of the Meta-Cost using MLP for classification with a _FN-Cost Factor_ equals 200. The lift chart indicates the ratio of having positive responses (bank clients accept to subscribe to a term deposit) to all positive responses. According to this lift chart, reaching only 10% of the total clients by the developed prediction model we will hit 4.3 times more than if no model was applied (clients reached randomly). Figure 4: The effect of Adjusting FN-Cost Factor on Type I and Type II Errors. Figure 5: Lift Curve, Meta-Cost using MLP with FN-Cost Factor of 200. To illustrate more the achieved enhancement in classification performance, Table 7 summarizes the performance of conventional machine-learning classifiers in Weka environment against the best cost-sensitive prediction models. The algorithms are namely: Lib-LINEAR (LL), i.e., an implementation of Support Vector Machines, Decision Table (DT), Very Fast Decision Rules (VFDR), Random Forest Trees (RF), Multilayer Perceptron (MLP), J48, and Deep Learning for MLP Classifier (DL-MLP). The experiments were repeated 10 times. 10-Fold Cross Validation is used to evaluate each run. Results are reported in terms of TPR, TNR, Geometric Mean, Type \(|\) Error, Type \(|\) Error, and Total Accuracy metrics. FPR for target class is reported in terms of the average and standard deviation (SD) of the 10 runs, while average of the 10 runs used to report the remaining metrics.