# Intelligent Decision Forest Models for Customer Churn Prediction
## 4 Results and Discussion
### CCP Performance Comparison of the DF Models and ML Classifiers  
In this section, the CCP performance of LMT, FT, and RF as DF models are compared with the ML classifiers on original and SMOTE-balanced Dataset 1 and Dataset 2 as outlined in Section 3.3 (Phase 1). Table 3 illustrates the CCP performances of LMT, FT, and RF and the selected ML classifiers (NB, kNN, and DS) on Dataset 1 (Kaggle Dataset). It can be observed that the DF models were superior in prediction performances when compared to the experimented ML classifiers based on the studied performance metrics. Concerning accuracy values, LMT recorded the highest of 94.75% amongst the DF models, followed by FT and RF, respectively. As for the ML classifiers, NB performed best with 88.24% prediction accuracy. However, LMT, FT, and RF had +7.38, +7.0, and +3.09% accuracy increments compared to NB. The LMT, FT, and RF's superior prediction accuracy values over NB, kNN, and DS, even on an imbalanced Dataset 1, highlights their resilience and usefulness for CCP. A similar occurrence can be observed regarding AUC values as the duo of LMT and FT both had the highest AUC values of 0.905 while RF had 0.896. The AUC values of these DF models were superior to that of NB (0.834), kNN (0.603), and DS (0.603). Also, the DF models obtained a strong ratio of sensitivity and recall, with LMT and FT having f-measure values of 0.945 and 00942, respectively. It is worth noting that the ML classifiers also had comparable f-measures, but they were still lower than those of the DF models. This observation confirms the performance stability of the DF models in CCP compared to experimented ML classifiers. In the case of the MCC values, the performances of the DF models are notably comparable, as LMT (0.777) was slightly better than FT (0.763). The other ML classifier had lower MCC values which indicate no conformity between the predicted and observed values. Figure 2 illustrates a graphical depiction of the CCP of the experimented models. From Dataset 2 (presented in Table 4), the DF models had comparable performance to the ML classifiers in terms of accuracy, AUC, f-measure, and MCC values. It was also discovered that the DF model experimental results on Dataset 2 are somewhat lower than those of Dataset 1. LMT, FT, and NB had similar prediction accuracy values (89.86%), and their respective AUC values are average (LMT: 0.5, FT: 0.5, NB: 0.503). Also, it can be observed that the f-measure and MCC values of some of the implemented models (NB, DS, LMT, and FT) are missing, and some models recorded poor performance based on f-measure and MCC values. This observation can be attributed to the nature and data quality of Dataset 2. That is, the presence of class imbalance (as indicated in Table 2) (IR of 8.86) had a derogatory effect on the implemented DF and ML models. Figure 3 depicts DF's and the ML classifiers' CPP performances. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy (\%)** & **AUC** & **F-Measure** & **MCC** \\ \hline LMT & 91.59 & 0.896 & 0.876 & 0.752 \\ \hline FT & 94.26 & 0.973 & 0.942 & 0.883 \\ \hline RF & 90.32 & 0.880 & 0.503 & 0.806 \\ \hline WSVEDFM & 96.57 & 0.988 & 0.965 & 0.981 \\ \hline WSEDFM & 96.43 & 0.986 & 0.964 & 0.971 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: The CCP performance of DF models and their Enhanced ensemble variants on balanced Dataset 2. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy (\%)** & **AUC** & **F-Measure** & **MCC** \\ \hline RF & 92.14 & 0.943 & 0.945 & 0.843 \\ \hline WSVEDFM & 96.31 & 0.990 & 0.987 & 0.940 \\ \hline WSEDFM & 96.31 & 0.989 & 0.983 & 0.946 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: _Cont._Based on the relative prediction performances of the DF models, which may have been affected by the latent class imbalance problem, the IR values for Dataset 1 and Dataset 2 are 5.9 and 8.86, respectively (See Table 2). Hence, this research work explored the prediction performances of the DF models and ML classifiers on balanced (SMOTE) CCP datasets.