# Intelligent Decision Forest Models for Customer Churn Prediction
## 4 Results and Discussion  
In this section, results and findings obtained from the experimental procedure shown in Section 3.3 are presented and analyzed. The effectiveness of the CCP models will be discussed based on their respective prediction performances with or without the class imbalance problem. That is, the efficacies of the CCP models studied on both original and balanced (SMOTE) CCP datasets will be investigated. Tables 3 and 4 present the comparison of CCP performances of DF models (LMT, FT, and RF) against ML classifiers (NB, kNN, and DS) on the original Datasets 1 and 2. The selected ML classifiers were chosen based on their prediction performances in ML tasks and distinct computational properties. In addition, Tables 5 and 6 show the CCP performance of the DF models with the ML classifiers on SMOTE-balanced Dataset 1 and Dataset 2. The data sampling method (SMOTE) was deployed to address the inherent class imbalance problem in CCP datasets. In addition, the deployment of balanced datasets on the CCP models will indicate the impact of data sampling on CCP models. To develop effective CCP models, enhanced ensemble variants of the DT models (WSVEDFM and WSEDFM) were deployed on the original and balanced versions of Dataset 1 and Dataset 2. Specifically, Tables 7-10 present the CCP performances of each of the DF models and their enhanced ensemble variants on the original and balanced studied customer churn datasets respectively. This analysis will show how the different DF models can work with imbalanced and balanced datasets. For a fair comparison, the CCP performances of DF models are further compared with renowned ensemble methods such as Bagging and Boosting. Finally, the CCP performances of the high-performing DF models are contrasted with current state-of-the-art CCP models. Consequently, the experimental results are aided by graphical representations to demonstrate the relevance of the observed experimental findings. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy (\%)** & **AUC** & **F-Measure** & **MCC** \\ \hline NB & 77.21 & 0.825 & 0.772 & 0.545 \\ \hline kNN & 83.91 & 0.839 & 0.839 & 0.678 \\ \hline DS & 87.84 & 0.499 & 0.500 & 0.257 \\ \hline \multicolumn{4}{l}{* LMT} & 91.59 & 0.896 & 0.876 & 0.752 \\ \hline \multicolumn{4}{l}{* FT} & **94.26** & **0.973** & **0.942** & **0.883** \\ \hline \multicolumn{4}{l}{* RF} & 90.32 & 0.880 & 0.503 & 0.806 \\ \hline \hline \end{tabular}  
* The superior CCP performances are bold and type-faced, and the proposed DF models are designated by an asterisk.  
\end{table} 
Table 6: The CCP performance of DF models and ML classifiers on Balanced (SMOTE) Dataset 2. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Accuracy (\%)** & **AUC** & **F-Measure** & **MCC** \\ \hline LMT & 89.86 & 0.5 &? &? \\ \hline FP & 89.86 & 0.5 &? &? \\ \hline RF & 89.49 & 0.508 & 0.850 & 0.003 \\ \hline WSVEDFM & 89.86 & 0.555 & 0.855 & 0.030 \\ \hline WSDFM & 89.86 & 0.545 & 0.850 & 0.025 \\ \hline \hline \end{tabular}  
\end{table} 
Table 8: The CCP performance of DF models and their Enhanced ensemble variants on original Dataset 2. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy (\%)** & **AUC** & **F-Measure** & **MCC** \\ \hline LMT & 93.60 & 0.971 & 0.966 & 0.872 \\ \hline FP & 94.83 & 0.975 & 0.968 & 0.897 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: The CCP performance of DF models and their Enhanced ensemble variants on balanced Dataset 1. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Accuracy (\%)** & **AUC** & **F-Measure** & **MCC** \\ \hline LMT & 94.75 & 0.905 & 0.945 & 0.777 \\ \hline FP & 94.42 & 0.905 & 0.942 & 0.763 \\ \hline RF & 90.97 & 0.896 & 0.895 & 0.581 \\ \hline WSVEDFM & 95.81 & 0.951 & 0.958 & 0.879 \\ \hline WSDFM & 95.53 & 0.948 & 0.955 & 0.865 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The CCP performance of DF models and their Enhanced ensemble variants on original Dataset 1.