# Intelligent Decision Forest Models for Customer Churn Prediction
## 4 Results and Discussion
### CCP Performance Comparison of Enhanced DFEnsemble Variants, Bagging and Boosting Ensemble Methods  
In this section, further assessment and performance comparisons were conducted to validate the effectiveness of WSVEDFM and WSEDFM for CCP processes. Specifically, the proposed DF ensemble variants were compared with Bagged and Boosted DF models on both original and balanced (SMOTE) Dataset 1 and Dataset 2. Tables 11-14 outline the CCP performance and comparison of WSVEDFM and WSEDFM with the Bagged DF and Boosted DF models on original and balanced CCP datasets. As shown in Tables 10 and 11, the Bagged and Boosted DF models had comparable performances as the WSVEDFM and WSEDFM on Dataset 1 and Dataset 2, respectively. In some cases, the differences in the prediction accuracy and f-measure values of the proposed DF ensemble variants and the Bagged and Boosted DF modes are insignificant. However, the WSVEDFM and WSEDFM are still superior in performance. For instance, WSVEDFM (0.951) and WSEDFM (0.948) had a +3.59% and +3.27% increment in AUC values over \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy (\%)** & **AUC** & **F-Measure** & **MCC** \\ \hline BaggedLMT & 95.50 & 0.986 & 0.955 & 0.910 \\ \hline BaggedFT & 95.96 & 0.985 & 0.960 & 0.919 \\ \hline BaggedRF & 92.30 & 0.971 & 0.923 & 0.846 \\ \hline BoostedLMT & 95.43 & 0.984 & 0.954 & 0.909 \\ \hline BoostedFT & 95.36 & 0.984 & 0.954 & 0.907 \\ \hline BoostedRF & 92.30 & 0.973 & 0.923 & 0.847 \\ \hline \multicolumn{4}{l}{\({}^{\star}\) WSVEDFM} & **96.31** & **0.990** & **0.987** & 0.940 \\ \hline \multicolumn{4}{l}{\({}^{\star}\) WSEDFM} & 96.31 & 0.989 & 0.983 & **0.946** \\ \hline \end{tabular}
\end{table} 
Table 13: The CCP performance of DF models and their Enhanced ensemble variants on balanced Dataset 1. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy (\%)** & **AUC** & **F-Measure** & **MCC** \\ \hline BaggedLMT & 95.21 & 0.914 & 0.951 & 0.801 \\ \hline BaggedFT & 95.44 & 0.918 & 0.953 & 0.807 \\ \hline BaggedRF & 90.16 & 0.898 & 0.881 & 0.533 \\ \hline BoostedLMT & 94.18 & 0.911 & 0.932 & 0.756 \\ \hline BoostedFT & 93.49 & 0.903 & 0.940 & 0.721 \\ \hline BoostedRF & 91.36 & 0.899 & 0.900 & 0.602 \\ \hline \multicolumn{4}{l}{\({}^{\star}\) WSVEDFM} & **95.81** & **0.951** & **0.958** & **0.879** \\ \hline \multicolumn{4}{l}{\({}^{\star}\) WSEDFM} & 95.53 & 0.948 & 0.955 & 0.865 \\ \hline \end{tabular}
\end{table} 
Table 11: The CCP performance comparison of WSVEDFM and WSEDFM with Bagged and Boosted DF models on Dataset 1. Bagged FT (0.918), which had the highest AUC value amongst the Bagged and Boosted DF models. A similar trend was observed with the MCC values with WSVEDFM (0.879) and WSEDFM (0.865), recording a +9.74% and +7.99% over Bagged FT (0.801). Contrary to the findings from Table 11, the CCP performance of the studied models on Dataset 2, as shown in Table 12, is relatively good. This could be related to the observed data quality problem (high IR) in Dataset 2. However, on a general note, the WSVEDFM and WSEDFM outperformed the Bagged and Boosted DF models, although the CCP performances of the Bagged DF models are better than the Boosted DF models. This finding can be due to the independent and parallel mode of model development in the Bagging method, which reduces variances amongst its base models and avoids model overfitting. In addition, the CCP performances of WSVEDFM and WSEDFM with Bagged and Boosted DF models on balanced Dataset 1 and Dataset 2 were compared. Tables 13 and 14 display the CCP performances of WSVEDFM, WSEDFM, Bagged DF models, and Boosted DF models on balanced Dataset 1 and Dataset 2, respectively. As shown in Tables 13 and 14, it can be observed that there are significant improvements in the performances of the WSVEDFM and WSEDFM over the Bagged and Boosted DF models on the balanced Dataset 1 and Dataset 2. Specifically, from Table 13, WSVEDFM and WSEDFM had a +0.36% increment in prediction accuracy values more than the best performer (Bagged FT) in this case.