# ChurnKB: A Generative AI-Enriched Knowledge Base for Customer Churn Feature Engineering
## 5 Discussion  
As illustrated in Figure 6, ChurnKB was validated in terms of its elements, content, and performance, leveraging human expertise for refinement. Participants' responses support Hypotheses 1-4. Part (i) confirms that all participants find the components of the developed knowledge base relevant to customer churn. In Part (ii), ten participants recognize the three sub-instances (i.e., the use of the pronoun \(I\), _absolute words_, and _certainty words_) as relevant to churn, while the rest perceive them as neutral. Since the majority consider these sub-instances relevant, we decided to incorporate them into ChurnKB.  
Furthermore, Part (iii) shows that all experts, except one, recognize the relevance of the sub-instances used in the developed classifier for identifying churn-related behavioural patterns. One expert, however, considers them only weakly relevant. Part (iv), which assesses the churn classifier's outcomes using the modified version of ChurnKB, indicates that all but one participant find the results relevant.  
However, it is essential to note that there are some validity concerns that should be considered before drawing conclusive results from our study. Although the overall survey results support Hypotheses 1, 2, 3, and 4, there is still room for improvement in the proposed approach--for instance, considering weights for some of the concepts and instances to be indicative of being more influential with respect to churn decisions of customers. These matters will be further investigated in our future works.  
On the other hand, the classifier was developed and evaluated using the extracted features from ChurnKB. Although the study is currently limited to a telecom dataset, the framework can be applied to other industries, such as banking, insurance, and e-commerce. For instance, in banking, customer churn often manifests through declining transaction frequency, reduced credit card usage, and negative sentiment in customer  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Model Setup** & **F1 Score** &
\begin{tabular}{c} **Improvement over** \\ **Baseline** \\ \end{tabular} \\ \hline Baseline (A)—numerical Only & 0.5752 & — \\ ChurnKB-enabled (B)—textual features only & 0.6698 & +9.4\% \\ Full model (C)—numerical + textual features & 0.7891 & +21.3\% \\ \hline \hline \end{tabular}
\end{table}
Table 4: Ablation study results, measuring the impact of ChurnKB-enabled features.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Metric** & **T-Statistic** & \(p\)**-Value** & **Metric** & **T-Statistic** & \(p\)**-Value** \\ \hline AUC & 5.297949 & 0.000253 & Recall & 7.819004 & 0.000008 \\ Precision & 6.428685 & 0.000049 & F1 & 8.786979 & 0.000003 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Paired \(t\)-test results comparing model performance with and without ChurnKB-derived features across multiple evaluation metrics. Statistically significant improvements (\(p\)-value \(<\) 0.05) in all metrics suggest that ChurnKB features enhance model performance.  
support interactions. In insurance, churn indicators include policy cancellations, inquiries about competitor policies, and dissatisfaction with claim processes. ChurnKB can be extended to these domains by adapting its feature extraction pipeline to industry-specific text sources (e.g., customer service transcripts, complaint records, and online reviews). Additionally, integrating industry-specific lexicons and knowledge bases would further refine ChurnKB's feature representation, enhancing its applicability across sectors.