# Customer Churn Prediction Approach Based on LLM Embeddings and Logistic Regression
## 6 Result and Discussion
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Embedding Model**} & \multirow{2}{*}{**Classifier**} & \multicolumn{4}{c}{**Measure**} \\ \cline{3-6}  & & **Accuracy** & **Precision** & **F1 Score** & **Recall** \\ \hline \multirow{6}{*}{Sent2 vec-distilbert-base-uncased} & LR & **86\%** & 86\% & 86\% & 86\% \\ \cline{2-6}  & SVM & 79\% & 78\% & 77\% & 76\% \\ \cline{2-6}  & RF & 77\% & 77\% & 77\% & 77\% \\ \cline{2-6}  & MLP & 85\% & 86\% & 85\% & 86\% \\ \cline{2-6}  & NB & 81\% & 83\% & 81\% & 83\% \\ \cline{2-6}  & KNN & 65\% & 65\% & 65\% & 65\% \\ \cline{2-6}  & DT & 69\% & 69\% & 69\% & 69\% \\ \cline{2-6}  & Zero-shot classification & 54\% & 50\% & 40\% & 50\% \\ \hline \multirow{6}{*}{Doc2 vec-Gensim’s word2 vec trained on ‘text8’ corpus} & LR & 56\% & 56\% & 53\% & 53\% \\ \cline{2-6}  & SVM & **57\%** & 73\% & 42\% & 53\% \\ \cline{2-6}  & RF & 55\% & 54\% & 52\% & 53\% \\ \cline{2-6}  & MLP & 55\% & 54\% & 54\% & 54\% \\ \cline{2-6}  & NB & 54\% & 53\% & 52\% & 53\% \\ \cline{2-6}  & KNN & 52\% & 51\% & 51\% & 51\% \\ \cline{2-6}  & DT & 52\% & 52\% & 52\% & 52\% \\ \cline{2-6}  & Zero-shot classification & 46\% & 47\% & 35\% & 33\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: _Cont._  
Figure 4: Final Model Deployment. Logistic regression emerges as the most effective model in this case due to its strong ability to identify linear relationships between features and the target variable. The embeddings used in this study preserve the semantic similarities between the data points, meaning that objects with similar characteristics are mapped closer together in the feature space. Logistic regression excels in such scenarios because it is designed to capture simple, linear decision boundaries, which is often sufficient when the data are relatively straightforward and linearly separable. Additionally, logistic regression is less prone to overfitting, particularly when working with high-dimensional data. In contrast, more complex models like random forest and multilayer perceptron (MLP) are more prone to overfitting, especially when the dataset is small or contains noisy features. Random forest, while powerful for handling complex and non-linear relationships, can struggle with high-dimensional spaces where it might unintentionally model noise rather than meaningful patterns. Similarly, MLPs require extensive hyperparameter tuning and often need larger datasets to generalize well, making them more vulnerable to overfitting if not properly optimized. The simplicity and interpretability of logistic regression allow it to efficiently handle these challenges, making it an ideal choice for this particular task. By capturing the essential linear relationships in the data and being less affected by overfitting, logistic regression provides a robust solution even in high-dimensional feature spaces with limited data. While the developed model demonstrated high performance, achieving an accuracy of 89% in predicting churn in the telecommunications sector, its implications extend beyond just prediction. By identifying potential churners before they leave, telecom companies can proactively implement targeted retention strategies such as offering exclusive offers to retain as many customers as possible. This predictive capability enables telecom companies to improve customer relationships, reduce revenue loss, enhance customer satisfaction, and improve their competitiveness in a highly dynamic market. Although our model has proven to be a highly effective method in prediction churn, the fact that customer feedback was limited to a set of nine predefined responses rather than allowing free text input could impact the model's ability to fully capture the diversity of customer opinions, complaints, and comments. This limitation may affect the model's performance in predicting churn more accurately.