# Customer Churn Prediction Approach Based on LLM Embeddings and Logistic Regression
## 1 Introduction  
The success and growth of companies are highly related to customer satisfaction. Consequently, in any domain, there is intense competition to attract new customers and retain existing ones by applying various strategies. However, in some cases, customer churn occurs despite these efforts. Customer churn refers to the situation where customers stop purchasing a company's product or service for any reason. A high number of churners indicates that a larger number of customers are disengaging from the business. This phenomenon affects various sectors such as telecommunications [1], e-commerce [2], gaming [3], and subscription-based services [4]. Losing existing customers can significantly impact a company's revenue [5]. Like any other business, the telecommunications industry is built on the loyalty of its customers. However, customers may cancel their subscriptions and switch to another telecom operator for various reasons, given the availability of alternatives. This disengagement impacts telecom companies financially and their reputation in the market, which affects directly this business progress [5]. Furthermore, according to an article in reference [6], obtaining a new customer is more expensive than maintaining an existing one. The first sale is particularly expensive and time-consuming, requiring extensive efforts in advertising, promotion, calls, contacts, demonstrations, and training. Therefore, customer retention has become the foundation for every industry to plan for future growth. Consequently,predicting customers at risk of churning becomes crucial, as it enables companies to implement retention strategies and plans that offer more satisfactory services to prevent losing customers. Machine learning (ML) is a valuable tool in predicting customer churn across various industries [7]. ML models have shown notable results in learning complex patterns and predicting outcomes accurately on unseen data. The main objective of this work is to build an ML model that provides telecommunication companies with an effective customer churn prediction model. We recognize the effectiveness of traditional preprocessing techniques for handling categorical and numerical data. However, we intended to investigate whether LLM embeddings, typically designed for textual data, can provide advantages in uncovering complex patterns through semantic representations. By emerging categorical and numerical features into coherent sentences, we aim to allow the LLM to capture relationships between different data elements that may be missed through one-hot encoding or numerical standardization. State-of-the-art studies in this field typically build predictive models using numerical datasets, such as the date of customer registration, number of complaints, and years of subscription. However, textual data also holds massive amounts of valuable information. Therefore, combining numerical and textual data can lead to more comprehensive and accurate insights. In our work, we leveraged both numerical and textual data, such as complaints and user feedback, to build an effective model. We conducted numerous experiments to create our dataset and test the effectiveness of different embedding techniques. These embedded vectors were subsequently used to train and test various machine-learning models. The main contributions of this work are summarized as follows:  
* We conducted a comparison study between various embedding techniques, including the OpenAI embedding model, Google Gemini Text Embedding, Google Bert, Sentence-Transformers, Sent2 vec, and Doc2 vec. * We evaluated the performance of various classifiers in predicting customer churn: SVM, LR, RF, zero-shot classification, KNN, MLP, NB, and DT. * We built an effective model that predicts customer churn with a notable accuracy of 89%. The rest of this paper is organized as follows: Section 2 provides background information and surveys the existing research work. The algorithms and techniques used in this study are presented in Section 3.