Machine Learning Analytics — Telco Churn Prediction
By: Lucio Chen
Tags: Machine Learning, Telco, Mba, Projects, Churn Prediction

Image Creator: Dong Wenjie | Credit: Getty Images

Paul Merage School of Business MBA Project; Advisor: Professor Vibs Abhishek

Author: Axel Eyer, Kevin Ly, Lucio Chen, Shimpei Tange;

Introduction:

Churn rate is defined as a measure of the number of individuals or items moving out of a collective group over a specific period. Churn rate is one the most important measures for customer satisfaction and operational performance for telcos. Research has shown that on average, it is 5~25 times more expensive to acquire a new customer than to retain existing customer and a 5% increase in customer retention can lead to revenue increase by 25%~95%[i]. Therefore, it is paramount to understand why churn happens and how we can minimize the churn rate. In this project, we hope to leverage telco customer data to understand customer churn and provide a meaningful business decision.

Objective:

The objective is to accurately identify customers that will potentially churn with independent variables such as gender, tenure, and opt-in business services like phone lines and internet services. The business impact can be measured by customer retention and revenue generated from retention, re-subscription and even cross-selling revenues that contributes to the total customer lifetime value (LTV). The success of the system could be measured by analyzing the accuracy of the model in classifying whether or not the customer will churn. The ultimate goal of the model is to help drive revenue and profit for telcos.

Dataset:

The data was provided from Kaggle. It includes a total of 7,000 rows of unique customer data with minimal missing values. There is telco specific data such as phone lines, Internet services, tenure of service, contract types, payment methods, monthly and total charges. There is also demographic data such as gender, senior citizenship status, and whether customers have dependents or not.

Data Cleaning & Analysis:

Data exploration and data cleaning are the first step before modeling. Pivot tables have been created to provide an overview of the data. The data can be categorized into demographics, business service variables (which is binary), various types of contracts, and payment methods.

Figure 1 Descriptive visualization of the independent variables (business service)

Figure 2 Descriptive visualization of demographic data

As shown in figure 1 and figure 2, 7,043 customers were distributed between female or male, having dependents or not having dependents, and having partners, or not having partners. A fraction of the customers are seniors citizens (16.2%). There are a total of 11 independent variables related to both phone and internet services. 90.3% of the customers have phone service and 42.2% of total customers have multiple lines. For internet services, about 78.3% of the customers have internet services — which is made up of DSL (44.0%) and fiber (34.4%).

Another set of data is related to contract and payment methods of the customers. Contract type is distributed among month-to-month (55%), one-year (21%) and two-year (24%). Payment method is slightly more evenly distributed with bank transfer (22%), credit card (22%), mailed check (23%), and electronic checking (34%).

Figure 3 Whisker Plot of Numeric Data

Figure 3 summarizes the numeric data such as tenure (total number of months), total monthly charges, and total charges per customer. The median tenure is 21.5 months with an interquartile range of 40 months. Total monthly charges have a median of $56.5 and an interquartile range of $64. The total charges have a median of $1,432 with an interquartile range of $2,880. Finally, the figure provides a breakdown of customer churned or not churn, this classification will help us in training the model.

Approaches

Approach I: Training with percentage splits

Models evaluation with Percentage Splits

In this approach, we have trained Naïve Bayes, Logistic Regression, and J48 decision tree with percentage splits across 60%, 66%, 70% and 80%. Across the three models, the highest accuracy comes from the Logistic Regression model consistently with more than 80% of accuracy rate. The highest precision also comes from the logistic regression model. On the other hand, the highest recall comes from Naïve Bayes which has consistently over 80% of recall.

Approach II: Training the model with k-fold

Models evaluation with Cross Validation

In the second approach, we have trained Naïve Bayes, Logistic Regression, and J48 decision tree with k-fold across 5, 10, 15, and 20 folds. The highest accuracy rate comes from Logistic Regression with an accuracy rate of at least 80%. The precision is also the highest for logistics regression model throughout, whereas the recall is the highest with the Naïve Bayes model.

Approach III: Logistic Regression — Coefficient (odds)

We will now examine the coefficients of the Logistic Regression’s independent variables by transforming it to the odds. What we can determine from this is that there is a relationship between a larger odd of the independent variable and the likelihood of the customer to stay, whereas a smaller value of the odds decreases the likelihood of a customer to stay.

Figure 4 Logistics Regression Coefficiency I

Figure 5 Logistics Regression Coefficiency II

Overall, we can say that the longer the tenure and the higher the monthly charge, the lower the customer churn rate. Also, the longer the contract, the lower the customer’s churn rate. What is interesting here is that customers who choose fiber optic for their internet service are more likely to churn than customers who choose DSL or no internet. This could be due to other factors such as other available fiber options and increased marketing efforts from competitors to offer fiber internet services.

Approach IV: Improvement with Cost & Benefit (Type II vs Type I)

In the case of churn, accuracy allows us to understand the percentage of customers accurately predicted to churn and not churn. It is important to understand the implication of Type I errors (False Positive) and Type II errors (False Negative) and their business costs.

Type I error (False Positive) refers to the customers who were predicted to churn but were planning to stay. If we assume the telco invests more in keeping the risky customers from churning, such as providing incentives to upgrade or free services, then potential Type I error cost is the cost of investment given to customers who were not planning to change services to another telco.

Type II error (False Negative) refers to the customers who are planning to leave but were categorized to stay. Assuming the telco focuses its investments in keeping the risky customers from churning and continues to provide the same level of services to other customers, then the potential cost of Type II error is lost customers. This can be very costly since the cost of acquiring a new customer is almost 5~25 times higher compared to retaining current customers. Therefore, a Type II error is much more expensive compared to Type I error.

Since Type II errors are more expensive, we have decided to adjust the cost of Type II error to be 1.5 times of the cost of Type I error.

Before the cost adjustment: Original Confusion Matrix (The ratio of Type II cost to Type I cost is 1:1)

Before: Weka Cost/Benefit Analysis: Accuracy = 80.12%; Type II error = 14.01%

After the cost adjustment: New Confusion Matrix (The ratio of Type II cost to Type I cost is 1.5:1)

After: Weka Cost/Benefit Analysis: Accuracy = 79.18%; Type II error = 9.37%

The cost of Type II errors is 1.5 times higher than that of Type I errors. By changing the threshold, the total accuracy would decrease by about 1% from 80.12% to 79.18%, but the Type II error was reduced by about 4.5% from 14.01% to 9.37%.

After choosing our “best” model, additional Cost-Benefit Analysis can be done to manage the total cost of our project. However, it should be noted that reducing one-side error causes a higher number of errors of the opposite Type. To accurately determine the threshold, an accurate understanding of the cost of errors is essential.

Business Implication and Recommendation:

It is more important to understand what proportion of actual positives were identified correctly (recall) rather than what proportion of positive identifications were correct (precision). In other words, it is more important to know the percentage of churning customers that the algorithms successfully find rather than the percentage of the actual customer churn from the customers that the algorithms predict to churn. Improving recall percentage and reducing the total number of Type II errors is important for the telco to retain its revenue.

Figure 6 Approach I Percentage Split Comparison

Figure 7 Approach II Cross Validation Comparison

Regardless of percentage split or cross-validation, Naïve Bayes had the best recall percentage compared to Logistic Regression and Decision Tree. Even though accuracy was lower, the substantially higher recall means that the telco should use Naïve Bayes to predict churn. After the Type II error customers have been identified, the telco should invest in incentives to entice these customers to stay. It may be easier to get these customers to stay compared to those that were accurately predicted to churn outright. Actions could include special loyalty programs, improved customer feedback, enhanced customer support, priority wait times, and experienced agents. We want to know what these customers really care about and make it easier to express their concerns.

As discussed previously, customers with higher charges, higher tenure, internet services and extended year or two-year contracts are the least likely to churn. More attention should be given to the recently onboarded month to month customers to make sure they are happy with their current service. After that, the telco should invest in marketing and sales programs to encourage all their customers to upgrade their services with bundles and long-term contracts. Once a customer is locked into the Telco’s ecosystem, they are less likely to look for alternative services.

Future Improvements:

The results found here are limited by the data available. To get better results in the future, a greater variety of data is needed. Some examples include time series data, qualitative data, and other business variables. Time series data such as date and time would help to plot and monitor the impact of business improvements over time. Qualitative data such as events, competitor’s offerings, new product launches would help to understand the impact of events on churn rate. Other business variables such as locations, customer satisfaction, and data usage would allow for more sophisticated modeling that could shed further insights into potential reasons for churn.

[i] The Value of Keeping the Right Customers, Harvard Business Review — https://hbr.org/2014/10/the-value-of-keeping-the-right-customers