Comprehensive Churn Prediction and Analysis
By: Mandy Gu
Tags: Data Analysis, Machine Learning, Data, Data Science, Programming

Telco Dataset

We will be training our churn model over the Telco-Customer-Churn Dataset to predict the likelihood of customers leaving the fictional telecommunications company, Telco. This synthetic dataset was put together by IBM and includes a label indicating whether or not the customer left within the last month.

Goal: predict whether a customer will churn based on their demographic and service information.

Data Exploration

The exploration and modelling will be conducted using a Jupyter notebook. We begin by loading the required libraries and importing the data as a DataFrame.

Indexing the first row shows us the columns and a sample record from our dataset.

Inspecting the Dataset

This dataset is fairly clean with no missing entries. There is a total of 7043 observations. Around 27% of the customers churned (not a good sign for Telco).

We can use the info(), describe() methods from pandas for a quick overview of data integrity. The dataset does not have any missing values. Although pandas interpreted SeniorCitizen as a continuous variable, it is actually a binary indicator.

Continuous Variables

We can use the seaborn library to further visualize and inspect the dataset. Let’s examine the continuous variables tenure, MonthlyCharges first. Since TotalCharges can be approximatedas a function of these two variables, we will exclude it from the plot.

Seaborn’s pairplot functionality plots pairwise relationships in the data.

Interpreting the Pair plot:

the diagonal axis is the histogram for the particular variable with the y-axis measuring the number of occurrences

the bottom left triangle of the plots and the top right triangle capture the same information, except the axis are flipped

The histograms tell us that the tenure distribution of churned clients is right skewed while the same distribution for churned clients is more uniform. It also shows that clients with higher MonthlyCharges see a higher rate of churn.

The other two charts capture very similar information with a distinctive divide between the orange churn and blue unchurned.

Next, we use boxplots to illustrate the quantile differences between customers who churned and who did not churn.

The first box plot compares the tenure quantiles between the two groups. The median tenure for churned customers is a lot shorter than unchurned customers. 75% of customers who end up leaving Telco do so within their first 30 months. There are a few outliers that churned after being with Telco for 70 months.

The second box plot compares MonthlyCharges against churn. The median MonthlyCharges of churned customers is significantly higher than those who did not churn. This suggests that discounts and promotions can be an enticing reason for customers to stay.

Demographic Variables

Next, we can examine the categorical variables, starting with the demographic variables: gender, SeniorCitizen, Partner and Dependents.

The top left graph counts the intersection between gender and churn. There is very little difference in churn proportionsbetween male and female customers.

There is a higher proportionof churn amongst SeniorCitizen (top right chart), customers with no Partners (bottom left chart) and customers with no dependents (bottom right chart).

The second finding might be of interest to business stakeholders. Non-senior citizens with no partners and dependents describe a particular intersection of customers. In lieu of other indicators, less frequent churn in this demographic speaks positively towards their lifetime value.

Other Categorical Variables

The other categorical variables can be divided between service and billing information.

These charts capture the service variables. There is a higher proportion of churn in customers without OnlineSecurity, OnlineBackup, DeviceProtection, and TechSupport. Based on my fairly limited knowledge about telecommunication services, I don’t think most people are frequently paying for these services. It suggests that those with increased reliabilityon Telco’s services, or those who need their device for particular uses, tend to churn less.

Next, we examine the intersection between billing method and churn. The top left chart compares churn across contract type. To no surprise, those with shorter plans (month to month) see a higher rate of churn. Those with longer plans face additional barriers when cancelling prematurely.

Surprisingly, those who opt out of PaperlessBilling are more frequently to churn as well as those who pay by electronic check. Some of these behaviors may be confounded with other variables (for instance, older customers might have a preference towards paper bills).

Training a Predictive Model

Here are the libraries that we are going to use to build our model. I am using a new notebook, so I will load the data in again.

In the TotalCharges column, 11 records are an empty string. All of these records have a valid MonthlyCharges value, but a tenure of 0. We can impute these values using 0.

Preparing the Data for Modeling

First, we one hot encode categorical variables and randomly partition the data between training and testing. After partitioning the dataset, we do not touch the test set except for evaluation purposes.

Feature Selection

The prior exploration already gave a lot of information regarding which features are important. We can use that in conjunction with a Chi-squared test to make a more informed decision.

Note: the Chi-squared test can only be used to assess categorical variables.

We will need to understand p-values to interpret the Chi-squared output.

The P-value indicates the % of more “extreme” observations if the null hypothesis is true and if we repeat the same experiment many times. The null hypothesis here is that the feature has no effect on the target (i.e. we should not use it as a predictor). Low p-values suggest we should reject our null hypothesis — in other words, reject the claim that the feature has no effect.

The p-values for most of the features are very low. Only gender, MultipleLines, PhoneService have p-values > 0.05, which means that we do not have enough evidence to reject the claim that these features have no effect. This is well aligned with the story from the charts.

For a better visual, let’s plot all categorical variables with p-value > 0.05.

We will also drop the TotalCharges since the same information is captured by MonthlyCharges, Tenure to further reduce the dimensionality (the other two columns appear to be more reliable than this one, which had data issues).

Model Training

We will use a Logistic Regression model to predict churn. This might not necessarily be the most performant algorithm or the algorithm which best separates the data (the only way to find out is to try), but let’s assume it is good enough for simplification. One additional advantage of Logistic Regression is the ease of interpretation.

We can leverage the GridSearchCV function from Sklearn for hyperparameter tuning. . Since we do not want to touch our test set until the very end, we will further partition our training set into training and validation.

We will use grid search to determine the optimal values for these hyperparameters: — whether we should use a L1 or L2 penalty — the C value — whether we should fit an intercept — class weight (whether we should put more emphasis on the Churn class as opposed to the non-Churn class).

Under the hood, GridSearchCV will be training and evaluating numerous models against the validation set to determine which hyperparameters yield the best performance.

Note: while I have not done so here, GridSearchCV can be used to test candidate algorithms.

We can find the best hyperparameter values using grid.best_params_ , which in this case is:

{'C': 1, 'fit_intercept': False, 'penalty': 'l1'}

With this information, we can formally train our model. We will be training the model on the training and validation set combined.

Evaluating the Model

Let’s test the model against our test set. The classification report gives the precision, recall and f1-score (the support is the number of observations in each class). The micro f1-score is 0.81 while the macro f1-score is 0.74.

We can visualize this result using a confusion matrix.