K-Nearest Neighbors (KNN) on Customer Churn Data
By: Akhil Sharma
Tags: Knn Classification, Customer Churn, Knn, Supervised Learning, Machine Learning

If a few of my neighbors leave the telecom service provider, should I leave it too? No, not really. That’s not what I meant.

When choosing a telecommunication service provider, customers usually have many choices. They can choose any service provider and may move away from the current provider. When a customer chooses to move away from the current provider to a new provider, it results in loss of business and revenue to the current provider. The percentage of customers moving out and disconnecting the service is known as “churn”. A stable customer base is a key to the success of any business. Businesses try to keep the customers satisfied, to retain them as long as possible. However, in the real world, the customer churn can be as high as 25% annually in the telecommunication industry. Also, the cost of acquiring a new customer is 10 times more than the cost to retain an existing customer. This poses a serious challenge to business owners.

Note:

All data used for this analysis is available at: https://github.com/microbhai/CustomerChurnAnalysis/blob/master/telecommunication_customer_churn.csv

All code available at: https://github.com/microbhai/CustomerChurnAnalysis/tree/master/ClassificationAnalysisKNN

Role of Data Analytics

Customer Churn data analysis can help the company understand the underlying reasons why customers may choose to leave the company. By implementing predictive analytics techniques and applying them to existing customer churn data from records, it is possible to understand the likelihood of customers switching or discontinuing the service. Customers with high a probability of switching can then be worked with to make sure they remain with the current provider.

Data used from this analysis is available on Kaggle. A cleaned copy is available at GitHub link. It has several different pieces of information about customers. Using this data predictive model for customer churn can be created. And it can then be used to understand which customers should be worked on for retention.

What will we do?

The overall problem can be summarized:

1. Creation of a predictive model using the available customer churn data to predict and find customers likely to discontinue the service.

2. The final prediction outcome for any particular customer should be a “Yes” or a “No” ( a binary output )

Based on the result of the predictions, the company can choose to take appropriate action in the form of various customer retention strategies and reduce the churn. In the next section, we will find out which method can be used to model the scenario to predict a binary outcome. We will also see how the predictive model works and under what assumptions it should be applied.

K-Nearest Neighbors

Based on the problem summary, we need a predictive model that can do a binary classification or predict Yes/No or 1/0 type of output variable. In one of my previous articles, I used Logistic Regression as the predictive model for customer churn analysis (here). In this article, we will be using another method, KNN or k-nearest neighbors. The KNN classification algorithm is often used for classifying the data, as well as for regression on continuous numeric data. It is used for not just binary classification but for multilevel classifications too. It is a non-parametric machine learning method implying that it doesn’t make any assumptions about the data. It also doesn’t make any generalizations, and simply checks the neighboring data points to determine the classification of unknown or uncategorized data points. In this task, we will be considering the classification aspect of KNN, not the regression part.

The KNN machine learning method works on the distance between the data points and is based on the basic premise that similar data points are close to each other. It is widely used in customer recommendation systems, image recognition, and decision-making models. If your neighbors are categorized in a certain way, you will likely fall under the same category too based on the virtue of you being near to them. Conversely, those who are not near but farther away are likely to fall under a different classification than yours. There can be exceptions (points belonging to a different category being near to points in a different category) and consequently, there can be errors in the predictions based on distance, especially around the decision boundaries. We can implement the KNN model and check how accurately it can predict the classification of points. Think of these data points as points in an n-dimensional space and the KNN algorithm predicting their categories based on the distance between the uncategorized data points and their k nearest neighbors, where k is a parameter.

KNN Algorithm in 2D Space

As shown in the image above, gray color points are predicted as red and blue. A predictive algorithm creates a decision boundary or a contour (high-dimension space) and depending on which side of the boundary or contour a point falls, a prediction is made. Decision boundary shows which points will be predicted as red or blue. And this decision boundary changes as variable k changes.

By varying the value of k, we can get a different boundary and that boundary may be better at predicting the outcomes. Choosing a value of k which results in higher prediction accuracy, is a part of parameter tuning. There are many different parameters to the KNN model, just like k and we will use different values of these parameters to determine a set of values best suited for the customer churn data set.

Data Preparation

Data preparation starts with an understanding of the available data for analysis. Customer churn data has 50 fields. One of the important tasks is to determine which fields can be used for regression analysis. There are categorical data fields like Martial, Gender, etc., and continuous numeric data fields like Tenure, Age, etc. Some of these fields may not be important for analysis such as customer ID, interaction, and UID (which are related to customer service interactions). Some other fields which are not important for analysis are the Latitude and Longitude of the customer, case order (used as a serial number). We will also check data for null values and if found they will need to be handled appropriately.

Dummy Variables and Normalization

Categorical data fields, such as gender, internet service, phone service, etc. will have to be converted to corresponding columns containing either 0 or 1. Categorical data like Yes/ No, good/bad/ugly can’t be used for mathematical operations. Hence, these columns containing categorical data will end up becoming columns with 0 or 1 entries.

For continuous numeric data, the scale of data for every column is different. For example, the age column has values ranging from 10 to 89, but the salary column has figures running in 10s or 100s of thousands. There is a big difference in the scale of these variables. Before we can use this data for analysis, it needs to be normalized. Standardization on the data points is even more important for the KNN algorithm, as the processing is based on the distance between data points. The accuracy of prediction increases when these points are plotted using similarly scaled dimensions, instead of dimensions that vary greatly (orders of magnitude).

Using both scalars, MinMaxScalar and StandardScalar for comparison purposes

Model Creation

We have the following variables in customer churn data after the initial data cleanup. There are many categorical variables include the Churn variable which will be used as the dependent variable. All other variables will be used as predictor variables.

Categorical variables:

‘Churn’, ‘Area’, ‘Marital’, ‘Gender’, ‘Techie’, ‘Contract’, ‘Port_modem’, ‘Tablet’, ‘InternetService’, ‘Phone’, ‘Multiple’, ‘OnlineSecurity’, ‘OnlineBackup’, ‘DeviceProtection’, ‘TechSupport’, ‘StreamingTV’, ‘StreamingMovies’, ‘PaperlessBilling’, ‘PaymentMethod’

Continuous Numeric variables:

‘Population’, ‘Children’, ‘Age’, ‘Income’, ‘Outage_sec_perweek’, ‘Email’, ‘Contacts’, ‘Yearly_equip_failure’, ‘Tenure’, ‘MonthlyCharge’, ‘Bandwidth_GB_Year’

Steps to analyze this data using a KNN classification algorithm:

1. Create dummy variables for categorical variables

2. Standardize continuous numeric data using StandardScalar

3. Create 2 data sets:

a. With all of the predictor variables or independent variables (X)

b. With the dependent variable Churn (y)

4. Split data set into train and test set (70% data for training the model and 30% for testing it)

5. Run the KNN model without any parameters (default) and check the accuracy and AUC statistics

6. Run Logistic Regression for comparing the model with the KNN model

7. Run parameter tuning and check the improvement in model performance

Analysis of KNN Model

The performance of a classification model can be assessed by accuracy and AUC (area under the curve). Accuracy for the binary prediction outcome can be computed from the confusion matrix. The confusion matrix tells us how many 0s were predicted as 0s, how many 0s were predicted as 1s, how many 1s were predicted as 0s, and how many 1s were predicted as 1s. Accuracy score can be computed as:

Accuracy Score = (# of True Positive+ # of False Negative) / (# of True Positive+ # of False Negative + # of True Negative+ # of False Positive)

Initial model parameter details:

{'algorithm': 'auto', 'leaf_size': 30, 'metric': 'minkowski', 'metric_params': None, 'n_jobs': -1, 'n_neighbors': 5, 'p': 2, 'weights': 'uniform'}

Confusion Matrix:

[[2018 183]

[ 341 458]]

AUC: 0.8253

As you can see, the model has approximately 82% accuracy in predicting customer churn. We already improved the accuracy by using MinMaxScalar for data standardization. Now, in the next step, we will tune various other parameters to arrive at a model better at predicting the customer churn.

Parameter Tuning

The KNN model takes several parameters. In this step, we will be working with different parameters and running various models for comparison to find a good set of parameters with the best possible accuracy score on the data.

1. One of the parameters for the KNN model is p. Value of p determines what type of distance should be used between the points for finding the nearest neighbors. The distance can be Euclidean, Manhattan, or Minkowski. Changing the parameter value from default 2 to p=1, the accuracy and AUC score got better. The new accuracy score came out to be 0.8266 with an AUC of 0.865

2. The parameter “n_neighbors” determines how many nearest neighbors should be used for consideration before predicting uncategorized data points. The default value of this parameter is 5.

3. The parameter “weights” determine, if all points should be weighed equally or if closer neighbors have a higher weight

4. The parameter “algorithm” determines which algorithm to compute the nearest neighbors. Values can be “auto” when it automatically determines the algorithm or a specific model like BallTree, KDTree, or brute force search, etc.

5. The parameter leaf_size is used for ball_tree or KDtree algorithms.

It is not possible to run the model comparison manually over 100s or 1000s of combinations of these parameter values. Hence, we will use GridSearchCV from sklearn. model_selection to run all the various models and determine the best model performance.

Based on the output from GridSearchCV, the following model seemed to be the best for customer churn predictive analysis using KNN.

{'algorithm': 'auto', 'leaf_size': 1, 'metric': 'minkowski', 'metric_params': None, 'n_jobs': -1, 'n_neighbors': 18, 'p': 1, 'weights': 'distance'}

Confusion Matrix:

[[2110 91]

[ 340 459]]

Accuracy Score: 0.8563

AUC: 0.918

Comparison of KNN models with Logistic Regression Model

For comparison purposes, I ran the same predictive analysis using Logistic Regression and got model performance statistics to show how these models compare.

We can see that these different models overall have a high degree of accuracy. Logistic regression seems to be much better at predicting 1s as 1s, meaning predicting the customer who would churn. This percent is close to 80% for Logistic regression while the number is <60% for KNN models. As this metric is critical to find which customers may discontinue the service, the logistics regression model seems to provide a better prediction model about those customers compared to the KNN model.

Model Summary and Interpretation

We have created a pretty accurate predictive model, with approximately 85% accuracy in predicting the behavior of the customers using KNN. The model can find out if a customer planning to disconnect with approximately 56% accuracy. AUC score of the final model is 0.916 which is very close to a perfect 1. With this model, the company can now have great insight into which customers may discontinue their service.

Our final confusion matrix is provided below. This means that 2110 customers who were predicted to continue with the company, actually continued. A total of 91 customers were predicted to continue with the company, but they discontinued the service. Also, 459 customers who were predicted to discontinue the services, actually discontinued. And 340 customers who were predicted to continue with the company, actually discontinued.

Confusion Matrix:

[[2110 91]

[ 340 459]

Provided below is the ROC curve for the final KNN model and a comparison with the logistic regression ROC curve. Notice how the logistic regression curve has more AUC (area under the curve)than KNN, hence slightly more accurate.

Logistic Regression Vs KNN — AUC

An AUC of 0.5 is as good a random guess. An AUC of 1 shows that predictions can be done with 100% accuracy. Also, an AUC of 0 would mean that all the predictions by the model are wrong (all 0s predicted as 1s and vice versa).

Notably, Logistic regression is much better at predicting the customers likely to disconnect the service. Unlike, a logistic regression model, the KNN model does not tell us the impact of various variables on the churn. We don’t have model coefficients using which we can determine the significance of variables in determining customer churn. Rather the classification is based on nearest neighbors of the unpredicted data points, being categorized in a certain way leading to a prediction for the data point. Using just the output of the KNN model, it is hard to come up with recommendations for the company to implement which will result in lower customer churn.

Further analysis should be done on the data to understand the variables which have a higher impact on churn. For example, the cost is always an important factor in the decision of discontinuing or continuing service. Higher costs and increasing costs over time can move the customers to a point when they would switch in search of better prices and better quality of service. For a better analysis of which dependent variables have a significant impact on churn, the company should do data analysis using logistic regression or other methods and based on the outcomes, work on the elimination of issues that lead to customer churn in the respective areas which those variables represent.