Survival Analysis To Understand Customer Retention | by Rashid Kazmi, Ph.D. | Towards Data Science

Customer retention is an increasingly pressing issue in today’s ever-competitive commercial arena. Companies are eager to develop a customer retention focus and initiatives to maximise long-term customer value. Specifically, the importance of customer retention; conceptualises an integrated customer value/retention model; and explains how usage segmentation can assist in relationship-building, retention strategy and profit planning. I am proposing a solution that integrates various techniques of customer data analysis, modelling and mining multiple concept-level associations to form an intuitive and novel approach to gauging customer loyalty and predicting their likelihood of defection. Immediate action triggered by these “early-warnings’’ resulting this could be the key to eventual customer retention
At the heart of any contractual or subscription‐oriented business model is the notion of the retention rate. An important managerial task is to take a series of past retention numbers for a given group of customers and project them into the future to make more accurate predictions about customer tenure, lifetime value, and so on. Thus, Customer lifetime value (LTV) is one of the cornerstones of database marketing. Customer Lifetime Value is usually defined as the total net income a company can expect from a customer (Novo 2001). The exact mathematical definition and its calculation method depend on many factors, such as whether customers are “subscribers” (as in most online subscription products) or “visitors” (as indirect marketing or e-business).
In this article, I will discuss the calculation and business uses of Customer Lifetime Value (LTV). The Business Intelligence Unit of the CRM any company which tailors analytical solutions to business problems, which are a high priority in the industry: chum and retention analysis, fraud analysis, campaign management, credit and collection, risk management and more. LTV plays a major role in several of these applications, in particular, Churn analysis and retention campaign management. In the context of churn analysis, the LTV of a customer or a segment is important complementary information to their churn probability, as it gives a sense of how much is really being lost due to churn and how much effort should be concentrated on this segment. In the context of retention campaigns, the main business issue is the relation between the resources invested in retention and the corresponding change in LTV of the target segments.
In general, an LTV model has three components: customer’s value over time, the customer’s length of service and a discounting factor. Each component can be calculated or estimated separately or their modelling can be combined. When modelling LTV in the context of a retention campaign, there is an additional issue, which is the need to calculate a customer’s LTV before and after the retention effort. In other words, we would need to calculate several LTV’s for each customer or segment, corresponding to each possible retention campaign we may want to run (i.e. the different incentives we may want to suggest). Being able to estimate these different LTV’s is the key to a successful and useful LTV application.
Let’s get with a quick motivation and the question that sometimes I do get asked what is the average subscription length and how long customers are at the company. For example, we have nine customers and the bars are tracking their average subscription lengths. The green bars are the customer that is still active and the red bars are the customer that are no longer active customers.
So say what’s the average subscription length? One question that needs to be asked, however, is whether just take the average of all of these customers in the CRM database or customer that are still active?
What if one takes average subscription length next month, probably going to to get totally different. Essentially, its s a moving target we are trying to look at. The second scenario can be one just ignore the active people and just take the inactive people and look at the average of that. This also does not resolve the problem as well because again some customer will become inactive. Thus, we are massively biasing our dataset so the customer who’ve already cancelled so neither way of taking the straightforward out which really gives us what we want. Taken together, these discussions suggest that, we’ve got a sense of data where we haven’t actually observed the endpoint that we’re trying to measure yet so essentially taking an average like this is never really going to be a sensible thing to do so.
To measure the problem mentioned above we need survival analysis for estimating the time to an event for a particular population when you may not have all you know see all the events happen it’s all your data points. Survival analysis is really quite an old idea in statistics and it’s used quite a lot so, for example, in medical statistics, not a very cheery example to start with. Where survival rates after cancer and the probability that people are surviving five-ten years are all survival analysis. Going back to what we care about is our customers and what we’re looking at is from when customers have signed up and started and moved off their free trial on to pay for subscription and how long do we think they’re going to be open to using the service which obviously then goes into the lifetime value side of it.
To begin with, its good idea to walk through some of the definition to understand survival analysis conceptually. We’ll take care of capital T which is the time to a subscription end for a customer. This is obviously greater than zero. However, it could be infinite if the customer never churns. This is quantity we care about and this will help us to understand lifetime value which is basically the probability that the customer hasn’t churned at any day T into the future. This is non increasing function. The overall probability is also important which is what would happen on each infinitesimal day. We define this by the hazard function which is the probability that on any given day T i.e. the exact day the customer churn not one day before not one day afterwards. These two things as you expect are related but it was just like a little bit of elementary probability you can show that essentially the survival function is expressed in terms of the hazard function in a relatively simple way.
In Python, we’ve got two main package lifelines and scikit-survival package. Lifelines are longer standing package and are very lightweight. Nonetheless, it also offers really nice visualisations. Both of these are based on their scikit-learn. One can use them how you’d use any scikit-learn package and put it in pipelines.
The dataset — here we used customer churn. The idea is to use data to walk the reader through the full cycle of customer retention with a data science perspective. We have started with understanding the business perspective of the problem. Then, we will use the available data set to gain insights and build a predictive model for use with future data.
We’ve got a lot of categorical data so particularly stuff like a partner, dependent, contract etc. There is standard one hot entertaining approach where you just turn it into like n or n minus one binary feature based on category. This would be great for X if you remember how cox model looks: it means we’d have a coefficient attached to every single categorical variable. So we will get really good clarity on the impact of each individual category. This would result in a massive data set as we have got the curse of dimensionality.


HTML




CSS




Result



Skip Results Iframe

<script src="https://gist.github.com/rhkaz/f6afe75ab2151a4a3a91f98b396ab030.js"></script>
/* https://github.com/lonekorean/gist-syntax-themes */
@import url('https://cdn.rawgit.com/lonekorean/gist-syntax-themes/848d6580/stylesheets/monokai.css');

@import url('https://fonts.googleapis.com/css?family=Open+Sans');
body {
  margin: 20px;
  font: 16px 'Open Sans', sans-serif;
}



This Pen is owned by Rashid Kazmi on CodePen.



See more by @rhkaz on CodePen



This Pen doesn't use any external CSS resources.


This Pen doesn't use any external JavaScript resources.