Level-up your Kaplan-Meier curves with Tableau | by Eryk Lewinson | Towards Data Science

In a previous article, I showed how we can create the Kaplan-Meier curves using Python. As much as I love Python and writing code, there might be some alternative approaches with their unique set of benefits. Enter Tableau!
Tableau is a business intelligence tool used for creating elegant and interactive visualizations on top of data coming from a vast number of sources (you would be surprised how many distinct ones are there!). To make the definition even shorter, Tableau is used for building dashboards.
So why would a data scientist be interested in using Tableau instead of Python? When creating a Notebook/report with the results of a survival analysis exercise in Python, the reader will always be limited to:
what the creator of the visualization had in mind,
what data was available at the moment of creating the report.
In other words, there is little freedom for the reader to explore some alternative angles. What is more, if someone in the company will accidentally find the report a few years later, the only way to make the analysis up-to-date would be to find the data scientist and make them rerun the Notebook and generate another report. Definitely not the best situation.
This is where a solution based on Tableau (or other business intelligence tools such as PowerBI, Looker, etc.) shines. As the visualizations are built directly on top of a data source, the visualization will be updated together with the data. Less work for the data scientist!
Another extra benefit is the possibility to include some filters, so the readers can play around and try to explore different subsets of the data. From experience, this is a feature often used by product owners, who want to dive deep into the details and at the same time do not want to constantly come to the data person with another request for a new filter or feature. Another win :)
Lastly, by using such tools, the analysts democratize the access to the data and analyses, as basically anyone in the company can access the dashboard and try to answer their own questions or verify their hypotheses.
After this introduction, let’s jump right into re-creating the very same Kaplan-Meier curves we created in the previous article. Once again, we use the Telco Churn dataset, which requires close to no extra preparation before the analysis. Please refer to that article if you need a refresher on the Kaplan-Meier estimator, as we will not cover theory this time. Also, we assume some basic knowledge of Tableau.
Note: Tableau is a commercial software and requires a license. You can get access to a 14-day trial by following the instructions here.
The first approach is dubbed easy, as it will favor speed and simplicity, while at the same time introducing some shortcomings. First, we load the data from a text file (available here).
To carry out the survival analysis in Tableau, we will need the following variables:
time-to-event — expressed as time periods (for example, days or months) elapsed since joining the sample until the event of interest or censoring.
event-of-interest — expressed as a binary variable, where 1 indicates that the event happened, 0 otherwise.
additional categorical variables — used for filtering and/or grouping.
The tenure variable does not require any preparation, as it already expresses the number of months since signing up for the services of the Telco company. But the Churn variable is expressed as a yes/no string, so we need to encode it to binary using a calculated field:
To create this field, right-click on the Churn variable in the variable selector on the left (Data tab), select Create -> Calculated Field.
As the next step, we create a new calculated field, d_i, which represents the number of events that occur over time:
The names we used for the variables correspond to the elements you can find in the formula for the Kaplan-Meier estimator.
The next variable we create will be the denominator used for calculating the hazard function at a given time. It represents the total number of observations since the last time period:
The Number of Records variable is a helper variable used for, as you might have guessed, counting the observations. For that purpose, newer versions of Tableau create a variable based on the name of the data source. However, you can easily create this variable manually by creating a calculated field and placing 1 in the field’s definition. Lastly, we define the Kaplan-Meier curve as:
Here, the probability of survival is defined as 1 - hazard function.
All the building blocks are ready. Now, we place the tenure on the x-axis, the Kaplan-Meier Curve on the y-axis, format the curve as a percentage, add the tile and place the PaymentMethod variable as a color. This way, we create the following visualization:
Which is very similar to what we obtained last time using lifelines:
Some quick observations:
the survival curves obtained in Tableau are more or less straight, without the characteristic step structure,
there are no confidence intervals, as their calculation is not that simple in Tableau.
Using Tableau, we can easily add some additional filters to the visualization, such as the cohort date, age, or any of the available categorical variables.
In this approach, we will focus on recreating the characteristic step-like shape of the Kaplan-Meier curves. This approach is dubbed the normal mode, as it requires a bit more preparation.
For the additional data preprocessing, we need to complete two steps. First, add a column called link to the CSV file with the Telco Customer Churn data. The column should be populated with a ‘link’ string. As a matter of fact, this string can be arbitrary, just as the column name. What matters is consistency, but all will become clear in a second. The second step is to create a new CSV file (we called it blending.csv), which contains the following:
link, setlink, 1link, 2
Yep, that’s pretty much it. For your convenience, I stored both files on my GitHub.
Armed with the two files, we load them to Tableau and left join the tables using the link variable. You can see that in the following image.
As this is the “normal mode”, we will combine a few steps at the same time and create a calculated field called Kaplan-Meier Dots:
You can easily recognize the contents of this field from the “easy mode”, this time, we have put everything into one field. After doing so, comes the new part. We define the Kaplan-Meier Curve as:
This convoluted formula will enable us to obtain the step-like shape of the curves. Lastly, we need one more helper variable:
When doing so, please click on the Default Table Calculation and specify to compute the results along tenure.
Finally, we have all the building blocks to create the curves. We approach the setup similarly to the “easy mode”, with the difference of placing the Index as the Path and set as the Detail. To recreate the curves from Python, we once again use the PaymentMethod as the Color.
In the picture above, we accurately recreated the curves we previously obtained using the lifelines library in Python. This definitely required a bit more work but can pay off in the end.
We can additionally use the Kaplan-Meier Dots to visualize the events as they happen along the curve. In this case, I believe this would simply clutter the visualization. It would be more suitable for a smaller dataset.
We can further improve the dashboard by adding some filters/splits and then share it with our colleagues via the company’s reporting portal (in this case, an instance of Tableau Server).
In this article, I explained the potential benefits of using business intelligence tools such as Tableau for survival analysis and showed how to create dashboards with the Kaplan-Meier curves.
As is often the case, nothing comes for free and there are also some disadvantages to this approach:
Calculating the confidence intervals is definitely harder and needs quite some effort.
In Tableau, there is no simple way to carry out the log-rank test to compare different survival curves (unless we use R from Tableau, but this might be an idea for a future article).
If new features are added to the data, for example, new customer segmentation or another category for each observation, this will still require some work from an analyst to add to an already existing dashboard. However, most of the time this does not happen often or requires little extra work.
I hope you enjoyed this alternative approach to visualizing the Kaplan-Meier curves. As always, any constructive feedback is welcome. You can reach out to me on Twitter or in the comments.
If you liked this article, you might also like the other ones in the series:
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
[1] Kaplan Meier Survival Test complete solution (without R) — available here