Cohort Analysis Visualization with R | by Pararawendy Indarjo | Towards Data Science

If you are a data practitioner in the B2C (Business to Consumer) industry, or simply a data-geek who has analyzed some consumer data, chances are you must be familiar with cohort analysis.
Cohort analysis is an analytic method where we first group individuals who belong to the dataset into different cohorts with specific time-span, within which the individuals in the same cohort group shared the same certain experience. An appropriate analysis is then carried out, taking these cohorts as the unit of analysis.
Among other use cases, cohort analysis allows us to look at the movement of user retention in performing a specific metric over time. This is done by comparing the performance of the cohorts on the basis of their ages.
In this article, I will share how to create cohort analysis visualization. To this end, we will use R and its popular ggplot2 package. There are two types of cohort analysis visualization that will be shown: line plot and heatmap.
The data used is relay-food data available at this Github repository. The data contains the order details of a certain food retailer. Throughout this article, we are only interested to visualize the user purchasing cohort on a monthly basis. Therefore, we will only consider two columns of the data as follows:
OrderDate : the date when the order occurredUserId : the user ID
OrderDate : the date when the order occurred
UserId : the user ID
As always, the first thing to do is to import the necessary libraries.
# import librarieslibrary(dplyr) #handy data manipulationlibrary(ggplot2) #our today's super starlibrary(stringr) #to manipulate string datelibrary(ggthemes) #many nice themeslibrary(mdthemes) #handy text in plot formattinglibrary(gghighlight) #will abuse it a bit to show nice label
Loading the data comes next.
#load the datadf = read.csv("relay-foods.csv")#inspect first 5 rows of the datadf %>% select(OrderDate, UserId) %>% head(5)
Apparently, the date format in the columnOrderDate is not standard yet. Therefore we need to adjust it.
# adjust date formatting of OrderDate columndf$OrderDate = format(as.Date(df$OrderDate, '%m/%d/%Y'), '%Y/%m/%d')df$OrderDate = gsub('00','20',df$OrderDate)
Next, since we want to look at the monthly cohort, we need to derive OrderMonth column from OrderDate column.
# create OrderMonth column from OrderDate (to  make monthly cohort)df$OrderMonth = str_sub(df$OrderDate,end = 7)
Cohort analysis requires cohort group information for each user in the dataset, so we build one.
# create reference data frame of cohort group (month level)cohort_group_df = df %>% group_by(UserId) %>%                     summarize(CohortGroup = min(OrderDate))cohort_group_df$CohortGroup =  str_sub(cohort_group_df$CohortGroup,                    end = 7) # join with the initial dfdf = inner_join(df, cohort_group_df, by = 'UserId')
Next, we create a reference dataframe containing the total users for each cohort group, as well as the monthly count of users who made purchases.
# create reference data frame of total users for each cohort groupbase_cohort_df = df %>% group_by(CohortGroup) %>%  summarise(   TotalUsers = n_distinct(UserId)  )# create purchase activity data frameactivity_cohort_df = df %>% group_by(CohortGroup, OrderMonth) %>% summarise(  BuyingUsers = n_distinct(UserId) )# join activity_cohort_df and base_cohort_dfuser_cohort_df = inner_join(activity_cohort_df, base_cohort_df,                   by = 'CohortGroup')
Data checkpoint. What we have got so far?
user_cohort_df %>% head(5)
Notice OrderMonth column is still in string format as above. For plotting, we want it to be in integer format to become the x-axis.
# transform OrderMonth to integeruser_cohort_df = user_cohort_df %>% group_by(CohortGroup) %>% mutate(MonthNumber = 1:n())
Finally, we will only plot a subset of the data to avoid overplotting.
# subsetting the dataplot_user_cohort_df = inner_join(base_cohort_df[seq(1,11,2),c(“CohortGroup”)], user_cohort_df, by = “CohortGroup”)
In the plot to be drawn, I wanted to have nice legends where they are put right at the end of the lines. To do so, I use gghighlight package. I know this is somewhat hacky, so please comment below if you guys know a more proper way to do it.
# dummy column for in-place legendplot_user_cohort_df$dummy_col = 1
Alright, we are done with data preparation for the line plot. Below is the final form of the data.
plot_user_cohort_df %>% head(5)
So, we are ready to have our first plot!
# plotting line plotggplot(plot_user_cohort_df) +  geom_line(aes(x = MonthNumber,                y = BuyingUsers/TotalUsers,                col = CohortGroup)) +  gghighlight(dummy_col == 1) +  scale_x_continuous(breaks = seq(from = 1, to = 15, by = 2)) +  scale_y_continuous(labels = scales::percent_format()) +  mdthemes::md_theme_solarized() +  labs(    title = "**Monthly User Purchasing Cohort**",    caption = "*Data: Relay Food order details (Source: github.com/ethen8181)*",     x = "K-th Month",    y = "Retention",    col = "Cohort Group"  )
It can be concluded from the plot that Relay Food tends to experience reduced user retention issues, as the younger cohort retention rate (2009–11) is smaller than that of older cohorts (2009–01).
According to my experience, some people prefer to see a heatmap kind of plot to visualize the cohorts. So let’s draw one too.
Instead of working from scratch, we will continue our work in the above. Particularly we will start upon user_cohort_df .
# create base dataframe for heat map visualizationcohort_heatmap_df = user_cohort_df %>% select(CohortGroup,      MonthNumber, BuyingUsers) %>%          spread(MonthNumber, BuyingUsers)# inspect datacohort_heatmap_df %>% head(5)
We also create the percentage version of the data.
# the percentage version of the dataframecohort_heatmap_df_pct = data.frame(  cohort_heatmap_df$CohortGroup,  cohort_heatmap_df[,2:ncol(cohort_heatmap_df)] / cohort_heatmap_df[["1"]])# assign the same column namescolnames(cohort_heatmap_df_pct) = colnames(cohort_heatmap_df)
Next, we melt both dataframes to please ggplot2.
# melt the dataframes for plottingplot_data_abs = gather(cohort_heatmap_df, "MonthNumber", "BuyingUsers", 2:ncol(cohort_heatmap_df))plot_data_pct = gather(cohort_heatmap_df_pct, "MonthNumber", "Retention", 2:ncol(cohort_heatmap_df_pct))
We prepare labels for the heatmap.
# prepare label names containing absolute number of buyers for the first month and retention percentages for the rest monthslabel_names = c(plot_data_abs$BuyingUsers[1:(ncol(cohort_heatmap_df)-1)],plot_data_pct$Retention[(ncol(cohort_heatmap_df_pct)):(nrow(plot_data_pct))])
Finally, we put everything together into a dataframe ready for plotting.
# beautify percentage labelsbeauty_print <- function(n) {  case_when( n <= 1  ~ sprintf("%1.0f %%", n*100),             n >  1  ~ as.character(n),             TRUE    ~ " ") # for NA values, skip the label}# create dataframe ready for plottingplot_data = data.frame(  CohortGroup = plot_data_pct$CohortGroup,  MonthNumber = plot_data_pct$MonthNumber,  Retention = plot_data_pct$Retention,  Label = beauty_print(label_names))plot_data$MonthNumber = as.numeric(plot_data$MonthNumber)
The final form of the dataframe for heatmap visualization is as follows.
plot_data %>% head(5)
Drum roll! The code for generating the heatmap is as follows.
# plotting heatmapggplot(plot_data) +  geom_raster(aes(x = MonthNumber,                  y = reorder(CohortGroup, desc(CohortGroup)),                  fill = Retention)) +  scale_fill_continuous(guide = FALSE, type = "gradient",                        low = "deepskyblue", high = "darkblue") +  scale_x_continuous(breaks = seq(from = 1, to = 15, by = 1),                     expand = c(0,0)) +  geom_text(aes(x = MonthNumber,                y = reorder(CohortGroup, desc(CohortGroup)),                label = Label), col = "white") +  mdthemes::md_theme_gray() +  labs(    title = "**Monthly User Purchasing Cohort**",    caption = "*Data: Relay Food order details (Source: github.com/ethen8181)*",     x = "K-th Month",    y = "Cohort Group"  )
Notice that the first column of the heatmap displays the absolute number of users who made purchases in corresponding months. Thus, this heatmap version of cohort visualization is somewhat richer than its line plot counterpart since provides more context on the absolute size of the cohort groups.
In this article, we have gone through a step-by-step R code tutorial to generate elegant visualizations of cohort analysis, in the form of a line plot and heatmap. For readers who more enjoy following codes from Github, the full codes in this article are available on my Github repository here.
Finally, thanks for reading, and let’s connect with me on LinkedIn! 👋