An RFM Analysis with Python
By: Wenling Yao
Tags: Rfm, Engagement, Segmentation, Python, CRM

Convert absolute values into relative ones to reduce the bias of customer lifetime.

One thing I noticed in many tutorials is that absolute numbers are used for the calculation. In other words, we have the following:

Recency = the interval between the last action and today

Frequency = the number of actions between the user’s signup date and today

Monetary value = the sum of revenue within the user’s lifetime

After this step, one typical approach is to calculate the Recency/Frequency/Monetary Score by creating buckets based on the quartiles of each individual column. For example, on the column “Frequency” we do the following:

customer_data["FrequencyScore"] = pd.cut(customer_data["Frequency"],

bins=[-1,

np.percentile(customer_data["Frequency"], 25),

np.percentile(customer_data["Frequency"], 50),

np.percentile(customer_data["Frequency"], 75),

customer_data["Frequency"].max()],

labels=[1, 2, 3, 4]).astype("int")

Note that for Recency Score we need to give inverse labels as the more active the customer is the lower the value of Recency. :)

Once we obtain the scores of each individual dimension, we calculate the overall RFM score by summing up the three scores. The higher the overall RFM score (in a range from 3 to 12), the more valuable the customer.

customer_data["RFM"] = customer_data["RecencyScore"] + customer_data["FrequencyScore"] + customer_data["MonetaryScore"]

This method is easy to implement, yet it is prone to bias associated with the actual customer lifetime. For example, compared to the customers who sign up in the last 6 months, those who signed up 12 months ago would by nature have lower recency (assuming generally decreasing user engagement over lifetime), higher frequency, and higher monetary value (assuming cohorts are homogenous in terms of user activity and monetization). Hence, the RFM score of customers who signed up 12 months may be overestimated.

To account for this bias, I would first normalize Recency, Frequency, and Monetary Value before calculating individual scores. For example, I would do Relative Frequency = Recency / Lifetime.

customer_data["RelFrequency"] = customer_data["Frequency"] / customer_data["Lifetime"]

Then I would calculate the Frequency Score based on the Relative Frequency:

customer_data["FrequencyScore"] = pd.cut(customer_data["RelFrequency"],

bins=[-1,

np.percentile(customer_data["RelFrequency"], 25),

np.percentile(customer_data["RelFrequency"], 50),

np.percentile(customer_data["RelFrequency"], 75),

customer_data["RelFrequency"].max()],

labels=[1, 2, 3, 4]).astype("int")

In my notebook, you can find rfm_abs the RFM segmentation using absolute values and rfm_rel using relative values. Here I attach only the median customer lifetime of each RFM segment under these two approaches. As we can see, the segments from the RFM segmentation using absolute values have a larger variance in customer lifetime whereas those from the RFM segmentation using relative values have rather a constant customer lifetime. More specifically, with the 1st approach we see that those with an RFM score of 5 and 6 (thus at the low bound of activity) have a significantly shorter lifetime (and thus are more recent customers). This approach seems to “penalize” the more recent customers for their lower frequency and monetary amount.

On the other hand, our approach manages to reduce the bias associated with customer lifetime in RFM segmentation!