Find Your Best Customers with Customer Segmentation in Python
By: Susan Li
Tags: Machine Learning, Python, Customer Success, Data Science, Towards Data Science

Overview

When it comes to finding out who your best customers are, the old RFM matrix principle is the best. RFM stands for Recency, Frequency and Monetary. It is a customer segmentation technique that uses past purchase behavior to divide customers into groups.

RFM Score Calculations

RECENCY (R): Days since last purchase

FREQUENCY (F): Total number of purchases

MONETARY VALUE (M): Total money this customer spent

Step 1: Calculate the RFM metrics for each customer.

Source: Slideshare

Step 2: Add segment numbers to RFM table.

Source: Slideshare

Step 3: Sort according to the RFM scores from the best customers (score 111).

Source: Blast Analytics Marketing

Since RFM is based on user activity data, the first thing we need is data.

Data

The dataset we will use is the same as when we did Market Basket Analysis — Online retail dataset that can be downloaded from UCI Machine Learning Repository.

import pandas as pd

import warnings

warnings.filterwarnings('ignore') df = pd.read_excel("Online_Retail.xlsx")

df.head()

df1 = df

The dataset contains all the transactions occurring between 01/12/2010 and 09/12/2011 for a UK-based and registered online retailer.

It took a few minutes to load the data, so I kept a copy as a backup.

Explore the data — validation and new variables

Missing values in important columns; Customers’ distribution in each country; Unit price and Quantity should > 0; Invoice date should < today.

df1.Country.nunique()

38

There were 38 unique countries as follows:

df1.Country.unique()

array([‘United Kingdom’, ‘France’, ‘Australia’, ‘Netherlands’, ‘Germany’,

‘Norway’, ‘EIRE’, ‘Switzerland’, ‘Spain’, ‘Poland’, ‘Portugal’,

‘Italy’, ‘Belgium’, ‘Lithuania’, ‘Japan’, ‘Iceland’,

‘Channel Islands’, ‘Denmark’, ‘Cyprus’, ‘Sweden’, ‘Austria’,

‘Israel’, ‘Finland’, ‘Bahrain’, ‘Greece’, ‘Hong Kong’, ‘Singapore’,

‘Lebanon’, ‘United Arab Emirates’, ‘Saudi Arabia’, ‘Czech Republic’,

‘Canada’, ‘Unspecified’, ‘Brazil’, ‘USA’, ‘European Community’,

‘Malta’, ‘RSA’], dtype=object

customer_country=df1[['Country','CustomerID']].drop_duplicates() customer_country.groupby(['Country'])['CustomerID'].aggregate('count').reset_index().sort_values('CustomerID', ascending=False)

More than 90% of the customers in the data are from the United Kingdom. There’s some research indicating that customer clusters vary by geography, so here I’ll restrict the data to the United Kingdom only.

df1 = df1.loc[df1['Country'] == 'United Kingdom']

Check whether there are missing values in each column.

There are 133,600 missing values in the CustomerID column, and since our analysis is based on customers, we will remove these missing values.

df1 = df1[pd.notnull(df1['CustomerID'])]

Check the minimum values in UnitPrice and Quantity columns.

df1 = df1[pd.notnull(df1['CustomerID'])]

0.0

df1.Quantity.min()

-80995

Remove the negative values in Quantity column.

df1 = df1[(df1['Quantity']>0)]

df1.shape

df1.info()

(354345, 8)

After cleaning up the data, we are now dealing with 354,345 rows and 8 columns.

Check unique value for each column.

def unique_counts(df1):

for i in df1.columns:

count = df1[i].nunique()

print(i, ": ", count)

unique_counts(df1)

InvoiceNo : 16649

StockCode : 3645

Description : 3844

Quantity : 294

InvoiceDate : 15615

UnitPrice : 403

CustomerID : 3921

Country : 1

Add a column for total price.

df1['TotalPrice'] = df1['Quantity'] * df1['UnitPrice']

Find out the first and last order dates in the data.

df1['InvoiceDate'].min()

Timestamp(‘2010–12–01 08:26:00’)

df1['InvoiceDate'].max()

Timestamp(‘2011–12–09 12:49:00’)

Since recency is calculated for a point in time, and the last invoice date is 2011–12–09, we will use 2011–12–10 to calculate recency.

import datetime as dt

NOW = dt.datetime(2011,12,10) df1['InvoiceDate'] = pd.to_datetime(df1['InvoiceDate'])

RFM Customer Segmentation

RFM segmentation starts from here.

Create a RFM table

rfmTable = df1.groupby('CustomerID').agg({'InvoiceDate': lambda x: (NOW - x.max()).days, 'InvoiceNo': lambda x: len(x), 'TotalPrice': lambda x: x.sum()}) rfmTable['InvoiceDate'] = rfmTable['InvoiceDate'].astype(int)

rfmTable.rename(columns={'InvoiceDate': 'recency',

'InvoiceNo': 'frequency',

'TotalPrice': 'monetary_value'}, inplace=True)

Calculate RFM metrics for each customer

Interpretation:

CustomerID 12346 has frequency: 1, monetary value: $77,183.60 and recency: 325 days.

CustomerID 12747 has frequency: 103, monetary value: $4,196.01 and recency: 2 days

Let’s check the details of the first customer.