Credit Card Customer Clustering with K-means | by Luke Sun | Towards Data Science

In a previous post, we attempted to detect fraud in credit card transactions using various models. Here we will turn to another popular topic - Customer Clustering. This post aims to help friends in the marketing department. As usual, it is split into 5 parts.
Problem statementData ReviewData processingk-means clusteringTakeaways
Problem statement
Data Review
Data processing
k-means clustering
Takeaways
Now let’s begin the journey 🏃‍♀️🏃‍♂️.
1. Problem statement
Marketing, as well known, is crucial for the growth and sustainability of any business. However, one of the key pain points for any marketing professionals is to know the customers and identify their needs. Tasked by colleagues in the marketing department, your job is to create a model to perform customer grouping.
2. Data Review
The dataset contains 17 features and 8,950 records. As illustrated in Fig.1, it is about customer’s purchase and payment habits, such as how often a customer makes one-off or instalment purchases, or how often they make cash advances, how much payments are made, etc. By inspecting each customer, we can find which type of purchase he/she is keen on, or if he/she prefers cash advance over purchases.
3. Data processing
Generally, to get a deep understanding and preliminary findings on any raw data, we perform exploratory data analysis (EDA). I covered this topic and the general steps in several articles. If you are a newbie on EDA, have a look at my previous articles. Here we will focus on some steps necessary for the following modelling.
3.1 Fill missing values
Due to various reasons, missing values commonly appear in raw data. To check how many missing values in each column,
creditcard_df = pd.read_csv(‘Marketing_data.csv’)creditcard_df.isnull().sum()
As shown in Fig.2, the column ‘CREDIT_LIMIT’ has 1 missing value, and ‘MINIMUM_PAYMENTS’ has 313 missing values.
There are many approaches to dealing with missing values, including:
Drop columns with missing values
Imputation, filling in missing values with some number, such as the mean value along each column.
An extension to Imputation. Imputation is the standard approach. However, imputed values may be above or below their actual values. To help model make a better prediction, we can add a column to tag the missing value as True or False.
In this case, we will fill the missing values the mean of the column. Specifically,
creditcard_df.loc[(creditcard_df[‘MINIMUM_PAYMENTS’].isnull() == True), ‘MINIMUM_PAYMENTS’] = creditcard_df[‘MINIMUM_PAYMENTS’].mean()creditcard_df.loc[(creditcard_df[‘MINIMUM_PAYMENTS’].isnull() == True), ‘MINIMUM_PAYMENTS’] = creditcard_df[‘MINIMUM_PAYMENTS’].mean()
3.2 Drop features
The dataset contains 17 features. Now we need to consider if all of them is necessary to train the model. In our case, ‘CUST_ID’ is of no help. So we will drop it.
creditcard_df.drop(‘CUST_ID’, axis = 1, inplace = True)
3.3 Scale features
Some features such as ‘PURCHASES’ vary in a wide range, while other features such as ‘PURCHASES_FREQUENCY’ vary between 0 to 1. We need to scale all features on the same range. Here we use StandardScaler() from sklearn to remove mean and scale to unit variance.
scaler = StandardScaler()creditcard_df_scaled = scaler.fit_transform(creditcard_df)
4. k-means clustering
k-means clustering is an unsupervised machine learning algorithm. According to Wikipedia, it aims to partition the observations into k sets so as to minimize the within-cluster sum of squares (WCSS). WCSS represents the sum of distances of all points to the centroid in a cluster. It starts with a group of randomly initialized centroids and then performs iterative calculations to optimize the position of centroids until the centroids stabilize, or the defined number of iterations is reached.
4.1 Find K
The first step in k-means is to pick the number of clusters. The elbow method is one of the most popular methods. To implement it, we apply k-means with a different number of clusters and compare their WCSS. Specifically,
score_1 = []range_values = range(1, 20)for i in range_values:    kmeans = KMeans(n_clusters = i)    kmeans.fit(creditcard_df_scaled)    score_1.append(kmeans.inertia_)
Fig.3 shows the WCSS variation with the number of clusters. To determine the optimal number of clusters, we have to select the value of k at the “elbow” i.e. the point after which the distortion/inertia start decreasing in a linear fashion. Thus, we conclude that the optimal number of clusters is 8.
4.2 Apply k-means
Having determined the optimal number of clusters, let’s apply k-means and predict each sample’s cluster. Specifically,
kmeans = KMeans(n_clusters = 8, init = ‘k-means++’, max_iter = 300, n_init = 10, random_state = 0)labels = kmeans.fit_predict(creditcard_df_scaled)
Note we set the number of time that k-means will be run to be 10. This means k-means will run 10 times with different centroid seeds, and the final results will be the best output in terms of WCSS. In addition, we set the maximum number of iterations of k-means for a single run to be 300.
4.3 Visualize Clusters
With the prediction results, the first visualization is to review the feature distribution for each cluster.
For instance, let’s take cluster 4 for analysis. As seen in Fig.4, compared to other clusters, we find customers in cluster 4 tend to have a high account balance, and update the balance very often, but the amount of their purchase is very skewed towards 0. This indicates the customers are keener on the cash advances, as shown in Fig.5.
As a credit card user, myself, I regularly make purchases but am not a fan of cash advances. So, I find myself may fit into cluster 0, which represents customers who make a frequent one-off purchase but are not keen on instalment and cash advances. But if you make a lot of instalment purchases and not cash advances, you may fit into cluster 2.
Personally, I find this is so much fun to analyse how different groups of customers behave on their transactions ✨✨. Maybe have a try and see where you fit into 📣📣?
4.4 PCA
Obviously, the histogram chart is great to analyse the cluster features, but not so great to visualize the overall cluster distribution. Because we have 17 features, we have to compress the dimensions of freedom to visually observe the cluster distribution. To do such, we use principal component analysis (PCA) to compress features into a 2D space.
For those who are not familiar with PCA, PCA is commonly used for dimensionality reduction by projecting data only onto the first principle components to obtain low-dimension data while preserving or maximizing the variances along the projected direction (Btw, this is from Wikipedia 🤭).
So, to implement PCA,
from sklearn.decomposition import PCApca = PCA(n_components = 2)principal_comp = pca.fit_transform(creditcard_df_scaled)pca_df = pd.DataFrame(data = principal_comp, columns = [‘pca1’, ‘pca2’])pca_df = pd.concat([pca_df, pd.DataFrame({‘cluster’: labels})], axis = 1)
Finally, with 17 features compressed to 2 features, Fig.6 shows the scatter plot of the k-means prediction results. You may notice the red cluster is somehow scattered among the blue cluster, pink cluster and green cluster. This may indicate the red cluster has feature similarity with the others. Maybe we can recreate a new cluster with a new technique 🤔🤔?
5. Takeaways
We created a k-means model that managed to segment credit card customers into 8 groups. We found some interesting purchases and payment patterns for cluster 0, 2 and 4, and more remains to be explored. We also noticed some overlaps between clusters, indicating there are spaces to improve.
Great. Hope you find this journey an interesting one. To continue the topic, I will dig further to see if we can improve the clustering results in the next post. See you there 💕💕.