Market Segment Analysis Part 1 — Analyzing and Clustering Data
By: Mariana Almeida
Tags: Machine Learning, Data Science, Kaggle, K Means

An application of k-means algorithm together with dimensionality reduction to credit card data

Photo by Mark OFlynn on Unsplash

This is a practice project I did to learn about clustering and dimensionality reduction. Cluster algorithms have a broad list of applications in terms of Data Science studies. One of the main applications is in market research and an example of this will be shown in this project.

Because the analysis is a bit long, it will be split into two posts:

Part 1: Data Analysis and Clustering Part 2: Dimensionality Reduction and final evaluation

The dataset contains credit card data with 18 variables. The variables describe consumer usage behaviours of about 9000 credit card holders during a period of 6 months.

The original dataset in kaggle can be found here. The code for this project can be found here.

The data science skills applied in this project are:

Exploratory data analysis

Unsupervised learning (k-means algorithm)

Principal component analysis (for dimensionality reduction)

Autoencoders (for dimensionality reduction)

Why consider dimensionality reduction?

As you will see later, there are just too many features in this dataset. To implement machine learning problems, one thing to have in mind is that we must choose the best features to represent the data. Having too many features may result in overfitting the model. When a model is overfit, it means the model adjusted too well to the training data which makes it difficult to generalize beyond the examples of the training set.

Our goal as data scientists is to create models that represent well the data and can be applied to new data fed to the model, once it is put into production.

1 — Loading the dataset and getting to know the data

See below a description of all the 18 variables:

CUSTID : Identification of Credit Card holder

: Identification of Credit Card holder BALANCE : Balance amount left in their account to make purchases

: Balance amount left in their account to make purchases BALANCEFREQUENCY : How frequently the Balance is updated, score between 0 and 1 (1 = frequently updated, 0 = not frequently updated)

: How frequently the Balance is updated, score between 0 and 1 (1 = frequently updated, 0 = not frequently updated) PURCHASES : Amount of purchases made from account

: Amount of purchases made from account ONEOFFPURCHASES : Maximum purchase amount done in one-go

: Maximum purchase amount done in one-go INSTALLMENTSPURCHASES : Amount of purchase done in installment

: Amount of purchase done in installment CASHADVANCE : Cash in advance given by the user

: Cash in advance given by the user PURCHASESFREQUENCY : How frequently the Purchases are being made, score between 0 and 1 (1 = frequently purchased, 0 = not frequently purchased)

How frequently the Purchases are being made, score between 0 and 1 (1 = frequently purchased, 0 = not frequently purchased) ONEOFFPURCHASESFREQUENCY : How frequently Purchases are happening in one-go (1 = frequently purchased, 0 = not frequently purchased)

: How frequently Purchases are happening in one-go (1 = frequently purchased, 0 = not frequently purchased) PURCHASESINSTALLMENTSFREQUENCY : How frequently purchases in installments are being done (1 = frequently done, 0 = not frequently done)

: How frequently purchases in installments are being done (1 = frequently done, 0 = not frequently done) CASHADVANCEFREQUENCY : How frequently the cash in advance being paid

: How frequently the cash in advance being paid CASHADVANCETRX : Number of Transactions made with “Cash in Advanced”

: Number of Transactions made with “Cash in Advanced” PURCHASESTRX : Number of purchase transactions made

: Number of purchase transactions made CREDITLIMIT : Limit of Credit Card for user

: Limit of Credit Card for user PAYMENTS : Amount of Payment done by user

: Amount of Payment done by user MINIMUM_PAYMENTS : Minimum amount of payments made by user

: Minimum amount of payments made by user PRCFULLPAYMENT : Percent of full payment paid by user

: Percent of full payment paid by user TENURE: Tenure of credit card service for user

Getting the statistics for all variables with the .describe() command we can note the following:

On average, clients maintain 1564 dollars in the bank account for use with the debit card.

About the purchase mode, on average clients spend 592 dollars on one-off purchases and 411 dollars on purchases with installments.

Good news for the bank: clients, on average, use 978 dollars as cash advancement. One must have in mind that, in general, the taxes for cash advancement are higher than the credit card taxes.

In regards to frequency, clients more frequently make purchases with installments (mean = 0.364) than one-off (mean = 0.202).

Regarding credit limits on the credit card, the maximum limit is 30,000 dollars with the minimum being 50 dollars. On average, clients have a credit card limit of 4494 dollars.

2 — Exploratory Data Analysis

Before visualization, we need to clean and make sure that the data have no missing values.

Checking for null values

Running the command below:

credit_data.isnull().sum()

We can check that there are 2 variables with null numbers: CREDIT_LIMIT (1 null value) and MINIMUM_PAYMENTS (313 null values).

The null values will be replaced with the mean, as both variables are continuous variables.

credit_data.loc[(credit_data['MINIMUM_PAYMENTS'].isnull() == True), 'MINIMUM_PAYMENTS'] = credit_data['MINIMUM_PAYMENTS'].mean()

Checking for duplicated data

credit_data.duplicated().sum()

There are no duplicated data.

Deleting unnecessary data

Custom id has no value for the behavioral analysis that will be carried out so this column can be dropped.

Visualizing the frequency for all variables plotting histograms:

From the plots, we can extract some insights:

BALANCE left in the account is more frequent around 1,000 dollars.

left in the account is more frequent around 1,000 dollars. PURCHASES values concentrate below 5,000 dollars.

values concentrate below 5,000 dollars. BALANCE FREQUENCY — we can see that clients frequently update the balance in their accounts.

— we can see that clients frequently update the balance in their accounts. ONEOFF_PURCHASES and INSTALLMENT_PURCHASES — looking at the scale of the graph we notice that purchases with installments are more frequent for values no greater than 5,000 dollars and one-off purchases are more frequent for values no greater than 10,000 dollars.

— looking at the scale of the graph we notice that purchases with installments are more frequent for values no greater than 5,000 dollars and one-off purchases are more frequent for values no greater than 10,000 dollars. PURCHASE FREQUENCY show a segmentation of clients: one group make purchases very frequently, while the other group rarely make purchases.

show a segmentation of clients: one group make purchases very frequently, while the other group rarely make purchases. MINIMUM PAYMENTS and PRC FULL PAYMENT — these variables show us that many clients opt for paying the minimum of their credit card bill. Very few clients pay the full bill. This is also good for the bank as taxes are high for credit card bills.

— these variables show us that many clients opt for paying the minimum of their credit card bill. Very few clients pay the full bill. This is also good for the bank as taxes are high for credit card bills. TENURE shows that most of the clients are long term clients (more than 11 years)

Getting correlations between variables

Instead of just printing a table with the coefficient of correlation values for the variables, we can make it easier to analyze by plotting on a heatmap.

correlations = credit_data.corr()

f, ax = plt.subplots(figsize=(20,15))

sns.heatmap(correlations, annot=True);

Some variables seem to have a strong correlation, for example: PURCHASE and ONE-OFF PURCHASES with a coefficient of correlation of 0.92. On the other hand, CREDIT LIMIT does not seem to be strongly correlated with any variable.

As there are many features and some seem to be redundant in the dataset, clustering the data may enable us to draw a deeper analysis and find similarities in the data.

3 — Clustering the data with K-means

Scaling the data

There are 17 numerical variables in this dataset. Some of them are within a wide range (for example BALANCE in the accounts) others are within a smaller range, from 0 to 1 (frequency data). Using Standard Scaler from sklearn preprocessing to scale the data:

scaler = StandardScaler()

credit_data_scaled = scaler.fit_transform(credit_data)

Standard Scaler centers the input value by subtracting the mean and dividing the results by the standard deviation. The result is that the distribution of values for that variable will have mean of zero and standard deviation of 1.

Determining the optimum number of clusters

To choose the best number of clusters the elbow method was implemented. This is one of the most popular methods to determine the number of clusters.

The objective of the elbow method is to minimize WCSS, which measures the within cluster sum of squares. This is the sum of squares of the distances of each data point in all clusters to their respective centroids. When WCSS is minimum, you have less variability of the data inside the cluster.

So, considering that the optimum number of clusters is between 1 and 20 we can run the code below and plot the results:

range_values = range(1, 20)

for i in range_values:

kmeans = KMeans(n_clusters=i)

kmeans.fit(credit_data_scaled)

wcss.append(kmeans.inertia_)

Using the elbow method it seems that the optimum number of clusters is between 7 and 10, this is the range which indicates the formation of an ‘elbow’ on the graph.

Running k-means and getting insights on the data

Running k-means algorithm considering the number of clusters equal to 8, as a first iteration:

kmeans = KMeans(n_clusters=8)

kmeans.fit(credit_data_scaled)

labels = kmeans.labels_

Checking the number of clients assigned to each cluster (or group):

np.unique(labels, return_counts=True)

Out[27]:

(array([0, 1, 2, 3, 4, 5, 6, 7], dtype=int32),

array([1044, 82, 2596, 1241, 1178, 200, 623, 1986]))

Getting the centroids:

cluster_centers = pd.DataFrame(data = kmeans.cluster_centers_, columns = [credit_data.columns])

cluster_centers = scaler.inverse_transform(cluster_centers)

cluster_centers = pd.DataFrame(data = cluster_centers, columns = [credit_data.columns])

cluster_centers

Exploring the 8 groups from the table we can draw some conclusions:

GROUP 0 : This seems to be an intermediate group — it is the fourth in the list for the amount of money spent on purchases ( 923 dollars on average). Most of the purchases were with the method of payment by installments. Credit limit is around 3,000 dollars, which is a low credit limit considering the 8 clusters.

: This seems to be an intermediate group — it is the fourth in the list for the amount of money spent on purchases ( 923 dollars on average). Most of the purchases were with the method of payment by installments. Credit limit is around 3,000 dollars, which is a low credit limit considering the 8 clusters. GROUP 1 :This also seems to be an intermediate group. At some points seems to present a similar behaviour to group 0, buy they spent more money on purchases, being the third on the list in terms of money spent on purchases in general. Most of the purchases were one-off purchases and this is the second group in the list which is likely to pay the credit card bill in full. Maybe this could be a group of clients that could be offered to increase the credit card limit.

:This also seems to be an intermediate group. At some points seems to present a similar behaviour to group 0, buy they spent more money on purchases, being the third on the list in terms of money spent on purchases in general. Most of the purchases were one-off purchases and this is the second group in the list which is likely to pay the credit card bill in full. GROUP 2 : This group has the highest frequency for asking for money in advance (0,52) and are not much likely to pay for the credit card bill in full (PRC_FULL_PAYMENT = 0,039, second last on the list). This does not to seem a target group to increase credit card limit, having one of the lowest purchase frequency of all (0,302) and being less likely to pay the credit card bill in full.

: This group has the highest frequency for asking for money in advance (0,52) and are not much likely to pay for the credit card bill in full (PRC_FULL_PAYMENT = 0,039, second last on the list). GROUP 3 : spends a lot on purchases (24,957) being the group which spends more on the credit card. Most of the purchases are one-off purchases which means this is a group of clients with high potential for consuming. Not surprisingly this is the group with one of the highest purchase frequencies (0.91). This is also the most likely group to pay the credit card bill in full. It could be a target group for marketing campaigns since they buy a lot and are ‘good payers’.

: spends a lot on purchases (24,957) being the group which spends more on the credit card. Most of the purchases are one-off purchases which means this is a group of clients with high potential for consuming. Not surprisingly this is the group with one of the highest purchase frequencies (0.91). This is also the most likely group to pay the credit card bill in full. GROUP 4 : Presents one of the lowest purchase frequencies from the list (0,267), lowest balance and balance frequency.

: Presents one of the lowest purchase frequencies from the list (0,267), lowest balance and balance frequency. GROUP 5: This is the group with highest balance and purchase frequencies. Their credit limit is the second highest on the list and most of their purchases were, on average, one-off purchases.

This is the group with highest balance and purchase frequencies. Their credit limit is the second highest on the list and most of their purchases were, on average, one-off purchases. GROUP 6 : This is the group that spent less money on purchases, with, as expected, the lowest purchase frequency. This is also the group less likely to pay for the credit card bill on full, which may indicate a group of clients that may have difficulties their financial situation.

: This is the group that spent less money on purchases, with, as expected, the lowest purchase frequency. This is also the group less likely to pay for the credit card bill on full, GROUP 7: This seems an to be another intermediate group, closer to the group of clients with less probability of spending money on purchases. A reduction of dimensionality is necessary to make this separation more clear. This is the group with the lowest credit limit and one of the lowest purchase frequencies. On the other hand, they seemed to ask for a considerable amount of cash in advance (1,122 dollars), which was a higher amount than the money they spent on purchases with the credit card over the period.

Below are the plots representing some of the variables discussed above for each of the 8 groups:

Balance variations for the 8 groups

Balance Frequency variations for the 8 groups

Overall expenditure on one-off purchases for the 8 groups

Overall expenditure on installment purchases for the 8 groups

Purchase frequency for the 8 groups

Cash advance frequency for the 8 groups