Customer Segmentation: Unsupervised Machine Learning Algorithms In Python
By: Lama Ali Alzahrani
Tags: Dbscan, Kmean, Unsupervised Learning, Clustering, Customer Segmentation

Business Problem:

A primary goal for any company and business is to understand their targeted customers. How their consumers operate and use their services. Every consumer may use a companies services differently. The problem we’re trying to solve is to define this delivery company’s consumers. To define certain behaviors and methods these consumers use the companies services for.

Data Exploration:

We were fortunate enough to be given a dataset of a startup delivery company. The dataset is a private dataset, therefore we won’t be linking it. The recorded data is data that has been collected in a span of six months. It’s divided into three sheets users,payment and orders.

Figure1: Orders data sheet columns and data types

The following is the orders data sheet, to give an idea of the data types we’re mostly working with. The orders data sheet contains 2604 records and 26 columns. The users contains 4663 records and 12 columns while the payment data sheet contains 3169 records and 6 columns.

To know a little more about the data before starting to implement our machine learning algorithms, we created a few EDA plots.

Figure2: Order status

Which product are user’s mostly using the app’s services for?

Figure3: Orders Pie chart

What is the time line for registered users ?

Figure4: Line graph of registered users over time.

Data Preparation:

We started by checking if there were any missing values.

Figure5: Missing values in the orders data sheet.

As the figure shows there are 6 columns with missing data points. The following code will demonstrate how we dealt with the missing values.

Next,we decided to divide the orders into 6 distinct categories for easier analysis, as shown in the figure below.

Figure6: Separating the orders into 6 categories.

Considering that there are three datasheets to work with, we read each data sheet using pandas dataframe then combined all three into one dataframe.

Figure7: Combining 3 dataframes into one.

Model Implementation:

Initially, before we decided to go with the customer segmentation route we were planning on implementing a supervised machine learning algorithm.However, we later realized that picking out an optimal target to base the supervised algorithm on wasn’t a suitable method given this dataset. After a few hours of brainstorming, we decided to do customer segmentation using two clustering algorithms: K-means and DBSCAN . Furthermore, we decided we’d like to test out the two different models.

Imported libraries:

We then went on and chose our features for the two clustering algorithms. we selected the most affected features related to buyers, such as Number of Orders , Total amount paid by this buyer , Number of orders paid Cash , Number of Orders paid by Card , Number of orders paid using STCPay and Count of unique product categorical ordered.

Note: STCPay is a digital wallet used in Saudi Arabia. Similar to Apple Pay.

Note: The clustering algorithms and EDA were conducted on two different google colab sheets that’s why the dataframe names differ from the figures shown above.

The next step was to scale the data and reduce the dimension using PCA.

We decided to reduce number of features before training our models. We used PCA technique for features reduction process. First, we created a Scree plot to help us to select the best number of components for PCA.

Figure8: Scree Plot.

As figure 8 shows, the best number of components for our PCA is 3,it describes the data best.Preserving 80% of the data’s variance.

After successfully choosing the right number of components,it’s time to fit the PCA with 3 components to our chosen features.

K-Mean:

Now, we start building our clustering model using K-mean algorithm. The most important parameter for K-mean algorithm is number of clusters. So, we should decide how many clusters we have in our data. There are many ways to select the best number of clusters for K-mean algorithm, in this project we applied two methods called Elbow method and Silhouette Analysis method.

1.Elbow method:

Elbow is the common method used to determine the best value of K. This method calculates the variance between data points within a cluster using the Sum of Squared Error. The best value of k to select is the point of inflection on the curve. As we see in the Elbow plot the best k for our model is 5.

Figure9: K-means Elbow Plot

2.Silhouette Analysis Method:

This method calculates the average silhouette value for each data point in the cluster, this value represents how similar a data point is to its own cluster. The range of this measure from -1 to 1. A value of 1 means the sample is far away from the neighboring clusters. The negative value refers to samples that might have been assigned to the wrong cluster.

The following GIF illustrates the idea of Silhouette Analysis Method, each row on the Silhouette plot represents one data point in the scatter plot and the X-axis refers to silhouette coefficient value. The red line indicates the average silhouette coefficient value for all samples in clusters. The cluster that has a high silhouette coefficient value is the best to choose. Finally, from the silhouette coefficient results we decided to go with 5 clusters.

Figure10: Silhouette Analysis

After the implementation of the elbow and silhouette analysis methods, we start building the K-mean algorithm with 5 clusters.

Below is a 3D scatter plot of the clusters created by K-means.

Figure11:K-means Clusters 3D Plot

DBSCAN:

For detecting outliers and anomalies in our dataset DBSCAN(density-based spatial clustering of applications with noise) is the most productive.The two determining parameters of DBSCAN are the eps and min_samples.

The eps is the distance that determines a data point’s neighbor. Points are neighbors if the distance between them is less than or equal to eps. To determine the right value of eps 3 methods were used: sklearn’s NearestNeighbors,k-distance elbow plot and an eps visualization.The optimal value for the eps is at the point of maximum curvature.

The min_samples is the number of points to form a cluster .It is determined based on domain knowledge and how big or small a dataset is. Given the number of dimensions of the dataset, min_samples is chosen.A good rule of thumb is minPts >= D + 1 and since our dataset is 3D that makes min_sample=4.For larger datasets minPts >= D*2.

There are 3 main data points in DBSCAN the core point, border point and finally what DBSCAN is good at detecting the outlier point. The core points are points within the eps’s radius. The border points are points that are accessible from the core points and with a less number of min_samples.An outlier is a point that isn’t a core or a border point.

Figure12:DBSCAN 3D Plot

In order to gain confidence regarding the eps value we went with the Elbow plot method again and an eps visualization graph.

Figure13:Eps Elbow Plot

Let’s zoom in on the plot:

Figure14:zoomed eps Elbow Plot

Figure15:Eps visualization graph

As the elbow plot and eps visualization graph show, the optimal value for eps is 1.

Results:

Using the K-means algorithm with a cluster count of 5 we see that the customers in cluster 3 prefer to pay using STCPay rather than card or cash in comparison to the other 4 clusters. In contrast customers in cluster 0 prefer to pay with cash the most.Cluster 0 also has the highest number of orders with 2072 orders while cluster 1 has 93 orders, being the least amount of orders out of the 5 clusters.In terms of profit cluster 0 generates the app the most amount of profit followed by cluster 3,2,1, and 4.

Using the DBSCAN algorithm with a cluster count of 5 we see that the customers in cluster 3 prefer to pay using STCPay rather than card or cash in comparison to the other 4 clusters. In contrast customers in cluster 0 prefer to pay with cash the most.Cluster 0 also has the highest number of orders with 2053 orders while cluster 2 has 26 orders, being the least amount of orders out of the 5 clusters.In terms of profit cluster 0 generates the app the most amount of profit followed by cluster -1,2,3, and 2.

A few actionable insights found that the delivery company could execute:

As seen from the K-mean results, the number of orders created by buyers in cluster 1 is greater than in other clusters, with an average number of is 46 orders during 6 months. This is a large number of orders for buyers, so it is possible these accounts are stores registered as buyers. We observe the same with cluster -1 in DBSCAN, with an average of 10 orders during 6 months for an individual buyer.

Most buyers pay with cash. In this case, maybe the buyers don’t know about the availability of different payment methods through the app. So, the marketing team should focus on clarifying the power of the app by providing different payment methods.

For more information regarding the code and it’s implementation, please visit our github repository:

Or you could checkout our dashboard to view the full insights that we’ve found: https://customer-segmentation-tuwaiq-2.herokuapp.com/

Note: The dashboard is a Dash App dashboard hosted on Heroku cloud server, it might take a few minutes to load.

Future Work:

Considering that the collected data was data of six months time, the models would be more robust with more data points to plot. The algorithms could be optimized in the future by feeding it more data. The company now may market to each segmented customers appropriately and show different advertisements to different customer segments.