K-Means Clustering for Beginners using Python from scratch.
By: Ankit Prasad
Tags: Machine Learning, Blogger, Neural Networks, Writer, Technology

In this article, we will take a real-world problem and try to solve it using clustering. So let's get our hands dirty with clustering.

Introduction:

Cluster analysis is a multivariate statistical technique that groups observations on the basis of features or variables they are described by.

Was that too boring ok let's try to understand this with an example.

In the example given below there two figure, one on the left side has three clusters and this is done on the basis of geographic proximity, the first cluster shows the countries in North America and the second and third cluster shows the countries in Europe and Australia respectively.

In the figure to the right, the clustering is done on the basis of the official language of the country.

In simple terms we can understand this as observations in a dataset is divided

into different groups and this is very useful.

EXAMPLE

K-Means Algorithm:

The algorithm is very simple given data we first initialize seeds randomly. Then we go on calculating the euclidean distance of every point with every seeds. The one with the minimum distance becomes the part of the given seed. After each and every data is covered we place the seeds into the centroid of the clusters formed. And now that centroid is the representative of that cluster.

When to use Cluster Analysis?

This is one of the decision we have to take while dealing with problems. Taking decision is not a tedious task as it solely depends upon the type of data we are using. If we are using a labeled data we can use classification technique whereas in case when the data is not labeled we can cluster the data based on certain feature and try to label it on our own.

So when we use cluster analysis we don’t have labels(ie..data is not labeled)

in the context of machine learning this is called as unsupervised learning.

Final Goal:

The goal of clustering is to maximize the similarity of observation within the cluster and maximize the dissimilarity between the clusters.

We will be achieving this goal very soon. Let’s dive into it.

Choosing a problem

Well to start of choosing a problem is the first and the foremost step of a data scientist. We should always choose a problem such that after solving it, the solution should benefit the end user.

In this article we will take an example of market segmentation. There will be certain features due to which the market is segmented. We will try to analyse the the type of customers in the market based on the features.

The data set consist of 30 samples and features are satisfaction and loyalty respectively.

You can download the data set through this link: https://drive.google.com/file/d/1nxr5XYg4JrwB_EtdWdyIwCT4gr85xfzb/view?usp=sharing

Lets Begin:

Well I hope you have downloaded the data set from the link given above. Before beginning make sure you have jupyter notebook installed in you pc with anaconda package manager as we will be importing certain libraries.

To install certain package through anaconda command prompt you just have to type the following command:

pip install the package name you want to install. If you already have the package installed you are ready to go.

Command to check the packages installed : conda list

#1. Importing relevant libraries:

First we will import certain libraries required for performing K-means Clustering. We will also look into these packages in details.

1. Numpy: It is a third party package that helps us to deal with multidimensional arrays.

2. Pandas: Allows us to organize data in tabular form.

3. Matplotlib: Helps in visualizing the numpy computation.

4. Seaborn: This also adds to the visualization of Matplotlib

5. Scikit-learn: This is the most widely used machine learning library. It

has various functionality as in this case we are importing KMeans from it.

Importing Libraries

#2. Reading the data:

In this step we will load the data set into the variable data using pandas data frame. There are thirty observations with features satisfaction and Loyalty .The data here is in .csv format. Remember pandas helps us to organize data in tabular form.

READING DATA

#3. Plotting the data:

Now we will simply plot the scatter plot of the given data using. These are simple python code we will get accustomed to it once we start using it regularly.

PLOTTING

#4. Clustering:

For the first section in Selecting Feature just ignore the title for now we will see it later. We are just creating a copy of our data and storing it in variable x.

So now we will create a variable kmeans and by passing the argument 2 in KMeans we just said that we want to create 2 clusters. We don’t have to do anything this simple line of code will perform the kmeans algorithm and will create two clusters. It will classify our data into two clusters.

You can see below the number of iterations that has been perfomed, here 300.

This all runs behind the scenes. Python will take care of everything.

CLUSTERING

#5. Clustering Result:

In this step we will again create a copy of x and store it in clusters. We will create a new column called cluster_predict which will have the value as predicted by our kmeans algorithm. Things will become more clear as we move ahead.

CLUSTERING RESULT

#6. Plot:

Now we will plot the clustered data, note here we have two parameters/features here ‘Satisfaction’ and ‘Loyalty’. We can easily see the two clusters the one with all the red and the other with all the blues. But there is a problem.

PLOTTING

The Problem

The biggest problem here is that Satisfaction is choosen as a feature and loyalty has been neglected. We can see in the figure that all the element to the right of 6 forms one cluster and the other on the left forms another. This is a bias result because our algorithm has discarded the Loyalty feature. It has done the clustering only on the basis of satisfaction. This does not give an appropriate result through which we can analyze things.

Satisfaction was choosen as the feature because it had large values.

So here is the problem both the data are not scaled. First we have to standardize the data, so that both the data have equal weights in our clustering.

We can’t neglect loyalty as it has an important role in the analyses of market segmentation.

PROBLEM (FEATURE SELECTION)

#7. Standardizing the variables:

We will not go in depth of this as sklearn helps us to scale the data.

The data is scaled around zero mean. Now we can see that both the data are equally scaled and now both will have equal chance of being selected as feature.

STANDARDIZING

#8. The Elbow Method:

Have we ever wondered why we initialized kmeans with 2 clusters only.

Yes, we could have initialized it with any value we wanted we could have got any number of clusters. But the analyses becomes difficult when there are a large number of clusters. So how we will know the exact number of cluster to start off. Note there are no such exact number as it changes with the problem in hand.

Here the elbow method comes handy when we are confused as to how may clusters do we need. What elbow method does is it starts of with making one cluster to the number of clusters in our sample and with the kmeans inertia value we determine what would be the appropriate number of clusters.

Remember our goal- Our final goal was to minimize the within the cluster sum of square and maximize the distance between clusters.

With this simple line of code we get all the inertia value or the within the cluster sum of square.

ELBOW METHOD

#9. Visualizing the Elbow Method:

This graph looks like elbow and we have to determine that elbow point.

Here the elbow point comes at around 4 and this our optimal number of clusters for the above data which we should choose.

If we look at the figure carefully after 4 when we go on increasing the number of cluster there is no big change in the wcss and it remains constant.

Hurrah..!! we have got the optimal number of clusters for our problem.

We will now quickly perform the kmeans clustering with the new number of clusters which is 4 and then dive into some analysis.

ELBOW METHOD VISUALIZATION

#10. Stronger Clustering:

This is a simple code which perform clustering with 4 clusters. Here there are four clusters so our whole data is categorized into either 0,1,2 or 3.

#11. Plotting the newly cluster:

#12. Analysis (The final step):

Through the given figure following things can be interpreted:

1. The purple dots are the people who are less satisfied and less loyal and therefore can be termed as alienated.

2. The red dots are people with high loyalty and less satisfaction.

3. The yellow dots are the people with high loyalty and high satisfaction and they are the fans.

4. The sky blue dots are the people who are in the midst of things.

The ultimate goal of any businessman would be to have as many people up there in the fans category. We are ready with a solution and we can target the audience as per our analysis. For example, the crowd who are supporters can easily be turned into fans by fulfilling their satisfaction level.

ANALYSIS

Footnotes

I hope I was successful in writing my first blog it took me around 4 long hours. Also please forgive me if I have missed something. I am not that good in writing I am just getting used to it. Hopefully, I will be back with another one.

Thanks for reading..!!