A Noob’s Guide to K-Means
By: Tanya Gupta
Tags: K Means Clustering, Machine Learning, Beginner, Unsupervised Learning

Today we will be learning together a very interesting algorithm that uses similarity among items to group them. The idea seems pretty similar to that of KNN algorithm but the algorithm is not. Let’s get started with the k-means algorithm

What is clustering and what are its types?

Clustering is a process of grouping up data points or items based on their similar features. It does not involve labelled data and it is left on the machine to find a pattern according to which we cluster. There are 3 types of clustering:

Exclusive Clustering: The clusters are non-overlapping ( example: K-means)

Source: analyticsprofile.com

Overlapping Clustering: some data points may belong to one or more clusters(example: Fuzzy or C-means clustering)

Source: analyticsprofile.com

Hierarchical Clustering: creates an hierarchy of clusters. Creation of such hierarchy could be either in a bottom-up approach(agglomerative) or top-down approach(divisive).

Introduction

K-means clustering is an unsupervised learning algorithm. Unlike in KNN algorithm, we do not group them by choosing the group that majority of the K neighbours belong to. Instead, we choose random k centroids, find the data points near those centroid and group them accordingly. We will get more into this as we will move to its implementation.

Let’s take an example to understand the idea of clusters:

Source:lbb.in

Imagine you are at a restaurant. You will see that a bunch of people are sharing a table. This suggests that these “clusters” of people, are familiar with each other more than the people sharing another table.

Another example could be that if you are in an “all-you-could-eat-buffet”, you would always see the dessert, main course and the appetizers separated.

Source:theatlantic.com

From both of these examples, we can see some properties of the clusters formed via any clustering algorithm:

Within a cluster, the data points are as similar to each other as possible.

If we compare each point within one cluster to each point within any other cluster, then, the data points should be as dissimilar as possible.

Working and Implementation:

Steps to perform K-means:

Assuming we have selected the number of clusters, k, we choose any k random positions as our centroids. For each of our centroids, we find their distance from each of our data points. If a centroid has least distance from a point, that point is grouped with that centroid. After grouping each point of our dataset, we find the mean of points in each of the clusters. This mean becomes our new centroid. If our new centroid values and old centroid values are equal then, this means we have reached “convergence” or the process of clustering is over. Else we would repeat steps 2 to 4 until we reach convergence.

Given below is the implementation on crime.csv dataset:

You can find the dataset here.

You might find this code implementation a bit intimidating but don’t worry I will explain each and every function in this code as best as I can. That being said, given below is the description of this code:

We will be importing all the important libraries to process and visualize our data like as given in the first cell in our embed.

For clustering data, I found it easier to remove the columns “Date” and “Magnitude” as they didn’t seem to have any significance to our data. I just dropped the “Date” column here. Feel free to do so for both.

For clustering algorithms, you might find scatterplot from seaborn or scatter() from matplotlib.pyplot a very convenient tool for visualizing data points and seeing how similar they are to each other.

After removing all the unnecessary columns, I converted our dataframe into a numpy array which I find easier to work with.

Some common arguments and their meanings:

df = dataset

k = number of clusters

centroid = center (or centroid) for each and every group. The number of centers is equal to k.

randomCentroid(df , k):

The above function is used for finding the random centroids required for our first step. It uses uniform() function from the random library to generate a random float value in the range of the minimum and maximum elements of a particular column.

I used df.shape[1]-1 here because I am only concerned with the first and second column and not the “Magnitude” column. If you are removing that column all together, just remove the -1 here.

FindDistance( ):

Generates the square of Euclidean distance. It takes 2 rows as the argument. In our case each row would only contain the elements of “Latitude” and “Longitude” columns.

group_data(df , k , centroid) :

It groups the data points based on their closest centroid values.

For example, if a particular point is closest to the centroid associated with group 1 then, this data point would be added to the said group.

It creates a dictionary named “groups” which has the group number/label as the key and for each group it stores the row of all those data points which are closest to the associated centroids.

KMeans(df , k):

cen: it is a list storing the initial set of randomized centroids for each group.

it is a list storing the initial set of randomized centroids for each group. ncen: stores the list of new centroids that we calculate by finding the mean of each point in each cluster.

stores the list of new centroids that we calculate by finding the mean of each point in each cluster. If the values of ncen and cen are equal then, we have reached a point of convergence and we return the group details and the final centroid values.

For k=5 we get the following result,

Applications:

used in recommendation systems in online shopping and our “beloved” Netflix.

You can find dominant colours in an image

Is used in feature-learning step in either semi-supervised learning or unsupervised learning.

Advantages:

It works for dataset where the data points cannot be separated by a straight line.

It works for unlabelled data which is pretty common in the real-world.

It is pretty easy to implement.

It is pretty efficient on large datasets.

Disadvantages:

Finding the appropriate k value is difficult.

The clustering is dependent on the initial set of centroids especially for high values of k.

Outliers might affect the centroid positions which inadvertently affects the clustering.

If the shape of the clusters are complex, then k-means might not be able to give satisfactory results.

That wraps our blog. Hope it helps you to understand k-means algorithm a little bit better. Thank you very much for reading!!!