k-means in a nutshell
Tags: Iteration, Python Programming, Clustering, Scikit Learn, K Means

In this post I will explain how the k-means algorithm of the scikit-learn library for python works. But first, it is important to know a few facts about the algorithm itself.

k-means needs to know the number of clusters we are searching for. There are many different ways how to do so, but for the seek of this short tutorial we already know how many clusters we have. Also important to know is that k-means is an iterative algorithm. In each iteration, the cluster-centre is moved a bit until there is either no better solution or the upper limit of iterations which we can define is already reached. With a little Python project, I will visualize how the centres are moving from iteration to iteration. We will use the famous iris data set which contains information about some iris plants which can be separated into 3 clusters.

To get started with our code we need to install a few libraries. To do so open your terminal and execute the following commands:

pip install NumPy

pip install matplotlib

pip install scikit-learn

In the next steps, you will understand why my program works how it does. If you arent interested in this, you can skip this tutorial and get the full code on my GitHub Repository. Just make sure all libraries are installed and execute the kmeans.py file… Have fun :)

Having said that, we can start coding our little script. We start of with importing the required libraries:

import section

Having done that, we can load our iris data set and define how many clusters we are looking for and how many iterations we will loop through.

some variables

Now we have to find a way to start the kmeans algorithm. Doing this we have some parameters which are important. First of all, the init parameter. In our case, we will set it to random, which means our starting centroids are randomly generated. Due to the fact, that there isn't a built-in way to visualize each iteration of kmeans we have to find another solution. We will use the max_iter parameter. This defines how many iterations are allowed. With a for loop we will loop through 100 iterations

basic k-means setup

So let's create the first version of our algorithm class. In the __init__ section we load our dataset. The KMeansAlgorithm function needs three parameters: Self, maxIteration and clusters. After that, we define 3 lists. In yKMeans we will store every iteration. In centres, we will store the centre points of every iteration and these two lists are combined in the output list because a function in python can just return a single object. In the for loop, we increase the max_iter parameter in the kmeans setup. Every iteration is stored in the two lists yKMeans and centers. When the loop is done the two lists are combined in the output list which then gets returned.

But we have a problem with this implementation because now every time we call kmeans different starting points are used due to the fact our init is random. But I have a fix for this problem: We store the centres of our first iteration and use them as starting point of every other iteration. Therefore our starting points are random but also constant :)

Okay, after we created this class we must call it too. To do so we implement this code snippet:

Now everything we have to do is to find a way how we wanna display our results. In my opinion, a scatter plot is the best way to visualize this dataset. We will use matplotlib to visualize the iterations. Unfortunately, there is no zoom function. Therefore we are only able to display 8 iterations at once because else the plots would be too small. In mytplotlib the code to set up a window for 8 scatter plots looks like the following:

The next step is to fill our 8 scatter plots with data. But wich of our 100 iterations should be displayed? I decided to print the first 4 iterations because in these iterations the centres should move the most. Then I want to display 3 other iterations with larger gaps between them. In this case, we will display the iterations 10, 20 and 30. The centres should be much more constant. To check if the centres move after the thirtieth iteration we also display the last iteration. In our case, the last iteration is iteration 100. The code for this looks like this:

To bring the code in the correct order here is the whole code. Also available on my GitHub.

If we execute our program we will receive this window. As predicted the centres move in the first iterations but the iteration 30 and 100 have the same scatter plot.

Thanks a lot for reading my tutorial about the kmeans algorithm, I hope it helped you understand how the algorithm works. If you have any questions, I would be happy to receive your comments. Stay tuned which post I write next til then I wish you all the best

~Jublebi