Introduction to K-means Clustering: Implementation and Image Compression.
By: Chingis Oinar
Tags: Artificial Intelligence, Clustering, Image Processing, Machine Learning

Introduction to K-means Clustering: Implementation and Image Compression.

This article is an introduction to K-means clustering through its Python implementation. Also, it includes its application to image compression. Chingis Oinar Feb 13·4 min read

K-means clustering is one of the most popular unsupervised clustering algorithms. A cluster implies a collection of data points grouped together because of some similarities. There are a large number of applications of clustering such as segmentation, sorting, marketing and sales, and etc. In this article, I would like to introduce the idea of K-means clustering through its Python implementation and how it can be used for image compression.

Algorithm

K-means Algorithm. Source: Incheol via Wikimedia

Initialization: Randomly sample k colors from the input image. These are the initial k means

2. For each pixel in the image, assign it to its nearest mean given by

Euclidean distance

3. Update the means using the pixel assignments from Step 2.

Find a mean point of each cluster

4. Repeat Steps 2 and 3 until convergence.

Python Implementation

Now I am attaching my Python implementation of the above algorithm. However, this implementation is for educational purposes only, hence it is not efficient in terms of time. To see that, I will compare the time taken to process the same image by both implementations.

Image Compression

Why are image compression techniques needed?

There might be a variety of needs for image compression such as :

Data needs lesser space when compressed allowing much more data to be stored with less disk space. This is crucial in healthcare, where medical images need to be archived, and dataset volume is massive. Sometimes we need to extract and store the most useful components of an image, which is represented as embedding, so an image compression might be a very beneficial approach to store more data.

Saint Basils Cathedral. Source: Julius Silver via Wikimedia

Let’s take a closer look at how we can use K-means clustering to tackle this problem using an image shown above as an example. I will break down our task into following sub-tasks:

Data Preprocessing Visualizing the Color Space using Point Clouds Visualizing the K-means Reduced Color Space

D ata Preprocessing

As you might have already known an image consists of three channels, which are Red, Green and Blue, each having values in the range of [0, 255]. Thus, a certain image might have 255 * 255 * 255 different colors. So, before we feed in our image, we need to preprocess it. First, we need to normalize our image to fall in the range of [0, 1] by dividing it by 255. Finally, we need to flatten our image since we can input 3D data into our algorithm.

V isualizing the Color Space using Point Clouds

Original Color Space

As can be seen, our original image is very rich in color. In fact, there are over 16 million possible colors. Also, we can notice that an image contains multiple shades of blue that might be an entire cluster representing all the representations. Using K-means clustering algorithm we can generalize all these colors by their corresponding centroids. Thus, the image is compressed by using centroids only.

Implementation comparison

Visualizing the K-means Reduced Color Space

Compressed Color Space

After we cluster our image, we can see that the colors in our color space change more sharply since we only use our 16 centroids to represent the image. Also, a comparison of our implementation with the original one shows us that it is almost three times slower; therefore, I provided it for your understanding only. Finally, if the original image takes about 96 kB, the compressed representation takes only 60 kB while preserving enough information to represent Saint Basils Cathedral.

Compressed image of Saint Basils Cathedral

Now imagine that you have 1GB of space to store your dataset. You can either store about 10,922 images of the size of 96kB, or you can compress images to take 60kB each and store about 17,476 images. These numbers impress even more with larger space.

Some Last Words

There are different approaches to compress our images. So, if you are interested in studying it further, I am attaching some useful articles. Also, I encourage you to get familiar with other applications of clustering algorithms. This work can be easily found on my GitHub:)

Related articles