A continuously updating k-means algorithm
By: Abhishek Jhunjhunwala
Tags: Machine Learning, K Means, Algorithms, Unsupervised Learning, Data Science

A continuously updating k-means algorithm

Photo by JESHOOTS.COM on Unsplash

Overview

We have all heard about the popular k-means algorithm used to find clusters in our data which is basically grouping of points that are similar based on the features present. Anyway, I will not bore you with the details about the algorithm which most of you might be well versed with already. What I am interested in discussing is extended version of k-means : A k-means model that can be updated with new data from time to time and become more information rich. Say, today we have a million data points roughly for the user behavior of a particular product company. We apply the k-means algorithm on our data and come up with some k clusters using the elbow diagram. A month later, we get an additional 10k data points which contains latest information on the user behavior which we want to include in our model.

Should we run k-means on entire data till date combined? Remember that running k-means on huge data sets with large dimensions can be computationally intensive. And we are talking about updating the model periodically to include new data coming in regular intervals. Clever thing to do here would be to update the existing model with the new information which we can call a delta model.

New model = Old Model + Delta Model

This is a simple yet powerful technique to maintain model at production level. I will discuss the algorithm in detail but lets take a look at the classical k-means algorithm first.

Classical K-means algorithm

Lets try to code the k-means algorithm from scratch. Here are the steps involved in implementing a k-means algorithm :

Initialise the k cluster centers called centroids by randomly selecting k points from the data. There is another technique used to initialize centers called k-means++ that we won’t discuss here. Calculate the distance of each data point from these centers and assign them to the centers closest or with minimum distance. Recalculate the cluster centers by taking the average/mean of all points in a cluster. These might or might not be in the data set. Calculate the sum of distances of the new centers from old ones and lets call it error. Check if the error is greater than the tolerance value and repeat steps 2 and 3 until the error falls below the tolerance value which means that the algorithm has converged. This will be the final cluster centers and respective points.

The implementation in python will look somewhat like this :

Improved k-means algorithm

The Class k-means can be used to train a model and predict on new data. Now, lets see how to integrate new data and update our k-means model.

Calculate the distance of a new data point from all the cluster centers and find the minimum of those distances. Check if the minimum distance is less than the threshold value. If true, we assign the new data point to the corresponding cluster. Then, update the cluster center of that cluster. If False, create a new cluster and assign the new data point as its center. Also, the data point becomes a part of the cluster. We can repeat the same steps with a list of new data points to update our model. Our model now is better than previous because it has seen more data.

The code for our improved k-means class with the update functionality can be written somewhat like this :

An example has been included on a test array. Cheers !