K-Means Clustering. A simpler intuitive explanation. | by Abhishek Kumar | Towards Data Science

K-means is one of the simplest unsupervised learning algorithms. The algorithm follows a simple and easy way to group a given data set into a certain number of coherent subsets called as clusters. The idea is to find K centres, called as cluster centroids, one for each cluster, hence the name K-means clustering. On seeing a new example, the algorithm reports the closest cluster to which the example belongs based on a distance measure such as euclidean distance.
Let’s consider an example for a better understanding — assume a social networking site wants to find groups (clusters) of people with like interests and direct relevant ads to them. One way to approach the problem is through clustering where the firm may use features such as pages liked, posts shared and places checked into to build a dataset and run it through a clustering algorithm to identify the relevant groups. One thing to note though is that, the dataset is unlabelled.
A note on the notation. x_{i} means x subscript i, x_{^th} means x superscript th and x_{^th}{i} means x superscript th and subscript i.
The K-means clustering algorithm begins with an initialisation step — called as the random initialisation step. The goal of this step is to randomly select a centroid, u_{i} i ∈ {1,2,3...K} for each of the K-clusters.
These K centroids are then converged to an optimal value using an iterative algorithm. Each step of the iteration performs the following two steps —
Cluster assignment
Move centroid
The cluster assignment step goes through each of the example x_{^i} in the dataset and assigns it to its closest cluster centroid based on a distance measure say euclidean distance. For the example discussed below one may say that — given an example, the cluster assignment step assigns it to a colour, green or yellow depending on the distance of that example from the cluster centroids.
The move centroid step computes new cluster centroids by taking an average of the examples assigned to same cluster centroid. Alternatively, the average of all the points coloured yellow and green, give the new values for cluster centroids. Just to be clear this is the step responsible for convergence of randomly initialised cluster centroid to an optimal value.
The algorithm continues until convergence — which is to say, until two subsequent iterations do not give nearly same values for cluster centroids.
As discussed earlier that algorithm starts with random initialisation step — but random initialisation might lead to the algorithm getting stuck at a local optima.
One way to improve our odds of ending up with better values for cluster centroids is to perform the random initialisation step multiple times and choose a set of values that leads to the minimum value of distortion cost function given by —
The above cost function is computing the average of the squared distances between each of the training example, x_{^i} and the cluster to which it has been assigned, u_{^i}{c}. Squaring has been done to solely avoid square root arising due to euclidean distance. It is obvious that, smaller values of cost function correspond to better initialisation.
This technique is effective for values of K≤10 but, it has been observed that for larger values of K, it does not make that significant a difference. For larger values of K, the K-means algorithm will pretty much converge to acceptable values of cluster centroids. Also, this method of optimising the random initialisation step is computationally expensive.
Up until now, you must have developed an intuitive understanding of K-means clustering algorithm but there is one more aspect to be dealt with, the parameter K. Let’s take a look as to how we may know the number of clusters in which our data set needs to be divided.
Elbow method can be used to find the optimal value of K. The method plots the number of clusters against the value of distortion cost function and chooses a value of K where the graph elbows out or has a sharp turn.
But practically this method is not used as much, because when real world data is plotted the graph does not usually has a prominent elbow to pick a value of K.
Unfortunately, there is no automatic way of picking up a value for K. Most of the times a value for the parameter K is manually picked by looking at the visualisations.