K-Means Clustering Algorithm Implementation
By: Karan Kharecha
Tags: Machine Learning Python, K Means, Unsupervised Learning, Clustering Algorithm, Data Science

When it comes to popularity among clustering algorithms, k-means is the one. The approach behind this simple algorithm is just about some iterations and updating clusters as per distance measures that are computed repeatedly. k is the number of clusters that are to be formed. This content represents the implementation of K-Means algorithm from the scratch using numpy, pandas and plotly.

Imports that are required for implementing:

import pandas as pd

import numpy as np

import plotly.offline as plt

import plotly.graph_objs as go

Data set that is taken here consists of 788 data points and have 7 shapes that can be identified visually.

Data points initially before clustering

One notable thing about this algorithm is that, it changes the cluster groups whenever it is re-evaluated. Hence, a data point may become a part of different cluster after the program is executed again.

Initially, two dataframes are created that will be used as set of current means and previous means. Before the iteration, one dataframe from these two contains randomly selected means from the targeted data set itself.

k_means = (data.sample(k, replace=False)) # store current means

k_means2 = pd.DataFrame() # store previous means

clusters = pd.DataFrame() # store distances

Here, first statement shows assigning set of random series from the data set itself to the variable named k_means, using the method sample() whose return type is dataframe. Another variable is k_means2 that is currently an empty dataframe which will eventually store previous means. Along with these, a new dataframe named clusters is created for storing the calculated distances.

Second step is to create a parent loop that iterates until current and previous means (i.e., k_means and k_means2) are equal.

(i.e., k_means and k_means2) are equal. Next, calculate distances of all the data points from each mean stored in k_means and put it in dataframe named clusters.

A new column named ‘MDCluster’ will be appended to the existing dataframe that denotes the label of the cluster to which all the data points are assigned. This assignment is done by fetching index (which will act as label) of minimum value from the clusters dataframe.

will be appended to the existing dataframe that denotes the to which all the data points are assigned. This assignment is done by fetching index (which will act as label) of minimum value from the dataframe. After setting each data point to specific cluster using ‘MDCluster’. It’s time to calculate new means of updated clusters and store older means in k_means2. These new means will be allocated to k_means and calculated repeatedly from ‘MDCluster’ using groupby and aggregate function.

while not k_means2.equals(k_means):



# distance matrix (euclidean distance)

cluster_count = 0

for idx, k_mean in k_means.iterrows():

clusters[cluster_count] = (data[k_means.columns] -

np.array(k_mean)).pow(2).sum(1).pow(0.5)

cluster_count += 1



# update cluster

data['MDCluster'] = clusters.idxmin(axis=1)



# store previous cluster

k_means2 = k_means

k_means = pd.DataFrame()

k_means_frame = data.groupby('MDCluster').agg(np.mean)



k_means[k_means_frame.columns] =

k_means_frame[k_means_frame.columns]

Changes in the dataframe regarding clustering labels in ‘MDCluster’ in each iteration of updating means are as follows:

Shows data points in different clusters in iterations

If these intermediate clusters formed by changing means are plotted, then they would seem like:

Iterative steps and cluster reformations

Final step is to plot all the points using plotly library and see how the clusters are formed. Here, along with the data points, centres of each cluster are also plotted as bold black dots. Following code snippet shows how the points are plotted: