K-Means Clustering: Python Implementation from Scratch
Tags: Machine Learning, K Means, Data Science, Unsupervised Learning, Artificial Intelligence

Image source: Towards AI

Clustering is the process of dividing the entire data into groups (known as clusters) based on the patterns in the data. It is an unsupervised machine learning problem because here we do not have a target variable and we only group the data points based on their similarities.

Properties of cluster

All the data points in a cluster are similar to each other. The data points from different clusters are as different as possible.

Applications of Clustering

Customer Segmentation Recommendation system Image Segmentation Document Clustering

K-Means Algorithm

The main objective of the K-Means algorithm is to minimize the sum of distances between the data points and their respective cluster’s centroid.

The scope of this article is only the implementation of k-means from scratch using python. If you are new to k-means clustering and want to learn more, you can refer to this amazing article.

Now, let’s jump straight into the implementation. You can also access the code given below from here. Download the dataset from here.

Importing Libraries

import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns import warnings warnings.filterwarnings('ignore')

Import dataset

data = pd.read_csv('clustering.csv') data.head()

Here, we will use only two features [‘ApplicantIncome’, ‘LoanAmount’] so that we can visualize the clusters on a 2D plane.

data = data.loc[:, ['ApplicantIncome', 'LoanAmount']] data.head(2)

Convert to numpy array

X = data.values

Visualize the data points

sns.scatterplot(X[:,0], X[:, 1]) plt.xlabel('Income') plt.ylabel('Loan') plt.show()

Calculate WCSS

def calculate_cost(X, centroids, cluster): sum = 0 for i, val in enumerate(X): sum += np.sqrt((centroids[int(cluster[i]), 0]-val[0])**2 +(centroids[int(cluster[i]), 1]-val[1])**2) return sum

Implement K-Means

def kmeans(X, k): diff = 1 cluster = np.zeros(X.shape[0]) centroids = data.sample(n=k).values while diff: # for each observation for i, row in enumerate(X): mn_dist = float('inf') # dist of the point from all centroids for idx, centroid in enumerate(centroids): d = np.sqrt((centroid[0]-row[0])**2 + (centroid[1]-row[1])**2) # store closest centroid if mn_dist > d: mn_dist = d cluster[i] = idx new_centroids = pd.DataFrame(X).groupby(by=cluster).mean().values # if centroids are same then leave if np.count_nonzero(centroids-new_centroids) == 0: diff = 0 else: centroids = new_centroids return centroids, cluster

Find K value using elbow method

cost_list = [] for k in range(1, 10): centroids, cluster = kmeans(X, k) # WCSS (Within cluster sum of square) cost = calculate_cost(X, centroids, cluster) cost_list.append(cost)

Plot a line plot between WCSS and k

sns.lineplot(x=range(1,10), y=cost_list, marker='o') plt.xlabel('k') plt.ylabel('WCSS') plt.show()

WCSS doesn’t reduces much after k=4, so make 4 clusters

Make clusters

k = 4 centroids, cluster = kmeans(X, k)

Visualize the clusters formed

sns.scatterplot(X[:,0], X[:, 1], hue=cluster) sns.scatterplot(centroids[:,0], centroids[:, 1], s=100, color='y') plt.xlabel('Income') plt.ylabel('Loan') plt.show()

We can clearly see the four clusters formed. The green dots represent the centroid for each cluster. With this we are done implementing the K-Means algorithm. Isn’t it simple?

Hope you found this blog useful. If you like it then give this a clap and follow me to read my upcoming blogs. Also check out some of my previous blogs:

Happy Learning!