Data Science : K-Means Clustering
By: Anjani Kumar
Tags: Machine Learning, K Means, Data Science, K Means Clustering

Introduction:

To understand K-Means Clustering, lets first understand about a cluster in machine learning .

A cluster is groups of homogeneous or similar data points. They share same information within a cluster but these clusters are heterogeneous/dissimilar to each other.

Let us give a proper definition

“A cluster is therefore collection of objects which are “similar” between them and are “dissimilar” to the objects belonging to the other cluster.”

Below Picture shows three clusters Red,Blue and Green.

Two types of Clustering Techniques:

1.Hierarchical Clustering

2.Non-Hierarchical Clustering.

Be we will focus here on Non-Hierarchical Clustering and K-Means is one of the Non-Hierarchical clustering techniques.

Intuition :

K-Means Clustering is an unsupervised machine learning algorithm. In unsupervised machine learning technique there will NOT be any target variable given. We need to analyze the data points and find the clusters between them.

We use distance as metrics to find the similarity or dissimilarity between data points and cluster variance as a measure to form homogeneous groups.

There are various distance metrics as mention below.

Euclidean Distance Manhattan Distance Minkowski Distance

Euclidean Distance:

Euclidean Distance represents the shortest distance between two points.

Here (X2,y2) and (X1,Y1) are two points in 2-Dimensional space and we calculate distance d between two using the given formula.

We will be using Euclidean distance for our K-Means clustering so lets stick with that.

What is K in K-Means ?

K is numbers of clusters, we need to find between the data sets.Choosing right number of K-value will help finding the essence from the data set.

Steps to perform for K-Means Clustering :

Assume K Centroids (for K Clusters). Compute Euclidean distance of each objects with these Centroids. Assign the objects to clusters with shortest distance. Compute the new centroid (mean) of each cluster based on the

objects assigned to each clusters. The K number of means

obtained will become the new centroids for each cluster. Repeat steps 2–4 again till there is convergence (i.e. there is no movement of objects from one cluster to another Or threshold number of iterations have occurred)

That is It.We are done.Now Clusters we get will contains all the similar data points.(when i say similar i mean most of the features of each data points are related.I am not saying ‘same’ where ‘same’ means all the features are matching exactly).

Check this picture below:

Step (a) : Its having all the data points

Step (b) Initialized two centroids(red and blue cross)

Step (c )(d)(e) : Moving the data points and centroids

Step(f) :Shows final clusters along its similar data points.

image credit: standford.edu

How to decide Optimal Number of Clusters ?

WSS Plot or Within Sum of Square Error Plot is used to identify the optimal number of clusters.

WSS plot is also called as Scree Plot or Elbow Curve within the Analytics Industry.Elbow in the graph represents the optimal value of the clusters.

We should select the optimal number of clusters(basically K) from the graph where we see sudden decrease in the elbow curve.Like here K=3 would be good choice for K.

Step by Step implementation of K-means using python code:

Advantage of K-Means Clustering :

Fast and preferable to use with large datasets. Uses within cluster variance as a measure of similarity. K-Means Guarantees convergence. Generalizes to clusters of different shapes and sizes, such as elliptical clusters.

Disadvantages :

Choosing Value of K manually. Centroids can be dragged by outliers, or outliers might get their own cluster instead of being ignored. Consider removing or clipping outliers before clustering. K-means clustering gives varying results on different runs of an algorithm. A random choice of cluster patterns yields different clustering results resulting in inconsistency. Sensitive to scaling.Changing or rescaling the dataset either through normalization or standardization will completely change the final results. K-means algorithm can be performed in numerical data only.

Conclusion: K-Means is one of the most used unsupervised machine learning algorithm used to find the clusters if value of K chosen wisely and works well with large data set.

Hope you like my article.Please hit a clap(upto 50 times) which will motivate me to write more.

Want to connect :

Linked In : https://www.linkedin.com/in/anjani-kumar-9b969a39/

If you like my posts here on Medium and would wish for me to continue doing this work, consider supporting me on patreon