K Means clustering with python code explained | by Yogesh Chauhan | Towards Data Science

K means clustering is another simplified algorithm in machine learning. It is categorized into unsupervised learning because here we don’t know the result already (no idea about which cluster will be formed). This algorithm is used for vector quantization of the data and has been taken from signal processing methodology. Here the data is divided into several groups, data points in each group have similar characteristics. These clusters are decided by calculating the distance between data points. This distance is a measure of the relationship among numerous data points lying unclaimed.
K means should not be confused with KNN algorithm as both use the same distance measurement technique. There is a basic difference between the two popular machine learning algorithms. K means works on data and divides it into various clusters/groups whereas KNN works on new data points and places them into the groups by calculating the nearest neighbor method. Data point will move to a cluster having a maximum number of neighbors.
K means clustering algorithm steps
Choose a random number of centroids in the data. i.e k=3.Choose the same number of random points on the 2D canvas as centroids.Calculate the distance of each data point from the centroids.Allocate the data point to a cluster where its distance from the centroid is minimum.Recalculate the new centroids.Recalculate the distance from each data point to new centroids.Repeat the steps from point 3, till no data point change its cluster.
Choose a random number of centroids in the data. i.e k=3.
Choose the same number of random points on the 2D canvas as centroids.
Calculate the distance of each data point from the centroids.
Allocate the data point to a cluster where its distance from the centroid is minimum.
Recalculate the new centroids.
Recalculate the distance from each data point to new centroids.
Repeat the steps from point 3, till no data point change its cluster.
K means divides the data into various clusters and the number of clusters is equal to the value of k i.e. if k=3 then the data will be divided into 3 clusters. each value of k is a centroid around which the data points will gather.
Distance calculation can be done by any of the four methods i.e. Euclidean, Manhattan, Correlation, and Eisen. Here we are using the Euclidean method for distance measurement i.e. distance between two points (x1,y1) and (x2,y2) will be
Here are some pros and cons for using k means clustering algorithm in machine learning
Pros:
A relatively simple algorithm to applyFlexible and work well with large dataConvergence is guaranteed
A relatively simple algorithm to apply
Flexible and work well with large data
Convergence is guaranteed
Cons:
We have to manually define the number of centroidsNot immune to outliersDepends on initial values of centroid chosen
We have to manually define the number of centroids
Not immune to outliers
Depends on initial values of centroid chosen
Now, we will try to create an algorithm in python language. Here, we will call some basic and important libraries to work.
import pandas as pdimport numpy as np import matplotlib.pyplot as pltfrom sklearn.cluster import KMeans
sklearn is one of the most important packages in machine learning and it provides the maximum number of functions and algorithms. To use k means clustering we need to call it from sklearn package.
To get a sample dataset, we can generate a random sequence by using numpy
x1=10*np.random.rand(100,2)
By the above line, we get a random code having 100 points and they are into an array of shape (100,2), we can check it by using this command
x1.shape
Now, we will train our algorithm by processing all the data. Here the number of clusters will be 3. This number is given arbitrarily by us. we can choose any number to define the number of clusters
kmean=KMeans(n_clusters=3)kmean.fit(x1)
we can see our three centers by using the following command
kmean.cluster_centers_
To check the labels created, we can use the following command. It gives the labels created for our data
kmean.labels_
We can see that because of manually choosing the number of centroids our clusters are not very much segregated. A cluster have a similar set of information and our aim is to make the cluster as unique as they could. It helps in extracting more information from our given dataset. Thus we can plot an elbow curve which can clearly depict a trade-off between the number of centroids and information gain.
wcss = []for i in range(1,20): kmeans = KMeans(n_clusters=i,init=’k-means++’,max_iter=300,n_init=10,random_state=0) kmeans.fit(x1) wcss.append(kmeans.inertia_) print(“Cluster”, i, “Inertia”, kmeans.inertia_)plt.plot(range(1,20),wcss)plt.title(‘The Elbow Curve’)plt.xlabel(‘Number of clusters’)plt.ylabel(‘WCSS’) ##WCSS stands for total within-cluster sum of squareplt.show()
You can see there is K-means++ as the method than conventional k-means. The former method overcomes the disadvantage of wrong selection of centroids which usually happens because of manual selection. Sometimes the centroids chosen are too far away from the points that they don’t have any data points in their cluster.
The output graph can help us in determining the number of centroids to be chosen for a better clustering.
The curve clearly states that if we choose the number of centroids as 7, 8, 9, or 10 then we have a better chance of fine clustering. The data is so scattered that we can even select 14 as the number of centroids. This curve helps in deciding between computational expense and knowledge gain from a dataset. Now, let us select the centroids as 10 so that we have 10 separate clusters.
Conclusion:
We have successfully created three clusters from our random data set. All data clusters are shown in different colors. We can use the same code for making clusters on other data also and can even change the number of clusters in the algorithm. Then by using the elbow method we predicted that more centroids can improve the clustering. So after choosing more clusters we get better clusters with improved information gain.