Cluster Analysis using R (Easy way)
By: Satish Mishra
Tags: Clustering, Data Science, R, Cluster Analysis

Cluster Analysis is an exploratory analysis that tries to identify structures within the Data.After clustering we create subset for each cluster and form equation for each cluster.

“Cluster Analysis is the task of Grouping a set of objects in such a way that objects in same group(Cluster) are more similar (homogeneous) to each other then to those in other group(Clusters)”

Typical Cluster model include :

1: Connectivity Models: Example : Hierarchical Clustering builds models based on distance connectivity.

2: Centroid Models: Example: The kmeans clustering algorithm represents each cluster by single mean vector.

3:Distribution Models:This type of Clusters are modeled using statistical distribution such as multivariate normal distribution used by the expectations maximization algorithm.

Hierarchical and Kmeans

Hierarchical and Kmeans are most frequently used clustering technique , I have covered both here:

1: Hierarchical Clustering: This type of clustering produces a set of nested clusters organized as a hierarchical tree. It can be visualized as a dendogram (A tree like diagram that records the sequence of merge or splits).

There are two main type of hierarchical clustering.

a: Agglomerative: This type of clustering starts with the point as individuals clusters . Then at each step the closest pair of clusters are merged until one cluster left.

b: Divisive: This type of clustering starts with one all inclusive cluster . Then at each step cluster is split until each cluster contains a point or (There are K cluster).

Example Using R:

Step1: Calculate Distance d and pass to function “hclust” to create dendogram and cut a tree.

We will use mtcars data

data(“mtcars”)

View(mtcars)

# Find Distance now using “euclidean”

library(stats)

d<-dist(mtcars , method = “euclidean”)

d

plot(d)

hist(d , col = “Yellow” , labels = T)

Create a dendogram using hclust function

h_1<- hclust(d,method = “single”)

plot(h_1)

We can apply hierarchical clustering using Single linkage method. Other methods are “average” and “Complete” default is complete method.

Single: Shortest distance between two points in each cluster.

Average: Average distance between all points in each cluster.

Complete: Longest distance between two points in each cluster.

h_1<- hclust(d,method = “single”)

plot(h_1)

To draw rectangle on cluster in dendogram use

rect.hclust(h_1, k=2 ,border = 2:6)

Note: On the basis of height we can cut and form group.

Cutting Tree: To get cluster number to subset data set

help(“cutree”)

h_ct<-cutree( h_1, k=2 )

Now two cluster formed we add this column to actual mtcars

new_mtcars <- cbind(mtcars , h_ct)

head(new_mtcars)

unique(new_mtcars$h_ct)

Dunn’s Index: Calculate the Dunn index for given clustering partition.

For using Dunn’s we need cluster and clvalid packages .

Dunn’s Index helps to choose which method we need to use while doing hierarchical clustering using hclus.

We try with all three methods 1: Single 2: Average 3: complete and check Dunn’s Index every time and the method returns high value of Dunn’s Index will be Preferred for hclustering. A higher DI implies better clustering.

install.packages(“clusters”)

library(cluster)

install.packages(“clValid”)

library(clValid)

help(dunn)

dunn_hct<-dunn(clusters = h_ct , Data = mtcars)

dunn_hct<-dunn(d,h_ct)

— — — — — — — — —

2:Kmeans Clustering: This is a prototype based partitional clustering technique that attempts to find a user specified number of cluster(k), which are presented by their centroids.

In this clustering number of cluster denote by K must be specified.

K-means algorithm:

Goal: Partition Data in K disjoint subsets.

1: Randomly Assign K centroids

2:Assign Data to closest Centroid

3:Move Centroids to Average Loacation

4: Repeat Steps 2 and 3

R Code: K-means clustering

help(kmeans) # to check info about kmeans

km_mt_cars<-kmeans(mtcars , centers =2 , nstart=20)

centers: Starting centroid or clusters

nstart: Times R restarts with different centroid.

help(kmeans) # to check info about kmeans

km_mtcars<-kmeans(mtcars, centers =2, nstart=20)

km_mtcars

km_mtcars$cluster

km_mtcars$centers

km_mtcars$totss

km_mtcars$withinss

km_mtcars$betweenss

totalss=WSS(within Sum of square)+BSS(Between Sum of square)

totalss<-km_mtcars$tot.withinss+km_mtcars$betweenss

Higher Value of WSS(within sum of square) is not good.

Higher Value of BSS(Between sum of square) is good.

Adding Cluster to source dataset

km_mtcars1<-cbind(mtcars ,km_mtcars$cluster)

View(km_mtcars1)

#Subsetting clusters:

mtcars_c1<-subset(km_mtcars1,km_mtcars$cluster==1)

mtcars_c2<-subset(km_mtcars1,km_mtcars$cluster==2)

head(mtcars_c1)

head(mtcars_c2)

Happy Learning