Unsupervised Learning: Clustering
By: Taylor Fogarty
Tags: Pokemon, Unsupervised Learning, Clustering, Pop Culture, Machine Learning

First, an introduction to unsupervised learning:

When it comes to the heavy subject of “machine learning” there are really two types: supervised and unsupervised. Supervised is a process where you’re teaching the algorithm how to label things, or you’re giving it a y-value for every observation and training it to predict y-values for unlabeled observations.

This is what was happening in my aforementioned Game of Thrones article- I was training a model to classify a battle as either a win or a loss based on the previous battles which were already labeled as wins or losses. Supervised learning is more useful for predictions because it learns the differences between one group and another, allowing it to classify observations outside the training set. However, what if our goal is not to classify, but to organize? That’s where unsupervised learning comes in.

So what is unsupervised learning?

There are three types of unsupervised learning: clustering (what we’re going to focus on), dimensionality reduction, and autoencoding.

Dimensionality reduction (aka data compression) does exactly what it sounds like it does. It finds ways to shrink and encode your data so that it’s easier, faster, and cheaper to run through a model. It’s commonly used for images in order to break them down but retain most of the information. Data compression falls on the assumption that most data is somewhat redundant and can be reformatted to contain information more efficiently. There are two types of data compression: principal component analysis which finds linear combinations of variables that communicate most of the variance and singular-value decomposition which factorizes a dataset into three smaller matrices. An Autoencoder is very similar to data compression in the sense that shrinks the data, but it does this through deep learning where the data is input into a neural network that, through weighting, mold an output that is the best representation of the data.

Clustering also does its name justice. It takes the unlabeled data and organizes it into similar groups. There are three ways it can do this. First, there is k-means clustering which creates k mutually exclusive groups. It does this by assigning k random centroids to the data and assigning the observations to the centroid they’re closest to. The centroid is centered within these observations and the process repeats until the centroids effectively stop moving. The difficult part with this is process of choosing a proper k can be complex. A larger k means smaller groups and therefore more granularity, but you might want the groups to be clustered more broadly. Below is a source with an interactive visualization that clearly explains k-means clustering further.

from sklearn.cluster import KMeans

k = 10

kmeans = KMeans(n_clusters=k).fit(X)

kmeans.labels_

There’s also hierarchical clustering which begins with n clusters, a cluster for each observation. From there, it combines the closest two clusters into a larger cluster and repeats this until all observations are in a single cluster. This is called agglomerative clustering and its reverse (one group splitting into many) is called divisive clustering. You can divide the resulting dendrogram to give you the desired number of clusters.

from sklearn.cluster import AgglomerativeClustering

clusters = AgglomerativeClustering(n_clusters=10).fit(X)

clusters.labels_

Lastly, there is probabilistic clustering which is a softer form of clustering which instead of assigning a group to each observation, it assigns a probability of a group. This is helpful if you want to know how similar an observation is to a group rather than just the group is most similar to.

There are two main challenges of unsupervised learning. First, specifically with clustering, there is required exploration into the resulting clusters. The algorithm will split the data, but it will not tell you how it did so or what the similarities are within the clusters which may be the goal of the execution. Second, it’s difficult to know if it worked properly. Unlike with supervised learning, there is no accuracy metric that can be used to evaluate it.