Practical Clustering with K-Means | by Henry Blais | Towards Data Science

Unsupervised learning is often looked on as a little ‘unconventional’ in the data science world, especially when empirically provable results are desired. K-Means clustering enjoys some enduring popularity, however, because it is relatively simple to employ, and because it functions as a powerful, if temperamental, exploratory data analysis tool.
As promised in my last article, I’ll walk through some of the basics of simple K-means clustering below!
Review of Unsupervised Learning versus Clustering:
As opposed to some other practical examples in this blog, discerning readers will notice that we have no “true”, testable data to compare to! K-means (KM) and other Unsupervised Learning algorithms are most helpful when no test case exists — we will proceed on the assumption that testable data for our dataset either does not exist or has been excluded from consideration.Lets Begin!
K-Means clustering is just one branch of a family of clustering algorithms that we will gloss over here, for the time being. KM is one of the most popular members of this family because it is fairly simple, and easy to visualize, and uses conceptually easy metrics. Specifically, KM algorithms seek to cluster data observations around “Centroid” points. Before we get into Centroids, have a glance at this sample data plot prior to clustering:
As you can see, there are three clearly separate clusters of data, that almost certainly represent trends in the base data which we may or may not be aware of at this point. The data used in this example is very neat, because we’re going to generate it with the following cell in a Python 3 environment. Go ahead and copy this code to follow along if you are so inclined;
### Building k-means example, step by step!#### Made sure to import and/or download the following libraries:from sklearn.datasets import make_blobs   # tool to create our sample dataimport matplotlib.pylab as plt            # standard plottingimport pandas as pd, numpy as np          %matplotlib inline                        # very handy 'magic' commandfigsize=(8, 6)colors = ["r", "b", "g"]columns = ["x1", "x2", "color"]# We'll work with a managable sample of 50 observations in 3 distinct clusters.  You can follow these specifications in the arguments below: X, color = make_blobs(n_samples=50, random_state=42, centers=3, cluster_std=1.75)# setting the colors we'll use in a little whileX = map(lambda x: (x[0][0], x[0][1], colors[x[1]]), zip(X, color))# Starting centroid points.  These can be arbitrary, but it helps to give intelligently-designated starting points:centroids = pd.DataFrame([(-8, 0, "g"), (-7.5, 8, "r"), (2.5, -2, "b")], columns=columns)# Training pointsgaus_df = pd.DataFrame(list(X), columns=columns)
Now, with the code below, we can apply colors to our data, and visualize it a little more clearly!
ax = gaus_df.plot(kind="scatter", x="x1", y="x2", figsize=figsize)ax.set_title("Begin: Load Data Points", fontsize=15)
Go ahead and run that cell, and you should generate the simple plot on the right!
Notice that the data itself is still monochrome, and the colorful Centroids we created are all a bit, well, uncentered from the clusters they seem to relate to.
As I mentioned earlier, K-Means clustering is all about minimizing the mean-squared distance (MSD) between data observations and their Centroids. In some cases (like in this example), we will even use pure Euclidean Distance as a measurement here, so K-Means is sometimes confused with the K Nearest Neighbors Classification model, but the two algorithms operate a bit differently.
Very importantly, KM will iteratively “walk” the chosen Centroids from their starting position, towards one that minimizes MSD from the data to the nearest, or most similar, Centroid. I’ll include a code to generate a visual review of Euclidean Distance here, as a reference:
import numpy as npimport pandas as pdfrom sklearn.metrics import euclidean_distancesfrom sklearn.datasets.samples_generator import make_blobsimport matplotlib.pyplot as plt%matplotlib inlineX = np.array([[2, -1], [-2, 2]])plt.figure(figsize=(12, 7))for i, (x, y) in enumerate(X):    plt.scatter(y, x)    plt.text(y + .05, x -.05, "$X_%d$" % (i+1), size=22)    plt.plot(X[0], X[1], "g--")plt.text(.5, .5, "$\sqrt{\sum_{i=1}^N (x_{1i} - x_{2i})^2}$", size=25)plt.title("Euclidean Distance Between $x_1$ and $x_2$", size=15)
As KM cycles through data to shift the Centroids, it learns to recognize the distinct clusters that you and I can intuitively see in the initial data plot! This is a very simple example, so little work is required for KM to converge to nice, distinct clusters. Check out the sample plots below, laid out step-by-step to visualize this process:
We’ve gone over this cell above, but I’ll include it again to walk through the logical flow of K-Means
Here, our Centroids are ‘arbitrarily’ chosen, so they can be applied to clustering the observed data
On the Right, notice how each Centroid considers EVERY point of data. This algorithm can be a bit computationally expensive when run on very large datasets!
This is new territory for us, as far as coding goes. We’ll review how to produce this kind of result below, but for now just observe how the data has been neatly divided into distinct groups!
Furthermore, notice that the Centroids have not moved — this works for simple data, but on more complex data, more steps are needed.
Here we loop through Step 2 and 3 (above) until the data converges! This just means that the Centroids will gradually shift across the plot until they have minimized the total MSD from themselves, to their nearest data points!
Once convergence is achieved, we are left with neatly distinct clusters of data! These clusters will share characteristics in the original data, some of which might not be obvious from the DataFrame itself.
Now let’s walk through a practical example, front-to-back:
First, we’ll generate some sample data:
%matplotlib inlinefrom sklearn.cluster import KMeans, k_meansfrom sklearn.metrics import silhouette_scorefrom sklearn.datasets.samples_generator import make_blobsimport numpy as npimport pandas as pdimport seaborn as snsimport matplotlib.pyplot as pltimport matplotlibmatplotlib.style.use('ggplot')# # Let's make some more blobs to test K-Means onX, color = make_blobs(n_samples=100, random_state=29, centers=3, cluster_std=1.5)df = pd.DataFrame(X, columns=["x1", "x2"])df['color'] = colordf.head()
Plotting the data with Matplotlib:
# Plot the scatter -- we need to code the previous cell to get "df"ax = df.plot(kind="scatter", x="x1", y="x2", figsize=(12,6), s=50)plt.xlabel("$X_1$", fontsize=18)plt.ylabel("$X_2$", fontsize=18)ax.tick_params(axis='both', which='major', labelsize=18)
Now we can move on to the interesting part; fitting the data with packages from Sklearn:
## Take note that the latest version of Kmeans my behave slightly different.  kmeans = KMeans(n_clusters=3)model = kmeans.fit(df[['x1', 'x2']])# Now, we can get the predicted model labels, or Centroids, in the form of an array:model.cluster_centers_
Now we’ll attach these labels to the DataFrame for future plots, and assign the Centroids to their own DataFrame, as well:
# attach predicted cluster to original pointsdf['predicted'] = model.labels_# Create a dataframe for cluster_centers (centroids)centroids = pd.DataFrame(model.cluster_centers_, columns=["x1", "x2"])
This should generate a basic, but demonstrative, K-Means clustered plot!
from matplotlib import pyplot as pltplt.figure(figsize=(7,7))## Plot scatter by cluster / color, and centroidscolors = ["red", "green", "blue"]df['color'] = df['predicted'].map(lambda p: colors[p])ax = df.plot(        kind="scatter",     x="x1", y="x2",    figsize=(10,8),    c = df['color'])centroids.plot(    kind="scatter",     x="x1", y="x2",     marker="*", c=["r", "g", "b"], s=550,    ax=ax)
That’s a simple overview of K-Means clustering! Note that many exceptions and mis-labeled data points are likely to occur in less-perfect, real-world data, but this should do to get you started. In particular, look out for “whispy” data — or data that clusters into long, thin clusters along the X, Y, or both axes — this simple model is not able to discern between these kinds of clusters, and will find the NEAREST points for cluster assignment, resulting in some cross-contamination! I’ll follow up in the future with examples for how to avoid this. Thanks for reading!