How to Apply K-means Clustering to Time Series Data | by Alexandra Amidon | Towards Data Science

Clustering is an unsupervised learning task where an algorithm groups similar data points without any “ground truth” labels. Similarity between data points is measured with a distance metric, commonly Euclidean distance.
Clustering different time series into similar groups is a challenging clustering task because each data point is an ordered sequence.
The most common approach to time series clustering is to flatten the time series into a table, with a column for each time index (or aggregation of the series) and directly apply standard clustering algorithms like k-means. (K-means is a common clustering algorithm that constructs clusters of data by splitting samples into k groups and minimizing the sum-of-squares in each cluster).
As shown below, this doesn’t always work well. Each subfigure in the chart plots a cluster generated by k-means clustering with Euclidian distance. The cluster centroids in red do not capture the shape of the series.
Intuitively, the distance measures used in standard clustering algorithms, such as Euclidean distance, are often not appropriate to time series. A better approach is to replace the default distance measure with a metric for comparing time series, such as Dynamic Time Warping.
In this article, I will explain how to adapt the k-means clustering to time series using dynamic time warping. I will provide easy code examples from the tslearn package.
But first, why is the common Euclidean distance metric is unsuitable for time series? In short, it is invariant to time shifts, ignoring the time dimension of the data. If two time series are highly correlated, but one is shifted by even one time step, Euclidean distance would erroneously measure them as further apart. Click here for a detailed example.
Instead, it is better to use dynamic time warping (DTW) to compare series. DTW is a technique to measure similarity between two temporal sequences that do not align exactly in time, speed, or length.
Given series X=(x0, ..., xn) and series Y=(y0, ..., ym), the DTW distance from X to Y is formulated as the following optimization problem:
To summarize the DTW equation: DTW is calculated as the squared root of the sum of squared distances between each element in X and its nearest point in Y. Note that DTW(X, Y) ≠ DTW(Y, X).
Let’s break this down further.
DTW compares each element in series X with each element in series Y (n x m comparisons). The comparison, d(xi, yj), is just the simple subtraction xi — yj.
Then for each xi in X, DTW selects the nearest point in Y for distance calculation.
This creates a warped “path” between X and Y that aligns each point in X to the nearest point in Y. The path is a temporal alignment of time series that minimizes the Euclidean distance between aligned series.
Dynamic Time Warping is computed using dynamic programming with complexity O(MN). Click here or here for more details about the specific algorithm.
It is easy to compare two time series with DTW in python:
from tslearn.metrics import dtwdtw_score = dtw(x, y)
soft-DTW is a differentiable variant of DTW that replaces the non-differentiable min operation with a differentiable soft-min operation:
Per the function, soft-DTW depends on a hyper-parameter γ that controls the smoothing of the resulting metric. like DTW, soft-DTW can be computed in quadratic time using dynamic programming.
Footnote: The main advantage of soft-DTW stems from the fact that it is differentiable everywhere. This allows soft-DTW to be used as a neural networks loss function, comparing a ground-truth series and a predicted series.
from tslearn.metrics import soft_dtwsoft_dtw_score = soft_dtw(x, y, gamma=.1)
The k-means clustering algorithm can be applied to time series with dynamic time warping with the following modifications.
Dynamic Time Warping (DTW) is used to collect time series of similar shapes.Cluster centroids, or barycenters, are computed with respect to DTW. A barycenter is the average sequence from a group of time series in DTW space. The DTW Barycenter Averaging (DBA) algorithm minimizes sum of squared DTW distance between the barycenter and the series in the cluster. The soft-DTW algorithm minimizes the weighted sum of soft-DTW distances between the barycenter and the series in the cluster. The weights can be tuned but must sum to 1.
Dynamic Time Warping (DTW) is used to collect time series of similar shapes.
Cluster centroids, or barycenters, are computed with respect to DTW. A barycenter is the average sequence from a group of time series in DTW space. The DTW Barycenter Averaging (DBA) algorithm minimizes sum of squared DTW distance between the barycenter and the series in the cluster. The soft-DTW algorithm minimizes the weighted sum of soft-DTW distances between the barycenter and the series in the cluster. The weights can be tuned but must sum to 1.
As a result, the centroids have an average shape that mimics the shape of the members of the cluster, regardless of where temporal shifts occur amongst the members.
Using the tslearn Python package, clustering a time series dataset with k-means and DTW simple:
from tslearn.clustering import TimeSeriesKMeansmodel = TimeSeriesKMeans(n_clusters=3, metric="dtw", max_iter=10)model.fit(data)
To use soft-DTW instead of DTW, simply set metric="softdtw".
Note that tslearn expects a single time series to be formatted as two-dimensional array. A set of time series should be formatted as a three-dimensional array with shape (num_series, max_length, 1). If series in the set are not of equal length, the shorter series are augmented with NaN values. tslearn has easy-to-use utility functions for properly formatting data and integrates easily with other time series packages and data formats.
I hope you enjoyed reading this piece. To learn about time series machine learning, please check out my other articles:
link.medium.com
towardsdatascience.com
medium.com
Understanding Dynamic Time Warping by DataBricks
Paparrizos, John and Gravano, Luis, 2015. k-Shape: Efficient and Accurate Clustering of Time Series
Time Series Classification and Clustering by Alex Minnaar
TSLearn User Guide for Clustering
Dynamic Time Warping Wikipedia article
Time Series Aggregation by LumenAI