K-Means Clustering and PCA to categorize music by similar audio features
By: Sejal Dua
Tags: Music, Unsupervised Learning, Spotify, Machine Learning, K Means Clustering

K-Means Clustering and PCA to categorize music by similar audio features

An unsupervised machine learning project to organize my music

A year ago, life was much simpler. It was a bustling Tuesday morning in early October, and I was riding the metro to my Artificial Neural Network class in downtown Copenhagen. I was browsing my Spotify library for a playlist to shuffle for the duration of my commute when I bumped into my friend Liddy. I asked for her input, and she told me something which actually kind of changed my life. She said, “Oh Sejal, I am far too indecisive to pick a playlist based on my mood. I actually create a new playlist and add to it on the first day of each month.”

Why music genres are outdated

With the increased diversity in music that we have seen on streaming platforms such as Spotify, Apple Music, and Soundcloud in the past decade, the lines separating genres have become even more blurred than they were previously.

Genre labels are broad umbrella terms that are used to describe music that vary greatly in their characteristics. If someone asks you what kind of music you like, responding with “Rock” or “Indie” does not really say much without namedropping a few artists.

Genres are always evolving over time. If someone says they are a fan of pop music, how do you know if they are referring to Michael Jackson or Justin Bieber?

They are often socially driven with little to do with the actual characteristics of the music.

Fortunately, there has been a movement towards labeling music solely based on musical characteristics or attributes. However, organizing music based on these audio features is still a complicated task, even for a human.

Motivation for my machine learning project

Ever since Liddy inspired me with the idea to organize my music by time period, I have been hooked. It helps me to embrace the chaos of my diverse and far-ranging music taste while still being able to compartmentalize my music based on certain phases of life.

The issue, though, is that when I clicking “Shuffle” on my 2020 playlist of 449 songs, I end up skipping 9/10 songs because I can never seem to find a song that fits the mood. So I set out to find a solution to my problem using a machine learning technique called k-means clustering and Principal Component Analysis (PCA), a dimensionality reduction technique.

Data Acquisition via Spotify API

The Spotify Developer API allows you to get audio features for a track. These features are described in more detail below:

acousticness : [0–1] Confidence measure of whether the track is acoustic.

: [0–1] Confidence measure of whether the track is acoustic. danceability : [0–1] Describes how suitable a track is for dancing based on musical attributes including tempo, rhythm, stability, beat strength, and overall regularity.

: [0–1] Describes how suitable a track is for dancing based on musical attributes including tempo, rhythm, stability, beat strength, and overall regularity. energy : [0–1] Perceptual measure of intensity and activity. Energetic tracks feel fast, loud, and noisy (e.g. death metal: high energy, Bach prelude: low energy).

: [0–1] Perceptual measure of intensity and activity. Energetic tracks feel fast, loud, and noisy (e.g. death metal: high energy, Bach prelude: low energy). instrumentalness : [0–1] Predicts whether a track contains no vocals (values above 0.5 represent instrumental tracks whereas rap songs would have a score close to 0).

: [0–1] Predicts whether a track contains no vocals (values above 0.5 represent instrumental tracks whereas rap songs would have a score close to 0). liveness : [0–1] Detects the presence of an audience in the recording.

: [0–1] Detects the presence of an audience in the recording. loudness : [-60–0 dB] The average volume across an entire track.

: [-60–0 dB] The average volume across an entire track. speechiness : [0–1] Detects the presence of spoken words in a track (values above 0.66 describe tracks that are probably made entirely of spoken words, 0.33–0.66 describe tracks that may contain both music and speech, and values below 0.33 most likely represent music and other non-speech-like tracks).

: [0–1] Detects the presence of spoken words in a track (values above 0.66 describe tracks that are probably made entirely of spoken words, 0.33–0.66 describe tracks that may contain both music and speech, and values below 0.33 most likely represent music and other non-speech-like tracks). valence : [0–1] Describes the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry).

: [0–1] Describes the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry). tempo: [0–300 BPM] The speed or pace of a given piece, as derived from the estimated average beat duration.

I use Spotipy, which is a Python wrapper for Spotify’s Developer API. First, we must create a token with our API credentials.

screenshot of my giant Spotify playlist (the data source for this project)

I then extracted metadata (name, artist name, and URI) for each track in the playlist and acquired the audio feature data described above.

A preview of the raw data

Next, we can form our training data by excluding columns in the DataFrame that are not audio features. We also have to do one more critical step: standardization. Standardization is the process of putting different features on the same scale, with each scaled feature having a mean of 0 and a standard deviation of 1. This is important because the model is not familiar with the context of the data. If we do not standardize our X_train data, the model would place a much larger weight on tempo and loudness, since those variables vary by much more than the variables that are distributed in the range from 0 to 1. Standardization allows for all features to be treated equally by the model. We can accomplish this step by using sklearn’s StandardScaler import.

from sklearn.preprocessing import StandardScaler scaler = StandardScaler()

X_std = scaler.fit_transform(df_X)

Dimensionality Reduction

What is dimensionality reduction? Well, as we can see from the image above, it is a process that involves using linear algebraic operations to transform n-dimensional data to (n-k)-dimensional data. In this example, we are just reducing 3-dimensional data into 2-dimensional data, so n equals 3 and k equals 1. As you might imagine, some information is lost in this transformation, but we still get a fairly decent approximation with the tremendous benefit of being able to visualize the data. Moreover, we can also effectively reduce the complexity of the problem by reducing the number of variables.

Principal Component Analysis (PCA) is a dimensionality reduction approach which attempts to find the best possible subspace that explains most of the variance in the data. It works by deriving components from the original scaled features. From these components, the two most significant ones are typically plotted, depicting the simplified data space.

After fitting a PCA object to the standardized matrix, we can see how much of the variance is explained by each of the nine features.

from sklearn.decomposition import PCA pca = PCA()

pca.fit(X_std)

evr = pca.explained_variance_ratio_

print(evr)

To inform our decision of how many features to use for our k-means clustering algorithm, it is helpful to make a cumulative variance plot using the evr.cumsum() function and matplotlib.

Cumulative Variance Plot after PCA

From this plot, we can observe that each of the principal components explain a pretty considerable amount of variance. However, we do not need to keep all of these components. In general, it is a good rule of thumb to preserve around 80% of the variance. Therefore, in this instance, we can select the 6 most important principal components to incorporate in the k-means algorithm. This can be accomplished by instantiating a new PCA object with the n_components parameter set to 6.

Having finished our PCA, we now have components which explain most of the variance in the data. From this point on, it is going to be very difficult to interpret meaning when analyzing the numeric differences in tracks because the vectorized audio features have been transformed into a new subspace. However, after implementing k-means Clustering, we will be able to inspect the raw data, and we can even visualize our clusters of similar songs on the same 2D-plane.

In order to implement k-means clustering, we must select a number of clusters, k, which distinctly splits the data. Given that we do not have a desired amount of groupings for the huge playlist that we would like to split, and we want this code to be reproducible, we must use an algorithmic approach to determine the optimal value for k. There are actually numerous algorithmic approaches, including but not limited to the elbow method, the silhouette method, and the gap statistic. Let’s take a closer look into the elbow method, which is arguably the most popular technique.

The basic principle of k-means clustering is to define clusters such that the total intra-cluster variation is minimized. The within-cluster sum of squares (WCSS) is a metric which can quantify this partitioning effect. It essentially measures the compactness of the clustering. The optimal number of clusters can be found by executing a simple procedure:

Implement clustering algorithm for varying values of k, ranging from 1 to 20 clusters. For each k, compute the within-cluster sum of squares (WCSS) by storing the inertia value in a list after the model has been fitted. Plot the WCSS curve according to the number of clusters k. Look for a kink or elbow in the WCSS graph. Usually, the part of the graph before the elbow would be steeply declining, while the part after it — much smoother. [NOTE: there are ways to locate the elbow (a.k.a. “knee”) point programmatically, including an open source Python library called kneed or a visualization library called Yellowbrick]

Running the code snippet above yields the following WCSS curve shown below. The KneeLocator method has determined that 7 is the ideal number of clusters or distinct groupings to be used in order to separate the 449 tracks in my absurdly lengthy playlist.

WCSS curve used for selecting the optimal number of clusters

Now, all that is left to do is to implement k-means clustering once and for all. With the help of sklearn, we can obtain the cluster labels for each track in just 3 lines of code.

kmeans_pca = KMeans(n_clusters=n_clusters, init='k-means++', random_state=42)

kmeans_pca.fit(scores_pca)

df['Cluster'] = kmeans_pca.labels_

Cluster labels have been assigned in the rightmost column of the DataFrame

The cluster labels in the rightmost column of the DataFrame screenshot above are the “results” of this ML technique. Okay… this was a bit anticlimactic. What good do these cluster labels do? Well, let’s investigate them further.

Analysis and Visualization

Analyzing the results of PCA and K-Means Clustering

While k-means clustering is not a supervised machine learning technique—meaning we cannot verify whether the results are accurate or not—we can visualize the clusters on a 2D plane to ensure that there is some sensical separation of data based on the sources of signals that are encapsulated by the components. In the figure above, it is clear that Cluster 5 is most similar to Clusters 1 and 3 and that Cluster 4 is quite loosely defined, overlapping with a few other clusters. Despite some noise, there are clear groupings of tracks, as indicated by the concentrated areas of green, red, orange, and blue data points. Additionally, this is only a 2D visualization of a 6-dimensional vector space, so more analysis is needed in order to determine the efficacy of the k-means algorithm.

Since meaning cannot be inferred from the transformed components, let’s take a look at the raw values for the audio features of the tracks after they have been clustered. A radar chart can be really powerful in this type of analysis because it allows us to make some quick observations about each cluster.

In the figure below, the a radar trace has been plotted for the average audio feature values in each cluster, after normalizing the entire dataframe. Acousticness is a Spotify-defined variable between 0 and 1 while tempo can be in the 100s. Normalization is important because it effectively scales all variables to have values between 0 and 1, making the following visualization far more meaningful.

Radar chart comparison of average audio features for each cluster

After a quick glance, it is evident that Cluster 2 is most acoustic-sounding, Cluster 4 consists of live-sounding tracks, Cluster 5 contains the most verbose tracks, and Cluster 6 is characterized by more instrumental tracks. Features like energy, danceability, valence, tempo, and loudness did not contain strong enough sources of signal on which to split the data. That being said, some combination of high danceability, tempo, and energy — may have characterized tracks in Cluster 1. The shapes of these traces are quite telling of which types of tracks we can expect to see in each cluster.

The code above allows us to create new Spotify playlists based on the cluster groupings of tracks after PCA and k-means. As we initially observed, Cluster 5 is characterized by tracks having high levels of “speechiness”. In other words, we can expect to find Rap and R&B tracks in this grouping.

As we can observe above, Burna Boy, blackbear, 6LACK, Playboi Carti, Lil Uzi Vert, Kota the Friend, A Boogie Wit da Hoodie, and Polo G are some the artists included in this subset, which validates the conclusion suggested by the data.

Cluster 2 contains more instrumental music, as suggested by the radar chart. There are some tracks by the xx, Lauv, Henry Green, and other lesser known artists. Many of the tracks in this cluster have a very synth-y feel and not too many words.

Finally, there is one last thing we can do to analyze the findings. We can make some bar charts, plotting a particular feature for each track in a given cluster in order to see if this feature is, indeed, characteristically high with respect to the average feature value in the entire dataset.

Looking at a few select clusters (2, 5, and 6) and features of interest (acousticness, speechiness, and instrumentalness), a trend is observable in the subplots along the diagonal. In the top left subplot, the tracks in Cluster 2 have mostly above average scores for acousticness; in the middle subplot, the tracks in Cluster 5 all have above average speechiness scores; in the bottom right subplot, Cluster 6 is characterized by extremely high instrumentalness scores, relative to the average for the full dataset.

Conclusions

This was a fun and interesting ML project which served a practical purpose for me and potentially others. Hopefully people can use this audio-based clustering methodology and save themselves a few clicks of the “skip” button next time they wish to listen to music of a particular mood or vibe. I believe it can be especially useful for people who curate monthly or yearly playlists which lack a cohesive theme. Additionally, it can be useful for large collaborative playlists which, at times, can feel a bit all over the place.

Thank you very much for reading! All of the source code is accessible here. If you would like to send me any feedback or inspiration, please contact me at sejaldua@gmail.com or through my website. I love connecting with people of similar interests! Also, stay tuned for a potential extension of this work! In the meantime, I will be busy thinking of better names for my new playlists because “cluster1” doesn’t have a great ring to it.

References