Explore your activity on Spotify with R and “spotifyr”: How to analyze and visualize your streaming history and music tastes | by Saúl Buentello | Towards Data Science

I can say with great pride that I was one of the people who still had the pleasure of going to the best music CD stores to buy the latest releases from their favorite artists. I remember that being only about 15 years old (age in which most of us adopt a musical genre, bands, and artists with whom we feel identified), it represented a whole ritual for me, from the tireless search to that magical moment in which that first note of the first track played on the stereo or on the Discman with the headphones on properly, and at full volume. Music is most likely one of those particular things that any civilization from another galaxy (if any) would signal us by identifying ourselves as the human race.
“After silence, that which comes nearest to expressing the inexpressible is music”- Aldous Huxley.
Like everything in life, as time goes by, music also changes and transforms, evolves (or strangely mutates?). And this also happens with the way we listen to and consume music. Today, 17 years later, my ritual ceased to exist and there are very few stores that sell musical CDs, over time I completely replaced them with the convenience of being subscribed to the most popular music streaming service in the world: Spotify. According to recent statistics, 35% of subscribers to a music streaming service worldwide have a subscription with Spotify, almost double the proportion who are subscribed to Apple Music, and well above Amazon, Deezer, and other platforms.
That is why I found it interesting to be able to take advantage of the fact that Spotify, like almost any application nowadays, allows you to download the personal data that it protects from your account. This includes your playback activity, search history, among other things in JSON format.
Yes, I know that Spotify year after year very kindly offers you your “year in review”, with information about your habits and consumption. Surely in December, you received an email inviting you to explore your activity during the year.
However, it is always interesting and more fun to see how things are done on your own, in addition to scrutinizing other data that Spotify would not provide you on its own.
I also want you to know an interesting package, called “spotifyr” that allows you to easily take a deeper look at your musical tastes with the Spotify API. But hey, as Jack the Ripper would say ... Let’s go in parts.
You will have to log in with your username and password to be able to enter the URL of privacy settings of your personal Spotify account: https://www.spotify.com/us/account/privacy/. There you will find that almost at the end of the page there is a section with the text “Download your data”, which invites you in three simple steps to download your data.
Once you make the request, for safety Spotify will be sending you an email to verify that it was indeed you who requested this copy of the data. It is necessary to confirm the email you receive.
When confirming, it will be almost immediately reaching you one more email, where Spotify will notify you that your request has been successful, but that you must wait up to 30 days in which it collects your information to send it to you. Do not despair and wait until further notice.
I actually think it depends a lot on how intense your activity has been with the platform. Testing with other accounts with greater activity than mine, took almost thirty days, while in my case, three days after making the request, I received the final email inviting me to download my data. You will find yourself after a couple of days then with an email like the following.
By clicking on the “Download” button you will be redirected once more to the privacy settings page of your Spotify account, where for security reasons, even if you are already logged in, it will ask you to enter your password again. Once this is done, you will automatically be seeing in your browser an active download of a file with the name “my_spotify_data.zip”. Remember that as mentioned in the last email, you only have a maximum of 14 days from when it was sent to you to be able to download your file, otherwise, you will have to make a new request.
Congratulations, you now have your copy of personal data on Spotify. I hope you didn’t have to wait too many days.
When you unzip the file you have downloaded, you will find a folder called “My Data” that contains different files in JSON format, including a very particular one with the information that interests you: your playback history, with the file name “StreamingHistory0.json”.
If you have had more intense activity than mine on your account, you will most likely find yourself with more than one file, followed by the next number. For example: “StreamingHistory1.json”, “StreamingHistory2.json”, etc. Spotify will be dividing your entire history into files no larger than 1.5 MB. A detail to take into account: Spotify keeps only your data and history for one year. It would be great to have a complete history, for example in my case I have been using the platform for just over six years, but hey, for now, it will be enough to analyze the data from the last year.
Now, you can get to work creating a new R script. The first thing you have to establish is the load of the libraries that you will be using, in addition to the reading of your history file in JSON format.
# REQUIRED LIBRARIESlibrary(jsonlite)library(lubridate)library(gghighlight)library(spotifyr)library(tidyverse)library(knitr)library(ggplot2)library(plotly)# READING JSON STREAMING HISTORYstreamHistory <- fromJSON("StreamingHistory0.json", flatten = TRUE)
You will find that your history contains basically four variables that you will be playing with “endTime”, “artistName”, “trackName” and “msPlayed”.
ou will also notice that you have loaded the “spotifyr” library, of which I am telling you for now as an introduction, it belongs to the package created by the user in GitHub “charlie86”, and it will allow you to easily use the Spotify API to obtain additional details. You can find beyond what you do with the examples in this article, other interesting examples in its repository: https://github.com/charlie86/spotifyr.
This is the first question that you could answer. Starting from our variables, you can define the hours, minutes and seconds, and basically any temporality. You can make a first plot to observe the behavior of your activity on Spotify throughout the year, per week for example.
# ADDING DATE AND TIMINGmySpotify <- streamHistory %>%   as_tibble() %>%   mutate_at("endTime", ymd_hm) %>%   mutate(endTime = endTime - hours(6)) %>%   mutate(date = floor_date(endTime, "day") %>% as_date, seconds = msPlayed / 1000, minutes = seconds / 60)# PLAYBACK ACTIVITY PER WEEK AND HOURSstreamingHours <- mySpotify %>%   filter(date >= "2020-01-01") %>%   group_by(date) %>%   group_by(date = floor_date(date, "week")) %>%  summarize(hours = sum(minutes) / 60) %>%   arrange(date) %>%   ggplot(aes(x = date, y = hours)) +   geom_col(aes(fill = hours)) +  scale_fill_gradient(low = "yellow", high = "red") +   labs(x= "Date", y= "Hours of music playback") +   ggtitle("On what dates I've listened to more or less music on Spotify?", "Playback activity per week")streamingHours
As a result, you will get a plot like the following one, wherein in my case, the decrease in the use of Spotify after April is very visible. Perhaps to a large extent, this is because in April 2020, when we were forced to work from home (and to date) by covid-19, the times when I most enjoyed listening to music were at the office, and today working at home we take turns playing each one their music on their platforms.
Another question that you could answer in a very simple way and visualize with a line chart, you can also highlight the results you are looking for using the “gghighlight” library, and through the artistName variable.
# PLAYBACK ACTIVITY PER SPECIFIC ARTISThoursArtist <- mySpotify %>%   group_by(artistName, date = floor_date(date, "month")) %>%   summarize(hours = sum(minutes) / 60) %>%   ggplot(aes(x = date, y = hours, group = artistName)) +   labs(x= "Date", y= "Hours of music playback") +   ggtitle("On what dates I've listened to more or less music by a specific artist?", "E.g: Alton Ellis and Jarabe de Palo") +  geom_line() +   gghighlight(artistName == "Alton Ellis" || artistName == "Jarabe De Palo") hoursArtist
For example, in my case, I took two artists that I usually listen to: “Alton Ellis”, a rocksteady genre icon, and “Jarabe de Palo”, a rock in Spanish icon, obtaining the following plot as a result. There is a very notorious peak listening to the latter since it was only in June 2020 that his vocalist, Pau Donés, unfortunately, passed away, the date on which I was playing many of his tracks as a tribute.
You can set a minimum of playing hours and find out from there which artists you listened to the most. For example, let’s say that you consider that the most listened to were those artists to whom you dedicated at least 3 hours of playback or more.
# MOST LISTENED ARTISTS (MORE THAN 3 HOURS)minutesMostListened <- mySpotify %>%   filter(date >= "2020-01-01") %>%   group_by(artistName) %>%   summarize(minutesListened = sum(minutes)) %>%   filter(minutesListened >= 180) %>%  ggplot(aes(x = artistName, y = minutesListened)) +   geom_col(aes(fill = minutesListened)) +  scale_fill_gradient(low = "yellow", high = "red") +   labs(x= "Artist", y= "Minutes of music playback") +   ggtitle("What were the most listened artists on my Spotify?", "> 3 hours listened") +  theme(axis.text.x = element_text(angle = 90))minutesMostListened
You will get a plot similar to the following one, in which you can view these artists. In my case, like many of you, I share my account with my girlfriend, so I can mostly observe a variety of Latin pop artists of the moment.
You could view by the hour of the day inclusive, the activity log of your complete history with a heatmap, to observe how this habit has changed over time if it is the case.
# PLAYBACK ACTIVITY BY DATE AND TIME OF DAYtimeDay <- mySpotify %>%   filter(date >= "2020-01-01") %>%   group_by(date, hour = hour(endTime)) %>%   summarize(minutesListened = sum(minutes)) %>%   ggplot(aes(x = hour, y = date, fill = minutesListened)) +   geom_tile() +   labs(x= "Time of the day", y= "Date") +   ggtitle("When has there been more playback activity on my Spotify?", "Activity by date and time of day") +  scale_fill_gradient(low = "yellow", high = "red")timeDay
As a result, you will find the following plot, wherein a very visual way you can see if there have been changes in your usual hours when you usually listen to music. In my case, for example, after April 2020 the constant habit of listening to music mostly at night changed.
You can also create a bar chart to take a closer look in detail at the times of the day when there is a record of the highest activity on your account.
# PLAYBACK ACTIVITY BY TIME OF THE DAYhoursDay <- mySpotify %>%   filter(date >= "2019-01-01") %>%   group_by(date, hour = hour(endTime), weekday = wday(date, label = TRUE))%>%   summarize(minutesListened = sum(minutes))hoursDay %>%   ggplot(aes(x = hour, y = minutesListened, group = date)) +   geom_col(fill = "#ff6600") +  labs(x= "Time of the day", y= "Minutes of music playback") +   ggtitle("What time of day I've listened to the most music on Spotify?", "Activity from 0 to 24 hours")
You will be able to observe the specific time in which there are more accumulated minutes of reproduction. In my case, for example, there is no time of day in which I have not used my account for at least a couple of minutes to listen to something, even in those moments of insomnia.
You can also answer this question by creating another heatmap, but this time visualizing the relationship between day of the week and time of day.
# PLAYBACK ACTIVITY BY TIME OF THE DAY AND WEEKDAYhoursDay %>%   group_by(weekday, hour) %>%   summarize(minutes = sum(minutesListened)) %>%   ggplot(aes(x = hour, weekday, fill = minutes)) +   geom_tile() +   scale_fill_gradient(low = "yellow", high = "red") +  labs(x= "Time of the day", y= "Weekday") +   ggtitle("What weekday and time of day I've listened to the most music on Spotify?", "Weekly activity from 0 to 24 hours")
You will be able to observe which day (s) of the week mark the trend of greater activity in your account. For example, in my case, I can clearly see that on Tuesdays, Thursdays, and Fridays there is a greater record of activity.
Another way in which you can take a look at the details previously obtained could be by creating a line chart.
# PLAYBACK ACTIVITY BY TIME OF THE DAY AND WEEKDAY - LINE CHARTweekDay <- hoursDay %>%   group_by(weekday, hour) %>%   summarize(minutes = sum(minutesListened)) %>%   ggplot(aes(x = hour, y = minutes, color = weekday)) +   geom_line() +  labs(x= "Time of the day", y= "Minutes of music playback") +   ggtitle("What weekday and time of day I've listened to the most music on Spotify?", "Line chart - Weekly activity from 0 to 24 hours") weekDay
I can see in my case that Sundays are definitely the days that I listen to music the least. You will get as a result a plot similar to the following one where you can also see which is the day when you have the least playback activity.
Although it is to be obviated after the visualizations obtained previously, you could also create one more plot to see what type of day (weekdays or weekends) you have had the most activity in your account.
# PLAYBACK ACTIVITY BY DAY TYPEdayType <- hoursDay %>%   mutate(day_type = if_else(weekday %in% c("Sat", "Sun"), "weekend", "weekday")) %>%   group_by(day_type, hour) %>%   summarize(minutes = sum(minutesListened)) %>%   ggplot(aes(x = hour, y = minutes, color = day_type)) +   geom_line() +  labs(x= "Time of the day", y= "Minutes of music playback") +   ggtitle("What day type I've listened to the most music on Spotify?", "Weekday and weekend activity from 0 to 24 hours") dayType
Then you will obtain a plot like the following one, identifying the activity of your account by the type of day and time of day.
It’s time to take a look at how the “spotifyr” package works. As I mentioned at the beginning, basically this package will allow us to connect with the Spotify API. Spotify stores extremely interesting data about the tracks, artists, and albums that it hosts on its platform, such as popularity, energy, valence, how “danceable” it is, among many other things.
The first thing you have to do to be able to start consuming such data through “spotifyr” is to establish a connection from your application in R. For this you need to obtain mainly two parameters: SPOTIFY_CLIENT_ID and SPOTIFY_CLIENT_SECRET. These are obtained from the Spotify portal for developers, creating a new application at https://developer.spotify.com/dashboard/applications where you will be asked to log in with the same credentials as with your Spotify account.
Once logged in you will find a dashboard where you can create and manage your created applications. You have to click on the “Create an app” button to establish a new application and assign it a name and a description.
Immediately then you will see the two parameters that we need, both the client ID and the Client secret. Obviously, I edited the image that I put as an example below hehe, but yours should appear both complete.
The author of the package “spotifyr” in his repository lets us know the following, I quote:
For certain functions and applications, you’ll need to log in as a Spotify user. To do this, your Spotify Developer application needs to have a callback URL. You can set this to whatever you want that will work with your application, but a good default option is http://localhost:1410/
In other words, finally, you have to add that URL callback (http:// localhost:1410/), clicking on the green “Edit settings” button that you will find right there, and proceed to save the changes.
Excellent, now you have everything ready to start using “spotifyr”. In your R script, you must first execute a function to authorize it with Spotify.
# ESTABLISH CONNECTION SPOTIFY APISys.setenv(SPOTIFY_CLIENT_ID = 'YOUR_CLIENT_ID_HERE')Sys.setenv(SPOTIFY_CLIENT_SECRET = 'YOUR_CLIENT_SECRET_HERE')get_spotify_authorization_code()
Once the “get_spotify_authorization_code()” function is executed, you will be seeing a new window in your browser where your application will be requesting permissions.
When accepting, you will immediately see a text indicating that the authentication was successful, so you can now close that window, to return to working in R.
Everything okay up to here? Perfect, so now, you can take advantage of all the scope that “spotifyr” has for you from R.
You can give a first approach to the data that you can obtain with “spotifyr”, at the same time that you answer this question. I am very sure that like me, dear reader, you have one or more playlists that you have created either with your favorite songs, or your favorite genres or for those special occasions. Logged into your account on Spotify, choose any of your playlists from your library, and copy the unique identifier of its URL.
For example, I have a playlist with my favorite songs in English, I’ll take that playlist as an example, but you can do it with anyone. For this, you need that identifier that I mentioned, in addition to your username, to pass them as parameters and make use of the “get_playlist_audio_features()” function, with which you will be obtaining a lot of information from that playlist generating a new data frame.
# GET SPECIFIC PLAYLIST FEATURESplaylist_username <- 'cosmoduende'playlist_uris <- c('0PV31w7ireeI6d0oSYJ2H0')playlistFavsEnglish <- get_playlist_audio_features(playlist_username, playlist_uris)
Stopping to see each variable that this new data frame contains would take us a long time, I suggest you take a look at its content in R carefully later. You will be amazed at the valuable data you can play with.
Returning to the initial question, one of the variables with which you can answer it is with “track.popularity”, which contains the popularity of each track on a scale from 0 to 100. Considering this, you can establish that the least popular tracks are those ranging from 0 to 35 and visualize which are those rarities that you hear within your playlist, for Spotify.
# PLOT LESS POPULARITY TRACKS ON SPECIFIC PLAYLISTplaylistFavsEnglish %>%   group_by(track.popularity) %>%   filter(track.popularity <= "35") %>%  ggplot(aes(x = track.name, y = track.popularity)) +   geom_col(aes(fill = track.album.name)) +  labs(x= "Track name", y= "Popularity") +   ggtitle("What are the least popular songs I listen to on Spotify?", "Popularity ranking < 35 in a specific playlist") +  theme(axis.text.x = element_text(angle = 90))
As a result, you will get a plot similar to the following one, where you can see which are those less popular songs from the playlist you chose. Clearly depending on how big your playlist is, you may get a greater or lesser amount of tracks.
You may have noticed that as many applications, with Spotify it is no stranger to give “likes” a.k.a. “♥” to songs or even entire albums that you listen to. Either from your desktop application, from the app, or from the web version, it is always available.
This greatly allows Spotify to improve its recommendation algorithm as well, in case you were wondering, to be able to make new suggestions on what else you could listen to based on your tastes.
You can get an overview of all those tracks that you have added over time to your Spotify account at https://open.spotify.com/collection/tracks
It is time to use the “get_my_saved_tracks()” function to obtain all the information regarding this complete collection of songs that at some point you have “liked”, generating a new data frame with a lot of new information. Spotify sets a request limit, you can get up to 50 items per request but you can get more with “offset”, to skip items and get more.
# GET FAVORITE TRACKSmyFavTracks <- ceiling(get_my_saved_tracks(include_meta_info = TRUE)[['total']] / 50) %>%  seq() %>%  map(function(x) {    get_my_saved_tracks(limit = 50, offset = (x - 1) * 50)  }) %>%   reduce(rbind) %>%  write_rds('raw_myFavTracks.rds')
It would be interesting if you got and remembered what was that first track that you gave a “like” since you opened your account on Spotify. You will use the package “lubridate” to handle the date, to convert the date from a character class to a date class.
# GET FIRST ADDED LIKED TRACKmyFavTracks %>%  mutate(added_at = ymd_hms(added_at)) %>%  arrange(added_at) %>%  head(1, wt = added_at) %>%  select(track.name, track.album.name, added_at)  %>%  kable()
For example in my case, I can see that the first song was by Vicentico in 2015, lead singer of “Los Fabulosos Cadillacs”, one of my favorite “Rock in Spanish” bands.
The variable “track.artists” is stored as a list, keep in mind that not all songs are made by a single artist, so you need to extract from lists into lists. You will select the “id” as the artist brand in this case to avoid duplicate names and then, you will add “n”, as the column of the number of tracks. Finally, you can create the plot to view the results obtained. To avoid altering the order of the frequency group, I would suggest converting the category variables as factor variables.
# GET TOP ARTISTS BASED ON LIKED TRACKSfavTracksArtist <- myFavTracks %>%  select(track.artists) %>%  reduce(rbind) %>%  reduce(rbind) %>%  select(id, name)trackNumArtist <- favTracksArtist %>%  count(id, sort = TRUE) %>%  left_join(favTracksArtist, by = 'id',.) %>%  unique() %>%  select(-id) %>%  top_n(10, n)# PLOT TOP 10 ARTISTS BASED ON LIKED TRACKSplotMyFavs <- trackNumArtist %>%  mutate(freq = case_when(n > 100 ~ '> 100 tracks',      between(n, 50, 99) ~ '50-99 tracks',      between(n, 20, 49) ~ '20-49 tracks',      TRUE ~ '< 20 tracks')) %>%  mutate(freq = factor(freq, levels = c('> 100 tracks', '50-99 tracks', '20-49 tracks', '< 20 tracks'))) %>%  ggplot(mapping = aes(x = reorder(name, -n), y = n, fill = freq)) +  geom_col() +  scale_fill_brewer(palette="Dark2") +  labs(x= "Artist name", y= "Number of tracks", fill = NULL) +  ggtitle("What are my Top 10 favorite artists?", "Based on my ♥ tracks") +  theme(axis.text.x = element_text(angle = 90))plotMyFavs
You will get as a result a plot similar to the following one, in which you can see your top 10 favorite artists, segmented by the number of tracks of each one that you have “liked”. Yes, you can tell I’m a Bunbury fan, for example.
We are especially attracted to certain artists or certain musical genres because we feel identified, so we make it part of ourselves and our personal culture. In my opinion, especially music can reveal a lot about your personality.
Continuing with the information obtained above, it would also be interesting to be able to see what feeling your favorite artists express in their musical creations mostly. Do they convey peace, joy, anger, sadness? Well, you can find this out using two measures that Spotify has established: Valence and Energy. Defined by Spotify, valence defines it as follows:
A measure from 0.0 to 1.0 describing the musical positiveness conveyed by a track. Tracks with high valence sound more positive (e.g. happy, cheerful, euphoric), while tracks with low valence sound more negative (e.g. sad, depressed, angry).
On the other hand, energy defines it as follows:
Energy is a measure from 0.0 to 1.0 and represents a perceptual measure of intensity and activity. Typically, energetic tracks feel fast, loud, and noisy. For example, death metal has high energy, while a Bach prelude scores low on the scale. Perceptual features contributing to this attribute include dynamic range, perceived loudness, timbre, onset rate, and general entropy.
With these two measurements, you can create a quadrant with a scatter plot that reveals this information. You can make use of the “get_artist_audio_features()” function this time, which basically expects the name of your artist as a parameter, to return you a lot of information about her albums and tracks, including valence and energy. To exemplify, take only the first four favorite artists of your previously obtained top 10, to later create and visualize the results obtained in an “emotional quadrant”.
# GET FEATURES TOP FOUR FAVORITE ARTISTSfavArtist1 <- get_artist_audio_features(artist= "Bunbury")favArtist2 <- get_artist_audio_features(artist= "Kevin Johansen")favArtist3 <- get_artist_audio_features(artist= "Vicentico")favArtist4 <- get_artist_audio_features(artist= "Los Auténticos Decadentes")# MAKE A SINGLE DATA FRAMEtopFourArtists <- rbind(favArtist1, favArtist2, favArtist3, favArtist4)# PLOT EMOTIONAL QUADRANT TOP FOUR ARTISTSemotionalQuadrant <- ggplot(data = topFourArtists, aes(x = valence, y = energy, color = artist_name)) +  geom_jitter() +  geom_vline(xintercept = 0.5) +  geom_hline(yintercept = 0.5) +  scale_x_continuous(expand = c(0, 0), limits = c(0, 1)) +  scale_y_continuous(expand = c(0, 0), limits = c(0, 1)) +  annotate('text', 0.25 / 2, 0.95, label = "Angry / Turbulent") +  annotate('text', 1.75 / 2, 0.95, label = "Joyful / Happy") +  annotate('text', 1.75 / 2, 0.05, label = "Peace / Chill") +  annotate('text', 0.25 / 2, 0.05, label = "Depressing / Sad") +  labs(x= "Valence", y= "Energy") +  ggtitle("Emotional quadrant Top four artists", "Based on energy y valence")  emotionalQuadrant
As you can see, each point represents a track by a specific artist, showing what feeling she expresses. Take into account that you are viewing the tracks of the entire discography of each artist, or at least of all the albums that Spotify has a record of. What do you think, does this reveal something about your personality too?
Thank you very much for your kind reading. As with most of my articles, I share the plots generated with plotly in a flexdashboard that I put together a little more aesthetic: https://rpubs.com/cosmoduende/spotify-history-analysis
Here you can find the complete code: https://github.com/cosmoduende/r-spotify-history-analysis
I thank you for making it to the end, I wish you a very happy analysis, that you can put everything into practice, and that you are surprised and have as much fun as I am with the results!
Some rights reserved