Create Music Recommendation System Using Python | by Ajinkya Khobragade | Towards Data Science

Hello folks! Recommendation Systems are everywhere and pretty standard all over the web. Amazon, Netflix, and many such companies are using Recommendation Systems. The one we are going to build is pretty common to what Spotify or Youtube Music uses but much more straightforward.
Tools required to follow along are Tableau and Python3. You can download Tableau from here. Let me take you through the dataset first, and later I will provide a brief introduction of Tableau in the data visualization section.
The Million Song Dataset is a freely-available collection of audio features and metadata for a million contemporary popular music tracks.
Below are the links to the datasets :
https://static.turi.com/datasets/millionsong/10000.txt
https://static.turi.com/datasets/millionsong/song_data.csv
These are the two datasets that we need to combine. We are going to use Python3 to connect the data set and import it into Tableau for data visualization.
#importing all the required librariesimport numpy as npimport pandas as pd#importing both the datasetssongmetadata = pd.read_csv(r'Path where the file is located')#one of the file is a text file hence we import it with pd.read_fwf#fwf stands for fixed width fileothersongdata = pd.read_fwf(r'Path where the file is located')#naming the columns for the othersongdataothersongdata.columns = ['user_id','song_id','listen_count’]#merging both the datasets and removing duplicatessong_df = pd.merge(othersongdata, songmetadata.drop_duplicates(['song_id']), on="song_id", how="left")#writing the file in .csv to visualize in Tableausong_df.to_csv(r'Path where you want to store the exported CSV file\File Name.csv', index = False)
After combining the dataset, what we get is a million song dataset. The dataset has more than a million observations. The dataset consists of seven variables.
Song_id = Object#Unique ID for every song in the dataset, in total there are 1000 songs in the datasetUser_id = Object #Unique ID for every userListen_count = int #Number of times a song was listened by an userArtist_name = Str #Name of Artist Title = Str #Title of each songYear = int #Categorical variable with lot of zero valuesRelease = Str#Album to which the songs belong to
“Visualization gives you answers to questions you didn’t know you had.” — Ben Schneiderman
It’s always a good practice to visualize the data, to understand it, and find insights. I use Tableau because it’s quick and gives a ton of options to visualize datasets. Tableau has significantly improved my workflow, and also, did I mention that it’s free for students?
I am not going to lie, and I love TreeMaps. I have used them extensively in this project to display data. They are beautiful and customizable and demand attention.
Wikipedia has already explained it in an extremely digestible format than I ever could, check it out below:
Treemaps display hierarchical (tree-structured) data as a set of nested rectangles. Each branch of the tree is given a rectangle, which is then tiled with smaller rectangles representing sub-branches. A leaf node’s rectangle has an area proportional to a specified dimension of the data.[1] Often the leaf nodes are colored to show a separate dimension of the data. — Wikipedia
While writing this article, I thought, what can be the best way to explain someone, “How to create TreeMaps in Tableau?”. And I didn’t want to go through the screenshots route. What could I possibly do? So I made an easy to understand video.
To get the data into Tableau, you need to convert the .csv into excel workbook. You can easily do it with the command below in python:
song_df.to_excel("millionsong.xlsx")
Some times conversion takes a tremendous amount of time, so I am providing a ready to download excel workbook here.
In no way, this is a comprehensive guide to Tableau, and it is just an overview to get you familiar with how Tableau works and how TreeMaps are made. To learn Tableau, check out the free training, the official Tableau website provides here.
The first TreeMap represents the Proportion of Year variable in the dataset with respect to Listen_count. The graph displays that we have many zeros in the dataset. Since we are not going to use the Year variable while creating the music recommendation system, we are not going to remove these observations.
The second TreeMap represents the Artist with respect to total Listen_count in the dataset. Coldplay is the most listened artist in the dataset with 36,406 hits, followed by Kings Of Leon with 32,552 hits.
Too many choices can overwhelm users. If offered too many options, the user may not buy anything. Streaming services like Spotify have massive catalogs. Identifying the tracks a user might like and recommending the product they might like is crucial for their business.
We are going to create two playlists, one being a Popular Playlist, and the second is to identify songs that are similar to a song a user already likes — hence giving them personalized recommendations.
CBRS recommends items based on its features and the similarity between elements of other items. Assume a user has already seen a movie from the genre of Comedy, CBRS will recommend movies that also belong to the Comedy genre.
User preferences and attitude is considered to create CFRS. CFRS recommends items similar to what the user has already chosen. We are going to use the Pearson Correlation to calculate the relationship between similar tracks.
Why are we not using Association rather than Correlation?
Association only describes the relationship between two variables, while Correlation describes not only the relationship but also the strength. Hence when we create a Collaborative-Filtering Recommendation System, we use Correlation.
We are going to create the Popular Playlist first. All we need is some data wrangling. I have tried explaining the code in the comments. Still, if you have questions, you can comment them below.
Now we are going to make the CFRS and create Personalized Playlist. The observation that I used was, and the results of the Recommendation System are given below.
If you are still with me, Thank you. Someone taking the time out and going through my work means a lot. Don’t be shy, because I am. Please connect with me on my LinkedIn and say ‘Hello!’, it would make my day.