A Simple Song Recommender System in Python (Tutorial) | by Jack Bandy | Towards Data Science

This material is based on content from a workshop I taught in April 2020.
Most of us now use AI-based recommender systems every day: looking for a show on Netflix, scrolling through social media feeds, watching “up next” videos on YouTube, and listening to “Discover Weekly” playlists on Spotify.
So how do these recommenders work?
They have become really good, and in some cases, creepily good. Many of us have received recommendations and advertisements that make us ask: “were they recording me?” Generally speaking, companies do not make recommendations by hacking your microphone and recording your conversations. Instead, they make a model of you, or as Tristan Harris called it, “a little voodoo doll, avatar-like version of you,” and then make predictions about that doll. I’ll refer to this doll as a mathematical model of your tastes.
The methods in this tutorial are nowhere near as sophisticated as what Google, Facebook, and Spotify are using, and they will not produce any creepily good recommendations. Still, most recommendations principally work the same, and are bound up in three steps:
Companies make a mathematical model of you and everyone elseThey use those models to find people who are similar to youThey find what those similar people liked, and recommend it to you
Companies make a mathematical model of you and everyone else
They use those models to find people who are similar to you
They find what those similar people liked, and recommend it to you
I’ll go over these three steps twice, first in a simple visual example, and then in a larger code-based example.
Let’s start by considering three people (Jack, Nick, and Trevor) and how they rated three songs (One Dance, Lean On, and Sunflower) on a scale of 1 to 5. Here they are in a simple table:
We now have a simple mathematical model of musical taste. In the real world, Spotify uses other signals, such as how many times you listen to a song, to determine this rating. Regardless, the principle is to translate your tastes into a list of numbers, which is also called a vector, a matrix, or an array. Whatever you call them, it’s helpful to visualize these numbers on a graph. Below, we put the rating for “One Dance” on the x-axis, and the rating for “Lean On” on the y-axis:
We now have a list of numbers that capture your musical tastes, so we can go to the next step: finding people with similar tastes. Let’s go back to the graph:
In the visual representation, similar people are just nearby people. So figuring out who is “most similar” to Jack is just a matter of figuring out who is closest to Jack. And we can measure this distance using the old Pythagorean theorem from middle school geometry:
Cool! We’re almost there. We can already see that Jack is most similar to Nick and not Trevor, but let’s make it official:
The maths confirm: in this tiny universe of song tastes, Jack is 3.2 units away from Trevor, but only 1.4 units away from Nick.
Now that we know Jack and Nick have similar taste, we can predict whether Jack will like “Sunflower” based on whether Nick liked sunflower. It’s just a matter of looking it up in the table:
Nick gave “Sunflower” a 5/5 rating, so we can predict that Jack will probably like the song, too.
Obviously, the scale of this tiny universe is a lot smaller than Spotify’s universe. We looked at three songs and three people, while Spotify purportedly has over 50 million tracks and over 271 million active users. So now, we’ll scale up.
To make a bigger dataset, we will load in a larger table. Here is one with 25 songs and 8 people:
In Python, you can use Pandas to load this data with the following command:
from pandas import read_csvdata_url = 'https://gist.githubusercontent.com/jackbandy/5cd988ab5c3d95b79219364dce7ee5ae/raw/731ecdbecc7b33030f23cd919e6067dfbaf42feb/song-ratings.csv'ratings = read_csv(data_url,index_col=0)
Now, let’s try to predict whether I (still Jack) will like the song “7 Rings.” As in the visual example, we will find the person closest to me. Using the Pythagorean theorem, the distance function in two dimensions was just distance=sqrt(a^2 + b^2) , which we could write like this:
def distance(person1,person2):  a_squared = (person1[0] - person2[0])**2  b_squared = (person1[1] - person2[1])**2  c = sqrt(a_squared+b_squared)  return c
But this time, there are 25 dimensions, not just two. How do we find the closest person when we have 25 dimensions instead of just an x-axis and a y-axis?
It turns out the trusty Pythagorean theorem generalizes pretty well. (Most recommenders use other distance metrics like cosine distance, but we’ll ignore that for this tutorial). If there are three dimensions, we can call them a , b , and c , and then just add it under the square root:distance=sqrt(a^2 + b^2 + c^2) . And we can actually do this with any number of dimensions: distance=sqrt(a^2 + b^2 + c^2 + d^2 + ... + n^2) .
The scipy library has written a function to do exactly what we need: calculate Euclidean distance in any number of dimensions. Let’s use it:
from scipy.spatial.distance import euclideandef distance(person1,person2):  distance = euclidean(person1,person2)  return distance
Now, we can use that distance function to look at the similarity between any two people. The only catch is that we need to fill in blanks with “0,” using fillna . The following block does that, and then retrieves the rows of data for the three people:
ratings=ratings.fillna(0)jack=ratings.loc['Jack']nick=ratings.loc['Nick']trevor=ratings.loc['Trevor']
Then, we can look at the distance between Jack and the other two people:
print("Distance between Jack and Nick:")print(distance(jack,nick))# 10.63014581273465print("Distance between Jack and Trevor:")print(distance(jack,trevor))# 13.490737563232042
Just like in the tiny universe, Jack is closer to Nick (10.63 units away) than to Trevor (13.49 units away). But really, we want to look at the distance between Jack and every other person. Here’s a function that uses a “for loop” to do just that:
def most_similar_to(name):  person = ratings.loc[name]  closest_distance=float('inf')  closest_person=''  for other_person in ratings.itertuples():    if other_person.Index==name:      # don't compare a person to themself      continue    distance_to_other_person = distance(person,ratings.loc[other_person.Index])  if distance_to_other_person < best_similarity:      # new high score! save it      closest_distance = distance_to_other_person      closest_person = other_person.Index  return closest_person
It starts by saying: “the closest person so far is infinitely far away.” Then, it loops through to look at each person and asks “is this person closer than the closest person so far?” It’s not very helpful to answer “the person most similar to Jack is Jack,” which is why the loops skips (i.e., continue ) if the names are the same.
Now, we can just call most_similar_to to see who is closest to any person. Let’s try it for me (Jack):
print("Jack is most similar to:")person_most_similar_to_jack = most_similar_to('Jack')print(person_most_similar_to_jack)print(distance)# Meg# 8.660254037844387
At last, we can predict whether I will like “7 Rings” by looking at the user who is most similar to me. At the time, the data said this was Meg, so let’s see how Meg rated the song:
print(ratings.at['Meg', '7 Rings (Ariana Grande)'])# 3.0
Meg rated the song at 3/5, so I will probably not be too crazy about it.
A Google Colab notebook with all the code is available here.
If you really wanted to do what Spotify is doing, you would make predictions using musical tastes of every person, not just the closest person. For example, maybe if I knew that Trevor and I consistently had very different tastes, recognizing that Trevor liked “7 Rings” might tell me that I would dislike the song. I may cover this in a future tutorial, so let me know if you are interested!