How to build a Movie Recommender System in Python using LightFm | by Arun Mathew Kurian | Towards Data Science

In this blog post, we will be creating a movie recommender system in python, that suggest new movies to the user based on their viewing history. Before we start let's have a quick look at what a recommender system is.
You may not know the definition of a Recommender system yet, but you have definitely encountered one before. This is because recommender systems are present everywhere on the internet. The purpose of a recommender system is to suggest users something based on their interest or usage history. So next time Amazon suggests you a product, or Netflix recommends you a tv show or medium display a great post on your feed, understand that there is a recommendation system working under the hood.
There are two types of recommendation systems. They are
Content-Based Recommender SystemCollaborative Recommender System
Content-Based Recommender System
Collaborative Recommender System
A content-based recommender system works on the data generated from a user. The data can be generated either explicitly (like clicking likes) or implicitly (like clicking on links). This data will be used to create a user profile for the user which contain the metadata of the items user interacted. More the data it receives more accurate the system or engine becomes.
A collaborative recommender system makes a recommendation based on how similar users liked the item. The system will group users with similar tastes. In addition to user similarity, recommender systems can also perform collaborative filtering using item similarity (like ‘Users who liked this item X also liked Y’).
Most systems will be a combination of these two methods.
Lest start coding
First, we need to install some packages.
LightFM is a Python implementation of a number of popular recommendation algorithms. LightFM includes implementations of BPR and WARP ranking losses(A loss function is a measure of how good a prediction model does in terms of being able to predict the expected outcome.).
BPR: Bayesian Personalised Ranking pairwise loss: It maximizes the prediction difference between a positive example and a randomly chosen negative example. It is useful when only positive interactions are present.
WARP: Weighted Approximate-Rank Pairwise loss: Maximises the rank of positive examples by repeatedly sampling negative examples until rank violating one is found
LightFm also contains a large set of datasets related to the movie rating. We will be working on this dataset. So we will install this library also.
pip install lightfm
Next, we will be installing two packages for mathematical operations namely numpy and scipy
pip install numpy
pip install scipy
We will create a python file called recommender.py.
We can start by importing the libraries into this file
import numpy as npfrom lightfm.datasets import fetch_movielensfrom lightfm import LightFM
fetch_movielens method is the method from lightfm that can be used to fetch movie data. We can fetch the movie data with a minimum rating of 4.
data = fetch_movielens(min_rating = 4.0)
The ‘data’ variable will contain the movie data that is divided into many categories test and train.
In a supervised learning, you use a training dataset, that contains outcomes, to train the machine. You then use testing dataset that has no outcomes to predict outcomes.Training Data is data for build model and Testing Data is data for test model.
We can check this by printing these data
print(repr(data[‘train’]))print(repr(data[‘test’]))
We can see that the amount of train data is much greater than the test data. This because typically when you separate a dataset into a training set and testing set, most of the data is used for training.
Next, we will be creating a lightfm model with ‘warp’ loss function
model = LightFM(loss = ‘warp’)
We can now train this model using our train data, with an epoch or iteration value of 30.
model.fit(data[‘train’], epochs=30, num_threads=2)
Now that's done let's build the function that process this data to recommend movies for any number of users. Our function will take the model, data and an array of user_ids.
def sample_recommendation(model, data, user_ids):
In the function definition, first, we need to get the number of all users and movies.
n_users, n_items = data[‘train’].shape
Next, we need to iterate the user_ids.
for user_id in user_ids:
For each user, we need to find the known positives or the movies they liked. This can be obtained from the data we have.
known_positives = data[‘item_labels’][data[‘train’].tocsr()[user_id].indices]
Next, we need to find the movies the user like. This can be done by the predict method of LightFm. The parameters of this function are user_id and the n_items variable arrange by numpy arrange
scores = model.predict(user_id, np.arange(n_items))
We can now sort the scores based on the order of most liked to least liked.
top_items = data[‘item_labels’][np.argsort(-scores)]
Now that the prediction is done we can print the first 3 known positives and 3 predictions.
print(“User %s” % user_id)print(“ Known positives:”)for x in known_positives[:3]: print(“ %s” % x)print(“ Recommended:”)for x in top_items[:3]: print(“ %s” % x)
Thus our sample_recommendation method becomes
def sample_recommendation(model, data, user_ids):    n_users, n_items = data['train'].shape    for user_id in user_ids:        known_positives = data['item_labels'][data['train'].tocsr()                                                              [user_id].indices]                scores = model.predict(user_id, np.arange(n_items))        top_items = data['item_labels'][np.argsort(-scores)]        print("User %s" % user_id)        print("     Known positives:")                for x in known_positives[:3]:            print("        %s" % x)                print("     Recommended:")                for x in top_items[:3]:            print("        %s" % x)
We can finish by invoking the function in our program, by providing three random user_ids.
sample_recommendation(model, data, [3, 25, 451])
Let's run the code in the terminal by the command
python recommender.py
and we get the output
Cool...We can see that our system recommends movies to our users.
This is an example of how easily a recommender system can be implemented. I hope you find it useful.
The code can be found at https://github.com/amkurian/movie-recommendation-system