Make your Own Book and Movie Recommender System using Surprise | by Maha Amami | Towards Data Science

I spent my Ph.D. years building recommender systems and benchmarking them all from scratch and it is only when I finished it that I heard about the sikit library Surprise.
It was a Surprise that I wished to know before. It would be spare me time and effort in implementing many recommender system baselines.
So I want to share information about this library here and give some code on how to use it as it may help one of the readers. Let’s dive in!
Surprise (stands for Simple Python RecommendatIon System Engine) is a Python library for building and analyzing recommender systems that deal with explicit rating data. It provides various ready-to-use prediction algorithms such as baseline algorithms, neighborhood methods, matrix factorization-based ( SVD, PMF, SVD++, NMF), and many others. Also, various similarity measures (cosine, MSD, Pearson...) are built-in. Source: http://surpriselib.com/
I will use Surprise on two different datasets:
To build a movie recommender system.To build a book recommender system.
To build a movie recommender system.
To build a book recommender system.
In both, I will use collaborative filtering techniques and content-based techniques to filter items, and no worries I will explain the differences between them.
pip install surprise
I use here the MovieLens dataset. It contains 25 million users ratings. the data are in the ./data/raw folder. We could load directly the .csv file with a built-in Surprise function, but it’s more convenient to load it through a Pandas data frame for later flexibility purposes.
I use only a subset of this dataset for complexity matters but you can use it all. Then, I upload the ratings to a Surprise Dataset object which contains the following fields in this order:
userIdmovieIdThe corresponding rating (usually on a scale such as 1–5)
userId
movieId
The corresponding rating (usually on a scale such as 1–5)
Collaborative filtering methods rely on the preferences of the user as well as those of other users in the system. The idea is that if two users are like-minded (agreed in the past on some preferences of items), relevant items to one user are recommended to the other user and vice versa. They derive recommendations only from the user-item rating matrix. Source: my PhD thesis report
Surprise implements a handful of algorithms that are collaborative such as SVD, NMF, etc.
0it [00:00, ?it/s]RMSE: 0.9265RMSE: 1.0959Computing the msd similarity matrix...Done computing similarity matrix.1it [00:12, 12.72s/it]RMSE: 0.9919RMSE: 0.9299RMSE: 1.1025Computing the msd similarity matrix...Done computing similarity matrix.2it [00:26, 13.54s/it]RMSE: 0.9915RMSE: 0.9270RMSE: 1.0996Computing the msd similarity matrix...Done computing similarity matrix.3it [00:42, 14.14s/it]RMSE: 0.9905
Content-based filtering methods analyze a set of features of items relevant to the user and learn a user profile based on these features. The filtering process basically consists in matching up the features of the user profile against the features of an item content (i.e., item profile). Source: my PhD thesis report
So, here I will rely directly on items attributes i.e. the movie title. First, I have to describe a user profile with an attributes vector. Then I will use these vectors to generate recommendations based on similarity measures.
Most of the content-based filtering methods are based on heuristic models which represent users and items as vectors of TF-IDF (or BM25) in a Vector Space Model.
I used TF-IDF in this code which considers terms that occur frequently in one document (TF =term-frequency), but rarely in the rest of the corpus (IDF = inverse-document-frequency).
Since items and user profiles are represented as vectors of TF-IDF, items arerecommended in decreasing order of similarity with the user (the user profiles are represented in the same form of the items). The most common measure is the cosine similarity which I m gonna use below to calculate the similarity between both profiles.
Suppose a user wants the 10 most ‘similar’ movies from the movie ‘Scientology and the Prison of Belief (2015)’.
Recommending 10 products similar to Going Clear: Scientology and the Prison of Belief (2015)...-------	Recommended: Soulless 2 (2015) (score:0.09111541613391295)	Recommended: Víkend (2015) (score:0.09111541613391295)	Recommended: No Longer Heroine (2015) (score:0.06179864475000222)	Recommended: In the Shadow of Women (2015) (score:0.06179864475000222)	Recommended: Marco Polo: One Hundred Eyes (2015) (score:0.05026525144746273)	Recommended: Ugly American, The (1963) (score:0.0)	Recommended: Snitch Cartel, The (El cartel de los sapos) (2011) (score:0.0)	Recommended: Drone (2014) (score:0.0)	Recommended: Kenny Rogers as The Gambler (1980) (score:0.0)	Recommended: Le Rossignol (2005) (score:0.0)
This is gonna be a similar code to the above but by applying it to a different dataset.
I used the goodbooks-10k dataset available on Kaggle. It contains ratings and books csv files. The first one contains rating data for 10,000 books rated by over 53,000 users. The second file contains the metadata of the 10,000 books (title, author, ISBN, etc.).
I shared the book and movie recommender systems via Github.
That’s it! You can apply this recommender system code template to any data you want. I will also share in the next few days about contextual recommender systems and how to implement one.
So, stay tuned!