Recommendation Systems: Collaborative Filtering using Matrix Factorization — Simplified
By: Sachin Prabhu Thandapani
Tags: Machine Learning, Movie Recommendation, Matrix Factorization, Recommendation System, Collaborative Filtering

Recommendation Systems are one of the most successful and fascinating applications of Machine Learning. Various sites like Netflix, YouTube, Yelp use them to recommend movies, videos or services that we might like. Different businesses and product needs have enabled a variety of algorithms that could be used for recommender systems. Being extensively used in video streaming, music streaming or retail applications makes recommendation systems a must-know for Data Science enthusiasts.

Recommender systems typically produce a list of recommendations either through collaborative filtering or through content-based filtering. Modern recommenders combine both approaches. Collaborative filtering methods are based on similarity from user interaction and content-based filtering methods calculate the similarity of attributes of an item. In this article, we will focus on understanding Matrix Factorization — a collaborative filtering solution for recommendations, using simple examples.

In the rest of the article, we will introduce the recommendation task, briefly discuss the collaborative filtering technique for recommender systems and explain Matrix Factorization in detail.

Recommendation Task

We have a set of users U and a set of items I. The goal is to recommend items from I to the users based on their preferences. To solve this problem, we have to learn a function based on the past data that predicts how much each item i∈I will be an appropriate recommendation for user u∈U.

Collaborative Filtering

Collaborative filtering methods are based on collecting and analyzing a large amount of information on user behaviors, activities or preferences and predicting what users will like based on their similarity to other users.

The fundamental assumption behind collaborative filtering technique is that similar user preferences over the items could be exploited to recommend those items to a user who has not seen or used it before. In simpler terms, we assume that users who agreed in the past (purchased the same product or viewed the same movie) will agree in the future.

Working of Collaborative Filtering (source)

Matrix Factorization

Since “Netflix Price Challenge”, Matrix Factorization has been one of the most famous and widely used Collaborative Filtering technique. To explain Matrix Factorization, we will use a simple example of a movie recommendation. Let us assume the following actors to be the users of a recommender system: Ben Affleck, Robert Downey Jr., Ryan Reynolds, Chris Evans, Henry Cavill. Following is the list of movies seen and rated on a 5-point scale by the users: Avengers Infinity War, Batman vs Superman, Deadpool, Aquaman, Green Lantern. For simplification, we will denote them using u1-u5 and i1-i5 respectively.

Everyone seems to love Deadpool except Captain, because “Whoa! Language!!”

Rating Matrix

From the rating matrix, it is easy to observe that u2 and u4 have similar movie preferences - they’re basically bros. Also, u1 and u5 have similar movie preferences - because of their mother’s names. And u3 is unique - because he’s awesome. Matrix Factorization exploits this similarity in user’s preferences and interactions to provide recommendations.

Think about the situation where u4 did not watch i5. We want to know if u4 will like i5. The logic is to find a user with similar preferences to the active user u4, take the rating given for i5 by that user and assume it to be the rating by u4 for that item. Using this rating obtained from the user dependency, we can tell if u4 will like i5 or not.

Aim is to find all these user and item dependencies in the Matrix.

Factorization

Solution to this problem is Matrix Factorization. So, what is factorization? In simpler terms, Factorization is the method of expressing something big as a product of smaller factors.

Similarly, Matrix Factorization finds two rectangular matrices with smaller dimensions to represent a big rating matrix (RM). These factors retain the dependencies and properties of the rating matrix. One matrix can be seen as the user matrix (UM) where rows represent users and columns are k latent factors. The other matrix is the item matrix (IM)where rows are k latent factors and columns represent items. Here k < number of items and k < number of users.

Rating Matrix (RM) = User Matrix (UM) . Item Matrix (IM)

The latent factors are otherwise called as features. Features are characteristics of an item. In our case, features of the movies can be its genre, actors, plot, etc. The features chosen by the Matrix Factorization is irrelevant to the solution. Now the rating matrix is represented as a dot product of user and item matrix.

We’ll now try to understand the features and the dot product of the factor matrices using the figure below. We will ignore other ratings in the rating matrix image for simplification. We’ll assume out factored matrices have only two features F1 and F2. Having Ryan Reynolds in mind, we will assume F1 to be “If its a movie with Marvel characters?” and F2 to be “ If Ryan is there in the movie?”.

User Matrix: According to Ryan, if its a Marvel movie, he’ll give it 3 points and if he is in the movie, he’ll give it 2 more points (Typical Ryan!). Item Matrix: Item Matrix contains binary values where the value is 1 if conditions of features mentioned above are satisfied and 0 otherwise. By performing dot product of the user matrix and item matrix, Infinity War gets 3 and Deadpool gets a 5.

Now the important question here is:

“HOW DO WE FIND THE RIGHT FACTORIZATION FOR A MATRIX?”

Machine Learning is the solution. The model learns to find latent factors to factorize the rating matrix. To arrive at the best approximation of the factors, RMSE(root mean squared error) is the cost function to be minimized. After Matrix factorization is done, squared error is calculated for every movie rating in the rating matrix and the root value of the mean of squared error values are minimized. In order to minimize RMSE to learn the factors, Gradient Descent and Alternating Least Squares are the two most used techniques.

Voila! We have our USER Matrix and ITEM Matrix now.

How to use Matrix Factorization for Recommendation?

I believe everyone will have a doubt about why we are creating this matrix. What is the use of these matrices we have spent so much time understanding?

In real-life scenarios, every user wouldn’t have seen all possible movies. So the rating matrix will look more empty - Sparse Matrix. In this case, the model still learns to factorize the sparse matrix, but RMSE will be calculated only for the ratings that are actually present in the matrix. After obtaining the best factors to approximate the ratings we have, we then perform dot product of the factor matrices to fill in the missing entries in the rating matrix. These filled in ratings are then used to provide recommendations to the users.