Recommender System. A Recommender System refers to a system... | by Rishabh Mall | Towards Data Science

A Recommender System refers to a system that is capable of predicting the future preference of a set of items for a user, and recommend the top items. One key reason why we need a recommender system in modern society is that people have too much options to use from due to the prevalence of Internet. In the past, people used to shop in a physical store, in which the items available are limited. For instance, the number of movies that can be placed in a Blockbuster store depends on the size of that store. By contrast, nowadays, the Internet allows people to access abundant resources online. Netflix, for example, has an enormous collection of movies. Although the amount of available information increased, a new problem arose as people had a hard time selecting the items they actually want to see. This is where the recommender system comes in. This article will give you a brief introduction to two typical ways for building a recommender system, Collaborative Filtering and Singular Value Decomposition.
There are two methods to construct a recommender system :
1. Content-based recommendation : The goal of a recommendation system is to predict the scores for unrated items of the users. The basic idea behind content filtering is that each item have some features x. For example, the movie “Love at last” is a romance movie and has a high score for feature x1, but a low score for feature x2.
Each person has a parameter θ which indicates how much they romance movies, and how much they love action movies. If θ is [1, 0.1], it means this person loves romance movies but not action movies. Then for each person, we can find the optimal θ with linear regression.
(Notation)r(i,j): 1 if user j has rated movie i (0 otherwise)y(i,j): rating by user j on movie i (if defined)θ(j): parameter vector for user x(i): feature vector for movie ipredicted rating [user j, movie i]: (θ(j))Tx(i)m(j): # of movies rated by user jnu: # of usersn: # of features of a movie
Given the notation above, the optimization objective for user j can be written as the follow:
Since we want to find the optimal parameters for all the users, we have many of these.
We want to have a single expression for these objectives. Note m(j) is just a constant that doesn’t affect the optimizations. Therefore, ignoring the m(j) factors, we can add up all the objectives and obtain a single expression.
Therefore, the gradient descent can be performed with the follow expressions:
2. Collaborative Filtering : The downside of the content filtering is that it needs the side information for each item. For example, the genres such as romance and action are the side information of movies. It is very expensive to have someone watch movies and add side information for every movie out there and every movie in the future. Moreover how can one possibly list out all the features of a movie? What if one want to add a new feature? Do we want to add the new feature to all the movies? Collaborative filtering solve this problem. To start with, let’s think of the opposite approach to content filtering. Namely, can we predict the features of movies based on user preference parameters?
Again we will user linear regression to predict the features.
Note this time we are minimizing the cost over the feature x instead of parameter θ. We want to predict the features for all the movies.
Similar to what we did earlier, we want to express these objectives with a single expression.
Right now, we have two optimization expressions — One for finding the optimal parameters θ, and the other for finding the optimal features x.
We can find the optimal parameters θ and optimal features x simultaneously. We can combine the two optimization objectives into one. Note both double summations are just looping through all the data where r(i,j)=1. Therefore, we can add the two optimizations together.
We put 1/2 instead of 1 because it’s more mathematically convenient to do so. One very interesting thing to note here is that we don’t need to add the intercept term 1 to x. Why? Because if the algorithm needs the intercept term 1, it can do that by setting the first term of x to be 1. (In reality, we don’t know what x1, x2, ... represent. So we set up arbitary number of features. The first feature could represent the bias.) Therefore, the cost updating functions look as follow:
We discussed before how collaborative filtering doesn’t require any information about the users or items. Well, is there another way we can figure out how the users and the items are related to each other?
It turns out we can if we apply matrix factorization. Often, matrix factorization is applied in the realm of dimensionality reduction, where we are trying to reduce the number of features while still keeping the relevant information. This is the case with principal component analysis (PCA) and the very similar singular value decomposition (SVD).
Essentially, can we take a large matrix of user/item interactions and figure out the latent (or hidden) features that relate them to each other in a much smaller matrix of user features and item features? That’s exactly what ALS is trying to do through matrix factorization.
As the image below demonstrates, let’s assume we have an original ratings matrix RR of size MxNMxN, where MM is the number of users and NN is the number of items. This matrix is quite sparse, since most users only interact with a few items each. We can factorize this matrix into two separate smaller matrices: one with dimensions MxKMxK which will be our latent user feature vectors for each user (U)(U) and a second with dimensions KxNKxN, which will have our latent item feature vectors for each item (V)(V). Multiplying these two feature matrices together approximates the original matrix, but now we have two matrices that are dense including a number of latent features KK for each of our items and users.
In order to solve for UU and VV, we could either utilize SVD (which would require inverting a potentially very large matrix and be computationally expensive) to solve the factorization more precisely or apply ALS to approximate it. In the case of ALS, we only need to solve one feature vector at a time, which means it can be run in parallel! (This large advantage is probably why it is the method of choice for Spark). To do this, we can randomly initialize UU and solve for VV. Then we can go back and solve for UU using our solution for VV. Keep iterating back and forth like this until we get a convergence that approximates RR as best as we can.
Stochastic gradient descent (SGD) optimization is a parallel algorithm that can solve the same problem of Recommender System. The algorithm loops through all ratings in the training data in a random order, and for each known rating r, it makes a prediction r* (based on the dot product of vectors x and y) and computes prediction error e. Then we modify x and y by moving them in the opposite direction of the gradient, yielding certain update formulas for each of the features of x and y.