Intro to Recommender Systems with TensorFlow and TFRS | by Dimitris Poulopoulos | Towards Data Science

In his 2004 article entitled “The Long Tail,” Chris Anderson said that we are leaving the information age and entering the age of recommendation. Unless we can filter the information overload we absorb every day and retain only what is important to us, everything reduces to noise.
A recommender system aims to predict the user’s behavioral patterns, preferences, or dislikes, and provide personalized recommendations on items users are likely to interact with.
Moreover, we seem to diverge from the notion of search and embrace that of discovery. The difference is that when searching, a user is actively looking for something. Discovery suggests that something the user did not know existed, or did not know how to ask for, finds him.
A recommender system aims to predict the user's behavioral patterns, preferences, or dislikes and provide personalized recommendations on items users are likely to interact with.
So, are you curious about how such systems work? This story examines how to formulate a recommendation problem, the difference between implicit and explicit feedback, and how to build a movie recommender system with TensorFlow and TFRS.
Learning Rate is my weekly newsletter for those who are curious about the world of AI and MLOps. You’ll hear from me every Friday with updates and thoughts on the latest AI news, research, repos and books. Subscribe here!
The recommendation problem can be conceptually structured as a two-dimensional matrix, where rows correspond to users and columns correspond to items. Then, each cell’s value represents a user's preference for an item.
A recommender system algorithm tries to predict the missing values in that matrix, and recommend the items that it predicts with a big score to an active user.
Sometimes we have the user's explicit opinion for an item in the form of a rating score; we know what the user liked and what they could go without. When explicit feedback from the users is available, the system tries to solve a surrogate problem, where ratings are viewed as a proxy to preference. This is usually solved as a regression problem; thus, the algorithm tries to predict a real-valued rating to complete the user-item matrix's missing values.
However, most of the time, we do not have access to explicit user feedback. In such cases, the system works passively in the background, trying to collect meaningful features. Those features can be used to implicitly discover user habits, preferences, and behavior, with some level of confidence. In binary implicit feedback, the user-item matrix is simplified to a Boolean matrix, where true values correspond to positive user-item interaction, and false values indicate no interaction. In any case, implicit feedback has consequences to both algorithmic design and evaluation measures.
For this example, we use the well-known movielens dataset. However, we won’t consider the ratings given by users, but we will treat a user-movie interaction (i.e., a user u has seen movie m) as positive implicit feedback. On the other hand, we do not have any information about the void interactions; if a user has not seen a movie, we cannot be sure if this is because she doesn’t like the movie or hasn’t stumbled upon it yet.
Although we could directly predict a user's recommendations, we will split the process into two parts: candidate extraction and candidate ranking. This is closer to what a real-world recommender, looking through millions of items, would do:
Create a list of candidate items that a user might like, filtering out all the noise
Rank those candidate items to produce a sorted list of personalized recommendations for the user
This is actually what YouTube also does for generating video recommendations. You can read more in their relevant publication.
In this article, we implement the first part of this process, the candidate extraction. In subsequent stories, we will see how to deploy this efficiently and how to rank those candidates.
To generate valid candidates for our users, we build a retrieval model. Retrieval models are often composed of two sub-models:
A query model computing the query representation, which essentially produces an embedding for each user.
A candidate model computing the candidate representation, an embedding vector for each item.
The two models' outputs are then multiplied together to give a query-candidate affinity score. This is the dot product between the user embedding vector and the corresponding item embedding vector; a higher score indicates a better match between the user and the item, or, in other words, the query and the candidate.
In this tutorial, we see how to:
Get our data and split it into a training and test set.
Implement a retrieval model.
Fit and evaluate the model.
Serve the model efficiently by building an ANN index.
First, let’s install the project’s dependencies and import the necessary libraries. We will install tensorflow-recommenders, tensorflow-datasets, and snann an optional dependency of TFRS, which will make our inference service orders of magnitude faster. We will see this last part in the next article, where we will talk about efficient deployment.
!pip install -q tensorflow-recommenders!pip install -q --upgrade tensorflow-datasets!pip install -q scann
Let’s now turn our attention to the data set. In this example, we use the MovieLens dataset from Tensorflow Datasets. Loading movielens/100k_ratings yields a tf.data.Dataset object containing the “ratings” data and loading movielens/100k_movies yields a tf.data.Dataset object containing only the “movies” data.
The MovieLens data set does not have predefined splits, so we will load the train split.
The ratings dataset holds a lot of info, such as the movie’s genre and the user’s age. Although you can use everything to build a more accurate model. Let’s simplify our problem a bit and keep only the user’s ID and the movie’s title. In the same way, let’s keep only the movie’s title from the movie dataset.
Since we don’t have a train and test split predefined, let’s create one ourselves. First, we shuffle the examples and then take 80% for the training set and the remaining 20% for the validation set.
Finally, we need to extract the unique user and movie IDs. This helps create our vocabularies, the list of users, and items we will embed.
Choosing the architecture of our model is key in Deep Learning. In this case, our model consists of two parts:
The query tower: the model that is responsible for learning the user embeddings
The candidate tower: the model that is responsible for learning the movie embeddings
We can build each tower separately and then combine them in the final model. Also, we should add an additional embedding to account for unknown tokens, and let’s agree on an embedding size of 32. Usually, more dimensions for the embedding layer will correspond to models that may be more accurate but will also be slower to fit and more prone to overfitting.
We are now ready to combine the two towers in a single model:
Instead of a forward method, we have a compute_loss method. This method will take the user and movie embeddings, compute the loss, update the weights, and evaluate the metrics, in one package. Under the hood, it’s still a plain Keras model.
You could achieve the same functionality by inheriting from tf.keras.Model and overriding the train_step and test_step functions. But this is where TFRS shines!
Next, let’s see how to define our loss and metric, and what those values mean.
Using implicit feedback as our signal means that we only have info about the positive user-item interactions. Thus, to figure out how good our model is, we compare the score that the model calculates for a specific pair to the scores of all the other possible candidates; if the score for the positive pair is higher than for all other candidates, our model is highly accurate.
To do this, we can use the tfrs.metrics.FactorizedTopK metric. The metric has one required argument: the dataset of candidates that are used as implicit negatives for evaluation.
As for the loss, TFRS makes it easy, as it provides a wrapper called task. We are building a retrieval model, so we can use the Retrieval task object. A task object bundles together the loss function and metric computation. So, let’s specify the metric and the task:
Finally, we are ready to train and evaluate the model. As we said, our recommender is nothing more than a Keras model, so we should compile it and fit it to the data.
During training, and also after evaluation, you can monitor the performance of your model. We have this weird looking metric that says:
factorized_top_k/top_10_categorical_accuracy': 0.02215000055730343
It’s actually quite simple: all it’s saying is that the movie that a user watched is in the top-10 recommended ones 2.2% of the time. This might look a bit low, but it’s actually a good result. To the best of my knowledge, the state of the art is around 5–6%.
Moreover, the model is re-recommending some movies that the users have already watched. These known-positive watches can push test movies out of top K recommendations. This can be tackled by simply excluding previously seen movies from test recommendations.
This story describes what a recommender is, how it works, and what’s the difference between implicit and explicit feedback. Next, we saw how to design modern, real-world recommenders by splitting the problem into a retrieval and a ranking challenge. Finally, we implemented a retrieval model using TensorFlow and TFRS.
In the next articles, we will see how to efficiently deploy such a retrieval model and conclude our example by coding the ranking algorithm.
Learning Rate is my weekly newsletter for those who are curious about the world of AI and MLOps. You’ll hear from me every Friday with updates and thoughts on the latest AI news, research, repos and books. Subscribe here!
My name is Dimitris Poulopoulos, and I’m a machine learning engineer working for Arrikto. I have designed and implemented AI and software solutions for major clients such as the European Commission, Eurostat, IMF, the European Central Bank, OECD, and IKEA.
If you are interested in reading more posts about Machine Learning, Deep Learning, Data Science, and DataOps, follow me on Medium, LinkedIn, or @james2pl on Twitter.
Opinions expressed are solely my own and do not express the views or opinions of my employer.