Recommendation System: A problem statement view
By: Louis Wang
Tags: Deep Learning, Machine Learning, Recommendation System

Three types of a problem statement on recommendation systems.

Introduction

When we start to do a data science project, the first thing that we should always do is define the problem, or translate the business problem into a data science problem. It’s not only about dividing the big project into small parts, but also representing how to think about the problem, which may have varying performance in our final solution.

The recommendation system is because of information overload, and we can call it an information filter system. It greatly influences what we interact with the world: shopping (Amazon, Best Buy), music(Spotify), video(Youtube, Netflix), etc. To build a recommendation system providing recommendations to millions of users with millions of items, the first thing is, define the problem.

In this post, I summary three ways of problem statement in the recommendation system: Rating Prediction, Sequence Prediction, and Ranking Prediction.

Rating Prediction

Given a user and his preference score on item 1 to item n, what is his preference score on item n+1?

The preference score here can be either explicit feedback (ratings, reviews)or implicit feedbacks(click, views, purchases). Before modeling, quantify the feedbacks with some techniques like TFIDF for text, feature-importance for implicit events, or normalization and standardization.

Machine Factorization (MF)is a classical type of models to solve this type of problem. Given a MxN matrix, where M is # unique users and N is # unique items, Rij is the ratings for user ion item j, MF can decompose the MxN matrix into two small matrix: MxK and KxN, where K is # dimension of latent factors. However, one possible disadvantage for this is that the matrix can be pretty sparse because one user can only interact with several items.

Bonus: MF can be implemented as a Factorization Machines(FM) model. FM takes features’ second-order interaction into consideration. You can consider MF as an FM model with only user and item features. However, you can add more features to FM models.

matrix factorization

Sequence Prediction

Given a sequence of items that the user interacted with in his previous shopping history, what will he interact in his next visit?

In my last post, I mentioned how sequence-based recommendation benefits us. Actually, in real work, it performed very well compared with rating prediction-based models. The idea behind this type of problem is pretty straightforward, learning the evolution of user affinity’s change in the timeline from the shopping sequence. The good news is that it’s very similar to NLP tasks, and we can apply NLP techniques like embeddings, attentions into the models. Basically, the embeddings can better represent a user or item information in the model, and the attention can decide what in the previous sequence should play a more important role in the future sequence.

One naive approach is to use time-decay on the product embeddings to differentiate the importance of more recent events and more previous events. However, the RNN(LSTM/GRU) based models could do better if computing power is enough.

More precisely, the sequence can be either the click sequence or the session sequence. I am doing a session-based RS model now and will have a post on that later. The session-based approach can detect user’s in-session interests and cross-session interests, which may more fit the real-world recommendation.

Ranking Prediction

Given a pool of items, how to rank them for each user?

This problem is aiming to identify a ranked list of items for the user after he engaged with the website. Tie-Yan Liu categorized them into three groups by their input representation and loss function: the pointwise, pairwise, and listwise approach. In practice, listwise approaches often outperform pairwise approaches and pointwise approaches.

Conclusion

Rating prediction is a good starting point: easy to implement and good for the explainability.

Sequence prediction is actively researched and pretty effective. However, it requires a good ML system/pipeline and a huge amount of data.

Ranking prediction needs more exploration.