Often bought together: Pitfalls and Opportunities
By: Dennis Van Der Voorn
Tags: Recommender Systems, Recommendation System, Matrix Factorization, Machine Learning

Business Case

Trying to come up with a reason as to why one would want to create a model that gives a customer possible complementary products seems clear: to help the customer get the most out of their journey with our company right? And some more revenue related incentives might play a certain role as well.

The specific place to start also became clear soon: To avoid long development processes for a proof of concept, we started with an email after someone purchased a product. In the email, we would thank the customer for their purchase and show products that might be very complementary to the already bought product.

Challenges

Method

As we started out, this process did not seems like such a big deal. After all, we had made recommenders already, and used various methods that we knew. We had used models close to Collaborative filtering ( e.g. https://medium.com/kunalrdeshmukh/collaborative-filtering-in-recommendation-systems-2fa49be8f518 ) and had been in contact with Matrix Factorization ( e.g. https://medium.com/@paritosh_30025/recommendation-using-matrix-factorization-5223a8ee1f4 ). Since Matrix Factorization is generally seen as one of the best methods currently in the market, we decided to go with this.

If you’re not too familiar with this model, here is a short explanation (if you want more details please check out one of the many articles written about it):

if you want to recommend movies (as Netflix did when they hosted the challenge that ultimately produced this type of model) you can basically create a matrix of users on the Y axis, and movies on the X axis, and for every user — movie combination create a score through either rating or behavioural patterns (e.g. watch behavior). If (in an ideal world) you’d be able to note down a score for every combination, you could use matrix algebra to create latent user and movie matrices that ‘explain’ the variance for every user and movie.

In an everyday world you obviously cannot note down scores for every combination since most users don’t interact with every movie. This is where Matrix Factorization comes in. by initiating the latent matrices with random numbers, you can use a machine learning oriented approach to come as close as possible to the correct latent matrices, in order to eventually calculate scores for movies that a user did not interact with. This takes about the following general route:

go through the user-movie combination that are available

multiply the user row with the specific movie row of the latent matrices to calculate what the model now says is the score of the movie

Compare this predicted score with the actual score and calculate a loss

use some optimization method to alter the latent matrices to slowly get closer to the correct scores

matrix factorization as explained by Netflix papers

But then came the question: what data do we actually link together for this use case?

Y-axis — Users, X-axis—bought products: Made sense for the model to perform, but not necessarily for the use case we were trying to create. We wanted to focus on products that were actually bought together within the same order.

Made sense for the model to perform, but not necessarily for the use case we were trying to create. We wanted to focus on products that were actually bought together within the same order. Y-axis — Orders, X-axis — bought products: This made more sense for the use case, but proved an interesting challenge: how do we give a different score to different products? Basically everything in the same order should get a score of 1, and there is really no negative score to work from.

This made more sense for the use case, but proved an interesting challenge: how do we give a different score to different products? Basically everything in the same order should get a score of 1, and there is really no negative score to work from. Y-axis — bought products, X-axis — bought products: This may seem strange, but it is what we actually went for ultimately. A co occurrence matrix of products that were bought together in the same order, where the score is just taken as the number of times a co occurrence takes place.

With this co occurrence matrix, we actually could fill all the scores within the matrix, and therefore we adopted a slightly different method to start our recommendation: Singular Value Decomposition (SVD). This is actually the name for the matrix algebra calculation that we hinted towards earlier. The approach is the following:

fill the entire co-occurrence matrix

use SVD to create the latent matrices of the X and Y axes

Take the latent matrix of X to calculate distances (euclidian in our case) between every products latent vector

recommend the N products that have the smallest distance for every bought product

We’ll show a bit more on how to do this under ‘implementation’.

Data

After we hatched out our implementation plan, we got to work, and created 10 recommended products per ‘bought product’. However, when we started evaluating the recommendations, we came to some interesting (read illogical) recommendations, with one example below:

As it turned out, we overestimated the density of data we had. The dataset included a relatively low percentage of orders with 2 or more products, and some products had none or very little co occurrence with others. As a result, one combination of a dress and a pan in an order could lead to these kinds of recommendations.

Ultimately, we solved this by slightly changing our co occurrence dataset to:

Y-axis — bought product brand category combination, X-axis — bought product brand category combination: In this scenario ‘bought product brand category combination’ means the brand category group that a bought product belongs to (e.g. ‘Ladies Dresses Vanilia’). With this change we could calculate the co occurrence between a larger group of products, which becomes an aggregation of the co occurrence between all the products within those groups. This helped us a lot in actually being able to spot patterns in the data.

The only added challenge in this scenario was the fact that we now got recommendations in the form of brand category combinations, and we had to translate them back to products. We used a combination of popularity, margin and stock metrics to be able to determine the best products to give back from each brand-category group.

Implementation

So how did the implementation look like in terms of code? Let’s assume that we are already in the stage where our data is ready for usage and looks like the following (only example data):

We can then use the following code to calculate what would be the best (with smallest distance) brand category group combinations:

Concluding

From this implementation we derived the following learning: