Product Recommender using Amazon Review dataset
By: Uma Gajendragadkar
Tags: Machine Learning, Recommender, Pyspark, Apache Spark, Data Science

What is the Problem? Why do we need Recommender Systems?

Online stores have millions of products available in their catalogs. Finding the right product becomes difficult because of this ‘Information overload’. Users get confused and this puts a cognitive overload on the user in choosing a product.

What are recommender systems?

Recommender systems help customers by suggesting probable list of products from which they can easily select the right one. They make customers aware of new and/or similar products available for purchase by providing comparable costs, features, delivery times etc.

Recommender systems have become an integral part of e-commerce sites and other businesses like social networking, movie/music rendering sites. They have a huge impact on the revenue earned by these businesses and also benefit users by reducing the cognitive load of searching and sifting through an overload of data. Recommender systems personalize customer experience by understanding their usage of the system and recommending items they would find useful.

You might have come across examples like below for amazon recommendation system:

Recommender systems can be built with two different methods: Content Based Filtering, and Collaborative Filtering.

Content Based Filtering

In content-based filtering, the similarity between different products is calculated on the basis of the attributes of the products. For instance, in a content-based book recommender system, the similarity between the books is calculated on the basis of genres, the author of the book, the publisher of the book, title of the book etc.

Collaborative filtering

Collaborative filtering is commonly used for recommender systems. These techniques aim to fill in the missing entries of a user-item association matrix. We are going to use collaborative filtering (CF) approach.

CF is based on the idea that the best recommendations come from people who have similar tastes. In other words, it uses historical item ratings of like-minded people to predict how someone would rate an item.Collaborative filtering has two sub-categories that are generally called memory based and model-based approaches.

Memory-based

There are two approaches: the first one identifies clusters of users and utilizes the interactions of one specific user to predict the interactions of other similar users. The second approach identifies clusters of items that have been rated by user A and utilizes them to predict the interaction of user A with a different but similar item B. These methods usually encounter major problems with large sparse matrices, since the number of user-item interactions can be too low for generating high quality clusters.

Model-based

These methods are based on machine learning and data mining techniques. The goal is to train models to be able to make predictions. For example, we could use existing user-item interactions to train a model to predict the top-5 items that a user might like the most. One advantage of these methods is that they are able to recommend a larger number of items to a larger number of users, compared to other methods like memory based approach. They have large coverage, even when working with large sparse matrices. [10]

Matrix factorization

A very popular collaborative filtering method is matrix factorization. It has a set of users and a set of items, and a very sparse matrix that represents known user-to-item ratings. We want to predict missing values in this matrix. In order to do this, we represent each user and each item as a vector of latent features, such that dot products of these vectors closely match known user-to-item ratings. The expectation is that unknown user-to-item ratings can be approximated by dot products of corresponding feature vectors as well. The simplest form of objective function, which we want to minimize, is:

Here, r are known user-to-item ratings, and x and y are the user and item feature vectors that we are trying to find. As there are many free parameters, we need the regularization part to prevent overfitting and numerical problems, with gamma being the regularization factor. After some number of iterations, changes in feature vectors become very small, and convergence is reached.

Alternating least square

Alternating least square (ALS) is one of the methods for Matrix factorization. It is used with nonlinear regression models, when there are two dependent variables (in our case, vectors x and y). The algorithm fixes one of the parameters (user vectors x), while optimally solving for the other (item vectors y) by minimizing the quadratic form. The algorithm alternates between fixing user vectors and updating item vectors, and fixing item vectors and updating user vectors, until the convergence criteria are satisfied.

There are several efficient and scalable implementations of Matrix Factorization in the industry. Among them, a prominent one is that provided by Apache Spark, a distributed data-processing engine that can be run easily on Amazon Web Services with an Elastic Mapreduce cluster.

Apache Spark implements a distributed version of the Alternating Least Square (ALS) method with Weight Regularization.

Data acquisition and selection

The dataset is available on the UCSD website. Thanks to Professor McAuley and team for making this dataset available. [1][2] This dataset contains product reviews and metadata from Amazon, including 142.8 million reviews spanning May 1996 — July 2014. [1] Because of the vast size of the data, it is quite a challenge to handle it all. So the recommender is built using datasets of 5 product categories namely Patio, lawn and garden , Musical instruments, Office products, Automotive, Instant video.

Data cleaning

To bring the data into a consistent format, steps taken are:

Drop unnecessary columns Drop duplicate records Check for invalid data Check ranges for applicable columns (such as ratings between 1 and 5) Deal with missing values and outliers

Exploratory Data Analysis

It’s good practice to know the features and their data types and to take a look at the data distribution. Plotting the data can provide insights into the patterns that the data follows. ‘Patio, Lawn & Garden’ product category dataset is used for plotting graphs 1 to 6.

1)Distribution of overall product ratings

Many users have given a rating of 5 to products followed by 4 and 3 whereas very few users have given a low rating of 1 or 2.

2) Mean and median of ratings over years

Looking at below plot, we can infer that over the years 2000 to 2014, the mean rating of the products has reduced. Median of ratings given to products remains at 5 from 2000 to 2014 except for years 2010 and 2011

3) Trend of number of ratings across years

There is an increasing trend for number of ratings given by the users to products on Amazon which indicates that a greater number of users started using the Amazon e-commerce site for online shopping and a greater number of users started giving feedback on the products purchased from 2000 to 2014. There is a significant increase in number of ratings given by users from 2012 to 2014.

4) Distribution of ratings per user

Distribution of ratings per user shows a long-tail normal distribution. Total number of users in Garden and Patio dataset is 1685. Maximum number of ratings given by single user is 63 and minimum number of ratings by single user is 1. On an average a user gives 7.55 ratings on Amazon as per the data present (till 2014).

5) Distribution of ratings by month

From the box plot below, we can say that highest number of ratings are given in the month of June followed by May and July. It indicates that in these three months, there is a high volume of purchases. As against that, February shows the lowest number of ratings given by users so one can infer that lowest sales might have happened in the month of February. This makes sense because in summer months there is a higher purchasing of gardening products.

6) Year wise count of Amazon products with particular rating

Below plot shows that number of products with ratings ‘1’, ‘2’ or ‘3’ is significantly less over the years as compared to those with ratings ‘4’ and ‘5’.

7) Breakdown of Amazon ratings by product category

I have used 5 different datasets from Amazon review dataset to plot below chart namely ‘Automotive’, ‘Garden and Patio’, ‘Musical Instruments’, ‘Office’ and ‘Instant Video’. In general, ‘Office category’ products and ‘Instant Video’ seem to be more popular in these 5 categories. In all five categories, number of good ratings(4,5) is more than number of bad ratings (1,2).

Implementation

Spark set up

I have installed Spark on a Windows 10 machine with 32 GB RAM Intel i5 processor.

Spark MLLib

ALS implementation provided by Apache Spark MLlib is used for this recommender. MLlib is Apache Spark’s scalable machine learning library. Spark excels at iterative computation, enabling MLlib to run fast.

Apache Spark implements a distributed version of the Alternating Least Square (ALS) method with Weight Regularization. Apache’s spark.mllib currently supports model-based collaborative filtering, in which users and products are described by a small set of latent factors that can be used to predict missing entries. spark.mllib uses the alternating least squares (ALS) algorithm to learn these latent factors. [6] [12]

Model selection and hyper-parameters

Main hyper parameters set for the recommendation engine are rank (number of latent factors), lambda (regularization parameter) and iterations. Rank was set with values [8, 10, 20] for [0.6,0.2,0.2] split and was [ 2,5,10,20] for no split in dataset. Lambda used ranges from [0.001 to 50]. Iterations vary between 5 and 20 for no split in dataset and are set to 20 for [0.6,0.2,0.2] split in dataset.

I have used ALS implementation from spark.mllib where these hyper-parameters can be set and tested.

Model Evaluation

For recommendation engine evaluation, different metrics like RMSE, MAE, Precision and recall are used. I have used RMSE for the model evaluation. Lesser the RMSE, better is the recommendation engine.

Results

1)The dataset was split into training, testing and validation as training — 60%, testing -20% and validation — 20%. In addition to using the split, we vary the rank hyper-parameter in the range 8–20. A few key observations are as follows:

We consistently see low RMSE values for all categories evaluated here.

Rather surprisingly, the RMSE for the office supplies category is 0, indicating perfect predictions (we checked the data for bugs or systematic issues like all null or same values but did not find any issues).

We see very little or no sensitivity of the metric to the rank hyper-parameter. A search in a bigger range for this parameter would be an interesting next step.

2) In the second method, we use a 60–20–20 split for train, validation and test set again. In addition to using the split, we vary the regularization hyper-parameter in the range 0.001–50. A few key observations are as follows:

a) For all product categories except ‘Office product’ category, setting regularization parameter equal to 0.001 gives best RMSE. For ‘Office product’ category regularization parameter of 0.5 and above gives best RMSE.

b) We see very little or no sensitivity of the metric to the regularization hyper-parameter except in case of Office product category.

Conclusion

In this article, we have discussed a product recommender with five different product categories from Amazon review dataset built using Apache PySpark MLLib and have discussed major steps involved in building a recommender as a data science project.

Next Steps

We would like to implement this recommender in cloud. Customizing the recommender for remaining product categories in Amazon Review dataset would be the next step. This recommender makes use of ratings given by users. Making use of review text given by users would be interesting as a further step.

Acknowledgement

I sincerely thank my data science mentor -Dr. Ramakrishna Soma for his guidance and encouragement to complete this project.

References

1) R. He, J. McAuley, Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering, WWW, 2016 pdf

2) J. McAuley, C. Targett, J. Shi, A. van den Hengel, Image-based recommendations on styles and substitutes, SIGIR, 2015, pdf

3) The Good, Bad and Ugly: Apache Spark for Data Science Work

https://thenewstack.io/the-good-bad-and-ugly-apache-spark-for-data-science-work/

4) Pythonic Data Cleaning With NumPy and Pandas

https://realpython.com/python-data-cleaning-numpy-pandas/

5) Data Cleaning with Python and Pandas: Detecting Missing Values

https://towardsdatascience.com/data-cleaning-with-python-and-pandas-detecting-missing-values-3e9c6ebcf78b

6) Apache Spark https://spark.apache.org/

7) Apache PySpark

https://spark.apache.org/docs/0.9.1/python-programming-guide.html

8) J.Bobadilla, F.OrtegaA., HernandoA.Gutiérrez, Recommender systems survey in Knowledge-Based Systems, Volume 46, July 2013, Pages 109–132

9) What is Collaborative Filtering?

https://dzone.com/articles/building-sales-recommendation-engine-with-apache-s

10) Charu C. Aggarwal, Recommender Systems, a comprehensive book

11) Greg Linden, Brent Smith, Jeremy York, Amazon.com Recommendations: Item-to-Item Collaborative Filtering, IEEE Internet Computing, v.7 n.1, p.76–80, January 2003

12) Apache Spark — ALS

https://spark.apache.org/docs/2.2.0/ml-collaborative-filtering.html

13) Simple Matrix Factorization example on the Movielens dataset using Pyspark

https://medium.com/@connectwithghosh/simple-matrix-factorization-example-on-the-movielens-dataset-using-pyspark-9b7e3f567536