Identifying Fake Review on Shopee Using Logistic Regression
By: Andri Oktavianto
Tags: Data Science, Supervised Learning, Machine Learning, Logistic Regression, E Commerce Business

Identifying Fake Review on Shopee Using Logistic Regression

Why the fake review is bad and how to identify them

Shopee is the leading e-commerce platform in Southeast Asia and Taiwan. Especially in my country, Indonesia, peoples love it because of the free shipping policy. I choose Shopee because I accidentally found fake reviews when I try to search for a cheap Netflix Premium account.

So.. Why the fake review is harmful?

There are two reasons why the fake review is harmful for Shopee:

According to Bright Local (2017), 85% of consumers trust online reviews as much as personal recommendations. That’s why it's extremely important to take the time to weed out fake reviews before they can harm your brand. When a user convinced to buy a product after reading a fake review, but then the product isn’t as good as the review said, it would make the customer disappointed. Hypothetically, this would reduce customer trust on Shopee as e-commerce platform, hindering their motivation from buying another product on Shopee. It would also leads to ‘damage control’ for customer support team.

Manually removing fake reviews would be laborious work. With that in mind, I build a machine learning algorithm to classify fake review using Logistic Regression.

Example of fake reviews. Similar review content by the same user at the same time

Methodology

Programming Language: Python

Software: Jupyter Notebook

Data Source: Scraping https://shopee.co.id using Scrapy

At first, I tried using only one feature: review content, because I’m new on this field and only have known how to do sentiment analysis, so I thought that feature would be enough.

But, I’m wrong. The classifier accuracy didn’t satisfy me.

After reading several paper and GitHub projects related to fake review, now I understand we can use multiple features to increase the classifier accuracy. So I try using multiple features from the data set and ditch out review text as a feature. Those multiple features are:

Number of Review by user per day Review Length Cosine Similarity

Steps

First, we must create training set. For those who didn’t know, a supervised learning algorithm such as Logistic Regression need to be trained with a training set. The training set is a data set with labeled class (value you want to predict using machine learning algorithm).

I created a training set containing 100 reviews, manually labeled by myself. With proportion: 51 fake review; 49 original/authentic review.

Snippet of training set containing 100 reviews

After we load the training set, we need to check whether there is a missing value.

We have 100 missing values in orderid column

So, we will drop orderid column because we don’t need it anyway.

We need to preprocess review text and convert UNIX timestamp to date and time format before training the model. After the preprocess has been done, now we can define our three features:

Number of Reviews by user per day

mnr_df1 = df[[‘userid’, ‘date’]].copy()

mnr_df2 = mnr_df1.groupby(by=[‘date’, ‘userid’]).size().reset_index(name=’mnr’)

mnr_df2[‘mnr’] = mnr_df2[‘mnr’] / mnr_df2[‘mnr’].max() df = df.merge(mnr_df2, on=[‘userid’, ‘date’], how=’inner’)

Review length

df[‘review_length’] = df[‘comment’].apply(lambda x: len(x.split()))

Cosine similarity

cosine = 1 — pairwise_distances(tfidf, metric=’cosine’)

These are snippets code, click here if you want to look at the full code.

Results

After we feed those training set to our model — with 70:30 train-test split ratio, we got this result:

Accuracy for C=0.75: 0.7666666666666667

Accuracy for C=1: 0.7666666666666667

Accuracy for C=2: 0.8333333333333334

Accuracy for C=3: 0.8333333333333334

Accuracy for C=4: 0.8333333333333334

Accuracy for C=5: 0.8333333333333334

Accuracy for C=10: 0.8333333333333334 # With C = 10

Accuracy Score : 0.8333333333333334

Precision Score : 0.7222222222222222

Recall Score : 1.0

F1 Score : 0.8387096774193548

Confusion Matrix with 70:30 train-test split

From 4.630 reviews, the algorithm classify 906 as fake reviews and 3.724 as authentic reviews.

Review length histogram

Sample of fake reviews content that has been classified by the algorithm:

We can see from the above figure there are a lot of fake reviews with the same review content. While review content in authentic reviews are more diverse, as you can see below:

In this project I demonstrate how machine learning algorithm such as Logistic Regression is useful to weed out fake reviews, even with small training set and few features we can produce great accuracy— as you can see from the sample classified reviews.

This projects could be improved by comparing classifier accuracy from various supervised learning algorithm, because there might be better classifier than logistic regression.