How to Compare Sentiments for Movies with Amazon Comprehend in R? — a Use Case for AWS Machine Learning Services
By: Xinqi Wang
Tags: Web Scraping, The Social Dilemma, Machine Learning, R, AWS

Now let’s get started! My favourite movie of the year is “The Social Dilemma (2020)”, and I am going to use this movie as an example. It is a documentary that explores the rise of social media and the damage it brings to today’s society. I found this movie to be rather alarming to today’s young generation and I am very curious about how the others think of this movie!

The Official Netflix Trailer of “The Social Dilemma”

First, we would like to scrape comments from this movie’s official trailer in R. To access YouTube’s metadata like comments, views, likes, and dislikes are simple by using the “tuber” package. It is an easy-to-use package for accessing the YouTube data API. Below is the R code:

R Script for Scrapping Comments from YouTube Video

Next, we would like to find a video review on “The Social Dilemma (2020)” and download it from MP3.

The most Popular Review of “The Social Dilemma”

Meanwhile, go to Amazon S3, and create a S3 bucket, and then store the converted MP3 into the S3 bucket.

R Script for Creating S3 Bucket and Storing File

After that, we need to go to Amazon Transcribe, fetch the file from S3 bucket, and run the speech-to-text function. You should get an output similar to this after if successfully run:

AWS Transcribe Output

And then go to Amazon Transcribe, convert our MP3 speech to text, and load it to R studio (remember to limit your character numbers within 5000 for later sentiment analysis)

R Script for Loading Converted Text

Next, I found one English movie review site here on “The Social Dilemma (2020)”:

And below is a review article from a French movie review site — Moustique:

We would like to scrape both of the online article using Google Chrome’s add-ins: SelectorGadget. And then use R package “rvest” to paste the HTML nodes into R studio. Don’t forget to take a look at your inputs before proceeds, as there might be some cleanings you need to do given the websites you have picked. Below is the R code block showing the whole process in R. In the very end, I also choose to translate the French review into English for later comparison.

R Script for Scrapping and Cleaning

Finally, let us use Amazon Comprehend in R to check the tone of the reviews, and complete the sentiment analysis in R. Let’s see how the reviews are different from site to site:

R Script for Language Detection

R Results for Language Detection

R Script for Sentiment Analysis

R Results for Sentiment Analysis

L et us take a look at our results here in R output. We could find that more than half (65%) of people who commented under the movie’s official trailer have positive sentiments toward this movie. Around 5% of people have negative sentiments, 12% of people have neutral sentiments, and the rest are in the mixed sentiments category. From the movie review video, 62% are in the negative sentiments category, only 5% are in the positive, and 7% are in the neutral. From the New York Times review article, 68% of them are in the neutral category, only 5% in positive, and 7% in negative. Similarly, the article from French site, Moustique, has 94% neutral sentiments, and 6% negative.

Sentiment categories from Different Sources

Overall Sentiment Toward the Movie

Overall, I would say the results are a bit surprising. Clearly, people have mixed feeling towards this movie. One reason could be that this is a documentary on the dangerous human impact of social networking. People may have more critical thoughts on contents like this.

I also put two “ggplot” graphs to have a more visual views of the results. We can see that overall, we have a neutral sentiment towards this movie from the four sources where people have comments and reviews, despite the fact that from the trailer, more people have a positive sentiment. Also bear in mind that in this analysis, I only use the first 3000 characters from the sources due to limitations from the sentiment analysis, and that might cause bias to some extend in our analysis.