Easy Sentiment Analysis with Sentimentr in R | Towards Data Science

The Sentimentr package for R is beneficial in analyzing text for psychological or sociological studies. Its first big advantage is that it makes sentiment analysis simple and achievable within a few lines of code. Its second big advantage is that it corrects for inversions, meaning that while a more basic sentiment analysis would judge “I am not good” as positive due to the adjective good, Sentimentr recognizes the inversion of good and classifies it as negative.
All in all, Sentimentr allows you to quickly do a sophisticated sentiment analysis and directly use it as an input for your regression or any other further analysis.
This article covers how to get started. Please refer to other articles such as Tyler Rinker’s Github Repo’s Readme if you are looking for advanced analyzing techniques. For this tutorial, I will be analyzing Amazon Reviews on Beauty products from the He & McAuley (2016) Dataset. However, you can easily adapt the code to make it fit your own dataset.
By default, Sentimentr uses the Jockers (2017) dictionary, which should be perfect for most circumstances.
install.packages("sentimentr")library(sentimentr)
The first two commands install and load the Sentimentr package. Next, I am loading the data. As it is in JSON format, I need to load the ndjson package. I can then use the package’s stream_in function to load the Amazon Beauty Data.
install.packages("ndjson")library(ndjson)df = stream_in("AmazonBeauty.json")head(df)
I also used the head function to quickly look at the first couple of rows of the data. As you will see when performing this on your own machine, there is a column called reviewText that contains the reviews.
sentiment=sentiment_by(df$reviewText)
This command runs the sentiment analysis. In this case, I used the sentiment_by command to get an aggregate sentiment measure for the entire review. In other cases, you could use the sentiment command (without _by) to get the sentiment per sentence.
While this command runs (it does take a while), I will discuss what the function will return. The sentiment object in this example will be a data.table including the following columns:
element_id — The id number of the review
word_count — The word count of the review
sd — The standard deviation of the sentiment score of the sentences in the review
ave_sentiment — The average sentiment score of the sentences in the review
The most interesting variable is the ave_sentiment, which is the sentiment of the review in one number. The number can take positive or negative values and expresses the valence and the polarity of the sentiment.
We can look at some summary statistics of the calculated sentiment scores.
summary(sentiment$ave_sentiment)
As you can see, most reviews tend to be moderately positive, but there are some extreme outliers, with the most positive review being 3.44 and the most negative being -1.88. These are quite far from the mean and the median, and one should consider removing them for further analysis.
I also did a quick histogram to look at the sentiment of the reviews.
library(ggplot2)qplot(sentiment$ave_sentiment,   geom="histogram",binwidth=0.1,main="Review Sentiment Histogram")
As I am most interested in the sentiment scores, I will conclude this tutorial by integrating the sentiment scores and their standard deviation back into the main dataset.