Sentiment Analysis (VADER Sentiment vs TextBlob) : Part 1
By: Jeffrey Triandi Sabarman
Tags: Python Programming, Analysis, Sentiment Analysis, Text Mining, Data Science

Sentiment analysis is very important to know for businesses this days. The easiest way to conduct sentiment analysis is from text or review. We will be testing the two most common sentiment analysis from text examples.

What is Sentiment Analysis?

Sentiment Analysis : Negative, Neutral, and Positive

Read the following text :

“I love this movie so much!”

Compare the text above with this :

“I hate that movie so much!”

Which one is more positive?

The first one right?? But how do we know? We know that because our brain have a memory about a positive word and negative word. How do we know that the first sentence is a positive statement? I bet that you know it from ‘love’ and ‘so much!’. You know that the second sentence is a negative statement from the word ‘hate’. That is how our brain work and that is sentiment analysis. We analyze the text and determine the sentiment of the text. Sentiment can be positive, negative, or neutral. But how does machine read this text and tag that sentence a positive statement or a negative statement?

We will do it with Python programming. The package that we are using is VADER Sentiment and TextBlob. We will compare those packages and show you how to make sentiment analysis from text using those two packages. Here we go!

STEP 1 : Install the package

Before we start, make sure you have Python install on your device and have the IDE. IDE is Integrated Development Environment. This the place to type your code and run it. I recommend PyCharm for Python IDE and install Kite for the autofill code. Here’s the link to download PyCharm and Kite.

PyCharm : https://www.jetbrains.com/pycharm/

Kite : https://kite.com/download/

Install PyCharm Edu for a free access.

After you have Python and the IDE, you have to install the package that we are using for this project. To install package, open your command prompt and type this:

pip install vadersentiment -- user

and type this :

pip install textblob --user

If your device doesn’t need “ — user”, you can skip it. Make sure your Python PATH is in the command prompt. The easiest way to make sure that, you can type python and enter on your command prompt. If it turns out error, you have to search it on the internet. After you download those packages, open your Python IDE. I’m using PyCharm Edu. We will also use pandas to make data frame. To install pandas, just type this and enter on your command prompt.

Other way to install the package, you can use PyCharm or other IDE to install those packages.

STEP 2 : Import the Packages

To import the packages that we have install, type this on your IDE.

import pandas as pd

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

from textblob import TextBlob

From the vaderSentiment package, we have to import SentimentIntensityAnalyzer for the sentiment analysis. From the textblob package, we have to import TextBlob.

STEP 3 : VADER Sentiment Analysis

We will analyse the two sentence above using VADER sentiment. We can start with typing these on your IDE.

analyser = SentimentIntensityAnalyzer()

sentence1 = "I love this movie so much!"

sentence2 = "I hate this move so much!"

So, what we do is analyser is the sentiment analysis that we use when we import vader package. Sentence1 and sentence2 is the sentence we use to compare the sentiment analysis.

score1 = analyser.polarity_scores(sentence1)

score2 = analyser.polarity_scores(sentence2)

The score1 and score2 indicate the score of the sentiment analysis from sentence1 and sentence2. If we run those statement using the print() funciton. This is the result.

The result of score1 and score2

You can see that the VADER sentiment analysis provide us with a negative, neutral, and positive score. Because the result is hard to see. We will make a table form from the result using this.

df1 = pd.DataFrame(list(score1.items()), columns=['Sentiment Metric', 'Score'])

df2 = pd.DataFrame(list(score2.items()), columns=['Sentiment Metric', 'Score'])

df1 is the dataframe for the score1 and df2 is the dataframe for the score2. Here’s the result.

The result of df1 and df2 using print() function

The dataframe give us a better visualization of the overall score. As you can see, the first sentence have a score for a positive sentiment = 0,529 and a negative sentiment = 0. The overall score or compound is 0.6696. The second sentence have a score for a negative sentiment = 0.501 and a positive sentiment = 0. The overall score or compound is -0.6114.

How to interpret the overall score? The rule of thumb is this :

Score >= 0.05 : Positive Sentence

-0.05 < Score < 0.05 : Neutral Sentence

Score <= -0.05 : Negative Sentence

To do that requirement and get the positive or negative sentence from sentence1 and sentence2, we use this :

sentiment1 = score1.get('compound')

sentiment2 = score2.get('compound')

if sentiment1 >= 0.05:

print("The first sentence is a positive sentiment :)")

elif sentiment1 > -0.05 and sentiment1 <0.05:

print("The first sentence is a neutral sentiment :|")

elif sentiment1 <= -0.05 : #or else

print("The first sentence is a negative sentiment :(")



if sentiment2 >= 0.05:

print("The second sentence is a positive sentiment :)")

elif sentiment2 > -0.05 and sentiment2 <0.05:

print("The second sentence is a neutral sentiment :|")

elif sentiment2 <= -0.05 : #or else

print("The second sentence is a negative sentiment :(")

We get the score compound from the score1 and score2 using get() function and put that in a new variable. The new variable is sentiment1 and sentiment2. The result of the code above is this :

The result of the sentence

So, the result is the first sentence is a positive sentiment and the second sentence is a negative sentiment. That’s the same as our answer earlier! So how it works is the VADER Sentiment have a data about the word. Each of the word have a score and it’s classify to positive, neutral, or negative. When there’s a sentence, each of the word in the sentence are classified to the positive, neutral, or negative sentiment and have a score on their own base on the VADER sentiment data. The step is very simple but very powerful to use. The next part, we are going to do another text mining sentiment analysis using different package : TextBlob.

So, there you go! a step by step guide to implement VADER sentiment analysis using Python.

This is the overall code :

After this, go check out the part 2 for the TextBlob part!

You can check other resources about Vader and TextBlob right here by neptune.ai.

Make sure to check out other stuff at neptune.ai medium and website to learn more!

If you have any trouble or a confusion about any the step, hit me up on instagram or check out my github.