Automate Sentiment Analysis Process for Reddit Post: TextBlob and VADER | by Manmohan Singh | Towards Data Science

Stuck behind the paywall? Read this article with my friend link here.
Have you ever wondered how to find the sentiment of a subreddit from a user’s comments? Or wanted to check the subreddit polarity on different topics before joining? If you are looking for answers to these questions, then this article will help you a lot. In this article, we will be finding the overall sentiment of the subreddit.
Here is an explanation for those who don’t know what is a subreddit and Reddit.
Reddit is a social sharing website that categorizes into smaller communities called subreddits. A subreddit is simply a board devoted to a specific topic.
Users, building blocks of Reddit, join these communities and submit their thoughts and experience. Other users then show their sentiments via comments on those thoughts.
Getting started with knowing the required libraries for sentiment analysis:
Reddit provides an API to fetch information from their site. We will be using PRAW, a Python library, to interact with Reddit API.
VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media. We will be using VADER for sentiment analysis of Reddit comments and topics.
TextBlob is a Python library for processing textual data. It provides an API for natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, classification, translation, and more.
Note: The final version of the code is available at the end of this article.
import prawfrom textblob import TextBlob# VADER can be accessed by the NLTK library.import nltk# Download the VADAR tool and access it through the NLTK library.nltk.download(‘vader_lexicon’)from nltk.sentiment.vader import SentimentIntensityAnalyzer
Before we start collecting data for sentiment analysis, we need to create a Reddit app. Visit this page to create an app and get the required authentication and details.
reddit_api = praw.Reddit(client_id=’my_client_id’,                         client_secret=’my_client_secret’,                         user_agent=’my_user_agent’)
Reddit_api object establishes a connection with Reddit API. We will be fetching subreddit information such as posts and comments using its functions.
We are using showerthoughts subreddit to get the top 5 weekly posts data. If you want to check the polarity of other subreddit, just replace showerthoughts with your subreddit name.
top_posts = Reddit.subreddit(‘showerthoughts’).top(‘week’, limit=5)
Also, we can replace week with hour, month, and year. The limit will restrict the post to 5. More information on Reddit API configuration are available on this page.
The next step is getting data in each subreddit: post title, comments, and replies. Posts on subreddit are divided into two parts, the title and the comment section. Some titles have a small description which represents a view of the post author. Most voted comment and its replies visible on top of the comment section.
for submission in top_posts:     print("Title of the post :", submission.title)
I have replicated the entire comment section of the subreddit post using recursion.
def replies_of(top_level_comment,               count_comment,               sub_entries_textblob,               sub_entries_nltk):    if len(top_level_comment.replies) == 0:        count_comment = 0        return    else:        for num, comment in enumerate(top_level_comment.replies):            try:                count_comment += 1                   print(‘-’ * count_comment, comment.body)                   text_blob_sentiment(comment.body,                                       sub_entries_textblob)                   nltk_sentiment(comment.body, sub_entries_nltk)            except:               continue            replies_of(comment,                        count_comment,                       sub_entries_textblob,                       sub_entries_nltk)
Subreddit data required is now available for sentiment analysis. We will pass the data in the TextBlob tool and get the sentiment.
def text_blob_sentiment(review, sub_entries_textblob):    analysis = TextBlob(review)    if analysis.sentiment.polarity >= 0.0001:        if analysis.sentiment.polarity > 0:            sub_entries_textblob[‘positive’]=                              sub_entries_textblob[‘positive’]+1            return ‘Positive’    elif analysis.sentiment.polarity <= -0.0001:        if analysis.sentiment.polarity <= 0:            sub_entries_textblob[‘negative’]=                             sub_entries_textblob[‘negative’] + 1            return ‘Negative’    else:        sub_entries_textblob[‘neutral’]=                             sub_entries_textblob[‘neutral’] + 1        return ‘Neutral’
All the functions of TextBlob can be access from the analysis variable. We will use analysis.sentiment to know the polarity score of the sentence. From the polarity score, we will categorize the sentence into positive, negative, and neutral. The sentiment of all sentences has been counted and stored in variable sub_entries_textblob.
Follow the same procedure for the VADER tool, pass the data, and store the sentiment in sub_entries_nltk.
sia = SentimentIntensityAnalyzer()def nltk_sentiment(review, sub_entries_nltk):    vs = sia.polarity_scores(review)    if not vs[‘neg’] > 0.05:        if vs[‘pos’] — vs[‘neg’] > 0:            sub_entries_nltk[‘positive’]=                                 sub_entries_nltk[‘positive’] + 1            return ‘Positive’        else:            sub_entries_nltk[‘neutral’]=                                 sub_entries_nltk[‘neutral’] + 1            return ‘Neutral’    elif not vs[‘pos’] > 0.05:        if vs[‘pos’] — vs[‘neg’] <= 0:            sub_entries_nltk[‘negative’]=                                sub_entries_nltk[‘negative’] + 1            return ‘Negative’        else:            sub_entries_nltk[‘neutral’]=                                 sub_entries_nltk[‘neutral’] + 1            return ‘Neutral’    else:        sub_entries_nltk[‘neutral’]=sub_entries_nltk[‘neutral’] + 1        return ‘Neutral’
We can access the VADER function using sia object. The sia.polarity takes a sentence as input and gives sentiment score as output. From nltk_sentiment() sentences categorizes into positive, negative, and neutral sentiment.
If you print sub_entries_nltk and sub_entries_textblob variable, we will get the total count of positive, negative, and neutral sentiments. From the count, we will conclude the overall polarity of subreddit showerthoughts.
In this article, one can learn how to fetch information from Reddit using the PRAW python library and discover the sentiment of subreddit. Also, learn how TextBlob and VADER tools are easy to implement for sentiment analysis. This article helps you to replicate the entire comment section of the subreddit post and sentiment tag attach with each comment and its replies.
Other Articles by Author
First step in EDA : Descriptive Statistic AnalysisText Data Analysis and Visualization of Reuters ArticlesDiscover the Sentiment of Reddit Subgroup using RoBERTa Model
First step in EDA : Descriptive Statistic Analysis
Text Data Analysis and Visualization of Reuters Articles
Discover the Sentiment of Reddit Subgroup using RoBERTa Model