Beyond the emotions of the Twitteratis regarding the presidential debate.
By: Soumyadeep Basu
Tags: Data Visualization, 2020 Presidential Race, Presidential Debates, Assignment 3, Elections

“Research is formalized curiosity. It is poking and prying with a purpose.” — Zora Neale Hurston

Introduction

The media often predicts the outcomes of an election by the debates and they consider the debates as a national event of great importance — a kind of Super Bowl of American democracy. The political scientists on the other hand believe that debates do not have any major effects on voters. If however an election is neck to neck, then the result of the debates can give a slight push to help the winner towards victory. The main advantage of a debate maybe there for the challenging candidate who the people know less and he or she has a chance to reach out to the public and make his presence known to them. [1]

This article is based on my last post “Visualization of networks related to the presidential debate.”[2], where I got some interesting insights. The most important being: The two main debate contenders viz. the Republican and Democratic nominees fell short to leave a mark on the Twitteratis and as a result the Libertarian nominee came to the limelight.

My previous research contained Tweets from all over the world with the hashtag Debates2020. It was a three hour timeframe after the debate completed from 10:30 pm to roughly 1:30 am the next day. Since I wanted to visualize the network and find out what impact the candidates had on the Twitter users, hence I did not put a location constraint for tweet collection at that time.

Curiosity led me to dive more deep into this topic and investigate what the actual voters who will vote and elect their president on November 3 i.e. the American people are discussing about the presidential candidates on Twitter and what are their positive and negative views for their candidates that will help them to choose a new president.

Process:

The important attribute to filter for my research idea is the users location. Hence, I filtered out the users based on their location. This was easier said than done. Since the location data did not contain formatted values and I had to create a list of all fifty state names, their short form and a few ways to write the country name along with city names from US United States [A to E] [F to J] [K to O] [P to T] [U to Z] and the UM United States Minor Outlying Islands lists found on the United Nations Code for Trade and Transport Locations website and the list of all 50 state abbreviations website. Then I compared the location column with the list and filtered out values which were not in the list. I started with 2500 rows and at the end of the process, I eliminated 500 rows.

In order to verify the results, I used Microsoft Excel, copied the location column and discarded the duplicate rows and 251 unique location values remained. Since that was a decent number to browse manually, I verified that the locations that are included are US locations. This did not have errors since, it compared the values from the list and was able to get the values which are there in the list. So, verifying this was just an extra layer of precaution.

However, the main concern was if I was excluding any US location that was not there in my defined list or any specially formatted string which the program missed. I exported the rows that I have discarded in a csv file and again followed the manual process utilizing Microsoft Excel. Since the dataset contained less data it was easier to verify the results manually.

After filtering the users, I used the same method as my previous post[2] but this time, instead of user, I used hashtags and got a hashtag graph in order to determine what topics the twitter users are discussing about the leaders.

Next, I filtered out the target column, only keeping the hashtags that consisted of Trump, Donald, Joe and Biden. I used Microsoft Excel’s filter for this purpose.

At the end of the process, I had roughly 200 rows of data which contained all the hashtag topics in the Source column and “trump”, “donald”, “joe” and “biden” keyword containing hashtags in the Target column[not case sensitive]. Hence, in order to easily identify the two concerned communities in the visualization, I selected all the “trump” and “donald” keyword related hashtags in the Target column and named them as “Donald Trump” similarly, I selected the “joe” and “biden” keyword containing hashtags in the Target column and renamed them as “JoeBiden”.

Hence, my laborious data processing task was done and next I imported this data to Gephi similar to my previous post[2] in order to visualize and find an answer to my initial question- What are the Twitter users discussing about the presidential candidates after the first presidential debate of 2020 and what are their topics of discussion for each of them before they elect a candidate?

Discussion: