Data Stories for U.S. Election 2020
By: Yingjie, Byron
Tags: Election, Data Analysis, Qlik Sense, Data Visualization, Covid 19

Introduction

The idea of applying data analytics for the United States 2020 Presidential Election was sparked when I picked up Qlik Sense, a powerful data visualization tool. “Nothing beats getting your hands dirty.” Even though I am not a very political person, I figure this is a great chance to challenge myself by applying and upskilling my newly honed knowledge. I started my work on 6th November, the third day since Election Day, while the world was much concerned and uncertain about what was going on with this race.

Disclaimer: The election has come to its finale with the confirmed winner Joe Biden when this blog is edited. This blog thus only intends to share the analytical views based on data acquired on 6th Nov. from a neutral point of view.

Datasets

The first thing to start off is to acquire datasets with adequate dimensions for data analytics from credible sources. I checked on the live-update results from well-known press media such as Bloomberg, New York Times, and Reuters where they usually specify the data sources of their results. However, it turns out that their data sources are acquired from either research companies or individual jurisdictions. The Federal Election Commission and the Electoral College do not provide the voting results but only the campaign finance data and legal resources about the presidential candidates.

Therefore, I resorted to 270toWin, a nonpartisan website that provides up-to-date results data. My preliminary goal is to visualize the results map on the state level, on top of which I continue to build the insights. Unfortunately, the website does not allow web scraping with Selenium. For that reason, I had to manually collect the results data of 50 electoral states and Washington D.C. into an Excel file.

With efforts and research on several federal and state government websites, I managed to gather the following datasets.

Results by state from 270toWin

from 270toWin Campaign finance from the Federal Election Commission

from the Federal Election Commission Ballots breakdown from Michael McDonald, Uni. of Florida

from Michael McDonald, Uni. of Florida COVID-19 cases & deaths by state from CDC

from CDC Demographics by state from Census Bureau

from Census Bureau Exit Polls from New York Times

For the rest of the resources, I was able to batch download or use Python Pandas to read and export into a tabular format.

Data Wrangling & Statistics Aggregation

Before I could dump the datasets into Qlik for analysis, they need to be transformed into a consistent and neat structure for the cross-table association without unnecessary fields to speed up the analysis.

The methods of doing so are mainly Excel functions, Pandas data frames, and preprocessing functions by Qlik Sense.

An example step is to aggregate the results by state data which originally stored in individual sheets into a single table.

Sample Raw Result Data Table (Alaska)

Hence, I have 51 such sheets on hand and have to unpivot them into a single table with rows of states and leave the columns only for Democrat, Republican, Independent/Minor, and Write-in. The process is handled by Pandas with the resource code shown below.

#create an empty df to store the unpivoted data

df_unpivoted = pd.DataFrame(columns = ["state"]+ parties) candidates = {'Democrat': 'Joe Biden',

'Republican': 'Donald Trump',

'Minority': {},

'Write-in': 'Unknown'}

minority_votes = {} #iterate over each state

for state in states:

df_state = pd.read_excel('./results by state.xlsx', state)

state_result = {'state': state}



#iter over each party voted

for index, row in df_state.iterrows():

candidate = row['Unnamed: 0']

party = row['PARTY']

votes = row['VOTES']

#percent = row['PCT.'] if party in candidates.keys(): #if democrat or republican

state_result[party] = votes

else: #if a minor party/independent

candidates['Minority'] = {'party':party,'candiate': candidate}

state_result["Independent"] = votes



if candidate not in minority_votes.keys():

minority_votes[candidate] = votes

else:

minority_votes[candidate] += votes



df_unpivoted=df_unpivoted.append(state_result,ignore_index=True)

Unpivoted Result Table

Analysis 1: The Results & Unsettled States

The first dashboard I built was the high-level overview of the results as of 6th November.

Result Overview Page

On the top, it gives an overview of how many votes for senate and house seats each party has secured. The left bottom corner shows the results map whereby a gradient of blue/red color indicates the percentage of votes won by Biden or Trump. A tooltip shows the electoral votes of the state since different states have decisively different weights in this race.

On 7th Nov., the election still had no projected winner, leaving 5 pivotal states unsettled while counting their mail ballots.

Charts for the Unsettled States

With a closer look at the ongoing results, the closing situation counts on the outstanding mail ballots. From the histogram, it proves that a majority of returned mail ballots go to Biden, who had advocated mail voting for his supporters amid the pandemic.

For the outstanding mail ballots, Biden enjoys an advantage in all states over Trump except that Georgia does not report its result breakdown.

Vote Results Polarity

To forecast the decision among these swing states, a comparison between 2016 and 2020 might also tell us something. In 2016, Trump wins Clinton by a significant amount in these states except Nevada. However, the previous firmly red states lost all their advantage and even flip into blue with a decisive amount of mail ballots left to count, which gives a hint of Biden’s awaiting success.

Analysis 2: Election Amid Pandemic: what has been affected?

COVID-19 Page

2020 is an unprecedented year with uncertainty and pressure. The effects of the pandemic shifted people’s faith in the candidates and the way the election is conducted.

From the map, clearly, the general pattern is Trump wins states with a more severe situation while Biden wins the better-controlled states. This may be explained by their respective diverged COVID policy and proposition. The bar charts on the right further illustrate that states with top cases per 100k almost all voted for Trump while states with the least cases mostly come in blue. I have also made an option of showing the cases per 100k during the week before the election instead. But the result has no clear pattern, which means voters probably do not deviate their decision by near-term factors.

Mail Ballots & COVID-19 Severity Correlation

Based on the assumption that the COVID-19 severity may have a positive correlation with mail ballots demand, a scattered plot interestingly rejects my hypothesis. There is no clear linear correlation shown between the percentage of mail ballots requested over total registered voters and cases per 100k. Nonetheless, we can see that Democrat-winning states do follow Biden’s advocate and requested for more mail ballots.

Analysis 3: Campaign Finance

Campaign Finance Page

The election in the United States is a cash-burning game. To some extent, the campaign finance reflects how “bullish” a candidate is deemed to be by industries and financing investors. Trump raised his fund in an even manner even closer to the election month, whilst Biden received drastically increasing receipts since March 2020, which is exactly the time when the U.S. started to experience a worsening pandemic situation. Among all candidates since 2008, Biden raised the record highest finance to arrive at his today’s success. In terms of spending, the trend is similar to that of fundraising, Biden made a final dash near to election day at a high monetary cost and support. The cash-on-hand summary approves Biden’s solid financial strength ever since he joins the game.

Analysis 4: Electorate Demographics Distribution

Electorate Demographics Page

Last but not least, combined insight into the demographical distribution of different states is worth exploring to understand which candidate wins which group.

With the data extracted from the U.S. Census Bureau, the map covers categories of

Race

Age & Sex ratio

Education level

Child/Old-age dependency ratio

Household income level

Yearly Household Income with 200k+ by State

Some patterns are identified such as the map above shows that on the east coast, states with more rich upper classes tend to support Biden over Trump. This contradicts the fact that Biden’s claim he would legislate higher taxes for the richest top 1%. Therefore, the income level may overlay with the education literacy factor, meaning that these states may also happen to be the most well-educated population.

Education Demographical Distribution

Looking into the education distribution, it is identified that the states with the lowest percentage of college degree attainment are all taken by Trump. This is consistent with the exit polls conducted by New York Times as shown below. Biden is more trusted by people with a college degree while the other group makes a tie between the two.

Exit Polls: Education Attainment Groups

2016 vs 2020: Which States Have Flipped?

With the dots representing results of 2020, we witness that Democrat keeps all their states but Republican loses its previous weakly supporting swing states such as Arizona, Michigan, and Wisconsin.

Although the exit polls tend to be controversial for being not representative of the actual voting results, it is still valuable to see how the mainstream values and certain social groups make their voices by choosing the side of candidates. For example, election 2020 is reported to be a unique election where gender division is at an all-time high.

Conclusion

This has been a great full experience from data gathering to uncovering insights with data-backed storytelling. Looking into a relatively holistic scope of perspectives in such a short time was really challenging yet enjoyable. With the hands-on practices, I have upskilled my knowledge of Qlik Sense’s advanced topics such as set expression, master items, interactive page navigation, and dynamic field association.

Based on the analysis, Biden holds a prominent advantage ahead in the race and we see more division and shift of trust in different states in accordance with different identities of voters. After all, the election is tremendously complex with an uncertain and intractable pool of factors. Hence, the above analysis might be superficial or simply fundamental. To have a more accountable prediction or in-depth analysis requires multi-disciplinary expertise and larger datasets.

Some possible future work for this project is to get more granular datasets such as historical voting results with timestamps to model the result trend and thus a predictive model for the next election. Social media listening can also be applied by using Twitter’s API to analyze sentiment movement during the election period.

Github repo: https://github.com/yingjiebyron/U.S.-Election-2020