Twitter as a Political Barometer
By: Chris Brownlie
Tags: Politics, Analytics, Social Media, Data Science, Data Visualization

Twitter as a Political Barometer

Analysing the content of 1.5 million tweets by current UK politicians

This is the third standalone post in a series analysing the UK government on Twitter. The first looked at the basic overview of how many MPs were on Twitter and who has the biggest following. The second article investigated this a bit more and investigated the following of each MP - in particular, how many people only follow a single MP or party (hint: a lot).

This article will get to grips with the actual content shared by UK MPs on Twitter. I have scraped every tweet that the current 580 MPs on Twitter have ever tweeted: a dataset of over 1.5 million tweets. In this article I will be exploring and analysing this rich dataset.

As in the previous two posts, all analysis and visualisations were produced in R using the rtweet and ggplot2 packages. This article also uses tidytext for analysing the content of tweets.

Twitter on the rise

To begin with, I will address something that should have been discussed in the very first article: why Twitter?

I chose to look at Twitter because it - and social media in general - is becoming increasingly integral to everyday life and this is also true in the world of politics. The graphs below prove this point.

First, the cumulative number of MPs on Twitter since the first one joined back in 2007:

Secondly, the density of tweets over time:

As you can see the frequency of tweets has been quickly increasing in the last few years.

If we combine these two graphs we get a more meaningful graph - the number of daily tweets by MPs, divided by the number of MPs on Twitter (zoomed in to show from 2015). This graph of adjusted tweets per day shows that in recent years there has been a consistent increase in the levels of activity MPs have on Twitter:

From these graphs you can see that Twitter has become increasingly important in the world of politics and this is a trend that doesn’t look like stopping any time soon.

The Political Barometer

One way to analyse the tweets of UK MPs is to look at the number of times they mention certain political topics and how this changes over time. As there are numerous possible areas of analysis, I have narrowed it down to mentions of the following 13 words, all of which are or have been hot topics in the media:

4 current UK MPs - [Boris] ‘johnson’, [Jeremy] ‘hunt’, [Jeremy] ‘corbyn’ & ‘theresa’ [May]. (I use ‘theresa’ due to the issue of ‘may’ being a commonly used word in English).

4 other notable politicians - [Donald] ‘trump’, [Nigel] ‘farage’, [Barack] ‘obama’ and [David] ‘cameron’.

4 words to represent events or topics - ‘grenfell’, ‘wedding’, ‘nhs’, ‘education’.

Brexit - I explore mentions of ‘brexit’ in a separate graph as it is mentioned so much more often than other topics that it requires different scales to properly visualise.

When I initially examined the number of tweets that mentioned each topic, I ran into an issue whereby the overall number of tweets being sent by MPs each day is increasing (see graph above) and this widespread increase hid any interesting patterns for individual words. To get around this I took the total number of tweets by MPs each day and looked at the proportion of them that mentioned each word. This means that if there were 10 tweets on a particular day and 2 mentioned the word ‘brexit’, it would be equivalent to if there were 1000 tweets and 200 mentioned ‘brexit’. This standardisation helps to deal with the vast disparity in tweeting volumes between the present day and several years ago (when only a handful of MPs had Twitter accounts).

Below you can see how the proportion of mentions of each word varies from 1st January 2015 until 1st July 2019.

Graph showing the proportion of MPs’ tweets mentioning each of the above words over time.

There are several features to point out here, starting from the top left:

Johnson & Hunt: the mentions of these two have risen rapidly in the first few months of 2019, as they are the two final candidates in the Conservative leadership contest.

Corbyn: barely mentioned until September 2015, when he becomes leader of the Labour party. Consistently mentioned since then with the biggest spikes coming after his victory in the Labour leadership contest and around the time of the 2017 general election, which was relatively successful for Labour.

May: similiar pattern to Corbyn, rarely mentioned until 2016 when she took over leadership of the Conservative party and became Prime Minister.

Cameron: see the mirroring with Theresa May in the plot above - he fades into (relative) obscurity after stepping down as PM.

Obama: the spike you see is the day that Obama made a statement saying that if the UK voted for Brexit, a UK-US trade deal would not be a priority. This prompted backlash and discussion from many MPs.

Farage: the two main spikes here (in 2016 and 2019) correspond to Brexit - as will be seen later on in the article.

Trump: big spikes in mentions when he took office in January 2017 and in July 2018 when he made his first visit to the UK, sparking significant protests.

Grenfell: not mentioned until the tragic event occurred, with spikes each year on the anniversary of the fire.

Wedding: we can see the royal wedding of Harry and Megan in May 2018.

NHS: biggest spike came on 5th July 2018 when the NHS turned 70. Also in November 2016 and January 2017 when Labour held rallies across the country focussed on the NHS.

Education: consistently a relevant topic in the political world but doesn’t seem to have many big events or developments that attract the attention of more than 7–8% the ‘digital government’.

These graphs are useful for seeing outliers and extremes but it can be hard to see patterns, so below you can see the same graphs but with a smoothed average rather than each individual day plotted- making it easier to see the patterns over time.

Graph showing the smoothed average proportion of all MPs’ tweets each day, containing each of the above words.

This also allows us to take into account the amount of time a topic stays relevant. For example, see the fact that the new Grenfell graph barely registers the anniversary of the event, as MPs tend to only tweet on the exact day of the anniversary and rarely in the days following or preceding.

Contrastingly we can see that for the NHS, the beginning of 2015 was a significant period - something that wasn’t apparent in the original graph. This is because it was subject of a sizeable proportion of MPs’ tweets over a sustained period of time. In this case it was the announcement by Ed Miliband & the Labour party that they were going to focus on the NHS as a priority in the 2015 general election, and the subsequent response from other parties.

Brexit is the Word

Below you can see this same analysis for tweets containing the word ‘brexit’. As I mentioned before it is so frequently mentioned that if it were included in the same graph as the topics above it would dwarf them and make it impossible to distinguish between them.

Graph showing the number of mentions of ‘brexit’ by all MPs since 2015.

The graph above shows the actual proportion of tweets each day that mentioned the word ‘brexit’ (black line) as well as a smoothed average (blue line). A few points can be made here:

Here can be seen the never-ending political topic that is Brexit. Barely mentioned at all before 2016, it then shot up around the time of the referendum. Having remained relatively constant it then rapidly rose again at the beginning of 2019 as it returned to the political spotlight with a vengeance - coinciding with the formation of Nigel Farage’s ‘Brexit Party’.

The other very interesting point is that the daily proportion of tweets (black line) hasn’t touched zero since the beginning of 2018- since Boxing Day 2017 to be precise. This means that - as staggering as this fact is - since Boxing Day 2017 there hasn’t been a single day where an MP hasn’t tweeted about Brexit. Wow. Thats 568 consecutive days of tweets about Brexit, as of 16th July 2019.

The streak was nearly broken on 14th April 2018 when there was only one single tweet about Brexit - Vince Cable tweeted at approximately 6pm:

“The #LocalElections2018 are important test of opinion on local council but also on #schoolcuts #NHS #crime #exitfrombrexit . Opportunity to support @LibDems . Must register by Tuesday April 17 @ElectoralCommUK . Can do online https://t.co/cIDlBMrfQf" @vincecable; 2018–04–14 18:04:20

If we exclude Christmas Day of 2017 (the last time we went a full day without a Brexit tweet) then this unbroken run extends to 700 days. It has also been 1259 days since we went more than 2 days without a tweet about Brexit - nearly 3 and a half years ago.

Since this unbroken streak that started Boxing Day 2017, the most Brexit tweets by MPs in a single day came on 15th January 2019 when 532 of the 1880 tweets (28.3%) were about Brexit. They have also averaged 64 Brexit tweets every single day in that time.

Name Dropping

Finally, we can look at which accounts are mentioned the most in MPs tweets. In particular I’ll examine which official party accounts (e.g. ‘@UKLabour’, ‘@Conservatives’ etc.) are mentioned the most by MPs of each party.

This can be seen below. Along the x and y axes are the 9 parties with elected MPs on Twitter (Independents are excluded as they have no official party account). The colour of the tile represents for each party’s MPs (along the y-axis), the proportion of all mentions of official party Twitter accounts, that each of the parties along the x-axis accounts for…in other words: the lighter the colour of the tile, the more times MPs from the party on the left side of the graph, have mentioned the official Twitter account of the party on the bottom of the graph.

Graph showing which parties are mentioned the most by MPs of each elected party

The results are unsurprising - MPs mention their own parties Twitter account significantly more than any other. What is potentially interesting to note, however, is where there are no mentions between parties (blank tiles), which may suggest that they are politically isolated from each other.

The graph below shows the same, but with self-mentions excluded (i.e. the diagonal of the graph above removed). In this new graph we can see which other parties MPs are most likely to mention.

Graph to show which other parties are mentioned most by MPs of each elected party.

This is more interesting as we can see the interactions and relationships between political parties.

First off, it is interesting that for almost all parties, Labour is the opponent that is most often mentioned. This may have something to do with the fact that Labour is the party most actively engaged with social media (see my previous post ‘The Twitter Elections’).

You can also clearly see some of the political alliances and rivalries here in twittersphere form:

Alliances: The Green Party (i.e. Caroline Lucas) mentions Labour frequently - Lucas is a friend of Jeremy Corbyn and they have aligned left-wing values.

Alliances: The Conservative Party accounts for 43% of mentions by DUP MPs, however the DUP accounts for 2.4% of mentions by Conservative MPs. I think it is clear who the dominant party is in that relationship.

Rivalries also tend to be geographically contested, with the ‘opposition’ MPs in each region mentioning the ‘in power’ party the most:

Rivalry (England): The Labour Party accounts for 42% of mentions by Conservative MPs and the Conservative Party accounts for 45% of mentions by Labour MPs.

Rivalry (Northern Ireland): Sinn Fein MPs reference the DUP in 56% of their mentions and their official twitter handle accounts for 13.5% of mentions by DUP MPs.

Rivalry (Scotland): Of all mentions of other political parties by Conservative MPs, the SNP account for 39%. The Conservative party appear in 15% of mentions by SNP MPs.

Rivalry (Wales): 43% of mentions by Plaid Cymru MPs are the official Labour party Twitter account. Plaid Cymru only account for 4.3% of mentions by Labour MPs, likely due to their significantly smaller size.

Conclusion

Thank you for reading I hope you found this enjoyable and interesting! If you did and you haven’t checked out the first two articles in this trilogy then click the links below!

Part 1 - The Political Twittersphere of the UK

Part 2 - The Twitter Elections

I plan to do a follow up on this trilogy, renewing a (not particularly slick) piece I did a year ago about Donald Trump on Twitter — I plan to update this, follow a similiar structure to the original and compare how he has changed!

If that sounds interesting to you, or if you found this post interesting and enjoyed finding out about something new using a Data Science perspective then follow me here or on Twitter(!) for more analytical posts, covering a wide variety of topics not just politics!