Graphing Brexit: Clustering Edition
By: Mark Needham
Tags: Graph Visualization, Brexit, Data Science, Towards Data Science, Neo4j

This is the 2nd in a series of posts showing how to analyse Brexit with graphs. In this post we cluster MPs based on voting records.

Brexit — EU

On Friday I wrote a blog post showing how to do graph analysis of Brexit data using Neo4j, and towards the end of the first post I showed how to compute the similarities of MPs to Boris Johnson, based on the position that they took on different motions.

While doing this analysis I noticed that there were a lot of people who voted in an identical way on every motion, so I wanted to explore that further.

Robin Bramley also pointed me towards the CommonsVotes website, which has the voting records available in CSV format.

This will be helpful for future Brexit analysis, so let’s quickly learn how to import data from there into Neo4j.

Importing CommonsVotes data

I’ve downloaded the data for all the indicative votes and put the files into the data/commonsvotes directory of the graphing-brexit GitHub repository.

We can import the data for all the indicative motions using the Cypher query below:



LOAD CSV FROM " UNWIND [655,656,657,658,659,660,661,662] AS divisionLOAD CSV FROM " https://github.com/mneedham/graphing-brexit/raw/master/data/commonsvotes/Division " + division + ".csv" AS row // Create motion nodes

WITH division, collect(row) AS rows

MERGE (motion:Motion {division: trim(split(rows[0][0], ":")[1]) })

SET motion.name = rows[2][0] // Skip the first 6 rows as they have metadata we don't need

WITH motion, rows

UNWIND rows[7..] AS row // Create person, party, constituency, and corresponding rels

MERGE (person:Person {name: row[0]})

MERGE (party:Party {name: row[1]})

MERGE (constituency:Constituency {name: row[2]})

MERGE (person)-[:MEMBER_OF]->(party)

MERGE (person)-[:REPRESENTS]->(constituency)

WITH person, motion,

CASE WHEN row[3] = "Aye" THEN "FOR"

WHEN row[3] = "No" THEN "AGAINST"

ELSE "DID_NOT_VOTE" END AS vote

CALL apoc.merge.relationship(person, vote, {}, {}, motion)

YIELD rel

RETURN count(*)

Identical Voters

Now that we’ve got the data loaded we’re going to explore those identical voters. I tried a few different approaches, and eventually settled on the following:

Create a similarity graph based on people who voted in an identical way Run the connected components algorithm over that similarity graph to find clusters of people

After we’ve done these two steps we want to have a node representing each community or cluster that we find, and a relationship from each person to their community.

Let’s get started!

Similarity Graph

We’re going to create a similarity graph of MPs, which is a fancy way of saying that we’re going to create relationships between pairs of MPs that voted in an identical way.

Cosine Similarity

As in the first post, we’re going to use the Cosine Similarity algorithm from the Neo4j Graph Algorithms Library to do this. Before we create any relationships in our database, we’re going to run the non streaming version of the procedure with write: false and similarityCutoff: 1.0 (i.e. they behaved identically) to see how many pairs of identical voters we have without writing to the database.

The following Cypher query does this:

MATCH (p:Person), (c:Motion)

OPTIONAL MATCH (p)-[vote]->(c)

WITH p, c,

CASE WHEN type(vote) = "FOR" THEN 1

WHEN type(vote) = "DID_NOT_VOTE" THEN 0.5

ELSE 0 END AS score

WITH {item:id(p), weights: collect(score)} as userData

WITH collect(userData) as data

CALL algo.similarity.cosine(data, {

similarityCutoff: 1.0, write: false

})

YIELD nodes, similarityPairs, writeRelationshipType, writeProperty

RETURN nodes, similarityPairs, writeRelationshipType, writeProperty

How many identical votes do we have?

There are more than 10,000 similarity pairs for our 649 MPs! This should be a fun graph to work with.