Neo4j: Graphing the 'My name is...I work' Twitter meme
Date: 2017-02-28 15:50:27

Over the last few days I've been watching the chain of 'My name is\...' tweets kicked off by https://twitter.com/dhh[DHH] with interest. As I understand it, the idea is to show that coding interview riddles/hard tasks on a whiteboard are ridiculous.

[,DHH (@dhh) https://twitter.com/dhh/status/834146806594433025[February 21, 2017]]
____
Hello, my name is David. I would fail to write bubble sort on a whiteboard. I look code up on the internet all the time. I don&#39;t do riddles.
____+++<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8">++++++</script>+++

Other people quoted that tweet and added their own piece and yesterday Eduardo Hernacki suggested that traversing this chain of tweets seemed tailor made for Neo4j.

[,Eduardo Hernacki (@eduardohki) https://twitter.com/eduardohki/status/836402386440761347[February 28, 2017]]
____
https://twitter.com/eduardohki[@eduardohki] is someone traversing all this stuff? https://twitter.com/hashtag/Neo4j?src=hash[#Neo4j]
____+++<script async="" src="//platform.twitter.com/widgets.js" charset="utf-8">++++++</script>+++

https://twitter.com/mesirii[Michael] was quickly on the scene and created a Cypher query which calls the Twitter API and creates a Neo4j graph from the resulting JSON response. The only tricky bit is creating a 'bearer token' but Jason Kotchoff has https://gist.github.com/jkotchoff/03add042c9b1b7db350c[a helpful gist] showing how to generate one from your Twitter consumer key and consumer secret.

Now that we're got our bearer token let's create a parameter to store it. Type the following in the Neo4j browser:

[source,cypher]
----

:param bearer: '<your-bearer-token-goes-here>'
----

Now we're ready to query the Twitter API. We'll start with the https://dev.twitter.com/rest/reference/get/search/tweets[search API] and find all tweets which contain the text '"my name" "I work"'. That will return a JSON response containing lots of tweets. We'll then create a node for each tweet it returns, a node for the user who posted the tweet, a node for the tweet it quotes, and relationships to glue them all together.

We're going to use the +++<cite>+++apoc.load.jsonParams+++</cite>+++ procedure from the https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases[APOC library] to help us import the data. If you want to follow along you can use a https://neo4j.com/sandbox-v2/[Neo4j sandbox instance] which comes with APOC installed. For your local Neo4j installation, grab the APOC jar and put it into your +++<cite>+++plugins+++</cite>+++ folder before restarting Neo4j.

This is the query in full:

[source,cypher]
----

WITH 'https://api.twitter.com/1.1/search/tweets.json?count=100&result_type=recent&lang=en&q=' as url, {bearer} as bearer

CALL apoc.load.jsonParams(url + "%22my%20name%22%20is%22%20%22I%20work%22",{Authorization:"Bearer "+bearer},null) yield value

UNWIND value.statuses as status
WITH status, status.user as u, status.entities as e
WHERE status.quoted_status_id is not null

// create a node for the original tweet
MERGE (t:Tweet {id:status.id})
ON CREATE SET t.text=status.text,t.created_at=status.created_at,t.retweet_count=status.retweet_count, t.favorite_count=status.favorite_count

// create a node for the author + a POSTED relationship from the author to the tweet
MERGE (p:User {name:u.screen_name})
MERGE (p)-[:POSTED]->(t)

// create a MENTIONED relationship from the tweet to any users mentioned in the tweet
FOREACH (m IN e.user_mentions | MERGE (mu:User {name:m.screen_name}) MERGE (t)-[:MENTIONED]->(mu))

// create a node for the quoted tweet and create a QUOTED relationship from the original tweet to the quoted one
MERGE (q:Tweet {id:status.quoted_status_id})
MERGE (t)–[:QUOTED]->(q)

// repeat the above steps for the quoted tweet
WITH t as t0, status.quoted_status as status WHERE status is not null
WITH t0, status, status.user as u, status.entities as e

MERGE (t:Tweet {id:status.id})
ON CREATE SET t.text=status.text,t.created_at=status.created_at,t.retweet_count=status.retweet_count, t.favorite_count=status.favorite_count

MERGE (t0)-[:QUOTED]->(t)

MERGE (p:User {name:u.screen_name})
MERGE (p)-[:POSTED]->(t)

FOREACH (m IN e.user_mentions | MERGE (mu:User {name:m.screen_name}) MERGE (t)-[:MENTIONED]->(mu))

MERGE (q:Tweet {id:status.quoted_status_id})
MERGE (t)–[:QUOTED]->(q);
----

The resulting graph looks like this:

[source,cypher]
----

MATCH p=()-[r:QUOTED]->() RETURN p LIMIT 25
----

image::{{<siteurl>}}/uploads/2017/02/graph-21.png[Graph  21,407]

A more interesting query would be to find the path from DHH to Eduardo which we can find with the following query:

[source,cypher]
----

match path = (dhh:Tweet {id: 834146806594433025})<-[:QUOTED*]-(eduardo:Tweet{id: 836400531983724545})
UNWIND NODES(path) AS tweet
MATCH (tweet)<-[:POSTED]->(user)
RETURN tweet, user
----

This query:

* starts from DHH's tweet
* traverses all +++<cite>+++QUOTED+++</cite>+++ relationships until it finds Eduardo's tweet
* collects all those tweets and then finds the author
* returns the tweet and the author

And this is the output:

image::{{<siteurl>}}/uploads/2017/02/graph-22-2.png[Graph  20,700]

I ran a couple of other queries against the Twitter API to hydrate some nodes that we hadn't set all the properties on - you can https://gist.github.com/mneedham/438649fbbac0e4124ae19f12586d529f[see all the queries on this gist].

For the next couple of days I also have a sandbox running https://10-0-1-157-32898.neo4jsandbox.com/browser/. You can login using the credentials +++<cite>+++readonly/twitter+++</cite>+++.

</p>

If you have any questions/suggestions let me know in the comments, https://twitter.com/markhneedham[@markhneedham] on twitter, or email the Neo4j DevRel team - devrel@neo4j.com.