edsu/twarc — pull request #6 by pbinkley (2013-11-30 04:30:21+00:00)
Title: added lookup when even expanded_url is a shortened url

Sometimes the expanded_url in a tweet's json is still a shortened url: a t.co url in the tweet might expand to a bit.ly url. This enhancement looks for domains of common url shorteners in the expanded_url (or in the plain url if there is no expanded_url), and if it finds one it does an HTTP HEAD to discover the source url. (I was mostly interested in finding hidden GitHub links.) It could be improved by making the list of shorteners configurable.