seomoz/g-crawl-py — pull request #2 by dlecocq (2013-01-28 23:29:07+00:00)
Title: Using gevent timeout to ensure that we never block indefinitely

It turns out that the `gevent` timeout only applies to connecting to a remote site, and not the whole transfer. This explicitly forces us to abort a fetch if the provided timeout is exceeded.