Use requests instead of urllib2
Remove threaded page downloads

Threaded page downloads was causing issues with the requests
pool being closed. It also makes this area of code harder to debug
and quite often the pool has a size of 1.

Further more the pool only affects the scaning of the index which
should not have as great of a need for parallezation now that there
is only a single request per download by default.