Improve the url grabber code.

Use the newer urllib2 that supports HTTP 1.1 properly.
Remove some obsolete code.