Changed timeout length, should now follow redirects and then crawl.