Added a bunch of skip_suffixes and one skip_string.
.
Don't import sitemaps; fixed indentation on several lines.
Removed testNeedsCrawledSitemap now that we want to use the sitemap.
Factored out a log_in method from the make_http_request method.  Also removed all the passing of full_url that was happening.