Package to crawl the web. Used by main program Crawl.