xcrawler (pypi): A multi-threaded, open source web crawler
Keywords: crawler|spider|scraper