Decouple parse_urls method in CrawlWorker. Fixes #1