Parallerize the crawler.