Parallelize crawling.