Parallel version of crawler.