Concurrently crawl the target url list.