A distributed crawling framework.