BUbiNG: massive crawling for the masses
UbiCrawler: a scalable fully distributed Web crawler