High-performance web crawling
Efficient Crawling Through URL Ordering