Web Crawlers: Taxonomy, Issues & Challenges
Focused crawling: a new approach to topic-specific web resource discovery