Extracted all index html get logic as crawlIndex

The crawl methods are mixing so many steps. Separating these logics
makes it easier to do unit test.

Added a crawlIndex() method in all crawlers to handle only the
downloading of indexing HTMLs in the targeting news sites.
Separated link discovery logics into findLinksIn()

Add a method findLinksIn() in all crawlers. The methods accept
HTML of some index pages then discover all useful HTML links in it.