replaced java.net api with apache httpclient api for crawling, updated the crawling mapreduce jobs