Added robots, cheerio, sleep, request to package.json.
Server for sitemap api added.
Added findByStatus function. Used in crawler to find currently crawling sitemaps.
Finally crawler added. SiteMapper task done.