Added crawl job to be run by cron.