Made some changes to the process_url task to start using the crawler.