Added support for parsing crawl-delay