A python class to facilitate compliance with the crawl-delay directive in a robots.txt file