t1gor/Robots.txt-Parser-Class — pull request #5 by troelskn (2014-02-28 13:55:52+00:00)
Title: Added support for parsing crawl-delay

Sorry for the white space changes. My editor automatically strips end-of-line spaces.