reppy2 (pypi): Replacement robots.txt Parser in pure Python
Keywords: utilities