GitHub repository: jwmorley73/jwm.robotstxt
Primary language: Python
Description: Provides python access to Googles parser for robot.txt files as used by their GoogleBot webscraper.