GitHub repository: knu/webrobots
Primary language: Ruby
Description: A Ruby library to help write robots.txt compliant web robots