GitHub repository: m4ta1l/RobotsDisallowed
Description: A curated list of the most common and most interesting robots.txt disallowed directories.