GitHub repository: scrapy/dirbot
Primary language: Python
Description: Scrapy project to scrape public web directories (educational) [DEPRECATED]