GitHub repository: xiaosimao/scrapy-redis
Primary language: Python
Description: Redis-based components for scrapy that allows distributed crawling