GitHub repository: Geolem/scrapy-redis
Primary language: Python
Description: Redis-based components for scrapy that allows distributed crawling