GitHub repository: tianhuil/domain_scheduler
Primary language: Python
Created: 2018-02-25T01:29:59Z
Description: A domain-aware scheduler for Scrapy that round robin cycles through the domains as it outputs requests to reduce the load on servers and accelerate scraping