GitHub repository: zanachka/scrapy-deltafetch
Primary language: Python
Description: Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls