GitHub repository: scrapy-plugins/scrapy-deltafetch
Primary language: Python
Created: 2016-06-16T18:21:13Z
Description: Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls