scrapy-crawl-once (pypi): Scrapy middleware which allows to crawl only new content
Keywords: scrapy