darkrho/scrapy-redis — pull request #17 by parisholley (2014-06-28 18:13:05+00:00)
Title: SET support

In my use case, I queue up URLs based on user activity, not crawling. Pushing into a list causes a tons of duplications, this allows me to change the behavior to use a SET without breaking existing behavior.