scrapy/scrapy — pull request #533 by chekunkov (2014-01-15 11:15:07+00:00)
Title: Make dupefilter easily subclassable

If I need to subclass dupefilter to implement different duplicate check - I should overwrite entire request_seen method, which looks good to me and possibly could be changed in future, while only change I need - change the way request fingerprint is calculated. So I propose to add request_fingerprint method to dupefilter classes which will allow easily override the way duplicates are checked without touching request_seen method.