scrapy/scrapy — pull request #732 by nyov (2014-05-24 20:50:56+00:00)
Title: change Scraper API to call internal `_parse` method (fix #712)

As a fix to the CrawlSpider-can't-have-parse useability issue (#712) ,
I propose having another internal `_parse` method called by the `Scraper` instead of `parse`, and using that in spiders who want internal "pre-processing", then exposing `parse` as the public, documented, no-baggage method to implement/override from a spider.

`Spider` simply calls the public `parse` method from `_parse`, while `CrawlSpider` and cohorts can do their thing in `_parse` and give the cold shoulder to users trying to break them by re-defining `parse` without calling `super()`.

Since all spiders presently should subclass from `Spider`, this also doesnt't break any existing spider classes who keep using `parse` exclusively.