Repository: scrapy/scrapy
Pull request #2865 (2017-07-30 01:37:27)

[MRG+1] Explicit message for scrapy parse callback
The scrapy parse method raises a NotImplementedError when not defined,
but for new comers it can be hard to debug what might be going wrong.

Adding an explicit message for NotImplementedError will help new users.

Refer #2831

codecov-io: # [Codecov](https://codecov.io/gh/scrapy/scrapy/pull/2865?src=pr&el=h1) Report
> Merging [#2865](https://codecov.io/gh/scrapy/scrapy/pull/2865?src=pr&el=desc) into [master](https://codecov.io/gh/scrapy/scrapy/commit/9d34b2fe166f34400b15b21e95e37cb5a62aff6d?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `0%`.


```diff
@@           Coverage Diff           @@
##           master    #2865   +/-   ##
=======================================
  Coverage   84.69%   84.69%           
=======================================
  Files         164      164           
  Lines        9189     9189           
  Branches     1369     1369           
=======================================
  Hits         7783     7783           
  Misses       1154     1154           
  Partials      252      252
```

| [Impacted Files](https://codecov.io/gh/scrapy/scrapy/pull/2865?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [scrapy/spiders/\_\_init\_\_.py](https://codecov.io/gh/scrapy/scrapy/pull/2865?src=pr&el=tree#diff-c2NyYXB5L3NwaWRlcnMvX19pbml0X18ucHk=) | `97.14% <0%> (ø)` | :arrow_up: |

dangra: Good idea @kirankoduru. Any chance you can add a test case?
kirankoduru: Thanks @dangra. I will add a test.
dangra: @kirankoduru commit history doesn't look good, can you rebase your changes on top of _master_ branch?
kiran-sp: Ah @dangra, thanks for catching that. I will rebase or open a new PR.
dangra: @kiran-sp just rebase please. no need for another PR (I hope you are the same guy than @kirankoduru ) :)

Patch:
diff --git a/scrapy/spiders/__init__.py b/scrapy/spiders/__init__.py
index 30cb7590a..e9c131e3b 100644
--- a/scrapy/spiders/__init__.py
+++ b/scrapy/spiders/__init__.py
@@ -87,7 +87,7 @@ class Spider(object_ref):
         return Request(url, dont_filter=True)
 
     def parse(self, response):
-        raise NotImplementedError
+        raise NotImplementedError('{}.parse callback is not defined'.format(self.__class__.__name__))
 
     @classmethod
     def update_settings(cls, settings):


Test patch:
diff --git a/tests/test_spider.py b/tests/test_spider.py
index e55f0fa6d..6a52b3ea7 100644
--- a/tests/test_spider.py
+++ b/tests/test_spider.py
@@ -429,3 +429,17 @@ class DeprecationTest(unittest.TestCase):
             self.assertEqual(len(requests), 1)
             self.assertEqual(requests[0].url, 'http://example.com/foo')
             self.assertEqual(len(w), 1)
+
+
+class NoParseMethodSpiderTest(unittest.TestCase):
+
+    spider_class = Spider
+
+    def test_undefined_parse_method(self):
+        spider = self.spider_class('example.com')
+        text = b'Random text'
+        resp = TextResponse(url="http://www.example.com/random_url", body=text)
+
+        exc_msg = 'Spider.parse callback is not defined'
+        with self.assertRaisesRegexp(NotImplementedError, exc_msg):
+            spider.parse(resp)