internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: scrapy/scrapy

Issue:
lots of warnings `RuntimeWarning: Could not load referrer policy ''` when running tests
Check `tox -e py36 -- tests/test_crawl.py -s` output. @redapple is it a testing issue, or a problem with the warning? I've first noticed it in https://travis-ci.org/scrapy/scrapy/jobs/208185821 output.


Discussion:
I can see it in "normal" crawls as well, it doesn't look like a testing-only issue.
Right. I'll submit a patch shortly. Thanks for reporting

Resolving patch:
diff --git a/scrapy/spidermiddlewares/referer.py b/scrapy/spidermiddlewares/referer.py
--- a/scrapy/spidermiddlewares/referer.py
+++ b/scrapy/spidermiddlewares/referer.py
@@ -261,6 +261,9 @@ class DefaultReferrerPolicy(NoReferrerWhenDowngradePolicy):
     DefaultReferrerPolicy,
 )}
 
+# Reference: https://www.w3.org/TR/referrer-policy/#referrer-policy-empty-string
+_policy_classes[''] = NoReferrerWhenDowngradePolicy
+
 
 def _load_policy_class(policy, warning_only=False):
     """
@@ -317,8 +320,9 @@ def policy(self, resp_or_url, request):
         policy_name = request.meta.get('referrer_policy')
         if policy_name is None:
             if isinstance(resp_or_url, Response):
-                policy_name = to_native_str(
-                    resp_or_url.headers.get('Referrer-Policy', '').decode('latin1'))
+                policy_header = resp_or_url.headers.get('Referrer-Policy')
+                if policy_header is not None:
+                    policy_name = to_native_str(policy_header.decode('latin1'))
         if policy_name is None:
             return self.default_policy()
 


Test patch:
diff --git a/tests/test_spidermiddleware_referer.py b/tests/test_spidermiddleware_referer.py
--- a/tests/test_spidermiddleware_referer.py
+++ b/tests/test_spidermiddleware_referer.py
@@ -526,6 +526,13 @@ class TestPolicyHeaderPredecence003(MixinNoReferrerWhenDowngrade, TestRefererMid
     settings = {'REFERRER_POLICY': 'scrapy.spidermiddlewares.referer.OriginWhenCrossOriginPolicy'}
     resp_headers = {'Referrer-Policy': POLICY_NO_REFERRER_WHEN_DOWNGRADE.title()}
 
+class TestPolicyHeaderPredecence004(MixinNoReferrerWhenDowngrade, TestRefererMiddleware):
+    """
+    The empty string means "no-referrer-when-downgrade"
+    """
+    settings = {'REFERRER_POLICY': 'scrapy.spidermiddlewares.referer.OriginWhenCrossOriginPolicy'}
+    resp_headers = {'Referrer-Policy': ''}
+
 
 class TestReferrerOnRedirect(TestRefererMiddleware):
Repository: scrapy/scrapy

Issue:
MEMUSAGE_REPORT doesn't seem to do anything
Moved from https://github.com/scrapy/scrapy/issues/2187#issuecomment-241032451
MEMUSAGE_REPORT doesn't seem to do anything
Moved from https://github.com/scrapy/scrapy/issues/2187#issuecomment-241032451


Discussion:

Resolving patch:
diff --git a/scrapy/extensions/memusage.py b/scrapy/extensions/memusage.py
--- a/scrapy/extensions/memusage.py
+++ b/scrapy/extensions/memusage.py
@@ -35,7 +35,6 @@ def __init__(self, crawler):
         self.notify_mails = crawler.settings.getlist('MEMUSAGE_NOTIFY_MAIL')
         self.limit = crawler.settings.getint('MEMUSAGE_LIMIT_MB')*1024*1024
         self.warning = crawler.settings.getint('MEMUSAGE_WARNING_MB')*1024*1024
-        self.report = crawler.settings.getbool('MEMUSAGE_REPORT')
         self.check_interval = crawler.settings.getfloat('MEMUSAGE_CHECK_INTERVAL_SECONDS')
         self.mail = MailSender.from_settings(crawler.settings)
         crawler.signals.connect(self.engine_started, signal=signals.engine_started)
diff --git a/scrapy/settings/default_settings.py b/scrapy/settings/default_settings.py
--- a/scrapy/settings/default_settings.py
+++ b/scrapy/settings/default_settings.py
@@ -219,7 +219,6 @@
 MEMUSAGE_ENABLED = True
 MEMUSAGE_LIMIT_MB = 0
 MEMUSAGE_NOTIFY_MAIL = []
-MEMUSAGE_REPORT = False
 MEMUSAGE_WARNING_MB = 0
 
 METAREFRESH_ENABLED = True


Test patch:
Repository: scrapy/scrapy

Issue:
DeprecationWarning: inspect.getargspec()
Hi,

I've tried searching for a similar problem but couldn't find a discussion in scrapy about it, so starting one.

```bash
build  │   func_args, _, _, _ = inspect.getargspec(func)
build  │ /usr/local/lib/python3.6/site-packages/scrapy/utils/python.py:201: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
```

We've stumbled into this log a LOT, actually getting spammed by it currently when we execute some `twisted.trial.unittest` tests where each unittest spins up a `CrawlerRunner` for our live tests that do exactly what unittests do but with live data fetching.

Could we leave the current behavior for py2 but port the `getargspec` to `getfullargspec` as is suggested in the warning? I see latter for py3 includes additional return values, but should be a quick port. Would you be ok with a patch? Thanks


Discussion:
Sure, it would be a good addition!

Resolving patch:
diff --git a/scrapy/utils/python.py b/scrapy/utils/python.py
--- a/scrapy/utils/python.py
+++ b/scrapy/utils/python.py
@@ -10,7 +10,6 @@
 import six
 from functools import partial, wraps
 import sys
-import time
 
 from scrapy.utils.decorators import deprecated
 
@@ -198,10 +197,30 @@ def binary_is_text(data):
     return all(c not in _BINARYCHARS for c in data)
 
 
+def _getargspec_py23(func):
+    """_getargspec_py23(function) -> named tuple ArgSpec(args, varargs, keywords,
+                                                        defaults)
+
+    Identical to inspect.getargspec() in python2, but uses
+    inspect.getfullargspec() for python3 behind the scenes to avoid
+    DeprecationWarning.
+
+    >>> def f(a, b=2, *ar, **kw):
+    ...     pass
+
+    >>> _getargspec_py23(f)
+    ArgSpec(args=['a', 'b'], varargs='ar', keywords='kw', defaults=(2,))
+    """
+    if six.PY2:
+        return inspect.getargspec(func)
+
+    return inspect.ArgSpec(*inspect.getfullargspec(func)[:4])
+
+
 def get_func_args(func, stripself=False):
     """Return the argument name list of a callable"""
     if inspect.isfunction(func):
-        func_args, _, _, _ = inspect.getargspec(func)
+        func_args, _, _, _ = _getargspec_py23(func)
     elif inspect.isclass(func):
         return get_func_args(func.__init__, True)
     elif inspect.ismethod(func):
@@ -248,9 +267,9 @@ def get_spec(func):
     """
 
     if inspect.isfunction(func) or inspect.ismethod(func):
-        spec = inspect.getargspec(func)
+        spec = _getargspec_py23(func)
     elif hasattr(func, '__call__'):
-        spec = inspect.getargspec(func.__call__)
+        spec = _getargspec_py23(func.__call__)
     else:
         raise TypeError('%s is not callable' % type(func))
 


Test patch:
Repository: scrapy/scrapy

Issue:
ur'strings' are a Syntax Error in Python 3
You can have __u__'strings' or you can have __r__'strings' but you can not have __ur__'strings' in Python 3.

flake8 testing of https://github.com/scrapy/scrapy on Python 3.6.2

$ __flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics__
```
./docs/conf.py:194:51: E901 SyntaxError: invalid syntax
  ('index', 'Scrapy.tex', ur'Scrapy Documentation',
                                                  ^

./docs/utils/linkfix.py:23:72: E901 SyntaxError: invalid syntax
line_re = re.compile(ur'(.*)\:\d+\:\s\[(.*)\]\s(?:(.*)\sto\s(.*)|(.*))')
                                                                       ^
```


Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -191,7 +191,7 @@
 # Grouping the document tree into LaTeX files. List of tuples
 # (source start file, target name, title, author, document class [howto/manual]).
 latex_documents = [
-  ('index', 'Scrapy.tex', ur'Scrapy Documentation',
+  ('index', 'Scrapy.tex', u'Scrapy Documentation',
    ur'Scrapy developers', 'manual'),
 ]
 
diff --git a/docs/utils/linkfix.py b/docs/utils/linkfix.py
--- a/docs/utils/linkfix.py
+++ b/docs/utils/linkfix.py
@@ -20,7 +20,7 @@
 _contents = None
 
 # A regex that matches standard linkcheck output lines
-line_re = re.compile(ur'(.*)\:\d+\:\s\[(.*)\]\s(?:(.*)\sto\s(.*)|(.*))')
+line_re = re.compile(u'(.*)\:\d+\:\s\[(.*)\]\s(?:(.*)\sto\s(.*)|(.*))')
 
 # Read lines from the linkcheck output file
 try:


Test patch:
Repository: scrapy/scrapy

Issue:
LinkExtractor is not ignoring .m4v extension (video) by default
By chance I found out that LinkExtractor is not ignoring the video extension m4v in the same way it is ignoring other video formats.

https://en.wikipedia.org/wiki/M4V



Discussion:

Resolving patch:
diff --git a/scrapy/linkextractors/__init__.py b/scrapy/linkextractors/__init__.py
--- a/scrapy/linkextractors/__init__.py
+++ b/scrapy/linkextractors/__init__.py
@@ -28,7 +28,7 @@
 
     # video
     '3gp', 'asf', 'asx', 'avi', 'mov', 'mp4', 'mpg', 'qt', 'rm', 'swf', 'wmv',
-    'm4a',
+    'm4a', 'm4v',
 
     # office suites
     'xls', 'xlsx', 'ppt', 'pptx', 'pps', 'doc', 'docx', 'odt', 'ods', 'odg',


Test patch:
Repository: scrapy/scrapy

Issue:
Line-ends (unnecessary blank lines) problem in CSV export on Windows 
CSV export on Windows create unnecessary blank lines after each line.

You can fix the problem just by adding 
newline='' 
as parameter to io.TextIOWrapper in the __init__ method of the CsvItemExporter class in scrapy.exporters

Details are over here:
https://stackoverflow.com/questions/39477662/scrapy-csv-file-has-uniform-empty-rows/43394566#43394566


Discussion:
Ah, i will try that.  I came across this problem several months ago with Windows10 and solved it by adding code in the pipelines.py file to re-open the cvs file with the blank lines and eliminate those lines once the spider had run.

Details are here: https://github.com/andrewnuk1/Python3-Scrapy-Projects/blob/master/Pipelines.py starting at line 42 “#given I am using Windows i need to elimate the blank lines in the csv file”

Yep,  would like to have this incorporated in next version.  Until then, if anyone wants to try this fix, just dont forget the compile package source code after modifying.     **python -m compileall .**
 command in in the package source folder will compile all files.  I just modified the **exporters.py** scrapy source in "site-packages\scrapy" folder of python installation. 
I think it's a nice and easy fix to have implemented in scrapy. Let's see @kmike thoughts on it.
It looks reasonable to me; I'm fine with merging such pull requests, given that tests pass.

Resolving patch:
diff --git a/scrapy/exporters.py b/scrapy/exporters.py
--- a/scrapy/exporters.py
+++ b/scrapy/exporters.py
@@ -214,7 +214,8 @@ def __init__(self, file, include_headers_line=True, join_multivalued=',', **kwar
             file,
             line_buffering=False,
             write_through=True,
-            encoding=self.encoding
+            encoding=self.encoding,
+            newline='' # Windows needs this https://github.com/scrapy/scrapy/issues/3034
         ) if six.PY3 else file
         self.csv_writer = csv.writer(self.stream, **kwargs)
         self._headers_not_written = True


Test patch:
Repository: scrapy/scrapy

Issue:
DEPTH_STATS has no effect
It looks like this flag stopped functioning in #99. `crawler.stats` are always passed to the middleware, and are always collected.

Additionally, it looks like this middleware would error if stats weren't provided, as the initial depth value would otherwise never be set. 

https://github.com/scrapy/scrapy/blob/c4f096d3a55def4c986174d50ff8b84c8f00503d/scrapy/spidermiddlewares/depth.py#L53-L56


Discussion:

Resolving patch:
diff --git a/scrapy/settings/default_settings.py b/scrapy/settings/default_settings.py
--- a/scrapy/settings/default_settings.py
+++ b/scrapy/settings/default_settings.py
@@ -55,7 +55,7 @@
 }
 
 DEPTH_LIMIT = 0
-DEPTH_STATS = True
+DEPTH_STATS_VERBOSE = False
 DEPTH_PRIORITY = 0
 
 DNSCACHE_ENABLED = True
diff --git a/scrapy/spidermiddlewares/depth.py b/scrapy/spidermiddlewares/depth.py
--- a/scrapy/spidermiddlewares/depth.py
+++ b/scrapy/spidermiddlewares/depth.py
@@ -13,7 +13,7 @@
 
 class DepthMiddleware(object):
 
-    def __init__(self, maxdepth, stats=None, verbose_stats=False, prio=1):
+    def __init__(self, maxdepth, stats, verbose_stats=False, prio=1):
         self.maxdepth = maxdepth
         self.stats = stats
         self.verbose_stats = verbose_stats
@@ -41,7 +41,7 @@ def _filter(request):
                         extra={'spider': spider}
                     )
                     return False
-                elif self.stats:
+                else:
                     if self.verbose_stats:
                         self.stats.inc_value('request_depth_count/%s' % depth,
                                              spider=spider)
@@ -50,7 +50,7 @@ def _filter(request):
             return True
 
         # base case (depth=0)
-        if self.stats and 'depth' not in response.meta:
+        if 'depth' not in response.meta:
             response.meta['depth'] = 0
             if self.verbose_stats:
                 self.stats.inc_value('request_depth_count/0', spider=spider)


Test patch:
Repository: scrapy/scrapy

Issue:
Cannot use contracts with inherited callbacks
### Description

If you want to `scrapy check` a spider that has inherited methods, these methods' contracts will be ignored.

### Reproduce

```python
class BaseSpider(Spider):

    def returns_request(self, response):
        """ method which returns request
        @url https://docs.scrapy.org/en/latest/
        @returns requests 1
        """
        return Request('http://scrapy.org', callback=self.returns_item)


class DemoSpider(BaseSpider):
    name = 'demo_spider'
```

And then run `scrapy check`.

You'll get the following output:

```
----------------------------------------------------------------------
Ran 0 contracts in 0.000s

OK
```

### Reason

`ContractsManager.tested_methods_from_spidercls` uses `vars(spidercls).items()` to get methods.

### Solution

Use `inspect.getmembers(spidercls)` instead.


Discussion:

Resolving patch:
diff --git a/scrapy/contracts/__init__.py b/scrapy/contracts/__init__.py
--- a/scrapy/contracts/__init__.py
+++ b/scrapy/contracts/__init__.py
@@ -1,6 +1,7 @@
 import sys
 import re
 from functools import wraps
+from inspect import getmembers
 from unittest import TestCase
 
 from scrapy.http import Request
@@ -17,7 +18,7 @@ def __init__(self, contracts):
 
     def tested_methods_from_spidercls(self, spidercls):
         methods = []
-        for key, value in vars(spidercls).items():
+        for key, value in getmembers(spidercls):
             if (callable(value) and value.__doc__ and
                     re.search(r'^\s*@', value.__doc__, re.MULTILINE)):
                 methods.append(key)


Test patch:
diff --git a/tests/test_contracts.py b/tests/test_contracts.py
--- a/tests/test_contracts.py
+++ b/tests/test_contracts.py
@@ -101,6 +101,10 @@ def parse_no_url(self, response):
         pass
 
 
+class InheritsTestSpider(TestSpider):
+    name = 'inherits_demo_spider'
+
+
 class ContractsManagerTest(unittest.TestCase):
     contracts = [UrlContract, ReturnsContract, ScrapesContract]
 
@@ -202,3 +206,9 @@ def test_errback(self):
 
         self.assertFalse(self.results.failures)
         self.assertTrue(self.results.errors)
+
+    def test_inherited_contracts(self):
+        spider = InheritsTestSpider()
+
+        requests = self.conman.from_spider(spider, self.results)
+        self.assertTrue(requests)
Repository: scrapy/scrapy

Issue:
about the signal retry_complete
I didn't find the singnal in the singnal list,how can I use it


Discussion:
[Searching for the signal](https://github.com/scrapy/scrapy/search?utf8=%E2%9C%93&q=retry_complete&type=), I only found a mention of it in the docstrings for middleware, but it appears nowhere in the code. On a local copy of the Scrapy git project, I ran:

`$ git rev-list --all | xargs git grep retry_complete | grep -v 'so other extensions'`

..and got nothing. The signal seems to be mentioned only in the docstrings, and it predates the migration to Git. It may be a case of outdated docs?

Resolving patch:
diff --git a/scrapy/downloadermiddlewares/retry.py b/scrapy/downloadermiddlewares/retry.py
--- a/scrapy/downloadermiddlewares/retry.py
+++ b/scrapy/downloadermiddlewares/retry.py
@@ -7,9 +7,7 @@
 RETRY_HTTP_CODES - which HTTP response codes to retry
 
 Failed pages are collected on the scraping process and rescheduled at the end,
-once the spider has finished crawling all regular (non failed) pages. Once
-there is no more failed pages to retry this middleware sends a signal
-(retry_complete), so other extensions could connect to that signal.
+once the spider has finished crawling all regular (non failed) pages.
 """
 import logging
 


Test patch:
Repository: scrapy/scrapy

Issue:
Copying items not work correctly with list values
> > item = Item()
> > item['some_field'] = []
> > 
> > c_item = Item(item)
> > c_item['some_field'].append(1)
> > 
> > print item['some_field'], c_item['some_field']
> > [1] [1]



Discussion:
Not a bug.
See https://docs.python.org/2/library/copy.html

What you do is not meant to deepcopy.
This is the constructor to which you pass an `Item` instance
and since it receives a mapping, it makes a shallow copy.
It's the same thing you'd expect from `dict2 = dict(dict1)`
See the documentation link above.

Yeah, I agree with @Digenis: we document Items as providing dictionary-like API.
But the example at http://doc.scrapy.org/en/latest/topics/items.html#other-common-tasks can be improved. So I think it is a documentation issue.

I wonder if we want to provide a deepcopy shortcut.

I like an idea of a deepcopy shortcut.


Resolving patch:
diff --git a/scrapy/item.py b/scrapy/item.py
--- a/scrapy/item.py
+++ b/scrapy/item.py
@@ -6,6 +6,7 @@
 
 from pprint import pformat
 from collections import MutableMapping
+from copy import deepcopy
 
 from abc import ABCMeta
 import six
@@ -96,6 +97,13 @@ def __repr__(self):
     def copy(self):
         return self.__class__(self)
 
+    def deepcopy(self):
+        """Return a `deep copy`_ of this item.
+
+        .. _deep copy: https://docs.python.org/library/copy.html#copy.deepcopy
+        """
+        return deepcopy(self)
+
 
 @six.add_metaclass(ItemMeta)
 class Item(DictItem):


Test patch:
diff --git a/tests/test_item.py b/tests/test_item.py
--- a/tests/test_item.py
+++ b/tests/test_item.py
@@ -249,6 +249,14 @@ class TestItem(Item):
         copied_item['name'] = copied_item['name'].upper()
         self.assertNotEqual(item['name'], copied_item['name'])
 
+    def test_deepcopy(self):
+        class TestItem(Item):
+            tags = Field()
+        item = TestItem({'tags': ['tag1']})
+        copied_item = item.deepcopy()
+        item['tags'].append('tag2')
+        assert item['tags'] != copied_item['tags']
+
 
 class ItemMetaTest(unittest.TestCase):
Repository: scrapy/scrapy

Issue:
Name parameters uniformly (Cache Policy API)
https://github.com/scrapy/scrapy/blob/92801850546121f2faf95fd21c923904fde35c28/scrapy/extensions/httpcache.py#L34

It will look better if we name the second parameter as `cachedresponse` (in place of just `response`).

In fact, implementation of policy for RFC2616 compliant HTTP cache names it correctly.
https://github.com/scrapy/scrapy/blob/92801850546121f2faf95fd21c923904fde35c28/scrapy/extensions/httpcache.py#L101

I am sending a pull request for the same.


Discussion:

Resolving patch:
diff --git a/scrapy/extensions/httpcache.py b/scrapy/extensions/httpcache.py
--- a/scrapy/extensions/httpcache.py
+++ b/scrapy/extensions/httpcache.py
@@ -31,7 +31,7 @@ def should_cache_request(self, request):
     def should_cache_response(self, response, request):
         return response.status not in self.ignore_http_codes
 
-    def is_cached_response_fresh(self, response, request):
+    def is_cached_response_fresh(self, cachedresponse, request):
         return True
 
     def is_cached_response_valid(self, cachedresponse, response, request):


Test patch:
Repository: scrapy/scrapy

Issue:
Incorrect link mentioned in comments (scrapy.extensions.httpcache)
https://github.com/scrapy/scrapy/blob/92801850546121f2faf95fd21c923904fde35c28/scrapy/extensions/httpcache.py#L73

Link mentioned : https://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec14.9.1
It should have been https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.1

I am sending a pull request for the same.


Discussion:

Resolving patch:
diff --git a/scrapy/extensions/httpcache.py b/scrapy/extensions/httpcache.py
--- a/scrapy/extensions/httpcache.py
+++ b/scrapy/extensions/httpcache.py
@@ -70,7 +70,7 @@ def should_cache_request(self, request):
         return True
 
     def should_cache_response(self, response, request):
-        # What is cacheable - https://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec14.9.1
+        # What is cacheable - https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.1
         # Response cacheability - https://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.4
         # Status code 206 is not included because cache can not deal with partial contents
         cc = self._parse_cachecontrol(response)


Test patch:
Repository: scrapy/scrapy

Issue:
Issue with Twisted and Python 3.4 
Twisted had a patch 3 days ago and it's causing test suite to fail for py34 environment. 
Twisted , according to their Readme, support Python 3.5+. This needs to be fixed if the builds need to pass


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -65,7 +65,8 @@ def has_environment_marker_platform_impl_support():
     ],
     python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*',
     install_requires=[
-        'Twisted>=13.1.0',
+        'Twisted>=13.1.0;python_version!="3.4"',
+        'Twisted>=13.1.0,<=19.2.0;python_version=="3.4"',
         'w3lib>=1.17.0',
         'queuelib',
         'lxml',


Test patch:
Repository: scrapy/scrapy

Issue:
File descriptor leak in FeedExporter
Test suite under windows fails to remove a file at https://github.com/scrapy/scrapy/pull/3315/files#diff-0f004a6e06393cc5e29012b83956eed2L245 because FeedExporter instances keeps an open file descriptor created for the test case.

We need to find the leak and close the file descriptor properly on close_spider signal. 


Discussion:
~@dangra Seeing as [tests are passing on Windows](https://ci.appveyor.com/project/redapple/scrapy/history), should we close this?~

Oh, you probably want to be able to remove `ignore_errors=True` from there.

Resolving patch:
diff --git a/scrapy/extensions/feedexport.py b/scrapy/extensions/feedexport.py
--- a/scrapy/extensions/feedexport.py
+++ b/scrapy/extensions/feedexport.py
@@ -242,7 +242,9 @@ def open_spider(self, spider):
     def close_spider(self, spider):
         slot = self.slot
         if not slot.itemcount and not self.store_empty:
-            return
+            # We need to call slot.storage.store nonetheless to get the file
+            # properly closed.
+            return defer.maybeDeferred(slot.storage.store, slot.file)
         if self._exporting:
             slot.exporter.finish_exporting()
             self._exporting = False


Test patch:
diff --git a/tests/test_feedexport.py b/tests/test_feedexport.py
--- a/tests/test_feedexport.py
+++ b/tests/test_feedexport.py
@@ -417,7 +417,7 @@ def run_and_export(self, spider_cls, settings=None):
                 content = f.read()
 
         finally:
-            shutil.rmtree(tmpdir, ignore_errors=True)
+            shutil.rmtree(tmpdir)
 
         defer.returnValue(content)