Repository: scrapy/scrapy

Issue:
Support pathlib.Path in FEED_URI
Make things work the same when the value assigned to the `FEED_URI` setting is a string containing a path or an instance of `pathlib.Path`.


Discussion:
Hi @Gallaecio, I'm interested in picking this one up. Do you have a bit more of a description of what is required?  
You want to use pathlib.Path instead of os.path thus replacing 
if path and not os.path.isdir(path):
to 
if path and not pathlib.Path.is_dir(path)
and places like these. These changes will mostly be in feedexport.py .
You mean to replace os.path with pathlib.Paths.
Is my understanding of the issue correct?

I’ve updated the description. It may be as simple as type-casting the setting to a string before using it internally.
Ok. I understand now.

@Gallaecio. Is it Ok for me to work on this?
Yes :)

You don’t have to ask for permission, just open a pull request and mention this issue on it.
Please take a look at the [PathLike](https://docs.python.org/3/library/os.html#os.PathLike) interface, which was added in 3.6 to deal with precisely these issues.

Resolving patch:
diff --git a/scrapy/extensions/feedexport.py b/scrapy/extensions/feedexport.py
--- a/scrapy/extensions/feedexport.py
+++ b/scrapy/extensions/feedexport.py
@@ -199,9 +199,9 @@ class FeedExporter(object):
 
     def __init__(self, settings):
         self.settings = settings
-        self.urifmt = settings['FEED_URI']
-        if not self.urifmt:
+        if not settings['FEED_URI']:
             raise NotConfigured
+        self.urifmt = str(settings['FEED_URI'])
         self.format = settings['FEED_FORMAT'].lower()
         self.export_encoding = settings['FEED_EXPORT_ENCODING']
         self.storages = self._load_components('FEED_STORAGES')


Test patch:
diff --git a/tests/test_feedexport.py b/tests/test_feedexport.py
--- a/tests/test_feedexport.py
+++ b/tests/test_feedexport.py
@@ -29,6 +29,8 @@
 from scrapy.utils.python import to_native_str
 from scrapy.utils.project import get_project_settings
 
+from pathlib import Path
+
 
 class FileFeedStorageTest(unittest.TestCase):
 
@@ -405,6 +407,7 @@ def run_and_export(self, spider_cls, settings=None):
         defaults = {
             'FEED_URI': res_uri,
             'FEED_FORMAT': 'csv',
+            'FEED_PATH': res_path
         }
         defaults.update(settings or {})
         try:
@@ -413,7 +416,7 @@ def run_and_export(self, spider_cls, settings=None):
                 spider_cls.start_urls = [s.url('/')]
                 yield runner.crawl(spider_cls)
 
-            with open(res_path, 'rb') as f:
+            with open(str(defaults['FEED_PATH']), 'rb') as f:
                 content = f.read()
 
         finally:
@@ -843,3 +846,17 @@ def test_init_exporters_storages_with_crawler(self):
         yield self.exported_data({}, settings)
         self.assertTrue(FromCrawlerCsvItemExporter.init_with_crawler)
         self.assertTrue(FromCrawlerFileFeedStorage.init_with_crawler)
+
+    @defer.inlineCallbacks
+    def test_pathlib_uri(self):
+        tmpdir = tempfile.mkdtemp()
+        feed_uri = Path(tmpdir) / 'res'
+        settings = {
+            'FEED_FORMAT': 'csv',
+            'FEED_STORE_EMPTY': True,
+            'FEED_URI': feed_uri,
+            'FEED_PATH': feed_uri
+        }
+        data = yield self.exported_no_data(settings)
+        self.assertEqual(data, b'')
+        shutil.rmtree(tmpdir, ignore_errors=True)