Repository: scrapy/scrapy

Issue:
Type error when we try to retrieve the `FEEDS` setting via CLI and it has a `Path` objects as a key
<!--

Thanks for taking an interest in Scrapy!

If you have a question that starts with "How to...", please see the Scrapy Community page: https://scrapy.org/community/.
The GitHub issue tracker's purpose is to deal with bug reports and feature requests for the project itself.

Keep in mind that by filing an issue, you are expected to comply with Scrapy's Code of Conduct, including treating everyone with respect: https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md

The following is a suggested template to structure your issue, you can find more guidelines at https://doc.scrapy.org/en/latest/contributing.html#reporting-bugs

-->

### Description

When a `Path` object is used as a key in the `FEEDS` dictionary and we try to obtain the `FEEDS` setting via CLI `scrapy settings --get FEEDS`, a type error is raised:

```
TypeError: keys must be str, int, float, bool or None, not PosixPath
```

A complete stack trace is attached below under "Actual behavior".

`Path` objects as keys are allowed as documented in [the FEEDS section of the Feed exports chapter](https://docs.scrapy.org/en/master/topics/feed-exports.html#feeds).

### Steps to Reproduce

1. In a freshly generated scrapy project (e.g. via `scrapy project feeds_bug`), prepend the following lines to `settings.py`:
```python
from pathlib import Path

FEEDS = {
    Path("some_path/file.csv"): {
        "format": "csv"
    }
}
```
2. Execute `scrapy settings --get FEEDS`

**Expected behavior:** Some string representation of the `FEEDS` setting gets printed to standard out, maybe like this:
```
{"some_path/file.csv": {"format": "csv"}}
```

**Actual behavior:** A type error is raised:
```
Traceback (most recent call last):
  File "/Users/paul/Projects/feeds-bug/.direnv/python-3.9.7/bin/scrapy", line 8, in <module>
    sys.exit(execute())
  File "/Users/paul/Projects/feeds-bug/.direnv/python-3.9.7/lib/python3.9/site-packages/scrapy/cmdline.py", line 145, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/Users/paul/Projects/feeds-bug/.direnv/python-3.9.7/lib/python3.9/site-packages/scrapy/cmdline.py", line 100, in _run_print_help
    func(*a, **kw)
  File "/Users/paul/Projects/feeds-bug/.direnv/python-3.9.7/lib/python3.9/site-packages/scrapy/cmdline.py", line 153, in _run_command
    cmd.run(args, opts)
  File "/Users/paul/Projects/feeds-bug/.direnv/python-3.9.7/lib/python3.9/site-packages/scrapy/commands/settings.py", line 37, in run
    print(json.dumps(s.copy_to_dict()))
  File "/Users/paul/.pyenv/versions/3.9.7/lib/python3.9/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/Users/paul/.pyenv/versions/3.9.7/lib/python3.9/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Users/paul/.pyenv/versions/3.9.7/lib/python3.9/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: keys must be str, int, float, bool or None, not PosixPath
```

**Reproduces how often:** 100%

### Versions

```
Scrapy       : 2.5.1
lxml         : 4.7.1.0
libxml2      : 2.9.12
cssselect    : 1.1.0
parsel       : 1.6.0
w3lib        : 1.22.0
Twisted      : 21.7.0
Python       : 3.9.7 (default, Dec 11 2021, 11:25:57) - [Clang 13.0.0 (clang-1300.0.29.3)]
pyOpenSSL    : 22.0.0 (OpenSSL 1.1.1m  14 Dec 2021)
cryptography : 36.0.1
Platform     : macOS-11.6-x86_64-i386-64bit
```



Discussion:
(I didn't even know we have a `settings` command)
So it simply takes all settings and if a setting value is a dict serializes it into JSON, which can't always work as not all Python dicts are JSON-serializable (not only because of the key type reqs). As the command just prints the values it should be fine if the representation is not 100% true, but I'm not sure how to handle serialization problems without explicit handling of special cases, maybe providing a `default` function that calls `str()` would be good enough.

Resolving patch:
diff --git a/scrapy/settings/__init__.py b/scrapy/settings/__init__.py
--- a/scrapy/settings/__init__.py
+++ b/scrapy/settings/__init__.py
@@ -375,9 +375,13 @@ def __len__(self):
         return len(self.attributes)
 
     def _to_dict(self):
-        return {k: (v._to_dict() if isinstance(v, BaseSettings) else v)
+        return {self._get_key(k): (v._to_dict() if isinstance(v, BaseSettings) else v)
                 for k, v in self.items()}
 
+    def _get_key(self, key_value):
+        return (key_value if isinstance(key_value, (bool, float, int, str, type(None)))
+                else str(key_value))
+
     def copy_to_dict(self):
         """
         Make a copy of current settings and convert to a dict.


Test patch:
diff --git a/tests/test_cmdline/__init__.py b/tests/test_cmdline/__init__.py
--- a/tests/test_cmdline/__init__.py
+++ b/tests/test_cmdline/__init__.py
@@ -64,3 +64,7 @@ def test_override_dict_settings(self):
         settingsdict = json.loads(settingsstr)
         self.assertCountEqual(settingsdict.keys(), EXTENSIONS.keys())
         self.assertEqual(200, settingsdict[EXT_PATH])
+
+    def test_pathlib_path_as_feeds_key(self):
+        self.assertEqual(self._execute('settings', '--get', 'FEEDS'),
+                         json.dumps({"items.csv": {"format": "csv", "fields": ["price", "name"]}}))
diff --git a/tests/test_cmdline/settings.py b/tests/test_cmdline/settings.py
--- a/tests/test_cmdline/settings.py
+++ b/tests/test_cmdline/settings.py
@@ -1,5 +1,14 @@
+from pathlib import Path
+
 EXTENSIONS = {
     'tests.test_cmdline.extensions.TestExtension': 0,
 }
 
 TEST1 = 'default'
+
+FEEDS = {
+    Path('items.csv'): {
+        'format': 'csv',
+        'fields': ['price', 'name'],
+    },
+}