Repository: getpelican/pelican

Issue:
Booleans specified with  -e / --extra-settings do not evaluate to False
<!--
  Hi there! Thank you for discovering and submitting an issue.

  Before you submit this, let’s make sure of a few things.
  Please make sure the following boxes are ticked if they are correct.
  If not, please try and fulfill them first.
-->

<!-- Checked checkbox should look like this: [x] -->
- [x] I have read the [Filing Issues](https://docs.getpelican.com/en/latest/contribute.html#filing-issues) and subsequent “How to Get Help” sections of the documentation.
- [x] I have searched the [issues](https://github.com/getpelican/pelican/issues?q=is%3Aissue) (including closed ones) and believe that this is not a duplicate.

<!--
  Once the above boxes are checked, if you are able to fill in the following list
  with your information, it would be very helpful for maintainers.
-->

- **OS version and name**: macOS Mojave
- **Python version**: 3.9.7
- **Pelican version**: 4.7.1
- **Link to theme**: default produced by `pelican-quickstart`
- **Links to plugins**: default produced by `pelican-quickstart`
- **Link to your site**: –
- **Link to your source**: https://github.com/r4victor/pelican/tree/boolean_overrides_demo/website
- **Link to a [Gist](https://gist.github.com/) with the contents of your settings file**: https://github.com/r4victor/pelican/blob/boolean_overrides_demo/website/pelicanconf.py

## Issue

1. Pick any boolean settings parameter that is set to `True` by default or in `pelicanconf.py`.
2. Try to override it to `False` with `-e / --extra-settings`.
3. Observe that parameter is still `True`.

For example, I set `DELETE_OUTPUT_DIRECTORY = True` in `pelicanconf.py` and then run:
```
pelican --debug -e DELETE_OUTPUT_DIRECTORY=false
```
I see how the old files in the `output` directory are deleted. Here's the debug log: https://github.com/r4victor/pelican/blob/boolean_overrides_demo/website/debug.txt

Setting `-e param=False` or `-e param=0` doesn't work as well. The only way is to set the parameter to an empty string like `-e param=`. This is definitely not what users are supposed to do according to the docs (https://docs.getpelican.com/en/latest/settings.html#settings).

It seems the problem was introduced when fixing issue #2789 and originates in the [`coerce_overrides()`](https://github.com/getpelican/pelican/blob/bb10d286a6ac2f283d332f7e80c5655b9997f827/pelican/settings.py#L664) function in `settings.py`:

```python
def coerce_overrides(overrides):
    if overrides is None:
        return {}
    coerced = {}
    types_to_cast = {int, str, bool}
    for k, v in overrides.items():
        if k not in DEFAULT_CONFIG:
            logger.warning('Override for unknown setting %s, ignoring', k)
            continue
        setting_type = type(DEFAULT_CONFIG[k])
        if setting_type not in types_to_cast:
            coerced[k] = json.loads(v)
        else:
            try:
                coerced[k] = setting_type(v)
            except ValueError:
                logger.debug('ValueError for %s override with %s, try to '
                             'load as json', k, v)
                coerced[k] = json.loads(v)
    return coerced
```
So, when `types_to_cast` is `bool`, setting `-e k=v` results in `coerced[k] = bool(v)`, but in Python `bool(v)` is True for any string except for the empty string. `bool("false")` is `True`.

The solution would be to evaluate `"false", "False", "0", ""` (something else?) to `False`, and everything else to `True`.





Discussion:
Hi Victor. Many thanks for reporting this problem and posting such a detailed analysis. Much appreciated! 👏 

Since you seem to have a good handle on a potential solution, would you consider submitting a PR that addresses the problem, preferably including first a commit with a (failing) test and then another that fixes it? That would _really_ help us out!

cc: @jwodder @sabaini 
Hi, Justin. Sure, I'll work on it.
I now realized that `coerce_overrides()` and thus `-e` / `--extra-settings` are completely broken. It's not just booleans.

To parse a value of some parameter, `coerce_overrides()` does `coerced[k] = json.loads(v)`. But to avoid parsing string parameters, it also looks at the type of the parameter's default value and so does `coerced[k] = str(v)` if it's str. (It does the same for `int`, and for `bool` before my PR for no good reason). The problem with this is that in Pelican a settings parameter can take values of different Python types. For instance, `PAGE_TRANSLATION_ID` is `'slug'` by default but can be `None` or `False`. Many other parameters are nullable strings. `DEFAULT_PAGINATION` can be `False` or int. All those parameters are parsed incorrectly.

With default `PAGINATED_TEMPLATES` run `pelican -e DEFAULT_PAGINATION=123`. You'll get one blog entry per page because Pelican sees that `DEFAULT_PAGINATION` is `False` in defaults and parses `123` as `True`. Then `Paginator(... per_page=True)` works as `Paginator(... per_page=1)`, but that's another story. The same for strings and so on. My PR doesn't affect this behaviour.

Another observation is that though we allowed non-json booleans like `False` and `True` for boolean types, it's still not possible to do `pelican -e JINJA_ENVIRONMENT='{"trim_blocks": True}'`. You'll get `JSONDecodeError`.

I see two possible solutions:

1. Associate each parameter with a type (maybe a compound type). Then parse each type accordingly.
2. Give up on custom values like `True` and `False` and rely completely on `json.loads()`. Show users a useful error message on `JSONDecodeError`. 

I'm 100% for the second solution since it's much simpler and predictable. I wouldn't be afraid of breaking code that relies on `True` and `False` since it was so broken anyway.

I should note that relying on `json.loads()` will force users to quote string parameters like so: `pelican -e k='"some string"'`. We'll parse `'null'` as `None` and `'"some string"'` as `'some string'`, but parsing `'some string'` will throw an error. 

If you want to accept `'some string'` literally, you need to handle `'null'`, and this again means you need to introduce some kind of type system for parameters.

I'd go with `json.loads()` + informative error messages + overview in the docs.

Resolving patch:
diff --git a/pelican/__init__.py b/pelican/__init__.py
--- a/pelican/__init__.py
+++ b/pelican/__init__.py
@@ -1,4 +1,5 @@
 import argparse
+import json
 import logging
 import multiprocessing
 import os
@@ -24,7 +25,7 @@
 from pelican.plugins._utils import get_plugin_name, load_plugins
 from pelican.readers import Readers
 from pelican.server import ComplexHTTPRequestHandler, RootedHTTPServer
-from pelican.settings import coerce_overrides, read_settings
+from pelican.settings import read_settings
 from pelican.utils import (FileSystemWatcher, clean_output_dir, maybe_pluralize)
 from pelican.writers import Writer
 
@@ -259,16 +260,29 @@ def __call__(self, parser, namespace, values, option_string):
         parser.exit()
 
 
-class ParseDict(argparse.Action):
+class ParseOverrides(argparse.Action):
     def __call__(self, parser, namespace, values, option_string=None):
-        d = {}
-        if values:
-            for item in values:
-                split_items = item.split("=", 1)
-                key = split_items[0].strip()
-                value = split_items[1].strip()
-                d[key] = value
-        setattr(namespace, self.dest, d)
+        overrides = {}
+        for item in values:
+            try:
+                k, v = item.split("=", 1)
+            except ValueError:
+                raise ValueError(
+                    'Extra settings must be specified as KEY=VALUE pairs '
+                    f'but you specified {item}'
+                )
+            try:
+                overrides[k] = json.loads(v)
+            except json.decoder.JSONDecodeError:
+                raise ValueError(
+                    f'Invalid JSON value: {v}. '
+                    'Values specified via -e / --extra-settings flags '
+                    'must be in JSON notation. '
+                    'Use -e KEY=\'"string"\' to specify a string value; '
+                    '-e KEY=null to specify None; '
+                    '-e KEY=false (or true) to specify False (or True).'
+                )
+        setattr(namespace, self.dest, overrides)
 
 
 def parse_arguments(argv=None):
@@ -366,13 +380,13 @@ def parse_arguments(argv=None):
 
     parser.add_argument('-e', '--extra-settings', dest='overrides',
                         help='Specify one or more SETTING=VALUE pairs to '
-                             'override settings. If VALUE contains spaces, '
-                             'add quotes: SETTING="VALUE". Values other than '
-                             'integers and strings can be specified via JSON '
-                             'notation. (e.g., SETTING=none)',
+                             'override settings. VALUE must be in JSON notation: '
+                             'specify string values as SETTING=\'"some string"\'; '
+                             'booleans as SETTING=true or SETTING=false; '
+                             'None as SETTING=null.',
                         nargs='*',
-                        action=ParseDict
-                        )
+                        action=ParseOverrides,
+                        default={})
 
     args = parser.parse_args(argv)
 
@@ -385,6 +399,8 @@ def parse_arguments(argv=None):
 
 
 def get_config(args):
+    """Builds a config dictionary based on supplied `args`.
+    """
     config = {}
     if args.path:
         config['PATH'] = os.path.abspath(os.path.expanduser(args.path))
@@ -409,7 +425,7 @@ def get_config(args):
     if args.bind is not None:
         config['BIND'] = args.bind
     config['DEBUG'] = args.verbosity == logging.DEBUG
-    config.update(coerce_overrides(args.overrides))
+    config.update(args.overrides)
 
     return config
 
diff --git a/pelican/settings.py b/pelican/settings.py
--- a/pelican/settings.py
+++ b/pelican/settings.py
@@ -1,7 +1,6 @@
 import copy
 import importlib.util
 import inspect
-import json
 import locale
 import logging
 import os
@@ -659,25 +658,3 @@ def configure_settings(settings):
             continue            # setting not specified, nothing to do
 
     return settings
-
-
-def coerce_overrides(overrides):
-    if overrides is None:
-        return {}
-    coerced = {}
-    types_to_cast = {int, str, bool}
-    for k, v in overrides.items():
-        if k not in DEFAULT_CONFIG:
-            logger.warning('Override for unknown setting %s, ignoring', k)
-            continue
-        setting_type = type(DEFAULT_CONFIG[k])
-        if setting_type not in types_to_cast:
-            coerced[k] = json.loads(v)
-        else:
-            try:
-                coerced[k] = setting_type(v)
-            except ValueError:
-                logger.debug('ValueError for %s override with %s, try to '
-                             'load as json', k, v)
-                coerced[k] = json.loads(v)
-    return coerced


Test patch:
diff --git a/pelican/tests/test_cli.py b/pelican/tests/test_cli.py
new file mode 100644
--- /dev/null
+++ b/pelican/tests/test_cli.py
@@ -0,0 +1,72 @@
+import unittest
+
+from pelican import get_config, parse_arguments
+
+
+class TestParseOverrides(unittest.TestCase):
+    def test_flags(self):
+        for flag in ['-e', '--extra-settings']:
+            args = parse_arguments([flag, 'k=1'])
+            self.assertDictEqual(args.overrides, {'k': 1})
+
+    def test_parse_multiple_items(self):
+        args = parse_arguments('-e k1=1 k2=2'.split())
+        self.assertDictEqual(args.overrides, {'k1': 1, 'k2': 2})
+
+    def test_parse_valid_json(self):
+        json_values_python_values_map = {
+            '""': '',
+            'null': None,
+            '"string"': 'string',
+            '["foo", 12, "4", {}]': ['foo', 12, '4', {}]
+        }
+        for k, v in json_values_python_values_map.items():
+            args = parse_arguments(['-e', 'k=' + k])
+            self.assertDictEqual(args.overrides, {'k': v})
+
+    def test_parse_invalid_syntax(self):
+        invalid_items = ['k= 1', 'k =1', 'k', 'k v']
+        for item in invalid_items:
+            with self.assertRaises(ValueError):
+                parse_arguments(f'-e {item}'.split())
+
+    def test_parse_invalid_json(self):
+        invalid_json = {
+            '', 'False', 'True', 'None', 'some other string',
+            '{"foo": bar}', '[foo]'
+        }
+        for v in invalid_json:
+            with self.assertRaises(ValueError):
+                parse_arguments(['-e ', 'k=' + v])
+
+
+class TestGetConfigFromArgs(unittest.TestCase):
+    def test_overrides_known_keys(self):
+        args = parse_arguments([
+            '-e',
+            'DELETE_OUTPUT_DIRECTORY=false',
+            'OUTPUT_RETENTION=["1.txt"]',
+            'SITENAME="Title"'
+        ])
+        config = get_config(args)
+        config_must_contain = {
+            'DELETE_OUTPUT_DIRECTORY': False,
+            'OUTPUT_RETENTION': ['1.txt'],
+            'SITENAME': 'Title'
+        }
+        self.assertDictEqual(config, {**config, **config_must_contain})
+
+    def test_overrides_non_default_type(self):
+        args = parse_arguments([
+            '-e',
+            'DISPLAY_PAGES_ON_MENU=123',
+            'PAGE_TRANSLATION_ID=null',
+            'TRANSLATION_FEED_RSS_URL="someurl"'
+        ])
+        config = get_config(args)
+        config_must_contain = {
+            'DISPLAY_PAGES_ON_MENU': 123,
+            'PAGE_TRANSLATION_ID': None,
+            'TRANSLATION_FEED_RSS_URL': 'someurl'
+        }
+        self.assertDictEqual(config, {**config, **config_must_contain})
diff --git a/pelican/tests/test_settings.py b/pelican/tests/test_settings.py
--- a/pelican/tests/test_settings.py
+++ b/pelican/tests/test_settings.py
@@ -7,7 +7,7 @@
 
 from pelican.settings import (DEFAULT_CONFIG, DEFAULT_THEME,
                               _printf_s_to_format_field,
-                              coerce_overrides, configure_settings,
+                              configure_settings,
                               handle_deprecated_settings, read_settings)
 from pelican.tests.support import unittest
 
@@ -304,18 +304,3 @@ def test_deprecated_slug_substitutions_from_file(self):
                          [(r'C\+\+', 'cpp')] +
                          self.settings['SLUG_REGEX_SUBSTITUTIONS'])
         self.assertNotIn('SLUG_SUBSTITUTIONS', settings)
-
-    def test_coerce_overrides(self):
-        overrides = coerce_overrides({
-            'ARTICLE_EXCLUDES': '["testexcl"]',
-            'READERS': '{"foo": "bar"}',
-            'STATIC_EXCLUDE_SOURCES': 'true',
-            'THEME_STATIC_DIR': 'theme',
-            })
-        expected = {
-            'ARTICLE_EXCLUDES': ["testexcl"],
-            'READERS': {"foo": "bar"},
-            'STATIC_EXCLUDE_SOURCES': True,
-            'THEME_STATIC_DIR': 'theme',
-        }
-        self.assertDictEqual(overrides, expected)