Repository: meltano/meltano

Issue:
Bug: Setting array values with `--interactive` doesn't support common array syntaxes, and doesn't provide a format hint.
In configuring values that are stored as string arrays, we don't provide any hint about how to set those value.

These all do not work:

- `asdf`
- `asdf, asdf2`
- `'asdf', 'asdf2'`
- `"asdf", "asdf2"`
- `[asdf]`
- `['asdf']`

The only error message is something like

> Failed to set value: Expecting value: line 1 column 2 (char 1)

Or:

> Failed to set value: Extra data: line 1 column 7 (char 6)

The expected syntax is square brackets, with double-quoted strings:

- `["asdf"]`
- `["asdf", "asdf2"]`

A simple fix would be to update the error handling to print a message with the required syntax whenever the input fails to parse. Something like:

> Failed to set value: Extra data: line 1 column 7 (char 6).
> Expected array syntax: ["item1", "item2"]


Raise in slack:

- https://meltano.slack.com/archives/C01TCRBBJD7/p1676926823957639


Discussion:
While we're addressing this, we should consider updating the places where we currently use `json.loads` for these values to use [`ast.literal_eval`](https://docs.python.org/3/library/ast.html#ast.literal_eval) as a fallback.

Making this change would broaden the accepted formats to a superset which includes e.g. `['asdf', "asdf",]` (note the trailing comma), or using parentheses instead of square brackets, or using no brackets at all.

Type checking would be required to ensure that a `collections.abc.Sequence` instance is received, but that's currently true as well since a user could enter something like `{"key":"value"}` as a JSON input.

This may be worth splitting off into a separate issue. Thoughts? @aaronsteers @tayloramurphy
I'm not familiar with ast.literal_eval, but if that improves the UX experience, I think it's a valid solve for this issue. As always, we still need to watch out for regressions and scope creep, but I'm in favor of that as a fix here if it's relatively small scope.
@aaronsteers I've updated my comment with a link to the docs for `ast.literal_eval`. In summary (paraphrasing):

> Evaluate a string containing only a Python literal, which consists of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, `None` and `Ellipsis` (i.e. `...`). This is specifically designed not to execute Python code, unlike the more general `eval()`.
Nice.

<details>
<summary>Details</summary>

Parsing works (basically) as should be expected. When we know we are looking for an array type, we might want to auto-convert sets and tuples to lists.

```py
aj@AJs-MacBook-Pro ~ % python
Python 3.10.6 (main, Sep 30 2022, 13:17:42) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ast import literal_eval
>>> literal_eval("['foo','bar']")
['foo', 'bar']
>>> literal_eval("[\"foo\",'bar']")
['foo', 'bar']
>>> literal_eval("\"foo\",'bar'")
('foo', 'bar')
>>> literal_eval("{\"foo\",'bar'}")
{'foo', 'bar'}
>>> literal_eval("{\"foo\": 'bar'}")
{'foo': 'bar'}
>>> literal_eval("'foobar'")
'foobar'
>>> literal_eval("'foobar'")
```

Failure message (`ValueError: malformed node or string on line 1`) is not much better/worse than existing:

```
>>> literal_eval("[foo,bar]")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/aj/.pyenv/versions/3.10.6/lib/python3.10/ast.py", line 108, in literal_eval
    return _convert(node_or_string)
  File "/Users/aj/.pyenv/versions/3.10.6/lib/python3.10/ast.py", line 88, in _convert
    return list(map(_convert, node.elts))
  File "/Users/aj/.pyenv/versions/3.10.6/lib/python3.10/ast.py", line 107, in _convert
    return _convert_signed_num(node)
  File "/Users/aj/.pyenv/versions/3.10.6/lib/python3.10/ast.py", line 81, in _convert_signed_num
    return _convert_num(node)
  File "/Users/aj/.pyenv/versions/3.10.6/lib/python3.10/ast.py", line 72, in _convert_num
    _raise_malformed_node(node)
  File "/Users/aj/.pyenv/versions/3.10.6/lib/python3.10/ast.py", line 69, in _raise_malformed_node
    raise ValueError(msg + f': {node!r}')
ValueError: malformed node or string on line 1: <ast.Name object at 0x109100dc0>
```

</details>

Resolving patch:
diff --git a/src/meltano/core/setting_definition.py b/src/meltano/core/setting_definition.py
--- a/src/meltano/core/setting_definition.py
+++ b/src/meltano/core/setting_definition.py
@@ -2,8 +2,10 @@
 
 from __future__ import annotations
 
+import ast
 import json
 import typing as t
+from collections.abc import Mapping, Sequence
 from datetime import date, datetime
 from enum import Enum
 
@@ -59,9 +61,8 @@ def get(self, env) -> str:
         Returns:
             Env var value for given env var.
         """
-        if self.negated:
-            return str(not utils.truthy(env[self.key]))
-        return env[self.key]
+        value = env[self.key]
+        return str(not utils.truthy(value)) if self.negated else value
 
 
 class SettingMissingError(Error):
@@ -144,6 +145,9 @@ class SettingKind(YAMLEnum):
     HIDDEN = "hidden"
 
 
+ParseValueExpectedType = t.TypeVar("ParseValueExpectedType")
+
+
 class SettingDefinition(NameEq, Canonical):
     """Meltano SettingDefinition class."""
 
@@ -353,6 +357,55 @@ def env_vars(
 
         return [EnvVar(key) for key in utils.uniques_in(env_keys)]
 
+    @staticmethod
+    def _parse_value(
+        unparsed: str,
+        expected_type_name: str,
+        expected_type: type[ParseValueExpectedType],
+    ) -> ParseValueExpectedType:
+        """Parse a JSON string.
+
+        Parsing is attempted first with `json.loads`, and then with
+        `ast.literal_eval` as a fallback. It is used as a fallback because it
+        correctly parses most inputs that `json.loads` can parse, but is more
+        liberal about what it accepts. For example, `json.loads` requires
+        double quotes for strings, but `ast.literal_eval` can use either single
+        or double quotes.
+
+        Args:
+            unparsed: The JSON string.
+            expected_type_name: The name of the expected type, e.g. "array".
+                Used in the error message if parsing fails or the type is not
+                as expected.
+            expected_type: The Python type class of the expected type. Used to
+                ensure that the parsed value is of the expected type.
+
+        Raises:
+            parse_error: Parsing failed, or the parsed value had an unexpected type.
+
+        Returns:
+            The parsed value.
+        """
+        parse_error = ValueError(
+            f"Failed to parse JSON {expected_type_name} from string: {unparsed!r}"
+        )
+        try:
+            parsed = json.loads(unparsed)
+        except json.JSONDecodeError:
+            try:  # noqa: WPS505
+                parsed = ast.literal_eval(unparsed)
+            except (
+                ValueError,
+                TypeError,
+                SyntaxError,
+                MemoryError,
+                RecursionError,
+            ) as ex:
+                raise parse_error from ex
+        if not isinstance(parsed, expected_type):
+            raise parse_error
+        return parsed
+
     def cast_value(self, value: t.Any) -> t.Any:  # noqa: C901
         """Cast given value.
 
@@ -361,9 +414,6 @@ def cast_value(self, value: t.Any) -> t.Any:  # noqa: C901
 
         Returns:
             Value cast according to specified setting definition kind.
-
-        Raises:
-            ValueError: if value cannot be cast to setting kind.
         """
         value = value.isoformat() if isinstance(value, (date, datetime)) else value
 
@@ -373,13 +423,13 @@ def cast_value(self, value: t.Any) -> t.Any:  # noqa: C901
             elif self.kind == SettingKind.INTEGER:
                 return int(value)
             elif self.kind == SettingKind.OBJECT:
-                value = json.loads(value)
-                if not isinstance(value, dict):
-                    raise ValueError(f"JSON value '{value}' is not an object")
+                value = dict(
+                    self._parse_value(value, "object", Mapping),  # type: ignore
+                )
             elif self.kind == SettingKind.ARRAY:
-                value = json.loads(value)
-                if not isinstance(value, list):
-                    raise ValueError(f"JSON value '{value}' is not an array")
+                value = list(
+                    self._parse_value(value, "array", Sequence),  # type: ignore
+                )
 
         processor = self.value_processor
         if value is not None and processor:


Test patch:
diff --git a/tests/meltano/core/test_setting_definition.py b/tests/meltano/core/test_setting_definition.py
new file mode 100644
--- /dev/null
+++ b/tests/meltano/core/test_setting_definition.py
@@ -0,0 +1,57 @@
+from __future__ import annotations
+
+import pytest
+
+from meltano.core.setting_definition import SettingDefinition, SettingKind
+
+
+class TestSettingDefinition:
+    @pytest.mark.parametrize(
+        ("setting_definition", "uncast_expected_pairs"),
+        (
+            (
+                SettingDefinition("test_setting", kind=SettingKind.ARRAY),
+                (
+                    ('["abc", "xyz"]', ["abc", "xyz"]),
+                    ('["abc", "xyz",]', ["abc", "xyz"]),
+                    ("['abc', 'xyz']", ["abc", "xyz"]),
+                    ("[abc, xyz]", ValueError),
+                    ("'abc', 'xyz'", ["abc", "xyz"]),
+                    ("'abc', 'xyz',", ["abc", "xyz"]),
+                    ('[null, "xyz"]', [None, "xyz"]),
+                    ("{'abc', 'xyz'}", ValueError),
+                    ("1234", ValueError),
+                    ("[1234, 5678]", [1234, 5678]),
+                ),
+            ),
+            (
+                SettingDefinition("test_setting", kind=SettingKind.OBJECT),
+                (
+                    (
+                        '{"key_1": "value_1", "key_2": "value_2"}',
+                        {"key_1": "value_1", "key_2": "value_2"},
+                    ),
+                    ('{"key": "value",}', {"key": "value"}),
+                    ('{"key": null}', {"key": None}),
+                    ("{'key': null}", ValueError),
+                    ("key: value", ValueError),
+                    ("{'key': 'value'}", {"key": "value"}),
+                    ("{key: value}", ValueError),
+                    ("{'abc', 'xyz'}", ValueError),
+                    ("1234", ValueError),
+                ),
+            ),
+        ),
+        ids=("array", "object"),
+    )
+    def test_cast_value_array(
+        self,
+        setting_definition: SettingDefinition,
+        uncast_expected_pairs,
+    ):
+        for uncast, expected in uncast_expected_pairs:
+            if isinstance(expected, type) and issubclass(expected, Exception):
+                with pytest.raises(expected):
+                    setting_definition.cast_value(uncast)
+            else:
+                assert setting_definition.cast_value(uncast) == expected