Repository: explosion/thinc

Issue:
Configuration: interpolation of resolved values
I have a use case where I need to interpolate resolved values in a `thinc.config` configuration file:

```ini
[task]
@task = "MyTask"

[architecture]
task = ${task}   <-- interpolated here...

[training]
task = ${task}  <-- ... and also here
```

`Config.from_disk("config.cfg")` gives me the following error:

```python
InterpolationDepthError: Recursion limit exceeded in value substitution: option 'task' in section 'architecture' contains an interpolation key which cannot be substituted in 10 steps. Raw value: '${task}'
```

This is not a suprise: this limitation is described in the documentation.

> Interpolation happens when the config is parsed, i.e. when you create a Config object from a file, string, etc. and before registered functions are resolved. This means that you can only use it for hard-coded values like numbers and strings, not to define placeholders for resolved values. For instance, the [model] section in this example will be resolved to an instance of Relu – but you wouldn’t be able to use ${model} as the argument to another function defined in the config. 

However, the documentation also says
> To achieve this, you can either define another nested block or variable positional arguments.

I am not sure what this means exactly. How would one achieve what I am trying to achieve with `positional arguments` or `nested block` in my particular case?  






Discussion:
Hey, nice to see you on the Thinc tracker! 👋

I just had to re-read that paragraph (that I likely wrote myself) a couple of times to refresh my memory, so it's definitely more confusing than it should have been and we should adjust this. 

I'm pretty sure what it means is that you can rewrite your config to look like this:

```ini
[architecture]

[architecture.task]
@task = "MyTask"

[training]

[training.task]
@task = "MyTask"
```

If `MyTask` has arguments, you could make those variables, so you're only defining them once. However, it's of course not fully equivalent, because here you'd be creating _two separate instances_ of the task and you're not passing the same instance into both.

I don't think there's currently a way to use the function registry to create an object once and then pass that same instance into multiple other functions. At least I can't think of one and couldn't make it work so far? Maybe @honnibal has an idea.
Agree that something like this should be possible, and isn't currently.

In the original design I figured something like this would be possible, likely with nicer syntax:

```
[task]
name = "hi"
data = null

[task.data]
@task = "MyTask"

[obj1]
arg = ${task:data}

[obj2]
arg = ${task:data}
```

This works because the Python configparser actually doesn't do early substitution of the references. It keeps them as pointers, so when we replace `task.data` using the section flattening, the references will resolve to the new value.

However, as we've added more validation, even this ugly workaround no longer functions: we check that `task.data` isn't a previously set value, and we raise on the conflict (correctly).

So in summary, there are two problems to solve before we can make this feature work.

1) `${task}` refers to a block, not a value-within-a-block. That's not legal with the current interpolation, but IMO we do want it to be legal.

2) `[task.block]` will be filled to `cfg["task"]["block"]` after `${task:block}` checks that the value exists. We'll currently get an exception from that, because it's a reference to a missing value. We don't want to force the user to specify a dummy value.

(If we instead forced the user to specify a dummy value, we'd have to solve the different problem of config validation and making sure the block gets assigned to overwrite the dummy value. But this is all bad, I'm sure we don't want this path.)

I think we can implement this one of two ways.

1) Initially, we could set `interpolation=False`. This will give us a chance to get in and fix the data while it's still just a dict. We can then activate the interpolation using the built-in `ExtendedInterpolation` class.
2) We can implement our own interpolation handler, which does things our way.

I think 2) seems better? It sets us up for more changes in future, and is less weird and hacky.
Here's the class we need to subclass: https://github.com/python/cpython/blob/master/Lib/configparser.py#L448 . It needs to be passed into our `get_configparser` function.

We need to implement four hooks. I wouldn't inherit, I think that just makes it less clear. If we want to use the private method of the existing `ExtendedInterpolation` class, I'd rather we vendored it in. It's kind of ugly to be calling into private methods of the standard library.

```python
class ConfigInterpolation:
    def before_get(self, parser: ConfigParser, section: str, option: str, value: str, defaults: dict):
        return value

    def before_set(self, parser: ConfigParser, section: str, option: str, value: str):
        return value

    def before_read(self, parser: ConfigParser, section: str, option: str, value: st:r):
        return value

    def before_write(self, parser: ConfigParser, section: str, option: str, value: str):
        return value
```

The `option` argument is the key, e.g. `data`. It's not the name I would've chosen but we're sort of stuck with it.

I'm kind of baffled by the `ExtendedInterpolation` implementation that's in the standard library. Is this..to support nested interpolation? Like can you write `${${some:value}:${another:value}}`. If so, I'm pretty sure we don't want that.
Thanks for looking into this so quickly.

I went with a workaround for now (i.e. set the "task" attribute manually instead of relying on `Config.from_disk`) but this new feature would definitely make my code less ugly.

Resolving patch:
diff --git a/thinc/config.py b/thinc/config.py
--- a/thinc/config.py
+++ b/thinc/config.py
@@ -1,7 +1,8 @@
 from typing import Union, Dict, Any, Optional, List, Tuple, Callable, Type, Sequence
 from types import GeneratorType
-from configparser import ConfigParser, ExtendedInterpolation
-from configparser import InterpolationMissingOptionError
+from configparser import ConfigParser, ExtendedInterpolation, MAX_INTERPOLATION_DEPTH
+from configparser import InterpolationMissingOptionError, InterpolationSyntaxError
+from configparser import NoSectionError, NoOptionError, InterpolationDepthError
 from pathlib import Path
 from pydantic import BaseModel, create_model, ValidationError
 from pydantic.main import ModelMetaclass
@@ -16,8 +17,91 @@
 from .types import Decorator
 
 
+SECTION_PREFIX = "__SECTION__:"
+
+
+class CustomInterpolation(ExtendedInterpolation):
+    def before_read(self, parser, section, option, value):
+        # If we're dealing with a quoted string as the interpolation value,
+        # make sure we load and unquote it so we don't end up with '"value"'
+        try:
+            json_value = srsly.json_loads(value)
+            if isinstance(json_value, str):
+                value = json_value
+        except Exception:
+            pass
+        return super().before_read(parser, section, option, value)
+
+    def before_get(self, parser, section, option, value, defaults):
+        # Mostly copy-pasted from the built-in configparser implementation.
+        L = []
+        self.interpolate(parser, option, L, value, section, defaults, 1)
+        return "".join(L)
+
+    def interpolate(self, parser, option, accum, rest, section, map, depth):
+        # Mostly copy-pasted from the built-in configparser implementation.
+        # We need to overwrite this method so we can add special handling for
+        # block references :( All values produced here should be strings –
+        # we need to wait until the whole config is interpreted anyways so
+        # filling in incomplete values here is pointless. All we need is the
+        # section reference so we can fetch it later.
+        rawval = parser.get(section, option, raw=True, fallback=rest)
+        if depth > MAX_INTERPOLATION_DEPTH:
+            raise InterpolationDepthError(option, section, rawval)
+        while rest:
+            p = rest.find("$")
+            if p < 0:
+                accum.append(rest)
+                return
+            if p > 0:
+                accum.append(rest[:p])
+                rest = rest[p:]
+            # p is no longer used
+            c = rest[1:2]
+            if c == "$":
+                accum.append("$")
+                rest = rest[2:]
+            elif c == "{":
+                m = self._KEYCRE.match(rest)
+                if m is None:
+                    err = f"bad interpolation variable reference {rest}"
+                    raise InterpolationSyntaxError(option, section, err)
+                path = m.group(1).split(":")
+                rest = rest[m.end() :]
+                sect = section
+                opt = option
+                try:
+                    if len(path) == 1:
+                        opt = parser.optionxform(path[0])
+                        if opt in map:
+                            v = map[opt]
+                        else:
+                            # We have block reference, store it as a special key
+                            section_name = parser[parser.optionxform(path[0])]._name
+                            v = f"{SECTION_PREFIX}{section_name}"
+                    elif len(path) == 2:
+                        sect = path[0]
+                        opt = parser.optionxform(path[1])
+                        v = parser.get(sect, opt, raw=True)
+                    else:
+                        err = f"More than one ':' found: {rest}"
+                        raise InterpolationSyntaxError(option, section, err)
+                except (KeyError, NoSectionError, NoOptionError):
+                    raise InterpolationMissingOptionError(
+                        option, section, rawval, ":".join(path)
+                    ) from None
+                if "$" in v:
+                    new_map = dict(parser.items(sect, raw=True))
+                    self.interpolate(parser, opt, accum, v, sect, new_map, depth + 1)
+                else:
+                    accum.append(v)
+            else:
+                err = "'$' must be followed by '$' or '{', " "found: %r" % (rest,)
+                raise InterpolationSyntaxError(option, section, err)
+
+
 def get_configparser():
-    config = ConfigParser(interpolation=ExtendedInterpolation())
+    config = ConfigParser(interpolation=CustomInterpolation())
     # Preserve case of keys: https://stackoverflow.com/a/1611877/6400719
     config.optionxform = str
     return config
@@ -39,7 +123,7 @@ def __init__(
             data = {}
         self.update(data)
 
-    def interpret_config(self, config: Union[Dict[str, Any], "ConfigParser"]):
+    def interpret_config(self, config: "ConfigParser") -> None:
         """Interpret a config, parse nested sections and parse the values
         as JSON. Mostly used internally and modifies the config in place.
         """
@@ -62,6 +146,7 @@ def interpret_config(self, config: Union[Dict[str, Any], "ConfigParser"]):
                     raise ConfigValidationError(self, err, message=err_title)
                 else:
                     node = node[part]
+            # TODO: add error if node not in list
             node = node.setdefault(parts[-1], {})
             if not isinstance(node, dict):
                 # Happens if both value *and* subsection were defined for a key
@@ -77,17 +162,30 @@ def interpret_config(self, config: Union[Dict[str, Any], "ConfigParser"]):
                 )
                 raise ConfigValidationError(f"{e}\n\n{err_msg}", [])
             for key, value in keys_values:
+                config_v = config.get(section, key)
                 try:
-                    node[key] = srsly.json_loads(config.get(section, key))
-                except Exception as e:
-                    err_msg = (
-                        f"Error reading key '{key}' in section '{section}': {e}. "
-                        f"If your value is a string, make sure it was provided "
-                        f"in quotes. If your value is a boolean, make sure it was "
-                        f"written in lowercase. If you want to specify a 'None' "
-                        f"value, write it as 'null' (without quotes)."
-                    )
-                    raise ValueError(err_msg)
+                    node[key] = srsly.json_loads(config_v)
+                except Exception:
+                    node[key] = config_v
+        self.replace_section_refs(self)
+
+    def replace_section_refs(self, config: Union[Dict[str, Any], "Config"]) -> None:
+        """Replace references to section blocks in the final config."""
+        for key, value in config.items():
+            if isinstance(value, dict):
+                self.replace_section_refs(value)
+            elif isinstance(value, str) and value.startswith(SECTION_PREFIX):
+                parts = value.replace(SECTION_PREFIX, "").split(".")
+                result = self
+                for item in parts:
+                    try:
+                        result = result[item]
+                    except (KeyError, TypeError):  # This should never happen
+                        err_title = "Error parsing reference to config section"
+                        err_msg = f"Section '{'.'.join(parts)}' is not defined"
+                        err = [{"loc": parts, "msg": err_msg}]
+                        raise ConfigValidationError(self, err, message=err_title)
+                config[key] = result
 
     def copy(self) -> "Config":
         """Deepcopy the config."""


Test patch:
diff --git a/thinc/tests/test_config.py b/thinc/tests/test_config.py
--- a/thinc/tests/test_config.py
+++ b/thinc/tests/test_config.py
@@ -823,14 +823,61 @@ def test_config_to_str_order():
     assert config.to_str() == expected
 
 
-@pytest.mark.xfail(reason="interpolation doesn't work because json.loads")
 def test_config_interpolation():
     config_str = """[a]\nfoo = "hello"\n\n[b]\nbar = ${a.foo}"""
     with pytest.raises(ConfigValidationError):
         Config().from_str(config_str)
     config_str = """[a]\nfoo = "hello"\n\n[b]\nbar = ${a:foo}"""
-    config = Config().from_str(config_str)
-    assert config["b"]["bar"] == "hello"
+    assert Config().from_str(config_str)["b"]["bar"] == "hello"
     config_str = """[a]\nfoo = "hello"\n\n[b]\nbar = ${a:foo}!"""
+    assert Config().from_str(config_str)["b"]["bar"] == "hello!"
+    config_str = """[a]\nfoo = "hello"\n\n[b]\nbar = "${a:foo}!\""""
+    assert Config().from_str(config_str)["b"]["bar"] == "hello!"
+    config_str = """[a]\nfoo = 15\n\n[b]\nbar = ${a:foo}!"""
+    assert Config().from_str(config_str)["b"]["bar"] == "15!"
+    config_str = """[a]\nfoo = ["x", "y"]\n\n[b]\nbar = ${a:foo}"""
+    assert Config().from_str(config_str)["b"]["bar"] == ["x", "y"]
+
+
+def test_config_interpolation_sections():
+    # Simple block references
+    config_str = """[a]\nfoo = "hello"\nbar = "world"\n\n[b]\nc = ${a}"""
+    config = Config().from_str(config_str)
+    assert config["b"]["c"] == config["a"]
+    # References with non-string values
+    config_str = """[a]\nfoo = "hello"\n\n[a.x]\ny = ${a.b}\n\n[a.b]\nc = 1\nd = [10]"""
+    config = Config().from_str(config_str)
+    assert config["a"]["x"]["y"] == config["a"]["b"]
+    # Multiple references in the same string
+    config_str = """[a]\nx = "string"\ny = 10\n\n[b]\nz = "${a:x}/${a:y}\""""
+    config = Config().from_str(config_str)
+    assert config["b"]["z"] == "string/10"
+    # Non-string references in string (converted to string)
+    config_str = """[a]\nx = ["hello", "world"]\n\n[b]\ny = "result: ${a:x}\""""
+    config = Config().from_str(config_str)
+    assert config["b"]["y"] == 'result: ["hello", "world"]'
+    # References to sections referencing sections
+    config_str = """[a]\nfoo = "x"\n\n[b]\nbar = ${a}\n\n[c]\nbaz = ${b}"""
     config = Config().from_str(config_str)
-    assert config["b"]["bar"] == "hello!"
+    assert config["b"]["bar"] == config["a"]
+    assert config["c"]["baz"] == config["b"]
+    # References to section values referencing other sections
+    config_str = """[a]\nfoo = "x"\n\n[b]\nbar = ${a}\n\n[c]\nbaz = ${b:bar}"""
+    config = Config().from_str(config_str)
+    assert config["c"]["baz"] == config["b"]["bar"]
+    # References to sections with subsections
+    config_str = """[a]\nfoo = "x"\n\n[a.b]\nbar = 100\n\n[c]\nbaz = ${a}"""
+    config = Config().from_str(config_str)
+    assert config["c"]["baz"] == config["a"]
+    # Infinite recursion
+    config_str = """[a]\nfoo ="x"\n\n[a.b]\nbar = ${a}"""
+    config = Config().from_str(config_str)
+    assert config["a"]["b"]["bar"] == config["a"]
+    config_str = """[a]\nfoo = "x"\n\n[b]\nbar = ${a}\n\n[c]\nbaz = ${b.bar:foo}"""
+    # We can't reference not-yet interpolated subsections
+    with pytest.raises(ConfigValidationError):
+        config = Config().from_str(config_str)
+    # Generally invalid references
+    config_str = """[a]\nfoo = ${b.bar}"""
+    with pytest.raises(ConfigValidationError):
+        config = Config().from_str(config_str)