Repository: omni-us/jsonargparse

Issue:
Custom instantiator does not run when using nested dependency injection
First of all, thanks for the package, been using it for 5 years already as it simplifies my code bases quite a bit. 

## 🐛 Bug report

A custom class instantiator, added with `parser.add_instantiator(...)`, does not run when the class objects are added via _nested_ dependency injection. See the provided code and below explanations. I am using version `4.32.1`, but from the change logs it doesn't seem that this has been fixed.

### Case 1 (instantiator runs)

The instantiator runs if we do not use nesting. For example see Case 1 in the below, which executes the custom instantiator correctly.

### Case 2 (instantiator does not run)

In case 2, the objects of `Dependency` class are provided via nested dependency injection. In this case, the custom instantiator is not run.

### To reproduce

```python
from typing import Iterable, Callable, Type, Any, Dict

import jsonargparse
from jsonargparse._util import ClassType


class Dependency():
    def __init__(self, val: int, val2: float = 0.2):
        self.val = val
        self.val2 = val2

class Runner():
    def __init__(self, dependency: Callable[[Iterable], Dependency]):
        self.dependency = dependency(self)

class Wrapper():
    def __init__(self, runner: Runner):
        self.runner = runner

class CustomDependencyInstantiator():
        """
        A custom instantiator to be used in jsonargparse.

        Allows settings some default values based on the dynamically provided arguments.
        """
        def __call__(self, class_type: Type[ClassType], *args, **scheduler_kwargs) -> ClassType:
            runner = args[0]

            # TODO: Do something with the scheduler_kwargs using the arguments
            # This does not run when using dependency injection
            print('Running instantiator')

            return class_type(1, **scheduler_kwargs)

if __name__ == '__main__':
    #
    # Case 1: Using dependency injection without nesting, runs custom instantiator
    #
    config = {
        "dependency": {
            "val2": 0.3,
        }
    }

    parser = jsonargparse.ArgumentParser()
    parser.add_class_arguments(Runner)
    parser.add_instantiator(CustomDependencyInstantiator(), Dependency, subclasses=True)

    config = parser.parse_object(config)
    config = parser.instantiate_classes(config)

    wrapper = Runner(dependency=config.dependency)

    #
    # Case 2: Using dependency injection with nesting, does not run custom instantiator
    #
    config = {
        "runner": {
            "dependency": {
                "val2": 0.3,
            }
        }
    }

    parser = jsonargparse.ArgumentParser()
    parser.add_class_arguments(Wrapper)
    parser.add_instantiator(CustomDependencyInstantiator(), Dependency, subclasses=True)

    config = parser.parse_object(config)
    config = parser.instantiate_classes(config)

    wrapper = Wrapper(runner=config.runner)
```

### Expected behavior

<!-- Describe how the behavior or output of the reproduction snippet should be. -->

The custom instantiator runs even when using nested dependency injection.

### Environment

<!-- Fill in the list below. -->

- jsonargparse version: 4.32.1
- Python version: 3.10
- How jsonargparse was installed: `pip install jsonargparse[signatures]==4.32.1` in a conda environment
- OS: linux pop-os 6.9.3

Discussion:
@mauvilsa: Thank you very much for reporting! I ran the reproduction script and did see what the problem is. I will fix it later.

A couple of small comments which did not affect the understanding of the problem. First, the line `self.dependency = dependency(self)` is weird since `self` is not of type `Iterable`. Second, both cases are dependency injection, see [class-type-and-sub-classes](https://jsonargparse.readthedocs.io/en/v4.33.2/#class-type-and-sub-classes). The difference is that case 2 is nested.@vsimkus: Thanks for looking into this and confirming so quickly!

> First, the line `self.dependency = dependency(self)` is weird since `self` is not of type `Iterable`. 

Yeah, I couldn't use `Runner` class (instead of `Iterable`) in the specification of the `__init__` method, since it's not yet defined, so I chose `Iterable`; doesn't matter for this example. 

> Second, both cases are dependency injection, see [class-type-and-sub-classes](https://jsonargparse.readthedocs.io/en/v4.33.2/#class-type-and-sub-classes). The difference is that case 2 is nested.

Thanks for the correction, I have updated the bug report to be more accurate. 

@mauvilsa: This should be fixed with #608.@vsimkus: Awesome, looks like it worked! Thanks for doing this.

Solution patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index c5db0c5f..489125ef 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -29,6 +29,8 @@ Fixed
   <https://github.com/omni-us/jsonargparse/pull/597>`__).
 - Callables that return class not considering previous values (`#603
   <https://github.com/omni-us/jsonargparse/pull/603>`__).
+- Custom instantiators not working for nested dependency injection (`#608
+  <https://github.com/omni-us/jsonargparse/pull/608>`__).
 
 Changed
 ^^^^^^^
diff --git a/jsonargparse/_common.py b/jsonargparse/_common.py
index f4e99bad..9af90e53 100644
--- a/jsonargparse/_common.py
+++ b/jsonargparse/_common.py
@@ -59,7 +59,7 @@ def __call__(self, class_type: Type[ClassType], *args, **kwargs) -> ClassType:
 defaults_cache: ContextVar[Optional[Namespace]] = ContextVar("defaults_cache", default=None)
 lenient_check: ContextVar[Union[bool, str]] = ContextVar("lenient_check", default=False)
 load_value_mode: ContextVar[Optional[str]] = ContextVar("load_value_mode", default=None)
-class_instantiators: ContextVar[Optional[InstantiatorsDictType]] = ContextVar("class_instantiators")
+class_instantiators: ContextVar[Optional[InstantiatorsDictType]] = ContextVar("class_instantiators", default=None)
 nested_links: ContextVar[List[dict]] = ContextVar("nested_links", default=[])
 
 
diff --git a/jsonargparse/_core.py b/jsonargparse/_core.py
index ce89003e..86a575a0 100644
--- a/jsonargparse/_core.py
+++ b/jsonargparse/_core.py
@@ -39,6 +39,7 @@
 from ._common import (
     InstantiatorCallable,
     InstantiatorsDictType,
+    class_instantiators,
     debug_mode_active,
     is_dataclass_like,
     lenient_check,
@@ -1139,6 +1140,10 @@ def _get_instantiators(self):
             parent_instantiators = self.parent_parser._get_instantiators()
             instantiators = instantiators.copy()
             instantiators.update({k: v for k, v in parent_instantiators.items() if k not in instantiators})
+        context_instantiators = class_instantiators.get()
+        if context_instantiators:
+            instantiators = instantiators.copy()
+            instantiators.update({k: v for k, v in context_instantiators.items() if k not in instantiators})
         return instantiators
 
     def instantiate_classes(


Test patch:
diff --git a/jsonargparse_tests/test_subclasses.py b/jsonargparse_tests/test_subclasses.py
index f85fe50f..e433fc2a 100644
--- a/jsonargparse_tests/test_subclasses.py
+++ b/jsonargparse_tests/test_subclasses.py
@@ -479,6 +479,21 @@ def test_custom_instantiation_replace(parser):
     assert list(parser._instantiators.values())[0] is second_instantiator
 
 
+class CustomInstantiationNested:
+    def __init__(self, sub: CustomInstantiationBase):
+        self.sub = sub
+
+
+def test_custom_instantiation_nested(parser):
+    parser.add_argument("--cls", type=CustomInstantiationNested)
+    parser.add_instantiator(instantiator("nested"), CustomInstantiationBase, subclasses=True)
+    cfg = parser.parse_args(["--cls=CustomInstantiationNested", "--cls.sub=CustomInstantiationSub"])
+    init = parser.instantiate_classes(cfg)
+    assert isinstance(init.cls, CustomInstantiationNested)
+    assert isinstance(init.cls.sub, CustomInstantiationSub)
+    assert init.cls.sub.call == "nested"
+
+
 # environment tests