Repository: omni-us/jsonargparse

Issue:
Support for subcommands and `instantiate_classes`
I'm trying to use subcommands and `instantiate_classes` together.

Without subcommand, everything works properly:

```python
from jsonargparse import ArgumentParser

class Foo:
    def __init__(self, a: int = 1):
        ...

parser = ArgumentParser(parse_as_dict=True)
key = "foo"
parser.add_class_arguments(Foo, key)

config = parser.parse_args()
print(config)
config_init = parser.instantiate_classes(config)
# the class should be instantiated
print(config_init)
assert isinstance(config_init[key], Foo)
```

```python
{'foo': {'a': 1}}
{'foo': <__main__.Foo object at 0x102fc90d0>}
```

However, with subcommand enabled, seems like `instantiate_classes` does not act recursively.

```python
from jsonargparse import ArgumentParser

class Foo:
    def __init__(self, a: int = 1):
        ...

parser = ArgumentParser(parse_as_dict=True)
subcommands = parser.add_subcommands()
subparser = ArgumentParser()
key = "foo"
subparser.add_class_arguments(Foo, key)
subcommand = "cmd"
subcommands.add_subcommand(subcommand, subparser)

config = parser.parse_args([subcommand])
print(config)
# the class should be instantiated even if it is under a subcommand
config_init = parser.instantiate_classes(config)
print(config_init)
assert isinstance(config_init[subcommand][key], Foo)
```

```python
{'subcommand': 'cmd', 'cmd': {'foo': {'a': 1}}}
{'subcommand': 'cmd', 'cmd': {'foo': {'a': 1}}}
Traceback (most recent call last):
  File "script.py", line 33, in <module>
    assert isinstance(config_init[subcommand][key], Foo)
AssertionError
```

I would expect the second print to be

```python
{'subcommand': 'cmd', 'cmd': {'foo': <__main__.Foo object at 0x102fc90d0>}}
```

Not sure if this is a bug or a feature. Is this not a valid way to instantiate classes under subcommands?

If I try to do instead

```python
config_init = parser.instantiate_classes(config["subcommand"])
print(config_init)
```

I get:

```python
Traceback (most recent call last):
  File "/Users/carlosmocholi/Library/Application Support/JetBrains/PyCharmCE2020.3/scratches/scratch_12.py", line 32, in <module>
    config_init = parser.instantiate_classes(config["subcommand"])
  File "/Users/carlosmocholi/.pyenv/versions/3.9.1/envs/lightning-3.9/lib/python3.9/site-packages/jsonargparse/core.py", line 1103, in instantiate_classes
    cfg = strip_meta(cfg)
  File "/Users/carlosmocholi/.pyenv/versions/3.9.1/envs/lightning-3.9/lib/python3.9/site-packages/jsonargparse/util.py", line 247, in strip_meta
    cfg = namespace_to_dict(cfg)
  File "/Users/carlosmocholi/.pyenv/versions/3.9.1/envs/lightning-3.9/lib/python3.9/site-packages/jsonargparse/util.py", line 233, in namespace_to_dict
    return expand_namespace(cfg_ns)
  File "/Users/carlosmocholi/.pyenv/versions/3.9.1/envs/lightning-3.9/lib/python3.9/site-packages/jsonargparse/util.py", line 224, in expand_namespace
    cfg = dict(vars(cfg))
TypeError: vars() argument must have __dict__ attribute
```

Discussion:
sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=CODE_SMELL)

[![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=78) No Coverage information  
[![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=78&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=78&metric=new_duplicated_lines_density&view=list)


sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=78&resolved=false&types=CODE_SMELL)

[![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=78) No Coverage information  
[![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=78&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=78&metric=new_duplicated_lines_density&view=list)


codecov-commenter: # [Codecov](https://codecov.io/gh/omni-us/jsonargparse/pull/78?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) Report
> Merging [#78](https://codecov.io/gh/omni-us/jsonargparse/pull/78?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) (a0b8eb9) into [master](https://codecov.io/gh/omni-us/jsonargparse/commit/bdc7635685220fa7b108c06626ae1902bad84fc2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) (bdc7635) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/omni-us/jsonargparse/pull/78/graphs/tree.svg?width=650&height=150&src=pr&token=nEueHrMNDh&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us)](https://codecov.io/gh/omni-us/jsonargparse/pull/78?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us)

```diff
@@           Coverage Diff           @@
##           master      #78   +/-   ##
=======================================
  Coverage   99.58%   99.59%           
=======================================
  Files          13       13           
  Lines        2926     2936   +10     
=======================================
+ Hits         2914     2924   +10     
  Misses         12       12           
```

| Flag | Coverage Δ | |
|---|---|---|
| py3.5 | `86.78% <100.00%> (+0.01%)` | :arrow_up: |
| py3.5_all | `94.12% <100.00%> (+0.02%)` | :arrow_up: |
| py3.6 | `86.78% <100.00%> (+0.04%)` | :arrow_up: |
| py3.6_all | `97.28% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.7 | `88.08% <100.00%> (+0.04%)` | :arrow_up: |
| py3.7_all | `98.76% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.8 | `88.17% <100.00%> (+0.04%)` | :arrow_up: |
| py3.8_all | `98.77% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.9 | `?` | |
| py3.9_all | `?` | |
| pypy3 | `?` | |
| pypy3_all | `?` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/omni-us/jsonargparse/pull/78?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Coverage Δ | |
|---|---|---|
| [jsonargparse/actions.py](https://codecov.io/gh/omni-us/jsonargparse/pull/78/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#diff-anNvbmFyZ3BhcnNlL2FjdGlvbnMucHk=) | `100.00% <100.00%> (ø)` | |
| [jsonargparse/core.py](https://codecov.io/gh/omni-us/jsonargparse/pull/78/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#diff-anNvbmFyZ3BhcnNlL2NvcmUucHk=) | `99.53% <100.00%> (+<0.01%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/omni-us/jsonargparse/pull/78?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/omni-us/jsonargparse/pull/78?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us). Last update [bdc7635...a0b8eb9](https://codecov.io/gh/omni-us/jsonargparse/pull/78?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 5cfda88..3230abe 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -23,6 +23,7 @@ Fixed
 - Don't ignore ``KeyError`` in call to instantiate_classes #81.
 - Optional subcommands fail with a KeyError #68.
 - Conflicting namespace for subclass key in subcommand.
+- ``instantiate_classes`` not working for subcommand keys #70.
 
 Changed
 ^^^^^^^
diff --git a/jsonargparse/actions.py b/jsonargparse/actions.py
index ac3b78a..6cedc61 100644
--- a/jsonargparse/actions.py
+++ b/jsonargparse/actions.py
@@ -733,14 +733,10 @@ class _ActionSubCommands(_SubParsersAction):
 
 
     @staticmethod
-    def handle_subcommands(parser, cfg, env, defaults, prefix='', fail_no_subcommand=True):
-        """Adds sub-command dest if missing and parses defaults and environment variables."""
+    def get_subcommand(parser, cfg_dict:dict, prefix:str='', fail_no_subcommand:bool=True):
+        """Returns the sub-command name and corresponding subparser."""
         if parser._subparsers is None:
-            return
-
-        cfg_dict = cfg.__dict__ if isinstance(cfg, Namespace) else cfg
-        cfg_keys = set(vars(_dict_to_flat_namespace(cfg)).keys())
-        cfg_keys = cfg_keys.union(set(cfg_dict.keys()))
+            return None, None
 
         # Get subcommands action
         for action in parser._actions:
@@ -760,11 +756,27 @@ class _ActionSubCommands(_SubParsersAction):
             cfg_dict[dest] = subcommand
 
         if subcommand is None and not (fail_no_subcommand and action._required):
-            return
+            return None, None
         if action._required and subcommand not in action._name_parser_map:
             raise KeyError('Sub-command "'+dest+'" is required but not given or its value is None.')
 
-        subparser = action._name_parser_map[subcommand]
+        subparser = action._name_parser_map.get(subcommand)
+
+        return subcommand, subparser
+
+
+    @staticmethod
+    def handle_subcommands(parser, cfg, env, defaults, prefix='', fail_no_subcommand=True):
+        """Adds sub-command dest if missing and parses defaults and environment variables."""
+
+        cfg_dict = cfg.__dict__ if isinstance(cfg, Namespace) else cfg
+
+        subcommand, subparser = _ActionSubCommands.get_subcommand(parser, cfg_dict, prefix=prefix, fail_no_subcommand=fail_no_subcommand)
+        if subcommand is None:
+            return
+
+        cfg_keys = set(vars(_dict_to_flat_namespace(cfg)).keys())
+        cfg_keys = cfg_keys.union(set(cfg_dict.keys()))
 
         # merge environment variable values and default values
         subnamespace = None
diff --git a/jsonargparse/core.py b/jsonargparse/core.py
index 89d3790..b78ca77 100644
--- a/jsonargparse/core.py
+++ b/jsonargparse/core.py
@@ -1117,6 +1117,11 @@ class ArgumentParser(_ActionsContainer, argparse.ArgumentParser, LoggerProperty)
                 component.instantiate_class(component, cfg)
                 _ActionLink.apply_instantiation_links(self, cfg, component.dest)
 
+        cfg_dict = cfg.__dict__ if isinstance(cfg, Namespace) else cfg
+        subcommand, subparser = _ActionSubCommands.get_subcommand(self, cfg_dict, fail_no_subcommand=False)
+        if subcommand:
+            cfg[subcommand] = subparser.instantiate_classes(cfg[subcommand], instantiate_groups=instantiate_groups)
+
         return cfg
 
 


Test patch:
diff --git a/jsonargparse_tests/signatures_tests.py b/jsonargparse_tests/signatures_tests.py
index e0506e9..966b7da 100755
--- a/jsonargparse_tests/signatures_tests.py
+++ b/jsonargparse_tests/signatures_tests.py
@@ -644,6 +644,24 @@ class SignaturesTests(unittest.TestCase):
         self.assertIsInstance(cfg['e'], EmptyInitClass)
 
 
+    def test_instantiate_classes_subcommand(self):
+        class Foo:
+            def __init__(self, a: int = 1):
+                self.a = a
+
+        parser = ArgumentParser(parse_as_dict=True)
+        subcommands = parser.add_subcommands()
+        subparser = ArgumentParser()
+        key = "foo"
+        subparser.add_class_arguments(Foo, key)
+        subcommand = "cmd"
+        subcommands.add_subcommand(subcommand, subparser)
+
+        config = parser.parse_args([subcommand])
+        config_init = parser.instantiate_classes(config)
+        self.assertIsInstance(config_init[subcommand][key], Foo)
+
+
     def test_implicit_optional(self):
 
         def func(a1: int = None):