Repository: omni-us/jsonargparse
Pull request #549 (2024-07-15 02:53:16)

Fix default_config_files making parse fail for subcommands and nested subclass types
## What does this PR do?

Fixes https://lightning.ai/forums/t/problem-lightningcli-with-default-config-files/5963

## Before submitting

- [x] Did you read the [contributing guideline](https://github.com/omni-us/jsonargparse/blob/main/CONTRIBUTING.rst)?
- [n/a] Did you update **the documentation**? (readme and public docstrings)
- [x] Did you write **unit tests** such that there is 100% coverage on related code? (required for bug fixes and new features)
- [x] Did you verify that new and existing **tests pass locally**?
- [x] Did you make sure that all changes preserve **backward compatibility**?
- [x] Did you update **the CHANGELOG**? (not for typos, docs, test updates, or minor internal changes/refactors)


sonarcloud[bot]: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=549) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=549&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=549&resolutions=WONTFIX)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=549&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=549&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=549&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=549)


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 100.00%. Comparing base [(`ed25652`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/ed256524fc8cda35018f8b40319988d19c2f863b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) to head [(`f83befc`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/f83befc7e8c44f0e238a4adae7d613b433c0657e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main      #549   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           22        22           
  Lines         6428      6428           
=========================================
  Hits          6428      6428           
```

| [Flag](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Coverage Δ | |
|---|---|---|
| [py3.10](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `85.69% <100.00%> (ø)` | |
| [py3.10_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.69% <100.00%> (ø)` | |
| [py3.10_pydantic1](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `48.52% <100.00%> (+0.09%)` | :arrow_up: |
| [py3.10_pydantic2](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `48.30% <100.00%> (+0.09%)` | :arrow_up: |
| [py3.10_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.70% <100.00%> (ø)` | |
| [py3.11](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `85.68% <100.00%> (ø)` | |
| [py3.11_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.69% <100.00%> (ø)` | |
| [py3.11_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.70% <100.00%> (ø)` | |
| [py3.12](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `85.82% <100.00%> (ø)` | |
| [py3.12_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.69% <100.00%> (ø)` | |
| [py3.12_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.70% <100.00%> (ø)` | |
| [py3.7](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `86.14% <100.00%> (ø)` | |
| [py3.7_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.19% <100.00%> (ø)` | |
| [py3.7_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.24% <100.00%> (ø)` | |
| [py3.8](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `86.33% <100.00%> (ø)` | |
| [py3.8_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.36% <100.00%> (ø)` | |
| [py3.8_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.39% <100.00%> (ø)` | |
| [py3.9](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `86.22% <100.00%> (ø)` | |
| [py3.9_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.28% <100.00%> (ø)` | |
| [py3.9_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.29% <100.00%> (ø)` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/omni-us/jsonargparse/pull/549?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index e38a17e..7b47285 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -31,6 +31,9 @@ Fixed
 - ``dump`` failing when a link target requires serialization and
   ``skip_link_targets=False`` (`#542
   <https://github.com/omni-us/jsonargparse/pull/542>`__).
+- ``default_config_files`` making parse fail for subcommands and nested subclass
+  types (`lightning-forums#5963
+  <https://lightning.ai/forums/t/problem-lightningcli-with-default-config-files/5963>`__).
 - Import path of inherited classmethod not resolving correctly (`lightning#19863
   comment
   <https://github.com/Lightning-AI/pytorch-lightning/discussions/19863#discussioncomment-10010226>`__).
diff --git a/jsonargparse/_actions.py b/jsonargparse/_actions.py
index e681c2d..e2bb047 100644
--- a/jsonargparse/_actions.py
+++ b/jsonargparse/_actions.py
@@ -580,12 +580,12 @@ parse_kwargs: ContextVar = ContextVar("parse_kwargs", default={})
 @contextmanager
 def parent_parsers_context(key, parser):
     prev = parent_parsers.get()
-    curr = prev + [(key, parser)]
-    t = parent_parsers.set(curr)
+    curr = [] if parser is None else prev + [(key, parser)]
+    token = parent_parsers.set(curr)
     try:
         yield
     finally:
-        parent_parsers.reset(t)
+        parent_parsers.reset(token)
 
 
 class _ActionSubCommands(_SubParsersAction):
diff --git a/jsonargparse/_typehints.py b/jsonargparse/_typehints.py
index 30c3ca6..19aeb9e 100644
--- a/jsonargparse/_typehints.py
+++ b/jsonargparse/_typehints.py
@@ -39,6 +39,7 @@ from ._actions import (
     _find_action,
     _find_parent_action,
     _is_action_value_list,
+    parent_parsers_context,
     remove_actions,
 )
 from ._common import (
@@ -453,7 +454,7 @@ class ActionTypeHint(Action):
                 or (isinstance(v, dict) and any(is_subclass_spec(e) for e in v.values()))
             )
 
-        with ActionTypeHint.sub_defaults_context():
+        with ActionTypeHint.sub_defaults_context(), parent_parsers_context(None, None):
             parser._apply_actions(cfg, skip_fn=skip_sub_defaults_apply)
 
     @staticmethod


Test patch:
diff --git a/jsonargparse_tests/test_subcommands.py b/jsonargparse_tests/test_subcommands.py
index 117b49f..a181bbb 100644
--- a/jsonargparse_tests/test_subcommands.py
+++ b/jsonargparse_tests/test_subcommands.py
@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import json
 import os
 import warnings
 from pathlib import Path
@@ -255,6 +256,46 @@ def test_subcommand_required_arg_in_default_config(parser, subparser, tmp_cwd):
     assert strip_meta(cfg) == Namespace(output="test", prepare=Namespace(media="test"), subcommand="prepare")
 
 
+class SubModel:
+    def __init__(self, p1: int, p2: str = "-"):
+        pass
+
+
+class Model:
+    def __init__(self, submodel: SubModel):
+        self.submodel = submodel
+
+
+def test_subcommand_default_config_add_subdefaults(parser, subparser, tmp_cwd):
+    config = {
+        "fit": {
+            "model": {
+                "class_path": f"{__name__}.Model",
+                "init_args": {
+                    "submodel": {
+                        "class_path": f"{__name__}.SubModel",
+                        "init_args": {"p1": 1},
+                    }
+                },
+            }
+        }
+    }
+    Path("config.json").write_text(json.dumps(config))
+    parser.default_config_files = ["config.json"]
+    subcommands = parser.add_subcommands()
+    subparser = ArgumentParser()
+    subparser.add_argument("--model", type=Model, required=True)
+    subcommands.add_subcommand("fit", subparser)
+    cfg = parser.parse_args([])
+    assert cfg.fit.model.class_path == f"{__name__}.Model"
+    assert list(cfg.fit.model.init_args.__dict__.keys()) == ["submodel"]
+    assert cfg.fit.model.init_args.submodel.class_path == f"{__name__}.SubModel"
+    assert cfg.fit.model.init_args.submodel.init_args == Namespace(p1=1, p2="-")
+    init = parser.instantiate_classes(cfg)
+    assert isinstance(init.fit.model, Model)
+    assert isinstance(init.fit.model.submodel, SubModel)
+
+
 def test_subsubcommands_parse_args(subtests):
     parser_s1_a = ArgumentParser(exit_on_error=False)
     parser_s1_a.add_argument("--os1a", default="os1a_def")