Repository: omni-us/jsonargparse

Issue:
``discard_init_args_on_class_path_change`` does not handle nested ``dict`` subclass specs in some contexts
## 🐛 Bug report
Firstly, thanks again for all your work maintaining this remarkably valuable package!

While a ``subclass_spec`` can be either a ``dict`` or ``Namespace`` 
https://github.com/omni-us/jsonargparse/blob/ac790ba09bcc920cab1acf712422152b931d337a/jsonargparse/typehints.py#L790-L795
when entering ``ActionTypeHint.discard_init_args_on_class_path_change``:
https://github.com/omni-us/jsonargparse/blob/ac790ba09bcc920cab1acf712422152b931d337a/jsonargparse/typehints.py#L295-L310


``jsonargparse.typehints.discard_init_args_on_class_path_change`` below currently assumes any previous value for a nested subclass will already have been cast as a ``Namespace``: 
https://github.com/omni-us/jsonargparse/blob/ac790ba09bcc920cab1acf712422152b931d337a/jsonargparse/typehints.py#L934-L942

with ``L941`` resulting in:
```
argparse.ArgumentError: Parser key "some_class":
  Problem with given class_path '__main__.CustomModule':
    - 'dict' object has no attribute 'init_args'
```
Right now, nested generic ``dict`` configurations like the repro case I include below don't trigger the ``subclass_spec_as_namespace`` ``Namespace`` wrapping that ``jsonargparse.typehints.discard_init_args_on_class_path_change`` requires when removing overridden subclass spec configuration in this context.

With the below minor patch to ``jsonargparse.typehints.discard_init_args_on_class_path_change``, this use case can be accommodated (I haven't had a chance to run the full jsongarparse test suite but can't immediately think of any problem with this additional check/transformation):

```python
def discard_init_args_on_class_path_change(parser_or_action, prev_val, value):
    if prev_val and 'init_args' in prev_val and prev_val['class_path'] != value['class_path']:
        parser = parser_or_action
        if isinstance(parser_or_action, ActionTypeHint):
            sub_add_kwargs = getattr(parser_or_action, 'sub_add_kwargs', {})
            parser = ActionTypeHint.get_class_parser(value['class_path'], sub_add_kwargs)
        del_args = {}
        if isinstance(prev_val, dict):
            prev_val = subclass_spec_as_namespace(prev_val)
        for key, val in list(prev_val.init_args.__dict__.items()):
            action = _find_action(parser, key)
```
### To reproduce
```python
from typing import Any, Dict

import yaml
import lightning.pytorch as pl
from jsonargparse import ArgumentParser


override_config = yaml.safe_dump(
    {
      "class_path": "CustomModule",
      "init_args": {
        "some_custom_init": {
          "class_path": "torch.optim.lr_scheduler.StepLR",
          "init_args": {"step_size": 1, "gamma": 0.7},
        }
      }
    }
  )

default_config = yaml.safe_dump(
  {
    "class_path": "CustomModule",
    "init_args": {
      "some_custom_init": {
        "class_path": "torch.optim.lr_scheduler.CosineAnnealingWarmRestarts",
        "init_args": {"T_0": 1, "T_mult": 2, "eta_min": 1.0e-07}
      }
    }
  }
)


class CustomModule(pl.LightningModule):
    def __init__(self, some_custom_init: Dict[str, Any],):
        super().__init__()


parser = ArgumentParser()
parser.add_argument("--some_class", type=pl.LightningModule)
result = parser.parse_args(["--some_class", default_config, "--some_class", override_config])
print(f"expected_result= {result}")
```
Current behavior:
```
argparse.ArgumentError: Parser key "some_class":
  Problem with given class_path '__main__.CustomModule':
    - 'dict' object has no attribute 'init_args'
```

Expected result, generated with additional handling discussed above:
```
expected_result= Namespace(some_class=Namespace(class_path='__main__.CustomModule', init_args=Namespace(some_custom_init={'class_path': 'torch.optim.lr_scheduler.StepLR', 'init_args': {'gamma': 0.7, 'step_size': 1}})))
```
### Environment

- jsonargparse version (e.g., 4.8.0): 4.20.0
- Python version (e.g., 3.9): 3.10
- How jsonargparse was installed (e.g. `pip install jsonargparse[all]`): pip install "jsonargparse[signatures]==4.20.0"
- OS (e.g., Linux): Ubuntu 20.04


Discussion:
sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=248)

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL)

[![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248) No Coverage information  
[![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248&metric=new_duplicated_lines_density&view=list)


sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=248)

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL)

[![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248) No Coverage information  
[![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248&metric=new_duplicated_lines_density&view=list)


sonarcloud[bot]: Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=248)

[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=BUG)  
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=VULNERABILITY)  
[![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=SECURITY_HOTSPOT)  
[![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=248&resolved=false&types=CODE_SMELL)

[![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248) No Coverage information  
[![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=248&metric=new_duplicated_lines_density&view=list)


codecov-commenter: # [Codecov](https://codecov.io/gh/omni-us/jsonargparse/pull/248?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) Report
Patch coverage: **`100.00`**% and no project coverage change
> Comparison is base [(`14b4842`)](https://codecov.io/gh/omni-us/jsonargparse/commit/14b484266115f79040b6a5c555e201daddf54f12?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) 100.00% compared to head [(`cb2d122`)](https://codecov.io/gh/omni-us/jsonargparse/pull/248?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) 100.00%.

:mega: This organization is not using Codecov’s [GitHub App Integration](https://github.com/apps/codecov). We recommend you install it so Codecov can continue to function properly for your repositories. [Learn more](https://about.codecov.io/blog/codecov-is-updating-its-github-integration/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us)

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##            master      #248   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           20        20           
  Lines         5189      5190    +1     
=========================================
+ Hits          5189      5190    +1     
```

| Flag | Coverage Δ | |
|---|---|---|
| py3.10 | `87.93% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.10_all | `99.88% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.6 | `87.57% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.6_all | `99.63% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.7 | `87.59% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.7_all | `99.80% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.8 | `87.91% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.8_all | `99.84% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.9 | `87.91% <100.00%> (+<0.01%)` | :arrow_up: |
| py3.9_all | `99.86% <100.00%> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/omni-us/jsonargparse/pull/248?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Coverage Δ | |
|---|---|---|
| [jsonargparse/typehints.py](https://codecov.io/gh/omni-us/jsonargparse/pull/248?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#diff-anNvbmFyZ3BhcnNlL3R5cGVoaW50cy5weQ==) | `100.00% <100.00%> (ø)` | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/omni-us/jsonargparse/pull/248?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 4faf1b4..1aafaf5 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -12,11 +12,14 @@ The semantic versioning only considers the public API as described in
 paths are considered internals and can change in minor and patch releases.
 
 
-v4.20.1 (2023-02-??)
+v4.20.1 (2023-03-??)
 --------------------
 
 Fixed
 ^^^^^
+- Allow ``discard_init_args_on_class_path_change`` to handle more nested contexts `#247
+  <https://github.com/omni-us/jsonargparse/issues/247>`__.
+
 - Dump not working for partial callable with return instance
   `pytorch-lightning#15340 (comment)
   <https://github.com/Lightning-AI/lightning/issues/15340#issuecomment-1439203008>`__.
diff --git a/jsonargparse/typehints.py b/jsonargparse/typehints.py
index fc11db1..7f67ae1 100644
--- a/jsonargparse/typehints.py
+++ b/jsonargparse/typehints.py
@@ -939,6 +939,7 @@ def discard_init_args_on_class_path_change(parser_or_action, prev_val, value):
             sub_add_kwargs = getattr(parser_or_action, 'sub_add_kwargs', {})
             parser = ActionTypeHint.get_class_parser(value['class_path'], sub_add_kwargs)
         del_args = {}
+        prev_val = subclass_spec_as_namespace(prev_val)
         for key, val in list(prev_val.init_args.__dict__.items()):
             action = _find_action(parser, key)
             if action:


Test patch:
diff --git a/jsonargparse_tests/test_signatures.py b/jsonargparse_tests/test_signatures.py
index 463be12..3c85032 100755
--- a/jsonargparse_tests/test_signatures.py
+++ b/jsonargparse_tests/test_signatures.py
@@ -1427,6 +1427,46 @@ class SignaturesConfigTests(TempDirTestCase):
                     self.assertIsInstance(init.main.sub, Sub2)
                     self.assertTrue(any("discarding init_args: {'s1': 'x'}" in o for o in log.output))
 
+    def test_config_nested_dict_discard_init_args(self):
+        class Base:
+            def __init__(self, b: float = 0.5):
+                pass
+
+        class Sub1(Base):
+            def __init__(self, s1: int = 3, **kwargs):
+                super().__init__(**kwargs)
+
+        class Sub2(Base):
+            def __init__(self, s2: int = 4, **kwargs):
+                super().__init__(**kwargs)
+
+        class Main:
+            def __init__(self, sub: Optional[Dict] = None) -> None:
+                self.sub = sub
+
+        configs, subconfigs, config_paths = {}, {}, {}
+        with mock_module(Base, Sub1, Sub2, Main) as module:
+            parser = ArgumentParser(exit_on_error=False, logger={'level': 'DEBUG'})
+            parser.add_argument('--config', action=ActionConfigFile)
+            parser.add_subclass_arguments(Main, 'main')
+            parser.set_defaults(main=lazy_instance(Main))
+            for c in [1, 2]:
+                subconfigs[c] = {
+                    'sub': {
+                        'class_path': f'{module}.Sub{c}',
+                        'init_args': {f's{c}': c},
+                    }
+                }
+                configs[c] = {'main': {'class_path': f'{module}.Main','init_args': subconfigs[c],}}
+                config_paths[c] = Path(f'config{c}.yaml')
+                config_paths[c].write_text(yaml.safe_dump(configs[c]))
+
+            with self.assertLogs(logger=parser.logger, level='DEBUG') as log:
+                cfg = parser.parse_args([f'--config={config_paths[1]}', f'--config={config_paths[2]}'])
+            init = parser.instantiate_classes(cfg)
+            self.assertIsInstance(init.main, Main)
+            self.assertTrue(init.main.sub['init_args']['s2'], 2)
+            self.assertTrue(any("discarding init_args: {'s1': 1}" in o for o in log.output))
 
 @dataclasses.dataclass(frozen=True)
 class MyDataClassA: