Repository: omni-us/jsonargparse
Pull request #506 (2024-05-22 05:38:27)

Fix nested values not validated when type not subclass and nested keys for subclass
## What does this PR do?

Resolves the issue mentioned in https://github.com/omni-us/jsonargparse/issues/503#issuecomment-2119724341

## Before submitting

- [x] Did you read the [contributing guideline](https://github.com/omni-us/jsonargparse/blob/main/CONTRIBUTING.rst)?
- [n/a] Did you update **the documentation**? (readme and public docstrings)
- [x] Did you write **unit tests** such that there is 100% coverage on related code? (required for bug fixes and new features)
- [x] Did you verify that new and existing **tests pass locally**?
- [x] Did you make sure that all changes preserve **backward compatibility**?
- [x] Did you update **the CHANGELOG**? (not for typos, docs, test updates, or minor internal changes/refactors)


sonarcloud[bot]: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=506) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=506&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=506&resolutions=WONTFIX)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=506&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=506&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=506)


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) Report
Attention: Patch coverage is `94.73684%` with `1 lines` in your changes are missing coverage. Please review.
> Project coverage is 99.98%. Comparing base [(`797f004`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/797f004c48bb077c20f30a7e682373e864559c5c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) to head [(`18dcc2f`)](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).

| [Files](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Patch % | Lines |
|---|---|---|
| [jsonargparse/\_core.py](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506?src=pr&el=tree&filepath=jsonargparse%2F_core.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#diff-anNvbmFyZ3BhcnNlL19jb3JlLnB5) | 94.73% | [1 Missing :warning: ](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) |

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##              main     #506      +/-   ##
===========================================
- Coverage   100.00%   99.98%   -0.02%     
===========================================
  Files           21       21              
  Lines         6139     6149      +10     
===========================================
+ Hits          6139     6148       +9     
- Misses           0        1       +1     
```

| [Flag](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Coverage Δ | |
|---|---|---|
| [py3.10](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.28% <94.73%> (+<0.01%)` | :arrow_up: |
| [py3.10_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.63% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.10_pydantic1](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `49.09% <78.94%> (+0.01%)` | :arrow_up: |
| [py3.10_pydantic2](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `48.86% <78.94%> (+0.01%)` | :arrow_up: |
| [py3.10_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.11](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.26% <94.73%> (+<0.01%)` | :arrow_up: |
| [py3.11_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.63% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.11_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.12](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.41% <94.73%> (+<0.01%)` | :arrow_up: |
| [py3.12_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.63% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.12_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.7](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.82% <94.73%> (+<0.01%)` | :arrow_up: |
| [py3.7_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.24% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.7_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.24% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.8](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.99% <94.73%> (+<0.01%)` | :arrow_up: |
| [py3.8_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.39% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.8_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.39% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.9](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.83% <94.73%> (+<0.01%)` | :arrow_up: |
| [py3.9_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.26% <94.73%> (-0.02%)` | :arrow_down: |
| [py3.9_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.28% <94.73%> (-0.02%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#carryforward-flags-in-the-pull-request-comment) to find out more.


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/omni-us/jsonargparse/pull/506?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).

sonarcloud[bot]: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=506) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=506&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=506&resolutions=WONTFIX)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=506&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=506&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=506)



Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 8e61671..ef6e802 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -32,7 +32,15 @@ Fixed
 - ``format_usage()`` not working (`#501
   <https://github.com/omni-us/jsonargparse/issues/501>`__).
 - Not able to modify init args for callable with class return and default class
-  (`#5?? <https://github.com/omni-us/jsonargparse/pull/5??>`__).
+  (`#504 <https://github.com/omni-us/jsonargparse/pull/504>`__).
+- Nested values not validated when type not subclass and nested keys for
+  subclass (`#503 comment
+  <https://github.com/omni-us/jsonargparse/issues/503#issuecomment-2119724341>`__).
+
+Changed
+^^^^^^^
+- When parsing fails due to unexpected key, now there are specific error
+  messages for the cases of groups and subcommands.
 
 
 v4.28.0 (2024-04-17)
diff --git a/jsonargparse/_core.py b/jsonargparse/_core.py
index 0e845b4..4a00234 100644
--- a/jsonargparse/_core.py
+++ b/jsonargparse/_core.py
@@ -29,7 +29,6 @@ from ._actions import (
     _ActionSubCommands,
     _find_action,
     _find_action_and_subcommand,
-    _find_parent_action,
     _find_parent_action_and_subcommand,
     _is_action_value_list,
     _is_branch_key,
@@ -67,6 +66,7 @@ from ._namespace import (
     recreate_branches,
     split_key,
     split_key_leaf,
+    split_key_root,
     strip_meta,
 )
 from ._optionals import (
@@ -1050,20 +1050,19 @@ class ArgumentParser(ParserDeprecations, ActionsContainer, ArgumentLinking, argp
                 check_required(cfg.get(subcommand), subparser, subcommand + ".")
 
         def check_values(cfg):
-            for key in cfg.get_sorted_keys():
-                val = cfg[key]
-                action = _find_action(self, key)
+            sorted_keys = {k: _find_action(self, k) for k in cfg.get_sorted_keys()}
+            for key, action in sorted_keys.items():
+                parent_action = None
                 if action is None:
-                    if (
-                        _is_branch_key(self, key)
-                        or key.endswith(".class_path")
-                        or key.endswith(".dict_kwargs")
-                        or ".init_args" in key
-                    ):
-                        continue
-                    action = _find_parent_action(self, key, exclude=_ActionConfigLoad)
-                    if action and not ActionTypeHint.is_subclass_typehint(action):
+                    if _is_branch_key(self, key):
                         continue
+                    parent_action, subcommand = _find_parent_action_and_subcommand(self, key, exclude=_ActionConfigLoad)
+                    if parent_action:
+                        parent_key = subcommand + "." + parent_action.dest if subcommand else parent_action.dest
+                        if key.startswith(parent_key + ".") and sorted_keys.get(parent_key) is parent_action:
+                            # only check action once with entire value
+                            continue
+                val = cfg[key]
                 if action is not None:
                     if val is None and skip_none:
                         continue
@@ -1074,10 +1073,15 @@ class ArgumentParser(ParserDeprecations, ActionsContainer, ArgumentLinking, argp
                             val == {} and ActionTypeHint.is_subclass_typehint(action) and key not in self.required_args
                         ):
                             raise ex
-                elif key in self.groups and hasattr(self.groups[key], "instantiate_class"):
-                    raise TypeError(f"Class group {key!r} got an unexpected value: {val}.")
                 else:
-                    raise NSKeyError(f'No action for key "{key}" to check its value.')
+                    if isinstance(parent_action, _ActionSubCommands) and "." in key:
+                        subcommand, subkey = split_key_root(key)
+                        raise NSKeyError(f"Subcommand '{subcommand}' does not accept nested key '{subkey}'")
+                    group_key = next((g for g in self.groups if key.startswith(g + ".")), None)
+                    if group_key:
+                        subkey = key[len(group_key) + 1 :]
+                        raise NSKeyError(f"Group '{group_key}' does not accept nested key '{subkey}'")
+                    raise NSKeyError(f"Key '{key}' is not expected")
 
         try:
             if not skip_required and not lenient_check.get():


Test patch:
diff --git a/jsonargparse_tests/test_dataclass_like.py b/jsonargparse_tests/test_dataclass_like.py
index 765d1bf..375d71a 100644
--- a/jsonargparse_tests/test_dataclass_like.py
+++ b/jsonargparse_tests/test_dataclass_like.py
@@ -196,6 +196,16 @@ def test_add_argument_dataclass_type(parser):
     assert isinstance(init.b.b2, DataClassA)
 
 
+def test_add_argument_dataclass_unexpected_keys(parser):
+    parser.add_argument("--b", type=DataClassB)
+    invalid = {
+        "class_path": f"{__name__}.DataClassB",
+    }
+    with pytest.raises(ArgumentError) as ctx:
+        parser.parse_args([f"--b={invalid}"])
+    ctx.match("Group 'b' does not accept nested key 'class_path'")
+
+
 @dataclasses.dataclass
 class DataRequiredAttr:
     a1: str
diff --git a/jsonargparse_tests/test_signatures.py b/jsonargparse_tests/test_signatures.py
index 6b051be..54b1346 100644
--- a/jsonargparse_tests/test_signatures.py
+++ b/jsonargparse_tests/test_signatures.py
@@ -193,7 +193,7 @@ def test_add_class_without_parameters(parser):
     config = {"no_params": {"class_path": f"{__name__}.NoParams"}}
     with pytest.raises(ArgumentError) as ctx:
         parser.parse_args([f"--cfg={config}"])
-    ctx.match("'no_params' got an unexpected value")
+    ctx.match("Group 'no_params' does not accept nested key 'class_path'")
 
 
 class NestedWithParams:
diff --git a/jsonargparse_tests/test_subclasses.py b/jsonargparse_tests/test_subclasses.py
index fb2275b..b580712 100644
--- a/jsonargparse_tests/test_subclasses.py
+++ b/jsonargparse_tests/test_subclasses.py
@@ -550,7 +550,7 @@ def test_subclass_class_name_then_invalid_init_args(parser):
     parser.add_argument("--op", type=Union[Calendar, GzipFile])
     with pytest.raises(ArgumentError) as ctx:
         parser.parse_args(["--op=TextCalendar", "--op=GzipFile", "--op.firstweekday=2"])
-    ctx.match('No action for key "firstweekday"')
+    ctx.match("Key 'firstweekday' is not expected")
 
 
 # dict parameter tests
@@ -1360,7 +1360,7 @@ def test_subclass_print_config(parser):
     assert obtained == {"a1": {"class_path": "calendar.Calendar", "init_args": {"firstweekday": 0}}, "a2": 7}
 
     err = get_parse_args_stderr(parser, ["--g.a1=calendar.Calendar", "--g.a1.invalid=1", "--print_config"])
-    assert 'No action for key "invalid"' in err
+    assert "Key 'invalid' is not expected" in err
 
 
 class PrintConfigRequired:
@@ -1438,7 +1438,7 @@ def test_subclass_error_unexpected_init_arg(parser):
     init_args = '"init_args": {"unexpected_arg": True}'
     with pytest.raises(ArgumentError) as ctx:
         parser.parse_args(["--op={" + class_path + ", " + init_args + "}"])
-    ctx.match('No action for key "unexpected_arg"')
+    ctx.match("Key 'unexpected_arg' is not expected")
 
 
 def test_subclass_invalid_class_path_value(parser):
@@ -1507,7 +1507,7 @@ def test_subclass_implicit_class_path(parser):
     assert cfg.implicit.init_args == Namespace(a=3, b="")
     with pytest.raises(ArgumentError) as ctx:
         parser.parse_args(['--implicit={"c": null}'])
-    ctx.match('No action for key "c" to check its value')
+    ctx.match("Key 'c' is not expected")
 
 
 # error messages tests
diff --git a/jsonargparse_tests/test_subcommands.py b/jsonargparse_tests/test_subcommands.py
index 0d147e9..306996f 100644
--- a/jsonargparse_tests/test_subcommands.py
+++ b/jsonargparse_tests/test_subcommands.py
@@ -121,7 +121,7 @@ def test_subcommands_parse_string_implicit_subcommand(subcommands_parser):
     assert cfg["a"] == {"ap1": "ap1_cfg", "ao1": "ao1_def"}
     with pytest.raises(ArgumentError) as ctx:
         subcommands_parser.parse_string('{"a": {"ap1": "ap1_cfg", "unk": "unk_cfg"}}')
-    ctx.match('No action for key "unk"')
+    ctx.match("Subcommand 'a' does not accept nested key 'unk'")
 
 
 def test_subcommands_parse_string_first_implicit_subcommand(subcommands_parser):