Repository: omni-us/jsonargparse
Pull request #523 (2024-06-12 05:14:47)

Fix: Callable that returns class not using required parameter default from lambda
## Before submitting

- [x] Did you read the [contributing guideline](https://github.com/omni-us/jsonargparse/blob/main/CONTRIBUTING.rst)?
- [n/a] Did you update **the documentation**? (readme and public docstrings)
- [x] Did you write **unit tests** such that there is 100% coverage on related code? (required for bug fixes and new features)
- [x] Did you verify that new and existing **tests pass locally**?
- [x] Did you make sure that all changes preserve **backward compatibility**?
- [x] Did you update **the CHANGELOG**? (not for typos, docs, test updates, or minor internal changes/refactors)


sonarcloud[bot]: ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=523) **Quality Gate failed**  
Failed conditions  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=523) (required ≥ A)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=523)

##   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)


sonarcloud[bot]: ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=523) **Quality Gate failed**  
Failed conditions  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=523) (required ≥ A)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=523)

##   
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarLint](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 100.00%. Comparing base [(`54fde77`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/54fde77da35400539fcc1698b31d86c8eabfca20?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) to head [(`e1f0bdf`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/e1f0bdf6d05d1b3c53349e4cbde21fe3558c7d8e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main      #523   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           21        21           
  Lines         6176      6188   +12     
=========================================
+ Hits          6176      6188   +12     
```

| [Flag](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Coverage Δ | |
|---|---|---|
| [py3.10](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.35% <100.00%> (+0.02%)` | :arrow_up: |
| [py3.10_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.64% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.10_pydantic1](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `49.28% <64.70%> (+<0.01%)` | :arrow_up: |
| [py3.10_pydantic2](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `49.06% <64.70%> (+<0.01%)` | :arrow_up: |
| [py3.10_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.11](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.34% <100.00%> (+0.02%)` | :arrow_up: |
| [py3.11_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.64% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.11_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.12](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.48% <100.00%> (+0.02%)` | :arrow_up: |
| [py3.12_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.64% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.12_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.7](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.86% <100.00%> (+0.02%)` | :arrow_up: |
| [py3.7_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.21% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.7_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.24% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.8](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `88.05% <100.00%> (+0.02%)` | :arrow_up: |
| [py3.8_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.38% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.8_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.40% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.9](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.9_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.9_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.28% <100.00%> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#carryforward-flags-in-the-pull-request-comment) to find out more.


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/omni-us/jsonargparse/pull/523?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).

sonarcloud[bot]: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=523) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [1 New issue](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=523&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=523&resolutions=WONTFIX)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=523&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=523&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=523)



Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index d1dbe87..232e5aa 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -30,6 +30,8 @@ Fixed
   <https://github.com/omni-us/jsonargparse/issues/516>`__).
 - ``--print_config`` failing in some cases (`#517
   <https://github.com/omni-us/jsonargparse/issues/517>`__).
+- Callable that returns class not using required parameter default from lambda
+  (`#523 <https://github.com/omni-us/jsonargparse/pull/523>`__).
 
 
 v4.29.0 (2024-05-24)
diff --git a/jsonargparse/_core.py b/jsonargparse/_core.py
index 5e49960..546c69a 100644
--- a/jsonargparse/_core.py
+++ b/jsonargparse/_core.py
@@ -1326,12 +1326,14 @@ class ArgumentParser(ParserDeprecations, ActionsContainer, ArgumentLinking, argp
         Returns:
             A new object with the merged configuration.
         """
-        cfg = cfg_to.clone()
+        cfg_from = cfg_from.clone()
+        cfg_to = cfg_to.clone()
         with parser_context(parent_parser=self):
-            ActionTypeHint.discard_init_args_on_class_path_change(self, cfg, cfg_from)
-        cfg.update(cfg_from)
-        ActionTypeHint.apply_appends(self, cfg)
-        return cfg
+            ActionTypeHint.discard_init_args_on_class_path_change(self, cfg_to, cfg_from)
+        ActionTypeHint.delete_init_args_required_none(cfg_from, cfg_to)
+        cfg_to.update(cfg_from)
+        ActionTypeHint.apply_appends(self, cfg_to)
+        return cfg_to
 
     def _check_value_key(self, action: argparse.Action, value: Any, key: str, cfg: Optional[Namespace]) -> Any:
         """Checks the value for a given action.
diff --git a/jsonargparse/_typehints.py b/jsonargparse/_typehints.py
index fee45a7..06aa92b 100644
--- a/jsonargparse/_typehints.py
+++ b/jsonargparse/_typehints.py
@@ -396,6 +396,21 @@ class ActionTypeHint(Action):
                     keys = keys[: num + 1] + [k for k in keys[num + 1 :] if not k.startswith(key + ".")]
             num += 1
 
+    @staticmethod
+    def delete_init_args_required_none(cfg_from, cfg_to):
+        for key, val in cfg_from.items(branches=True):
+            if isinstance(val, Namespace) and val.get("class_path") and val.get("init_args"):
+                skip_keys = [
+                    k
+                    for k, v in val.init_args.__dict__.items()
+                    if v is None and cfg_to.get(f"{key}.init_args.{k}") is not None
+                ]
+                if skip_keys:
+                    parser = ActionTypeHint.get_class_parser(val.class_path)
+                    for skip_key in skip_keys:
+                        if skip_key in parser.required_args:
+                            del val.init_args[skip_key]
+
     @staticmethod
     @contextmanager
     def subclass_arg_context(parser):


Test patch:
diff --git a/jsonargparse_tests/test_typehints.py b/jsonargparse_tests/test_typehints.py
index b4d237d..be5161f 100644
--- a/jsonargparse_tests/test_typehints.py
+++ b/jsonargparse_tests/test_typehints.py
@@ -816,9 +816,10 @@ class StepLR:
 
 
 class ReduceLROnPlateau:
-    def __init__(self, optimizer: Optimizer, monitor: str):
+    def __init__(self, optimizer: Optimizer, monitor: str, factor: float = 0.1):
         self.optimizer = optimizer
         self.monitor = monitor
+        self.factor = factor
 
 
 def test_callable_args_return_type_union_of_classes(parser, subtests):
@@ -846,7 +847,7 @@ def test_callable_args_return_type_union_of_classes(parser, subtests):
         }
         cfg = parser.parse_args([f"--scheduler={value}"])
         assert f"{__name__}.ReduceLROnPlateau" == cfg.scheduler.class_path
-        assert Namespace(monitor="loss") == cfg.scheduler.init_args
+        assert Namespace(monitor="loss", factor=0.1) == cfg.scheduler.init_args
         init = parser.instantiate_classes(cfg)
         scheduler = init.scheduler(optimizer)
         assert isinstance(scheduler, ReduceLROnPlateau)
@@ -948,6 +949,40 @@ def test_callable_zero_args_return_type_class(parser):
     assert activation.negative_slope == 0.05
 
 
+class ModelRequiredCallableArg:
+    def __init__(
+        self,
+        scheduler: Callable[[Optimizer], ReduceLROnPlateau] = lambda o: ReduceLROnPlateau(o, monitor="acc"),
+    ):
+        self.scheduler = scheduler
+
+
+def test_callable_return_class_required_arg_from_default(parser):
+    parser.add_argument("--cfg", action="config")
+    parser.add_argument("--model", type=ModelRequiredCallableArg)
+
+    cfg = parser.parse_args(["--model=ModelRequiredCallableArg"])
+    assert cfg.model.init_args.scheduler.class_path == f"{__name__}.ReduceLROnPlateau"
+    assert cfg.model.init_args.scheduler.init_args == Namespace(monitor="acc", factor=0.1)
+
+    config = {
+        "model": {
+            "class_path": f"{__name__}.ModelRequiredCallableArg",
+            "init_args": {
+                "scheduler": {
+                    "class_path": f"{__name__}.ReduceLROnPlateau",
+                    "init_args": {
+                        "factor": 0.5,
+                    },
+                },
+            },
+        }
+    }
+    cfg = parser.parse_args([f"--cfg={config}"])
+    assert cfg.model.init_args.scheduler.class_path == f"{__name__}.ReduceLROnPlateau"
+    assert cfg.model.init_args.scheduler.init_args == Namespace(monitor="acc", factor=0.5)
+
+
 # lazy_instance tests