Repository: omni-us/jsonargparse

Issue:
`lazy_instance` no longer accepts arguments that go into kwargs
<!-- If you like this project, please ⭐ star it https://github.com/omni-us/jsonargparse/ -->

## 🐛 Bug report

The last release of jsonargparse 4.27.6 made our CI fail. We have a test that does the following:


### To reproduce

```
pip install lightning
```

```py
import torch
from jsonargparse import lazy_instance

from lightning.pytorch import LightningModule
from lightning.pytorch.cli import LightningCLI
from lightning.pytorch.strategies import DDPStrategy


class BoringModel(LightningModule):
    def __init__(self):
        super().__init__()
        self.layer = torch.nn.Linear(32, 2)

    def training_step(self, batch, batch_idx):
        loss = self.layer(batch).sum()
        self.log("train_loss", loss)
        return {"loss": loss}

    def configure_optimizers(self):
        return torch.optim.SGD(self.layer.parameters(), lr=0.1)


def run():
    strategy_default = lazy_instance(DDPStrategy, find_unused_parameters=True)
    LightningCLI(
        BoringModel,
        run=False,
        trainer_defaults={"strategy": strategy_default},
    )


if __name__ == "__main__":
    run()

```
Error:
```
Traceback (most recent call last):
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_core.py", line 1083, in check_config
    check_values(cfg)
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_core.py", line 1077, in check_values
    raise NSKeyError(f'No action for key "{key}" to check its value.')
jsonargparse._namespace.NSKeyError: No action for key "find_unused_parameters" to check its value.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_core.py", line 441, in parse_object
    parsed_cfg = self._parse_common(
                 ^^^^^^^^^^^^^^^^^^^
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_core.py", line 319, in _parse_common
    self.check_config(cfg, skip_required=skip_required)
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_core.py", line 1089, in check_config
    raise type(ex)(message) from ex
jsonargparse._namespace.NSKeyError: Validation failed: No action for key "find_unused_parameters" to check its value.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_typehints.py", line 1277, in check_lazy_kwargs
    parser.parse_object(lazy_kwargs)
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_deprecated.py", line 124, in patched_parse
    cfg = parse_method(*args, _skip_check=_skip_check, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_core.py", line 451, in parse_object
    self.error(str(ex), ex)
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_core.py", line 1002, in error
    raise argument_error(message) from ex
argparse.ArgumentError: Validation failed: No action for key "find_unused_parameters" to check its value.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/adrian/repositories/lightning/examples/pytorch/bug_report/bug_report_model.py", line 33, in <module>
    run()
  File "/Users/adrian/repositories/lightning/examples/pytorch/bug_report/bug_report_model.py", line 24, in run
    strategy_default = lazy_instance(DDPStrategy, find_unused_parameters=True)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_typehints.py", line 1349, in lazy_instance
    return lazy_init_class(class_type, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_typehints.py", line 1285, in __init__
    check_lazy_kwargs(class_type, lazy_kwargs)
  File "/Users/adrian/miniconda3/envs/lightning/lib/python3.11/site-packages/jsonargparse/_typehints.py", line 1279, in check_lazy_kwargs
    raise ValueError(str(ex)) from ex
ValueError: Validation failed: No action for key "find_unused_parameters" to check its value.
```

### Expected behavior

I'm not sure, maybe it's intentionally no longer supported, but it worked in 4.27.5. 
The `find_unused_parameters` in the above usage is not explicitly listed in the DDPStrategy signature, but is supposed to be eaten by the `**kwargs`:
https://github.com/Lightning-AI/pytorch-lightning/blob/5232cfdc77df1df2fa712c4ec7781266ae572daa/src/lightning/pytorch/strategies/ddp.py#L85

Do you suggest I change the test in Lightning?

### Environment

<!-- Fill in the list below. -->

- jsonargparse version (e.g., 4.8.0): 4.27.6
- Python version (e.g., 3.9): 3.10
- How jsonargparse was installed (e.g. `pip install jsonargparse[all]`): pip install jsonargparse[signatures]
- OS (e.g., Linux): Linux


Discussion:
sonarcloud[bot]: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=476) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=476&resolved=false&inNewCodePeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=476&metric=new_accepted_issues&view=list)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=476&resolved=false&inNewCodePeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=476&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=476)


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 99.80%. Comparing base [(`bb83fdc`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/bb83fdc76546743ad0e4e7012715febd965f34ac?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) to head [(`e654d11`)](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##              main     #476      +/-   ##
===========================================
- Coverage   100.00%   99.80%   -0.20%     
===========================================
  Files           21       21              
  Lines         6008     6008              
===========================================
- Hits          6008     5996      -12     
- Misses           0       12      +12     
```

| [Flag](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Coverage Δ | |
|---|---|---|
| [py3.10](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.10_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.10_pydantic1](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.10_pydantic2](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.10_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.11](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.11_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.11_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.12](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.12_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.73% <100.00%> (ø)` | |
| [py3.12_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.73% <100.00%> (ø)` | |
| [py3.7](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `88.96% <100.00%> (ø)` | |
| [py3.7_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.49% <100.00%> (ø)` | |
| [py3.7_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.47% <100.00%> (ø)` | |
| [py3.8](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `89.11% <100.00%> (ø)` | |
| [py3.8_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.51% <100.00%> (ø)` | |
| [py3.8_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.50% <100.00%> (ø)` | |
| [py3.9](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `88.96% <100.00%> (ø)` | |
| [py3.9_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.9_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#carryforward-flags-in-the-pull-request-comment) to find out more.


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/omni-us/jsonargparse/pull/476?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index bdded86..fd6d23f 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -12,6 +12,18 @@ The semantic versioning only considers the public API as described in
 paths are considered internals and can change in minor and patch releases.
 
 
+v4.27.7 (2024-03-??)
+--------------------
+
+Fixed
+^^^^^
+- Regression from `14456c2
+  <https://github.com/omni-us/jsonargparse/commit/14456c21ff7a11ba420f010d2b21bcfdb14977a2>`__
+  that prevented ``**kwargs`` parameter resolving when an ``Optional[Callable]``
+  type is used (`#473
+  <https://github.com/omni-us/jsonargparse/issues/473>`__).
+
+
 v4.27.6 (2024-03-15)
 --------------------
 
diff --git a/jsonargparse/_typehints.py b/jsonargparse/_typehints.py
index 67ec5eb..6dab180 100644
--- a/jsonargparse/_typehints.py
+++ b/jsonargparse/_typehints.py
@@ -955,7 +955,11 @@ def get_callable_return_type(typehint):
 
 def get_subclass_types(typehint, callable_return=True):
     subclass_types = None
-    if callable_return and ActionTypeHint.is_callable_typehint(typehint, all_subtypes=False) and typehint.__args__:
+    if (
+        callable_return
+        and ActionTypeHint.is_callable_typehint(typehint, all_subtypes=False)
+        and getattr(typehint, "__args__", None)
+    ):
         typehint = get_optional_arg(typehint)
         typehint = typehint.__args__[-1]
     if ActionTypeHint.is_subclass_typehint(typehint, all_subtypes=False):


Test patch:
diff --git a/jsonargparse_tests/test_parameter_resolvers.py b/jsonargparse_tests/test_parameter_resolvers.py
index 22b89e9..9cc6f80 100644
--- a/jsonargparse_tests/test_parameter_resolvers.py
+++ b/jsonargparse_tests/test_parameter_resolvers.py
@@ -5,7 +5,7 @@ import inspect
 import xml.dom
 from calendar import Calendar
 from random import shuffle
-from typing import Any, Callable, Dict, List, Union
+from typing import Any, Callable, Dict, List, Optional, Union
 from unittest.mock import patch
 
 import pytest
@@ -541,6 +541,14 @@ def conditional_calls(**kwargs):
         cond_3(**kwargs)
 
 
+def function_optional_callable(p1: Optional[Callable] = None, **kw):
+    """
+    Args:
+        p1: help for p1
+    """
+    function_no_args_no_kwargs(**kw)
+
+
 def assert_params(params, expected, origins={}):
     assert expected == [p.name for p in params]
     docs = [f"help for {p.name}" for p in params] if docstring_parser_support else [None] * len(params)
@@ -864,6 +872,10 @@ def test_conditional_calls_kwargs():
         assert get_params(conditional_calls) == []
 
 
+def test_get_params_optional_callable():
+    assert_params(get_params(function_optional_callable), ["p1", "pk1", "k2"])
+
+
 # unsupported cases