Repository: omni-us/jsonargparse
Pull request #537 (2024-06-25 03:50:55)

Fix: --print_shtab crashing on failure to get signature parameters from one class
## What does this PR do?

Fixes https://github.com/Lightning-AI/pytorch-lightning/discussions/10858#discussioncomment-9846252

## Before submitting

- [x] Did you read the [contributing guideline](https://github.com/omni-us/jsonargparse/blob/main/CONTRIBUTING.rst)?
- [n/a] Did you update **the documentation**? (readme and public docstrings)
- [x] Did you write **unit tests** such that there is 100% coverage on related code? (required for bug fixes and new features)
- [x] Did you verify that new and existing **tests pass locally**?
- [x] Did you make sure that all changes preserve **backward compatibility**?
- [x] Did you update **the CHANGELOG**? (not for typos, docs, test updates, or minor internal changes/refactors)


sonarcloud[bot]: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=537) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=537&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=537&resolutions=WONTFIX)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=537&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=537&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=537&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=537)


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 100.00%. Comparing base [(`82273f9`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/82273f9867ded25214d7719584f7c2bf011dc7da?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) to head [(`090685d`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/090685d92692e0023079580375fc61576628551f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main      #537   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           22        22           
  Lines         6377      6381    +4     
=========================================
+ Hits          6377      6381    +4     
```

| [Flag](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Coverage Δ | |
|---|---|---|
| [py3.10](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `85.59% <30.00%> (-0.06%)` | :arrow_down: |
| [py3.10_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.68% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.10_pydantic1](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `48.53% <30.00%> (-0.04%)` | :arrow_down: |
| [py3.10_pydantic2](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `48.31% <30.00%> (-0.04%)` | :arrow_down: |
| [py3.10_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.69% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.11](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `85.57% <30.00%> (-0.06%)` | :arrow_down: |
| [py3.11_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.68% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.11_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.69% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.12](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `85.71% <30.00%> (-0.06%)` | :arrow_down: |
| [py3.12_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.68% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.12_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.7](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `86.07% <30.00%> (-0.06%)` | :arrow_down: |
| [py3.7_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.22% <100.00%> (-0.02%)` | :arrow_down: |
| [py3.7_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.25% <100.00%> (-0.02%)` | :arrow_down: |
| [py3.8](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `86.26% <30.00%> (-0.06%)` | :arrow_down: |
| [py3.8_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.38% <100.00%> (-0.02%)` | :arrow_down: |
| [py3.8_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.40% <100.00%> (-0.02%)` | :arrow_down: |
| [py3.9](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `86.12% <30.00%> (-0.06%)` | :arrow_down: |
| [py3.9_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.27% <100.00%> (-0.02%)` | :arrow_down: |
| [py3.9_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.29% <100.00%> (-0.02%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/omni-us/jsonargparse/pull/537?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 207108c..e62c433 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -19,6 +19,9 @@ Fixed
 ^^^^^
 - Resolving of import paths for some ``torch`` functions not working (`#535
   <https://github.com/omni-us/jsonargparse/pull/535>`__).
+- ``--print_shtab`` crashing on failure to get signature parameters from one
+  class (`lightning#10858 comment
+  <https://github.com/Lightning-AI/pytorch-lightning/discussions/10858#discussioncomment-9846252>`__).
 
 Changed
 ^^^^^^^
diff --git a/jsonargparse/_completions.py b/jsonargparse/_completions.py
index c693916..ee722e5 100644
--- a/jsonargparse/_completions.py
+++ b/jsonargparse/_completions.py
@@ -25,10 +25,6 @@ from ._typehints import (
 )
 from ._util import NoneType, Path, import_object, unique
 
-shtab_shell: ContextVar = ContextVar("shtab_shell")
-shtab_prog: ContextVar = ContextVar("shtab_prog")
-shtab_preambles: ContextVar = ContextVar("shtab_preambles")
-
 
 def handle_completions(parser):
     if find_spec("argcomplete") and "_ARGCOMPLETE" in os.environ:
@@ -76,6 +72,10 @@ def argcomplete_warn_redraw_prompt(prefix, message):
 
 # shtab
 
+shtab_shell: ContextVar = ContextVar("shtab_shell")
+shtab_prog: ContextVar = ContextVar("shtab_prog")
+shtab_preambles: ContextVar = ContextVar("shtab_preambles")
+
 
 class ShtabAction(argparse.Action):
     def __init__(
@@ -236,7 +236,7 @@ def get_typehint_choices(typehint, prefix, parser, skip, choices=None, added_sub
         origin = get_typehint_origin(typehint)
         if origin == Union:
             for subtype in typehint.__args__:
-                if subtype in added_subclasses:
+                if subtype in added_subclasses or subtype is object:
                     continue
                 get_typehint_choices(subtype, prefix, parser, skip, choices, added_subclasses)
         elif ActionTypeHint.is_subclass_typehint(typehint):
@@ -261,8 +261,12 @@ def add_subactions_and_get_subclass_choices(typehint, prefix, parser, skip, adde
     subclasses = defaultdict(list)
     for path in paths:
         choices.append(path)
-        cls = import_object(path)
-        params = get_signature_parameters(cls)
+        try:
+            cls = import_object(path)
+            params = get_signature_parameters(cls, None, parser._logger)
+        except Exception as ex:
+            parser._logger.debug(f"Unable to get signature parameters for '{path}': {ex}")
+            continue
         num_skip = next((s for s in skip if isinstance(s, int)), 0)
         if num_skip > 0:
             params = params[num_skip:]


Test patch:
diff --git a/jsonargparse_tests/test_shtab.py b/jsonargparse_tests/test_shtab.py
index 83704e3..ae463a9 100644
--- a/jsonargparse_tests/test_shtab.py
+++ b/jsonargparse_tests/test_shtab.py
@@ -13,9 +13,10 @@ import pytest
 
 from jsonargparse import ArgumentParser
 from jsonargparse._completions import norm_name
+from jsonargparse._parameter_resolvers import get_signature_parameters
 from jsonargparse._typehints import type_to_str
 from jsonargparse.typing import Path_drw, Path_fr
-from jsonargparse_tests.conftest import get_parse_args_stdout
+from jsonargparse_tests.conftest import capture_logs, get_parse_args_stdout
 
 
 @pytest.fixture(autouse=True)
@@ -203,6 +204,22 @@ class SubB(Base):
         pass
 
 
+def test_bash_subclasses_fail_get_perams(parser, logger):
+    def get_params_patch(cls, method, logger):
+        if cls == SubB:
+            raise Exception("test get params failure")
+        return get_signature_parameters(cls, method, logger)
+
+    parser.logger = logger
+    parser.add_argument("--cls", type=Base)
+    with capture_logs(logger) as logs, patch("jsonargparse._completions.get_signature_parameters", get_params_patch):
+        shtab_script = get_shtab_script(parser, "bash")
+    assert "'--cls' '--cls.p1' '--cls.p2'" in shtab_script
+    assert f"'{__name__}.SubB'" in shtab_script
+    assert "'--cls.p3'" not in shtab_script
+    assert "test_shtab.SubB': test get params failure" in logs.getvalue()
+
+
 def test_bash_subclasses_help(parser):
     parser.add_argument("--cls", type=Base)
     shtab_script = get_shtab_script(parser, "bash")