Repository: omni-us/jsonargparse

Issue:
Cannot use optional dataclasses defaulting to not-null in the configuration hierarchy
<!-- If you like this project, please ⭐ star it https://github.com/omni-us/jsonargparse/ -->

## 🐛 Bug report

Cannot use optional dataclasses defaulting to not-null in the configuration hierarchy.

### To reproduce

Run script:

```
import jsonargparse
from typing import Optional
from dataclasses import dataclass, field

@dataclass
class B:
    c: int = 3

@dataclass
class A:
    # THIS CAUSES AN ERROR
    b: Optional[B] = field(default_factory=B)
    # THIS IS FINE
    #b: B = field(default_factory=B)

def fun(a: A):
    print(a)

jsonargparse.CLI(fun)
```

We get 
```
usage: python-jsonargparse-optional-dataclass.py [-h] [--config CONFIG] [--print_config[=flags]] [--a CONFIG] [--a.b B]
error: Validation failed: Parser key "a.b":
  Does not validate against any of the Union subtypes
  Subtypes: (<class '__main__.B'>, <class 'NoneType'>)
  Errors:
    - Type <class '__main__.B'> expects a dict or Namespace
    - Expected a <class 'NoneType'>
  Given value type: <class '__main__.B'>
  Given value: B(c=3)
```

### Expected behavior

Script should write:
```A(b=B(c=3))```

### Environment

- jsonargparse version: 4.28.0:
- Python version: 3.10.12:
- How jsonargparse was installed: `pip install jsonargparse`
- OS: Ubuntu Jammy


Discussion:
sonarcloud[bot]: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=508) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=508&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=omni-us_jsonargparse&pullRequest=508&resolutions=WONTFIX)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=omni-us_jsonargparse&pullRequest=508&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=omni-us_jsonargparse&pullRequest=508&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=omni-us_jsonargparse&pullRequest=508)


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 99.96%. Comparing base [(`ef1a267`)](https://app.codecov.io/gh/omni-us/jsonargparse/commit/ef1a26791cd5a924db2b1721dc904fc3f65591aa?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) to head [(`02998e5`)](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##              main     #508      +/-   ##
===========================================
- Coverage   100.00%   99.96%   -0.04%     
===========================================
  Files           21       21              
  Lines         6147     6149       +2     
===========================================
  Hits          6147     6147              
- Misses           0        2       +2     
```

| [Flag](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | Coverage Δ | |
|---|---|---|
| [py3.10](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.29% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.10_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.10_pydantic1](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `49.34% <100.00%> (+0.24%)` | :arrow_up: |
| [py3.10_pydantic2](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.10_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.66% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.11](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.11_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.11_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.66% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.12](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.42% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.12_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `98.65% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.12_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `?` | |
| [py3.7](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.84% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.7_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.26% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.7_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.26% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.8](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `88.01% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.8_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.41% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.8_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.41% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.9](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `87.84% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.9_all](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.28% <100.00%> (+<0.01%)` | :arrow_up: |
| [py3.9_types](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us) | `99.30% <100.00%> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us#carryforward-flags-in-the-pull-request-comment) to find out more.


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/omni-us/jsonargparse/pull/508?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=omni-us).


Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index ef6e802..0f4d636 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -36,6 +36,8 @@ Fixed
 - Nested values not validated when type not subclass and nested keys for
   subclass (`#503 comment
   <https://github.com/omni-us/jsonargparse/issues/503#issuecomment-2119724341>`__).
+- Dataclass with nested optional dataclass and default field factory failing to
+  parse (`#507 <https://github.com/omni-us/jsonargparse/issues/507>`__).
 
 Changed
 ^^^^^^^
diff --git a/jsonargparse/_typehints.py b/jsonargparse/_typehints.py
index f90c44e..f406264 100644
--- a/jsonargparse/_typehints.py
+++ b/jsonargparse/_typehints.py
@@ -215,6 +215,10 @@ class ActionTypeHint(Action):
         is_subclass_type = self.is_subclass_typehint(self._typehint, all_subtypes=False)
         if isinstance(default, LazyInitBaseClass):
             default = default.lazy_get_init_data()
+        elif is_dataclass_like(default.__class__):
+            from ._signatures import dataclass_to_dict
+
+            default = dataclass_to_dict(default)
         elif is_subclass_type and isinstance(default, dict) and "class_path" in default:
             default = subclass_spec_as_namespace(default)
             default.class_path = normalize_import_path(default.class_path, self._typehint)
@@ -896,11 +900,9 @@ def adapt_typehints(
 
     # Dataclass-like
     elif is_dataclass_like(typehint):
-        if is_dataclass_like(type(prev_val)) and is_subclass(type(prev_val), typehint):
-            from ._signatures import dataclass_to_dict
-
+        if isinstance(prev_val, (dict, Namespace)):
             assert isinstance(sub_add_kwargs, dict)
-            sub_add_kwargs["default"] = lazy_instance(type(prev_val), **dataclass_to_dict(prev_val))
+            sub_add_kwargs["default"] = lazy_instance(typehint, **prev_val)
         parser = ActionTypeHint.get_class_parser(typehint, sub_add_kwargs=sub_add_kwargs)
         if instantiate_classes:
             init_args = parser.instantiate_classes(val)


Test patch:
diff --git a/jsonargparse_tests/test_dataclass_like.py b/jsonargparse_tests/test_dataclass_like.py
index 375d71a..3e30216 100644
--- a/jsonargparse_tests/test_dataclass_like.py
+++ b/jsonargparse_tests/test_dataclass_like.py
@@ -375,6 +375,28 @@ def test_optional_dataclass_type_null_value():
     assert cfg == parser_optional_data.instantiate_classes(cfg)
 
 
+@dataclasses.dataclass
+class DataWithOptionalB:
+    c: int = 3
+
+
+@dataclasses.dataclass
+class DataWithOptionalA:
+    b: Optional[DataWithOptionalB] = dataclasses.field(default_factory=DataWithOptionalB)
+
+
+def data_with_optional(a: DataWithOptionalA):
+    pass
+
+
+def test_dataclass_with_optional_default(parser):
+    parser.add_function_arguments(data_with_optional, "data")
+    cfg = parser.parse_args([])
+    assert cfg.data == Namespace(a=Namespace(b={"c": 3}))
+    init = parser.instantiate_classes(cfg)
+    assert init.data.a == DataWithOptionalA()
+
+
 @dataclasses.dataclass
 class SingleParamChange:
     p1: int = 0
@@ -397,7 +419,9 @@ class ModelConfig:
 def test_dataclass_optional_dict_attribute(parser):
     parser.add_argument("--model", type=Optional[ModelConfig], default=ModelConfig(data={"A": 1, "B": 2}))
     cfg = parser.parse_args(["--model.data.A=4"])
-    assert cfg.model.data == {"A": 4, "B": 2}
+    assert cfg.model["data"] == {"A": 4, "B": 2}
+    init = parser.instantiate_classes(cfg)
+    assert init.model == ModelConfig(data={"A": 4, "B": 2})
 
 
 def test_dataclass_in_union_type(parser):