Repository: koxudaxi/datamodel-code-generator (Python)

Issue:
Regression where _deep_merge() mutates list values in passed dictionary
**Describe the bug**

I think #2318 introduced a regression, by changing

```
result[key] = result[key] + value
```

in `_deep_merge()` in `src/datamodel_code_generator/parser/jsonschema.py` to

```
result[key] += value
```

In the old / good version, we create a new list and assign it to `result[key]`. In the new / buggy version, we mutate `result[key]` in place, which results in incorrect models being generated in some cases.

It looks like the change was introduced by adopting ruff's [PLR6104](https://docs.astral.sh/ruff/rules/non-augmented-assignment/#non-augmented-assignment-plr6104). That rule is unsafe, i.e. it changes the semantics.

I opened a PR with a fix: #2400.

**To Reproduce**

Example schema:

```json
{
  "type": "object",
  "properties": {
    "foo": {
      "type": "object",
      "required": [
        "bar"
      ],
      "anyOf": [{
          "required": [
            "baz"
          ]
        },
        {
          "required": [
            "qux"
          ]
        }
      ],
      "properties": {
        "bar": {
          "type": "integer"
        },
        "baz": {
          "type": "integer"
        },
        "qux": {
          "type": "integer"
        }
      }
    }
  }
}
```

Used commandline:

```
$ uv tool run --from datamodel-code-generator datamodel-codegen --input-file-type jsonschema --input my_schema.json
```

**Expected output**

```python
# generated by datamodel-codegen:
#   filename:  my_schema.json
#   timestamp: 2025-05-09T22:13:24+00:00

from __future__ import annotations

from typing import Optional, Union

from pydantic import BaseModel


class Foo(BaseModel):
    bar: int
    baz: int
    qux: Optional[int] = None


class Foo1(BaseModel):
    bar: int
    baz: Optional[int] = None
    qux: int


class Model(BaseModel):
    foo: Optional[Union[Foo, Foo1]] = None
```

**Actual output**

```python
# generated by datamodel-codegen:
#   filename:  my_schema.json
#   timestamp: 2025-05-09T22:13:24+00:00

from __future__ import annotations

from typing import Optional, Union

from pydantic import BaseModel


class Foo(BaseModel):
    bar: int
    baz: int
    qux: Optional[int] = None


class Foo1(BaseModel):
    bar: int
    baz: int
    qux: int


class Model(BaseModel):
    foo: Optional[Union[Foo, Foo1]] = None
```

**Version:**

 - OS: macOS sequoia 15.4.1
 - Python version: 3.13.2
 - datamodel-code-generator version: 0.30.1

**Additional context**

I've confirmed that reverting the change results in the expected output in the above example. See #2400.

I found the commit that introduced the regression using `git bisect`: 66e5876 is the last good; 1acaa9b is the first bad.

To find the bug, I speculatively set a breakpoint in `parse_combined_schema()` in `src/datamodel_code_generator/parser/jsonschema.py`, and poked around with `pdb`:

```python
    def parse_combined_schema(
        self,
        name: str,
        obj: JsonSchemaObject,
        path: list[str],
        target_attribute_name: str,
    ) -> list[DataType]:
        base_object = obj.dict(exclude={target_attribute_name}, exclude_unset=True, by_alias=True)
        combined_schemas: list[JsonSchemaObject] = []
        refs = []
        for index, target_attribute in enumerate(getattr(obj, target_attribute_name, [])):
            if target_attribute.ref:
                combined_schemas.append(target_attribute)
                refs.append(index)
                # TODO: support partial ref
            else:
                combined_schemas.append(
                    self.SCHEMA_OBJECT_TYPE.parse_obj(
                        self._deep_merge(
                            base_object,
                            target_attribute.dict(exclude_unset=True, by_alias=True),
                        )
                    )
                )
    ...
```

The key observation was that when `base_object` is assigned, it's symmetrical in `baz` and `qux`:

```
(Pdb) pp base_object
{'#-datamodel-code-generator-#-extras-#-special-#': {},
 'properties': {'bar': {'#-datamodel-code-generator-#-extras-#-special-#': {},
                        'type': 'integer'},
                'baz': {'#-datamodel-code-generator-#-extras-#-special-#': {},
                        'type': 'integer'},
                'qux': {'#-datamodel-code-generator-#-extras-#-special-#': {},
                        'type': 'integer'}},
 'required': ['bar'],
 'type': 'object'}
```

but after the first run through the `for` loop body it no longer is, because `_deep_merge()` has mutated `base_object["required"]`:

```
(Pdb) pp base_object
{'#-datamodel-code-generator-#-extras-#-special-#': {},
 'properties': {
    'bar': {'#-datamodel-code-generator-#-extras-#-special-#': {},
                        'type': 'integer'},
    'baz': {'#-datamodel-code-generator-#-extras-#-special-#': {},
                        'type': 'integer'},
    'qux': {'#-datamodel-code-generator-#-extras-#-special-#': {},
                        'type': 'integer'}},
 'required': ['bar', 'baz'],   <--- HERE
 'type': 'object'}
```

This means that on the second run through the loop, `baz` is wrongly taken to be required. That's why in the actual output above, `baz` is required in _both_ `Foo` and `Foo1`.


Discussion:

Patch:
diff --git a/src/datamodel_code_generator/parser/jsonschema.py b/src/datamodel_code_generator/parser/jsonschema.py
index 21a7de169..61e8e70d0 100644
--- a/src/datamodel_code_generator/parser/jsonschema.py
+++ b/src/datamodel_code_generator/parser/jsonschema.py
@@ -627,7 +627,7 @@ def _deep_merge(self, dict1: dict[Any, Any], dict2: dict[Any, Any]) -> dict[Any,
                     result[key] = self._deep_merge(result[key], value)
                     continue
                 if isinstance(result[key], list) and isinstance(value, list):
-                    result[key] += value
+                    result[key] = result[key] + value  # noqa: PLR6104
                     continue
             result[key] = value
         return result
diff --git a/uv.lock b/uv.lock
index 76f3a2f5f..0ee17fe0f 100644
--- a/uv.lock
+++ b/uv.lock
@@ -6,9 +6,9 @@ resolution-markers = [
     "python_full_version < '3.9.17'",
 ]
 conflicts = [[
+    { package = "datamodel-code-generator", group = "black24" },
     { package = "datamodel-code-generator", group = "black22" },
     { package = "datamodel-code-generator", group = "black23" },
-    { package = "datamodel-code-generator", group = "black24" },
     { package = "datamodel-code-generator", group = "dev" },
 ]]
 


Test patch:
diff --git a/tests/data/expected/main/jsonschema/required_and_any_of_required.py b/tests/data/expected/main/jsonschema/required_and_any_of_required.py
new file mode 100644
index 000000000..3e0bd7171
--- /dev/null
+++ b/tests/data/expected/main/jsonschema/required_and_any_of_required.py
@@ -0,0 +1,25 @@
+# generated by datamodel-codegen:
+#   filename:  required_and_any_of_required.json
+#   timestamp: 2019-07-26T00:00:00+00:00
+
+from __future__ import annotations
+
+from typing import Optional, Union
+
+from pydantic import BaseModel
+
+
+class Foo(BaseModel):
+    bar: int
+    baz: int
+    qux: Optional[int] = None
+
+
+class Foo1(BaseModel):
+    bar: int
+    baz: Optional[int] = None
+    qux: int
+
+
+class Model(BaseModel):
+    foo: Union[Foo, Foo1]
diff --git a/tests/data/jsonschema/required_and_any_of_required.json b/tests/data/jsonschema/required_and_any_of_required.json
new file mode 100644
index 000000000..6c859ab60
--- /dev/null
+++ b/tests/data/jsonschema/required_and_any_of_required.json
@@ -0,0 +1,37 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "type": "object",
+  "required": [
+    "foo"
+  ],
+  "properties": {
+    "foo": {
+      "type": "object",
+      "required": [
+        "bar"
+      ],
+      "anyOf": [{
+          "required": [
+            "baz"
+          ]
+        },
+        {
+          "required": [
+            "qux"
+          ]
+        }
+      ],
+      "properties": {
+        "bar": {
+          "type": "integer"
+        },
+        "baz": {
+          "type": "integer"
+        },
+        "qux": {
+          "type": "integer"
+        }
+      }
+    }
+  }
+}
diff --git a/tests/main/jsonschema/test_main_jsonschema.py b/tests/main/jsonschema/test_main_jsonschema.py
index 234e49fea..4ad83198f 100644
--- a/tests/main/jsonschema/test_main_jsonschema.py
+++ b/tests/main/jsonschema/test_main_jsonschema.py
@@ -3114,3 +3114,20 @@ def test_main_jsonschema_field_has_same_name(output_model: str, expected_output:
         ])
         assert return_code == Exit.OK
         assert output_file.read_text() == (EXPECTED_JSON_SCHEMA_PATH / expected_output).read_text()
+
+
+@pytest.mark.benchmark
+@freeze_time("2019-07-26")
+def test_main_jsonschema_required_and_any_of_required() -> None:
+    with TemporaryDirectory() as output_dir:
+        output_file: Path = Path(output_dir) / "output.py"
+        return_code: Exit = main([
+            "--input",
+            str(JSON_SCHEMA_DATA_PATH / "required_and_any_of_required.json"),
+            "--output",
+            str(output_file),
+            "--input-file-type",
+            "jsonschema",
+        ])
+        assert return_code == Exit.OK
+        assert output_file.read_text() == (EXPECTED_JSON_SCHEMA_PATH / "required_and_any_of_required.py").read_text()