Repository: getmoto/moto
Pull request #8884 (2025-05-10 10:39:19)

IOTData: update_thing_shadow() should keep empty dicts
## Motivation

Improves the parity between how `update_thing_shadow()` behaves when sending empty dictionaries. The existing implementation had two problems.

### State treated the same as Attributes

First: when a user calls `update_thing_shadow` with an empty desired or reported state, we should ignore this empty dict.
```json
{
    "state": {
        "reported": {}
    }
}
```

However, if an attribute within the state is empty, we should keep it:
```json
{
    "state": {
        "reported": {
            "attr": {}
        }
    }
}
```

Our implementation treated the two cases the same, and always removed empty dictionaries. With this change, we make it explicit that empty reported/desired states are effectively ignored

### Type changes

The second problem is that we always assumed that types would not change - that is, if a new value for attribute `a` is a dictionary, it was assumed that the original value for `a` was also a dictionary.
This is not the case - a user could update `'a': 3` to `'a': {}`.

#### API change
The `merge_dicts`-method now no longer removes empty dictionaries from the input if `remove_nulls=True`.
This makes sense IMO, as the naming is very specific. It also should not matter, as this method is not used anywhere else (despite being part of `moto.core`)

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/getmoto/moto/pull/8884?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 92.85%. Comparing base [(`a8d0784`)](https://app.codecov.io/gh/getmoto/moto/commit/a8d0784621d2e4def9c9318be96da0161eb4ec72?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) to head [(`940b251`)](https://app.codecov.io/gh/getmoto/moto/commit/940b251151aa9881ca9e9b07668aa7b8fe2e26bc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #8884      +/-   ##
==========================================
- Coverage   92.85%   92.85%   -0.01%     
==========================================
  Files        1282     1282              
  Lines      111874   111877       +3     
==========================================
+ Hits       103882   103884       +2     
- Misses       7992     7993       +1     
```

| [Flag](https://app.codecov.io/gh/getmoto/moto/pull/8884/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | Coverage Δ | |
|---|---|---|
| [servertests](https://app.codecov.io/gh/getmoto/moto/pull/8884/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | `27.96% <0.00%> (-0.01%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/getmoto/moto/pull/8884/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto) | `92.83% <100.00%> (-0.01%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/getmoto/moto/pull/8884?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=getmoto).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>

Patch:
diff --git a/moto/core/utils.py b/moto/core/utils.py
index d370d55d2..fafb9ad70 100644
--- a/moto/core/utils.py
+++ b/moto/core/utils.py
@@ -331,19 +331,17 @@ def merge_dicts(
     :param dict dict1: the dictionary to be updated.
     :param dict dict2: a dictionary of keys/values to be merged into dict1.
 
-    :param bool remove_nulls: If true, updated values equal to None or an empty dictionary
+    :param bool remove_nulls: If true, updated values equal to None
         will be removed from dict1.
     """
     for key in dict2:
         if isinstance(dict2[key], dict):
-            if key in dict1 and key in dict2:
+            if key in dict1 and isinstance(dict1[key], dict):
                 merge_dicts(dict1[key], dict2[key], remove_nulls)
             else:
                 dict1[key] = dict2[key]
                 if isinstance(dict1[key], dict):
                     remove_null_from_dict(dict1)
-            if dict1[key] == {} and remove_nulls:
-                dict1.pop(key)
         else:
             dict1[key] = dict2[key]
             if dict1[key] is None and remove_nulls:
diff --git a/moto/iotdata/models.py b/moto/iotdata/models.py
index 009a293b6..98f183aae 100644
--- a/moto/iotdata/models.py
+++ b/moto/iotdata/models.py
@@ -60,13 +60,22 @@ class FakeShadow(BaseModel):
             shadow = FakeShadow(None, None, None, version, deleted=True)
             return shadow
 
+        desired_payload = payload.get("state", {}).get("desired")
+        reported_payload = payload.get("state", {}).get("reported")
+
+        state_document = previous_payload.copy()
+
+        desired = state_document.get("state", {}).get("desired", {})
+        reported = state_document.get("state", {}).get("reported", {})
+
         # Updates affect only the fields specified in the request state document.
         # Any field with a value of None is removed from the device's shadow.
-        state_document = previous_payload.copy()
-        merge_dicts(state_document, payload, remove_nulls=True)
-        desired = state_document.get("state", {}).get("desired")
-        reported = state_document.get("state", {}).get("reported")
-        return FakeShadow(desired, reported, payload, version)
+        if desired_payload:
+            merge_dicts(desired, desired_payload, remove_nulls=True)
+        if reported_payload:
+            merge_dicts(reported, reported_payload, remove_nulls=True)
+
+        return FakeShadow(desired or None, reported or None, payload, version)
 
     @classmethod
     def parse_payload(cls, desired: Any, reported: Any) -> Any:  # type: ignore[misc]


Test patch:
diff --git a/tests/test_iotdata/test_iotdata.py b/tests/test_iotdata/test_iotdata.py
index 6b6b80d68..8e1b8daf1 100644
--- a/tests/test_iotdata/test_iotdata.py
+++ b/tests/test_iotdata/test_iotdata.py
@@ -342,3 +342,48 @@ def test_delta_calculation(
     res = client.get_thing_shadow(thingName=name)
     payload = json.loads(res["payload"].read())
     assert payload["state"] == delta_after_report
+
+
+@iot_aws_verified()
+@pytest.mark.aws_verified
+@pytest.mark.parametrize(
+    "initial_state,updated_state",
+    [
+        # Insert new dicts
+        (
+            {"a": 1, "b": {"c": 3}},
+            {"a": 1, "b": {"c": 3}, "d": {}},
+        ),
+        # Update existing value with new dicts
+        (
+            {"a": 1, "b": {"c": 3}, "d": {}},
+            {"a": 1, "b": {"c": {}}, "d": {}},
+        ),
+        # Update existing value with full dicts
+        (
+            {"a": 1, "b": {"c": 3}},
+            {"a": 1, "b": {"c": {"d": 3}}},
+        ),
+    ],
+)
+def test_update_desired(
+    initial_state: dict[str, str],
+    updated_state: dict[str, str],
+    name: Optional[str] = None,
+) -> None:
+    client = boto3.client("iot-data", region_name="ap-northeast-1")
+
+    # CREATE
+    payload = json.dumps({"state": {"desired": initial_state}}).encode("utf-8")
+    client.update_thing_shadow(thingName=name, payload=payload)
+
+    # UPDATE
+    payload = json.dumps({"state": {"desired": updated_state}}).encode("utf-8")
+    client.update_thing_shadow(thingName=name, payload=payload)
+
+    # GET --> Verify the updated state is returned as-is
+    res = client.get_thing_shadow(thingName=name)
+    result_payload = json.loads(res["payload"].read())
+
+    assert result_payload["state"]["delta"] == updated_state
+    assert result_payload["state"]["desired"] == updated_state