Repository: getmoto/moto

Issue:
DynamoDB: update_item does not return Item in error response
This issue is related to #5204 but affects a different dynamodb operation (`update_item`). The essential issue is the same, but the manifestation is slightly different, due to the different response structures.

moto does not return the `Item` element in `err.response` when an exception is thrown with an `ConditionalCheckFailedException` error code and `ReturnValuesOnConditionCheckFailure="ALL_OLD"` is supplied in the `update_item` request structure.

I've adapted @magicmark's repro (updated to use `Table.update_item()` instead of `transact_write_items`: [https://github.com/windhamg/moto-issue-demo.git](https://github.com/windhamg/moto-issue-demo.git).

I've created a minimal repro with a failing test case here: https://github.com/magicmark/moto-issue-demo.git

Versions:
```
python --version
Python 3.11.4
```
```
poetry show boto3
 name         : boto3                  
 version      : 1.28.62
```
```
poetry show moto
 name         : moto  
 version      : 4.2.6
```

### Repro instructions 

```bash
$ git clone https://github.com/windhamg/moto-issue-demo.git
$ cd moto-issue-demo
$ poetry install
$ poetry run pytest -vvv
```

### Output

```
        try:
            order_breakfast()
        except BaseException as err:
>           assert 'Item' in err.response
E           AssertionError: assert 'Item' in {'Error': {'Code': 'ConditionalCheckFailedException', 'Message': 'The conditional request failed'}, 'ResponseMetadata': {'HTTPHeaders': {'content-type': 'application/json', 'x-amzn-errortype': 'ConditionalCheckFailedException'}, 'HTTPStatusCode': 400, 'RetryAttempts': 0}, 'message': 'The conditional request failed'}
E            +  where {'Error': {'Code': 'ConditionalCheckFailedException', 'Message': 'The conditional request failed'}, 'ResponseMetadata': {'HTTPHeaders': {'content-type': 'application/json', 'x-amzn-errortype': 'ConditionalCheckFailedException'}, 'HTTPStatusCode': 400, 'RetryAttempts': 0}, 'message': 'The conditional request failed'} = ConditionalCheckFailedException('An error occurred (ConditionalCheckFailedException) when calling the UpdateItem operation: The conditional request failed').response

tests/test_order_breakfast.py:40: AssertionError
========================================================================= short test summary info ==========================================================================
FAILED tests/test_order_breakfast.py::test_breakfast_order - AssertionError: assert 'Item' in {'Error': {'Code': 'ConditionalCheckFailedException', 'Message': 'The conditional request failed'}, 'ResponseMetadata': {'HTTPHeaders'...
============================================================================ 1 failed in 0.42s =============================================================================
```

### What's the problem?

With DynamoDB+boto3, when an `update_item` operation fails due to a `ConditionExpression` failing, AND we've set `ReturnValuesOnConditionCheckFailure="ALL_OLD",` we should see the current item in the error response.

For example, if we ran `order-breakfast` when hitting DynamoDB for real, here's the output of catching the exception and pprinting `ex.response`:

```python
{
    'Error': {
        'Code': 'ConditionalCheckFailedException',
        'Message': 'The conditional request failed'
    },
    'Item': {
        'customer': {
            'S': 'mark'
        },
        'lock': {
            'M': {
                'acquired_at': {
                    'N': '123'
                }
            }
        },
        'mealtime': {
            'S': 'breakfast'
        }
    },
    'ResponseMetadata': {
        'HTTPHeaders': {
            'connection': 'keep-alive',
            'content-length': '223',
            'content-type': 'application/x-amz-json-1.0',
            'date': 'Mon, 23 Oct 2023 23:20:42 GMT',
            'server': 'Server',
            'x-amz-crc32': '3895319109',
            'x-amzn-requestid': '7RVFJQ170IU2JEIGB6VFNN73R7VV4KQNSO5AEMVJF66Q9ASUAAJG'
        },
        'HTTPStatusCode': 400,
        'RequestId': '7RVFJQ170IU2JEIGB6VFNN73R7VV4KQNSO5AEMVJF66Q9ASUAAJG',
        'RetryAttempts': 0
    }
}
```

Note that we see, `Error`, `Item`, and `ResponseMetadata`.

Under moto3, we do not see `Item`:

```python
{
    'Error': {
        'Code': 'ConditionalCheckFailedException',
        'Message': 'The conditional request failed'
    },
    'ResponseMetadata': {
        'HTTPHeaders': {
            'content-type': 'application/json',
            'x-amzn-errortype': 'ConditionalCheckFailedException'
        },
        'HTTPStatusCode': 400,
        'RetryAttempts': 0
    },
    'message': 'The conditional request failed'
}
```


Discussion:
Hi @windhamg, welcome to Moto! Thanks for letting us know, and for providing the repro - that is much appreciated.

Marking it as an enhancement to add this field.

Resolving patch:
diff --git a/moto/dynamodb/exceptions.py b/moto/dynamodb/exceptions.py
--- a/moto/dynamodb/exceptions.py
+++ b/moto/dynamodb/exceptions.py
@@ -1,5 +1,5 @@
 import json
-from typing import Any, List, Optional
+from typing import Any, Dict, List, Optional
 from moto.core.exceptions import JsonRESTError
 from moto.dynamodb.limits import HASH_KEY_MAX_LENGTH, RANGE_KEY_MAX_LENGTH
 
@@ -195,10 +195,31 @@ def __init__(self) -> None:
 class ConditionalCheckFailed(DynamodbException):
     error_type = ERROR_TYPE_PREFIX + "ConditionalCheckFailedException"
 
-    def __init__(self, msg: Optional[str] = None):
-        super().__init__(
-            ConditionalCheckFailed.error_type, msg or "The conditional request failed"
-        )
+    def __init__(
+        self, msg: Optional[str] = None, item: Optional[Dict[str, Any]] = None
+    ):
+        _msg = msg or "The conditional request failed"
+        super().__init__(ConditionalCheckFailed.error_type, _msg)
+        if item:
+            self.description = json.dumps(
+                {
+                    "__type": ConditionalCheckFailed.error_type,
+                    # Note the uppercase Message
+                    # This ensures the message is only part of the 'error': {'message': .., 'code': ..}
+                    "Message": _msg,
+                    "Item": item,
+                }
+            )
+        else:
+            self.description = json.dumps(
+                {
+                    "__type": ConditionalCheckFailed.error_type,
+                    # Note that lowercase 'message'
+                    # This ensures that 'message' is a top-level field in the response
+                    # (in addition to being part of the 'error': {'message': .., 'code': ..}
+                    "message": _msg,
+                }
+            )
 
 
 class TransactionCanceledException(DynamodbException):
@@ -212,10 +233,13 @@ def __init__(self, errors: List[Any]):
         super().__init__(
             error_type=TransactionCanceledException.error_type, message=msg
         )
-        reasons = [
-            {"Code": code, "Message": message, **item} if code else {"Code": "None"}
-            for code, message, item in errors
-        ]
+        reasons = []
+        for code, message, item in errors:
+            r = {"Code": code, "Message": message} if code else {"Code": "None"}
+            if item:
+                r["Item"] = item
+            reasons.append(r)
+
         self.description = json.dumps(
             {
                 "__type": TransactionCanceledException.error_type,
diff --git a/moto/dynamodb/models/__init__.py b/moto/dynamodb/models/__init__.py
--- a/moto/dynamodb/models/__init__.py
+++ b/moto/dynamodb/models/__init__.py
@@ -385,6 +385,7 @@ def update_item(
         attribute_updates: Optional[Dict[str, Any]] = None,
         expected: Optional[Dict[str, Any]] = None,
         condition_expression: Optional[str] = None,
+        return_values_on_condition_check_failure: Optional[str] = None,
     ) -> Item:
         table = self.get_table(table_name)
 
@@ -424,7 +425,13 @@ def update_item(
             expression_attribute_values,
         )
         if not condition_op.expr(item):
-            raise ConditionalCheckFailed
+            if (
+                return_values_on_condition_check_failure == "ALL_OLD"
+                and item is not None
+            ):
+                raise ConditionalCheckFailed(item=item.to_json()["Attributes"])
+            else:
+                raise ConditionalCheckFailed
 
         # Update does not fail on new items, so create one
         if item is None:
@@ -538,6 +545,7 @@ def check_unicity(table_name: str, key: Dict[str, Any]) -> None:
             Union[Tuple[str, str, Dict[str, Any]], Tuple[None, None, None]]
         ] = []  # [(Code, Message, Item), ..]
         for item in transact_items:
+            original_item: Optional[Dict[str, Any]] = None
             # check transact writes are not performing multiple operations
             # in the same item
             if len(list(item.keys())) > 1:
@@ -586,7 +594,7 @@ def check_unicity(table_name: str, key: Dict[str, Any]) -> None:
                         return_values_on_condition_check_failure == "ALL_OLD"
                         and current
                     ):
-                        item["Item"] = current.to_json()["Attributes"]
+                        original_item = current.to_json()["Attributes"]
 
                     self.put_item(
                         table_name,
@@ -643,7 +651,7 @@ def check_unicity(table_name: str, key: Dict[str, Any]) -> None:
                 self.tables = original_table_state
                 raise MultipleTransactionsException()
             except Exception as e:  # noqa: E722 Do not use bare except
-                errors.append((type(e).__name__, e.message, item))  # type: ignore[attr-defined]
+                errors.append((type(e).__name__, e.message, original_item))  # type: ignore
         if any([code is not None for code, _, _ in errors]):
             # Rollback to the original state, and reraise the errors
             self.tables = original_table_state
diff --git a/moto/dynamodb/responses.py b/moto/dynamodb/responses.py
--- a/moto/dynamodb/responses.py
+++ b/moto/dynamodb/responses.py
@@ -851,6 +851,9 @@ def update_item(self) -> str:
             raise MockValidationException(
                 "Can not use both expression and non-expression parameters in the same request: Non-expression parameters: {AttributeUpdates} Expression parameters: {UpdateExpression}"
             )
+        return_values_on_condition_check_failure = self.body.get(
+            "ReturnValuesOnConditionCheckFailure"
+        )
         # We need to copy the item in order to avoid it being modified by the update_item operation
         existing_item = copy.deepcopy(self.dynamodb_backend.get_item(name, key))
         if existing_item:
@@ -887,6 +890,7 @@ def update_item(self) -> str:
             expression_attribute_values=expression_attribute_values,
             expected=expected,
             condition_expression=condition_expression,
+            return_values_on_condition_check_failure=return_values_on_condition_check_failure,
         )
 
         item_dict = item.to_json()


Test patch:
diff --git a/tests/test_dynamodb/exceptions/test_dynamodb_exceptions.py b/tests/test_dynamodb/exceptions/test_dynamodb_exceptions.py
--- a/tests/test_dynamodb/exceptions/test_dynamodb_exceptions.py
+++ b/tests/test_dynamodb/exceptions/test_dynamodb_exceptions.py
@@ -5,6 +5,7 @@
 from botocore.exceptions import ClientError
 from unittest import SkipTest
 from moto import mock_dynamodb, settings
+from .. import dynamodb_aws_verified
 
 table_schema = {
     "KeySchema": [{"AttributeName": "partitionKey", "KeyType": "HASH"}],
@@ -1113,3 +1114,53 @@ def test_query_with_missing_expression_attribute():
         err["Message"]
         == "Invalid condition in KeyConditionExpression: Multiple attribute names used in one condition"
     )
+
+
+@pytest.mark.aws_verified
+@dynamodb_aws_verified
+def test_update_item_returns_old_item(table_name=None):
+    dynamodb = boto3.resource("dynamodb", region_name="us-east-1")
+    table = dynamodb.Table(table_name)
+    table.put_item(Item={"pk": "mark", "lock": {"acquired_at": 123}})
+
+    with pytest.raises(ClientError) as exc:
+        table.update_item(
+            Key={"pk": "mark"},
+            UpdateExpression="set #lock = :lock",
+            ExpressionAttributeNames={
+                "#lock": "lock",
+                "#acquired_at": "acquired_at",
+            },
+            ExpressionAttributeValues={":lock": {"acquired_at": 124}},
+            ConditionExpression="attribute_not_exists(#lock.#acquired_at)",
+        )
+    resp = exc.value.response
+    assert resp["Error"] == {
+        "Message": "The conditional request failed",
+        "Code": "ConditionalCheckFailedException",
+    }
+    assert resp["message"] == "The conditional request failed"
+    assert "Item" not in resp
+
+    with pytest.raises(ClientError) as exc:
+        table.update_item(
+            Key={"pk": "mark"},
+            UpdateExpression="set #lock = :lock",
+            ExpressionAttributeNames={
+                "#lock": "lock",
+                "#acquired_at": "acquired_at",
+            },
+            ExpressionAttributeValues={":lock": {"acquired_at": 123}},
+            ReturnValuesOnConditionCheckFailure="ALL_OLD",
+            ConditionExpression="attribute_not_exists(#lock.#acquired_at)",
+        )
+    resp = exc.value.response
+    assert resp["Error"] == {
+        "Message": "The conditional request failed",
+        "Code": "ConditionalCheckFailedException",
+    }
+    assert "message" not in resp
+    assert resp["Item"] == {
+        "lock": {"M": {"acquired_at": {"N": "123"}}},
+        "pk": {"S": "mark"},
+    }
diff --git a/tests/test_dynamodb/test_dynamodb.py b/tests/test_dynamodb/test_dynamodb.py
--- a/tests/test_dynamodb/test_dynamodb.py
+++ b/tests/test_dynamodb/test_dynamodb.py
@@ -1,20 +1,20 @@
+import boto3
+import pytest
 import uuid
+import re
+from botocore.exceptions import ClientError
 from datetime import datetime
 from decimal import Decimal
 
-import boto3
 from boto3.dynamodb.conditions import Attr, Key
 from boto3.dynamodb.types import Binary
-import re
 from moto import mock_dynamodb, settings
 from moto.core import DEFAULT_ACCOUNT_ID as ACCOUNT_ID
 from moto.dynamodb import dynamodb_backends
-from botocore.exceptions import ClientError
 
 import moto.dynamodb.comparisons
 import moto.dynamodb.models
-
-import pytest
+from . import dynamodb_aws_verified
 
 
 @mock_dynamodb
@@ -3685,17 +3685,12 @@ def test_transact_write_items_put():
     assert len(items) == 5
 
 
-@mock_dynamodb
-def test_transact_write_items_put_conditional_expressions():
-    table_schema = {
-        "KeySchema": [{"AttributeName": "id", "KeyType": "HASH"}],
-        "AttributeDefinitions": [{"AttributeName": "id", "AttributeType": "S"}],
-    }
+@pytest.mark.aws_verified
+@dynamodb_aws_verified
+def test_transact_write_items_put_conditional_expressions(table_name=None):
     dynamodb = boto3.client("dynamodb", region_name="us-east-1")
-    dynamodb.create_table(
-        TableName="test-table", BillingMode="PAY_PER_REQUEST", **table_schema
-    )
-    dynamodb.put_item(TableName="test-table", Item={"id": {"S": "foo2"}})
+
+    dynamodb.put_item(TableName=table_name, Item={"pk": {"S": "foo2"}})
     # Put multiple items
     with pytest.raises(ClientError) as ex:
         dynamodb.transact_write_items(
@@ -3703,12 +3698,12 @@ def test_transact_write_items_put_conditional_expressions():
                 {
                     "Put": {
                         "Item": {
-                            "id": {"S": f"foo{i}"},
+                            "pk": {"S": f"foo{i}"},
                             "foo": {"S": "bar"},
                         },
-                        "TableName": "test-table",
+                        "TableName": table_name,
                         "ConditionExpression": "#i <> :i",
-                        "ExpressionAttributeNames": {"#i": "id"},
+                        "ExpressionAttributeNames": {"#i": "pk"},
                         "ExpressionAttributeValues": {
                             ":i": {
                                 "S": "foo2"
@@ -3726,27 +3721,20 @@ def test_transact_write_items_put_conditional_expressions():
     assert {
         "Code": "ConditionalCheckFailed",
         "Message": "The conditional request failed",
-        "Item": {"id": {"S": "foo2"}, "foo": {"S": "bar"}},
     } in reasons
     assert {"Code": "None"} in reasons
     assert ex.value.response["ResponseMetadata"]["HTTPStatusCode"] == 400
     # Assert all are present
-    items = dynamodb.scan(TableName="test-table")["Items"]
+    items = dynamodb.scan(TableName=table_name)["Items"]
     assert len(items) == 1
-    assert items[0] == {"id": {"S": "foo2"}}
+    assert items[0] == {"pk": {"S": "foo2"}}
 
 
-@mock_dynamodb
-def test_transact_write_items_put_conditional_expressions_return_values_on_condition_check_failure_all_old():
-    table_schema = {
-        "KeySchema": [{"AttributeName": "id", "KeyType": "HASH"}],
-        "AttributeDefinitions": [{"AttributeName": "id", "AttributeType": "S"}],
-    }
+@pytest.mark.aws_verified
+@dynamodb_aws_verified
+def test_transact_write_items_failure__return_item(table_name=None):
     dynamodb = boto3.client("dynamodb", region_name="us-east-1")
-    dynamodb.create_table(
-        TableName="test-table", BillingMode="PAY_PER_REQUEST", **table_schema
-    )
-    dynamodb.put_item(TableName="test-table", Item={"id": {"S": "foo2"}})
+    dynamodb.put_item(TableName=table_name, Item={"pk": {"S": "foo2"}})
     # Put multiple items
     with pytest.raises(ClientError) as ex:
         dynamodb.transact_write_items(
@@ -3754,12 +3742,13 @@ def test_transact_write_items_put_conditional_expressions_return_values_on_condi
                 {
                     "Put": {
                         "Item": {
-                            "id": {"S": f"foo{i}"},
+                            "pk": {"S": f"foo{i}"},
                             "foo": {"S": "bar"},
                         },
-                        "TableName": "test-table",
+                        "TableName": table_name,
                         "ConditionExpression": "#i <> :i",
-                        "ExpressionAttributeNames": {"#i": "id"},
+                        "ExpressionAttributeNames": {"#i": "pk"},
+                        # This man right here - should return item as part of error message
                         "ReturnValuesOnConditionCheckFailure": "ALL_OLD",
                         "ExpressionAttributeValues": {
                             ":i": {
@@ -3778,14 +3767,14 @@ def test_transact_write_items_put_conditional_expressions_return_values_on_condi
     assert {
         "Code": "ConditionalCheckFailed",
         "Message": "The conditional request failed",
-        "Item": {"id": {"S": "foo2"}},
+        "Item": {"pk": {"S": "foo2"}},
     } in reasons
     assert {"Code": "None"} in reasons
     assert ex.value.response["ResponseMetadata"]["HTTPStatusCode"] == 400
     # Assert all are present
-    items = dynamodb.scan(TableName="test-table")["Items"]
+    items = dynamodb.scan(TableName=table_name)["Items"]
     assert len(items) == 1
-    assert items[0] == {"id": {"S": "foo2"}}
+    assert items[0] == {"pk": {"S": "foo2"}}
 
 
 @mock_dynamodb