Repository: localstack/localstack

Issue:
ContentBasedDeduplication not working correctly in FIFO queue
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

I created an SQS FIFO queue with content based deduplication and sent a message that contains the same messageBody and attributes twice.

The message gets sent and received by the queue twice.

Im using the latest version of localstack with docker-compose.


### Expected Behavior

When ContetBasedDeduplication is set to true a message with the same body should only be recieved once by the queue during the 5 minute deduplication interval.

### How are you starting LocalStack?

With a docker-compose file

### Steps To Reproduce

create queue
```
sqs_client.create_queue(
            QueueName='my_queue.fifo',
            Attributes={
                'FifoQueue': 'true',
                'ContentBasedDeduplication': 'true'
            },
        )
```

send message
```
item = {'hello': 'hola'}
sqs_client.send_message(
            QueueUrl='my_queue.fifo',
            MessageBody=json.dumps(item),
            MessageAttributes= {'hello': {'StringValue': 'hola', 'DataType': 'String'}},
            MessageGroupId='group-1'
        )
```





### Environment

```markdown
- OS:Ubuntu 20.04
- LocalStack: latest
```


### Anything else?

_No response_


Discussion:
Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our [Slack Community](https://localstack.cloud/contact/) for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our [contributing guidelines](https://github.com/localstack/.github/blob/main/CONTRIBUTING.md) and our [developer guide](https://docs.localstack.cloud/developer-guide/).
in addition this bug only occurs when `--message-deduplication-id` parameter is not passed while sending a message. 

this is absolutely fine according to the AWS documentation: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SetQueueAttributes.html:
> If you aren't able to provide a MessageDeduplicationId and you enable ContentBasedDeduplication for your queue, Amazon SQS uses a SHA-256 hash to generate the MessageDeduplicationId using the body of the message (but not the attributes of the message).

summary:
```sh
awslocal sqs create-queue --queue-name input.fifo --attributes '{"FifoQueue": "true", "FifoThroughputLimit": "perMessageGroupId", "ContentBasedDeduplication": "true", "DeduplicationScope": "messageGroup"}'
awslocal sqs send-message --queue-url http://localhost:4566/000000000000/input.fifo --message-body A --message-group-id G1
awslocal sqs send-message --queue-url http://localhost:4566/000000000000/input.fifo --message-body A --message-group-id G1
awslocal sqs receive-message --max-number-of-messages 10 --visibility-timeout 2 --queue-url http://localhost:4566/000000000000/input.fifo # 2 messages instead of 1
```

using this, will work:
```sh
awslocal sqs send-message --queue-url http://localhost:4566/000000000000/input.fifo --message-body A --message-group-id G1 --message-deduplication-id A
```
any updates?

Resolving patch:
diff --git a/localstack/services/sqs/provider.py b/localstack/services/sqs/provider.py
--- a/localstack/services/sqs/provider.py
+++ b/localstack/services/sqs/provider.py
@@ -581,7 +581,7 @@ def put(
             self.visible.put_nowait(qm)
             if not original_message_group:
                 self.deduplication[message_group_id] = {}
-            self.deduplication[message_group_id][message_deduplication_id] = qm
+            self.deduplication[message_group_id][dedup_id] = qm
 
     def _assert_queue_name(self, name):
         if not name.endswith(".fifo"):


Test patch:
diff --git a/tests/integration/test_sqs.py b/tests/integration/test_sqs.py
--- a/tests/integration/test_sqs.py
+++ b/tests/integration/test_sqs.py
@@ -1444,6 +1444,34 @@ def test_fifo_queue_send_multiple_messages_multiple_single_receives(
             assert message["MessageId"] == sent_messages[i]["MessageId"]
             sqs_client.delete_message(QueueUrl=queue_url, ReceiptHandle=message["ReceiptHandle"])
 
+    @pytest.mark.aws_validated
+    def test_fifo_content_based_message_deduplication_arrives_once(
+        self, sqs_client, sqs_create_queue
+    ):
+        # created for https://github.com/localstack/localstack/issues/6327
+        queue_url = sqs_create_queue(
+            QueueName=f"test-queue-{short_uid()}.fifo",
+            Attributes={"FifoQueue": "true", "ContentBasedDeduplication": "true"},
+        )
+        item = '{"foo": "bar"}'
+        group = "group-1"
+
+        sqs_client.send_message(QueueUrl=queue_url, MessageBody=item, MessageGroupId=group)
+        sqs_client.send_message(QueueUrl=queue_url, MessageBody=item, MessageGroupId=group)
+
+        # first receive has the item
+        response = sqs_client.receive_message(
+            QueueUrl=queue_url, MaxNumberOfMessages=1, WaitTimeSeconds=2
+        )
+        assert len(response["Messages"]) == 1
+        assert response["Messages"][0]["Body"] == item
+
+        # second doesn't since the message has the same content
+        response = sqs_client.receive_message(
+            QueueUrl=queue_url, MaxNumberOfMessages=1, WaitTimeSeconds=1
+        )
+        assert response.get("Messages", []) == []
+
     @pytest.mark.xfail(
         reason="localstack allows queue names with slashes, but this should be deprecated"
     )