Repository: getmoto/moto
Pull request: Add sqs error handling
I had to rewrite `sqs.send_message_batch` a bit to be able to handle most errors.

Also fixes #1799

Over time I will also check other functions in the SQS implementation and update the coverage file while doing so.


Problem statement:
Moto should raise an error when SQS send_message_batch pass an empty entries
When doing `client.send_message_batch(QueueUrl=queue_url, Entries=[])`, boto will raise an error.

```
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/botocore/client.py", line 314, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/botocore/client.py", line 612, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.EmptyBatchRequest: An error occurred (AWS.SimpleQueueService.EmptyBatchRequest) when calling the SendMessageBatch operation: There should be at least one SendMessageBatchRequestEntry in the request.
```

moto should raise the same error, rather than to be silent on that.

Environment:

```
$ python --version
Python 2.7.11
$ python -c "import boto3; print(boto3.__version__)"
1.4.7
$ python -c "import moto; print(moto.__version__)"
1.3.1
```

Hints from issue discussion:
Thanks; setting as a feature request.

Commit message: Update implementation coverage

Patch to review:
diff --git a/IMPLEMENTATION_COVERAGE.md b/IMPLEMENTATION_COVERAGE.md
index 57f169b8ab01..060f9f7a7ddd 100644
--- a/IMPLEMENTATION_COVERAGE.md
+++ b/IMPLEMENTATION_COVERAGE.md
@@ -5902,7 +5902,7 @@
 - [x] untag_resource
 
 ## sqs
-65% implemented
+75% implemented
 - [X] add_permission
 - [X] change_message_visibility
 - [ ] change_message_visibility_batch
@@ -5913,13 +5913,13 @@
 - [ ] get_queue_attributes
 - [ ] get_queue_url
 - [X] list_dead_letter_source_queues
-- [ ] list_queue_tags
+- [x] list_queue_tags
 - [X] list_queues
 - [X] purge_queue
 - [ ] receive_message
 - [X] remove_permission
 - [X] send_message
-- [ ] send_message_batch
+- [x] send_message_batch
 - [X] set_queue_attributes
 - [X] tag_queue
 - [X] untag_queue
diff --git a/moto/sqs/responses.py b/moto/sqs/responses.py
index 747fa23637ac..75f121b76574 100644
--- a/moto/sqs/responses.py
+++ b/moto/sqs/responses.py
@@ -242,25 +242,59 @@ def send_message_batch(self):
 
         queue_name = self._get_queue_name()
 
+        try:
+            self.sqs_backend.get_queue(queue_name)
+        except QueueDoesNotExist as e:
+            return self._error('AWS.SimpleQueueService.NonExistentQueue',
+                               e.description)
+
+        if self.querystring.get('Entries'):
+            return self._error('AWS.SimpleQueueService.EmptyBatchRequest',
+                               'There should be at least one SendMessageBatchRequestEntry in the request.')
+
+        entries = {}
+        for key, value in self.querystring.items():
+            match = re.match(r'^SendMessageBatchRequestEntry\.(\d+)\.Id', key)
+            if match:
+                entries[match.group(1)] = {
+                    'Id': value[0],
+                    'MessageBody': self.querystring.get(
+                        'SendMessageBatchRequestEntry.{}.MessageBody'.format(match.group(1)))[0]
+                }
+
+        if any(not re.match(r'^[\w-]{1,80}$', entry['Id']) for entry in entries.values()):
+            return self._error('AWS.SimpleQueueService.InvalidBatchEntryId',
+                               'A batch entry id can only contain alphanumeric characters, '
+                               'hyphens and underscores. It can be at most 80 letters long.')
+
+        body_length = next(
+            (len(entry['MessageBody']) for entry in entries.values() if len(entry['MessageBody']) > 262144),
+            False
+        )
+        if body_length:
+            return self._error('AWS.SimpleQueueService.BatchRequestTooLong',
+                               'Batch requests cannot be longer than 262144 bytes. '
+                               'You have sent {} bytes.'.format(body_length))
+
+        duplicate_id = self._get_first_duplicate_id([entry['Id'] for entry in entries.values()])
+        if duplicate_id:
+            return self._error('AWS.SimpleQueueService.BatchEntryIdsNotDistinct',
+                               'Id {} repeated.'.format(duplicate_id))
+
+        if len(entries) > 10:
+            return self._error('AWS.SimpleQueueService.TooManyEntriesInBatchRequest',
+                               'Maximum number of entries per request are 10. '
+                               'You have sent 11.')
+
         messages = []
-        for index in range(1, 11):
+        for index, entry in entries.items():
             # Loop through looking for messages
-            message_key = 'SendMessageBatchRequestEntry.{0}.MessageBody'.format(
-                index)
-            message_body = self.querystring.get(message_key)
-            if not message_body:
-                # Found all messages
-                break
-
-            message_user_id_key = 'SendMessageBatchRequestEntry.{0}.Id'.format(
-                index)
-            message_user_id = self.querystring.get(message_user_id_key)[0]
             delay_key = 'SendMessageBatchRequestEntry.{0}.DelaySeconds'.format(
                 index)
             delay_seconds = self.querystring.get(delay_key, [None])[0]
             message = self.sqs_backend.send_message(
-                queue_name, message_body[0], delay_seconds=delay_seconds)
-            message.user_id = message_user_id
+                queue_name, entry['MessageBody'], delay_seconds=delay_seconds)
+            message.user_id = entry['Id']
 
             message_attributes = parse_message_attributes(
                 self.querystring, base='SendMessageBatchRequestEntry.{0}.'.format(index))
@@ -273,6 +307,14 @@ def send_message_batch(self):
         template = self.response_template(SEND_MESSAGE_BATCH_RESPONSE)
         return template.render(messages=messages)
 
+    def _get_first_duplicate_id(self, ids):
+        unique_ids = set()
+        for id in ids:
+            if id in unique_ids:
+                return id
+            unique_ids.add(id)
+        return None
+
     def delete_message(self):
         queue_name = self._get_queue_name()
         receipt_handle = self.querystring.get("ReceiptHandle")[0]
@@ -397,6 +439,20 @@ def tag_queue(self):
         queue_name = self._get_queue_name()
         tags = self._get_map_prefix('Tag', key_end='.Key', value_end='.Value')
 
+        try:
+            self.sqs_backend.get_queue(queue_name)
+        except QueueDoesNotExist as e:
+            return self._error('AWS.SimpleQueueService.NonExistentQueue',
+                               e.description)
+
+        if len(tags) == 0:
+            return self._error('MissingParameter',
+                               'The request must contain the parameter Tags.')
+
+        if len(tags) > 50:
+            return self._error('InvalidParameterValue',
+                               'Too many tags added for queue {}.'.format(queue_name))
+
         self.sqs_backend.tag_queue(queue_name, tags)
 
         template = self.response_template(TAG_QUEUE_RESPONSE)
@@ -418,7 +474,11 @@ def untag_queue(self):
     def list_queue_tags(self):
         queue_name = self._get_queue_name()
 
-        queue = self.sqs_backend.get_queue(queue_name)
+        try:
+            queue = self.sqs_backend.get_queue(queue_name)
+        except QueueDoesNotExist as e:
+            return self._error('AWS.SimpleQueueService.NonExistentQueue',
+                               e.description)
 
         template = self.response_template(LIST_QUEUE_TAGS_RESPONSE)
         return template.render(tags=queue.tags)
diff --git a/tests/test_sqs/test_sqs.py b/tests/test_sqs/test_sqs.py
index bc9fa8e4d335..1ad2e1a80972 100644
--- a/tests/test_sqs/test_sqs.py
+++ b/tests/test_sqs/test_sqs.py
@@ -882,6 +882,127 @@ def test_delete_message_after_visibility_timeout():
     assert new_queue.count() == 0
 
 
+@mock_sqs
+def test_send_message_batch_errors():
+    client = boto3.client('sqs', region_name = 'us-east-1')
+
+    response = client.create_queue(QueueName='test-queue-with-tags')
+    queue_url = response['QueueUrl']
+
+    client.send_message_batch.when.called_with(
+        QueueUrl=queue_url + '-not-existing',
+        Entries=[
+            {
+                'Id': 'id_1',
+                'MessageBody': 'body_1'
+            }
+        ]
+    ).should.throw(
+        ClientError,
+        'The specified queue does not exist for this wsdl version.'
+    )
+
+    client.send_message_batch.when.called_with(
+        QueueUrl=queue_url,
+        Entries=[]
+    ).should.throw(
+        ClientError,
+        'There should be at least one SendMessageBatchRequestEntry in the request.'
+    )
+
+    client.send_message_batch.when.called_with(
+        QueueUrl=queue_url,
+        Entries=[
+            {
+                'Id': '',
+                'MessageBody': 'body_1'
+            }
+        ]
+    ).should.throw(
+        ClientError,
+        'A batch entry id can only contain alphanumeric characters, '
+        'hyphens and underscores. It can be at most 80 letters long.'
+    )
+
+    client.send_message_batch.when.called_with(
+        QueueUrl=queue_url,
+        Entries=[
+            {
+                'Id': '.!@#$%^&*()+=',
+                'MessageBody': 'body_1'
+            }
+        ]
+    ).should.throw(
+        ClientError,
+        'A batch entry id can only contain alphanumeric characters, '
+        'hyphens and underscores. It can be at most 80 letters long.'
+    )
+
+    client.send_message_batch.when.called_with(
+        QueueUrl=queue_url,
+        Entries=[
+            {
+                'Id': 'i' * 81,
+                'MessageBody': 'body_1'
+            }
+        ]
+    ).should.throw(
+        ClientError,
+        'A batch entry id can only contain alphanumeric characters, '
+        'hyphens and underscores. It can be at most 80 letters long.'
+    )
+
+    client.send_message_batch.when.called_with(
+        QueueUrl=queue_url,
+        Entries=[
+            {
+                'Id': 'id_1',
+                'MessageBody': 'b' * 262145
+            }
+        ]
+    ).should.throw(
+        ClientError,
+        'Batch requests cannot be longer than 262144 bytes. '
+        'You have sent 262145 bytes.'
+    )
+
+    # only the first duplicated Id is reported
+    client.send_message_batch.when.called_with(
+        QueueUrl=queue_url,
+        Entries=[
+            {
+                'Id': 'id_1',
+                'MessageBody': 'body_1'
+            },
+            {
+                'Id': 'id_2',
+                'MessageBody': 'body_2'
+            },
+            {
+                'Id': 'id_2',
+                'MessageBody': 'body_2'
+            },
+            {
+                'Id': 'id_1',
+                'MessageBody': 'body_1'
+            }
+        ]
+    ).should.throw(
+        ClientError,
+        'Id id_2 repeated.'
+    )
+
+    entries = [{'Id': 'id_{}'.format(i), 'MessageBody': 'body_{}'.format(i)} for i in range(11)]
+    client.send_message_batch.when.called_with(
+        QueueUrl=queue_url,
+        Entries=entries
+    ).should.throw(
+        ClientError,
+        'Maximum number of entries per request are 10. '
+        'You have sent 11.'
+    )
+
+
 @mock_sqs
 def test_batch_change_message_visibility():
     if os.environ.get('TEST_SERVER_MODE', 'false').lower() == 'true':
@@ -987,6 +1108,73 @@ def test_tags():
     })
 
 
+@mock_sqs
+def test_list_queue_tags_errors():
+    client = boto3.client('sqs', region_name='us-east-1')
+
+    response = client.create_queue(
+        QueueName='test-queue-with-tags',
+        tags={
+            'tag_key_1': 'tag_value_X'
+        }
+    )
+    queue_url = response['QueueUrl']
+
+    client.list_queue_tags.when.called_with(
+        QueueUrl=queue_url + '-not-existing',
+    ).should.throw(
+        ClientError,
+        'The specified queue does not exist for this wsdl version.'
+    )
+
+
+@mock_sqs
+def test_tag_queue_errors():
+    client = boto3.client('sqs', region_name='us-east-1')
+
+    response = client.create_queue(
+        QueueName='test-queue-with-tags',
+        tags={
+            'tag_key_1': 'tag_value_X'
+        }
+    )
+    queue_url = response['QueueUrl']
+
+    client.tag_queue.when.called_with(
+        QueueUrl=queue_url + '-not-existing',
+        Tags={
+            'tag_key_1': 'tag_value_1'
+        }
+    ).should.throw(
+        ClientError,
+        'The specified queue does not exist for this wsdl version.'
+    )
+
+    client.tag_queue.when.called_with(
+        QueueUrl=queue_url,
+        Tags={}
+    ).should.throw(
+        ClientError,
+        'The request must contain the parameter Tags.'
+    )
+
+    too_many_tags = {'tag_key_{}'.format(i): 'tag_value_{}'.format(i) for i in range(51)}
+    client.tag_queue.when.called_with(
+        QueueUrl=queue_url,
+        Tags=too_many_tags
+    ).should.throw(
+        ClientError,
+        'Too many tags added for queue test-queue-with-tags.'
+    )
+
+    # when the request fails, the tags should not be updated
+    client.list_queue_tags(QueueUrl=queue_url)['Tags'].should.equal(
+        {
+            'tag_key_1': 'tag_value_X'
+        }
+    )
+
+
 @mock_sqs
 def test_untag_queue_errors():
     client = boto3.client('sqs', region_name='us-east-1')
@@ -1006,7 +1194,7 @@ def test_untag_queue_errors():
         ]
     ).should.throw(
         ClientError,
-        "The specified queue does not exist for this wsdl version."
+        'The specified queue does not exist for this wsdl version.'
     )
 
     client.untag_queue.when.called_with(


Code review comments:
diff_hunk: @@ -397,6 +439,20 @@ def tag_queue(self):
         queue_name = self._get_queue_name()
         tags = self._get_map_prefix('Tag', key_end='.Key', value_end='.Value')
 
+        try:
+            self.sqs_backend.get_queue(queue_name)
+        except QueueDoesNotExist as e:
path: moto/sqs/responses.py
text: @user1:
Ditto on if this is needed.

diff_hunk: @@ -397,6 +439,20 @@ def tag_queue(self):
         queue_name = self._get_queue_name()
         tags = self._get_map_prefix('Tag', key_end='.Key', value_end='.Value')
 
+        try:
+            self.sqs_backend.get_queue(queue_name)
+        except QueueDoesNotExist as e:
+            return self._error('AWS.SimpleQueueService.NonExistentQueue',
+                               e.description)
+
+        if len(tags) == 0:
path: moto/sqs/responses.py
text: @user1:
Make this `if not len(tags):`

@author:
Done

diff_hunk: @@ -242,25 +242,59 @@ def send_message_batch(self):
 
         queue_name = self._get_queue_name()
 
+        try:
+            self.sqs_backend.get_queue(queue_name)
+        except QueueDoesNotExist as e:
path: moto/sqs/responses.py
text: @user1:
Is this needed? If it raises a `QueueDoesNotExist`, that should have all the info required to format the exception to the client.

Unless of course this API is different than the others?

@author:
The problem is the general implementation of the `QueueDoesNotExist` exception. It results in an 503 error while running moto in server mode. I would need to change it to a `RESTError` then the handling would be right, I think.

@user1:
Thank You -- that makes sense.