internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: getmoto/moto

Issue:
DescribeInstances: Add block_device_mapping to XML
DescribeInstances currently has a basic "rootDeviceName" in the XML response, and a basic "block_device_mapping" on the instance object, but the "blockDeviceMapping" in the XML is empty.



Discussion:

Resolving patch:
diff --git a/moto/ec2/models.py b/moto/ec2/models.py
--- a/moto/ec2/models.py
+++ b/moto/ec2/models.py
@@ -263,7 +263,7 @@ def __init__(self, image_id, user_data, security_groups, **kwargs):
         self.key_name = kwargs.get("key_name")
 
         self.block_device_mapping = BlockDeviceMapping()
-        self.block_device_mapping['/dev/sda1'] = BlockDeviceType()
+        self.block_device_mapping['/dev/sda1'] = BlockDeviceType(volume_id=random_volume_id())
 
         amis = ec2_backend.describe_images(filters={'image-id': image_id})
         ami = amis[0] if amis else None
diff --git a/moto/ec2/responses/instances.py b/moto/ec2/responses/instances.py
--- a/moto/ec2/responses/instances.py
+++ b/moto/ec2/responses/instances.py
@@ -340,7 +340,17 @@ def _dot_value_instance_attribute_handler(self):
                     <kernelId>{{ instance.kernel }}</kernelId>
                     <rootDeviceType>ebs</rootDeviceType>
                     <rootDeviceName>/dev/sda1</rootDeviceName>
-                    <blockDeviceMapping />
+                    <blockDeviceMapping>
+                      <item>
+                        <deviceName>/dev/sda1</deviceName>
+                          <ebs>
+                            <volumeId>{{ instance.block_device_mapping['/dev/sda1'].volume_id }}</volumeId>
+                            <status>attached</status>
+                            <attachTime>YYYY-MM-DDTHH:MM:SS.SSSZ</attachTime>
+                            <deleteOnTermination>true</deleteOnTermination>
+                        </ebs>
+                      </item>
+                    </blockDeviceMapping>
                     <virtualizationType>{{ instance.virtualization_type }}</virtualizationType>
                     <clientToken>ABCDE1234567890123</clientToken>
                     <tagSet>


Test patch:
diff --git a/tests/test_ec2/test_instances.py b/tests/test_ec2/test_instances.py
--- a/tests/test_ec2/test_instances.py
+++ b/tests/test_ec2/test_instances.py
@@ -50,6 +50,10 @@ def test_instance_launch_and_terminate():
     instances[0].id.should.equal(instance.id)
     instances[0].state.should.equal('running')
 
+    root_device_name = instances[0].root_device_name
+    instances[0].block_device_mapping[root_device_name].status.should.equal('attached')
+    instances[0].block_device_mapping[root_device_name].volume_id.should.match(r'vol-\w+')
+
     conn.terminate_instances([instances[0].id])
 
     reservations = conn.get_all_instances()
Repository: getmoto/moto

Issue:
Multiple region support
We'd like to support multiple regions.  For example, an SNS instance could pass messages to SQS instances located in different regions.



Discussion:
It's frustrating that this _almost_ works, but at least _some_ calls in the second region just use `moto.ec2.models.ec2_backend`, which is not region-specific.


Resolving patch:
diff --git a/moto/ec2/responses/subnets.py b/moto/ec2/responses/subnets.py
--- a/moto/ec2/responses/subnets.py
+++ b/moto/ec2/responses/subnets.py
@@ -2,7 +2,6 @@
 from jinja2 import Template
 
 from moto.core.responses import BaseResponse
-from moto.ec2.models import ec2_backend
 from moto.ec2.utils import filters_from_querystring
 
 
@@ -10,19 +9,19 @@ class Subnets(BaseResponse):
     def create_subnet(self):
         vpc_id = self.querystring.get('VpcId')[0]
         cidr_block = self.querystring.get('CidrBlock')[0]
-        subnet = ec2_backend.create_subnet(vpc_id, cidr_block)
+        subnet = self.ec2_backend.create_subnet(vpc_id, cidr_block)
         template = Template(CREATE_SUBNET_RESPONSE)
         return template.render(subnet=subnet)
 
     def delete_subnet(self):
         subnet_id = self.querystring.get('SubnetId')[0]
-        subnet = ec2_backend.delete_subnet(subnet_id)
+        subnet = self.ec2_backend.delete_subnet(subnet_id)
         template = Template(DELETE_SUBNET_RESPONSE)
         return template.render(subnet=subnet)
 
     def describe_subnets(self):
         filters = filters_from_querystring(self.querystring)
-        subnets = ec2_backend.get_all_subnets(filters)
+        subnets = self.ec2_backend.get_all_subnets(filters)
         template = Template(DESCRIBE_SUBNETS_RESPONSE)
         return template.render(subnets=subnets)
 
diff --git a/moto/ec2/responses/vpcs.py b/moto/ec2/responses/vpcs.py
--- a/moto/ec2/responses/vpcs.py
+++ b/moto/ec2/responses/vpcs.py
@@ -2,27 +2,26 @@
 from jinja2 import Template
 
 from moto.core.responses import BaseResponse
-from moto.ec2.models import ec2_backend
 from moto.ec2.utils import filters_from_querystring, vpc_ids_from_querystring
 
 
 class VPCs(BaseResponse):
     def create_vpc(self):
         cidr_block = self.querystring.get('CidrBlock')[0]
-        vpc = ec2_backend.create_vpc(cidr_block)
+        vpc = self.ec2_backend.create_vpc(cidr_block)
         template = Template(CREATE_VPC_RESPONSE)
         return template.render(vpc=vpc)
 
     def delete_vpc(self):
         vpc_id = self.querystring.get('VpcId')[0]
-        vpc = ec2_backend.delete_vpc(vpc_id)
+        vpc = self.ec2_backend.delete_vpc(vpc_id)
         template = Template(DELETE_VPC_RESPONSE)
         return template.render(vpc=vpc)
 
     def describe_vpcs(self):
         vpc_ids = vpc_ids_from_querystring(self.querystring)
         filters = filters_from_querystring(self.querystring)
-        vpcs = ec2_backend.get_all_vpcs(vpc_ids=vpc_ids, filters=filters)
+        vpcs = self.ec2_backend.get_all_vpcs(vpc_ids=vpc_ids, filters=filters)
         template = Template(DESCRIBE_VPCS_RESPONSE)
         return template.render(vpcs=vpcs)
 


Test patch:
Repository: getmoto/moto

Issue:
Python 2.6 issues with wheels and dependencies
My Travis tests failed on Python 2.6: piskvorky/smart_open#15 .

After some digging around it appears this is because of `moto`. Moto apparently depends on some `ordereddict` package, but that package is not installed (nor mentioned anywhere in the docs, AFAICS).

Do you think you could make `ordereddict` a dependency for moto, when installing on Python 2.6?

In other words, after I successfully run `pip install moto`, I'd expect moto to work, even on Python 2.6.

And thanks for the great package!



Discussion:
This should already be the case as far as I know: https://github.com/spulec/moto/blame/master/setup.py#L18-21

In that case, I don't know why it fails, but it does.

I just checked: compare the Travis fail on
https://travis-ci.org/piskvorky/smart_open/builds/57924387 (no explicit `pip install ordereddict`)
vs. Travis pass on
https://travis-ci.org/piskvorky/smart_open/builds/57924974 (does an explicit `pip install ordereddict`)

The only difference between the two is installing `ordereddict`: https://github.com/piskvorky/smart_open/pull/17/files

Any ideas?

Okay, I have confirmed the issue. It looks like something to do with the wheel distribution files. If you install with `pip install moto --no-use-wheel` it will work. I'm going to keep digging into the root cause.


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -13,11 +13,10 @@
     "werkzeug",
 ]
 
-import sys
-
-if sys.version_info < (2, 7):
-    # No buildint OrderedDict before 2.7
-    install_requires.append('ordereddict')
+extras_require = {
+    # No builtin OrderedDict before 2.7
+    ':python_version=="2.6"': ['ordereddict'],
+}
 
 setup(
     name='moto',
@@ -34,6 +33,7 @@
     },
     packages=find_packages(exclude=("tests", "tests.*")),
     install_requires=install_requires,
+    extras_require=extras_require,
     license="Apache",
     test_suite="tests",
     classifiers=[


Test patch:
Repository: getmoto/moto

Issue:
SQS MD5 Hashing Issues
I've started using Moto as a standalone server to aid testing a PHP stack.  I've discovered that once I create a message which contains encapsulated (escaped) JSON - it starts to fail with the AWS PHP SDK, although it works fine with Boto2.

The issue appears to be in and around the calculation of the MD5 sum.  I suspect Boto must be ignoring the MD5, otherwise I'd think it would be reporting the same problem.

The exception I get from PHP SDK:

```
PHP Warning:  Uncaught Aws\Sqs\Exception\SqsException: AWS Error Code: , Status Code: , AWS Request ID: , AWS Error Type: , AWS Error Message: Body MD5 mismatch for array (
  'MessageId' => '97f171c9-b7a5-b764-f3e0-4234555f509f',
  'ReceiptHandle' => 'nntoxkpevzvvbvbvylufszndstdeplilaxnckhsceeztjvmdqtzpxptfoeyndfgscncydyntjilbppbgsrwlldsjpksxklybpayijnoewirfexullvcdtmbvuablunaykrqomudptfmnznriseoegwopnaxidtwwsmoikjndpaxilaicgcbpisdpt',
  'MD5OfBody' => '08ab38f810e137a6cce4990c3952be77',
  'Body' => '{
```

Trying to reproduce that MD5 of the body using the same body contents from a json file:

PHP:

```
php > $body = file_get_contents(__DIR__ . '/test.json');
php > echo md5($body);
6d8dc937d72f4cdfad4b76be545dda6b
```

Python:

```
>>> import hashlib
>>> with open('git_src/api/data/sqs/ses/temp_bounce.json') as myfile:
...     data=myfile.read()
>>> hashlib.md5(data).hexdigest()
'6d8dc937d72f4cdfad4b76be545dda6b'
>>> from xml.sax.saxutils import escape
>>> hashlib.md5(escape(data).encode('utf-8')).hexdigest()
'08ab38f810e137a6cce4990c3952be77'
```

So it seems the XML escaping is causing the problem.  Before I put together a PR I'll confirm how the real AWS SQS service calculates this MD5.



Discussion:

Resolving patch:
diff --git a/moto/sqs/models.py b/moto/sqs/models.py
--- a/moto/sqs/models.py
+++ b/moto/sqs/models.py
@@ -34,7 +34,7 @@ def __init__(self, message_id, body):
     @property
     def md5(self):
         body_md5 = hashlib.md5()
-        body_md5.update(self.body.encode('utf-8'))
+        body_md5.update(self._body.encode('utf-8'))
         return body_md5.hexdigest()
 
     @property


Test patch:
Repository: getmoto/moto

Issue:
DescribeVolumes fails on moto 0.4.20 with "NotImplementedError: The filter 'volume-id' for DescribeVolumes has not been implemented in Moto yet" message
The volume-id filter seems to work fine on moto 0.4.19



Discussion:
Solved by passing volume_ids as a non-filter arg to ec2_conn.get_all_volumes()...

```
volumes = ec2_conn.get_all_volumes(volume_ids=volume_ids)
```

...which replaced the code that doesn't play nicely with moto 0.4.20...

```
volumes = ec2_conn.get_all_volumes(filters={'volume_id': volume_ids})
```


Resolving patch:
diff --git a/moto/ec2/models.py b/moto/ec2/models.py
--- a/moto/ec2/models.py
+++ b/moto/ec2/models.py
@@ -1453,6 +1453,9 @@ def get_filter_value(self, filter_name):
         if filter_name == 'status':
             return self.status
 
+        if filter_name == 'volume-id':
+            return self.id
+
         filter_value = super(Volume, self).get_filter_value(filter_name)
 
         if filter_value is None:


Test patch:
diff --git a/tests/test_ec2/test_elastic_block_store.py b/tests/test_ec2/test_elastic_block_store.py
--- a/tests/test_ec2/test_elastic_block_store.py
+++ b/tests/test_ec2/test_elastic_block_store.py
@@ -95,6 +95,9 @@ def test_volume_filters():
     volumes_by_status = conn.get_all_volumes(filters={'status': 'in-use'})
     set([vol.id for vol in volumes_by_status]).should.equal(set([block_mapping.volume_id]))
 
+    volumes_by_id = conn.get_all_volumes(filters={'volume-id': volume1.id})
+    set([vol.id for vol in volumes_by_id]).should.equal(set([volume1.id]))
+
     volumes_by_tag_key = conn.get_all_volumes(filters={'tag-key': 'testkey1'})
     set([vol.id for vol in volumes_by_tag_key]).should.equal(set([volume1.id]))
Repository: getmoto/moto

Issue:
Fix S3 issues with botocore 1.3.29
botocore 1.3.29 breaks s3 in tests



Discussion:

Resolving patch:
diff --git a/moto/__init__.py b/moto/__init__.py
--- a/moto/__init__.py
+++ b/moto/__init__.py
@@ -31,3 +31,13 @@
 from .sts import mock_sts  # flake8: noqa
 from .route53 import mock_route53  # flake8: noqa
 from .swf import mock_swf  # flake8: noqa
+
+
+try:
+    # Need to monkey-patch botocore requests back to underlying urllib3 classes
+    from botocore.awsrequest import HTTPSConnectionPool, HTTPConnectionPool, HTTPConnection, VerifiedHTTPSConnection
+except ImportError:
+    pass
+else:
+    HTTPSConnectionPool.ConnectionCls = VerifiedHTTPSConnection
+    HTTPConnectionPool.ConnectionCls = HTTPConnection


Test patch:
Repository: getmoto/moto

Issue:
botocore.exceptions.ConnectionClosedError raised when calling change_resource_record_sets (Boto3)
I am not sure whether or not I should expect this to work, but I see there are currently similar tests in moto against boto so I thought I would inquire.

When using the Route53 client from boto3, a call to the change_resource_record_set method raises a botocore.exceptions.ConnectionClosedError. (botocore.exceptions.ConnectionClosedError: Connection was closed before we received a valid response from endpoint URL: "https://route53.amazonaws.com/2013-04-01/hostedzone/cc11c883/rrset/")

A test case to reproduce is below.

``` python
import boto3
import uuid
from moto import mock_route53

def guid():
    return str(uuid.uuid4())

@mock_route53
def test_route53_rrset_fail():

    client = boto3.client('route53')

    # Create a new zone
    zone_name = '{0}.com'.format(guid())
    zone = client.create_hosted_zone(
        Name=zone_name,
        CallerReference=guid(),
        HostedZoneConfig={'Comment': guid()}
    )
    zone_id = zone['HostedZone']['Id']

    # Verify the zone is retrievable
    z = client.get_hosted_zone(Id=zone_id)
    assert z['HostedZone']['Id'] == zone_id

    # Try to create a record set
    # Raises botocore.exceptions.ConnectionClosedError
    client.change_resource_record_sets(
        HostedZoneId=zone_id,
        ChangeBatch={
            'Comment': guid(),
            'Changes': [{
                'Action': 'CREATE',
                'ResourceRecordSet': {
                    'Name': 'foo.{0}'.format(zone_name),
                    'Type': 'A',
                    'ResourceRecords': [{'Value': '1.2.3.4'}]
                }
            }]
        }
    )
```



Discussion:

Resolving patch:
diff --git a/moto/route53/urls.py b/moto/route53/urls.py
--- a/moto/route53/urls.py
+++ b/moto/route53/urls.py
@@ -8,7 +8,7 @@
 url_paths = {
     '{0}hostedzone$': responses.list_or_create_hostzone_response,
     '{0}hostedzone/[^/]+$': responses.get_or_delete_hostzone_response,
-    '{0}hostedzone/[^/]+/rrset$': responses.rrset_response,
+    '{0}hostedzone/[^/]+/rrset/?$': responses.rrset_response,
     '{0}healthcheck': responses.health_check_response,
     '{0}tags|trafficpolicyinstances/*': responses.not_implemented_response,
 }


Test patch:
Repository: getmoto/moto

Issue:
Unable to create a key with a trailing slash using OrdinaryCallingFormat
When using OrdinaryCallingFormat, it's not possible to create a key ending with a slash (e.g. when mimicking directory creation), since this is stripped off when parsing the key name. I can't comment on S3, but this is at least different behaviour from Ceph.

For example, the below fails as is, but works if the connection uses SubdomainCallingFormat instead.

```
import boto
import moto
import unittest


class TestCreatingKeyEndingWithSlash(unittest.TestCase):

    @moto.mock_s3
    def test_ordinary_calling_format(self):
        bucket_name = 'testbucket'
        key_name = 'key_ending_with_slash/'

        conn = boto.connect_s3('access_key', 'secret_key',
                               calling_format=boto.s3.connection.OrdinaryCallingFormat())
        bucket = conn.create_bucket(bucket_name)

        key = boto.s3.key.Key(bucket)
        key.key = key_name
        key.set_contents_from_string('')

        self.assertIn(key_name, [k.name for k in bucket.get_all_keys()])
```



Discussion:

Resolving patch:
diff --git a/moto/s3bucket_path/utils.py b/moto/s3bucket_path/utils.py
--- a/moto/s3bucket_path/utils.py
+++ b/moto/s3bucket_path/utils.py
@@ -12,7 +12,7 @@ def bucket_name_from_url(url):
 
 
 def parse_key_name(path):
-    return "/".join(path.rstrip("/").split("/")[2:])
+    return "/".join(path.split("/")[2:])
 
 
 def is_delete_keys(request, path, bucket_name):


Test patch:
diff --git a/tests/test_s3/test_s3.py b/tests/test_s3/test_s3.py
--- a/tests/test_s3/test_s3.py
+++ b/tests/test_s3/test_s3.py
@@ -1076,3 +1076,20 @@ def test_website_configuration_xml():
     bucket = conn.create_bucket('test-bucket')
     bucket.set_website_configuration_xml(TEST_XML)
     bucket.get_website_configuration_xml().should.equal(TEST_XML)
+
+
+@mock_s3
+def test_key_with_trailing_slash_in_ordinary_calling_format():
+    conn = boto.connect_s3(
+        'access_key',
+        'secret_key',
+        calling_format=boto.s3.connection.OrdinaryCallingFormat()
+    )
+    bucket = conn.create_bucket('test_bucket_name')
+
+    key_name = 'key_with_slash/'
+
+    key = Key(bucket, key_name)
+    key.set_contents_from_string('some value')
+
+    [k.name for k in bucket.get_all_keys()].should.contain(key_name)
Repository: getmoto/moto

Issue:
DynamoDB update_item() is fragile
```
self._dc.update_item(
    TableName="FakeTable",
    Key={"obj_id": {"S": "identifier"}},
    UpdateExpression='SET mode=READONLY',
)
```

works but

```
self._dc.update_item(
    TableName="FakeTable",
    Key={"obj_id": {"S": "identifier"}},
    UpdateExpression='SET mode = READONLY',
    # Added spaces here       ^ ^
)
```

Does not.  This is a glitch in the expression parser here:
https://github.com/spulec/moto/blob/master/moto/dynamodb2/models.py#L130
Which treats spaces as boundaries in a weird way.



Discussion:

Resolving patch:
diff --git a/moto/dynamodb2/responses.py b/moto/dynamodb2/responses.py
--- a/moto/dynamodb2/responses.py
+++ b/moto/dynamodb2/responses.py
@@ -419,6 +419,12 @@ def update_item(self):
         expression_attribute_names = self.body.get('ExpressionAttributeNames', {})
         expression_attribute_values = self.body.get('ExpressionAttributeValues', {})
         existing_item = dynamodb_backend2.get_item(name, key)
+
+        # Support spaces between operators in an update expression
+        # E.g. `a = b + c` -> `a=b+c`
+        if update_expression:
+            update_expression = re.sub('\s*([=\+-])\s*', '\\1', update_expression)
+
         item = dynamodb_backend2.update_item(name, key, update_expression, attribute_updates, expression_attribute_names, expression_attribute_values)
 
         item_dict = item.to_json()


Test patch:
diff --git a/tests/test_dynamodb2/test_dynamodb_table_with_range_key.py b/tests/test_dynamodb2/test_dynamodb_table_with_range_key.py
--- a/tests/test_dynamodb2/test_dynamodb_table_with_range_key.py
+++ b/tests/test_dynamodb2/test_dynamodb_table_with_range_key.py
@@ -1267,6 +1267,39 @@ def test_update_item_add_value_does_not_exist_is_created():
     })
 
 
+@mock_dynamodb2
+def test_update_item_with_expression():
+    table = _create_table_with_range_key()
+
+    table.put_item(Item={
+        'forum_name': 'the-key',
+        'subject': '123',
+        'field': '1'
+    })
+
+    item_key = {'forum_name': 'the-key', 'subject': '123'}
+
+    table.update_item(
+        Key=item_key,
+        UpdateExpression='SET field=2',
+    )
+    dict(table.get_item(Key=item_key)['Item']).should.equal({
+        'field': '2',
+        'forum_name': 'the-key',
+        'subject': '123',
+    })
+
+    table.update_item(
+        Key=item_key,
+        UpdateExpression='SET field  = 3',
+    )
+    dict(table.get_item(Key=item_key)['Item']).should.equal({
+        'field': '3',
+        'forum_name': 'the-key',
+        'subject': '123',
+    })
+
+
 @mock_dynamodb2
 def test_boto3_query_gsi_range_comparison():
     table = _create_table_with_range_key()
Repository: getmoto/moto

Issue:
Metadata being returned with some keys with wrong format
**Problem description**
When using AWS Java SDK against Moto an issue was uncovered. Metadata is being returned with keys like "Etag" when they should be returned with a specific format (in this example, "ETag").

**Symptoms**
The above mentioned problem leads the Java SDK to generate an NPE as it's counting with a header called "ETag" and not "Etag".

**Steps to reproduce**
- Download https://github.com/awslabs/aws-java-sample.
- Change the sample to use Moto Server by setting the endpoint. Locally, it would be something like ```s3.setEndpoint("http://127.0.0.1:5000");```
- Start Moto Server at ```127.0.0.1:5000```
- Run the sample

**Output**
```
Exception in thread "main" java.lang.NullPointerException
	at com.amazonaws.util.BinaryUtils.fromHex(BinaryUtils.java:61)
	at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1450)
	at com.amazonaws.samples.S3Sample.main(S3Sample.java:107)
```



Discussion:

Resolving patch:
diff --git a/moto/s3/models.py b/moto/s3/models.py
--- a/moto/s3/models.py
+++ b/moto/s3/models.py
@@ -91,7 +91,7 @@ def metadata(self):
     @property
     def response_dict(self):
         res = {
-            'Etag': self.etag,
+            'ETag': self.etag,
             'last-modified': self.last_modified_RFC1123,
             'content-length': str(len(self.value)),
         }


Test patch:
diff --git a/tests/test_s3/test_server.py b/tests/test_s3/test_server.py
--- a/tests/test_s3/test_server.py
+++ b/tests/test_s3/test_server.py
@@ -34,6 +34,7 @@ def test_s3_server_bucket_create():
     res = test_client.put(
         '/bar', 'http://foobaz.localhost:5000/', data='test value')
     res.status_code.should.equal(200)
+    assert 'ETag' in dict(res.headers)
 
     res = test_client.get('/bar', 'http://foobaz.localhost:5000/')
     res.status_code.should.equal(200)