internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: boto/boto

Issue:
Support addtional regions in CloudSearch 
It appears that boto currently only supports two regions. It would be nice to support the more recently added ones. 

Current list of CloudSearch regions is here: http://aws.amazon.com/cloudsearch/faqs/#Which_AWS_regions_is_CloudSearch_available_in?

I think it just needs to be added to boto/cloudsearch/**init**.py:

return [RegionInfo(name='us-east-1',
                       endpoint='cloudsearch.us-east-1.amazonaws.com',
                       connection_cls=boto.cloudsearch.layer1.Layer1),
            RegionInfo(name='eu-west-1',
                       endpoint='cloudsearch.eu-west-1.amazonaws.com',
                       connection_cls=boto.cloudsearch.layer1.Layer1),

And yes, you can use another region by changing the default field in .boto 



Discussion:
Note that the tutorial at http://boto.readthedocs.org/en/latest/cloudsearch_tut.html current uses us-west-2, which doesn't seem to work yet. Works fine with us-east-1.


Resolving patch:
diff --git a/boto/cloudsearch/__init__.py b/boto/cloudsearch/__init__.py
--- a/boto/cloudsearch/__init__.py
+++ b/boto/cloudsearch/__init__.py
@@ -38,6 +38,16 @@ def regions():
             RegionInfo(name='eu-west-1',
                        endpoint='cloudsearch.eu-west-1.amazonaws.com',
                        connection_cls=boto.cloudsearch.layer1.Layer1),
+            RegionInfo(name='us-west-1',
+                       endpoint='cloudsearch.us-west-1.amazonaws.com',
+                       connection_cls=boto.cloudsearch.layer1.Layer1),
+            RegionInfo(name='us-west-2',
+                       endpoint='cloudsearch.us-west-2.amazonaws.com',
+                       connection_cls=boto.cloudsearch.layer1.Layer1),
+            RegionInfo(name='ap-southeast-1',
+                       endpoint='cloudsearch.ap-southeast-1.amazonaws.com',
+                       connection_cls=boto.cloudsearch.layer1.Layer1),
+
             ]
 
 


Test patch:
Repository: boto/boto

Issue:
Setting S3 key common metadata headers isn't case-insensitive
When creating a new key on s3, the following fails with `SignatureDoesNotMatch`:

```
        k = bucket.new_key(path)
        k.set_metadata('Content-type', 'application/json')
        k.set_contents_from_string(datastr)
```

But if I change it to `k.set_metadata('Content-Type', 'application/json')`, it works.  I've always been under the impression that these HTTP headers are case insensitive, and other python utilities seem to treat them as such.  IMO the code above should also work, i.e. boto should put known headers in amazon's canonical form before submitting the request.  Is this a bug or is it by (sadistic) design?



Discussion:
I believe this is a side effect of the signing mechanism in play on S3 (see http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#ConstructingTheAuthenticationHeader). We don't do anything special with any of the headers you provide, save for `Date` AFAICT (https://github.com/boto/boto/blob/develop/boto/auth.py#L138-L170).

I'll see what I can do about adding code to make handling this a bit more graceful.


Resolving patch:
diff --git a/boto/s3/key.py b/boto/s3/key.py
--- a/boto/s3/key.py
+++ b/boto/s3/key.py
@@ -522,7 +522,14 @@ def get_metadata(self, name):
         return self.metadata.get(name)
 
     def set_metadata(self, name, value):
-        self.metadata[name] = value
+        # Ensure that metadata that is vital to signing is in the correct
+        # case. Applies to ``Content-Type`` & ``Content-MD5``.
+        if name.lower() == 'content-type':
+            self.metadata['Content-Type'] = value
+        elif name.lower() == 'content-md5':
+            self.metadata['Content-MD5'] = value
+        else:
+            self.metadata[name] = value
 
     def update_metadata(self, d):
         self.metadata.update(d)


Test patch:
diff --git a/tests/integration/s3/test_key.py b/tests/integration/s3/test_key.py
--- a/tests/integration/s3/test_key.py
+++ b/tests/integration/s3/test_key.py
@@ -383,3 +383,14 @@ def test_setting_date(self):
         check = self.bucket.get_key('test_date')
         self.assertEqual(check.get_metadata('date'), u'20130524T155935Z')
         self.assertTrue('x-amz-meta-date' in check._get_remote_metadata())
+
+    def test_header_casing(self):
+        key = self.bucket.new_key('test_header_case')
+        # Using anything but CamelCase on ``Content-Type`` or ``Content-MD5``
+        # used to cause a signature error (when using ``s3`` for signing).
+        key.set_metadata('Content-type', 'application/json')
+        key.set_metadata('Content-md5', 'XmUKnus7svY1frWsVskxXg==')
+        key.set_contents_from_string('{"abc": 123}')
+
+        check = self.bucket.get_key('test_header_case')
+        self.assertEqual(check.content_type, 'application/json')
Repository: boto/boto

Issue:
Support for dynamodb local
Dynamodb local support was recently announced by amazon:

http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.html

As far as I can tell (and I have been monkey-punching the DynamoDBConnection class for many hours now) there is no way to change the host to point to a locally running instance.  This is understandable as it was never expected that there would be a dynamo service running anywhere but on their servers.

Please add support for this as it will become very useful for development environments.



Discussion:
It looks like `DynamoDBConnection` is a subclass of `AWSQueryConnection`, which in turn takes `host` and `port` keyword parameters. You should be able to do something like this:

``` python
from boto.dynamodb2.layer1 import DynamoDBConnection

conn = DynamoDBConnection(aws_access_key_id='...', aws_secret_access_key='...',
                          host='localhost', port=8000)
```

If this doesn't work please let me know.

Thanks for looking into this!

I just tried to pass in the host and port, and I got the following response:
"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.  The Canonical String for this request should have been\n'POST\n/\n\nhost:dynamodb.us-east-1.amazonaws.com ...".  It looks like it is still trying to post to the default region.

According to the dynamodb local documentation, they ignore the security keys passed, so I tried to pass blank access key / secret, and I got this:

"The security token included in the request is invalid."

Which makes sense.  Let me know if I can pass along any more info.

Ah, you're right. There's a bug in the constructor that always overwrites the host. I'm working on a fix.


Resolving patch:
diff --git a/boto/dynamodb2/layer1.py b/boto/dynamodb2/layer1.py
--- a/boto/dynamodb2/layer1.py
+++ b/boto/dynamodb2/layer1.py
@@ -67,7 +67,11 @@ def __init__(self, **kwargs):
                 if reg.name == region_name:
                     region = reg
                     break
-        kwargs['host'] = region.endpoint
+
+        # Only set host if it isn't manually overwritten
+        if 'host' not in kwargs:
+            kwargs['host'] = region.endpoint
+
         AWSQueryConnection.__init__(self, **kwargs)
         self.region = region
         self._validate_checksums = boto.config.getbool(


Test patch:
diff --git a/tests/unit/dynamodb2/test_layer1.py b/tests/unit/dynamodb2/test_layer1.py
--- a/tests/unit/dynamodb2/test_layer1.py
+++ b/tests/unit/dynamodb2/test_layer1.py
@@ -43,3 +43,11 @@ def test_init_region(self):
             aws_secret_access_key='aws_secret_access_key',
         )
         self.assertEqual(dynamodb.region.name, 'us-west-2')
+
+    def test_init_host_override(self):
+        dynamodb = DynamoDBConnection(
+            aws_access_key_id='aws_access_key_id',
+            aws_secret_access_key='aws_secret_access_key',
+            host='localhost', port=8000)
+        self.assertEqual(dynamodb.host, 'localhost')
+        self.assertEqual(dynamodb.port, 8000)
Repository: boto/boto

Issue:
attrs parameter to Item.__init__ and table.new_item don't work as documented
In dynamodb/table.py, `new_item` is documented to take an `attrs` parameter which will be used to populate the new item.

However, this doesn't work:
`
    In [3]: item = table.new_item('key', attrs={"a":"test"})
    In [4]: item.save()
    Out[4]: {'ConsumedCapacityUnits': 1.0}
    In [5]: read = table.get_item('key')
    In [6]: print read
    {'key': 'key'}
`

The bug is in item.py in `__init__` - `self._updates` is `None` until after the `attrs` dictionary is copied - but `__setitem__` doesn't update the `_updates` dictionary if it is None.



Discussion:

Resolving patch:
diff --git a/boto/dynamodb/item.py b/boto/dynamodb/item.py
--- a/boto/dynamodb/item.py
+++ b/boto/dynamodb/item.py
@@ -50,11 +50,11 @@ def __init__(self, table, hash_key=None, range_key=None, attrs=None):
             if range_key == None:
                 range_key = attrs.get(self._range_key_name, None)
             self[self._range_key_name] = range_key
+        self._updates = {}
         for key, value in attrs.items():
             if key != self._hash_key_name and key != self._range_key_name:
                 self[key] = value
         self.consumed_units = 0
-        self._updates = {}
 
     @property
     def hash_key(self):


Test patch:
diff --git a/tests/integration/dynamodb/test_layer2.py b/tests/integration/dynamodb/test_layer2.py
--- a/tests/integration/dynamodb/test_layer2.py
+++ b/tests/integration/dynamodb/test_layer2.py
@@ -160,7 +160,7 @@ def test_layer2_basic(self):
                                     consistent_read=True)
         assert item1_copy.hash_key == item1.hash_key
         assert item1_copy.range_key == item1.range_key
-        for attr_name in item1_copy:
+        for attr_name in item1_attrs:
             val = item1_copy[attr_name]
             if isinstance(val, (int, long, float, basestring)):
                 assert val == item1[attr_name]
Repository: boto/boto

Issue:
Documentation Issue?
I found that the documentation here:

http://boto.readthedocs.org/en/latest/ref/cloudwatch.html#boto.ec2.cloudwatch.alarm.MetricAlarm

Says 'dimensions' takes a list of dicts. Doing so I ran into issues creating an alarm:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/jenkins/.virtualenvs/warmachine/local/lib/python2.7/site-packages/boto/ec2/cloudwatch/__init__.py", line 529, in put_metric_alarm
    self.build_dimension_param(alarm.dimensions, params)
  File "/jenkins/.virtualenvs/warmachine/local/lib/python2.7/site-packages/boto/ec2/cloudwatch/__init__.py", line 126, in build_dimension_param
    dim_value = dimension[dim_name]
TypeError: list indices must be integers, not dict
```

From looking at the source code it takes a dict. This seems to check out with the rest of the documentation on the page linked above.

Lastly the example seems really bad. It looks like:

Dimensions of alarm, such as: [{‘InstanceId’:[‘i-0123456,i-0123457’]}]

Which IMO should read:

Dimensions of alarm, such as: [{‘InstanceId’:‘i-0123456,i-0123457’}]

Making it much more explicit that it is a string containing the values. With instance id you can only specify one value, showing a list doesn't make this very explicit... one might assume you could pass a second string with more instances.



Discussion:
It does look like this doc string is different form the rest. I'll make a quick fix - thanks :+1: 


Resolving patch:
diff --git a/boto/ec2/cloudwatch/alarm.py b/boto/ec2/cloudwatch/alarm.py
--- a/boto/ec2/cloudwatch/alarm.py
+++ b/boto/ec2/cloudwatch/alarm.py
@@ -112,9 +112,16 @@ def __init__(self, connection=None, name=None, metric=None,
         :type description: str
         :param description: Description of MetricAlarm
 
-        :type dimensions: list of dicts
-        :param dimensions: Dimensions of alarm, such as:
-                            [{'InstanceId':['i-0123456,i-0123457']}]
+        :type dimensions: dict
+        :param dimensions: A dictionary of dimension key/values where
+                           the key is the dimension name and the value
+                           is either a scalar value or an iterator
+                           of values to be associated with that
+                           dimension.
+                           Example: {
+                               'InstanceId': ['i-0123456', 'i-0123457'],
+                               'LoadBalancerName': 'test-lb'
+                           }
         
         :type alarm_actions: list of strs
         :param alarm_actions: A list of the ARNs of the actions to take in


Test patch:
Repository: boto/boto

Issue:
s3 bucket operation should validate kwarg params
In bucket operations that take `**params`, there's no validation to ensure the keys passed are valid.  This makes it hard to debug if you, for example, have a typo in a kwarg parameter:

```
# typo delimeter instead of delimiter
bucket.get_all_keys(delimeter='/', prefix='foo/bar/')
```



Discussion:

Resolving patch:
diff --git a/boto/s3/bucket.py b/boto/s3/bucket.py
--- a/boto/s3/bucket.py
+++ b/boto/s3/bucket.py
@@ -342,6 +342,11 @@ def _get_all(self, element_map, initial_query_string='',
             raise self.connection.provider.storage_response_error(
                 response.status, response.reason, body)
 
+    def _validate_kwarg_names(self, kwargs, names):
+        for kwarg in kwargs:
+            if kwarg not in names:
+                raise TypeError('Invalid argument %s!' % kwarg)
+
     def get_all_keys(self, headers=None, **params):
         """
         A lower-level method for listing contents of a bucket.  This
@@ -371,6 +376,8 @@ def get_all_keys(self, headers=None, **params):
         :return: The result from S3 listing the keys requested
 
         """
+        self._validate_kwarg_names(params, ['maxkeys', 'max_keys', 'prefix',
+                                            'marker', 'delimiter'])
         return self._get_all([('Contents', self.key_class),
                               ('CommonPrefixes', Prefix)],
                              '', headers, **params)
@@ -408,6 +415,9 @@ def get_all_versions(self, headers=None, **params):
         :rtype: ResultSet
         :return: The result from S3 listing the keys requested
         """
+        self._validate_kwarg_names(params, ['maxkeys', 'max_keys', 'prefix',
+                                            'key_marker', 'version_id_marker',
+                                            'delimiter'])
         return self._get_all([('Version', self.key_class),
                               ('CommonPrefixes', Prefix),
                               ('DeleteMarker', DeleteMarker)],
@@ -451,6 +461,8 @@ def get_all_multipart_uploads(self, headers=None, **params):
         :return: The result from S3 listing the uploads requested
 
         """
+        self._validate_kwarg_names(params, ['max_uploads', 'key_marker',
+                                            'upload_id_marker'])
         return self._get_all([('Upload', MultiPartUpload),
                               ('CommonPrefixes', Prefix)],
                              'uploads', headers, **params)


Test patch:
diff --git a/tests/unit/s3/test_bucket.py b/tests/unit/s3/test_bucket.py
--- a/tests/unit/s3/test_bucket.py
+++ b/tests/unit/s3/test_bucket.py
@@ -51,6 +51,13 @@ def test_bucket_delete_key_missing_name(self):
         with self.assertRaises(ValueError):
             key = bucket.delete_key('')
 
+    def test_bucket_kwargs_misspelling(self):
+        self.set_http_response(status_code=200)
+        bucket = self.service_connection.create_bucket('mybucket')
+
+        with self.assertRaises(TypeError):
+            bucket.get_all_keys(delimeter='foo')
+
     def test__get_all_query_args(self):
         bukket = Bucket()
Repository: boto/boto

Issue:
setup.py and cheeseshop releases do not list dependencies  
The boto release packages do not enumerate dependencies in setup.py, so they are not installed if you easy_install or pip install boto.  As it is, the cloudsearch module is not functional out-of-the-box due to missing dependencies after install (requires requests in some version range)

I notice that you have a requirements.txt, which is useful, but is there any reason the set needed for deployment is not also listed setup.py in the "install_requires" / "tests_require" sections as most packages do? 



Discussion:
I agree with @ltucker on this one. It would be nice to have them clearly stated somewhere.

This was done because an original goal of boto was to lean on as few external dependencies. Because there's a big install base, just adding a bunch of deps (that most people won't use) isn't a great idea.

That said, we'll work up a way for all the dependencies to be explicitly listed & installable, perhaps with optional dependencies/an `extra_requirements.txt`/docs/etc.


Resolving patch:
diff --git a/boto/cloudsearch/document.py b/boto/cloudsearch/document.py
--- a/boto/cloudsearch/document.py
+++ b/boto/cloudsearch/document.py
@@ -191,12 +191,9 @@ def commit(self):
         session = requests.Session()
         adapter = requests.adapters.HTTPAdapter(
             pool_connections=20,
-            pool_maxsize=50
+            pool_maxsize=50,
+            max_retries=5
         )
-        # Now kludge in the right number of retries.
-        # Once we're requiring ``requests>=1.2.1``, this can become an
-        # initialization parameter above.
-        adapter.max_retries = 5
         session.mount('http://', adapter)
         session.mount('https://', adapter)
         r = session.post(url, data=sdf, headers={'Content-Type': 'application/json'})


Test patch:
Repository: boto/boto

Issue:
Documentation of security_groups argument doesn't mention VPC
The documentation for the 'security_groups' parameter for a LaunchConfiguration object could be better. As stated on:

http://docs.aws.amazon.com/cli/latest/reference/autoscaling/create-launch-configuration.html

"--security-groups (list)

The names of the security groups with which to associate Amazon EC2 or Amazon VPC instances. Specify Amazon EC2 security groups using security group names, such as websrv . Specify Amazon VPC security groups using security group IDs, such as sg-12345678 ."

In the Boto documentation for creating a launch configuration (http://boto.readthedocs.org/en/latest/ref/autoscale.html#boto.ec2.autoscale.launchconfig.LaunchConfiguration), the comment for the 'security_groups' parameter says:

"Names of the security groups with which to associate the EC2 instances."

In this comment, there is no mention of VPC instances or security group id's as seen in Amazon's own docs. I lost about a day of productivity figuring this out so I'm hoping that the documentation can be improved.



Discussion:

Resolving patch:
diff --git a/boto/ec2/autoscale/launchconfig.py b/boto/ec2/autoscale/launchconfig.py
--- a/boto/ec2/autoscale/launchconfig.py
+++ b/boto/ec2/autoscale/launchconfig.py
@@ -110,8 +110,9 @@ def __init__(self, connection=None, name=None, image_id=None,
         :param key_name: The name of the EC2 key pair.
 
         :type security_groups: list
-        :param security_groups: Names of the security groups with which to
-            associate the EC2 instances.
+        :param security_groups: Names or security group id's of the security
+            groups with which to associate the EC2 instances or VPC instances,
+            respectively.
 
         :type user_data: str
         :param user_data: The user data available to launched EC2 instances.


Test patch:
Repository: boto/boto

Issue:
AttributeError: _strptime (Python issue 7980) when using S3Connection on multiple threads
Provider._populate_keys_from_metadata_server is affected by Python issue 7980 (http://bugs.python.org/issue7980), where an AttributeError can be thrown when datetime.strptime is called from multiple threads at the same time. This can happen when using multiple S3Connection's at the same time on different threads. The workaround listed in the issue is to call datetime.strptime once on the main thread during initialization.

This issue is reproducible using s3funnel (https://github.com/sstoiana/s3funnel) on a Ubuntu 13.10 EC2 instance running Python 2.7.6 and boto 2.19.0.



Discussion:

Resolving patch:
diff --git a/boto/__init__.py b/boto/__init__.py
--- a/boto/__init__.py
+++ b/boto/__init__.py
@@ -27,6 +27,7 @@
 from boto.pyami.config import Config, BotoConfigLocations
 from boto.storage_uri import BucketStorageUri, FileStorageUri
 import boto.plugin
+import datetime
 import os
 import platform
 import re
@@ -39,6 +40,9 @@
 __version__ = '2.21.0'
 Version = __version__  # for backware compatibility
 
+# http://bugs.python.org/issue7980
+datetime.datetime.strptime('', '')
+
 UserAgent = 'Boto/%s Python/%s %s/%s' % (
     __version__,
     platform.python_version(),


Test patch:
diff --git a/tests/unit/utils/test_utils.py b/tests/unit/utils/test_utils.py
--- a/tests/unit/utils/test_utils.py
+++ b/tests/unit/utils/test_utils.py
@@ -26,8 +26,9 @@
 
 import hashlib
 import hmac
-
 import mock
+import thread
+import time
 
 import boto.utils
 from boto.utils import Password
@@ -39,6 +40,21 @@
 
 from boto.compat import json
 
+
+@unittest.skip("http://bugs.python.org/issue7980")
+class TestThreadImport(unittest.TestCase):
+    def test_strptime(self):
+        def f():
+            for m in xrange(1, 13):
+                for d in xrange(1,29):
+                    boto.utils.parse_ts('2013-01-01T00:00:00Z')
+
+        for _ in xrange(10):
+            thread.start_new_thread(f, ())
+
+        time.sleep(3)
+
+
 class TestPassword(unittest.TestCase):
     """Test basic password functionality"""
Repository: boto/boto

Issue:
Binary object's __str__ method returns non string
Depending on the value given to it, the boto.dynamodb.types.Binary object's **str**() function does not always return a valid string.  This makes it cumbersome when printing debug values.

```
>>> from boto.dynamodb.types import Binary
>>> str(Binary(1))
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: __str__ returned non-string (type int)
>>>
>>> Binary(1).__str__()
1
>>> type(Binary(1).__str__())
<type 'int'>
```



Discussion:

Resolving patch:
diff --git a/boto/dynamodb/types.py b/boto/dynamodb/types.py
--- a/boto/dynamodb/types.py
+++ b/boto/dynamodb/types.py
@@ -136,6 +136,9 @@ def dynamize_value(val):
 
 class Binary(object):
     def __init__(self, value):
+        if not isinstance(value, basestring):
+            raise TypeError('Value must be a string of binary data!')
+
         self.value = value
 
     def encode(self):


Test patch:
diff --git a/tests/unit/dynamodb/test_types.py b/tests/unit/dynamodb/test_types.py
--- a/tests/unit/dynamodb/test_types.py
+++ b/tests/unit/dynamodb/test_types.py
@@ -78,5 +78,17 @@ def test_lossy_float_conversions(self):
         self.assertEqual(dynamizer.decode({'NS': ['1.1', '2.2', '3.3']}),
                          set([1.1, 2.2, 3.3]))
 
+
+class TestBinary(unittest.TestCase):
+    def test_bad_input(self):
+        with self.assertRaises(TypeError):
+            data = types.Binary(1)
+
+    def test_good_input(self):
+        data = types.Binary(chr(1))
+
+        self.assertEqual('\x01', str(data))
+
+
 if __name__ == '__main__':
     unittest.main()