nebius/SWE-rebench-V2-PRs huggingface dataset:
Repository: boto/botocore
Pull request #1060 (2016-10-22 16:14:52)

sigv4 presigned urls should preserve blank params
A unit test and fix for #1059, which is a sigv4 bug related to presigned urls where the base url contains a query parameter without a value, such as '/bucket/key?uploads' for an s3 create_multipart_upload url.

The first commit contains a unit test that fails, and the second commit updates SigV4QueryAuth._modify_request_before_signing to use the keep_blank_values param of parse_qs in order to preserve parameters with no value.



Patch:
diff --git a/.changes/next-release/bugfix-s3-27146.json b/.changes/next-release/bugfix-s3-27146.json
new file mode 100644
index 000000000..bba2a4389
--- /dev/null
+++ b/.changes/next-release/bugfix-s3-27146.json
@@ -0,0 +1,5 @@
+{
+  "category": "s3", 
+  "type": "bugfix", 
+  "description": "fixes `#1059 <https://github.com/boto/botocore/issues/1059>`__ (presigned s3v4 URL bug related to blank query parameters being filtered incorrectly)"
+}
diff --git a/botocore/auth.py b/botocore/auth.py
index a1ecb24b3..480d28c0f 100644
--- a/botocore/auth.py
+++ b/botocore/auth.py
@@ -443,7 +443,8 @@ class SigV4QueryAuth(SigV4Auth):
         # have repeated keys so we know we have single element lists which we
         # can convert back to scalar values.
         query_dict = dict(
-            [(k, v[0]) for k, v in parse_qs(url_parts.query).items()])
+            [(k, v[0]) for k, v in
+             parse_qs(url_parts.query, keep_blank_values=True).items()])
         # The spec is particular about this.  It *has* to be:
         # https://<endpoint>?<operation params>&<auth params>
         # You can't mix the two types of params together, i.e just keep doing


Test patch:
diff --git a/tests/unit/auth/test_signers.py b/tests/unit/auth/test_signers.py
index f084576e1..ec9ac58c6 100644
--- a/tests/unit/auth/test_signers.py
+++ b/tests/unit/auth/test_signers.py
@@ -680,6 +680,15 @@ class TestSigV4Presign(BasePresignTest):
         # Verify we encode spaces as '%20, and we don't use '+'.
         self.assertIn('Description=With%20Spaces', request.url)
 
+    def test_presign_with_empty_param_value(self):
+        request = AWSRequest()
+        request.method = 'POST'
+        # actual URL format for creating a multipart upload
+        request.url = 'https://s3.amazonaws.com/mybucket/mykey?uploads'
+        self.auth.add_auth(request)
+        # verify that uploads param is still in URL
+        self.assertIn('uploads', request.url)
+
     def test_s3_sigv4_presign(self):
         auth = botocore.auth.S3SigV4QueryAuth(
             self.credentials, self.service_name, self.region_name, expires=60)
Repository: boto/botocore
Pull request #1092 (2016-11-23 21:34:49)

Add support for timestamp in ArgumentGenerator
Uses epoch time as a ``datetime`` for its default value. Right now it defaults to ``None`` which is in appropriate representation of the ``timestamp`` type in our models.

This is needed for this PR: https://github.com/aws/aws-cli/pull/2293

cc @jamesls @JordonPhillips @stealthycoin 


Patch:
diff --git a/botocore/utils.py b/botocore/utils.py
index 8c57e694b..d9c49535e 100644
--- a/botocore/utils.py
+++ b/botocore/utils.py
@@ -563,6 +563,8 @@ class ArgumentGenerator(object):
                 return 0.0
             elif shape.type_name == 'boolean':
                 return True
+            elif shape.type_name == 'timestamp':
+                return datetime.datetime(1970, 1, 1, 0, 0, 0)
         finally:
             stack.pop()
 


Test patch:
diff --git a/tests/unit/test_utils.py b/tests/unit/test_utils.py
index e17e05e81..17f30f1a1 100644
--- a/tests/unit/test_utils.py
+++ b/tests/unit/test_utils.py
@@ -371,12 +371,14 @@ class TestArgumentGenerator(unittest.TestCase):
                 'B': {'type': 'integer'},
                 'C': {'type': 'float'},
                 'D': {'type': 'boolean'},
+                'E': {'type': 'timestamp'}
             },
             generated_skeleton={
                 'A': '',
                 'B': 0,
                 'C': 0.0,
                 'D': True,
+                'E': datetime.datetime(1970, 1, 1, 0, 0, 0)
             }
         )
Repository: boto/botocore
Pull request #1094 (2016-11-29 19:51:59)

Use member names in lists for ArgumentGenerator
Use the member name for strings in lists. Catches and edge case pointed out by @jamesls in this PR: https://github.com/aws/aws-cli/pull/2293

cc @jamesls @JordonPhillips @stealthycoin 


Patch:
diff --git a/botocore/utils.py b/botocore/utils.py
index d9c49535e..26e6a877e 100644
--- a/botocore/utils.py
+++ b/botocore/utils.py
@@ -580,8 +580,11 @@ class ArgumentGenerator(object):
     def _generate_type_list(self, shape, stack):
         # For list elements we've arbitrarily decided to
         # return two elements for the skeleton list.
+        name = ''
+        if self._use_member_names:
+            name = shape.member.name
         return [
-            self._generate_skeleton(shape.member, stack),
+            self._generate_skeleton(shape.member, stack, name),
         ]
 
     def _generate_type_map(self, shape, stack):


Test patch:
diff --git a/tests/unit/test_utils.py b/tests/unit/test_utils.py
index 17f30f1a1..c19e304bf 100644
--- a/tests/unit/test_utils.py
+++ b/tests/unit/test_utils.py
@@ -399,6 +399,33 @@ class TestArgumentGenerator(unittest.TestCase):
             }
         )
 
+    def test_will_use_member_names_for_string_values_of_list(self):
+        self.arg_generator = ArgumentGenerator(use_member_names=True)
+        # We're not using assert_skeleton_from_model_is
+        # because we can't really control the name of strings shapes
+        # being used in the DenormalizedStructureBuilder. We can only
+        # control the name of structures and list shapes.
+        shape_map = ShapeResolver({
+            'InputShape': {
+                'type': 'structure',
+                'members': {
+                    'StringList': {'shape': 'StringList'},
+                }
+            },
+            'StringList': {
+                'type': 'list',
+                'member': {'shape': 'StringType'},
+            },
+            'StringType': {
+                'type': 'string',
+            }
+        })
+        shape = shape_map.get_shape_by_name('InputShape')
+        actual = self.arg_generator.generate_skeleton(shape)
+
+        expected = {'StringList': ['StringType']}
+        self.assertEqual(actual, expected)
+
     def test_generate_nested_structure(self):
         self.assert_skeleton_from_model_is(
             model={
Repository: boto/botocore
Pull request #1105 (2016-12-12 16:53:26)

Do not sign trace header
The trace header can be mutated along the way, so it should be blacklisted.

cc @kyleknap @jamesls @stealthycoin


Patch:
diff --git a/.changes/next-release/bugfix-sigv4-897.json b/.changes/next-release/bugfix-sigv4-897.json
new file mode 100644
index 000000000..f645cb68d
--- /dev/null
+++ b/.changes/next-release/bugfix-sigv4-897.json
@@ -0,0 +1,5 @@
+{
+  "type": "bugfix",
+  "category": "sigv4",
+  "description": "Do not sign x-amzn-trace-id as it can be mutated along the way."
+}
diff --git a/botocore/auth.py b/botocore/auth.py
index 6313c8791..f3fd11746 100644
--- a/botocore/auth.py
+++ b/botocore/auth.py
@@ -47,7 +47,8 @@ ISO8601 = '%Y-%m-%dT%H:%M:%SZ'
 SIGV4_TIMESTAMP = '%Y%m%dT%H%M%SZ'
 SIGNED_HEADERS_BLACKLIST = [
     'expect',
-    'user-agent'
+    'user-agent',
+    'x-amzn-trace-id',
 ]
 
 


Test patch:
diff --git a/tests/unit/auth/test_signers.py b/tests/unit/auth/test_signers.py
index c388fd270..b420a53c5 100644
--- a/tests/unit/auth/test_signers.py
+++ b/tests/unit/auth/test_signers.py
@@ -341,6 +341,10 @@ class TestS3SigV4Auth(BaseTestWithFixedDate):
     def test_blacklist_expect_headers(self):
         self._test_blacklist_header('expect', '100-continue')
 
+    def test_blacklist_trace_id(self):
+        self._test_blacklist_header('x-amzn-trace-id',
+                                    'Root=foo;Parent=bar;Sampleid=1')
+
     def test_blacklist_headers(self):
         self._test_blacklist_header('user-agent', 'botocore/1.4.11')
Repository: boto/botocore
Pull request #1111 (2016-12-19 20:26:59)

Add an external_link method
This generates an inlink external link.  This is modeled off of the [sphinx_reference_label](https://github.com/boto/botocore/blob/develop/botocore/docs/bcdoc/style.py#L206)


Patch:
diff --git a/botocore/docs/bcdoc/style.py b/botocore/docs/bcdoc/style.py
index fd5570d4d..df4833327 100644
--- a/botocore/docs/bcdoc/style.py
+++ b/botocore/docs/bcdoc/style.py
@@ -387,3 +387,9 @@ class ReSTStyle(BaseStyle):
         docstring_lines = docstring.splitlines()
         for docstring_line in docstring_lines:
             self.doc.writeln(docstring_line)
+
+    def external_link(self, title, link):
+        if self.doc.target == 'html':
+            self.doc.write('`%s <%s>`_' % (title, link))
+        else:
+            self.doc.write(title)


Test patch:
diff --git a/tests/unit/docs/bcdoc/test_style.py b/tests/unit/docs/bcdoc/test_style.py
index b89e0d310..31ffd35a5 100644
--- a/tests/unit/docs/bcdoc/test_style.py
+++ b/tests/unit/docs/bcdoc/test_style.py
@@ -279,3 +279,17 @@ class TestStyle(unittest.TestCase):
 
         self.assertEqual(style.doc.getvalue(),
                          six.b("\n\n\n* foo\n\n\n  \n  * bar\n  "))
+
+    def test_external_link(self):
+        style = ReSTStyle(ReSTDocument())
+        style.doc.target = 'html'
+        style.external_link('MyLink', 'http://example.com/foo')
+        self.assertEqual(style.doc.getvalue(),
+                         six.b('`MyLink <http://example.com/foo>`_'))
+
+
+    def test_external_link_in_man_page(self):
+        style = ReSTStyle(ReSTDocument())
+        style.doc.target = 'man'
+        style.external_link('MyLink', 'http://example.com/foo')
+        self.assertEqual(style.doc.getvalue(), six.b('MyLink'))