Repository: boto/botocore
Pull request #2811 (2022-11-14 21:40:04)

Fix: S3 on Outpost requests miss x-amz-content-sha256 headers
Addresses an issue where requests for some S3 API operations that should include the `x-amz-content-sha256` header do not have this header for S3 on Outpost. This issue was introduced in version 1.28.0 of botocore.

codecov-commenter: # [Codecov](https://codecov.io/gh/boto/botocore/pull/2811?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) Report
Base: **93.45**% // Head: **93.42**% // Decreases project coverage by **`-0.02%`** :warning:
> Coverage data is based on head [(`2f2b1ac`)](https://codecov.io/gh/boto/botocore/pull/2811?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) compared to base [(`15b62f7`)](https://codecov.io/gh/boto/botocore/commit/15b62f7348a0a9f6110cfee835afbe1bab9977cd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           develop    #2811      +/-   ##
===========================================
- Coverage    93.45%   93.42%   -0.03%     
===========================================
  Files           63       63              
  Lines        13298    13298              
===========================================
- Hits         12427    12424       -3     
- Misses         871      874       +3     
```


| [Impacted Files](https://codecov.io/gh/boto/botocore/pull/2811?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) | Coverage Δ | |
|---|---|---|
| [botocore/handlers.py](https://codecov.io/gh/boto/botocore/pull/2811/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvaGFuZGxlcnMucHk=) | `96.43% <100.00%> (ø)` | |
| [botocore/compat.py](https://codecov.io/gh/boto/botocore/pull/2811/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvY29tcGF0LnB5) | `91.33% <0.00%> (-1.34%)` | :arrow_down: |
| [botocore/utils.py](https://codecov.io/gh/boto/botocore/pull/2811/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvdXRpbHMucHk=) | `78.40% <0.00%> (-0.07%)` | :arrow_down: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/boto/botocore/pull/2811?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto).


Patch:
diff --git a/.changes/next-release/bugfix-s3-62528.json b/.changes/next-release/bugfix-s3-62528.json
new file mode 100644
index 000000000..59fa1d26b
--- /dev/null
+++ b/.changes/next-release/bugfix-s3-62528.json
@@ -0,0 +1,5 @@
+{
+  "type": "bugfix",
+  "category": "s3",
+  "description": "fixes missing x-amz-content-sha256 header for s3 on outpost"
+}
diff --git a/botocore/handlers.py b/botocore/handlers.py
index 30a3d3358..18607750b 100644
--- a/botocore/handlers.py
+++ b/botocore/handlers.py
@@ -219,7 +219,7 @@ def set_operation_specific_signer(context, signing_name, **kwargs):
             context['payload_signing_enabled'] = False
 
         # s3 and s3-control have customized signers "s3v4" and "s3v4a".
-        if signing_name == 's3':
+        if signing_name in ('s3', 's3-outposts'):
             signature_version = f's3{signature_version}'
 
         return signature_version


Test patch:
diff --git a/tests/functional/test_s3.py b/tests/functional/test_s3.py
index 66f77e7d0..a20081d3e 100644
--- a/tests/functional/test_s3.py
+++ b/tests/functional/test_s3.py
@@ -716,6 +716,9 @@ class TestAccesspointArn(BaseS3ClientConfigurationTest):
             "s3-outposts.us-west-2.amazonaws.com"
         )
         self.assert_endpoint(request, expected_endpoint)
+        sha_header = request.headers.get("x-amz-content-sha256")
+        self.assertIsNotNone(sha_header)
+        self.assertNotEqual(sha_header, b"UNSIGNED-PAYLOAD")
 
     def test_basic_outpost_arn_custom_endpoint(self):
         outpost_arn = (
@@ -1402,6 +1405,10 @@ class TestS3SigV4(BaseS3OperationTest):
         self.assertIn("content-md5", self.get_sent_headers())
 
     def test_content_sha256_set_if_config_value_is_true(self):
+        # By default, put_object() does not include an x-amz-content-sha256
+        # header because it also includes a `Content-MD5` header. The
+        # `payload_signing_enabled` config overrides this logic and forces the
+        # header.
         config = Config(
             signature_version="s3v4", s3={"payload_signing_enabled": True}
         )
@@ -1431,6 +1438,56 @@ class TestS3SigV4(BaseS3OperationTest):
         sha_header = sent_headers.get("x-amz-content-sha256")
         self.assertEqual(sha_header, b"UNSIGNED-PAYLOAD")
 
+    def test_content_sha256_set_if_config_value_not_set_put_object(self):
+        # The default behavior matches payload_signing_enabled=False. For
+        # operations where the `Content-MD5` is present this means that
+        # `x-amz-content-sha256` is present but not set.
+        config = Config(signature_version="s3v4")
+        self.client = self.session.create_client(
+            "s3", self.region, config=config
+        )
+        self.http_stubber = ClientHTTPStubber(self.client)
+        self.http_stubber.add_response()
+        with self.http_stubber:
+            self.client.put_object(Bucket="foo", Key="bar", Body="baz")
+        sent_headers = self.get_sent_headers()
+        sha_header = sent_headers.get("x-amz-content-sha256")
+        self.assertEqual(sha_header, b"UNSIGNED-PAYLOAD")
+
+    def test_content_sha256_set_if_config_value_not_set_list_objects(self):
+        # The default behavior matches payload_signing_enabled=False. For
+        # operations where the `Content-MD5` is not present, this means that
+        # `x-amz-content-sha256` is present and set.
+        config = Config(signature_version="s3v4")
+        self.client = self.session.create_client(
+            "s3", self.region, config=config
+        )
+        self.http_stubber = ClientHTTPStubber(self.client)
+        self.http_stubber.add_response()
+        with self.http_stubber:
+            self.client.list_objects(Bucket="foo")
+        sent_headers = self.get_sent_headers()
+        sha_header = sent_headers.get("x-amz-content-sha256")
+        self.assertIsNotNone(sha_header)
+        self.assertNotEqual(sha_header, b"UNSIGNED-PAYLOAD")
+
+    def test_content_sha256_set_s3_on_outpost(self):
+        # S3 on Outpost bucket names should behave the same way.
+        config = Config(signature_version="s3v4")
+        bucket = (
+            'test-accessp-e0000075431d83bebde8xz5w8ijx1qzlbp3i3kuse10--op-s3'
+        )
+        self.client = self.session.create_client(
+            "s3", self.region, config=config
+        )
+        self.http_stubber = ClientHTTPStubber(self.client)
+        self.http_stubber.add_response()
+        with self.http_stubber:
+            self.client.list_objects(Bucket=bucket)
+        sent_headers = self.get_sent_headers()
+        sha_header = sent_headers.get("x-amz-content-sha256")
+        self.assertNotEqual(sha_header, b"UNSIGNED-PAYLOAD")
+
     def test_content_sha256_set_if_md5_is_unavailable(self):
         with mock.patch("botocore.compat.MD5_AVAILABLE", False):
             with mock.patch("botocore.utils.MD5_AVAILABLE", False):