Repository: getmoto/moto
Pull request #7459 (2024-03-11 22:45:04)

Bump softprops/action-gh-release from 1 to 2
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 1 to 2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/softprops/action-gh-release/releases">softprops/action-gh-release's releases</a>.</em></p>
<blockquote>
<h2>v2.0.0</h2>
<ul>
<li>update actions.yml declaration to node20 to address warnings</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md">softprops/action-gh-release's changelog</a>.</em></p>
<blockquote>
<h2>0.1.12</h2>
<ul>
<li>fix bug leading to empty strings subsituted for inputs users don't provide breaking api calls <a href="https://redirect.github.com/softprops/action-gh-release/pull/144">#144</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/softprops/action-gh-release/commit/3198ee18f814cdf787321b4a32a26ddbf37acc52"><code>3198ee1</code></a> prep release</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/7ee8e0638151a48bb421b567d7d53a07cb158121"><code>7ee8e06</code></a> declare an update docs for make_latest input (<a href="https://redirect.github.com/softprops/action-gh-release/issues/419">#419</a>)</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/d99959edae48b5ffffd7b00da66dcdb0a33a52ee"><code>d99959e</code></a> prep release</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/0e39c679e8bac647fe5ad41070fe2295b0393d58"><code>0e39c67</code></a> make pattern error opt in (<a href="https://redirect.github.com/softprops/action-gh-release/issues/417">#417</a>)</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/20e085ccc73308c2c8e43ab8da4f8d7ecbb94d4e"><code>20e085c</code></a> kick off 2.0.1 release</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/9f5c4d39bc56b5ed07f92f260dd05d33000058a6"><code>9f5c4d3</code></a> update changelog</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/0bea76b22707a1871a52b97e07c80657071adb76"><code>0bea76b</code></a> Add support for make_latest property (<a href="https://redirect.github.com/softprops/action-gh-release/issues/304">#304</a>)</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/762fe13dfbc8921c36ebb81da5626a394a6a5b95"><code>762fe13</code></a> typo in changelog</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/c0f2569fb90b35a0e2b043535a5778f437ec86ea"><code>c0f2569</code></a> remove unused script</li>
<li><a href="https://github.com/softprops/action-gh-release/commit/5743e060246b253faba6cb8e051511b63e7639c9"><code>5743e06</code></a> update changelog</li>
<li>Additional commits viewable in <a href="https://github.com/softprops/action-gh-release/compare/v1...v2">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softprops/action-gh-release&package-manager=github_actions&previous-version=1&new-version=2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>


Patch:
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 5033b5fee..f7801a2ff 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -64,7 +64,7 @@ jobs:
         git tag ${{ env.VERSION }}
         git push origin ${{ env.VERSION }}
     - name: Create GitHub release
-      uses: softprops/action-gh-release@v1
+      uses: softprops/action-gh-release@v2
       with:
         name: ${{ env.VERSION }}
         tag_name: ${{ env.VERSION }}
diff --git a/moto/core/botocore_stubber.py b/moto/core/botocore_stubber.py
index bb248d182..d1637ad82 100644
--- a/moto/core/botocore_stubber.py
+++ b/moto/core/botocore_stubber.py
@@ -1,15 +1,14 @@
 import re
 from io import BytesIO
 from typing import Any, Optional, Union
-from urllib.parse import urlparse
 
 from botocore.awsrequest import AWSResponse
 
 import moto.backend_index as backend_index
-from moto import settings
 from moto.core.base_backend import BackendDict
 from moto.core.common_types import TYPE_RESPONSE
 from moto.core.config import passthrough_service, passthrough_url
+from moto.core.utils import get_equivalent_url_in_aws_domain
 
 
 class MockRawResponse(BytesIO):
@@ -43,30 +42,11 @@ class BotocoreStubber:
             return response
 
     def process_request(self, request: Any) -> Optional[TYPE_RESPONSE]:
+        # Handle non-standard AWS endpoint hostnames from ISO regions or custom
+        # S3 endpoints.
+        parsed_url, _ = get_equivalent_url_in_aws_domain(request.url)
         # Remove the querystring from the URL, as we'll never match on that
-        x = urlparse(request.url)
-        host = x.netloc
-
-        # https://github.com/getmoto/moto/pull/6412
-        # Support ISO regions
-        iso_region_domains = [
-            "amazonaws.com.cn",
-            "c2s.ic.gov",
-            "sc2s.sgov.gov",
-            "cloud.adc-e.uk",
-            "csp.hci.ic.gov",
-        ]
-        for domain in iso_region_domains:
-            if host.endswith(domain):
-                host = host.replace(domain, "amazonaws.com")
-
-        # https://github.com/getmoto/moto/issues/2993
-        # Support S3-compatible tools (Ceph, Digital Ocean, etc)
-        for custom_endpoint in settings.get_s3_custom_endpoints():
-            if host == custom_endpoint or host == custom_endpoint.split("://")[-1]:
-                host = "s3.amazonaws.com"
-
-        clean_url = f"{x.scheme}://{host}{x.path}"
+        clean_url = f"{parsed_url.scheme}://{parsed_url.netloc}{parsed_url.path}"
 
         if passthrough_url(clean_url):
             return None
diff --git a/moto/core/responses_custom_registry.py b/moto/core/responses_custom_registry.py
index 135e47c8e..64c9422df 100644
--- a/moto/core/responses_custom_registry.py
+++ b/moto/core/responses_custom_registry.py
@@ -1,10 +1,12 @@
 # This will only exist in responses >= 0.17
 from collections import defaultdict
 from typing import Any, Dict, List, Optional, Tuple
+from urllib.parse import urlunparse
 
 import responses
 
 from .custom_responses_mock import CallbackResponse, not_implemented_callback
+from .utils import get_equivalent_url_in_aws_domain
 
 
 class CustomRegistry(responses.registries.FirstMatchRegistry):
@@ -41,8 +43,20 @@ class CustomRegistry(responses.registries.FirstMatchRegistry):
         )
         found = []
         match_failed_reasons = []
+
+        # Handle non-standard AWS endpoint hostnames from ISO regions or custom S3 endpoints.
+        parsed_url, url_was_modified = get_equivalent_url_in_aws_domain(request.url)
+        if url_was_modified:
+            url_with_standard_aws_domain = urlunparse(parsed_url)
+            request_with_standard_aws_domain = request.copy()
+            request_with_standard_aws_domain.prepare_url(
+                url_with_standard_aws_domain, {}
+            )
+        else:
+            request_with_standard_aws_domain = request
+
         for response in all_possibles:
-            match_result, reason = response.matches(request)
+            match_result, reason = response.matches(request_with_standard_aws_domain)
             if match_result:
                 found.append(response)
             else:
diff --git a/moto/core/utils.py b/moto/core/utils.py
index 7735e30a7..5a8d32bb3 100644
--- a/moto/core/utils.py
+++ b/moto/core/utils.py
@@ -3,10 +3,11 @@ import inspect
 import re
 from gzip import decompress
 from typing import Any, Callable, Dict, List, Optional, Tuple
-from urllib.parse import urlparse
+from urllib.parse import ParseResult, urlparse
 
 from botocore.exceptions import ClientError
 
+from ..settings import get_s3_custom_endpoints
 from .common_types import TYPE_RESPONSE
 from .versions import PYTHON_311
 
@@ -398,3 +399,47 @@ def get_partition_from_region(region_name: str) -> str:
     if region_name.startswith("cn-"):
         return "aws-cn"
     return "aws"
+
+
+def get_equivalent_url_in_aws_domain(url: str) -> Tuple[ParseResult, bool]:
+    """Parses a URL and converts non-standard AWS endpoint hostnames (from ISO
+    regions or custom S3 endpoints) to the equivalent standard AWS domain.
+
+    Returns a tuple: (parsed URL, was URL modified).
+    """
+
+    parsed = urlparse(url)
+    original_host = parsed.netloc
+    host = original_host
+
+    # https://github.com/getmoto/moto/pull/6412
+    # Support ISO regions
+    iso_region_domains = [
+        "amazonaws.com.cn",
+        "c2s.ic.gov",
+        "sc2s.sgov.gov",
+        "cloud.adc-e.uk",
+        "csp.hci.ic.gov",
+    ]
+    for domain in iso_region_domains:
+        if host.endswith(domain):
+            host = host.replace(domain, "amazonaws.com")
+
+    # https://github.com/getmoto/moto/issues/2993
+    # Support S3-compatible tools (Ceph, Digital Ocean, etc)
+    for custom_endpoint in get_s3_custom_endpoints():
+        if host == custom_endpoint or host == custom_endpoint.split("://")[-1]:
+            host = "s3.amazonaws.com"
+
+    if host == original_host:
+        return (parsed, False)
+    else:
+        result = ParseResult(
+            scheme=parsed.scheme,
+            netloc=host,
+            path=parsed.path,
+            params=parsed.params,
+            query=parsed.query,
+            fragment=parsed.fragment,
+        )
+        return (result, True)


Test patch:
diff --git a/tests/test_s3/test_s3_custom_endpoint.py b/tests/test_s3/test_s3_custom_endpoint.py
index e3384f873..b252570df 100644
--- a/tests/test_s3/test_s3_custom_endpoint.py
+++ b/tests/test_s3/test_s3_custom_endpoint.py
@@ -4,6 +4,7 @@ from unittest.mock import patch
 
 import boto3
 import pytest
+import requests
 
 from moto import mock_aws, settings
 
@@ -96,3 +97,27 @@ def test_put_and_list_objects(url):
             contents = s3_client.list_objects(Bucket=bucket)["Contents"]
             assert len(contents) == 3
             assert "two" in [c["Key"] for c in contents]
+
+
+@pytest.mark.parametrize("url", [CUSTOM_ENDPOINT, CUSTOM_ENDPOINT_2])
+def test_get_presigned_url(url):
+    if not settings.TEST_DECORATOR_MODE:
+        raise SkipTest("Unable to set ENV VAR in ServerMode")
+    with patch.dict(os.environ, {"MOTO_S3_CUSTOM_ENDPOINTS": url}):
+        with mock_aws():
+            bucket = "mybucket"
+            key = "file.txt"
+            contents = b"file contents"
+            conn = boto3.resource(
+                "s3", endpoint_url=url, region_name=DEFAULT_REGION_NAME
+            )
+            conn.create_bucket(Bucket=bucket)
+
+            s3_client = boto3.client("s3", endpoint_url=url)
+            s3_client.put_object(Bucket=bucket, Key=key, Body=contents)
+
+            signed_url = s3_client.generate_presigned_url(
+                "get_object", Params={"Bucket": bucket, "Key": key}, ExpiresIn=86400
+            )
+            response = requests.get(signed_url, stream=False)
+            assert contents == response.content