Repository: boto/botocore
Pull request #2762 (2022-09-16 03:03:35)

aws_is_virtual_hostable_bucket endpoint provider standard library function
Implements the `aws.isVirtualHostableS3Bucket` standard library function for endpoint provider rulesets. This is a small addition to #2737.

The spec for the function is:

> Evaluates whether a string is a DNS compatible bucket name and can be used with virtual hosted style addressing. In addition to the restrictions defined in [RFC 1123](https://www.ietf.org/rfc/rfc1123.txt) and `isValidHostLabel`, the `isVirtualHostableS3Bucket` function must also validate that the bucket name is between [3,63] characters, does not contain upper case characters, and is not formatted as an IP address.

codecov-commenter: # [Codecov](https://codecov.io/gh/boto/botocore/pull/2762?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) Report
> Merging [#2762](https://codecov.io/gh/boto/botocore/pull/2762?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) (7afdca8) into [develop](https://codecov.io/gh/boto/botocore/commit/5c9aac2112a0d14ff53e6ac011330173c682dba4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) (5c9aac2) will **decrease** coverage by `0.05%`.
> The diff coverage is `100.00%`.

```diff
@@             Coverage Diff             @@
##           develop    #2762      +/-   ##
===========================================
- Coverage    95.24%   95.19%   -0.06%     
===========================================
  Files           62       62              
  Lines        12845    12851       +6     
===========================================
- Hits         12234    12233       -1     
- Misses         611      618       +7     
```


| [Impacted Files](https://codecov.io/gh/boto/botocore/pull/2762?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto) | Coverage Δ | |
|---|---|---|
| [botocore/endpoint\_provider.py](https://codecov.io/gh/boto/botocore/pull/2762/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvZW5kcG9pbnRfcHJvdmlkZXIucHk=) | `99.67% <100.00%> (+<0.01%)` | :arrow_up: |
| [botocore/compat.py](https://codecov.io/gh/boto/botocore/pull/2762/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvY29tcGF0LnB5) | `91.33% <0.00%> (-1.34%)` | :arrow_down: |
| [botocore/httpsession.py](https://codecov.io/gh/boto/botocore/pull/2762/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvaHR0cHNlc3Npb24ucHk=) | `91.33% <0.00%> (-0.40%)` | :arrow_down: |
| [botocore/utils.py](https://codecov.io/gh/boto/botocore/pull/2762/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvdXRpbHMucHk=) | `93.37% <0.00%> (-0.22%)` | :arrow_down: |
| [botocore/credentials.py](https://codecov.io/gh/boto/botocore/pull/2762/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto#diff-Ym90b2NvcmUvY3JlZGVudGlhbHMucHk=) | `98.54% <0.00%> (-0.11%)` | :arrow_down: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=boto)


Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index d0894355e..521a14760 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -7,6 +7,7 @@ exclude: "\
     botocore/compat.py|\
     docs/|\
     tests/unit/auth/aws4_testsuite|\
+    tests/unit/data/endpoints/|\
     tests/unit/response_parsing/xml|\
     CHANGELOG.rst\
     )"
diff --git a/botocore/endpoint_provider.py b/botocore/endpoint_provider.py
index 750b4d45b..a7042001f 100644
--- a/botocore/endpoint_provider.py
+++ b/botocore/endpoint_provider.py
@@ -29,7 +29,7 @@ from string import Formatter
 from typing import NamedTuple
 
 from botocore import xform_name
-from botocore.compat import quote, urlparse
+from botocore.compat import IPV4_RE, quote, urlparse
 from botocore.exceptions import EndpointResolutionError
 from botocore.utils import (
     ArnParser,
@@ -377,6 +377,37 @@ class RuleSetStandardLibary:
         """
         return not value
 
+    def aws_is_virtual_hostable_s3_bucket(self, value, allow_subdomains):
+        """Evaluates whether a value is a valid bucket name for virtual host
+        style bucket URLs. To pass, the value must meet the following criteria:
+        1. is_valid_host_label(value) is True
+        2. length between 3 and 63 characters (inclusive)
+        3. does not contain uppercase characters
+        4. is not formatted as an IP address
+
+        If allow_subdomains is True, split on `.` and validate
+        each component separately.
+
+        :type value: str
+        :type allow_subdomains: bool
+        :rtype: bool
+        """
+        if (
+            value is None
+            or len(value) < 3
+            or value.lower() != value
+            or IPV4_RE.match(value) is not None
+        ):
+            return False
+
+        if allow_subdomains is True:
+            return all(
+                self.aws_is_virtual_hostable_s3_bucket(label, False)
+                for label in value.split(".")
+            )
+
+        return self.is_valid_host_label(value, allow_subdomains=False)
+
 
 class BaseRule:
     """Base interface for individual endpoint rules."""


Test patch:
diff --git a/tests/unit/data/endpoints/test-cases/is-virtual-hostable-s3-bucket.json b/tests/unit/data/endpoints/test-cases/is-virtual-hostable-s3-bucket.json
new file mode 100644
index 000000000..9641304ac
--- /dev/null
+++ b/tests/unit/data/endpoints/test-cases/is-virtual-hostable-s3-bucket.json
@@ -0,0 +1,121 @@
+{
+  "version": "1.0",
+  "testCases": [
+    {
+      "documentation": "bucket-name:  isVirtualHostable",
+      "params": {
+        "BucketName": "bucket-name"
+      },
+      "expect": {
+        "endpoint": {
+          "url": "https://bucket-name.s3.amazonaws.com"
+        }
+      }
+    },
+    {
+      "documentation": "bucket-with-number-1: isVirtualHostable",
+      "params": {
+        "BucketName": "bucket-with-number-1"
+      },
+      "expect": {
+        "endpoint": {
+          "url": "https://bucket-with-number-1.s3.amazonaws.com"
+        }
+      }
+    },
+    {
+      "documentation": "BucketName: not isVirtualHostable (uppercase characters)",
+      "params": {
+        "BucketName": "BucketName"
+      },
+      "expect": {
+        "error": "not isVirtualHostableS3Bucket"
+      }
+    },
+    {
+      "documentation": "bucket_name: not isVirtualHostable (underscore)",
+      "params": {
+        "BucketName": "bucket_name"
+      },
+      "expect": {
+        "error": "not isVirtualHostableS3Bucket"
+      }
+    },
+    {
+      "documentation": "bucket.name: isVirtualHostable (http only)",
+      "params": {
+        "BucketName": "bucket.name"
+      },
+      "expect": {
+        "endpoint": {
+          "url": "http://bucket.name.s3.amazonaws.com"
+        }
+      }
+    },
+    {
+      "documentation": "bucket.name.multiple.dots1: isVirtualHostable (http only)",
+      "params": {
+        "BucketName": "bucket.name.multiple.dots1"
+      },
+      "expect": {
+        "endpoint": {
+          "url": "http://bucket.name.multiple.dots1.s3.amazonaws.com"
+        }
+      }
+    },
+    {
+      "documentation": "-bucket-name: not isVirtualHostable (leading dash)",
+      "params": {
+        "BucketName": "-bucket-name"
+      },
+      "expect": {
+        "error": "not isVirtualHostableS3Bucket"
+      }
+    },
+    {
+      "documentation": "bucket-name-: not isVirtualHostable (trailing dash)",
+      "params": {
+        "BucketName": "bucket-name-"
+      },
+      "expect": {
+        "error": "not isVirtualHostableS3Bucket"
+      }
+    },
+    {
+      "documentation": "aa: not isVirtualHostable (< 3 characters)",
+      "params": {
+        "BucketName": "aa"
+      },
+      "expect": {
+        "error": "not isVirtualHostableS3Bucket"
+      }
+    },
+    {
+      "documentation": "'a'*64: not isVirtualHostable (> 63 characters)",
+      "params": {
+        "BucketName": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
+      },
+      "expect": {
+        "error": "not isVirtualHostableS3Bucket"
+      }
+    },
+    {
+      "documentation": ".bucket-name: not isVirtualHostable (leading dot)",
+      "params": {
+        "BucketName": ".bucket-name"
+      },
+      "expect": {
+        "error": "not isVirtualHostableS3Bucket"
+      }
+    },
+    {
+      "documentation": "bucket-name.: not isVirtualHostable (trailing dot)",
+      "params": {
+        "BucketName": "bucket-name."
+      },
+      "expect": {
+        "error": "not isVirtualHostableS3Bucket"
+      }
+    }
+  ]
+}
\ No newline at end of file
diff --git a/tests/unit/data/endpoints/valid-rules/is-virtual-hostable-s3-bucket.json b/tests/unit/data/endpoints/valid-rules/is-virtual-hostable-s3-bucket.json
new file mode 100644
index 000000000..7ece9b534
--- /dev/null
+++ b/tests/unit/data/endpoints/valid-rules/is-virtual-hostable-s3-bucket.json
@@ -0,0 +1,48 @@
+{
+  "version": "1.3",
+  "parameters": {
+    "BucketName": {
+      "type": "string",
+      "required": true,
+      "documentation": "the input used to test isVirtualHostableS3Bucket"
+    }
+  },
+  "rules": [
+    {
+      "conditions": [
+        {
+          "fn": "aws.isVirtualHostableS3Bucket",
+          "argv": [
+            "{BucketName}",
+            false
+          ]
+        }
+      ],
+      "endpoint": {
+        "url": "https://{BucketName}.s3.amazonaws.com"
+      },
+      "type": "endpoint"
+    },
+    {
+      "conditions": [
+        {
+          "fn": "aws.isVirtualHostableS3Bucket",
+          "argv": [
+            "{BucketName}",
+            true
+          ]
+        }
+      ],
+      "endpoint": {
+        "url": "http://{BucketName}.s3.amazonaws.com"
+      },
+      "type": "endpoint"
+    },
+    {
+      "conditions": [
+      ],
+      "error": "not isVirtualHostableS3Bucket",
+      "type": "error"
+    }
+  ]
+}
diff --git a/tests/unit/test_endpoints_v2.py b/tests/unit/test_endpoints_v2.py
index a71e60082..4b5bd8b8b 100644
--- a/tests/unit/test_endpoints_v2.py
+++ b/tests/unit/test_endpoints_v2.py
@@ -135,6 +135,7 @@ def ruleset_testcases():
         "eventbridge",
         "fns",
         "headers",
+        "is-virtual-hostable-s3-bucket",
         "local-region-override",
         "parse-arn",
         "parse-url",