Repository: ansible-collections/community.aws

Issue:
cloudfront_distribution doesn't recognise S3 origin
### Summary

When I refer to an S3 bucket domain in the form `{bucket_name}.s3.{region}.amazonaws.com`, as per [Origin Domain spec](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-values-specify.html#DownloadDistValuesDomainName), it isn't recognised as an S3 domain [here](https://github.com/ansible-collections/community.aws/blob/e80bf933412ea5c7ab2a94af945170cb2ebd900f/plugins/modules/cloudfront_distribution.py#L1821), so a `custom_origin_config` entry is added automatically [here](https://github.com/ansible-collections/community.aws/blob/e80bf933412ea5c7ab2a94af945170cb2ebd900f/plugins/modules/cloudfront_distribution.py#L1842), which results in an error:

"botocore.errorfactory.InvalidOrigin: An error occurred (InvalidOrigin) when calling the CreateDistribution operation: You must specify either a CustomOrigin or an S3Origin. You cannot specify both."

The problem is in the method used for recognising S3 domains: whether it contains `.s3.amazonaws.com` or not (note the missing region part).

### Issue Type

Bug Report

### Component Name

cloudfront_distribution

### Ansible Version

```console (paste below)
$ ansible --version
ansible [core 2.14.5]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/fules/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = /home/fules/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] (/usr/bin/python3)
  jinja version = 3.0.3
  libyaml = True
```


### Collection Versions

```console (paste below)
$ ansible-galaxy collection list
# /home/fules/.ansible/collections/ansible_collections
Collection    Version
------------- -------
amazon.aws    6.0.1
community.aws 6.0.0

# /usr/lib/python3/dist-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    5.4.0
ansible.netcommon             4.1.0
ansible.posix                 1.5.2
ansible.utils                 2.9.0
ansible.windows               1.13.0
arista.eos                    6.0.1
awx.awx                       21.14.0
azure.azcollection            1.15.0
check_point.mgmt              4.0.0
chocolatey.chocolatey         1.4.0
cisco.aci                     2.6.0
cisco.asa                     4.0.0
cisco.dnac                    6.7.1
cisco.intersight              1.0.27
cisco.ios                     4.5.0
cisco.iosxr                   4.1.0
cisco.ise                     2.5.12
cisco.meraki                  2.15.1
cisco.mso                     2.4.0
cisco.nso                     1.0.3
cisco.nxos                    4.3.0
cisco.ucs                     1.8.0
cloud.common                  2.1.3
cloudscale_ch.cloud           2.2.4
community.aws                 5.4.0
community.azure               2.0.0
community.ciscosmb            1.0.5
community.crypto              2.12.0
community.digitalocean        1.23.0
community.dns                 2.5.3
community.docker              3.4.3
community.fortios             1.0.0
community.general             6.6.0
community.google              1.0.0
community.grafana             1.5.4
community.hashi_vault         4.2.0
community.hrobot              1.8.0
community.libvirt             1.2.0
community.mongodb             1.5.2
community.mysql               3.6.0
community.network             5.0.0
community.okd                 2.3.0
community.postgresql          2.3.2
community.proxysql            1.5.1
community.rabbitmq            1.2.3
community.routeros            2.8.0
community.sap                 1.0.0
community.sap_libs            1.4.1
community.skydive             1.0.0
community.sops                1.6.1
community.vmware              3.5.0
community.windows             1.12.0
community.zabbix              1.9.3
containers.podman             1.10.1
cyberark.conjur               1.2.0
cyberark.pas                  1.0.17
dellemc.enterprise_sonic      2.0.0
dellemc.openmanage            6.3.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
dellemc.powerflex             1.6.0
dellemc.unity                 1.6.0
f5networks.f5_modules         1.23.0
fortinet.fortimanager         2.1.7
fortinet.fortios              2.2.3
frr.frr                       2.0.2
gluster.gluster               1.0.2
google.cloud                  1.1.3
grafana.grafana               1.1.1
hetzner.hcloud                1.11.0
hpe.nimble                    1.1.4
ibm.qradar                    2.1.0
ibm.spectrum_virtualize       1.11.0
infinidat.infinibox           1.3.12
infoblox.nios_modules         1.4.1
inspur.ispim                  1.3.0
inspur.sm                     2.3.0
junipernetworks.junos         4.1.0
kubernetes.core               2.4.0
lowlydba.sqlserver            1.3.1
mellanox.onyx                 1.0.0
microsoft.ad                  1.0.0
netapp.aws                    21.7.0
netapp.azure                  21.10.0
netapp.cloudmanager           21.22.0
netapp.elementsw              21.7.0
netapp.ontap                  22.5.0
netapp.storagegrid            21.11.1
netapp.um_info                21.8.0
netapp_eseries.santricity     1.4.0
netbox.netbox                 3.12.0
ngine_io.cloudstack           2.3.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.3
openstack.cloud               1.10.0
openvswitch.openvswitch       2.1.0
ovirt.ovirt                   2.4.1
purestorage.flasharray        1.17.2
purestorage.flashblade        1.11.0
purestorage.fusion            1.4.2
sensu.sensu_go                1.13.2
splunk.es                     2.1.0
t_systems_mms.icinga_director 1.32.2
theforeman.foreman            3.10.0
vmware.vmware_rest            2.3.1
vultr.cloud                   1.7.0
vyos.vyos                     4.0.2
wti.remote                    1.0.4
```


### AWS SDK versions

```console (paste below)
$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.26.137
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.29.137
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.10/dist-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed
CONFIG_FILE() = /etc/ansible/ansible.cfg
```


### OS / Environment

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
-
  hosts: localhost
  collections:
    - community.aws
    - amazon.aws
  vars_files:
    - external_vars.yaml
  tasks:
    -
      name: Reading landing page domain from certificate
      acm_certificate_info:
        profile: "{{ aws_profile }}"
        region: "us-east-1"
        certificate_arn: "{{ landing_page_certificate_arn }}"
      register: landing_page_certificates
    -
      set_fact:
        landing_page_domain: "{{ landing_page_certificates.certificates[0].domain_name }}"
    -
      name: Create the S3 bucket
      s3_bucket:
        profile: "{{ aws_profile }}"
        state: present
        name: "{{ landing_page_domain }}"
        object_ownership: "BucketOwnerEnforced"
        encryption: "AES256"
        versioning: false
        public_access:
          block_public_acls: false
          block_public_policy: false
          ignore_public_acls: false
          restrict_public_buckets: false
      register: landing_page_bucket
    -
      name: Create CloudFront Access Identity
      cloudfront_origin_access_identity:
        state: present
        caller_reference: "LandingPageAccessIdentity"
        comment: "no comment"
      register: landing_page_access_identity
    -
      name: Create CloudFront Distribution
      cloudfront_distribution:
        profile: "{{ aws_profile }}"
        state: present
        http_version: "http2"
        caller_reference: "LandingPageDistribution"
        comment: "no comment"
        alias: "{{ landing_page_domain }}"
        viewer_certificate:
          acm_certificate_arn: "{{ landing_page_certificate_arn }}"
          ssl_support_method: "sni-only"
          minimum_protocol_version: "TLSv1.2_2021"
        origins:
          -
            id: "{{ landing_page_domain }}"
            domain_name: "{{ landing_page_bucket.name }}.s3.{{ aws_region }}.amazonaws.com"
            s3_origin_config:
              origin_access_identity: "origin-access-identity/cloudfront/{{ landing_page_access_identity.cloud_front_origin_access_identity.id }}"
            origin_shield:
              enabled: false
        # default_origin_domain_name: "{{ landing_page_bucket.name }}"
        default_root_object: "index.html"
        price_class: "PriceClass_200"
        wait: true
      register: landing_page_distribution
```


### Expected Results

I expected that only the `s3_origin_config` is generated in the origin, and the `custom_origin_config` isn't.

### Actual Results

```console (paste below)
botocore.errorfactory.InvalidOrigin: An error occurred (InvalidOrigin) when calling the CreateDistribution operation: You must specify either a CustomOrigin or an S3Origin. You cannot specify both.
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:

Resolving patch:
diff --git a/plugins/modules/cloudfront_distribution.py b/plugins/modules/cloudfront_distribution.py
--- a/plugins/modules/cloudfront_distribution.py
+++ b/plugins/modules/cloudfront_distribution.py
@@ -1417,6 +1417,7 @@
 
 from collections import OrderedDict
 import datetime
+import re
 
 try:
     import botocore
@@ -1676,7 +1677,7 @@ def __init__(self, module):
                 "http2and3",
             ]
         )
-        self.__s3_bucket_domain_identifier = ".s3.amazonaws.com"
+        self.__s3_bucket_domain_regex = re.compile(r"\.s3(?:\.[^.]+)?\.amazonaws\.com$")
 
     def add_missing_key(self, dict_object, key_to_set, value_to_set):
         if key_to_set not in dict_object and value_to_set is not None:
@@ -1818,7 +1819,7 @@ def validate_origin(self, client, existing_config, origin, default_origin_path):
                         )
                     else:
                         origin_shield_region = origin_shield_region.lower()
-            if self.__s3_bucket_domain_identifier in origin.get("domain_name").lower():
+            if self.__s3_bucket_domain_regex.search(origin.get("domain_name").lower()):
                 if origin.get("s3_origin_access_identity_enabled") is not None:
                     if origin["s3_origin_access_identity_enabled"]:
                         s3_origin_config = self.validate_s3_origin_configuration(client, existing_config, origin)
@@ -1834,10 +1835,10 @@ def validate_origin(self, client, existing_config, origin, default_origin_path):
 
                     origin["s3_origin_config"] = dict(origin_access_identity=oai)
 
-                    if "custom_origin_config" in origin:
-                        self.module.fail_json(
-                            msg="s3_origin_access_identity_enabled and custom_origin_config are mutually exclusive"
-                        )
+                if "custom_origin_config" in origin:
+                    self.module.fail_json(
+                        msg="s3 origin domains and custom_origin_config are mutually exclusive"
+                    )
             else:
                 origin = self.add_missing_key(
                     origin, "custom_origin_config", existing_config.get("custom_origin_config", {})


Test patch:
diff --git a/tests/integration/targets/cloudfront_distribution/tasks/main.yml b/tests/integration/targets/cloudfront_distribution/tasks/main.yml
--- a/tests/integration/targets/cloudfront_distribution/tasks/main.yml
+++ b/tests/integration/targets/cloudfront_distribution/tasks/main.yml
@@ -568,6 +568,43 @@
     ignore_errors: true
 
   - name: check that custom origin with origin access identity fails
+    # "s3 origin domains and custom_origin_config are mutually exclusive"
+    assert:
+      that:
+        - update_origin_to_s3_with_origin_access_and_with_custom_origin_config.failed
+
+  - name: check that custom_origin_config can't be used with an region-agnostic S3 domain
+    cloudfront_distribution:
+      distribution_id: "{{ distribution_id }}"
+      origins:
+      - domain_name: "{{ resource_prefix }}-bucket.s3.{{ aws_region }}.amazonaws.com"
+        id: "{{ resource_prefix }}3.example.com"
+        custom_origin_config:
+          http_port: 8080
+      state: present
+    register: update_origin_to_s3_with_origin_access_and_with_custom_origin_config
+    ignore_errors: true
+
+  - name: check that custom origin with region-agnostic S3 domain fails
+    # "s3 origin domains and custom_origin_config are mutually exclusive"
+    assert:
+      that:
+        - update_origin_to_s3_with_origin_access_and_with_custom_origin_config.failed
+
+  - name: check that custom_origin_config can't be used with an region-aware S3 domain
+    cloudfront_distribution:
+      distribution_id: "{{ distribution_id }}"
+      origins:
+      - domain_name: "{{ resource_prefix }}-bucket.s3.amazonaws.com"
+        id: "{{ resource_prefix }}3.example.com"
+        custom_origin_config:
+          http_port: 8080
+      state: present
+    register: update_origin_to_s3_with_origin_access_and_with_custom_origin_config
+    ignore_errors: true
+
+  - name: check that custom origin with region-aware S3 domain fails
+    # "s3 origin domains and custom_origin_config are mutually exclusive"
     assert:
       that:
         - update_origin_to_s3_with_origin_access_and_with_custom_origin_config.failed