Repository: ansible-collections/community.aws

Issue:
route53: error when creating records in private zone with vpc_id
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
`route53` has a special behaviour when working on private zones and `vpc_id` option is provided. As stated in documentation:
> When used in conjunction with private_zone: true, this will only modify records in the private hosted zone attached to this VPC.
> This allows you to have multiple private hosted zones, all with the same name, attached to different VPCs.

The following error occurs when creating or updating a record in such zone: `AttributeError: 'dict' object has no attribute 'id'`.

In case anyone wonders, this is not a duplicate of https://github.com/ansible-collections/community.aws/issues/434 because no `vpc_id` option is used there.

I'll provide a PR soon.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
`route53`

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.10.5
  config file = /home/admin/ansible/ansible.cfg
  configured module search path = ['/home/admin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
ALLOW_WORLD_READABLE_TMPFILES(/home/admin/ansible/ansible.cfg) = True
ANSIBLE_SSH_CONTROL_PATH(/home/admin/ansible/ansible.cfg) = %(directory)s/%%h-%%p-%%r
DEFAULT_NO_TARGET_SYSLOG(/home/admin/ansible/ansible.cfg) = True
DEFAULT_ROLES_PATH(/home/admin/ansible/ansible.cfg) = ['/home/admin/ansible/roles']
INTERPRETER_PYTHON(/home/admin/ansible/ansible.cfg) = /usr/bin/python3
RETRY_FILES_ENABLED(/home/admin/ansible/ansible.cfg) = False
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
Debian 10

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->
Ensure a private zone exists, then run the following task.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- hosts: all
  tasks:
      route53:
        aws_access_key: "{{ aws_access_key }}"
        aws_secret_key: "{{ aws_secret_key }}"
        overwrite: true
        private_zone: true
        record: "test-ansible.internal.example.com"
        state: present
        type: A
        value: "1.2.3.4"
        vpc_id: vpc-xyz1234
        zone: "internal.example.com"
```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
The record is created inside the zone.

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
An error occured.

<!--- Paste verbatim command output between quotes -->
```paste below
The full traceback is:
Traceback (most recent call last):
  File "/home/admin/.ansible/tmp/ansible-tmp-1617023331.5792427-11521-49668929273150/AnsiballZ_route53.py", line 102, in <module>
    _ansiballz_main()
  File "/home/admin/.ansible/tmp/ansible-tmp-1617023331.5792427-11521-49668929273150/AnsiballZ_route53.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/admin/.ansible/tmp/ansible-tmp-1617023331.5792427-11521-49668929273150/AnsiballZ_route53.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.route53', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_route53_payload_n6znsp0f/ansible_route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 626, in <module>
  File "/tmp/ansible_route53_payload_n6znsp0f/ansible_route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 529, in main
  File "/tmp/ansible_route53_payload_n6znsp0f/ansible_route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 413, in get_zone_id_by_name
AttributeError: 'dict' object has no attribute 'id'
```

route53: error when creating records in private zone with vpc_id
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
`route53` has a special behaviour when working on private zones and `vpc_id` option is provided. As stated in documentation:
> When used in conjunction with private_zone: true, this will only modify records in the private hosted zone attached to this VPC.
> This allows you to have multiple private hosted zones, all with the same name, attached to different VPCs.

The following error occurs when creating or updating a record in such zone: `AttributeError: 'dict' object has no attribute 'id'`.

In case anyone wonders, this is not a duplicate of https://github.com/ansible-collections/community.aws/issues/434 because no `vpc_id` option is used there.

I'll provide a PR soon.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
`route53`

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.10.5
  config file = /home/admin/ansible/ansible.cfg
  configured module search path = ['/home/admin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
ALLOW_WORLD_READABLE_TMPFILES(/home/admin/ansible/ansible.cfg) = True
ANSIBLE_SSH_CONTROL_PATH(/home/admin/ansible/ansible.cfg) = %(directory)s/%%h-%%p-%%r
DEFAULT_NO_TARGET_SYSLOG(/home/admin/ansible/ansible.cfg) = True
DEFAULT_ROLES_PATH(/home/admin/ansible/ansible.cfg) = ['/home/admin/ansible/roles']
INTERPRETER_PYTHON(/home/admin/ansible/ansible.cfg) = /usr/bin/python3
RETRY_FILES_ENABLED(/home/admin/ansible/ansible.cfg) = False
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
Debian 10

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->
Ensure a private zone exists, then run the following task.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- hosts: all
  tasks:
      route53:
        aws_access_key: "{{ aws_access_key }}"
        aws_secret_key: "{{ aws_secret_key }}"
        overwrite: true
        private_zone: true
        record: "test-ansible.internal.example.com"
        state: present
        type: A
        value: "1.2.3.4"
        vpc_id: vpc-xyz1234
        zone: "internal.example.com"
```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
The record is created inside the zone.

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
An error occured.

<!--- Paste verbatim command output between quotes -->
```paste below
The full traceback is:
Traceback (most recent call last):
  File "/home/admin/.ansible/tmp/ansible-tmp-1617023331.5792427-11521-49668929273150/AnsiballZ_route53.py", line 102, in <module>
    _ansiballz_main()
  File "/home/admin/.ansible/tmp/ansible-tmp-1617023331.5792427-11521-49668929273150/AnsiballZ_route53.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/admin/.ansible/tmp/ansible-tmp-1617023331.5792427-11521-49668929273150/AnsiballZ_route53.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.aws.plugins.modules.route53', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_route53_payload_n6znsp0f/ansible_route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 626, in <module>
  File "/tmp/ansible_route53_payload_n6znsp0f/ansible_route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 529, in main
  File "/tmp/ansible_route53_payload_n6znsp0f/ansible_route53_payload.zip/ansible_collections/community/aws/plugins/modules/route53.py", line 413, in get_zone_id_by_name
AttributeError: 'dict' object has no attribute 'id'
```



Discussion:

Resolving patch:
diff --git a/plugins/modules/route53.py b/plugins/modules/route53.py
--- a/plugins/modules/route53.py
+++ b/plugins/modules/route53.py
@@ -410,15 +410,9 @@ def get_zone_id_by_name(route53, module, zone_name, want_private, want_vpc_id):
             if want_vpc_id:
                 # NOTE: These details aren't available in other boto methods, hence the necessary
                 # extra API call
-                hosted_zone = route53.get_hosted_zone(aws_retry=True, Id=zone.id)
-                zone_details = hosted_zone['HostedZone']
-                # this is to deal with this boto bug: https://github.com/boto/boto/pull/2882
-                if isinstance(zone_details['VPCs'], dict):
-                    if zone_details['VPCs']['VPC']['VPCId'] == want_vpc_id:
-                        return zone_id
-                else:  # Forward compatibility for when boto fixes that bug
-                    if want_vpc_id in [v['VPCId'] for v in zone_details['VPCs']]:
-                        return zone_id
+                hosted_zone = route53.get_hosted_zone(aws_retry=True, Id=zone_id)
+                if want_vpc_id in [v['VPCId'] for v in hosted_zone['VPCs']]:
+                    return zone_id
             else:
                 return zone_id
     return None


Test patch:
diff --git a/tests/integration/targets/route53/tasks/main.yml b/tests/integration/targets/route53/tasks/main.yml
--- a/tests/integration/targets/route53/tasks/main.yml
+++ b/tests/integration/targets/route53/tasks/main.yml
@@ -16,6 +16,14 @@
     route53:
       region: null
   block:
+
+  - name: create VPC
+    ec2_vpc_net:
+      cidr_block: 192.0.2.0/24
+      name: '{{ resource_prefix }}_vpc'
+      state: present
+    register: vpc
+
   - route53_zone:
       zone: '{{ zone_one }}'
       comment: Created in Ansible test {{ resource_prefix }}
@@ -39,13 +47,39 @@
       that:
         - hosted_zones.HostedZone.ResourceRecordSetCount == 2
 
+  - route53_zone:
+      zone: '{{ zone_two }}'
+      vpc_id: '{{ vpc.vpc.id }}'
+      vpc_region: '{{ aws_region }}'
+      comment: Created in Ansible test {{ resource_prefix }}
+    register: z2
+
+  - assert:
+      that:
+        - z2 is success
+        - z2 is changed
+        - "z2.comment == 'Created in Ansible test {{ resource_prefix }}'"
+
+  - name: Get zone details
+    route53_info:
+      query: hosted_zone
+      hosted_zone_id: '{{ z2.zone_id }}'
+      hosted_zone_method: details
+    register: hosted_zones
+
+  - name: Assert newly created hosted zone only has NS and SOA records
+    assert:
+      that:
+        - hosted_zones.HostedZone.ResourceRecordSetCount == 2
+        - hosted_zones.HostedZone.Config.PrivateZone
+
   - name: Create A record using zone fqdn
     route53:
       state: present
       zone: '{{ zone_one }}'
       record: 'qdn_test.{{ zone_one }}'
       type: A
-      value: 1.2.3.4
+      value: 192.0.2.1
     register: qdn
   - assert:
       that:
@@ -63,7 +97,7 @@
       that:
         - get_result.nameservers|length > 0
         - get_result.set.Name == "qdn_test.{{ zone_one }}"
-        - get_result.set.ResourceRecords[0].Value == "1.2.3.4"
+        - get_result.set.ResourceRecords[0].Value == "192.0.2.1"
         - get_result.set.Type == "A"
 
   - name: Create same A record using zone non-qualified domain
@@ -72,7 +106,7 @@
       zone: '{{ zone_one[:-1] }}'
       record: 'qdn_test.{{ zone_one[:-1] }}'
       type: A
-      value: 1.2.3.4
+      value: 192.0.2.1
     register: non_qdn
   - assert:
       that:
@@ -85,7 +119,7 @@
       hosted_zone_id: '{{ z1.zone_id }}'
       record: 'zid_test.{{ zone_one }}'
       type: A
-      value: 1.2.3.4
+      value: 192.0.2.1
     register: zid
   - assert:
       that:
@@ -99,8 +133,8 @@
       record: 'order_test.{{ zone_one }}'
       type: A
       value:
-        - 4.5.6.7
-        - 1.2.3.4
+        - 192.0.2.2
+        - 192.0.2.1
     register: mv_a_record
   - assert:
       that:
@@ -114,8 +148,8 @@
       record: 'order_test.{{ zone_one }}'
       type: A
       value:
-        - 4.5.6.7
-        - 1.2.3.4
+        - 192.0.2.2
+        - 192.0.2.1
     register: mv_a_record
   - assert:
       that:
@@ -134,8 +168,8 @@
       that:
         - records.ResourceRecordSets|length == 3
         - records.ResourceRecordSets[0].ResourceRecords|length == 2
-        - records.ResourceRecordSets[0].ResourceRecords[0].Value == "4.5.6.7"
-        - records.ResourceRecordSets[0].ResourceRecords[1].Value == "1.2.3.4"
+        - records.ResourceRecordSets[0].ResourceRecords[0].Value == "192.0.2.2"
+        - records.ResourceRecordSets[0].ResourceRecords[1].Value == "192.0.2.1"
 
   - name: Remove a member from multi-value A record with values in different order
     route53:
@@ -144,7 +178,7 @@
       record: 'order_test.{{ zone_one }}'
       type: A
       value:
-        - 4.5.6.7
+        - 192.0.2.2
     register: del_a_record
     ignore_errors: true
   - name: This should fail, because `overwrite` is false
@@ -160,7 +194,7 @@
       overwrite: true
       type: A
       value:
-        - 4.5.6.7
+        - 192.0.2.2
     register: del_a_record
     ignore_errors: true
   - name: This should not fail, because `overwrite` is true
@@ -181,7 +215,7 @@
       that:
         - records.ResourceRecordSets|length == 3
         - records.ResourceRecordSets[0].ResourceRecords|length == 1
-        - records.ResourceRecordSets[0].ResourceRecords[0].Value == "4.5.6.7"
+        - records.ResourceRecordSets[0].ResourceRecords[0].Value == "192.0.2.2"
 
   - name: Create a LetsEncrypt CAA record
     route53:
@@ -232,6 +266,93 @@
         - caa is not failed
         - caa is not changed
 
+  # Tests on zone two (private zone)
+  - name: Create A record using zone fqdn
+    route53:
+      state: present
+      zone: '{{ zone_two }}'
+      record: 'qdn_test.{{ zone_two }}'
+      type: A
+      value: 192.0.2.1
+      private_zone: true
+    register: qdn
+  - assert:
+      that:
+        - qdn is not failed
+        - qdn is changed
+
+  - name: Get A record using 'get' method of route53 module
+    route53:
+      state: get
+      zone: "{{ zone_two }}"
+      record: "qdn_test.{{ zone_two }}"
+      type: A
+      private_zone: true
+    register: get_result
+  - assert:
+      that:
+        - get_result.nameservers|length > 0
+        - get_result.set.Name == "qdn_test.{{ zone_two }}"
+        - get_result.set.ResourceRecords[0].Value == "192.0.2.1"
+        - get_result.set.Type == "A"
+
+  - name: Create same A record using zone non-qualified domain
+    route53:
+      state: present
+      zone: '{{ zone_two[:-1] }}'
+      record: 'qdn_test.{{ zone_two[:-1] }}'
+      type: A
+      value: 192.0.2.1
+      private_zone: true
+    register: non_qdn
+  - assert:
+      that:
+        - non_qdn is not failed
+        - non_qdn is not changed
+
+  - name: Create A record using zone ID
+    route53:
+      state: present
+      hosted_zone_id: '{{ z2.zone_id }}'
+      record: 'zid_test.{{ zone_two }}'
+      type: A
+      value: 192.0.2.2
+      private_zone: true
+    register: zid
+  - assert:
+      that:
+        - zid is not failed
+        - zid is changed
+
+  - name: Create A record using zone fqdn and vpc_id
+    route53:
+      state: present
+      zone: '{{ zone_two }}'
+      record: 'qdn_test_vpc.{{ zone_two }}'
+      type: A
+      value: 192.0.2.3
+      private_zone: true
+      vpc_id: '{{ vpc.vpc.id }}'
+    register: qdn
+  - assert:
+      that:
+        - qdn is not failed
+        - qdn is changed
+
+  - name: Create A record using zone ID and vpc_id
+    route53:
+      state: present
+      hosted_zone_id: '{{ z2.zone_id }}'
+      record: 'zid_test_vpc.{{ zone_two }}'
+      type: A
+      value: 192.0.2.4
+      private_zone: true
+      vpc_id: '{{ vpc.vpc.id }}'
+    register: zid
+  - assert:
+      that:
+        - zid is not failed
+        - zid is changed
 
   always:
   - route53_info:
@@ -247,6 +368,20 @@
       type: '{{ item.Type }}'
       value: '{{ item.ResourceRecords | map(attribute="Value") | join(",") }}'
     loop: '{{ z1_records.ResourceRecordSets | selectattr("Type", "in", ["A", "AAAA", "CNAME", "CAA"]) | list }}'
+  - route53_info:
+      query: record_sets
+      hosted_zone_id: '{{ z2.zone_id }}'
+    register: z2_records
+  - debug: var=z2_records
+  - name: Loop over A/AAAA/CNAME records and delete them
+    route53:
+      state: absent
+      zone: '{{ zone_two }}'
+      record: '{{ item.Name }}'
+      type: '{{ item.Type }}'
+      value: '{{ item.ResourceRecords | map(attribute="Value") | join(",") }}'
+      private_zone: true
+    loop: '{{ z2_records.ResourceRecordSets | selectattr("Type", "in", ["A", "AAAA", "CNAME", "CAA"]) | list }}'
   - name: Delete test zone one '{{ zone_one }}'
     route53_zone:
       state: absent
@@ -264,3 +399,13 @@
     retries: 10
     until: delete_two is not failed
     when: false
+  - name: destroy VPC
+    ec2_vpc_net:
+      cidr_block: 192.0.2.0/24
+      name: '{{ resource_prefix }}_vpc'
+      state: absent
+    register: remove_vpc
+    retries: 10
+    delay: 5
+    until: remove_vpc is success
+    ignore_errors: true