Repository: ansible-collections/amazon.aws

Issue:
'NoneType' object is not subscriptable error in ec2_vpc_igw module
### Summary

When submitting #646 to fix bug #622, the integration tests fail in the `ec2_vpc_igw` module. This seems to be unrelated to the changes I made in the `aws_acm` module.

The `ec2_vpc_igw` module fails with `TypeError: 'NoneType' object is not subscriptable`. I don't think the module should ever fail with a NoneType error.

### Issue Type

Bug Report

### Component Name

ec2_vpc_igw

### Ansible Version

The environment (ansible version, collection version, SDK, OS...) is entirely determined by the PR process. The failures of the integration tests are here: https://github.com/ansible-collections/amazon.aws/pull/646 and https://ansible.softwarefactory-project.io/zuul/build/cb5198eefa104eeb9cd5c8bc6e12bdc1

It's not clear which part of the zuul logs contain the details about the environment. I've copy-pasted some data from the zuul logs.

```console (paste below)
2022-01-28 14:24:20.068273 | 
2022-01-28 14:24:20.068478 | TASK [emit-job-header : Print job information]
2022-01-28 14:24:20.662964 | # Job Information
2022-01-28 14:24:20.663164 | Ansible Version: 2.9.16
2022-01-28 14:24:20.663198 | Job: ansible-test-cloud-integration-aws-py38_3
2022-01-28 14:24:20.663225 | Pipeline: check
2022-01-28 14:24:20.663250 | Executor: ze06.softwarefactory-project.io
2022-01-28 14:24:20.663275 | Triggered by: https://github.com/ansible-collections/amazon.aws/pull/646
2022-01-28 14:24:20.663299 | Log URL (when completed): https://ansible.softwarefactory-project.io/logs/46/646/d444627c887c44c244ef257e68d45a892f10f772/check/ansible-test-cloud-integration-aws-py38_3/cb5198e/
2022-01-28 14:24:20.663324 | Event ID: 79a6d1b0-8044-11ec-8e57-1426c1318593

```


### Collection Versions

```console (paste below)
$ ansible-galaxy collection list
```


### AWS SDK versions

```console (paste below)
$ pip show boto boto3 botocore
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed

```


### OS / Environment

_No response_

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)

```


### Expected Results

I expected the PR integration tests to complete without error. Or at least to complete without any error outside the changes I made.
More specifically:
1. The integration tests for the `ec2_vpc_igw` module should not fail. 
1. When a problem does occur in the `ec2_vpc_igw` module, it should handle the error more gracefully, not by raising a`'NoneType' object is not subscriptable` exception

"/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 158, in get_igw_info
TypeError: 'NoneType' object is not subscriptable

### Actual Results

The PR integration tests fail in the following modules:

```console
ERROR: The 2 integration test(s) listed below (out of 8) failed. See error output above for details:
ec2_vpc_igw
ec2_vpc_route_table
```

Here is a specific stack trace for the ec2_vpc_igw module.

```console (paste below)
Traceback (most recent call last):
  File "<stdin>", line 121, in <module>
  File "<stdin>", line 113, in _ansiballz_main
  File "<stdin>", line 61, in invoke_module
  File "/usr/lib64/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 248, in <module>
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 242, in main
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 132, in process
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 220, in ensure_igw_present
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 158, in get_igw_info
TypeError: 'NoneType' object is not subscriptable
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/modules/ec2_vpc_igw.py`](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/ec2_vpc_igw.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @jillr @s-hertel @tremble
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
This PR https://github.com/ansible-collections/amazon.aws/pull/672 should fix it.
@alinabuzachis I'm still running into this issue when running integration tests for `ec2_vpc_route_table`

```
Traceback (most recent call last):
  File "<stdin>", line 121, in <module>
  File "<stdin>", line 113, in _ansiballz_main
  File "<stdin>", line 61, in invoke_module
  File "/usr/lib64/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 249, in <module>
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 243, in main
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 132, in process
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 221, in ensure_igw_present
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 158, in get_igw_info
TypeError: 'NoneType' object is not subscriptable
fatal: [testhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 121, in <module>\n  File \"<stdin>\", line 113, in _ansiballz_main\n  File \"<stdin>\", line 61, in invoke_module\n  File \"/usr/lib64/python3.8/runpy.py\", line 207, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.8/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib64/python3.8/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 249, in <module>\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 243, in main\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 132, in process\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 221, in ensure_igw_present\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 158, in get_igw_info\nTypeError: 'NoneType' object is not subscriptable\n",
```

Resolving patch:
diff --git a/plugins/module_utils/waiters.py b/plugins/module_utils/waiters.py
--- a/plugins/module_utils/waiters.py
+++ b/plugins/module_utils/waiters.py
@@ -54,6 +54,24 @@
                 },
             ]
         },
+        "InternetGatewayAttached": {
+            "operation": "DescribeInternetGateways",
+            "delay": 5,
+            "maxAttempts": 40,
+            "acceptors": [
+                {
+                    "expected": "available",
+                    "matcher": "pathAll",
+                    "state": "success",
+                    "argument": "InternetGateways[].Attachments[].State"
+                },
+                {
+                    "matcher": "error",
+                    "expected": "InvalidInternetGatewayID.NotFound",
+                    "state": "retry"
+                },
+            ]
+        },
         "NetworkInterfaceAttached": {
             "operation": "DescribeNetworkInterfaces",
             "delay": 5,
@@ -716,6 +734,12 @@ def route53_model(name):
         core_waiter.NormalizedOperationMethod(
             ec2.describe_internet_gateways
         )),
+    ('EC2', 'internet_gateway_attached'): lambda ec2: core_waiter.Waiter(
+        'internet_gateway_attached',
+        ec2_model('InternetGatewayAttached'),
+        core_waiter.NormalizedOperationMethod(
+            ec2.describe_internet_gateways
+        )),
     ('EC2', 'network_interface_attached'): lambda ec2: core_waiter.Waiter(
         'network_interface_attached',
         ec2_model('NetworkInterfaceAttached'),
diff --git a/plugins/modules/ec2_vpc_igw.py b/plugins/modules/ec2_vpc_igw.py
--- a/plugins/modules/ec2_vpc_igw.py
+++ b/plugins/modules/ec2_vpc_igw.py
@@ -207,6 +207,11 @@ def ensure_igw_present(self, vpc_id, tags, purge_tags):
                     InternetGatewayId=igw['internet_gateway_id'],
                     VpcId=vpc_id
                 )
+
+                # Ensure the gateway is attached before proceeding
+                waiter = get_waiter(self._connection, 'internet_gateway_attached')
+                waiter.wait(InternetGatewayIds=[igw['internet_gateway_id']])
+
                 self._results['changed'] = True
             except botocore.exceptions.WaiterError as e:
                 self._module.fail_json_aws(e, msg="No Internet Gateway exists.")


Test patch: