Repository: ansible-collections/amazon.aws

Issue:
'NoneType' object is not subscriptable error in ec2_vpc_igw module
### Summary

When submitting #646 to fix bug #622, the integration tests fail in the `ec2_vpc_igw` module. This seems to be unrelated to the changes I made in the `aws_acm` module.

The `ec2_vpc_igw` module fails with `TypeError: 'NoneType' object is not subscriptable`. I don't think the module should ever fail with a NoneType error.

### Issue Type

Bug Report

### Component Name

ec2_vpc_igw

### Ansible Version

The environment (ansible version, collection version, SDK, OS...) is entirely determined by the PR process. The failures of the integration tests are here: https://github.com/ansible-collections/amazon.aws/pull/646 and https://ansible.softwarefactory-project.io/zuul/build/cb5198eefa104eeb9cd5c8bc6e12bdc1

It's not clear which part of the zuul logs contain the details about the environment. I've copy-pasted some data from the zuul logs.

```console (paste below)
2022-01-28 14:24:20.068273 | 
2022-01-28 14:24:20.068478 | TASK [emit-job-header : Print job information]
2022-01-28 14:24:20.662964 | # Job Information
2022-01-28 14:24:20.663164 | Ansible Version: 2.9.16
2022-01-28 14:24:20.663198 | Job: ansible-test-cloud-integration-aws-py38_3
2022-01-28 14:24:20.663225 | Pipeline: check
2022-01-28 14:24:20.663250 | Executor: ze06.softwarefactory-project.io
2022-01-28 14:24:20.663275 | Triggered by: https://github.com/ansible-collections/amazon.aws/pull/646
2022-01-28 14:24:20.663299 | Log URL (when completed): https://ansible.softwarefactory-project.io/logs/46/646/d444627c887c44c244ef257e68d45a892f10f772/check/ansible-test-cloud-integration-aws-py38_3/cb5198e/
2022-01-28 14:24:20.663324 | Event ID: 79a6d1b0-8044-11ec-8e57-1426c1318593

```


### Collection Versions

```console (paste below)
$ ansible-galaxy collection list
```


### AWS SDK versions

```console (paste below)
$ pip show boto boto3 botocore
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed

```


### OS / Environment

_No response_

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)

```


### Expected Results

I expected the PR integration tests to complete without error. Or at least to complete without any error outside the changes I made.
More specifically:
1. The integration tests for the `ec2_vpc_igw` module should not fail. 
1. When a problem does occur in the `ec2_vpc_igw` module, it should handle the error more gracefully, not by raising a`'NoneType' object is not subscriptable` exception

"/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 158, in get_igw_info
TypeError: 'NoneType' object is not subscriptable

### Actual Results

The PR integration tests fail in the following modules:

```console
ERROR: The 2 integration test(s) listed below (out of 8) failed. See error output above for details:
ec2_vpc_igw
ec2_vpc_route_table
```

Here is a specific stack trace for the ec2_vpc_igw module.

```console (paste below)
Traceback (most recent call last):
  File "<stdin>", line 121, in <module>
  File "<stdin>", line 113, in _ansiballz_main
  File "<stdin>", line 61, in invoke_module
  File "/usr/lib64/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 248, in <module>
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 242, in main
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 132, in process
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 220, in ensure_igw_present
  File "/tmp/ansible_ec2_vpc_igw_payload_g4o1kl_r/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 158, in get_igw_info
TypeError: 'NoneType' object is not subscriptable
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/modules/ec2_vpc_igw.py`](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/ec2_vpc_igw.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @jillr @s-hertel @tremble
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
This PR https://github.com/ansible-collections/amazon.aws/pull/672 should fix it.
@alinabuzachis I'm still running into this issue when running integration tests for `ec2_vpc_route_table`

```
Traceback (most recent call last):
  File "<stdin>", line 121, in <module>
  File "<stdin>", line 113, in _ansiballz_main
  File "<stdin>", line 61, in invoke_module
  File "/usr/lib64/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 249, in <module>
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 243, in main
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 132, in process
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 221, in ensure_igw_present
  File "/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py", line 158, in get_igw_info
TypeError: 'NoneType' object is not subscriptable
fatal: [testhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 121, in <module>\n  File \"<stdin>\", line 113, in _ansiballz_main\n  File \"<stdin>\", line 61, in invoke_module\n  File \"/usr/lib64/python3.8/runpy.py\", line 207, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.8/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib64/python3.8/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 249, in <module>\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 243, in main\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 132, in process\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 221, in ensure_igw_present\n  File \"/tmp/ansible_ec2_vpc_igw_payload_jxieg1a_/ansible_ec2_vpc_igw_payload.zip/ansible_collections/amazon/aws/plugins/modules/ec2_vpc_igw.py\", line 158, in get_igw_info\nTypeError: 'NoneType' object is not subscriptable\n",
```
I still see the issue when I run the integration tests in https://github.com/ansible-collections/amazon.aws/pull/631
So adding a waiter after attaching the igw still isnt enough it seems? Thoughts on waiting for igw after ensuring tags? @alinabuzachis
> So adding a waiter after attaching the igw still isnt enough it seems? Thoughts on waiting for igw after ensuring tags? @alinabuzachis

Ok, I will also try to have a look if the error persists. BTW, the CI seems green now. @sebastien-rosset please feel free to  open an issue if you encounter this issue again. 
The waiter I added after attaching definitely helps - it used to fail about 10% of the time and now its about 2% of the time (I ran bash scripts to run 50 iterations) - but the main problem is just the instability of calling describe_internet_gateways. We might just have to do a manual waiter prior to calling ensure_ec2_tags then again after and just return the gw directly to make 100% sure we have the object. Not pretty but I'm not sure what else we can do.

Resolving patch:
diff --git a/plugins/modules/ec2_vpc_igw.py b/plugins/modules/ec2_vpc_igw.py
--- a/plugins/modules/ec2_vpc_igw.py
+++ b/plugins/modules/ec2_vpc_igw.py
@@ -112,6 +112,12 @@
 from ..module_utils.tagging import boto3_tag_list_to_ansible_dict
 
 
+@AWSRetry.jittered_backoff(retries=10, delay=10)
+def describe_igws_with_backoff(connection, **params):
+    paginator = connection.get_paginator('describe_internet_gateways')
+    return paginator.paginate(**params).build_full_result()['InternetGateways']
+
+
 class AnsibleEc2Igw():
 
     def __init__(self, module, results):
@@ -135,10 +141,8 @@ def process(self):
 
     def get_matching_igw(self, vpc_id):
         filters = ansible_dict_to_boto3_filter_list({'attachment.vpc-id': vpc_id})
-        igws = []
         try:
-            response = self._connection.describe_internet_gateways(aws_retry=True, Filters=filters)
-            igws = response.get('InternetGateways', [])
+            igws = describe_igws_with_backoff(self._connection, Filters=filters)
         except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
             self._module.fail_json_aws(e)
 
@@ -211,17 +215,20 @@ def ensure_igw_present(self, vpc_id, tags, purge_tags):
                 # Ensure the gateway is attached before proceeding
                 waiter = get_waiter(self._connection, 'internet_gateway_attached')
                 waiter.wait(InternetGatewayIds=[igw['internet_gateway_id']])
-
                 self._results['changed'] = True
             except botocore.exceptions.WaiterError as e:
                 self._module.fail_json_aws(e, msg="No Internet Gateway exists.")
             except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
                 self._module.fail_json_aws(e, msg='Unable to create Internet Gateway')
 
+        # Modify tags
         self._results['changed'] |= ensure_ec2_tags(
             self._connection, self._module, igw['internet_gateway_id'],
-            resource_type='internet-gateway', tags=tags, purge_tags=purge_tags
+            resource_type='internet-gateway', tags=tags, purge_tags=purge_tags,
+            retry_codes='InvalidInternetGatewayID.NotFound'
         )
+
+        # Update igw
         igw = self.get_matching_igw(vpc_id)
         igw_info = self.get_igw_info(igw, vpc_id)
         self._results.update(igw_info)


Test patch: