Repository: ansible/ansible

Issue:
Unstable integration test ec2_vpc_subnet
##### ISSUE TYPE

Bug Report

##### COMPONENT NAME

ec2_vpc_subnet integration test

##### ANSIBLE VERSION

2.5

##### CONFIGURATION

N/A

##### OS / ENVIRONMENT

Shippable

##### SUMMARY

Unstable integration test ec2_vpc_subnet:

https://app.shippable.com/github/ansible/ansible/runs/54187/55/tests

##### STEPS TO REPRODUCE

Run tests on Shippable.

##### EXPECTED RESULTS

Tests pass.

##### ACTUAL RESULTS

Tests are unstable or fail.

> test/integration/targets/ec2_vpc_subnet/tasks/main.yml:350 / [localhost] amazon: ec2_vpc_subnet : tidy up subnet aws_secret_key={{ aws_secret_key }}, aws_access_key={{ aws_access_key }}, security_token={{ security_token }}, state=absent, vpc_id={{ vpc_result.vpc.id }}, region={{ ec2_region }}, cidr=10.232.232.128/28

> failure: The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'vpc' The error appears to have been in '/root/ansible/test/integration/targets/ec2_vpc_subnet/tasks/main.yml': line 350, column 7, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: - name: tidy up subnet ^ here

```
{
"failed": true, 
"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'vpc'\n\nThe error appears to have been in '/root/ansible/test/integration/targets/ec2_vpc_subnet/tasks/main.yml': line 350, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n    - name: tidy up subnet\n      ^ here\n"
}
```

> test/integration/targets/ec2_vpc_subnet/tasks/main.yml:14 / [localhost] amazon: ec2_vpc_subnet : create a VPC aws_secret_key={{ aws_secret_key }}, aws_access_key={{ aws_access_key }}, name={{ resource_prefix }}-vpc, security_token={{ security_token }}, tags={u'Name': u'{{ resource_prefix }}-vpc', u'Description': u'Created by ansible-test'}, region={{ ec2_region }}, state=present, cidr_block=10.232.232.128/26

> error: ClientError: An error occurred (InvalidVpcID.NotFound) when calling the DescribeVpcClassicLink operation: The vpc ID 'vpc-1b2bda60' does not exist

```
Traceback (most recent call last):
  File "/tmp/ansible_dPE6PL/ansible_module_ec2_vpc_net.py", line 203, in get_vpc
    classic_link = connection.describe_vpc_classic_link(VpcIds=[vpc_id])['Vpcs'][0].get('ClassicLinkEnabled')
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 317, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 615, in _make_api_call
    raise error_class(parsed_response, operation_name)
ClientError: An error occurred (InvalidVpcID.NotFound) when calling the DescribeVpcClassicLink operation: The vpc ID 'vpc-1b2bda60' does not exist
```
Unstable integration test ec2_vpc_subnet
##### ISSUE TYPE

Bug Report

##### COMPONENT NAME

ec2_vpc_subnet integration test

##### ANSIBLE VERSION

2.5

##### CONFIGURATION

N/A

##### OS / ENVIRONMENT

Shippable

##### SUMMARY

Unstable integration test ec2_vpc_subnet:

https://app.shippable.com/github/ansible/ansible/runs/54187/55/tests

##### STEPS TO REPRODUCE

Run tests on Shippable.

##### EXPECTED RESULTS

Tests pass.

##### ACTUAL RESULTS

Tests are unstable or fail.

> test/integration/targets/ec2_vpc_subnet/tasks/main.yml:350 / [localhost] amazon: ec2_vpc_subnet : tidy up subnet aws_secret_key={{ aws_secret_key }}, aws_access_key={{ aws_access_key }}, security_token={{ security_token }}, state=absent, vpc_id={{ vpc_result.vpc.id }}, region={{ ec2_region }}, cidr=10.232.232.128/28

> failure: The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'vpc' The error appears to have been in '/root/ansible/test/integration/targets/ec2_vpc_subnet/tasks/main.yml': line 350, column 7, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: - name: tidy up subnet ^ here

```
{
"failed": true, 
"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'vpc'\n\nThe error appears to have been in '/root/ansible/test/integration/targets/ec2_vpc_subnet/tasks/main.yml': line 350, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n    - name: tidy up subnet\n      ^ here\n"
}
```

> test/integration/targets/ec2_vpc_subnet/tasks/main.yml:14 / [localhost] amazon: ec2_vpc_subnet : create a VPC aws_secret_key={{ aws_secret_key }}, aws_access_key={{ aws_access_key }}, name={{ resource_prefix }}-vpc, security_token={{ security_token }}, tags={u'Name': u'{{ resource_prefix }}-vpc', u'Description': u'Created by ansible-test'}, region={{ ec2_region }}, state=present, cidr_block=10.232.232.128/26

> error: ClientError: An error occurred (InvalidVpcID.NotFound) when calling the DescribeVpcClassicLink operation: The vpc ID 'vpc-1b2bda60' does not exist

```
Traceback (most recent call last):
  File "/tmp/ansible_dPE6PL/ansible_module_ec2_vpc_net.py", line 203, in get_vpc
    classic_link = connection.describe_vpc_classic_link(VpcIds=[vpc_id])['Vpcs'][0].get('ClassicLinkEnabled')
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 317, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/botocore/client.py", line 615, in _make_api_call
    raise error_class(parsed_response, operation_name)
ClientError: An error occurred (InvalidVpcID.NotFound) when calling the DescribeVpcClassicLink operation: The vpc ID 'vpc-1b2bda60' does not exist
```


Discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py)
* [test/integration/targets/ec2_vpc_subnet/aliases](https://github.com/ansible/ansible/blob/devel/test/integration/targets/ec2_vpc_subnet/aliases)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @brandond @willthames
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
This test continues to be unstable:

https://app.shippable.com/github/ansible/ansible/runs/57503/66/tests

Here's another failure: https://app.shippable.com/github/ansible/ansible/runs/57626/66/tests
Another failure: https://app.shippable.com/github/ansible/ansible/runs/57929/66/tests

However, the failure is different than the one originally reported:

> test/integration/targets/ec2_vpc_subnet/tasks/main.yml:337 / [localhost] amazon: ec2_vpc_subnet : assert subnet ipv6 cidr removed (expected changed=true) that=[u'vpc_remove_ipv6_cidr.changed', u"vpc_remove_ipv6_cidr.subnet.ipv6_cidr_block == ''", u'not vpc_remove_ipv6_cidr.subnet.assign_ipv6_address_on_creation']

> failure: rc=0

```
{
    "assertion": "vpc_remove_ipv6_cidr.subnet.ipv6_cidr_block == ''", 
    "changed": false, 
    "evaluated_to": false
}
```
Another failure: https://app.shippable.com/github/ansible/ansible/runs/57935/66/tests
Partially fixed by: #37354
Still needs ec2_vpc_subnet stabilization
Another failure: https://app.shippable.com/github/ansible/ansible/runs/58876/66/tests
Disabled test in https://github.com/ansible/ansible/commit/af15a46f20b152a80a6d714bd519950d6fd9ab0c
Reenabled test in 873a9ddf8dc37086aa8998c248310a23ae518c9d. 
It looks like these tests are still unstable:

https://app.shippable.com/github/ansible/ansible/runs/60100/67/tests
Files identified in the description:
* [lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py)
* [test/integration/targets/ec2_vpc_subnet/aliases](https://github.com/ansible/ansible/blob/devel/test/integration/targets/ec2_vpc_subnet/aliases)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @brandond @willthames
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
This test continues to be unstable:

https://app.shippable.com/github/ansible/ansible/runs/57503/66/tests

Here's another failure: https://app.shippable.com/github/ansible/ansible/runs/57626/66/tests
Another failure: https://app.shippable.com/github/ansible/ansible/runs/57929/66/tests

However, the failure is different than the one originally reported:

> test/integration/targets/ec2_vpc_subnet/tasks/main.yml:337 / [localhost] amazon: ec2_vpc_subnet : assert subnet ipv6 cidr removed (expected changed=true) that=[u'vpc_remove_ipv6_cidr.changed', u"vpc_remove_ipv6_cidr.subnet.ipv6_cidr_block == ''", u'not vpc_remove_ipv6_cidr.subnet.assign_ipv6_address_on_creation']

> failure: rc=0

```
{
    "assertion": "vpc_remove_ipv6_cidr.subnet.ipv6_cidr_block == ''", 
    "changed": false, 
    "evaluated_to": false
}
```
Another failure: https://app.shippable.com/github/ansible/ansible/runs/57935/66/tests
Partially fixed by: #37354
Still needs ec2_vpc_subnet stabilization
Another failure: https://app.shippable.com/github/ansible/ansible/runs/58876/66/tests
Disabled test in https://github.com/ansible/ansible/commit/af15a46f20b152a80a6d714bd519950d6fd9ab0c
Reenabled test in 873a9ddf8dc37086aa8998c248310a23ae518c9d. 
It looks like these tests are still unstable:

https://app.shippable.com/github/ansible/ansible/runs/60100/67/tests

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py b/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py
--- a/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py
+++ b/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py
@@ -362,14 +362,6 @@ def ensure_map_public(conn, module, subnet, map_public, check_mode, start_time):
     except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
         module.fail_json_aws(e, msg="Couldn't modify subnet attribute")
 
-    if module.params['wait']:
-        if map_public:
-            handle_waiter(conn, module, 'subnet_has_map_public',
-                          {'SubnetIds': [subnet['id']]}, start_time)
-        else:
-            handle_waiter(conn, module, 'subnet_no_map_public',
-                          {'SubnetIds': [subnet['id']]}, start_time)
-
 
 def ensure_assign_ipv6_on_create(conn, module, subnet, assign_instances_ipv6, check_mode, start_time):
     if check_mode:
@@ -379,14 +371,6 @@ def ensure_assign_ipv6_on_create(conn, module, subnet, assign_instances_ipv6, ch
     except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
         module.fail_json_aws(e, msg="Couldn't modify subnet attribute")
 
-    if module.params['wait']:
-        if assign_instances_ipv6:
-            handle_waiter(conn, module, 'subnet_has_assign_ipv6',
-                          {'SubnetIds': [subnet['id']]}, start_time)
-        else:
-            handle_waiter(conn, module, 'subnet_no_assign_ipv6',
-                          {'SubnetIds': [subnet['id']]}, start_time)
-
 
 def disassociate_ipv6_cidr(conn, module, subnet, start_time):
     if subnet.get('assign_ipv6_address_on_creation'):
@@ -512,6 +496,10 @@ def ensure_subnet_present(conn, module):
             changed = True
 
     subnet = get_matching_subnet(conn, module, module.params['vpc_id'], module.params['cidr'])
+    if not module.check_mode and module.params['wait']:
+        # GET calls are not monotonic for map_public_ip_on_launch and assign_ipv6_address_on_creation
+        # so we only wait for those if necessary just before returning the subnet
+        subnet = ensure_final_subnet(conn, module, subnet, start_time)
 
     return {
         'changed': changed,
@@ -519,6 +507,36 @@ def ensure_subnet_present(conn, module):
     }
 
 
+def ensure_final_subnet(conn, module, subnet, start_time):
+    for rewait in range(0, 10):
+        map_public_correct = False
+        assign_ipv6_correct = False
+
+        if module.params['map_public'] == subnet['map_public_ip_on_launch']:
+            map_public_correct = True
+        else:
+            if module.params['map_public']:
+                handle_waiter(conn, module, 'subnet_has_map_public', {'SubnetIds': [subnet['id']]}, start_time)
+            else:
+                handle_waiter(conn, module, 'subnet_no_map_public', {'SubnetIds': [subnet['id']]}, start_time)
+
+        if module.params['assign_instances_ipv6'] == subnet.get('assign_ipv6_address_on_creation'):
+            assign_ipv6_correct = True
+        else:
+            if module.params['assign_instances_ipv6']:
+                handle_waiter(conn, module, 'subnet_has_assign_ipv6', {'SubnetIds': [subnet['id']]}, start_time)
+            else:
+                handle_waiter(conn, module, 'subnet_no_assign_ipv6', {'SubnetIds': [subnet['id']]}, start_time)
+
+        if map_public_correct and assign_ipv6_correct:
+            break
+
+        time.sleep(3)
+        subnet = get_matching_subnet(conn, module, module.params['vpc_id'], module.params['cidr'])
+
+    return subnet
+
+
 def ensure_subnet_absent(conn, module):
     subnet = get_matching_subnet(conn, module, module.params['vpc_id'], module.params['cidr'])
     if subnet is None:


Test patch: