Repository: ansible/ansible

Issue:
ec2_vpc_route_table can't update routes
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
ec2_vpc_route_table

##### ANSIBLE VERSION
```
ansible 2.1.3.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
-->
N/A

##### SUMMARY
<!--- Explain the problem briefly -->
Ran a script to create a NAT instance, created the routes going through the NAT using ec2_vpc_route_table 
Deleted the NAT and ran the same script again.
ec2_vpc_route_table was not able to update the route with the new instance id, but left the old network interface (which no longer existed) in place, thereby resulting in a black hole.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: Create Backend route 1 and route it through NAT 1
  ec2_vpc_route_table:
    vpc_id: '{{ vpc_id }}'
    region: '{{ vpc_region }}'
    tags:
      Name: "{{ vpc_name }} Backend network 1"
    routes:
      - dest: 0.0.0.0/0
        instance_id: '{{ instance_id }}'
    subnets:
      - "{{ vpc_subnet['web_subnet']['subnet_one'].resource_tags.Name }}"
      - "{{ vpc_subnet['db_subnet']['subnet_one'].resource_tags.Name }}"

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
I naturally expected that the route table would be updated

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
changed: [10.77.200.10] => {"changed": true, "invocation": {"module_args": {"aws_access_key": null, "aws_secret_key": null, "ec2_url": null, "lookup": "tag", "profile": null, "propagating_vgw_ids": null, "region": "us-west-2", "route_table_id": null, "routes": [{"destination_cidr_block": "0.0.0.0/0", "instance_id": "i-1234567890123456"}], "security_token": null, "state": "present", "subnets": ["test - web - us-west-2c",  "test - database - us-west-2c"], "tags": {"Name": "test Backend network 1"}, "validate_certs": true, "vpc_id": "vpc-12345678"}, "module_name": "ec2_vpc_route_table"}, "route_table": {"id": "rtb-23456789", "routes": [{"destination_cidr_block": "10.99.0.0/16", "gateway_id": null, "instance_id": "i-0987654321098765", "interface_id": "eni-12345678", "origin": "CreateRoute", "state": "active", "vpc_peering_connection_id": null}, {"destination_cidr_block": "10.77.0.0/16", "gateway_id": "local", "instance_id": null, "interface_id": null, "origin": "CreateRouteTable", "state": "active", "vpc_peering_connection_id": null}, {"destination_cidr_block": "0.0.0.0/0", "gateway_id": null, "instance_id": null, "interface_id": "eni-87654321", "origin": "CreateRoute", "state": "blackhole", "vpc_peering_connection_id": null}], "tags": {"Name": "test Backend network 1"}, "vpc_id": "vpc-12345678"}}

```



Discussion:
cc @wimnat
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
I will take a look into this.
I got something similar with VPC peers.  Since AWS allows for this case, it's not so much a fault of the module, but it could be kind and do the clean up with a flag.  +1.
I'm have the same issue in 2.3
```
ansible 2.3.0.0
  config file = /var/svc-ac2sp-ansible/playbooks/gmd-ansible_core/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.5 (default, Nov  6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]
```

I should note that in my task, I've set `purge_routes: false` because I just want to update a single route.

```
- name: "Set this proxy as the route to 0.0.0.0/0 on non-dmz subnets in this VPC"
  delegate_to: localhost
  vars:
    subnets_q: "[? ! (tags.dmz == 'true' || tags.dmz == 'True') ].id"
  ec2_vpc_route_table:
    aws_access_key: "{{ instance.vpc.NETADMIN.access_key }}"
    aws_secret_key: "{{ instance.vpc.NETADMIN.secret_key }}"
    security_token: "{{ instance.vpc.NETADMIN.security_token }}"
    vpc_id: "{{ instance.vpc.id }}"
    region: "{{ instance.region }}"
    subnets: "{{ instance.vpc.subnets | json_query(subnets_q) }}"
    tags:
      Name: "{{ instance.vpc.tags.Name | default(instance.vpc.id) }}-routes"
    routes:
      - dest: 0.0.0.0/0
        instance_id: "{{ instance.id }}"
    state: present
    purge_routes: false
    purge_subnets: false

```
For those of you who are interested, here's the work-around I used.  It assumes you have the aws cli installed on localhost

```
- name: "Set this proxy as the route to 0.0.0.0/0 on non-dmz subnets in this VPC"
  delegate_to: localhost
  vars:
    subnets_q: "[? ! (tags.dmz == 'true' || tags.dmz == 'True') ].id"
  ec2_vpc_route_table:
    aws_access_key: "{{ instance.vpc.NETADMIN.access_key }}"
    aws_secret_key: "{{ instance.vpc.NETADMIN.secret_key }}"
    security_token: "{{ instance.vpc.NETADMIN.security_token }}"
    vpc_id: "{{ instance.vpc.id }}"
    region: "{{ instance.region }}"
    subnets: "{{ instance.vpc.subnets | json_query(subnets_q) }}"
    tags:
      Name: "{{ instance.vpc.tags.Name | default(instance.vpc.id) }}-routes"
    routes:
      - dest: 0.0.0.0/0
        instance_id: "{{ instance.id }}"
    state: present
    purge_routes: false
    purge_subnets: false
  register: ec2_vpc_route_table_result

- name: "Work around ec2_vpc_route_table not replacing a route"
  delegate_to: localhost
  shell:
    cmd: |
      export AWS_ACCESS_KEY_ID="{{ instance.vpc.NETADMIN.access_key }}"
      export AWS_SECRET_ACCESS_KEY="{{ instance.vpc.NETADMIN.secret_key }}"
      export AWS_SESSION_TOKEN="{{ instance.vpc.NETADMIN.session_token }}"
      aws ec2 replace-route \
        --route-table-id "{{ ec2_vpc_route_table_result.route_table.id }}" \
        --destination-cidr-block "0.0.0.0/0" \
        --instance-id "{{ instance.id }}" \
        --region "{{ instance.region }}"

```

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/ec2_vpc_route_table.py b/lib/ansible/modules/cloud/amazon/ec2_vpc_route_table.py
--- a/lib/ansible/modules/cloud/amazon/ec2_vpc_route_table.py
+++ b/lib/ansible/modules/cloud/amazon/ec2_vpc_route_table.py
@@ -334,6 +334,11 @@ def route_spec_matches_route(route_spec, route):
     return True
 
 
+def route_spec_matches_route_cidr(route_spec, route):
+    cidr_attr = 'destination_cidr_block'
+    return route_spec[cidr_attr] == getattr(route, cidr_attr)
+
+
 def rename_key(d, old_key, new_key):
     d[new_key] = d[old_key]
     del d[old_key]
@@ -343,16 +348,21 @@ def index_of_matching_route(route_spec, routes_to_match):
     for i, route in enumerate(routes_to_match):
         if route_spec_matches_route(route_spec, route):
             return i
+        elif route_spec_matches_route_cidr(route_spec, route):
+            return "replace"
 
 
 def ensure_routes(vpc_conn, route_table, route_specs, propagating_vgw_ids,
                   check_mode, purge_routes):
     routes_to_match = list(route_table.routes)
     route_specs_to_create = []
+    route_specs_to_recreate = []
     for route_spec in route_specs:
         i = index_of_matching_route(route_spec, routes_to_match)
         if i is None:
             route_specs_to_create.append(route_spec)
+        elif i == "replace":
+            route_specs_to_recreate.append(route_spec)
         else:
             del routes_to_match[i]
 
@@ -372,7 +382,7 @@ def ensure_routes(vpc_conn, route_table, route_specs, propagating_vgw_ids,
             else:
                 routes_to_delete.append(r)
 
-    changed = bool(routes_to_delete or route_specs_to_create)
+    changed = bool(routes_to_delete or route_specs_to_create or route_specs_to_recreate)
     if changed:
         for route in routes_to_delete:
             try:
@@ -392,6 +402,11 @@ def ensure_routes(vpc_conn, route_table, route_specs, propagating_vgw_ids,
                 if e.error_code == 'DryRunOperation':
                     pass
 
+        for route_spec in route_specs_to_recreate:
+            if not check_mode:
+                vpc_conn.replace_route(route_table.id,
+                                       **route_spec)
+
     return {'changed': bool(changed)}
 
 


Test patch: