Repository: ansible/ansible
Pull request: Add execution_role_arn parameter
##### SUMMARY
Fargate support requires containers to have the executionRoleArn defined. This PR adds that parameter.
Fixes #41848
##### ISSUE TYPE
 - Feature Pull Request

##### COMPONENT NAME
ecs_taskdefinition

##### ANSIBLE VERSION
```
ansible 2.7.0.dev0 (devel 46dbb3e1e8) last updated 2018/06/22 10:32:41 (GMT -700)
  config file = None
  configured module search path = [u'/home/mjmayer/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mjmayer/ansible/lib/ansible
  executable location = /home/mjmayer/ansible/bin/ansible
  python version = 2.7.5 (default, Apr 11 2018, 17:41:36) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28.0.1)]

```


Problem statement:
ecs_taskdefinition does not support execution_role_arn
##### SUMMARY
Deploying fargate containers requires the containers to have a execution role arn assigned. esc_taskdefinition does not currently support this parameter.

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
ecs_taskdefinition

##### ANSIBLE VERSION
```
ansible 2.7.0.dev0 (devel 46dbb3e1e8) last updated 2018/06/22 10:32:41 (GMT -700)
  config file = None
  configured module search path = [u'/home/mjmayer/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mjmayer/ansible/lib/ansible
  executable location = /home/mjmayer/ansible/bin/ansible
  python version = 2.7.5 (default, Apr 11 2018, 17:41:36) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28.0.1)]

```

##### CONFIGURATION

##### OS / ENVIRONMENT
N/A

##### STEPS TO REPRODUCE
```yaml
- name: Create task definition
  ecs_taskdefinition:
    family: flask
    containers:
    - name: flask
      essential: true
      image: "id.dkr.ecr.us-west-2.amazonaws.com/flask:latest"
      portMappings:
      - containerPort: 5000
        hostPort:      5000
      logConfiguration:
        logDriver: awslogs
        options:
          awslogs-region: us-west-2
          awslogs-group: /ecs/flask
          awslogs-stream-prefix: ecs
    launch_type: FARGATE
    cpu: 512
    memory: 1024
    network_mode: awsvpc
    state: present
```

##### EXPECTED RESULTS

##### ACTUAL RESULTS
```
TASK [flask/app : Create task definition] ***************************************************************************************************************************************************************************
task path: /home/mjmayer/ait-cluster-np/roles/flask/app/tasks/main.yml:8
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: mjmayer
<127.0.0.1> EXEC /bin/sh -c 'echo ~mjmayer && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/mjmayer/.ansible/tmp/ansible-tmp-1529688836.67-2731985955396 `" && echo ansible-tmp-1529688836.67-2731985955396="` echo /home/mjmayer/.ansible/tmp/
ansible-tmp-1529688836.67-2731985955396 `" ) && sleep 0'
Using module file /home/mjmayer/ansible/lib/ansible/modules/cloud/amazon/ecs_taskdefinition.py
<127.0.0.1> PUT /home/mjmayer/.ansible/tmp/ansible-local-48933eLALRW/tmpeplZIR TO /home/mjmayer/.ansible/tmp/ansible-tmp-1529688836.67-2731985955396/ecs_taskdefinition.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/mjmayer/.ansible/tmp/ansible-tmp-1529688836.67-2731985955396/ /home/mjmayer/.ansible/tmp/ansible-tmp-1529688836.67-2731985955396/ecs_taskdefinition.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /home/mjmayer/.ansible/tmp/ansible-tmp-1529688836.67-2731985955396/ecs_taskdefinition.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/mjmayer/.ansible/tmp/ansible-tmp-1529688836.67-2731985955396/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_vTzbf6/ansible_module_ecs_taskdefinition.py", line 240, in register_task
    response = self.ecs.register_task_definition(**params)
  File "/usr/lib/python2.7/site-packages/botocore/client.py", line 314, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python2.7/site-packages/botocore/client.py", line 612, in _make_api_call
    raise error_class(parsed_response, operation_name)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "error": {
        "code": "ClientException",
        "message": "Fargate requires task definition to have execution role ARN to support log driver awslogs."
    },
    "invocation": {
        "module_args": {
            "arn": null,
            "aws_access_key": null,
            "aws_secret_key": null,
            "containers": [
                {
                    "essential": true,
                    "image": "989838493125.dkr.ecr.us-west-2.amazonaws.com/flaskr:latest",
                    "logConfiguration": {
                        "logDriver": "awslogs",
                        "options": {
                            "awslogs-group": "/ecs/flask",
                            "awslogs-region": "us-west-2",
                            "awslogs-stream-prefix": "ecs"
                        }
                    },
                    "name": "flask",
                    "portMappings": [
                        {
                            "containerPort": 5000,
                            "hostPort": 5000
                        }
                    ]
                }
            ],
            "cpu": "512",
            "ec2_url": null,
            "family": "flask",
            "force_create": false,
            "launch_type": "FARGATE",
            "memory": "1024",
            "network_mode": "awsvpc",
            "profile": null,
            "region": null,
            "revision": null,
            "security_token": null,
            "state": "present",
            "task_role_arn": "",
            "validate_certs": true,
            "volumes": null
        }
    },
    "msg": "An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Fargate requires task definition to have execution role ARN to support log driver awslogs.",
    "response_metadata": {
        "http_headers": {
            "connection": "keep-alive",
            "content-length": "131",
            "content-type": "application/x-amz-json-1.1",
            "date": "Fri, 22 Jun 2018 17:33:58 GMT",
            "server": "Server",
            "x-amzn-requestid": "71a2219e-7642-11e8-b904-39778a45d6a7"
        },
        "http_status_code": 400,
        "request_id": "71a2219e-7642-11e8-b904-39778a45d6a7",
        "retry_attempts": 0
    }
}
        to retry, use: --limit @/home/mjmayer/ait-cluster-np/aws.retry


```

Hints from issue discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/amazon/ecs_taskdefinition.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/ecs_taskdefinition.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Java1Guy @willthames
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Commit message: Add test for execution role parameter

Patch to review:
diff --git a/lib/ansible/modules/cloud/amazon/ecs_taskdefinition.py b/lib/ansible/modules/cloud/amazon/ecs_taskdefinition.py
index ac336fce816254..37aacf681a3ac0 100644
--- a/lib/ansible/modules/cloud/amazon/ecs_taskdefinition.py
+++ b/lib/ansible/modules/cloud/amazon/ecs_taskdefinition.py
@@ -69,6 +69,11 @@
               the permissions that are specified in this role.
         required: false
         version_added: 2.3
+    execution_role_arn:
+        description:
+            - The Amazon Resource Name (ARN) of the task execution role that the Amazon ECS container agent and the Docker daemon can assume.
+        required: false
+        version_added: 2.7
     volumes:
         description:
             - A list of names of volumes to be attached
@@ -180,7 +185,7 @@
 except ImportError:
     HAS_BOTO3 = False
 
-from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils.aws.core import AnsibleAWSModule
 from ansible.module_utils.ec2 import boto3_conn, camel_dict_to_snake_dict, ec2_argument_spec, get_aws_connection_info
 from ansible.module_utils._text import to_text
 
@@ -201,7 +206,7 @@ def describe_task(self, task_name):
         except botocore.exceptions.ClientError:
             return None
 
-    def register_task(self, family, task_role_arn, network_mode, container_definitions, volumes, launch_type, cpu, memory):
+    def register_task(self, family, task_role_arn, execution_role_arn, network_mode, container_definitions, volumes, launch_type, cpu, memory):
         validated_containers = []
 
         # Ensures the number parameters are int as required by boto
@@ -235,6 +240,8 @@ def register_task(self, family, task_role_arn, network_mode, container_definitio
             params['memory'] = memory
         if launch_type:
             params['requiresCompatibilities'] = [launch_type]
+        if execution_role_arn:
+            params['executionRoleArn'] = execution_role_arn
 
         try:
             response = self.ecs.register_task_definition(**params)
@@ -279,12 +286,6 @@ def deregister_task(self, taskArn):
         response = self.ecs.deregister_task_definition(taskDefinition=taskArn)
         return response['taskDefinition']
 
-    def ecs_api_supports_requirescompatibilities(self):
-        from distutils.version import LooseVersion
-        # Checking to make sure botocore is greater than a specific version.
-        # Support for requiresCompatibilities is only available in versions beyond 1.8.4
-        return LooseVersion(botocore.__version__) >= LooseVersion('1.8.4')
-
 
 def main():
     argument_spec = ec2_argument_spec()
@@ -297,16 +298,17 @@ def main():
         containers=dict(required=False, type='list'),
         network_mode=dict(required=False, default='bridge', choices=['bridge', 'host', 'none', 'awsvpc'], type='str'),
         task_role_arn=dict(required=False, default='', type='str'),
+        execution_role_arn=dict(required=False, default='', type='str'),
         volumes=dict(required=False, type='list'),
         launch_type=dict(required=False, choices=['EC2', 'FARGATE']),
         cpu=dict(),
         memory=dict(required=False, type='str')
     ))
 
-    module = AnsibleModule(argument_spec=argument_spec,
-                           supports_check_mode=True,
-                           required_if=[('launch_type', 'FARGATE', ['cpu', 'memory'])]
-                           )
+    module = AnsibleAWSModule(argument_spec=argument_spec,
+                              supports_check_mode=True,
+                              required_if=[('launch_type', 'FARGATE', ['cpu', 'memory'])]
+                              )
 
     if not HAS_BOTO3:
         module.fail_json(msg='boto3 is required.')
@@ -316,9 +318,13 @@ def main():
     results = dict(changed=False)
 
     if module.params['launch_type']:
-        if not task_mgr.ecs_api_supports_requirescompatibilities():
+        if not module.botocore_at_least('1.8.4'):
             module.fail_json(msg='botocore needs to be version 1.8.4 or higher to use launch_type')
 
+    if module.params['execution_role_arn']:
+        if not module.botocore_at_least('1.10.44'):
+            module.fail_json(msg='botocore needs to be version 1.10.44 or higher to use execution_role_arn')
+
     for container in module.params.get('containers', []):
         for environment in container.get('environment', []):
             environment['value'] = to_text(environment['value'])
@@ -442,6 +448,7 @@ def _task_definition_matches(requested_volumes, requested_containers, existing_t
                 volumes = module.params.get('volumes', []) or []
                 results['taskdefinition'] = task_mgr.register_task(module.params['family'],
                                                                    module.params['task_role_arn'],
+                                                                   module.params['execution_role_arn'],
                                                                    module.params['network_mode'],
                                                                    module.params['containers'],
                                                                    volumes,
diff --git a/test/integration/targets/ecs_cluster/playbooks/network_fail.yml b/test/integration/targets/ecs_cluster/playbooks/network_fail.yml
index 72c2f8876e7bbf..7d026d806fac17 100644
--- a/test/integration/targets/ecs_cluster/playbooks/network_fail.yml
+++ b/test/integration/targets/ecs_cluster/playbooks/network_fail.yml
@@ -44,6 +44,26 @@
             <<: *aws_connection_info
           register: ecs_taskdefinition_creation_vpc
 
+        - name: create ecs_taskdefinition and execution_role_arn (expected to fail)
+          ecs_taskdefinition:
+            containers:
+              - name: my_container
+                image: ubuntu
+                memory: 128
+            family: "{{ resource_prefix }}-vpc"
+            execution_role_arn: not_a_real_arn
+            state: present
+            network_mode: awsvpc
+            <<: *aws_connection_info
+          ignore_errors: yes
+          register: ecs_taskdefinition_arn
+
+        - name: check that graceful failure message is returned from ecs_taskdefinition_arn
+          assert:
+            that:
+              - ecs_taskdefinition_arn.failed
+              - 'ecs_taskdefinition_arn.msg == "botocore needs to be version 1.10.44 or higher to use execution_role_arn"'
+
         - name: ecs_taskdefinition works fine even when older botocore is used
           assert:
             that:
diff --git a/test/integration/targets/ecs_cluster/playbooks/roles/ecs_cluster/tasks/main.yml b/test/integration/targets/ecs_cluster/playbooks/roles/ecs_cluster/tasks/main.yml
index c30cdb2a999889..893c4c3e336734 100644
--- a/test/integration/targets/ecs_cluster/playbooks/roles/ecs_cluster/tasks/main.yml
+++ b/test/integration/targets/ecs_cluster/playbooks/roles/ecs_cluster/tasks/main.yml
@@ -541,6 +541,23 @@
     # ============================================================
     # Begin tests for Fargate
 
+    - name: ensure AmazonECSTaskExecutionRolePolicy exists
+      iam_role:
+        name: ecsTaskExecutionRole
+        assume_role_policy_document: "{{ lookup('file','ecs-trust-policy.json') }}"
+        description: "Allows ECS containers to make calls to ECR"
+        state: present
+        create_instance_profile: no
+        managed_policy:
+        - AmazonEC2ContainerServiceRole
+        <<: *aws_connection_info
+
+    - name: gather facts for the ecsTaskExecutionRole
+      iam_role_facts:
+        name: ecsTaskExecutionRole
+        <<: *aws_connection_info
+      register: iam_execution_role
+
     - name: create Fargate VPC-networked task definition with host port set to 8080 and unsupported network mode (expected to fail)
       ecs_taskdefinition:
         containers: "{{ ecs_fargate_task_containers }}"
@@ -579,7 +596,7 @@
           - ecs_fargate_task_definition_vpc_no_mem is failed
           - 'ecs_fargate_task_definition_vpc_no_mem.msg == "launch_type is FARGATE but all of the following are missing: cpu, memory"' 
 
-    - name: create Fargate VPC-networked task definition with CPU or Memory 
+    - name: create Fargate VPC-networked task definition with CPU or Memory and execution role
       ecs_taskdefinition:
         containers: "{{ ecs_fargate_task_containers }}"
         family: "{{ ecs_task_name }}-vpc"
@@ -587,6 +604,7 @@
         launch_type: FARGATE
         cpu: 512
         memory: 1024
+        execution_role_arn: "{{ iam_execution_role.iam_roles.0.arn }}"
         state: present
         <<: *aws_connection_info
       vars:


Code review comments:
diff_hunk: @@ -541,6 +541,23 @@
     # ============================================================
     # Begin tests for Fargate
 
+    - name: ensure AmazonECSTaskExecutionRolePolicy exists
+      iam_role:
+        name: ecsTaskExecutionRole
+        assume_role_policy_document: "{{ lookup('file','ecs-trust-policy.json') }}"
+        description: "Allows ECS containers to make calls to ECR"
+        state: present
+        create_instance_profile: no
+        managed_policy:
+        - AmazonEC2ContainerServiceRole
+        <<: *aws_connection_info
+
+    - name: gather facts for the ecsTaskExecutionRole
path: test/integration/targets/ecs_cluster/playbooks/roles/ecs_cluster/tasks/main.yml
text: @user1:
Should be able to access this from `register`ing the results of the previous task

@author:
I registered the output from iam_role task and reran the tests. The tests passed.