Repository: ansible/ansible

Issue:
Batch mode for delegate_to and local_action (take 2)
I'll try this again.  I had suggested a feature in #2389 which was closed
before I think anyone even understood what the suggestion was.  I tried to
clarify what I meant, but it seems like comments to a closed issue go to
/dev/null.  

We have roughly 1200 servers which are currently configured with cfengine.
Cfengine is good for determining what to do based on dynamic data, but I have
wanted to find something that can handle group complexity better without
becoming unwieldy or be a resource hog.  Obviously, I'm toying around
with ansible, and trying to get it to wrap around a lot of existing stuff.

I have several times wished that delegate_to had an option to do batch
processing instead of firing off for each host in the loop and killing the
delegate server.  A batch job is supposedly possible by working around the problem as
described here:
http://jpmens.net/2012/11/03/an-action-plugin-for-ansible-to-handle-ssh-host-keys/

That solution seems less than obvious.  As a newbie working through this, the
most intuitive thing I could see would be a "batch=yes" option or something
similar, which would cause delegate_to to run once, and just send the entire
list of hosts ansible is going to loop over.  

Anyway, just a suggestion! :)



Discussion:
I think I understand now.

Right now, the functionality you seek needs to exist in the module itself.    I think what you are asking for is exactly how yum and apt work.

This can be added to basically any module, but then we also have to teach the module how to work with lists (like apt and yum are told they are special).   It would be nice if the module itself could declare this versus the hard code, which is reasonable.

What load balancer are you using?  (Or just the shell command?)

We could teach 'shell' to do "with_items" using the ";" as a seperator, perhaps. 

I think you might be understanding it, just backwards.  Switch around which
direction the list is going, and you'll have it.  The module isn't accepting a
list, the command being run is getting one, maybe with a special variable.

Perhaps I can make it clear with a ridiculous example.  Pretend that I work
just like ansible, and I have to help orchestrate a birthday party.

<pre>
---
- name: Birthday party
  guests: Joe Jimmy John Susy our_kids kids_from_school
  tasks:
    - name: make somthing to eat
      shell: /home/prepare.sh --food cupcakes --food pizza --for ${guest_name}
      delegate_to: wife
      register: food
  
    - name: feed the kids
      action: feed ${food}
  
    - name: distribute pins for pin-the-tail-on-the-donkey
      action: give_pin
</pre>

Here would be a good example of a chance for a special variable and a
"run_once" or "batch" option.   I could call my wife and say, "Hi Honey, here is
who is coming to the party.  It's just going to be Joe.  Go ahead and get the
food ready".  After she has ordered a small pizza, cooked one cupcake and cleaned
the kitchen, I call her, and tell her about another guest.  I do the same
thing for every guest.  She would most likely come at me with a kitchen knife when I
got home.  If I could do something like the following, I think the party would
be a lot less stressful for everyone:

<pre>
---
- name: Birthday party
  guests: Joe Jimmy John Susy our_kids kids_from_school
  tasks:
    - name: prepare somthing to eat
      shell: /home/prepare.sh --food cupcakes --food pizza --for ${everyone_who_is_coming}
      delegate_to: wife
      register: food
      run_once: yes
</pre>

She could use the "everyone_who_is_coming" special variable, and only do the task once.
Hopefully that gives the idea.

This feels like complex_args to me and making a module take a list and knowing what to do with it, but I am closing this because we don't support cupcakes :)

cheribral, I will be glad to hear if you found a solution for this issue.


Resolving patch:
diff --git a/lib/ansible/module_utils/ec2.py b/lib/ansible/module_utils/ec2.py
--- a/lib/ansible/module_utils/ec2.py
+++ b/lib/ansible/module_utils/ec2.py
@@ -354,6 +354,28 @@ def value_is_list(camel_list):
     return snake_dict
 
 
+def snake_dict_to_camel_dict(snake_dict):
+
+    def camelize(complex_type):
+        if complex_type is None:
+            return
+        new_type = type(complex_type)()
+        if isinstance(complex_type, dict):
+            for key in complex_type:
+                new_type[camel(key)] = camelize(complex_type[key])
+        elif isinstance(complex_type, list):
+            for i in range(len(complex_type)):
+                new_type.append(camelize(complex_type[i]))
+        else:
+            return complex_type
+        return new_type
+
+    def camel(words):
+        return words.split('_')[0] + ''.join(x.capitalize() or '_' for x in words.split('_')[1:])
+
+    return camelize(snake_dict)
+
+
 def ansible_dict_to_boto3_filter_list(filters_dict):
 
     """ Convert an Ansible dict of filters to list of dicts that boto3 can use
@@ -556,3 +578,45 @@ def value_is_list(my_list):
             ordered_policy_dict[key] = value
 
     return ordered_policy_dict
+
+
+def map_complex_type(complex_type, type_map):
+    """
+        Allows to cast elements within a dictionary to a specific type
+        Example of usage:
+
+        DEPLOYMENT_CONFIGURATION_TYPE_MAP = {
+            'maximum_percent': 'int',
+            'minimum_healthy_percent': 'int'
+        }
+
+        deployment_configuration = map_complex_type(module.params['deployment_configuration'],
+                                                    DEPLOYMENT_CONFIGURATION_TYPE_MAP)
+
+        This ensures all keys within the root element are casted and valid integers
+    """
+
+    if complex_type is None:
+        return
+    new_type = type(complex_type)()
+    if isinstance(complex_type, dict):
+        for key in complex_type:
+            if key in type_map:
+                if isinstance(type_map[key], list):
+                    new_type[key] = map_complex_type(
+                        complex_type[key],
+                        type_map[key][0])
+                else:
+                    new_type[key] = map_complex_type(
+                        complex_type[key],
+                        type_map[key])
+            else:
+                return complex_type
+    elif isinstance(complex_type, list):
+        for i in range(len(complex_type)):
+            new_type.append(map_complex_type(
+                complex_type[i],
+                type_map))
+    elif type_map:
+        return vars(globals()['__builtins__'])[type_map](complex_type)
+    return new_type
\ No newline at end of file
diff --git a/lib/ansible/modules/cloud/amazon/ecs_service.py b/lib/ansible/modules/cloud/amazon/ecs_service.py
--- a/lib/ansible/modules/cloud/amazon/ecs_service.py
+++ b/lib/ansible/modules/cloud/amazon/ecs_service.py
@@ -33,6 +33,9 @@
 author:
     - "Mark Chance (@java1guy)"
     - "Darek Kaczynski (@kaczynskid)"
+    - "Stephane Maarek (@simplesteph)"
+    - "Zac Blazic (@zacblazic)"
+
 requirements: [ json, boto, botocore, boto3 ]
 options:
     state:
@@ -78,6 +81,11 @@
           - The number of times to check that the service is available
         required: false
         default: 10
+    deployment_configuration:
+        description:
+          - Optional parameters that control the deployment_configuration; format is '{"maximum_percent":<integer>, "minimum_healthy_percent":<integer>}
+        required: false
+        version_added: 2.3
 extends_documentation_fragment:
     - aws
     - ec2
@@ -103,6 +111,17 @@
     name: default
     state: absent
     cluster: new_cluster
+
+# With custom deployment configuration
+- ecs_service:
+    name: test-service
+    cluster: test-cluster
+    task_definition: test-task-definition
+    desired_count: 3
+    deployment_configuration:
+      minimum_healthy_percent: 75
+      maximum_percent: 150
+    state: present
 '''
 
 RETURN = '''
@@ -164,6 +183,19 @@
             description: list of service deployments
             returned: always
             type: list of complex
+        deploymentConfiguration:
+            description: dictionary of deploymentConfiguration
+            returned: always
+            type: complex
+            contains:
+                maximumPercent:
+                    description: maximumPercent param
+                    returned: always
+                    type: int
+                minimumHealthyPercent:
+                    description: minimumHealthyPercent param
+                    returned: always
+                    type: int
         events:
             description: lost of service events
             returned: always
@@ -180,6 +212,12 @@
 '''
 import time
 
+DEPLOYMENT_CONFIGURATION_TYPE_MAP = {
+    'maximum_percent': 'int',
+    'minimum_healthy_percent': 'int'
+}
+
+
 try:
     import boto
     import botocore
@@ -194,7 +232,7 @@
     HAS_BOTO3 = False
 
 from ansible.module_utils.basic import AnsibleModule
-from ansible.module_utils.ec2 import boto3_conn, ec2_argument_spec, get_aws_connection_info
+from ansible.module_utils.ec2 import boto3_conn, ec2_argument_spec, get_aws_connection_info, snake_dict_to_camel_dict, map_complex_type
 
 
 class EcsServiceManager:
@@ -204,7 +242,6 @@ def __init__(self, module):
         self.module = module
 
         try:
-            # self.ecs = boto3.client('ecs')
             region, ec2_url, aws_connect_kwargs = get_aws_connection_info(module, boto3=True)
             if not region:
                 module.fail_json(msg="Region must be specified as a parameter, in EC2_REGION or AWS_REGION environment variables or in boto configuration file")
@@ -212,15 +249,6 @@ def __init__(self, module):
         except boto.exception.NoAuthHandlerFound as e:
             self.module.fail_json(msg="Can't authorize connection - %s" % str(e))
 
-    # def list_clusters(self):
-    #     return self.client.list_clusters()
-    # {'failures=[],
-    # 'ResponseMetadata={'HTTPStatusCode=200, 'RequestId='ce7b5880-1c41-11e5-8a31-47a93a8a98eb'},
-    # 'clusters=[{'activeServicesCount=0, 'clusterArn='arn:aws:ecs:us-west-2:777110527155:cluster/default', 'status='ACTIVE', 'pendingTasksCount=0, 'runningTasksCount=0, 'registeredContainerInstancesCount=0, 'clusterName='default'}]}
-    # {'failures=[{'arn='arn:aws:ecs:us-west-2:777110527155:cluster/bogus', 'reason='MISSING'}],
-    # 'ResponseMetadata={'HTTPStatusCode=200, 'RequestId='0f66c219-1c42-11e5-8a31-47a93a8a98eb'},
-    # 'clusters=[]}
-
     def find_in_array(self, array_of_services, service_name, field_name='serviceArn'):
         for c in array_of_services:
             if c[field_name].endswith(service_name):
@@ -259,7 +287,7 @@ def is_matching_service(self, expected, existing):
         return True
 
     def create_service(self, service_name, cluster_name, task_definition,
-        load_balancers, desired_count, client_token, role):
+                       load_balancers, desired_count, client_token, role, deployment_configuration):
         response = self.ecs.create_service(
             cluster=cluster_name,
             serviceName=service_name,
@@ -267,16 +295,18 @@ def create_service(self, service_name, cluster_name, task_definition,
             loadBalancers=load_balancers,
             desiredCount=desired_count,
             clientToken=client_token,
-            role=role)
+            role=role,
+            deploymentConfiguration=deployment_configuration)
         return self.jsonize(response['service'])
 
     def update_service(self, service_name, cluster_name, task_definition,
-        load_balancers, desired_count, client_token, role):
+        load_balancers, desired_count, client_token, role, deployment_configuration):
         response = self.ecs.update_service(
             cluster=cluster_name,
             service=service_name,
             taskDefinition=task_definition,
-            desiredCount=desired_count)
+            desiredCount=desired_count,
+            deploymentConfiguration=deployment_configuration)
         return self.jsonize(response['service'])
 
     def jsonize(self, service):
@@ -297,7 +327,6 @@ def jsonize(self, service):
     def delete_service(self, service, cluster=None):
         return self.ecs.delete_service(cluster=cluster, service=service)
 
-
 def main():
 
     argument_spec = ec2_argument_spec()
@@ -311,7 +340,8 @@ def main():
         client_token=dict(required=False, default='', type='str'),
         role=dict(required=False, default='', type='str'),
         delay=dict(required=False, type='int', default=10),
-        repeat=dict(required=False, type='int', default=10)
+        repeat=dict(required=False, type='int', default=10),
+        deployment_configuration=dict(required=False, default={}, type='dict')
     ))
 
     module = AnsibleModule(argument_spec=argument_spec,
@@ -329,6 +359,12 @@ def main():
       module.fail_json(msg='boto3 is required.')
 
     service_mgr = EcsServiceManager(module)
+
+    deployment_configuration = map_complex_type(module.params['deployment_configuration'],
+                                                            DEPLOYMENT_CONFIGURATION_TYPE_MAP)
+
+    deploymentConfiguration = snake_dict_to_camel_dict(deployment_configuration)
+
     try:
         existing = service_mgr.describe_service(module.params['cluster'], module.params['name'])
     except Exception as e:
@@ -360,7 +396,8 @@ def main():
                         loadBalancers,
                         module.params['desired_count'],
                         clientToken,
-                        role)
+                        role,
+                        deploymentConfiguration)
                 else:
                     # doesn't exist. create it.
                     response = service_mgr.create_service(module.params['name'],
@@ -369,7 +406,8 @@ def main():
                         loadBalancers,
                         module.params['desired_count'],
                         clientToken,
-                        role)
+                        role,
+                        deploymentConfiguration)
 
                 results['service'] = response
 


Test patch: