Repository: ansible/ansible

Issue:
cloudformation_facts fails if stack doesn't exist
##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
cloudformation_facts module

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.2.3.0 (stable-2.2 7ff9fa52cf) last updated 2017/04/07 11:55:42 (GMT -700)
  lib/ansible/modules/core: (detached HEAD cfe3c844f4) last updated 2017/04/07 11:55:49 (GMT -700)
  lib/ansible/modules/extras: (detached HEAD 8b78d56a08) last updated 2017/04/07 11:55:49 (GMT -700)
  config file = /Users/guy.knights/.ansible.cfg
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
* Added 'hash_behaviour = merge'

##### OS / ENVIRONMENT
Control machine: OSX 10.12.3
Managed machine: same (playbook execution=local)

##### SUMMARY
The cloudformation_facts module causes ansible-playbook to fail if the stack name given doesn't exist. In my use case, I have a playbook that's creating IAM user keys, and it's possible to pass in an incremented serial number if the keys are to be rotated. I want to get this number from the existing stack, or set it to 0 if the stack doesn't yet exist, but the playbook fails when cloudformation_facts is run.

I'm currently using ignore_errors=true on this task, which works, but it spits out a big red error message before continuing. It would be nicer if it just continued but returned a variable with a "stack_exists" value set to false, or something similar.

##### STEPS TO REPRODUCE
Create at task using the cloudformation_facts module, passing in the name of a stack that doesn't exist.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- cloudformation_facts:
    stack_name: "{{ vpcenvironment }}-iam"
    region: "{{ whitelabel_data[wl_name].region }}"
    profile: "{{ wl_name }}"
  register: stack_iam_data

- block:
  - set_fact:
      key_index: "{{ stack_iam_data['ansible_facts']['cloudformation'][vpcenvironment + '-iam']['stack_description']['parameters']|selectattr('parameter_key', 'equalto', 'AccessKeySerial')|map(attribute='parameter_value')|list }}"

  - set_fact:
      key_serial: "{{ key_index[0]|int }}"
    when: rotate_keys == 'n'

  - set_fact:
      key_serial: "{{ key_index[0]|int + 1 }}"
    when: rotate_keys == 'y'
  when: stack_iam_data['failed'] is not defined

- name: create IAM stack
  cloudformation:
    stack_name: "{{ vpcenvironment }}-iam"
    state: present
    region: "{{ whitelabel_data[wl_name].region }}"
    template_url: https://bbcloudformation.s3.amazonaws.com/GTP/GTPIAMConfig.json
    profile: "{{ wl_name }}"
    template_parameters:
      GTPWhitelabelName: "{{ wl_name }}"
      Environment: "{{ vpcenvironment }}"
      AccessKeySerial: "{{ key_serial | default(0) }}"
    tags:
      Name: "{{ wl_name }}"
      Type: iam
      Creator: ansible
      Environment: "{{ vpcenvironment }}"
```

##### EXPECTED RESULTS
Playbook execution would continue, but stack_iam_data would be empty or have a key to indicate no data was returned.

##### ACTUAL RESULTS
Playbook execution fails with big red error message saying stack doesn't exist.

```
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ClientError: An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id dev-iam does not exist
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Error describing stack - An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id dev-iam does not exist"}
```

cloudformation_facts fails if stack doesn't exist
##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
cloudformation_facts module

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.2.3.0 (stable-2.2 7ff9fa52cf) last updated 2017/04/07 11:55:42 (GMT -700)
  lib/ansible/modules/core: (detached HEAD cfe3c844f4) last updated 2017/04/07 11:55:49 (GMT -700)
  lib/ansible/modules/extras: (detached HEAD 8b78d56a08) last updated 2017/04/07 11:55:49 (GMT -700)
  config file = /Users/guy.knights/.ansible.cfg
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
* Added 'hash_behaviour = merge'

##### OS / ENVIRONMENT
Control machine: OSX 10.12.3
Managed machine: same (playbook execution=local)

##### SUMMARY
The cloudformation_facts module causes ansible-playbook to fail if the stack name given doesn't exist. In my use case, I have a playbook that's creating IAM user keys, and it's possible to pass in an incremented serial number if the keys are to be rotated. I want to get this number from the existing stack, or set it to 0 if the stack doesn't yet exist, but the playbook fails when cloudformation_facts is run.

I'm currently using ignore_errors=true on this task, which works, but it spits out a big red error message before continuing. It would be nicer if it just continued but returned a variable with a "stack_exists" value set to false, or something similar.

##### STEPS TO REPRODUCE
Create at task using the cloudformation_facts module, passing in the name of a stack that doesn't exist.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- cloudformation_facts:
    stack_name: "{{ vpcenvironment }}-iam"
    region: "{{ whitelabel_data[wl_name].region }}"
    profile: "{{ wl_name }}"
  register: stack_iam_data

- block:
  - set_fact:
      key_index: "{{ stack_iam_data['ansible_facts']['cloudformation'][vpcenvironment + '-iam']['stack_description']['parameters']|selectattr('parameter_key', 'equalto', 'AccessKeySerial')|map(attribute='parameter_value')|list }}"

  - set_fact:
      key_serial: "{{ key_index[0]|int }}"
    when: rotate_keys == 'n'

  - set_fact:
      key_serial: "{{ key_index[0]|int + 1 }}"
    when: rotate_keys == 'y'
  when: stack_iam_data['failed'] is not defined

- name: create IAM stack
  cloudformation:
    stack_name: "{{ vpcenvironment }}-iam"
    state: present
    region: "{{ whitelabel_data[wl_name].region }}"
    template_url: https://bbcloudformation.s3.amazonaws.com/GTP/GTPIAMConfig.json
    profile: "{{ wl_name }}"
    template_parameters:
      GTPWhitelabelName: "{{ wl_name }}"
      Environment: "{{ vpcenvironment }}"
      AccessKeySerial: "{{ key_serial | default(0) }}"
    tags:
      Name: "{{ wl_name }}"
      Type: iam
      Creator: ansible
      Environment: "{{ vpcenvironment }}"
```

##### EXPECTED RESULTS
Playbook execution would continue, but stack_iam_data would be empty or have a key to indicate no data was returned.

##### ACTUAL RESULTS
Playbook execution fails with big red error message saying stack doesn't exist.

```
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ClientError: An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id dev-iam does not exist
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Error describing stack - An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id dev-iam does not exist"}
```



Discussion:
cc @jmenga
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
You could use the Ansible blocks feature to handle the failure - http://docs.ansible.com/ansible/playbooks_blocks.html#error-handling

```
- block:
  - cloudformation_facts:
       stack_name: "{{ vpcenvironment }}-iam"
       region: "{{ whitelabel_data[wl_name].region }}"
       profile: "{{ wl_name }}"
     register: stack_iam_data
  - set_fact:
      key_index: "{{ stack_iam_data['ansible_facts']['cloudformation'][vpcenvironment + '-iam']['stack_description']['parameters']|selectattr('parameter_key', 'equalto', 'AccessKeySerial')|map(attribute='parameter_value')|list }}"
  - set_fact:
      key_serial: "{{ key_index[0]|int }}"
    when: rotate_keys == 'n'
  - set_fact:
      key_serial: "{{ key_index[0]|int + 1 }}"
    when: rotate_keys == 'y'
  rescue:
  - debug: msg="Perform some exception handling tasks here"
```
Interesting, I will check that out. It's been a while since I've looked at the block feature and I hadn't read the error handling section.

This should be a satisfactory workaround for my issue but I still kind of feel like the best workflow would be that the module shouldn't fail, just return a message to indicate the stack wasn't found. It's not like a facts module (or, at least this one) makes a change which could break things.
Sounds reasonable - the failed key signals failure to Ansible, so perhaps setting the return code to non-zero with empty output is the best approach based upon the best practices guide - 
http://docs.ansible.com/ansible/dev_guide/developing_modules_best_practices.html
Yes, and actually I just tried the block/rescue approach you outlined and while it does indeed work fine, it still spits out a big red error message which looks nasty and in certain situations possibly confusing.
You can suppress (**ALL**) errors from any task using `failed_when: false` but I agree it is not ideal
Good suggestion. I did that and it's helped me achieve what I was after.

Thanks!
cc @jmenga
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
You could use the Ansible blocks feature to handle the failure - http://docs.ansible.com/ansible/playbooks_blocks.html#error-handling

```
- block:
  - cloudformation_facts:
       stack_name: "{{ vpcenvironment }}-iam"
       region: "{{ whitelabel_data[wl_name].region }}"
       profile: "{{ wl_name }}"
     register: stack_iam_data
  - set_fact:
      key_index: "{{ stack_iam_data['ansible_facts']['cloudformation'][vpcenvironment + '-iam']['stack_description']['parameters']|selectattr('parameter_key', 'equalto', 'AccessKeySerial')|map(attribute='parameter_value')|list }}"
  - set_fact:
      key_serial: "{{ key_index[0]|int }}"
    when: rotate_keys == 'n'
  - set_fact:
      key_serial: "{{ key_index[0]|int + 1 }}"
    when: rotate_keys == 'y'
  rescue:
  - debug: msg="Perform some exception handling tasks here"
```
Interesting, I will check that out. It's been a while since I've looked at the block feature and I hadn't read the error handling section.

This should be a satisfactory workaround for my issue but I still kind of feel like the best workflow would be that the module shouldn't fail, just return a message to indicate the stack wasn't found. It's not like a facts module (or, at least this one) makes a change which could break things.
Sounds reasonable - the failed key signals failure to Ansible, so perhaps setting the return code to non-zero with empty output is the best approach based upon the best practices guide - 
http://docs.ansible.com/ansible/dev_guide/developing_modules_best_practices.html
Yes, and actually I just tried the block/rescue approach you outlined and while it does indeed work fine, it still spits out a big red error message which looks nasty and in certain situations possibly confusing.
You can suppress (**ALL**) errors from any task using `failed_when: false` but I agree it is not ideal
Good suggestion. I did that and it's helped me achieve what I was after.

Thanks!

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/cloudformation_facts.py b/lib/ansible/modules/cloud/amazon/cloudformation_facts.py
--- a/lib/ansible/modules/cloud/amazon/cloudformation_facts.py
+++ b/lib/ansible/modules/cloud/amazon/cloudformation_facts.py
@@ -80,6 +80,13 @@
     stack_resources: true
     stack_policy: true
 
+# Fail if the stack doesn't exist
+- name: try to get facts about a stack but fail if it doesn't exist
+  cloudformation_facts:
+    stack_name: nonexistent-stack
+    all_facts: yes
+  failed_when: cloudformation['nonexistent-stack'] is undefined
+
 # Example dictionary outputs for stack_outputs, stack_parameters and stack_resources:
 # "stack_outputs": {
 #     "ApplicationDatabaseName": "dazvlpr01xj55a.ap-southeast-2.rds.amazonaws.com",
@@ -102,38 +109,38 @@
 RETURN = '''
 stack_description:
     description: Summary facts about the stack
-    returned: always
+    returned: if the stack exists
     type: dict
 stack_outputs:
     description: Dictionary of stack outputs keyed by the value of each output 'OutputKey' parameter and corresponding value of each
                  output 'OutputValue' parameter
-    returned: always
+    returned: if the stack exists
     type: dict
 stack_parameters:
     description: Dictionary of stack parameters keyed by the value of each parameter 'ParameterKey' parameter and corresponding value of
                  each parameter 'ParameterValue' parameter
-    returned: always
+    returned: if the stack exists
     type: dict
 stack_events:
     description: All stack events for the stack
-    returned: only if all_facts or stack_events is true
+    returned: only if all_facts or stack_events is true and the stack exists
     type: list
 stack_policy:
     description: Describes the stack policy for the stack
-    returned: only if all_facts or stack_policy is true
+    returned: only if all_facts or stack_policy is true and the stack exists
     type: dict
 stack_template:
     description: Describes the stack template for the stack
-    returned: only if all_facts or stack_template is true
+    returned: only if all_facts or stack_template is true and the stack exists
     type: dict
 stack_resource_list:
     description: Describes stack resources for the stack
-    returned: only if all_facts or stack_resourses is true
+    returned: only if all_facts or stack_resourses is true and the stack exists
     type: list
 stack_resources:
     description: Dictionary of stack resources keyed by the value of each resource 'LogicalResourceId' parameter and corresponding value of each
                  resource 'PhysicalResourceId' parameter
-    returned: only if all_facts or stack_resourses is true
+    returned: only if all_facts or stack_resourses is true and the stack exists
     type: dict
 '''
 
@@ -148,6 +155,7 @@
 except ImportError:
     HAS_BOTO3 = False
 
+from ansible.module_utils._text import to_native
 from ansible.module_utils.basic import AnsibleModule
 from ansible.module_utils.ec2 import (get_aws_connection_info, ec2_argument_spec, boto3_conn,
                                       camel_dict_to_snake_dict, AWSRetry, boto3_tag_list_to_ansible_dict)
@@ -180,22 +188,25 @@ def describe_stacks(self, stack_name=None):
             kwargs = {'StackName': stack_name} if stack_name else {}
             func = partial(self.client.describe_stacks, **kwargs)
             response = self.paginated_response(func, 'Stacks')
-            if response:
+            if response is not None:
                 return response
             self.module.fail_json(msg="Error describing stack(s) - an empty response was returned")
         except Exception as e:
-            self.module.fail_json(msg="Error describing stack(s) - " + str(e), exception=traceback.format_exc())
+            if 'does not exist' in e.response['Error']['Message']:
+                # missing stack, don't bail.
+                return {}
+            self.module.fail_json(msg="Error describing stack - " + to_native(e), exception=traceback.format_exc())
 
     def list_stack_resources(self, stack_name):
         try:
-            func = partial(self.client.list_stack_resources,StackName=stack_name)
+            func = partial(self.client.list_stack_resources, StackName=stack_name)
             return self.paginated_response(func, 'StackResourceSummaries')
         except Exception as e:
             self.module.fail_json(msg="Error listing stack resources - " + str(e), exception=traceback.format_exc())
 
     def describe_stack_events(self, stack_name):
         try:
-            func = partial(self.client.describe_stack_events,StackName=stack_name)
+            func = partial(self.client.describe_stack_events, StackName=stack_name)
             return self.paginated_response(func, 'StackEvents')
         except Exception as e:
             self.module.fail_json(msg="Error describing stack events - " + str(e), exception=traceback.format_exc())
@@ -222,7 +233,7 @@ def paginated_response(self, func, result_key, next_token=None):
         Returns expanded response for paginated operations.
         The 'result_key' is used to define the concatenated results that are combined from each paginated response.
         '''
-        args=dict()
+        args = dict()
         if next_token:
             args['NextToken'] = next_token
         response = func(**args)
@@ -232,6 +243,7 @@ def paginated_response(self, func, result_key, next_token=None):
             return result
         return result + self.paginated_response(func, result_key, next_token)
 
+
 def to_dict(items, key, value):
     ''' Transforms a list of items to a Key/Value dictionary '''
     if items:
@@ -239,6 +251,7 @@ def to_dict(items, key, value):
     else:
         return dict()
 
+
 def main():
     argument_spec = ec2_argument_spec()
     argument_spec.update(dict(


Test patch:
diff --git a/test/sanity/pep8/legacy-files.txt b/test/sanity/pep8/legacy-files.txt
--- a/test/sanity/pep8/legacy-files.txt
+++ b/test/sanity/pep8/legacy-files.txt
@@ -12,7 +12,6 @@ lib/ansible/modules/cloud/openstack/_os_server_actions.py
 lib/ansible/modules/cloud/ovirt/_ovirt_affinity_groups.py
 lib/ansible/modules/cloud/amazon/aws_kms.py
 lib/ansible/modules/cloud/amazon/cloudformation.py
-lib/ansible/modules/cloud/amazon/cloudformation_facts.py
 lib/ansible/modules/cloud/amazon/cloudfront_facts.py
 lib/ansible/modules/cloud/amazon/dynamodb_table.py
 lib/ansible/modules/cloud/amazon/ec2_ami_copy.py