Repository: ansible/ansible
Pull request: Handle gracefully a bad module 'results' key
  fixes #53962

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
- Bugfix Pull Request

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below -->
core

Problem statement:
Unexpected Exception, this is probably a bug: argument of type 'int' is not iterable
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
<!--- Explain the problem briefly below -->
When I create a custom module, return a list of integers, and name the return variable "results", I get the following error:
> ERROR! Unexpected Exception, this is probably a bug: argument of type 'int' is not iterable

If I change the return value to anything other than "results", it works.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below, use your best guess if unsure -->
Custom modules

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.7.5
  config file = /home/matt/ansible_playbooks/ansible.cfg
  configured module search path = ['/home/matt/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.7 (default, Dec  5 2018, 15:02:05) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
DEPRECATION_WARNINGS(/home/matt/ansible_playbooks/ansible.cfg) = False
HOST_KEY_CHECKING(/home/matt/ansible_playbooks/ansible.cfg) = False
PERSISTENT_COMMAND_TIMEOUT(/home/matt/ansible_playbooks/ansible.cfg) = 60
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
OS: CentOS Linux 7 (Core)

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->

<!--- Paste example playbooks or commands between quotes below -->
Create the following custom Python module in the "library" folder.  It can be named anything.  Here it is "test_lib.py"
```python
#!/usr/bin/python3

from ansible.module_utils.basic import AnsibleModule


def main():
    """Main function."""

    module = AnsibleModule(
        argument_spec=dict(
        )
    )

    return_list = [1, 2, 3, 4, 5]
    module.exit_json(results=return_list)


if __name__ == "__main__":
    main()
```

Create the following test playbook in the root ansible playbooks directory:
```yaml
---
- name: Demonstrate bug
  hosts: localhost
  gather_facts: false
  vars:
    ansible_python_interpreter: ./venv/bin/python3

  tasks:
    - name: Test output
      test_lib:
      register: numbers

    - debug:
        var: numbers
```

Run the playbook with "ansible-playbook <pb-name>.yml"

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
The playbook should print out the list [1, 2, 3, 4, 5]

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
It throws the following error on the first task:

> ERROR! Unexpected Exception, this is probably a bug: argument of type 'int' is not iterable
> to see the full traceback, use -vvv

<!--- Paste verbatim command output between quotes -->
```paste below
ansible-playbook 2.7.5
  config file = /home/matt/ansible_playbooks/ansible.cfg
  configured module search path = ['/home/matt/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.6.7 (default, Dec  5 2018, 15:02:05) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
Using /home/matt/ansible_playbooks/ansible.cfg as config file
setting up inventory plugins
/etc/ansible/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/etc/ansible/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /etc/ansible/hosts inventory source with ini plugin
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Loading callback plugin default of type stdout, v2.0 from /usr/lib/python3.6/site-packages/ansible/plugins/callback/default.py

PLAYBOOK: test.yml ***********************************************************************************************************************************************************************************************************************************************************
1 plays in test.yml

PLAY [Demonstrate bug] *******************************************************************************************************************************************************************************************************************************************************
META: ran handlers

TASK [Test output] ***********************************************************************************************************************************************************************************************************************************************************
task path: /home/matt/ansible_playbooks/test.yml:9
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: matt
<127.0.0.1> EXEC /bin/sh -c 'echo ~matt && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/matt/.ansible/tmp/ansible-tmp-1552918894.7886205-102956225615398 `" && echo ansible-tmp-1552918894.7886205-102956225615398="` echo /home/matt/.ansible/tmp/ansible-tmp-1552918894.7886205-102956225615398 `" ) && sleep 0'
Using module file /home/matt/ansible_playbooks/library/test_lib.py
<127.0.0.1> PUT /home/matt/.ansible/tmp/ansible-local-28981nnl4ulrc/tmp8wf8hk63 TO /home/matt/.ansible/tmp/ansible-tmp-1552918894.7886205-102956225615398/AnsiballZ_test_lib.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/matt/.ansible/tmp/ansible-tmp-1552918894.7886205-102956225615398/ /home/matt/.ansible/tmp/ansible-tmp-1552918894.7886205-102956225615398/AnsiballZ_test_lib.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c './venv/bin/python3 /home/matt/.ansible/tmp/ansible-tmp-1552918894.7886205-102956225615398/AnsiballZ_test_lib.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/matt/.ansible/tmp/ansible-tmp-1552918894.7886205-102956225615398/ > /dev/null 2>&1 && sleep 0'
ERROR! Unexpected Exception, this is probably a bug: argument of type 'int' is not iterable
the full traceback was:

Traceback (most recent call last):
  File "/usr/bin/ansible-playbook", line 118, in <module>
    exit_code = cli.run()
  File "/usr/lib/python3.6/site-packages/ansible/cli/playbook.py", line 122, in run
    results = pbex.run()
  File "/usr/lib/python3.6/site-packages/ansible/executor/playbook_executor.py", line 156, in run
    result = self._tqm.run(play=play)
  File "/usr/lib/python3.6/site-packages/ansible/executor/task_queue_manager.py", line 291, in run
    play_return = strategy.run(iterator, play_context)
  File "/usr/lib/python3.6/site-packages/ansible/plugins/strategy/linear.py", line 325, in run
    results += self._wait_on_pending_results(iterator)
  File "/usr/lib/python3.6/site-packages/ansible/plugins/strategy/__init__.py", line 712, in _wait_on_pending_results
    results = self._process_pending_results(iterator)
  File "/usr/lib/python3.6/site-packages/ansible/plugins/strategy/__init__.py", line 117, in inner
    results = func(self, iterator, one_pass=one_pass, max_passes=max_passes)
  File "/usr/lib/python3.6/site-packages/ansible/plugins/strategy/__init__.py", line 474, in _process_pending_results
    if task_result.is_failed():
  File "/usr/lib/python3.6/site-packages/ansible/executor/task_result.py", line 59, in is_failed
    'results' in self._result and True in [True for x in self._result['results'] if 'failed_when_result' in x]:
  File "/usr/lib/python3.6/site-packages/ansible/executor/task_result.py", line 59, in <listcomp>
    'results' in self._result and True in [True for x in self._result['results'] if 'failed_when_result' in x]:
TypeError: argument of type 'int' is not iterable
```

Hints from issue discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/amazon/ec2_customer_gateway.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/ec2_customer_gateway.py)
* [lib/ansible/modules/cloud/amazon/ec2_customer_gateway_facts.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/ec2_customer_gateway_facts.py)
* [lib/ansible/modules/cloud/cloudstack/cs_vpn_customer_gateway.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/cloudstack/cs_vpn_customer_gateway.py)
* [lib/ansible/modules/cloud/vmware/vmware_guest_custom_attribute_defs.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/vmware/vmware_guest_custom_attribute_defs.py)
* [lib/ansible/modules/cloud/vmware/vmware_guest_custom_attributes.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/vmware/vmware_guest_custom_attributes.py)
* [lib/ansible/modules/cloud/vmware/vmware_guest_customization_facts.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/vmware/vmware_guest_customization_facts.py)
* [lib/ansible/modules/network/avi/avi_customipamdnsprofile.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/avi/avi_customipamdnsprofile.py)
* [lib/ansible/modules/network/fortios/fortios_application_custom.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/fortios/fortios_application_custom.py)
* [lib/ansible/modules/network/fortios/fortios_firewall_internet_service_custom.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/fortios/fortios_firewall_internet_service_custom.py)
* [lib/ansible/modules/network/fortios/fortios_firewall_service_custom.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/fortios/fortios_firewall_service_custom.py)
* [lib/ansible/modules/network/fortios/fortios_firewall_wildcard_fqdn_custom.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/fortios/fortios_firewall_wildcard_fqdn_custom.py)
* [lib/ansible/modules/network/fortios/fortios_ips_custom.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/fortios/fortios_ips_custom.py)
* [lib/ansible/modules/network/fortios/fortios_log_custom_field.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/fortios/fortios_log_custom_field.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Akasurde @Madhura-CSI @MichaelBaydoun @Tomorrow9 @bjolivot @ckotte @dav1x @dpassante @ericsysmin @grastogi23 @khaltore @lparkes @mamunozgonzalez @resmo @rhtyd @thomnico @warthog9
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
!component =lib/ansible/executor/task_result.py
Files identified in the description:
* [lib/ansible/executor/task_result.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/executor/task_result.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
`results` is used internally to handle loops, so your module should never return this key

Commit message: typo

Patch to review:
diff --git a/changelogs/fragments/clean_results.yml b/changelogs/fragments/clean_results.yml
new file mode 100644
index 00000000000000..71c4fd423b2e0e
--- /dev/null
+++ b/changelogs/fragments/clean_results.yml
@@ -0,0 +1,2 @@
+bugfixes:
+    - disallow non dict results from module and allow user to continue using with a warning.
diff --git a/lib/ansible/plugins/action/__init__.py b/lib/ansible/plugins/action/__init__.py
index b56438dd8aef31..d378fcd392e804 100644
--- a/lib/ansible/plugins/action/__init__.py
+++ b/lib/ansible/plugins/action/__init__.py
@@ -21,6 +21,7 @@
 from ansible.errors import AnsibleError, AnsibleConnectionFailure, AnsibleActionSkip, AnsibleActionFail
 from ansible.executor.module_common import modify_module
 from ansible.executor.interpreter_discovery import discover_interpreter, InterpreterDiscoveryRequiredError
+from ansible.module_utils.common._collections_compat import Mapping
 from ansible.module_utils.json_utils import _filter_non_json_lines
 from ansible.module_utils.six import binary_type, string_types, text_type, iteritems, with_metaclass
 from ansible.module_utils.six.moves import shlex_quote
@@ -909,6 +910,12 @@ def _execute_module(self, module_name=None, module_args=None, tmp=None, task_var
         if data.pop("_ansible_suppress_tmpdir_delete", False):
             self._cleanup_remote_tmp = False
 
+        # NOTE: yum returns results .. but that made it 'compatible' with squashing, so we allow mappings, for now
+        if 'results' in data and not isinstance(data['results'], Mapping):
+            data['ansible_module_results'] = data['results']
+            del data['results']
+            display.warning("Found internal 'results' key in module return, renamed to 'ansible_module_results'.")
+
         # remove internal keys
         remove_internal_keys(data)
 


Code review comments:
diff_hunk: @@ -909,6 +910,12 @@ def _execute_module(self, module_name=None, module_args=None, tmp=None, task_var
         if data.pop("_ansible_suppress_tmpdir_delete", False):
             self._cleanup_remote_tmp = False
 
+        # NOTE: yum returns results .. but that made it 'compatible' with squashing, so we allow mappings, for now
+        if 'results' in data and not isinstance(data['results'], Mapping):
path: lib/ansible/plugins/action/__init__.py
text: @user1:
IIRC `results` in the `yum` module is a `list`.

@author:
actually, `results` should always be a list .. of dicts, so this was wrong