Repository: ansible/ansible

Issue:
local_action and with_items mix up facts between hosts
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### SUMMARY
<!--- Explain the problem briefly -->
When using a `local_action` in conjunction with `with_items`, ansible mixes up variables between hosts.

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
local_action

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.7.0.dev0
  config file = /home/user/ansible/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.6.6 (default, Jun 27 2018, 13:11:40) [GCC 8.1.1 20180531]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
```
DEFAULT_HOST_LIST(/home/user/ansible/ansible.cfg) = ['/home/user/ansible/hosts.yml']
DEFAULT_PRIVATE_KEY_FILE(/home/user/ansible/ansible.cfg) = /home/user/.ssh/id_ansible
DEFAULT_REMOTE_USER(/home/user/ansible/ansible.cfg) = ansible
```

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
N/A

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- hosts: all
  tasks:
    - name: Copy var
      command: "echo Hostname: {{ inventory_hostname }}"
      with_items: [dummy]
      register: myvar

      # Needs to be a local action
    - local_action: debug
      args:
        msg: "{{ inventory_hostname }} got {{ item.stdout }}"
      # Needs to be with_items, printing myvar is ok
      with_items: "{{ myvar.results }}"
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Each hostname should get its own inventory name printed out.
```
ok: [host1 -> localhost] => ... {
    "msg": "host1 got Hostname: host1"
}
ok: [host2 -> localhost] => ... {
    "msg": "host2 got Hostname: host2"
}
```

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
Hosts get the facts of other hosts, in the minimal example it seems to be deterministic, in a larger playbook it looked random to me.
```
ok: [host1 -> localhost] => ... {
    "msg": "host1 got Hostname: host1"
}
ok: [host2 -> localhost] => ... {
    "msg": "host2 got Hostname: host1"
}
```

<!--- Paste verbatim command output between quotes below -->
```

PLAY [all] ********************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************
ok: [host1]
ok: [host2]
ok: [host3]

TASK [Copy var] ***************************************************************************************************************************************
changed: [host1] => (item=dummy)
changed: [host2] => (item=dummy)
changed: [host3] => (item=dummy)

TASK [debug] ******************************************************************************************************************************************
ok: [host1 -> localhost] => (item={'cmd': ['echo', 'Hostname:', 'host1'], 'stdout': 'Hostname: host1', 'stderr': '', 'rc': 0, 'start': '2018-07-19 12:44:07.021396', 'end': '2018-07-19 12:44:07.023243', 'delta': '0:00:00.001847', 'changed': True, 'invocation': {'module_args': {'_raw_params': 'echo Hostname: host1', 'warn': True, '_uses_shell': False, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, '_ansible_parsed': True, 'stdout_lines': ['Hostname: host1'], 'stderr_lines': [], '_ansible_no_log': False, 'failed': False, 'item': 'dummy', '_ansible_item_result': True, '_ansible_ignore_errors': None, '_ansible_item_label': 'dummy'}) => {
    "msg": "host1 got Hostname: host1"
}
ok: [host3 -> localhost] => (item={'cmd': ['echo', 'Hostname:', 'host1'], 'stdout': 'Hostname: host1', 'stderr': '', 'rc': 0, 'start': '2018-07-19 12:44:07.021396', 'end': '2018-07-19 12:44:07.023243', 'delta': '0:00:00.001847', 'changed': True, 'invocation': {'module_args': {'_raw_params': 'echo Hostname: host1', 'warn': True, '_uses_shell': False, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, '_ansible_parsed': True, 'stdout_lines': ['Hostname: host1'], 'stderr_lines': [], '_ansible_no_log': False, 'failed': False, 'item': 'dummy', '_ansible_item_result': True, '_ansible_ignore_errors': None, '_ansible_item_label': 'dummy'}) => {
    "msg": "host3 got Hostname: host1"
}
ok: [host2 -> localhost] => (item={'cmd': ['echo', 'Hostname:', 'host1'], 'stdout': 'Hostname: host1', 'stderr': '', 'rc': 0, 'start': '2018-07-19 12:44:07.021396', 'end': '2018-07-19 12:44:07.023243', 'delta': '0:00:00.001847', 'changed': True, 'invocation': {'module_args': {'_raw_params': 'echo Hostname: host1', 'warn': True, '_uses_shell': False, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, '_ansible_parsed': True, 'stdout_lines': ['Hostname: host1'], 'stderr_lines': [], '_ansible_no_log': False, 'failed': False, 'item': 'dummy', '_ansible_item_result': True, '_ansible_ignore_errors': None, '_ansible_item_label': 'dummy'}) => {
    "msg": "host2 got Hostname: host1"
}

PLAY RECAP ********************************************************************************************************************************************
host1                      : ok=3    changed=1    unreachable=0    failed=0   
host3                      : ok=3    changed=1    unreachable=0    failed=0   
host2                      : ok=3    changed=1    unreachable=0    failed=0   

```



Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
This is indeed of adding `local_action` to the `debug` task which is the equivalent of `delegate_to: localhost`.

I should note here, that debug _always_ executes locally, and does not require the use of `local_action` or `delegate_to: localhost`, so the note of `# Needs to be a local action` is unnecessary, unless that was purely a way to document that was the trigger to the problem.
You are right, the comment is about triggering the problem.
Just using debug is fine, but using it with `local_action` or the equivalent `delegate_to: localhost` shows the bug.

As you noted, a `debug` `local_action` is pretty useless, I was actually using it to fetch some files with `rsync`.

Resolving patch:
diff --git a/lib/ansible/vars/manager.py b/lib/ansible/vars/manager.py
--- a/lib/ansible/vars/manager.py
+++ b/lib/ansible/vars/manager.py
@@ -503,24 +503,16 @@ def _get_delegated_vars(self, play, task, existing_variables):
                     # This task will be skipped later due to this, so we just setup
                     # a dummy array for the later code so it doesn't fail
                     items = [None]
-                # Update task.loop with templated items, this ensures that delegate_to+loop
-                # doesn't produce different restuls than TaskExecutor which may reprocess the loop
-                # Set loop_with to None, so we don't do extra unexpected processing on the cached items later
-                # in TaskExecutor
-                task.loop_with = None
-                task.loop = items
             else:
                 raise AnsibleError("Failed to find the lookup named '%s' in the available lookup plugins" % task.loop_with)
         elif task.loop is not None:
             items = templar.template(task.loop)
-            # Update task.loop with templated items, this ensures that delegate_to+loop
-            # doesn't produce different restuls than TaskExecutor which may reprocess the loop
-            task.loop = items
         else:
             items = [None]
 
         delegated_host_vars = dict()
         item_var = getattr(task.loop_control, 'loop_var', 'item')
+        cache_items = False
         for item in items:
             # update the variables with the item value for templating, in case we need it
             if item is not None:
@@ -528,6 +520,8 @@ def _get_delegated_vars(self, play, task, existing_variables):
 
             templar.set_available_variables(vars_copy)
             delegated_host_name = templar.template(task.delegate_to, fail_on_undefined=False)
+            if delegated_host_name != task.delegate_to:
+                cache_items = True
             if delegated_host_name is None:
                 raise AnsibleError(message="Undefined delegate_to host for task:", obj=task._ds)
             if delegated_host_name in delegated_host_vars:
@@ -583,6 +577,16 @@ def _get_delegated_vars(self, play, task, existing_variables):
                 include_delegate_to=False,
                 include_hostvars=False,
             )
+
+        if cache_items:
+            # delegate_to templating produced a change, update task.loop with templated items,
+            # this ensures that delegate_to+loop doesn't produce different results than TaskExecutor
+            # which may reprocess the loop
+            # Set loop_with to None, so we don't do extra unexpected processing on the cached items later
+            # in TaskExecutor
+            task.loop_with = None
+            task.loop = items
+
         return delegated_host_vars
 
     def clear_facts(self, hostname):


Test patch: