Repository: ansible/ansible
Pull request: Clear 'connection related' plugin vars for next loop iteration
##### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
Fixes #58876
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
- Bugfix Pull Request

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below -->
`lib/ansible/executor/task_executor.py`

Problem statement:
assigning variable to remote_user in loop only works first time
<!--- Verify first that your issue is not already reported on GitHub -->
it is not
<!--- Also test if the latest release and devel branch are affected too -->

<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
<!--- Explain the problem briefly below -->
I have a loop which tries to assign a user to remote_user in my playbook.  The first assignment works, subsequent ones do not and the first assignment is used in subsequent iterations of the loop
##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below, use your best guess if unsure -->
ssh connection plugin perhaps? or the calling of it

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.8.1
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/bi003do/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Apr  9 2019, 14:30:50) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
DEFAULT_TIMEOUT(/etc/ansible/ansible.cfg) = 30
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False


```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
target OS is AWS Linux instances of various flavors


##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->
run the following playbook using inventory file with a single known linux host with user: ec2-user
```ini
[test]
xxx.xxx.xxx.xxx
```

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- hosts: test
  gather_facts: false
  tasks:
  - name: try users
    remote_user: "{{ item }}"
    ping:
    ignore_unreachable: true
    failed_when: false
    with_items:
      - centos
      - ec2-user
      - root

```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
I would expect this to loop 3 times trying to ping with the 3 users listed in with_items above

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
```
 ansible-playbook TryUsers.yml -i inventory -vvvv
First iteration

<xxx.xxx.xxx.xxx> ESTABLISH SSH CONNECTION FOR USER: centos
Second iteration
<xxx.xxx.xxx.xxx> ESTABLISH SSH CONNECTION FOR USER: centos
third iteration
<xxx.xxx.xxx.xxx> ESTABLISH SSH CONNECTION FOR USER: centos

```

<!--- Paste verbatim command output between quotes -->
```
bi003do@usbou-ansible TryUsers]$ ansible-playbook TryUsers.yml -i inventory

PLAY [test] *********************************************************************************************************************************************************************************

TASK [try users] ****************************************************************************************************************************************************************************
failed: [xxx.xxx.xxx.xxx] (item=centos) => {"ansible_loop_var": "item", "item": "centos", "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).", "unreachable": true}
failed: [10.5.162.71] (item=root) => {"ansible_loop_var": "item", "item": "root", "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).", "unreachable": true}
failed: [10.5.162.71] (item=ec2-user) => {"ansible_loop_var": "item", "item": "ec2-user", "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).", "unreachable": true}
fatal: [10.5.162.71]: UNREACHABLE! => {"changed": false, "msg": "All items completed", "results": [{"ansible_loop_var": "item", "item": "centos", "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).", "unreachable": true}, {"ansible_loop_var": "item", "item": "root", "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).", "unreachable": true}, {"ansible_loop_var": "item", "item": "ec2-user", "msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).", "unreachable": true}], "skip_reason": "Host 10.5.162.71 is unreachable"}

PLAY RECAP **********************************************************************************************************************************************************************************
10.5.162.71                : ok=0    changed=0    unreachable=1    failed=0    skipped=1    rescued=0    ignored=0


```

Hints from issue discussion:
Files identified in the description:
* [`lib/ansible/plugins/connection/ssh.py`](https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/connection/ssh.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Commit message: Clear 'connection related' plugin vars for next loop iteration

Fixes #58876

Patch to review:
diff --git a/changelogs/fragments/58876-do-not-reuse-remote_user-from-prev-loop.yaml b/changelogs/fragments/58876-do-not-reuse-remote_user-from-prev-loop.yaml
new file mode 100644
index 00000000000000..d345b2254be2bf
--- /dev/null
+++ b/changelogs/fragments/58876-do-not-reuse-remote_user-from-prev-loop.yaml
@@ -0,0 +1,2 @@
+bugfixes:
+  - Do not re-use remote_user from previous loop iteration (https://github.com/ansible/ansible/issues/58876)
diff --git a/lib/ansible/executor/task_executor.py b/lib/ansible/executor/task_executor.py
index 0865a3a50bc58f..c4f7e510bd4122 100644
--- a/lib/ansible/executor/task_executor.py
+++ b/lib/ansible/executor/task_executor.py
@@ -410,6 +410,19 @@ def _run_loop(self, items):
             results.append(res)
             del task_vars[loop_var]
 
+            # clear 'connection related' plugin variables for next iteration
+            clear_plugins = {
+                'connection': self._connection._load_name,
+                'shell': self._connection._shell._load_name
+            }
+            if self._play_context.become:
+                clear_plugins['become'] = self._get_become(self._play_context.become_method)._load_name
+
+            for plugin_type, plugin_name in iteritems(clear_plugins):
+                for var in C.config.get_plugin_vars(plugin_type, plugin_name):
+                    if var in task_vars:
+                        del task_vars[var]
+
         self._task.no_log = no_log
 
         return results


Code review comments:
diff_hunk: @@ -410,6 +410,19 @@ def _run_loop(self, items):
             results.append(res)
             del task_vars[loop_var]
 
+            # clear 'connection related' plugin variables for next iteration
+            clear_plugins = {
+                'connection': self._connection._load_name,
+                'shell': self._connection._shell._load_name
+            }
+            if self._play_context.become:
+                clear_plugins['become'] = self._get_become(self._play_context.become_method)._load_name
+
+            for plugin_type, plugin_name in iteritems(clear_plugins):
+                for var in C.config.get_plugin_vars(plugin_type, plugin_name):
+                    if var in task_vars:
+                        del task_vars[var]
+
path: lib/ansible/executor/task_executor.py
text: @user1:
soo, instead of removing them .. why don't we just 'not add them' in the first place?
https://github.com/ansible/ansible/blob/devel/lib/ansible/executor/task_executor.py#L198

we might also need to finalize things like #55219

@user2:
They're not being added in _get_loop_items, it seems like they're added in the internal execution so this may be the easiest way to clean them up.

@author:
Yeah, they're added in `TaskExecutor._execute()`:
https://github.com/ansible/ansible/blob/9489433a4d3e2c7bcd01ec9e93684b53ec853277/lib/ansible/executor/task_executor.py#L373
->
https://github.com/ansible/ansible/blob/9489433a4d3e2c7bcd01ec9e93684b53ec853277/lib/ansible/executor/task_executor.py#L520