Repository: ansible/ansible

Issue:
ios_config breaks when using loops
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
iOS_config

##### ANSIBLE VERSION
```
ansible 2.3.1.0
  config file = /Users/sander/IdeaProjects/retevia-ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Dec 17 2016, 23:03:43) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
```

##### CONFIGURATION
default settings

##### OS / ENVIRONMENT
Running Ansible from Mac OS
Darwin MacPro.local 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64

Tested with a Cisco 3850
Cisco IOS Software [Denali], Catalyst L3 Switch Software (CAT3K_CAA-UNIVERSALK9-M), Version 16.3.3, RELEASE SOFTWARE (fc3)

And a Cisco 2960G
Cisco IOS Software, C2960 Software (C2960-LANBASEK9-M), Version 15.0(2)SE10a, RELEASE SOFTWARE (fc3)

##### SUMMARY
`ios_config` fails when performing the second and subsequent tasks in the loop. The first one is executed fine.

##### STEPS TO REPRODUCE
Take for example this task:

```yaml
- name: "interfaces : set switchport mode"
  ios_config:
    parents:
    - "interface {{ item.name }}"
    lines:
    - "switchport mode {{ item.mode }}"
  with_items:
  - name: TenGigabitEthernet1/0/1
    mode: access
  - name: TenGigabitEthernet1/0/2
    mode: access
  - name: TenGigabitEthernet1/0/3
    mode: access
```

##### EXPECTED RESULTS
```
TASK [switch-common : interfaces : set switchport mode] *****************************************************************************************
ok: [sw1.example] => (item={u'name': u'TenGigabitEthernet1/0/1', u'mode': u'access'})
ok: [sw1.example] => (item={u'name': u'TenGigabitEthernet1/0/2', u'mode': u'access'})
ok: [sw1.example] => (item={u'name': u'TenGigabitEthernet1/0/3', u'mode': u'access'})
```

##### ACTUAL RESULTS
```
$ ansible-playbook switches.yml -vvvv
No config file found; using defaults
statically included: /Users/sander/IdeaProjects/retevia-ansible/roles/switch-common/tasks/interfaces.yml
Loading callback plugin default of type stdout, v2.0 from /usr/local/Cellar/ansible/2.3.1.0/libexec/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc

PLAYBOOK: switches.yml **************************************************************************************************************************
1 plays in switches.yml

PLAY [switches] *********************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************
Using module file /usr/local/Cellar/ansible/2.3.1.0/libexec/lib/python2.7/site-packages/ansible/modules/system/setup.py
<sw1.retevia.eu> ESTABLISH LOCAL CONNECTION FOR USER: sander
<sw1.retevia.eu> EXEC /bin/sh -c 'echo ~ && sleep 0'
<sw1.retevia.eu> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/sander/.ansible/tmp/ansible-tmp-1498511911.79-48589097989550 `" && echo ansible-tmp-1498511911.79-48589097989550="` echo /Users/sander/.ansible/tmp/ansible-tmp-1498511911.79-48589097989550 `" ) && sleep 0'
<sw1.retevia.eu> PUT /var/folders/xs/nwtp76d53vv5_4mvq3cyfsrh0000gn/T/tmpv01e_d TO /Users/sander/.ansible/tmp/ansible-tmp-1498511911.79-48589097989550/setup.py
<sw1.retevia.eu> EXEC /bin/sh -c 'chmod u+x /Users/sander/.ansible/tmp/ansible-tmp-1498511911.79-48589097989550/ /Users/sander/.ansible/tmp/ansible-tmp-1498511911.79-48589097989550/setup.py && sleep 0'
<sw1.retevia.eu> EXEC /bin/sh -c '/usr/bin/python /Users/sander/.ansible/tmp/ansible-tmp-1498511911.79-48589097989550/setup.py; rm -rf "/Users/sander/.ansible/tmp/ansible-tmp-1498511911.79-48589097989550/" > /dev/null 2>&1 && sleep 0'
ok: [sw1.retevia.eu]
META: ran handlers

TASK [switch-common : interfaces : set switchport mode] *****************************************************************************************
task path: /Users/sander/IdeaProjects/retevia-ansible/roles/switch-common/tasks/interfaces.yml:1
<sw1.retevia.eu> using connection plugin network_cli
<sw1.retevia.eu> socket_path: /Users/sander/.ansible/pc/3bccdcdc61
open_shell() returned 0 ok 
Using module file /usr/local/Cellar/ansible/2.3.1.0/libexec/lib/python2.7/site-packages/ansible/modules/network/ios/ios_config.py
<sw1.retevia.eu> ESTABLISH LOCAL CONNECTION FOR USER: sander
<sw1.retevia.eu> EXEC /bin/sh -c 'echo ~ && sleep 0'
<sw1.retevia.eu> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/sander/.ansible/tmp/ansible-tmp-1498511916.1-174888287427276 `" && echo ansible-tmp-1498511916.1-174888287427276="` echo /Users/sander/.ansible/tmp/ansible-tmp-1498511916.1-174888287427276 `" ) && sleep 0'
<sw1.retevia.eu> PUT /var/folders/xs/nwtp76d53vv5_4mvq3cyfsrh0000gn/T/tmpRsgoej TO /Users/sander/.ansible/tmp/ansible-tmp-1498511916.1-174888287427276/ios_config.py
<sw1.retevia.eu> EXEC /bin/sh -c 'chmod u+x /Users/sander/.ansible/tmp/ansible-tmp-1498511916.1-174888287427276/ /Users/sander/.ansible/tmp/ansible-tmp-1498511916.1-174888287427276/ios_config.py && sleep 0'
<sw1.retevia.eu> EXEC /bin/sh -c '/usr/bin/python /Users/sander/.ansible/tmp/ansible-tmp-1498511916.1-174888287427276/ios_config.py; rm -rf "/Users/sander/.ansible/tmp/ansible-tmp-1498511916.1-174888287427276/" > /dev/null 2>&1 && sleep 0'
changed: [sw1.retevia.eu] => (item={u'name': u'TenGigabitEthernet1/0/1', u'mode': u'access'}) => {
    "banners": {}, 
    "changed": true, 
    "commands": [
        "interface TenGigabitEthernet1/0/1", 
        "switchport mode access"
    ], 
    "invocation": {
        "module_args": {
            "after": null, 
            "auth_pass": null, 
            "authorize": null, 
            "backup": false, 
            "before": null, 
            "config": null, 
            "defaults": false, 
            "force": false, 
            "host": null, 
            "lines": [
                "switchport mode access"
            ], 
            "match": "line", 
            "multiline_delimiter": "@", 
            "parents": [
                "interface TenGigabitEthernet1/0/1"
            ], 
            "password": null, 
            "port": null, 
            "provider": null, 
            "replace": "line", 
            "save": false, 
            "src": null, 
            "ssh_keyfile": null, 
            "timeout": null, 
            "username": null
        }
    }, 
    "item": {
        "mode": "access", 
        "name": "TenGigabitEthernet1/0/1"
    }, 
    "updates": [
        "interface TenGigabitEthernet1/0/1", 
        "switchport mode access"
    ]
}
<sw1.retevia.eu> using connection plugin network_cli
<sw1.retevia.eu> socket_path: /Users/sander/.ansible/pc/ba633a4d9e
open_shell() returned 255  failed to connect to control socket
failed: [sw1.retevia.eu] (item={u'name': u'TenGigabitEthernet1/0/2', u'mode': u'access'}) => {
    "failed": true, 
    "item": {
        "mode": "access", 
        "name": "TenGigabitEthernet1/0/2"
    }, 
    "msg": "unable to open shell. Please see: https://docs.ansible.com/ansible/network_debug_troubleshooting.html#unable-to-open-shell", 
    "rc": 255
}
<sw1.retevia.eu> using connection plugin network_cli
<sw1.retevia.eu> socket_path: /Users/sander/.ansible/pc/ba633a4d9e
open_shell() returned 255  failed to connect to control socket
failed: [sw1.retevia.eu] (item={u'name': u'TenGigabitEthernet1/0/3', u'mode': u'access'}) => {
    "failed": true, 
    "item": {
        "mode": "access", 
        "name": "TenGigabitEthernet1/0/3"
    }, 
    "msg": "unable to open shell. Please see: https://docs.ansible.com/ansible/network_debug_troubleshooting.html#unable-to-open-shell", 
    "rc": 255
}
	to retry, use: --limit @/Users/sander/IdeaProjects/retevia-ansible/switches.retry

PLAY RECAP **************************************************************************************************************************************
sw1.retevia.eu             : ok=1    changed=0    unreachable=0    failed=1   
```

When removing `TenGigabitEthernet1/0/1` then `TenGigabitEthernet1/0/2` gets executed and `TenGigabitEthernet1/0/3` fails, etc.

Interesting is also that if the first item is skipped (when adding a `when: item.name != 'TenGigabitEthernet1/0/1'` condition) then it still triggers the error:

```
TASK [switch-common : interfaces : set switchport mode] *****************************************************************************************
task path: /Users/sander/IdeaProjects/retevia-ansible/roles/switch-common/tasks/interfaces.yml:1
skipping: [sw1.retevia.eu] => (item={u'name': u'TenGigabitEthernet1/0/1', u'mode': u'access'})  => {
    "changed": false, 
    "item": {
        "mode": "access", 
        "name": "TenGigabitEthernet1/0/1"
    }, 
    "skip_reason": "Conditional result was False", 
    "skipped": true
}
<sw1.retevia.eu> using connection plugin network_cli
<sw1.retevia.eu> socket_path: /Users/sander/.ansible/pc/ba633a4d9e
open_shell() returned 255  failed to connect to control socket
failed: [sw1.retevia.eu] (item={u'name': u'TenGigabitEthernet1/0/2', u'mode': u'access'}) => {
    "failed": true, 
    "item": {
        "mode": "access", 
        "name": "TenGigabitEthernet1/0/2"
    }, 
    "msg": "unable to open shell. Please see: https://docs.ansible.com/ansible/network_debug_troubleshooting.html#unable-to-open-shell", 
    "rc": 255
}
<sw1.retevia.eu> using connection plugin network_cli
<sw1.retevia.eu> socket_path: /Users/sander/.ansible/pc/ba633a4d9e
open_shell() returned 255  failed to connect to control socket
failed: [sw1.retevia.eu] (item={u'name': u'TenGigabitEthernet1/0/3', u'mode': u'access'}) => {
    "failed": true, 
    "item": {
        "mode": "access", 
        "name": "TenGigabitEthernet1/0/3"
    }, 
    "msg": "unable to open shell. Please see: https://docs.ansible.com/ansible/network_debug_troubleshooting.html#unable-to-open-shell", 
    "rc": 255
}
```

Also: when splitting this into three separate tasks everything works:

```
- name: "interfaces : set switchport mode (1)"
  ios_config:
    parents:
    - "interface {{ item.name }}"
    lines:
    - "switchport mode {{ item.mode }}"
  with_items:
  - name: TenGigabitEthernet1/0/1
    mode: access

- name: "interfaces : set switchport mode (2)"
  ios_config:
    parents:
    - "interface {{ item.name }}"
    lines:
    - "switchport mode {{ item.mode }}"
  with_items:
  - name: TenGigabitEthernet1/0/2
    mode: access

- name: "interfaces : set switchport mode (3)"
  ios_config:
    parents:
    - "interface {{ item.name }}"
    lines:
    - "switchport mode {{ item.mode }}"
  with_items:
  - name: TenGigabitEthernet1/0/3
    mode: access
```

Results in:

```
$ ansible-playbook switches.yml 

PLAY [switches] *********************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************
ok: [sw1.retevia.eu]

TASK [switch-common : interfaces : set switchport mode (1)] *************************************************************************************
changed: [sw1.retevia.eu] => (item={u'name': u'TenGigabitEthernet1/0/1', u'mode': u'access'})

TASK [switch-common : interfaces : set switchport mode (2)] *************************************************************************************
changed: [sw1.retevia.eu] => (item={u'name': u'TenGigabitEthernet1/0/2', u'mode': u'access'})

TASK [switch-common : interfaces : set switchport mode (3)] *************************************************************************************
changed: [sw1.retevia.eu] => (item={u'name': u'TenGigabitEthernet1/0/3', u'mode': u'access'})

PLAY RECAP **************************************************************************************************************************************
sw1.retevia.eu             : ok=4    changed=3    unreachable=0    failed=0   
```



Discussion:
cc @privateip @rcarrillocruz
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
I'm afraid I'm unable to reproduce this now, we don't seem to have a switch image with IOS in our lab at the moment, we will address once we get a new VIRL setup.

In the meantime, can you confirm that none of your items contains your management interface? i.e. the interface you use to connect to the device with ansible itself.
Confirmed, I'm doing management over a dedicated management port.
Looks like it's related to #26432 

Resolving patch:
diff --git a/lib/ansible/playbook/play_context.py b/lib/ansible/playbook/play_context.py
--- a/lib/ansible/playbook/play_context.py
+++ b/lib/ansible/playbook/play_context.py
@@ -432,6 +432,11 @@ def set_task_and_variable_override(self, task, variables, templar):
                 if exe_var in variables:
                     setattr(new_info, 'executable', variables.get(exe_var))
 
+        # we store original in 'connection_user' for use of network/other modules that fallback to it as login user
+        # this needs to be done before the MAGIC_VARIABLE_MAPPING happens below
+        if new_info.connection == 'local':
+            new_info.connection_user = new_info.remote_user
+
         attrs_considered = []
         for (attr, variable_names) in iteritems(MAGIC_VARIABLE_MAPPING):
             for variable_name in variable_names:
@@ -493,9 +498,7 @@ def set_task_and_variable_override(self, task, variables, templar):
 
         # if the final connection type is local, reset the remote_user value to that of the currently logged in user
         # this ensures any become settings are obeyed correctly
-        # we store original in 'connection_user' for use of network/other modules that fallback to it as login user
         if new_info.connection == 'local':
-            new_info.connection_user = new_info.remote_user
             new_info.remote_user = pwd.getpwuid(os.getuid()).pw_name
 
         # set no_log to default if it was not previouslly set


Test patch: