Repository: ansible/ansible

Issue:
onyx_config module failed while using python = 3.5
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and devel branch are affected too.
Always add information AFTER of these html comments.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report


##### COMPONENT NAME
onyx_config.py

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.0
  config file = /home/user/git/docs/ansible/ansible.cfg
  configured module search path = [u'/home/user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.5/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118]
```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
DEFAULT_GATHERING(/home/user/git/docs/ansible/ansible.cfg) = explicit
DEFAULT_HOST_LIST(/home/user/git/docs/ansible/ansible.cfg) = [u'/home/user/git/docs/ansible/hosts']
DEFAULT_TIMEOUT(/home/user/git/docs/ansible/ansible.cfg) = 30
HOST_KEY_CHECKING(/home/user/git/docs/ansible/ansible.cfg) = False
PARAMIKO_HOST_KEY_AUTO_ADD(/home/user/git/docs/ansible/ansible.cfg) = True
PARAMIKO_LOOK_FOR_KEYS(/home/user/git/docs/ansible/ansible.cfg) = False
PERSISTENT_COMMAND_TIMEOUT(/home/user/git/docs/ansible/ansible.cfg) = 30
PERSISTENT_CONNECT_RETRY_TIMEOUT(/home/user/git/docs/ansible/ansible.cfg) = 45
PERSISTENT_CONNECT_TIMEOUT(/home/user/git/docs/ansible/ansible.cfg) = 60

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
Debian 9.4
##### SUMMARY
<!--- Explain the problem briefly -->
An exception is thrown when saving config using ansible module onyx_config on Mellanox Switch running 3.6.6000 with python 3.5
##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
-  hosts: mlnx_os
   gather_facts: false
   connection: network_cli
   become: yes
   become_method: enable
   tasks:
   - name: change config on MLNX-OS device
     onyx_config:
       backup: yes
```
hosts:
```yaml
[mlnx_os]
1.1.1.1
2.2.2.2
[mlnx_os:vars]
ansible_network_os=onyx
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
Config fetched and saved in ./backup/ directory, which happens with
python version = 2.7.14+ (default, Feb  6 2018, 19:12:18) [GCC 7.3.0]

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
user@somewhere:~/git/noc-docs/ansible/playbooks$ ansible-playbook onyx_config.yml  -vvvv 
ansible-playbook 2.5.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.5/dist-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
Parsed /home/user/git/noc-docs/ansible/hosts inventory source with ini plugin
Loading callback plugin default of type stdout, v2.0 from /usr/local/lib/python3.5/dist-packages/ansible/plugins/callback/default.py

PLAYBOOK: onyx_config.yml ***************************************************************************************************************************************************************
1 plays in onyx_config.yml

PLAY [mlnx_os] **************************************************************************************************************************************************************************
META: ran handlers

TASK [change config on MLNX-OS device] **************************************************************************************************************************************************
task path: /home/user/git/noc-docs/ansible/playbooks/onyx_config.yml:7
<1.1.1.1> attempting to start connection
<1.1.1.1> using connection plugin network_cli
<2.2.2.2> attempting to start connection
<2.2.2.2> using connection plugin network_cli
<1.1.1.1> local domain socket does not exist, starting it
<1.1.1.1> control socket path is /home/user/.ansible/pc/e7d6fea450
<1.1.1.1> <1.1.1.1> ESTABLISH CONNECTION FOR USER: None on PORT 22 TO 1.1.1.1
<1.1.1.1> <1.1.1.1> ssh connection done, setting terminal
<1.1.1.1> <1.1.1.1> loaded terminal plugin for network_os onyx
<1.1.1.1> <1.1.1.1> loaded cliconf plugin for network_os onyx
<1.1.1.1> <1.1.1.1> firing event: on_open_shell()
<1.1.1.1> <1.1.1.1> firing event: on_become
<1.1.1.1> <1.1.1.1> ssh connection has completed successfully
<1.1.1.1> connection to remote device started successfully
<1.1.1.1> local domain socket listeners started successfully
<1.1.1.1> 
<1.1.1.1> local domain socket path is /home/user/.ansible/pc/e7d6fea450
Using module file /usr/local/lib/python3.5/dist-packages/ansible/modules/network/onyx/onyx_config.py
<1.1.1.1> ESTABLISH LOCAL CONNECTION FOR USER: user
<1.1.1.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<1.1.1.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/user/.ansible/tmp/ansible-tmp-1522919162.2716436-173180906683447 `" && echo ansible-tmp-1522919162.2716436-173180906683447="` echo /home/user/.ansible/tmp/ansible-tmp-1522919162.2716436-173180906683447 `" ) && sleep 0'
<1.1.1.1> PUT /home/user/.ansible/tmp/ansible-local-104891g0op533/tmpffkdau9n TO /home/user/.ansible/tmp/ansible-tmp-1522919162.2716436-173180906683447/onyx_config.py
<1.1.1.1> EXEC /bin/sh -c 'chmod u+x /home/user/.ansible/tmp/ansible-tmp-1522919162.2716436-173180906683447/ /home/user/.ansible/tmp/ansible-tmp-1522919162.2716436-173180906683447/onyx_config.py && sleep 0'
<1.1.1.1> EXEC /bin/sh -c '/usr/bin/python /home/user/.ansible/tmp/ansible-tmp-1522919162.2716436-173180906683447/onyx_config.py && sleep 0'
<2.2.2.2> local domain socket does not exist, starting it
<2.2.2.2> control socket path is /home/user/.ansible/pc/89c607bbb3
<2.2.2.2> <2.2.2.2> ESTABLISH CONNECTION FOR USER: None on PORT 22 TO 2.2.2.2
<2.2.2.2> <2.2.2.2> ssh connection done, setting terminal
<2.2.2.2> <2.2.2.2> loaded terminal plugin for network_os onyx
<2.2.2.2> <2.2.2.2> loaded cliconf plugin for network_os onyx
<2.2.2.2> <2.2.2.2> firing event: on_open_shell()
<2.2.2.2> <2.2.2.2> firing event: on_become
<2.2.2.2> <2.2.2.2> ssh connection has completed successfully
<2.2.2.2> connection to remote device started successfully
<2.2.2.2> local domain socket listeners started successfully
<2.2.2.2> 
<2.2.2.2> local domain socket path is /home/user/.ansible/pc/89c607bbb3
Using module file /usr/local/lib/python3.5/dist-packages/ansible/modules/network/onyx/onyx_config.py
<2.2.2.2> ESTABLISH LOCAL CONNECTION FOR USER: user
<2.2.2.2> EXEC /bin/sh -c 'echo ~ && sleep 0'
<2.2.2.2> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/user/.ansible/tmp/ansible-tmp-1522919171.0372546-12351055057161 `" && echo ansible-tmp-1522919171.0372546-12351055057161="` echo /home/user/.ansible/tmp/ansible-tmp-1522919171.0372546-12351055057161 `" ) && sleep 0'
<2.2.2.2> PUT /home/user/.ansible/tmp/ansible-local-104891g0op533/tmp6wz2e7ji TO /home/user/.ansible/tmp/ansible-tmp-1522919171.0372546-12351055057161/onyx_config.py
<2.2.2.2> EXEC /bin/sh -c 'chmod u+x /home/user/.ansible/tmp/ansible-tmp-1522919171.0372546-12351055057161/ /home/user/.ansible/tmp/ansible-tmp-1522919171.0372546-12351055057161/onyx_config.py && sleep 0'
<2.2.2.2> EXEC /bin/sh -c '/usr/bin/python /home/user/.ansible/tmp/ansible-tmp-1522919171.0372546-12351055057161/onyx_config.py && sleep 0'
<1.1.1.1> EXEC /bin/sh -c 'rm -f -r /home/user/.ansible/tmp/ansible-tmp-1522919162.2716436-173180906683447/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/ansible/executor/task_executor.py", line 138, in run
    res = self._execute()
  File "/usr/local/lib/python3.5/dist-packages/ansible/executor/task_executor.py", line 554, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/local/lib/python3.5/dist-packages/ansible/plugins/action/onyx_config.py", line 57, in run
    for key in result.keys():
RuntimeError: dictionary changed size during iteration

fatal: [1.1.1.1]: FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}
<2.2.2.2> EXEC /bin/sh -c 'rm -f -r /home/user/.ansible/tmp/ansible-tmp-1522919171.0372546-12351055057161/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/ansible/executor/task_executor.py", line 138, in run
    res = self._execute()
  File "/usr/local/lib/python3.5/dist-packages/ansible/executor/task_executor.py", line 554, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/local/lib/python3.5/dist-packages/ansible/plugins/action/onyx_config.py", line 57, in run
    for key in result.keys():
RuntimeError: dictionary changed size during iteration

fatal: [2.2.2.2]: FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}
	to retry, use: --limit @/home/user/git/noc-docs/ansible/playbooks/onyx_config.retry

PLAY RECAP ******************************************************************************************************************************************************************************
1.1.1.1              : ok=0    changed=0    unreachable=0    failed=1   
2.2.2.2              : ok=0    changed=0    unreachable=0    failed=1   

```



Discussion:
@samerd
cc @atabachnik
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/plugins/action/onyx_config.py b/lib/ansible/plugins/action/onyx_config.py
--- a/lib/ansible/plugins/action/onyx_config.py
+++ b/lib/ansible/plugins/action/onyx_config.py
@@ -54,7 +54,7 @@ def run(self, tmp=None, task_vars=None):
 
         # strip out any keys that have two leading and two trailing
         # underscore characters
-        for key in result.keys():
+        for key in list(result.keys()):
             if PRIVATE_KEYS_RE.match(key):
                 del result[key]
 


Test patch: