Repository: ansible/ansible

Issue:
zabbix_host module doesn't wotk with python3
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
zabbix_host

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.0
  config file = /home/user/work/ansible/ansible.cfg
  configured module search path = [u'/home/user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
```
ANSIBLE_NOCOWS(/home/user/work/ansible/ansible.cfg) = True
DEFAULT_FORKS(/home/user/work/ansible/ansible.cfg) = 50
DEFAULT_GATHERING(/home/user/work/ansible/ansible.cfg) = smart
DEFAULT_HOST_LIST(/home/user/work/ansible/ansible.cfg) = [u'/home/user/work/ansible/inventory']
DEFAULT_JINJA2_EXTENSIONS(/home/user/work/ansible/ansible.cfg) = jinja2.ext.do,jinja2.ext.loopcontrols
DEFAULT_MANAGED_STR(/home/user/work/ansible/ansible.cfg) = Ansible template: {file}
DEFAULT_MODULE_NAME(/home/user/work/ansible/ansible.cfg) = shell
DEFAULT_ROLES_PATH(/home/user/work/ansible/ansible.cfg) = [u'/home/user/work/ansible/roles']
RETRY_FILES_ENABLED(/home/user/work/ansible/ansible.cfg) = False
```
##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
Control host OS: Ubuntu GNU/Linux 16.04 Xenial
Managed host OS: Ubuntu GNU/Linux 16.04 Xenial
Managed host Zabbix server version: 3.4.8
Managed host python 3 module zabbix-api version: 0.5.3

##### SUMMARY
<!--- Explain the problem briefly -->
Module zabbix_host does not work if python 3.5 is used, but works fine with python 2.7. With python 3.5 it reports error "Missing module", but module for python 3.5 is installed and working:

```
root@managed-host # python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from zabbix_api import ZabbixAPI, ZabbixAPISubClass
>>> 
```

##### STEPS TO REPRODUCE
Host should have set `ansible_python_interpreter=/usr/bin/python3` in inventory.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- hosts: all
  become: yes
  tasks:
    - name: test
      zabbix_host:
        server_url: http://127.0.0.1
        login_user: Admin
        login_password: *******
        host_name: Zabbix server
        state: present
        status: enabled
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Enable Zabbix host "Zabbix server".

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
TASK [test] *****************************************************************************************************************************************************************************
task path: /home/user/work/ansible/nms-test.yml:4
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/monitoring/zabbix/zabbix_host.py
<nms1-dev> ESTABLISH LXD CONNECTION FOR USER: root
<nms1-dev> EXEC /bin/sh -c 'echo ~ && sleep 0'
<nms1-dev> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1524099837.23-18108060523735 `" && echo ansible-tmp-1524099837.23-18108060523735="` echo /root/.ansible/tmp/ansible-tmp-1524099837.23-18108060523735 `" ) && sleep 0'
<nms1-dev> PUT /home/user/.ansible/tmp/ansible-local-477aMRPmy/tmpITRrIY TO /root/.ansible/tmp/ansible-tmp-1524099837.23-18108060523735/zabbix_host.py
<nms1-dev> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1524099837.23-18108060523735/ /root/.ansible/tmp/ansible-tmp-1524099837.23-18108060523735/zabbix_host.py && sleep 0'
<nms1-dev> EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-mizxmcxwwxjmzgemzmabbcfycewmqkyq; /usr/bin/python /root/.ansible/tmp/ansible-tmp-1524099837.23-18108060523735/zabbix_host.py'"'"' && sleep 0'
<nms1-dev> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1524099837.23-18108060523735/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_IdqsKH/ansible_module_********_host.py", line 233, in <module>
    from ********_api import ZabbixAPI, ZabbixAPISubClass

fatal: [nms1-dev]: FAILED! => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "description": null, 
            "force": true, 
            "host_groups": null, 
            "host_name": "Zabbix server", 
            "http_login_password": null, 
            "http_login_user": null, 
            "interfaces": null, 
            "inventory_mode": null, 
            "inventory_zabbix": null, 
            "ipmi_authtype": null, 
            "ipmi_password": null, 
            "ipmi_privilege": null, 
            "ipmi_username": null, 
            "link_templates": null, 
            "login_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", 
            "login_user": "Admin", 
            "proxy": null, 
            "server_url": "http://127.0.0.1", 
            "state": "present", 
            "status": "enabled", 
            "timeout": 10, 
            "tls_accept": 1, 
            "tls_connect": 1, 
            "tls_issuer": null, 
            "tls_psk": null, 
            "tls_psk_identity": null, 
            "tls_subject": null, 
            "validate_certs": true, 
            "visible_name": null
        }
    }, 
    "msg": "Missing required ********-api module (check docs or install with: pip install ********-api)"
}
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/monitoring/zabbix/zabbix_host.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/monitoring/zabbix/zabbix_host.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @D3DeFi @cove @eikef @harrisongu
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
So, how was python3 installed and how was zabbix-api installed ?
@mscherer 

Python3 is installed in Ubuntu Xenial by default, package `python3.5-minimal`.

zabbix-api: `apt install python3-pip; pip3 install zabbix-api`. 

```
# pip3 list | grep zabbix
zabbix-api (0.5.3)
```
so the error is likely somewhere else in the try/except clause, since look ok.
@mscherer yep, `from zabbix_api import ZabbixAPI, ZabbixAPISubClass` works fine in python3. Btw, why does it change "zabbix" to "********" ? Even in error report: 

```
The full traceback is:
  File "/tmp/ansible_IdqsKH/ansible_module_********_host.py", line 233, in <module>
    from ********_api import ZabbixAPI, ZabbixAPISubClass
```

Could it literally try to execute `from ********_api import ZabbixAPI, ZabbixAPISubClass` ?
No, I think that's the filtering of the password. If the password is 'zabbix', it would be filtered.
@mscherer Got it, it's default Admin password for a fresh Zabbix installation. Is there any way I can help debug this?
I would start by using the git version, and remove the try/expect from code in zabbix_host. It should show a better backtrace. Now, there isn't anything more on your side, I just need to see if I can reproduce it.
https://github.com/ansible/ansible/commits/devel/lib/ansible/modules/monitoring/zabbix/zabbix_host.py

Last change was fix for zabbix <= 3.0, I got 3.4, and change before that was long time ago, it's already in ansible 2.5.1.

Changed in `zabbix_host.py` `try:` to `if True:`, here is what I got running playbook with `-vvvv`:

```
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_sixogx86/ansible_module_zabbix_host.py", line 862, in <module>
    main()
  File "/tmp/ansible_sixogx86/ansible_module_zabbix_host.py", line 798, in main
    for key in interface.keys():
RuntimeError: dictionary changed size during iteration

fatal: [nms1-dev]: FAILED! => {
    "changed": false, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_sixogx86/ansible_module_zabbix_host.py\", line 862, in <module>\n    main()\n  File \"/tmp/ansible_sixogx86/ansible_module_zabbix_host.py\", line 798, in main\n    for key in interface.keys():\nRuntimeError: dictionary changed size during iteration\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 1
}
```
@mscherer I even manged to fix it this way:

Changed line 798 in `zabbix_host.py` this:

```
                    for key in interface.keys():
                        if key in ['interfaceid', 'hostid', 'bulk']:
                            interface.pop(key, None)
```
to this:
```
                    for key in tuple(interface.keys()):
                        if key in ['interfaceid', 'hostid', 'bulk']:
                            interface.pop(key, None)
```

I'm not sure this it correct pythonic way to do things, but it works.

Resolving patch:
diff --git a/lib/ansible/modules/monitoring/zabbix/zabbix_host.py b/lib/ansible/modules/monitoring/zabbix/zabbix_host.py
--- a/lib/ansible/modules/monitoring/zabbix/zabbix_host.py
+++ b/lib/ansible/modules/monitoring/zabbix/zabbix_host.py
@@ -789,7 +789,7 @@ def main():
             if not force or not interfaces:
                 for interface in copy.deepcopy(exist_interfaces):
                     # remove values not used during hostinterface.add/update calls
-                    for key in interface.keys():
+                    for key in tuple(interface.keys()):
                         if key in ['interfaceid', 'hostid', 'bulk']:
                             interface.pop(key, None)
 


Test patch: