Issue 1:
ISSUE TYPE

Bug Report

COMPONENT NAME

ANSIBLE VERSION

ansible 2.4.2.0
config file = /Users/user/Desktop/ansible2/ansible.cfg
configured module search path = [u'/Users/user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /Library/Python/2.7/site-packages/ansible
executable location = /usr/local/bin/ansible
python version = 2.7.10 (default, Jul 15 2017, 17:16:57) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]

CONFIGURATION

DEFAULT_REMOTE_TMP(/Users/user/Desktop/ansible2/ansible.cfg) = ~/.ansible/tmp

HOST_KEY_CHECKING(/Users/user/Desktop/ansible2/ansible.cfg) = False

OS / ENVIRONMENT

macOS High Sierra 10.13.3

SUMMARY

I'm running NX-OSv7.3.0 in GNS3 2.1.3, attempting to run a simple ansible script as seen below and get hit with the error.

STEPS TO REPRODUCE

a.yml

- hosts: switch
tasks:
- name: Nexus
nxos_facts:
gather_subset: all

inventory file:

[all:vars]
ansible_connection=ssh
ansible_ssh_user=admin
ansible_ssh_pass=admin

[switch]
172.16.0.2

I also have in my local hosts file:

switch 172.16.0.2

EXPECTED RESULTS

Expected ansible to connect via SSH and get some facts from NX-OSv.

ACTUAL RESULTS

TASK [Gathering Facts] ***********************************************************************************
Using module file /Library/Python/2.7/site-packages/ansible/modules/system/setup.py
<172.16.0.2> ESTABLISH SSH CONNECTION FOR USER: admin
<172.16.0.2> SSH: EXEC sshpass -d42 ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o User=admin -o ConnectTimeout=10 -o ControlPath=/Users/user/.ansible/cp/bb64dc9435 172.16.0.2 '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
<172.16.0.2> (16, "Syntax error while parsing '/bin/sh -c 'echo ~ && sleep 0''\n\n\nCmd exec error.\n", 'OpenSSH_7.6p1, LibreSSL 2.6.2\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 48: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/Users/user/.ansible/cp/bb64dc9435" does not exist\r\ndebug2: resolving "172.16.0.2" port 22\r\ndebug2: ssh_connect_direct: needpriv 0\r\ndebug1: Connecting to 172.16.0.2 [172.16.0.2] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9817 ms remain after connect\r\ndebug1: identity file /Users/user/.ssh/id_rsa type 0\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /Users/user/.ssh/id_rsa-cert type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /Users/user/.ssh/id_dsa type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /Users/user/.ssh/id_dsa-cert type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /Users/user/.ssh/id_ecdsa type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /Users/user/.ssh/id_ecdsa-cert type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /Users/user/.ssh/id_ed25519 type -1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /Users/user/.ssh/id_ed25519-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_7.6\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_6.2 FIPS\r\ndebug1: match: OpenSSH_6.2 FIPS pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to 172.16.0.2:22 as \'admin\'\r\ndebug3: hostkeys_foreach: reading file "/Users/user/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type RSA in file /Users/user/.ssh/known_hosts:6\r\ndebug3: load_hostkeys: loaded 1 keys from 172.16.0.2\r\ndebug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c\r\ndebug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519\r\ndebug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com\r\ndebug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com\r\ndebug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: zlib@openssh.com,zlib,none\r\ndebug2: compression stoc: zlib@openssh.com,zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1\r\ndebug2: host key algorithms: ssh-rsa\r\ndebug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr\r\ndebug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr\r\ndebug2: MACs ctos: hmac-sha1\r\ndebug2: MACs stoc: hmac-sha1\r\ndebug2: compression ctos: none,zlib@openssh.com\r\ndebug2: compression stoc: none,zlib@openssh.com\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: ecdh-sha2-nistp256\r\ndebug1: kex: host key algorithm: ssh-rsa\r\ndebug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha1 compression: zlib@openssh.com\r\ndebug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha1 compression: zlib@openssh.com\r\ndebug3: send packet: type 30\r\ndebug1: sending SSH2_MSG_KEX_ECDH_INIT\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ssh-rsa SHA256:oSmRIx+fcmRlvRewAQwBMA1YWG6vjlsMuwTrJwh+b88\r\ndebug3: hostkeys_foreach: reading file "/Users/user/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type RSA in file /Users/user/.ssh/known_hosts:6\r\ndebug3: load_hostkeys: loaded 1 keys from 172.16.0.2\r\ndebug1: Host \'172.16.0.2\' is known and matches the RSA host key.\r\ndebug1: Found key in /Users/user/.ssh/known_hosts:6\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey after 4294967296 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey after 4294967296 blocks\r\ndebug2: key: /Users/user/.ssh/id_rsa (0x7ffd92f00570)\r\ndebug2: key: /Users/user/.ssh/id_dsa (0x0)\r\ndebug2: key: /Users/user/.ssh/id_ecdsa (0x0)\r\ndebug2: key: /Users/user/.ssh/id_ed25519 (0x0)\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 53\r\ndebug3: input_userauth_banner\r\nUser Access Verification\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password,keyboard-interactive\r\ndebug3: start over, passed a different list publickey,password,keyboard-interactive\r\ndebug3: preferred publickey,keyboard-interactive,password\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: keyboard-interactive,password\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Offering public key: RSA SHA256:7ArvlnJl2z7pIaLMoi3os8c3CYG7vTlBevLlXPVkNOY /Users/user/.ssh/id_rsa\r\ndebug3: send_pubkey_test\r\ndebug3: send packet: type 50\r\ndebug2: we sent a publickey packet, wait for reply\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password,keyboard-interactive\r\ndebug1: Trying private key: /Users/user/.ssh/id_dsa\r\ndebug3: no such identity: /Users/user/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /Users/user/.ssh/id_ecdsa\r\ndebug3: no such identity: /Users/user/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /Users/user/.ssh/id_ed25519\r\ndebug3: no such identity: /Users/user/.ssh/id_ed25519: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug3: authmethod_lookup keyboard-interactive\r\ndebug3: remaining preferred: password\r\ndebug3: authmethod_is_enabled keyboard-interactive\r\ndebug1: Next authentication method: keyboard-interactive\r\ndebug2: userauth_kbdint\r\ndebug3: send packet: type 50\r\ndebug2: we sent a keyboard-interactive packet, wait for reply\r\ndebug3: receive packet: type 60\r\ndebug2: input_userauth_info_req\r\ndebug2: input_userauth_info_req: num_prompts 1\r\ndebug3: send packet: type 61\r\ndebug3: receive packet: type 60\r\ndebug2: input_userauth_info_req\r\ndebug2: input_userauth_info_req: num_prompts 0\r\ndebug3: send packet: type 61\r\ndebug3: receive packet: type 52\r\ndebug1: Enabling compression at level 6.\r\ndebug1: Authentication succeeded (keyboard-interactive).\r\nAuthenticated to 172.16.0.2 ([172.16.0.2]:22).\r\ndebug1: setting up multiplex master socket\r\ndebug3: muxserver_listen: temporary control path /Users/user/.ansible/cp/bb64dc9435.P0TkokpfkhewzqzW\r\ndebug2: fd 4 setting O_NONBLOCK\r\ndebug3: fd 4 is O_NONBLOCK\r\ndebug3: fd 4 is O_NONBLOCK\r\ndebug1: channel 0: new [/Users/user/.ansible/cp/bb64dc9435]\r\ndebug3: muxserver_listen: mux listener channel 0 fd 4\r\ndebug2: fd 3 setting TCP_NODELAY\r\ndebug3: ssh_packet_set_tos: set IP_TOS 0x08\r\ndebug1: control_persist_detach: backgrounding master process\r\ndebug2: control_persist_detach: background process is 60991\r\ndebug2: fd 4 setting O_NONBLOCK\r\ndebug1: forking to background\r\ndebug1: Entering interactive session.\r\ndebug1: pledge: id\r\ndebug2: set_control_persist_exit_time: schedule exit in 60 seconds\r\ndebug1: multiplexing control connection\r\ndebug3: fd 5 is O_NONBLOCK\r\ndebug3: fd 5 is O_NONBLOCK\r\ndebug1: channel 1: new [mux-control]\r\ndebug3: channel_post_mux_listener: new mux channel 1 fd 5\r\ndebug3: mux_master_read_cb: channel 1: hello sent\r\ndebug2: set_control_persist_exit_time: cancel scheduled exit\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x00000001 len 4\r\ndebug2: process_mux_master_hello: channel 1 slave version 4\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x10000004 len 4\r\ndebug2: process_mux_alive_check: channel 1: alive check\r\ndebug3: mux_client_request_alive: done pid = 60993\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_master_read_cb: channel 1 packet type 0x10000002 len 98\r\ndebug2: process_mux_new_session: channel 1: request tty 0, X 0, agent 0, subsys 0, term "xterm-256color", cmd "/bin/sh -c \'echo ~ && sleep 0\'", env 1\r\ndebug3: process_mux_new_session: got fds stdin 6, stdout 7, stderr 8\r\ndebug2: fd 7 setting O_NONBLOCK\r\ndebug2: fd 8 setting O_NONBLOCK\r\ndebug1: channel 2: new [client-session]\r\ndebug2: process_mux_new_session: channel_new: 2 linked to control channel 1\r\ndebug2: channel 2: send open\r\ndebug3: send packet: type 90\r\ndebug3: receive packet: type 91\r\ndebug2: channel_input_open_confirmation: channel 2: callback start\r\ndebug2: client_session2_setup: id 2\r\ndebug1: Sending environment.\r\ndebug1: Sending env LC_CTYPE = UTF-8\r\ndebug2: channel 2: request env confirm 0\r\ndebug3: send packet: type 98\r\ndebug1: Sending command: /bin/sh -c \'echo ~ && sleep 0\'\r\ndebug2: channel 2: request exec confirm 1\r\ndebug3: send packet: type 98\r\ndebug3: mux_session_confirm: sending success reply\r\ndebug2: channel_input_open_confirmation: channel 2: callback done\r\ndebug2: channel 2: open confirm rwindow 0 rmax 32768\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: channel 2: rcvd adjust 2097152\r\ndebug3: receive packet: type 99\r\ndebug2: channel_input_status_confirm: type 99 id 2\r\ndebug2: exec request accepted on channel 2\r\ndebug3: receive packet: type 96\r\ndebug2: channel 2: rcvd eof\r\ndebug2: channel 2: output open -> drain\r\ndebug2: channel 2: obuf empty\r\ndebug2: channel 2: close_write\r\ndebug2: channel 2: output drain -> closed\r\ndebug3: receive packet: type 98\r\ndebug1: client_input_channel_req: channel 2 rtype exit-status reply 0\r\ndebug3: mux_exit_message: channel 2: exit message, exitval 16\r\ndebug3: receive packet: type 98\r\ndebug1: client_input_channel_req: channel 2 rtype eow@openssh.com reply 0\r\ndebug2: channel 2: rcvd eow\r\ndebug2: channel 2: close_read\r\ndebug2: channel 2: input open -> closed\r\ndebug3: receive packet: type 97\r\ndebug2: channel 2: rcvd close\r\ndebug3: channel 2: will not send data after close\r\ndebug2: channel 2: send close\r\ndebug3: send packet: type 97\r\ndebug2: channel 2: is dead\r\ndebug2: channel 2: gc: notify user\r\ndebug3: mux_master_session_cleanup_cb: entering for channel 2\r\ndebug2: channel 1: rcvd close\r\ndebug2: channel 1: output open -> drain\r\ndebug2: channel 1: close_read\r\ndebug2: channel 1: input open -> closed\r\ndebug2: channel 2: gc: user detached\r\ndebug2: channel 2: is dead\r\ndebug2: channel 2: garbage collecting\r\ndebug1: channel 2: free: client-session, nchannels 3\r\ndebug3: channel 2: status: The following connections are open:\r\n #1 mux-control (t16 nr0 i3/0 o1/16 fd 5/5 cc -1)\r\n #2 client-session (t4 r0 i3/0 o3/0 fd -1/-1 cc -1)\r\n\r\ndebug2: channel 1: obuf empty\r\ndebug2: channel 1: close_write\r\ndebug2: channel 1: output drain -> closed\r\ndebug2: channel 1: is dead (local)\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: channel 1: gc: notify user\r\ndebug3: mux_master_control_cleanup_cb: entering for channel 1\r\ndebug2: channel 1: gc: user detached\r\ndebug2: channel 1: is dead (local)\r\ndebug2: channel 1: garbage collecting\r\ndebug1: channel 1: free: mux-control, nchannels 2\r\ndebug3: channel 1: status: The following connections are open:\r\n #1 mux-control (t16 nr0 i3/0 o3/0 fd 5/5 cc -1)\r\n\r\ndebug2: set_control_persist_exit_time: schedule exit in 60 seconds\r\ndebug2: Received exit status from master 16\r\n')
<172.16.0.2> ESTABLISH SSH CONNECTION FOR USER: admin
<172.16.0.2> SSH: EXEC sshpass -d42 ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o User=admin -o ConnectTimeout=10 -o ControlPath=/Users/user/.ansible/cp/bb64dc9435 172.16.0.2 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo Cmd exec error./.ansible/tmp/ansible-tmp-1517295845.92-147957596299827 `" && echo ansible-tmp-1517295845.92-147957596299827="` echo Cmd exec error./.ansible/tmp/ansible-tmp-1517295845.92-147957596299827 `" ) && sleep 0'"'"''
<172.16.0.2> (16, 'Syntax error while parsing \'/bin/sh -c \'( umask 77 && mkdir -p "` echo Cmd exec error./.ansible/tmp/ansible-tmp-1517295845.92-147957596299827 `" && echo ansible-tmp-1517295845.92-147957596299827="` echo Cmd exec error./.ansible/tmp/ansible-tmp-1517295845.92-147957596299827 `" ) && sleep 0\'\'\n\n\nCmd exec error.\n', 'OpenSSH_7.6p1, LibreSSL 2.6.2\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 48: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 60993\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 16\r\n')
fatal: [172.16.0.2]: UNREACHABLE! => {
"changed": false,
"msg": "Authentication or permission failure. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote temp path in ansible.cfg to a path rooted in \"/tmp\". Failed command was: ( umask 77 && mkdir -p \"` echo Cmd exec error./.ansible/tmp/ansible-tmp-1517295845.92-147957596299827 `\" && echo ansible-tmp-1517295845.92-147957596299827=\"` echo Cmd exec error./.ansible/tmp/ansible-tmp-1517295845.92-147957596299827 `\" ), exited with result 16, stdout output: Syntax error while parsing '/bin/sh -c '( umask 77 && mkdir -p \"` echo Cmd exec error./.ansible/tmp/ansible-tmp-1517295845.92-147957596299827 `\" && echo ansible-tmp-1517295845.92-147957596299827=\"` echo Cmd exec error./.ansible/tmp/ansible-tmp-1517295845.92-147957596299827 `\" ) && sleep 0''\n\n\nCmd exec error.\n, stderr output: OpenSSH_7.6p1, LibreSSL 2.6.2\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 48: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 60993\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 16\r\n",
"unreachable": true
}
to retry, use: --limit @/Users/user/Desktop/ansible2/1.retry

PLAY RECAP ***********************************************************************************************
172.16.0.2 : ok=0 changed=0 unreachable=1 failed=0

Issue 2:
chaoslemur should return ok sometimes, failure sometimes, nothing sometimes, and invalid JSON sometimes, so that you can load up your system with lots of duplicate host aliase and check that failed hosts get filtered out and reported on correctly.

we do not have ship chaoslemur but it would be very useful for testing.