Issue 1:
Issue Type:

Bug Report

Ansible Version:

ansible 2.0.0.2
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides

Ansible Configuration:

Using default ansible.cfg.

Inventory:

[all:vars]
ansible_python_interpreter=/usr/bin/python2

[kubernetes]
kubemaster1 ansible_ssh_host=192.168.0.150
kubemaster2 ansible_ssh_host=192.168.0.220
kubenode1 ansible_ssh_host=192.168.0.157
kubenode2 ansible_ssh_host=192.168.0.73

Environment:

Arch Linux (up to date) on all machines.

Summary:

When using any module (I'm using ping for testing), Ansible sometimes fails on a host. It never happens when I use one single host and it fails in about 8/10 cases. The failing hosts are differing.

I'm calling Ansible like this:

ansible -vvvv -e 'ansible_ssh_user=root' kubernetes -m ping

Full log output of a failed run:

Using /etc/ansible/ansible.cfg as config file
Loaded callback minimal of type stdout, v2.0
<192.168.0.220> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.150> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.220> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.220 'mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.3-129748284741880 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.3-129748284741880 )"'
<192.168.0.150> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.150 'mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.3-110669559663006 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.3-110669559663006 )"'
<192.168.0.73> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.157> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.73> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.73 'mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.32-132910215829706 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.32-132910215829706 )"'
<192.168.0.220> PUT /tmp/janne/tmpqd8UFn TO /root/.ansible/tmp/ansible-tmp-1455401136.3-129748284741880/ping
<192.168.0.220> SSH: EXEC sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.0.220]'
<192.168.0.157> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.157 'mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.32-263198601926357 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.32-263198601926357 )"'
<192.168.0.150> PUT /tmp/janne/tmpvvf2uM TO /root/.ansible/tmp/ansible-tmp-1455401136.3-110669559663006/ping
<192.168.0.150> SSH: EXEC sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.0.150]'
kubenode2 | FAILED! => {
"failed": true,
"msg": "ERROR! failed to resolve remote temporary directory from ansible-tmp-1455401136.32-132910215829706: `mkdir -p \"$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.32-132910215829706 )\" && echo \"$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.32-132910215829706 )\"` returned empty string"
}
<192.168.0.220> ESTABLISH SSH CONNECTION FOR USER: root
kubenode1 | FAILED! => {
"failed": true,
"msg": "ERROR! failed to resolve remote temporary directory from ansible-tmp-1455401136.32-263198601926357: `mkdir -p \"$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.32-263198601926357 )\" && echo \"$( echo $HOME/.ansible/tmp/ansible-tmp-1455401136.32-263198601926357 )\"` returned empty string"
}
<192.168.0.220> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.220 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1455401136.3-129748284741880/ping; rm -rf "/root/.ansible/tmp/ansible-tmp-1455401136.3-129748284741880/" > /dev/null 2>&1'
<192.168.0.150> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.150> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.150 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1455401136.3-110669559663006/ping; rm -rf "/root/.ansible/tmp/ansible-tmp-1455401136.3-110669559663006/" > /dev/null 2>&1'
kubemaster2 | SUCCESS => {
"changed": false,
"invocation": {
"module_args": {
"data": null
},
"module_name": "ping"
},
"ping": "pong"
}
kubemaster1 | SUCCESS => {
"changed": false,
"invocation": {
"module_args": {
"data": null
},
"module_name": "ping"
},
"ping": "pong"
}

Note that different hosts fail sometimes.

Steps To Reproduce:

Run the command above on a small group of Arch Linux hosts. Sometimes, it fails.

Expected Results:

Using /etc/ansible/ansible.cfg as config file
Loaded callback minimal of type stdout, v2.0
<192.168.0.157> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.150> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.157> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.157 'mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401324.79-221794005648841 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401324.7
9-221794005648841 )"'
<192.168.0.150> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.150 'mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401324.79-182183570705768 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401324.7
9-182183570705768 )"'
<192.168.0.73> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.73> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o C
onnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.73 'mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401324.79-134588512123476 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401324.79-
134588512123476 )"'
<192.168.0.220> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.220> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.220 'mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401324.8-245049243481998 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455401324.8-
245049243481998 )"'
<192.168.0.157> PUT /tmp/janne/tmpU2Q_8L TO /root/.ansible/tmp/ansible-tmp-1455401324.79-221794005648841/ping
<192.168.0.157> SSH: EXEC sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ro
ot -o ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.0.157]'
<192.168.0.150> PUT /tmp/janne/tmpiBTinc TO /root/.ansible/tmp/ansible-tmp-1455401324.79-182183570705768/ping
<192.168.0.150> SSH: EXEC sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ro
ot -o ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.0.150]'
<192.168.0.73> PUT /tmp/janne/tmpV_eY8S TO /root/.ansible/tmp/ansible-tmp-1455401324.79-134588512123476/ping
<192.168.0.73> SSH: EXEC sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=roo
t -o ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.0.73]'
<192.168.0.220> PUT /tmp/janne/tmpbllWlE TO /root/.ansible/tmp/ansible-tmp-1455401324.8-245049243481998/ping
<192.168.0.220> SSH: EXEC sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ro
ot -o ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.0.220]'
<192.168.0.150> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.150> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.150 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1455401324.79-182183570705768/ping
; rm -rf "/root/.ansible/tmp/ansible-tmp-1455401324.79-182183570705768/" > /dev/null 2>&1'
<192.168.0.157> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.157> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.157 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1455401324.79-221794005648841/ping
; rm -rf "/root/.ansible/tmp/ansible-tmp-1455401324.79-221794005648841/" > /dev/null 2>&1'
<192.168.0.220> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.220> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o
ConnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.220 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1455401324.8-245049243481998/ping;
rm -rf "/root/.ansible/tmp/ansible-tmp-1455401324.8-245049243481998/" > /dev/null 2>&1'
<192.168.0.73> ESTABLISH SSH CONNECTION FOR USER: root
<192.168.0.73> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o C
onnectTimeout=10 -o ControlPath=/home/janne/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.0.73 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1455401324.79-134588512123476/ping;
rm -rf "/root/.ansible/tmp/ansible-tmp-1455401324.79-134588512123476/" > /dev/null 2>&1'
kubemaster1 | SUCCESS => {
"changed": false,
"invocation": {
"module_args": {
"data": null
},
"module_name": "ping"
},
"ping": "pong"
}
kubenode1 | SUCCESS => {
"changed": false,
"invocation": {
"module_args": {
"data": null
},
"module_name": "ping"
},
"ping": "pong"
}
kubemaster2 | SUCCESS => {
"changed": false,
"invocation": {
"module_args": {
"data": null
},
"module_name": "ping"
},
"ping": "pong"
}
kubenode2 | SUCCESS => {
"changed": false,
"invocation": {
"module_args": {
"data": null
},
"module_name": "ping"
},
"ping": "pong"
}

Actual Results:

The error that I added above.

Issue 2:
After I upgrade ansible 2.0.0.1 from 1.9.4, when performing a playlook always encounter intermittent fault "failed to resolve remote temporary directory from".

$ ansible-playbook playlooks/playlook-filters.yml

PLAY ***************************************************************************

TASK [command] *****************************************************************
fatal: [YT_8_22]: FAILED! => {"failed": true, "msg": "ERROR! failed to resolve remote temporary directory from ansible-tmp-1452759681.1-95441304852350: `( umask 22 && mkdir -p \"$( echo /tmp/.ansible/tmp/ansible-tmp-1452759681.1-95441304852350 )\" && echo \"$( echo /tmp/.ansible/tmp/ansible-tmp-1452759681.1-95441304852350 )\" )` returned empty string"}
...ignoring

TASK [debug] *******************************************************************
ok: [YT_8_22] => {
"msg": "it failed"
}

PLAY RECAP *********************************************************************
YT_8_22 : ok=2 changed=0 unreachable=0 failed=0

This is happening for one of the tasks randomly, following are those tasks:

---
- hosts: YT_8_22
tasks:
- shell: /bin/true
register: result
ignore_errors: True

- debug: msg="it failed"
when: result|failed

I try on 1.9.4, have not seen the phenomenon. Can anybody tell me how to do that ?

I found another links to similar problems: https://groups.google.com/forum/#!searchin/ansible-project/Intermittent$20error%7Csort:relevance/ansible-project/FyK6au2O9KY/tWuf31P9AQAJ