Issue 1:
$ ansible -m raw localhost -a 'echo connection' --sudo -K -vvvv -c local
Using /etc/ansible/ansible.cfg as config file
SUDO password:
Loaded callback minimal of type stdout, v2.0
ESTABLISH LOCAL CONNECTION FOR USER: badger
127.0.0.1 EXEC /bin/sh -c 'sudo -H -S -p "[sudo via ansible, key=dsvxgavwyxalsjrucewuzwdyxmpasolo] password: " -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-dsvxgavwyxalsjrucewuzwdyxmpasolo; echo connection'"'"''
localhost | FAILED | rc=0 >>
ERROR! privilege output closed while waiting for password prompt:
BECOME-SUCCESS-dsvxgavwyxalsjrucewuzwdyxmpasolo
connection

[pts/36@roan /srv/ansible/ansible-friday]$ ansible -m raw localhost -a '/bin/echo connection' --sudo -K -vvvv -c local
SUDO password:
localhost | SUCCESS | rc=0 >>
connection

[pts/36@roan /srv/ansible/ansible-friday]$ ansible -m raw -i 'rhel7-test,' '*' -a 'echo connection' --sudo -K
SUDO password:
rhel7-test | SUCCESS | rc=0 >>

connection

[pts/36@roan /srv/ansible/ansible-friday]$ ansible -m raw -i 'rhel7-test,' '*' -a '/bin/echo connection' --sudo -K
SUDO password:
rhel7-test | SUCCESS | rc=0 >>

connection

[pts/36@roan /srv/ansible/ansible-friday]$ *[devel] (20:03:38)

The first run is the problem run. The rest work. Could it be a problem with the echo shell builtin (since the version with full path to echo works?) I also notice that the full path to echo but still running on localhost has one less newline in its output (between the header and "connection") than the output when run over ssh. Maybe that's a clue?

This does work in 1.9.x:

$ ansible -m raw localhost -a 'echo connection' --sudo -K
SUDO password:
localhost | success | rc=0 >>
connection

$

EDIT: pipelining must be on

Issue 2:
ISSUE TYPE

Bug Report

COMPONENT NAME

plugins/connection/ssh

ANSIBLE VERSION

ansible 2.3.2.0
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides
python version = 2.7.5 (default, Aug 4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]

But I believe the bug also exists in v2.4

CONFIGURATION

[ssh_connection]

control_path = %(directory)s/%%h-%%p-%%r

OS / ENVIRONMENT

CentOS 7 control & target machines

SUMMARY

This is another incarnation of #29095 but in case of ssh retry due to ControlPersist broken pipe (#16731 fix).

STEPS TO REPRODUCE

I'm sorry, but it is hard for me to specify reproduction steps.

The issue has occurred in various different tasks of a long playbook.

It was executed on a fairly slow VM control machine and was provisioning over 30 remote machines.

EXPECTED RESULTS

sshpass pipe should be recreated in case of retry due to ControlPersist broken pipe

ACTUAL RESULTS

(...)
<192.168.0.22> ESTABLISH SSH CONNECTION FOR USER: XXX
(...)
<192.168.0.22> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<192.168.0.22> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<192.168.0.22> SSH: ANSIBLE_REMOTE_PORT/remote_port/ansible_port set: (-o)(Port=22)
<192.168.0.22> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=XXX)
<192.168.0.22> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<192.168.0.22> SSH: PlayContext set ssh_common_args: (-o)(StrictHostKeyChecking=no)
<192.168.0.22> SSH: PlayContext set ssh_extra_args: (-o)(UserKnownHostsFile=/dev/null)
<192.168.0.22> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/XXX/.ansible/cp/%h-%p-%r)
<192.168.0.22> SSH: EXEC sshpass -d14 ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o User=XXX -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ControlPath=/home/XXX/.ansible/cp/%h-%p-%r 192.168.0.22 '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
(...)
RETRYING BECAUSE OF CONTROLPERSIST BROKEN PIPE
<192.168.0.22> SSH: EXEC sshpass -d14 ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o User=XXX -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ControlPath=/home/XXX/.ansible/cp/%h-%p-%r 192.168.0.22 '/bin/sh -c '"'"'echo ~ && sleep 0'"'"''
(...)
close failed in file object destructor:
IOError: [Errno 9] Bad file descriptor
(...)
The full traceback is:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 125, in run
res = self._execute()
File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 522, in _execute
result = self._handler.run(task_vars=variables)
File "/usr/lib/python2.7/site-packages/ansible/plugins/action/package.py", line 69, in run
result.update(self._execute_module(module_name=module, module_args=new_module_args, task_vars=task_vars, wrap_async=self._task.async))
File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 637, in _execute_module
tmp = self._make_tmp_path()
File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 248, in _make_tmp_path
tmpdir = self._remote_expand_user(C.DEFAULT_REMOTE_TMP, sudoable=False)
File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 545, in _remote_expand_user
data = self._low_level_execute_command(cmd, sudoable=False)
File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 883, in _low_level_execute_command
rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 788, in exec_command
(returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 85, in wrapped
return_tuple = func(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 461, in _run
os.write(self.sshpass_pipe[1], to_bytes(self._play_context.password) + b'
')
OSError: [Errno 9] Bad file descriptor
fatal: [192.168.0.22]: FAILED! => {
"failed": true,
"msg": "Unexpected failure during module execution.",
"stdout": ""
}