Repository: ansible/ansible

Issue:
Pipelining seems to be disabled since Ansible 2.4
This issue is related to https://github.com/ansible/ansible/issues/31091 , which was closed yesterday. I investigated a bit, tried my best to test on multiple Ansible versions with the same tasks. I feel that pipelining behavior has changed between Ansible 2.3 and recent releases, starting at 2.4 and affecting also devel.

Let me know if you need more information or if I missed something. I'll be glad to help!

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
SSH Pipelining.

##### ANSIBLE VERSION
Ansible 2.3 is not affected, Ansible 2.4 and greater (most notably devel, ran straight from source) are affected.

Here is an example with Ansible 2.4, straight from source:
```
$ git checkout stable-2.4
$ source hacking/env-setup
$ ansible --version
ansible 2.4.1.0 (stable-2.4 6945f6d3b2) last updated 2017/09/30 14:39:20 (GMT +200)
  config file = None
  configured module search path = [u'/Users/pmougel/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /private/tmp/ansible/lib/ansible
  executable location = /private/tmp/ansible/bin/ansible
  python version = 2.7.10 (default, Feb  7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
```

##### CONFIGURATION
<!---
$ ansible-config dump --only-changed
$
-->

##### OS / ENVIRONMENT
My machine: Mac OS Sierra 10.12.6 (16G29).
Remote: Debian GNU/Linux 8

##### SUMMARY

Enabling pipelining does not work on Ansible 2.4+.

##### STEPS TO REPRODUCE

*See [this gist](https://gist.github.com/PaulMougel/a00997b4ac9757c653674d65e6b6c1a4)* for the full log.

Basically: default ansible configuration, test with `ANSIBLE_SSH_PIPELINING=False` and `ANSIBLE_SSH_PIPELINING=True` the following command:

```
ansible -vvv -i /tmp/my-host my-host.com -m shell -a 'echo ok'
```

On Ansible 2.3, the expected behavior matches the actual results:
- If pipelining is disabled, 4 `EXEC ssh` and one `EXEC sftp` commands are launched. It takes around 0.84s to execute the command.
- If pipelining is enabled, only one `EXEC ssh` command is launched. It takes around 0,46s to execute the command.

On Ansible 2.4 and devel:
- If pipelining is disabled, 4 `EXEC ssh` and one `EXEC sftp` commands are launched. It takes around 1s to execute the command.
- If pipelining is disabled, 4 `EXEC ssh` and one `EXEC sftp` commands are launched. It takes around 1s to execute the command.

##### EXPECTED RESULTS

- Similar behavior between Ansible 2.4 & devel and Ansible 2.3.
- No `sftp` commands when pipelining is enabled, as in Ansible 2.3
- Thus, shorter execution time when pipelining is enabled, as show in Ansible 2.3

##### ACTUAL RESULTS

Activating the pipelining option does not seem to have any effect.


Discussion:
#30760 Shows similar behaviour 
Workaround:  Looks like ANSIBLE_PIPELINING is working.

@bcoca I looked in base.yml and noticed that there's both ANSIBLE_PIPELINING and ANSIBLE_SSH_PIPELINING and could not see nay link between the two entries.  Could it be that ANSIBLE_SSH_PIPELINING is being overridden by ANSBILE_PIPELINING?
@PaulMougel Excellent work on this. Hoping to see this fixed (if it is a bug) on the next release
@abadger You are correct!

It looks like:
- `ANSIBLE_PIPELINING` works as intended
- `ANSIBLE_SSH_PIPELINING` is ignored
- the `pipelining` option in the `[ssh_connection]` section of the `ansible.cfg` configuration file is ignored as well

Resolving patch:
diff --git a/lib/ansible/plugins/connection/ssh.py b/lib/ansible/plugins/connection/ssh.py
--- a/lib/ansible/plugins/connection/ssh.py
+++ b/lib/ansible/plugins/connection/ssh.py
@@ -116,11 +116,18 @@
             - However this conflicts with privilege escalation (become).
               For example, when using sudo operations you must first disable 'requiretty' in the sudoers file for the target hosts,
               which is why this feature is disabled by default.
-          env: [{name: ANSIBLE_SSH_PIPELINING}]
+          env:
+            - name: ANSIBLE_PIPELINING
+            #- name: ANSIBLE_SSH_PIPELINING
           ini:
-          - {key: pipelining, section: ssh_connection}
+            - section: defaults
+              key: pipelining
+            #- section: ssh_connection
+            #  key: pipelining
           type: boolean
-          vars: [{name: ansible_ssh_pipelining}]
+          vars:
+            - name: ansible_pipelining
+            - name: ansible_ssh_pipelining
       private_key_file:
           description:
               - Path to private key file to use for authentication


Test patch:
diff --git a/test/legacy/Makefile b/test/legacy/Makefile
--- a/test/legacy/Makefile
+++ b/test/legacy/Makefile
@@ -58,7 +58,7 @@ $(CREDENTIALS_FILE):
 	@exit 1
 
 amazon: $(CREDENTIALS_FILE)
-	ANSIBLE_HOST_KEY_CHECKING=False ANSIBLE_SSH_PIPELINING=no BOTO_CONFIG=/dev/null ansible-playbook amazon.yml -i $(INVENTORY) -e @$(VARS_FILE) $(CREDENTIALS_ARG) -e "resource_prefix=$(CLOUD_RESOURCE_PREFIX)" -v $(TEST_FLAGS) ; \
+	ANSIBLE_HOST_KEY_CHECKING=False ANSIBLE_PIPELINING=no BOTO_CONFIG=/dev/null ansible-playbook amazon.yml -i $(INVENTORY) -e @$(VARS_FILE) $(CREDENTIALS_ARG) -e "resource_prefix=$(CLOUD_RESOURCE_PREFIX)" -v $(TEST_FLAGS) ; \
     RC=$$? ; \
     CLOUD_RESOURCE_PREFIX="$(CLOUD_RESOURCE_PREFIX)" make amazon_cleanup ; \
     exit $$RC;