Repository: ansible/ansible

Issue:
Network module using persistent connection fails when debug logs are enabled
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report


##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
ios_config
junos_config


##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
2.5
```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"

Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).

-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->

##### SUMMARY
<!--- Explain the problem briefly -->

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
Enable debug logs

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
-  name: test
   hosts: vsrx
   connection: local
   gather_facts: no

   tasks:
   - name: test
     junos_command:
          commands:
            - show version
     register: response
```
OR
```yaml
---
-  name: test
   hosts: vsrx
   connection: netconf
   gather_facts: no

   tasks:
   - name: test
     junos_command:
          commands:
            - show version
     register: response
```
<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Should run successfully

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
The full traceback is:
Traceback (most recent call last):
  File "/Users/gnalawad/ansible/ganeshrn/ansible/lib/ansible/executor/task_executor.py", line 119, in run
    res = self._execute()
  File "/Users/gnalawad/ansible/ganeshrn/ansible/lib/ansible/executor/task_executor.py", line 523, in _execute
    result = self._handler.run(task_vars=variables)
  File "/Users/gnalawad/ansible/ganeshrn/ansible/lib/ansible/plugins/action/junos.py", line 73, in run
    socket_path = connection.run()
  File "/Users/gnalawad/ansible/ganeshrn/ansible/lib/ansible/plugins/connection/persistent.py", line 68, in run
    socket_path = self._start_connection()
  File "/Users/gnalawad/ansible/ganeshrn/ansible/lib/ansible/plugins/connection/persistent.py", line 96, in _start_connection
    result = json.loads(to_text(stdout, errors='surrogate_then_replace'))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 369, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 8 - line 4 column 121 (char 7 - 405)

fatal: [38.145.35.195]: FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}
```



Discussion:

Resolving patch:
diff --git a/lib/ansible/executor/task_executor.py b/lib/ansible/executor/task_executor.py
--- a/lib/ansible/executor/task_executor.py
+++ b/lib/ansible/executor/task_executor.py
@@ -813,6 +813,7 @@ def _start_connection(self):
         stdin.write(src)
 
         stdin.write(b'\n#END_INIT#\n')
+        stdin.flush()
 
         (stdout, stderr) = p.communicate()
         stdin.close()
@@ -829,7 +830,7 @@ def _start_connection(self):
         if 'error' in result:
             if self._play_context.verbosity > 2:
                 msg = "The full traceback is:\n" + result['exception']
-                display.display(result['exception'], color=C.COLOR_ERROR)
+                display.display(msg, color=C.COLOR_ERROR)
             raise AnsibleError(result['error'])
 
         return result['socket_path']
diff --git a/lib/ansible/plugins/connection/persistent.py b/lib/ansible/plugins/connection/persistent.py
--- a/lib/ansible/plugins/connection/persistent.py
+++ b/lib/ansible/plugins/connection/persistent.py
@@ -88,6 +88,7 @@ def _start_connection(self):
         stdin.write(src)
 
         stdin.write(b'\n#END_INIT#\n')
+        stdin.flush()
 
         (stdout, stderr) = p.communicate()
         stdin.close()
@@ -104,7 +105,7 @@ def _start_connection(self):
         if 'error' in result:
             if self._play_context.verbosity > 2:
                 msg = "The full traceback is:\n" + result['exception']
-                display.display(result['exception'], color=C.COLOR_ERROR)
+                display.display(msg, color=C.COLOR_ERROR)
             raise AnsibleError(result['error'])
 
         return result['socket_path']
diff --git a/lib/ansible/utils/display.py b/lib/ansible/utils/display.py
--- a/lib/ansible/utils/display.py
+++ b/lib/ansible/utils/display.py
@@ -126,6 +126,8 @@ def display(self, msg, color=None, stderr=False, screen_only=False, log_only=Fal
                 # characters that are invalid in the user's locale
                 msg2 = to_text(msg2, self._output_encoding(stderr=stderr), errors='replace')
 
+            # Note: After Display() class is refactored need to update the log capture
+            # code in 'bin/ansible-connection' (and other relevant places).
             if not stderr:
                 fileobj = sys.stdout
             else:


Test patch: