Repository: ansible/ansible

Issue:
Service module fails in gentoo when checking status
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
ansible/module/service.py

##### ANSIBLE VERSION
```
ansible 2.2.1.0
  config file = 
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
None

##### OS / ENVIRONMENT
OS Host: OSX 10.11.6
OS Client: Gentoo

##### SUMMARY
Ansible fails when checking the status of a service.
It fails on any service whether it is running or not.

##### STEPS TO REPRODUCE

The problem occurs on my gentoo server but I am also able to reproduce the error in vagrant found here: https://github.com/thomaslorentsen/ansible-gentoo

```yaml
- hosts: localhost
  become: true
  tasks:
  - service:
      name: net.eth0
      state: started
```

##### EXPECTED RESULTS
I would expect that when run on Gentoo, the service is specified is started without error.

##### ACTUAL RESULTS
```
ansible-playbook playbook.yml -vvvv
No config file found; using defaults
 [WARNING]: Host file not found: /etc/ansible/hosts

 [WARNING]: provided hosts list is empty, only localhost is available

Loading callback plugin default of type stdout, v2.0 from /usr/lib64/python3.4/site-packages/ansible/plugins/callback/__init__.py

PLAYBOOK: playbook.yml *********************************************************
1 plays in playbook.yml

PLAY [localhost] ***************************************************************

TASK [setup] *******************************************************************
Using module file /usr/lib64/python3.4/site-packages/ansible/modules/core/system/setup.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: vagrant
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1485728475.1546552-136258276406474 `" && echo ansible-tmp-1485728475.1546552-136258276406474="` echo ~/.ansible/tmp/ansible-tmp-1485728475.1546552-136258276406474 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpgb4ji_2c TO /home/vagrant/.ansible/tmp/ansible-tmp-1485728475.1546552-136258276406474/setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/vagrant/.ansible/tmp/ansible-tmp-1485728475.1546552-136258276406474/ /home/vagrant/.ansible/tmp/ansible-tmp-1485728475.1546552-136258276406474/setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-lpcxxfjkwxbepqicrhaitlvwbgeeoftw; /usr/bin/python3.4 /home/vagrant/.ansible/tmp/ansible-tmp-1485728475.1546552-136258276406474/setup.py; rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1485728475.1546552-136258276406474/" > /dev/null 2>&1'"'"' && sleep 0'
ok: [localhost]

TASK [service] *****************************************************************
task path: /vagrant/playbook.yml:5
Running service
Using module file /usr/lib64/python3.4/site-packages/ansible/modules/core/system/service.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: vagrant
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1485728480.7279906-46765693117684 `" && echo ansible-tmp-1485728480.7279906-46765693117684="` echo ~/.ansible/tmp/ansible-tmp-1485728480.7279906-46765693117684 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmphqtgchb0 TO /home/vagrant/.ansible/tmp/ansible-tmp-1485728480.7279906-46765693117684/service.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/vagrant/.ansible/tmp/ansible-tmp-1485728480.7279906-46765693117684/ /home/vagrant/.ansible/tmp/ansible-tmp-1485728480.7279906-46765693117684/service.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-wmmnestsglioaloffnejbxcdunpmikze; /usr/bin/python3.4 /home/vagrant/.ansible/tmp/ansible-tmp-1485728480.7279906-46765693117684/service.py; rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1485728480.7279906-46765693117684/" > /dev/null 2>&1'"'"' && sleep 0'
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_ckjkjhx8/ansible_module_service.py", line 1552, in <module>
    main()
  File "/tmp/ansible_ckjkjhx8/ansible_module_service.py", line 1508, in main
    service.get_service_status()
  File "/tmp/ansible_ckjkjhx8/ansible_module_service.py", line 575, in get_service_status
    rc, status_stdout, status_stderr = self.service_control()
  File "/tmp/ansible_ckjkjhx8/ansible_module_service.py", line 899, in service_control
    rc_state, stdout, stderr = self.execute_command("%s %s %s" % (svc_cmd, self.action, arguments), daemonize=True)
  File "/tmp/ansible_ckjkjhx8/ansible_module_service.py", line 262, in execute_command
    return json.loads(data)
  File "/usr/lib64/python3.4/json/__init__.py", line 318, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.4/json/decoder.py", line 343, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.4/json/decoder.py", line 361, in raw_decode
    raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)

fatal: [localhost]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "name": "net.eth0",
            "state": "started"
        },
        "module_name": "service"
    },
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_ckjkjhx8/ansible_module_service.py\", line 1552, in <module>\n    main()\n  File \"/tmp/ansible_ckjkjhx8/ansible_module_service.py\", line 1508, in main\n    service.get_service_status()\n  File \"/tmp/ansible_ckjkjhx8/ansible_module_service.py\", line 575, in get_service_status\n    rc, status_stdout, status_stderr = self.service_control()\n  File \"/tmp/ansible_ckjkjhx8/ansible_module_service.py\", line 899, in service_control\n    rc_state, stdout, stderr = self.execute_command(\"%s %s %s\" % (svc_cmd, self.action, arguments), daemonize=True)\n  File \"/tmp/ansible_ckjkjhx8/ansible_module_service.py\", line 262, in execute_command\n    return json.loads(data)\n  File \"/usr/lib64/python3.4/json/__init__.py\", line 318, in loads\n    return _default_decoder.decode(s)\n  File \"/usr/lib64/python3.4/json/decoder.py\", line 343, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/usr/lib64/python3.4/json/decoder.py\", line 361, in raw_decode\n    raise ValueError(errmsg(\"Expecting value\", s, err.value)) from None\nValueError: Expecting value: line 1 column 1 (char 0)\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE"
}
        to retry, use: --limit @/vagrant/playbook.retry

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1   
```



Discussion:
This looks like a python3 issue, Ansible 2.2 might not really support this, try using with python2 on the target.
I think I know what's going wrong on python3:

* The exec_command() code forks a child to daemonize and return the commands output
* This child is supposed to return the output as a json string but on python3, it is backtracing because the code combines a text string (stdout="") with a byte string (the output from subprocess).
* The parent then takes the information from the child and attempts to json.loads() it.
* The parent then backtraces because the data from the child isn't valid json.
* We see the traceback from the parent.

If my guess is correct, I can fix the sequence of events that starts this by making sure the child does not combine text and bytes.  Processing the output of the child could still be made more robust, though.

Resolving patch:
diff --git a/lib/ansible/modules/system/service.py b/lib/ansible/modules/system/service.py
--- a/lib/ansible/modules/system/service.py
+++ b/lib/ansible/modules/system/service.py
@@ -128,16 +128,21 @@
 
 '''
 
-import platform
+import glob
 import os
+import platform
 import re
-import tempfile
-import shlex
 import select
-import time
+import shlex
 import string
-import glob
-from ansible.module_utils.service import fail_if_missing
+import subprocess
+import tempfile
+import time
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
 
 # The distutils module is not shipped with SUNWPython on Solaris.
 # It's in the SUNWPython-devel package which also contains development files
@@ -146,6 +151,12 @@
 if platform.system() != 'SunOS':
     from distutils.version import LooseVersion
 
+from ansible.module_utils.basic import AnsibleModule, load_platform_subclass
+from ansible.module_utils.service import fail_if_missing
+from ansible.module_utils.six import PY2, b
+from ansible.module_utils._text import to_bytes, to_text
+
+
 class Service(object):
     """
     This is the generic Service manipulation class that is subclassed
@@ -242,11 +253,19 @@ def execute_command(self, cmd, daemonize=False):
                 os._exit(0)
 
             # Start the command
-            if isinstance(cmd, basestring):
+            if PY2:
+                # Python 2.6's shlex.split can't handle text strings correctly
+                cmd = to_bytes(cmd, errors='surrogate_or_strict')
                 cmd = shlex.split(cmd)
+            else:
+                # Python3.x shex.split text strings.
+                cmd = to_text(cmd, errors='surrogate_or_strict')
+                cmd = [to_bytes(c, errors='surrogate_or_strict') for c in shlex.split(cmd)]
+            # In either of the above cases, pass a list of byte strings to Popen
+
             p = subprocess.Popen(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, preexec_fn=lambda: os.close(pipe[1]))
-            stdout = ""
-            stderr = ""
+            stdout = b("")
+            stderr = b("")
             fds = [p.stdout, p.stderr]
             # Wait for all output, or until the main process is dead and its output is done.
             while fds:
@@ -265,7 +284,8 @@ def execute_command(self, cmd, daemonize=False):
                     stderr += dat
             p.wait()
             # Return a JSON blob to parent
-            os.write(pipe[1], json.dumps([p.returncode, stdout, stderr]))
+            blob = json.dumps([p.returncode, to_text(stdout), to_text(stderr)])
+            os.write(pipe[1], to_bytes(blob, errors='surrogate_or_strict'))
             os.close(pipe[1])
             os._exit(0)
         elif pid == -1:
@@ -274,7 +294,7 @@ def execute_command(self, cmd, daemonize=False):
             os.close(pipe[1])
             os.waitpid(pid, 0)
             # Wait for data from daemon process and process it.
-            data = ""
+            data = b("")
             while True:
                 rfd, wfd, efd = select.select([pipe[0]], [], [pipe[0]])
                 if pipe[0] in rfd:
@@ -282,7 +302,7 @@ def execute_command(self, cmd, daemonize=False):
                     if not dat:
                         break
                     data += dat
-            return json.loads(data)
+            return json.loads(to_text(data, errors='surrogate_or_strict'))
 
     def check_ps(self):
         # Set ps flags
@@ -1583,7 +1603,5 @@ def main():
 
     module.exit_json(**result)
 
-from ansible.module_utils.basic import *
-
 if __name__ == '__main__':
     main()


Test patch: