Repository: dmwm/wmcore
Issue: [py2py3] investigate WMCore/WebTools/RESTFormatter.to_string() return type in py3

**Impact of the new feature**

It should affect only the internals cherrypy when running WMCore from a py3 environment, [this function](https://github.com/cherrypy/cherrypy/blob/c524a0da73a85ec4dd4ae6b1512c25854b19c8de/cherrypy/_cprequest.py#L775) in particular.

**Is your feature request related to a problem? Please describe.**

In #10117  we modernized ` src/python/WMCore/WebTools/RESTFormatter.py ` as follows

```python
+ from builtins import str, bytes
    def to_string(self, data):
        if isinstance(data, GeneratorType):
            return self.json(data)
        if isinstance(data, dict) or isinstance(data, list):
            return json.dumps(data)
-        return str(data)
+        return bytes(str(data), "utf-8") #  equivalent to `str(data).encode("utf-8")`
```

as discussed in [this comment in PR 10117](https://github.com/dmwm/WMCore/pull/10117#discussion_r555779338). This is based on the assumption that the internals of cherrypy use bytes string in py3 also, which is only guessed by looking at [cherrypy py3 code](https://github.com/cherrypy/cherrypy/blob/c524a0da73a85ec4dd4ae6b1512c25854b19c8de/cherrypy/_cprequest.py#L775), this has not been tested yet.

**Describe the solution you'd like**

We should test this assumption in dev, testbed and/or pre-production.

**Describe alternatives you've considered**

If cherrypy requires `RESTFormatter.to_string() ` to return a bytes string in py3 also, then we are good to go. If it requires unicode string in py3, then we need to change our code.



Changed files:
src/python/WMCore/WebTools/RESTFormatter.py

diff --git a/src/python/WMCore/WebTools/RESTFormatter.py b/src/python/WMCore/WebTools/RESTFormatter.py
index 2eba89fad..aae571ee7 100644
--- a/src/python/WMCore/WebTools/RESTFormatter.py
+++ b/src/python/WMCore/WebTools/RESTFormatter.py
@@ -8,7 +8,7 @@ appropriate format and sets the CherryPy header appropriately.
 Could add YAML via http://pyyaml.org/
 """
 
-from builtins import str, bytes
+from builtins import str
 
 import json
 from types import GeneratorType
@@ -18,6 +18,9 @@ from cherrypy import response, HTTPError, request
 from WMCore.WebTools.Page import TemplatedPage, _setCherryPyHeaders
 from WMCore.Wrappers.JsonWrapper.JSONThunker import JSONThunker
 
+from Utils.Utilities import decodeBytesToUnicodeConditional
+from Utils.PythonVersion import PY3
+
 class RESTFormatter(TemplatedPage):
     def __init__(self, config):
         self.supporttypes = {
@@ -66,16 +69,21 @@ class RESTFormatter(TemplatedPage):
                                 path = request.path_info)
 
     def to_string(self, data):
+        """
+        Since json.dumps returns unicode in py3 and bytes in py2 (it behaves
+        returning "native str" in both versions), then we do the same with all
+        the data that is not GeneratorType, dict nor list by calling str(data)
+        Moreover, we need to properly decode bytes.
+
+        :returns: "native str" (unicode in py3, bytes in py2)
+        """
         if isinstance(data, GeneratorType):
             return self.json(data)
         if isinstance(data, dict) or isinstance(data, list):
             return json.dumps(data)
-        # Do not be misguided by the function name, this must return bytes, 
-        # otherwise the unit test
-        # testSupportedFormat (WMCore_t.WebTools_t.RESTFormat_t.RESTFormatTest)
-        # fails with "ValueError: Page handlers MUST return bytes. 
-        # Use tools.encode if you wish to return unicode."
-        return bytes(str(data), "utf-8")
+        if not isinstance(data, bytes):
+            return str(data)
+        return decodeBytesToUnicodeConditional(data, condition=PY3)
 
     def format(self, data, datatype, expires):
         response_data = ''