Repository: dmwm/wmcore
Issue: pycurl_manager fails with TypeError in python3

**Impact of the bug**
Every class that inherits from `WMCore.Services.Services` and uses pycurl to make a request.  

**Describe the bug**
While trying to run the micro services under python3 environment I got stuck with the following error: [4]

After a deeper look and running the whole long chain of calls through `pdb` I managed to pin the culprit. This is a type mismatch between the header we get from the call of `curl.perform()` in `pycurl_manager`, which is a `byte` [1], and the expression we use for splitting it, which is a hardcoded string [2]. We have two options to proceed here:
   * casting the header we are trying to parse to a `str`
   * or making the split `str` a `byte` 
I already tried the former and it seems to work [3], but it requires a change of the type of a header returned by a regular library (such as pycurl) and I would consider it risky because I am not sure if it is not to be expected as a byte somewhere else down the code. Even though in my case it did work.  

[1]
```
> /afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py(72)parse()
-> def parse(self, response):
(Pdb) l
 67  	        self.header = {}
 68  	        self.reason = ''
 69  	        self.fromcache = False
 70  	        self.parse(response)
 71  	
 72  ->	    def parse(self, response):
 73  	        """Parse response header and assign class member data"""
 74  	        startRegex = r"^HTTP/\\d.\\d \\d{3}"
 75  	        continueRegex = r"^HTTP/\\d.\\d 100"  # Continue: client should continue its request
 76  	        replaceRegex = r"^HTTP/\\d.\\d"
 77  	
(Pdb) s
> /afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py(74)parse()
-> startRegex = r"^HTTP/\\d.\\d \\d{3}"
(Pdb) response
b'HTTP/1.1 200 OK\\r\\nDate: Fri, 05 Mar 2021 21:12:25 GMT\\r\\nServer: CherryPy/17.4.0\\r\\nContent-Length: 18\\r\\nContent-Encoding: deflate\\r\\nX-Rest-Time: 22814.035 us\\r\\nX-Rest-Status: 100\\r\\nVary: Accept,Accept-Encoding\\r\\nEtag: "9c66ca948c2c34405e231689f5286bb9b0cc7c20"\\r\\nCache-Control: max-age=3600\\r\\nContent-Type: application/json\\r\\nCMS-Server-Time: D=39530 t=1614978745314628\\r\\n\\r\\n'
```

[2]
```
> /afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py:
 78        for row in response.split('\\r'):  
```

[3]
```
(Pdb) response=str(response)
(Pdb) s
> /afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py(79)parse()
-> row = row.replace('\\n', '')
(Pdb) 
> /afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py(80)parse()
-> if not row:
...
```


[4]
```
2021-03-05 19:43:15,206:ERROR:MSRuleCleaner:execute(): ('Unknown exception while fetching requests from ReqMgr2. Error: %s', "a bytes-like object is required, not 'str'")
Traceback (most recent call last):
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/MicroService/Unified/MSRuleCleaner.py", line 207, in execute
    requestRecords.update(self.getRequestRecords(status))
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/MicroService/Unified/MSRuleCleaner.py", line 691, in getRequestRecords
    result = self.reqmgr2.getRequestByStatus([reqStatus], detail=True)
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/ReqMgr/ReqMgr.py", line 131, in getRequestByStatus
    return self._getResult(callname, verb="GET")
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/ReqMgr/ReqMgr.py", line 50, in _getResult
    verb=verb, contentType=contentType)
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/Service.py", line 209, in refreshCache
    self.getData(cachefile, url, inputdata, incoming_headers, encoder, decoder, verb, contentType)
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/Service.py", line 288, in getData
    contentType=contentType)
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/Requests.py", line 159, in makeRequest
    result, response = self.makeRequest_pycurl(uri, data, verb, headers)
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/Requests.py", line 177, in makeRequest_pycurl
    ckey=ckey, cert=cert, capath=capath)
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py", line 282, in request
    header = self.parse_header(hbuf.getvalue())
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py", line 270, in parse_header
    return ResponseHeader(header)
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py", line 70, in __init__
    self.parse(response)
  File "/afs/cern.ch/work/t/tivanov/private/WMCoreDev.d/WMCore/src/python/WMCore/Services/pycurl_manager.py", line 78, in parse
    for row in response.split('\\r'):
TypeError: a bytes-like object is required, not 'str'
```
 
**How to reproduce it**
Try to run anything which calls `WMCore.Services.pycurl_manager` under python3 environment.

**Expected behavior**
I expect the have a header and a parsing string of the same type (regardless `str` or `byte`) so that  any call to `pycurl_manager` should succeed.

**Additional context and error message**
Add any other context about the problem here, like error message and/or traceback. You might want to use triple back ticks to properly format it.


Changed files:
src/python/WMCore/Services/pycurl_manager.py

diff --git a/src/python/WMCore/Services/pycurl_manager.py b/src/python/WMCore/Services/pycurl_manager.py
index 207c9a406..8f68941b1 100644
--- a/src/python/WMCore/Services/pycurl_manager.py
+++ b/src/python/WMCore/Services/pycurl_manager.py
@@ -57,7 +57,7 @@ from io import BytesIO
 import http.client
 from urllib.parse import urlencode
 
-from Utils.Utilities import encodeUnicodeToBytes
+from Utils.Utilities import encodeUnicodeToBytes, decodeBytesToUnicode
 from Utils.PortForward import portForward, PortForward
 
 
@@ -77,6 +77,8 @@ class ResponseHeader(object):
         continueRegex = r"^HTTP/\\d.\\d 100"  # Continue: client should continue its request
         replaceRegex = r"^HTTP/\\d.\\d"
 
+        response = decodeBytesToUnicode(response)
+
         for row in response.split('\\r'):
             row = row.replace('\\n', '')
             if not row: