Repository: dmwm/wmcore
Issue: [py2py3] WMAgent Py3 AgentStatusWatcher failing to calculate disk usage

**Impact of the bug**
PY3 WMAgent

**Describe the bug**
Testing wmagentpy3 1.5.0.pre3 (plus some patches), we manage to run services + components, but there are exceptions being recorded in the AgentStatusWatcher logs, usual bytes vs text string issues. See traceback in [1]

**How to reproduce it**
See above

**Expected behavior**
This TypeError needs to be fixed - compatible for both python versions - and the AgentStatusWatcher component should be running without any issues.

**Additional context and error message**
[1]
```
2021-06-28 15:06:27,200:140063939860224:INFO:DrainStatusPoller:Agent not in drain mode. Resetting flags and skipping drain check...
2021-06-28 15:06:27,200:140063939860224:INFO:BaseWorkerThread:DrainStatusPoller took 0.048 secs to execute
2021-06-28 15:06:27,517:140063948252928:INFO:Grafana:Calling API: https://monit-grafana.cern.ch/api/datasources/proxy/9475/_msearch with query: ('{"search_type":"query_then_fetch","ignore_unavailable":true,"index":["monit_prod_cmssst_*"]}\\n'
 '{"size":500,"query":{"bool":{"filter":[{"range":{"metadata.timestamp":{"gte":"now-1d","lte":"now","format":"epoch_millis"}}},{"query_string":{"analyze_wildcard":true,"query":"metadata.type: '
 'ssbmetric AND metadata.type_prefix:raw AND metadata.path: '
 'sts15min"}}]}},"sort":{"metadata.timestamp":{"order":"desc","unmapped_type":"boolean"}},"script_fields":{},"docvalue_fields":["metadata.timestamp"]}\\n')
2021-06-28 15:06:27,571:140063956645632:ERROR:AgentStatusPoller:Error occurred, will retry later.
Details: a bytes-like object is required, not 'str'
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/AgentStatusWatcher/AgentStatusPoller.py", line 139, in algorithm
    agentInfo = self.collectAgentInfo()
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/AgentStatusWatcher/AgentStatusPoller.py", line 213, in collectAgentInfo
    agentInfo['disk_warning'] = listDiskUsageOverThreshold(self.config, updateDB=True)
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMCore/Services/ReqMgrAux/ReqMgrAux.py", line 404, in listDiskUsageOverThreshold
    diskUseList = diskUse()
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/Utils/Utilities.py", line 96, in diskUse
    output = df.communicate()[0].split("\\n")
TypeError: a bytes-like object is required, not 'str'
2021-06-28 15:06:27,573:140063956645632:INFO:BaseWorkerThread:AgentStatusPoller took 0.822 secs to execute
```


Changed files:
test/python/Utils_t/Utilities_t.py, src/python/Utils/Utilities.py

diff --git a/src/python/Utils/Utilities.py b/src/python/Utils/Utilities.py
index 8f908bffc..c2cf85c8d 100644
--- a/src/python/Utils/Utilities.py
+++ b/src/python/Utils/Utilities.py
@@ -93,7 +93,8 @@ def diskUse():
     """
     diskPercent = []
     df = subprocess.Popen(["df", "-klP"], stdout=subprocess.PIPE)
-    output = df.communicate()[0].split("\\n")
+    output = df.communicate()[0]
+    output = decodeBytesToUnicode(output).split("\\n")
     for x in output:
         split = x.split()
         if split != [] and split[0] != 'Filesystem':
@@ -107,7 +108,8 @@ def numberCouchProcess():
     This returns the number of couch process
     """
     ps = subprocess.Popen(["ps", "-ef"], stdout=subprocess.PIPE)
-    process = ps.communicate()[0].count('couchjs')
+    process = ps.communicate()[0]
+    process = decodeBytesToUnicode(process).count('couchjs')
 
     return process
 
diff --git a/test/python/Utils_t/Utilities_t.py b/test/python/Utils_t/Utilities_t.py
index af37fbd07..309886ab0 100644
--- a/test/python/Utils_t/Utilities_t.py
+++ b/test/python/Utils_t/Utilities_t.py
@@ -11,7 +11,7 @@ import unittest
 
 from Utils.Utilities import makeList, makeNonEmptyList, strToBool, \\
     safeStr, rootUrlJoin, zipEncodeStr, lowerCmsHeaders, getSize, \\
-    encodeUnicodeToBytes
+    encodeUnicodeToBytes, diskUse, numberCouchProcess
 
 
 class UtilitiesTests(unittest.TestCase):
@@ -167,5 +167,22 @@ cms::Exception caught in CMS.EventProcessor and rethrown
         print(getSize(cls2)) # py2: 426, py3: 205
         self.assertTrue(getSize(cls2) > 200)
 
+    def testDiskUse(self):
+        """
+        Test the `diskUse` function.
+        """
+        data = diskUse()
+        # assuming nodes will always have at least 3 partitions/mount points
+        self.assertTrue(len(data) > 2)
+
+    def testNumberCouchProcess(self):
+        """
+        Test the `numberCouchProcess` function.
+        """
+        data = numberCouchProcess()
+        # there should be at least one process, but who knows...
+        self.assertTrue(data >= 0)
+
+
 if __name__ == '__main__':
     unittest.main()