Repository: dmwm/wmcore
Issue: [py2py3] JobCreatorPoller crashing when dumping wmbsJobGroup jobs

**Impact of the bug**
Python3 WMAgent

**Describe the bug**
As we keep testing Python3 WMAgent 1.5.0.pre3, another component error-related popped up, this time with JobCreator.
Again problems with text vs byte strings, see component log snippet in [1]

**How to reproduce it**
Let a Py3 WMAgent node acquire any workflow

**Expected behavior**
Jobs should be properly created in the component, in both python2 and python3.

**Additional context and error message**
[1]
```
2021-06-28 19:52:35,939:139920120297216:INFO:EventBased:Job created with mask: {'inclusivemask': True, 'FirstEvent': 8980, 'LastEvent': 9000, 'FirstLumi': 220, 'LastLumi': 220, 'FirstRun': 1, 'LastRun': 1, 'runAndLumis': {}}
2021-06-28 19:52:37,263:139920120297216:INFO:JobCreatorPoller:Retrieved 1 jobGroups from jobSplitter
2021-06-28 19:52:37,283:139920120297216:ERROR:JobCreatorPoller:Exception in processing wmbsJobGroup 2
. Error: write() argument must be str, not bytes
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/JobCreator/JobCreatorPoller.py", line 215, in creatorProcess
    saveJob(job=job, workflow=workflow,
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/JobCreator/JobCreatorPoller.py", line 159, in saveJob
    pickle.dump(job, output, pickle.HIGHEST_PROTOCOL)
TypeError: write() argument must be str, not bytes
2021-06-28 19:52:37,731:139920120297216:ERROR:BaseWorkerThread:Error in worker algorithm (1):
Backtrace:
  <WMComponent.JobCreator.JobCreatorPoller.JobCreatorPoller object at 0x7f41b404d8e0> <@========== WMException Start ==========@>
Exception Class: JobCreatorException
Message: Exception in processing wmbsJobGroup 2
. Error: write() argument must be str, not bytes
        ClassName : None
        ModuleName : WMComponent.JobCreator.JobCreatorPoller
        MethodName : creatorProcess
        ClassInstance : None
        FileName : /data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/JobCreator/JobCreatorPoller.py
        LineNumber : 236
        ErrorNr : 0

Traceback: 
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/JobCreator/JobCreatorPoller.py", line 215, in creatorProcess
    saveJob(job=job, workflow=workflow,

  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/JobCreator/JobCreatorPoller.py", line 159, in saveJob
    pickle.dump(job, output, pickle.HIGHEST_PROTOCOL)

<@---------- WMException End ----------@>  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMCore/WorkerThreads/BaseWorkerThread.py", line 183, in __call__
    tSpent, results, _ = algorithmWithDBExceptionHandler(parameters)
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMCore/Database/DBExceptionHandler.py", line 40, in wrapper
    return f(*args, **kwargs)
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/Utils/Timers.py", line 26, in wrapper
    res = func(*arg, **kw)
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/JobCreator/JobCreatorPoller.py", line 418, in algorithm
    self.pollSubscriptions()
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/JobCreator/JobCreatorPoller.py", line 613, in pollSubscriptions
    jobGroup = creatorProcess(work=tempDict,
  File "/data/srv/wmagent/v1.5.0.pre3/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.0.pre3/lib/python3.8/site-packages/WMComponent/JobCreator/JobCreatorPoller.py", line 236, in creatorProcess
    raise JobCreatorException(msg)

2021-06-28 19:52:37,731:139920120297216:INFO:Harness:>>>Terminating worker threads
```

Changed files:
src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py, src/python/WMComponent/JobCreator/JobCreatorPoller.py

diff --git a/src/python/WMComponent/JobCreator/JobCreatorPoller.py b/src/python/WMComponent/JobCreator/JobCreatorPoller.py
index 145faf5c9..61895783d 100644
--- a/src/python/WMComponent/JobCreator/JobCreatorPoller.py
+++ b/src/python/WMComponent/JobCreator/JobCreatorPoller.py
@@ -155,7 +155,7 @@ def saveJob(job, workflow, sandbox, wmTask=None, jobNumber=0,
     job['inputPileup'] = inputPileup
     job['allowOpportunistic'] = allowOpportunistic
 
-    with open(os.path.join(cacheDir, 'job.pkl'), 'w') as output:
+    with open(os.path.join(cacheDir, 'job.pkl'), 'wb') as output:
         pickle.dump(job, output, pickle.HIGHEST_PROTOCOL)
 
     return
diff --git a/src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py b/src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py
index f6d42fde4..001bf055d 100644
--- a/src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py
+++ b/src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py
@@ -315,7 +315,7 @@ class JobSubmitterPoller(BaseWorkerThread):
                 badJobs[71104].append(newJob)
                 continue
             try:
-                with open(pickledJobPath, 'r') as jobHandle:
+                with open(pickledJobPath, 'rb') as jobHandle:
                     loadedJob = pickle.load(jobHandle)
             except Exception as ex:
                 logging.warning("Failed to load job pickle object %s", pickledJobPath)