anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: dmwm/wmcore
Issue: Missing validation campaign configurations for 2021

**Impact of the bug**
Validation

**Describe the bug**
We have the test/validation campaign names hardcoded in the `parseUnifiedCampaigns.py` script and the those for the current year are missing.

**How to reproduce it**
Any injection with a campaign name such as `HG_21**_Val` would fail to move workflows beyond the `assigned` status.  

**Expected behavior**
Have the validation campaign identifiers for the whole year at least

**Additional context and error message**
Add any other context about the problem here, like error message and/or traceback. You might want to use triple back ticks to properly format it.


Changed files:
bin/adhoc-scripts/parseUnifiedCampaigns.py

diff --git a/bin/adhoc-scripts/parseUnifiedCampaigns.py b/bin/adhoc-scripts/parseUnifiedCampaigns.py
index 31c6cad85..10cb5eac6 100644
--- a/bin/adhoc-scripts/parseUnifiedCampaigns.py
+++ b/bin/adhoc-scripts/parseUnifiedCampaigns.py
@@ -286,9 +286,13 @@ def insertTestCampaigns(mgr):
 
     testCamp = ("CMSSW_10_6_1_Step3", "CMSSW_10_6_1_patch1_Step1",  "CMSSW_10_6_1_patch1_Step2",
                 "CMSSW_9_4_0__test2inwf-1510737328", "CMSSW_7_3_2__test2inwf-1510737328",
-                "Sept2020_Val", "Oct2020_Val", "Nov2020_Val", "Dec2020_Val",
                 "Agent140_Val", "Agent142_Val", "Agent144_Val",
-                "HG2009_Val", "HG2010_Val", "HG2011_Val", "HG2012_Val")
+                "Jan2021_Val", "Feb2021_Val", "Mar2021_Val", "Apr2021_Val",
+                "May2021_Val", "Jun2021_Val", "Jul2021_Val", "Aug2021_Val",
+                "Sept2021_Val", "Oct2021_Val", "Nov2021_Val", "Dec2021_Val",
+                "HG2101_Val", "HG2102_Val", "HG2103_Val", "HG2104_Val",
+                "HG2105_Val", "HG2106_Val", "HG2107_Val", "HG2108_Val",
+                "HG2109_Val", "HG2110_Val", "HG2111_Val", "HG2112_Val")
     for campName in testCamp:
         defaultCamp['CampaignName'] = campName
         upload(mgr, defaultCamp)
Repository: dmwm/wmcore
Issue: Pileup with multi-RSE rules can have a wrong block location

**Impact of the bug**
WMAgent

**Describe the bug**
This bug has been exposed while I was trying to hack one of the agents to let the P&R team to run a backfill workflow with container only partially available.
Problem affects WMAgent, through PileupFetcher module, when looking for pileup availability and locks and producing the usual pileupconf.json file. And the origin of the problem is at this line:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Services/Rucio/Rucio.py#L1039

which assigns `finalRSEs` to result by reference. Which means, as soon as we add any other RSE to the `result` variable, it gets reflect to all the other blocks that point to that memory address.

**How to reproduce it**
Resolve pileup location for a container having a rule locking the data at multiple RSEs.

**Expected behavior**
Block location should be properly built, considering only the container-level rules and dataset_locks matching those rules.
Thus, this block needs fixing:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Services/Rucio/Rucio.py#L1046-L1051

**Additional context and error message**
none

Changed files:
src/python/WMCore/Services/Rucio/Rucio.py

diff --git a/src/python/WMCore/Services/Rucio/Rucio.py b/src/python/WMCore/Services/Rucio/Rucio.py
index 1d9238a39..e0b71bfab 100644
--- a/src/python/WMCore/Services/Rucio/Rucio.py
+++ b/src/python/WMCore/Services/Rucio/Rucio.py
@@ -1044,11 +1044,13 @@ class Rucio(object):
         # if we got here, then there is a third step to be done.
         # List every single block lock and check if the rule belongs to the WMCore system
         for blockName in result:
+            blockRSEs = set()
             for blockLock in self.cli.get_dataset_locks(scope, blockName):
                 if isTapeRSE(blockLock['rse']):
                     continue
                 if blockLock['state'] == 'OK' and blockLock['rule_id'] in multiRSERules:
-                    result[blockName].add(blockLock['rse'])
+                    blockRSEs.add(blockLock['rse'])
+            result[blockName] = finalRSEs | blockRSEs
         return result
 
     def getParentContainerRules(self, **kwargs):
Repository: dmwm/wmcore
Issue: Global WorkQueue using the wrong 'grouping' for pileup data location

**Impact of the bug**
Global WorkQueue

**Describe the bug**
While testing workflows in my VM, I noticed that global workqueue was creating elements which had an empty list of RSEs for the pileup data, even though the container rule had recently being satisfied.
Upon global workqueue restart, the LocationUpdateTask cherrypy thread gets executed and those elements get updated with the correct pileup location.

Both threads use the same method `getDataLockedAndAvailable`, but they provide a different set of arguments. For the workqueue, here is where the pileup gets resolved:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py#L255

while the LocationUpdateTask thread calls this:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WorkQueue/DataLocationMapper.py#L99

**How to reproduce it**
inject any workflow that requires pileup data.

**Expected behavior**
Pileup data location has to work during the workqueue element creation. Which means, the `grouping="DATASET"` key/value pair has to be removed (this internally uses the containe ruler RSEs for rules in the OK state).

**Additional context and error message**
There are no error messages, just the wrong information in the GQ element.

Changed files:
src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py

diff --git a/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py b/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py
index ea40075f1..4b6b89a1a 100644
--- a/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py
+++ b/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py
@@ -252,7 +252,7 @@ class StartPolicyInterface(PolicyInterface):
         for dbsUrl in datasets:
             for datasetPath in datasets[dbsUrl]:
                 if hasattr(self, "rucio"):
-                    locations = self.rucio.getDataLockedAndAvailable(name=datasetPath, grouping="DATASET",
+                    locations = self.rucio.getDataLockedAndAvailable(name=datasetPath,
                                                                      account=self.args['rucioAcct'])
                 else:
                     locations = set()
Repository: dmwm/wmcore
Issue: MSMonitor fails to calculate rule progress: division by zero

**Impact of the bug**
ReqMgr2MS - MSMonitor

**Describe the bug**
There is one specific workflow which we hit this exception:
```
2020-09-29 23:54:53,133:ERROR:MSMonitor: Unknown exception checking workflow amaltaro_TaskChain_Prod_Sept2020_Val_200929_232610_2012. Error: division by zero
```

apparently the locks_* parameters are all 0. It's likely related to the current state of the rule `state=INJECT`, because this is a very large container (500TB).

**How to reproduce it**
Make a container rule for a very large container (perhaps > 10k blocks)

**Expected behavior**
If we cannot calculate the rule progress, then we should set its progress to 0%.
There might be room for improvement on the error message as well.

**Additional context and error message**
Error from the service log:
```
2020-09-29 23:44:52,078:DEBUG:MSMonitor: Rule ID: 4a2f8d276dbc40a98f2ea2a3582f2076, DID: /Cosmics/Commissioning2015-v1/RAW#487c3a70-b47d-11e4-8732-02163e00d625, state: OK, grouping: DATASET, rse_expression: T1_US_FNAL_Disk
2020-09-29 23:44:52,078:DEBUG:MSMonitor: Checking transfers for: amaltaro_TaskChain_Prod_Sept2020_Val_200929_232610_2012
2020-09-29 23:44:52,098:ERROR:MSMonitor: Unknown exception checking workflow amaltaro_TaskChain_Prod_Sept2020_Val_200929_232610_2012. Error: division by zero
```

Changed files:
src/python/WMCore/MicroService/Unified/MSMonitor.py

diff --git a/src/python/WMCore/MicroService/Unified/MSMonitor.py b/src/python/WMCore/MicroService/Unified/MSMonitor.py
index 2fc4cccaa..1290becd0 100644
--- a/src/python/WMCore/MicroService/Unified/MSMonitor.py
+++ b/src/python/WMCore/MicroService/Unified/MSMonitor.py
@@ -165,7 +165,7 @@ class MSMonitor(MSCore):
                 doc['lastUpdate'] = tstamp
             except Exception as exc:
                 msg = "Unknown exception checking workflow %s. Error: %s"
-                self.logger.error(msg, doc['workflowName'], str(exc))
+                self.logger.exception(msg, doc['workflowName'], str(exc))
                 skippedWorkflows.append(doc['workflowName'])
         return skippedWorkflows
 
@@ -260,7 +260,11 @@ class MSMonitor(MSCore):
                 lockCompletion = 100.0
             else:
                 totalLocks = data['locks_ok_cnt'] + data['locks_replicating_cnt'] + data['locks_stuck_cnt']
-                lockCompletion = (data['locks_ok_cnt'] / totalLocks) * 100
+                try:
+                    lockCompletion = (data['locks_ok_cnt'] / totalLocks) * 100
+                except ZeroDivisionError:
+                    self.logger.warning("Rule does not have any lock counts yet. Rule data: %s", data)
+                    lockCompletion = 0
             completion.append(lockCompletion)
             self.logger.info("Rule ID: %s has a completion rate of: %s%%", ruleID, lockCompletion)
             self.logger.debug("Rule ID: %s, DID: %s, state: %s, grouping: %s, rse_expression: %s",
Repository: dmwm/wmcore
Issue: LogCollect uses stageIn when it shouldn't

**Impact of the bug**
LogCollect uses stagein to retrieve logArchives from storage, possibly overloading storage

**Describe the bug**
LogCollect uses stageIn when it should use edmCopyUtil. The problem was that 2 years ago the cutoff release in this check

https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WMSpec/Steps/Executors/LogCollect.py#L103

was updated to a newer release and now the SL6 only check doesn't work anymore because we have releases older that are available as SL7. This includes some releases used in the UL Campaigns Gen steps.

Fix is easy, just add another conditional check for SL7 scram archs with the appropriate overrides.


Changed files:
src/python/WMCore/WMSpec/Steps/Executors/LogCollect.py

diff --git a/src/python/WMCore/WMSpec/Steps/Executors/LogCollect.py b/src/python/WMCore/WMSpec/Steps/Executors/LogCollect.py
index 9d7971791..05b9a4025 100644
--- a/src/python/WMCore/WMSpec/Steps/Executors/LogCollect.py
+++ b/src/python/WMCore/WMSpec/Steps/Executors/LogCollect.py
@@ -103,6 +103,11 @@ class LogCollect(Executor):
         useEdmCopyUtil = True
         if isCMSSWSupported(cmsswVersion, "CMSSW_10_4_0"):
             pass
+        elif scramArch.startswith('slc7_amd64_'):
+            msg = "CMSSW too old or not fully functional to support edmCopyUtil, using CMSSW_10_4_0 instead"
+            logging.warning(msg)
+            cmsswVersion = "CMSSW_10_4_0"
+            scramArch = "slc7_amd64_gcc820"
         elif scramArch.startswith('slc6_amd64_'):
             msg = "CMSSW too old or not fully functional to support edmCopyUtil, using CMSSW_10_4_0 instead"
             logging.warning(msg)
Repository: dmwm/wmcore
Issue: MSTransferor crashing to check the final pileup location - testbed

**Impact of the bug**
ReqMgr2MS - MSTransferor

**Describe the bug**
The service is failing the cycle because it's accessing an unbound variable. Traceback will be provided below.

**How to reproduce it**
none

**Expected behavior**
MSTransferor should be able to check the final location of pileup samples ;)

**Additional context and error message**
Traceback from vocms0731:
```
2020-09-12 02:29:11,629:INFO:MSTransferor: Finding final pileup destination for request: wmagent_SC_MultiPU_khurtado_newsubmit_SC_MultiPU_200909_180630_7065
2020-09-12 02:29:11,631:ERROR:MSManager: MS daemon error: local variable 'commonPsns' referenced before assignment
Traceback (most recent call last):
  File "/data/srv/beHG2009h/sw/slc7_amd64_gcc630/cms/reqmgr2ms/0.4.0.pre8/lib/python2.7/site-packages/WMCore/MicroService/Unified/MSManager.py", line 43, in daemon
    func(reqStatus)
  File "/data/srv/beHG2009h/sw/slc7_amd64_gcc630/cms/reqmgr2ms/0.4.0.pre8/lib/python2.7/site-packages/WMCore/MicroService/Unified/MSManager.py", line 141, in transferor
    res = self.msTransferor.execute(reqStatus)
  File "/data/srv/beHG2009h/sw/slc7_amd64_gcc630/cms/reqmgr2ms/0.4.0.pre8/lib/python2.7/site-packages/WMCore/MicroService/Unified/MSTransferor.py", line 211, in execute
    self.checkPUDataLocation(wflow)
  File "/data/srv/beHG2009h/sw/slc7_amd64_gcc630/cms/reqmgr2ms/0.4.0.pre8/lib/python2.7/site-packages/WMCore/MicroService/Unified/MSTransferor.py", line 515, in checkPUDataLocation
    pnns = self._findFinalPULocation(wflow)
  File "/data/srv/beHG2009h/sw/slc7_amd64_gcc630/cms/reqmgr2ms/0.4.0.pre8/lib/python2.7/site-packages/WMCore/MicroService/Unified/MSTransferor.py", line 561, in _findFinalPULocation
    if not commonPsns:
UnboundLocalError: local variable 'commonPsns' referenced before assignment
[12/Sep/2020:02:29:22] vocms0731.cern.ch 127.0.0.1 "GET /ms-transferor/data
```

Changed files:
src/python/WMCore/MicroService/Unified/MSTransferor.py

diff --git a/src/python/WMCore/MicroService/Unified/MSTransferor.py b/src/python/WMCore/MicroService/Unified/MSTransferor.py
index 4097ae663..d7005fa78 100644
--- a/src/python/WMCore/MicroService/Unified/MSTransferor.py
+++ b/src/python/WMCore/MicroService/Unified/MSTransferor.py
@@ -534,6 +534,7 @@ class MSTransferor(MSCore):
                 dsetName = dataIn["name"]
                 campConfig = self.campaigns[dataIn['campaign']]
 
+                commonPsns = set()
                 # if the dataset has a location list, use solely that one
                 if campConfig['Secondaries'].get(dsetName, []):
                     commonPsns = set(psns) & set(campConfig['Secondaries'][dsetName])
Repository: dmwm/wmcore
Issue: Requests stuck in staged state for weeks

**Impact of the bug**
Requests stuck in staged state for weeks

**Describe the bug**
There are 10 requests stuck in `staged` state for weeks: https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?stuck_state=staged&stuck_min=21 It might be an indication for a bug in work-queue

**How to reproduce it**
Check the request logs

**Expected behavior**
They should not get stuck in this state for days

**Additional context and error message**
None


Changed files:
src/python/WMCore/WorkQueue/Policy/Start/Block.py

diff --git a/src/python/WMCore/WorkQueue/Policy/Start/Block.py b/src/python/WMCore/WorkQueue/Policy/Start/Block.py
index 3f1a5b0e5..505bcf91e 100644
--- a/src/python/WMCore/WorkQueue/Policy/Start/Block.py
+++ b/src/python/WMCore/WorkQueue/Policy/Start/Block.py
@@ -223,7 +223,7 @@ class Block(StartPolicyInterface):
             return dict()
         # blocks with 0 files in Rucio should be ignored as well
         blockRucio = self.rucio.getDID(didName=blockName, dynamic=False)
-        if not blockRucio['length']:
+        if not blockRucio.get('length'):
             logging.warning("Block %s being rejected for lack of files in Rucio to process", blockName)
             self.badWork.append(blockName)
             return dict()
Repository: dmwm/wmcore
Issue: [py2py3] Syntax error when dropping MariaDB database

**Impact of the bug**
WMCore unit tests

**Describe the bug**
While commissioning the Python3 Jenkins CI, I noticed many unit tests failing with a syntax error when clearing the test SQL database, see [1].

**How to reproduce it**
Running this module unit tests:
WMCore_t/WMBS_t/JobSplitting_t/EventBased_t.py
with this option:
```
--reallyDeleteMyDatabaseAfterEveryTest=true
```

**Expected behavior**
Fix the SQL syntax such that we can run unit tests in BOT mode and cleaning up the database between tests.

**Additional context and error message**
[1]
```
======================================================================
ERROR: _test150EventMultipleFileSplit_
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/cmsbld/jenkins/workspace/DMWM-WMAgentPy3-TestAll/SLICE/2/label/cms-dmwm-cc7/code/test/python/WMCore_t/WMBS_t/JobSplitting_t/EventBased_t.py", line 43, in setUp
    self.testInit.setDatabaseConnection(destroyAllDatabase=True)
  File "/build/cmsbld/jenkins/workspace/DMWM-WMAgentPy3-TestAll/SLICE/2/label/cms-dmwm-cc7/code/src/python/WMQuality/TestInit.py", line 154, in setDatabaseConnection
    self.clearDatabase()
  File "/build/cmsbld/jenkins/workspace/DMWM-WMAgentPy3-TestAll/SLICE/2/label/cms-dmwm-cc7/code/src/python/WMQuality/TestInit.py", line 285, in clearDatabase
    self.init.clearDatabase()
  File "/build/cmsbld/jenkins/workspace/DMWM-WMAgentPy3-TestAll/SLICE/2/label/cms-dmwm-cc7/code/src/python/WMCore/WMInit.py", line 205, in clearDatabase
    raise WMInitException(msg)
WMCore.WMInit.WMInitException: <@========== WMException Start ==========@>
Exception Class: WMInitException
Message: Critical error while attempting to delete entire DB!
(MySQLdb._exceptions.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'b'WMCore_unit_test'' at line 1")
[SQL: DROP DATABASE b'WMCore_unit_test']
```

Changed files:
src/python/WMCore/Database/DBFormatter.py

diff --git a/src/python/WMCore/Database/DBFormatter.py b/src/python/WMCore/Database/DBFormatter.py
index 4175ec6c1..b02c8c3e8 100644
--- a/src/python/WMCore/Database/DBFormatter.py
+++ b/src/python/WMCore/Database/DBFormatter.py
@@ -11,6 +11,7 @@ from builtins import str, bytes, zip, range
 import datetime
 import time
 import types
+import sys
 
 from WMCore.DataStructs.WMObject import WMObject
 
@@ -77,7 +78,7 @@ class DBFormatter(WMObject):
                 entry = {}
                 for index in range(0, len(descriptions)):
                     # WARNING: Oracle returns table names in CAP!
-                    if isinstance(i[index], str):
+                    if isinstance(i[index], str) and sys.version_info[0] == 2:
                         entry[str(descriptions[index].lower())] = i[index].encode("utf-8")
                     else:
                         entry[str(descriptions[index].lower())] = i[index]
Repository: dmwm/wmcore
Issue: Email alerts coming without subject (and alert name too long)

**Impact of the bug**
Any system using the new AlertManager module (only MicroServices so far)

**Describe the bug**
Over the weekend I received 3 MSTransferor email alerts with no subject `(no subject)`. We should adopt the correct alert arguments such that alert emails are sent with a meaningful and short subject.

**How to reproduce it**
I guess it's just a matter of creating an alert in AlertManager.

**Expected behavior**
There are a couple of things to be fixed in this issue:
* email alerts must contain a short and meaningful subject. Something like "MSTransferor: input transfer over threshold for workflow ABC..."
* the set of rule ids should likely not be part of the alert name (otherwise it's very hard to see them in Slack)

**Additional context and error message**
Follow up of: https://github.com/dmwm/WMCore/pull/10308
Screenshot (cropped): http://amaltaro.web.cern.ch/amaltaro/forWMCore/Issue_10468/wrong_alert_email.png

Changed files:
src/python/WMCore/MicroService/MSTransferor/MSTransferor.py

diff --git a/src/python/WMCore/MicroService/MSTransferor/MSTransferor.py b/src/python/WMCore/MicroService/MSTransferor/MSTransferor.py
index 3446205bf..1036cd2bf 100644
--- a/src/python/WMCore/MicroService/MSTransferor/MSTransferor.py
+++ b/src/python/WMCore/MicroService/MSTransferor/MSTransferor.py
@@ -746,18 +746,21 @@ class MSTransferor(MSCore):
         """
         # Warn about data transfer subscriptions going above some threshold
         if aboveWarningThreshold:
-            alertName = "ms-transferor: Transfer over threshold: {}".format(transferId)
+            alertName = "{}: input data transfer over threshold: {}".format(self.alertServiceName,
+                                                                            wflowName)
             alertSeverity = "high"
             alertSummary = "[MS] Large pending data transfer under request id: {}".format(transferId)
-            alertDescription = "Workflow: {}\\nhas a large amount of ".format(wflowName)
-            alertDescription += "data subscribed: {} TB,\\n".format(teraBytes(dataSize))
+            alertDescription = "Workflow: {} has a large amount of ".format(wflowName)
+            alertDescription += "data subscribed: {} TB, ".format(teraBytes(dataSize))
             alertDescription += "for {} data: {}.""".format(dataIn['type'], dataIn['name'])
 
             try:
-                self.alertManagerApi.sendAlert(alertName, alertSeverity, alertSummary, alertDescription, self.alertServiceName)
+                # alert to expiry in an hour from now
+                self.alertManagerApi.sendAlert(alertName, alertSeverity, alertSummary, alertDescription,
+                                               self.alertServiceName, endSecs=1 * 60 * 60)
             except Exception as ex:
                 self.logger.exception("Failed to send alert to %s. Error: %s", self.alertManagerUrl, str(ex))
-            self.logger.info(alertDescription)
+            self.logger.warning(alertDescription)
 
     def _getValidSites(self, wflow, dataIn):
         """
Repository: dmwm/wmcore
Issue: MSMonitor: Rucio lock completion has a mix of fraction and percentage values

**Impact of the bug**
MSMonitor

**Describe the bug**
When calculating the progress of Rucio rules (locks), I noticed values much greater than 100, which come from a mixed usage of fraction (0.0 - 1.0) and percentage (0 - 100). PhEDEx transfers progress is made in percentage, so we should keep using percentage for the rucio rule progress as well, otherwise it can affect request state transition.

**How to reproduce it**
Have a rucio rule going to OK, it would result in progress 100 * 100. Other ways to reproduce it too.

**Expected behavior**
Rule progress should be a number between 0 and 100, where 100 means that the rule has been completely served and data is at the final destination (which then triggers a request state transition from staging to staged).

**Additional context and error message**
Follow up of the work done in https://github.com/dmwm/WMCore/pull/9881


Changed files:
src/python/WMCore/MicroService/Unified/MSMonitor.py

diff --git a/src/python/WMCore/MicroService/Unified/MSMonitor.py b/src/python/WMCore/MicroService/Unified/MSMonitor.py
index 08023de89..2fc4cccaa 100644
--- a/src/python/WMCore/MicroService/Unified/MSMonitor.py
+++ b/src/python/WMCore/MicroService/Unified/MSMonitor.py
@@ -260,9 +260,9 @@ class MSMonitor(MSCore):
                 lockCompletion = 100.0
             else:
                 totalLocks = data['locks_ok_cnt'] + data['locks_replicating_cnt'] + data['locks_stuck_cnt']
-                lockCompletion = data['locks_ok_cnt'] / totalLocks
-            completion.append(lockCompletion * 100)
-            self.logger.info("Rule ID: %s has a completion rate of: %s", ruleID, lockCompletion)
+                lockCompletion = (data['locks_ok_cnt'] / totalLocks) * 100
+            completion.append(lockCompletion)
+            self.logger.info("Rule ID: %s has a completion rate of: %s%%", ruleID, lockCompletion)
             self.logger.debug("Rule ID: %s, DID: %s, state: %s, grouping: %s, rse_expression: %s",
                               ruleID, data['name'], data['state'], data['grouping'], data['rse_expression'])
         if not completion:
Repository: dmwm/wmcore
Issue: Keep microservice transfer documents for workflows in the failed status

**Impact of the bug**
ReqMgr2MS

**Describe the bug**
As noted by Erik, on the issue #9579 , we should not cleanup transfer documents for workflows sitting in the `failed` status.

**How to reproduce it**
none

**Expected behavior**
Make sure the workflow transfer document is kept around for workflows under the `failed` status.
Thus, add that status to this list:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/ReqMgr/CherryPyThreads/CouchDBCleanup.py#L22

**Additional context and error message**
Follow up of this issue: https://github.com/dmwm/WMCore/issues/9478

Changed files:
src/python/WMCore/ReqMgr/CherryPyThreads/CouchDBCleanup.py

diff --git a/src/python/WMCore/ReqMgr/CherryPyThreads/CouchDBCleanup.py b/src/python/WMCore/ReqMgr/CherryPyThreads/CouchDBCleanup.py
index 1ee65dd1b..6dad693a7 100644
--- a/src/python/WMCore/ReqMgr/CherryPyThreads/CouchDBCleanup.py
+++ b/src/python/WMCore/ReqMgr/CherryPyThreads/CouchDBCleanup.py
@@ -20,7 +20,7 @@ class CouchDBCleanup(CherryPyPeriodicTask):
         self.reqmgrAux = ReqMgrAux(config.reqmgr2_url, logger=self.logger)
         # statuses that we want to keep the transfer documents
         self.transferStatuses = ["assigned", "staging", "staged", "acquired",
-                                 "running-open", "running-closed"]
+                                 "failed", "running-open", "running-closed"]
 
         baseURL, acdcDB = splitCouchServiceURL(config.acdc_url)
         self.acdcService = CouchService(url=baseURL, database=acdcDB)
Repository: dmwm/wmcore
Issue: Review Services cacheduration parameter (it's in hours, not seconds!)

**Impact of the bug**
WMCore in general

**Describe the bug**
Not a bug per se, but one can misuse this parameter and cause nasty bugs downstream.
This issue is made such that we can review all Services wrapper APIs and make sure the use a sound value for the `cacheduration` parameter.

Here is a mistake a made for the MicroServices:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/MicroService/Unified/MSCore.py#L36-L45

**How to reproduce it**
none

**Expected behavior**
Review all the WMCore/Services module and make sure they are setting what they want to set as cacheduration (i.e., make sure the caching expiration makes sense!).

**Additional context and error message**
none

Changed files:
src/python/WMCore/MicroService/Unified/MSCore.py

diff --git a/src/python/WMCore/MicroService/Unified/MSCore.py b/src/python/WMCore/MicroService/Unified/MSCore.py
index cf11ba434..dce22e155 100644
--- a/src/python/WMCore/MicroService/Unified/MSCore.py
+++ b/src/python/WMCore/MicroService/Unified/MSCore.py
@@ -29,12 +29,11 @@ class MSCore(object):
         """
         self.logger = getMSLogger(getattr(msConfig, 'verbose', False), logger)
         self.msConfig = msConfig
-        self.logger.info(
-            "Configuration including default values:\\n%s", self.msConfig)
+        self.logger.info("Configuration including default values:\\n%s", self.msConfig)
 
         self.reqmgr2 = ReqMgr(self.msConfig['reqmgrUrl'], logger=self.logger)
         self.reqmgrAux = ReqMgrAux(self.msConfig['reqmgrUrl'],
-                                   httpDict={'cacheduration': 60},
+                                   httpDict={'cacheduration': 1.0},
                                    logger=self.logger)
 
         # hard code it to production DBS otherwise PhEDEx subscribe API fails to match TMDB data
@@ -42,10 +41,10 @@ class MSCore(object):
         if usingRucio():
             # FIXME: we cannot use Rucio in write mode yet
             # self.rucio = Rucio(self.msConfig['rucioAccount'], configDict={"logger": self.logger})
-            self.phedex = PhEDEx(httpDict={'cacheduration': 10 * 60},
+            self.phedex = PhEDEx(httpDict={'cacheduration': 0.5},
                                  dbsUrl=dbsUrl, logger=self.logger)
         else:
-            self.phedex = PhEDEx(httpDict={'cacheduration': 10 * 60},
+            self.phedex = PhEDEx(httpDict={'cacheduration': 0.5},
                                  dbsUrl=dbsUrl, logger=self.logger)
 
     def unifiedConfig(self):
@@ -65,7 +64,7 @@ class MSCore(object):
         """
         Update the request status in ReqMgr2
         """
-        self.logger.info('%s updating %s status to %s', prefix, reqName, reqStatus)
+        self.logger.info('%s updating %s status to: %s', prefix, reqName, reqStatus)
         try:
             if not self.msConfig['readOnly']:
                 self.reqmgr2.updateRequestStatus(reqName, reqStatus)
Repository: dmwm/wmcore
Issue: New FNAL agents failed to check the proxy lifetime

**Impact of the bug**
WMAgent

**Describe the bug**
Jen reported during the P&R meeting that 2 of the FNAL agents had proxy error in WMStats (submit3 and submit4), which are running the newest WMAgent version 1.2.6.patch1. The other FNAL agents are doing just fine, running on the old version.

Checking the lifetime of my proxy, I see everything is fine:
```
notBefore=Sep 19 05:50:03 2019 GMT
notAfter=Sep 26 05:55:02 2019 GMT
```

**How to reproduce it**
none

**Expected behavior**
Properly fetch the proxy timeleft and properly report it ;)

**Additional context and error message**
It might be coming from one of the security/crypto libraries that have been updated in cmsdist/comp over the last month or two...

Changed files:
src/python/WMComponent/AgentStatusWatcher/AgentStatusPoller.py

diff --git a/src/python/WMComponent/AgentStatusWatcher/AgentStatusPoller.py b/src/python/WMComponent/AgentStatusWatcher/AgentStatusPoller.py
index 6ee07f040..8feb302ca 100644
--- a/src/python/WMComponent/AgentStatusWatcher/AgentStatusPoller.py
+++ b/src/python/WMComponent/AgentStatusWatcher/AgentStatusPoller.py
@@ -42,7 +42,7 @@ class AgentStatusPoller(BaseWorkerThread):
         self.agentInfo = initAgentInfo(self.config)
         self.summaryLevel = config.AnalyticsDataCollector.summaryLevel
 
-        proxyArgs = {'logger': logging.getLogger()}
+        proxyArgs = {'logger': logging.getLogger(), 'cleanEnvironment': True}
         self.proxy = Proxy(proxyArgs)
         self.proxyFile = self.proxy.getProxyFilename()  # X509_USER_PROXY
         self.userCertFile = self.proxy.getUserCertFilename()  # X509_USER_CERT
Repository: dmwm/wmcore
Issue: UserFileCache.removeFile throws on success

**Impact of the bug**
CRAB

**Describe the bug**
UserFileCache.removeFile throws TypeError on success.

**How to reproduce it**
```crab purge -d <CRAB Directory>```

Alternatively, without crab:

```
from WMCore.Services.UserFileCache.UserFileCache import UserFileCache
ufc=UserFileCache({'endpoint': u'https://cmsweb.cern.ch/crabcache', 'pycurl': True})
hashkey=ufc.upload('some_file.tgz')['hashkey']
ufc.removeFile(hashkey)
```

**Expected behavior**
```UserFileCache.removeFile``` returns successfully.

**Additional context and error message**
Traceback:
```
ufc.removeFile throws
Traceback (most recent call last):
  File "purge_test.py", line 10, in <module>
    res=crab('purge',dir=sys.argv[1])
  File "/cvmfs/cms.cern.ch/crab3/slc6_amd64_gcc493/cms/crabclient/3.3.2001/lib/python2.7/site-packages/CRABAPI/RawCommand.py", line 26, in crabCommand
    return execRaw(command, arguments)
  File "/cvmfs/cms.cern.ch/crab3/slc6_amd64_gcc493/cms/crabclient/3.3.2001/lib/python2.7/site-packages/CRABAPI/RawCommand.py", line 46, in execRaw
    res = cmdobj()
  File "/cvmfs/cms.cern.ch/crab3/slc6_amd64_gcc493/cms/crabclient/3.3.2001/lib/python2.7/site-packages/CRABClient/Commands/purge.py", line 65, in __call__
    ufcresult = ufc.removeFile(hashkey)
  File "/cvmfs/cms.cern.ch/crab3/slc6_amd64_gcc493/cms/crabclient/3.3.2001/lib/python2.7/site-packages/WMCore/Services/UserFileCache/UserFileCache.py", line 123, in removeFile
    return result[0]['result'][0]
TypeError: string indices must be integers, not str
```

Changed files:
src/python/WMCore/Services/UserFileCache/UserFileCache.py

diff --git a/src/python/WMCore/Services/UserFileCache/UserFileCache.py b/src/python/WMCore/Services/UserFileCache/UserFileCache.py
index 88df4bc61..b69fb7a0f 100644
--- a/src/python/WMCore/Services/UserFileCache/UserFileCache.py
+++ b/src/python/WMCore/Services/UserFileCache/UserFileCache.py
@@ -120,7 +120,7 @@ class UserFileCache(Service):
 
     def removeFile(self, haskey):
         result=self['requests'].makeRequest(uri = 'info', data = {'subresource':'fileremove', 'hashkey': haskey})
-        return result[0]['result'][0]
+        return json.loads(result[0])['result'][0]
 
     def download(self, hashkey, output, username=None):
         """
Repository: dmwm/wmcore
Issue: ParentageResolved flag needs to be updated for workflows with no parents too

**Impact of the bug**
ReqMgr2

**Describe the bug**
As reported in this JIRA:
https://its.cern.ch/jira/browse/CMSCOMPPR-16710

there are workflows that are not marked as parentage resolved, thus blocking Unified from announcing workflows.

**How to reproduce it**
Run StepChain workflow producing only one dataset from scratch, and move it to closed-out.

**Expected behavior**
The ReqMgr2 cherrypy thread should go through all the StepChain workflows and mark them with:
```
ParentageResolved = true
```
when there is no parentage information to be updated, besides the normal operation of the thread.

**Additional context and error message**
Example of workflow producing - from scratch - only a MINIAODSIM dataset, thus no parentage information needs to be updated:
https://cmsweb.cern.ch/reqmgr2/fetch?rid=cmsunified_task_EXO-RunIIFall18wmLHEGS-02833__v1_T_200220_091412_4231

Changed files:
src/python/WMCore/ReqMgr/CherryPyThreads/StepChainParentageFixTask.py

diff --git a/src/python/WMCore/ReqMgr/CherryPyThreads/StepChainParentageFixTask.py b/src/python/WMCore/ReqMgr/CherryPyThreads/StepChainParentageFixTask.py
index 681d2844a..0b3e74885 100644
--- a/src/python/WMCore/ReqMgr/CherryPyThreads/StepChainParentageFixTask.py
+++ b/src/python/WMCore/ReqMgr/CherryPyThreads/StepChainParentageFixTask.py
@@ -21,9 +21,8 @@ def getChildDatasetsForStepChainMissingParent(reqmgrDB, status):
     for reqName, info in results.items():
 
         for dsInfo in info.values():
-            if dsInfo["ParentDset"]:
-                for childDS in dsInfo["ChildDsets"]:
-                    requestsByChildDataset[childDS].add(reqName)
+            for childDS in dsInfo["ChildDsets"]:
+                requestsByChildDataset[childDS].add(reqName)
     return requestsByChildDataset
Repository: dmwm/wmcore
Issue: Change default RucioInjector.listTiersToInject value

**Impact of the bug**
WMAgent

**Describe the bug**
While we are still not using RucioInjector, that default value:
```
config.RucioInjector.listTiersToInject = ['NANOAOD', 'NANOAODSIM']
```
will actually cause problems in PhEDExInjector.
I wanted to make sure they would not work on the same datatier, the problem is that, if RucioInjector is disabled in the configuration, but still has that default value, those two datatiers will actually be skipped by PhEDExInjector as well.
In short, that default value only makes sense once we start gradually using RucioInjector.

**How to reproduce it**
none

**Expected behavior**
Default `listTiersToInject` to a empty list; and change the configuration once we are ready to switch some data over to RucioInjector.

**Additional context and error message**
none

Changed files:
etc/WMAgentConfig.py

diff --git a/etc/WMAgentConfig.py b/etc/WMAgentConfig.py
index 8415ae257..604f93200 100644
--- a/etc/WMAgentConfig.py
+++ b/etc/WMAgentConfig.py
@@ -368,7 +368,7 @@ config.RucioInjector.pollIntervalRules = 43200
 config.RucioInjector.cacheExpiration = 2 * 24 * 60 * 60  # two days
 config.RucioInjector.createBlockRules = True
 config.RucioInjector.RSEPostfix = False  # enable it to append _Test to the RSE names
-config.RucioInjector.listTiersToInject = ["NANOAOD", "NANOAODSIM"]
+config.RucioInjector.listTiersToInject = []  # ["NANOAOD", "NANOAODSIM"]
 config.RucioInjector.skipRulesForTiers = ["NANOAOD", "NANOAODSIM"]
 config.RucioInjector.rucioAccount = "OVER_WRITE_BY_SECRETS"
 config.RucioInjector.rucioUrl = "OVER_WRITE_BY_SECRETS"
Repository: dmwm/wmcore
Issue: Error when ssbStatus is unknown

**Impact of the bug**
WMAgent

**Describe the bug**
ResourceControlUpdater scans the list of site status, if a site is in an "unknown" state it tries to pop a item from ssbStatus (a string) instead of ssbState (a dict).

**How to reproduce it**
Start AgentStatusWatcher with PR #9584 

**Expected behavior**
Pop the item from ssbState instead of ssbStatus.
https://github.com/dmwm/WMCore/blob/2a7ebaaaa108859802a84f73abdce014e3b8e280/src/python/WMComponent/AgentStatusWatcher/ResourceControlUpdater.py#L269

**Additional context and error message**
```
2020-04-02 20:36:57,725:139635865409280:WARNING:ResourceControlUpdater:Site T3_US_PuertoRico has an unknown SSB status 'unknown'. Skipping it!
2020-04-02 20:36:57,725:139635865409280:ERROR:ResourceControlUpdater:Error occurred, will retry later:
2020-04-02 20:36:57,725:139635865409280:ERROR:ResourceControlUpdater:'unicode' object has no attribute 'pop'
2020-04-02 20:36:57,726:139635865409280:ERROR:ResourceControlUpdater:Trace back: 
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.3.0.patch1/sw/slc7_amd64_gcc630/cms/wmagent/1.3.0.patch1/lib/python2.7/site-packages/WMComponent/AgentStatusWatcher/ResourceControlUpdater.py", line 96, in algorithm
    ssbSiteStatus = self.getSiteStatus()
  File "/data/srv/wmagent/v1.3.0.patch1/sw/slc7_amd64_gcc630/cms/wmagent/1.3.0.patch1/lib/python2.7/site-packages/WMComponent/AgentStatusWatcher/ResourceControlUpdater.py", line 271, in getSiteStatus
    ssbStatus.pop(site, None)
AttributeError: 'unicode' object has no attribute 'pop'
```

Changed files:
src/python/WMComponent/AgentStatusWatcher/ResourceControlUpdater.py

diff --git a/src/python/WMComponent/AgentStatusWatcher/ResourceControlUpdater.py b/src/python/WMComponent/AgentStatusWatcher/ResourceControlUpdater.py
index 6e3a6eb0a..c70e3b86b 100644
--- a/src/python/WMComponent/AgentStatusWatcher/ResourceControlUpdater.py
+++ b/src/python/WMComponent/AgentStatusWatcher/ResourceControlUpdater.py
@@ -266,7 +266,7 @@ class ResourceControlUpdater(BaseWorkerThread):
             wmcoreStatus = self.getState(str(ssbStatus))
             if not wmcoreStatus:
                 logging.warning("Site %s has an unknown SSB status '%s'. Skipping it!", site, ssbStatus)
-                ssbStatus.pop(site, None)
+                ssbState.pop(site, None)
             else:
                 ssbState[site] = {'state': wmcoreStatus}
Repository: dmwm/wmcore
Issue: Variable `file` out of scope - pylint complains - not critical

**Impact of the bug**
`src/python/WMCore/WMRuntime/Monitors/TestMonitor.py`, so i assume this is not critical.

**Describe the bug**
While running `pylint --py3k` working on #10440, pylint complained about the line

https://github.com/dmwm/WMCore/blob/dd63a901a9b929d83cbe83e9a2f9004e0901a90c/src/python/WMCore/WMRuntime/Monitors/TestMonitor.py#L155

with the warning `file built-in referenced (file-builtin)`. However, (as far as i understand) here we are using the variable `file` out of scope (it is not defined in the class `TestMonitor`), so pylint thinks we are using the python2 reserved keyword `file` (which is not a reserved keyword anymore in py3) and complains.

I propose to define the variable file as a `TestMonitor` class member, and maybe also change its name to a non-reserved keyword while we are at it. Maybe `self.cmsswFile` is what we should use in that line instead of `file`?

**How to reproduce it**
Run
```bash
pylint --py3k -d W1618,W1619 src/python/WMCore/WMRuntime/Monitors/TestMonitor.py
```

**Expected behavior**
We would like pylint not to complain (not too much, at least)



Changed files:
src/python/WMCore/WMRuntime/Monitors/TestMonitor.py

diff --git a/src/python/WMCore/WMRuntime/Monitors/TestMonitor.py b/src/python/WMCore/WMRuntime/Monitors/TestMonitor.py
index dd4fbf46b..f7a0ffb85 100644
--- a/src/python/WMCore/WMRuntime/Monitors/TestMonitor.py
+++ b/src/python/WMCore/WMRuntime/Monitors/TestMonitor.py
@@ -152,7 +152,7 @@ class TestMonitor(WMRuntimeMonitor):
 
         # Check for events
         if self.cmsswFile:
-            run, event = searchForEvent(file)
+            run, event = searchForEvent(self.cmsswFile)
             if run and event:
                 # Then we actually found something, otherwise do nothing
                 # Right now I don't know what to do
Repository: dmwm/wmcore
Issue: Problem parsing ProcessingVersion on StoreResults script

**Impact of the bug**
Script used by Ops to create StoreResults workflows doesn't work on some cases of InputDataset.

**Describe the bug**
Provided that there is no strong schema on USER dataset creation, it could be that the InputDataset doesn't have a format/length expected by the script and eventually fails to parse the input json used to feed the script.

In this case ProcVer was a string, failing to be casted to integer.

**How to reproduce it**
Steps to reproduce the behavior:

**Expected behavior**
If it can't parse ProcVer for any reason - like dataset name too large - then default it to 1.

**Additional context and error message**
Sharad's output error:
```
[sagarwal@vocms0275 WmAgentScripts]$ python createStoreResults_updated.py ~/public/dataelevation.json 
Migrating dataset /GluGluHToTauTau_M125_13TeV_powheg_pythia8_nospinner/dwinterb-GluGluHToTauTau_M125_13TeV_powheg_pythia8_nospinner-miniAOD-28028af67189b3de7224b79195bd0e1d/USER from https://cmsweb.cern.ch/dbs/prod/phys03/DBSReader to prod/global
Traceback (most recent call last):
  File "createStoreResults_updated.py", line 160, in <module>
    main()
  File "createStoreResults_updated.py", line 69, in main
    newDict = buildRequest(entry)
  File "createStoreResults_updated.py", line 112, in buildRequest
    newSchema["ProcessingVersion"] = int(procVer[1:])
ValueError: invalid literal for int() with base 10: '8028af67189b3de7224b79195bd0e1d'
```


Changed files:
bin/createStoreResults.py

diff --git a/bin/createStoreResults.py b/bin/createStoreResults.py
index 6daa1b183..ad054d554 100644
--- a/bin/createStoreResults.py
+++ b/bin/createStoreResults.py
@@ -109,12 +109,17 @@ def buildRequest(userDict):
     # Truncate the ProcessingString, otherwise it can be larger than allowed
     primDset, procDset, _tier = newSchema['InputDataset'].split("/")[1:]
     acqEra, procStr = procDset.split("-", 1)
-    newSchema["AcquisitionEra"] = acqEra  # should we worry about lenght limits?
+    newSchema["AcquisitionEra"] = acqEra  # should we worry about length limits?
     procStr, procVer = procStr.rsplit("-", 1)
     newSchema["ProcessingString"] = "StoreResults_" + procStr[:67]  # limit to 80 chars
-    newSchema["ProcessingVersion"] = int(procVer[1:])
+    # ProcessingString cannot have a dash char
+    newSchema["ProcessingString"] = newSchema["ProcessingString"].replace("-", "_")
+    try:
+        newSchema["ProcessingVersion"] = int(procVer[1:])
+    except ValueError:
+        newSchema["ProcessingVersion"] = 1
     # Use PrimaryDataset and ProcessedDataset in the RequestString
-    newSchema["RequestString"] = primDset[:50] + "-" + procDset[:50]
+    newSchema["RequestString"] = primDset[:35] + "-" + procDset[:35]
     return newSchema
Repository: dmwm/wmcore
Issue: MSRuleCleaner: local variable 'transferInfo' referenced before assignment

**Impact of the bug**
MSRuleCleaner

**Describe the bug**
While testing another and unrelated PR, one unit test failed and part of the error message is:
```
  File "/build/cmsbld/jenkins/workspace/DMWM-WMCorePy3-PR-unittests/SLICE/3/label/cms-dmwm-cc7/code/src/python/WMCore/MicroService/MSRuleCleaner/MSRuleCleaner.py", line 632, in getMSOutputTransferInfo
    if transferInfo is None:
local variable 'transferInfo' referenced before assignment
```

here is the full unit test log (which will become unavailable in 30 days from now):
https://cmssdt.cern.ch/dmwm-jenkins/job/DMWM-WMCore-PR-test/13970/testReport/junit/WMCore_t.MicroService_t.MSRuleCleaner_t.MSRuleCleaner_t/MSRuleCleanerTest/testRunning/

**How to reproduce it**
Make that code to execute the exception block

**Expected behavior**
That unexpected exception should not occur.

**Additional context and error message**
None

Changed files:
src/python/WMCore/MicroService/MSRuleCleaner/MSRuleCleaner.py

diff --git a/src/python/WMCore/MicroService/MSRuleCleaner/MSRuleCleaner.py b/src/python/WMCore/MicroService/MSRuleCleaner/MSRuleCleaner.py
index 8d5807aa3..54d1cd049 100644
--- a/src/python/WMCore/MicroService/MSRuleCleaner/MSRuleCleaner.py
+++ b/src/python/WMCore/MicroService/MSRuleCleaner/MSRuleCleaner.py
@@ -625,6 +625,7 @@ class MSRuleCleaner(MSCore):
             data = json.loads(res)['result'][0]
             transferInfo = data['transferDoc']
         except Exception as ex:
+            transferInfo = None
             msg = "General exception while fetching TransferInfo from MSOutput for %s. "
             msg += "Error: %s"
             self.logger.exception(msg, wflow['RequestName'], str(ex))
Repository: dmwm/wmcore
Issue: CRAB REST cannot write logs to named pipe with PR #11431

**Impact of the bug**
CRAB REST

**Describe the bug**
CRAB REST cannot write logs to named pipe with Valentin PR [dmwm/WMCore#11431](https://github.com/dmwm/WMCore/pull/11431).

Last week, I opened PR [dmwm/WMCore#11455](https://github.com/dmwm/WMCore/pull/11455), and we agreed to merge [dmwm/WMCore#11431](https://github.com/dmwm/WMCore/pull/11431).
But, when I tested it again after PR merged, I found that I ran the wrong script that did write logs to file instead of named pipe. When I switched to named pipe this time, I got `UnsupportedOperation: File or stream is not seekable` error.

CRAB REST want to write to named pipe in order to stream logs to the container stdout using our [solution here](https://github.com/dmwm/CRABServer/issues/7463#issuecomment-1344306384).

**How to reproduce it**
1. Make named pipe with `mkfifo /crabserver-fifo`.
2. Run `cat crabserver-fifo` to start reading data (named pipe need to read before CRAB's process can write).
3. In another terminal, start python and try to open file with `a+` mode. You will see this error:
```
>>> open('/crabserver-fifo', 'a+')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
io.UnsupportedOperation: File or stream is not seekable.
```

**Expected behavior**
[WMCore/REST/Main.py#L406](https://github.com/dmwm/WMCore/blob/81c7d43e71283b9fca163895d295975dc8655c89/src/python/WMCore/REST/Main.py#L406) should open named pipe without error.

**Additional context and error message**
Changing the open file mode to `a` is enough to fix this.

Mode `a` works for writing logs to both file and named pipe because logs operation (at least for StreamHandler, default of CherryPy's WMCore) is write-only operation.

Sorry for my mistake in my last PR.

Changed files:
src/python/WMCore/REST/Main.py

diff --git a/src/python/WMCore/REST/Main.py b/src/python/WMCore/REST/Main.py
index b8427080d..6cc9a628c 100644
--- a/src/python/WMCore/REST/Main.py
+++ b/src/python/WMCore/REST/Main.py
@@ -403,7 +403,9 @@ class RESTDaemon(RESTMain):
                                 bufsize=0, close_fds=True, shell=False)
                 logger = subproc.stdin
             elif isinstance(self.logfile, str):
-                logger = open(self.logfile, "a+")
+                # if a unix pipe is set as the logfile, it must be opened to append to the end of the file
+                # if file/pipe does not exist, create it
+                logger = open(self.logfile, "a")
             else:
                 raise TypeError("'logfile' must be a string or array")
             os.dup2(logger.fileno(), sys.stdout.fileno())