anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: dmwm/wmcore
Issue: Possible misconfiguration with the secondary input

Also related to https://github.com/dmwm/WMCore/issues/8983
and initially reported and discussed over this JIRA ticket: https://its.cern.ch/jira/browse/CMSCOMPPR-4523

I'm still debugging it, but I have strong suspicious on a badly configured pileup task. More news this afternoon.

Changed files:
src/python/WMCore/WMRuntime/Scripts/SetupCMSSWPset.py

diff --git a/src/python/WMCore/WMRuntime/Scripts/SetupCMSSWPset.py b/src/python/WMCore/WMRuntime/Scripts/SetupCMSSWPset.py
index 346c80eef..67a7c7b45 100755
--- a/src/python/WMCore/WMRuntime/Scripts/SetupCMSSWPset.py
+++ b/src/python/WMCore/WMRuntime/Scripts/SetupCMSSWPset.py
@@ -499,7 +499,7 @@ class SetupCMSSWPset(ScriptInterface):
         prodsAndFilters.update(self.process.producers)
         prodsAndFilters.update(self.process.filters)
         for key, value in prodsAndFilters.items():
-            if value.type_() in ["MixingModule", "DataMixingModule"]:
+            if value.type_() in ["MixingModule", "DataMixingModule", "PreMixingModule"]:
                 mixModules.append(value)
             if value.type_() == "DataMixingModule":
                 dataMixModules.append(value)
Repository: dmwm/wmcore
Issue: ACDC of recovery workflows

https://cmsweb.cern.ch/reqmgr2/fetch?rid=vlimant_ACDC0_recovery-1-fabozzi_Run2016H-v1-BTagCSV-07Aug17_8029__171013_133417_8968

went in failed. it's an ACDC of a recovery.
how hard will it be to make it work ?
FYI @prozober


Changed files:
src/python/WMCore/ReqMgr/Utils/Validation.py

diff --git a/src/python/WMCore/ReqMgr/Utils/Validation.py b/src/python/WMCore/ReqMgr/Utils/Validation.py
index f095683dc..c6f408261 100644
--- a/src/python/WMCore/ReqMgr/Utils/Validation.py
+++ b/src/python/WMCore/ReqMgr/Utils/Validation.py
@@ -116,6 +116,9 @@ def validate_resubmission_create_args(request_args, config, reqmgr_db_service, *
     response = reqmgr_db_service.getRequestByNames(request_args["OriginalRequestName"])
     originalArgs = response.values()[0]
 
+    ### not a nice fix for #8245, but we cannot inherit the CollectionName attr
+    originalArgs.pop("CollectionName", None)
+
     chainArgs = None
     if originalArgs["RequestType"] == 'Resubmission':
         # ACDC of ACDC, we can't validate this case
Repository: dmwm/wmcore
Issue: Add pileup support to StepChain MC from scratch

While working on something else, I noticed the `pileup` section is not created IF the StepChain contains a single step and the first step does not have an input dataset, i.e., if `setupGeneratorTask` is called instead of `setupTask`.

Changed files:
test/python/WMCore_t/WMSpec_t/StdSpecs_t/StepChain_t.py, src/python/WMCore/WMSpec/StdSpecs/StepChain.py

diff --git a/src/python/WMCore/WMSpec/StdSpecs/StepChain.py b/src/python/WMCore/WMSpec/StdSpecs/StepChain.py
index fa5f5a30d..02709c558 100644
--- a/src/python/WMCore/WMSpec/StdSpecs/StepChain.py
+++ b/src/python/WMCore/WMSpec/StdSpecs/StepChain.py
@@ -225,6 +225,9 @@ class StepChainWorkloadFactory(StdBase):
                                            globalTag=taskConf.get("GlobalTag", None),
                                            taskConf=taskConf)
 
+        if taskConf["PileupConfig"]:
+            self.setupPileup(task, taskConf['PileupConfig'])
+
         # outputModules were added already, we just want to create merge tasks here
         if strToBool(taskConf.get('KeepOutput', True)):
             self.setupMergeTask(task, taskConf, "cmsRun1", outMods)
diff --git a/test/python/WMCore_t/WMSpec_t/StdSpecs_t/StepChain_t.py b/test/python/WMCore_t/WMSpec_t/StdSpecs_t/StepChain_t.py
index 2873c0260..1cbbafb2b 100644
--- a/test/python/WMCore_t/WMSpec_t/StdSpecs_t/StepChain_t.py
+++ b/test/python/WMCore_t/WMSpec_t/StdSpecs_t/StepChain_t.py
@@ -490,6 +490,32 @@ class StepChainTests(EmulatedUnitTestCase):
 
         return
 
+    def testPileupWithoutInputData(self):
+        """
+        Test whether we can properly setup the Pileup information even
+        when the top level step has no input dataset (so pileup over a
+        MC from scratch step)
+        """
+        testArguments = StepChainWorkloadFactory.getTestArguments()
+        testArguments.update(deepcopy(REQUEST))
+
+        configId = injectStepChainConfigSingle(self.configDatabase)
+        testArguments['Step1'].update(ConfigCacheID=configId,
+                                      MCPileup=testArguments['Step2']['MCPileup'])
+        testArguments.pop("Step2", None)
+        testArguments.pop("Step3", None)
+        testArguments['StepChain'] = 1
+
+        factory = StepChainWorkloadFactory()
+        testWorkload = factory.factoryWorkloadConstruction("TestWorkload", testArguments)
+
+        # workload level check
+        self.assertEqual(testWorkload.getDashboardActivity(), "production")
+        self.assertEqual(testWorkload.listInputDatasets(), [])
+        pileups = testWorkload.listPileupDatasets().values()
+        pileups = [item for puSet in pileups for item in puSet]
+        self.assertItemsEqual(pileups, [testArguments['Step1']['MCPileup']])
+
     def testStepChainIncludeParentsValidation(self):
         """
         Check that the test arguments pass basic validation,
Repository: dmwm/wmcore
Issue: DrainStatusPoller crash when couch is unavailable

I saw this error in one of the agents this morning
```
2018-11-06 11:10:19,930:139824323180288:ERROR:BaseWorkerThread:Error in worker algorithm (1):
Backtrace:
  <WMComponent.AgentStatusWatcher.DrainStatusPoller.DrainStatusPoller object at 0x7f2b641db910> 'NoneType' object has no attribute 'items'  File "/data/srv/wmagent/v1.1.15.patch4/sw/slc7_amd64_gcc630/cms/wmagent/1.1.15.patch4/lib/python
2.7/site-packages/WMCore/WorkerThreads/BaseWorkerThread.py", line 187, in __call__
    tSpent, results, _ = algorithmWithDBExceptionHandler(parameters)
  File "/data/srv/wmagent/v1.1.15.patch4/sw/slc7_amd64_gcc630/cms/wmagent/1.1.15.patch4/lib/python2.7/site-packages/WMCore/Database/DBExceptionHandler.py", line 28, in wrapper
    return f(*args, **kwargs)
  File "/data/srv/wmagent/v1.1.15.patch4/sw/slc7_amd64_gcc630/cms/wmagent/1.1.15.patch4/lib/python2.7/site-packages/Utils/Timers.py", line 24, in wrapper
    res = func(*arg, **kw)
  File "/data/srv/wmagent/v1.1.15.patch4/sw/slc7_amd64_gcc630/cms/wmagent/1.1.15.patch4/lib/python2.7/site-packages/WMComponent/AgentStatusWatcher/DrainStatusPoller.py", line 47, in algorithm
    thresholdsHit = self.checkSpeedDrainThresholds()
  File "/data/srv/wmagent/v1.1.15.patch4/sw/slc7_amd64_gcc630/cms/wmagent/1.1.15.patch4/lib/python2.7/site-packages/WMComponent/AgentStatusWatcher/DrainStatusPoller.py", line 144, in checkSpeedDrainThresholds
    for k, v in speedDrainConfig.items():

2018-11-06 11:10:19,931:139824323180288:INFO:Harness:>>>Terminating worker threads
2018-11-06 11:10:28,469:139824323180288:ERROR:BaseWorkerThread:Error in event loop (2): <WMComponent.AgentStatusWatcher.DrainStatusPoller.DrainStatusPoller object at 0x7f2b641db910> 'NoneType' object has no attribute 'items'
Backtrace:
  File "/data/srv/wmagent/v1.1.15.patch4/sw/slc7_amd64_gcc630/cms/wmagent/1.1.15.patch4/lib/python2.7/site-packages/WMCore/WorkerThreads/BaseWorkerThread.py", line 213, in __call__
    raise ex
```

so it fails to fetch the agent info from the auxiliary db.

Changed files:
src/python/WMComponent/AgentStatusWatcher/DrainStatusPoller.py

diff --git a/src/python/WMComponent/AgentStatusWatcher/DrainStatusPoller.py b/src/python/WMComponent/AgentStatusWatcher/DrainStatusPoller.py
index 0e2c0d3be..c4a9e29f5 100644
--- a/src/python/WMComponent/AgentStatusWatcher/DrainStatusPoller.py
+++ b/src/python/WMComponent/AgentStatusWatcher/DrainStatusPoller.py
@@ -41,6 +41,9 @@ class DrainStatusPoller(BaseWorkerThread):
         """
         logging.info("Running agent drain algorithm...")
         self.agentConfig = self.reqAuxDB.getWMAgentConfig(self.config.Agent.hostName)
+        if not self.agentConfig:
+            logging.error("Failed to fetch agent configuration from the auxiliary DB")
+            return
 
         if isDrainMode(self.config):
             # check to see if the agent hit any speed drain thresholds
Repository: dmwm/wmcore
Issue: SubmitFailed not reporting site in wmstats

for pdmvserv_task_HIG-RunIIFall18wmLHEGS-00388__v1_T_181126_083424_8286 for example, job failed with SubmitFailed but the site is not reported in wmstat (while it should to my understanding, @amaltaro ?)

Changed files:
src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py, src/python/WMCore/FwkJobReport/Report.py

diff --git a/src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py b/src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py
index 5d326981b..82cf7d5d5 100644
--- a/src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py
+++ b/src/python/WMComponent/JobSubmitter/JobSubmitterPoller.py
@@ -482,7 +482,8 @@ class JobSubmitterPoller(BaseWorkerThread):
             job['fwjr'] = Report()
             if exitCode in [71102, 71104]:
                 job['fwjr'].addError("JobSubmit", exitCode, "SubmitFailed",
-                                     WM_JOB_ERROR_CODES[exitCode] + ', '.join(job['possibleSites']))
+                                     WM_JOB_ERROR_CODES[exitCode] + ', '.join(job['possibleSites']),
+                                     ', '.join(job['possibleSites']))
             elif exitCode in [71101]:
                 # there is no possible site
                 if job.get("fileLocations"):
diff --git a/src/python/WMCore/FwkJobReport/Report.py b/src/python/WMCore/FwkJobReport/Report.py
index eace07937..06e93c611 100644
--- a/src/python/WMCore/FwkJobReport/Report.py
+++ b/src/python/WMCore/FwkJobReport/Report.py
@@ -278,6 +278,18 @@ class Report(object):
         """
         return getattr(self.data, 'siteName', {})
 
+
+    def _setSiteName(self, site):
+        """
+        _setSiteName_
+
+        Set the site name attribute (no step specific)
+        """
+        setattr(self.data, 'siteName', site)
+
+        return
+
+
     def getExitCodes(self):
         """
         _getExitCodes_
@@ -554,12 +566,13 @@ class Report(object):
         self.report.cleanup.removed.fileCount += 1
         return
 
-    def addError(self, stepName, exitCode, errorType, errorDetails):
+    def addError(self, stepName, exitCode, errorType, errorDetails, siteName=None):
         """
         _addError_
 
         Add an error report with an exitCode, type/class of error and
-        details of the error as a string
+        details of the error as a string.
+        Also, report attempted site if error happened before landing on it.
         """
         if self.retrieveStep(stepName) is None:
             # Create a step and set it to failed
@@ -583,6 +596,10 @@ class Report(object):
 
         setattr(stepSection.errors, "errorCount", errorCount + 1)
         self.setStepStatus(stepName=stepName, status=exitCode)
+
+        if siteName:
+            self._setSiteName(site=siteName)
+
         return
 
     def addSkippedFile(self, lfn, pfn):
Repository: augerai/a2ml
Issue: Azure predict proba issue

Run project with use_a2ml_hub: true in config.yml
train and deploy model
then run predict with threshold
target will contain class numbers, not actual classes. For iris:
prediction_id,sepal_length,sepal_width,petal_length,petal_width,class,proba_0,proba_1,proba_2
e37f8e1b-c70e-4ef3-9cec-fe914d963720,6.0,2.2,5.0,1.5,2,0.0006133182633167796,0.02319485741752154,0.9761918243191617


Changed files:
a2ml/tasks_queue/tasks_hub_api.py, a2ml/api/auger/impl/cloud/pipeline.py

diff --git a/a2ml/api/auger/impl/cloud/pipeline.py b/a2ml/api/auger/impl/cloud/pipeline.py
index 18b3513..9f85d52 100644
--- a/a2ml/api/auger/impl/cloud/pipeline.py
+++ b/a2ml/api/auger/impl/cloud/pipeline.py
@@ -16,7 +16,7 @@ class AugerPipelineApi(AugerBaseApi):
             ['creating_files', 'packaging', 'deploying'])
 
     def remove(self, trial_id):
-        return self._call_update({'id': trial_id, 'status': 'undeploying'})
+        return self._call_update({'id': trial_id, 'status': 'undeploying'}, progress=['ready','packaged_with_error'])
 
     def predict(self, records, features, threshold=None, file_url=None, predicted_at=None):
         if self.object_id is None:
diff --git a/a2ml/tasks_queue/tasks_hub_api.py b/a2ml/tasks_queue/tasks_hub_api.py
index 9225952..812b161 100644
--- a/a2ml/tasks_queue/tasks_hub_api.py
+++ b/a2ml/tasks_queue/tasks_hub_api.py
@@ -397,11 +397,14 @@ def undeploy_model_task(params):
     res = A2MLModel(ctx).undeploy(model_id = model_id)
     _update_hub_objects(ctx, params.get('provider'), params)
 
+    return res
+
 @celeryApp.task(ignore_result=True, after_return=process_task_result)
 def predict_by_model_task(params):
     from a2ml.api.utils.crud_runner import CRUDRunner
 
     ctx = _create_provider_context(params)
+    ctx = _read_hub_experiment_session(ctx, params)
 
     ctx.config.clean_changes()
     runner = CRUDRunner(ctx, "%s"%params.get('provider'), 'model')