Repository: dmwm/wmcore
Issue: Global WorkQueue setting using PNN instead of PSN for PileupData

**Impact of the bug**
Global WorkQueue and then work acquisition by WMAgent.

**Describe the bug**
A WorkQueue element contains a `SiteWhitelist` with a list of PSNs, and the input data attributes are also meant to be set with a list of PSNs (thus site locations instead of storage location). However, it seems we do not map PNN to PSN for the `PileupData` attribute, which then makes a workqueue element not to pass the work restriction logic, see:
https://github.com/dmwm/WMCore/blob/master/src/couchapps/WorkQueue/lists/workRestrictions.js

This is how pileup data discovery is made at workqueue level:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py#L234

while the input primary data (and parent) are followed by a CRIC call to convert from PNN to PSN, see:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WorkQueue/Policy/Start/Block.py#L195

**How to reproduce it**
Not really how to reproduce, but we use PhEDEx to find block replicas location, where we can see it returns the FNAL PNN:
```
>>> from WMCore.Services.PhEDEx.PhEDEx import PhEDEx
>>> ph = PhEDEx()
>>> ph.getReplicaPhEDExNodesForBlocks(block="/RelValMinBias_13/CMSSW_11_0_0_pre4-106X_upgrade2018_design_v5-v1/GEN-SIM#4c0f1601-7008-4733-9e23-33599d26f1c1",  complete='y')
{u'/RelValMinBias_13/CMSSW_11_0_0_pre4-106X_upgrade2018_design_v5-v1/GEN-SIM#4c0f1601-7008-4733-9e23-33599d26f1c1': [u'T1_US_FNAL_Disk']}
```

if converted to PSN using CRIC:
```
>>> from WMCore.Services.CRIC.CRIC import CRIC
>>> cric = CRIC()
>>> cric.PNNstoPSNs('T1_US_FNAL_Disk')
[u'T1_US_FNAL']
```

**Expected behavior**
Given that the agent provides the PSN and its slots - when requesting work to the parent queue - and that PSN is used to check whether the element and its data pass all the restrictions, we need to make sure a WQE contains the PSN as location, instead of PNN.
So mapping the PNNs to PSNs will do the trick.

**Additional context and error message**
Issue was reported via this JIRA ticket:
https://its.cern.ch/jira/browse/CMSCOMPPR-8666

hand picking one of those workflows, e.g.:
muahmad_RVCMSSW_11_0_0_pre5TTbar_13_PU25ns__190810_113803_761

I saw its WQE had the following data
```
{u'ACDC': {},
 u'ChildQueueUrl': None,
 u'CreationTime': 0,
 u'Dbs': u'https://cmsweb.cern.ch/dbs/prod/global/DBSReader',
 u'EndPolicy': {u'policyName': u'SingleShot'},
 u'EventsWritten': 0,
 u'FilesProcessed': 0,
 u'Inputs': {u'/RelValTTbar_13/CMSSW_11_0_0_pre4-106X_upgrade2018_design_v5-v1/GEN-SIM#0e1d6193-8e3c-407a-a433-d8f34d610025': [u'T1_US_FNAL']},
 u'Jobs': 18.0,
 u'Mask': None,
 u'NoInputUpdate': False,
 u'NoPileupUpdate': False,
 u'NumOfFilesAdded': 0,
 u'NumberOfEvents': 9000,
 u'NumberOfFiles': 4,
 u'NumberOfLumis': 180,
 u'OpenForNewData': False,
 u'ParentData': {},
 u'ParentFlag': False,
 u'ParentQueueId': u'muahmad_RVCMSSW_11_0_0_pre5TTbar_13_PU25ns__190810_113803_761',
 u'ParentQueueUrl': None,
 u'PercentComplete': 0,
 u'PercentSuccess': 0,
 u'PileupData': {u'/RelValMinBias_13/CMSSW_11_0_0_pre4-106X_upgrade2018_design_v5-v1/GEN-SIM': [u'T1_US_FNAL_Disk']},
 u'Priority': 500000,
 u'ProcessedInputs': [],
 u'RejectedInputs': [],
 u'RequestName': u'muahmad_RVCMSSW_11_0_0_pre5TTbar_13_PU25ns__190810_113803_761',
 u'SiteBlacklist': [],
 u'SiteWhitelist': [u'T1_US_FNAL'],
 u'StartPolicy': u'Block',
 u'Status': u'Available',
 u'SubscriptionId': None,
 u'TaskName': u'DigiFullPU_2018DesignPU',
 u'TeamName': u'relval',
 u'TimestampFoundNewData': 0,
 u'WMBSUrl': None}
```

I updated that element in place and replaced `T1_US_FNAL_Disk` by `T1_US_FNAL`. That change made the workflow to go through and get acquired by the agent.

Changed files:
src/python/WMCore/WorkQueue/Policy/Start/Dataset.py, src/python/WMQuality/Emulators/EmulatedUnitTestCase.py, src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py, src/python/WMCore/WorkQueue/Policy/Start/Block.py, src/python/WMCore/WorkQueue/Policy/Start/ResubmitBlock.py

diff --git a/src/python/WMCore/WorkQueue/Policy/Start/Block.py b/src/python/WMCore/WorkQueue/Policy/Start/Block.py
index fea249aae..71dd032cd 100644
--- a/src/python/WMCore/WorkQueue/Policy/Start/Block.py
+++ b/src/python/WMCore/WorkQueue/Policy/Start/Block.py
@@ -8,7 +8,6 @@ from __future__ import print_function, division
 import logging
 from math import ceil
 from WMCore.WorkQueue.Policy.Start.StartPolicyInterface import StartPolicyInterface
-from WMCore.Services.CRIC.CRIC import CRIC
 from WMCore.WorkQueue.WorkQueueExceptions import WorkQueueWMSpecError
 from WMCore.WorkQueue.WorkQueueUtils import makeLocationsList
 from WMCore import Lexicon
@@ -28,8 +27,6 @@ class Block(StartPolicyInterface):
 
         # Initialize modifiers of the policy
         self.blockBlackListModifier = []
-        self.cric = CRIC()
-
 
     def split(self):
         """Apply policy to spec"""
diff --git a/src/python/WMCore/WorkQueue/Policy/Start/Dataset.py b/src/python/WMCore/WorkQueue/Policy/Start/Dataset.py
index 91b29571d..a074fb613 100644
--- a/src/python/WMCore/WorkQueue/Policy/Start/Dataset.py
+++ b/src/python/WMCore/WorkQueue/Policy/Start/Dataset.py
@@ -11,7 +11,6 @@ make it generic enough that could be used by other spec types.
 import logging
 from math import ceil
 from WMCore import Lexicon
-from WMCore.Services.CRIC.CRIC import CRIC
 from WMCore.WorkQueue.Policy.Start.StartPolicyInterface import StartPolicyInterface
 from WMCore.WorkQueue.WorkQueueExceptions import WorkQueueWMSpecError
 from WMCore.WorkQueue.WorkQueueUtils import makeLocationsList
@@ -26,7 +25,6 @@ class Dataset(StartPolicyInterface):
         self.args.setdefault('SliceSize', 1)
         self.lumiType = "NumberOfLumis"
         self.sites = []
-        self.cric = CRIC()
 
     def split(self):
         """Apply policy to spec"""
diff --git a/src/python/WMCore/WorkQueue/Policy/Start/ResubmitBlock.py b/src/python/WMCore/WorkQueue/Policy/Start/ResubmitBlock.py
index cec6b9d16..2b6c8af93 100644
--- a/src/python/WMCore/WorkQueue/Policy/Start/ResubmitBlock.py
+++ b/src/python/WMCore/WorkQueue/Policy/Start/ResubmitBlock.py
@@ -20,7 +20,6 @@ ACDC unsupported:
 
 from math import ceil
 from WMCore.WorkQueue.Policy.Start.StartPolicyInterface import StartPolicyInterface
-from WMCore.Services.CRIC.CRIC import CRIC
 from WMCore.WorkQueue.WorkQueueExceptions import WorkQueueWMSpecError
 from WMCore.WorkQueue.WorkQueueUtils import makeLocationsList
 from WMCore.WorkQueue.DataStructs.ACDCBlock import ACDCBlock
@@ -47,7 +46,6 @@ class ResubmitBlock(StartPolicyInterface):
         self.unsupportedAlgos = ['WMBSMergeBySize', 'SiblingProcessingBased']
         self.defaultAlgo = self.fixedSizeChunk
         self.sites = []
-        self.cric = CRIC()
 
     def split(self):
         """Apply policy to spec"""
diff --git a/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py b/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py
index 8c238368c..c5cc3058f 100644
--- a/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py
+++ b/src/python/WMCore/WorkQueue/Policy/Start/StartPolicyInterface.py
@@ -10,6 +10,7 @@ from WMCore.WorkQueue.DataStructs.WorkQueueElement import WorkQueueElement
 from WMCore.DataStructs.LumiList import LumiList
 from WMCore.WorkQueue.WorkQueueExceptions import WorkQueueWMSpecError, WorkQueueNoWorkError
 from dbs.exceptions.dbsClientException import dbsClientException
+from WMCore.Services.CRIC.CRIC import CRIC
 from WMCore.Services.DBS.DBSErrors import DBSReaderError
 from WMCore import Lexicon
 
@@ -31,6 +32,7 @@ class StartPolicyInterface(PolicyInterface):
         self.rejectedWork = []  # List of inputs that were rejected
         self.badWork = []  # list of bad work unit (e.g. without any valid files)
         self.pileupData = {}
+        self.cric = CRIC()
 
     def split(self):
         """Apply policy to spec"""
@@ -238,5 +240,5 @@ class StartPolicyInterface(PolicyInterface):
             dbs = self.dbs(dbsUrl)
             for datasetPath in datasets[dbsUrl]:
                 locations = dbs.listDatasetLocation(datasetPath)
-                result[datasetPath] = locations
+                result[datasetPath] = self.cric.PNNstoPSNs(locations)
         return result
diff --git a/src/python/WMQuality/Emulators/EmulatedUnitTestCase.py b/src/python/WMQuality/Emulators/EmulatedUnitTestCase.py
index df8e0084e..abcc69f4e 100644
--- a/src/python/WMQuality/Emulators/EmulatedUnitTestCase.py
+++ b/src/python/WMQuality/Emulators/EmulatedUnitTestCase.py
@@ -116,9 +116,7 @@ class EmulatedUnitTestCase(unittest.TestCase):
             patchCRICAt = ['WMCore.ReqMgr.Tools.cms.CRIC',
                            'WMCore.WorkQueue.WorkQueue.CRIC',
                            'WMCore.WorkQueue.WorkQueueUtils.CRIC',
-                           'WMCore.WorkQueue.Policy.Start.Dataset.CRIC',
-                           'WMCore.WorkQueue.Policy.Start.ResubmitBlock.CRIC',
-                           'WMCore.WorkQueue.Policy.Start.Block.CRIC']
+                           'WMCore.WorkQueue.Policy.Start.StartPolicyInterface.CRIC']
             for module in patchCRICAt:
                 self.cricPatchers.append(mock.patch(module, new=MockCRICApi))
                 self.cricPatchers[-1].start()