Repository: dmwm/wmcore
Issue: DQMHarvesting workflow does not trigger a container-level input rule

**Impact of the bug**
ReqMgr2MS - MSTransferor

**Describe the bug**
When MSTransferor is dealing with input data for DQMHarvest workflows, it's supposed to make a container-level rule instead of scattering blocks around (or making N-blocks rules). DQMHarvest workflows have a `Dataset` workqueue start policy, which means the chunk of work is actually the whole dataset (rucio container).

Testing the latest tag 1.4.1.pre3 in my VM, this is the rule that it tries to create:
```
2020-09-29 22:24:35,669:ERROR:MSTransferor: Hit a bad exception while creating replication rules for DID: set([u'/NoBPTX/Run2016F-23Sep2016-v1/DQMIO#c386453a-c100-11e6-80ad-001e67abf094', u'/NoBPTX/Run2016F-23Sep2016-v1/DQMIO#c3d28f34-c106-11e6-9206-001e67abefa8'])
```

**How to reproduce it**
inject a DQMHarvest workflow and set MSTransferor to Rucio mode

**Expected behavior**
DQMHarvesting workflows should have input data placement done at the container level.
In addition to that, we should have a clear message on what the "bad exception" was.

**Additional context and error message**
Logs from MSTransferor:
```
2020-09-29 22:24:35,616:INFO:MSTransferor: Secondary data placement, picked RSE: T2_CH_CERN out of an RSE list: T2_CH_CERN
2020-09-29 22:24:35,616:INFO:MSTransferor: Creating a Rucio rule for workflow amaltaro_DQMHarvesting_RunWhitelist_Sept2020_Val_200929_215620_386, container /NoBPTX/Run2016F-23Sep2016-v1/DQMIO, RSEs: T2_CH_CERN, grouping: ALL
2020-09-29 22:24:35,666:DEBUG:connectionpool: http://cms-rucio.cern.ch:80 "POST /rules/ HTTP/1.1" 500 112
2020-09-29 22:24:35,669:ERROR:MSTransferor: Hit a bad exception while creating replication rules for DID: set([u'/NoBPTX/Run2016F-23Sep2016-v1/DQMIO#c386453a-c100-11e6-80ad-001e67abf094', u'/NoBPTX/Run2016F-23Sep2016-v1/DQMIO#c3d28f34-c106-11e6-9206-001e67abefa8'])
2020-09-29 22:24:35,669:WARNING:MSTransferor: There were failures transferring data for workflow: amaltaro_DQMHarvesting_RunWhitelist_Sept2020_Val_200929_215620_386. Will retry again later.
```

Changed files:
src/python/WMCore/MicroService/Unified/MSTransferor.py

diff --git a/src/python/WMCore/MicroService/Unified/MSTransferor.py b/src/python/WMCore/MicroService/Unified/MSTransferor.py
index fab2065bd..d592a8c6c 100644
--- a/src/python/WMCore/MicroService/Unified/MSTransferor.py
+++ b/src/python/WMCore/MicroService/Unified/MSTransferor.py
@@ -634,6 +634,7 @@ class MSTransferor(MSCore):
                     # enforce a dataset level PhEDEx subscription / container-level Rucio rule
                     subLevel = "dataset"
                     dataBlocks = None
+                    blocks = None
                 elif blocks:
                     subLevel = "block"
                     dataBlocks = {dataIn['name']: blocks}
@@ -641,6 +642,7 @@ class MSTransferor(MSCore):
                     # then it's a dataset level subscription
                     subLevel = "dataset"
                     dataBlocks = None
+                    blocks = None
 
                 if self.msConfig['useRucio']:
                     success, transferId = self.makeTransferRucio(wflow, dataIn, subLevel,
@@ -692,7 +694,7 @@ class MSTransferor(MSCore):
         #   "block" level in PhEDEx --> "DATASET" in Rucio (whole block at same location)
         success, transferId = True, set()
         subLevel = "ALL" if subLevel == "dataset" else "DATASET"
-        dids = blocks if blocks else dataIn['name']
+        dids = blocks if blocks else [dataIn['name']]
 
         rule = {'copies': 1,
                 'activity': 'Production Input',
@@ -707,14 +709,15 @@ class MSTransferor(MSCore):
             msg = "Primary data placement with parent blocks, putting all in the same RSE: {}".format(rseExpr)
             self.logger.info(msg)
         elif rule['grouping'] == "ALL":
-            # then it means we are doing the secondary data placement.
+            # this means we are placing the whole container under the same RSE.
             # Ask Rucio which RSE we should use, provided a list of them
             rseExpr = "|".join(nodes)
             rseTuple = self.rucio.pickRSE(rseExpr)
             if not rseTuple:
                 self.logger.error("PickRSE did not return any valid RSE for expression: %s", rseExpr)
                 return False, transferId
-            self.logger.info("Secondary data placement, picked RSE: %s out of an RSE list: %s", rseTuple[0], rseExpr)
+            self.logger.info("Placing whole container, picked RSE: %s out of an RSE list: %s",
+                             rseTuple[0], rseExpr)
             rseExpr = rseTuple[0]
         else:
             # then grouping is by DATASET, and there is no parent dataset
@@ -731,13 +734,13 @@ class MSTransferor(MSCore):
                                     dataSize > self.msConfig.get('warningTransferThreshold')
 
             # Then make the data subscription, for real!!!
-            self.logger.info("Creating a Rucio rule for workflow %s, container %s, RSEs: %s, grouping: %s",
-                             wflow.getName(), dataIn['name'], rseExpr, subLevel)
+            self.logger.info("Creating rule for workflow %s with %d DIDs in container %s, RSEs: %s, grouping: %s",
+                             wflow.getName(), len(dids), dataIn['name'], rseExpr, subLevel)
             try:
                 res = self.rucio.createReplicationRule(dids, rseExpr, **rule)
-            except Exception:
-                msg = "Hit a bad exception while creating replication rules for DID: %s"
-                self.logger.error(msg, dids)
+            except Exception as exc:
+                msg = "Hit a bad exception while creating replication rules for DID: %s. Error: %s"
+                self.logger.error(msg, dids, str(exc))
                 success = False
             else:
                 if res: