Repository: rucio/rucio

Issue:
The client suggests that an invalid DID has replicas on tape
Motivation
----------

Also, ‘No files will be download.’ requires a grammatical fix.

```
$ rucio -v get foo:bar
2021-03-25 12:27:31,093	INFO	Processing 1 item(s) for input
2021-03-25 12:27:31,094	DEBUG	downloadclient.py	num_unmerged_items=1; num_dids=1; num_merged_items=1
2021-03-25 12:27:31,094	INFO	Getting sources of DIDs
2021-03-25 12:27:31,122	DEBUG	downloadclient.py	schemes: None
2021-03-25 12:27:31,122	DEBUG	downloadclient.py	rse_expression: None
2021-03-25 12:27:31,122	DEBUG	downloadclient.py	num DIDs for list_replicas call: 1
2021-03-25 12:27:31,151	DEBUG	downloadclient.py	num resolved files: 0
2021-03-25 12:27:31,151	ERROR	DID does not exist: foo:bar
2021-03-25 12:27:31,151	WARNING	Requested did foo:bar has only replicas on tape. No files will be download.
2021-03-25 12:27:31,165	DEBUG	downloadclient.py	"unzip -v" returned with exitcode 0
2021-03-25 12:27:31,185	DEBUG	downloadclient.py	"tar --version" returned with exitcode 0
2021-03-25 12:27:31,185	DEBUG	downloadclient.py	Queueing file: foo:bar
2021-03-25 12:27:31,185	DEBUG	downloadclient.py	real parents: set([])
2021-03-25 12:27:31,186	DEBUG	downloadclient.py	options: {'foo:bar': {'ignore_checksum': False, 'transfer_timeout': 360, 'destinations': set([('.', False)])}}
2021-03-25 12:27:31,187	DEBUG	downloadclient.py	Prepared sources: num_sources=0/0; num_non_cea_sources=0; num_cea_ids=0
2021-03-25 12:27:31,188	INFO	Using main thread to download 1 file(s)
2021-03-25 12:27:31,188	DEBUG	downloadclient.py	Start processing queued downloads
2021-03-25 12:27:31,188	INFO	Preparing download of foo:bar
2021-03-25 12:27:31,189	WARNING	No available source found for file: foo:bar
2021-03-25 12:27:31,240	ERROR	None of the requested files have been downloaded.
```

Modification
------------





Discussion:
It’s equally as confusing when the dataset is empty.

```
$ rucio list-content mc16_13TeV.364196.Sherpa_221_NNPDF30NNLO_Wtaunu_MAXHTPTV500_1000.deriv.DAOD_HIGG4D2.e5340_s3126_r10201_r10210_p3585_tid14607441_00
+--------------+--------------+
| SCOPE:NAME   | [DID TYPE]   |
|--------------+--------------|
+--------------+--------------+
$ rucio download mc16_13TeV.364196.Sherpa_221_NNPDF30NNLO_Wtaunu_MAXHTPTV500_1000.deriv.DAOD_HIGG4D2.e5340_s3126_r10201_r10210_p3585_tid14607441_00
2021-04-06 20:17:48,998 ERROR   DID does not exist: mc16_13TeV:mc16_13TeV.364196.Sherpa_221_NNPDF30NNLO_Wtaunu_MAXHTPTV500_1000.deriv.DAOD_HIGG4D2.e5340_s3126_r10201_r10210_p3585_tid14607441_00
2021-04-06 20:17:48,999 WARNING Requested did mc16_13TeV:mc16_13TeV.364196.Sherpa_221_NNPDF30NNLO_Wtaunu_MAXHTPTV500_1000.deriv.DAOD_HIGG4D2.e5340_s3126_r10201_r10210_p3585_tid14607441_00 has only replicas on tape. No files will be download.
2021-04-06 20:17:49,016 WARNING No available source found for file: mc16_13TeV:mc16_13TeV.364196.Sherpa_221_NNPDF30NNLO_Wtaunu_MAXHTPTV500_1000.deriv.DAOD_HIGG4D2.e5340_s3126_r10201_r10210_p3585_tid14607441_00
2021-04-06 20:17:49,284 ERROR   None of the requested files have been downloaded.
```

Resolving patch:
diff --git a/lib/rucio/client/downloadclient.py b/lib/rucio/client/downloadclient.py
--- a/lib/rucio/client/downloadclient.py
+++ b/lib/rucio/client/downloadclient.py
@@ -1152,12 +1152,13 @@ def _get_sources(self, merged_items, resolve_archives=True):
             # filtering out tape sources
             if self.is_tape_excluded:
                 for file_item in file_items:
-                    sources = file_item['sources']
-                    for src in file_item['sources']:
+                    unfiltered_sources = copy.copy(file_item['sources'])
+                    for src in unfiltered_sources:
                         if src in tape_rses:
-                            sources.remove(src)
-                    if not sources:
-                        logger(logging.WARNING, 'Requested did {} has only replicas on tape. No files will be download.'.format(file_item['did']))
+                            file_item['sources'].remove(src)
+                    if unfiltered_sources and not file_item['sources']:
+                        logger(logging.WARNING, 'The requested DID {} only has replicas on tape. Direct download from tape is prohibited. '
+                                                'Please request a transfer to a non-tape endpoint.'.format(file_item['did']))
 
             nrandom = item.get('nrandom')
             if nrandom:


Test patch: