anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: rucio/rucio
Issue: Unable to create new meta key with string type

Motivation
----------
If one attempts to make a DID key with a string value type:
```python
client.add_key(
    key="asdf",
    key_type="COLLECTION",
    value_type=str,
    value_regexp="\\w{,50}",
)
```
then an exception is raised:
```
rucio.common.exception.UnsupportedValueType: Unsupported type for the value. List of supported types: (<type 'float'>, <type 'int'>, (<type 'basestring'>,)).
Details: The type '<type 'str'>' is not supported for values!
```

Proposal
---------
This is due to https://github.com/rucio/rucio/blob/master/lib/rucio/common/constraints.py#L25 where `six.string_types` is a tuple and should be added rather than nested. But that is not enough, as depending on whether you are in python2 or python3, different types will be valid. Really this should be an enum.

Note: tested in v1.22.2

Changed files:
lib/rucio/common/constraints.py

diff --git a/lib/rucio/common/constraints.py b/lib/rucio/common/constraints.py
index 7ca02e470..95988f5bc 100644
--- a/lib/rucio/common/constraints.py
+++ b/lib/rucio/common/constraints.py
@@ -1,4 +1,4 @@
-# Copyright 2012-2018 CERN for the benefit of the ATLAS collaboration.
+# Copyright 2012-2020 CERN for the benefit of the ATLAS collaboration.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -16,11 +16,12 @@
 # - Vincent Garonne <vgaronne@gmail.com>, 2012-2018
 # - Frank Berghaus <frank.berghaus@cern.ch>, 2018
 # - Hannes Hansen <hannes.jakob.hansen@cern.ch>, 2019
+# - Martin Barisits <martin.barisits@cern.ch>, 2020
 #
 # PY3K COMPATIBLE
 
 from six import string_types
 
 # List of authorized value types for key
-AUTHORIZED_VALUE_TYPES = (float, int, string_types)
+AUTHORIZED_VALUE_TYPES = (float, int) + string_types
 STRING_TYPES = string_types
Repository: rucio/rucio
Issue: Suspicious file reporting broken in Kronos

Motivation
----------
The reporting of suspicious files in Kronos is broken


Changed files:
lib/rucio/daemons/tracer/kronos.py

diff --git a/lib/rucio/daemons/tracer/kronos.py b/lib/rucio/daemons/tracer/kronos.py
index c5357ac4b..6fe45c751 100644
--- a/lib/rucio/daemons/tracer/kronos.py
+++ b/lib/rucio/daemons/tracer/kronos.py
@@ -143,11 +143,15 @@ class AMQConsumer(object):
                     for pattern in self.__bad_files_patterns:
                         if 'stateReason' in report and pattern.match(report['stateReason']):
                             reason = report['stateReason'][:255]
-                            try:
-                                declare_bad_file_replicas(report['url'], reason=reason, issuer='root', status=BadFilesStatus.SUSPICIOUS)
-                                logging.info('Declare suspicious file %s with reason %s' % (report['url'], reason))
-                            except Exception as error:
-                                logging.error('Failed to declare suspicious file' + str(error))
+                            if 'url' not in report or not report['url']:
+                                logging.error('Missing url in the following trace : ' + str(report))
+                            else:
+                                try:
+                                    surl = report['url']
+                                    declare_bad_file_replicas([surl, ], reason=reason, issuer='root', status=BadFilesStatus.SUSPICIOUS)
+                                    logging.info('Declare suspicious file %s with reason %s' % (report['url'], reason))
+                                except Exception as error:
+                                    logging.error('Failed to declare suspicious file' + str(error))
 
                 # check if scope in report. if not skip this one.
                 if 'scope' not in report:
Repository: rucio/rucio
Issue: Documentation build fails

Motivation
----------
Currently the documentation builds fail with 

```
Running Sphinx v1.6.5
making output directory...
loading translations [en]... done

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/rucio/envs/latest/local/lib/python2.7/site-packages/sphinx/cmdline.py", line 305, in main
    opts.warningiserror, opts.tags, opts.verbosity, opts.jobs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/rucio/envs/latest/local/lib/python2.7/site-packages/sphinx/application.py", line 196, in __init__
    self.setup_extension(extension)
  File "/home/docs/checkouts/readthedocs.org/user_builds/rucio/envs/latest/local/lib/python2.7/site-packages/sphinx/application.py", line 456, in setup_extension
    self.registry.load_extension(self, extname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/rucio/envs/latest/local/lib/python2.7/site-packages/sphinx/registry.py", line 199, in load_extension
    raise ExtensionError(__('Could not import extension %s') % extname, err)
ExtensionError: Could not import extension rucio.common.doc.argparse.ext (exception: No module named rucio.common.doc.argparse.ext)

Extension error:
Could not import extension rucio.common.doc.argparse.ext (exception: No module named rucio.common.doc.argparse.ext)
```

I did not look too much into the issue yet, but I wonder why we added this `rucio.common.doc.argparse.ext` to the repository instead of using `sphinx-argparse`? @vingar do you maybe remember?

Changed files:
doc/source/conf.py

diff --git a/doc/source/conf.py b/doc/source/conf.py
index 60e19bdfa..99bdb9c1f 100644
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -20,7 +20,7 @@ import os
 import sys
 from mock import Mock as MagicMock
 
-sys.path.insert(len(sys.path) - 1, os.path.abspath('.'))
+sys.path.insert(len(sys.path), os.path.abspath('../../lib'))
 
 
 class Mock(MagicMock):
Repository: rucio/rucio
Issue: core: rse listing needs additional handling of legacy 0/1 values

Motivation
----------

Existing RSE attributes which use 0/1 as true/false are not found if they are the sole solution to an RSE expression.

Changed files:
lib/rucio/core/rse.py

diff --git a/lib/rucio/core/rse.py b/lib/rucio/core/rse.py
index 71d69bcf4..d132c9ebb 100644
--- a/lib/rucio/core/rse.py
+++ b/lib/rucio/core/rse.py
@@ -15,7 +15,7 @@
 # Authors:
 # - Vincent Garonne <vgaronne@gmail.com>, 2012-2018
 # - Ralph Vigne <ralph.vigne@cern.ch>, 2012-2015
-# - Mario Lassnig <mario.lassnig@cern.ch>, 2012-2018
+# - Mario Lassnig <mario.lassnig@cern.ch>, 2012-2019
 # - Martin Barisits <martin.barisits@cern.ch>, 2013-2018
 # - Cedric Serfon <cedric.serfon@cern.ch>, 2013-2018
 # - Thomas Beermann <thomas.beermann@cern.ch>, 2014-2017
@@ -302,8 +302,13 @@ def list_rses(filters={}, session=None):
                 # ATLAS RSE listing workaround (since booleans are capital 'True'/'False')
                 # remove elif branch after appropriate database fix has been applied
                 # see also db/types.py
-                query = query.filter(or_(t.value == v,
-                                         t.value == 'tmp_atlas_%s' % v))
+                if isinstance(v, bool):
+                    query = query.filter(or_(t.value == v,
+                                             t.value == 'tmp_atlas_%s' % v,
+                                             t.value == 'tmp_atlas_%s' % 1 if v else 0))
+                else:
+                    query = query.filter(or_(t.value == v,
+                                             t.value == 'tmp_atlas_%s' % v))
 
         condition1, condition2 = [], []
         for i in range(0, 8):
Repository: rucio/rucio
Issue: Release: Alembic upgrade script for rule notification references wrong column

Motivation
----------
The upgrade script should be fixed in a hotfix release `1.19.0.post1` and the `1.19.0` release should be pulled from PyPi. This only impacts database upgrades from an older version.

Changed files:
lib/rucio/db/sqla/migrate_repo/versions/01eaf73ab656_add_new_rule_notification_state_progress.py

diff --git a/lib/rucio/db/sqla/migrate_repo/versions/01eaf73ab656_add_new_rule_notification_state_progress.py b/lib/rucio/db/sqla/migrate_repo/versions/01eaf73ab656_add_new_rule_notification_state_progress.py
index 45c19452a..da93d7a01 100644
--- a/lib/rucio/db/sqla/migrate_repo/versions/01eaf73ab656_add_new_rule_notification_state_progress.py
+++ b/lib/rucio/db/sqla/migrate_repo/versions/01eaf73ab656_add_new_rule_notification_state_progress.py
@@ -36,7 +36,7 @@ def upgrade():
     if context.get_context().dialect.name not in ('sqlite'):  # pylint: disable=no-member
         drop_constraint('RULES_NOTIFICATION_CHK', 'rules', type_='check')
         create_check_constraint(name='RULES_NOTIFICATION_CHK', source='rules',
-                                condition="state in ('Y', 'N', 'C', 'P')")
+                                condition="notification in ('Y', 'N', 'C', 'P')")
 
 
 def downgrade():
@@ -46,4 +46,4 @@ def downgrade():
     if context.get_context().dialect.name not in ('sqlite'):  # pylint: disable=no-member
         drop_constraint('RULES_NOTIFICATION_CHK', 'rules', type_='check')
         create_check_constraint(name='RULES_NOTIFICATION_CHK', source='rules',
-                                condition="state in ('Y', 'N', 'C')")
+                                condition="notification in ('Y', 'N', 'C')")
Repository: rucio/rucio
Issue: Dark Reaper rse_id crash

Motivation
----------
The dark reaper raises an error in 1.20.3


Modification
------------
Fix the query in core/quarantined_replica.py -> list_rses()



Changed files:
lib/rucio/core/quarantined_replica.py

diff --git a/lib/rucio/core/quarantined_replica.py b/lib/rucio/core/quarantined_replica.py
index 749ae77cb..29f52e067 100644
--- a/lib/rucio/core/quarantined_replica.py
+++ b/lib/rucio/core/quarantined_replica.py
@@ -161,7 +161,7 @@ def list_rses(session=None):
 
     :returns: a list of RSEs.
     """
-    query = session.query(models.RSE.rse_id).distinct(models.RSE.rse_id).\\
+    query = session.query(models.RSE.id).distinct(models.RSE.id).\\
         filter(models.QuarantinedReplica.rse_id == models.RSE.id).\\
         filter(models.RSE.deleted == false())
     return [rse for (rse,) in query]
Repository: rucio/rucio
Issue: delete_replica query on archive_contents uses too many bind variables

Motivation
----------
In the `delete_replicas` method the archive_contents clause uses too many bind variables (700).


Modification
------------
Reduce the chunking from `100` to `30`



Changed files:
lib/rucio/core/replica.py

diff --git a/lib/rucio/core/replica.py b/lib/rucio/core/replica.py
index f4dafed9b..0572d0146 100644
--- a/lib/rucio/core/replica.py
+++ b/lib/rucio/core/replica.py
@@ -1545,7 +1545,7 @@ def delete_replicas(rse_id, files, ignore_availability=True, session=None):
                                      models.DataIdentifier.name == name))
 
     # Remove Archive Constituents
-    for chunk in chunks(archive_contents_condition, 100):
+    for chunk in chunks(archive_contents_condition, 30):
         session.query(models.ConstituentAssociation).\\
             with_hint(models.ConstituentAssociation, "INDEX(ARCHIVE_CONTENTS ARCH_CONTENTS_CHILD_IDX)", 'oracle').\\
             filter(or_(*chunk)).\\
Repository: rucio/rucio
Issue: Output formatting typo in `rse/protocols/gfal.py`

 ##Motivation
----------
Previous PR #2555 introduced this bug related to a missing parentheses at the formatting of a debug output string here: https://github.com/rucio/rucio/blob/916987976a1f994a64e4ef3dce837b49141a8f1b/lib/rucio/rse/protocols/gfal.py#L363.


Modification
------------
Add the parentheses surrounding the formatting elements.



Changed files:
lib/rucio/rse/protocols/gfal.py

diff --git a/lib/rucio/rse/protocols/gfal.py b/lib/rucio/rse/protocols/gfal.py
index d5ce34c36..18fa80b99 100644
--- a/lib/rucio/rse/protocols/gfal.py
+++ b/lib/rucio/rse/protocols/gfal.py
@@ -360,7 +360,7 @@ class Default(protocol.RSEProtocol):
                 ret[checksum_name] = ctx.checksum(str(path), str(checksum_name.capitalize()))
                 verified = True
             except Exception as error:
-                message += 'Error while processing gfal checksum call (%s). Error: %s \\n' % checksum_name, str(error)
+                message += 'Error while processing gfal checksum call (%s). Error: %s \\n' % (checksum_name, str(error))
 
         if not verified:
             raise exception.RSEChecksumUnavailable(message)
Repository: rucio/rucio
Issue: Dark Reaper rse_id crash

Motivation
----------
The dark reaper raises an error in 1.20.3


Modification
------------
Fix the query in core/quarantined_replica.py -> list_rses()



Changed files:
lib/rucio/rse/protocols/gfal.py, lib/rucio/core/quarantined_replica.py

diff --git a/lib/rucio/core/quarantined_replica.py b/lib/rucio/core/quarantined_replica.py
index 749ae77cb..29f52e067 100644
--- a/lib/rucio/core/quarantined_replica.py
+++ b/lib/rucio/core/quarantined_replica.py
@@ -161,7 +161,7 @@ def list_rses(session=None):
 
     :returns: a list of RSEs.
     """
-    query = session.query(models.RSE.rse_id).distinct(models.RSE.rse_id).\\
+    query = session.query(models.RSE.id).distinct(models.RSE.id).\\
         filter(models.QuarantinedReplica.rse_id == models.RSE.id).\\
         filter(models.RSE.deleted == false())
     return [rse for (rse,) in query]
diff --git a/lib/rucio/rse/protocols/gfal.py b/lib/rucio/rse/protocols/gfal.py
index c2a835b8b..d8056e4d7 100644
--- a/lib/rucio/rse/protocols/gfal.py
+++ b/lib/rucio/rse/protocols/gfal.py
@@ -394,7 +394,7 @@ class Default(protocol.RSEProtocol):
             params.timeout = int(transfer_timeout)
             watchdog = Timer(params.timeout + 60, self.__gfal2_cancel)
 
-        if not self.renaming and dest[:5] == 'https':
+        if not (self.renaming and dest[:5] == 'https'):
             dir_name = os.path.dirname(dest)
             # This function will be removed soon. gfal2 will create parent dir automatically.
             try:
Repository: rucio/rucio
Issue: Broken backwards compatibility for account usage and limits

Motivation
----------
Due to the change from simple limits/usage to local/global usage and limits, the backwards compatibility for old clients is currently broken. This is due to the fact that the REST endpoints where changed from
```
    '/(.+)/limits', 'AccountLimits',
    '/(.+)/limits/(.+)', 'AccountLimits',
    '/(.+)/usage/', 'Usage1',
    '/(.+)/usage/(.+)', 'Usage2',
```
to
```
    '/(.+)/limits/local', 'LocalAccountLimits',
    '/(.+)/limits/local/(.+)', 'LocalAccountLimits',
    '/(.+)/limits/global', 'GlobalAccountLimits',
    '/(.+)/limits/global/(.+)', 'GlobalAccountLimits',
```
thus the required endpoints are missing for old clients. 

Modification
------------
The endpoints need to be re-added and pointing to the local version of the usage/limit.


Changed files:
lib/rucio/web/rest/webpy/v1/account.py

diff --git a/lib/rucio/web/rest/webpy/v1/account.py b/lib/rucio/web/rest/webpy/v1/account.py
index e97acbdd4..5f41f2778 100755
--- a/lib/rucio/web/rest/webpy/v1/account.py
+++ b/lib/rucio/web/rest/webpy/v1/account.py
@@ -61,12 +61,16 @@ URLS = (
     '/(.+)/limits/local/(.+)', 'LocalAccountLimits',
     '/(.+)/limits/global', 'GlobalAccountLimits',
     '/(.+)/limits/global/(.+)', 'GlobalAccountLimits',
+    '/(.+)/limits', 'LocalAccountLimits',
+    '/(.+)/limits/(.+)', 'LocalAccountLimits',
     '/(.+)/rules', 'Rules',
     '/(.+)/usage/history/(.+)', 'UsageHistory',
     '/(.+)/usage/local', 'LocalUsage',
     '/(.+)/usage/local/(.+)', 'LocalUsage',
     '/(.+)/usage/global', 'GlobalUsage',
     '/(.+)/usage/global/(.+)', 'GlobalUsage',
+    '/(.+)/usage', 'LocalUsage',
+    '/(.+)/usage/(.+)', 'LocalUsage',
     '/(.+)', 'AccountParameter',
     '/?$', 'Account',
 )
Repository: rucio/rucio
Issue: Permission check for ddmadmin done wrong

Motivation
----------
In some of the permission checks we do
`if _is_root(issuer) or issuer == 'ddmadmin':`
which does not work anymore since we introduced the InternalAccount for the issuers.

Modification
------------
Change to
`if _is_root(issuer) or issuer.external == 'ddmadmin':`
or add ddmadmin to the `_is_root` check



Changed files:
lib/rucio/core/permission/atlas.py

diff --git a/lib/rucio/core/permission/atlas.py b/lib/rucio/core/permission/atlas.py
index 8972e2507..64b8450eb 100644
--- a/lib/rucio/core/permission/atlas.py
+++ b/lib/rucio/core/permission/atlas.py
@@ -14,7 +14,7 @@
 #
 # Authors:
 # - Vincent Garonne <vgaronne@gmail.com>, 2016
-# - Martin Barisits <martin.barisits@cern.ch>, 2016-2018
+# - Martin Barisits <martin.barisits@cern.ch>, 2016-2019
 # - Cedric Serfon <cedric.serfon@cern.ch>, 2016-2019
 # - Mario Lassnig <mario.lassnig@cern.ch>, 2018
 # - Hannes Hansen <hannes.jakob.hansen@cern.ch>, 2018-2019
@@ -115,7 +115,7 @@ def has_permission(issuer, action, kwargs):
 
 
 def _is_root(issuer):
-    return issuer.external == 'root'
+    return issuer.external == 'root' or issuer.external == 'ddmadmin'
 
 
 def perm_default(issuer, kwargs):
@@ -448,7 +448,7 @@ def perm_del_rule(issuer, kwargs):
     :param kwargs: List of arguments for the action.
     :returns: True if account is allowed to call the API call, otherwise False
     """
-    if _is_root(issuer) or issuer == 'ddmadmin':
+    if _is_root(issuer):
         return True
     if get_rule(kwargs['rule_id'])['account'] == issuer:
         return True
Repository: rucio/rucio
Issue: Improperly Raising Exception in FTS3 Transfertool

Motivation
----------
File "/usr/lib/python2.7/site-packages/rucio/transfertool/fts3.py", line 221, in submit
raise('No transfer id returned by %s' % self.external_host)
TypeError: exceptions must be old-style classes or derived from BaseException, not unicode


Modification
------------
Needs to raise an actual exception class.



Changed files:
lib/rucio/transfertool/fts3.py

diff --git a/lib/rucio/transfertool/fts3.py b/lib/rucio/transfertool/fts3.py
index 1463477f5..a9c9c6047 100644
--- a/lib/rucio/transfertool/fts3.py
+++ b/lib/rucio/transfertool/fts3.py
@@ -218,7 +218,7 @@ class FTS3Transfertool(Transfertool):
             record_counter('transfertool.fts3.%s.submission.failure' % self.__extract_host(self.external_host), len(files))
 
         if not transfer_id:
-            raise('No transfer id returned by %s' % self.external_host)
+            raise TransferToolWrongAnswer('No transfer id returned by %s' % self.external_host)
         return transfer_id
 
     def cancel(self, transfer_ids, timeout=None):
Repository: rucio/rucio
Issue: declare_bad_file_replicas broken

Motivation
----------
`declare_bad_file_replicas` is broken after the move to `InternalAccount`


Changed files:
lib/rucio/daemons/conveyor/finisher.py

diff --git a/lib/rucio/daemons/conveyor/finisher.py b/lib/rucio/daemons/conveyor/finisher.py
index 2874ae232..f1f7096f5 100644
--- a/lib/rucio/daemons/conveyor/finisher.py
+++ b/lib/rucio/daemons/conveyor/finisher.py
@@ -17,7 +17,7 @@
 # - Mario Lassnig <mario.lassnig@cern.ch>, 2015
 # - Vincent Garonne <vgaronne@gmail.com>, 2015-2018
 # - Martin Barisits <martin.barisits@cern.ch>, 2015-2019
-# - Cedric Serfon <cedric.serfon@cern.ch>, 2017-2018
+# - Cedric Serfon <cedric.serfon@cern.ch>, 2017-2019
 # - Hannes Hansen <hannes.jakob.hansen@cern.ch>, 2018
 # - Andrew Lister <andrew.lister@stfc.ac.uk>, 2019
 #
@@ -49,6 +49,7 @@ from dogpile.cache.api import NoValue
 from sqlalchemy.exc import DatabaseError
 
 from rucio.common.config import config_get
+from rucio.common.types import InternalAccount
 from rucio.common.utils import chunks
 from rucio.common.exception import DatabaseException, ConfigNotFound, UnsupportedOperation, ReplicaNotFound, RequestNotFound
 from rucio.core import request as request_core, heartbeat, replica as replica_core
@@ -351,7 +352,7 @@ def __check_suspicious_files(req, suspicious_patterns):
                     pfns.append(url['url'])
                 if pfns:
                     logging.debug("Found suspicious urls: %s", str(pfns))
-                    replica_core.declare_bad_file_replicas(pfns, reason=reason, issuer='root', status=BadFilesStatus.SUSPICIOUS)
+                    replica_core.declare_bad_file_replicas(pfns, reason=reason, issuer=InternalAccount('root'), status=BadFilesStatus.SUSPICIOUS)
     except Exception as error:
         logging.warning("Failed to check suspicious file with request: %s - %s", req['request_id'], str(error))
     return is_suspicious
@@ -444,7 +445,7 @@ def __update_replica(replica, session=None):
                                          replica['name'],
                                          replica['bytes'],
                                          pfn=replica['pfn'] if 'pfn' in replica else None,
-                                         account='root',  # it will deleted immediately, do we need to get the accurate account from rule?
+                                         account=InternalAccount('root'),  # it will deleted immediately, do we need to get the accurate account from rule?
                                          adler32=replica['adler32'],
                                          tombstone=datetime.datetime.utcnow(),
                                          session=session)
Repository: rucio/rucio
Issue: protocol_stat_retries in rsemanager is loaded as string instead of int

Motivation
----------
As it is loaded as string instead of int, the method https://github.com/rucio/rucio/blob/fffbe0919f179467f3e11cb4bcb3896906755122/lib/rucio/rse/rsemanager.py#L729 crashes on line 738.

Modification
------------
config_get_int should be used here, however, this method needs to be expanded with the raise_Exception and default options similar to config_get and config_get_bool


Changed files:
lib/rucio/common/config.py, lib/rucio/rse/rsemanager.py

diff --git a/lib/rucio/common/config.py b/lib/rucio/common/config.py
index e6cddc76d..43a045732 100644
--- a/lib/rucio/common/config.py
+++ b/lib/rucio/common/config.py
@@ -80,14 +80,24 @@ def config_add_section(section):
     return __CONFIG.add_section(section)
 
 
-def config_get_int(section, option):
+def config_get_int(section, option, raise_exception=True, default=None):
     """Return the integer value for a given option in a section"""
-    return __CONFIG.getint(section, option)
+    try:
+        return __CONFIG.getint(section, option)
+    except (ConfigParser.NoOptionError, ConfigParser.NoSectionError) as err:
+        if raise_exception and default is None:
+            raise err
+        return default
 
 
-def config_get_float(section, option):
+def config_get_float(section, option, raise_exception=True, default=None):
     """Return the floating point value for a given option in a section"""
-    return __CONFIG.getfloat(section, option)
+    try:
+        return __CONFIG.getfloat(section, option)
+    except (ConfigParser.NoOptionError, ConfigParser.NoSectionError) as err:
+        if raise_exception and default is None:
+            raise err
+        return default
 
 
 def config_get_bool(section, option, raise_exception=True, default=None):
diff --git a/lib/rucio/rse/rsemanager.py b/lib/rucio/rse/rsemanager.py
index 77c3f24c1..650cbf2cb 100644
--- a/lib/rucio/rse/rsemanager.py
+++ b/lib/rucio/rse/rsemanager.py
@@ -42,7 +42,7 @@ except ImportError:
     from urllib.parse import urlparse
 
 from rucio.common import exception, utils, constants
-from rucio.common.config import config_get
+from rucio.common.config import config_get_int
 from rucio.common.constraints import STRING_TYPES
 from rucio.common.utils import make_valid_did
 
@@ -733,7 +733,7 @@ def _retry_protocol_stat(protocol, pfn):
     :param protocol     The protocol to use to reach this file
     :param pfn          Physical file name of the target for the protocol stat
     """
-    retries = config_get('client', 'protocol_stat_retries', raise_exception=False, default=6)
+    retries = config_get_int('client', 'protocol_stat_retries', raise_exception=False, default=6)
 
     for attempt in range(retries):
         try:
Repository: rucio/rucio
Issue: Backwards compatibility for account usage and limits broken

Motivation
----------
The 1.21 release introduced local and global quotas and broke backwards compatibility to the old clients.



Changed files:
lib/rucio/daemons/conveyor/finisher.py, lib/rucio/web/rest/webpy/v1/account_limit.py, lib/rucio/web/rest/webpy/v1/account.py

diff --git a/lib/rucio/daemons/conveyor/finisher.py b/lib/rucio/daemons/conveyor/finisher.py
index 0a69bcde7..c8fdce9fc 100644
--- a/lib/rucio/daemons/conveyor/finisher.py
+++ b/lib/rucio/daemons/conveyor/finisher.py
@@ -408,7 +408,7 @@ def __update_bulk_replicas(replicas, session=None):
     :returns commit_or_rollback:  Boolean.
     """
     try:
-        replica_core.update_replicas_states(replicas, nowait=True, add_tombstone=True, session=session)
+        replica_core.update_replicas_states(replicas, nowait=True, add_tombstone=False, session=session)  # TODO: add_tombstone needs to be changed to True once set_tombstone is fixed
     except ReplicaNotFound as error:
         logging.warn('Failed to bulk update replicas, will do it one by one: %s', str(error))
         raise ReplicaNotFound(error)
@@ -432,7 +432,7 @@ def __update_replica(replica, session=None):
     """
 
     try:
-        replica_core.update_replicas_states([replica], nowait=True, add_tombstone=True, session=session)
+        replica_core.update_replicas_states([replica], nowait=True, add_tombstone=False, session=session)  # TODO: add_tombstone needs to be changed to True once set_tombstone is fixed
         if not replica['archived']:
             request_core.archive_request(replica['request_id'], session=session)
         logging.info("HANDLED REQUEST %s DID %s:%s AT RSE %s STATE %s", replica['request_id'], replica['scope'], replica['name'], replica['rse_id'], str(replica['state']))
diff --git a/lib/rucio/web/rest/webpy/v1/account.py b/lib/rucio/web/rest/webpy/v1/account.py
index 5f41f2778..d307176df 100755
--- a/lib/rucio/web/rest/webpy/v1/account.py
+++ b/lib/rucio/web/rest/webpy/v1/account.py
@@ -69,7 +69,7 @@ URLS = (
     '/(.+)/usage/local/(.+)', 'LocalUsage',
     '/(.+)/usage/global', 'GlobalUsage',
     '/(.+)/usage/global/(.+)', 'GlobalUsage',
-    '/(.+)/usage', 'LocalUsage',
+    '/(.+)/usage/', 'LocalUsage',
     '/(.+)/usage/(.+)', 'LocalUsage',
     '/(.+)', 'AccountParameter',
     '/?$', 'Account',
diff --git a/lib/rucio/web/rest/webpy/v1/account_limit.py b/lib/rucio/web/rest/webpy/v1/account_limit.py
index 45c2b8a6d..e61384eeb 100755
--- a/lib/rucio/web/rest/webpy/v1/account_limit.py
+++ b/lib/rucio/web/rest/webpy/v1/account_limit.py
@@ -14,7 +14,7 @@
 # limitations under the License.
 #
 # Authors:
-# - Martin Barisits <martin.barisits@cern.ch>, 2014
+# - Martin Barisits <martin.barisits@cern.ch>, 2014-2019
 # - Vincent Garonne <vincent.garonne@cern.ch>, 2017
 # - Mario Lassnig <mario.lassnig@cern.ch>, 2018
 # - Hannes Hansen <hannes.jakob.hansen@cern.ch>, 2018-2019
@@ -41,6 +41,7 @@ LOGGER.addHandler(SH)
 URLS = (
     '/local/(.+)/(.+)', 'LocalAccountLimit',
     '/global/(.+)/(.+)', 'GlobalAccountLimit',
+    '/(.+)/(.+)', 'LocalAccountLimit',
 )
Repository: rucio/rucio
Issue: Partitioning algorithm wrong for thread_id=0

Motivation
----------
The partitioning algorithm is not working correctly for `thread_id=0`. It then wrongly goes to a mode assuming there are no partitions.


Changed files:
lib/rucio/db/sqla/__init__.py

diff --git a/lib/rucio/db/sqla/__init__.py b/lib/rucio/db/sqla/__init__.py
index 4a0df8e60..d86c16af7 100644
--- a/lib/rucio/db/sqla/__init__.py
+++ b/lib/rucio/db/sqla/__init__.py
@@ -9,7 +9,7 @@ from sqlalchemy.sql.expression import bindparam, text
 
 def filter_thread_work(session, query, total_threads, thread_id, hash_variable=None):
     """ Filters a query to partition thread workloads based on the thread id and total number of threads """
-    if thread_id and total_threads and (total_threads - 1) > 0:
+    if thread_id is not None and total_threads is not None and (total_threads - 1) > 0:
         if session.bind.dialect.name == 'oracle':
             bindparams = [bindparam('thread_id', thread_id), bindparam('total_threads', total_threads - 1)]
             if not hash_variable:
Repository: rucio/rucio
Issue: Bug in archive content history

Motivation
----------
Traceback
```
2020-03-05 13:38:04,692 11088   CRITICAL        Thread [33/220] :  Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/rucio/daemons/reaper/reaper2.py", line 518, in reaper
    delete_replicas(rse_id=rse_id, files=deleted_files)
  File "/usr/lib/python2.7/site-packages/rucio/db/sqla/session.py", line 374, in new_funct
    raise DatabaseException(str(error))
DatabaseException: Database exception.
Details: (cx_Oracle.IntegrityError) ORA-02292: integrity constraint (ATLAS_RUCIO.DATASET_LOCKS_DID_FK) violated - child record found
```





Changed files:
lib/rucio/core/did.py

diff --git a/lib/rucio/core/did.py b/lib/rucio/core/did.py
index 326ccd6a6..cc425a66e 100644
--- a/lib/rucio/core/did.py
+++ b/lib/rucio/core/did.py
@@ -2005,6 +2005,7 @@ def insert_content_history(content_clause, did_created_at, session=None):
     :param did_created_at: Creation date of the did
     :param session: The database session in use.
     """
+    new_did_created_at = did_created_at
     query = session.query(models.DataIdentifierAssociation.scope,
                           models.DataIdentifierAssociation.name,
                           models.DataIdentifierAssociation.child_scope,
@@ -2022,6 +2023,8 @@ def insert_content_history(content_clause, did_created_at, session=None):
         filter(or_(*content_clause))
 
     for cont in query.all():
+        if not did_created_at:
+            new_did_created_at = cont.created_at
         models.DataIdentifierAssociationHistory(
             scope=cont.scope,
             name=cont.name,
@@ -2037,6 +2040,6 @@ def insert_content_history(content_clause, did_created_at, session=None):
             rule_evaluation=cont.rule_evaluation,
             updated_at=cont.updated_at,
             created_at=cont.created_at,
-            did_created_at=did_created_at,
+            did_created_at=new_did_created_at,
             deleted_at=datetime.utcnow()
         ).save(session=session, flush=False)
Repository: rucio/rucio
Issue: Problems with get_hops

Motivation
----------
Debugging with Katy her attempt to multi hop, we noticed that while there is a check in non-multihopping code which ignores distances set to None, there is no check in the multi hop link finder. We're now running a version which ignores such links in its test.

I also noticed the function is using [] as a default argument, which is a python no-no.

I can make a PR that fixes both issues. Hopefully after we get MH working.


Modification
------------




Changed files:
lib/rucio/core/transfer.py

diff --git a/lib/rucio/core/transfer.py b/lib/rucio/core/transfer.py
index 1c604bf1b..2cf0c22c8 100644
--- a/lib/rucio/core/transfer.py
+++ b/lib/rucio/core/transfer.py
@@ -455,7 +455,7 @@ def update_transfer_state(external_host, transfer_id, state, logging_prepend_str
 
 
 @transactional_session
-def get_hops(source_rse_id, dest_rse_id, include_multihop=False, multihop_rses=[], session=None):
+def get_hops(source_rse_id, dest_rse_id, include_multihop=False, multihop_rses=None, session=None):
     """
     Get a list of hops needed to transfer date from source_rse_id to dest_rse_id.
     Ideally, the list will only include one item (dest_rse_id) since no hops are needed.
@@ -468,6 +468,9 @@ def get_hops(source_rse_id, dest_rse_id, include_multihop=False, multihop_rses=[
     :raises:                   NoDistance
     """
 
+    if multihop_rses is None:
+        multihop_rses = []
+
     # TODO: Might be problematic to always load the distance_graph, since it might be expensive
     # TODO: Have an rse_expression to specify the eligible hops
 
@@ -522,6 +525,8 @@ def get_hops(source_rse_id, dest_rse_id, include_multihop=False, multihop_rses=[
 
             for out_v in distance_graph[current_node]:
                 # Check if the distance would be smaller
+                if distance_graph[current_node][out_v] is None:
+                    continue
                 if visited_nodes.get(out_v, {'distance': 9999})['distance'] > current_distance + distance_graph[current_node][out_v] + HOP_PENALTY\\
                    and local_optimum > current_distance + distance_graph[current_node][out_v] + HOP_PENALTY:
                     # Check if the intermediate RSE is enabled for multihop
@@ -655,6 +660,7 @@ def get_transfer_requests_and_source_replicas(total_workers=0, worker_number=0,
         try:
             list_hops = get_hops(source_rse_id, dest_rse_id, include_multihop=core_config_get('transfers', 'use_multihop', default=False, expiration_time=600, session=session), multihop_rses=multihop_rses, session=session)
             if len(list_hops) > 1:
+                logging.debug('From %s to %s requires multihop: %s', source_rse_id, dest_rse_id, list_hops)
                 multihop = True
                 multi_hop_dict[req_id] = (list_hops, dict_attributes, retry_count)
         except NoDistance:
Repository: rucio/rucio
Issue: Fix Belle2 schema for RSE and RSE attributes

Motivation
----------
The current schema for RSE and RSE attributes break the creation of the rules. Needs to be fixed



Changed files:
lib/rucio/common/schema/belleii.py

diff --git a/lib/rucio/common/schema/belleii.py b/lib/rucio/common/schema/belleii.py
index 9ba88ec2c..c8df95886 100644
--- a/lib/rucio/common/schema/belleii.py
+++ b/lib/rucio/common/schema/belleii.py
@@ -62,7 +62,7 @@ NAME_LENGTH = 250
 
 NAME = {"description": "Data Identifier name",
         "type": "string",
-        "pattern": r"^[\\/A-Za-z0-9][A-Za-z0-9\\\\.\\\\-\\\\_\\/]{1,%s}$" % NAME_LENGTH}
+        "pattern": r"^/[A-Za-z0-9\\.\\-\\_\\+\\/]{1,%s}$" % NAME_LENGTH}
 
 R_NAME = {"description": "Data Identifier name",
           "type": "string",
@@ -93,7 +93,7 @@ RSE_ATTRIBUTE = {"description": "RSE attribute",
 
 DEFAULT_RSE_ATTRIBUTE = {"description": "Default RSE attribute",
                          "type": "string",
-                         "pattern": r'([A-Z0-9]+([_-][A-Z0-9]+)*)'}
+                         "pattern": r'([A-Za-z0-9]+([_-][A-Za-z0-9]+)*)'}
 
 REPLICA_STATE = {"description": "Replica state",
                  "type": "string",
@@ -271,7 +271,7 @@ ATTACHMENT = {"description": "Attachement",
                              "name": NAME,
                              "rse": {"description": "RSE name",
                                      "type": ["string", "null"],
-                                     "pattern": "^([A-Z0-9]+([_-][A-Z0-9]+)*)$"},
+                                     "pattern": "^([A-Z])+((([A-Za-z0-9]+))*(-)?)*$"},
                              "dids": DIDS},
               "required": ["dids"],
               "additionalProperties": False}
Repository: rucio/rucio
Issue: grouping value is corrupted in case of add_rule call with multiple kids

Motivation
----------
it will always become DATASET from second did on


Modification
------------
move grouping translation line out of loop



Changed files:
lib/rucio/core/rule.py

diff --git a/lib/rucio/core/rule.py b/lib/rucio/core/rule.py
index 6ea43a951..2324d2869 100644
--- a/lib/rucio/core/rule.py
+++ b/lib/rucio/core/rule.py
@@ -24,6 +24,7 @@
 # - Andrew Lister <andrew.lister@stfc.ac.uk>, 2019
 # - Dimitrios Christidis <dimitrios.christidis@cern.ch>, 2019
 # - Brandon White <bjwhite@fnal.gov>, 2019-2020
+# - Luc Goossens <luc.goossens@cern.ch>, 2020
 #
 # PY3K COMPATIBLE
 
@@ -136,6 +137,8 @@ def add_rule(dids, account, copies, rse_expression, grouping, weight, lifetime,
     """
     rule_ids = []
 
+    grouping = {'ALL': RuleGrouping.ALL, 'NONE': RuleGrouping.NONE}.get(grouping, RuleGrouping.DATASET)
+
     with record_timer_block('rule.add_rule'):
         # 1. Resolve the rse_expression into a list of RSE-ids
         with record_timer_block('rule.add_rule.parse_rse_expression'):
@@ -221,8 +224,6 @@ def add_rule(dids, account, copies, rse_expression, grouping, weight, lifetime,
 
             # 4. Create the replication rule
             with record_timer_block('rule.add_rule.create_rule'):
-                grouping = {'ALL': RuleGrouping.ALL, 'NONE': RuleGrouping.NONE}.get(grouping, RuleGrouping.DATASET)
-
                 if meta is not None:
                     try:
                         meta = json.dumps(meta)
Repository: rucio/rucio
Issue: get_did returns None for bytes instead of zero if dataset has no files yet

Motivation
----------
this requires additional testing on clients to avoid numeric exceptions


Modification
------------




Changed files:
lib/rucio/core/did.py

diff --git a/lib/rucio/core/did.py b/lib/rucio/core/did.py
index cc425a66e..2276ac59c 100644
--- a/lib/rucio/core/did.py
+++ b/lib/rucio/core/did.py
@@ -25,8 +25,9 @@
 # - Hannes Hansen <hannes.jakob.hansen@cern.ch>, 2018-2019
 # - Tobias Wegner <twegner@cern.ch>, 2019
 # - Andrew Lister <andrew.lister@stfc.ac.uk>, 2019
-# - Ruturaj Gujar, <ruturaj.gujar23@gmail.com>, 2019
-# - Brandon White, <bjwhite@fnal.gov>, 2019
+# - Ruturaj Gujar <ruturaj.gujar23@gmail.com>, 2019
+# - Brandon White <bjwhite@fnal.gov>, 2019
+# - Goossens Luc <luc.goossens@cern.ch>, 2020
 #
 # PY3K COMPATIBLE
 
@@ -1142,6 +1143,9 @@ def get_did(scope, name, dynamic=False, session=None):
         else:
             if dynamic:
                 bytes, length, events = __resolve_bytes_length_events_did(scope=scope, name=name, session=session)
+                # replace None value for bytes with zero
+                if bytes is None:
+                    bytes = 0
             else:
                 bytes, length = result.bytes, result.length
             return {'scope': result.scope, 'name': result.name, 'type': result.did_type,
Repository: rucio/rucio
Issue: Reaper 2.0 Should Reap Greedy RSE's Without Needing MinFreeSpace/MaxBeingDeletedFiles Set

Motivation
----------
The current version of the Reaper 2.0 will not reap an RSE marked as greedy that does not have MinFreeSpace and MaxBeingDeletedFiles set. MinFreeSpace should not matter for a greedy RSE, and MaxBeingDeletedFiles is not used by the Reaper 2.


Modification
------------
Move the check for greedy mode in __check_rse_usage() to before the acquisition of RSE limits.



Changed files:
lib/rucio/daemons/reaper/reaper2.py

diff --git a/lib/rucio/daemons/reaper/reaper2.py b/lib/rucio/daemons/reaper/reaper2.py
index 788f016f9..c755a7ca9 100644
--- a/lib/rucio/daemons/reaper/reaper2.py
+++ b/lib/rucio/daemons/reaper/reaper2.py
@@ -195,6 +195,14 @@ def __check_rse_usage(rse, rse_id, prepend_str):
     if result is NO_VALUE:
         max_being_deleted_files, needed_free_space, used, free, obsolete = None, None, None, None, None
 
+        # First of all check if greedy mode is enabled for this RSE
+        attributes = list_rse_attributes(rse_id=rse_id)
+        greedy = attributes.get('greedyDeletion', False)
+        if greedy:
+            result = (max_being_deleted_files, 1000000000000, used, free)
+            REGION.set('rse_usage_%s' % rse_id, result)
+            return result
+
         # Get RSE limits
         limits = get_rse_limits(rse_id=rse_id)
         if not limits and 'MinFreeSpace' not in limits and 'MaxBeingDeletedFiles' not in limits:
@@ -207,20 +215,12 @@ def __check_rse_usage(rse, rse_id, prepend_str):
 
         # Check from which sources to get used and total spaces
         # Default is storage
-        attributes = list_rse_attributes(rse_id=rse_id)
         source_for_total_space = attributes.get('sourceForTotalSpace', 'storage')
         source_for_used_space = attributes.get('sourceForUsedSpace', 'storage')
-        greedy = attributes.get('greedyDeletion', False)
 
         logging.debug('%s RSE: %s, source_for_total_space: %s, source_for_used_space: %s',
                       prepend_str, rse, source_for_total_space, source_for_used_space)
 
-        # First of all check if greedy mode is enabled for this RSE
-        if greedy:
-            result = (max_being_deleted_files, 1000000000000, used, free)
-            REGION.set('rse_usage_%s' % rse_id, result)
-            return result
-
         # Get total, used and obsolete space
         rse_usage = get_rse_usage(rse_id=rse_id)
         usage = [entry for entry in rse_usage if entry['source'] == 'obsolete']
Repository: rucio/rucio
Issue: 1.20.4rc2 storm protocol bug

Motivation
----------
The Storm protocol in RSEManager returns the input lfn as the pfn in lfns2pfns. This causes a crash as an InternalScope is then used as a dictionary key in list_replicas.

Modification
------------
The lfns dictionary should be sanitised so that scope is returned as an external string.


Changed files:
lib/rucio/rse/protocols/storm.py

diff --git a/lib/rucio/rse/protocols/storm.py b/lib/rucio/rse/protocols/storm.py
index 16fde1b76..1d599199a 100644
--- a/lib/rucio/rse/protocols/storm.py
+++ b/lib/rucio/rse/protocols/storm.py
@@ -43,8 +43,23 @@ class Default(protocol.RSEProtocol):
         return '%s/%s' % (scope, name)
 
     def lfns2pfns(self, lfns):
-        """ In this case, just returns back lfn. """
-        return lfns
+        """ In this case, just returns back lfn with external scope. """
+        if type(lfns) == dict:
+            val = lfns.copy()
+            if 'scope' in val and val['scope'] is not None:
+                val['scope'] = val['scope'].external
+
+        elif type(lfns) == list:
+            val = []
+            for l in lfns:
+                v = l.copy()
+                if 'scope' in v and v['scope'] is not None:
+                    v['scope'] = v['scope'].external
+                val.append(v)
+
+        else:
+            val = lfns
+        return val
 
     def path2pfn(self, path):
         """
Repository: rucio/rucio
Issue: strict_copy mode not properly working

Motivation
----------
Conveyor : strict_copy mode not properly working




Changed files:
lib/rucio/core/transfer.py, lib/rucio/daemons/conveyor/common.py

diff --git a/lib/rucio/core/transfer.py b/lib/rucio/core/transfer.py
index 5ac30219f..1c604bf1b 100644
--- a/lib/rucio/core/transfer.py
+++ b/lib/rucio/core/transfer.py
@@ -35,6 +35,7 @@ import datetime
 import imp
 import json
 import logging
+import re
 import time
 import traceback
 
@@ -771,6 +772,11 @@ def get_transfer_requests_and_source_replicas(total_workers=0, worker_number=0,
                     overwrite = False
                     transfer_dst_type = "TAPE"
 
+                sign_url = rse_attrs[dest_rse_id].get('sign_url', None)
+                if sign_url == 'gcs':
+                    dest_url = re.sub('davs', 'gclouds', dest_url)
+                    dest_url = re.sub('https', 'gclouds', dest_url)
+
                 # IV - get external_host + strict_copy
                 strict_copy = rse_attrs[dest_rse_id].get('strict_copy', False)
                 fts_hosts = rse_attrs[dest_rse_id].get('fts', None)
diff --git a/lib/rucio/daemons/conveyor/common.py b/lib/rucio/daemons/conveyor/common.py
index c29f92d5d..e5bc9af63 100644
--- a/lib/rucio/daemons/conveyor/common.py
+++ b/lib/rucio/daemons/conveyor/common.py
@@ -303,6 +303,7 @@ def bulk_group_transfer(transfers, policy='rule', group_bulk=200, source_strateg
                 set_checksum_value(t_file, dest_supported_checksums)
 
         multihop = transfer.get('multihop', False)
+        strict_copy = transfer.get('strict_copy', False)
 
         external_host = transfer['external_host']
         scope = t_file['metadata']['scope']
@@ -328,6 +329,8 @@ def bulk_group_transfer(transfers, policy='rule', group_bulk=200, source_strateg
                       's3alternate': True}
         if multihop:
             job_params['multihop'] = True
+        if strict_copy:
+            job_params['strict_copy'] = True
 
         # Don't put optional & missing keys in the parameters
         if transfer['dest_spacetoken']:
Repository: rucio/rucio
Issue: gfal stat does checksum calls for all checksums

Motivation
----------
The stat command of gfal currently does a checksum call for each checksum in `GLOBALLY_SUPPORTED_CHECKSUMS` this currrently results in a very long processing time for SRM which finally times out after ~30 minutes.


Modification
------------
First try the `PREFERRED_CHCEKSUM` and if this validates successfully don't try the other checksums.



Changed files:
lib/rucio/rse/protocols/gfal.py

diff --git a/lib/rucio/rse/protocols/gfal.py b/lib/rucio/rse/protocols/gfal.py
index 18fa80b99..596bfb684 100644
--- a/lib/rucio/rse/protocols/gfal.py
+++ b/lib/rucio/rse/protocols/gfal.py
@@ -41,7 +41,7 @@ from threading import Timer
 
 from rucio.common import exception, config
 from rucio.common.constraints import STRING_TYPES
-from rucio.common.utils import GLOBALLY_SUPPORTED_CHECKSUMS
+from rucio.common.utils import GLOBALLY_SUPPORTED_CHECKSUMS, PREFERRED_CHECKSUM
 from rucio.rse.protocols import protocol
 
 try:
@@ -334,13 +334,15 @@ class Default(protocol.RSEProtocol):
 
             :raises ServiceUnavailable: if some generic error occured in the library.
 
-            :returns: a dict with two keys, filesize and adler32 of the file provided in path.
+            :returns: a dict with two keys, filesize and an element of GLOBALLY_SUPPORTED_CHECKSUMS.
         """
         ret = {}
         ctx = self.__ctx
 
+        path = str(path)
+
         try:
-            stat_str = str(ctx.stat(str(path)))
+            stat_str = str(ctx.stat(path))
         except Exception as error:
             msg = 'Error while processing gfal stat call. Error: %s'
             raise exception.ServiceUnavailable(msg % str(error))
@@ -350,22 +352,25 @@ class Default(protocol.RSEProtocol):
             msg = 'gfal stat call result has unknown format. Result: %s'
             raise exception.ServiceUnavailable(msg % stat_str)
 
-        ret['filesize'] = stat_str.split()[7]
+        ret['filesize'] = stats[7]
 
-        verified = False
         message = "\\n"
+        try:
+            ret[PREFERRED_CHECKSUM] = ctx.checksum(path, str(PREFERRED_CHECKSUM.capitalize()))
+            return ret
+        except Exception as error:
+            message += 'Error while processing gfal checksum call (%s). Error: %s \\n' % (PREFERRED_CHECKSUM, str(error))
 
         for checksum_name in GLOBALLY_SUPPORTED_CHECKSUMS:
+            if checksum_name == PREFERRED_CHECKSUM:
+                continue
             try:
-                ret[checksum_name] = ctx.checksum(str(path), str(checksum_name.capitalize()))
-                verified = True
+                ret[checksum_name] = ctx.checksum(path, str(checksum_name.capitalize()))
+                return ret
             except Exception as error:
                 message += 'Error while processing gfal checksum call (%s). Error: %s \\n' % (checksum_name, str(error))
 
-        if not verified:
-            raise exception.RSEChecksumUnavailable(message)
-
-        return ret
+        raise exception.RSEChecksumUnavailable(message)
 
     def __gfal2_cancel(self):
         """