Repository: rucio/rucio
Issue: Default multihop tombstone is effectively None

Motivation
----------
In
https://github.com/rucio/rucio/blob/d59c237f533f40116026dc9f347f4fc1297f1ff0/lib/rucio/core/transfer.py#L103
a `DEFAULT_MULTIHOP_TOMBSTONE_DELAY` is set, and then in 
https://github.com/rucio/rucio/blob/d59c237f533f40116026dc9f347f4fc1297f1ff0/lib/rucio/core/transfer.py#L1440
the default value is passed if no runtime configuration is set. The problem is, all cache values are converted to string in
https://github.com/rucio/rucio/blob/d59c237f533f40116026dc9f347f4fc1297f1ff0/lib/rucio/core/config.py#L191
which means that the actual default value ends up being the string `'2:00:00'`, and later on in `tombstone_from_delay` we get a ValueError which is converted to `None`
https://github.com/rucio/rucio/blob/d59c237f533f40116026dc9f347f4fc1297f1ff0/lib/rucio/core/replica.py#L1424-L1428
hence no tombstone unless `transfers.multihop_tombstone_delay` is explicitly set (or if the RSE has a `multihop_tombstone_delay` attribute)


Modification
------------
Set
```python
DEFAULT_MULTIHOP_TOMBSTONE_DELAY = int(datetime.timedelta(hours=2).total_seconds())
```



Changed files:
lib/rucio/core/replica.py, lib/rucio/core/transfer.py

diff --git a/lib/rucio/core/replica.py b/lib/rucio/core/replica.py
index 89a225773..436731a9a 100644
--- a/lib/rucio/core/replica.py
+++ b/lib/rucio/core/replica.py
@@ -1421,11 +1421,7 @@ def tombstone_from_delay(tombstone_delay):
     if not tombstone_delay:
         return None
 
-    if not isinstance(tombstone_delay, timedelta):
-        try:
-            tombstone_delay = timedelta(seconds=int(tombstone_delay))
-        except ValueError:
-            return None
+    tombstone_delay = timedelta(seconds=int(tombstone_delay))
 
     if not tombstone_delay:
         return None
diff --git a/lib/rucio/core/transfer.py b/lib/rucio/core/transfer.py
index 477b001f9..4b54f7b0f 100644
--- a/lib/rucio/core/transfer.py
+++ b/lib/rucio/core/transfer.py
@@ -100,7 +100,7 @@ REGION_SHORT = make_region().configure('dogpile.cache.memcached',
                                        arguments={'url': config_get('cache', 'url', False, '127.0.0.1:11211'), 'distributed_lock': True})
 WEBDAV_TRANSFER_MODE = config_get('conveyor', 'webdav_transfer_mode', False, None)
 
-DEFAULT_MULTIHOP_TOMBSTONE_DELAY = datetime.timedelta(hours=2)
+DEFAULT_MULTIHOP_TOMBSTONE_DELAY = int(datetime.timedelta(hours=2).total_seconds())
 
 
 class RseData:
@@ -1511,10 +1511,14 @@ def create_missing_replicas_and_requests(
         if rws.request_id:
             continue
 
-        if 'multihop_tombstone_delay' in rws.dest_rse.attributes:
-            tombstone = tombstone_from_delay(rws.dest_rse.attributes['multihop_tombstone_delay'])
-        else:
-            tombstone = tombstone_from_delay(default_tombstone_delay)
+        tombstone_delay = rws.dest_rse.attributes.get('multihop_tombstone_delay', default_tombstone_delay)
+        try:
+            tombstone = tombstone_from_delay(tombstone_delay)
+        except ValueError:
+            logger(logging.ERROR, "%s: Cannot parse multihop tombstone delay %s", initial_request_id, tombstone_delay)
+            creation_successful = False
+            break
+
         files = [{'scope': rws.scope,
                   'name': rws.name,
                   'bytes': rws.byte_count,