anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: hashgraph/hedera-services
Issue: Find out why job regression-validate-freeze fails in nightly regression.

**Summary of the defect**
The job regression-validate-freeze fails more frequently recently. I am thinking there might be something wrong hide behind it.

**How to reproduce (if possible)**
Just check hedera-services master branch's nightly regression. For example, this link: https://app.circleci.com/pipelines/github/hashgraph/hedera-services/1070/workflows/3245b8bc-1496-4f3d-b467-668734b9e897/jobs/13443.


**Service logs (if applicable)**
```
...
  _            _    ___               _    
 | |_  __ _ __| |_ / __|_ _ __ _ _ __| |_  
 | ' \\/ _` (_-< ' \\ (_ | '_/ _` | '_ \\ ' \\ 
 |_||_\\__,_/__/_||_\\___|_| \\__,_| .__/_||_|
                                |_|        
This AMI is from ATF-U18.04-OJDK12.0.1-PSQL10.9-BADGERIZE-V7
OpenJDK12.0(build 12.0.1+12) 
Host #0 still pending...
>>>> [CI] >> Time's up! ( 120 secs expired.)
>>>> [CI] >> Not all  hgcaa.log had  1 'current platform status = MAINTENANCE' occurrences!
>>>> [CI] >> 52.45.177.185
>>>> [CI] >> 0
Ubuntu 18.04.2 LTS
...
```
**Environment:**
- OS: [e.g. Ubuntu 18.04]
- Java: [e.g. OpenJDK 12.0.2]
- Hedera Services Version: [e.g. 0.0.5]
- HAPI Version: [e.g. 0.0.5]

**Additional Context**
Add any other context about the problem here. Attach any logs here, if applicable.


Changed files:
.circleci/scripts/summarize-test-results.py

diff --git a/.circleci/scripts/summarize-test-results.py b/.circleci/scripts/summarize-test-results.py
index c823f5c964..e9f613319c 100644
--- a/.circleci/scripts/summarize-test-results.py
+++ b/.circleci/scripts/summarize-test-results.py
@@ -181,13 +181,15 @@ def get_AWS_testnet_IPs(log_file):
     with open(log_file, "r") as f:
         data = f.read()
 
-        match = re.search("PLAY RECAP \\*+\\n((.*\\n){4})", data)
-        IP_lines = [line for line in match.group(
-            1).split(sep='\\n') if len(line) > 0]
-        for line in IP_lines:
-            IP = re.search('(\\d+\\.\\d+\\.\\d+\\.\\d+)[ \\t]+:[ \\t]+ok=\\d+.*', line)
-            IPs.append(IP.group(1))
-
+        match = re.search(r'>>>> \\[CI\\] >> (\\d+\\.\\d+\\.\\d+\\.\\d+) (\\d+\\.\\d+\\.\\d+\\.\\d+) (\\d+\\.\\d+\\.\\d+\\.\\d+) (\\d+\\.\\d+\\.\\d+\\.\\d+)\\n', data)
+        if match:
+            for i in range(4):
+                IP = match.group(i+1)
+                print("IP: {}".format(IP))
+                IPs.append(IP)
+        else:
+            print("No match")
+            IPs.append('NO_IP_ADDRESS')
     return IPs
Repository: qiboteam/qibo
Issue: cma optimizer ignores bounds silently

**Describe the bug**
When passing bounds to the `cma` optimizer, they are silently ignored.

**To Reproduce**
Simply call `optimize` with the `bounds` keyword and the `cma` method. The bounds are not passed down:

https://github.com/qiboteam/qibo/blob/4ed2fe67bd4b72f7394a13f1beaf5f1f270a5646/src/qibo/optimizers.py#L79

**Expected behavior**
Since the documentation specifies that the bounds are for scipy optimizers, I believe it should either return an error:

`The keyword "bounds" cannot be used with the cma optimizer. Please use options = {"bounds": <bound definition>} instead as defined by the cma documentation` (or something on that line, the cma documentation is not very prolific so not sure about how to reference it...).

Or manually put the `bounds` inside the `options` passed to the `cma`. However this might be tricky as the format I think is not the same as `scipy`.

Bounds from scipy: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.Bounds.html
Bounds from cma: https://github.com/CMA-ES/pycma/blob/master/notebooks/notebook-usecases-basics.ipynb

Changed files:
src/qibo/optimizers.py

diff --git a/src/qibo/optimizers.py b/src/qibo/optimizers.py
index 35b84024..c9208e16 100644
--- a/src/qibo/optimizers.py
+++ b/src/qibo/optimizers.py
@@ -1,3 +1,6 @@
+from qibo.config import log, raise_error
+
+
 def optimize(
     loss,
     initial_parameters,
@@ -76,6 +79,11 @@ def optimize(
             circuit.set_parameters(params)
     """
     if method == "cma":
+        if bounds is not None:  # pragma: no cover
+            raise_error(
+                RuntimeError,
+                "The keyword 'bounds' cannot be used with the cma optimizer. Please use 'options' instead as defined by the cma documentation: ex. options['bounds'] = [0.0, 1.0].",
+            )
         return cmaes(loss, initial_parameters, args, options)
     elif method == "sgd":
         if backend is None:
@@ -226,8 +234,6 @@ def sgd(loss, initial_parameters, args=(), options=None, compile=False, backend=
             - ``'nmessage'`` (int, default: ``1e3``): Every how many epochs to print
               a message of the loss function.
     """
-    from qibo.config import log, raise_error
-
     if not backend.name == "tensorflow":
         raise_error(RuntimeError, "SGD optimizer requires Tensorflow backend.")
Repository: googlefonts/gftools
Issue: builder: variable doesn't build but output message is still  kindda positive

Recently we allowed the variable build to fail without stopping the build, but it still yeld "Completed" at the end.


```
ERROR:cu2qu.ufo:Glyphs named 'gcommaaccent' have different number of segments
ERROR:GFBuilder:Could not build variable font: In 'Anybody.glyphs' -> 'master_ufo/Anybody.designspace': Generating fonts from Designspace failed: fonts contains incompatible glyphs: 'gcommaaccent'
INFO:GFBuilder:Building variable completed. All done!
```

Changed files:
Lib/gftools/builder/__init__.py

diff --git a/Lib/gftools/builder/__init__.py b/Lib/gftools/builder/__init__.py
index 32e8aea..58adeba 100644
--- a/Lib/gftools/builder/__init__.py
+++ b/Lib/gftools/builder/__init__.py
@@ -291,13 +291,10 @@ class GFBuilder:
             args["output_path"] = os.path.join(
                 self.config["vfDir"], sourcebase + "-VF.ttf",
             )
-            try:
-                output_files = self.run_fontmake(source, args)
-                newname = self.rename_variable(output_files[0])
-                ttFont = TTFont(newname)
-                ttFonts.append(ttFont)
-            except Exception as e:
-                self.logger.error("Could not build variable font: %s" % e)
+            output_files = self.run_fontmake(source, args)
+            newname = self.rename_variable(output_files[0])
+            ttFont = TTFont(newname)
+            ttFonts.append(ttFont)
 
         if not ttFonts:
             return
Repository: rucio/rucio
Issue: Access information from traces are not propagated to datasets

Motivation
----------

It has come to our attention that, while files have proper access times, datasets do not.

https://github.com/rucio/rucio/blob/58a5ff3bff3bdff2b7251167522dbfe4c0b444f8/lib/rucio/daemons/tracer/kronos.py#L410-L416

Effectively, `__update_datasets()` is not run during the daemon loop; it’s only run when the daemon exits. Most likely, there isn’t enough time to process all the datasets. This is a regression from #5687.

Modification
------------

Replace this obsolete and non-functional wait mechanism with the `run_daemon()`’s `sleep_time`.

Changed files:
lib/rucio/daemons/tracer/kronos.py

diff --git a/lib/rucio/daemons/tracer/kronos.py b/lib/rucio/daemons/tracer/kronos.py
index c80e72bf1..4dc67ad6d 100644
--- a/lib/rucio/daemons/tracer/kronos.py
+++ b/lib/rucio/daemons/tracer/kronos.py
@@ -389,7 +389,8 @@ def run_once_kronos_file(heartbeat_handler: HeartbeatHandler, return_values: Dic
             conn.subscribe(destination=config_get('tracer-kronos', 'queue'), ack='client-individual', id=subscription_id, headers={'activemq.prefetchSize': prefetch_size})
 
 
-def kronos_dataset(once: bool = False, dataset_queue: Queue = None, sleep_time: int = 60):
+def kronos_dataset(dataset_queue: Queue, once: bool = False, sleep_time: int = 60):
+    return_values = {'heartbeat_handler': HeartbeatHandler("kronos-dataset", 10)}
     run_daemon(
         once=once,
         graceful_stop=graceful_stop,
@@ -400,23 +401,21 @@ def kronos_dataset(once: bool = False, dataset_queue: Queue = None, sleep_time:
         run_once_fnc=functools.partial(
             run_once_kronos_dataset,
             dataset_queue=dataset_queue,
+            return_values=return_values,
         )
     )
 
-    # once again for the backlog
-    __update_datasets(dataset_queue)
-
+    # once again for potential backlog
+    run_once_kronos_dataset(dataset_queue=dataset_queue, return_values=return_values, heartbeat_handler=return_values['heartbeat_handler'], sleep_time=sleep_time)
 
-def run_once_kronos_dataset(heartbeat_handler: HeartbeatHandler, dataset_queue: Queue, **kwargs):
-    _, _, logger = heartbeat_handler.live()
-    dataset_wait = config_get_int('tracer-kronos', 'dataset_wait')
-    start = datetime.now()
-    if (datetime.now() - start).seconds > dataset_wait:
-        __update_datasets(dataset_queue=dataset_queue, logger=logger)
-        start = datetime.now()
 
+def run_once_kronos_dataset(dataset_queue: Queue, return_values: dict, heartbeat_handler: HeartbeatHandler, **kwargs):
+    if heartbeat_handler is None:
+        if "heartbeat_handler" not in return_values.keys():
+            return_values["heartbeat_handler"] = HeartbeatHandler("kronos-dataset", 10)
+        heartbeat_handler = return_values["heartbeat_handler"]
 
-def __update_datasets(dataset_queue, logger=logging.log):
+    _, _, logger = heartbeat_handler.live()
     len_ds = dataset_queue.qsize()
     datasets = {}
     dslocks = {}
Repository: rucio/rucio
Issue: cancelled job_state + failed file_state multi-source transfers never handled by poller

the issue is, again, in this state transition : 
https://github.com/rucio/rucio/blob/2b23466beec4372ae6ae25b1e3e2a69501788a27/lib/rucio/transfertool/fts3.py#L596

There were so many bugs already with it that I'm afraid to touch anything here :cry: 

Changed files:
lib/rucio/transfertool/fts3.py

diff --git a/lib/rucio/transfertool/fts3.py b/lib/rucio/transfertool/fts3.py
index 4ef6e86bd..7b36c2fd9 100644
--- a/lib/rucio/transfertool/fts3.py
+++ b/lib/rucio/transfertool/fts3.py
@@ -593,7 +593,7 @@ class FTS3ApiTransferStatusReport(Fts3TransferStatusReport):
         if file_state_is_final:
             if file_state == FTS_STATE.FINISHED:
                 new_state = RequestState.DONE
-            elif file_state == FTS_STATE.FAILED and job_state == FTS_STATE.FAILED or \\
+            elif file_state == FTS_STATE.FAILED and job_state_is_final or \\
                     file_state == FTS_STATE.FAILED and not self._multi_sources:  # for multi-source transfers we must wait for the job to be in a final state
                 if self._is_recoverable_fts_overwrite_error(self.request(session), reason, self._file_metadata):
                     new_state = RequestState.DONE
Repository: rucio/rucio
Issue: Failed multi-source transfers not transitioned to failed state

Motivation
----------

this condition is incomplete https://github.com/rucio/rucio/blob/5804712c7eb187ffb1fcfc3bbf49d89c02c0aeff/lib/rucio/transfertool/fts3.py#L576




Changed files:
lib/rucio/transfertool/fts3.py

diff --git a/lib/rucio/transfertool/fts3.py b/lib/rucio/transfertool/fts3.py
index 1f60a9989..77eeb8e41 100644
--- a/lib/rucio/transfertool/fts3.py
+++ b/lib/rucio/transfertool/fts3.py
@@ -573,7 +573,8 @@ class FTS3ApiTransferStatusReport(Fts3TransferStatusReport):
         if file_state_is_final:
             if file_state == FTS_STATE.FINISHED:
                 new_state = RequestState.DONE
-            elif file_state == FTS_STATE.FAILED and not self._multi_sources:  # for multi-source transfers we must wait for the job to be in a final state
+            elif file_state == FTS_STATE.FAILED and job_state == FTS_STATE.FAILED or \\
+                    file_state == FTS_STATE.FAILED and not self._multi_sources:  # for multi-source transfers we must wait for the job to be in a final state
                 if self._is_recoverable_fts_overwrite_error(self.request(session), reason, self._file_metadata):
                     new_state = RequestState.DONE
                 else:
Repository: rucio/rucio
Issue: receiver doesn't correctly handle intermediate hops on fts 3.12

Motivation
----------
The last major FTS release started to correctly fill "job_state" for those messages as "ACTIVE". This is a behavioral change which makes rucio skip those messages. 

https://github.com/rucio/rucio/blob/5e911923990cc9c3caa0afbfec919c3ea4121edc/lib/rucio/daemons/conveyor/receiver.py#L74



Changed files:
lib/rucio/daemons/conveyor/receiver.py

diff --git a/lib/rucio/daemons/conveyor/receiver.py b/lib/rucio/daemons/conveyor/receiver.py
index f6a4bf06f..bbecd50cf 100644
--- a/lib/rucio/daemons/conveyor/receiver.py
+++ b/lib/rucio/daemons/conveyor/receiver.py
@@ -71,7 +71,7 @@ class Receiver(object):
            and 'issuer' in msg['job_metadata'].keys() \\
            and str(msg['job_metadata']['issuer']) == str('rucio'):
 
-            if 'job_state' in msg.keys() and str(msg['job_state']) != str('ACTIVE'):
+            if 'job_state' in msg.keys() and (str(msg['job_state']) != str('ACTIVE') or msg.get('job_multihop', False) is True):
                 record_counter('daemons.conveyor.receiver.message_rucio')
 
                 self._perform_request_update(msg)
Repository: rucio/rucio
Issue: bad-str-strip-call errors in pylint

### Description

Pylint found 3 occurrences of E1310 bad-str-strip-call in the codebase.

### Steps to reproduce

1. run donkeyrider with `-d centos7 -s syntax --build`
2. examine output of `running pylint`

### Rucio Version

_No response_

### Additional Information

_No response_

Changed files:
lib/rucio/rse/protocols/ssh.py, lib/rucio/rse/protocols/rclone.py

diff --git a/lib/rucio/rse/protocols/rclone.py b/lib/rucio/rse/protocols/rclone.py
index de49dfe2f..9e0a21b3e 100644
--- a/lib/rucio/rse/protocols/rclone.py
+++ b/lib/rucio/rse/protocols/rclone.py
@@ -195,7 +195,7 @@ class Default(protocol.RSEProtocol):
 
             if status_query == 0:
                 chsum = 'md5'
-                val = out.strip('  ').split()
+                val = out.strip(' ').split()
                 ret[chsum] = val[0]
 
         except Exception as e:
diff --git a/lib/rucio/rse/protocols/ssh.py b/lib/rucio/rse/protocols/ssh.py
index 9b45e3c25..82d92792a 100644
--- a/lib/rucio/rse/protocols/ssh.py
+++ b/lib/rucio/rse/protocols/ssh.py
@@ -110,7 +110,7 @@ class Default(protocol.RSEProtocol):
 
             if status_query == 0:
                 chsum = 'md5'
-                val = out.strip('  ').split()
+                val = out.strip(' ').split()
                 ret[chsum] = val[0]
 
         except Exception as e:
@@ -324,7 +324,7 @@ class Rsync(Default):
 
             if status_query == 0:
                 chsum = 'md5'
-                val = out.strip('  ').split()
+                val = out.strip(' ').split()
                 ret[chsum] = val[0]
 
         except Exception as e:
Repository: rucio/rucio
Issue: Clients try to access memcached 

Motivation
----------
Some places in clients indirectly (via imports) try to access cache values. The old library used for memcache silently ignored those accesses, but pymemcache doesn't like it. For now, the code which disables the caching does the trick, but prints a warning message, which isn't very user-friendly. 

Modification
------------
Ensure that clients never try to access the cache. 



Changed files:
lib/rucio/rse/__init__.py

diff --git a/lib/rucio/rse/__init__.py b/lib/rucio/rse/__init__.py
index ba6da0177..5fc50100b 100644
--- a/lib/rucio/rse/__init__.py
+++ b/lib/rucio/rse/__init__.py
@@ -15,7 +15,6 @@
 
 from dogpile.cache import make_region
 
-from rucio.common.cache import make_region_memcached
 from rucio.common.utils import is_client
 from rucio.rse import rsemanager
 
@@ -81,6 +80,7 @@ if rsemanager.CLIENT_MODE:   # pylint:disable=no-member
 if rsemanager.SERVER_MODE:   # pylint:disable=no-member
     from rucio.core.rse import get_rse_protocols, get_rse_id
     from rucio.core.vo import map_vo
+    from rucio.common.cache import make_region_memcached
 
     def tmp_rse_info(rse=None, vo='def', rse_id=None, session=None):
         if rse_id is None:
Repository: rucio/rucio
Issue: globus RSE protocol silently ignores errors

This will potentially result in dark data. 

To avoid this, just throw an exception which will be picked by reaper and skip removing the replicas 

Changed files:
lib/rucio/rse/protocols/globus.py

diff --git a/lib/rucio/rse/protocols/globus.py b/lib/rucio/rse/protocols/globus.py
index 72b9d78d4..09b519fad 100644
--- a/lib/rucio/rse/protocols/globus.py
+++ b/lib/rucio/rse/protocols/globus.py
@@ -198,13 +198,14 @@ class GlobusRSEProtocol(RSEProtocol):
             try:
                 delete_response = send_delete_task(endpoint_id=self.globus_endpoint_id, path=path, logger=self.logger)
             except TransferAPIError as err:
-                print(err)
+                self.logger(logging.WARNING, str(err))
+                raise exception.RucioException(err)
         else:
-            print('No rse attribute found for globus endpoint id.')
+            raise exception.RucioException('No rse attribute found for globus endpoint id.')
 
         if delete_response['code'] != 'Accepted':
-            print('delete_task not accepted by Globus')
-            print('delete_response: %s' % delete_response)
+            self.logger(logging.DEBUG, 'delete_response: %s' % delete_response)
+            raise exception.RucioException('delete_task not accepted by Globus')
 
     def bulk_delete(self, pfns):
         """
@@ -218,13 +219,13 @@ class GlobusRSEProtocol(RSEProtocol):
             try:
                 bulk_delete_response = send_bulk_delete_task(endpoint_id=self.globus_endpoint_id, pfns=pfns, logger=self.logger)
             except TransferAPIError as err:
-                self.logger(logging.WARNING, str(err))
+                raise exception.RucioException(err)
         else:
-            self.logger(logging.WARNING, 'No rse attribute found for globus endpoint id.')
+            raise exception.RucioException('No rse attribute found for globus endpoint id.')
 
         if bulk_delete_response['code'] != 'Accepted':
-            self.logger(logging.WARNING, 'delete_task not accepted by Globus')
-            self.logger(logging.WARNING, 'delete_response: %s' % bulk_delete_response)
+            self.logger(logging.DEBUG, 'delete_response: %s' % bulk_delete_response)
+            raise exception.RucioException('delete_task not accepted by Globus')
 
     def connect(self):
         """
Repository: rucio/rucio
Issue: throttler releases less requests than the available quota

Motivation
----------
```
conveyor-throttler[8b7233a:0/4]: limit to UKI-NORTHGRID-LANCS-HEP-CEPH_DATADISK activity Data rebalancing: can release 41853 out of 1628126 waiting requests. Account share:
 {ddmadmin: {'waiting': 1627436, 'active': 42723, 'residual_capacity': 0}, panda: {'waiting': 690, 'active': 15424, 'residual_capacity': 690}}
```
this is due to the code which splits the quota per account to achieve fair share



Changed files:
lib/rucio/daemons/conveyor/throttler.py

diff --git a/lib/rucio/daemons/conveyor/throttler.py b/lib/rucio/daemons/conveyor/throttler.py
index 9dcd805b8..7e99f88a0 100644
--- a/lib/rucio/daemons/conveyor/throttler.py
+++ b/lib/rucio/daemons/conveyor/throttler.py
@@ -334,17 +334,11 @@ def _split_threshold_per_account(per_account_stats, total_to_release):
     nr_accounts = len(per_account_stats)
     remaining_to_release = total_to_release
     remaining_accounts = nr_accounts
-    for account, account_stat in sorted(per_account_stats.items(), key=lambda i: i[1]['active']):
+    for account, account_stat in sorted(per_account_stats.items(), key=lambda i: i[1]['waiting']):
         threshold_per_account = math.ceil(remaining_to_release / remaining_accounts)
 
         waiting = account_stat['waiting']
-        active = account_stat['active']
-        if active > threshold_per_account:
-            to_release_for_account = 0
-        elif active + waiting <= threshold_per_account:
-            to_release_for_account = waiting
-        else:
-            to_release_for_account = threshold_per_account - active
+        to_release_for_account = min(waiting, threshold_per_account)
 
         yield account, to_release_for_account
Repository: rucio/rucio
Issue: Request with one tape and one multihop tape source will crash submitter

Motivation
----------
Suppose in the `get_transfer_requests_and_source_replicas` main loop, a request is added with a multihop tape source. Then in another iteration, a direct tape source is identified. We end up in https://github.com/rucio/rucio/blob/740b825cac747280ca159089bca4635264c07fc1/lib/rucio/core/transfer.py#L968
Since the first source was multihop, we clear `transfers[req_id]['sources']` at https://github.com/rucio/rucio/blob/740b825cac747280ca159089bca4635264c07fc1/lib/rucio/core/transfer.py#L1005-L1009
Then shortly after, if the second source is tape, we will end up in https://github.com/rucio/rucio/blob/740b825cac747280ca159089bca4635264c07fc1/lib/rucio/core/transfer.py#L1042-L1050
where there is an unchecked access to `transfers[req_id]['sources'][0]` that crashes.

Modification
------------
In this situation, I believe we still want to prefer the direct tape source over the multihop tape source and want to take the path at L1047. So I would suggest modifying L1045 to
```python
if len(transfers[req_id]["sources"]) and (
    transfers[req_id]["sources"][0][3] > ranking
    or (
        transfers[req_id]["sources"][0][3] == ranking
        and transfers[req_id]["sources"][0][4] <= link_ranking
    )
):
    continue
```


Changed files:
lib/rucio/core/transfer.py

diff --git a/lib/rucio/core/transfer.py b/lib/rucio/core/transfer.py
index c71ca30d0..bc3c71ebe 100644
--- a/lib/rucio/core/transfer.py
+++ b/lib/rucio/core/transfer.py
@@ -1029,12 +1029,6 @@ def get_transfer_requests_and_source_replicas(total_workers=0, worker_number=0,
                 if multihop:
                     continue
 
-                # The transfer queued previously is a multihop, but this one is direct.
-                # Reset the sources, remove the multihop flag
-                if transfers[req_id].get('multihop', False):
-                    transfers[req_id].pop('multihop', None)
-                    transfers[req_id]['sources'] = []
-
                 current_source_is_tape = transfers[req_id]['bring_online']
                 new_source_is_tape = ctx.is_tape_rse(source_rse_id) or ctx.rse_attrs(source_rse_id).get('staging_required', False)
 
@@ -1074,6 +1068,12 @@ def get_transfer_requests_and_source_replicas(total_workers=0, worker_number=0,
                     else:
                         continue
 
+                # The transfer queued previously is a multihop, but this one is direct.
+                # Reset the sources, remove the multihop flag
+                if transfers[req_id].get('multihop', False):
+                    transfers[req_id].pop('multihop', None)
+                    transfers[req_id]['sources'] = []
+
                 transfers[req_id]['sources'].append((rse, source_url, source_rse_id, ranking, link_ranking))
                 # if one source has force IPv4, force IPv4 for the whole job
                 use_ipv4 = ctx.rse_attrs(source_rse_id).get('use_ipv4', False)
Repository: rucio/rucio
Issue: Fix UnicodeEncodeError while using PDB in baseclient.py _send_request

Motivation
----------
While using `remote_pdb` in a telnet client on `automatix` daemon pod in OKD4, when I try to use the `l` command to print the line numbers, `remote_pdb` throws `UnicodeEncodeError: 'ascii' codec can't encode character '\\u2019' in position 112: ordinal not in range(128)`. This is because the character on this comment line next to the word `server` in `baseclient.py` is not a ascii single quotation (27), but a unicode right single quotation (u2019): https://github.com/rucio/rucio/blob/9dc4238217a37b938f120b3bac2007487f6a063e/lib/rucio/client/baseclient.py#L367

The other single quotes in the code are standard ascii quotations, such as on this line: https://github.com/rucio/rucio/blob/9dc4238217a37b938f120b3bac2007487f6a063e/lib/rucio/client/baseclient.py#L92

Modification
------------
Change the right quotation to an ascii quotation.



Changed files:
lib/rucio/client/baseclient.py

diff --git a/lib/rucio/client/baseclient.py b/lib/rucio/client/baseclient.py
index e31f48c30..4ec9b9206 100644
--- a/lib/rucio/client/baseclient.py
+++ b/lib/rucio/client/baseclient.py
@@ -364,7 +364,7 @@ class BaseClient(object):
         :param get_token: (optional) if it is called from a _get_token function.
         :param cert: (optional) if String, path to the SSL client cert file (.pem). If Tuple, (cert, key) pair.
         :param auth: (optional) auth tuple to enable Basic/Digest/Custom HTTP Auth.
-        :param verify: (optional) either a boolean, in which case it controls whether we verify the server’s TLS
+        :param verify: (optional) either a boolean, in which case it controls whether we verify the server's TLS
                        certificate, or a string, in which case it must be a path to a CA bundle to use.
         :return: the HTTP return body.
         """
Repository: rucio/rucio
Issue: Regression in bulk_group_transfer due to missing key

Motivation
----------
Due to the new verify_checksum mapping.
```
2018-04-24 14:36:42,467 2991    INFO    0:0 Got 1 transfers for Functional Test XrootD
2018-04-24 14:36:42,468 2991    INFO    0:0 Starting to group transfers for Functional Test XrootD
2018-04-24 14:36:42,468 2991    CRITICAL        0:0 Traceback (most recent call last):
File “/usr/lib/python2.7/site-packages/rucio/daemons/conveyor/submitter.py”, line 157, in submitter
  grouped_jobs = bulk_group_transfer(transfers, group_policy, group_bulk, fts_source_strategy, max_time_in_queue)
File “/usr/lib/python2.7/site-packages/rucio/daemons/conveyor/common.py”, line 156, in bulk_group_transfer
  ‘verify_checksum’: transfer[‘verify_checksum’],
KeyError: ‘verify_checksum’
```

Changed files:
lib/rucio/daemons/conveyor/common.py

diff --git a/lib/rucio/daemons/conveyor/common.py b/lib/rucio/daemons/conveyor/common.py
index 1eacbbd3b..5e20fbf41 100644
--- a/lib/rucio/daemons/conveyor/common.py
+++ b/lib/rucio/daemons/conveyor/common.py
@@ -148,12 +148,13 @@ def bulk_group_transfer(transfers, policy='rule', group_bulk=200, fts_source_str
             grouped_transfers[external_host] = {}
             grouped_jobs[external_host] = []
 
+        verify_checksum = transfer.get('verify_checksum', 'both')
         file = {'sources': transfer['sources'],
                 'destinations': transfer['dest_urls'],
                 'metadata': transfer['file_metadata'],
                 'filesize': int(transfer['file_metadata']['filesize']),
                 'checksum': None,
-                'verify_checksum': transfer['verify_checksum'],
+                'verify_checksum': verify_checksum,
                 'selection_strategy': fts_source_strategy,
                 'request_type': transfer['file_metadata'].get('request_type', None),
                 'activity': str(transfer['file_metadata']['activity'])}
@@ -163,7 +164,7 @@ def bulk_group_transfer(transfers, policy='rule', group_bulk=200, fts_source_str
             if 'adler32' in list(file['metadata'].keys()) and file['metadata']['adler32']:
                 file['checksum'] = 'ADLER32:%s' % str(file['metadata']['adler32'])
 
-        job_params = {'verify_checksum': transfer['verify_checksum'],
+        job_params = {'verify_checksum': verify_checksum,
                       'copy_pin_lifetime': transfer['copy_pin_lifetime'] if transfer['copy_pin_lifetime'] else -1,
                       'bring_online': transfer['bring_online'] if transfer['bring_online'] else None,
                       'job_metadata': {'issuer': 'rucio'},  # finaly job_meta will like this. currently job_meta will equal file_meta to include request_id and etc.
Repository: rucio/rucio
Issue: Transfer : Fix compatible transfers mapping

Motivation
----------
The table mapping compatible mapping is erroneous 

Modification
------------
Fix the mapping table

Changed files:
lib/rucio/common/constants.py

diff --git a/lib/rucio/common/constants.py b/lib/rucio/common/constants.py
index 653473004..bd522e8d1 100644
--- a/lib/rucio/common/constants.py
+++ b/lib/rucio/common/constants.py
@@ -6,7 +6,7 @@
 #
 # Authors:
 # - Vincent Garonne, <vincent.garonne@cern.ch>, 2013
-# - Cedric Serfon, <cedric.serfon@cern.ch>, 2015
+# - Cedric Serfon, <cedric.serfon@cern.ch>, 2015-2018
 # - Mario Lassnig, <mario.lassnig@cern.ch>, 2018
 
 """
@@ -23,8 +23,9 @@ RESERVED_KEYS = ['scope', 'name', 'account', 'did_type', 'is_open', 'monotonic',
 KEY_TYPES = ['ALL', 'COLLECTION', 'FILE', 'DERIVED']
 # all(container, dataset, file), collection(dataset or container), file, derived(compute from file for collection)
 
-SCHEME_MAP = {'srm': ['srm', 'gsiftp', 's3'],
-              'gsiftp': ['srm', 'gsiftp', 's3'],
-              'https': ['https', 'davs'],
-              'davs': ['https', 'davs'],
-              's3': ['https', 'davs', 'gsiftp', 'srm']}
+SCHEME_MAP = {'srm': ['srm', 'gsiftp'],
+              'gsiftp': ['srm', 'gsiftp'],
+              'https': ['https', 'davs', 's3'],
+              'davs': ['https', 'davs', 's3'],
+              'root': ['root'],
+              's3': ['https', 'davs', 's3']}
Repository: rucio/rucio
Issue: findall with 2 arguments failing on SL6

Motivation
----------
```
cmd: rucio get user.desilva:user.desilva.dataset.1fe078e2-a369-42c4-9dd0-3b28d95032ac
2018-06-22 07:04:23,396	INFO	Processing 1 item(s) for input
2018-06-22 07:04:23,867	ERROR	findall() takes exactly 2 arguments (3 given)
```
It seems that the definition of findall on python 2.6 and python 2.7 uses 1 and 2 arguments respectively. As we need to be compatible to python 2.6 this needs to be fixed.

Changed files:
lib/rucio/client/downloadclient.py

diff --git a/lib/rucio/client/downloadclient.py b/lib/rucio/client/downloadclient.py
index 88cc481d7..a3d6af2cf 100644
--- a/lib/rucio/client/downloadclient.py
+++ b/lib/rucio/client/downloadclient.py
@@ -974,13 +974,13 @@ class DownloadClient:
         root = ElementTree.fromstring(metalink_str)
         files = []
 
-        # metalink namespace mapping
-        ns = {'list_replicas_ml': 'urn:ietf:params:xml:ns:metalink'}
+        # metalink namespace
+        ns = '{urn:ietf:params:xml:ns:metalink}'
 
         # loop over all <file> tags of the metalink string
-        for file_ml in root.findall('list_replicas_ml:file', ns):
+        for file_ml in root.findall(ns + 'file'):
             # search for identity-tag
-            cur_did = file_ml.find('list_replicas_ml:identity', ns)
+            cur_did = file_ml.find(ns + 'identity')
             if not ElementTree.iselement(cur_did):
                 raise RucioException('Failed to locate identity-tag inside %s' % ElementTree.tostring(file_ml))
 
@@ -994,16 +994,16 @@ class DownloadClient:
                         'md5': None,
                         'sources': []}
 
-            size = file_ml.find('list_replicas_ml:size', ns)
+            size = file_ml.find(ns + 'size')
             if ElementTree.iselement(size):
                 cur_file['bytes'] = int(size.text)
 
-            for cur_hash in file_ml.findall('list_replicas_ml:hash', ns):
+            for cur_hash in file_ml.findall(ns + 'hash'):
                 hash_type = cur_hash.get('type')
                 if hash_type:
                     cur_file[hash_type] = cur_hash.text
 
-            for rse_ml in file_ml.findall('list_replicas_ml:url', ns):
+            for rse_ml in file_ml.findall(ns + 'url'):
                 # check if location attrib (rse name) is given
                 rse = rse_ml.get('location')
                 pfn = rse_ml.text
Repository: rucio/rucio
Issue: Resurrect method doesn't remove expired_at for deleted_dids

Motivation
----------
The resurrect method doesn't remove the expired_at when a DID is resurrected which triggers the immediate deletion of the resurrected DID


Changed files:
lib/rucio/core/did.py

diff --git a/lib/rucio/core/did.py b/lib/rucio/core/did.py
index 68a065329..a38e4a63b 100644
--- a/lib/rucio/core/did.py
+++ b/lib/rucio/core/did.py
@@ -1591,6 +1591,8 @@ def resurrect(dids, session=None):
         #    raise exception.UnsupportedOperation("File '%(scope)s:%(name)s' cannot be resurrected" % did)
 
         kargs = del_did.to_dict()
+        if kargs['expired_at']:
+            kargs['expired_at'] = None
         kargs.pop("_sa_instance_state", None)
 
         session.query(models.DeletedDataIdentifier).\\
Repository: rucio/rucio
Issue: setup_clients.py classifiers needs to be a list, not tuples

Motivation
----------
Classifiers were changed to tuple, which does not work, needs to be a list.

Changed files:
setup_rucio_client.py

diff --git a/setup_rucio_client.py b/setup_rucio_client.py
index bbbdf0bb1..0e6a83298 100644
--- a/setup_rucio_client.py
+++ b/setup_rucio_client.py
@@ -190,7 +190,7 @@ setup(
     license="Apache License, Version 2.0",
     url="http://rucio.cern.ch/",
     python_requires=">=2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*",
-    classifiers=(
+    classifiers=[
         'Development Status :: 5 - Production/Stable',
         'License :: OSI Approved :: Apache Software License',
         'Intended Audience :: Information Technology',
@@ -207,7 +207,7 @@ setup(
         'Programming Language :: Python :: Implementation :: CPython',
         'Programming Language :: Python :: Implementation :: PyPy',
         'Environment :: No Input/Output (Daemon)'
-    ),
+    ],
     install_requires=REQUIRES,
     extras_require=EXTRAS_REQUIRES,
     dependency_links=DEPEND_LINKS,
Repository: rucio/rucio
Issue: DuplicateRule error creates index out of bound stacktrace

Motivation
----------
```
File "/usr/lib/python2.7/site-packages/rucio/web/rest/rule.py", line 271, in POST
raise generate_http_error(409, 'DuplicateRule', error.args[0])
IndexError: tuple index out of range
```
[0] needs to be removed.


Changed files:
lib/rucio/web/rest/webpy/v1/rule.py

diff --git a/lib/rucio/web/rest/webpy/v1/rule.py b/lib/rucio/web/rest/webpy/v1/rule.py
index 1f261ea78..173011322 100755
--- a/lib/rucio/web/rest/webpy/v1/rule.py
+++ b/lib/rucio/web/rest/webpy/v1/rule.py
@@ -1,15 +1,25 @@
-'''
-  Copyright European Organization for Nuclear Research (CERN)
+# Copyright 2012-2018 CERN for the benefit of the ATLAS collaboration.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Authors:
+# - Vincent Garonne <vgaronne@gmail.com>, 2012-2017
+# - Mario Lassnig <mario.lassnig@cern.ch>, 2013-2018
+# - Thomas Beermann <thomas.beermann@cern.ch>, 2013-2018
+# - Martin Barisits <martin.barisits@cern.ch>, 2013-2017
+# - Cedric Serfon <cedric.serfon@cern.ch>, 2014-2017
+# - Joaquin Bogado <jbogado@linti.unlp.edu.ar>, 2018
 
-  Licensed under the Apache License, Version 2.0 (the "License");
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
-
-  Authors:
-  - Vincent Garonne, <vincent.garonne@cern.ch>, 2012-2017
-  - Martin Barisits, <martin.barisits@cern.ch>, 2013-2017
-  - Cedric Serfon, <cedric.serfon@cern.ch>, 2015, 2017
-'''
 
 from logging import getLogger, StreamHandler, DEBUG
 from json import dumps, loads
@@ -268,7 +278,7 @@ class AllRule:
         except InvalidReplicationRule as error:
             raise generate_http_error(409, 'InvalidReplicationRule', error.args[0][0])
         except DuplicateRule as error:
-            raise generate_http_error(409, 'DuplicateRule', error.args[0])
+            raise generate_http_error(409, 'DuplicateRule', error.args)
         except InsufficientTargetRSEs as error:
             raise generate_http_error(409, 'InsufficientTargetRSEs', error.args[0][0])
         except InsufficientAccountLimit as error:
Repository: rucio/rucio
Issue: transfers: verify_checksum badly propagated

Motivation
----------
The verify_checksum option is nested too deep in the submission JSON to FTS. Needs to be elevated one level higher.

Changed files:
lib/rucio/daemons/conveyor/common.py

diff --git a/lib/rucio/daemons/conveyor/common.py b/lib/rucio/daemons/conveyor/common.py
index 5e20fbf41..aea665d3b 100644
--- a/lib/rucio/daemons/conveyor/common.py
+++ b/lib/rucio/daemons/conveyor/common.py
@@ -1,17 +1,27 @@
-# Copyright European Organization for Nuclear Research (CERN)
+# Copyright 2014-2018 CERN for the benefit of the ATLAS collaboration.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
-# You may not use this file except in compliance with the License.
-# You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
 #
 # Authors:
-# - Vincent Garonne, <vincent.garonne@cern.ch>, 2012-2014
-# - Mario Lassnig, <mario.lassnig@cern.ch>, 2013-2018
-# - Cedric Serfon, <cedric.serfon@cern.ch>, 2013-2018
-# - Wen Guan, <wen.guan@cern.ch>, 2014-2016
-# - Joaquin Bogado, <jbogadog@cern.ch>, 2016
-# - Martin Barisits, <martin.barisits@cern.ch>, 2017
-# - Eric Vaandering, <ewv@fnal.gov>, 2018
+# - Mario Lassnig <mario.lassnig@cern.ch>, 2014-2018
+# - Cedric Serfon <cedric.serfon@cern.ch>, 2014-2018
+# - Vincent Garonne <vgaronne@gmail.com>, 2014-2016
+# - Martin Barisits <martin.barisits@cern.ch>, 2014-2017
+# - Wen Guan <wguan.icedew@gmail.com>, 2014-2016
+# - Joaquin Bogado <jbogado@linti.unlp.edu.ar>, 2016
+# - Thomas Beermann <thomas.beermann@cern.ch>, 2016
+# - Brian Bockelman <bbockelm@cse.unl.edu>, 2018
+# - Eric Vaandering <ericvaandering@gmail.com>, 2018
 
 """
 Methods common to different conveyor submitter daemons.
@@ -148,7 +158,7 @@ def bulk_group_transfer(transfers, policy='rule', group_bulk=200, fts_source_str
             grouped_transfers[external_host] = {}
             grouped_jobs[external_host] = []
 
-        verify_checksum = transfer.get('verify_checksum', 'both')
+        verify_checksum = transfer['file_metadata'].get('verify_checksum', 'both')
         file = {'sources': transfer['sources'],
                 'destinations': transfer['dest_urls'],
                 'metadata': transfer['file_metadata'],
Repository: rucio/rucio
Issue: RSE manager: remove .part from filename if adler32 is set and ignore_checksum is true

Motivation
----------
rsemanager.download, downloads the file as <filename>.part if an adler32 checksum is given.
If ignore_checksum is set to True the file will not be renamed thus the .part will not be removed from the file name.

Modification
------------
Make the  os.rename call independent of ignore_checksum

Changed files:
lib/rucio/rse/rsemanager.py

diff --git a/lib/rucio/rse/rsemanager.py b/lib/rucio/rse/rsemanager.py
index a78d0df7b..4bb5ffb0f 100755
--- a/lib/rucio/rse/rsemanager.py
+++ b/lib/rucio/rse/rsemanager.py
@@ -267,16 +267,19 @@ def download(rse_settings, files, dest_dir=None, force_scheme=None, ignore_check
                     if printstatements:
                         print('File downloaded. Will be validated')
 
-                    if not ignore_checksum:
+                    if ignore_checksum:
+                        if printstatements:
+                            print('Skipping checksum validation')
+                    else:
                         ruciochecksum = f['adler32'] if f['adler32'] else f['md5']
                         localchecksum = utils.adler32(tempfile) if f['adler32'] else utils.md5(tempfile)
                         if localchecksum == ruciochecksum:
                             if printstatements:
                                 print('File validated')
-                            os.rename(tempfile, finalfile)
                         else:
                             os.unlink(tempfile)
                             raise exception.FileConsistencyMismatch('Checksum mismatch : local %s vs recorded %s' % (str(localchecksum), str(ruciochecksum)))
+                    os.rename(tempfile, finalfile)
                 else:
                     protocol.get(pfn, '%s/%s' % (target_dir, f['name']), transfer_timeout=transfer_timeout)
                 ret['%s:%s' % (f['scope'], f['name'])] = True
Repository: rucio/rucio
Issue: Transfers : Bug with verify checksum

Motivation
----------
The conveyor crash if verify_checksum is a bool instead of a string




Changed files:
lib/rucio/core/transfer.py

diff --git a/lib/rucio/core/transfer.py b/lib/rucio/core/transfer.py
index d4e5acd1b..0e6b1c172 100644
--- a/lib/rucio/core/transfer.py
+++ b/lib/rucio/core/transfer.py
@@ -662,13 +662,13 @@ def get_transfer_requests_and_source_replicas(process=None, total_processes=None
                 fts_list = fts_hosts.split(",")
 
                 verify_checksum = 'both'
-                if rse_attrs[dest_rse_id].get('verify_checksum', 'True').lower() == 'false':
-                    if rse_attrs[source_rse_id].get('verify_checksum', 'True').lower() == 'false':
+                if not rse_attrs[dest_rse_id].get('verify_checksum', True):
+                    if not rse_attrs[source_rse_id].get('verify_checksum', True):
                         verify_checksum = 'none'
                     else:
                         verify_checksum = 'source'
                 else:
-                    if rse_attrs[source_rse_id].get('verify_checksum', 'True').lower() == 'false':
+                    if not rse_attrs[source_rse_id].get('verify_checksum', True):
                         verify_checksum = 'destination'
                     else:
                         verify_checksum = 'both'
Repository: rucio/rucio
Issue: core.account_limit.get_rse_account_usage should liste accounts who have usage, but no quota

Motivation
----------
Due to the join of the two tables, only accounts with usage and quota are listed, however, accounts without quota should be listed as well.

Changed files:
lib/rucio/core/account_limit.py

diff --git a/lib/rucio/core/account_limit.py b/lib/rucio/core/account_limit.py
index 77409dece..2f9021aba 100644
--- a/lib/rucio/core/account_limit.py
+++ b/lib/rucio/core/account_limit.py
@@ -1,16 +1,21 @@
-'''
-  Copyright European Organization for Nuclear Research (CERN)
-
-  Licensed under the Apache License, Version 2.0 (the "License");
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-  http://www.apache.org/licenses/LICENSE-2.0
-
-  Authors:
-  - Martin Barisits, <martin.barisits@cern.ch>, 2013-2015
-  - Cedric Serfon, <cedric.serfon@cern.ch>, 2015
-  - Vincent Garonne, <vincent.garonne@cern.ch>, 2015
-'''
+# Copyright 2013-2018 CERN for the benefit of the ATLAS collaboration.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Authors:
+#  - Martin Barisits, <martin.barisits@cern.ch>, 2013-2018
+#  - Cedric Serfon, <cedric.serfon@cern.ch>, 2015
+#  - Vincent Garonne, <vincent.garonne@cern.ch>, 2015
 
 from sqlalchemy.orm.exc import NoResultFound
 from sqlalchemy.sql.expression import and_, or_
@@ -32,7 +37,7 @@ def get_rse_account_usage(rse, session=None):
     result = []
     rse_id = get_rse_id(rse=rse, session=session)
     query = session.query(models.AccountUsage.account, models.AccountUsage.files, models.AccountUsage.bytes, models.AccountLimit.bytes)
-    query = query.join(models.AccountLimit, and_(models.AccountUsage.account == models.AccountLimit.account, models.AccountUsage.rse_id == models.AccountLimit.rse_id)).filter(models.AccountUsage.rse_id == rse_id)
+    query = query.outerjoin(models.AccountLimit, and_(models.AccountUsage.account == models.AccountLimit.account, models.AccountUsage.rse_id == models.AccountLimit.rse_id)).filter(models.AccountUsage.rse_id == rse_id)
     account_limits_tmp = query.all()
     for row in account_limits_tmp:
         result.append({'rse': rse, 'account': row[0], 'used_files': row[1], 'used_bytes': row[2], 'quota_bytes': row[3]})
Repository: rucio/rucio
Issue: clients: download api, logger

Motivation
----------

STAGEIN FAILED: Get error: Staging input file failed: lfn=EVNT.04972714._000032.pool.root.1, error=stageIn with API faied: Logger instance has no attribute 'getChild'

Modification
------------

can be realted to:
https://github.com/dsoprea/GDriveFS/issues/104



Changed files:
lib/rucio/client/uploadclient.py, lib/rucio/client/downloadclient.py

diff --git a/lib/rucio/client/downloadclient.py b/lib/rucio/client/downloadclient.py
index 78a67eb66..02d94c46e 100644
--- a/lib/rucio/client/downloadclient.py
+++ b/lib/rucio/client/downloadclient.py
@@ -50,8 +50,8 @@ class DownloadClient:
         :param logger: logging.Logger object to use for downloads. If None nothing will be logged.
         """
         if not logger:
-            logger = logging.getLogger(__name__).getChild('null')
-            logger.addHandler(logging.NullHandler())
+            logger = logging.getLogger('%s.null' % __name__)
+            logger.disabled = True
 
         self.logger = logger
         self.is_human_readable = True
diff --git a/lib/rucio/client/uploadclient.py b/lib/rucio/client/uploadclient.py
index 0d4fad6fe..5527ebf82 100644
--- a/lib/rucio/client/uploadclient.py
+++ b/lib/rucio/client/uploadclient.py
@@ -49,8 +49,8 @@ class UploadClient:
         :param logger:      - logging.Logger object to use for uploads. If None nothing will be logged.
         """
         if not logger:
-            logger = logging.getLogger(__name__).getChild('null')
-            logger.addHandler(logging.NullHandler())
+            logger = logging.getLogger('%s.null' % __name__)
+            logger.disabled = True
 
         self.logger = logger
         self.client = _client if _client else Client()