Repository: rucio/rucio

Issue:
Provide a better error description for list_dataset_replicas_bulk client API
Motivation
----------
As mentioned in this GH issue comment:
https://github.com/rucio/rucio/issues/2459#issuecomment-686940735

if a call to the `list_dataset_replicas_bulk` API is not properly constructed and it fails on the server side, the server should respond with an error that best describe what the problem was, instead of responding with:
```
rucio.common.exception.RucioException: An unknown exception occurred.
Details: no error information passed (http status code: 500 ('internal_server_error', 'server_error', '/o\\', '\xe2\x9c\x97'))
``` 

In addition to that, perhaps the server http-error-logs could be a bit more verbose when these errors happen, instead of simply have a single line such as:
```
[core:error] ... Premature end of script headers: replica.py
```

NOTE: I'm not sure whether this same generic error can be returned for other APIs, but I have definitely seen it while calling a few REST APIs that require parameters to be encoded (maybe the `/rules` one).

Modification
------------
Ideally, the end user should be able to know what the problem was with his/her query/parameters. Which means, the server should report a more specific error message.



Discussion:
In principle I agree and we try to do that, but I think in this case this was really the issue with the schema, which is not all that easy to catch. 
> In principle I agree and we try to do that, but I think in this case this was really the issue with the schema, which is not all that easy to catch.

@bari12 I can remember seeing a nice and descriptive error on the server-side, which was not propagated to the client-side. I will see if I can fix this, since I am already looking at a lot of the APIs right now

Resolving patch:
diff --git a/lib/rucio/common/utils.py b/lib/rucio/common/utils.py
--- a/lib/rucio/common/utils.py
+++ b/lib/rucio/common/utils.py
@@ -480,7 +480,10 @@ def strip_newlines(msg):
     exc_msg = strip_newlines(exc_msg)
     if exc_msg:
         # Truncate too long exc_msg
-        exc_msg = exc_msg[:min(len(exc_msg), 15000)]
+        oldlen = len(exc_msg)
+        exc_msg = exc_msg[:min(oldlen, 125)]
+        if len(exc_msg) != oldlen:
+            exc_msg = exc_msg + '...'
     headers = {'Content-Type': 'application/octet-stream',
                'ExceptionClass': strip_newlines(exc_cls),
                'ExceptionMessage': exc_msg}


Test patch:
diff --git a/lib/rucio/tests/test_dataset_replicas.py b/lib/rucio/tests/test_dataset_replicas.py
--- a/lib/rucio/tests/test_dataset_replicas.py
+++ b/lib/rucio/tests/test_dataset_replicas.py
@@ -1,4 +1,5 @@
-# Copyright 2015-2020 CERN for the benefit of the ATLAS collaboration.
+# -*- coding: utf-8 -*-
+# Copyright 2015-2020 CERN
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -26,10 +27,13 @@
 
 import unittest
 
+import pytest
+
 from rucio.client.didclient import DIDClient
 from rucio.client.replicaclient import ReplicaClient
 from rucio.client.ruleclient import RuleClient
 from rucio.common.config import config_get, config_get_bool
+from rucio.common.exception import InvalidObject
 from rucio.common.types import InternalAccount, InternalScope
 from rucio.common.utils import generate_uuid
 from rucio.core.did import attach_dids, add_did, add_dids
@@ -77,6 +81,10 @@ def test_list_dataset_replicas_bulk(self):
         rule_client.add_replication_rule(dids=dids,
                                          account='root', copies=1, rse_expression='MOCK',
                                          grouping='DATASET')
+
+        with pytest.raises(InvalidObject):
+            replica_client.list_dataset_replicas_bulk(dids=[{'type': "I'm Different"}])
+
         replicas = list(replica_client.list_dataset_replicas_bulk(dids=dids))
 
         assert len(replicas) == 2