Repository: StackStorm/st2

Issue:
When trying to get a key from the datastore, st2api logs an error but also prints the traceback
## SUMMARY

Provide a quick summary of your bug report.

### STACKSTORM VERSION

v3.2.0

##### OS, environment, install method

Post what OS you are running this on, along with any other relevant information/

CentOS 7 / 8

## Steps to reproduce the problem

```
def get_value(self, name, local=True, scope=SYSTEM_SCOPE, decrypt=False):
        """
        Retrieve a value from the datastore for the provided key.
        By default, value is retrieved from the namespace local to the pack/class. If you want to
        retrieve a global value from a datastore, pass local=False to this method.
        :param name: Key name.
        :type name: ``str``
        :param local: Retrieve value from a namespace local to the pack/class. Defaults to True.
        :type: local: ``bool``
        :param scope: Scope under which item is saved. Defaults to system scope.
        :type: local: ``str``
        :param decrypt: Return the decrypted value. Defaults to False.
        :type: local: ``bool``
        :rtype: ``str`` or ``None``
        """
        if scope != SYSTEM_SCOPE:
            raise ValueError('Scope %s is unsupported.' % scope)

        name = self._get_full_key_name(name=name, local=local)

        client = self.get_api_client()
        self._logger.debug('Retrieving value from the datastore (name=%s)', name)

        try:
            params = {'decrypt': str(decrypt).lower(), 'scope': scope}
            kvp = client.keys.get_by_id(id=name, params=params)
        except Exception as e:
            self._logger.exception(
                'Exception retrieving value from datastore (name=%s): %s',
                name,
                e
            )
            return None

        if kvp:
            return kvp.value
```

## Expected Results

I expect to get an ERROR, but not the traceback

## Actual Results

```
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]: 2020-06-26 18:27:35,134 ERROR [-] Failed to call controller function "get_one" for operation "st2api.controllers.v1.keyvalue:key_value_pair_controller.get_one": The key "aws_inspector.AWSInspector:us-east-2.last_start_time_" does not exist in the StackStorm datastore.
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]: Traceback (most recent call last):
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]:  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/router.py", line 515, in __call__
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]:    resp = func(**kw)
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]:  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2api/controllers/v1/keyvalue.py", line 124, in get_one
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]:    from_model_kwargs=from_model_kwargs
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]:  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2api/controllers/resource.py", line 402, in _get_one_by_scope_and_name
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]:    instance = self.access.get_by_scope_and_name(scope=scope, name=name)
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]:  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/persistence/keyvalue.py", line 115, in get_by_scope_and_name
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]:    raise StackStormDBObjectNotFoundError(msg % name)
Jun 26 18:27:35 ip-10-15-1-121 gunicorn[184385]: st2common.exceptions.db.StackStormDBObjectNotFoundError: The key "aws_inspector.AWSInspector:us-east-2.last_start_time_" does not exist in the StackStorm datastore.sod
```
Thanks!



Discussion:

Resolving patch:
diff --git a/st2common/st2common/exceptions/keyvalue.py b/st2common/st2common/exceptions/keyvalue.py
--- a/st2common/st2common/exceptions/keyvalue.py
+++ b/st2common/st2common/exceptions/keyvalue.py
@@ -14,9 +14,11 @@
 
 from __future__ import absolute_import
 from st2common.exceptions import StackStormBaseException
+from st2common.exceptions.db import StackStormDBObjectNotFoundError
 
 __all__ = [
     'CryptoKeyNotSetupException',
+    'DataStoreKeyNotFoundError',
     'InvalidScopeException'
 ]
 
@@ -25,6 +27,10 @@ class CryptoKeyNotSetupException(StackStormBaseException):
     pass
 
 
+class DataStoreKeyNotFoundError(StackStormDBObjectNotFoundError):
+    pass
+
+
 class InvalidScopeException(StackStormBaseException):
     pass
 
diff --git a/st2common/st2common/persistence/keyvalue.py b/st2common/st2common/persistence/keyvalue.py
--- a/st2common/st2common/persistence/keyvalue.py
+++ b/st2common/st2common/persistence/keyvalue.py
@@ -19,7 +19,7 @@
 from st2common.constants.triggers import KEY_VALUE_PAIR_UPDATE_TRIGGER
 from st2common.constants.triggers import KEY_VALUE_PAIR_VALUE_CHANGE_TRIGGER
 from st2common.constants.triggers import KEY_VALUE_PAIR_DELETE_TRIGGER
-from st2common.exceptions.db import StackStormDBObjectNotFoundError
+from st2common.exceptions.keyvalue import DataStoreKeyNotFoundError
 from st2common.models.api.keyvalue import KeyValuePairAPI
 from st2common.models.db.keyvalue import keyvaluepair_access
 from st2common.models.system.common import ResourceReference
@@ -112,7 +112,7 @@ def get_by_scope_and_name(cls, scope, name):
 
         if not query_result:
             msg = 'The key "%s" does not exist in the StackStorm datastore.'
-            raise StackStormDBObjectNotFoundError(msg % name)
+            raise DataStoreKeyNotFoundError(msg % name)
 
         return query_result.first() if query_result else None
 
diff --git a/st2common/st2common/router.py b/st2common/st2common/router.py
--- a/st2common/st2common/router.py
+++ b/st2common/st2common/router.py
@@ -32,6 +32,7 @@
 
 from st2common.exceptions import rbac as rbac_exc
 from st2common.exceptions import auth as auth_exc
+from st2common.exceptions.keyvalue import DataStoreKeyNotFoundError
 from st2common import log as logging
 from st2common.persistence.auth import User
 from st2common.rbac.backends import get_rbac_backend
@@ -513,6 +514,10 @@ def __init__(self, **entries):
 
         try:
             resp = func(**kw)
+        except DataStoreKeyNotFoundError as e:
+            LOG.warning('Failed to call controller function "%s" for operation "%s": %s' %
+                       (func.__name__, endpoint['operationId'], six.text_type(e)))
+            raise e
         except Exception as e:
             LOG.exception('Failed to call controller function "%s" for operation "%s": %s' %
                           (func.__name__, endpoint['operationId'], six.text_type(e)))


Test patch: