Repository: StackStorm/st2

Issue:
Decrypting K/V values in action parameters causing misleading errors if the keys don't exist and another K/V parameter needs to be converted to bool, array, or object
## SUMMARY

In an action's parameters, decrypting an encrypted value in the K/V store results in a misleading error message if that key either doesn't exist or is not encrypted

### STACKSTORM VERSION

3.5.0

##### OS, environment, install method

RHEL 8, Python 3.6.8

## Steps to reproduce the problem

st2 key set test_bool "true"

test_action.yaml
```
---
name: 'test_action'
runner_type: 'remote-shell-cmd'
description: 'Test action inputs from the k/v store'
enabled: true
entry_point: ''
pack: 'test_pack'
parameters:
    username:
        type: 'string'
        description: 'Username to login to the remote system'
        required: true
    password:
        type: 'string'
        description: 'Password to login to the remote system'
        required: true
    hosts:
        type: 'string'
        description: 'Comma separated list of hosts to run the command on'
        required: true
    cmd:
        immutable: true
        default: 'echo "TEST: {{ test_bool }}"'
    connect_timeout:
        type: 'integer'
        description: 'SSH connect timeout in seconds'
        default: 30
    test_bool:
        type: boolean
        description: "Test boolean variable"
        default: "{{ st2kv.system.test_bool }}"
    test_encrypt:
      	type: string
        description: "Test encrypted variable"
        required: false
        secret: true
        default: "{{ st2kv.system.test_encrypt | decrypt_kv }}"
```

st2 run test_pack.test_action username="root" password="pass" hosts="localhost"

## Expected Results

Prior to 3.5, if test_encrypt does not exist in the K/V store then a None value would be used for that parameter. If that can't be done then an error message about the test_encrypt parameter would be expected

## Actual Results

Running the action above will result in the following misleading error if test_encrypt either is not in the k/v store or if it exists but isn't encrypted:
```
ERROR: 400 Client Error: Bad Request
MESSAGE: '{{ st2kv.system.test_bool }}' is not of type 'boolean' for url: http://127.0.0.1:9101/executions
```



Discussion:
@jschoewe The error being reported is correct, as it is not about the value of test_encrypt that is the problem.

Test_bool is being set as a string value of "true", and therefore ST2 complains that is not of type boolean. Hence the error you see.

If I change the action so test_bool is a string, and dont' have test_encrypt specified then I get an error:
```
MESSAGE: Failed to render parameter "test_encrypt": Referenced datastore item "st2kv.system.test_encrypt" doesn't exist or it contains an empty string for url: http://127.0.0.1:9101/v1/executions
```
@amanda11 ```it is not about the value of test_encrypt that is the problem```
Yes it is because when I add a correct test_encrypt value to the k/v store the action runs fine.
```Test_bool is being set as a string value of "true", and therefore ST2 complains that is not of type boolean. Hence the error you see.```
This is not right, stackstorm is smart enough to convert strings from the k/v store into booleans, arrays, objects, etc.
```If I change the action so test_bool is a string, and dont' have test_encrypt specified then I get an error:```
I don't want test_bool to be a string, and this doesn't change the fact that when it is a boolean it will cause hours of needless debugging due to a misleading error. This is still an issue that needs to be addressed.

The error message in the example is about the test_bool parameter, not test_encrypt.

If I took test_bool out of the problem, then I couldn't reproduce the problem you are seeing. And got a clear error when test_encrypt was not in the store.

Could you perhaps provide an example that just has the encrypted parameter and provide the error you get?

As from my attempts to reproduce with the information given it is complaining that it has a string being provided for a Boolean, and not related to the test_encrypt parameter. 
(It is possible that the behaviour of string to Boolean on key value store has changed - so trying to narrow down the problem).
I am wondering as well if it's a combination of the two things, test_bool and test_encrypt.

I will try and confirm that if I keep the set on the test_bool as a Boolean and put the encrypted string in for test_encrypt if it works for me.

I am wondering if the action has just a parameter that is encrypted, and nothing else from the key value store - do you get the right error? 

But if have both a variable that it has to convert type and a encrypted that isn't set then you get the wrong error.

As when I tried to simplify down to just the encrypted field I couldn't reproduce.

So it would be good if we can simplify the case to see if it's a combination of parameters that is causing your issue.
Sorry, I probably could have been more descriptive but I do think it's an issue with a combination of the 2. I am getting the correct error if I don't have to convert anything and I'm also getting bad type errors trying to pass arrays and objects from the k/v store too.
I've verified it on my system, and agree it's a combination of the two that causes the error.

1. Action expects test_bool as string, and test_encrypt as encrypted.
Send in no value for test_encrypt get error:
```
MESSAGE: Failed to render parameter "test_encrypt": Referenced datastore item "st2kv.system.test_encrypt" doesn't exist or it contains an empty string for url: http://127.0.0.1:9101/v1/executions
```

2. Action expects test_bool as boolean, and test_encrypt as encrypted.
Send in no value for test_encrypt get error:
```
ERROR: 400 Client Error: Bad Request
MESSAGE: '{{ st2kv.system.test_bool }}' is not of type 'boolean' for url: http://127.0.0.1:9101/v1/executions
```
 
3. Action expects test_bool as boolean, and test_encrypt as encrypted
Set test_encrypt to an encrypted string.
Result. Input parameters expected 

So I think we agree that it's the combination of both that causes the error. I'll change status to reproduced.

Would you mind amending the title so that we know it's a combination of the two that is the problem, that will make it clearer when trying to identify a fix.
The problem occurs because when the ParamException is thrown for the invalid value for test_encrypt, it is caught at:
https://github.com/StackStorm/st2/blob/ce1abf95477741d10bcebce358034f90b80b6ab8/st2api/st2api/controllers/v1/actionexecutions.py#L218

However in processing that failure, the call to create_request does a validate of parameters, and this doesn't seem to take into account the key/value conversion, and so throws a json error about test_bool.

Not sure, what has changed as couldn't find anything that broke it. Have applied a workaround fix locally so that it doesn't validate the parameter when it's just creating that request so that it can report it as failure. This does mean we then get the correct error returned, but need to do further investigation to see whether that is a viable fix or not.



Resolving patch:
diff --git a/st2api/st2api/controllers/v1/actionexecutions.py b/st2api/st2api/controllers/v1/actionexecutions.py
--- a/st2api/st2api/controllers/v1/actionexecutions.py
+++ b/st2api/st2api/controllers/v1/actionexecutions.py
@@ -219,6 +219,7 @@ def _schedule_execution(
                 liveaction=liveaction_db,
                 action_db=action_db,
                 runnertype_db=runnertype_db,
+                validate_params=False,
             )
 
             # By this point the execution is already in the DB therefore need to mark it failed.
diff --git a/st2common/st2common/services/action.py b/st2common/st2common/services/action.py
--- a/st2common/st2common/services/action.py
+++ b/st2common/st2common/services/action.py
@@ -55,7 +55,9 @@ def _get_immutable_params(parameters):
     return [k for k, v in six.iteritems(parameters) if v.get("immutable", False)]
 
 
-def create_request(liveaction, action_db=None, runnertype_db=None):
+def create_request(
+    liveaction, action_db=None, runnertype_db=None, validate_params=True
+):
     """
     Create an action execution.
 
@@ -66,6 +68,9 @@ def create_request(liveaction, action_db=None, runnertype_db=None):
     :param runnertype_db: Runner model to operate one. If not provided, one is retrieved from the
                           database using values from "liveaction".
     :type runnertype_db: :class:`RunnerTypeDB`
+    :param validate_params: Whether to validate parameters against schema. Default to True, but
+                          set to False when raising a request to report an error.
+    :type validate_params: ``bool``
 
     :return: (liveaction, execution)
     :rtype: tuple
@@ -108,13 +113,14 @@ def create_request(liveaction, action_db=None, runnertype_db=None):
     # Validate action parameters.
     schema = util_schema.get_schema_for_action_parameters(action_db, runnertype_db)
     validator = util_schema.get_validator()
-    util_schema.validate(
-        liveaction.parameters,
-        schema,
-        validator,
-        use_default=True,
-        allow_default_none=True,
-    )
+    if validate_params:
+        util_schema.validate(
+            liveaction.parameters,
+            schema,
+            validator,
+            use_default=True,
+            allow_default_none=True,
+        )
 
     # validate that no immutable params are being overriden. Although possible to
     # ignore the override it is safer to inform the user to avoid surprises.


Test patch:
diff --git a/st2api/tests/unit/controllers/v1/test_executions.py b/st2api/tests/unit/controllers/v1/test_executions.py
--- a/st2api/tests/unit/controllers/v1/test_executions.py
+++ b/st2api/tests/unit/controllers/v1/test_executions.py
@@ -198,6 +198,26 @@
         "non_secret_param_2": {"type": "string", "required": True},
     },
 }
+ACTION_DEFAULT_ENCRYPT_AND_BOOL = {
+    "name": "st2.dummy.default_encrypted_and_bool",
+    "description": "An action that uses a jinja template with decrypt_kv filter "
+    "in default parameter",
+    "enabled": True,
+    "pack": "starterpack",
+    "runner_type": "local-shell-cmd",
+    "parameters": {
+        "encrypted_param": {
+            "type": "string",
+            "default": "{{ st2kv.system.secret | decrypt_kv }}",
+            "secret": True,
+        },
+        "bool_param": {
+            "type": "boolean",
+            "default": "{{ st2kv.system.test_bool }}",
+        },
+    },
+}
+
 
 LIVE_ACTION_1 = {
     "action": "sixpack.st2.dummy.action1",
@@ -298,6 +318,9 @@
 LIVE_ACTION_DEFAULT_ENCRYPT = {
     "action": "starterpack.st2.dummy.default_encrypted_value",
 }
+LIVE_ACTION_DEFAULT_ENCRYPT_AND_BOOL = {
+    "action": "starterpack.st2.dummy.default_encrypted_and_bool",
+}
 LIVE_ACTION_DEFAULT_ENCRYPT_SECRET_PARAM = {
     "action": "starterpack.st2.dummy.default_encrypted_value_secret_param",
 }
@@ -359,6 +382,10 @@ def setUpClass(cls):
         post_resp = cls.app.post_json("/v1/actions", cls.action_decrypt)
         cls.action_decrypt["id"] = post_resp.json["id"]
 
+        cls.action_decrypt_and_bool = copy.deepcopy(ACTION_DEFAULT_ENCRYPT_AND_BOOL)
+        post_resp = cls.app.post_json("/v1/actions", cls.action_decrypt_and_bool)
+        cls.action_decrypt_and_bool["id"] = post_resp.json["id"]
+
         cls.action_decrypt_secret_param = copy.deepcopy(
             ACTION_DEFAULT_ENCRYPT_SECRET_PARAMS
         )
@@ -390,6 +417,8 @@ def tearDownClass(cls):
         cls.app.delete("/v1/actions/%s" % cls.action_inquiry["id"])
         cls.app.delete("/v1/actions/%s" % cls.action_template["id"])
         cls.app.delete("/v1/actions/%s" % cls.action_decrypt["id"])
+        cls.app.delete("/v1/actions/%s" % cls.action_decrypt_secret_param["id"])
+        cls.app.delete("/v1/actions/%s" % cls.action_decrypt_and_bool["id"])
         cls.app.delete(
             "/v1/actions/%s" % cls.action_with_output_schema_secret_param["id"]
         )
@@ -913,6 +942,26 @@ def test_template_param(self):
         # was not rendered, and the provided parameter was used
         self.assertEqual(post_resp.json["parameters"]["intparam"], live_int_param)
 
+    def test_template_encrypted_and_bool(self):
+        # register datastore values which are used in this test case
+        KeyValuePairAPI._setup_crypto()
+        register_items = [
+            {
+                "name": "test_bool",
+                "value": "true",
+            },
+        ]
+        [KeyValuePair.add_or_update(KeyValuePairDB(**x)) for x in register_items]
+
+        post_resp = self._do_post(
+            LIVE_ACTION_DEFAULT_ENCRYPT_AND_BOOL, expect_errors=True
+        )
+        self.assertEqual(post_resp.status_int, 400)
+        self.assertEqual(
+            post_resp.json["faultstring"],
+            'Failed to render parameter "encrypted_param": Referenced datastore item "st2kv.system.secret" doesn\'t exist or it contains an empty string',
+        )
+
     def test_template_encrypted_params(self):
         # register datastore values which are used in this test case
         KeyValuePairAPI._setup_crypto()
diff --git a/st2common/tests/unit/services/test_action.py b/st2common/tests/unit/services/test_action.py
--- a/st2common/tests/unit/services/test_action.py
+++ b/st2common/tests/unit/services/test_action.py
@@ -624,3 +624,25 @@ def test_root_liveaction(self):
             child, expected_root = self._create_nested_executions(depth=i)
             actual_root = action_service.get_root_liveaction(child)
             self.assertEqual(expected_root["id"], actual_root["id"])
+
+    def test_invalid_json_request_validate(self):
+        parameters = {"hosts": "127.0.0.1", "cmd": "uname -a", "arg_default_value": 123}
+        liveaction = LiveActionDB(action=ACTION_REF, parameters=parameters)
+
+        self.assertRaisesRegexp(
+            jsonschema.ValidationError,
+            "123 is not of type 'string'",
+            action_service.create_request,
+            liveaction,
+        )
+
+    def test_invalid_json_request_skipvalidate(self):
+        parameters = {"hosts": "127.0.0.1", "cmd": "uname -a", "arg_default_value": 123}
+        liveaction = LiveActionDB(action=ACTION_REF, parameters=parameters)
+
+        # Validate that if skip validation that no exception raised
+        (action, execution) = action_service.create_request(
+            liveaction, validate_params=False
+        )
+        self.assertTrue(action)
+        self.assertTrue(execution)