Repository: StackStorm/st2

Issue:
Jinja is not rendered in action default values prior to schema checking
# Problem

When trying to call an action with Jinja for its default value an error is raised complaining about a type mismatch.

## Action metadata

```yaml
---
description: Run a local linux command
enabled: true
runner_type: mistral-v2
entry_point: workflows/mistral-basic.yaml
name: mistral-basic
pack: examples
parameters:
  cmd:
    required: true
    type: string
  timeout:
    type: integer
    default: 60
  kv_test:
    type: integer
    default: "{{ st2kv.system.nick_test | int }}"
```

## Error - CLI

``` shell
[root@stackstorm st2]# st2 key set nick_test 123
+------------------+-----------+
| Property         | Value     |
+------------------+-----------+
| name             | nick_test |
| value            | 123       |
| expire_timestamp |           |
+------------------+-----------+

[root@stackstorm st2]# st2 run examples.mistral-basic cmd="ls"
ERROR: 400 Client Error: Bad Request
MESSAGE: '{{ st2kv.system.nick_test | int }}' is not of type 'integer' for url: http://127.0.0.1:9101/v1/executions
```

## Error - Logs

``` shell
2017-10-30 16:03:59,072 107609136 ERROR actionexecutions [-] Unable to execute action. Parameter validation failed.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2api/controllers/v1/actionexecutions.py", line 128, in _handle_schedule_execution
    pack=action_db.pack)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2api/controllers/v1/actionexecutions.py", line 177, in _schedule_execution
    liveaction_db, actionexecution_db = action_service.create_request(liveaction_db)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/services/action.py", line 87, in create_request
    allow_default_none=True)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/util/schema/__init__.py", line 293, in validate
    jsonschema.validate(instance=instance, schema=schema, cls=cls, *args, **kwargs)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/jsonschema/validators.py", line 541, in validate
    cls(schema, *args, **kwargs).validate(instance)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/jsonschema/validators.py", line 130, in validate
    raise error
ValidationError: u'{{ st2kv.system.nick_test | int }}' is not of type u'integer'

Failed validating u'type' in schema['properties'][u'kv_test']:
    {u'default': u'{{ st2kv.system.nick_test | int }}',
     u'type': u'integer'}

On instance[u'kv_test']:
    u'{{ st2kv.system.nick_test | int }}'
2017-10-30 16:03:59,072 107609136 ERROR router [-] Failed to call controller function "post" for operation "st2api.controllers.v1.actionexecutions:action_executions_controller.post": '{{ st2kv.system.nick_test | int }}' is not of type 'integer'
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/router.py", line 434, in __call__
    resp = func(**kw)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2api/controllers/v1/actionexecutions.py", line 575, in post
    show_secrets=show_secrets)
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2api/controllers/v1/actionexecutions.py", line 134, in _handle_schedule_execution
    abort(http_client.BAD_REQUEST, re.sub("u'([^']*)'", r"'\1'", e.message))
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/router.py", line 53, in abort
    raise exc.status_map[status_code](message)
HTTPBadRequest: '{{ st2kv.system.nick_test | int }}' is not of type 'integer'
2017-10-30 16:03:59,073 107609136 INFO logging [-] 2c636a0c-d2ff-4405-9632-8268bd09d76b - 400 86 19.061ms (content_length=86,request_id='2c636a0c-d2ff-4405-9632-8268bd09d76b',runtime=19.061,remote_addr='127.0.0.1',status=400,method='POST',path='/v1/executions')
```


Discussion:
@nmaludy: The problem doesn't seem to be jinja filters. It is the quotes around the integer. The default value for integer type should be a number instead of string (this is expected behavior). 
Example:

```bash
$ st2  run examples.mistral-basic cmd="ls"
ERROR: 400 Client Error: Bad Request
MESSAGE: '2020' is not of type 'integer' for url: http://127.0.0.1:9101/v1/executions
```
### Action metadata
```yaml
---
description: Run a local linux command
enabled: true
runner_type: mistral-v2
entry_point: workflows/mistral-basic.yaml
name: mistral-basic
pack: examples
parameters:
  cmd:
    required: true
    type: string
  timeout:
    type: integer
    default: 60
  kv_test:
    type: integer
    default: "2020" 
```
-----------
and you are interested in providing default value for params. in action meta from datastore. For that you can do something like this:

### Action metadata (does't need jinja filter)
```yaml
---
description: Run a local linux command
enabled: true
runner_type: mistral-v2
entry_point: workflows/mistral-basic.yaml
name: mistral-basic
pack: examples
parameters:
  cmd:
    required: true
    type: string
  timeout:
    type: integer
    default: 60
  kv_test:
    type: string   #----> Use string and datastore will automatically take care of datatype for you.
    default: "{{ st2kv.system.nick_test}}"
```

### Workflow
```yaml
version: '2.0'

examples.mistral-basic:
    description: A basic workflow that runs an arbitrary linux command.
    type: direct
    input:
        - cmd
        - timeout
        - kv_test
    output:
        stdout: <% $.stdout %>
    tasks:
        task1:
            action: core.local cmd="if echo <% $.kv_test %> | egrep -q '^[0-9]+$'; then echo 'yes'; else echo 'no';fi"  #cmd=<% $.cmd %> timeout=<% $.timeout %>
            publish:
                stdout: <% task(task1).result.stdout %>
                stderr: <% task(task1).result.stderr %>
```

### String in datastore

```bash
$ st2 key set nick_test "123isnotnumber"
+------------------+----------------+
| Property         | Value          |
+------------------+----------------+
| name             | nick_test      |
| value            | 123isnotnumber |
| expire_timestamp |                |
+------------------+----------------+

$ st2  run examples.mistral-basic cmd=""
....
id: 5a0124a3de59a42542829c21
action.ref: examples.mistral-basic
parameters:
  cmd: ''
status: succeeded
result_task: task1
result:
  failed: false
  return_code: 0
  stderr: ''
  stdout: 'no'  #------> not a number
  succeeded: true
start_timestamp: 2017-11-07T03:12:35.552628Z
end_timestamp: 2017-11-07T03:12:42.013246Z
+--------------------------+------------------------+-------+------------+-------------------------------+
| id                       | status                 | task  | action     | start_timestamp               |
+--------------------------+------------------------+-------+------------+-------------------------------+
| 5a0124a3de59a42542829c24 | succeeded (1s elapsed) | task1 | core.local | Tue, 07 Nov 2017 03:12:35 UTC |
+--------------------------+------------------------+-------+------------+-------------------------------+
```
### Integer in datastore

```bash
$ st2 key set nick_test "123"
+------------------+-----------+
| Property         | Value     |
+------------------+-----------+
| name             | nick_test |
| value            | 123       |
| expire_timestamp |           |
+------------------+-----------+

$ st2  run examples.mistral-basic cmd=""
...
id: 5a0124bbde59a42542829c26
action.ref: examples.mistral-basic
parameters:
  cmd: ''
status: succeeded
result_task: task1
result:
  failed: false
  return_code: 0
  stderr: ''
  stdout: 'yes'  #----> number
  succeeded: true
start_timestamp: 2017-11-07T03:12:59.067428Z
end_timestamp: 2017-11-07T03:13:05.018635Z
+--------------------------+------------------------+-------+------------+-------------------------------+
| id                       | status                 | task  | action     | start_timestamp               |
+--------------------------+------------------------+-------+------------+-------------------------------+
| 5a0124bbde59a42542829c29 | succeeded (0s elapsed) | task1 | core.local | Tue, 07 Nov 2017 03:12:59 UTC |
+--------------------------+------------------------+-------+------------+-------------------------------+

```
Hope this helps!
@humblearner The example i gave was simplified to demonstrate the problem.

This also breaks when doing something like:

### Objects

``` YAML
---
description: Run a local linux command
enabled: true
runner_type: mistral-v2
entry_point: workflows/mistral-basic.yaml
name: mistral-basic
pack: examples
parameters:
  cmd:
    required: true
    type: string
  timeout:
    type: integer
    default: 60
  kv_test:
    type: object
    default: "{{ st2kv.system.nick_test | from_json_string }}"
```

``` shell
st2 key set nick_test '{"test_key": "test_value}'
```

In this case, in the workflow i want to utilize data within the object natively using `<% $.kv_test.test_key %>`


### Arrays

``` YAML
---
description: Run a local linux command
enabled: true
runner_type: mistral-v2
entry_point: workflows/mistral-basic.yaml
name: mistral-basic
pack: examples
parameters:
  cmd:
    required: true
    type: string
  timeout:
    type: integer
    default: 60
  kv_test:
    type: array
    default: "{{ st2kv.system.nick_test | from_json_string }}"
```

``` shell
st2 key set nick_test '["a", "b", "c"]'
```

In this case, in the workflow i want to utilize data within the arra natively using `<% $.kv_test[1] %>`
This is happening because of two reasons. First, the [actual execution request](https://github.com/StackStorm/st2/blob/ac6c00a129e1dc29f8a84a4fab33b94ff3382cd5/st2api/st2api/controllers/v1/actionexecutions.py#L177) takes place before any rendering takes place. Second, even if this wasn't true, the ActionExecutions API controller [is calling out to param_utils.render_live_params](https://github.com/StackStorm/st2/blob/ac6c00a129e1dc29f8a84a4fab33b94ff3382cd5/st2api/st2api/controllers/v1/actionexecutions.py#L183) which only renders "live" params (meaning params being explicitly passed in, which doesn't include default values. So, [when the controller requests an execution](https://github.com/StackStorm/st2/blob/ac6c00a129e1dc29f8a84a4fab33b94ff3382cd5/st2api/st2api/controllers/v1/actionexecutions.py#L199), the schema is checked, and this value is still an unrendered template, which is of course a string. So with this behavior, the value you're intending to get better also be a string. This behavior is more coincidental than intentional, so we should fix this.

Note in the docs, [each example of this is a string value](https://docs.stackstorm.com/actions.html#parameters-in-actions) - this is a bit misleading because the actual call to render final params doesn't happen until actual execution of the action. 

I'm looking at alternative approaches now, but figured I'd update the discussion with a root cause. I also see the reference to https://github.com/StackStorm/st2/issues/3828 but as @kami suggests, that's a much bigger change, so I'm looking to see if there's a less dangerous, temporary solution to this problem.

Bottom line, we **at least** need to update docs to say only string values are supported for this, but ideally we can render these values before sending the request.

Resolving patch:
diff --git a/st2api/st2api/controllers/v1/actionexecutions.py b/st2api/st2api/controllers/v1/actionexecutions.py
--- a/st2api/st2api/controllers/v1/actionexecutions.py
+++ b/st2api/st2api/controllers/v1/actionexecutions.py
@@ -174,8 +174,6 @@ def _schedule_execution(self,
 
         # Schedule the action execution.
         liveaction_db = LiveActionAPI.to_model(liveaction)
-        liveaction_db, actionexecution_db = action_service.create_request(liveaction_db)
-
         action_db = action_utils.get_action_by_ref(liveaction_db.action)
         runnertype_db = action_utils.get_runnertype_by_name(action_db.runner_type['name'])
 
@@ -184,6 +182,10 @@ def _schedule_execution(self,
                 runnertype_db.runner_parameters, action_db.parameters, liveaction_db.parameters,
                 liveaction_db.context)
         except param_exc.ParamException:
+
+            # We still need to create a request, so liveaction_db is assigned an ID
+            liveaction_db, actionexecution_db = action_service.create_request(liveaction_db)
+
             # By this point the execution is already in the DB therefore need to mark it failed.
             _, e, tb = sys.exc_info()
             action_service.update_status(
@@ -191,9 +193,12 @@ def _schedule_execution(self,
                 new_status=action_constants.LIVEACTION_STATUS_FAILED,
                 result={'error': str(e), 'traceback': ''.join(traceback.format_tb(tb, 20))})
             # Might be a good idea to return the actual ActionExecution rather than bubble up
-            # the execption.
+            # the exception.
             raise validation_exc.ValueValidationException(str(e))
 
+        # The request should be created after the above call to render_live_params
+        # so any templates in live parameters have a chance to render.
+        liveaction_db, actionexecution_db = action_service.create_request(liveaction_db)
         liveaction_db = LiveAction.add_or_update(liveaction_db, publish=False)
 
         _, actionexecution_db = action_service.publish_request(liveaction_db, actionexecution_db)
diff --git a/st2common/st2common/util/param.py b/st2common/st2common/util/param.py
--- a/st2common/st2common/util/param.py
+++ b/st2common/st2common/util/param.py
@@ -21,6 +21,7 @@
 from jinja2 import meta
 from st2common import log as logging
 from st2common.util.config_loader import get_config
+from st2common.util.jinja import is_jinja_expression
 from st2common.constants.action import ACTION_CONTEXT_KV_PREFIX
 from st2common.constants.pack import PACK_CONFIG_CONTEXT_KV_PREFIX
 from st2common.constants.keyvalue import DATASTORE_PARENT_SCOPE, SYSTEM_SCOPE, FULL_SYSTEM_SCOPE
@@ -220,12 +221,43 @@ def _cast_params_from(params, context, schemas):
     Pick a list of parameters from context and cast each of them according to the schemas provided
     '''
     result = {}
+
+    # First, cast only explicitly provided live parameters
     for name in params:
         param_schema = {}
         for schema in schemas:
             if name in schema:
                 param_schema = schema[name]
         result[name] = _cast(context[name], param_schema)
+
+    # Now, iterate over all parameters, and add any to the live set that satisfy ALL of the
+    # following criteria:
+    #
+    # - Have a default value that is a Jinja template
+    # - Are using the default value (i.e. not being overwritten by an actual live param)
+    #
+    # We do this because the execution API controller first determines live params before
+    # validating params against the schema. So, we want to make sure that if the default
+    # value is a template, it is rendered and added to the live params before this validation.
+    for schema in schemas:
+        for param_name, param_details in schema.items():
+
+            # Skip if the parameter doesn't have a default, or if the
+            # value in the context is identical to the default
+            if 'default' not in param_details or \
+                    param_details.get('default') == context[param_name]:
+                continue
+
+            # Skip if the default value isn't a Jinja expression
+            if not is_jinja_expression(param_details.get('default')):
+                continue
+
+            # Skip if the parameter is being overridden
+            if param_name in params:
+                continue
+
+            result[param_name] = _cast(context[param_name], param_details)
+
     return result
 
 


Test patch:
diff --git a/st2api/tests/unit/controllers/v1/test_executions.py b/st2api/tests/unit/controllers/v1/test_executions.py
--- a/st2api/tests/unit/controllers/v1/test_executions.py
+++ b/st2api/tests/unit/controllers/v1/test_executions.py
@@ -142,6 +142,20 @@
     'runner_type': 'inquirer',
 }
 
+ACTION_DEFAULT_TEMPLATE = {
+    'name': 'st2.dummy.default_template',
+    'description': 'An action that uses a jinja template as a default value for a parameter',
+    'enabled': True,
+    'pack': 'starterpack',
+    'runner_type': 'local-shell-cmd',
+    'parameters': {
+        'intparam': {
+            'type': 'integer',
+            'default': '{{ st2kv.system.test_int | int }}'
+        }
+    }
+}
+
 LIVE_ACTION_1 = {
     'action': 'sixpack.st2.dummy.action1',
     'parameters': {
@@ -211,6 +225,12 @@
     }
 }
 
+# Do not add parameters to this. There are tests that will test first without params,
+# then make a copy with params.
+LIVE_ACTION_DEFAULT_TEMPLATE = {
+    'action': 'starterpack.st2.dummy.default_template',
+}
+
 FIXTURES_PACK = 'generic'
 TEST_FIXTURES = {
     'runners': ['testrunner1.yaml'],
@@ -247,6 +267,10 @@ def setUpClass(cls):
         post_resp = cls.app.post_json('/v1/actions', cls.action_inquiry)
         cls.action_inquiry['id'] = post_resp.json['id']
 
+        cls.action_template = copy.deepcopy(ACTION_DEFAULT_TEMPLATE)
+        post_resp = cls.app.post_json('/v1/actions', cls.action_template)
+        cls.action_template['id'] = post_resp.json['id']
+
     @classmethod
     def tearDownClass(cls):
         cls.app.delete('/v1/actions/%s' % cls.action1['id'])
@@ -254,6 +278,7 @@ def tearDownClass(cls):
         cls.app.delete('/v1/actions/%s' % cls.action3['id'])
         cls.app.delete('/v1/actions/%s' % cls.action4['id'])
         cls.app.delete('/v1/actions/%s' % cls.action_inquiry['id'])
+        cls.app.delete('/v1/actions/%s' % cls.action_template['id'])
         super(BaseActionExecutionControllerTestCase, cls).tearDownClass()
 
     def test_get_one(self):
@@ -439,7 +464,8 @@ def test_post_parameter_validation_failed(self):
         execution['parameters'] = {"hosts": "127.0.0.1", "cmd": 1000}
         post_resp = self._do_post(execution, expect_errors=True)
         self.assertEqual(post_resp.status_int, 400)
-        self.assertEqual(post_resp.json['faultstring'], "1000 is not of type 'string', 'null'")
+        self.assertIn('Value "1000" must either be a string or None. Got "int"',
+                      post_resp.json['faultstring'])
 
         # Runner type expects parameters "cmd" to be str.
         execution['parameters'] = {"hosts": "127.0.0.1", "cmd": "1000", "c": 1}
@@ -541,7 +567,31 @@ def test_re_run_failure_parameter_override_invalid_type(self):
         re_run_resp = self.app.post_json('/v1/executions/%s/re_run' % (execution_id),
                                          data, expect_errors=True)
         self.assertEqual(re_run_resp.status_int, 400)
-        self.assertIn('1000 is not of type \'string\'', re_run_resp.json['faultstring'])
+        self.assertIn('Value "1000" must either be a string or None. Got "int"',
+                      re_run_resp.json['faultstring'])
+
+    def test_template_param(self):
+
+        # Test with default value containing template
+        post_resp = self._do_post(LIVE_ACTION_DEFAULT_TEMPLATE)
+        self.assertEqual(post_resp.status_int, 201)
+
+        # Assert that the template in the parameter default value
+        # was rendered and st2kv was used
+        self.assertEqual(post_resp.json['parameters']['intparam'], 0)
+
+        # Test with live param
+        live_int_param = 3
+        livaction_with_params = copy.deepcopy(LIVE_ACTION_DEFAULT_TEMPLATE)
+        livaction_with_params['parameters'] = {
+            "intparam": live_int_param
+        }
+        post_resp = self._do_post(livaction_with_params)
+        self.assertEqual(post_resp.status_int, 201)
+
+        # Assert that the template in the parameter default value
+        # was not rendered, and the provided parameter was used
+        self.assertEqual(post_resp.json['parameters']['intparam'], live_int_param)
 
     def test_re_run_workflow_success(self):
         # Create a new execution
diff --git a/st2common/tests/unit/test_param_utils.py b/st2common/tests/unit/test_param_utils.py
--- a/st2common/tests/unit/test_param_utils.py
+++ b/st2common/tests/unit/test_param_utils.py
@@ -647,6 +647,7 @@ def test_get_live_params_with_additional_context(self):
 
         expected_params = {
             'r1': 'generic',
+            'r2': 'generic',
             'r3': 'lolcathost'
         }
         self.assertEqual(live_params, expected_params)
@@ -695,3 +696,82 @@ def test_unsatisfied_dependency_friendly_error_message(self):
         expected_msg = 'Dependency unsatisfied in variable "variable_not_defined"'
         self.assertRaisesRegexp(ParamException, expected_msg, param_utils.render_live_params,
                                 runner_param_info, action_param_info, params, action_context)
+
+    def test_add_default_templates_to_live_params(self):
+        """Test addition of template values in defaults to live params
+        """
+
+        # Test with no live params, and two parameters - one should make it through because
+        # it was a template, and the other shouldn't because its default wasn't a template
+        schemas = [
+            {
+                'templateparam': {
+                    'default': '{{ 3 | int }}',
+                    'type': 'integer'
+                }
+            }
+        ]
+        context = {
+            'templateparam': '3'
+        }
+        result = param_utils._cast_params_from({}, context, schemas)
+        self.assertEquals(result, {'templateparam': 3})
+
+        # Ensure parameter is skipped if the value in context is identical to default
+        schemas = [
+            {
+                'nottemplateparam': {
+                    'default': '4',
+                    'type': 'integer'
+                }
+            }
+        ]
+        context = {
+            'nottemplateparam': '4',
+        }
+        result = param_utils._cast_params_from({}, context, schemas)
+        self.assertEquals(result, {})
+
+        # Ensure parameter is skipped if the parameter doesn't have a default
+        schemas = [
+            {
+                'nottemplateparam': {
+                    'type': 'integer'
+                }
+            }
+        ]
+        context = {
+            'nottemplateparam': '4',
+        }
+        result = param_utils._cast_params_from({}, context, schemas)
+        self.assertEquals(result, {})
+
+        # Skip if the default value isn't a Jinja expression
+        schemas = [
+            {
+                'nottemplateparam': {
+                    'default': '5',
+                    'type': 'integer'
+                }
+            }
+        ]
+        context = {
+            'nottemplateparam': '4',
+        }
+        result = param_utils._cast_params_from({}, context, schemas)
+        self.assertEquals(result, {})
+
+        # Ensure parameter is skipped if the parameter is being overridden
+        schemas = [
+            {
+                'templateparam': {
+                    'default': '{{ 3 | int }}',
+                    'type': 'integer'
+                }
+            }
+        ]
+        context = {
+            'templateparam': '4',
+        }
+        result = param_utils._cast_params_from({'templateparam': '4'}, context, schemas)
+        self.assertEquals(result, {'templateparam': 4})